Analyzing media in real-time is of great importance with social media platforms at the epicenter of crunching, digesting and disseminating content to individuals connected to these platforms. Within this context, topic models, specially LDA, have gained strong momentum due to their scalability, inference power and their compact semantics. Although, state of the art topic models come short in handling streaming large chunks of data arriving dynamically onto the platform, thus hindering their quality of interpretation as well as their adaptability to information overload. As a result, in this manuscript we propose for a labelled and online extension to LDA (OLLDA), which incorporates supervision through external labeling and capability of quickly digesting real-time updates thus making it more adaptive to Twitter and platforms alike. Our proposed extension has capability of handling large quantities of newly arrived documents in a stream, and at the same time, is capable of achieving high topic inference quality given the short and often sloppy text of tweets. Our approach mainly uses an approximate inference technique based on variational inference coupled with a labeled LDA model. We conclude by presenting experiments using a one year crawl of Twitter data that shows significantly improved topical inference as well as temporal user profile classification when compared to state of the art baselines.

With the advent of social media, our online feeds increasingly consist of short, informal, and unstructured text. This textual data can be analyzed for the purpose of improving user recommendations and detecting trends. Instagram is one of the largest social media platforms, containing both text and images. However, most of the prior research on text processing in social media is focused on analyzing Twitter data, and little attention has been paid to text mining of Instagram data. Moreover, many text mining methods rely on annotated training data, which in practice is both difficult and expensive to obtain. In this paper, we present methods for unsupervised mining of fashion attributes from Instagram text, which can enable a new kind of user recommendation in the fashion domain. In this context, we analyze a corpora of Instagram posts from the fashion domain, introduce a system for extracting fashion attributes from Instagram, and train a deep clothing classifier with weak supervision to classify Instagram posts based on the associated text. With our experiments, we confirm that word embeddings are a useful asset for information extraction. Experimental results show that information extraction using word embeddings outperforms a baseline that uses Levenshtein distance. The results also show the benefit of combining weak supervision signals using generative models instead of majority voting. Using weak supervision and generative modeling, an F1 score of 0.61 is achieved on the task of classifying the image contents of Instagram posts based solely on the associated text, which is on level with human performance. Finally, our empirical study provides one of the few available studies on Instagram text and shows that the text is noisy, that the text distribution exhibits the long-tail phenomenon, and that comment sections on Instagram are multi-lingual.

Online networking sites tried their best to have right privacy mechanisms in place for users, enabling them to share the right content with the right audience. With all these efforts, privacy customizations remain hard for users across the sites. Existing research that addresses this problem mainly focuses on semi-supervised strategies that introduce extra complexity by requiring the user to manually specify initial privacy preferences for their friends. In this work, we suggest a deep reinforcement learning framework that can dynamically generate privacy labels for users in OSNs. We evaluated our framework on a 1 year crawl of Twitter data, using different types of recurrent units in recurrent neural networks (RNN): Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Simple RNN. Our experiments revealed that LSTM performed better than GRU in terms of top users detection accuracy and the ranked dependence between the generated privacy labels and estimated user trust values.

With the increasing number of online shopping services, the number of users and the quantity of visual and textual information on the Internet, there is a pressing need for intelligent recommendation systems that analyze the user's behavior amongst multiple domains and help them to find the desirable information without the burden of search. However, there is little research that has been done on complex recommendation scenarios that involve knowledge transfer across multiple domains. This problem is especially challenging when the involved data sources are complex in terms of the limitations on the quantity and quality of data that can be crawled. The goal of this paper is studying the connection between visual and textual inputs for better analysis of a certain domain, and to examine the possibility of knowledge transfer from complex domains for the purpose of efficient recommendations. The methods employed to achieve this study include both design of architecture and algorithms using deep learning technologies to analyze the effect of deep pixel-wise semantic segmentation and text integration on the quality of recommendations. We plan to develop a practical testing environment in a fashion domain.

With around 2.77 billion users using online social media platforms nowadays, it is becoming more attractive for business retailers to reach and to connect to more potential clients through social media. However, providing more effective recommendations to grab clients’ attention requires a deep understanding of users’ interests. Given the enormous amounts of text and images that users share in social media, deep learning approaches play a major role in performing semantic analysis of text and images. Moreover, object localisation and pixel-by-pixel semantic segmentation image analysis neural architectures provide an enhanced level of information. However, to train such architectures in an end-to-end manner, detailed datasets with specific meta-data are required. In our paper, we present a complete framework that can be used to tag images in a hierarchical fashion, and to perform object localisation and semantic segmentation. In addition to this, we show the value of using neural word embeddings in providing additional semantic details to annotators to guide them in annotating images in the system. Our framework is designed to be a fully functional solution capable of providing fine-grained annotations, essential localisation and segmentation services while keeping the core architecture simple and extensible. We also provide a fine-grained labelled fashion dataset that can be a rich source for research purposes.

Instagram as an online photo-sharing and social-networking service is becoming more powerful in enabling fashion brands to ramp up their business growth. Nowadays, a single post by a fashion influencer attracts a wealth of attention and a magnitude of followers who are curious to know more about the brands and style of each clothing item sitting inside the image. To this end, the development of efficient Deep CNN models that can accurately detect styles and brands have become a research challenge. In addition, current techniques need to cope with inherent fashion-related data issues. Namely, clothing details inside a single image only cover a small proportion of the large and hierarchical space of possible brands and clothing item attributes. In order to cope with these challenges, one can argue that neural classifiers should become adapted to large-scale and hierarchical fashion datasets. As a remedy, we propose two novel techniques to incorporate the valuable social media textual content to support the visual classification in a dynamic way. The first method is adaptive neural pruning (DynamicPruning) in which the clothing item category detected from posts' text analysis can be used to activate the possible range of connections of clothing attributes' classifier. The second method (DynamicLayers) is a dynamic framework in which multiple-attributes classification layers exist and a suitable attributes' classifier layer is activated dynamically based upon the mined text from the image. Extensive experiments on a dataset gathered from Instagram and a baseline fashion dataset (DeepFashion) have demonstrated that our approaches can improve the accuracy by about 20% when compared to base architectures. It is worth highlighting that with Dynamiclayers we have gained 35% accuracy for the task of multi-class multi-labeled classification compared to the other model.