Change search
Link to record
Permanent link

Direct link
BETA
Jaradat, Shatha
Publications (4 of 4) Show all publications
Jaradat, S., Dokoohaki, N., Hammar, K., Matskin, M. & Matskin, M. (2019). Dynamic CNN models for fashion recommendation in Instagram. In: Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018: . Paper presented at 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, 11 December 2018 through 13 December 2018 (pp. 1144-1151). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Dynamic CNN models for fashion recommendation in Instagram
Show others...
2019 (English)In: Proceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 1144-1151Conference paper, Published paper (Refereed)
Abstract [en]

Instagram as an online photo-sharing and socialnetworking service is becoming more powerful in enabling fashion brands to ramp up their business growth. Nowadays, a single post by a fashion influencer attracts a wealth of attention and a magnitude of followers who are curious to know more about the brands and style of each clothing item sitting inside the image. To this end, the development of efficient Deep CNN models that can accurately detect styles and brands have become a research challenge. In addition, current techniques need to cope with inherent fashion-related data issues. Namely, clothing details inside a single image only cover a small proportion of the large and hierarchical space of possible brands and clothing item attributes. In order to cope with these challenges, one can argue that neural classifiers should become adapted to large-scale and hierarchical fashion datasets. As a remedy, we propose two novel techniques to incorporate the valuable social media textual content to support the visual classification in a dynamic way. The first method is adaptive neural pruning (DynamicPruning) in which the clothing item category detected from posts' text analysis can be used to activate the possible range of connections of clothing attributes' classifier. The second method (DynamicLayers) is a dynamic framework in which multiple-attributes classification layers exist and a suitable attributes' classifier layer is activated dynamically based upon the mined text from the image. Extensive experiments on a dataset gathered from Instagram and a baseline fashion dataset (DeepFashion) have demonstrated that our approaches can improve the accuracy by about 20% when compared to base architectures. It is worth highlighting that with Dynamiclayers we have gained 35% accuracy for the task of multi-class multilabeled classification compared to the other model.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2019
Keywords
Dynamic CNN, Dynamic Computation Graph, Fashion Recommendation, Image Classification, Neural Pruning, Text Mining, Classification (of information), Cloud computing, Data mining, Large dataset, Text processing, Dynamic computations, Multiple attributes, Online Photo Sharing, Research challenges, Visual classification, Ubiquitous computing
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-252212 (URN)10.1109/BDCloud.2018.00169 (DOI)000467843200155 ()2-s2.0-85063866491 (Scopus ID)9781728111414 (ISBN)
Conference
16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, 11 December 2018 through 13 December 2018
Note

QC 20190611

Available from: 2019-06-11 Created: 2019-06-11 Last updated: 2019-08-19Bibliographically approved
Hammar, K., Jaradat, S., Dokoohaki, N. & Matskin, M. (2018). Deep Text Mining of Instagram Data Without Strong Supervision. In: Proceedings - 2018 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2018: . Paper presented at 18th IEEE/WIC/ACM International Conference on Web Intelligence, WI 2018; Santiago; Chile; 3 December 2018 through 6 December 2018 (pp. 158-165). IEEE
Open this publication in new window or tab >>Deep Text Mining of Instagram Data Without Strong Supervision
2018 (English)In: Proceedings - 2018 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2018, IEEE, 2018, p. 158-165Conference paper, Published paper (Refereed)
Abstract [en]

With the advent of social media, our online feeds increasingly consist of short, informal, and unstructured text. This textual data can be analyzed for the purpose of improving user recommendations and detecting trends. Instagram is one of the largest social media platforms, containing both text and images. However, most of the prior research on text processing in social media is focused on analyzing Twitter data, and little attention has been paid to text mining of Instagram data. Moreover, many text mining methods rely on annotated training data, which in practice is both difficult and expensive to obtain. In this paper, we present methods for unsupervised mining of fashion attributes from Instagram text, which can enable a new kind of user recommendation in the fashion domain. In this context, we analyze a corpora of Instagram posts from the fashion domain, introduce a system for extracting fashion attributes from Instagram, and train a deep clothing classifier with weak supervision to classify Instagram posts based on the associated text. With our experiments, we confirm that word embeddings are a useful asset for information extraction. Experimental results show that information extraction using word embeddings outperforms a baseline that uses Levenshtein distance. The results also show the benefit of combining weak supervision signals using generative models instead of majority voting. Using weak supervision and generative modeling, an F1 score of 0.61 is achieved on the task of classifying the image contents of Instagram posts based solely on the associated text, which is on level with human performance. Finally, our empirical study provides one of the few available studies on Instagram text and shows that the text is noisy, that the text distribution exhibits the long-tail phenomenon, and that comment sections on Instagram are multi-lingual.

Place, publisher, year, edition, pages
IEEE, 2018
Keywords
Information extraction, Instagram, Weak Supervision, Word Embeddings
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-245979 (URN)10.1109/WI.2018.00-94 (DOI)000458968200021 ()2-s2.0-85061892408 (Scopus ID)9781538673256 (ISBN)
Conference
18th IEEE/WIC/ACM International Conference on Web Intelligence, WI 2018; Santiago; Chile; 3 December 2018 through 6 December 2018
Note

QC 20190313

Available from: 2019-03-13 Created: 2019-03-13 Last updated: 2019-03-13Bibliographically approved
Jaradat, S., Dokoohaki, N., Hammar, K., Wara, U. & Matskin, M. (2018). Dynamic CNN Models For Fashion Recommendation in Instagram. In: Chen, JJ Yang, LT (Ed.), 2018 IEEE INT CONF ON PARALLEL & DISTRIBUTED PROCESSING WITH APPLICATIONS, UBIQUITOUS COMPUTING & COMMUNICATIONS, BIG DATA & CLOUD COMPUTING, SOCIAL COMPUTING & NETWORKING, SUSTAINABLE COMPUTING & COMMUNICATIONS: . Paper presented at 16th IEEE ISPA / 17th IEEE IUCC / 8th IEEE BDCloud / 11th IEEE SocialCom / 8th IEEE SustainCom, DEC 11-13, 2018, Melbourne, AUSTRALIA (pp. 1144-1151). IEEE COMPUTER SOC
Open this publication in new window or tab >>Dynamic CNN Models For Fashion Recommendation in Instagram
Show others...
2018 (English)In: 2018 IEEE INT CONF ON PARALLEL & DISTRIBUTED PROCESSING WITH APPLICATIONS, UBIQUITOUS COMPUTING & COMMUNICATIONS, BIG DATA & CLOUD COMPUTING, SOCIAL COMPUTING & NETWORKING, SUSTAINABLE COMPUTING & COMMUNICATIONS / [ed] Chen, JJ Yang, LT, IEEE COMPUTER SOC , 2018, p. 1144-1151Conference paper, Published paper (Refereed)
Abstract [en]

Instagram as an online photo-sharing and social-networking service is becoming more powerful in enabling fashion brands to ramp up their business growth. Nowadays, a single post by a fashion influencer attracts a wealth of attention and a magnitude of followers who are curious to know more about the brands and style of each clothing item sitting inside the image. To this end, the development of efficient Deep CNN models that can accurately detect styles and brands have become a research challenge. In addition, current techniques need to cope with inherent fashion-related data issues. Namely, clothing details inside a single image only cover a small proportion of the large and hierarchical space of possible brands and clothing item attributes. In order to cope with these challenges, one can argue that neural classifiers should become adapted to large-scale and hierarchical fashion datasets. As a remedy, we propose two novel techniques to incorporate the valuable social media textual content to support the visual classification in a dynamic way. The first method is adaptive neural pruning (DynamicPruning) in which the clothing item category detected from posts' text analysis can be used to activate the possible range of connections of clothing attributes' classifier. The second method (DynamicLayers) is a dynamic framework in which multiple-attributes classification layers exist and a suitable attributes' classifier layer is activated dynamically based upon the mined text from the image. Extensive experiments on a dataset gathered from Instagram and a baseline fashion dataset (DeepFashion) have demonstrated that our approaches can improve the accuracy by about 20% when compared to base architectures. It is worth highlighting that with Dynamiclayers we have gained 35% accuracy for the task of multi-class multi-labeled classification compared to the other model.

Place, publisher, year, edition, pages
IEEE COMPUTER SOC, 2018
Series
IEEE International Symposium on Parallel and Distributed Processing with Applications, ISSN 2158-9178
Keywords
Neural Pruning, Dynamic Computation Graph, Dynamic CNN, Image Classification, Text Mining, Fashion Recommendation
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-252671 (URN)10.1109/BDCloud.2018.00169 (DOI)000467843200155 ()2-s2.0-85063866491 (Scopus ID)978-1-7281-1141-4 (ISBN)
Conference
16th IEEE ISPA / 17th IEEE IUCC / 8th IEEE BDCloud / 11th IEEE SocialCom / 8th IEEE SustainCom, DEC 11-13, 2018, Melbourne, AUSTRALIA
Note

QC 20190603

Available from: 2019-06-03 Created: 2019-06-03 Last updated: 2019-06-03Bibliographically approved
Jaradat, S. (2017). Deep cross-domain fashion recommendation. In: RecSys 2017 - Proceedings of the 11th ACM Conference on Recommender Systems: . Paper presented at 11th ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27 August 2017 through 31 August 2017 (pp. 407-410). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>Deep cross-domain fashion recommendation
2017 (English)In: RecSys 2017 - Proceedings of the 11th ACM Conference on Recommender Systems, Association for Computing Machinery (ACM), 2017, p. 407-410Conference paper (Refereed)
Abstract [en]

With the increasing number of online shopping services, the number of users and the quantity of visual and textual information on the Internet, there is a pressing need for intelligent recommendation systems that analyze the user's behavior amongst multiple domains and help them to find the desirable information without the burden of search. However, there is little research that has been done on complex recommendation scenarios that involve knowledge transfer across multiple domains. This problem is especially challenging when the involved data sources are complex in terms of the limitations on the quantity and quality of data that can be crawled. The goal of this paper is studying the connection between visual and textual inputs for better analysis of a certain domain, and to examine the possibility of knowledge transfer from complex domains for the purpose of efficient recommendations. The methods employed to achieve this study include both design of architecture and algorithms using deep learning technologies to analyze the effect of deep pixel-wise semantic segmentation and text integration on the quality of recommendations. We plan to develop a practical testing environment in a fashion domain.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2017
Keywords
CNN, Cross-domain knowledge transfer, Deep learning, Domain adaptation, Fashion recommendation, Transfer learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-218546 (URN)10.1145/3109859.3109861 (DOI)2-s2.0-85030469064 (Scopus ID)9781450346528 (ISBN)
Conference
11th ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, 27 August 2017 through 31 August 2017
Note

QC 20171130

Available from: 2017-11-30 Created: 2017-11-30 Last updated: 2018-01-13Bibliographically approved
Organisations

Search in DiVA

Show all publications