Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Toward automated veracity assessment of data from open sources using features and indicators
KTH, Skolan för elektroteknik och datavetenskap (EECS), Datavetenskap, Programvaruteknik och datorsystem, SCS.ORCID-id: 0000-0002-0408-1421
2024 (engelsk)Doktoravhandling, med artikler (Annet vitenskapelig)
Abstract [en]

This dissertation hypothesizes that the key to automated veracity assessment of data from open sources is the careful estimation and extraction of relevant features and indicators. These features and indicators provide added value to a quantifiable veracity assessment, either directly or indirectly. The importance and usefulness of a veracity assessment largely depend on the specific situation and reason for which it is being conducted. Factors such as the recipient of the veracity assessment, the scope of the assessment, and the metrics used to measure accuracy and performance, all play a role in determining the value and perceived quality of the assessment.

Five peer-reviewed publications; two journal articles, two conference articles, and one workshop article, are included in this compilation thesis.

The main contributions of the work presented in this dissertation are: i) a compilation of challenges with manual methods of veracity assessment, ii) a road map for addressing the identified challenges, iii) identification of the state-of-the-art and gap analysis of veracity assessment of open-source data, iv) exploration of indicators such as topic geo-location tracking over time and stance classification, and v) evaluation of various feature types, model transferability, and style obfuscation attacks and the impact on accuracy for automated veracity assessment of a type of deception: fake reviews.

Abstract [sv]

Denna avhandling har som hypotes att nyckeln till automatiserad trovärdighetsbedömning av data från öppna källor ligger i det noggranna urvalet och estimeringen av relevanta särdrag och indikatorer. Dessa särdrag och indikatorer ger ett direkt eller indirekt mervärde till en kvantifierbar trovärdighetsbedömning. Betydelsen och användbarheten av en trovärdighetsbedömning beror till stor del på den specifika kontexten och anledningen till att den genomförs. Faktorer som mottagaren av trovärdighetsbedömningen, omfattningen av bedömningen och de mått som används för att mäta noggrannhet och prestanda, spelar alla in för att bestämma värdet och den upplevda kvalitén på bedömningen.

Fem referentgranskade publikationer ingår i denna sammanläggningsavhandling; två tidskriftsartiklar, två konferensartiklar och en workshopartikel.

De huvudsakliga bidragen från arbetet som presenteras i denna avhandling är: i) en sammanställning av utmaningar relaterade till manuella metoder för trovärdighetsbedömning, ii) en plan för att ta itu med de identifierade utmaningarna, iii) identifiering av forskningsfronten och en gapanalys av trovärdighetsbedömning av data från öppna källor, iv) studie av indikatorer såsom geolokalisering av ämnen och spårning av dem över tid samt klassificering av individers reaktioner i inlägg på sociala medier, och v) en utvärdering av särdragstyper som påverkar noggrannheten för automatisk trovärdighetsbedömning applicerat på en typ av bedrägeri: falska recensioner.

sted, utgiver, år, opplag, sider
Stockholm, Sweden: KTH Royal Institute of Technology, 2024. , s. 71
Serie
TRITA-EECS-AVL ; 2024:47
Emneord [en]
Veracity assessment, natural language processing, machine learning, open-source data
Emneord [sv]
Trovärdighetsbedömning, naturlig språkbehandling, maskininlärning, data från öppna källor
HSV kategori
Forskningsprogram
Informations- och kommunikationsteknik
Identifikatorer
URN: urn:nbn:se:kth:diva-346353ISBN: 978-91-8040-927-8 (tryckt)OAI: oai:DiVA.org:kth-346353DiVA, id: diva2:1857437
Disputas
2024-06-03, https://kth-se.zoom.us/j/63226866138, Sal C, Kistagången 16, Stockholm, 13:30 (engelsk)
Opponent
Veileder
Merknad

QC 20240514

Tilgjengelig fra: 2024-05-14 Laget: 2024-05-13 Sist oppdatert: 2024-05-21bibliografisk kontrollert
Delarbeid
1. Towards Automatic Veracity Assessment of Open Source Information
Åpne denne publikasjonen i ny fane eller vindu >>Towards Automatic Veracity Assessment of Open Source Information
2015 (engelsk)Inngår i: 2015 IEEE International Congress on Big Data (BigData Congress), IEEE Computer Society, 2015, s. 199-206Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Intelligence analysis is dependent on veracity assessment of Open Source Information (OSINF) which includes assessment of the reliability of sources and credibility of information. Traditionally, OSINF veracity assessment is done by intelligence analysts manually, but the large volumes, high velocity, and variety make it infeasible to continue doing so, and calls for automation. Based on meetings, interviews and questionnaires with military personnel, analysis of related work and state of the art, we identify the challenges and propose an approach and a corresponding framework for automated veracity assessment of OSINF. The framework provides a basis for new tools which will give the intelligence analysts the ability to automatically or semi-automatically assess veracity of larger amounts of data in a shorter amount of time. Instead of spending their time working with irrelevant, ambiguous, contradicting, biased, or plain wrong data, they can spend more time on analysis.

sted, utgiver, år, opplag, sider
IEEE Computer Society, 2015
Emneord
Big Data, public domain software, Big Data, OSINF, automatic data veracity assessment, intelligence analysis, open source information, Automation, Big data, Interviews, Probabilistic logic, Reliability, Semantics, Twitter, NATO STANAG 2511, OSINF, big data, data veracity, reliability and credibility, trust, veracity assessment
HSV kategori
Identifikatorer
urn:nbn:se:kth:diva-179402 (URN)10.1109/BigDataCongress.2015.36 (DOI)000380443700026 ()2-s2.0-84959501022 (Scopus ID)978-1-4673-7277-0 (ISBN)
Eksternt samarbeid:
Konferanse
Big Data (BigData Congress), 2015 IEEE International Congress on, New York, USA, June 27 - July 2, 2015.
Merknad

QC 20151217

Tilgjengelig fra: 2015-12-16 Laget: 2015-12-16 Sist oppdatert: 2024-05-14bibliografisk kontrollert
2. Veracity assessment of online data
Åpne denne publikasjonen i ny fane eller vindu >>Veracity assessment of online data
Vise andre…
2020 (engelsk)Inngår i: Decision Support Systems, ISSN 0167-9236, E-ISSN 1873-5797, Vol. 129, artikkel-id 113132Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

Fake news, malicious rumors, fabricated reviews, generated images and videos, are today spread at an unprecedented rate, making the task of manually assessing data veracity for decision-making purposes a daunting task. Hence, it is urgent to explore possibilities to perform automatic veracity assessment. In this work we review the literature in search for methods and techniques representing state of the art with regard to computerized veracity assessment. We study what others have done within the area of veracity assessment, especially targeted towards social media and open source data, to understand research trends and determine needs for future research. The most common veracity assessment method among the studied set of papers is to perform text analysis using supervised learning. Regarding methods for machine learning much has happened in the last couple of years related to the advancements made in deep learning. However, very few papers make use of these advancements. Also, the papers in general tend to have a narrow scope, as they focus on solving a small task with only one type of data from one main source. The overall veracity assessment problem is complex, requiring a combination of data sources, data types, indicators, and methods. Only a few papers take on such a broad scope, thus, demonstrating the relative immaturity of the veracity assessment domain.

sted, utgiver, år, opplag, sider
Elsevier, 2020
Emneord
Veracity assessment, Credibility, Data quality, Online data, Social media, Fake news
HSV kategori
Identifikatorer
urn:nbn:se:kth:diva-268789 (URN)10.1016/j.dss.2019.113132 (DOI)000510956500001 ()2-s2.0-85076227196 (Scopus ID)
Merknad

QC 20200224

Tilgjengelig fra: 2020-02-24 Laget: 2020-02-24 Sist oppdatert: 2024-05-14bibliografisk kontrollert
3. Tracking Geographical Locations using a Geo-Aware Topic Model for Analyzing Social Media Data
Åpne denne publikasjonen i ny fane eller vindu >>Tracking Geographical Locations using a Geo-Aware Topic Model for Analyzing Social Media Data
2017 (engelsk)Inngår i: Decision Support Systems, ISSN 0167-9236, E-ISSN 1873-5797, Vol. 99, nr SI, s. 18-29Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

Tracking how discussion topics evolve in social media and where these topics are discussed geographically over time has the potential to provide useful information for many different purposes. In crisis management, knowing a specific topic’s current geographical location could provide vital information to where, or even which, resources should be allocated. This paper describes an attempt to track online discussions geographically over time. A distributed geo-aware streaming latent Dirichlet allocation model was developed for the purpose of recognizing topics’ locations in unstructured text. To evaluate the model it has been implemented and used for automatic discovery and geographical tracking of election topics during parts of the 2016 American presidential primary elections. It was shown that the locations correlated with the actual election locations, and that the model provides a better geolocation classification compared to using a keyword-based approach.

sted, utgiver, år, opplag, sider
Elsevier, 2017
Emneord
Social media, Topic modeling, Geo-awareness, Trend analysis, Latent Dirichlet allocation, Streaming media
HSV kategori
Forskningsprogram
Datalogi
Identifikatorer
urn:nbn:se:kth:diva-210570 (URN)10.1016/j.dss.2017.05.006 (DOI)000405162500003 ()2-s2.0-85020801622 (Scopus ID)
Forskningsfinansiär
EU, FP7, Seventh Framework Programme, 312649
Merknad

QC 20170704

Tilgjengelig fra: 2017-07-02 Laget: 2017-07-02 Sist oppdatert: 2024-05-14bibliografisk kontrollert
4. Mama Edha at SemEval-2017 Task 8: Stance Classification with CNN and Rules
Åpne denne publikasjonen i ny fane eller vindu >>Mama Edha at SemEval-2017 Task 8: Stance Classification with CNN and Rules
2017 (engelsk)Inngår i: ACL 2017 - 11th International Workshop on Semantic Evaluations, SemEval 2017, Proceedings of the Workshop, Association for Computational Linguistics (ACL) , 2017, s. 481-485Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

For the competition SemEval-2017 we investigated the possibility of performing stance classification (support, deny, query or comment) for messages in Twitter conversation threads related to rumours. Stance classification is interesting since it can provide a basis for rumour veracity assessment. Our ensemble classification approach of combining convolutional neural networks with both automatic rule mining and manually written rules achieved a final accuracy of 74.9% on the competition's test data set for Task 8A. To improve classification we also experimented with data relabeling and using the grammatical structure of the tweet contents for classification.

sted, utgiver, år, opplag, sider
Association for Computational Linguistics (ACL), 2017
HSV kategori
Identifikatorer
urn:nbn:se:kth:diva-332056 (URN)2-s2.0-85097656375 (Scopus ID)
Konferanse
11th International Workshop on Semantic Evaluations, SemEval 2017, co-located with the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, Aug 4 2017 - Aug 3 2017
Merknad

Part of ISBN 9781945626555

QC 20230719

Tilgjengelig fra: 2023-07-19 Laget: 2023-07-19 Sist oppdatert: 2024-05-14bibliografisk kontrollert
5. Identifying deceptive reviews: Feature exploration, model transferability and classification attack
Åpne denne publikasjonen i ny fane eller vindu >>Identifying deceptive reviews: Feature exploration, model transferability and classification attack
2019 (engelsk)Inngår i: Proceedings of the 2019 European Intelligence and Security Informatics Conference, EISIC 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, s. 109-116Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

The temptation to influence and sway public opinion most certainly increases with the growth of open online forums where anyone anonymously can express their views and opinions. Since online review sites are a popular venue for opinion influencing attacks, there is a need to automatically identify deceptive posts. The main focus of this work is on automatic identification of deceptive reviews, both positive and negative biased. With this objective, we build a deceptive review SVM based classification model and explore the performance impact of using different feature types (TF-IDF, word2vec, PCFG). Moreover, we study the transferability of trained classification models applied to review data sets of other types of products, and, the classifier robustness, i.e., the accuracy impact, against attacks by stylometry obfuscation trough machine translation. Our findings show that i) we achieve an accuracy of over 90% using different feature types, ii) the trained classification models do not perform well when applied on other data sets containing reviews of different products, and iii) machine translation only slightly impacts the results and can not be used as a viable attack method. 

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers Inc., 2019
Emneord
Classification, Deceptive, Fake, PCFG, SVM, Word2vec, Automation, Computational linguistics, Computer aided language translation, Social aspects, Support vector machines, Attack methods, Classification models, Feature types, Machine translations, Model transferabilities, Online reviews, Performance impact, Public opinions, Classification (of information)
HSV kategori
Identifikatorer
urn:nbn:se:kth:diva-285414 (URN)10.1109/EISIC49498.2019.9108852 (DOI)2-s2.0-85087087979 (Scopus ID)9781728167350 (ISBN)
Konferanse
2019 European Intelligence and Security Informatics Conference, EISIC 2019, 26 November 2019 through 27 November 2019
Merknad

QC 20201130

Tilgjengelig fra: 2020-11-30 Laget: 2020-11-30 Sist oppdatert: 2024-05-14bibliografisk kontrollert

Open Access i DiVA

summary(879 kB)168 nedlastinger
Filinformasjon
Fil SUMMARY01.pdfFilstørrelse 879 kBChecksum SHA-512
39220bc334b5ee58a0de78a6a343c234465cc87f7929351711b6f3ffe84ed0fa8b35936533fe6d20ff0f12c4e912f081736d81a30e6c23c2c1f70fdc0ebab699
Type summaryMimetype application/pdf

Person

García Lozano, Marianela

Søk i DiVA

Av forfatter/redaktør
García Lozano, Marianela
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

isbn
urn-nbn

Altmetric

isbn
urn-nbn
Totalt: 1950 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf