kth.sePublications KTH
System disruptions
There are recurring service disruptions on some of the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Impact of Synthetic Data on Membership Inference Attacks
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Theoretical Computer Science, TCS.ORCID iD: 0000-0001-6934-0378
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Theoretical Computer Science, TCS.ORCID iD: 0000-0001-5742-5462
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Privacy of machine learning on Big Data has become a prominent issue in recent years due to the increased availability and usage of sensitive personal data to train the models. Membership inference attacks are one such issue that has been identified as a major privacy threat against machine learning models. Several techniques including applying differential privacy have been advocated to mitigate the effectiveness of inference attacks, however, they come at a cost of reduced utility/accuracy. Synthetic data is one approach that has been widely studied as a tool for privacy preservation recently but not much yet in the context of membership inference attacks. In this work, we aim to deepen the understanding of the impact of synthetic data on membership inference attacks. We compare models trained on original versus synthetic data, evaluate different synthetic data generation methods, and study the effect of overfitting in terms of membership inference attacks. Our investigation reveals that training on synthetic data can significantly reduce the effectiveness of membership inference attacks compared to models trained directly on the original data. This also holds for highly overfitted models that have been shown to increase the success rate of membership inference attacks. We also find that different synthetic data generation methods do not differ much in terms of membership inference attack accuracy but they do differ in terms of utility (i.e., observed based on train/test accuracy). Since synthetic data shows promising results for binary classification-based membership inference attacks on classification models explored in this work, exploring the impact on other attack types, models, and attribute inference attacks can be of worth. 

Keywords [en]
Synthetic Data, Machine Learning, Membership Inference Attack, Accuracy
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-326693OAI: oai:DiVA.org:kth-326693DiVA, id: diva2:1755632
Funder
Knut and Alice Wallenberg Foundation
Note

QC 20230510

Available from: 2023-05-08 Created: 2023-05-08 Last updated: 2023-05-10Bibliographically approved
In thesis
1. Towards Privacy Preserving Intelligent Systems
Open this publication in new window or tab >>Towards Privacy Preserving Intelligent Systems
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Intelligent systems, i.e., digital systems containing smart devices that can gather, analyze, and act in response to the data they collect from their surrounding environment, have progressed from theory to application especially in the last decade, thanks to the recent technological advances in sensors and machine learning. These systems can take decisions on users' behalf dynamically by learning their behavior over time. The number of such smart devices in our surroundings is increasing rapidly. Since these devices in most cases handle privacy-sensitive data, privacy concerns are also increasing at a similar rate. However, privacy research has not been in sync with these developments. Moreover, the systems are heterogeneous in nature (e.g., in terms of form factor, energy, processing power, use case scenarios, etc.) and continuously evolving which makes the privacy problem even more challenging.

In this thesis, we identify open privacy problems of intelligent systems and later propose solutions to some of the most prominent ones. We first investigate privacy concerns in the context of data stored on a single smart device. We identify that ownership change of a smart device can leak privacy-sensitive information stored on the device. To solve this, we propose a framework to enhance the privacy of owners during ownership change of smart devices based on context detection and data encryption. Moving from the single-device setting to more complex systems involving multiple devices, we conduct a systematic literature review and a review of commercial systems to identify the unique privacy concerns of home-based health monitoring systems. From the review, we distill a common architecture covering most commercial and academic systems, including an inventory of what concerns they address, their privacy considerations, and how they handle the data. Based on this, we then identify potential privacy intervention points of such systems.

For the publication of collected data or a machine-learning model trained on such data, we explore the potential of synthetic data as a tool for achieving a better trade-off between privacy and utility compared to traditional privacy-enhancing approaches. We perform a thorough assessment of the utility of synthetic tabular data. Our investigation reveals that none of the commonly used utility metrics for assessing how well synthetic data corresponds to the original data can predict whether for any given univariate or multivariate statistical analysis (when the analysis is not known beforehand) synthetic data achieves utility similar to the original data. For machine learning-based classification tasks, however, the metric Confidence Interval Overlap shows a strong correlation with how similarly the machine learning models (i.e., trained on synthetic vs. original) perform. Concerning privacy, we explore membership inference attacks against machine learning models which aim at finding out whether some (or someone's) particular data was used to train the model. We find from our exploration that training on synthetic data instead of original data can significantly reduce the effectiveness of membership inference attacks. For image data, we propose a novel methodology to quantify, improve, and tune the privacy utility trade-off of the synthetic image data generation process compared to the traditional approaches.

Overall, our exploration in this thesis reveals that there are several open research questions regarding privacy at different phases of the data lifespan of intelligent systems such as privacy-preserving data storage, possible inferences due to data aggregation, and the quantification and improvement of privacy utility trade-off for achieving better utility at an acceptable level of privacy in a data release. The identified privacy concerns and their corresponding solutions presented in this thesis will help the research community to recognize and address remaining privacy concerns in the domain. Solving the concerns will encourage the end-users to adopt the systems and enjoy the benefits without having to worry about privacy.

Abstract [sv]

Intelligenta system, d.v.s. digitala system som innehåller smarta enheter som kan samla in, analysera och agera beroende på den data de samlar in från sin omgivning, har gått från teori till tillämpning, särskilt under det senaste decenniet, tack vare tekniska framsteg inom sensorer och maskininlärning. Dessa system kan fatta beslut åt användarna på ett dynamiskt sätt genom att lära sig deras beteende över tid. Antalet sådana smarta enheter i vår omgivning  ökar snabbt. Eftersom dessa enheter i de flesta fall hanterar integritetskänsliga data, ökar integritetsproblemen också i samma takt. Dock har forskningen kring skydd av personlig information och integritet inte varit i synk med denna utveckling. Dessutom är systemenheterogena (t.ex. när det gäller formfaktor, energi, beräkningskapacitet, användningsområden, etc.) och de utvecklas ständigt vilket gör att integritetsproblem blir ännu mer utmanande.

I denna avhandling identifierar vi integritetsproblem för intelligenta system och föreslår lösningar på några av de mest framstående problemen. Vi undersöker först integritetsproblem i samband med data som lagras på en enda smart enhet. Vi noterar att när en smart enhete byter ägare kan integritetskänslig information lagrad på enheten komma i orätta händer. För att lösa detta föreslår vi ett ramverk för att förbättra integriteten för ägarna under sådana ägarbyten. Ramverket använder sig av tekniker för att detektera miljöombyte och kryptering av data. Sedan går vi från scenariot med en enda enhet till mer komplexa system som involverar flera enheter. Vi genomför en systematisk litteraturstudie och en genomgång av kommersiella system för att identifiera de unika integritetsproblemen som uppstår hos hembaserade hälsoövervakningssystem. Från studien destillerar vi en gemensam arkitektur som täcker de flesta kommersiella och akademiskt producerade system, samt en inventering av vilka problem de tar upp, deras integritetshänsyn och hur de hanterar ägarens data. Utifrån detta har vi då identifierat potentiella ställen för integritetsskydd för sådana system.

För att dela insamlad data eller en maskininlärningsmodell tränad på sådana data med andra utforksar vi huruvida syntetiskt data kan användas som ett verktyg för att uppnå en bättre avvägning mellan integritet och nytta jämfört med traditionella integritetshöjande tillvägagångssätt. Vi gör en grundlig bedömning av användbarheten av syntetiska tabelldata vad gäller korrekthet. Vår undersökning visar att ingen av de vanliga måtten för hur väl syntetisk data motsvarar originaldata kan förutsäga om, för en given univariat eller multivariat statistisk analys (när analysen inte är känd i förväg), syntetiska data uppnår nytta liknande originaldata. För maskininlärningsbaserade klassificeringsuppgifter visar dock metriken överlapp av konfidensintervaller en stark korrelation mellan hur lika maskininlärningsmodellerna (d.v.s. tränade på syntetiska vs. originaldata) presterar. När det gäller integritet utforskar vi attacker mot maskininlärningsmodeller som syftar till ta reda på om vissa (eller någons) särskilda data användes för att träna modellen. Vår forskning visar att träning på syntetisk data istället av originaldata kan avsevärt minska effektiviteten av sådana attacker. För bilddata föreslår vi en ny metod för att kvantifiera, förbättra och justera avvägningen mellan integritet och nytta jämfört med de traditionella metoderna.

Sammantaget visar vår utforskning i denna avhandling att det finns flera öppna forskningsfrågor angående integritet vid olika faser av databehandling inom intelligenta system, så som integritetsbevarande datalagring, möjliga oönskade slutsatser på grund av dataaggregering, och kvantifiering och förbätt-ring av avvägningen mellan integritet och nytta av data, för att uppnå bättre nytta på en acceptabel nivå av integritet när man delar data med andra. De identifierade integritetsproblemen och deras motsvarande lösningar som presenteras i denna avhandling kommer att hjälpa forskarsamhället att känna igen och åtgärda återstående integritetsproblem i domänen. Om problemen lösas kommer det att uppmuntra slutanvändarna att använda nya system och dra nytta av fördelarna utan att behöva oroa sig för integritet. 

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2023. p. xii, 41
Series
TRITA-EECS-AVL ; 2023:17
Keywords
Privacy, Intelligent Systems, Synthetic data, Machine Learning
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-326694 (URN)978-91-8040-582-9 (ISBN)
Public defence
2023-06-02, https://kth-se.zoom.us/j/66441177033, E2, Lindstedtsvägen 3, Floor 3, Stockholm, 09:00 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP), 7132
Note

QC 20230510

Available from: 2023-05-10 Created: 2023-05-09 Last updated: 2023-05-15Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Khan, Md Sakib NizamBuchegger, Sonja

Search in DiVA

By author/editor
Khan, Md Sakib NizamBuchegger, Sonja
By organisation
Theoretical Computer Science, TCS
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 1022 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf