kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Skillnad i bedömning av text beroende på om texten uppges vara skriven av en människa eller genererad av ChatGPT
KTH, School of Electrical Engineering and Computer Science (EECS).
2023 (Swedish)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesisAlternative title
Difference in evaluation of text depending on if the text is stated to be written by a human or generated by ChatGPT. (English)
Abstract [en]

AI is now more often being used by individuals to write entire texts. This makes it more difficult for readers to determine who or what has written the text. In the future, it may be necessary to disclose who or what has written the text. What does this mean for how we relate to the text? Do we evaluate the text equally regardless of the author? Previous research shows that people have a certain negative bias towards AI in different situations.This study investigates whether there is a difference in how people evaluate text depending on whether the text is stated to be written by a human or generated by ChatGPT. In the study an experiment was conducted with a between-group design. The participants were 20 students from the Media Technology program at KTH who were asked to read and evaluate texts. Half of the participants were informed that the texts were written by a human, while the remaining participants were informed that the texts were generated by ChatGPT. The participants evaluated the texts by responding to twelve statements about the text's quality in a questionnaire. Data was analyzed using the Mann-Whitney U-test to identify any differences in evaluations.The results showed significant differences in evaluations for six out of 48 statements. These were in the evaluations of the statements: mature, personal, emotional, liked and good. Although not all evaluations could show significant differences, the mean values of the evaluations show that all texts were perceived as more emotional and personal by participants who were informed that the texts were written by a human. Additionally, all texts were perceived as more analytical and intelligent by participants who were informed that the texts were generated by AI. This suggests that there is a certain bias among people towards AI.

Abstract [sv]

AI används alltmer av privatpersoner för att författa hela texter. Det gör det svårare att avgöra vem eller vad som skrivit texten. I framtiden kanske man kommer att behöva redovisa vem eller vad som skrivit texten. Vad betyder det för hur vi förhåller oss till texten? Bedömer vi texten lika oavsett vem som är författare? Tidigare forskning visar att människor i olika situationer har viss negativ bias gentemot AI. I denna studie undersöks ifall det finns skillnad i hur människor bedömer text beroende på om texten uppges vara skriven av en människa eller generad av ChatGPT. I studien genomfördes ett experiment med mellangruppsdesign. Deltagarna var 20 studenter från programmet Civilingenjör inom Medieteknik på KTH som fick läsa och bedöma texter. Hälften av deltagarna fick informationen att texterna var skrivna av en människa, resterande att texterna var genererade av ChatGPT. Deltagarna utvärderade texterna genom att svara på tolv påståenden om textens kvalitet i ett frågeformulär. Data analyserades med Mann-Whitney U-test för att identifiera eventuella skillnader i bedömningarna. Resultaten visade signifikanta skillnader i bedömning för sex av 48 bedömningar. Dessa var i bedömning av påståendena mogen, personlig, känslosam, gillade och bra. Även om inte signifikanta skillnader kunde påvisas i alla jämförelser visar medelvärden att samtliga texter bedömdes vara mer känslosamma och mer personliga av deltagare som blev informerade att texterna var skrivna av en människa. Samtliga texter bedömdes även vara mer analytiska och mer intelligenta av deltagare som blev informerade att texterna var genererade av AI. Detta tyder på att det finns en viss bias hos människor gentemot AI.

Place, publisher, year, edition, pages
2023. , p. 9
Series
TRITA-EECS-EX ; 2023:221
Keywords [en]
Bias, AI, Chat gpt
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-330732OAI: oai:DiVA.org:kth-330732DiVA, id: diva2:1778290
Supervisors
Examiners
Available from: 2023-09-25 Created: 2023-06-30 Last updated: 2023-10-26Bibliographically approved

Open Access in DiVA

No full text in DiVA

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 245 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf