kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
AI-assisted Image Manipulation with Eye Tracking
KTH, School of Electrical Engineering and Computer Science (EECS).
KTH, School of Electrical Engineering and Computer Science (EECS).
2023 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesisAlternative title
Bildbehandling med Eye Tracking och AI (Swedish)
Abstract [en]

Image editing tools can pose a challenge for motor impaired individuals who wish to perform image manipulation. The process includes many steps and can be difficult given a lack of tactile input such as mouse and keyboard. To increase the availability of image editing for motor impaired individuals, the potential for new tools and modalities have to be explored. In this project, a prototype was developed, which allows the user to edit images using eye tracking and deep learning models, specifically the DALL-E 2 model. This prototype was then tested on users who rated its functionality based on a set of human-computer interaction principles. The quality of the results varied a lot depending on the eye movements of the user, and the provided prompts. The results of the user testing found that there was potential for an editing tool implementing eye tracking and AI assistance, but that it requires further iteration and time to learn how to use. Most users enjoyed the experience of using the prototype and felt that continued experimentation would lead to improved results.

Abstract [sv]

Användandet av bildbehandlingsverktyg kan för någon med motoriska svårigheter, specifikt de utan möjlighet att använda sina händer, innebära flera svårigheter. Processen omfattas av många steg som kan vara särskilt besvärliga utan användningen av mus och tangentbord. För att öka tillgängligheten av dessa verktyg behöver nya system utforskas, till exempel sådana som använder AI system. I denna studie utvärderas ett sådant system, för vilken en prototyp utvecklades. Prototypen låter användaren redigera bilder med hjälp av eye tracking och maskininlärningsmodellen DALL-E 2. Deltagarna i studien utvärderade funktionaliteten baserat på utvalda människa-datorinteraktionsprinciper. Resultaten av utvärderingen skiljde sig en del, till stor del grundat i ögonrörelserna av användaren och den givna ändringsbeskrivningen. Resultaten visade på att det fanns potential för ett bildbehandlingsverktyg som implementerar både AI och eye tracking men att det krävs mer tid och iterering för användaren att lära sig modellen. Användare fann överlag ett nöje i att använda programmet och upplevde att de skulle kunna presterat bättre resultat om de fick mer tid att experimentera.

Place, publisher, year, edition, pages
2023. , p. 33
Series
TRITA-EECS-EX ; 2023:337
Keywords [en]
Human-computer interaction, Accessibility, Assistive technologies, Eye tracking, Input modalities, Image editing, Generative models
Keywords [sv]
Människa–datorinteraktion, Tillgänglighet, Eye Tracking, Inmatningsenheter, Bildredigering, Maskininlärningsmodeller
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-331009OAI: oai:DiVA.org:kth-331009DiVA, id: diva2:1779806
Supervisors
Examiners
Available from: 2023-08-01 Created: 2023-07-04 Last updated: 2023-08-01Bibliographically approved

Open Access in DiVA

fulltext(13580 kB)1212 downloads
File information
File name FULLTEXT01.pdfFile size 13580 kBChecksum SHA-512
f2fb33f295f35dfb0da5b8ca608966186da17527489714bab38f20625394f23c332f2d399db9eaba57e090db650669811cd81d2315f4c03eb339690dab81ff12
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 1213 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 640 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf