kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Automatic Description of AI Patches With AST Analysis
KTH, School of Electrical Engineering and Computer Science (EECS).
KTH, School of Electrical Engineering and Computer Science (EECS).
2023 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

Using artificial intelligence (AI) to write source code is soon to be reality. One research areais to use AI to automatically generate a patch that fixes a bug. However, most such tools onlygenerate the patch without any natural language description of it, making it difficult for thereviewer to understand the patch. Therefore in our work, we aim to develop a program thatgenerates descriptions of code patches generated by artificial intelligence. The program takestwo Java files as input and analyses the differences in the abstract syntax tree representationsof the code. First, the program finds what changes have been made and where those changesare located. Then, it generates a description of the change. The program has been tested on 50patches from open-source projects from GitHub. The generated comments were comparedwith comments generated by ChatGPT as well as those written by humans in a survey. Thesurvey also asked the participants to rate how useful the generated comments were in helpingthem understand the patches generated by artificial intelligence. The result shows that thecomments generated with our approach received 34.7 % of the participant votes whereas theexisting human-written descriptions received 20.8 % of the votes, although the generateddescriptions were sometimes thought to be too long when describing simple changes. Theprogram-generated descriptions were also rated a 3.09 out of 5 on average in how helpfulthey were in understanding patches generated by artificial intelligence.

Abstract [sv]

Användandet av artificiell intelligens (AI) för att skriva källkod börjar bli verklighet. Ettforskningsområde är användandet av AI för att automatiskt generera en programkorrigering till en bugg. Många av dessa verktyg genererar dock programkorrigeringar utan någonbeskrivning på naturligt språk, vilket gör det svårt för granskare att förståprogramkorrigeringen. Därför syftar detta projekt till att utveckla ett program som genererarbeskrivningar av programkorrigeringar genererade av artificiell intelligens. Programmet tartvå Java-filer som indata och analyserar skillnaden mellan de abstraktasyntaxträdsrepresentationerna av dessa. Först hittar programmet vilka ändringar som hargjorts och var i programmet dessa finns. Därefter genereras en beskrivning av ändringen.Programmet har testats på 50 programkorrigeringar från projekt med öppen källkod påGitHub. De genererade kommentarerna jämfördes med kommentarer genererade avChatGPT, utöver kommentarer skrivna av människor, i en enkät. Deltagarna i enkäten fickockså bedöma hur hjälpsamma de genererade kommentarerna var för att kunna förståprogramkorrigeringar skapade av artificiell intelligens. Resultatet visar att beskrivningarnagenerarade med vårt tillvägagångssätt fick 34.7 % av deltagarnas röster medan de redanexisterande beskrivningarna skrivna av människor fick 20.8 % av rösterna, men degenererade kommentarerna ansågs ibland vara för långa när små ändringar skulle beskrivas.De genererade beskrivningarna rankades som 3.09 av 5 i hjälpsamhet för att förståprogramkorrigeringar genererade av artificiell intelligens.

Place, publisher, year, edition, pages
2023. , p. 719-728
Series
TRITA-EECS-EX ; 2023:199
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:kth:diva-341790OAI: oai:DiVA.org:kth-341790DiVA, id: diva2:1823519
Supervisors
Examiners
Projects
Kandidatexjobb i elektroteknik 2023, KTH, StockholmAvailable from: 2024-01-02 Created: 2024-01-02 Last updated: 2024-01-12

Open Access in DiVA

fulltext(211487 kB)349 downloads
File information
File name FULLTEXT01.pdfFile size 211487 kBChecksum SHA-512
69786101c351a58f7bd524c3aeee40c661028b577366c4a725033372b88c624c87c2183b6acca2d3d43bbd2bb2f3942326c69263e70c99cf1db027ce9c4e9ae2
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 349 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 300 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf