kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A Comprehensive Study of Automatic Program Repair on the QuixBugs Benchmark
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Theoretical Computer Science, TCS.ORCID iD: 0000-0003-4807-2110
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Theoretical Computer Science, TCS.ORCID iD: 0000-0003-3505-3383
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Automatic program repair papers tend to repeatedly use the same benchmarks. This poses a threat to the external validity of the findings of the program repair research community. In this paper, we perform an automatic repair experiment on a benchmark called QuixBugs that has been recently published. This benchmark has never been studied in the context of program repair. In this study, we report on the characteristics of QuixBugs, and we design and perform an experiment about the effectiveness of test-suite based program repair on QuixBugs. We study two repair systems, Astor and Nopol, which are representatives of generate-and-validate repair technique and synthesis repair technique respectively. We propose three patch correctness assessment techniques to comprehensively study overfitting and incorrect patches. Our key results are: 1) 13/40 buggy programs in the QuixBugs can be repaired with a test-suite adequate patch; 2) a total of 22 different plausible patches for those 13 buggy programs in the QuixBugs are present in the search space of the considered tools; 3) the three patch assessment techniques discard in total 12/22 patches that are overfitting. This sets a baseline for future research of automatic repair on QuixBugs. Our experiment also highlights the major properties and challenges of how to perform automated correctness assessment of program repair patches. All experimental results are publicly available on Github in order to facilitate future research on automatic program repair.

National Category
Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-239890OAI: oai:DiVA.org:kth-239890DiVA, id: diva2:1268031
Note

QC 20181206

Available from: 2018-12-04 Created: 2018-12-04 Last updated: 2023-02-08Bibliographically approved
In thesis
1. Improving the Precision of Automatic Program Repair with Machine Learning
Open this publication in new window or tab >>Improving the Precision of Automatic Program Repair with Machine Learning
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Automatic program repair as a research field aims to eliminate software bugs and vulnerabilities in an automatic manner. Automatic program repair holds great promise to reduce the debugging cost and increase the productivity of software development. Test suite is one of the most widely used specifications for automatic program repair to specify correct program behavior and guide patch generation. However, test suite is an incomplete specification with limited input-output data. This results in automatic program repair generating patches that merely satisfy test suite specifications, yet fail to repair buggy programs in general. The generation of a great number of incorrect patches leads to a low precision of automatic program repair.

In this thesis, we focus on improving the precision of automatic program repair from three perspectives: patch generation, patch assessment in practice, and patch assessment for scientific usage. This thesis makes contributions to the following in automatic program repair. First of all, to increase the precision of patch generation, we propose two learning-based automatic program repair approaches to encourage the generation of more correct patches with fewer candidate patches. Second, to increase the precision of patch assessment in practice, we propose to build a probabilistic model based on static code features to discard incorrect patches and thus increase the ratio of correct patches to all generated patches. Third, to increase the patch assessment precision for scientific usage, we propose to use automatically generated test cases to discard incorrect patches.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2023. p. 97
Series
TRITA-EECS-AVL ; 2023:10
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-323295 (URN)978-91-8040-469-3 (ISBN)
Public defence
2023-02-24, https://kth-se.zoom.us/j/63393781380, Kollegiesalen, Brinellvägen 6, Stockholm, 13:30 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20230208

Available from: 2023-02-08 Created: 2023-02-07 Last updated: 2026-03-25Bibliographically approved

Open Access in DiVA

fulltext(426 kB)597 downloads
File information
File name FULLTEXT01.pdfFile size 426 kBChecksum SHA-512
336bc010d4aea5142877ada131095960f667633097aaef45cc61474b38d4251f3b6cf900bb28e7bbe1cc90628563dbdf9f8e610c131d3ac9a1a792d3722df89c
Type fulltextMimetype application/pdf

Authority records

Ye, HeMonperrus, Martin

Search in DiVA

By author/editor
Ye, HeMonperrus, Martin
By organisation
Theoretical Computer Science, TCS
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 598 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 428 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf