Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 22) Show all publications
Vera-Perez, O. L., Danglot, B., Monperrus, M. & Baudry, B. (2019). A comprehensive study of pseudo-tested methods. Journal of Empirical Software Engineering, 24(3), 1195-1225
Open this publication in new window or tab >>A comprehensive study of pseudo-tested methods
2019 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 24, no 3, p. 1195-1225Article in journal (Refereed) Published
Abstract [en]

Pseudo-tested methods are defined as follows: they are covered by the test suite, yet no test case fails when the method body is removed, i.e., when all the effects of this method are suppressed. This intriguing concept was coined in 2016, by Niedermayr and colleagues, who showed that such methods are systematically present, even in well-tested projects with high statement coverage. This work presents a novel analysis of pseudo-tested methods. First, we run a replication of Niedermayr's study with 28K+ methods, enhancing its external validity thanks to the use of new tools and new study subjects. Second, we perform a systematic characterization of these methods, both quantitatively and qualitatively with an extensive manual analysis of 101 pseudo-tested methods. The first part of the study confirms Niedermayr's results: pseudo-tested methods exist in all our subjects. Our in-depth characterization of pseudo-tested methods leads to two key insights: pseudo-tested methods are significantly less tested than the other methods; yet, for most of them, the developers would not pay the testing price to fix this situation. This calls for future work on targeted test generation to specify those pseudo-tested methods without spending developer time.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Software testing, Software developers, Pseudo-tested methods, Test quality, Program analysis
National Category
Software Engineering
Identifiers
urn:nbn:se:kth:diva-255215 (URN)10.1007/s10664-018-9653-2 (DOI)000472043800005 ()2-s2.0-85053658099 (Scopus ID)
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20190903

Available from: 2019-09-03 Created: 2019-09-03 Last updated: 2019-09-13Bibliographically approved
Harrand, N., Allier, S., Rodriguez-Cancio, M., Monperrus, M. & Baudry, B. (2019). A journey among Java neutral program variants. Genetic Programming and Evolvable Machines, 20(4), 531-580
Open this publication in new window or tab >>A journey among Java neutral program variants
Show others...
2019 (English)In: Genetic Programming and Evolvable Machines, ISSN 1389-2576, E-ISSN 1573-7632, Vol. 20, no 4, p. 531-580Article in journal (Refereed) Published
Abstract [en]

Neutral program variants are alternative implementations of a program, yet equivalent with respect to the test suite. Techniques such as approximate computing or genetic improvement share the intuition that potential for enhancements lies in these acceptable behavioral differences (e.g., enhanced performance or reliability). Yet, the automatic synthesis of neutral program variants, through program transformations remains a key challenge. This work aims at characterizing plastic code regions in Java programs, i.e., the code regions that are modifiable while maintaining functional correctness, according to a test suite. Our empirical study relies on automatic variations of 6 real-world Java programs. First, we transform these programs with three state-of-the-art program transformations: add, replace and delete statements. We get a pool of 23,445 neutral variants, from which we gather the following novel insights: developers naturally write code that supports fine-grain behavioral changes; statement deletion is a surprisingly effective program transformation; high-level design decisions, such as the choice of a data structure, are natural points that can evolve while keeping functionality. Second, we design 3 novel program transformations, targeted at specific plastic regions. New experiments reveal that respectively 60%, 58% and 73% of the synthesized variants (175,688 in total) are neutral and exhibit execution traces that are different from the original.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Neutral program variant, Program transformation, Java, Code plasticity
National Category
Software Engineering
Identifiers
urn:nbn:se:kth:diva-264174 (URN)10.1007/s10710-019-09355-3 (DOI)000492843200004 ()2-s2.0-85068185414 (Scopus ID)
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Swedish Foundation for Strategic Research , trustfull
Note

QC 20191122

Available from: 2019-11-22 Created: 2019-11-22 Last updated: 2019-11-29Bibliographically approved
Danglot, B., Vera-Perez, O., Yu, Z., Zaidman, A., Monperrus, M. & Baudry, B. (2019). A snowballing literature study on test amplification. Journal of Systems and Software, 157, Article ID UNSP 110398.
Open this publication in new window or tab >>A snowballing literature study on test amplification
Show others...
2019 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 157, article id UNSP 110398Article in journal (Refereed) Published
Abstract [en]

The adoption of agile approaches has put an increased emphasis on testing, resulting in extensive test suites. These suites include a large number of tests, in which developers embed knowledge about meaningful input data and expected properties as oracles. This article surveys works that exploit this knowledge to enhance manually written tests with respect to an engineering goal (e.g., improve coverage or refine fault localization). While these works rely on various techniques and address various goals, we believe they form an emerging and coherent field of research, which we coin "test amplification". We devised a first set of papers from DBLP, searching for all papers containing "test" and "amplification" in their title. We reviewed the 70 papers in this set and selected the 4 papers that fit the definition of test amplification. We use them as the seeds for our snowballing study, and systematically followed the citation graph. This study is the first that draws a comprehensive picture of the different engineering goals proposed in the literature for test amplification. We believe that this survey will help researchers and practitioners entering this new field to understand more quickly and more deeply the intuitions, concepts and techniques used for test amplification.

Place, publisher, year, edition, pages
Elsevier, 2019
Keywords
Test amplification, Test augmentation, Test optimization, Test regeneration, Automatic testing
National Category
Computer and Information Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-262765 (URN)10.1016/j.jss.2019.110398 (DOI)000488137500009 ()2-s2.0-85070967135 (Scopus ID)
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20191022

Available from: 2019-10-22 Created: 2019-10-22 Last updated: 2019-11-26Bibliographically approved
Rodriguez-Cancio, M., Combemale, B. & Baudry, B. (2019). Approximate Loop Unrolling. In: CF '19 - PROCEEDINGS OF THE 16TH ACM INTERNATIONAL CONFERENCE ON COMPUTING FRONTIERS: . Paper presented at 16th ACM International Conference on Computing Frontiers, CF 2019; Alghero, Sardinia; Italy; 30 April 2019 through 2 May 2019 (pp. 94-105). ASSOC COMPUTING MACHINERY
Open this publication in new window or tab >>Approximate Loop Unrolling
2019 (English)In: CF '19 - PROCEEDINGS OF THE 16TH ACM INTERNATIONAL CONFERENCE ON COMPUTING FRONTIERS, ASSOC COMPUTING MACHINERY , 2019, p. 94-105Conference paper, Published paper (Refereed)
Abstract [en]

We introduce Approximate Unrolling, a compiler loop optimization that reduces execution time and energy consumption, exploiting code regions that can endure some approximation and still produce acceptable results. Specifically, this work focuses on counted loops that map a function over the elements of an array. Approximate Unrolling transforms loops similarly to Loop Unrolling. However, unlike its exact counterpart, our optimization does not unroll loops by adding exact copies of the loop's body. Instead, it adds code that interpolates the results of previous iterations.

Place, publisher, year, edition, pages
ASSOC COMPUTING MACHINERY, 2019
Keywords
approximate computing, compiler optimizations
National Category
Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-255520 (URN)10.1145/3310273.3323841 (DOI)000474686400013 ()2-s2.0-85066046859 (Scopus ID)
Conference
16th ACM International Conference on Computing Frontiers, CF 2019; Alghero, Sardinia; Italy; 30 April 2019 through 2 May 2019
Note

QC 20190819

Available from: 2019-08-19 Created: 2019-08-19 Last updated: 2019-08-19Bibliographically approved
Danglot, B., Vera-Perez, O. L., Baudry, B. & Monperrus, M. (2019). Automatic test improvement with DSpot: a study with ten mature open-source projects. Journal of Empirical Software Engineering, 24(4), 2603-2635
Open this publication in new window or tab >>Automatic test improvement with DSpot: a study with ten mature open-source projects
2019 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 24, no 4, p. 2603-2635Article in journal (Refereed) Published
Abstract [en]

In the literature, there is a rather clear segregation between manually written tests by developers and automatically generated ones. In this paper, we explore a third solution: to automatically improve existing test cases written by developers. We present the concept, design and implementation of a system called DSpot, that takes developer-written test cases as input (JUnit tests in Java) and synthesizes improved versions of them as output. Those test improvements are given back to developers as patches or pull requests, that can be directly integrated in the main branch of the test code base. We have evaluated DSpot in a deep, systematic manner over 40 real-world unit test classes from 10 notable and open-source software projects. We have amplified all test methods from those 40 unit test classes. In 26/40 cases, DSpot is able to automatically improve the test under study, by triggering new behaviors and adding new valuable assertions. Next, for ten projects under consideration, we have proposed a test improvement automatically synthesized by DSpot to the lead developers. In total, 13/19 proposed test improvements were accepted by the developers and merged into the main code base. This shows that DSpot is capable of automatically improving unit-tests in real-world, large scale Java software.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Test improvement, Junit test, Pull request empirical study
National Category
Software Engineering
Identifiers
urn:nbn:se:kth:diva-256259 (URN)10.1007/s10664-019-09692-y (DOI)000477582700025 ()2-s2.0-85064832264 (Scopus ID)
Funder
Swedish Foundation for Strategic Research , trustfull
Note

QC 20191025

Available from: 2019-10-25 Created: 2019-10-25 Last updated: 2019-11-14Bibliographically approved
Alferez, M., Acher, M., Galindo, J. A., Baudry, B. & Benavides, D. (2019). Modeling variability in the video domain: language and experience report. Software quality journal, 27(1), 307-347
Open this publication in new window or tab >>Modeling variability in the video domain: language and experience report
Show others...
2019 (English)In: Software quality journal, ISSN 0963-9314, E-ISSN 1573-1367, Vol. 27, no 1, p. 307-347Article in journal (Refereed) Published
Abstract [en]

In an industrial project, we addressed the challenge of developing a software-based video generator such that consumers and providers of video processing algorithms can benchmark them on a wide range of video variants. This article aims to report on our positive experience in modeling, controlling, and implementing software variability in the video domain. We describe how we have designed and developed a variability modeling language, called VM, resulting from the close collaboration with industrial partners during 2 years. We expose the specific requirements and advanced variability constructs; we developed and used to characterize and derive variations of video sequences. The results of our experiments and industrial experience show that our solution is effective to model complex variability information and supports the synthesis of hundreds of realistic video variants. From the software language perspective, we learned that basic variability mechanisms are useful but not enough; attributes and multi-features are of prior importance; meta-information and specific constructs are relevant for scalable and purposeful reasoning over variability models. From the video domain and software perspective, we report on the practical benefits of a variability approach. With more automation and control, practitioners can now envision benchmarking video algorithms over large, diverse, controlled, yet realistic datasets (videos that mimic real recorded videos)-something impossible at the beginning of the project.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Variability modeling, Feature modeling, Software product line engineering, Configuration, Automated reasoning, Domain-specific languages, Video testing
National Category
Software Engineering
Identifiers
urn:nbn:se:kth:diva-249887 (URN)10.1007/s11219-017-9400-8 (DOI)000462236000009 ()2-s2.0-85043359422 (Scopus ID)
Note

QC 20190426

Available from: 2019-04-26 Created: 2019-04-26 Last updated: 2019-04-26Bibliographically approved
Laperdrix, P., Avoine, G., Baudry, B. & Nikiforakis, N. (2019). Morellian analysis for browsers: Making web authentication stronger with canvas fingerprinting. In: Roberto Perdisci, Clémentine Maurice, Giorgio Giacinto, Magnus Almgren (Ed.), Detection of Intrusions and Malware, and Vulnerability Assessment: 16th International Conference, DIMVA 2019, Gothenburg, Sweden, June 19–20, 2019, Proceedings. Paper presented at 16th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA 2019, Gothenburg, Sweden, 19-20 June 2019 (pp. 43-66). Springer Verlag
Open this publication in new window or tab >>Morellian analysis for browsers: Making web authentication stronger with canvas fingerprinting
2019 (English)In: Detection of Intrusions and Malware, and Vulnerability Assessment: 16th International Conference, DIMVA 2019, Gothenburg, Sweden, June 19–20, 2019, Proceedings / [ed] Roberto Perdisci, Clémentine Maurice, Giorgio Giacinto, Magnus Almgren, Springer Verlag , 2019, p. 43-66Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we present the first fingerprinting-based authentication scheme that is not vulnerable to trivial replay attacks. Our proposed canvas-based fingerprinting technique utilizes one key characteristic: it is parameterized by a challenge, generated on the server side. We perform an in-depth analysis of all parameters that can be used to generate canvas challenges, and we show that it is possible to generate unique, unpredictable, and highly diverse canvas-generated images each time a user logs onto a service. With the analysis of images collected from more than 1.1 million devices in a real-world large-scale experiment, we evaluate our proposed scheme against a large set of attack scenarios and conclude that canvas fingerprinting is a suitable mechanism for stronger authentication on the web.

Place, publisher, year, edition, pages
Springer Verlag, 2019
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 11543
Keywords
Authentication, Computer crime, Malware, Attack scenarios, Authentication scheme, Fingerprinting techniques, In-depth analysis, Key characteristics, Large scale experiments, Parameterized, Web authentication, Image analysis
National Category
Information Systems
Identifiers
urn:nbn:se:kth:diva-262441 (URN)10.1007/978-3-030-22038-9_3 (DOI)2-s2.0-85067800996 (Scopus ID)9783030220372 (ISBN)
Conference
16th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA 2019, Gothenburg, Sweden, 19-20 June 2019
Note

QC 20191104

Available from: 2019-11-04 Created: 2019-11-04 Last updated: 2019-11-04Bibliographically approved
Cabrera Arteaga, J., Monperrus, M. & Baudry, B. (2019). Scalable comparison of JavaScript V8 bytecode traces. In: : . Paper presented at VMIL 2019, SPLASH (pp. 22-31). New York, NY, USA, Article ID 3361228.
Open this publication in new window or tab >>Scalable comparison of JavaScript V8 bytecode traces
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

The comparison and alignment of runtime traces are essential, e.g., for semantic analysis or debugging. However, naive sequence alignment algorithms cannot address the needs of the modern web: (i) the bytecode generation process of V8 is not deterministic; (ii) bytecode traces are large.

We present STRAC, a scalable and extensible tool tailored to compare bytecode traces generated by the V8 JavaScript engine. Given two V8 bytecode traces and a distance function between trace events, STRAC computes and provides the best alignment. The key insight is to split access between memory and disk. STRAC can identify semantically equivalent web pages and is capable of processing huge V8 bytecode traces whose order of magnitude matches today's web like https://2019.splashcon.org, which generates approx. 150k of V8 bytecode instructions.

Place, publisher, year, edition, pages
New York, NY, USA: , 2019
National Category
Computer Systems
Identifiers
urn:nbn:se:kth:diva-262911 (URN)10.1145/3358504.3361228 (DOI)978-1-4503-6987-9 (ISBN)
Conference
VMIL 2019, SPLASH
Funder
Swedish Foundation for Strategic Research , 3066Swedish Foundation for Strategic Research , Trustfull
Note

QC 20191028

Available from: 2019-10-24 Created: 2019-10-24 Last updated: 2019-10-28Bibliographically approved
Halin, A., Nuttinck, A., Acher, M., Devroey, X., Perrouin, G. & Baudry, B. (2019). Test them all, is it worth it?: Assessing configuration sampling on the JHipster Web development stack. Journal of Empirical Software Engineering, 24(2), 674-717
Open this publication in new window or tab >>Test them all, is it worth it?: Assessing configuration sampling on the JHipster Web development stack
Show others...
2019 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 24, no 2, p. 674-717Article in journal (Refereed) Published
Abstract [en]

Many approaches for testing configurable software systems start from the same assumption: it is impossible to test all configurations. This motivated the definition of variability-aware abstractions and sampling techniques to cope with large configuration spaces. Yet, there is no theoretical barrier that prevents the exhaustive testing of all configurations by simply enumerating them if the effort required to do so remains acceptable. Not only this: we believe there is a lot to be learned by systematically and exhaustively testing a configurable system. In this case study, we report on the first ever endeavour to test all possible configurations of the industry-strength, open source configurable software system JHipster, a popular code generator for web applications. We built a testing scaffold for the 26,000+ configurations of JHipster using a cluster of 80 machines during 4 nights for a total of 4,376 hours (182 days) CPU time. We find that 35.70% configurations fail and we identify the feature interactions that cause the errors. We show that sampling strategies (like dissimilarity and 2-wise): (1) are more effective to find faults than the 12 default configurations used in the JHipster continuous integration; (2) can be too costly and exceed the available testing budget. We cross this quantitative analysis with the qualitative assessment of JHipster's lead developers.

Place, publisher, year, edition, pages
SPRINGER, 2019
Keywords
Configuration sampling, Variability-intensive system, Software testing, JHipster, Case study
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-249802 (URN)10.1007/s10664-018-9635-4 (DOI)000462654200005 ()2-s2.0-85049998068 (Scopus ID)
Note

QC 20190424

Available from: 2019-04-24 Created: 2019-04-24 Last updated: 2019-04-24Bibliographically approved
Soto Valero, C., Benelallam, A., Harrand, N., Barais, O. & Baudry, B. (2019). The Emergence of Software Diversity inMaven Central. In: 16th International Conference on Mining Software Repositories: . Paper presented at MSR (pp. 333-343). Montréal, QC, Canada: IEEE conference proceedings
Open this publication in new window or tab >>The Emergence of Software Diversity inMaven Central
Show others...
2019 (English)In: 16th International Conference on Mining Software Repositories, Montréal, QC, Canada: IEEE conference proceedings, 2019, p. 333-343Conference paper, Published paper (Refereed)
Abstract [en]

Maven artifacts are immutable: an artifact that is uploaded on Maven Central cannot be removed nor modified. The only way for developers to upgrade their library is to releasea new version. Consequently, Maven Central accumulates all the versions of all the libraries that are published there, and applications that declare a dependency towards a library can pick any version. In this work, we hypothesize that the immutability of Maven artifacts and the ability to choose any version naturally support the emergence of software diversity within Maven Central. We analyze 1,487,956 artifacts that represent all the versions of 73,653 libraries. We observe that more than 30% of libraries have multiple versions that are actively used by latest artifacts. In the case of popular libraries, more than 50% of their versions are used. We also observe that more than 17% of libraries have several versions that are significantly more used than the other versions. Our results indicate that the immutability of artifacts in Maven Central does support a sustained level of diversity among versions of libraries in the repository.

Place, publisher, year, edition, pages
Montréal, QC, Canada: IEEE conference proceedings, 2019
Keywords
Maven Central, Software Diversity, LibraryVersions, Evolution, Open-Source Software
National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-254553 (URN)10.1109/MSR.2019.00059 (DOI)
Conference
MSR
Note

QC 20190802

Available from: 2019-07-01 Created: 2019-07-01 Last updated: 2019-08-02Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-4015-4640

Search in DiVA

Show all publications