Tool Competition
This year as well we are pleased to announce the tenth edition of the testing tool competition. The competition has the goal to experiment with testing tools for a diversified set of traditional and emerging systems and domains.
Java tool competition: As for recent years, we invite researchers to participate in the competition with their unit test generation tool for Java. Tools will be evaluated against a benchmark with respect to code coverage and mutation score.
Cyber-physical systems (CPS) testing competition: In addition to the traditional Java tool competition, we also organize a CPS testing competition on self-driving cars simulation environments. Specifically, in collaboration with the BeamNG research team, this competition focuses on the generation of scenarios using BeamNG self-driving cars simulator.
For future editions of the SBST Tool competition we foresee the addition of a further "Python testing tool competition" where:
- an infrastructure will be provided to generate the tools
- it is conducted in an evaluation setting similar to the one performed for the traditional Java testing tool competition
How to participate in the two competitions?
To participate:
- notify the organizers by sending an email to Fiorella Zampetti, Alessio Gambi, Gunel Jahangirova, and Vincenzo Riccio (fiorella.zampetti@unisannio.it, alessio.gambi@uni-passau.de, gunel.jahangirova@usi.ch, vincenzo.riccio@usi.ch).
- check out the contest infrastructure and make sure your tool works well within the infrastructure.
Java Infrastructure:
The infrastructure concerning the Java tool competition is available on GitHub and can also be tried using Docker: https://github.com/JUnitContest/junitcontest. It is important to note that some changes are expected to the infrastructure during the end of 2021 and during the beginning of 2022.
CPS Infrastructure:
The infrastructure concerning the CPS tool competition can be found here: https://github.com/se2p/tool-competition-av.
Important Dates
- Tool submission: Friday January 21
- Benchmark results communicated to authors: Friday February 25
- Submission of camera-ready paper: Sunday March 18