SBST 2022

Paolo Tonella

Università della Svizzera Italiana, Switzerland

Deep Learning Testing

Abstract: Deep neural networks have outperformed classical techniques in domains such as natural language processing, computer vision and speech recognition. They found several real world applications, ranging from autonomous vehicles to medical diagnosis. Correspondingly, the need for testing approaches to ensure their dependability and quality has increased.

In recent years, we have seen an exponential growth in the number of research papers that address various aspects of deep learning testing. In this seminar, I will describe a selected set of core problems in the field. In particular, I will focus on the reasons why such problems differ from the corresponding, traditional testing ones. I will present some of the solutions that appeared recently in the area and I will comment on the issues ("elephants in the room") that still affect the existing approaches.

Biography: Paolo Tonella is Full Professor at the Faculty of Informatics and at the Software Institute of Università della Svizzera Italiana (USI) in Lugano, Switzerland. He is Honorary Professor at University College London, UK and he is Affiliated Fellow of Fondazione Bruno Kessler, Trento, Italy, where he has been Head of Software Engineering until mid 2018. Paolo Tonella holds an ERC Advanced grant as Principal Investigator of the project PRECRIME. Paolo Tonella wrote over 150 peer reviewed conference papers and over 50 journal papers. His H-index (according to Google scholar) is 59. He is/was in the editorial board of the ACM Transactions on Software Engineering and Methodology, of the IEEE Transactions on Software Engineering, of Empirical Software Engineering, Springer, and of the Journal of Software: Evolution and Process, Wiley. His current research interests are in software testing, in particular approaches to ensure the dependability of machine learning based systems, automated testing of cyber physical systems, and test oracle inference and improvement.

Rahul Gopinath

University of Sydney, Australia

Learning and Refining Input Grammars for Effective Fuzzing

Abstract: Fuzzing is one of the key techniques for evaluating robustness of programs against malicious inputs. To fuzz the program logic effectively, one needs the input specification of the program under fuzzing. However, such input specifications are rarely available, and even when present, can be obsolete, incomplete or incorrect leading to fuzzing blind spots. In this tutorial, I will show how to mine the input specification from a given program from the ground up, first generating sample inputs, then using such inputs to mine the program input grammar, and finally using the mined grammar to fuzz the program and find any bugs.

What should you do next once you find a bug? An input pattern rather than a particular input is likely to result in the bug, and to have any confidence in a bug fix, we should test the fix using the input pattern rather than a single input. In this talk, I will show how to abstract such input patterns corresponding to program behaviors such as bugs into a focused grammar, how to combine multiple input patterns together, and use such patterns to fuzz. The specialized grammars we generate can be used by any grammar fuzzer for precise control of produced inputs and hence the expected behavior.

Biography: Rahul Gopinath is a lecturer at the University of Sydney, Australia. He received his Ph.D. in 2017 from the Oregon State University, US. Rahul's research focus is on static and dynamic analysis of programs, especially mining input specifications, focused fuzzing, and debugging. He is one of the co-authors of the "Fuzzing Book - Tools and Techniques for Generating Software Tests". Rahul also has extensive industry experience, having worked in diverse fields such as telco, publishing, systems, and DevOps. He has also worked on empirical evaluation of the effectiveness of different coverage techniques and Mutation Analysis.