Dear AI, Which Tests Should We Run Now?

Teams have to test more and more functionality in less and less time, as successful software grows with each release, but release cycles are getting shorter and shorter. Many teams that Dr. Elmar Juergens works with manage thousands or even hundreds of thousands of tests across various stages and phases of testing. Typically, there are far too many tests to execute fully for each release, let alone during continuous integration. But how can one select the right tests to run at any given moment? Which tests will uncover the most bugs with minimal effort and within the available time? Dr. Juergens spent the last decade working on this challenge. In recent years, numerous AI-driven approaches have been proposed to address this issue. His team has implemented and tested these approaches, both with and without AI, in their own development processes and in collaboration with customer teams. In this talk, he will outline the best approaches, including test-impact-analysis, pareto-optimization, predictive test selection, AI clustering and similarity scoring of tests and code changes. He will share their failures and successes, and discuss which approach works best in which context.