STAREAST 2020 Concurrent Session : Setting Your Tests Up to Fail


Wednesday, May 6, 2020 - 2:45pm to 3:45pm

Setting Your Tests Up to Fail

It's a fact of life that we often have to write automated UI tests for features that have defects, or that interact with 3rd party APIs that aren't returning the right responses, or for items that we know aren't working right. When the team has decided that the behavior isn't going to be fixed, what's an automation engineer to do? Let the tests fail? Not write them? Champion harder for the defects? Jenny suggests writing your tests to pass. By creating tests that pass on the current expected behavior (the defect), we are in a perfect position to tell when the defect is resolved, or the api is returning the correct information, or any of the other error cases we may be encountering. This prevents failure fatigue (from seeing a test always fail), while still providing meaningful, actionable information out of our test suite. Jenny will discuss her experiences with several cases that she was able to use this method and how it kept the rest of the team informed through TODOs, Jira stories, and documentation. And--of course!--what to do when your test finally fails. You'll leave with a clear understanding of when to design your tests to fail, how to use TODOs, and other indicators to let the rest of the team know what's going on, and you’ll also hear a frank discussion of automation as information­–not as a bug detection system!

WillowTree Apps

Jenny Bramble came up through support and DevOps, cutting her teeth on that interesting role that acts as the 'translator' between customer requests from support and the development team before diving headlong into her career as a tester. Her love of support and the human side of problems lets her find a sweet spot between empathy for the user and empathy for her team. She's done testing, support, or human interfacing for most of her career. She finds herself happiest when she's making an impact on other people--whether it's helping find issues in applications, speaking at events, or just grabbing coffee and chatting.