STAREAST 2021 Concurrent Session : Testing Uncertainty for a Conversational AI


Thursday, May 7, 2020 - 1:30pm to 2:30pm

Testing Uncertainty for a Conversational AI

Uncertainty has always been a key challenge for testers. But testing a chatbot adds a completely new level of uncertainty. There are a lot of platforms and tools available for chatbot development, but what we lack is a standardized chatbot testing strategy. The way testing is performed on chatbots differs a lot from "traditional" testing (like for an app or web portal) due to the apparent randomness of a conversation with a chatbot. From testing numerous clients' chatbots. Rajni Singh has experienced that it is impossible to anticipate all the situations that can happen during a conversation. As they introduced learning components to conversational ai such as machine learning and intent training, the chatbot evolved and changed its behavior compared to previous test runs. This increased the need for regression tests and complicated them at the same time. Rajni will talk about the challenges faced during chatbot testing and how to mitigate them with different strategies. She'll share her experience using their own advanced automation framework as well as commercial and open source tools.

Rajni Singh

Rajni has twelve years of experience in the field of software testing and is currently a manager in quality assurance. She is a dynamic, self-driven, results-oriented leader with an excellent technical ability to deliver seamless solutions. She possesses excellent management and leadership skills with a focus on continuous improvement and client-centricity. With an understanding of various domains to deliver value to customers, her continuous focus is on digital testing, current industry testing trends like blockchain, IoT, chatbot, AI/ML, and agile transformation.