Automating the Testing of AI/ML Models: Tools, Skills, and Best Practices

As AI and ML models become integral to software systems, ensuring their accuracy and reliability through effective testing is paramount. Traditional testing approaches often fall short in addressing the unique challenges posed by these models, such as handling large datasets, verifying model predictions, and maintaining robustness against data drift. This presentation explores Otis Elevator's journey in automating the testing of AI/ML models using cutting-edge tools and techniques. Ayisha Tabbassum's team faced significant hurdles in manually testing their AI models, from the sheer volume of data to the complexity of model behaviors. To address these, they adopted a suite of automation tools, including TensorFlow Extended (TFX), Apache Airflow, and MLflow. These tools enabled data validation automation, model training, evaluation, and monitoring processes, and ensuring consistency and reliability across AI/ML pipelines. Attendees will gain insights into the specific challenges encountered and how the team overcame them through automation. Ayisha will share practical examples of test cases for AI models, strategies for integrating these tools into existing workflows, and the essential skills required for teams to successfully adopt AI/ML test automation. Key takeaways will include best practices for setting up automated testing frameworks for AI models and tips for upskilling team members to proficiently use these tools.