STAREAST 2018 Concurrent Session : Migrating from Test Cases to Real-World Telemetry Measures

SEE PRICING & PACKAGES

Wednesday, May 2, 2018 - 11:30am to 12:30pm

Migrating from Test Cases to Real-World Telemetry Measures

Add to calendar

Ken Johnston sees today’s software ecosystem in the light of Everything as a Service (EaaS). Operating systems like Windows, Android, and Chrome OS all ship regularly like a service. Browsers automatically update every few weeks, and apps are constantly updating through all the app stores. Although getting a test to pass once and signing off has gone by the wayside for software testing, still we run test cases over and over again. Ken shares how Microsoft took millions of test cases—yes, actually millions—and turned the important ones into measures based on real world telemetry. Massive amounts of data coming in from real devices and real users measure product quality and tie it to key customer satisfaction metrics. Join Ken to learn how your organization can shift from internal regression and other testing to dynamic telemetry from the real world and how you can identify just the right number of measures that truly matter.

Ken Johnston
Microsoft

A principal manager on the Microsoft core data science team, Ken Johnston and his team focus their research on Windows post sales monetization and device usage within the commercial and education segments. Since joining Microsoft in 1998, Ken has held roles of group program manager for Bing Data Quality and Measurements, group manager for Bing Shopping and Data Operations, test lead and test manager on MSN, and director of test excellence. Ken coauthored How We Test Software at Microsoft and is a contributing author to Experiences of Test Automation: Case Studies of Software Test Automation. Contact Ken on Twitter.