Targeting Quality 2019

There are two main challenges to testing systems that incorporate elements of artificial intelligence (AI). First, the same input can trigger different responses as an AI system learns and adapts to new conditions, and second, it is difficult to understand what the correct response really should be. Such behavior violates one of the main principles of traditional testing: the repeatability of test case execution. Testers lose confidence in the outcome of their testing when traditional approaches no longer apply. Yury Makedonov explains how testing can be improved if we have direct access to the system’s state (grey box testing). He provides a demonstration of a simple “machine learning” system to show some grey box testing techniques. In the second part of his presentation Yury discusses test data challenges using a “pattern recognition” model as an example. These data handling techniques can be used for both cases of grey box and black box testing (when we can’t access the state of AI system). You can apply these techniques to a wide range of AI systems, ranging from simple machine learning systems to complex neural networks.