Testing of AI - what about it?

Smart software. It’s everywhere. It’s changing the way we work, manufacture, communicate, entertain ourselves, manage our health, money, fitness, and more. Smart software is embedded in so much of what we do and use in today’s digital world. But I believe that this presents us with a real problem in terms of quality. Why? Because the actions and responses of intelligent machines will differ over time and thus are less predictable than traditional IT-systems. And also, because the manually intensive way we’ve developed, tested and delivered software up until now, is no longer enough to meet the escalating demand for new and innovative applications, in every walk of life.

Learning curve
Yet – as is always the case when new capabilities and technologies enter the fray, while we will still need the same solid base of testing knowledge as always, – there is a steep learning curve needed for everyone involved. Since currently there aren’t enough people skilled in the new technologies, including AI, data analytics, robotics and machine learning.

This is something that Humayun Shaukat, Toni Gansel and Rik Marselis, discuss in our new opinion paper ‘Testing of artificial intelligence – AI Quality Engineering Skills (An Introduction)’.

Embracing new skills

This testing of AI will require a new skillset related to interpreting a system’s boundaries or tolerances. Indeed, as our paper points out, the complex functioning of an AI system means that the focus of testing shifts from output to input to verify a robust solution.

That’s not to say people that test must throw out everything they’ve already learned. In fact, we will still see many of the established test approaches, test design techniques and testing practices being applied in the AI-led landscape. It’s more that their skillset will need to expand as AI plays an increasingly major role. For example, as well as skills from the likes of TMap and ISTQB, AI quality engineers should have expertise in A/B testing, metamorphic testing and multiple types of non-functional testing, amongst other techniques that have gained new importance.

Cross-functional teams must hone their skills in what our paper describes as “the six angles of quality” used for testing AI, robotics and other modern technologies. These six angles are: mechanical, electrical, information processing, machine intelligence, business impact, and social impact. The first three are well-known and enough knowledge and experience is available. The latter three need research and development of new approaches and skills of which our paper sketches the first outlines.

Tomorrow’s world
What does all of this mean for today’s – and tomorrow’s – QA and testing profession? We believe AI and other smart technologies broaden our horizons and opportunities. Even though the technology can take over many of the mundane, repetitive tasks, the need for skills and knowledge in areas such as software engineering, informatics, applied statistics, analytics, new programming languages, and so much more, make this an exciting new world in which to work.

Free download the paper "Testing of AI" here.

To find out more, please contact Rik Marselis at rik.marselis@sogeti.com or Humayun Shaukat at humayun.shaukat@sogeti.com.