PD ISO/IEC TR 29119-11:2020: Software and Systems Engineering - Software Testing Guidelines on the Testing of AI-based Systems

Standard Number: PD ISO/IEC TR 29119-11:2020

Pages: 60

Release Date: December 7, 2020

ISBN: 978 0 539 15872 4

Status: Standard

Overview

In the rapidly evolving world of technology, artificial intelligence (AI) is at the forefront of innovation, transforming industries and redefining the way we interact with software systems. As AI-based systems become increasingly complex and integral to business operations, ensuring their reliability and performance through rigorous testing is paramount. The PD ISO/IEC TR 29119-11:2020 standard provides comprehensive guidelines specifically tailored for the testing of AI-based systems, offering a structured approach to address the unique challenges posed by these advanced technologies.

Why Choose This Standard?

AI-based systems present unique testing challenges due to their inherent complexity, adaptability, and the often unpredictable nature of machine learning algorithms. Traditional testing methods may fall short in effectively evaluating these systems. The PD ISO/IEC TR 29119-11:2020 standard is designed to bridge this gap by providing a robust framework that encompasses the following key aspects:

• Comprehensive Coverage: With 60 pages of detailed guidelines, this standard covers a wide range of testing scenarios and methodologies specific to AI-based systems.
• Adaptability: The guidelines are adaptable to various types of AI technologies, including machine learning, neural networks, and natural language processing, ensuring relevance across different applications.
• Risk Management: Emphasizes the identification and mitigation of risks associated with AI systems, enhancing their reliability and safety.
• Quality Assurance: Focuses on maintaining high standards of quality and performance, crucial for AI systems that often operate in critical environments.

Key Features

The PD ISO/IEC TR 29119-11:2020 standard is a vital resource for software engineers, testers, and quality assurance professionals involved in the development and deployment of AI-based systems. Here are some of the key features that make this standard indispensable:

• Structured Testing Approach: Provides a systematic approach to testing AI systems, ensuring thorough evaluation and validation of their functionalities.
• Guidelines for Test Design: Offers detailed instructions on designing effective test cases that cater to the unique characteristics of AI algorithms.
• Performance Evaluation: Includes methodologies for assessing the performance and efficiency of AI systems under various conditions.
• Ethical Considerations: Addresses ethical concerns related to AI testing, promoting responsible and fair use of AI technologies.

Who Can Benefit?

This standard is an essential tool for a wide range of professionals and organizations, including:

• Software Developers: Gain insights into best practices for integrating testing into the AI development lifecycle.
• Quality Assurance Teams: Enhance testing strategies to ensure AI systems meet high standards of quality and reliability.
• Project Managers: Utilize the guidelines to manage AI projects effectively, ensuring timely delivery and compliance with industry standards.
• Regulatory Bodies: Reference the standard to establish benchmarks for AI system testing and certification.

Conclusion

As AI continues to revolutionize the technological landscape, the importance of robust testing frameworks cannot be overstated. The PD ISO/IEC TR 29119-11:2020 standard is a critical resource that empowers professionals to navigate the complexities of AI testing with confidence. By adhering to these guidelines, organizations can ensure the development of reliable, efficient, and ethically sound AI systems that meet the demands of today's dynamic market.

PD ISO/IEC TR 29119-11:2020

This standard PD ISO/IEC TR 29119-11:2020 Software and systems engineering. Software testing is classified in these ICS categories:

• 35.080 Software

This document provides an introduction to AI-based systems. These systems are typically complex (e.g. deep neural nets), are sometimes based on big data, can be poorly specified and can be non-deterministic, which creates new challenges and opportunities for testing them.

This document explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems.

This document presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduces white-box testing specifically for neural networks. It describes options for the test environments and test scenarios used for testing AI-based systems.

In this document an AI-based system is a system that includes at least one AI component.