Tutorial Systems Engineering Test & Evaluation Conference 2024

Foundations of Data Science and Explainability in AI Systems - a Systems Engineering Perspective  (20746)

Tanya Dixit 1 , Andrew Madry 2 , Jawahar Bhalla 3
  1. Google Cloud, Google, Sydney, NSW, 2000
  2. Madry Technologies Pty Ltd, Australia
  3. JB Engineering Systems, Australia

In the fast-evolving field of Artificial Intelligence (AI) and Machine Learning (ML), understanding the foundational principles and decision-making processes of AI systems is crucial for building trust and accountability. This comprehensive tutorial begins with an introduction to basic data science and ML methods, including essential statistics and an overview of specialized techniques such as Neural Networks, which significantly impact modern systems.

Participants will then explore the Systems Engineering (SE) approach necessary for AI-intensive projects. AI projects often face challenges like poorly defined requirements, changing stakeholder expectations, and unknown constraints. The session introduces SE principles, including systems thinking, modeling and simulation, and situation awareness, as foundational blocks for best practices in evolutionary SE life-cycle processes. It emphasizes the need for a distinct methodology for AI projects, different from traditional software projects, and discusses how Data Science life-cycle models can be effectively mapped to the SE approach.

Building on this foundation, the tutorial addresses the importance of AI transparency and explainability, particularly in the context of Generative AI and large language models (LLMs). Participants will gain insights into various explainability techniques such as attention visualization, feature attribution, and model-agnostic methods. Practical use cases will demonstrate how these techniques can enhance transparency and performance in real-world applications.

This tutorial is designed for AI practitioners, data scientists, engineers, researchers, and beginners in AI/ML, providing a thorough overview of the technology and practical methods for implementing and leveraging explainability techniques and SE principles in their AI projects. By the end of the session, attendees will be equipped with the knowledge to effectively integrate these concepts into their own Generative AI and ML projects.