Session 09: Making AI Trustworthy

Testing, Evaluating, and Securing Intelligent Systems About the Event AI applications can produce impressive results, but how do we know when they are actually working well? Traditional software can be tested with predictable inputs and expected outputs. AI systems are different: their responses can change, information can be misunderstood, and seemingly small changes can affect the quality of an entire system. As AI becomes responsible for searching information, using tools, and carrying out tasks, simply asking a few questions and deciding that "it looks good" is no longer enough. In Session 09 of this series, we focus on making intelligent systems measurable, testable, and secure. Building on the retrieval systems, AI tools, and workflows developed throughout the series, we will explore how to evaluate AI responses, test systems against unexpected or malicious inputs, add safeguards, and automatically detect when changes negatively affect system quality. What We'll Cover The Hands-On Codelab: Putting AI to the Test In this practical workshop, participants will: By the end of the session, participants will have a system that does more than work—it will have a way to measure its quality, identify weaknesses, and prevent regressions as it evolves. The Big Idea