Your Agent Hasn't Faced Adversity Yet!

Description

Please join us as we host FJ Collins coming in from NYC to discuss how the team at Intuit prepares their agents for the real world.AbstractEvery coach knows a play that looks perfect on a whiteboard can fall apart the moment a defender brings the heat. So why do we expect AI agents to be any different? In the lab, your agent is playing against air; in the real world, it's facing a defense it hasn't practiced for. This talk breaks down three field-tested strategies to move your AI from "meeting room ready" to "game day proven." First, we'll look at the Scout Team approach: deploying adversarial "confused user" agents to throw every unexpected look at your product before your customers do. Second, we'll discuss how to "watch the film"—systematically calibrating AI judges against human ground truth (using Cohen's κ) until you can trust the machine to make the right call unsupervised. Finally, we'll explore the role of the Head Coach: ensuring that while agents execute the plays, a human always retains the final say on the high-stakes decisions. These aren't drills run in a sandbox — they're the same practices behind the agents powering Intuit's Expert Network of 31,000 experts. The takeaway is simple: an AI doesn't earn trust by working once in practice. It earns it by surviving the defenses it was never designed to face.LocationThis event is hosted in the Teamworking by TechNexus space. Upon arrival, guests can come directly to the 12th floor. The reception team will be there to greet you and help guide you to your room.Parking & TransportationThe Civic Opera Building does not have a dedicated parking lot. There is limited metered parking nearby. We recommend using iParkit or SpotHero to find discounted parking options. The CME Building (10 S. Wacker) is the nearest parking garage, with several additional garages located within a two-block radius.The venue location is highly walkable and conveniently situated near multiple CTA train lines and bus stops, making it easy to access via public transportation.Agenda6:00 - 6:30 Networking and Bites6:30 - 6:45 Rising Star Speaker6:50 - 7:30 Presentation7:30 - 7:40 Q&A7:40 - 8:00+ Networking and Wrap UpSponsors:​​Food and Beverage: Focused an exclusive LangChain partner and boutique consultancy dedicated to building production-ready AI systems that work in the real world.Venue: Teamworking by TechNexusContinue the conversation between gatherings. Join our discord (https://discord.gg/bmdwxs2Q5F)