
Reading Group (+🧋): Senior SWE-Bench
Description
Join Snorkel AI's Reading Group for a presentation on Senior SWE-Bench, an open-source benchmark evaluating coding agents on real-world tasks.
Highlights
Presentation by lead researcher Henry Ehrenberg on Senior SWE-Bench. Discussion on coding agents evaluated as senior engineers. Introduction to validation agents for reliable grading. Tasks sourced from real PRs with significant runtime investigation. Taste judge scores on code minimality and craftsmanship. Top models' performance analysis on senior-level tasks. Benchmark includes 100 tasks from 12 real repositories. Open-source and Harbor-compatible benchmark. Collaboration with Princeton and UW–Madison. Boba tea and other refreshments provided.
Event location
Let your network know you`re going
Share this event to start conversations, invite colleagues, and connect before it begins.
