
Reading Group (+🧋): Senior SWE-Bench
Опис
Join Snorkel AI's Reading Group for a presentation on Senior SWE-Bench, an open-source benchmark evaluating coding agents on real-world tasks.
Особливості
Presentation by lead researcher Henry Ehrenberg on Senior SWE-Bench. Discussion on coding agents evaluated as senior engineers. Introduction to validation agents for reliable grading. Tasks sourced from real PRs with significant runtime investigation. Taste judge scores on code minimality and craftsmanship. Top models' performance analysis on senior-level tasks. Benchmark includes 100 tasks from 12 real repositories. Open-source and Harbor-compatible benchmark. Collaboration with Princeton and UW–Madison. Boba tea and other refreshments provided.
Місце проведення
Розкажіть своїй мережі, що ви йдете
Поділіться цією подією, щоб розпочати розмови, запросити колег та налагодити контакти до її початку.
