Worth knowing
Dayjob: Tax – evaluating AI agents on 50 professional tax assignments
This benchmark highlights that while frontier AI agents can handle isolated calculations and questions, they still struggle with the ambiguity, unverified assumptions, and proactive judgment required in real-world professional workflows.
Reported by Hacker News ·
What happened
Surge AI introduced DAYJOB: Tax, a benchmark evaluating AI agents across 50 realistic, multi-file professional tax assignments created by tax experts. The benchmark tests end-to-end professional judgment and messy record reconciliation, with top models achieving at most a 10.8% pass rate.
Who it’s about
- Surge AI
- DAYJOB: Tax
- Claude Sonnet 5.5
- Claude Opus 5.5
- Claude Fable 5.1
Source
More on benchmarks
Get a brief picked for you
Sign in to follow topics and companies, save stories and search the archive.
Sign in