Measuring Autonomous AI Research
We ran 153 autonomous runs across 18 frontier models on the nanoGPT optimizer speedrun.
Opus 5 is a step change improvement for the Opus tier powering long-running agents while delivering improvements in coding and professional work.
We ran 153 autonomous runs across 18 frontier models on the nanoGPT optimizer speedrun.
DataBench v1: 100 realistic analytical tasks across Q&A and open-ended prompts, run in a synthetic Hex workspace — built because existing analytics benchmarks test "overspecified pub trivia" rather than the vague, directional questions people actually ask.
Meet Toast 1, Mixedbread's search agent for knowledge-intensive tasks, matching or outperforming Claude Opus 5 and GPT-5.6 Sol while being up to 10× cheaper and 12× faster.
The prompting method behind Claude of Duty. Give the agent a bar it can't talk its way around, let it split the work, and never let the builder grade itself.
Opus 5 is a step change improvement for the Opus tier powering long-running agents while delivering improvements in coding and professional work.
I’ve written previously about how to best prompt the newest generation of Claude 5 models and work with them iteratively to discover what you want to build.
Pair Opus as an advisor with Sonnet or Haiku as an executor, and get Opus-level intelligence in your agents at a fraction of the cost.
Anthropic
Claude Opus 5.5 leads in agentic coding and knowledge work, and costs 40% less to run than Opus 5 on typical workloads.
Anthropic
Claude Sonnet 5.5 is a clear upgrade over Claude Sonnet 5, runs 30%+ faster, and costs up to 30% less for most work.
Anthropic
Mythos 5.1 is available only through our trusted access programs; its safeguards are specifically designed to support work in cybersecurity and the life sciences.
An occasional email when notable AI dev tools and models land in the directory. No spam, unsubscribe anytime.