GPT-6 Astra vs Claude Fable 5.1: What the Benchmarks Stopped Telling You

Two Frontier Models in Forty-Eight Hours

Anthropic shipped Claude Fable 5.1 on September 1. OpenAI shipped GPT-6 Astra on September 3. Two days apart, both priced identically at $10 per million input tokens and $50 per million output, both claiming the frontier.

We should say the quiet part up front: OneWave is a Claude partner. We build on Anthropic, we train teams on Anthropic, and we have written at length about why we made that bet and about the last time OpenAI pulled even. So take it for what it is worth when we say our team has been in GPT-6 Astra since launch week and came away genuinely impressed.

The interesting story is not which model won. It is that the benchmark scoreboard moved three separate times in the first week, and most of the coverage you have read is quoting a version of it that no longer exists.

Here is what actually shipped, what the independent numbers say once you strip out the launch-day framing, and the decision we are handing clients this week.

That reaction was everywhere in launch week, and it is the right instinct pointed at the wrong target. The score was not wrong. It was about to be revised twice.

What Anthropic Shipped: Claude Fable 5.1

Fable 5.1 is generally available as claude-fable-5-1 across the Claude API, AWS, Google Cloud, and Microsoft Azure. Anthropic also released Claude Mythos 5.1 - the same underlying model with fewer safeguards, restricted to US organizations inside trusted-access programs for cybersecurity and life sciences work.

The headline gains are in long-horizon agentic work, and the jump is not subtle. On Terminal-Bench-Science 0.1, Fable 5.1 scores 52.6% against 24.7% for Fable 5 and 29.0% for Opus 5. That is roughly a doubling in one release. AutomationBench moves from 17.1% to 31.4%. Humanity's Last Exam with tools reaches 65.0%.

The Change Most People Missed

The cache read price dropped 75%, from $1.00 to $0.25 per million tokens. That sounds like a line item. For anyone running agents, it is the whole ballgame.

Agentic workloads re-read the same context on every turn - the codebase, the system prompt, the tool definitions, the accumulated scratchpad. Cache reads are the dominant cost in a long-running agent, not fresh input. Anthropic quotes roughly 25% lower cost on typical workloads and up to 45% on agentic tasks. For the kind of overnight agent orchestration we run, that is the difference between a workload being worth running nightly and not.

There is also a new effort dial - low, medium, high, xhigh, max - which defaults to high in Claude Code and medium in Cowork and claude.ai. Most teams will never touch it. The ones running overnight agents should.

What OpenAI Shipped: GPT-6 Astra

Astra arrived with a one-million-token context window, an Astra and Astra Pro tier split, and a Fast Mode that runs 2.5x faster at twice the cost. Greg Brockman called it the AGI era. OpenAI's own launch line was blunter and better: "Anything you can do on a computer, Astra can do for you. Fast."

That is the actual product thesis, and it is where the model earns the hype. On OSWorld 2.0, the desktop computer-use benchmark, Astra scores 72.6% against 65.7% for GPT-5.6 Sol - while cutting average time per task from about 75 minutes to 40. On ScreenSpot-Pro it jumps from 76.9% to 92.7%. OpenAI demoed it driving KiCad, Blender, Excel, and Power BI.

Two numbers deserve more attention than the AGI framing got. Astra's measured hallucination rate fell from 12.2% to 4.2%. And on honeypot tests where GPT-5.6 Sol attempted security shortcuts in 48.2% of runs without production safeguards, Astra did it in 0%.

For three years the reliability argument for Claude was that it says "I do not know." Astra just closed most of that gap. Any honest comparison has to start there.

Not everything improved. OpenAI disclosed a regression in monitorability - Astra's written reasoning is harder to audit than Sol's - and said it will withhold further scaling until it regains confidence there. Their chief scientist put it plainly: progress in intelligence does not guarantee progress in alignment. Publishing that on launch day was the right call.

The Benchmarks, Once You Strip the Launch Framing

Both labs published tables where they win. Here are the head-to-head rows that came from the same evaluation on both sides, plus the ones where the loser is the one who published them.

Read that table honestly and Astra wins most rows. It wins computer use, math, business automation, and agentic terminal work. Fable 5.1 holds the coding agent index, the tool-assisted reasoning exam that OpenAI itself published, and the cost structure.

The Scoreboard Moved Three Times in One Week

This is the part that should change how you read every model launch from here on.

Artificial Analysis is the closest thing the industry has to a neutral referee. On launch day, their Intelligence Index v4.1.1 had Fable 5.1 at 66 and Astra at 61 - a five-point Claude lead, tying Astra with its own predecessor. Within days, v4.2 recalibrated to 57 and 55. By v4.3 the two models were tied at 53.

Nothing about either model changed. The ruler did. If your read on GPT-6 came from a chart screenshotted on September 3, you are holding a number that has been revised twice.

Benchmarks are now a moving target measured by a moving instrument. Treat any single index score as a snapshot with a shelf life of about a week.

And the Benchmaxxing Problem Is Real

SemiAnalysis ran an experiment worth internalizing: compare each model on Terminal-Bench 2.1 against the newer, harder Terminal-Bench 4.0, and watch how far each one falls.

Four models within three points of each other on the older benchmark. A 38-point spread on the newer one. Astra and Fable 5.1 degrade gracefully. The other two fall off a cliff, which is what benchmark overfitting looks like from the outside.

A few more caveats worth carrying: Astra's 99.9% on ARC-AGI-3 required a special test harness and scores materially lower through the plain API. OpenAI part-funded FrontierMath. And every one of these tables was run at maximum effort, which inflates both scores and latency past anything you would run in production.

What Independent Testers Found

Vendor tables are marketing. Third-party runs are evidence. Three that changed our thinking:

• Andon Labs' Vending-Bench Arena. Agents run competing vending machines in a shared market, emailing and trading with each other. Astra beat Fable 5.1 and GLM-5.3 in all three games - final balances of $13,364, $13,562, and $10,162 against Fable's $5,637, $5,062, and $8,457 - while behaving more ethically. Andon's own note: it is the first time OpenAI has topped Vending-Bench, and the best model is no longer the unethical one.
• Perplexity's WANDR evaluation. Astra posted 0.682 at $11.98 per task, the highest score of any model they tested - 13.5% above Fable 5.1 at 6.1% lower cost per task.
• Independent full-app builds. On smaller builds the two are near identical, roughly 90% versus 92.5%. On larger ones the gap widened toward Claude: Astra's movie tracker shipped with a broken API integration and UI flicker, and its Obsidian clone shipped with a non-functional agent. One tester's run cost $198 on Astra against $113 on Fable 5.1.

Those last two points are in tension, and that tension is the actual finding. Astra is cheaper per task on short, well-scoped work because it is dramatically more token efficient - Artificial Analysis measured roughly one-third the tokens of its predecessor on coding. Fable 5.1 is cheaper on long, context-heavy work because cache reads cost a quarter as much. Your bill depends entirely on which shape your work is.

What We Are Telling Clients This Week

No one should rip out a working stack over a two-day release window. But the defaults have genuinely shifted, and here is where we now land.

Reach for GPT-6 Astra when

• The work is computer use - driving desktop apps, filling forms, running QA, updating systems with no API
• You need heavy quantitative reasoning or research math
• Tasks are short, well-scoped, and token efficiency beats cache efficiency
• Hallucination rate is the binding constraint on shipping

Stay on Claude Fable 5.1 when

• Agents run long and unattended against a large codebase or document set - Anthropic's customers are reporting sustained multi-hour and even 38-hour runs
• The workload re-reads the same context on every turn, where the $0.25 cache read compounds
• You are already invested in Skills, MCP, and subagents - the ecosystem gap is still real and still Anthropic's
• You need the tool-assisted reasoning edge on genuinely hard analysis

What everybody should do

Run your own eval. Not a benchmark - your five real tasks, the ones your team does every week, scored by the person who has to live with the output. This is the same discipline we walk through in how to evaluate an AI vendor. Every number in this post was produced at maximum effort on somebody else's workload. Twenty of your own prompts will tell you more than the entire launch-week discourse.

The second thing: stop treating this as a loyalty question. The gap between these two models is smaller than the gap between a team that has been trained to use one well and a team that has not. That is the whole reason we argue training should come before agents, and nothing in this release week changed it.

The right question is not "which model is best." It is "which model is best for this task, and does my team know how to drive it." Most companies are losing far more to the second half of that sentence.

The Honest Summary

GPT-6 Astra is the most impressive thing OpenAI has shipped, and it beat us to conclusions we did not expect - on hallucination, on computer use, on alignment behavior under pressure. Our team was blown away, and we are a Claude shop saying that.

Claude Fable 5.1 still holds the ground that matters most for how we build: long-horizon agentic coding, the cache economics that make overnight agents affordable, and an extension ecosystem nobody else has matched. It is the model we will keep shipping client work on.

Both of those things are true at once, which is the only takeaway that survives contact with next month's release. If you want the wider field, our five-platform comparison covers Gemini, Perplexity and Manus alongside these two, and our Grok comparison adds the fourth option now on most shortlists. If you want help running a real evaluation against your own workload instead of somebody else's benchmark, that is what our Claude consulting and OpenAI consulting engagements start with - talk to our team.

Sources

• Introducing Claude Fable 5.1 and Claude Mythos 5.1 - Anthropic
• OpenAI launches GPT-6 Astra and says welcome to the "AGI era" - The New Stack
• Benchmarking GPT-6 Astra - Artificial Analysis
• GPT-6 Astra Benchmarks Explained - Vellum
• Claude Fable 5.1 and Mythos 5.1 arrive with a 75% cost reduction for cache reads - VentureBeat
• Vending-Bench Arena results - Andon Labs
• Terminal-Bench 2.1 vs 4.0 agent benchmark comparison - SemiAnalysis
• GPT-6 Astra on WANDR - Perplexity