perf-test.com
·

Claude Opus 5.5 Performance: Benchmarks, Token Efficiency, and Cost per Task

A performance engineer's read of Claude Opus 5.5: the benchmark scores, why fewer tokens per task beats the 20% price cut, what always-on thinking does to latency, and how to benchmark it yourself.

· 8 min read · By perf-test.com Editorial · AI-assisted
llmclaudeanthropicbenchmarkingtoken-economicslatency

Anthropic released Claude Opus 5.5 on 22 September 2026. The headline is a model that scores higher than Opus 5 at a lower price. For anyone who runs LLM workloads in production, the more useful story is efficiency. Opus 5.5 finishes the same work with fewer tokens, in fewer agent steps, and with faster output. The per-token price cut matters less than that, because tokens per task compound across every step of an agent run.

This article covers what the published numbers say, what they leave out, and how to measure the model on your own workload before you switch production traffic to it.

The launch numbers at a glance

Scores from Anthropic’s launch announcement. Treat them as vendor-reported until independent runs reproduce them.

BenchmarkWhat it measuresOpus 5.5Fable 5.1Opus 5
Terminal-Bench 4.0Agentic coding in a terminal66.4%55.8%52.3%
FrontierCode v1.1Agentic coding54.4%50.3%48.0%
CursorBench 4.0Coding in an IDE agent57.8%51.8%46.6%
GDPval-AA v2.1Knowledge work (Elo)184617351708
AutomationBenchBusiness workflows40.0%31.4%26.9%
Humanity’s Last Exam (tools)Multidisciplinary reasoning67.7%65.6%63.6%
Terminal-Bench-Science 0.1Agentic scientific research58.7%52.6%29.0%
OSWorld 2.0 (partial)Computer use81.8%80.7%74.0%
Chartography (tools)Reading charts89.0%88.4%83.4%

Opus 5.5 does not win every row against competitors. Anthropic’s own table shows GPT-6 Astra ahead on AutomationBench (41.4%) and Terminal-Bench-Science (64.6%). Against its own model family, though, it beats Opus 5 everywhere and edges past Fable 5.1, the more expensive tier, on every benchmark listed.

Read the effort level before you read the score. The launch table does not use one setting throughout. Terminal-Bench 4.0 was run at xhigh effort, FrontierCode at the default medium, and GDPval-AA at max. CursorBench is 57.8% at its best setting but 52.5% at the default. The row you care about is the one run at the effort level you will actually deploy.

The real headline: fewer tokens per task

A 20% price cut is easy to understand. The bigger change is that Opus 5.5 produces fewer tokens for the same result:

  • Independent measurement. Sonar ran its Java code-generation benchmark (HumanEval, MBPP, ComplexCodeEval; 4,444 tasks) on both models. Opus 5.5 used 12.96M output tokens against Opus 5’s 21.71M, about 40% fewer, and wrote 27.5% less code (664,890 lines against 916,813).
  • Vendor and early-tester reports. Anthropic cites testers who needed about half the turns and output tokens to match Opus 5’s quality, finished tasks in a third to half fewer steps, and used as few as a third of the tokens.
  • Speed. Anthropic says output generates more than 30% faster than on Opus 5. Fast mode, a research preview on the Claude API, runs the same model at up to 2.5× output speed for double the price.

For an agent this compounds. Each step re-sends the growing context and generates new output. Fewer steps means fewer round trips, less accumulated context, and fewer chances to fail partway through a run. Measure cost and latency per completed task, not per request.

Cost per task: a worked example

List prices per million tokens:

InputOutputCache read
Opus 5$5.00$25.00$0.50
Opus 5.5$4.00$20.00$0.20

Now take an illustrative task that uses 50K input tokens and 20K output tokens on Opus 5. Assume Opus 5.5 does it with 40% fewer output tokens, in line with the Sonar measurement above:

Opus 5    : 50K × $5/1M  + 20K × $25/1M = $0.25 + $0.50 = $0.75
Opus 5.5  : 50K × $4/1M  + 12K × $20/1M = $0.20 + $0.24 = $0.44   (≈41% less)

The price cut alone accounts for 20% of the saving. Token efficiency roughly doubles it. That matches Anthropic’s claim of about 40% lower cost on typical workloads. Output tokens carry most of the cost, so a model that says less does most of the saving.

The cache-read price falls 60%, to $0.20 per million tokens. That matters most for long agent sessions that re-read a large cached prefix on every turn. It also makes cache misses relatively more expensive, so keeping the cache warm matters more than before (see the prompt caching deep-dive).

You can plug your own token counts into the LLM Cost & Latency Estimator.

Does “less code” mean worse code?

Sonar’s static analysis gives the most useful independent quality signal so far. The results point in more than one direction:

  • Functional correctness was about the same: an 87.68% pass rate against Opus 5’s 88.6%.
  • Absolute issue counts fell sharply: 42% fewer findings in total, 34% fewer vulnerabilities, and BLOCKER-severity vulnerabilities down 53% per million lines.
  • Bug density rose: 644 bugs per million lines against 576. Absolute bug count still fell 19% (428 against 528), because there was much less code.

This is the familiar trap with density metrics. When the denominator (lines of code) shrinks by 27%, a per-line rate can go up while the actual number of defects goes down. Decide which one you care about before you compare models. For code review and maintenance, the absolute count usually matters more.

Latency: thinking is now always on

One change affects latency measurements directly. On Opus 5.5, extended thinking cannot be turned off. Requests that disable it or set a thinking budget return an error. The only control is effort (low / medium / high / xhigh / max), and the default is now medium where Opus 5’s was high.

What that means for your latency numbers:

  1. Time to first token (TTFT) now includes thinking time. By default, thinking content is not shown (display: "omitted"). The stream therefore starts with thinking blocks that have empty text, and visible text arrives later. Measure time to first visible text token separately from time to first stream event, or your TTFT figures will look better than what users experience.
  2. For routes where latency matters, test effort: "low" first. Anthropic’s guidance is that lower effort cuts thinking, latency, and cost more reliably than prompts asking the model to be brief. On several coding evaluations, low came close to Opus 5 at high.
  3. Set max_tokens with room for thinking. Thinking tokens count against max_tokens even when the text is hidden. A limit sized for a no-thinking route on Opus 5 can now cut replies off.
  4. Always set effort explicitly. If you omit it, a migrated route drops from high to medium without warning, which invalidates any before-and-after comparison.

A caveat for benchmarks: fallback routing

Opus 5.5 adds broader safety classifiers, with new biology and reasoning-extraction categories alongside cybersecurity. When one of them declines a request, the API returns stop_reason: "refusal". If you have opted into server-side fallbacks, the request is re-run on another model, such as Opus 5, within the same call.

For benchmarking, some of your “Opus 5.5” results may have been served by a different model. Before you attribute a number to Opus 5.5, check response.model and the fallback_message entries in usage.iterations, and count refusals as their own outcome.

How to benchmark Opus 5.5 yourself

The standard LLM API load-testing methodology applies, with a few additions specific to this model:

  1. Sweep effort levels. Run low, medium, and high at minimum. The quality-to-cost curve differs by workload. Coding and agentic tasks respond strongly to effort; chat and classification often don’t.
  2. Measure per task, not per request. Record steps, total input and output tokens, wall-clock time, and cost for each completed task. Include retries.
  3. Separate the two TTFT figures: first stream event and first visible text.
  4. Use realistic, fixed output buckets so decode time stays comparable between runs (see measuring LLM inference performance).
  5. Track cache-read tokens. With a 60% cheaper cache read, your cache hit rate moves cost more than before.
  6. Log which model served each request and exclude or label anything a fallback handled.
  7. Report p95 and p99, not the mean. Longer thinking at high effort widens the tail (why percentiles matter).

Should you switch?

For most agentic coding and knowledge-work workloads now on Opus 5, the published and independent evidence points the same way: similar or better quality, fewer tokens, faster output, and a lower price per token. Budget for the migration work, though:

  • Code that disables thinking, or forces a specific tool with tool_choice, now returns an error and has to change.
  • The effort default moved. Re-run your effort sweep rather than carrying settings over.
  • Streaming UIs may go quiet, because progress notes between tool calls now arrive as thinking blocks.

Run your own evaluation at the effort level you will deploy, compare cost and latency per completed task, and move traffic when your numbers hold up. The vendor’s numbers are not a substitute for that.

Sources

Comments

Loading…