Anthropic's Sonnet 5.5: Surprisingly Good, Yet Questionable Value

Anthropic unexpectedly launched Sonnet 5.5, a model the speaker initially did not anticipate would be significant, given his satisfaction with Opus's price and value.

The speaker's previous favorite code model was Fable 5.1, but Opus 5.5 had become a dominant part of his workflow for the preceding week, making the new Sonnet release a surprise that he views as a strategic move against OpenAI.

The speaker, however, champions Sonnet 5.5 as a 'cheapish' Anthropic option that aligns with the demands of the modern era.

The model is positioned as a direct competitor to OpenAI’s GPT-6 Soul, which the speaker implies is an underwhelming offering.

This feature provides automated summaries of specific changes, such as package installations or logic fixes, and includes a 'security blast radius' diagram to help identify high-risk changes.

The activity view significantly speeds up catching up on PR threads, offering a more efficient alternative to standard GitHub interfaces.

Sonnet 5.5 recorded a score of 70.6% on Terminal Bench 4, marking a substantial increase from its previous 10.3%, a result the speaker described as "kind of nuts" for a Sonnet model.

Despite general skepticism toward benchmarks, this model demonstrates high capability in long-horizon tasks and image understanding, even beating previous models in playing Pokémon Red from screenshots.

Previously, these models often failed to properly understand their assigned work, but current iterations now function like faster, though slightly less intelligent, versions of Opus.

Sonnet 5.5 is priced at $2 per million input tokens and $10 per million output tokens, with caching reads costing 20 cents per million tokens.

Unlike previous Anthropic models like Fable 5.1 and Opus 5.5, Sonnet 5.5 did not receive a substantial discount on cache read costs.

Cache reads account for a disproportionately large percentage of Sonnet 5.5's total expenses compared to higher-tier models, which is a significant concern for day-to-day agent code work.

The high cost of cache reads for Sonnet 5.5 persists, despite expectations for a price reduction, while output tokens remain a minor expense.

For real-world coding tasks, Sonnet 5.5 proves to be as costly as Fable 5.1, making it less cost-effective than Opus.

This forced overthinking can cause token usage to spike by as much as 1,500% compared to 'X High' mode.

The increased token consumption often leads to decreased performance and a higher likelihood of incorrect answers from the model.

Conversely, reducing the reasoning level to 'High' or 'Medium' drastically cuts costs to $0.18 and $0.09 respectively.

Despite its higher costs, Sonnet 5.5 is favored for daily coding due to fewer instances of 'dumb spikes' in performance.

This positions Sonnet 5.5 favorably against GPT-6 (Soul) in specific scenarios, a model the speaker generally finds underwhelming.

Despite Sonnet 5.5's strong performance in computer use, the speaker ultimately prefers Astra for tasks requiring advanced computer interaction.

Sonnet 3.5 used 271,920 tokens per task in Cursor Bench, which is substantially higher than Opus (218K) and Gemini 1.5 Flash (162K).

This token usage is more than five times greater than that of GPT-4o and xAI's Grok.

Despite a higher tokens-per-second (TPS) rate, the excessive token consumption results in longer actual task completion times, with Sonnet taking 43 minutes for a task that Opus completed in 36 minutes.

Critics labeled the animations and card designs from Sonnet 3.5 'hideous' or 'boring' during the Witch AI design showcase, where the model struggled with readability and layout coherence compared to Claude 3.5 Haiku.

Claude 3.5 Haiku is still considered the top model for front-end design, and Opus is noted for its better steerability and adherence to design instructions.

Despite these shortcomings, Sonnet 3.5 still significantly outperforms OpenAI and xAI counterparts in overall capability.

The model's high token usage, subpar front-end performance, and longer actual completion times mean it does not justify its selection over alternatives like Haiku or Opus within Claude Code.

The model demonstrates superior cost-effectiveness for specific research-heavy architectural tasks.

This specialized role allows Sonnet to contribute efficiently without being the primary interface for users.

Opus took almost twice as long, and Astra required nearly three times the duration for the same task.

This speed makes Sonnet an ideal utility for agents that need to quickly scope and plan work prior to execution.

Theo is currently engaged in rewriting the TypeScript compiler in Rust, a complex, long-running project.

He initially misinterpreted Sonnet's logical, thread-specific behavior as a regression in its ability to understand intent.

However, Sonnet correctly followed the specific pattern of the conversation thread, focusing on numerical aspects rather than attempting to guess external intentions, successfully deleting 1.6 million lines of legacy code.

Compared to models from just a few months ago, current generation models now produce usable and clearly defined 3D models.

The generated fish and environmental elements showcase unique and cohesive designs, indicating a substantial improvement in artistic coherence.

The simulation runs smoothly at 120 frames per second on the user's laptop, demonstrating robust game mechanics for the AI-generated content.

A demonstrated game demo was generated in just 40 minutes for a cost of only a few dollars.

The speaker estimates that if full API pricing were applied, the total cost would amount to approximately $16, which he notes is less than he has paid for worse games.

He contrasts this with the $200 Claude Pro monthly plan, which provides substantial value for heavy users, with a rough analysis suggesting it offers up to $2,300 worth of weekly usage.