OpenAI's GPT-6.1 Soul: Performance & Cost Leader

Previously, OpenAI's GPT-6 Soul and GPT-6 Luna models had not fully impressed, leading to anticipation for a more robust offering.

The creator noted that GPT-6.1 Soul is a surprisingly cheap, capable, and notably better alternative to Astra, making prior reviews of competing models challenging to publish.

Terminal Bench 4 results indicate state-of-the-art scores for OpenAI's new model, GPT-6.1 Soul, despite creator skepticism toward benchmarks due to manual testing complexities.

Deep SWE results show performance parity with Astra at a substantially reduced price point, making it an attractive option for developers.

The cost comparison is stark: GPT-6.1 Soul on low settings costs 21 cents, while Astra on low costs $1.46, and Opus 5.5 on max settings, achieving the same score, costs $14.65.

GPT-6.1 Soul scores identically to Opus in performance while being 73 times cheaper, a significant factor for widespread adoption.

The model demonstrates higher intelligence than GPT6, suggesting a fundamental shift in its architectural design.

Slower tokens per second indicate a potentially larger model size, which could contribute to its enhanced capabilities.

This recalculation by OpenAI is likely a strategic move to protect profit margins in response to the high value offered by competitors like Opus 5.5.

The estimated monthly usage capacity for the revamped Pro subscription is now approximately $9,000, adjusting the value proposition for users.

The cache read price has been reduced by 95% to 10 cents per million, marking the first time OpenAI has deviated from the standard 10% cache read pricing ratio, specifically targeting real-world agentic workflows.

The most expensive task run with GPT-6.1 Soul cost approximately $1.38, demonstrating its operational efficiency.

In contrast, the cheapest task run with Opus 5.5 came in at $5.12, revealing a real-world cost gap of roughly 4 to 5 times more expensive for Opus 5.5.

Astra, while highly intelligent, suffered from unpredictable performance spikes, leading the speaker to switch to Opus 5.5 for daily tasks due to frustration with its inconsistency.

Soul 6.1 is described as a less 'spiky' model, offering a more trustworthy and reliable experience for complex tasks, despite its peaks not being as high as Astra's revolutionary 3D capabilities.

Testers like Julius and Ben noted that Soul 6.1, while feeling 'boring' compared to Astra, consistently performs tasks effectively.

The speaker successfully deployed Soul 6.1 to automate email management and process financial invoices, highlighting its trustworthiness in critical applications.

The model demonstrated its capability by organizing Chrome tabs, filling in wire transfer details, and accurately identifying overlooked invoices and potential errors.

This level of reliability in automation stands out significantly, especially when contrasted with the inconsistent performance often observed in models like Astra.

GPT-6.1 Soul maintains the 'Rottweiler' characteristic of OpenAI models, demonstrating a deep capability to uncover code bugs.

In an audit of orchestrator v2, Soul 6.1 outperformed both Astra (83.8) and Grock 4.7 (80.7) by achieving a score of 87.4 in improving T3 code, showcasing its thoroughness despite burning more tokens.

The cost for the T3 code improvement task was notably lower for Soul 6.1 at $2.15, compared to $5.00 for Opus 5.5 and nearly $9.00 for Sonnet 5.5.

Sol 6.1 demonstrates performance levels competitive with Opus 5.5 and Sonnet 5.5, positioning it as a strong contender in the market.

The cost for running Sol 6.1 is a mere $7, significantly lower than $15 for Sonnet 5.5 and $50 for Opus 5.5, making it highly cost-effective.

While Sol 6.1 proves efficient by wasting fewer tokens than its predecessors, Opus 5.5 still retains an edge for long-term unattended development tasks.

The efficiency of Sol 6.1's context management led to a significant reduction in projected costs for the speaker's sessions, dropping from $5,700 to $1,550.

This efficiency is further evidenced by an average of 110,000 tokens read per request for 6.1, a substantial improvement over the 360,000 tokens for Sonnet 5, diminishing the financial concern over bloated system prompts.

Jev Router, designed to optimize costs by intelligently routing requests between different models, proved ineffective.

Tests revealed that the router indiscriminately sent 60% of requests to Deepseek 4.1 Flash, regardless of the task's complexity, leading to higher costs than directly using GPT-6 Astra.

The fundamental flaw was that a model lacking reasoning capabilities and context cannot accurately assess the difficulty of a given problem to make optimal routing decisions.

The speaker intends to utilize Opus 5.5 as the primary model for high-level tasks, leveraging its strengths for broader strategic functions.

Sonnet 6.1 will be specifically delegated for detailed investigative work, bug triage, and in-depth analysis of codebases, creating a specialized and efficient workflow for code review and development.

While the model generated high-quality 3D graphics and intricate plant details, it produced an unacceptably cluttered user interface with excessive, unnecessary text.

Movement mechanics and frame rates in the AI-generated game felt significantly inferior compared to versions created by Opus or Sonnet, indicating a regression in user experience quality.

Despite robust 3D modeling capabilities using Blender via CLI, the AI exhibits poor design taste for frontend components, necessitating human intervention.

The speaker emphasized that if a model for creating functional and appealing frontends is desired, funds should be allocated elsewhere due to the current model's deficiencies.

The cost of generating the game demo was approximately $5, signaling an imminent shift in game development economics.

The industry is approaching a critical threshold where AI models' enhanced understanding of three-dimensional space and necessary tooling could enable full game production in minutes on demand.

This improved comprehension marks a pivotal moment, poised to fundamentally transform traditional game development workflows.

The author deployed the model on internal projects like Lakebed, where it effectively identified bugs in T3 code that Fable and Opus had missed.

The model utilizes a combination of sophisticated code analysis and computer usage to detect regressions with high accuracy.

While the author places significant trust in Soul 6.1 for reviewing code, he currently does not trust it for writing production-level code.

This positions Soul 6.1 as an invaluable tool for quality assurance and bug detection within development workflows.

The author prefers using Soul 6.1 and Claude Opus in tandem, leveraging each model's distinct strengths.

Opus is favored as the primary coding collaborator for implementing solutions, offering a more pleasant and unblocked development experience.

Soul 6.1 excels in auditing and detailed investigation, effectively reviewing code produced by Opus and digging into granular issues, creating a robust professional workflow.

The release underscores the competitive pricing and high value of modern AI tools, signaling a potential shift away from the current 'subsidization era' for AI software.

Despite its capabilities, the author still finds Opus to be a superior overall collaborator, suggesting a nuanced integration of AI tools based on specific task requirements.