UltraFast AI: Breakthrough Speed, Staggering Cost

UltraFast AI models provide absurdly quick build times, enabling real-time interaction where code updates are visible to the user instantly.

This model speed creates an experience that feels faster than human interaction, opening practical use cases for live prototyping and immediate user experiences.

However, the cost escalates dramatically: standard Astra is priced at $50 per million output tokens, while UltraFast can jump to $300 per million, with long-context projects pushing costs as high as $450 per million output tokens.

Theo - t3.gg states, "Not only can it build absurdly quickly with a model like Astra, it also can burn through your rate limits and bills even faster."

Additional costs include $6 per million for cache reads and $75 per million for cache writes, contributing to the overall high expense.

Greptile introduced T-Rex, a new solution that allows review agents to run on a real computer, enabling actual testing of code changes.

This system moves beyond mere static code analysis, actively testing the code to catch regressions that might be missed by just reading the code, thereby improving review reliability.

Theo - t3.gg explains, "T-Rex is a new solution that Reptiles put together to allow your review agents to run on a real computer that can actually run and test the changes. So, it's no longer just looking at the code and giving it a thumbs up that it thinks it's fine. It's actually testing the code itself."

The $500 monthly subscription fails to provide sufficient volume to justify its API rates, leading Theo to conclude the service is impractical for most users.

Even the $500 per month UltraFast plan, a subscription tier offered, provides an insufficient volume of usage for the cost.

Theo - t3.gg concludes, "I don't think almost anyone should be touching ultraast right now. It's just far too expensive, especially over the API rates. But even the amount you get in the new $500 a month ultra fast plan they offer is just it's not enough."

These substantial credits are consumed at an alarming rate, taking approximately 2.1 hours of active use to burn through the entire weekly allowance.

Theo reports that during a demonstration, two simple prompts consumed 1% of the total monthly allowance in under a minute.

Theo states, "They are subsidizing it heavily. So in that ultra fast $500 plan, you get about $1,000 of usage per week, which takes a whopping 2.1 hours to burn when you're using ultraast."

UltraFast models enable a novel working relationship with AI, allowing users to observe code changes and live deployments as they occur, eliminating traditional waiting periods.

This setup encourages a split-screen approach, integrating the application and the AI model, and effectively removes the 'wait-and-return' pattern common with slower AI tools.

The experience closely mimics the traditional developer flow state, characterized by seamless transitions between coding and viewing immediate output, fostering a more continuous and focused development cycle.

Theo states, "You can have a very different working relationship with a setup like this. I found that it like actually makes sense to split screen with your app and your model again because I'm not just asking it to go do the thing and then looking at it when it's done."

The rapid inference capabilities of UltraFast models shift the bottleneck from model generation time to tool-call latency, requiring developers to rethink their optimization strategies.

To maximize efficiency, users must avoid unnecessary tool calls, such as browser previews, which can double the runtime and negate the benefits of speed.

Steerable models like Astra allow for iterative adjustments even while the AI is actively processing, facilitating a more dynamic and responsive development process.

Theo - t3.gg notes, "When your token generation took 10 minutes and your git commands or builds or whatever took under 30 seconds, that ratio meant that the token time was so much higher that you didn't care about optimizing those other pieces."

UltraFast's system leverages Nvidia infrastructure, delivering genuine high-speed performance that underpins its unique developer experience.

The integration of voice commands with live code execution fundamentally transforms the user experience, making interactions feel seamless and intuitive.

The enthusiasm among developers stems from this immediate, back-and-forth interactivity, which creates a 'magical' feeling during the coding process.

Theo - t3.gg explains, "It is magic. Combine it with voice and the experience feels fundamentally different. And I understand why all those OpenAI employees were so hyped about this. It's because of this unique feeling with the live back and forth."

Despite these advancements, the high costs associated with UltraFast remain a significant barrier to broader adoption for individual or non-corporate users.

Giving users a 'max reasoning' setting invites predictably expensive outcomes, as Theo pointed out when citing instances where individuals depleted half a monthly plan in one hour.

The example of Primagen, who burned over half of a $500 plan in a single hour by using max reasoning, highlights the financial risks associated with such unconstrained features.

Theo argues that releasing a feature with such high costs without adequate guardrails or clearer warnings constituted a design mistake by OpenAI, leading to poor decisions by both the platform and its users.

Theo states, "Funny enough, when you give users a knob, they will turn it all the way. And that has resulted in a lot of bad decisions by both OpenAI for giving us the knob and the users for using the knob."

UltraFast enabled users to correct the chart orientation from horizontal to vertical for better screen fit and to request consistent branding using logos and colors inspired by labs.

The system allowed for the rapid implementation of sorting logic changes, such as removing averages and sorting by the highest option, maintaining a continuous 'in-the-loop' workflow due to sub-minute response times.

The speaker drastically changed their prompting style to direct, brief instructions, accelerating the design iteration process.

Theo - t3.gg notes, "I was in the loop for this... I made the switch not because I was so scared of wasting all my money, but because my food was in the oven and I wanted to be able to get up and do it and not feel like I was missing as much."

Initial issues included 'max reasoning' misleadingly skewing cost and token comparisons, as well as confusing output token distributions for models like Sonnet, Opus, and Astra.

To improve clarity, models were grouped by family rather than individual performance, and interactive toggles (low, mid, high, max) were added to reveal model-specific efficiencies.

Theo states, "So instead, I group the numbers by the model family. And then the order is based on which one has the highest number, which has some fun side effects."

Despite Astra's inherent design limitations, rapid back-and-forth iteration enabled the creation of a superior design, demonstrating how developer-AI partnership can overcome individual model weaknesses.

Custom UI patterns were developed by testing multiple configurations within seconds, contrasting sharply with the standard speed experience of 'Soul 61', where waiting for results led to a loss of focus.

The developer executed five distinct prompts within four minutes, receiving responses in under 20 to 60 seconds for each, highlighting the efficiency of the UltraFast workflow.

Theo - t3.gg remarks, "And even though Astra is way, way, way, way worse at design than the anthropic models are, I made a better design because I was in the loop with it."

Real-time feedback enabled the developer to iterate rapidly on a comparison chart, facilitating adjustments to layout and panel positioning.

Rapid UI adjustments, such as panel positioning and layout, were implemented quickly during the session, demonstrating the efficiency gains.

UI features like hovering on 2D charts to isolate lines were also added swiftly, and the developer sent over 50 messages in a single thread to refine the design, indicating a highly interactive workflow.

Theo - t3.gg shares, "I only came up with this because I was staring at it while it was being worked on and could have the back and forth with the model and try different things."

UltraFast models showcased exceptional performance with tokens per second (TPS) rates between 320 and 340, significantly higher than regular models at 30 TPS and fast models at around 60 TPS.

The total cost for the project discussed was $286, comprising $36 for the main thread and $250 for follow-up work, a stark contrast to an estimated $90 with 'Astro' or merely $12 with 'Soul 61' for the same output.

This extreme price difference makes UltraFast modes financially prohibitive for the vast majority of use cases, despite their speed advantages.

Theo - t3.gg reports, "But I also found the cost for all of my threads working on this, that main thread was $36 and the follow-up where I did a bunch of additional work was $250."

Maintaining a developer’s cognitive flow and preventing context loss constitutes the primary advantage of UltraFast AI, according to the speaker.

This capability led the developer to adapt their behavior, opting for smaller, more frequent requests over large, complex ones.

Despite being objectively 'worse' at design than other models, UltraFast's speed allows for superior UI refinement through rapid iteration, and the developer suggests that current pricing tiers are misleading and require explicit warnings.

Theo - t3.gg explains, "The difference is that I just wouldn't have built it. If I had to wait between every step, I would have lost the context of what I'm thinking about."

The primary goal of current UltraFast usage is to explore and understand the model's capabilities through experimentation, even if it means substantial token expenditure by early adopters.

The speaker anticipates a future where such high speeds become affordable without incurring excessive costs, moving beyond its current experimental phase.

Theo - t3.gg clarifies, "The point of Astra UltraFast is not to be used. It is to be experimented with where lunatics like I can waste far too many tokens on it just to see what it's capable of."

UltraFast mode's exorbitant cost makes it nearly impossible to justify for most applications, as it is only deemed justifiable for extreme, high-severity incident responses.

It is generally not worth the expense unless an individual or organization has effectively unlimited funds, given the rapid consumption of credits.

A strategic use, such as deploying it right before account resets, is suggested to maximize the value from its prohibitive cost.

Theo states, "It's just so expensive it is basically impossible to justify. There is one other time you can justify it though and for me that time is right now."

Analysis of the $200 tier, which includes Sonnet 3.5 and standard Astra, is pending for a future video, suggesting potential for better value at that level.

Theo - t3.gg advises, "Don't upgrade to the $500 tier just for this yet. It's not worth it."