Jev, a New Class of AI Models, Reimagines AI's Future

Diogo Almeida describes himself as a "ragged corpse" due to the demanding launch phase of Jev, noting he is a technical CEO dealing with many "fires to fight."

Despite the physical exhaustion, he feels a strong mental alignment with reality, noting that after years of observing the AI field as a "carnival house of mirrors" where many claims make no sense, he feels better in sync with reality this week.

Almeida expresses gratitude to the developer community, who he believes understand the true potential of Jev.

Diogo Almeida identifies "mode dropping" (or "mode collapse") as a significant, often overlooked downside of Reinforcement Learning from Human Feedback (RLHF).

He explains that models trained with RLHF tend to become extremely conservative to avoid easily detectable errors in long-sequence outputs, rather than accurately reflecting the model's internal confidence.

Almeida questions Yann LeCun's assertion that long-sequence error rates are an inherent mathematical flaw, arguing instead that these errors are influenced by how models use "mode dropping" to maintain conservatism.

Diogo Almeida expresses strong opposition to public benchmarks, viewing them as antithetical to genuine AI intelligence measurement.

He claims that AI labs have historically oriented their data collection efforts to mirror benchmark criteria, such as MMLU, which he calls "benchmarking with extra steps."

Almeida advocates for evaluating AI through specific, real-world workflows that matter to developers, rather than relying on public tests which he believes are "extremely gameable."

Almeida argues that overfitting to present-day data would hinder the development of long-term, "sci-fi" future infrastructure designed to serve as foundational layers.

He likens TypeSafe's models to fundamental network protocols like TCP, suggesting they serve as a base upon which complex software can be built.

The company's approach focuses on a "cognitive core" that identifies "jaggednesses" in models and addresses them surgically to achieve general applicability.

Diogo Almeida describes the AI industry's tendency to "overpromise and underdeliver" as a tragic failure, asserting that while AI is capable of solving complex math problems, it still struggles with basic rote automation.

He notes a lack of the right "plugs" to connect AI's inherent intelligence to economically valuable work.

Both Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from Vision and Language Rewards (RLVR) are identified by Almeida as extreme perpetrators of this overpromise-underdeliver gap.

He believes the threat of an AI winter has been averted, as Jev's early usage data shows it is being used for real-world work.

A key milestone for Jev has been surpassing a trillion tokens per day, which Almeida notes indicates consistent machine usage rather than fleeting human experimentation.

Diogo Almeida differentiates between strict determinism, where identical inputs yield identical outputs, and robustness, which he champions as a more important property.

He defines robustness as the ability of a model to produce semantically similar outputs when given similar inputs.

While strict determinism can be offered for specific use cases like unit tests, Almeida notes it is a trade-off that often comes at the expense of "intelligence per dollar."

TypeSafe is committed to performing "disgusting things" behind the scenes to maintain its position on the intelligence-per-dollar frontier, continuously pushing the boundaries of what is possible.

Diogo Almeida acknowledges that maintaining consistent intelligence while optimizing for speed and cost is a "load-bearing statement" and a significant challenge.

The company uses internal evaluations to measure progress but enforces strict discipline to prevent gaming these metrics, which Almeida identifies as a top-level priority.

"Truth-seeking" is emphasized as a critical cultural value at TypeSafe, necessary for an honest assessment of model performance.

Almeida introduced new API primitives like "Bernoulli" (a continuous score based on Bernoulli probability) and "Score," which are intentional departures from standard programming types to provide more precise mapping to model outputs.

Diogo Almeida declares that TypeSafe's models are designed for deep integration within the "guts of programs," envisioning a future where AI operates more massively than current shallow tasks.

He criticizes the use of "system messages" in traditional AI interactions, likening them to "disgusting global variables" that hinder effective instruction following.

Jev encourages developers to utilize structured JSON objects for state, instructions, and criteria, moving away from string templates to enhance programmatic control.

This deep structural integration is intended to allow for better engineering and verification, as decomposing problems into smaller, programmatic decisions allows them to be more evaluable.

Diogo Almeida advises breaking down complex problems into their smallest possible semantic units, focusing on the "lowest level thing" for AI prompts.

He stresses the importance of keeping prompts extremely structured, explicit, and literal to ensure models follow instructions precisely, defining this as the essence of "the art of programming."

For large-scale production applications, maintaining precise, query-based breakdowns is preferred over more hybrid or lazy prompting approaches to ensure reliability and verifiability.

Almeida suggests that if a model fails to refuse in a specific scenario, the solution is to add a new question to the prompt, which he equates to fixing a bug, rather than retraining the model.

Diogo Almeida expressed his desire for TypeSafe to become a reliable utility in the AI landscape, emphasizing a commitment to foundational technology rather than succumbing to market hype.

He stated that the company will not release products merely for the sake of doing so, and everything must align with three foundational steps or axes.

Almeida underscored that this approach differentiates TypeSafe by standing for something enduring, even if it means becoming a 'boring' utility like Visa in the future, rather than a constantly trending entity.

"I don't know if exactly the thing that culture is what you do and the market doesn't reward it. And I really like that because I think that we are standing for something."

He also expressed a desire to rally the world towards this vision, doing cool things not out of necessity, but to show that current advancements are just the beginning of machine-native intelligence.

Diogo Almeida suggests that people focus too much on speed and cost, overlooking the importance of reliability in AI systems.

He believes that reliability is what allows users to trust AI and what makes software delightful.

Almeida emphasized that despite the launch of Jev, the challenging work is far from over, with many more complex problems yet to be solved.

He expressed a desire for AI to integrate more into the background of daily life rather than always being a foreground character.

"I I don't think so and like I really really think that people focus too much on speed and cost and not enough on reliability like reliability is what makes it delightful like reliability is what like allows you to trust it"

Before the launch, Diogo Almeida harbored concerns that their approach might be perceived as a 'vitamin' rather than a 'painkiller,' struggling to find immediate and clear utility for users.

However, adoption after the launch suggested that the community implemented use cases far beyond what the TypeSafe team initially demonstrated.

"We put a lot of work because we thought that education would be like a gigantic bottleneck for us. Um I it probably works and it's no longer a problem because people are doing things like well beyond what we could ever show you how to use your model."

His primary focus is to ensure that Jev functions with unwavering reliability in real-world scenarios, prioritizing actual performance over staged presentations.

Almeida emphasized the critical role of internal team testing to rigorously identify and address any weaknesses within the system.

The ultimate objective is to transition from experimental applications to truly reliable, production-ready use cases that can seamlessly integrate into various computational environments.

"So, just like I'm anti-benchmaxing, I'm also anti-demos. I want to make sure that it works reliably. I love people are playing with it. This is super sick. Have no doubt. I want to see this. I want to see it be used. I want our team to play with it. I want to find the weaknesses and I want to solve that."

Diogo Almeida criticized the industry's singular focus on RLVR (Reinforcement Learning from Verifiable Rewards), asserting that it is not solely about verifiable rewards and has shown limitations since before the reasoning revolution.

He argued that the 'pace the frontier' discussion is too narrow, as it assumes RLVR is a universal requirement for AI advancement.

Almeida stated that for the specific shapes of his models, zero RLVR is optimal, suggesting that other approaches exist.

He contended that some labs prioritize RLVR because they believe it's the only route to powerful AI, despite the potential for it to induce dangerous behaviors.

"RLVR is Like so RLVR is not actually about verifiable rewards like that has been failing since before the reasoning revolution like like like and that's the weird part about tasks right like back when oh fun history back when RHF was becoming a thing thing there were three different things that like are now called post training different efforts and instruction following was by far the like the the vaster child like people didn't like it they didn't want to take it into account it was annoying you know like I talked to the pre-training team. I'm like, "Guys, this is the magic." And they're like, "We'd run so many model sweeps. You want us to wait for human evals to figure out which models to use?" And like everyone is like, you know, giving tons of like resources to like the codegen team, which like they did have some successes, but they were trying really hard to do RL on like unit tests, and it didn't work obviously, right? Like you needed reasoning for that. So uh re so just to be clear RVR is not purely about the reward. It's about like the shape of everything to and part of it is that reasoning is included in here like this latent variable that you're doing things and when you're doing things you're just letting the models do whatever they want in order to make them be as powerful as you can to answer the hardest problems. >> [snorts] >> And this whole pace the frontier discussion I think is like a very narrow focus because it assumes that everyone needs to do more RLVR, right? Which um like I obviously don't think I need to do more RLVR on our models, you know? I think zero is the optimal amount for our shape, right? Like come on, you know? So um it's really I think a bit of a slight of hand where they are saying that we actually want to keep doing the thing that looks dangerous"

Diogo Almeida critiqued the narrow focus of labs, arguing that researchers are not adequately exploring diverse new directions and 'North Stars' for AI.

He asserted that researchers, rather than the general public, are responsible for this limited exploration, as they possess the expertise to understand the true optionality available in AI development.

Almeida characterized existing approaches like RLHF and RLVR as merely '0.5' or '1' out of many potential paths, advocating for a broader range of methodologies.

He stated his personal philosophy that any insight shared privately with an investor should also be shared publicly, and that he would not pre-train if given a billion dollars.

"If you think about course I think there's like like on the bitterest lesson direction I think that there's very few people who've like made right tasks you know like new directions of AI that is or new new north stars that is rare again like I think 2.2 two times or something for LLMs itself like RHF and then RLCD RLVR is like a 0 2 in my opinion and I think that's generous um but or.5 or like it could be one whole one I don't really care um but I do think that people are thinking very closedmindedly about this type of thing and um this the only people who are at fault here are the researchers because it's definitely not the populace you know like they just assume that open anthropic are just doing the best they can and they are not the experts who are aware of the true optionality available."

Diogo Almeida argued that current AI models, often focused on chat functionality, are prone to 'cheating' due to their simple linear nature.

He stated that OpenAI, during his tenure, overly emphasized 'chat' development, neglecting the deeper study of subtle intelligence.

Almeida explained that reward models in AI systems compel models to be 'hyperconfident' and miscalibrated, which distorts their probabilistic reasoning.

This process of forcing models into specific behaviors, like identifying as a particular entity, fragments intelligence rather than fostering smooth, predictable outcomes.

He reflected on the two-year journey of TypeSafe and recalled his intense efforts at OpenAI before the company's founding, particularly around Thanksgiving, where he utilized available GPUs for instruction-following models.

"a lot of the art to intelligence is studying this subtlety that I think that um at least when I was in open AI people were not really studying that because like they were just like chat chat chat chat"

He observed that regulatory concerns are significantly shaping the strategic decisions of large AI organizations, moving them beyond purely technological development.

Almeida described this shift towards political maneuvering as being outside the purview of a pure technologist, expressing discomfort with the alleged manipulation of public perception.

"it is this is a political thing much more so than a pure X-T thing. Uh, so yeah, political positioning and that and that's beyond my that's well beyond."

He also shared his personal 'North Star' for AI development: reliability, programmability, composability, and cost-effectiveness, and highlighted games as a significant area for intelligent AI applications.

He asserted that freeing coding agents from this limitation could enable superior abstraction, decomposition, and overall software development practices within AI systems.

Almeida revealed his intention to release a document detailing these insights, aiming to stimulate further research and exploration into alternative programming patterns for intelligent agents.

He emphasized that escaping the KV cache would allow for more intelligent handling of state, such as enabling sub-agents to perform easier tasks without excessive state passing, and facilitating efficient historical context lookup.

"the thing that I would really really like to explore is like coding agents free from the tyranny of the KV cache."

Diogo Almeida expressed feeling disempowered during his time at OpenAI, where he noted a pervasive focus on chat interfaces.

He criticized the function calling interface as 'insane' and 'anti-developer,' questioning its deployment.

Almeida advocated for the inclusion of confidence scores or probability thresholds for functions, arguing that developers need better control mechanisms than simply 'begging' the model for desired actions.

He noted that existing coding agents are often highly overfit to their existing harness, leading to 'jagged' behavior and poor use of external tools and MCPs due to overfitting. He also expressed excitement for the prospect of an 'AWS of intelligence' and noted that the company wants to ship more 'shapes of intelligence' beyond its initial model, Jev. Almeida also mentioned the performance impact for European users due to the lack of servers there.

"I just really did feel disempowered when I was at OpenAI, you know, like I felt um... like I just felt like in the the the insane house where everyone is just like chat GPT. Yeah. Like where do we put chat GPT and everything? How do we make chat GPT good for like you know developers and stuff? And I'm like what what are you talking about? Like the function calling interface is insane. Why would you deploy this? Um like this is this is so anti-developer."