Rémi Louf Argues Against AI Agent Frameworks, Advocates for Building Minimal Runtimes

Rémi Louf dedicated two weeks in January to intensively explore AI agent development, driven by the emergence of powerful models like Opus 4.6. His goal was to move beyond theoretical understanding and engage with the practical implementation of these technologies. Louf stated, "I just told my CTO I was like okay I'm just going to go away for two weeks and I'm just going to dive in this thing and try to understand what we can get out of it."

Louf compared his ideal AI agents to an autonomous robot mower that operates without constant supervision. He aimed to automate his morning routines, such as reviewing market news and CRM data, to achieve a 'set it and forget it' experience. Louf articulated, "I kind of wanted the same thing for my morning because my mornings are always the same thing the first couple hours."

Current AI agent interfaces demand continuous human oversight, akin to needing to stay on a self-driving tractor mower. Louf described early mobile agent interfaces as feeling like 'SSH with vibes,' implying a lack of true autonomy. Users often find themselves needing to constantly monitor and steer agents through their tasks, which defeats the purpose of automation. Louf explained, "It's kind of the equivalent of having a robot like a tractor mower that you still have to stay on even if it's driving by itself, right?"

Instead of relying on often complex and constraining frameworks, Louf opted to build the simplest possible solution for an agent runtime. He noted that frameworks frequently force developers into a cycle of constant prompt editing within their code, hindering flexibility and version control. By externalizing logic into separate files, tracking changes and collaborating becomes significantly easier. Louf stated, "I started to build the dumbest thing that could possibly work."

Louf used YAML for agent configurations to significantly enhance version control and development workflows. This approach allows for clear diffing, streamlined pull request reviews, and simpler implementation processes. He noted, "I still found that this was actually a lot easier to start implementing agents without code. You can version it, you can diff it, you can review in the PR." This system enables a straightforward workflow where agent definition files can simply be dropped into a folder to initiate their execution, further simplifying deployment.

Louf highlighted the limitations of traditional cron jobs, which operate on fixed time intervals rather than in response to dynamic state changes. He emphasized that event-driven systems, such as those triggered by voice notes, emails, or pull request updates, offer greater flexibility and responsiveness for agent workflows. Agents should be designed to define their inputs and outputs as events, enabling better modularity and seamless integration within a responsive system. Louf explained, "it's not just a prone job and that and that means that you know agents are the Voice note processor is just you know not a chron job but here you have accepts and returns."

Instead of relying on rigid, hard-coded graph structures, agents can be orchestrated by subscribing to events emitted by other processes. For instance, a daily brief agent can process the outputs of a voice note to trigger Slack notifications. This event-driven approach eliminates the need to maintain complex graph edges, as agents simply react to relevant events. Louf stated, "you do not need graph uh in this case all you need is events. You have no edges to maintain agents sub simply subscribe to events."

Initial failures in Louf's system included duplicate Slack messages, missing voice notes, and low-quality prompt outputs. A significant challenge was the lack of version control, which made it difficult to pinpoint the causes of inconsistent prompt performance. Louf explained that each of these failure modes became an opportunity to build fundamental infrastructure components, such as logs and queues, essential for a robust agent runtime. He noted, "each of these failure modes that you found actually led to building one piece of what turned out to be a runtime."

Louf developed a content-addressed system for managing prompts, similar to Git or Nix, to ensure reproducibility and track changes effectively. This system prevents the generation of 'garbage' output that often occurs when prompts lack proper versioning. He highlighted that this infrastructure was built out of practical necessity rather than an initial desire to design a complex framework. Louf elaborated, "I just ended up uh building a content like a content addressed system for this a content address system. You can think of git uh you can think of nyx and any other build system."

Louf emphasized that the log serves as the system's persistent memory, ensuring that no data is ever lost during agent execution. This fundamental component is critical for maintaining data integrity and system reliability.

All system events are meticulously captured in a single, append-only log, ensuring comprehensive observability. These events are causally linked, allowing developers to trace which event triggered another, which is invaluable for debugging complex multi-agent interactions. Louf stated, "everything is observed. You only have one append only event stable on the left. It's a real common line in like command zita events and you get all the events. They are causally linked as well like you know which event triggered which event which happens to be super useful when you're debugging."

Louf's system stores each component of a prompt—system prompts, tool descriptions, and user messages—as unique hash-based identifiers. Prompts are then represented as a list of these hashes, rather than raw text strings. This graph-based structure allows for precise tracing of the exact content sent to the model, similar to how build systems manage dependencies. Louf elaborated, "This is basically how build system works. You have different parts for a prompt. Each one of those is stored and addressed and you know stored somewhere as a identifier which is a hash. So when we build a prompt instead of building a piece of I mean before rendering the text we actually represent the prompt as a list of these hashes."

Representing prompts as graphs of hashes significantly simplifies context and KV cache management by allowing manipulation of graphs instead of raw strings. This approach provides full auditability, giving developers a clear understanding of why a model produced a specific output. Louf remarked, "it makes compaction a lot easier. You're just manipulating a graph right? You're not manipulating strings. It's just a lot easier and it makes KV cache management a lot easier as well indirectly. But I think that when you know I guess probably the main advantage that's when you use that skill is really auditability."

The graph-based prompt system enables developers to generate diffs, allowing them to precisely compare different agent runs. This functionality reveals exactly what components changed between executions, whether it was the user message, a tool provided, or an agent's skill. Such detailed visibility is crucial for understanding shifts in performance or output quality. Louf explained, "what you get once you have this graph is you get diffs. Like you can say, okay, what changed between these two runs? Like which components changed? Was it just my message? Did I like give the model a different skill? Did I give it a different tool?"

Louf described a kernel model where agents function similarly to processes in an operating system, allowing them to be scheduled and isolated. The markdown definition of an agent remains in user land, separate from the system's internal definition. This separation ensures that the system can manage and isolate agent processes effectively, regardless of their specific functions. Louf stated, "let's say a kernel like runs processes and your agent kind of is a process. It doesn't matter what it does actually. Uh but the system can schedule it is built to isolate it."

Reliable structured output is a core competency for Louf's company, as initial attempts with proprietary models resulted in a 20% failure rate. He stressed that the kernel's fundamental role is to prevent undesirable actions, not merely to reduce their likelihood. This focus ensures the integrity and predictability of agent operations. Louf emphasized, "the job of the kernel is actually to make bad actions impossible, not just unlikely."

Louf highlighted two non-negotiable boundaries for agents: typed tool calls for interaction with the external world, and strictly typed events for inter-agent communication. These stringent typing requirements are crucial for preventing common errors and ensuring robust system operation. By enforcing these boundaries, the system ensures clear and predictable interactions between agents and their environment. Louf affirmed, "you have this two boundaries with the between agents and the external world. The first one is type tool calls... and also the boundary of other agents which is typed events and this is non-negotiable."

Louf strongly recommends building internal agent runtimes before considering purchasing external solutions, especially for smaller companies or tech CEOs. This hands-on approach allows for a deeper understanding of specific needs and limitations, which is essential for effective implementation. He challenged current framework builders to 'eat their own dog food' and urged leaders to immerse themselves in the technology to grasp its intricacies. Louf advised, "I would advise that today like definitely start building before you buy uh so if you're a small company if for tech CEO it's kind of an advantage because you can just do this without you know tasking engineers to do this."