Unpacking True Recursive Self-Improvement

Dhaval Adjodah, co-founder of MakerMaker.AI, described this paradigm shift, stating, "For the first time, we could swap the optimizer as the optimiz. And this leads to a reinvestment of cognitive improvement into itself."

The objective extends to optimizing the entire AI stack, encompassing everything from user prompts to underlying hardware, enabling continuous reinvestment of cognitive enhancements within the system.

Owen He, a former research scientist at DeepMind, entered AI with a background in formal language and logic, motivated by the challenge of creating machines capable of proving theorems. He said, "I was taking a class called formal language and logic and they were showing how you can build a theorem prover and I was like oh that would be cool I can just build a machine that prove theorems for me that's how I got into AI."

Dhaval Adjodah explained that in autocatalysis, despite advanced AI tools, "The improver is still human with tools even though these tools are getting better and better but the idea and the improvement come from human brains."

Martin Casado noted that this human-centric model is not new, reflecting a historical pattern in human civilization of using tools to create better tools, whereas genuine RSI necessitates the AI itself serving as the optimizer.

Dhaval Adjodah stated, "If you have a gap in the system, you can't have a closed loop and autocatalytic has no closed loop, right? So whatever you improve, there's still like a gap."

Conversely, a true closed-loop system allows the AI model to function as its own feedback mechanism, fostering improvements that are either constant or increasing over time.

The fundamental objective is to eliminate the need for external human resource injection, thereby enabling the system to drive its own continuous self-improvement.

Martin Casado emphasized that claims of Recursive Self-Improvement (RSI) must account for all associated costs, including human intellectual capital and the expenses incurred from unsuccessful experiments.

The speakers underscored the importance of rigorous company discipline in meticulously tracking the true costs, time, and resources expended, and the team employs an internal 'bleederboard' to monitor experiment costs across the organization, a practice Dhaval Adjodah noted as good company discipline when he said, "I we do and I think it's it's good company discipline to do that."

Owen He highlighted that training costs in AI can significantly decrease through algorithmic improvements, data efficiency, and kernel optimizations, rather than solely relying on hardware scaling via Moore's Law, stating, "What has improved is like algorithmic techniques, data techniques, kernels, all of that kind of stuff."

The NanoGPT benchmark illustrates this trend, showing a reduction in GPT-2 training time from weeks on multiple GPUs to mere hours on a single GPU, with the overall cost to train GPT-2 reportedly dropping to approximately $100.

Dhaval Adjodah explained that even if only the denominator is improved, "you could get faster training because now you can get more rollouts," leading to the development of smarter and more capable models.

This emphasis on compute efficiency remains a highly guarded secret within the industry, recognizing its strategic value in accelerating model development and performance.

Dhaval Adjodah noted this issue, stating, "What we found is that a lot of the papers that claim to do RSI, when we test it in our internal infra, we find that they don't work."

The firm therefore prioritizes establishing a grounded mathematical framework to comprehensively map the problem space, aiming to generalize solutions across various challenges rather than constructing overly complex or Rube Goldberg-style systems.

Current AI research often operates as disparate 'STEM collections' rather than being integrated within unified, physics-based mathematical frameworks, which can lead to fragmented improvements.

Dhaval Adjodah and Owen He stated, "So without a unified math framework, all these research are sort of just stem collections... so what we have is that we're putting all these components that you could possibly recursively improve into a unified framework and look at it from a holistic view so that improving one part um does not negatively impact the other part."

The goal is to develop a holistic perspective where individual optimizations are harmonized, ensuring that improvements in one area do not inadvertently compromise other parts of the system.

Dhaval Adjodah and Owen He explained, "You can think of RSI at many different levels of the stack, right? You can think of um prompts, um harness, weights, and you can go all the way to basically chips, right? And our our goal here is to optimize all the levels of the stack."

The team focuses on co-optimizing interconnected components like weights and harnesses, balancing capital efficiency with incremental gains through iterative improvements across the entire stack.

Dhaval Adjodah and Owen He highlighted that inference optimization simultaneously improves both the 'numerator' (performance) and the 'denominator' (efficiency or cost), stating, "The more you can get improvements and and efficiency out of inference the the more everything kind of unlocks... there's two improvements going on here one of them is the ability of the model to make inference faster. And then the second one is now that the inference is faster, it can do that faster."

Faster inference generates a compounding effect, allowing the model to accelerate its own processes, and efficiently serving models is identified as a critical economic challenge for the current AI industry.

The system employs multiple large language models (LLMs) to manage diverse tasks, with an autonomous agent often determining the optimal model choice for each specific task.

Dhaval Adjodah and Owen He emphasized a 'hands-off' approach, stating, "It's our agent because it's recursively self-improvement and what we're trying to do is not to limit itself to only one component... even that master prompt that has been improving."

This approach allows the system to independently identify and pursue optimization opportunities, extending even to the 'master prompt' that governs these actions, which itself undergoes recursive improvement.

While many AI companies aim to replace humans with AI, the speakers from MakerMaker.AI contend that human researchers and engineers bring invaluable contributions that are difficult for AI agents to replicate.

Dhaval Adjodah/Owen He articulated this perspective, stating, "I think the way that a lot of AI companies are thinking is you know let let us use AI to to replace humans. But what we found is that there's there's so much that if you have good researchers and good engineers that they can bring to the table that's going to be very hard to to replace by agents. And so that's why from the very beginning we've tried to think of our basically entire company as a multi- aent system where humans are a kind of agent that's cohabitating with other AI agents."

The company is structured as a multi-agent system where human expertise coexists and synergizes with AI agency, recognizing that true systemic progress arises from this collaborative interaction.

Dhaval Adjodah/Owen He explained, "I think for at some point you're going to reach plank constant you know plank length plank energy plank time right... So, if you use um I don't know benchmark accuracy, it's upward bounded by 100%. You're never going to see exponential improvement on that because eventually bottleneck 100%."

Performance metrics like benchmark accuracy are inherently capped at 100%, implying that continuous exponential growth on these specific measures cannot be sustained indefinitely.

Dhaval Adjodah/Owen He explained their approach, stating, "No, it's not belief. It's it's empirical measurement. But but also as a as a company there's a there's a very funny thing that that is happening which is because we have been so thorough, we have not released things until we're super sure it's correct, right? And every few months, you know, there's like an announcement of something RSI related, we look at it, we're like, we should have released our results or something. And then we look at it, we try to replicate it, and we realize, oh, it's noise, you know, it's cherry picking."

The company employs a 'viceroy of variance' to stress-test results and uncover flaws, underscoring that deep intuition derived from manual 'quality time' and high-volume experimentation remains crucial for validating AI advancements.

Dhaval Adjodah stated, "Intelligence is not a single scalar. It's more like a multi-dimension vector and there will be domains and field where this RSI will give you take off and there will be dimensions where you might plateau."

Amdahl's Law dictates that the overall system speed-up is constrained by its slowest component, with physical world interactions, such as wet lab experiments in drug discovery, often acting as major bottlenecks that can cause plateaus in improvement.

Dhaval Adjodah hinted at this by saying, "The secret to RSI is somewhere in a garden hose company in Wisconsin."

This perspective implies that, similar to how the theory of relativity was essential for GPS, current RSI advancements might be empirically 'fudged' before a comprehensive theoretical framework fully catches up.

The speakers defined Recursive Self-Improvement (RSI) as a system that simultaneously enhances both its underlying architecture (harness) and its model weights.

Martin Casado summarized this approach, noting, "The numerator is like how good the system gets and the denominator is how much energy goes into it. You believe you're seeing glimmers of the sign where both improve at the same time."

The ultimate objective is to optimize the ratio of the system's performance (numerator) to its energy expenditure (denominator), with the team claiming to observe early indications of both metrics improving in tandem.