Naveen Rao on AI's Energy Challenge

He views AI as one of the most transformative technologies ever created, believing it will enable humanity to reach unprecedented levels of progress through hardware innovation.

Rao founded Nervana Systems in 2014, an early AI chip company that was later acquired by Intel, where he then led the AI group until 2020.

His work at Intel focused on building infrastructure to scale Large Language Model (LLM) training and platformizing GPUs, before he joined forces with Databricks in 2023, where his segment now contributes a quarter of the company's total revenue.

The initial goal was to achieve a 1000x improvement in power efficiency within five years, a target Rao has now accelerated to 3.5 years, driven by the increasing ability to use AI itself to solve complex scientific problems more rapidly.

Unconventional AI employs a comprehensive top-to-bottom strategy, involving a diverse team from theoretical researchers to physical engineers.

The company's theorists, who hold math and neuroscience PhDs, concentrate on developing concepts that minimize data movement to enhance power efficiency, which are then translated into models, trained on real data to validate their effectiveness, and finally built into physical circuits and systems by the hardware team for production.

This consumption is significant when compared to the U.S.'s total data center allocation of 40 gigawatts and the world's total capacity of under 100 gigawatts.

Such figures indicate that the energy demand for AI is projected to outstrip global supply within the next three years, creating a critical bottleneck for further development and deployment.

The priority in data center infrastructure development has dramatically shifted from floor space and networking capabilities to GPUs, and now predominantly to energy availability.

Energy contracts have become the essential prerequisite for initiating any new data center construction, with approximately 50% of the cost of serving a single AI token directly attributable to energy consumption.

Biological systems demonstrate remarkable energy efficiency, with the human brain operating on approximately 20 watts and a monkey brain on just 1 watt, comparable to a smartphone.

Even a squirrel's brain, capable of complex physical maneuvers, consumes a mere 8 milliwatts, highlighting biology's capacity as an ideal physical substrate for intelligence.

High-end GPUs, by comparison, move nearly 30 trillion bits in and out of memory per second, reflecting a fundamental architectural approach that has remained largely unchanged since digital computing emerged in the 1940s.

Early computers like ENIAC were designed primarily for speed, relative to manual human calculations, rather than energy efficiency, a prioritization that has largely continued.

Although transistor counts continue to increase, the traditional scaling of frequency and single-thread performance has plateaued, signaling that the era of efficiency gains solely from smaller transistors, often referred to as Moore's Law, is largely over.

Current digital systems, operating on binary ones and zeros, rely on multiple levels of lossy abstractions that are inefficient.

The human brain, in contrast, performs intelligent functions through the physical behavior of neurons, without relying on linear algebra or floating-point mathematics, suggesting that a more direct abstraction of semiconductor physics, tailored to neural network requirements, could significantly simplify and improve computing.

Nature provides numerous examples of computation through emergent behaviors, such as the synchronized movements of bird flocks or the collective decision-making in ant colonies.

Dynamical systems theory explores how complex, emergent properties arise from the simple behaviors of individual components, offering a framework for understanding such phenomena.

A physical model illustrating this concept involves synchronized metronomes placed on a movable plank, where their interconnectedness leads to system-wide states of synchronization, demonstrating a new model for computation.

The UNO model, developed by Unconventional AI, demonstrates the practical application of oscillator systems in real-world computation, specifically for image generation.

This model leverages a network of physical oscillators, which are simulated and trained to perform complex image generation tasks.

The successful implementation of UNO serves as concrete proof that scaling such dynamical systems can produce useful and tangible outputs, like advanced image generation capabilities.

This approach marks a significant departure from traditional digital computing methods, potentially paving the way for more energy-efficient and biologically inspired AI systems.