NVIDIA Unveils Vera Rubin for Agentic AI

He highlighted the definition of a new era of inference, driven by increasingly autonomous AI systems.

NVIDIA is introducing its Vera Rubin platform, designed to lead these innovations, with a specific emphasis on enabling advanced AI capabilities.

Ian Buck articulated that while data storage and communication are essential, the fundamental source of value generation in the modern data economy lies in computing. The transformation of raw data into meaningful calculations forms the core of this value creation.

This process of computing turns disparate information into actionable results, emphasizing its critical role over mere data accumulation.

The AI model ecosystem has experienced explosive growth, with over 3 million models now available on Hugging Face. Despite this vast number, approximately 3,000 models account for 90% of all downloads, indicating a concentration of popular and impactful models.

This diversity supports a wide array of workloads, encompassing critical applications in vision, life sciences, and robotics, showcasing the broad utility of AI.

This platform is built to handle the increasing complexity and scale of AI operations, moving beyond previous benchmarks.

The Vera Rubin platform maintains robust support for large-scale training, ensuring it can accommodate both model development and deployment. Its architecture is explicitly designed to power AI factories, which require a durable and efficient infrastructure for continuous operation.

Ian Buck noted that a 2023 benchmark, which laid the foundation for understanding value, focused on human-involved chat interactions.

Original metrics for these benchmarks utilized 1,000 input tokens and a 4,000 KV cache, reflecting the scale of early interactive AI. The current trend is moving away from simple chat turns towards sophisticated, automated agentic processes that operate with greater autonomy and complexity.

Agentic AI systems operate in continuous loops, interacting and adding to their context windows without constant human intervention. This persistent activity significantly increases the demands on KV cache and compute turns, as agents can perform turns as rapidly as compute resources allow, eliminating human wait times.

To support this massive state persistence, the KV cache must be effectively managed across an entire data center, reflecting the significant scale of agentic operations.

The shift from simple token-in/token-out models to complex agentic workloads introduces significant orchestration challenges. Modern AI operations require CPU-based orchestration to manage dynamic Kubernetes containers and helm charts effectively.

Models must now handle a wide range of tasks, including large language model (LLM) processing, advanced reasoning, and intelligent routing between various sub-agents. Furthermore, critical aspects like security and governance must be integrated, with models monitored to prevent issues such as hallucinations and inefficient token usage.

Modern AI models have grown too large to fit on a single GPU, making disaggregated inference the new standard for achieving large-scale AI factory throughput. NVIDIA's Dynamo software plays a crucial role by separating pre-fill and decode stages, maximizing token throughput.

To support the vast KV cache requirements across these disaggregated nodes, smart storage solutions and NVLink networking are indispensable components.

The Vera Rubin platform has achieved substantial performance gains in real-world agentic benchmarks, demonstrating its capability for high-throughput AI factory operations. New results from SemiAnalysis benchmarks indicate a 30x improvement in AI factory throughput for agentic workloads.

In some specific agentic workloads, the performance improvements have reached up to 60x, showcasing the platform's versatility and power. For data centers facing power constraints, the critical efficiency metric becomes total token throughput per megawatt, where Vera Rubin shows exceptional performance.

This collaboration is particularly effective for user interactivity, enabling rapid responses from AI models.

Performance benchmarks demonstrate impressive speeds, with the Gemma 4 model achieving 3,400 tokens per second and the Qwen 3.8 model reaching 2,529 tokens per second. These high throughput rates are critical for large context windows, such as 100K contexts, which demand significant compute resources.

This core is integral to the Vera Rubin racks, built for speed and fully loaded computational capabilities, aiming to produce answers in the quickest possible time.

The Olympus core features a monolithic compute die, eliminating traditional triple attacks common in CPU architectures. Its LPDDR memory subsystem significantly reduces latency by 40%, optimizing memory access for critical CPU calculations.

The design prioritizes minimizing memory latency, which is crucial for the rapid and iterative computations inherent in agentic AI workloads.

The criteria for evaluating data center CPUs have shifted significantly from previous eras, where single-threaded performance was paramount. Pre-cloud CPUs, often likened to gaming PCs, prioritized high single-threaded performance with low core counts, resulting in lower overall throughput but very fast individual operations.

The current AI era, however, necessitates massive core counts and superior efficiency to handle complex, parallel workloads. This marks a fundamental transition from legacy designs that favored individual task speed to architectures optimized for high-throughput server capabilities.

Cloud computing value is being redefined beyond traditional 'dollar-per-core' metrics, which historically dominated evaluations for generic tasks like web hosting. The Vera platform emphasizes a holistic approach to performance efficiency, moving past simple core count as the sole indicator of value.

NVIDIA's focus is on delivering not only high throughput but also superior single-thread performance, ensuring that the platform excels across diverse computational demands. This approach acknowledges that raw core count alone does not capture the full performance profile required for modern AI workloads.

Early adopters of the Vera Rubin platform, including Perplexity and ClickHouse, are reporting significant real-world deployment impacts. Perplexity, a company focused on quick tasks, observed a 1.5x improvement in sandbox startup time, demonstrating enhanced efficiency for running tasks.

ClickHouse, a real-time analytical database, achieved top benchmark results using the Vera platform, underscoring its capabilities in high-performance data processing. Frontier labs are utilizing ClickHouse for critical functions such as security, governance, and real-time operations, leveraging its speed and reliability.

NVLink is a critical component for scaling inference performance, and NVIDIA is now opening access to this technology, including switch trays and software, to the broader market. This initiative is designed to foster a more integrated and expansive hardware ecosystem.

Through a program called NVLink Fusion, third-party chip designers can integrate their XPUs into the NVIDIA ecosystem. This process involves designing their accelerator and partnering with NVIDIA to interface using enabling chiplet IP and advancements in HBM.

Early partners like The Matrix and Amazon's Annapurna Labs are already leveraging this open architecture to enhance their custom silicon solutions.

NVIDIA is collaborating with undisclosed partners on NVHBM, incorporating custom silicon technology directly into the base die of the NVHBM. This integration moves HBM logic and memory management to the base die itself.

This innovative approach frees up significant compute resources on the XPU, enhancing overall processing capability and efficiency.

This software dynamically manages the power of every single rack, ensuring it never exceeds a predetermined power cap.

MaxLPS allows for dynamic adjustment of rack power up and down based on workload changes, enabling higher GPU density within existing power budgets and optimizing overall token revenue at scale.

Lambda, a key early partner, has validated the benefits of MaxLPS through extensive benchmarking and testing, confirming its impact on compute capacity. Dave Ward from Lambda expressed significant excitement, noting the ability to increase GPU count and token offerings to customers.

MaxLPS demonstrated a 25% increase in compute on the B200 and is projected to deliver upwards of 40% more compute on the Vera Rubin platform within the same power consumption. NVIDIA plans a broad rollout of MaxLPS to its NCP and CSP partners, extending its dynamic power management capabilities across the industry.