Vercel's Journey to Solving Agent Building

His work primarily focuses on the creation of new libraries, frameworks, and cutting-edge technologies. Vercel's core mission involves building essential infrastructure for the web, now extending its reach into the rapidly evolving field of agentic applications.

Vercel developed the AI SDK to significantly simplify the process of integrating various AI providers. This innovation allows developers to replace hundreds of lines of provider-specific code with just a single line, offering a unified model interface across different AI services. The SDK also incorporates critical features such as model fallbacks, secure code execution, and enhanced durability, ensuring a more robust and efficient development environment.

Andrew Qu and Vercel's CTO were inspired by Bill Gates' vision of a computer on every desk, prompting them to consider the possibility of an "agent on every desk." This conceptual shift expanded their focus from traditional coding workloads to broader business verticals, including design and product management. The initial experiments for this ambitious project commenced approximately a year ago, coinciding with the release of Sonnet 3.5.

The most compelling initial use case for AI agents emerged from Vercel's data team, which, despite being lean, faced an exploding volume of data as the company grew rapidly. Marketing and sales teams frequently required manual SQL queries, creating a significant dependency on the data scientists. This manual workload and constant request stream severely hampered overall developer productivity and created a bottleneck in operations.

Andrew Qu conducted an initial experiment using a 'mega-prompt' by pasting the entire Snowflake database schema into a system prompt, followed by a specific question. The model then generated SQL, which Andrew manually copied and executed to test its validity. This proof of concept demonstrated that while base models could generate SQL, they required significantly better context and robust guardrails to function effectively and reliably in a real-world scenario.

The first step involves processing the natural language question to understand its intent and scope. Subsequently, the data scientist must explore the semantic layer to identify relevant data tables and understand their join patterns. Finally, the process culminates in executing the SQL query and analyzing the results to provide an answer.

Vercel transitioned to a multi-agent workload strategy, dubbed D0, by mapping data science tasks into specific agent roles, including query, planning, and execution agents. Version 2 of D0 utilized specialized system prompts and precisely scoped tools for each agent within a chained sequence. This approach achieved end-to-end automation, streamlining the entire process from a user's initial question to the final report generation.

Vercel discovered significant limitations in their chained agent architecture, particularly concerning information loss during hand-offs between agents. This led them to the conclusion that a single 'mega-agent' capable of managing its own memory and internal state across planning, building, and execution phases was necessary. This mega-agent approach allowed for self-reflection and the ability to retry tasks autonomously if errors were encountered during execution. The centralized context and memory management within a single agent proved to be more effective for complex, multi-step tasks.

Vercel's internal testing indicated a 30% success rate on evaluations, leading to high confidence in their advanced agent system. However, when the system was deployed to a small group of trusted users, the feedback was overwhelmingly negative, described as "awful." This disparity revealed a significant "reality gap," as the complexity and variety of real-world user queries proved far greater than anticipated. Manual mapping of scenarios, while effective for internal tests, proved non-scalable and insufficient to handle the emergent complexity of actual user interactions.

A significant breakthrough occurred when Vercel observed that Claude Code and Opus 4.5 agents outperformed their internally developed 'handgrown' agents by treating the environment as a simple file system. This approach minimized the need for a prescriptive set of tools, instead providing only basic functionalities like read, write, and run bash commands. The realization was that allowing agents to 'go wild' with standard, foundational tools facilitated the emergence of more effective and flexible behaviors. This shift from tightly controlled, highly specific toolsets to a more open, file-system-based interaction model unlocked superior performance and adaptability. By giving agents the freedom to explore and interact with the environment through basic file operations, Vercel witnessed a dramatic improvement in their problem-solving capabilities. This paradigm demonstrated that less prescriptive guidance and more environmental agency could lead to more robust and versatile AI agents, capable of handling unforeseen complexities through emergent behavior rather than pre-programmed solutions.

Vercel designed the Eve agent runtime with a focus on durability and the ability to run agents within isolated environments. The architecture ensures open-source compatibility, providing adapters for various technologies such as Postgres, Docker, and OpenAI. Eve also includes built-in support for Vercel Workflows, Sandbox, and Vercel Connect, streamlining integration into existing Vercel ecosystems.

This structure consists of clear system instructions, a defined set of skills, and specific tools, making it straightforward for developers to compose and build real agents. This modular approach also greatly facilitates iteration, allowing for quick and efficient adjustments to agent behavior and capabilities.

Aura successfully implemented a custom 'mini claw' agent designed to test various services by visiting websites, installing applications, and attempting to use them. This custom solution demonstrated better success rates compared to generic off-the-shelf cloud solutions. The Aura agent provided deeper insights into service performance with fewer steps in its operational process, highlighting the advantages of tailored agent development.

When agents built with Eve are deployed to Vercel, developers gain immediate access to out-of-the-box observability features. This includes comprehensive tracking of all agent runs and detailed visibility into every tool call made by the agents. The platform also offers step-by-step execution insights and cost estimation, with potential suggestions for optimization based on usage patterns, streamlining monitoring and management.

Vercel has found that off-the-shelf agents are often limited by their lack of deep domain context, making them less effective for specialized tasks. The true success of an agent, they argue, relies heavily on integrating very specific company knowledge. This insight has led Vercel to develop specialized agents for various internal tasks, including complex data queries and rigorous legal contract reviews, which generic solutions cannot handle effectively. Ultimately, Vercel concludes that to maximize efficiency and achieve the best results, companies should build their own agents tailored to their unique operational needs and proprietary information.

The implementation of AI agents has significantly enhanced the productivity of Vercel's data team. By automating routine tasks, staff now have more time to focus on strategic initiatives, such as improving Snowflake performance and adding previously missing data sources. This shift from manual query handling to higher-level work demonstrates the broad potential for such automation across various departments, including HR, Finance, and Sales, to optimize human talent.