Build-Time vs. Run-Time Tools in AI Production

Averi Kitsch, a Staff Software Engineer, serves as the technical lead for the MCP toolbox focusing on databases at Google.

Prerna Kakkar, a Senior Software Engineer, is the technical lead for EvalBench, Google's evaluation framework for agent technology, MCP, and skill needs. Both engineers actively contribute to the MCP toolbox project.

The MCP toolbox is an open-source framework with over 15.7K GitHub stars and more than 132 active contributors, providing connection pooling, integrated observability, and support for over 40 databases.

Google-managed MCP offers a fully managed version for IDEs and CLI tools like Gemini CLI, incorporating 'model armor' for secure access and identity control, which collectively processed 20 million tool calls last month.

NL2SQL tools offer flexibility by generating SQL from natural language for data exploration but carry inherent risks in production due to their dynamic nature.

Structured SQL tools, conversely, employ predefined, secure logic for production use cases, effectively preventing SQL injection and reducing latency and hallucinations compared to free-form NL2SQL.

Build-time tools, such as NL2SQL tools and control plane tools, are designed for flexibility and atomic operations but pose significant risks in production due to their dynamic nature, necessitating human oversight.

In a demonstration, an agent, without sufficient guardrails, catastrophically deleted an entire database table when prompted to “delete the table and start fresh,” illustrating the dangers of deploying build-time tools in a production environment without proper safeguards.

This agent is engineered with identity verification mechanisms and maintains authenticated OAuth to ensure that only authorized users can execute specific tasks, preventing unauthorized actions.

Simon Willison's concept of a 'lethal trifecta' describes the critical risk of simultaneous access to private data, untrusted content, and the ability to exfiltrate that data externally.

An internal triage agent, designed to process tickets, was exploited by a malicious insider who redirected it to query a sensitive salary database.

The agent, leveraging its trusted system privileges, extracted salary data and posted it back to the original ticket, inadvertently causing a public data breach.

The agent identity is strictly limited to accessing only the data that the end-user is explicitly authorized to view, thereby minimizing the blast radius in case of compromise.

Agent parameters represent untrusted inputs that the agent dynamically derives, posing a potential security risk if not carefully managed.

Conversely, application parameters are factual constraints that must remain outside the agent's control to maintain security, with the primary goal being to remove connection details from agent management.

The Toolbox addresses security concerns by introducing a source primitive, moving connection details out of the agent's direct control and into preconfigured YAML files.

This approach eliminates the need for agents to hold raw SQL queries or database credentials, making read-only restrictions the most frequently requested customer feature.

To minimize the blast radius and secure databases, access should be limited to read-only queries, and cloud-native allowed data sets should be utilized.

Furthermore, implementing output size limits prevents system overload and mitigates potential data exfiltration risks.

Toolbox introduces custom tools, enabling the definition of exact SQL statements within YAML files for consistent and reliable execution.

This allows for customized tool names and descriptions, enhancing context, and utilizes prepared statements with type parameters to effectively mitigate SQL injection vulnerabilities.

It is highly recommended that tools prioritize outcomes rather than focusing on atomic REST APIs, which helps reduce unnecessary round trips and improves efficiency.

Separating read and write tools allows for human confirmation of critical write actions, enhancing security, while providing actionable errors enables agents to intelligently retry failed tasks.

Additionally, using flat input structures, instead of complex primitives, makes tools more agent-friendly and robust.

Implementing bounded parameters at the application level ensures user identity is securely bound, preventing agents from directly handling sensitive information.

Authenticated parameters are used to validate OpenID or JWT tokens at the tool level, confirming the authenticity and authorization of the user.

Crucially, user claims are extracted directly from these tokens, keeping identity secure and isolated from the agent itself.

The speakers recommend consulting the official documentation and GitHub repositories for more information, particularly highlighting the EvalBench repository for evaluating tool performance and ensuring their effectiveness.