Google DeepMind Replaces Python with Files for AI Agents

Google DeepMind is introducing a significant change in how AI agents are orchestrated, moving away from Python code towards a file-driven system. This new paradigm defines agents as tools that operate in a continuous loop until a specific goal is achieved. The primary objective is to simplify the agent creation process by minimizing the volume of Python code developers traditionally had to write and maintain.

This transition marks a departure from monolithic code structures, favoring modular configurations managed through files. Philipp Schmid emphasized, "We're going to look into how files are basically replacing Python."

This API includes a built-in sandbox environment, enhancing security and isolating agent operations. Philipp Schmid stated, "The Interactions API is our new Gemini API. It's a unified interface for running models and agents."

Key features of the Interactions API include support for server-side state management and background execution, which allows agents to perform tasks asynchronously without constant user interaction. Furthermore, it provides a standardized interface for tool calls, multimodality, and generation tasks, streamlining how agents interact with various data types and external functions.

The approach to large language model (LLM) applications is evolving beyond traditional turn-based conversation histories, which typically involve simple user input followed by a model output. For sophisticated AI agents, complex reasoning, numerous function calls, and extensive environment data are required, necessitating a more robust interaction model. Philipp Schmid noted, "We moved away from this turn-based conversation history to steps."

The new 'steps' model is designed to capture intricate sequences of reasoning, function calls, and their results. This method avoids 'abusing' the user role by allowing the system to track internal agent actions and thought processes, providing a clearer and more comprehensive timeline of agent operations without needing to present every intermediate step as a user-facing turn.

Historically, developing AI agents largely involved writing complex loops in Python, a process that proved to be quite intricate and time-consuming. Philipp Schmid explained, "Roughly a year and a half ago, writing agents mostly meant writing a loop in Python." This legacy approach required developers to manually define JSON schemas for tool descriptions and write corresponding Python functions to execute those tools.

Moreover, developers were burdened with manually parsing the outputs from LLMs and implementing extensive error checking mechanisms. This process often involved constant feedback loops, where adjustments to the Python code were made based on the responses and behaviors of the LLM, making agent development a highly iterative and often frustrating task.

Philipp Schmid mentioned, "I built, or I let Gemini build, a basic implementation of this Python loop." This agent required explicit system instructions to guide its behavior, along with manual definition of each tool it could use, complete with detailed JSON schemas.

This implementation highlighted the constraints of such an agent: it is limited to a predefined set of tools and struggles if the request falls outside its explicitly programmed capabilities. For instance, if the agent is asked to perform a task beyond its specific pull request review functions, it would fail to execute, underscoring the lack of flexibility inherent in this manual loop methodology.

Building custom agent logic in traditional setups involves numerous complex and manual steps. Philipp Schmid highlighted, "There's a lot of things we need to do to get an agent running." These tasks include the manual generation of tokens and the management of function calls, which are crucial for agent operation. Developers also face the requirement of extensive loop management and tool routing to ensure the agent correctly processes information and uses the appropriate functions.

Furthermore, the necessity of creating JSON schemas for every tool and executing corresponding Python code adds significant overhead. The burden of state management, ensuring the agent maintains context and consistency across interactions, further compounds the complexity of traditional agent development.

Agent frameworks have emerged as a solution to abstract away much of the complexity associated with building custom agent logic. Philipp Schmid noted, "There were many different agent frameworks which abstracted away some of that complexity." These frameworks significantly reduce boilerplate code, allowing developers to focus more on the unique aspects of their agents rather than repetitive setup tasks.

They introduce agent classes that automatically handle common functionalities such as loop management and error handling, making the development process more robust and less prone to manual errors. An example of such a framework is ADK, which demonstrates how these tools streamline the creation and deployment of AI agents by providing pre-built solutions for recurring challenges.

Philipp Schmid explained, "Those agent frameworks now use the signature of our functions to create those JSON schemas on the fly to provide the model." This innovation means that JSON schemas are automatically generated directly from the function signatures, eliminating the cumbersome manual process of defining them.

This approach also streamlines system prompts, making them more concise and easier to manage. By automating the generation of schemas, developers can define tools and their functionalities more efficiently, allowing the model to dynamically understand and utilize the available functions without extensive manual configuration. This leads to a more agile and less error-prone agent development cycle.

Philipp Schmid outlined, "The framework solves the turn taking loops, the routing, the execution mapping, the JSON schema creation for like the different function calls, but we still own the Python plumbing." This means developers are still in charge of writing the core Python code for tools and setting up their specific functionalities.

Furthermore, developers must contend with the need for specific environment and hosting setups, which are not always fully abstracted by current frameworks. These implicit limitations highlight that while frameworks automate many aspects of agent development, the underlying infrastructure and tool implementation still require significant manual effort.

Philipp Schmid elaborated, "This environment parameter here allows the agent to get access to a hosted isolated cloud sandbox where it can run tools, where it can run bash commands and where it can save files." This development marks the launch of the anti-gravity remote agent on the Gemini API.

These hosted, isolated cloud sandboxes provide a secure and flexible environment for agents to operate. Within these sandboxes, agents gain robust execution capabilities, including the ability to utilize various tools, run bash commands, and save files persistently. This setup not only simplifies environment management but also ensures that agent operations are contained and secure.

Philipp Schmid explained, "We created a network proxy around the agent sandbox which basically injects the credentials when the agent makes a request from inside the sandbox to outside the sandbox." This network proxy acts as a critical layer, preventing direct exposure of sensitive credentials to the agent itself.

The injection of credentials is a dynamic process, occurring only at the moment an agent makes a request from inside the sandbox to an external service. This just-in-time credentialing minimizes the risk of credentials being compromised or misused. Additionally, the system provides the ability to limit domain access for network requests, allowing administrators to precisely control which external services an agent can interact with, further enhancing security and preventing unauthorized data access or malicious activities.