Building Deterministic Infrastructure for Non-Deterministic AI

Every production architecture must be driven by the incidents it is designed to survive, according to Nishant Gupta. He asserts that anticipating and planning for failures is not a reactive measure but a foundational principle for building resilient systems.

Gupta emphasizes that these identified failure modes serve as the critical starting point for developing reliable agentic systems. This approach ensures that the infrastructure is inherently robust against the unpredictable behaviors of AI agents.

Instead of waiting for AI models to achieve perfect determinism, engineers should design the surrounding infrastructure to account for their probabilistic nature. Nishant Gupta highlighted that this proactive design strategy acknowledges the current limitations of LLMs.

Gupta notes that models will be occasionally wrong and sometimes overconfident. He suggests that designing infrastructure with this understanding prevents system failures caused by unexpected model behavior.

Nishant Gupta explained that this transformation fundamentally changes the role of AI in production systems.

He elaborated that in the past, models merely returned text, and humans made subsequent decisions. Now, agentic systems empower models to select tools and execute actions directly. This enhanced capability means that failures can impact core infrastructure and alter customer states, necessitating more robust control mechanisms.

The primary engineering objective is not to correct the inherent non-determinism of AI models, but rather to contain them within a deterministic execution envelope. Nishant Gupta underscored that this approach allows for the reliable operation of probabilistic models in production environments.

He specified that engineers must be able to audit all aspects of agent behavior within this envelope, including beliefs, actions, schemas, and policies. This auditing capability ensures transparency and control over the agent's operations, even with its underlying probabilistic nature.

Every agent action must be reconstructible as a typed request, allowing for clear understanding and debugging. Nishant Gupta states that this requirement ensures full traceability of the agent's operational history.

This reconstructibility necessitates meticulous tracking of all policy decisions, tool results, and state transitions. By capturing these elements, engineers can replay and analyze any action, providing critical insights into system behavior and potential failure points.

The integration of traditional software with Large Language Models introduces novel compositional failure modes, increasing system complexity. Nishant Gupta points out that common issues like wrong actions, retry storms, cost spirals, and silent successes can now compound due to the inherent unpredictability of AI agents.

Ambiguous planning and self-directed tool selection by agents significantly amplify these risks, making incident management more challenging. He warns that if retries are unbounded and permissions are broad, even an ambiguous failure can escalate into a cascading incident, severely impacting the organization.

Nishant Gupta clarified that engineers must consider a broader range of factors when designing robust systems.

Prompt context, external tool calls, and concurrent execution introduce variability into agent behavior. These elements can cause an agent to produce different outputs or take different actions even when presented with seemingly identical inputs.

Additionally, state compaction when context windows are nearing capacity, and human feedback, further contribute to non-deterministic outcomes. Recognizing these diverse sources is crucial for developing infrastructure that can reliably manage AI agents.

No AI agent should be deployed unless it successfully passes standard software test protocols. Nishant Gupta mandates that these tests form a baseline for any agent's readiness for production environments.

He adds that agents also require testing against adversarial tool use scenarios and must undergo thorough rollback escalation drills. These additional testing phases ensure that agents can handle unexpected or malicious interactions and that recovery procedures are well-practiced before deployment.

Traditional observability metrics such as latency, errors, and logs are insufficient for understanding complex AI agent behavior. Nishant Gupta explained that agent observability requires these standard metrics augmented with semantic traces, which provide deeper insights into an agent's internal reasoning.

Semantic traces are crucial for revealing how an agent interpreted a task, formed its plan, selected tools, and perceived the outcome. This detailed understanding allows incident dashboards to precisely pinpoint whether failures originate from issues in context interpretation, reasoning, tool behavior, or human interaction delays, preventing misattribution of errors between infrastructure and model.

High cost or excessive token usage should be treated as a safety signal, indicating potential issues with an agent's operation. Nishant Gupta highlighted that an agent continuously looping, fanning out, or expanding its context may be confused, stuck, or operating outside its intended workflow.

He advocates for practical controls such as token budgets, step limits, and wall-clock time constraints to prevent runaway behavior. When these budgets are exhausted, agents should be designed to summarize their current state and escalate to human operators rather than attempting further autonomous reasoning, thus mitigating potential incidents and controlling operational expenses.

A reliable agent infrastructure ensures that the model never directly holds broad production credentials, enhancing security. Nishant Gupta described an architecture where the planner proposes bounded steps, which a validator then checks against established schemas and policies.

He added that tool brokers are responsible for minting scoped credentials, restricting the model's access to only what is necessary for a given task. Furthermore, every step is recorded in an append-only trace, ensuring replayability and auditability, with orchestration state, retry logic, and verification functions owned by deterministic services, guaranteeing reliability independent of the model's probabilistic nature.

Non-deterministic systems often directly execute model output as commands, creating inherent risks. Nishant Gupta argues that a deterministic system, however, treats each model response as a proposal, subject to validation and approval before execution.

He detailed that components like validators, policy engines, approval gates, idempotent executors, and verifiers each play a role in reducing the blast radius of potential failures. This layered execution fabric significantly enhances safety outcomes, even when the underlying AI model remains non-deterministic.

Nishant Gupta stressed that progressing an agent's autonomy must follow a structured path, starting from suggestions and moving through drafts, dry runs, and approval gates before reaching atomic execution.

He outlined that promotion between these stages should rely on objective criteria, such as successful rollbacks and complete audit trails. Gupta advises against directly transitioning an agent from basic chatbot functionality to an autonomous production operator, highlighting the need for careful, incremental advancement to mitigate risks effectively.

The production viability of AI agents relies heavily on robust systems engineering, not solely on advancements in models themselves. Nishant Gupta asserted that designing deterministic interfaces around non-deterministic reasoning is the critical skill set for making agents useful in real-world applications.

He clarified that agents should be granted creative freedom in planning but never in controlling production systems directly. This principle ensures that while AI can innovate in strategic thinking, its actions are always constrained and validated by a reliable, deterministic infrastructure, safeguarding operations.