A Pharma Data Engineer Built a Patched Doom to Test AI Agents

Chernov introduced himself as an associate principal data engineer at a pharmaceutical company, working in R&D on tools intended to help drug developers and researchers invent new drugs. He drew the line himself: 'So, but this talk is not about that.'

He describes himself as 'not a game developer or engineer or programmer yet', and says he finds the topic of agents in games fascinating to explore.

Chernov positioned the build against digital twins. In that established approach, he said, you construct a real-life system into agents, replicate the system model, and use it to run simulations and models to understand phenomena better.

His version is deliberately smaller: 'So we're trying to not rebuild entire Doom in agents.' What he wants instead is to understand the semantics of the game and replicate that in agents — a 'semantic mirror' rather than a full system replica. The stated goal is to capture the meanings of what happens in the world, not to reimplement the world.

The mirror has three tiers in his description: a structural tier with one observer state machine per visible actor, a semantic tier where perception brokers attach labels, and a behavior tier made up of agent swarm consumers.

The engine and the mod that give the demo its look come from other people. Chernov said he is not affiliated with GZDoom or its development, and the same goes for Project Brutality, the mod that shapes the game's look and feel.

He also flagged the graphical content before playing the demo live: it is gory, and he compared it to what audiences see in films such as Day of the Dead or Dawn of the Dead.

The problem Chernov set out is that agent behavior is not deterministic logic. It emerges from interaction with the environment rather than from if-then rules, the LLM behind it may decide one way or another, decision chains are long and their effects cascade, multiple agents talking to each other construct more complex behavior than any one of them holds, and outcomes depend on the environment. From that he posed the question driving the work: how do we design, test and govern agent behavior before deploying to a real system.

A game engine answers part of that with a strict control loop. GZDoom runs a fixed tick rate — the talk cites 35 ticks per second — with deterministic scheduling, giving the agents a metronome that does not drift.

The second half is real-time feedback. Physics, collision and line of sight are all simulated, and the engine limits how much any actor can see: rooms are bounded, so an actor only interacts with events taking place in the room it currently occupies. Chernov notes the approach is demonstrated with Doom but says it applies to any system and any problem.

The build is not something you can download and compile. 'So this is not standard Doom you can download and build yourself. It's actually patched Doom,' Chernov said, describing additional channels for communicating with the outside world where the agents plug in. GZDoom supplies a mature, extensible 3D environment.

Inside the engine, custom actor logic is written in ZScript, so agents are implemented partly in that native scripting language. The build uses sector and line def geometry with built-in physics and collision, event handlers for telemetry, and external control over standard in and FIFO.

As contributions, Chernov listed an agent state framework in ZScript, a policy guard system defining what agents can and cannot do, an external Rust control plane, MCP gateway integration, and an agent-cell framework adapter.

The architecture is held to three properties. Actions must be observable, meaning structured events are captured; attributable, meaning each one is linked to a specific agent or context; and reproducible, meaning it can be reconstructed through changes in world state.

Chernov said the game environment itself drives these architecture decisions. The consequence he points to is distribution: an action taken by one agent is transparent to the others rather than hidden inside a single reasoning chain.

The same three words return near the end of the talk, where he calls them the invariant guiding architecture decisions for any agent system — not just this one. Between the two mentions, the demo shows what enforcement looks like in practice: policy guards that run inside each state machine or agent instead of being applied from outside.

During the live demo a request to identify an object took roughly 3 to 5 seconds, most of that time attributed to network latency rather than model compute. Chernov attributed most of that to network latency — the round trip to OpenAI asking it to extract information from an image and return what it sees — rather than to model compute.

The output was specific. Looking at two characters whose backs were turned, the model identified 'zombie standing with chainsaw' and was correct. Chernov also pointed to a hidden house bonus object the agent spotted and reported an approximate position for.

The dots appearing on screen are not drawn by the game. They approximate where a given agent saw something — an armor bonus, a zombie — and they serve as the visible trace of what each observer registered.

It projects each actor to screen coordinates, producing a location for every object an observer can see. There is no LLM in this tier; Chernov describes it as deterministic. It assigns roles — idle, tracking, suspicion, alarm, loss — and a gateway picks what runs the observer reactor.

The second tier is the LLM broker swarm, which has four lenses: entity, HUD, motion and zone. Four different agents see through the broker and filter what they can observe in the game. Both tiers work from the same screen the player sees; the difference is which tier handles which work.

The reason for the split is speed, stated plainly. Anything that can be done without an LLM is pushed through the first tier, and anything that requires an LLM goes to the second. The vision tier's output is what Chernov calls the semantic mirror.

The screen Chernov showed is the control plane for the agentic system. It displays 1,000 agents attached to the consumer swarm, split between the observer path and the LLM path. He was careful about what that figure means: they are not all actively running all the time, and how many are live depends on the scenario and what kind of work is being done.

Three brokers sit alongside them. The pickup broker coordinates messages about picking items off the floor — medkits, team kits, clips, weapons. The entity broker looks for characters and works out what they are about, whether they attack or defend. The third, manifest, is a special scenario Chernov deferred to later in the talk, describing it as a game scenario implemented through agentic vision.

In the live narration he says four brokers and then corrects himself to three.

The mechanic is named after the Matrix films, where agents manifest in a room as soon as someone sees Neo there so he has to run. Chernov's version tracks which room the player is in and manifests enemies around them when conditions written into the game are met — behind or beside the player, close enough to attack.

One trigger is stalling. If the player stands too long in a certain location and the game wants them to move, a character is placed in front of or behind them and starts attacking.

A second trigger is a pickup about to happen. If the player is about to collect a particular weapon — a rocket launcher is the example — the system sees the pickup in progress and puts an enemy behind or around them to interrupt it. Armor pickups follow the same mechanism.

In both cases the placement is decided by the manifest scenario, which watches the player's route and position and chooses where the new actor appears.

The first layer is the game runtime itself. It contains an FSM agent base class plus five role subclasses, a policy guard, a zone manager, and telemetry that feeds the control plane.

The second layer is the Doom bridge, which exports whatever communication happens between the game engine and the agents over HTTP. It holds a policy engine and a replay store. The replay store lets the team reproduce what happened in the game and try other responses: as Chernov points out, an LM does not necessarily play the same way a second time, so they state what behavior is expected for certain characters and watch it play back. The scripting layer is game-native and compiled when the application runs, which makes it runtime-deployable and pushable while the game runs — Chernov calls it a kind of GitOps operation.

The MCP gateway implements three MCP services: Doom world for action execution, Doom vision for screenshot snapshots sent to GPT, and policy enforcement. Rate limiting and tenant policies live in the gateway. The policy guard supports three enforcement types — permissible, standard and strict — and can be switched from one to another, from permissible to more aggressive, while the system is running.

Four features are named for the gateway layer. The first is session routing with affinity, keeping related requests on a consistent path.

The second is token-based bucket rate limiting, which Chernov frames as defining how much of the LLM resources the system is consuming at any moment — the mechanism that keeps a thousand agents from becoming a thousand concurrent model calls.

The remaining two are tenant policy enforcement and semantic policy aimed at repeated queries and duplication — detecting when the same question is being asked again rather than paying to answer it twice.

Chernov walked through the correspondences one at a time. The 35 Hz tick loop corresponds to agents scheduling at fixed intervals in a production environment. State transitions correspond to workflow state machines, which exist in real applications. Policy guard corresponds to a policy engine such as OPA. Zone boundaries correspond to authorization domains.

Telemetry events correspond to structured logging. The replay store corresponds to an audit trace: whatever happens in the game can be recorded and replayed, and the same idea applies when replicating a process in a real design or application.

Behavioral regression corresponds to agent policy validation in CI/CD. Because the agent scripting can be deployed in a GitOps-style operation, changes can be pushed and run essentially in real time.

His conclusion from the list is that there are many analogies with real-world applications where agents can be employed, not necessarily game-related ones. The demo is the vehicle; the mappings are the argument.

Game engines, in his framing, supply the engineering discipline agent systems need: control loops, state management and event systems.

Policy is enforced from the inside. Each state machine or agent carries its own policy and runs autonomously, so enforcement does not have to be applied from outside the agent.

Simulation is the foundation. The agent system can be plugged into this game, another game, or a real-time process, and the semantic links between them can be reproduced and replayed. Chernov expects that to matter in other industries later, including the one he comes from.

On cost, he treats routing as an architectural decision rather than a budget line: semantic cache, local model and escalation belong to design. Semantic caching is the part he calls a big deal, because the cache is not over text — it is over images, trying to capture the semantics inside them, which is what the vision lenses produce. The named implementations for the whole build are gz doom, a Rust MCP implementation, and React DOOM.

Alongside the ticks, Chernov described an interruption mechanism he calls ERQ. Each agent normally follows a workflow — he calls it ABC — that it sticks to.

The interruption stops the agent and tells it that it no longer has to follow that workflow, because something happened in the game that should affect its reasoning. He frames the interruption as an analogy to real life rather than something specific to the demo.

There is no sound interaction in the current build, Chernov said. The capture pipeline is vision-only.

He outlined how it could extend. The same way vision capture works, sound could be captured: each demon makes its own specific noise, so noise alone could reveal which demons are in a room and feed that into the agents' reactions.

Player noise counts too — someone starting to shoot is a signal, and Chernov suggests it could be used to affect what the actors do in the game.