PostHog's AI Wizard Safely Executes Bash Commands

The Wizard streamlines setup by reading the codebase, installing the correct SDK, instrumenting events, and configuring dashboards automatically. This innovative tool aims to reduce setup time from the usual one to two hours down to five to six minutes, significantly enhancing developer experience. Sarah Sanders described it as: "It's an agentic CLI tool that reads your codebase. It installs the right SDK for your project. It instruments your events and it sets up dashboards for you." The Wizard also offers free inference, contributing to a seamless and improved user experience.

The speaker, Sarah Sanders, initially expressed unease, describing the agent's capabilities as "malware shaped." This realization highlighted the critical need for robust security measures within the tool. This concern directly led to the development of internal security tools aimed at protecting production deployments from potential threats inherent in agentic systems.

Users frequently question why they should use an autonomous agent instead of simply writing standard prompts. PostHog's rationale for building the Wizard is to elevate the developer experience and leverage the agent's capability to participate in an agent loop. Sarah Sanders emphasized, "because this developer experience and the capability of the wizard is the whole point. It's the whole product because we built a CLI tool that can fully take part in an agent loop." This design provides users with a 'mini implementation engineer' experience directly within their terminal, offering advanced automation beyond what simple prompts can achieve.

It utilizes various models, specific prompts tailored for different tasks, and a predefined set of tools to execute its functions. A core element is the custom, in-house context engine, affectionately dubbed the 'Wizard's brain,' which Sarah Sanders humorously likened to "marked down in a trench coat." The tool also features a custom terminal UI built using Ink for an enhanced user interaction. Furthermore, a dedicated security scanner named 'Warlock' was introduced to ensure the safety and integrity of production environments.

The Warlock scanner was specifically developed to counter these potential dangers, serving as a critical defense against the inherent horrors of production-ready agents that can execute arbitrary commands.

The Wizard began as a V0 prototype, developed out of necessity because existing tools like Cursor frequently hallucinated PostHog setups. The team found that a dedicated agent performed significantly better, validating their approach. This success quickly translated into widespread adoption, with Sarah Sanders proudly announcing, "We were dreaming of thousands of developers running this a week and yesterday we just hit 8,000 people running this a week." By the time of the presentation, the tool had grown to serve 8,000 weekly users, demonstrating its effectiveness and user demand.

Subsequently, the team advanced to 'layer one,' implementing an allow list to restrict the agent's capabilities. This was further augmented by developing a 'hacky' scanner designed to monitor for threat-shaped patterns, signaling a shift towards more proactive security measures.

Within this sandbox, Bash commands were denied by default, significantly limiting potential malicious actions. The agent was only permitted to install trusted packages that had been thoroughly vetted by the PostHog team. Sarah Sanders noted, "We had bash as deny by default. It could only install trusted packages that were vetted by us. Um, it could build, it could type check, it could lint, and pretty much nothing else." Additionally, access to environment variables and .env files was blocked, and a secure vault was used for managing secrets, reinforcing the strict security posture.

An audit by the security team revealed that vulnerabilities in the agent system did not originate from obviously malicious code. Instead, the identified gaps stemmed from seemingly innocent, well-intentioned interactions between different system components. Sarah Sanders explained, "The interesting part wasn't the specific gaps or bugs they found themselves, but it was the shape of them. Because almost none of them were obviously evil. They were all two very innocent, well-intentioned things that were shaking hands and opening a hole." This highlights a critical insight: attackers often exploit how separate, benign features can be combined to create vulnerabilities, a challenge that traditional code reviews, which often examine diffs in isolation, may overlook.

The agent's context engine, dubbed the 'Context Mill,' is a machine designed to inject content into an agent capable of running commands. Sarah Sanders described it as: "It's a machine whose whole job is to take content and inject it into an agent that can run commands." This engine sources data from documentation, prompts, and example applications, bundling it for loading into the agent's context at runtime. This process introduces significant security risks, particularly concerning prompt injection.

There is a potential for supply chain attacks where poisoned content in open-source repositories could be inadvertently ingested by the agent. Furthermore, the risk exists of accidentally shipping malicious prompt injections through AI-approved pull requests (PRs), which might appear benign to automated checks but harbor hidden threats. These factors necessitate robust scanning at multiple stages to mitigate such vulnerabilities.

This approach involves scanning content at two critical junctures: when a skill is initially built and released, and again when the wizard subsequently utilizes that skill. Sarah Sanders summarized the strategy, stating, "My methodology is catch it at the source, assume the source failed and catch it again at the point of use." This layered defense assumes that initial security checks at the source might fail, necessitating a second verification at the point of execution to catch any missed threats.

This standalone module replaces a previous 'hacky' regex scanner that was integrated directly within the Wizard. Sarah Sanders explained the naming choice, stating, "I called it the warlock because everything wizard shape needs a bodyguard." The primary goal of The Warlock is to scale security measures effectively as the usage of the developer agent continues to grow. By centralizing security within a dedicated and robust module, PostHog aims to maintain a high level of protection without impeding developer productivity. This specialized approach ensures that the agent's expanding capabilities are met with commensurate security protocols, allowing for safe and widespread adoption.

The system is designed to be fully deterministic, ensuring that the same input consistently produces the same output every single time. Sarah Sanders emphasized, "It's fully deterministic. It's the same input, same output every single time. It's boring on purpose. And in security, boring is a feature." This deliberate choice for a 'boring' and predictable system highlights that, in security, reliability and consistency are highly valued features over novelty or complexity, providing a stable foundation for threat detection.

PostHog implemented a Large Language Model (LLM) layer to assist with false positives in its security system, carefully defining its role. The team debated whether the LLM should act as a 'bouncer' that strictly enforces security decisions or an 'adviser' that provides recommendations. Sarah Sanders noted, "Should the layer be a bouncer or should the layer be an adviser? And the easiest choice probably could have been make the LLM the bouncer." Ultimately, they chose to position the LLM as an adviser, thereby avoiding reliance on the LLM's potentially unpredictable 'coin flips' for core security decisions. This approach ensures that critical security enforcement remains deterministic and reliable, while the LLM helps refine the detection process.

The LLM's function is to assess information only after these deterministic checks have successfully passed, providing insights and context rather than making critical pass/fail security decisions itself. This strategy ensures that human-level security remains robust, while AI assists in refining the process.

Sarah Sanders explained, "And every warlock rule has four parts. Part one is the metadata." These rules are engineered to track the flow direction, distinguishing between input and output to apply context-aware security measures. For instance, effectively blocking prompt injection requires more than simple keyword filtering; it demands sophisticated context-aware matching to identify and mitigate malicious content. This structured approach ensures comprehensive and precise security enforcement.

Effective security rules require rigorous negative testing to prevent false positives, which can quickly lead to user frustration and the disabling of security tools. Sarah Sanders highlighted the importance of context, noting, "RM-rf is scary, but it's also how we all delete node modules like 40 times a day." This illustrates that the severity of a command should be based on its real-world impact and context rather than its perceived scariness. Security tools that frequently generate false alarms on routine tasks are invariably disabled by users, undermining their effectiveness. Therefore, continuous testing and a deep understanding of operational context are crucial for maintaining usable and reliable security.

The core principle is that no single layer is sufficient on its own to provide complete protection. Instead, the architecture combines several 'boring,' honest layers, each performing specific security jobs. This layered approach ensures redundancy and resilience, significantly enhancing the overall security posture against potential threats.