While the technical hurdles to deploying AI agents have diminished significantly, their reliability remains compromised by a lack of access to real-world, human-generated context.
Previously, constructing an agent would demand the collective effort of an entire team over a full quarter, underscoring the significant resources and time commitment involved in the past.
my name is Jeff. I'm a founding engineer at Unblock and I'm here to talk to you about how building agents has actually gotten pretty easy, but unfortunately they still get things confidently wrong.
Ng detailed the significant infrastructural challenges that historically complicated agent development, likening each component to a separate company function. Among these, checkpointing and state persistence were paramount to ensure that agents could retain historical data and progress through operational loops without losing valuable information.
The ephemeral nature of infrastructure often led to expensive session restarts, wasting computational resources and time. Additionally, robust sandboxing mechanisms were essential to mitigate security risks, preventing agents from unauthorized network access or other potentially harmful actions.
The AI ecosystem has seen substantial maturation, with cloud infrastructure providers such as Cloudflare, Vercel, and AWS abstracting away much of the underlying complexity that once burdened developers.
These platforms now offer advanced primitives and services that significantly simplify the development process, complemented by frameworks like Flu and Mastra, which allow engineers to concentrate primarily on the core logic of their agents rather than infrastructure management.
Today, defining an AI agent has been streamlined to a few essential parameters, making it accessible to a broader range of developers.
This process involves selecting the desired model, formulating the system prompt or instructions, determining the specific tools or skills the agent will have access to, and specifying the sandbox location for its operation.
This agent is engineered to fetch Linear tickets, classify them as either a feature or a bug, and then perform comprehensive code repository searches.
Based on its analysis, the agent formulates a plan for the next steps to address the issue, effectively automating parts of the ticket triage and initial investigation process.
Despite its sophisticated analysis, the agent encountered a critical flaw when it recommended re-enabling async dispatch based on its code analysis. This recommendation, while technically derived from the available code, was fundamentally incorrect in the operational context.
Crucially, this exact setting had caused an outage just days prior, prompting support engineers to explicitly disable it. The agent, lacking this crucial real-world context, provided a solution that would have reintroduced a known problem.
Ng elaborated that the agent's failure stemmed from its incomplete understanding of the situation, specifically its lack of access to critical human-generated context. While the agent had full access to the code repository and relevant tickets, it was unaware of the discussions that occurred in Slack after the outage.
These discussions, where engineers collaborated to diagnose and resolve the issue, along with subsequent postmortem documents, contained vital "tribal knowledge" that was never fed into the agent's reasoning process. This highlights that access to unstructured human communication and incident reports is the next crucial frontier for improving the reliability and operational intelligence of AI agents.
자막에서 근거가 되는 대목을 찾아 답합니다.
AI Engineer의 다음 글도 받아볼까요?
AI Engineer에 새 영상이 올라오면, 방금 읽으신 것처럼 정리해서 메일로 보내 드릴게요.
이 채널은 지난 7일 동안 93편 올렸어요.
