Self-Improving AI Agents are Here

Marc Klingen of Langfuse notes that the period from 2024 to 2026 represents a new era where advanced AI models increasingly abstract away downstream complexity in agent operations.

Early applications in 2023 were often limited to basic functions, such as single-file edits or simple GitHub issue-to-pull-request automation, demonstrating initial capabilities for automated token generation.

Models have matured significantly, evolving from basic token generation to systems capable of autonomous reasoning and self-verification, with current engineering teams focused on transitioning from manual intervention to AI-driven optimization loops.

Traditional stacks separate observability tools like Datadog from MLOps platforms like MLflow, a disconnect that Langfuse addresses to prevent benchmarking on stale datasets.

Traditional development stacks typically separate observability tools like Datadog from MLOps tools such as MLflow or Weights & Biases, creating distinct operational silos.

Langfuse aims to bridge this gap, integrating these functions to prevent the risks associated with benchmarking agent performance against stale or inaccurate datasets, a process described as manually tedious and labor-intensive.

Modern agent autonomy relies on self-improving stacks that operate across layers, ranging from low-level token prediction to highly abstract human-involved reasoning processes.

Early iterations of these loops, around 2023-2024, were exemplified by tools like GitHub Copilot, representing a foundational stage.

Modern feedback loops, expected between 2025 and 2026, enable agents to not only identify errors but also propose fixes and verify their own work, though human involvement remains crucial for aligning datasets with actual user-facing questions.

Human intervention is essential for reviewing proposed changes to datasets and evaluation criteria within the agent's self-improvement loop.

Automated agents face the risk of overfitting to specific data or narrow failure definitions, potentially leading to suboptimal performance.

Furthermore, not all production anomalies necessitate application-level fixes; some patterns might be irrelevant to the agent's primary function and are best filtered out by human judgment.

Agents can be programmed to propose improvements to their own implementations, leveraging production traces and evaluation criteria.

Developers can engage in a process known as 'hill climbing,' iteratively testing various adjustments—such as modifying models or context aggregation strategies—against established datasets.

This iterative success depends on maintaining a clearly defined boundary for what aspects of the agent are being optimized.

Teams are increasingly deploying AI to ensure datasets remain synchronized with actual user activity in real-world scenarios.

For customer support agents, AI can dynamically update datasets to incorporate common user questions, improving relevance.

AI identifies error patterns within production data to refine offline testing criteria, streamlining the development cycle.

Unsupervised automation carries the risk of generating 'slop' rather than genuinely improving quality, especially in AI applications.

The initial targets for AI applications are often vague, requiring human intervention to define strategic direction while the development process is being assembled.

Therefore, humans should establish high-level goals and review critical changes, allowing AI to handle lower-level optimizations and iterative improvements.

Traditional manual tracing and benchmarking methods for agent development demand significant time investment but typically yield high-quality outcomes.

Conversely, fully automating agent loops without human oversight can lead to excessive token usage and a lack of meaningful improvements, as the AI may not distinguish valuable output.

The ultimate goal is to reach a 'promised land' where developers primarily intervene at critical high-level decision points, eliminating tedious and repetitive tasks.

AI-driven evaluation systems can process a substantially greater number of error cases than human evaluators, who are constrained by time and patience, thereby improving efficiency and coverage.

Implicit indicators, such as customer reactions within support channels, provide training data for AI agents without needing direct developer intervention.

Human team members accept or modify responses proposed by internal agent systems, which provides the AI with essential learning signals.

Continuous streams of such real-world performance metrics enable AI loops to adapt and improve based on live operational data.

Langfuse's engineering team sought to automate release communications as the velocity of new feature development increased, requiring a more efficient process.

A 'change writer' agent was implemented to draft documentation and change logs automatically following each successful pull request merge.

Integration of human feedback into the GitHub pull request review process establishes a primary evaluation signal for agent outputs.

Reviews and requests for changes are fed back into observability and evaluation tools, enabling the change writer agent to automatically improve its future performance.

Langfuse identified that their agent's documentation writer was factually accurate but consistently produced low-clarity output.

A key issue involved internal jargon frequently appearing in customer-facing documentation, hindering user understanding.

To eliminate jargon leakage, the agent underwent a reconfiguration where it began suggesting modifications to its own datasets and evaluators.

This involved replacing technical implementation details with language more aligned with the user's domain, significantly improving readability.

New agent implementations are rigorously back-tested against updated datasets to ensure performance improvements.

The system systematically compares a v1 baseline, representing the existing agent implementation, with a v2 candidate to quantify the extent of improvement.

While refining user-facing language and enhancing clarity, the system maintains strict adherence to format compliance and accuracy metrics.

Changes are merged automatically via pull requests, as the agent operates within the development pipeline and does not deploy directly to production environments.

Continuous collection of production signals and precise tracking of execution traces form the foundation of building robust AI agents.

Agents typically operate in a continuous loop, often scheduled as a cron job, to process newly gathered user data and feed it back into the improvement cycle.

Data sources for this continuous feedback include direct user feedback, programmatic labeling from production systems, and in-app user annotations.

Cost-effective, non-locked-in storage solutions allow companies to retain full contextual traces instead of sampling, which supports the heavy read volumes generated as agents iterate on historical data.

Historical traces are crucial for providing the necessary context for long-term agent improvement, making their retention vital.

Companies must retain full datasets rather than relying on sampling, which drives the demand for cost-effective, non-vendor-locked-in storage solutions to support this data-intensive approach.