GEPA Outperforms RL in AI Agent Optimization

Lakshya A. Agrawal developed reflective optimization to address the limitations of conventional AI training, which requires massive datasets like trillions of tokens or hundreds of thousands of rollouts.

Traditional methods like gradient descent typically demand massive datasets, often requiring trillions of tokens or hundreds of thousands of rollouts, which are resource-intensive.

Current AI development faces significant constraints due to the limited availability of domain-specific data.

Many agentic workflows involve slow, expensive, or long-running tool calls, making the hundreds of thousands of rollouts required by traditional online reinforcement learning impractical for real-world product applications.

Reinforcement learning with verified rewards, such as GRPO, simplifies complex agent traces into a single O(1) score, which overlooks crucial diagnostic information.

This approach effectively ignores valuable context like chains of thought, tool calls, and error messages embedded within interaction traces, preventing comprehensive learning.

GEPA utilizes a language model to analyze entire rollouts, identifying both successes and failures by examining the complete textual feedback rather than just a zero or one reward signal.

This reflective process incorporates intermediate outputs and facilitates external tool calls, such as retrieving data from company knowledge bases, allowing for significant behavioral shifts through single natural language prompt updates instead of numerous gradient adjustments.

GEPA incorporates an evolutionary loop combined with a Pareto-based candidate selection strategy.

In a demonstration, GEPA achieved twice the performance gains of GRPO after only one round of reflection using just three data points, whereas GRPO required 25,000 rollouts to reach comparable results. The Qwen 38B model independently optimized itself without external expert guidance.

By generating precise and detailed instructions for complex workflows, GEPA improves upon previous prompt optimizers that relied on vague emotional prompts.

This automation replaces manual prompt engineering, a process that often takes human teams weeks to refine, by discovering problem specifications in 30 minutes to an hour for tasks like multi-hop question answering.

GEPA successfully optimized GPT-4.1 mini to surpass GPT-4.1's performance on math tasks and significantly improved an existing agent's performance on AMD's NPU XDNA2 from 4.25% to 30.52%, a seven-fold increase.

The system also successfully identified that a legacy library, ADF.h, was incompatible with new hardware, demonstrating its capability in addressing sparse documentation challenges that even GPT-4 struggled with.

The GEPA algorithm refines prompts by collecting domain-specific feedback and using a Large Language Model (LLM) for reflection, iterating through improvements.

Unlike standard loop methods that keep only the single best candidate and often get stuck in local optima, GEPA maintains a Pareto pool, preserving every candidate that performs well on at least one training example, which contributes to a more balanced search process.

This strategy accounts for over half of GEPA’s success across four benchmarks, allowing it to achieve twice the performance gains of loop-based methods by converging to much higher scores.

GEPA consistently achieves approximately 10% performance increases, even on models already optimized for existing benchmarks, showcasing its robust capabilities.

This optimization framework is highly versatile, applicable across diverse fields such as question answering, instruction following, claim verification, and math, and can be used to improve any text-scorable artifact, including Python or JavaScript agent code.

This API functions by taking an evaluator's feedback, alongside actionable side information like compiler errors or profiler logs, to allow an LLM to generate better candidates while maintaining a Pareto pool to avoid local optima, and is applicable to diverse applications such as CUDA kernel code, numeric values, and cloud scheduling policies.

The API supports an open-ended dictionary for this side information, which can include documentation, compiler errors, or tool call messages, automating the search for an optimized solution based on the specified objective.

Using the Optimize Anything framework, GEPA transformed a 4-line Python program into a 6-step agent for RKGI problems in 16 rounds of reflection.

This agent significantly boosted Gemini Flash's RKGI accuracy from 32.5% to 89.5% and autonomously performs rule induction, code synthesis, and debugging.

Additionally, a 2-step agent created with GEPA improved GPT-4o mini's performance on the MATH-500 benchmark by 20%.

GEPA enhances agent skills for specific code repositories by optimizing based on trajectories, leading to significant improvements.

GPT-4o mini's performance on Go repository issue resolution increased from 24% to 93%, an almost three-fold jump, and these optimized skills were then successfully transferred to Claude 3.5 Sonnet, achieving 100% issue resolution.

GEPA's GSkill reduced execution and issue resolution times by nearly 50%, saving token usage.

GSkill achieves this by embedding crucial domain information such as repository organization, test case invocation methods, feature implementation locations, and build system details, and is available open source within the GEPA repository.

The interface facilitates three distinct modes—single-problem, multitask search, and generalization—enabling users to handle tasks from specific kernel tuning to complex, novel deployment queries.

These include a single-problem mode for direct optimization, a multitask search mode for related problems that enables information transfer between them, and a generalization mode for training on existing data to prepare for novel deployment queries.

Optimize Anything has been used across a broad set of domains, demonstrating its versatility and impact.

It reduced cloud scheduling costs by 40% compared to expert heuristics and cut OCR error rates for leading models by approximately 35%.

Databricks achieved a 90x cost reduction in their deployed agents' performance by tuning GPT-4 120B to outperform Claude Opus at a significantly lower cost, indicating that prompt optimization becomes even more effective with advanced models.

GEPA can learn evaluation metrics for subjective tasks directly from production traces, utilizing approximately 50 human-annotated trajectories with detailed feedback.

This process establishes a data flywheel where an LLM-as-a-judge prompt is optimized using human annotations, which then iteratively refines agents, allowing for continuous improvement and co-optimization of model weights and prompt harnesses through a fast-slow learning approach.

GEPA is currently deployed in production environments by companies such as Dropbox and Shopify, with OpenAI also highlighting its methodology for building self-improving AI systems.

The system has zero hard dependencies, ensuring simple integration into any existing framework or model and encouraging users to frame technical problems as optimization tasks by leveraging domain-specific side information.