Ramp Automates Engineering with AI Agents

Engineers evaluated Fable by assigning it non-product tasks, such as CI testing suites, to measure its performance on unfamiliar large codebases.

Engineers specifically sought to evaluate Fable's capabilities on large codebases that they had not personally reviewed, aiming for objective validation.

During this testing, Fable successfully addressed issues such as import cycles and implemented lazy loading for Python modules, with the generated code subsequently merged into the actual codebase.

Claude Fable significantly reduced Ramp's Continuous Integration (CI) P50 time from 18 minutes to just 6 minutes.

The AI agent iteratively profiled the codebase and scheduled self-runs to collect production data, automatically repeating this process over several days until multiple performance improvements were achieved.

This allows the model to utilize its maximum thinking capacity, with users able to adjust a 'thinking dial' to control the computational effort allocated to a problem.

Loops are best suited for repetitive tasks with predefined steps, such as managing pull requests or auto-rebasing code.

In contrast, dynamic workflows are designed for complex, non-linear problems like system optimization, where the exact steps are not known in advance, requiring a shift in how engineers delegate command execution.

Tasks such as deleting dead code can be handled by daily automated scripts, moving away from manual intervention for repetitive maintenance.

Austin Ray describes 'Claude tag' as an agent that manages the entire lifecycle of an experiment, from creation to monitoring.

This agent generates the experiment, lands the pull request, monitors performance, and balances exposure, with the engineer only approving the initial pull request and the agent handling the rest of the execution.

Developers are moving towards minimalist coding environments, favoring terminal-only setups like iTerm2 over complex Integrated Development Environments (IDEs).

These simple, 'vanilla' configurations help engineers better understand the capabilities and behaviors of AI models, utilizing Claude Code for adversarial reviews and background information gathering.

Ramp has implemented AI agents throughout its engineering lifecycle, covering processes from ideation and bug detection to production monitoring.

Security is a primary consideration in the implementation of these agent systems, guiding their integration across various stages.

The company's strategy involves building for anticipated future model capabilities, treating agents as coworkers by providing them with specific goals and necessary system access.

Ramp focuses on planning for future model capabilities rather than current limitations, recognizing that immediate solutions often become technical debt that hinders future progress.

Aiming for goals further in the future allows for more efficient use of current resources and prevents slowdowns caused by accumulating technical debt.

Ramp prioritizes studying individual interaction traces of AI agents over aggregate benchmarks, analyzing failed traces to identify issues like context problems or missing tool access.

Implementing multi-layered defense mechanisms enhances agent agency and access, while the principle of least privilege is applied by providing only necessary data access, such as read-only service keys for platforms like BigQuery or DataDog.

Ramp aims for an 'iPhone-like' user experience for its AI agents, featuring a simple text box for inputs.

The focus is on declarative prompting, where users state what they want done, not how, shaping agent behavior through optimized prompts, tools, and skill sets, and leveraging increasing model capabilities over time.

The security team serves as both the architect of the underlying infrastructure and network policies, and an active user of the systems, noting that their close involvement is the most exciting aspect of the project.

Significantly, these security teams are also active, regular users of the AI agents they help govern, fostering a close relationship with the technology.

Ramp cultivates a culture of experimentation where failed projects are accepted as a natural part of the development process.

Engineers are given free access to all necessary tools without restrictive token or budget limits, promoting a unified technical language across the team and encouraging innovation.

Serving as a home base, Project Glass allows non-technical teams to interact with coding agents and gain velocity without needing a technical background.

The tool is designed to meet non-technical users at their current skill level, eliminating the need for direct code interaction.

Operating as an AI Site Reliability Engineer (SRE), it automatically generates pull requests for fixes.

The system delivers a detailed root cause analysis report directly into Slack incident channels and has been in production at Ramp since late February or March.

Individual teams retain the autonomy to build their own automation solutions, while the platform team provides foundational abstractions like 'Inspect' to standardize tools once multiple groups demonstrate a clear need.

The platform team provides bedrock abstractions, such as 'Inspect,' to facilitate these builds, driving innovation through 'desire paths' where foundational tools are standardized once multiple teams identify similar needs.

Ramp avoids imposing token or dollar limits, ensuring engineers have unrestricted access to any level of AI intelligence.

Cost management is handled through optimized defaults, batch/flex APIs, and the use of cheaper models for non-human-controlled tasks.

The team monitors high spending to identify opportunities for 'platformizing' bespoke solutions rather than restricting usage, prioritizing return on investment (ROI) where value generation outweighs token costs.

CTOs should prioritize the rate of change and the trajectory of model intelligence over current feature sets when adopting AI agents.

Ramp emphasizes building for capabilities expected three to six months in the future, as focusing on currently available solutions can lead to being behind by the time a product is shipped, requiring an understanding of scaling patterns to anticipate future model agency.