Reducto CEO on Building Robust AI Agents

Adit Abraham, CEO of Reducto, emphasizes the platform's role in processing unstructured documents like PDFs, images, and spreadsheets to help AI-native firms like Harvey, Lora, and Rogo transition from demos to production-grade agents. Reducto processes billions of documents for clients ranging from AI-native companies like Harvey, Lora, and Rogo to large enterprises in finance and insurance.

The company focuses on converting unstructured data, including PDFs, images, and spreadsheets, into a usable format for AI teams. Their experience has provided insights into moving beyond basic demonstrations to deploying production-grade agents in real-world scenarios.

While early Retrieval-Augmented Generation products focused on basic synthesis, the industry is transitioning toward complex workflows where agents perform autonomous tasks in sectors like finance and healthcare. This progression means agents are moving beyond merely answering questions to generating and modifying files.

Sectors like finance, insurance, and healthcare are increasingly adopting end-to-end autonomous decision-making processes. This expanded scope necessitates different tools and a more robust approach to data processing than traditional retrieval systems.

The change underscores the need for agents to handle complex, multi-step tasks that interact dynamically with data.

When agents pull from multiple sources to make sequential decisions, errors compound across the entire workflow, a risk that Adit Abraham underscores for multi-step agent pipelines. When agents interact with multiple data sources and make sequential choices, inaccuracies in initial data can compromise the entire workflow.

As AI agents transition from straightforward Q&A to intricate, multi-stage processes, reliable document intelligence becomes increasingly vital. The pervasive nature of bad inputs can significantly degrade the quality of outputs.

Enterprises currently rely on unstructured, multimodal data scattered across platforms like Google Drive and Box, which lack the clean organization necessary for AI operations. It lacks a cleanly organized repository, presenting significant challenges for AI systems.

This fragmentation creates downstream difficulties in retrieval accuracy and effective context interaction. Without proper organization, agents struggle to efficiently access and interpret relevant information.

PDFs, designed in the early 1990s for consistent printing, remain a challenging problem for AI, with even frontier models achieving only about 30% accuracy on the 'GDP PDF' benchmark. Human-created documents often contain visual context, handwriting, and non-standard tabular structures that are difficult for AI to parse accurately.

The primary goal is to convert these complex documents into a machine-readable format, such as markdown, to enable effective agent reasoning. This conversion is crucial for moving beyond simple text extraction to deep semantic understanding.

The inherent complexity of PDFs, with their varied layouts and embedded elements, requires sophisticated processing techniques to bridge the gap between human readability and machine interpretability.

Traditional Natural Language Processing (NLP) pipelines relied on Optical Character Recognition (OCR) and rigid templates for documents with consistent layouts, such as W2 forms. However, this approach struggled with the variability of real-world documents.

Vision Language Models (VLMs) introduce a horizontal approach, enabling AI to read documents much like humans do. This allows for better handling of the 'long tail' of data, including complex handwriting and irregular structures, although they are not a one-size-fits-all solution for high-scale document processing.

Efficiency and determinism are crucial for processing hundreds of millions of documents, where traditional computer vision (CV) techniques still play a significant role. Object detection, for example, remains highly effective for layout analysis.

Small models with sub-million parameters can run on CPUs and efficiently handle specific document regions, avoiding the need for expensive foundation models. VLMs are then leveraged for semantic identification after the initial structural processing by these more efficient models.

Adit Abraham introduces 'agentic OCR' as an automated alternative to traditional manual annotation teams for correcting errors. This method utilizes Vision Language Models (VLMs) to perform specific, token-level edits, moving beyond full document regeneration.

Speculative decoding, akin to fast edit tools in Integrated Development Environments (IDEs), enables precise corrections for character-level mistakes, such as distinguishing periods from commas or zeros from 'O's. This prevents models from hallucinating or 'correcting' data by performing their own calculations instead of preserving the original document's intent.

The approach aims to achieve higher accuracy and reduce the labor involved in human-in-the-loop processes, making document processing more efficient and reliable. By focusing on granular edits, agentic OCR avoids common pitfalls of standard prompting, which can lead to unintended alterations.

Preserving structure in complex tables, especially those with merged cells, is crucial for accurate AI reasoning. While HTML provides a robust representation for reasoning, its verbosity can lead to excessive token consumption and increased processing costs.

Markdown, conversely, is efficient for simple tables, making a dynamic strategy essential: markdown for simple tables and HTML for complex ones. This approach optimizes both cost and accuracy.

Furthermore, embedding models often struggle to effectively retrieve information from messy, HTML-heavy data when queried with natural language. Therefore, balancing detailed structure with efficient representation is key for effective document processing pipelines.

A single data representation often fails to meet the distinct needs of both reasoning models and embedding models. Reasoning models require structured data like HTML, while embedding models need natural language summaries for accurate retrieval.

To achieve optimal performance in RAG systems, developers should provide a dual representation: a natural language summary for efficient retrieval and an HTML structure for detailed reasoning. This approach maximizes the effectiveness of both components, taking minimal effort to implement.

Adit Abraham highlights the 'GDF PDF benchmark' as evidence that superior input quality significantly boosts Large Language Model (LLM) performance. Providing structured PDF representations alongside raw files notably improves accuracy across models from Gemini, Anthropic, and OpenAI.

Better input data allows models like GPT-4.5 and Opus to outperform others out-of-the-box and reduces reasoning token usage and latency. This improvement occurs because models can concentrate on generating outputs rather than expending resources on complex parsing.

Human intervention remains necessary to route documents to appropriate pipelines and decompose complex tasks effectively. Simple 'needle in a haystack' context dumping often degrades quality beyond just token economics.

Effective orchestration involves classifying documents and isolating relevant snippets before processing, reducing the cognitive load on models. This pre-processing is crucial for handling large documents, such as multi-page paper mail packets, preventing models from being distracted by irrelevant content.

Agent harnesses empower models to solve previously 'unsolved' problems requiring high precision, such as extracting granular data from complex line charts. Traditional vision encoders often fail to capture individual data points accurately from such visual elements.

Reducto utilizes agents equipped with code interpreters and visualization tools to iteratively reconstruct markdown tables from charts. This process involves the agent repeatedly identifying and correcting its own mistakes, a capability impossible for a single-shot task.

The iterative refinement allows the agent to reach an accurate final output, demonstrating how specialized tools within an agentic framework can overcome limitations of conventional vision models for intricate data extraction.

By defining validation criteria for sub-agents, parent agents mitigate silent errors like dropped rows within complex CBP documents. This approach helps prevent issues like dropped content or rows, which are common in complex forms such as CBP documents.

A Micro-one benchmark highlights a market bifurcation between high-precision frontier models and high-recall dedicated processing services. Agent harnesses help bridge this trade-off between precision and recall, allowing for a local maximum in extraction accuracy.

Evaluation must underpin all decisions and extend beyond macro-level metrics, focusing on granular performance at each step of the pipeline. Key components requiring specific evaluation include parsing, retrieval, and final formatting.

Production data often differs significantly from contrived test sets, necessitating real-time monitoring to ensure accuracy. Ultimately, the objective of comprehensive evaluation is to continually improve the end-to-end performance of the AI agent.

Developers are transitioning from rigid, deterministic pipelines to dynamic, agent-directed workflows as AI capabilities advance. Agents can now navigate file systems and intelligently select specific tools based on the type of document encountered.

Pipelines are evolving to split documents into content for reading and metadata for tasks like citations, enhancing overall efficiency. The industry is also increasingly focusing on document generation as the next significant phase of AI development.

To achieve optimal accuracy, cost, and latency, it is recommended to decompose parsing problems and use the right tool for each specific task. Implementing agentic verification can significantly improve the reliability of pipelines by catching errors proactively.

Effective data orchestration, including proper classification and splitting of documents, is crucial for efficiency. Furthermore, evaluation should be conducted at every stage of the pipeline, moving beyond traditional, outdated methods to adapt strategies for current challenges.