Design AI as Knowledge Agents, Not Just Coders

He emphasizes that current agents often miss the full scope of what knowledge work entails.

Knowledge work involves processing inherently ambiguous information to yield actionable decisions or judgments, making coding merely one specific application within a much larger domain.

Early AI agents in 2022 primarily handled simple retrieval-augmented generation (RAG) tasks, offering basic trivia and lacking true tool-calling capabilities or complex conditional logic.

Subsequently, the industry's focus largely shifted to developing coding agents, which quickly became a dominant application, illustrating a specific, albeit limited, form of knowledge work.

This type of work involves a complex process of distilling ambiguous and diffuse information into precise, actionable judgments or decisions, a core function for professionals.

The service economy heavily relies on knowledge work, encompassing diverse roles such as lawyers, actuaries, and researchers, all of whom transform information into expert output.

Code constitutes a unique subset of knowledge work, benefiting from highly durable and structured references like IDs, file paths, and keywords.

This inherent structure provides clear, easily findable cues within codebases, making them more amenable to agent processing compared to the broader, often ambiguous, fields of law or academic research.

Consequently, the strong capabilities of current agents in programming tasks do not directly translate to deep research, due to the fundamental structural differences in knowledge domains.

In contrast, knowledge work inherently lacks clear function definitions or strict syntactic rules, making context and interpretation crucial.

For instance, the term '30 days' can mean a deadline, a grace period, or a retention rule depending on the specific context, highlighting the ambiguity present in knowledge work.

Meaning in knowledge work is often implicit and heavily relies on deep domain knowledge, which is difficult for current AI models to fully grasp without explicit instruction.

In knowledge work, the search process originates from a user's intent rather than predefined parameters, necessitating a more dynamic and adaptive approach.

Agents must therefore evaluate international norms and conditional information that is not explicitly contained within the initial task description.

This meaning-driven work is significantly more intricate than straightforward code execution, demanding a sophisticated understanding of context and nuance.

The 'tool loop' has evolved from oral traditions to written language, exemplified by Alexandria's library catalogs and culminating in modern search engines, each advancing how knowledge is accessed.

Concurrently, the 'organization loop' progressed from individual polymaths to structured institutions like monasteries and universities, eventually leading to the specialized bureaucracies of today.

These two loops function as a single self-optimizing system, where advancements in knowledge drive the creation of better tools, which in turn necessitate new organizational structures and roles for managing that knowledge.

Improved tools significantly reduce costs, thereby increasing the accessibility of knowledge and broadening the scope of what can be achieved.

For instance, finding a manuscript at the Library of Alexandria could take weeks without a catalog but only minutes with one, demonstrating how tools dramatically alter efficiency.

Enhanced accessibility directly influences how frequently and easily humans seek out and integrate new knowledge, fostering greater innovation and discovery.

Traditional BM25 search methods consistently fail to retrieve accurate information, even when granted unlimited search attempts, highlighting a fundamental bottleneck in their effectiveness.

Over-reliance on OCR (Optical Character Recognition) text introduces significant performance bottlenecks, as errors in text conversion propagate through the search process.

Consequently, modern search tools must evolve beyond standard text-based retrieval to overcome these inherent performance ceilings.

Tools developed by Jina and Mixedbread leverage full multimodality, integrating visual and semantic contexts to enhance search capabilities.

The integration of visual context significantly improves retrieval accuracy, making searches more precise and comprehensive.

Mixedbread introduces a decomposed agent architecture that breaks down complex queries into smaller, manageable sub-tasks, reflecting effective human problem-solving strategies.

In this system, sub-agents independently search for information and then summarize their findings for a main agent, which then synthesizes these summaries to answer the original complex question.

This task decomposition approach has demonstrated a 3.5% gain in accuracy, showing the benefits of structured, collaborative agent design.

The 'oracle gap' quantifies the difference between ideal document retrieval and the actual performance of a given search system, serving as a key metric for evaluation.

Through architectural improvements and agent utilization, this performance gap was successfully reduced from 10 points to 6 points.

These enhancements led to a notable 40% reduction in mistakes, significantly boosting the reliability and precision of the agent system.

Industries like legal and medical fields provide established templates for organizing knowledge and workflows, which serve as valuable blueprints for agent architecture.

Therefore, agent design should emulate these effective human knowledge management strategies rather than being confined to programming paradigms.

Tools are fundamentally designed to overcome performance ceilings and enable tasks that would otherwise be impossible or impractical, rather than merely existing for their own sake.

Models must be equipped to recognize specific tools as distinct primitives, each tailored for different needs and functions within a complex workflow.

Effective agents need explicit training to discern when to apply semantic search versus keyword retrieval, ensuring the most appropriate tool is selected for each query.

Effective orchestration agents are crucial for managing large-scale knowledge retrieval, ensuring that tasks are broken down and information is organized systematically.

Ultimately, effective agents must be capable of organizing knowledge, as no single model can encompass the entirety of global legal codes or other vast knowledge domains.