Agentic AI Production Needs Harnesses, Not Just Models

Harnesses are as crucial as models for production systems, managing intricate loops from reasoning to execution. These systems encompass tool routing, memory, observability, and security features that differentiate a mere lab prototype from a robust production system.

Open weights models are rapidly narrowing the performance gap with frontier models, currently trailing by only three to six points. This progress is enabling companies to construct completely new sovereign stack agentic AI systems that can run on local hardware, such as laptops, at an estimated six times lower cost than frontier alternatives.

This trend mirrors Jevons' paradox, where efficiency gains lead to increased resource usage; usage volume has surged by approximately 50 times or more, according to recent studies.

While metrics like tokens per dollar and tokens per watt are understood for models, there is no common framework for assessing an agent's success in completing tasks at the lowest cost. Crucial working groups for memory, context, and hardware remain unaddressed in current evaluation frameworks.

Consumers operate within rules set by others, while contributors can influence these rules, representing a significant difference in shaping the technology.

Influencing technology is far easier while it is still taking shape; once a technology becomes a default, altering its direction becomes much harder. Established technologies like Kubernetes, Prometheus, and OOS already serve as defaults, but areas such as identity and trust in agents remain open, requiring active participation for early contributions to have a higher impact on their direction.

Proof-of-concept setups for AI agents are typically simple, involving one agent, one tool, one trust domain, and often a single token. Production environments, however, are far more complex, necessitating multiple agents, tools, organizations, and trust domains. Fundamental questions regarding agent authority and trust in these multi-faceted environments largely remain unanswered.

Japan extensively uses agent technologies but has relatively few participants actively shaping them. The core issue is not a lack of technical skill but rather insufficient participation in governance, specification validation, and maintenance within the community. This limited engagement hinders Japan's potential influence in the evolving agentic technology landscape.

Providing an agent with connectivity access is not equivalent to establishing trust. Trust requires comprehensive visibility into what the AI can see and do, along with a robust tracking and auditing mechanism for all its actions. Seven specific requirements have been proposed for trustworthy agents, primarily focusing on defining clear boundaries and ensuring execution safety.

The control gap highlights difficulties in ensuring traceability and trust in AI actions, while the scale gap arises from the repeated need to build systems from scratch instead of leveraging shared capabilities.

Business processes often span disparate systems like Slack, Salesforce, and SAP, while AI frameworks currently lack a unified control layer, resulting in fragmented audit trails. This situation mirrors the early 'API sprawl' era, where effective governance was virtually impossible due to the lack of integration.

This layer connects the chosen AI model to the actual business systems where work is performed, applying consistent policies, permissions, orchestration, and auditability. It translates AI intent into governable actions without requiring modifications to the underlying models, thereby streamlining integration and ensuring compliance.

Currently, MCP servers and agent skills operate on separate distribution and access channels, creating a disjointed experience. Merging these skills into MCP would allow for 'shipping the manual' alongside the tools, offering a unified and more efficient AI agent experience.

This includes developing a two-part extension for listing and reading skills. The initiative also utilizes the established agent skills specification front matter for metadata, streamlining the integration process.

The finalization process focused on implementing minimal features to prevent fragmentation and ensure broader adoption.

Collaborative specification development highlighted the importance of avoiding premature feature support to maintain future flexibility, as adding features later is easier than removing them. Prioritizing a minimal implementation proved crucial in preventing fragmentation across different systems. The success of the initiative was significantly driven by broad participation from multiple companies, fostering a robust and adaptable specification.