From Laptop to Pipeline: Scaling AI Agents

His current role involves working with customers on how to adopt and scale the technology.

Autonomous agents run independently within a pipeline, while local agents handle specific manual tasks.

Understanding how to build, scale, and utilize each type effectively is central to Copilot's agent architecture.

Users can interact with coding agents through the IDE, GitHub CLI, or SDK.

GitHub CLI sessions can be streamed or delegated to GitHub.com, and mobile devices can remotely control active agents.

The SDK enables agent creation within third-party tools like Slack or Microsoft Teams, extending reach beyond GitHub's native interfaces.

Regardless of which interface a user chooses, all requests hit the Copilot API service, also called Cappy or the harness.

The harness handles the difficult engineering work: chunking input, security, authentication, and logging.

It routes requests to various model providers and filters final output before returning it to the user.

The Explorer agent uses smaller models, like Haiku, for efficient code base investigation when exploring a repository.

The Rubber Duck agent provides planning validation by using different model providers—such as GPT—to audit plans written by others, like Opus, demonstrating how embedded agents can layer different models for specific purposes.

The agent command guides users through setting up a workflow within their CLI environment, and agents can be scoped to a specific Git project or configured at the user or organization level.

Copilot handles the scaffolding process automatically, writing descriptions, formatting front matter, and structuring methodology without manual setup.

The builder is available in both the CLI and IDE interfaces, and users can edit the generated files afterward to refine sources or task logic.

A marketplace is a repository containing a specific JSON file that adds package management capabilities, making it the easiest way to distribute agents inside Copilot.

Subscribing to a marketplace allows teams to sync, version, and push updates to agents simultaneously, serving as a central hub for contributing skills and scaling Copilot adoption across an organization.

This mechanism ensures consistent tool availability, such as a shared compliance bot, across an entire team without requiring separate deployment steps.

The cheapest way to use AI products is through cache hits—exact matches between turns on common data.

Publishing agents to marketplaces or internal assets allows for organization-wide reuse, and standardizing agent workflows prevents reinventing processes repeatedly, both of which increase cache rates and reduce token costs.

The .github private directory is how agents are shared internally to everyone inside an organization, and marketplaces can be configured for internal use, allowing admins to distribute agents across organizations or even externally.

Agents restricted to their home repository struggle with cross-repo changes, which is why Orchestrate serves as a horizontal scaling tool for cross-repo tasks.

It allows assigning GitHub issues as goals and spinning up sub-agents in specific target repositories, making it useful for simultaneous changes such as modifying a microservice and its downstream dependencies.

Each sub-agent handles its portion of work while the parent agent coordinates the overall flow.

The Copilot CLI can run in headless mode using the -p (prompt) command, allowing it to accept a single prompt one-shot and complete work without user interaction.

Running this inside a GitHub Actions runner or other CI system turns that interactive agent into an autonomous pipeline agent using the repository's identity and budget.

Workflow triggers can be release events or scheduled maintenance via cron jobs, integrating human-in-the-loop approval points where needed to maintain cost and quality control.

A typical workflow has the product manager agent scope features and generate plans, then hand them off to GitHub Copilot for code implementation, with developers interacting at the acceptance phase to evaluate code.

Pipeline agents run autonomously in the platform, so they continue operating even if a developer's local device disconnects or becomes unavailable, enabling truly scalable, hands-off automation.

The framework contains approximately 200 agents running different parts of the pipeline, including meta-agents that check daily reports and orchestrate tasks by generating markdown outputs for other agents to consume.

Available agents include tools for code deduplication and various reviewer types—QA reviewers, humorous reviewers, and others—allowing teams to avoid building from scratch.

When agents run only on individual developer machines, there is no visibility into whether tool calls are successful, what they cost, or how long they take.

Moving agents to CI allows instrumentation with OpenTelemetry for full monitoring, enabling organizations to track metrics like average token usage, P90 cost thresholds, and tool call success rates.

Centralized logs enable A/B testing of models and prompts to identify cost-effective alternatives, and performance data helps diagnose whether agents are too verbose or failing to complete tasks correctly.

The presentation mapped a complete journey: building an agent locally, sharing agents through the platform via marketplaces or private repositories, and eventually moving those agents into the CI pipeline for autonomous execution.

Palafox works in a field capacity at GitHub and invites interested parties to connect via LinkedIn or through their GitHub Account Executive.