Akamai Functions Achieve Zero Cold Starts for AI

Cold starts in containers and VMs can stretch from seconds to minutes, and over-provisioning to combat them increases cloud costs and environmental impact, according to Thorsten Hans.

The performance of AI agents and Managed Control Planes (MCPs) can vary greatly depending on their deployment location, causing inconsistent user experiences, and container and virtual machine setups frequently encounter cold start delays that range from seconds to minutes, impeding real-time application responsiveness.

WebAssembly binaries are tiny, portable, and execute quickly—functioning for servers much as containers do for virtual machines.

Thorsten Hans indicated that Wasm functions for servers in a similar way that containers relate to virtual machines, emphasizing its small size, ease of distribution, and rapid execution capabilities.

Spin, a Cloud Native Computing Foundation project, consists of a CLI, a Wasmtime-based runtime, and language-specific SDKs that help developers handle common tasks like HTTP requests and database interactions.

This framework supports any language that compiles to WebAssembly and provides a command-line interface (CLI) for development, a Wasmtime-based runtime, and language-specific SDKs to manage common tasks such as HTTP requests and database interactions.

This platform manages all infrastructure, eliminating the need for manual configuration tools like Terraform, and enforces security through WebAssembly sandboxing while including a built-in, multi-tenant key-value store for persistence.

WMCP acts as a command-line interface (CLI) that integrates with Spin, streamlining the creation of Managed Control Plane (MCP) servers.

This integration allows developers to concentrate on core business logic, such as implementing tools, resources, and prompts, rather than the complexities of protocol implementation, as non-functional requirements like OAuth2 and OIDC are automatically handled by WMCP components.

Thorsten Hans demonstrated the use of 'wmcp', a CLI tool available on GitHub, to streamline Managed Control Plane (MCP) development.

Developers can select from Rust, TypeScript, or Python templates, define input schemas, and list tools with precise descriptions for Language Model (LLM) compatibility, centralizing tool execution logic in a 'call tool' function that routes payloads to specific handlers.

The 'make' command compiles JavaScript or other language code into a WebAssembly component, which is a step towards creating a full-fledged Managed Control Plane (MCP) server, Thorsten Hans explained.

The 'wasmcp compose' command then aggregates these tools into a complete MCP server, bypassing unnecessary authentication for demonstration purposes, and Spin is used to run the server locally on localhost:3000, while the MCP Inspector serves as a browser-based client to verify tool connectivity and execution.

A decision-log Managed Control Plane (MCP) server written in Rust demonstrates advanced functionality using state storage, according to Thorsten Hans.

During local development, Spin automatically uses SQLite for key-value storage, but on Akamai Functions, the same code seamlessly leverages a globally distributed key-value store without needing additional configuration.

The deployment process involves updating an application manifest and using the 'spin akamai deploy' command to push the workload to the distributed infrastructure.

During the deployment process, OCI layers are generated and uploaded to the service registry.

Akamai Functions distributes these files across its global regions, verifying workload capability and performing internal health checks, ensuring the application is ready for use worldwide.

Once all regions return an HTTP 200 status, a generic subdomain is automatically created for the server, making the MCP server accessible on the public internet, and the infrastructure can be further secured using Akamai's Property Manager to enforce specific security requirements.

Developers can utilize TypeScript, Python, or Rust templates to create tools, prompts, or resources, with the system managing request routing seamlessly across Akamai's edge network.

The agent is configured with specific tools, including coin flipping and dice rolling, along with defined instructions to guide its behavior.

The runtime loads an optimized 10MB JavaScript engine, referred to as 'stalling monkey,' to ensure high performance and responsiveness during execution.

The process of compiling WebAssembly (Wasm) applications involves loading the interpreter and user code, followed by evaluating the global scope.

A memory snapshot is then saved to WebAssembly, pre-loading all dependencies, which makes Wasm applications faster than native JavaScript apps during cold starts and runtime, despite the trade-off of shipping a 10MB JavaScript runtime alongside user code.

Network headers from a test request indicate Santa Clara as the entry point and LAX as the service region, providing insights into the distributed nature of the Akamai network.

The 'X-envoy-upstream-service-time' header confirms the total round-trip time, which includes the entire business logic execution and the large language model (LLM) inference, with a typical round-trip for a simple task through the LLM taking approximately 820 to 866 milliseconds.

Thorsten Hans emphasized that the developer workflow for Spin applications remains consistent across various use cases, including line-of-business applications, Managed Control Plane (MCP) servers, or AI agents, supporting languages like Rust, TypeScript, Python, and Go.

Developers can test applications locally using the Spin CLI and deploy them with a simple 'spin deploy' command, benefiting from portability across Wasm-compliant runtimes and reduced operational overhead due to minimal infrastructure management.