Modal's 50x Growth Driven by Strategic Shift to Compute Execution and Emergence of AI Sandboxes

Modal's CEO, Erik Bernhardsson, stated that the company has achieved at least 50x growth in revenue, users, and general platform usage since his last appearance on Latent Space two years prior.

Bernhardsson attributes this substantial growth to a significant shift in AI workloads, indicating a strong product-market fit for Modal's offerings in the current AI landscape.

Before founding Modal, Bernhardsson was instrumental in building Spotify's music recommendation system, an experience that highlighted the need for better infrastructure for AI applications, influencing Modal's core mission.

Initially, Modal considered focusing on workflow orchestration, similar to existing tools like Luigi or Airflow. However, Erik Bernhardsson realized that the more critical challenge was the efficient execution of compute.

This led to a strategic pivot, with Modal shifting its focus to build a platform that simplifies running code in the cloud, particularly for GPU-intensive and AI/machine learning tasks.

The company's early efforts focused on general-purpose cloud code execution, emphasizing scalability and an optimal developer experience, which required deep dives into the infrastructure stack, including file systems and container runtimes.

The emergence of Stable Diffusion as a 'killer app' for their platform, prior to ChatGPT, validated this focus, prompting Modal to further realize the potential of generative AI inference.

Modal made an early decision to adopt a multi-region architecture, treating its entire compute capacity as one unified, global pool rather than isolated regional clusters.

This approach was a critical and consequential decision, enabling them to effectively manage and scale large GPU requests, which is often a significant challenge in the AI compute market.

The company procures capacity from various hyperscalers, utilizing both spot and one-year reservations, and has also rented capacity from providers like Oracle for specific GPU types such as A10s and A100s, leveraging favorable rates.

This diverse sourcing and unified management strategy helps Modal navigate the complexities of securing hundreds, thousands, and even tens of thousands of GPUs for its diverse customer base.

Erik Bernhardsson disputes the notion of a permanent GPU scarcity, highlighting that Nvidia delivers hundreds of thousands of chips monthly, suggesting that supply is generally available if companies are willing to pay higher prices.

He views the current tightness in the GPU market as a temporary phase, comparing it to the 'toilet paper shortage' during the pandemic, and expects hardware prices to eventually trend downwards, following historical patterns.

While acknowledging that prices are currently increasing and capacity remains tight, Bernhardsson notes that this situation is not necessarily detrimental for Modal, as scarcity allows them to charge more from customers and their existing long-term reservations act as a natural hedge.

Modal's strategy of paying higher prices for capacity has consistently secured the necessary GPUs, demonstrating that the market, while not perfectly efficient, responds to pricing signals.

Modal deliberately avoids direct competition with major AI players like OpenAI and Anthropic for large, top-tier GPU clusters.

Instead, the company focuses on aggregating smaller capacity nodes from diverse global locations, such as 32 nodes in Armenia or 128 nodes in Indonesia, to secure better prices.

To ensure quality and reliability, Modal employs a rigorous internal 'spec sheet' for vendor acceptance testing, outlining specific requirements for IOPS, CPU performance (e.g., PassMark scores), and network bandwidth (ingress and egress).

Despite sometimes working with less established vendors, Modal conducts initial acceptance testing with single nodes and engages in significant financial commitments, relying on a degree of 'leap of faith' alongside their internal performance metrics.

Erik Bernhardsson explains that 'neo-clouds' like Core and ncale achieve rapid scaling by securing large, multi-year GPU purchase contracts from major AI buyers such as Anthropic or OpenAI.

These long-term contracts, which can involve commitments of billions of dollars, then serve as collateral to raise substantial financing from Wall Street.

This financial model allows neo-clouds to pre-sell capacity and acquire hardware even before it is physically installed, enabling extremely fast growth.

Modal, with its Product-Led Growth (PLG) focus, faces greater challenges in adopting this debt-backed scaling model due to its inability to lock in long-term contracts in the same manner, although Bernhardsson believes they will reach a similar scale in the long run.

Modal currently operates with two main product offerings: Modal Functions, launched in 2021, and Modal Sandboxes, introduced in 2023.

Modal Functions provide serverless, scalable GPU workloads, proving highly effective for inference, managing bursty demands, and efficiently aggregating capacity across thousands of customers.

Modal Sandboxes emerged from the observation that Large Language Models (LLMs) were generating code that required a runtime environment; initially, the product's necessity was debated internally, but it subsequently experienced explosive growth.

Sandboxes now account for approximately one-third of Modal's revenue, contributing to the company's evolution into a multi-product vendor, with future plans to expand into training and batch processing capabilities.

Erik Bernhardsson notes a shift in the fine-tuning landscape, stating that the necessity for fine-tuning has diminished significantly due to the rapid improvement of base open-source models.

He explains that users can now achieve 'frontier-level' quality and cost targets by directly deploying models like Kim K3, DeepSeek 4.1, or GLM 5.3, which are nearly as good as proprietary frontier models.

While fine-tuning remains an option, it is now considered 'very optional,' as off-the-shelf open-source models perform remarkably well, reducing the previous requirement to fine-tune smaller models to reach competitive performance.

Modal continues to see demand for fine-tuning and is developing products in this area, including SFT and reinforcement learning, but acknowledges the reduced imperative for it in many use cases.

Erik Bernhardsson identifies a prevailing 'lift and shift' approach in current sandbox usage, where users treat sandboxes as direct cloud equivalents of local development environments, which he believes is not ideal for long-term agent compute.

He proposes a more efficient model involving long-running agents that trigger short-lived, high-speed sandboxes, capable of spinning up in under 100 milliseconds.

This ephemeral model, enabled by snapshotting, drastically reduces startup times by eliminating the need for repeated installations and configurations, leading to more efficient compute.

Such a system facilitates advanced functionalities like branching execution and massive parallelization of tasks, akin to modern CI suites, where hundreds of agents can run concurrently, departing from traditional sequential workflows.

Modal has begun using AI agents internally to actively search for vulnerabilities and test the robustness of its sandbox environments.

The process involves deploying agents within a sandbox and instructing them to attempt 'escape' or exploit deep parts of the system to identify potential security weaknesses.

While these agents have not yet found vulnerabilities, Bernhardsson views this as a promising defensive strategy, believing that agents can identify and help fix security flaws faster than traditional human penetration testers.

This proactive approach to cybersecurity, which started a few months ago, aims to enhance system safety by leveraging AI breakthroughs to address vulnerabilities defensively.

Erik Bernhardsson notes that the widespread use of AI agents, particularly those trained on vast amounts of online code including Modal's own, makes deprecating features significantly more difficult.

Agents frequently prefer and utilize older syntax or legacy versions of features found in their training data, making it hard to 'nudge' them towards newer implementations.

As a result, Modal has adopted a more conservative approach to deprecating and removing features over the past year, requiring meticulous updates to every online example and documentation piece.

The company also conducts internal 'agent benchmarks' to monitor if AI agents recommend Modal for specific tasks, akin to search engine optimization, to understand and influence agent behavior.

Modal intentionally avoids aggressively hiring dedicated product managers (PMs), believing that effective product management in an infrastructure company requires a very deep level of strategic thinking.

Erik Bernhardsson explains that product functions at Modal are often handled by engineers or developer relations specialists who demonstrate strong aptitude for strategic planning and identifying foundational building blocks.

He cautions against the common tendency for infrastructure companies to hire PMs too eagerly, which can lead to the development of disjointed, 'widgety' features rather than core, strategic infrastructure.

The focus remains on building fundamental primitives and anticipating future needs for one to two years out, ensuring that infrastructure development remains robust and avoids feature proliferation without clear strategic alignment.

Erik Bernhardsson maintains his position as a 'cloud maximalist,' rejecting the necessity of an on-premise business model for Modal.

He views traditional on-premise infrastructure, along with concepts like Bring Your Own Cloud (BYOC) or VPC peering, as reflecting an outdated security model that relies on network perimeters rather than robust, modern paradigms like zero trust.

Bernhardsson argues that Modal's full control over its entire infrastructure stack, from renting hardware to deploying and managing all infrastructure, enables faster iteration and the development of a superior product.

While BYOC might be considered for specific capacity-pooling needs with large enterprise partners in the future, it is not seen as a primary product offering or a core part of their strategy.

Erik Bernhardsson states that prepayments for GPU hardware capacity represent the most critical use of Modal's cash flow, reflecting aggressive capacity ramping.

He describes other operating costs, such as salaries and office spaces, as 'rounding errors' when compared to the substantial expenditures on GPU capacity.

This financial structure highlights Modal's business model, which relies on extremely high operating leverage where infrastructure costs significantly outweigh headcount expenses, despite having around 200 employees.

The company's primary financial focus remains on navigating the complexities of capacity, pricing, and the balance between hardware ownership and renting, which Bernhardsson identifies as the biggest area of strategic thought.

Modal is establishing a research team dedicated to post-training, distillation, and quantization techniques to optimize models.

Initially, this team will work closely with customers to optimize their models, providing bespoke services.

However, the long-term goal is to convert these custom optimization efforts into repeatable, self-serve internal products and tooling, avoiding a service-heavy, non-scalable business model.

Erik Bernhardsson emphasizes that success means customers can optimize their models without direct intervention from Modal, enabling the company to scale more effectively in GPU performance and post-training spaces.

Modal Sandboxes have seen unexpected and significant growth, currently contributing one-third of the company's total revenue, a fact Erik Bernhardsson describes as 'insane' and a surprise even to himself.

Sandbox revenue continues to grow at approximately 30% month-over-month, consistently surpassing internal predictions of plateauing.

Initially, the market potential for sandboxes in 2023 was underestimated, but the company now invests heavily in evolving them from traditional 'dev-boxes' into ephemeral, snapshot-capable environments.

Modal's strategy focuses on massively parallelizing tasks within these sandboxes, aiming to drastically reduce startup and shutdown overhead to further enhance efficiency and accommodate sustained market demand.