Open AI Models Drive Enterprise Cost Savings

The shift to open-source models is particularly noticeable in enterprise environments. Many companies currently use a mix of U.S. and Chinese development models, primarily leveraging AI for coding agents and AI assistants. The adoption of open-source models like Open CLO and Hermes is steadily increasing.

Cases of large corporations adopting open-source models are becoming concrete. AT&T recently announced that it has already transitioned 40% of its total token usage to open-source models. Currently, it primarily uses U.S. and European models, but it is reportedly considering the introduction of Chinese models as well.

Token usage for open models is exploding. Ollama Cloud's total token usage has increased by a staggering 150 times compared to the beginning of this year. This surge is mainly due to the emergence of coding agent-driven models and the expansion of usage stemming from the introduction of Open CLO models.

Open models are becoming crucial in the field of security testing. Closed models like Claude often refuse penetration testing, which has constrained security research. Consequently, open models are gaining attention as a key use case for verifying software security, with their flexibility acting as a strong advantage.

AI infrastructure configuration is as complex as operating system development. It requires connecting every layer: hardware drivers, inference layer providers, application runtimes, and harness functionality. The core challenge in AI stack configuration is to organically integrate and connect all these elements, acting as the glue that binds them.

In the future, most of an enterprise's AI token usage is expected to be invested in open models. The open model community contributes to cost reduction, making AI more accessible to more people. Therefore, a hybrid approach, using closed models for cutting-edge research and open models for general tasks, is anticipated to become prevalent.

For AI utilization, a local and cloud hybrid strategy is important depending on the nature of the task. High-difficulty problems, such as coding agents that write code and perform tests, are most effective with large-scale cloud models. In contrast, relatively simple problem-solving processes, like document processing workflows, can run very efficiently in a local environment, offering advantages in terms of cost and efficiency.

The workstation hardware market is booming, enhancing AI infrastructure accessibility. High-performance devices like DGX Spark, unveiled at GTC, can be used on a desktop and provide 128GB of integrated memory capable of running models from 20B to 120B. This allows developers to efficiently advance AI development by leveraging local resources connected to the cloud without the hassle of purchasing expensive GPUs.

The AI market is seeing a trend towards optimizing model performance and cost efficiency. The emergence of low-cost models like DeepSeek Flash is a significant indicator, reducing costs not only per token but also per task. These types of models are expected to be the first to realize the concept of unlimited tokens, and the performance gap between state-of-the-art and open-source models is rapidly narrowing to less than three months.

When adopting AI models, customers assign different levels of importance to model origin and data security. Some customers are only concerned with where the model is executed, while others place great importance on the model's origin itself. U.S. customers, in particular, tend to prioritize model origin and data security, leading to increased demand for transparently developed models like NeMo.

Ollama's co-founder stated that after participating in YC in 2021, the product was officially launched only in July 2023. The journey involved finding customer problems that could satisfy developers, and trying various ideas and pivoting until 2023 was a fortunate outcome. This highlights the importance of solving customer problems and the necessity of pivoting.

Ollama initially started in Kubernetes security but later shifted its business direction to desktop developer security. This process took about two years to solidify the idea. The team of 10 spent an uncertain period feeling a lot of fear. This pivot demonstrates the need for early-stage startups to respond flexibly to market changes and customer needs.

Initially, the focus was on providing developer security solutions in Kubernetes environments. However, after analyzing market reactions and technological trends, the company decided that greater opportunities lay in the desktop developer tools market. This decision was a critical turning point that redefined the company's core competencies and sought new growth engines.

The process of changing business direction was not easy. Having already secured funding for the initial idea, transitioning to an entirely new field involved significant risk. For a small team, the psychological pressure of uncertainty was considerable.

Nevertheless, the Ollama team recognized the need for change and made a bold decision. They invested two years in developing a new product and finding market fit, which became the foundation for their current growth into a successful open-source AI model platform.

Open models are rapidly spreading in enterprise environments. Currently, 85% of Fortune 500 companies use Ollama products, largely because anyone can start for free and run it anywhere. This ease of access is quickly being adopted by in-house developers, becoming a key driver for the widespread adoption of open models, similar to the diffusion of databases like MongoDB.

The emergence of coding agents this year is cited as the moment open models dominated the mainstream AI market. In the past, closed models were considered to have higher market fit, but the situation reversed with the advent of coding agents. Now, as open models can finally meet the demand in the most AI-intensive areas, they have become a core driver of the AI industry.

Ollama's founder cited transparent information sharing among founders as the reason for joining the YC community. In the YC community, previous generations of companies share difficulties that don't need to be repeated, which helps alleviate the loneliness of entrepreneurship and learn effective methodologies. This is an important value that helps founders make better decisions without repeating similar mistakes.

In the AI infrastructure era, strategies different from traditional startup perceptions are effective. In the past, startups acting as layers on top of other systems were perceived as vulnerable, but this is not the case in the AI world. Moving higher up the stack can actually be a better strategy as it brings closer proximity to customers. This is because LLMs have non-deterministic characteristics and do not always work as designed.