Building AI Applications in Rust with Rig

Stephen Korzeniewski, the lead maintainer of Rig, introduced the library as a Rust-based tool designed for interacting with Large Language Models (LLMs). He explained that Rig supports all major LLM providers and facilitates the integration of AI agents.

Korzeniewski stated, "currently uh I'm the lead maintainer of rig, which is a a library written in Rust for uh interacting with LLMs. Basically we support all the major providers."

Korzeniewski recounted his decision to transition from a development stack involving C, Python, and JavaScript to Rust. He expressed frustration with managing multiple languages for distinct tasks.

His primary motivation was to find a single, powerful language that could serve as a unified replacement across his entire development stack. Korzeniewski aimed for a more streamlined and cohesive workflow.

Korzeniewski's interest in AI predates the recent surge in Large Language Models (LLMs), rooted in his background in computational linguistics and natural language processing. He has long been familiar with various text generation techniques.

He drew a connection between modern AI capabilities and the historical context of text-based games such as Zork, illustrating how the foundational concepts of text generation have evolved over time. This background informs his current work with AI.

Korzeniewski demonstrated a practical application of AI in his live video setup, which utilizes the Bevy game engine and Ratatouille. His system employs a machine learning model to convert webcam input into 3D bone tracking data.

These 3D bone data points are then fed into Bevy, enabling real-time control of his avatar. This integration showcases AI's role in creative and interactive applications.

Korzeniewski highlighted Rust's strictness as a significant advantage for ecosystem cohesion. Unlike languages such as JavaScript or C, Rust features a unified build system and consistent package management.

The borrow checker, a core feature of Rust, enforces a cohesive standard that ensures resiliency and compatibility across third-party libraries. This design philosophy makes the ecosystem feel like 'Lego pieces' where everything fits together seamlessly, enhancing overall productivity and reliability.

Korzeniewski emphasized that developer preference leans towards practical utility, prioritizing performance and WebAssembly (Wasm) compatibility. These factors are crucial for real-world applications and widespread adoption.

He appreciates Rust for its ability to balance power and stability without introducing unnecessary complexity, focusing on a robust and reliable programming experience rather than theoretical perfection in niche type-system complexities.

Korzeniewski explained that asynchronous programming is essential for LLM-powered Rust applications due to the extensive network communications involved. Many operations, such as interacting with external models, rely on efficient asynchronous handling.

While Rig abstracts much of this complexity, developers still need to operate within an asynchronous environment to manage these network interactions effectively. This ensures that applications remain responsive and performant.

Korzeniewski defined a 'tool' in Rig as a Rust function that an AI agent can execute. These tools define essential characteristics, including their name, input and output schemas, and specific error types, enabling structured interaction.

Crucially, an AI agent itself can be considered a tool, allowing for recursive calls where one agent can utilize another agent as a functional component. This architecture enables complex, nested AI workflows.

Rig incorporates a robust hook system that functions as middleware, allowing developers to observe and modify inputs and outputs at every stage of model execution. This system provides granular control over the AI pipeline.

Hooks enable advanced features such as the injection of RAG (Retrieval Augmented Generation) context and the recovery from failed tool calls. This capability enhances the flexibility and error handling of AI applications built with Rig.

Rig's design philosophy centers on providing a low-level library that gives users maximum power without hiding underlying complexities. The aim is to ensure developers have full control over their applications.

Simultaneously, Rig minimizes the burden of writing complex asynchronous Rust code, abstracting away difficulties where possible. The library also ensures that its internal types do not leak into user code, maintaining a clean and ergonomic developer experience.

Korzeniewski clarified that some of Rig's terminology, such as 'completions,' is largely legacy, having been in use since the library's inception about three years ago. This reflects the library's long-standing presence in the Rust AI ecosystem.

He stated that the naming convention is not particularly significant and encouraged community feedback via GitHub issues. This open approach allows the project to evolve based on user input.

Rig employs a static cassette system for its Continuous Integration (CI) testing, utilizing approximately 1,700 static HTTP recordings. This approach ensures consistency and reliability, especially given potential model regressions in live services.

By running tests against these static cassettes, Rig avoids dependencies on production services during CI, allowing tests to complete rapidly, often in just a few seconds. This method guarantees stable and reproducible test results.

Rig is actively used in various production environments, with one notable application being at St. Jude’s Children’s Hospital for medical chat applications. This highlights the library's robustness and reliability in critical sectors.

Korzeniewski noted a healthy and active community surrounding Rig, with frequent contributions from corporate users. This engagement underscores the library's growing adoption and utility in real-world scenarios.

Korzeniewski revealed a complete shift to AI-driven workflows for his projects, stating he hasn't written code by hand in months. Despite this, he has experienced his most productive coding year to date, demonstrating the efficiency of AI assistance.

He utilizes multiple AI agents concurrently to manage diverse tasks, advocating for a workflow that combines precise prompt engineering with Git-based verification. This methodology allows him to scale his output significantly.

Korzeniewski described his developer workflow for managing complex AI-powered projects as remarkably simple, relying primarily on terminal tools. He uses terminal multiplexers like Zellij to organize multiple project tabs efficiently.

Context switching is managed through detailed documentation and note-taking within text editors such as Helix. His development process involves reviewing AI-generated pull requests and performing human verification to maintain quality.

Korzeniewski noted a significant shift in his developer satisfaction, moving from the act of writing syntax to the gratification of architecting creative and impactful projects. He explained that AI adoption altered his 'dopamine receptors,' making project completion more rewarding than manual coding.

AI enables developers to undertake projects that were previously too time-consuming or complex due to the heavy manual effort required. This paradigm shift positions the developer's role closer to that of a senior engineering manager, directing AI 'juniors' rather than performing repetitive coding tasks.

Korzeniewski acknowledged the validity of concerns from both proponents and skeptics regarding AI adoption, emphasizing the importance of respecting diverse viewpoints. He noted the initial 'AI hype' led to a lot of 'slop' from inexperienced users, particularly before AI coding capabilities matured.

Experienced engineers often worry about security vulnerabilities and the quality of code generated by inexperienced AI users, especially in critical industries like automotive and aerospace. Other concerns include corporate constraints, job security, and the environmental impact of AI. Ideological debates around copyright and training data usage also contribute to resistance.

Korzeniewski acknowledged the significant societal changes that AI will bring and stressed the importance of societal awareness. He advocated for establishing a safety net to prevent adverse outcomes such as widespread job loss or social disarray.

He emphasized the continued importance of humans in the face of rapid AI advancement, urging caution to avoid negative consequences and ensure human well-being. Korzeniewski believes that careful planning is necessary to navigate the future with AI.

Rig is developing a new experimental runtime that integrates with the Bevy game engine. This initiative aims to leverage Bevy's Entity Component System (ECS) to manage Rig agents as Bevy entities.

The goal is to utilize Bevy's ecosystem and tooling to create a robust coding harness, providing enhanced visualization and advanced control for AI agents. This integration seeks to bring game engine capabilities to AI development.

Bevy has recently adapted its open-source policies, moving away from a strict 'no AI' policy for contributions to now allowing AI-assisted submissions. This change was prompted by practical difficulties encountered under the previous rule.

Stephen Korzeniewski noted that the strict ban led to internal paranoia among maintainers and reviewers, who found it challenging to distinguish between human-written and AI-generated code, sometimes resulting in baseless accusations. The new policy aims to foster a more transparent and trusting environment.