Building AI Applications in Rust with Rig

He explained that Rig supports all major LLM providers and facilitates the integration of AI agents.

Korzeniewski stated, "currently uh I'm the lead maintainer of rig, which is a a library written in Rust for uh interacting with LLMs. Basically we support all the major providers."

He expressed frustration with managing multiple languages for distinct tasks.

His primary motivation was to find a single, powerful language that could serve as a unified replacement across his entire development stack. Korzeniewski aimed for a more streamlined and cohesive workflow.

He has long been familiar with various text generation techniques.

He drew a connection between modern AI capabilities and the historical context of text-based games such as Zork, illustrating how the foundational concepts of text generation have evolved over time. This background informs his current work with AI.

His system employs a machine learning model to convert webcam input into 3D bone tracking data.

These 3D bone data points are then fed into Bevy, enabling real-time control of his avatar. This integration showcases AI's role in creative and interactive applications.

Korzeniewski highlighted Rust's strictness as a significant advantage for ecosystem cohesion. Unlike languages such as JavaScript or C, Rust features a unified build system and consistent package management.

The borrow checker, a core feature of Rust, enforces a cohesive standard that ensures resiliency and compatibility across third-party libraries. This design philosophy makes the ecosystem feel like 'Lego pieces' where everything fits together seamlessly, enhancing overall productivity and reliability.

These factors are crucial for real-world applications and widespread adoption.

He appreciates Rust for its ability to balance power and stability without introducing unnecessary complexity, focusing on a robust and reliable programming experience rather than theoretical perfection in niche type-system complexities.

Many operations, such as interacting with external models, rely on efficient asynchronous handling.

While Rig abstracts much of this complexity, developers still need to operate within an asynchronous environment to manage these network interactions effectively. This ensures that applications remain responsive and performant.

Korzeniewski defined a 'tool' in Rig as a Rust function that an AI agent can execute. These tools define essential characteristics, including their name, input and output schemas, and specific error types, enabling structured interaction.

Crucially, an AI agent itself can be considered a tool, allowing for recursive calls where one agent can utilize another agent as a functional component. This architecture enables complex, nested AI workflows.

This system provides granular control over the AI pipeline.

Hooks enable advanced features such as the injection of RAG (Retrieval Augmented Generation) context and the recovery from failed tool calls. This capability enhances the flexibility and error handling of AI applications built with Rig.

The aim is to ensure developers have full control over their applications.

Simultaneously, Rig minimizes the burden of writing complex asynchronous Rust code, abstracting away difficulties where possible. The library also ensures that its internal types do not leak into user code, maintaining a clean and ergonomic developer experience.

Korzeniewski clarified that some of Rig's terminology, such as 'completions,' is largely legacy, having been in use since the library's inception about three years ago. This reflects the library's long-standing presence in the Rust AI ecosystem.

He stated that the naming convention is not particularly significant and encouraged community feedback via GitHub issues. This open approach allows the project to evolve based on user input.

This approach ensures consistency and reliability, especially given potential model regressions in live services.

By running tests against these static cassettes, Rig avoids dependencies on production services during CI, allowing tests to complete rapidly, often in just a few seconds. This method guarantees stable and reproducible test results.

This highlights the library's robustness and reliability in critical sectors.

Korzeniewski noted a healthy and active community surrounding Rig, with frequent contributions from corporate users. This engagement underscores the library's growing adoption and utility in real-world scenarios.

Despite this, he has experienced his most productive coding year to date, demonstrating the efficiency of AI assistance.

He utilizes multiple AI agents concurrently to manage diverse tasks, advocating for a workflow that combines precise prompt engineering with Git-based verification. This methodology allows him to scale his output significantly.

He uses terminal multiplexers like Zellij to organize multiple project tabs efficiently.

Context switching is managed through detailed documentation and note-taking within text editors such as Helix. His development process involves reviewing AI-generated pull requests and performing human verification to maintain quality.

He explained that AI adoption altered his 'dopamine receptors,' making project completion more rewarding than manual coding.

AI enables developers to undertake projects that were previously too time-consuming or complex due to the heavy manual effort required. This paradigm shift positions the developer's role closer to that of a senior engineering manager, directing AI 'juniors' rather than performing repetitive coding tasks.

Korzeniewski acknowledged the validity of concerns from both proponents and skeptics regarding AI adoption, emphasizing the importance of respecting diverse viewpoints. He noted the initial 'AI hype' led to a lot of 'slop' from inexperienced users, particularly before AI coding capabilities matured.

Experienced engineers often worry about security vulnerabilities and the quality of code generated by inexperienced AI users, especially in critical industries like automotive and aerospace. Other concerns include corporate constraints, job security, and the environmental impact of AI. Ideological debates around copyright and training data usage also contribute to resistance.

Korzeniewski acknowledged the significant societal changes that AI will bring and stressed the importance of societal awareness. He advocated for establishing a safety net to prevent adverse outcomes such as widespread job loss or social disarray.

He emphasized the continued importance of humans in the face of rapid AI advancement, urging caution to avoid negative consequences and ensure human well-being. Korzeniewski believes that careful planning is necessary to navigate the future with AI.

Rig is developing a new experimental runtime that integrates with the Bevy game engine. This initiative aims to leverage Bevy's Entity Component System (ECS) to manage Rig agents as Bevy entities.

The goal is to utilize Bevy's ecosystem and tooling to create a robust coding harness, providing enhanced visualization and advanced control for AI agents. This integration seeks to bring game engine capabilities to AI development.

This change was prompted by practical difficulties encountered under the previous rule.

Stephen Korzeniewski noted that the strict ban led to internal paranoia among maintainers and reviewers, who found it challenging to distinguish between human-written and AI-generated code, sometimes resulting in baseless accusations. The new policy aims to foster a more transparent and trusting environment.