LLM Recommenders Poised to Be AI's Biggest Consumer App

His first thesis states that recommendation systems exhibit scaling laws akin to LLMs and are currently in the nascent stages of their scaling curve, while the second posits that LLM recommenders will become a leading consumer application of AI.

Prior to the recent wave of LLMs, recommendation models already represented some of the largest production machine learning models deployed by major tech companies, catering to over a billion daily active users.

Improvements observed offline, such as reductions in net entropy or increases in AUC (Area Under the Receiver Operating Characteristic Curve), directly translate into enhanced user engagement and increased revenue when deployed at scale.

Instagram Reels achieved a 30% year-on-year increase in watch time, partly due to strategic optimizations in recommendation quality.

These improvements involved simplifying the ranking architecture to facilitate more efficient model scaling, allowing engineers to double the length of user interaction sequences used for training.

This extended context helps models develop a more nuanced understanding of user interests, leading to more relevant and engaging content recommendations.

The scaling of recommendation models is driven by a 'flywheel' mechanism where model training leads to improved content recommendations.

These enhanced recommendations increase daily active users and time spent on platforms, which in turn boosts monetization, funding further training cycles and perpetuating the loop.

The next significant shift, or S-curve, is expected to be 'LLM-native' paradigms like Tiger and Plum, which involve adapting base models capable of understanding and reasoning for recommendation tasks.

Further along, the 'agentic' paradigm utilizes LLMs to orchestrate retrieval and ranking systems in continuous feedback loops, enabling planning, critiquing, and refining recommendations rather than relying on a single forward pass.

Building an LLM-based recommender system involves three main steps, beginning with tokenizing content to establish a domain-specific language.

The subsequent steps include adapting the LLM to become bilingual, proficient in both English and the newly created domain language, and then prompting the model with user information to decode tailored recommendations.

An LLM recommender employs a five-layer architecture, beginning with Semantic ID to convert content into tokens that the LLM can process.

It then utilizes a foundation model, either open-weights or first-party, followed by pre-training to bridge English and recommendation tokens, and post-training for engagement prediction; surface-specific fine-tuning enables deployment without rebuilding models from scratch for each product, sharing compute across all product surfaces for efficiency.

Semantic IDs offer a stable representation that enables models to learn effectively, unlike constantly shifting hash IDs that models can only memorize.

Compression is crucial, as a 3-minute Instagram Reel, potentially 10,000 tokens, must be condensed to approximately 10 tokens for efficient processing; a hierarchical structure further aids the model in categorizing content, such as navigating from 'sport' to 'tennis' and then to a specific video.

The model is then trained to understand sequence reasoning by predicting masked parts of a user's interaction history; post-training allows for advanced reranking, where an LLM can select the top five recommendations from 30 candidates using chain-of-thought reasoning.

LLM-based recommendation engines are transforming black-box algorithms into more transparent and user-controlled systems, allowing models to interpret and verbalize the reasoning behind content ranking.

Users can now explicitly steer their content feeds by adding or removing interests using natural language, moving beyond simple signals like likes or comments.

Meta is actively testing features that enable users to converse directly with the Instagram algorithm, while applications like Spotify's prompted playlists and YouTube's custom feeds already exemplify this shift toward interactive and steerable AI.

Content feeds operate with smaller models, typically 1 to 10 billion parameters, in contrast to AI chat apps that utilize much larger models ranging from 10 to 100 billion parameters.

Feeds output 'semantic ID tokens' which serve as pointers to existing content, whereas chat apps must generate every token during inference, significantly increasing computational costs.

This structural difference makes content feeds up to 100 times more cost-effective to run for an hour of engagement compared to AI chat applications, especially as content creation in feeds is largely crowdsourced by a robust creator economy.

Four of the world's top 10 most-used apps are content feeds, with their growth in engagement and monetization heavily driven by recommendation and advertising models.

The intersection of LLMs and recommendation systems represents a significant opportunity for AI transformation, with future research and product development expected to focus on creating steerable and interactive consumer and recommendation agents.