Leash Bio Explores AI's Understanding of Chemistry Versus Social Factors in Drug Discovery

Leash Bio's co-founder and CEO, Ian Quigley, shared that AI models can identify the chemist who created a molecule with a 60% top-five accuracy, based on the molecule alone. This capability suggests that the models might be learning sociological patterns from the data rather than fundamental chemical truths.

The company conducted an experiment where they downloaded molecules from all chemists in Reaxys, a large repository of chemical compounds. They included chemists who had published over 30 papers and 600 molecules, resulting in a dataset of 1,815 chemists.

The molecules were represented as an 1,800-element probability vector over the creator, rather than traditional chemical structures. Training a classifier on this representation to predict activity yielded results comparable to models using standard chemical formulas.

Quigley concluded that models are learning a lot of sociology, not ground truth, and warned that training models on the 'exhaust' of scientific literature could be risky due to these embedded biases.

Leash Bio's method for detecting molecular binding involves attaching a target protein to a magnetic bead, which is then immersed in a complex mixture of small molecules. Only molecules with the correct shape and electronic properties will bind to the protein.

After binding, the protein and attached molecules are pulled out, and unbound molecules are rinsed away. The DNA barcodes of the retained, bound molecules are then amplified using PCR and sequenced.

This process is analogous to a 'fish hook' mechanism, where the bead selectively captures desired interactions. Controls are essential to filter out non-specific binding, where molecules might stick to the bead or protein without a functional interaction, which can lead to misleading results.

The presence of a bulky DNA tag on small molecules in DNA-encoded libraries (DELs) is a known concern, as it can potentially alter the molecule's behavior. However, Leash Bio treats this barcode as an integral part of the measurement, conditioning its machine learning models on its existence.

For simple targets, approximately 90% of chemicals can be successfully manufactured off the DNA tag, while for more challenging targets, this figure drops to about 50%. Despite this attrition, the binding event with the barcode still represents a genuine signal for machine learning purposes.

Leash Bio reports high reproducibility and robustness in their experiments. Re-running experiments over several months, using fresh protein and new batches of chemical libraries, consistently yields largely the same hits in the same order.

Ian Quigley identified a group of four companies, including Leash Bio, Noetic, Inveterate, and Known Medicine (later acquired by PAS), as 'Recursion mafia companies.' These companies, founded by Recursion alumni, share a common approach to extracting meaningful signals from noisy biological measurements.

These organizations leverage the expertise gained at Recursion to address complex biological problems by applying a high degree of epistemic rigor to data collection and analysis.

The shared philosophy emphasizes careful experimentation and robust methods to ensure data quality and reliable interpretation, particularly in environments characterized by significant measurement noise.

Many toxicities in drug development arise from small molecules binding to unintended proteins. Leash Bio aims to predict these off-target bindings using its models, thereby flagging potential toxicity early in the process.

The company's aspiration is to screen every protein, including those from non-human species like mice and rats. This comprehensive approach is crucial because chemical behavior can vary significantly across different species due to slight protein differences.

Beyond initial binding predictions, Leash Bio also considers downstream cell-level assays. These assays evaluate emergent behaviors such as membrane permeability and interactions with specific cell types, like cardiac cells for arrhythmia, providing a more complex understanding of drug effects.

Leash Bio organized a Kaggle competition a few years ago to rigorously evaluate the ability of computational models to predict small molecules binding to chemicals. The dataset for this competition comprised approximately 4 billion physical interactions, a thousand times larger than previous datasets like BindingDB.

The competition's design emphasized epistemic rigor, moving beyond simple random train-test splits. Instead, the organizers carefully structured the data to test models across a range of evaluation difficulties, aiming to accommodate both beginners and advanced participants.

The goal was to provide a fair assessment of predictive power, with specific challenges to probe model behavior, ensuring that models were truly learning chemistry rather than memorizing data patterns.

Upon releasing a dataset for a Kaggle competition that was a thousand times larger than any previously seen, Leash Bio's founders estimated a 5% chance that recent advancements in co-folding approaches could render their company entirely obsolete. This fear, however, did not materialize.

The actual results of the competition showed that participants struggled to achieve high accuracy, suggesting that the field's predictive capabilities were not as advanced as initially feared. Ian Quigley speculated that competitors were not yet proficient in these tasks, and the data itself might help improve future performance.

Furthermore, the competition helped validate Leash Bio's choice of molecular representation. They found that basic SMILES representations outperformed other complex methods, including 3D graphs and spherical convolutional nets. This finding reinforced their decision to continue using the SMILES approach for its sufficient performance.

In response to criticisms that poor competition results were due to weak participants, Leash Bio made all raw data, including read counts and binary labels, publicly available. This transparency allows anyone to verify the findings and test their own models.

The entire dataset is hosted on Polaris, a nonprofit organization managed by Recursion, making it accessible to the broader scientific community.

Despite the full release, Ian Quigley noted that no one has yet demonstrated outstanding performance on the most challenging evaluations, reinforcing Leash Bio's initial assessment of the difficulty of the problem.

Quigley believes that truly understanding the physics of molecular interactions, rather than relying on structural models, is key to improving predictions, regardless of the training set's distribution.

Leash Bio explored different data split strategies to evaluate model performance, especially considering the combinatorial nature of DNA-encoded libraries (DELs). A random split of DELs can lead to models memorizing rather than learning, as building blocks might be shared between training and test sets, artificially inflating performance.

To address this, the team implemented a more rigorous split design: clustering building blocks by similarity and ensuring entire clusters were exclusively in either the training or test set. This approach revealed a significant drop in predictive performance, which they attribute to preventing memorization.

When comparing against baselines like XGBoost, their own model demonstrated consistent performance across various split regimes. XGBoost's performance deteriorated significantly as the test set moved further from the training distribution, particularly when building blocks were entirely held out. In contrast, Leash Bio's model remained relatively stable, indicating it was learning more generalizable principles rather than simply memorizing training data.

Around 2019, a medicinal chemist at Recursion, Mike Karesi, demonstrated an uncanny ability to identify the creator of a molecule simply by glancing at its structure. This observation raised concerns for Ian Quigley and his team about the potential for AI models to learn 'chemist intent' or sociological biases rather than pure chemistry.

To investigate this, Leash Bio collected molecules from 1,815 chemists who had published extensively in Kemble. They trained a classifier to predict the chemist based solely on the molecule, achieving 60% top-five accuracy. Notably, confusion was highest among chemists working on the same papers or at the same institution, suggesting that students often adopt the styles of their mentors.

This phenomenon highlights how chemists leave unique fingerprints through their choices, including options they considered but never pursued. These unrecorded decisions still influence the final product. Additionally, Quigley noted that heavier molecules often appear to bind better in public datasets not due to inherent chemical properties, but because chemists tend to add groups to molecules that show promising activity.

Ultimately, these findings reinforced Leash Bio's belief that AI models trained on such data learn the 'sociology' of the project, including human preferences and decision-making patterns, rather than purely objective chemical principles.

Leash Bio exclusively trains its models on internally generated data, deliberately avoiding external datasets. This strategy allows them to control for potential biases and noise present in publicly available data.

Instead of training on external data, Leash Bio uses these external molecule collections for prediction. They tested their models on a test set that included about 500 protein targets and roughly 13 million tests, a scale comparable to that in the Hermes paper.

Their models were benchmarked against existing state-of-the-art structural models, such as those developed by the Bolts team. Despite not training on external data, Leash Bio's models achieved comparable performance on public datasets for molecular binding tasks, demonstrating their ability to generalize predictions effectively.

Leash Bio focuses on identifying and exploiting 'activity cliffs' in drug discovery. An activity cliff refers to two nearly identical molecules where a tiny structural change, such as a 'magic methyl' group, leads to a significant difference in binding activity.

This concept is applied to DNA-encoded libraries (DELs), where two out of three building blocks can be kept similar across many examples while varying a single group. The building blocks themselves typically range from 6 to 30 atoms, excluding hydrogens, allowing for subtle yet impactful structural modifications.

On the protein side, Leash Bio intentionally engineers mutations in target proteins to create similar activity cliffs. This dual approach of varying both small molecules and proteins helps to map the precise interactions that drive binding and non-binding events, crucial for understanding and scaling chemical diversity.

Leash Bio has demonstrated through multiple independent DNA-encoded library (DEL) experiments that a single side group on a building block can clearly interact with a specific amino acid residue, leading to observable shifts in population behavior. They observed this in five independent examples, providing clear and robust evidence.

This level of detail is achievable because the DEL approach allows for the creation of vast numbers of similar molecules with only one systematic change. By analyzing how binding behavior shifts with these minute modifications, researchers can pinpoint specific interactions.

To further map these interactions, Leash Bio digitally 'breaks' 3-5 pocket residues per protein. This experimental engineering of mutations in protein binding regions, followed by re-screening, helps to understand which part of the molecule is facing which residue and how these interactions contribute to binding affinity.

Leash Bio extensively utilizes mutation-based screens to understand how small changes in protein pockets affect molecular binding. These screens reveal mutations that either enhance binding (appearing as upward shifts on plots) or entirely prevent molecules from binding.

This approach is particularly valuable for kinase families, which often contain around 100 members that differ by only a single residue within their important binding sites. Mapping these subtle differences in both compound and protein space provides critical insights into drug selectivity.

The team conducts hundreds of such mutation-based screens, systematically breaking and re-screening binding regions. This high-throughput method allows them to identify key amino acids that are critical for binding and to understand the precise nature of activity cliffs.

By identifying these causal changes, Leash Bio aims to develop models that deeply understand the underlying physics of molecular interactions, moving beyond simple memorization of binding events.

Leash Bio is developing LLM agents trained on natural language to mimic the reasoning process of medicinal chemists. These agents are taught medicinal chemistry by describing chemical structures and differences between molecules in natural English, using open-source models like Qwen and GLM.

The agents are given a starting point of 10 binders and 10 non-binders, along with a budget of 100 molecules, to identify the best binders from a catalog of millions. This process, akin to a coding sandbox, provides instant feedback, allowing agents to quickly update their hypotheses.

The ultimate goal is for these agents to find the best binder in the fewest possible steps, mirroring how a human medicinal chemist would operate. Last weekend, Leash Bio generated over 10,000 such campaigns, with each run involving approximately 100 to 200 calls from a pool of 6.5 million molecules.

Ian Quigley suggests this approach is more resistant to the memorization problem often seen in other models, as natural language acts as a strong regularizer, encouraging the agents to learn causal changes rather than simply memorizing training data.

Ian Quigley described the vision of combining the basic chemical intuition gained from large language model (LLM) pre-training corpora with an 'infinitely fast lab in the loop.' This approach aims to equip AI agents with foundational medicinal chemistry knowledge, akin to a mid-level medicinal chemist.

He emphasized that reasoning skills are crucial, allowing agents to learn from reinforcement learning campaigns and apply that knowledge to tackle progressively harder questions. This iterative process of making choices and receiving feedback is vital for continuous learning.

Currently a small company of 10 people, Leash Bio anticipates that as their experimental corpus grows exponentially (2x, 10x, 100x), the exploration space for these agents will dramatically expand. This growth in data will enable agents to make more sophisticated and counterintuitive decisions, and learn from their mistakes, ensuring unbiased learning.

The company aims to develop agents that can make causal changes and understand counterfactuals, focusing on the quality and unbiased nature of the data rather than just its volume to prevent the models from learning 'chemist sociology fingerprints.'

While Leash Bio primarily focuses on binding affinity, the company is also integrating ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) properties into its drug discovery process. This expansion addresses the broader challenge of developing effective and safe drug candidates beyond just target engagement.

Ian Quigley claims that Leash Bio has achieved promising results in PKPD (pharmacokinetics and pharmacodynamics) experiments, moving from initial chemical library hits to good PKPD in mice within six months, with a minimal number of med-chem cycles. This suggests their initial chemical material selection is of high quality.

The company incorporates ADMET prediction models into its optimization functions when designing chemical libraries. These functions score generated virtual libraries based on binding targets, avoidance targets, and various ADMET predictions, allowing for the selection of drug-like compounds early on.

In a specific example, they tracked inflammatory cytokine levels in mouse blood, and the results aligned with the drug's target mechanism, indicating successful in vivo performance and the potential for their approach to yield materials with desirable ADMET profiles.

Leash Bio is actively investigating whether cytokine targets, traditionally addressed by antibody therapies, can be effectively targeted by small molecule drugs. This endeavor aims to provide more accessible and less logistically challenging treatment options, as small molecule pills circumvent the need for cold chain transport and high costs associated with antibody-based drugs.

While many protein targets are challenging to drug with small molecules for various reasons, Ian Quigley asserts that Leash Bio's technology is capable of generating effective small molecule materials for these targets.

He confirmed that Leash Bio currently possesses selective materials for a range of cytokine targets. This achievement indicates potential breakthroughs in an area where small molecule intervention has historically been difficult, opening avenues for new therapeutic approaches.

Leash Bio has achieved significant operational milestones, having developed and screened approximately 2,000 different protein constructs. This includes a total of about 5,000 screens throughout the company's history, each across at least 6.5 million molecules.

Ian Quigley suggested that the number of mutations screened by Leash Bio is likely greater than that of any other entity, emphasizing their extensive exploration of chemical space and protein variations.

A core focus for the company is to make the entire screening process highly cost-effective and scalable. Quigley contrasted their approach with contract research organizations (CROs), where a single screen could cost around $100,000, highlighting Leash Bio's ability to conduct numerous campaigns at a fraction of the traditional cost.

Despite screening millions of molecules, Quigley acknowledged that 6.5 million molecules represent only a fraction of the vast chemical space. However, he noted that their internal-data-only training approach has allowed them to make generalizable predictions into very different corners of chemical space, indicating the effectiveness of their chosen methodology.

Ian Quigley identified a critical operational bottleneck in drug discovery: the lack of highly parallelizable tools for purifying small molecules at scale. While acknowledging the general industry focus on GPU limitations and sequencing costs, he highlighted this specific challenge as a unique hurdle in their workflow.

Currently, purification methods like HPLC are not easily scalable for the massive volumes of molecules handled by Leash Bio. A breakthrough in mass-scale purification would significantly enhance their ability to clean up reaction outputs, ensuring higher quality material for downstream validation.

The ability to purify small molecules at a massive scale would allow Leash Bio to orthogonally validate their measurements in 'pure' assays, providing a crucial check against the complex, potentially 'trauma-inducing' assays they currently use. This would lead to more reliable and reproducible drug discovery outcomes.