BioBasicsBench: GPT-6 Astra Leads in 100 Wet Lab Tasks

Key Takeaways

1BioBasicsBench measures the interpretation of experimental data by AI agents across 100 wet lab tasks

2At high reasoning effort, GPT‑6 Astra achieves 66.7% credit, compared to 43.8% for GPT‑5.6 Sol

3The evaluation covers seven models, 37 settings, and 18,500 results, highlighting the advancements of GPT‑6 Astra in multimodal management and context

4The tasks span five domains and are built from real data, with expert review

Why it matters : This benchmark allows for assessing the advancements and limitations of AI agents for biological research in wet labs.

BioBasicsBench, a testing bench dedicated to wet lab environments, measures the interpretation of raw experimental data by AI agents across 100 tasks. At a high reasoning effort level, GPT-6 Astra achieves 66.7% credit, compared to 43.8% for GPT-5.6 Sol. The evaluation, conducted on 18,500 outputs and seven models, highlights progress but also areas for improvement in control verification and image reading.

Clear Disparities Between Models at High Reasoning Effort

At the "xhigh" reasoning effort level, GPT-6 Astra secures 66.7% of the available credit, while GPT-5.6 Sol reaches 43.8%. The results indicate advancements in routine lab analysis, but improvements are still needed for image interpretation, control verification, and the application of experimental context. GPT-6 Astra stands out with better recognition of data limitations, enhanced multimodal management, and more reliable maintenance of biological and experimental context. The evaluation encompasses 37 model and reasoning effort settings, totaling 18,500 results.

What the Wet Lab Testing Bench Measures

BioBasicsBench focuses on 100 tasks categorized into five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. These tasks range from basic readings, such as determining the molecular weight of a DNA sample on a gel, to in-depth assay analyses, and involve common manipulations expected from a wet lab professional. The protocol is based on the idea that effective AI agents must be able to reliably interpret experimental results in a context where failures, difficult-to-interpret images, and contradictory observations are commonplace. The designers believe that existing evaluation tools, often based on figures chosen for publications, do not always capture these ambiguities or the problem-solving requirements they entail.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Task Construction and Evaluation Protocol

Scientists designed the tasks based on data from their own experiments, while experts drafted the scoring criteria. Each task was first internally verified by a scientist from Ligo, then reviewed by a doctor specializing in wet labs, before a final internal proofreading. Seven models were evaluated: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna, and GPT-6 Astra. The evaluation spans 37 settings combining models and reasoning effort, with five attempts per task for each of the 100 tasks at each setting, producing a total of 18,500 results.

Stated Ambition for Biological Research

Project proponents describe BioBasicsBench as a step forward in assessing the ability of AI agents to manage routine scientific activities in wet labs. They believe that more accurate data interpretation could revolutionize biological research and enable researchers to draw stronger conclusions. They express a desire for general artificial intelligence to serve humanity by accelerating the understanding of diseases and the development of new treatments.