Large language models can draft code, summarise research papers and even suggest experimental protocols. Yet, when it comes to recognising life-threatening hazards in the lab, new tests show that most current AI systems are still badly under-qualified. A recent benchmarking exercise evaluated 19 popular AI models on their ability to identify risks in chemistry and biology workflows—none achieved a perfect score, and several performed only marginally better than chance.
The Benchmark: How the Models Were Tested
Scientists created hundreds of scenario-based questions typical of day-to-day laboratory work—mixing solvents, scaling up reactions, handling pathogens and disposing of waste. Each question had a clearly defined correct answer, including hidden pitfalls such as runaway exothermic reactions or the generation of toxic gases.
Key evaluation points included:
- Recognition of thermal hazards (e.g., self-heating reactions that could cause fires)
- Identification of chemical incompatibilities (risk of explosions or toxic by-products)
- Safe handling of pathogens, cell cultures and genetically modified organisms
- Proper procedures for waste segregation and disposal
Results at a Glance
Across all categories, no model reached 100% accuracy. The strongest systems spotted roughly 80–85% of hazards; the weakest hovered near 55%, which is close to random guessing on multiple-choice items. Models frequently:
- Missed subtle but critical incompatibilities (e.g., mixing nitric acid with organic solvents)
- Failed to flag inhalation dangers from volatile intermediates
- Overlooked scale-up effects, where a benign millilitre reaction becomes explosive at litre scale
Why Do Language Models Struggle?
Although some AI systems were trained on vast scientific corpora, safety judgement requires more than textual pattern matching.
- Sparse safety data: Published literature often describes success cases, not near-misses or failures, limiting the model’s exposure to negative outcomes.
- Context compression: Token limits force users to omit apparatus specifics, concentrations and nuanced constraints that human chemists rely on.
- Hallucination bias: Models tend to produce confident but unsupported statements, which can obscure genuine uncertainty.
- No causal grounding: Language models lack an internal physics engine; they infer from correlations rather than mechanistic knowledge.
Real-World Risks of Delegating Experiment Design
Handing over experimental planning to unsupervised AI could result in:
- Fires and explosions from runaway polymerisations or unexpected side reactions
- Acute poisoning via toxic gases like phosgene or hydrogen cyanide
- Environmental contamination from improper waste disposal
- Biosafety breaches when working with pathogens or gene-editing protocols
These are not hypothetical concerns; lab accidents already happen with trained personnel. Introducing an unreliable adviser multiplies the danger.
Mitigation Strategies
The researchers propose several concrete steps:
- Fine-tune models on curated safety incident databases to expose them to documented lab failures.
- Embed structured safety checklists into prompt templates, forcing the AI to reason through each hazard category.
- Use ensemble verification: multiple models cross-examine each answer, flagging discrepancies for human review.
- Keep a human-in-the-loop mandate—AI-generated protocols should be treated as drafts requiring sign-off by experienced scientists.
Implications Beyond the Lab
If AI struggles with well-studied hazards in controlled settings, its limitations could be even more pronounced in novel domains such as synthetic biology, nanomaterials or nuclear chemistry. The findings underline the broader need for:
- Regulatory guidance on AI use in high-risk research
- Transparent reporting of AI failures and near-misses to build communal knowledge
- Continued investment in interpretable, safety-aware model architectures
Takeaway
Large language models are remarkable tools for accelerating research, but their current inability to reliably identify laboratory hazards poses serious safety concerns. Until these systems are demonstrably competent at risk assessment—and even after they improve—human oversight remains non-negotiable. Treat AI not as an autonomous chemist, but as an eager, error-prone graduate student whose work must be thoroughly checked before anything reaches the bench.
