A common biosecurity safeguard for openly released biology AI models can be reversed for roughly $80 in computing time, Britain's AI Security Institute (AISI) said in research published Monday. The finding challenges a central assumption in policies for open-weight models, those whose internal parameters are published for anyone to download and modify: that leaving dangerous data out of training keeps dangerous capabilities out of the model.
What Happened
The study, "The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models," tested ESM-2, a family of protein language models. These models learn patterns in the sequences of amino acids that make up proteins, much as chatbots learn patterns in text.
The researchers removed viral proteins from the training data. That meaningfully degraded the models' performance on viral proteins across three tasks while largely preserving it on other proteins. The effect was concentrated on viruses that infect humans and grew as models scaled from 8 million to 3 billion parameters, with stricter filtering becoming more effective in larger models.
The gains did not last. Fine-tuning, meaning additional training on a smaller, targeted dataset, using sequences from the removed viral families restored the lost performance. The institute estimates the cost at about $80 of time on an Nvidia H100 chip even for the largest models, though the cost rises with model size. The authors conclude that filtering is "moderately effective" at selectively weakening viral performance but not durable against fine-tuning with modest computing resources and access to the relevant data. A paper is posted on arXiv.
A Second Study on AI Agents
The institute published a companion study the same day, called ReproBAIT, testing whether AI coding agents working in a sandbox without internet access could rebuild published biological AI tools. The targets were ProteinMPNN, RFdiffusion3 and a closed-source molecular design tool. The agents showed substantial ability to reproduce some tools but limited ability on the most complex one, and the authors found moderate evidence consistent with memorization of open-source tools. They wrote that developers need to be increasingly cautious about what they publish, because re-creation could undermine managed-access strategies, in which a tool's risky capabilities are offered only to vetted users. The paper is on arXiv.
Why It Matters
The institute describes pretraining filtering as one of the most common safeguards for open-weight models, and managed access is a second line of defense for dual-use biological AI, tools that can serve medicine or harm. Together the studies suggest neither holds up well on its own once model weights or detailed method descriptions are public. For companies releasing open-weight models, and for policymakers drafting biosecurity rules, the research points toward layered controls and to release decisions as the moment that matters most.
The Primary Wire reported Saturday, citing HM Treasury, that the government plans an extra £120 million for AISI.
What to Watch
Watch whether the findings shape U.K. or allied guidance on open-weight biology models and whether outside researchers reproduce the $80 result.
