Researchers Train AI to Avoid Bioweapon Knowledge in Pre-Training

Researchers are exploring a novel approach to mitigating the risks of artificial intelligence inadvertently teaching people how to build bioweapons: simply avoid training AI models on the dangerous information in the first place [1]. This method, detailed in a new study titled Deep Ignorance, was led by Stella Biderman of Eleuther AI in collaboration with the UK government’s AI Security Institute and other researchers. The findings suggest that filtering out harmful content from AI training data can create more robust safety measures that are harder to bypass or reverse-engineer [1].

The study involved training open-source AI models on datasets that had been scrubbed of proxy information—such as bioweapon-related material—used as stand-ins for harmful content. The results showed that these models were less likely to generate harmful outputs, without a significant loss in performance on other tasks. This approach represents a departure from traditional post-training safety measures, which are often easier to tamper with and can lead to unintended consequences [1].

One of the researchers, Stephen Casper, emphasized that the goal was to create AI models that are inherently safe—not just at launch, but even if someone later tries to modify or manipulate them [1]. This is a critical challenge in the AI field, where many safety interventions are applied after a model has already been built. Pre-training filters, by contrast, aim to embed safety from the outset, making the model more resilient to tampering [1].

Biderman noted that such pre-training approaches are uncommon in public research due to high costs and complexity, which makes them largely the domain of private AI firms like OpenAI and Anthropic. However, these companies are secretive about their methods and do not typically disclose how they filter training data [1]. OpenAI, for example, has indicated that it filters pre-training data to exclude hazardous biosecurity knowledge, using filters similar to those applied in its proprietary GPT-4o model [1].

The Deep Ignorance project aims to challenge the narrative that AI training data is too large and opaque to be properly curated or understood. Biderman described this as a “story” that companies like OpenAI often use to justify a lack of transparency. By demonstrating that it is possible—and necessary—to filter harmful content during pre-training, the research team hopes to encourage broader adoption of these methods in both open-source and proprietary models [1].

The implications of this work extend beyond bioweapon risks. As AI becomes more integrated into domains like healthcare, finance, and national security, ensuring that models are trained on clean, ethical, and safe data is increasingly important. The Deep Ignorance approach could serve as a blueprint for embedding safety by design in AI systems—rather than retrofitting it later [1].

Source: [1] Worried AI could teach people to build bioweapons? Don’t teach it how, say researchers (https://fortune.com/2025/08/14/worried-ai-could-teach-people-to-build-bioweapons-dont-teach-it-how-say-researchers/)