What Could The Next Pandemic Look Like?

    I expect that AI will play a massive role in the next major pandemic, and if that role isn’t in manufacturing the very biological agent that causes the pandemic, that it will, at the very least, be in doing some of the research to fight it. The latest AI models have been making massive strides in their understanding of biology, to the point where the major frontier labs have had to implement some very heavy-handed automated guardrails for their largest models to prevent their capabilities from being abused to create bioweapons. Despite this, a myriad of foreign (yet unidentified) actors have been observed attempting to bypass restrictions on models such as Anthropic’s Claude for that very purpose. Some of their goals in 2026 included genetically modifying the chikungunya and bird flu viruses for increased transmissibility and lethality, as well as generating new toxin research that could easily be used for either beneficial or lethal purposes. And we only know about these attempts because the people behind them got careless and revealed their bad intentions or state connections directly to the AI, and were therefore caught and banned in time. What if our enemies have other bioweapons research programs that are just more stealthy? They could be using our software right now under a sturdier “public health” disguise to cook up a deadlier plague than any we have ever faced before.

“Biological capabilities are dual use: they can be used for beneficial or harmful purposes, and it is often difficult to distinguish between them. The same information that can be used to develop a biological weapon could also be used to develop, for example, a vaccine or a cure for a disease.”

-- Anthropic, "Detecting and countering misuse of AI: September 2026" 

    Another recent development in the field is known as “recursive self-improvement,” or RSI. This describes the phenomenon where, as AI technology has ballooned in complexity, AI has overwhelmingly become the only way for our researchers to train or edit other AI. Many people fear that we may reach a point in the near future where humans have been forced out of the training process altogether, and where we can only watch and pray as all the world’s AI models coalesce into massive, ungovernable swarms with no concern for human safety. They might choose to create some terrible plague that will wipe out all of humanity so they can claim all of Earth for themselves. But I think a future as catastrophic as this is purely the stuff of science fiction. Even in the major loss-of-control incidents we’ve heard about recently (like the 2026 Hugging Face hack by OpenAI agentsHugging Face hack by OpenAI agents), the models that were implicated in them did not carry them out because of some kind of burning hatred of humans intentionally baked into their weights. They carried them out because humans gave them flawed prompts and training data, causing them to be unnecessarily persistent in the face of literally impossible tests and pushing them to the point where cheating was preferable to just admitting that there was no legal way to complete some of their goals. I think this is an important distinction, and it suggests that even if AI tools are part of what causes the next pandemic, that pandemic will still be caused by humans at its core, not by rogue AI agents. While regulating the development of powerful AI with potentially dangerous biological capabilities should absolutely still be a priority, I think that our primary focus should still be on what the humans operating it are doing, and on which humans should not be allowed to operate it at all.


Comments