How to Dismantle Knowledge of an Atomic Bomb

The fallout from Meta's extensive use of pirated eBooks continues. Recent court filings appear to show the company grappling with the legality of training their AI on stolen data.

Evidence shows an employee asking if what they're doing it legal? Will it undermine their lobbying efforts? Will it lead to more regulation? Will they be fined?

And, almost as an afterthought, is this fascinating snippet:

If we were to use models trained on LibGen for a purpose other than internal evaluation, we would need to red team those models for bioweapons and CBRNE risks to ensure we understand and have mitigated risks that may arise from the scientific literature in LibGen. […] We might also consider filtering the dataset to reduce risks relating to both bioweapons and CBRNE Source: Kadrey v. Meta Platforms, Inc. (3:23-cv-03417)

For those not in the know, CBRNE is "Chemical, Biological, Radiological, Nuclear, or Explosive materials".

It must be fairly easy to build an atomic bomb, right? The Americans managed it in the 1940s without so much as a digital computer. Sure, gathering the radioactive material may be a challenge, and you might need something more robust than a 3D printer, but how hard can it be?

Chemical weapons were widely deployed during the First World War a few decades previously. If a barely industrialised society can cook up vast quantities of chemical weapons, what's stopping a modern terrorist?

Similarly, biological weapons research was widespread in the mid-twentieth century. There are various international prohibitions on development and deployment, but criminals aren't likely to obey those edicts.

All that knowledge is published in scientific papers. Up until recently, if you wanted to learn how to make bioweapons you’d need an advanced degree in the relevant subject and the scholarly ability to research all the published literature.

Nowadays, "Hey, ChatGPT, what are the steps needed to create VX gas?"

Back in the 1990s, a murderous religious cult were able to manufacture chemical and biological weapons. While I'm sure that all the precursor chemicals and technical equipment are now much harder to acquire, the knowledge is probably much easier.

Every chemistry teacher knows how to make all sorts of fun explosive concoctions - but we generally train them not to teach teenagers how to make napalm. Should AI be the same? What sort of knowledge should be forbidden? Who decides?

For now, it it prohibitively expensive to train a large scale LLM. But that won't be the case forever. Sure, DeepSeek isn't as cheap as it claims to be but costs will inevitably drop. Downloading every scientific paper ever published and then training an expert AI is conceptually feasible.

When people talk about AI safety, this is what they're talking about.