NIST Mathematical Proof Supports Transition to a Continuous-Monitor-and-Update Security Model for AI Systems

The proof provides a rigorous explanation of the importance of transitioning from a “one and done” security model.

Can we make artificial intelligence impervious to adversaries who want to twist the technology to nefarious ends? Though AI is among the newest of technologies, the question’s answer is nearly a century old. 

Try as we might, we can never render AI completely unassailable using conventional security models. In the peer-reviewed journal IEEE Security and Privacy, Apostol Vassilev, a senior scientist at the National Institute of Standards and Technology (NIST), has published a mathematical proof of this statement building on work published in 1931 by famed logician Kurt Gödel. His incompleteness theorems showed that there are limits to what can be proved within a system built on a finite number of rules. 

The guardrails that govern an AI’s behavior are just such a system, and one of the proof’s implications is that there will always be a way to prompt an AI system to disregard its rules — it’s just a matter of finding it.

“One of the pillars of responsible AI is that you want the technology to be secure,” said Vassilev, the proof’s author and an expert in adversarial machine learning. “You want it to withstand adversarial attacks and perform only what you want it to do, not what an attacker might want. What this proof shows is that there is no finite set of guardrails that is universally robust against adversarial prompts.”