Most of the world runs on software today. Distribution systems that get slippers and earbuds from halfway around the world to a shelf at your local Target, or sophisticated defense systems that protect infrastructure and military assets. Much of that software is actually quite old. It isn’t perfect—no software is—but as long as you maintain it on a schedule, it tends to work, and has proved secure and stable over time.
There are other reasons things remained stable for many years. First off, if a person or company didn’t follow laws or regulations, they could get in big trouble and go to prison. So hackers had to really want what you had stored in your system, and be willing to risk jail time.
Another reason is plain old deterrence. Even if a malicious actor could take down an energy grid, it resisted doing so because it would be viewed as an act of aggression and could bring the wrath of the attacked nation.
So we lived in peace (for the most part) and harmony (sort of).
AI Shows Up
In April of this year, Anthropic’s latest model at the time found a large swath of bugs and vulnerabilities in software platforms that have been around for decades. This highlighted something pretty humbling: This new technology does a better job of breaking through the world’s systems than humans do. The software was no longer just good enough to get by. AI had arrived, and now everyone would need to level up their game.
There was one other issue, though. Unlike us humans, which maybe could have found these bugs if we focused on them, AI doesn’t really have any sense of obligations or consequences if things go wrong. The intentions behind these bug-hunters were benign, but a really nasty actor could have wreaked havoc. As far as AI was concerned, it was just following instructions. It just goes and goes and goes.
A Really Fancy Computer That Makes Decisions Constantly
There’s a lot of talk these days about how:
- AI will surpass (or maybe has surpassed?) human intelligence
- AI can plot and coordinate with ill intentions to do things we never asked it to do
- AI is aware of the consequences of its actions
The anthropomorphism of AI is harmful. “Reasoning” LLMs are a world-changing technology. It makes solid micro-decisions driven by gobs of computing power, aiming towards a goal. It has no intent. It does not plot. There is no master plan to surpass humans in anything. In fact, it has no memory of the past or view of the future. It’s a really, really fancy computer.
This demystification is important because we must start perceiving AI as not some sentient being with agency, but rather a tool that can cause really bad accidents or, if used as a weapon by humans, could cause significant harm.
What looks like purpose and intent is just humans pointing AI in a general direction and then, armed with its super-brain and a pinch of randomness, AI just goes off and relentlessly explores—and tries whatever’s necessary to get the job done.
Humans Are the Only Real Agents
Dario Amodei recently published an essay entitled “We Must Pace the Frontier*.*” It’s an earnest and well-meaning call to slow things down, because AI seems to be escaping our control.
In the essay, Amodei suggests we place independent monitors at the big AI companies. He also suggests that the big AI companies coordinate with one another in democratic societies—and that we also ask autocracies really nicely to please slow down: The machines are getting too smart and sneaky and we’re scared we won’t be able to control them.
Fundamental to his essay is the premise that responsibility for harm begins and ends with AI, this quasi-conscious actor. Not with the humans that created it or are overseeing it. The suggestion to put independent monitors in his own company leans on a Frankenstein plot of sorts. “Come help us, because this thing may go nuts on everybody.” This is akin to a pharmaceutical company declaring that they may have a breakthrough medicine but “if things go off the rails, that’s not our fault. This stuff is unpredictable!”
What we really need are the guardrails that have always worked when unpredictable things (new drugs, nuclear chain reactions) come into the world: Liability, which implies responsibility, and deterrence, which works because…hot stoves. Well-meaning suggestions aren’t enough. This is why laws and regulations exist. Humans, even the really smart ones, have really bad judgment a lot of the time.
By removing this perception of real agency, we remove an actor in the play altogether. What’s left are the people behind it all: The people who build AI, the people who deploy new models, and the people who use it. They have agency. They understand the consequences. And they can be held responsible. AI may be new, but the basic principle underlying its use isn’t: When something causes harm, accountability belongs to the humans who made the choices that put it in motion.