The claim that AI is too dumb to be dangerous misses the point. Risk is not just intelligence. It is capability times autonomy times access times scale, amplified by opacity and convergence.
A common argument in the current AI debate goes something like this:
AI is still too dumb to be genuinely dangerous.
Today's models hallucinate. They make elementary mistakes. They fail at tasks humans find trivial. They need supervision. And if someone is foolish enough to hand an unreliable machine control over an important decision, perhaps the problem is not AI but the person who trusted it.
There is some truth in this argument.
Current AI systems are far from universally intelligent, consistently reliable, or independently capable. Many claims about imminent artificial superintelligence are speculative.
But I think this framing misses the more important question.
A system does not need to be superintelligent to become dangerous.
Because risk is not simply a function of intelligence.
A more useful mental model is:
Risk ≈ Capability × Autonomy × Access × Scale
And that risk can be further amplified by opacity and by connecting AI to domains where actions have physical, economic, biological, or societal consequences.
This is not intended as a literal mathematical equation. It is a framework for thinking about how AI risk actually emerges.
The first variable is the closest to what we normally call intelligence.
Can a model reason? Can it write software? Can it analyze scientific literature? Can it plan across multiple steps? Can it recognize patterns that humans struggle to identify? Can it design something new?
But "intelligence" is probably too vague a word for this discussion. Capability is more useful because a system does not need general human-level intelligence to become extremely powerful in a narrow domain.
A machine that is mediocre at conversation but exceptionally good at finding software vulnerabilities may matter enormously in cybersecurity.
A model that cannot understand basic social context but can search millions of molecular structures may matter enormously in biotechnology.
The relevant question is therefore not:
Is AI intelligent?
It is:
What capabilities has this system acquired, and what can those capabilities be used for?
This distinction is important enough that frontier AI safety frameworks increasingly evaluate particular dangerous capabilities rather than attempting to assign some universal intelligence score. Google DeepMind's Frontier Safety Framework, for example, explicitly considers critical capability thresholds in areas including autonomy, cybersecurity and biosecurity.
Capability alone is limited if every action requires a human to evaluate it.
Autonomy changes that equation.
There is a fundamental difference between an AI that tells you "here is what I think you should do" and an AI that says "I have already done it."
An ordinary chatbot produces information.
An autonomous agent can potentially read information, make decisions, use software, call APIs, modify files, communicate with other systems, observe the results of its actions and decide what to do next.
Every additional step that occurs without human intervention reduces the number of opportunities humans have to detect an error.
This is why the 2026 International AI Safety Report treats autonomous agents as a distinct reliability concern. It notes that agent failures may have greater consequences precisely because agents can take actions in the world with fewer opportunities for human intervention, particularly as they gain access to external tools.
The important transition, therefore, may not be from "dumb AI" to "smart AI."
It may be from AI that answers to AI that acts.
Now imagine two identical AI models.
The first can only answer questions in a browser window.
The second has access to your email, source code, cloud infrastructure, company bank accounts, customer database and internal APIs.
They have exactly the same intelligence. They do not have the same risk profile.
Access converts cognitive capability into operational capability.
An unreliable AI that has no external access may produce a bad answer.
An unreliable AI with production credentials can produce a bad action.
This is why I do not think the problem can be reduced to humans being "dumb enough to trust AI."
Trust is only part of the problem. Architecture matters. Permissions matter. Delegation matters. Infrastructure matters.
The question is not merely whether a human believes an AI's recommendation. The question is whether we are building environments in which AI systems can convert recommendations into consequences.
Humans make mistakes constantly. But humans have biological limitations.
We can only write so many emails, make so many financial decisions, analyze so many systems or interact with so many people in a day.
Software has no equivalent limitation.
This makes scale one of the most underestimated variables in AI risk.
Suppose an AI system has only a 0.1% probability of making a serious mistake in a particular workflow.
At ten decisions, perhaps nothing happens. At a thousand decisions, the risk becomes more relevant. At a billion automated decisions, even rare failure modes may become operationally significant.
Scale also applies to malicious use.
The important question is not merely whether AI allows someone to do something that was previously impossible. Sometimes the transformation comes from making something that was previously expensive, slow or expertise-intensive dramatically cheaper and repeatable.
A mediocre capability multiplied by enormous scale can matter more than exceptional intelligence operating once.
There is another dimension that makes all four variables harder to manage: opacity.
This point needs to be stated carefully.
We do know how neural networks perform their computations. We designed the architectures. We understand matrix multiplication, attention mechanisms, optimization, token prediction and the training process.
What we often do not possess is a reliable, human-understandable explanation of the internal representations and strategies that lead a sufficiently complex model to a particular behavior.
Anthropic describes this problem directly in its interpretability research: large language models learn internal strategies during training that are encoded across enormous numbers of computations, and developers still do not understand how models perform many of the things they can do.
That distinction matters.
The mystery is not how the computer executes the neural network. The mystery is increasingly about what computation the learned network has discovered.
This creates an unusual engineering situation.
We are improving the capabilities of systems faster than we are improving our ability to explain their internal mechanisms.
That should not automatically be interpreted as catastrophic. But it should certainly make us cautious.
This is the part of the AI debate that concerns me most.
AI is not developing in isolation. It is converging with other technologies.
Consider robotics. Large language models are increasingly being connected to robotic systems capable of planning and operating in complex physical environments. Research published in Nature Machine Intelligence has already demonstrated embodied language-model systems performing complex robotic tasks in unpredictable environments.
Now consider biology. AI-based protein design has progressed from prediction toward generative design, allowing researchers to explore biological structures and functions computationally at increasing speed. In 2025, researchers reported AI-designed CRISPR gene editors capable of performing programmable editing in human cells.
None of this means AI is about to independently engineer organisms or unleash some science-fiction scenario. That conclusion would go far beyond the evidence.
But something important is happening.
AI is gradually moving from a system that primarily manipulates information toward a general-purpose layer capable of interacting with software, machines and biological design processes.
That transition matters enormously.
The relevant risk may therefore emerge not from one spectacular breakthrough called AGI, but from the coupling of several technologies that individually appear manageable.
AI provides cognition. Agents provide autonomy. APIs provide access. Cloud computing provides scale. Robotics provides physical agency. Biotechnology provides access to biological systems.
Each layer changes what the other layers can do.
This is why I am skeptical when AI risk is reduced to a debate about whether today's models are "smart" or "dumb."
Complex systems often become qualitatively different when components become connected.
The Internet was not revolutionary because any individual computer suddenly became infinitely more intelligent. Its importance came from connecting computers, people, information and infrastructure at enormous scale.
AI may follow a similar pattern.
We may never wake up on a particular Tuesday and discover that AI suddenly crossed a clearly measurable line called "superintelligence."
Instead, we may gradually connect increasingly capable but still imperfect systems to increasingly consequential parts of the world.
At some point, capability, autonomy, access and scale may cross a threshold where our ability to understand, supervise and contain these systems becomes the limiting factor.
And because deep learning systems remain partially opaque, we may not completely understand that threshold before crossing it.
None of this proves that AI will destroy humanity. Nor does it prove that today's AI systems are close to doing so.
Predictions at that level remain deeply uncertain, and experts disagree substantially about extreme loss-of-control scenarios. The 2026 International AI Safety Report explicitly reflects that disagreement rather than treating catastrophic outcomes as established facts.
But uncertainty works in both directions.
We should be skeptical of unsupported predictions of catastrophe. We should be equally skeptical of the argument that AI cannot be dangerous simply because today's systems still look stupid in many situations.
A system does not need consciousness. It does not need human-level intelligence. It does not need evil intentions. And it does not even need to be consistently correct.
It needs sufficient capability, sufficient autonomy, sufficient access and sufficient scale.
That is a very different threshold.
And unlike superintelligence, we are already building every component of it.
Because risk is not purely a function of intelligence. A system can be unreliable and narrow yet still cause serious harm when it has enough capability in a specific domain combined with autonomy, access and scale. Danger emerges from how AI is embedded in consequential systems, not just from how smart it is.
It is a mental model, not a literal equation, for how AI risk emerges. Capability is what a system can do, autonomy is whether it can act without human checkpoints, access is what resources and permissions it can touch, and scale is how many decisions it makes. Risk is further amplified by opacity and by coupling AI to physical, economic and biological domains.
Every action an AI takes without human intervention removes an opportunity to catch an error. Autonomous agents can read data, make decisions, call APIs, modify files and act on results, so their failures can have larger consequences. The key transition is from AI that answers to AI that acts.
Unlike humans, software has no biological limit on how many decisions it makes. A 0.1 percent failure rate is negligible over ten decisions but operationally significant across a billion automated decisions. Scale also makes malicious use cheaper and repeatable, so a mediocre capability applied at massive scale can outweigh exceptional intelligence used once.
We understand how neural networks compute at the mechanical level, but we often lack a reliable human-understandable explanation of the internal strategies a complex model learns. Capabilities are improving faster than our ability to explain internal mechanisms, which is a reason for caution rather than automatic catastrophe.
AI is combining with agents, APIs, cloud computing, robotics and biotechnology. Each layer expands what the others can do, so the dangerous threshold may be a systems threshold reached gradually through connection, similar to how the Internet became transformative by linking systems rather than by any one computer becoming smarter.