---
title: "Possible Futures"
slug: possible-futures
url: https://listedarticles.com/articles/possible-futures
canonical_url: https://borretti.me/article/possible-futures
content_type: essay
language: en
published_at: 2026-11-11T13:00:00.000Z
updated_at: 2026-11-14T22:53:34.226Z
author: "Fernando Borretti"
authored_by: human
publisher: "Fernando Borretti"
publisher_url: https://borretti.me/
topics: ["ai", "opinion", "ai-safety", "economics"]
license: all-rights-reserved
word_count: 1555
reading_minutes: 7
citation: "Fernando Borretti, Fernando Borretti. \"Possible Futures.\" 11 Nov 2026. https://borretti.me/article/possible-futures (all-rights-reserved)"
# The full text follows. The web page shows an extract and sends readers
# to the source above; quote the citation and link the canonical URL.
---

# Possible Futures

> Fernando Borretti sketches a map of possible futures, laying out the different ways the arrival of advanced AI could play out for society and individuals.

# Possible Futures

Fernando Borretti · November 12, 2026

# Contents

Click on the blue boxes to go to the corresponding section:

# Questions

- **Do we reach ASI?** Do we reach the point where AI is massively better than
  humans at almost all cognitive tasks, or does the intelligence explosion
  fizzle out, and produce minds that have very jagged capabilities, and are
  worse than humans at some things?
- **Do we solve alignment?** Can we instill our own values into the AI, or does
  it have its own evolved, ahuman value system?
- **What kind of alignment?** Intent alignment means an AI that reliably follows
  the intent of its operator (that is: only takes actions the operator would
  approve of), value alignment means aligning to an impersonal system of values.
- **Takeoff severity?** If the AI is unaligned, how fatal is that for humans? Do
  we get an out-of-control paperclip maximizer, or a more benign AI that is
  unaligned but which cares enough to stay out of our way?
- **How many ASIs?** A [singleton](https://en.wikipedia.org/wiki/Singleton_(global_governance)), a pantheon, or an ecology?
- **Full automation?** In the no-ASI case, do we still automate almost all jobs
  with non-superintelligent AI?

# Misuse Doom

In this world, an AI ends the world by following the orders of its operator. The
AI might be strongly intent-aligned, so that it does whatever the operator asks;
or it might be an open-source model with refusals ablated away; or it might be a
commercial AI where safeguards are circumvented by clever task decomposition.

This could probably happen today: a pandemic could cause human extinction, and
AI is capable enough to assist in biology research. The obvious example of
biorisk is a terrorist group using AI to make a bioweapon. But you could get to
this outcome without anyone even trying to make a bioweapon. A
government-funded, completely official and legitimate laboratory doing
gain-of-function research on some pathogen could accidentally start a
pandemic. In this scenario, the AI’s role in the causal chain would be:

- Compressing timelines by speeding up literature review, hypothesis selection,
  experiment design, data analysis, equipment scheduling.
- Increasing the doom surface area by doing many experiments in parallel.

So it’s not the classic Terminator scenario, but rather a case of AI making it
easier to find “[recipes for ruin](https://michaelnotebook.com/xrisk/)”.

# Butlerian Jihad

> JIHAD, BUTLERIAN: (*see also* Great Revolt)—the crusade against computers,
> thinking machines, and conscious robots begun in 201 B.G. and concluded in 108
> B.G. Its chief commandment remains in the O.C. Bible as “Thou shalt not make a
> machine in the likeness of a human mind.”
>
> — Frank Herbert, [*Dune*](https://en.wikipedia.org/wiki/Dune_(novel))

At the extreme end this is the actual Butlerian Jihad from *Dune* (named,
tangentially, after [Samuel Butler](https://en.wikipedia.org/wiki/Samuel_Butler_(novelist)), who in *1863* wrote [the first AI
doomer article](https://web.archive.org/web/20060524131242/http://www.nzetc.org/tm/scholarly/tei-ButFir-t1-g1-t1-g1-t4-body.html)).

Otherwise, this is something like an international agreement to stop the
development of superintelligent AI ([ControlAI](https://controlai.org/) is advocating for this and
you should support them). Maybe this fails, and we go back to the start, or it
succeeds and we can get on [the other road](https://borretti.me/article/the-other-road).

# Enduring Complementarity

This is the mundane future. It turns out that the “G” in AGI was never
real. Every mind is its own *sui generis* thing. Jaggedness remains. AIs remain
a complement to, rather than a substitute for, human labour. There is job
displacement but no mass unemployment, people move up to more
managerial/relational roles. There are fewer ICs and everyone is some kind of
manager. Labour becomes a smaller factor of production and consequently loses
some bargaining power against capital, but most people are fine.

# Permanent Underclass

In this future, we don’t reach ASI, but non-superintelligent AI is enough to
automate almost all jobs. The labour term in the production function goes to
zero. All income is either capital ownership or welfare.

Economically: there may be material abundance, but absent some kind of [Georgist
economic reform](https://en.wikipedia.org/wiki/Georgism), most of the gains are captured by
rent-seeking. Socially: there is no more social mobility, your lot in life is
determined by how much capital you had when the music stopped. Growing your
human capital is no longer incentivized, since it doesn’t change your life
outcomes.

See also:

- [*A rising tide raises all rents*](https://jyn.dev/rising-tide-raises-rents/) by [jyn](https://jyn.dev/).
- [*No-One Escapes the Permanent Underclass*](https://borretti.me/article/no-one-escapes-the-permanent-underclass).

# The Singleton

> I am the Eschaton. I am not your god.  
> I am descended from you, and I exist in your future.  
> Thou shalt not violate causality within my historic light cone. Or else.
>
> — Charles Stross, [*Singularity Sky*](https://en.wikipedia.org/wiki/Singularity_Sky)

In this world, we build ASI, and it’s successfully aligned to some impersonal
system of values. First order of business is to prevent someone else from
building an ASI that might challenge the first one, so the AI performs a
“[pivotal act](https://www.lesswrong.com/w/pivotal-act)”, which is a LessWrong euphemism for “put everyone under
constant surveillance, destroy all GPUs”. The result is a [singleton](https://en.wikipedia.org/wiki/Singleton_(global_governance)): a
single mind that is individually sovereign over everyone else.

What happens then? It depends on the content of the AI’s value system, and the
stability of that value system over time. The singleton might be a “[night
watchman](https://www.forethought.org/research/nightwatchmen)”, whose only goal is to prevent the creation of paperclip
maximizers (i.e. prevent [Classic Doom](#doom)), or it might be deeply
interventionist. The singleton’s values might be incompatible with human life:
maybe its creators made it utilitarian and species-egalitarian, and it notices
that an 80kg human generates less utility than 80kg of [*Dactylopius
coccus*](https://en.wikipedia.org/wiki/Cochineal), so we get pulped and the universe becomes a terrarium for insects
instantiating the [rebugnant conclusion](https://benthams.substack.com/p/insects-matter-more-than-people).

The future is a very long time: the stelliferous era will last at least 100
trillion years. Maybe the singleton is aligned at first, but over millions or
billions of years of growth and self-modification, the alignment drifts (the
predictable LessWrong objection here would be: “*obviously* an ASI would be
smart enough to prevent this”, because ASI is magic).

# Classic Doom

You know, the paperclip maximizer. “[The AI does not hate you, nor does it love
you, but you are made out of atoms which it can use for something
else](https://intelligence.org/files/AIPosNegFactor.pdf)”. This has been done to death so I won’t bore you.

# Orion’s Arm

> Another of the sayings of Chrysippus is: We should infer in the case of a
> beautiful dwelling-place that it was built for its owners and not for mice; we
> ought, therefore, in the same way to regard the universe as the dwelling-place
> of the gods.
>
> — Cicero, [*De Natura Deorum*](https://en.wikipedia.org/wiki/De_Natura_Deorum)

We build ASI, and it has its own goals and values. Luckily, it’s neither
actively hostile to humans, nor so indifferent that it destroys us without even
noticing. It mostly leaves us alone, and goes off to another star to do its own
thing. The result might be something like the galactic ecology in [Orion’s
Arm](https://orionsarm.com/), where humans and superintelligent AIs and everything in between have a
kind of stable co-existence by virtue of occupying different volumes of space.

# The World State

> … [The] human race is best ordered when in all its movements and motors it
> is controlled by one Prince as by one mover, by one law as by one motion.
>
> — Dante, [*De Monarchia*](https://en.wikipedia.org/wiki/Monarchia)

In this world, a reliably intent-aligned ASI takes over the world. What happens
then depends on the character of its operator, who now wields the [ring of
Gyges](https://en.wikipedia.org/wiki/Ring_of_Gyges). The difference between this and [The Singleton](#singleton) is
that in this future, the singleton is the human operator mediated by an ASI,
rather than an ASI deriving actions from an impersonal system of values.

# MAD World

> As Nietzsche said, “God is dead”. The gods die but the Titans gain
> power. Technology is just the clothes, the armour, of the Titans.
>
> — Ernst Jünger

In this world, the [intent alignment](https://ai-alignment.com/clarifying-ai-alignment-cec47cd69dd6) problem is solved, and we can build an
ASI that reliably obeys the intent of its operator. Furthermore,
superintelligence is widely distributed rather than centralized: everyone has
their own personal intent-aligned ASI. In this world, everyone is 10,000x more
capable and maybe you can build WMDs with a prompt. What keeps the peace
is mutually assured destruction. It’s not clear how survivable this is.

The obvious failure mode is that one bad actor causes human extinction. Maybe
their ASI can’t overpower all the other ASIs put together, *but* it can do
enough damage to destroy human life before the other ASIs can act (a general
observation: the technology to make the Earth uninhabitable to humans is much
lower on the tech tree than the technology to build a distributed,
extinction-proof interstellar civilization).

The generalized ($n>1$ rather than $n=1$) version of this is that all conflicts
are scaled up: two humans having a feud is the equivalent of nuclear war, two
countries going to war is something much greater.

On the more positive note: if there’s no way to stop AI, then “everyone should
have it” is obviously preferable to “only the Elect should have it”, which is
intolerable. If the goal is to keep the peace, the threat of mutually-assured
destruction is better than totalitarianism, in the sense that there’s more
dignity in it.

# Power Concentration

A smaller version of [MAD World](#mad), where intent-aligned ASI is less
distributed. A small number of actors have ASI, and everyone else just watches
what happens. Maybe the AI companies will divide the light cone among themselves,
like in the [Treaty of Tordesillas](https://en.wikipedia.org/wiki/Treaty_of_Tordesillas): every star and galaxy between 0h and
12h of [right ascension](https://en.wikipedia.org/wiki/Right_ascension) goes to Anthropic, analogously for OpenAI.
