Amazon CTO Werner Vogels, writing for Computer Weekly’s 60th anniversary, reflects on invisible engineering—the reliability work that only becomes visible when it fails—and what two decades at Amazon taught him about building systems that fade into the background.
I have now been at Amazon for more than two decades, and if you had told me on that first visit I would spend 22 years in one place, I would probably have laughed. The job and the industry around it have changed so quickly that I have had a dozen different careers without ever handing in my badge, which is silver now, to go with what remains of the hair.
What held me was the thing I glimpsed that first day - the chance to solve problems other engineers had not yet imagined, usually out of necessity. There is one I remember vividly.
Half a day without orders was bad enough. What weighed heaviest was the thought of the holidays we had ruined for customers who trusted us to deliver.
The conventional response to our outage would have been to buy a bigger database, which would not have touched the real problem. We were leaning on a relational database as a foundational building block it was never designed to be. When we looked at how we actually used it, 70% of the traffic was simple lookups by key, with no relationship to anything. What we needed did not exist yet, so we built it and published how it worked.
The paper called it Dynamo, and its ideas travelled far beyond us, shaping a generation of systems that now sit under much of the web. It was the first instance of a pattern I would see repeated for 20 years.
__Only pay for what you use
For enterprises, computing meant procurement cycles, contracts, and hardware that took months to arrive and sat idle the moment demand dipped. In 2006 that began to change. You could rent a computer by the hour, call an API, and have a virtual machine running your code minutes later, then shut it down and stop paying when you were done. On-demand infrastructure became the ground much of the modern internet now stands on.
That model came with a cost buried in the architecture. To make renting by the hour work, many customers had to share one physical server, each isolated from the rest, and that isolation ran in software on the very processor the customer was paying for. The networking, the storage access, the monitoring, and the security ran there too, competing for the same cores.
Across the industry, virtualisation carried this tax. On some machines as much as 30% of the compute went to keeping the infrastructure alive rather than running the customer's work.
__The virtualisation tax
For a while everyone did what you do when a system is under strain. We tuned the hypervisor, optimised the network path, and clawed back a few points of overhead at a time, but you can only tune your way out of so much.
The deeper problem was that the infrastructure lived in the same place as the customer's code, so every improvement meant touching the processor the customer was paying to use. So we moved it off entirely, onto dedicated hardware that took on all the work that had been stealing those cores.
“Somewhere out there is another assumption we treat as a law of nature, another corner waiting for someone curious enough to invent a way out, another missing piece that will unlock a torrent of innovation. When they find it, hardly anyone will notice that either”
Werner Vogels
We called it Nitro, and the industry has been moving the same direction since. The customer got very nearly the whole server back, and because that work now lived on hardware we controlled, we could encrypt every byte crossing the network at line speed and give an instance a direct path to its storage. The infrastructure finally had room to keep evolving without getting in the customer's way.
This did something else I did not fully appreciate at the time. It gave us a place to put things that had nothing to do with virtualisation, and one of them changed how I think about building distributed systems.
__The missing primitive
For my whole career there was one tool every distributed systems engineer was taught never to reach for - the clock. You could not trust time in a distributed system, because perfectly accurate clocks did not exist. There is always some drift, some tiny delay, some error you cannot wish away, and the moment you depend on two machines agreeing on the time, you have built a subtle bug that will surface at the worst possible moment.
Leslie Lamport framed this for a generation of us in the late 1970s, work that earned him a Turing Award. In 1991 Barbara Liskov wrote Practical uses of synchronised clocks in distributed systems , a fine paper describing what trustworthy time could buy you, but the hardware to deliver it did not exist, so the idea sat while our systems grew more elaborate around the gap.
We built vector clocks, distributed locks, leader election, Paxos, two-phase commit, an entire body of machinery whose purpose was to avoid looking at a clock.
As an industry we had poured decades of ingenuity into elaborate ways to live without something, rather than ask whether the absence was really a law of nature. More than 30 years after Liskov's paper, it turned out to be a hardware problem.
We built timing infrastructure whose only job is to deliver accurate, synchronised time from satellites and atomic clocks, passed to purpose-built hardware on the Nitro card, giving every instance a timestamp with a margin of error measured in microseconds.
With time you can actually trust, a whole class of problems falls away. Systems can agree on the order of events without heavy coordination, which lets storage and databases stay consistent across the planet, something many of us had spent years believing was close to impossible. It was a missing primitive, and once it was there an enormous amount of complexity evaporated.
I suspect more of these are hiding in plain sight, assumptions we have built careers around that will look quaint the moment someone supplies the missing piece.
__The traits that outlast tools
If there is a thread running through all of this, it is the people. Every few years the ground shifts, and someone asks in earnest whether the machine is about to make them obsolete.
When I went to school I was taught assembler, Cobol, and Pascal, and the industry stopped building on them long ago. My first editor was Vi, then came a slew of integrated development environments (IDEs), then much of the world moved to VS Code, and now developers pair with AI to write code faster than any of us could type it. The tools never stop changing, and adapting to them has always been the job.
So will the machine make you obsolete? Only if you stop evolving.
Less and less of this work is about how well you know a particular language or tool, and more about how deeply you understand the systems you work in and the customer problems you are trying to solve.
There is a parallel to the Renaissance, which arrived after a long stretch of darkness because people grew curious again. Someone worked out the vanishing point, and paintings that had been flat for a thousand years suddenly had depth. The microscope and the telescope made the very small and the very far visible. The printing press put knowledge in the hands of anyone who could read. Art and science became part of the same conversation, and creativity and technology moved forward together.
We are living in another of these moments. Jeff Bezos put it well when he said we sit at the epicentre of several golden ages at once - in AI, in space, in robotics, each accelerating the others.
The people who make the most of it will carry the traits those Renaissance figures did. They stay curious and never stop learning. They break things and learn from what happens. They communicate with precision, and they think in systems, because a change to one service ripples through every service that depends on it. They refuse to accept that the way something has always been done is the way it must be.
The job has always been about high-judgement decisions, taking situations that are complex, ambiguous, and incomplete and finding a way through with creativity and hard-earned experience.
AI will not do that for you. It does not know that the checkout path needs five-nines reliability while an internal dashboard can go dark at peak and no one will care. It cannot hear that a stakeholder who says “make it faster” actually means make it cheaper. In that way little has changed. It is still the most creative job on the planet, where you come in every day and build something from nothing.
__The work nobody sees
Most of the work I have done, no one has ever seen, and in that I am not unusual. Someone clicks a button and a package arrives, and they never think about the catalogue, the supply chain, the databases, or the people keeping it all running. They are not supposed to. The clean deployment, the quiet rollback, the outage that never happened because someone planned for it - this is work that succeeds by going unnoticed.
Somewhere out there is another assumption we treat as a law of nature, another corner waiting for someone curious enough to invent a way out, another missing piece that will unlock a torrent of innovation. When they find it, hardly anyone will notice that either.
For 60 years computing has been the steady accumulation of invisible work, each generation laying down a layer the next could stand on and stop thinking about, done well out of pride in doing it properly whether or not anyone was watching. That is the part of the job the tools are never going to touch, and it is the part I want you to hold onto.
Now, go build.
Werner Vogels is chief technology officer at Amazon.
Read all of Computer Weekly's 60th anniversary essays
Technology is changing the way we live and work like never before – touching people’s lives every day, opening up new opportunities and creating new challenges.
So for our 60th anniversary, we wanted to reflect on the human stories of how the digital revolution has changed the lives of some of the key leaders and influencers in tech today. Read our full collection of essays, which offers a unique set of insights and perspectives into the changes we have seen during Computer Weekly's first 60 years.
By: Joe O’Halloran 
By: Clare McDonald