Photo by Dan Counsell on Unsplash
Thirty minutes into a recent review meeting at Amazon, an L7 principal product manager began a discussion on a six-page document. Two minutes into his presentation, it was blindingly obvious to everybody in the room that he was reading the document for the first time.
I know this happened, because someone in my community was an L6 in that room.
But because the presenter was the most senior person there, nobody was brave enough to to speak up .
I spent almost twenty years inside Amazon’s writing culture, and in my opinion, that culture was the company’s secret sauce. Jeff Bezos explained it this way:
Full sentences are harder to write. They have verbs. The paragraphs have topic sentences. There is no way to write a six-page, narratively structured memo and not have clear thinking.
That last sentence is no longer true. With AI, you can trivially produce six pages, no critical thinking required. But that writing process, as described (and prescribed) by Bezos, used to guarantee that the author of a memo would have done the necessary thinking before asking for an hour of everybody’s time
That guarantee has gone.
The Effect on Amazon
When I joined Amazon in 2006, I did not appreciate that its writing culture was a competitive advantage. I just thought it was a quirk.
Two years before I started, Jeff Bezos had banned the use of PowerPoint in meetings with his senior team. His reasoning was that the nature of PowerPoint presentations made it easy for their recipients to miss important ideas and information. It may have been easier for a presenter to create slides, but the greater value for an audience came from a well-written document.
So meetings at Amazon were unconventional to outsiders. In a one-hour meeting, the first thirty minutes were “study hall.” If you were presenting, you brought printouts of your six-pager, and everyone read it in silence. The only sounds came from people writing notes in the margins with their red pens. Thus, when the discussion started, everybody began from the same place.
Such a meeting simply could not take place until your ideas had been written down coherently enough to survive thirty minutes of scrutiny.
AI;DR
Amazon’s writing culture survived the pandemic. The printouts may have moved online, and study hall may have transformed into a shared doc with comments down the margin, but that was fine. People still wrote the documents, and people still read them.
But it did not survive the advent of AI.
From what I hear, the moment a model could produce something that passed as a human-written six-pager, they stopped being written by humans. A senior engineer in my community told me that he has started tagging documents as “AI;DR.” AI; didn’t read. In his view, “If a model wrote it, a model can summarize it.” He had better things to do with his time.
The L7 PM from above may have produced a six-pager, but because he had skipped the critical thinking step, he didn’t have a credible plan. I don’t know how the situation ended, but there are only two ways it could have gone. Either the plan got reworked, and a room full of senior engineers lost time to a meeting that should never have been scheduled, or it didn’t get reworked, and that team executed a plan nobody had thought through.
Both of those outcomes would be terrible, and each would be the result of a lack of respect for one’s coworkers on the part of the memo’s “author.”
Performance reviews are probably the worst example. The process used to require several paragraphs about each person you worked with. That turned out to be too big a burden at Amazon’s size, and some time before the pandemic, it was cut down to roughly two tweets per person: their superpower, and their areas for improvement.
The reports from my community are that the entire process is now just AI-driven. People lean on AI to write those two tweets, and managers lean on AI to collect them and create a summary. A year of someone’s work gets summarized by one LLM, rolled up by another, and the human in the loop is simply copy-and-pasting the output.
How absurd it is that the mechanism that was meant to be the primary driver of career growth conversations has devolved to managers being meat proxies for a conversation between two prompts.
It would be more efficient, and more honest, to skip the middle step and simply have everybody share their prompts.
If you’re having problems growing your tech career, Top Tier may be right for you. The premise is simple: work closely with a team of coaches who are a step or three ahead of you, who have seen it all before, and know how to get you unstuck. You’ll also be alongside hundreds of people working through the same problems you are. Nothing is going to change if nothing changes. If you’re serious about getting an exceeds rating or your next promotion, take a look.
What’s at stake
The obvious cost is bad decision-making. For twenty years, Amazon’s writing culture was a filter for half-formed ideas. The filter, it seems, has gone. Haphazard decisions will compound over time, leading to the decline of the business.
But the deeper cost is the trust between colleagues. A document used to be proof of thought. If you handed me six pages, I knew that you had spent significant time with the problem. Therefore, I knew that it would be worthwhile spending my time to understand your thoughts.
These aren’t arguments for going back to a pre-AI era. That ship has sailed. AI is fundamentally changing the workplace, and for better or worse, you cannot simply prohibit its use. No company is going to ban it completely.
But most of what gets written at a large company does not need a human author. A model can write a status update or a set of meeting notes better than any person. The real question is how we can do it in a way that doesn’t abdicate critical thinking for critical decisions.
The teams that stopped writing completely
Some teams, at Amazon and elsewhere, no longer do code reviews. They have extensive automated pipelines, end-to-end tests, automated rollbacks, effective monitoring, and loud alarms. They don’t write design docs for their changes. And of course, they don’t write the code, either.
I think this is perfectly fine and where teams should be headed.
I led several initiatives to get teams onto continuous deployment and off manual pipelines. If a bad commit rolls itself back in four minutes and doesn’t page anybody, that’s an ideal to aspire to.
Bezos explained the rationale for this way of working in his 2016 annual shareholder letter. He drew a line between one-way doors, the decisions you can’t walk back easily, and two-way doors, the ones you can.
One-way doors deserve slow, deliberate, clear thought.
Two-way door decisions should be made fast, because for them, the cost of being wrong is small.
Most day-to-day engineering decisions are two-way doors. The teams above did what engineering needed to be done to earn the right to not have to write anything.
Therefore, I propose the following simple rule: Use AI liberally wherever you have engineered a two-way door. Let it write the code, bypass the code review, and ship it. You are allowed to trust the rollback you’ve built.
But keep the full ritual of a human writing and a human reading for every decision that is a one-way door. If you don’t know whether the decision in front of you is easily reversible or not, then think about it.
What the rules should be
This summer, Sophie Alpert, an engineering lead at Clay noticed the same AI-induced dumbing down of writing that I described earlier in this article. Rather than banning the use of AI for the creation of documents, she wrote a policy for the engineering team to follow to ensure they communicated well and respected the people who read their documents. The rest of the company adopted it, and when a co-founder posted the policy on LinkedIn it went viral, which tells you how many companies have this problem.
Here are the salient points:
- You must stand behind every sentence. “Oh, AI wrote that, ignore it” is never an acceptable answer to a reviewer’s question.
- Writing is thinking. A spec or a retrospective is proof that someone has thought about the problem. If you skip the writing then you lose that proof.
- Spend more time writing a document than your readers will spend reading it. One author; many readers. The time you take to ensure you will be understood will have a multiplying benefit..
- Longer is not better. If you have generated a long document from a short prompt, share the prompt instead.
- Quoting generated text is fine as long as you label it as such.
Amazon should write its own version of this policy. If I were to write it, the policy would have at least three rules:
- Two-way doors can be AI. One-way doors require humans. If an issue can sort itself out or the stakes are low, let the machine do the writing, or don’t write anything at all. Otherwise, a human writes it, and a human reads it and gives feedback.
- You own every word. Generated or typed, if it is in your document, you are accountable for it. If you present a document where it’s clear that the AI did the thinking, the people in that room should feel free to tell your manager, and your manager will treat it as seriously as they treat any other failure to do the job. There must be consequences.
- Certain documents must always be human-written. Some documents only have value if a human has written them. The annual shareholder letters, planning documents (OP1 and OP2), PR-FAQs, any proposal for a new initiative, major rearchitectures, performance reviews, and promotion documents all belong on that list.
AI should definitely be used for brainstorming, organizing your thoughts, research, making sure your documents are consistent, and grammar. But, as we would say at Amazon, the human should always take the pen.
I’d like to be wrong about this
I left Amazon in 2024 to make content full time, so everything after that date is secondhand and filtered through former colleagues, and the people in my community who are still inside. I’m also still a shareholder. I want Amazon to prosper.
If you’re at Amazon, I’d like to hear from you. I’m making a YouTube video on this topic. If I’ve mischaracterized something, or if you have a story that paints an even worse picture than the one above, please let me know in the comments.
I read every one myself, which this week, feels like a point I need to make..