I had heard about the ouster of Sam Altman at OpenAI for a brief blip but didn’t pay much mind to it and was not really all that concerned either way. Going into this book I had essentially no prior knowledge of the company. The book gave me a much deeper understanding of the sorts of opinions I had heard about OpenAI which I had no reason to pay much heed to before, and it forced me to update on several things I had been dismissive about. But the book’s organizational metaphor — the empires framing that the title rests on — and taking that metaphor seriously generates better questions than what I think the book itself asks.

The book’s best work is at the object level. Hao’s argument that AI was not inevitable hit home with me, and her case for it is strong: the founders of OpenAI are next to one of a kind, and nowhere else can you find people willing to put that much capital up on a project that is so speculative and has no way of giving a return at all. I had always thought the water consumption story was drastically overstated, more of a political issue than a realizable one, and I had naively assumed data centers used closed loop cooling systems. They often don’t, and the picture Hao paints of companies effectively controlling the water market in places where the people are so poor in relation to the companies is much clearer than anything I had encountered before. The RLHF chapter was the most eye opening for me. How is an image generator or an LLM supposed to know what is crossing the line, and how do you correct that behavior? RLHF is one obvious way to curate answers without being able to explicitly state the requirements in words. To be one of the people whose job it was to train these models and have to look at horrific images or read heinous texts for hours and hours on end is something akin to hell. Humans have had to endure viewing horrific scenes in areas like law enforcement, military war times, disaster relief and many others. What makes this different is the volume, which is near incomparable to any of the aforementioned jobs, and the fact that workers push themselves to work as many hours as possible to make that little bit more because the jobs could stop at any time.

Hao’s account of what she calls the divorce — Dario and Daniela Amodei leaving OpenAI in late 2020 with seven to ten other senior researchers to found Anthropic — is a main point of the book. Their stated reason was that the commercialization pace at OpenAI was incompatible with what they thought safety first research required, and the deeper structural issue was that even though they had strong views on safety they had no real leverage to slow deployment decisions that Altman and Brockman wanted to push. The way they rationalized leaving rather than trying to change things from the inside is what Anthropic has publicly called the race to the top: frontier AI is going to get built by somebody, and if safety focused researchers are permanently behind the frontier then they can critique but they cannot actually shape how the frontier gets built. The move is to build a lab that stays at the frontier while prioritizing safety, publishing responsible scaling policies and doing interpretability work, rather than trying to slow the race from the outside. The counter argument people in the AI safety community make is that a race to the top can very easily become a race to the bottom wearing a mission statement. Every frontier lab says some version of this to justify scaling, and if you believe catastrophic AI risk is real then adding another well funded competitor to the race accelerates timelines, you have just multiplied the number of players who need to not mess up. Anthropic employees also earn equity that vests over years of scaling, a financial incentive structure basically identical to OpenAI’s regardless of what the stated mission is. The part that felt reckless to me was the gutting of OpenAI’s safety team after the divorce. The Amodeis’ line on this is that the safety team was never structurally empowered to stop anything anyway, so leaving did not functionally change OpenAI’s safety trajectory, it just made explicit what was already the case. Whether that is genuine analysis or post hoc rationalization is up to the reader. What makes this more interesting to me is that Jan Leike, who was running the superalignment team at OpenAI, resigned publicly in May 2024 and said outright that safety culture had taken a backseat to shiny products. That is some evidence for the Amodeis’ 2020 read that the safety team who stayed behind did keep losing — whether that vindicates the divorce or just shows that leaving did not actually help is another thing the reader has to decide.

The organizing idea of the book is the framing of OpenAI and the other frontier labs as empires, and I want to take that metaphor more seriously than the book itself does by pulling in Harari’s definition from Sapiens. Harari’s account is fairly specific. He argues that an empire is a political order that rules over a significant number of distinct peoples with different cultural identities, has flexible borders and a potentially unlimited appetite for absorbing more territory and more peoples without changing its essential character, and typically spreads a unifying ideology or culture across the populations it rules. He is also deliberately provocative in claiming that empires, despite being coercive and extractive, are often responsible for much of what we call civilization, and that the aggregate welfare of the ruled populations often improves under empire even when fine grained measures of liberty and consent look terrible. The actually interesting question is whether the AI empire metaphor holds up against that definition. Some of it fits well. The unifying ideology point fits very cleanly since an AI company distributing the same model across close to 180 countries flattens cultural and linguistic diversity in a way that Harari would immediately recognize as empire behavior. The unlimited appetite point fits cleanly along the capability axis — OpenAI’s internal scaling laws, the claim that another 1000x run gets you something qualitatively new, are basically unlimited appetite written as an engineering roadmap. It fits less cleanly on the organizational side: Sam explicitly resisted growing headcount because he thought the company culture would be lost if they added too many people, which cuts against Harari’s claim that empires absorb more peoples without changing their essential character. Where the metaphor strains more seriously is that Harari’s empires claim formal political authority and are acknowledged as rulers by the ruled, whereas the AI empires Hao is describing operate through market penetration and infrastructural dependence without ever claiming sovereignty over anyone. You could argue that informality is a feature of a stronger metaphor and that these are empires without the accountability of being called empires, or you could argue that it is a sign that the metaphor is mainly rhetorical rather than analytical. I lean toward the first reading, since the lack of a formal claim to authority does not really reduce the actual leverage these companies have over the populations they reach.

The part where I think Hao’s framing is weakest is the one that Harari would probably push on hardest. His read of empires is that they are often net positive for aggregate human welfare despite being coercive and extractive, and he is willing to sit with that uncomfortable possibility. Hao does not really grapple with the parallel question of whether tech empires might turn out the same way, bad by fine grained standards but good by aggregate civilizational standards. She sort of forecloses that question by treating extraction as self evidently wrong. A Harari style reading would at least ask, if AI empires end up accelerating scientific research, lowering the cost of education and healthcare, and raising aggregate welfare by large amounts, does that change the moral calculus of what they did to get there. The obvious pushback is that empires have historically sacrificed the people doing the actual work — workers whose daily conditions were often worse than their parents’ — and called it progress, which is exactly what the RLHF chapters describe happening now. That cuts against Harari’s framing as much as Hao’s: aggregate welfare is doing a lot of work in a sentence that includes the people whose lives were traded for it. I do not know the answer to that question and I am not sure Hao does either, but the version of the review that uses her metaphor against her by taking it more seriously than she does is more interesting than the version where I just agree with her.

After reading this book OpenAI to me has lived so long to see themselves become the thing that they swore to destroy. They seem to be caught up in profit seeking and have lost their roots of creating a safe environment for what will be humanity’s last invention for better or worse.