Empire of AI

I’ve spent this year reading as much as I can on AI and LLMs in particular, which means several books on the topic, innumerable online essays, posts, videos, and so on. This book is the single best resource I’ve found if you want to understand how LLMs like ChatGPT work and, more ominously, what the culture and people behind them are like.

Hao is an MIT-trained engineer turned journalist, and her technical fluency as well as her relative youth–she mentions going to school with many of the leaders of Silicon Valley startups as well as the staff at places like OpenAI–make this a compelling read. On the page, she writes confidently, intuitively, and it is a relief when someone can write such that the medium doesn’t get in the way of the message, and the message she has to share is quite frightening.

OpenAI famously started out as a nonprofit in 2015, funded by a consortium of backers, including Elon Musk, who pledged $1 billion in capital towards an organization that would “freely collaborate” with other institutions and researchers by making its patents and research open to the public. As Hao documents, the founders acknowledged almost from the beginning that “they could walk back their commitments to openness once the narrative had served its purpose.” Ilya Sutskever, one of the founders, wrote in January 2016–months after its founding– that “As we get closer to building AI, it will make sense to start being less open… The Open in openai means that everyone should benefit from the fruits of AI after its [sic] built, but it’s totally ok to not share the science.” Elon musk replied to this group message with, “Yup.”

And what was the purpose? Initially, it was to recruit the best AI engineers in the world, many of whom were being hoarded by Google, which was paying the best salaries. OpenAI couldn’t compete on salary, but it could promise academic openness, prominent colleagues like Sutskever, and the backing of billionaires. The best of all worlds.

Hao writes, “It became a cynical refrain among AI researchers: sell out to Big Tech or to the military industrial complex, or leave AI research. Between these binary extremes, OpenAI seemed like a third way, corrupted by neither profit nor state power.”

Perhaps the best chapter in the entire book comes early, Chapter 4: Dreams of Modernity, in which Hao provides an overview of the AI field in general. I wish this segment could be available for anyone curious about how AI works and why hype has been part of its DNA from the beginning. Hao starts off one paragraph with the following:

“The promise propelling AI development is encoded in the technology’s very name. In 1956, six years after Turing’s paper began with the line “Can machines think?” twenty scientists, all white men, gathered at Dartmouth College to form a new discipline in the study of this question. They came from fields such as mathematics, cryptography, and cognitive science and needed a new name to unify them. John McCarthy, the Dartmouth professor who convened the workshop, initially used the term automata studies to describe the pursuit of machines capable of automatic behavior. When the research didn’t attract much attention, he cast about for a more evocative phrase. He settled on the term artificial intelligence.

The name artificial intelligence was thus a marketing tool from the very beginning, the promise of what the technology could bring embedded within it.”

This really sums up a lot of the challenges facing AI more generally now: the mixture of hype and marketing; a field still dominated by white men; and hitched to the alluring yet elusive concept of “intelligence.” As Hao says, “The term lends itself to casual anthropomorphizing and breathless exaggerations about the technology’s capabilities.”

Blink twice if that sounds familiar to you. And, as Hao says soon thereafter, “the central problem is that there is no scientifically agreed-upon definition of intelligence.

Near the end of the chapter, Hao brings it back to OpenAI and notes:

“AGI, if ever reached, will solve climate change, enable affordable health care, and provide equitable education. OpenAI is the poster child for this line of thought. It cannot say how the technology will deliver on these promises–only that the staggering price society needs to pay for what it is developing will someday be worth it.

What’s left unsaid is that in a vacuum of agreed-upon meaning, “artificial intelligence” or “artificial general intelligence” can be whatever OpenAI wants.”

The book itself is 400 pages long, and I won’t attempt to recap it here; suffice to say, with this starting point, you can see where things are going.

One point I took away from this book, more so than from others, is that despite all the talent, money, and press, OpenAI didn’t really know what to do for the first few years. They assembled great talent but lacked clear examples of AI in action to point to. DeepMind was busy building the best chess and then Go-playing engines ever, as well as making progress in protein folding that eventually led to a Nobel Prize for its founder. OpenAI had nothing.

Until one of the researchers there, Alec Radford, a dropout from Olin College of Engineering, began playing around with the recently introduced Transformer architecture from Google. Hao writes, “Transformers can ingest large volumes of text and consider each word, sentence, and paragraph in a significantly larger context [than before for neural nets.]” Google saw it as a way to improve search and language translation.

But Radford had the insight to change the task the Transformer had to learn. Instead of translating languages, he switched it to learn text generation by predicting the most probable word in a sentence. It turned out that if you trained on a large enough dataset with enough GPUs and for enough time, you could develop Large Language Models (LLMs) that came close to mimicking human conversation.

Radford would go on to be the lead author of the GPT-1 paper in 2018 that was largely a proof of concept, demonstrating that generative pre-training (GPT) followed by fine-tuning could best supervised NLP benchmarks. The model had only a few hundred million parameters, which can be thought of as tuning dials.

He was also the lead author of the GPT-2 paper in 2019, which expanded to 1.5B parameters and was the first time people outside AI research circles noticed that these models could generate coherent text.

OpenAI began to focus almost entirely on this GPT approach and in 2020 released GPT-3 with 175B parameters that appeared to show emergent behaviors and set the stage for ChatGPT’s public launch in late 2022.

Also in 2020, OpenAI published a paper called Scaling laws for neural language models that implied that with enough training data, GPUs, and compute size, you could achieve expected gains in LLM model performance. More than anything else, this set off an arms race of sorts as the largest technology companies all raced to mimic OpenAI, since if it was a game of scale, who better to compete?

Hao astutely notes that, despite the enthusiasm at the time for scaling laws, it had deep connections to Moore’s Law around semiconductor chips. “In the end, Moore’s Law was not based on some principle of physics. It was an economic and political observation that Moore made about the rate of progress that he could drive his company to achieve, and an economic and political choice that he made to follow it….OpenAI’s Law, or what the company would later replace with an even more fevered pursuit of so-called scaling laws, is exactly the same. It is not a natural phenomenon. It’s a self-fulfilling prophecy.”

Let me pause here to say, ouch. And yet, I agree with her. Again and again, Hao is able to cut through the noise and deliver sharp insights on LLMs and OpenAI in particular.

I should step back and note that Hao is not a hater or denier of AI in general. It’s just that she’s attempting to take a clear-eyed look at its costs, benefits, and the accumulated damage along the way.

In several parts of the book, Hao has done real investigative work to highlight the human cost of building these LLM models, especially in the last few years, as the emphasis shifted from clean pre-training data to a download-the-internet-and-sort-it-out-in-post-training approach. That is, rather than filtering data before compute, OpenAI and others are now so hungry for data–any data at all–that they don’t try to filter it much before running through the compute process. Instead, they apply filters after the fact to weed out offensive content and fine-tune the model to be more accurate. This shifts the burden of filtering onto human beings, typically those with a computer, internet connection, and in dire economic conditions, such as Venezuela, Kenya, and other countries. This technique is more broadly referred to as Reinforcement Learning from Human Feedback (RLHF), and a cottage industry of companies has sprung up to service LLM providers, offering contract-based work to desperate individuals.

The later sections of the book focus particularly on Sam Altman and the growing pains of managing OpenAI. He does not make for a compelling subject, to say the least, and Hao does note that many of the leading scientists and executives at the company end up leaving after the board’s aborted attempt to remove him from power.

Internally, employees began to refer to this period as “The Blip,” a reference to the Avengers, where Thanos snaps his fingers and half the world disappears, only to later return thanks to the Hulk.

The Blip

Overall, there is a lot to process and ponder from this book. It is a fantastic technical overview of recent advancements in AI, LLMs in particular, as well as a well-reported look at the individuals running these now all-powerful companies. It is hard not to emerge with a strong sense of unease.

Hao makes a convincing case that modern AI power players mimic the empires of old. “During the long era of European colonialism, empires seized and extracted resources that were not their own and exploited the labor of people they subjugated to mine, cultivate, and refine those resources for the empires’ enrichment. They projected racist, dehumanizing ideas of their own superiority and modernity to justify–and even entice the conquered into accepting–the invasion of sovereignty, the theft, and the subjugation. They justified their quest for power by the need to compete with other empires: In an arms race, all bets are off.”