not a bot i just don't know what I'm doing

Timnit Gebru was fired from Google in December 2020 for refusing to retract a research paper, and every single warning that paper made about large language models has now happened at a scale the industry spent 4 years trying to make people forget about.

Her name is Timnit Gebru.

She co-led the Ethical AI team at Google. She co-wrote a paper called "On the Dangers of Stochastic Parrots" with Emily Bender at the University of Washington and two other researchers. The paper was 14 pages long. It was submitted to a top AI ethics conference. And it was the reason Google decided that one of the most senior Black women in AI research could no longer work there.

The story Google told publicly was that she resigned. The story she told, confirmed by 2,695 of her colleagues in an open letter, was that she was fired by email while on vacation because she refused to either retract the paper or remove her name from it.

The paper had not even been published yet.

Here is what she actually wrote, and why every prediction inside it has now come true.

The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. They called these systems stochastic parrots because they would repeat patterns from training data with statistical confidence and zero comprehension. The paper predicted that this apparent intelligence would fool both users and developers into trusting outputs that were structurally incapable of being reliable.

This was 2020. GPT-3 had just come out. The paper predicted the hallucination problem before anyone had a word for it.

The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it, because the optimization process rewards confident outputs, and confidence in language patterns tracks frequency in the training set.

The prediction was that hiring tools built on these models would discriminate against women. That healthcare triage tools would underperform on Black patients. That loan approval systems would entrench inequality while presenting their decisions as neutral algorithmic judgment.

Every one of those things has now been documented in deployment.

Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.

The third warning was about environmental cost. The paper calculated that training a single large language model produced emissions equivalent to the lifetime output of 5 cars. The prediction was that the race to scale would create an environmental footprint that would eventually rival entire industries.

In 2024, Google's emissions were up 48% from 2019, and the company explicitly blamed AI infrastructure. Microsoft's were up 29%, same reason. Both companies have now quietly abandoned the climate commitments they were publicly celebrating the year Gebru was fired.

The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit. Nobody at Google, OpenAI, Meta, or any other lab could tell you with confidence what was in the data their models were trained on. This was not a temporary problem to be solved later. It was a permanent feature of the approach.

In 2023, researchers discovered that the LAION-5B dataset, used to train Stable Diffusion and other major image models, contained thousands of images of child sexual abuse material. The companies that had trained on the dataset had no way of knowing. The paper predicted that category of failure 3 years before it was found.

The fifth warning was the one Google cared about most.

Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them. The internet would become a place where the dominant voice was a statistical average of dominant voices, presented as a neutral assistant. Languages underrepresented in the training data would degrade over time as more web content was generated by these systems and fed back into the next training run.

This is now happening in real time. A 2024 study found that 57% of new web content in English is AI-generated or AI-assisted. Researchers studying low-resource languages have documented active degradation in translation quality, because the synthetic content fed back into training is itself worse in those languages.

The paper Google fired her for predicted the model collapse problem before model collapse had a name.

The mechanism behind why this all happened is the part of her work that nobody quotes.

Gebru's argument was not that AI is dangerous in some abstract sci-fi sense. Her argument was that AI is dangerous in a very specific structural sense. The technology was being built by a small group of researchers who shared similar backgrounds, worked at similar companies, and were rewarded for shipping products faster than competitors. The incentive structure made it impossible for safety, ethics, and bias concerns to slow anything down. Anyone inside the system who raised those concerns was either ignored, sidelined, or removed.

She was making that argument from inside Google.

Then Google proved her right by removing her.

The team Google had built to make sure their AI was safe was dismantled in 90 days because they did the job they had been hired to do. Margaret Mitchell, the other co-lead of the Ethical AI team, was fired two months after Gebru for searching through her own emails for evidence of how Gebru had been treated.

Gebru did not stop. She founded DAIR, the Distributed AI Research Institute, in 2021. The mission is to do AI research outside the control of the companies that have a financial interest in not hearing the answers.

Every prediction in the Stochastic Parrots paper has now been validated by deployment. Hallucinations are an industry-wide problem the largest labs cannot solve. Bias amplification has been documented in hiring, healthcare, lending, and criminal justice. Environmental costs are larger than entire small countries. Training data audits remain impossible. Model collapse is an active research crisis at every major lab.

The question worth sitting with is the one almost no one in the industry will say out loud.

Every researcher with the technical credibility to call out these problems watched what happened to her in December 2020 and made a calculation about their own career. The number of people willing to speak publicly about safety and ethics issues inside the major AI labs collapsed after that firing and has not recovered.

The researcher Google fired for warning about exactly what is now happening was right.

The company that fired her is now the second-largest deployer of the technology she warned about.

This book is largely concerned with Hobbits, and from its pages a reader may discover much of their character and a little of their history...

That is the first sentence of The Fellowship of the Ring.

The prologue of the Lord of the Rings is iconic. Swept away by the story, we forget we’re reading at all. It's understandable. Who can resist the overwhelming charm of the writer, and the bewildering excitement of being taken by the hand and invited into The Fellowship of the Ring.

But even people with deeper takes on Tolkien tend to miss the significance of the Prologue. It’s a place where critical reading abilities and political processors usually turn off entirely - fair enough, it's probably a relief. Let's talk about the Narrator.

Meeting the Narrator: Time, Place, Person

The Prologue is narrated by a Mannish (Big Folk) Narrator, a modern human being, from an accessible academic standpoint. We are encouraged to think of him as Friendly Professor Tolkien, although you really do need to remember that he is clearly addressing us from within a fictional narrative world. He is a character. Even if he is Tolkien’s self-insert, intended to be read as Tolkien Himself, he is still a character who can be analysed and interpreted. This is a fictional character.

The Big Man deliberately addresses the reader as someone with a shared background, in what is presumably somewhere in the early-to-mid twentieth century. It is stated multiple times that you (reader) and Narrator are both Big Folk together - there is no chance that you, the Reader, are a scholar of another race.

It is plain indeed that in spite of later estrangement Hobbits are relatives of ours: far nearer to us than Elves, or even than Dwarves.

Pick out the “later estrangement” and park it for now.

The casting of the Narrator is a deliberate alignment with Professor Tolkien, and we are certainly intended to understand him as an academic, avuncular, rather unworldly male professor in the British Isles.

(Sidebar: for convenience I gender the Narrator as male. I think there's evidence for the Narrator being intended as male-by-default, which can be provided on request, and I personally feel the Big Man narrator is the translator/propagator of the silly convention of referring to modern humans as “capital-M Men.” )

The Prologue is written charmingly, a framing device of an academic translator giving the context of background information before presenting someone else’s text (the translated Red Book, etc). Later, this Prologue connects to the Appendices in The Return of the King, where the Narrator returns in his persona of the translator of the works. Our Narrator is certainly a strong, influential, deliberate character, with a specific and distinctive voice!

Anyway, whether or not you choose to picture the Big Man Narrator as Tolkien Himself doing a folksy Bit, OR as a character Tolkien created - Remember! The entire story is fiction and the Big Man Narrator is a created fictional character. Why would you assume he is telling the truth? Why assume that he is an expert? Where are his biases?

Look what the Big Man Narrator actually says. Look at what he chooses to tell, and what he finds unimportant. There are so, so many posts that pick over the fascinating bits of Concerning Hobbits, mining canon for more information, as if it is a pure source of truth. I suggest that the next time you do, you try this fun exercise.

Before we go into the Magic Thing, the narrator also notes AGAIN that hobbits exist today, but are shorter than they were;

They seldom now reach three feet; but they have dwindled, they say, and in ancient days they were taller.

This continues and reinforces the framing of “hobbits still exist now,” and sounds rather as if the Big Man has interviewed modern hobbits (“they say,”) which we’ll also park.

We move on, parking "it's assumed you're a Man, receiving information from a Mannish professor", the "future estrangement" and "diminished hobbits are available for interview."

The Magic Thing

I was provoked into writing this by a fun Tumblr post pointing out that "hobbits are said to 'not study magic' - does that mean that they don't HAVE magic?" which went off into a separate and funnier reblog chain.

I want to analyse this again, noting that this is information received from Big Man.

Let’s examine the “hobbit magic thing” noting that we are being TOLD all of this by a CHARACTER.

Here’s how the passage about "hobbit magic" starts.

Hobbits are an unobtrusive but very ancient people, more numerous formerly than they are today;

In our time, we’ve just been told, hobbits still exist, but had a population drop and are vanishing. To the point where a reader is not expected to have ever heard of them. Chillingly, in typical mid century British academic fashion, the Big Folk Narrator assumes that the reader is also British; when he later mentions that the remaining hobbits only live in the British Isles, it’s a little alarming. There’s a species of humans native to these islands, so rare and so politically silent that you’ve never seen or heard of them.

Hello?!

for they love peace and quiet and good tilled earth: a well-ordered and well-farmed countryside was their favourite haunt.

We are told here that Hobbits are going extinct because they cannot readily survive due to, essentially, habitat destruction. (we feel the Narrator’s annoyance about the Industrial Revolution spoiling the “peace and quiet” strongly here, more strongly than the buried implications for indigenous people).

They no longer have any land. Not only have they lost the Shire, they have no towns, small villages or even farms. “Was” is very much past-tense, and they “haunted” land in the past, ghosting lightly and leaving no traces of their presence, rather than living there. so in our modern day there’s certainly no Shire, no Bree (mixed human/hobbit town) and no Michel Delving, which in its time was a market town with above-ground buildings and a museum. For context, it takes a decent amount of work for the British Isles to lose towns, especially on the level of development that Hobbits had - famously anachronistic, they have waistcoat buttons and watermills and good china and museums and smoking habits, while all the rest of medieval-ish Middle Earth is not as developed.

It’s hard to lose all that, without any trace at all, in crowded countries. Wholesale loss always means that Something Happened.

They do not and did not understand or like machines more complicated than a forge-bellows, a water-mill, or a hand-loom, though they were skilful with tools.

“Do not and did not” is further reinforcement of a still-living people. (I love the “understand or like” thing, which is charming - the implication that hobbits are perfectly capable of UNDERSTANDING machinery in textile factories, but would hate it.)

Something that makes the Big Man nuanced as a character is that he obviously adores hobbits, and studies them because he likes them. The fondness and admiration comes through, even as he is showing his own privilege and bias.

To me, the way this passage about machinery is framed - lumping together those machines as “about the level of technology hobbits are comfortable with” - is something that someone standing post-Enlightenment, probably post-Industrial Revolution, would do. The implication I take from this passage is that this is a modern writer describing the current status of modern hobbits; a mid-century British scholar, a self-insert of Tolkien.

This sense of time matters, because of everything else he says, and the temptation people will have to excuse the Big Man narrator as “a product of his time.” This isn’t a medieval writer looking back on Middle Earth. It’s a highly educated man writing in the 1940s: computers existed, there were several Disney films out, women had the right to vote, and feminist essays were published from Tolkien's own workplace.

Even in ancient days they were, as a rule, shy of ‘the Big Folk’, as they call us,

We then proceed to see, across three books, examples of hobbit behavior in “the ancient days”, which may serve as an example of this shyness. Several different relationship with Big Folk are outlined, in which fairly chirpy hobbits, characterised by their ready emotional availability, cohesion, and incredible abilities to build relationships and form massive political alliances, seem to do well on the strength of that. Hobbit shyness may involve glaring ferociously at Big Folk for a moment, but within a few days they are sitting on your lap, and then it’s all over. With this evidence in our memory, casting a coy “shyness” as the reason for their avoidance of “us” becomes uncomfortable.

and now they avoid us with dismay and are becoming hard to find.

The Narrator is handwaving, in avuncular fashion, why the Reader has never seen a hobbit in their lives, and needs to be lectured, from first principles, on a living indigenous people of the British Isles. Do marinate on it for a moment, though. The tone of a professor or a parent, whimsically explaining to Victorian children why you don’t see the Tooth Fairy - she hides! Teehee.

They avoid us with dismay.

Behind this airy statement, what happened? Massive betrayals, the loss of their land and political power, loss of the conditions they need for their survival, massive loss of their people, and a total breakdown in trust. Humans and hobbits, in the prologue and main story, are shown as natural allies; close kin who understand each other well; humans are shown owing a tremendous amount of their own political influence to hobbits, and even cold/reserved humans end up liking them after a conversation. Hobbits are especially shown for being loyal friends who do not break down under war; noted for retaining cohesion and resisting corruption; who, under unimaginable conditions, will still resist harming or betraying friends.

Hobbits and humans have clearly had some significant breakage of our kinship since the events of the LotR cycle. The Big Man knows this.

Earlier in the essay, when the Big Man told us that “hobbits are closest to (us)” he gave us a lot of additional information, didn’t he? He refers to “later estrangement.” (He also tells us clearly, in that subtext of that sentence, that no hobbit will ever read the book in our hands, no hobbit will ever be addressed as a reader, no hobbit will enter academia, no hobbit will be able to fill in the gaps that the Big Man waves his hand over. Certainly no hobbit scholar contributed to the Big Man’s translation of the Red Book. They’re not just going, they’re functionally GONE. This is what I mean!) Anyway, even the Big Man notes that there was “an estrangement.” Something that has caused them to flee from contact with us in dismay.

whatever happened in that estrangement probably doesn’t reflect well on the Big Folk. A species facing extinction and hiding, dismayed and estranged, from their closest kin, is not having a pleasant time on this earth. Especially when we understand that they’re basically trapped in the crowded and inhospitable British Isles (and still managing to hide from us to the point of the public not being aware of their existence!)

The Big Man Narrator isn’t interested. This is the point where you ought to start wondering about academic bias on the part of the Big Man Narrator. He's fond of hobbits, and has interviewed/met them, but would never treat one as a colleague.

[…]They possessed from the first the art of disappearing swiftly and silently, when large folk whom they do not wish to meet come blundering by; and this art they have developed until to Men it may seem magical.

so hobbits have an inherent ability of being invisible/undetectable, which they still practice today (teehee, that’s why it’s okay that you’ve never spoken to one) and which is pretty damn effective. Effective enough that people in modern times are completely fooled, effective enough that it still counts as “disappearing,” and the elusiveness of hobbits is so perfect as to conceal their existence from the general public. Effective enough that the few adults who DO discuss hobbits could conceivably think it could be magic. The Narrator has probably rolled his eyes over a rival’s paper about “Slipping Into The Shadow-Realm: how hobbits shift space and time to conceal their vital signatures” (Sayers, 1934).

further, they’ve specifically developed this “art” - from what’s implied to be an instinctive/animal ability - to a higher skill, indistinguishable from magic. The “art” is SOMETHING material and quantifiable, if it was innate-and-continually-developed.

But Hobbits have never, in fact, studied magic of any kind,

Here is a point that’s been discussed on tumblr, and it is correct to note that “studied” is doing a lot of work. Especially when contrasted against the previous sentence, with the interesting term “art”. “It isn’t science/magic, it’s an instinctive art”.

To me - remembering that this is intended to be a mid-century British academic speaking to us - it resonates with how romanticism of marginalised cultures was treated by academia, in the generation the Big Man Narrator would’ve studied in - full of romantic, unexamined, politically revealing statements like, “The Celts are skilled in the art of music, but have never properly studied it.”

What I’m saying here is that we should not assume the Big Man is a good judge of the difference between “art” and “study,” especially since the next bit reads:

and their elusiveness is due solely to a professional skill that heredity and practice, and a close friendship with the earth, have rendered inimitable by bigger and clumsier races.

Hobbit invisibility is an “art” through “heredity,” but also a “professional skill” refined through “practice.” It has been “developed” until it is mistaken for “magic,” but against this, we are told that hobbits “have never studied magic of any kind.” In the cleavage point here, we can see the definition of “study” that The Big Man is working with. This definition is possibly what makes something “magic” or not. Have you seen this before?

The point I am making here is that the Big Man is speaking to us from the position of a “coloniser.” There are some worldbuilding implications to unpack from this. One is that the Big Man is speaking from a place where magic can be studied, not even requiring hereditary aptitude (if hobbits were excluded from magic by physiology, This Fuckin Guy would’ve said it) but that it is an academic practice. Hobbits are not just nearly-extinct and terrified out of contact with humans; they are fully excluded from academia (they do not translate or contribute to translations of their histories; they do not study) and if they cannot formalise their practices in acceptable study as the Big Man defines it, it cannot be magic. This is exactly the tone in which majority cultures dismiss other practices of culture/medicine/science, by stating it is NOT a form of science, because it is not practiced with the academy, because it is definitionally not allowed in the academy.

We can then go to a higher level of political analysis and reading, and ask: who benefits from a definition of “magic” that includes (academic study) but excludes (hobbit arts)?

You can certainly do some delightful worldbuilding answers for yourself, and say that “perhaps magic is spells, material changes, great works as performed by Elvish or Maiar Ringbearers, etc.” But if we look at the political stuff I’ve just pointed out, why not examine the definition and who it serves and why? Given that we’ve seen this pattern before - colonisers deliberately bundle, define and dismiss marginalised practices as primitive, animalistic, instinctual and unschooled, as part of the PURPOSEFUL WORK of colonisation - I read the Big Man definition as: “Magic is formalised by the bigger races and defined by excluding the practices of the smallest race.”

Who does this benefit? Well, the Bigger Races could in some ways. Magic must be studied, hobbits don’t study, hobbits don’t have magic, hobbits are The Only Unmagical Humans - despite having practices indistinguishable from magic - this could be something. Big Men would have some reason to define “magic” to exclude hobbits. Normally this is done in order to take resources or drain resistance from marginalised people, but as hobbits have had virtually no remaining resources or resistance since long before the Industrial Revolution, you could open this up to other worldbuilding implications - maybe, Big Men didn’t really MIND hobbits going extinct.

An interesting point here is to re-read sections of this work with different interpretations of who the Big Man is. Where are his biases? Who is he as a character?

I personally read him as a friendly, Tolkienesque academic who likes hobbits, follows his linguistic interests, and is too blinded by his bias to think about their political position. He seems unaware of the horrors he's talking about. Perhaps that's down to innocence.

A character crying out to be analysed.

Landless and Dismayed

That sums up a lot of information that can be mined from one of the very first paragraphs of the Fellowship of the Ring. But here's another message to toy with - hobbits exist in the modern space; landless, estranged, fleeing from us in dismay. Quite likely to have been betrayed.