All of you Undisturbed Cities...

Timnit Gebru was fired from Google in December 2020 for refusing to retract a research paper, and every single warning that paper made about large language models has now happened at a scale the industry spent 4 years trying to make people forget about.

Her name is Timnit Gebru.

She co-led the Ethical AI team at Google. She co-wrote a paper called "On the Dangers of Stochastic Parrots" with Emily Bender at the University of Washington and two other researchers. The paper was 14 pages long. It was submitted to a top AI ethics conference. And it was the reason Google decided that one of the most senior Black women in AI research could no longer work there.

The story Google told publicly was that she resigned. The story she told, confirmed by 2,695 of her colleagues in an open letter, was that she was fired by email while on vacation because she refused to either retract the paper or remove her name from it.

The paper had not even been published yet.

Here is what she actually wrote, and why every prediction inside it has now come true.

The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. They called these systems stochastic parrots because they would repeat patterns from training data with statistical confidence and zero comprehension. The paper predicted that this apparent intelligence would fool both users and developers into trusting outputs that were structurally incapable of being reliable.

This was 2020. GPT-3 had just come out. The paper predicted the hallucination problem before anyone had a word for it.

The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it, because the optimization process rewards confident outputs, and confidence in language patterns tracks frequency in the training set.

The prediction was that hiring tools built on these models would discriminate against women. That healthcare triage tools would underperform on Black patients. That loan approval systems would entrench inequality while presenting their decisions as neutral algorithmic judgment.

Every one of those things has now been documented in deployment.

Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.

The third warning was about environmental cost. The paper calculated that training a single large language model produced emissions equivalent to the lifetime output of 5 cars. The prediction was that the race to scale would create an environmental footprint that would eventually rival entire industries.

In 2024, Google's emissions were up 48% from 2019, and the company explicitly blamed AI infrastructure. Microsoft's were up 29%, same reason. Both companies have now quietly abandoned the climate commitments they were publicly celebrating the year Gebru was fired.

The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit. Nobody at Google, OpenAI, Meta, or any other lab could tell you with confidence what was in the data their models were trained on. This was not a temporary problem to be solved later. It was a permanent feature of the approach.

In 2023, researchers discovered that the LAION-5B dataset, used to train Stable Diffusion and other major image models, contained thousands of images of child sexual abuse material. The companies that had trained on the dataset had no way of knowing. The paper predicted that category of failure 3 years before it was found.

The fifth warning was the one Google cared about most.

Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them. The internet would become a place where the dominant voice was a statistical average of dominant voices, presented as a neutral assistant. Languages underrepresented in the training data would degrade over time as more web content was generated by these systems and fed back into the next training run.

This is now happening in real time. A 2024 study found that 57% of new web content in English is AI-generated or AI-assisted. Researchers studying low-resource languages have documented active degradation in translation quality, because the synthetic content fed back into training is itself worse in those languages.

The paper Google fired her for predicted the model collapse problem before model collapse had a name.

The mechanism behind why this all happened is the part of her work that nobody quotes.

Gebru's argument was not that AI is dangerous in some abstract sci-fi sense. Her argument was that AI is dangerous in a very specific structural sense. The technology was being built by a small group of researchers who shared similar backgrounds, worked at similar companies, and were rewarded for shipping products faster than competitors. The incentive structure made it impossible for safety, ethics, and bias concerns to slow anything down. Anyone inside the system who raised those concerns was either ignored, sidelined, or removed.

She was making that argument from inside Google.

Then Google proved her right by removing her.

The team Google had built to make sure their AI was safe was dismantled in 90 days because they did the job they had been hired to do. Margaret Mitchell, the other co-lead of the Ethical AI team, was fired two months after Gebru for searching through her own emails for evidence of how Gebru had been treated.

Gebru did not stop. She founded DAIR, the Distributed AI Research Institute, in 2021. The mission is to do AI research outside the control of the companies that have a financial interest in not hearing the answers.

Every prediction in the Stochastic Parrots paper has now been validated by deployment. Hallucinations are an industry-wide problem the largest labs cannot solve. Bias amplification has been documented in hiring, healthcare, lending, and criminal justice. Environmental costs are larger than entire small countries. Training data audits remain impossible. Model collapse is an active research crisis at every major lab.

The question worth sitting with is the one almost no one in the industry will say out loud.

Every researcher with the technical credibility to call out these problems watched what happened to her in December 2020 and made a calculation about their own career. The number of people willing to speak publicly about safety and ethics issues inside the major AI labs collapsed after that firing and has not recovered.

The researcher Google fired for warning about exactly what is now happening was right.

The company that fired her is now the second-largest deployer of the technology she warned about.

and this rage has very little to do with AI or this AI generated story, and a lot more to do with the epistemology of reading across cultural difference:

what assumptions are you making when you encounter prose that doesn't do what you're used to, and how do you distinguish between:

this is doing something I don't have the framework to follow/yet

and

this is doing nothing

the uncomfortable answer is that a lot of people, at levels high above the average reader mind you, being prize judges and all, don't make that distinction. they experience the unfamiliarity and name it as failure, as excess, as incoherence, as the literary equivalent of noise, without asking whether the problem is in the text or in the reading, or they fall prey to a manifestation of ‘trim the fat culture’ (good post on this).

this is not an accusation of bad faith reading necessarily; it is just what happens when you read without the relevant context and without the intellectual curiosity to notice that you're missing something and attempt to find it. telling, however, is how quickly that experience of unfamiliarity, in this particular case, became a generalisation. not "I find this story's specific metaphors incoherent" but "I find this kind of writing incoherent", as if “this kind of writing” is a stable category and not just something this AI slapped together. a sliding from the fraudulent to the traditional that happens with striking confidence, and one which you do not see applied with the same ease to, say, Western European modernism, where the response to difficulty tends toward "I need to read more Woolf to understand Woolf" rather than "yucky stinky Woolf is AI-slop”.

anyway. here is my favourite sentence from the shitty AI story:

"she had the kind of walking that made benches become men."

and like it’s my all time favourite sentence ever because like. what does it mean. what is it doing. why is it there. what decision was made in its construction and to what end? and I just could not come to a conclusion because the real answer is that no actual decision was made, because decision-making requires an engagement with the writing, requires a reasoning for the sentence to exist in the way it does, and this exists across all literary prose styles, from the sparsest to the lushest. the bench-men sentence is difficult to interpret, but not in a ‘this is difficult to interpret which makes the reward of interpretation sweeter’ way, it is difficult to interpret in a ‘there is nothing under this sentence’ way, and that is made very clear when even the slightest interpretative pressure is laid on the story.

anyway, turns out the judges of one of the world’s biggest literary competitions did not apply that pressure. caribbean regional judge Sharma Taylor described Nazir's language as "sublime — precise yet richly evocative — conjuring vivid, lush imagery with remarkable economy" and like man this isn’t to dunk on Taylor personally but i think that sentence, in being a diagnostic object, is in itself a diagnostic object as to the whole scandal here: it’s evaluative language that doesn’t touch the text itself, a string of compliments whose terms don’t require a unique object. "precise yet richly evocative" is a sentence that could describe anyone from Chekhov to MT Vasudevan Nair.

what it cannot do is tell you what is precise about Nazir's objectively vague, dreary sentences, or where exactly economy manifests in a story that opens with three subordinate images somehow being unable to create even half an image. the judges either didn't notice or didn't give a fuck, and imo the honest interpretation there is that the evaluation was matching the text against a prior model of what this kind of writing is supposed to feel like, rather than what it actually does.

the main vulnerability of this kind of matching-against-model judging criteria is that it can only flag deviation from the expected shape, not absence within it. a story that inhabits the expected form, even hollowly, passes muster. a story that does something actually unexpected might fail on those same grounds, whether or not it's extraordinary. the AI machine got through to the prize list not because it fooled sophisticated readers into thinking they were reading a great work of literature, but because the reading operation in use did not require that experience of reading great literature to complete successfully. you just needed the vague shape, and the machines are good at making vague shapes.

what shape?

seemingly lyrical, lush, image-dense, located in rural poverty or landscape-as-metaphysical-weight, threaded with folk memory and unresolved grief, incantatory, myth-grabbing, rum shops, zinc rooftops, zinc-hair. what the AI has done is precisely what it is built to do: grab tiny scraps and fragments from actual prize-winning postcolonial stories and shoved them all together into an amorphous, senseless mass, knowing what it is supposed to do but not knowing how to do it. and so to me the most astounding/horrifying aspect of this scandal is how the judges who one can safely assume, based on their credentials, are very familiar with ‘world literature’, proved unable to tell the difference between a form inhabited and a form vacated.

and I really don’t like bringing up my literary/academic credentials (derogatory) etc etc on here anymore, because it at times positions me in an uncritical way I don’t intend or enjoy and I spent my early months in fandom realising just how very uncomfortable I was with the image I inadvertently curated as a result of coming straight from that sort of literary-academic space. so to put it very basically: I have spent my academic career broadly specialising in the very style and period of postcolonial literature that this AI story is attempting (badly) to emulate. my focus has always been south asia but i have also worked extensively with caribbean lit especially early on, and i’ve been taught/examined by some very well known caribbean writers and literary scholars, etc etc. ie i’m just trying to say that this post isn’t just me talking about a vague grievance with literary cultures but something i’ve been neck deep in for 10+ years now, ie i do know my shit and am not just knee jerk wanking, even though frankly i don’t think i should have to explain my background because way too many people are being way too confident with the ‘i have been writing for THREE BILLION years and they gave ARUNDHATI ROY THAT BITCH the booker prize’ atm…

anyway the reason I’m so brainrotted about this is because this exact literary-cultural problem was one of the things that led me to structure my longfic, Prayers to Broken Stone, in the way I did. the fic itself is totally irrelevant here so you’re not missing anything if you haven’t read it or are unfamiliar with the Silmarillion, I’m just referring to how the first quarter of that fic deliberately contains every single postcolonial miserycore cliché that appeals to a literary-prize, Western Anglophone, and diasporic audience’s ideas of what ‘Global South’ world-literatures should look like (and ngl I feel like I probably went too hard on this because so far I know at least 5 ppl familiar with the genre who justifiably almost dropped the fic before the mic drop because of the beginning being Like That… sorry guys. i will probably do it again 😭).

anyway after that, and very abruptly, the story takes a hard pivot to what it actually is, which is not an apolitical portrait of India, not diasporic literature about the Indian subcontinent, not even an Indian novel about Kozhikode, but a Kozhikodan novel about India, down to the style: my writing in general tends to lean on carnivaleque and incongruous tonal whiplashes between ‘lowbrow’ humour, abject tragedy and direct critical fourth-wallfucking commentary, but that whiplash is turned all the way up to 100 in Prayers and the humour especially is taken to borderline slapstick levels, and that style is evocative of Kozhikodan literary cultures (see—writings of Vaikom Muhammad Basheer, who is mentioned in the story in that Comrade Maedhros lies claims they are great buddies lmao), only that most writing from the region is in Malayalam, etc.

the reason i mention the fic here is that objectively speaking, those first few chapters, the ‘series of clichés’ ones, are the ‘clearest’ part of the story when it came to writing it. those chapters were written to directly evoke the vague shape of ‘prizewinning postcolonial giants’ of South Asian literature, both the brilliant and incisive writers and the floggers of diaspora-gaze miserycore, providing a series of aesthetic signals to those texts: the joint-family ‘madhouse’, the separated twins, the daddy-issues-as-father-of-the-nation-issues, the family-as-country, the dried rivers, the symbolic heirloom bangle, the utopian pre-imperial regional historiography, the diasporic returnee, the rotting house, the familial disconnect. Roy, Rushdie, Mistry, Lahiri, Desai, Seth, Ghosh, rinse and repeat.

do I personally enjoy every single one of these authors? no, I would probably cagefight two of them at least. what I am saying though, is that that their writing isn’t some kind of incomprehensible mess that nobody aside from their little tiny id-group can understand, not amorphous or vague or too overwrought to comprehend. their prose, all differing styles, can be rich, lush, playful, meandering, yes. but they are not unclear: they’re so clear that the positionality of the authors, their class and caste backgrounds, their educational and migratory trajectories, are often painfully evident (hence the cagefighting). the reason i used those aspects in my fic to signal towards a particular kind of globally lauded postcolonial literature is because those signals are clear, not confusing.

ie it is not a case of ‘global south’ writers being incomprehensible, it is a case of readers walking into a garden with a few flowers they haven’t seen before and immediately going ‘damn, look at this jungle. can’t navigate it but i’m sure it’s great, ok bye’ then turning the fuck around and writing the travelogue anyway. which is to say, applying a colonial reading practice to postcolonial writing.

and there’s a similar, though differently approached, aspect in globally renowned caribbean anglophone writing: a history of deliberate formal difficulty. where the difficulty isn’t some ambient mystery or marker of ‘serious’ literature but a formal consequence of a model of storytelling. eg. Selvon's Creole narration in The Lonely Londoners was a decision with costs+purposes about what it would mean for Moses Aloetta's interiority to be rendered in standard English versus in a voice that had not been, at the time, admitted to the Anglophone literary canon, rather than being the inevitable default of a Caribbean writer. Harris's dissolving frames in Palace of the Peacock are not difficult because Harris was apathetic to comprehensibility but because the Guyanese historical consciousness the novel examines does not easily resolve into stable subjectivity.

form is so often part of the argument across literature, across the English canon itself, and normally in literary criticism, ‘difficulty’ is approached epistemologically alongside aesthetically. this is common knowledge yet the first part is something that appears to be hard to grasp for people reading and commentating on ‘world literature’.

what is this form doing that another form cannot?

you can answer that question for Harris and Selvon and Ghosh and Roy and man, I think he’s so fucking annoying sometimes, but you can even do it for Rushdie. you cannot do it for "coffee and cocoa leaned wild on a slope that wanted either rain in teeth or none at all". and this impossibility has nothing to do with foreignness or excessiveness but because the question, when applied to this AI generated piece of writing, has no answer.

and like… what does that tell us about what the judges were evaluating? imo it tells us they were at least in part evaluating surface-level compliance. compliance with recognisable genre conventions and an expected register, and so with the right signals of “authenticity”. and in the case of ‘Global South literature’, these conventions include an emphasis on the rural, the embodied, the rooted, the mythical.

an AI is very good at compliance because compliance is, quite literally, what AI does: every LLM is trained on the corpus of what has been rewarded before and thus it reproduces the patterns of that reward. if the judges were themselves rewarding compliance with a known type, then of course the AI passed with flying colours, because they were, in effect, running the same operation as the LLM model: matching input against a predetermined template instead of engaging with the work itself.

not to use my favourite cliché, but this specific scandal having played out in the way it did pretty much evidences how these two things, the upper echelons of the global literary prize circuit judging panels and generative AI, are less ‘warring factions’ and more ‘two frat bros fisting each other while saying no homo bro’, ie comorbid counterparts.

and so imo the question that should haunt every future Commonwealth Prize shortlist is not "did an AI write this?" but "what model of literary value are we using to judge Anglophone literature?”, and “why the fuck are we doing that???”

bc if your aesthetic criteria are vague enough that a sentence like "the grove isn't a ledger; it's a mouth — it closes only when it's satisfied" reads as "vivid, lush imagery" delivered with "quiet authority," then your judging criteria is less criteria and more vibes. you are literally just playing a high-stakes vibes-based game of Pin the Tail on the Mango whilst wilfully ignoring how vibes are precisely what AI large language models are the best at faking.

anyway, like I said in my intro, this scandal is already sliding into a secondary discourse in which ‘Oriental™️ opacity/incomprehensibility’ is being treated as the general category, of which this AI-generated confusion is just the most recent instance. you can watch it happening in real time, unbearably prolonged: people who rightly found the Nazir story incoherent, reaching way too easily for other examples of postcolonial prose they also apparently found incoherent or “purple”, prose that is, in fact, doing things they just didn't know how to follow. the AI story has handed a lazy, sneering and dismissive reading practice the veneer of clinical diagnosis.

that is the horribly ironic thing here. reader after reader, openly admitting to doing the exact same lazy, apathetic reading of postcolonial literature as the literary prize judges they are (rightly) criticizing have done with this AI story, have been doing for human-writing from the global south for all this time. “ewww this is what that writing looks like when a machine does it" (correct) is sliding so so so easily into "ewww this is what that writing looks like" (not correct). dog after dog, chasing tail after tail.

and that slide, from a machine having ‘successfully’ impersonated prize-winning prose, to a panel of judges who clearly weren't really reading, to the genre itself being defined as imitable machinery, is imo the most damaging thing to come out of this whole affair, and the people most hurt by it are the writers who have fuck all to do with Jamir Nazir, who is clearly just a chancer who fucked around and found out.

because somewhere in those 8000 entries, there is a writer, possibly many writers, who solved their riddle, who knew what every sentence was doing, who had made the thousand small decisions that constitute a story, and whose difficulty (if their story was difficult: difficulty is subjective and not a default, as we have established) could easily be accounted for. that writer did not win, because the judges were not looking for them. and now, in the aftermath, the interrogation of the incident continues to refuse to ask the questions that would have found them.

I first thought it would be blowing smoke up my own ass to finish this post with a quote from my own story. and then I remembered that this is my circus and you are all my monkeys, so I will indeed be ending with a (spoiler-free, context-unnecessary) quote from the final chapter of Prayers, from one of the fic’s multiple fourth-wall breaches, this one explicitly addressing both the character of Maedhros, a gay Muslim man in postcolonial India, as well as the attritional impact of global Anglophone prize cultures on ‘national literatures’, explaining the structure of the story and touching on the reading-practice I talk about in this post, this cold, dismissive flattening based on the reader’s refusal to comprehend the unfamiliar. Emphasis obviously made just for this excerpt:

Humanity has tried many times, with fanfare and floodlights, to hold the great white shark within glass walls. When a young female was placed in the Steinhart Aquarium in San Francisco, its keepers marvelled for a day, two days, then watched as she rammed herself against the tank walls, snout bloodied and refusing food until her body yielded to exhaustion. In San Diego, one was found dead within two weeks. More recently, in a public aquarium, a six-foot juvenile circling its tank like a condemned spirit, colliding with the corners until its skin peeled raw, was released after months only to die on the way back to the sea. Each attempt ended the same: a slow unravelling, a remarkable animal’s vast strength curdling inward, its shimmering blue-mapped body drifting in a pale echo of the life denied to them.

I do not deny they are vicious creatures. But it is not viciousness that makes it impossible for them to survive in the aquarium. The old fables and new films, the man-eater, the blood-frothed wave, the lurking fin, have all mistaken the matter entirely. The thing that kills the great white shark in captivity is the billowing cage: the narrowing circle of water, no current to guide their gills, the confiscation of the horizon. In captivity they turned to self-excoriation, scraping themselves to ribbons on the glass, starving in protest, dragging their bodies into stillness. As if potential had been so thoroughly written into their marrow that the denial of it was a kind of murder. What we mistake for noble resilience is in fact the beginning of a long derangement. A creature built to know the endless universe, driven into madness by the closing-in of incomprehensible walls.

And so we, in our hunger for marvels, have reduced an oceanic immensity to an ornament, a sole symbolic bangle on a slender wrist, a riddle turned spectacle. In that act of enclosure, the essential vastness of the creature is stripped away, its thousand-mile wanderings and salt-scored pilgrimages compressed into a parody of itself in a ghost story projected on glass.

What is offered to the crowd is no shark but the space where a shark once was: a wonder gutted and repackaged, its enforced silence masquerading as our unspoken understanding even as a scream writhes in every bubble.

As we behold the captive great white shark, Arwen, we do naught but applaud its absence in our lives, gild the blade which vanquished its truth, and heave a sigh of relief for the barrier between ourselves and the beast. We build shrines to the wonders we swallow whole. We raise gardens tomorrow from the cities we raze today.

But perhaps there is light on the horizon for humanity. Perhaps one day, we will learn how to keep the great white shark in a cage. And in turn, maybe it will learn how to rasp itself down for the onlooker and pace circles into borrowed water, each turn narrower, each wall closer than the last. What is witnessed is not the beast but its mutilation, a spectre stripped of horizon and turned inward on itself, a hollow spectacle mistaken for a radiant life.

The tank allows for neither possibility nor invention, and so the tale of the great white shark contracts into a pattern of bruises, the persistence of a body against limits it was never meant to know. The water becomes a neverending sentence, telling the story of a ruin that can only end in its own undoing. I wanted to be a writer, Arwen. I have always wanted to be a writer. You know that. You have always known that. And yet anything I ever write will only ever be an un-writing of the things other people have already written of me. Even my letters to you.