If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Douglas Adams wrote, "Anything that is in the world when you're born is normal and ordinary and is just a natural part of the way the world works. Anything that's invented between when you’re 15 and 35 is new and exciting and revolutionary and you can probably get a career in it. Anything invented after you're 35 is against the natural order of things."
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
I think about this quote whenever I get angry at the technology around me. When I rail against the Great Enshittening, am I simply committing the sin of nostalgia ("Nostalgia is a toxic impulse" -J. Hodgman)? I am, after all, old.
I've written before how conservatives' yearning for "simpler times" is really just a wish to be a child again. The reason times seemed simpler during your childhood is that you were a child, and if your parents did their job, they shielded you from a lot of the complexity of their adulthood so you could enjoy your childhood:
That's where the "National Customer Rage Survey" comes in. It's been surveying a panel of 1,000 representative consumers every three years for a decade, continuing a research project that started in 1976. The survey measures respondents' attitudes towards the businesses they deal with, and as of 2025, it's fair to say, customers are pissed:
We're experiencing more problems with the products and services we use. Those problems are more severe, they make us angrier, and they produce lingering stress. More and more, we are seeking revenge on the businesses that piss us off.
So it's not just me, an old man yelling at the cloud. The world is getting shittier.
The latest Customer Rage Survey inspired The Guardian's Heather Timmons to launch a new investigative series looking at how fucked up everything is. Her inaugural installment is very good, and it's drawn a massive reader response:
I spoke with Timmons this week about the series. She told me she's been deluged with emails from readers who feel that the world is different now – and many of them cite my work on enshittification. Timmons wanted to know what advice I had for her readers. I told her that I don't think you can solve this as a consumer, because this isn't a market problem, it's a political problem, and shopping isn't politics:
Later, Timmons forwarded one of those emails to me. It gave an eloquent and evocative account of just how rancid the vibe is these days. The writer said that when they and their spouse encounter this rot, they cite Stephen King's Dark Tower novels, quoting the oft-repeated phrase from that series: "The world has moved on."
At this point, I should warn you that the following contains some Dark Tower spoilers, so if you're planning to read a decades-old (but very good) dystopian western/science fiction crossover series, and if spoilers bug you, this might not be the essay for you.
Spoiler alert!
Still with me? OK, then.
In the Dark Tower novels, we crisscross a fallen world in which decay is all around us. The buildings are rotten, the machines have stopped working and no one knows how to fix them, babies and livestock alike are frequently born with deadly congenital defects. Much of the world has fallen into wasteland, cracked and barren. An army of wreckers, led by the demagogue John Farson (who styles himself "The Good Man") are slowly but surely conquering the land, laying waste to those few remaining outposts of civilization and conscripting the young men in the conquered lands to march on their neighbors.
It wasn't always this way. There was a time when the world was defined by hope and virtue and light, when the machines were fixed and the crops were harvested. Life wasn't golden – there were still squabbles and sorrows and even wars – but life was good.
And then the world moved on.
For reasons that no one truly understands, the normal push/pull of decay and renewal turned into a one-way, irreversible process in which everything that crumbled or snapped or burned up couldn't be repaired or replaced or recovered. Our mysterious ability to beat back the Second Law of Thermodynamics – an absurdity we probably should have always treated as an aberration – has collapsed. The world has moved on.
The Dark Tower series is a long, long, long Bildungsroman, with many detours through the life-stories of the characters in the ensemble cast, as well as the biographies of many of the figures they meet along the road. It's mostly an adventure novel, as road-trip tales tend to be, but those character studies and the lore that they surface – from our world and theirs – creates an overwhelming, many-layered, richly textured sense of loss and worse, of despair. For the world has moved on, and despite the love and care and bravery of many of the people in that world, the world cannot be redeemed. Each terrible day of those people's lives is the best day of the rest of their lives. From here on in, it only gets worse.
When Timmons' reader and their spouse greet every fresh depredation in modern life – hours on the phone with customer service to resolve a billing error that the company repeats every month, say – with "the world has moved on," they are invoking something heavy. This isn't just a rancid vibe, it's the fucking end-times.
For all that the Dark Tower novels are a series of cracking adventures and thoughtful character studies, they are also a mystery. Over and over again, we are made to ask ourselves, why has the world moved on? Was it John Farson and his army? Was it the Man in Black, the evil wizard whom the book's protagonist has pursued across time and space? Was it the Crimson King, the evil force whom the Man in Black serves?
Well, yes – and no.
Midway through the novels, we learn that the Crimson King and his evil minions have laid siege to "the beams," vast ley-lines that span the universe and provide the force that pushes away entropy, creating breathing room where repair and care can live. "All things serve the beams," we're told. The beams are the organizing force of the universe, the answer to the riddle of how such pitiful things as we could have fought back remorseless entropy for so long. By attacking the beams, the villains of the series have all but snuffed out that force, and so the world has moved on.
When I read that email and the invocation of the Dark Tower, I was immediately struck by how apt this comparison is. Because, as I've written many times, there were always enshittifiers who would have plundered your data and money and treated you with naked contempt:
There were always enshittifiers, but those enshittifiers faced external forces that checked their wreckers' urge. They were held in check by competition, and regulation, and workers' sense of fairness and duty, and by the threat of new products and services that might pop up to correct the defects they deliberately introduced into their products by enshittifying them.
And the foundation – the Dark Tower upon which all the beams converged- was antitrust enforcement, grounded in the idea that we could not afford to let any company – not a "good" company, nor a "bad" company – get so large that it could no longer be regulated, lest its executives become "autocrats of trade":
The same people who laid siege to antitrust law would later come after all forms of checks and balances. These are the people who gave us the "unitary executive" and Project 2025, and the collapse of accountability that has allowed the worst people to commit the gravest sins they could imagine and still reap vast fortunes. These beam-breakers wanted kings, and they got them.
I collect definitions of "conservatism," and one of my favorites comes from Corey Robin's book, The Reactionary Mind. Robins asks how it is that we can call so many disparate, irreconcilable ideologies – various ethno-nationalisms, imperialism, financialism, patriarchy, Christian nationalism, libertarianism, white supremacy, etc – "conservative"? What binds all these views together?
Robin's answer: the foundation that all these otherwise disparate views share is that some people are born to rule, while others are born to be ruled over. When these lesser people are elevated to positions of power, their inferiority creates a system of misrule, by which we all suffer. The best outcome for everyone is for us all to know our place and defer to our social betters.
That's why conservatives are obsessed with affirmative action, DEI, and any form of anti-racism. For them, the discriminatory outcomes we see in the wild are natural, reflecting the in-born defects in the people at the bottom of the social order. That's why, after every plane crash, every collision between a cargo ship and a bridge, every spectacular corporate bankruptcy, conservatives race to uncover the race, gender, religion and sexual orientation of the captain, the pilot or the CEO.
If the person who oversaw the catastrophe has anything remotely resembling a marginalized identity, then this is loudly trumpeted as confirmation that "diversity hires," promoted above their station, are ruining our society and wrecking our bridges. Naturally, if the person in charge was a wealthy, well-born, straight white guy, that's just proof that shit happens – it definitely doesn't prove that white straight guys, as a class, should be removed from positions of power.
For conservatives, virtue is "whatever the people who are born to rule desire." Hence Frank Wilhoit's definition of conservativism, "exactly one proposition, to wit: There must be in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect." It's not a crime if the president does it. It's also not a crime if your boss does it, or if a monopolist does it, or if ICE does it. It's not a crime if the IDF do it, or if the Epstein Class do it. "Taxes are for the little people":
The attack on antitrust law was part of the attack on the rule of law, the campaign to put everyone back in the their place. It's a piece of the effort to establish a new hereditary aristocracy, and every hereditary aristocracy requires heredity serfs (that would be us):
The ideology of economism – which says that market outcomes are the only way to govern a society – cashes out to "the strong do what they can and the weak suffer what they must." If we interfere with mergers, or labor practices, or commercial conduct, we "distort the market," which is literally going against nature:
That's why Trump dismantled the consumer protection agencies, the antitrust agencies, the labor protection agencies, the environmental protection agencies. When someone in power cheats the system, that's not a crime, no matter how many people they rob, maim or kill. As Trump told us on the debate stage in 2016, that kind of cheating "makes me smart":
That's why Elon Musk (almost) got to force every pension saver in America to bail out his money-incinerating AI business and his failed social media takeover – because the rules that protect everyday investors are "for the little people." Musk's mistake was trying to get a bunch of billionaires to hold the bag, too. The one form of systemic violence our society will not tolerate is trillionaire-on-billionaire violence:
The world has moved on. 50 years of neoliberal rule has weakened and snapped the beams – the rule of law, consumer and labor rights, civil rights – that radiated from our Dark Tower – antitrust law, which blocked the emergence of the "autocrats of trade." The people who besieged these beams had the same motives as the Crimson King and John Farson and the Man in Black: they were willing to pay any price for a world free from consequences for people like them. They knew they were born to rule, and that the rules were "for the little people," that breaking those rules "made them smart."
They wanted "bossism." Or, as rendered in the original Afrikaans, "baasskap," which means, "the social, political and economic domination of South Africa by its minority white population":
https://en.wikipedia.org/wiki/Baasskap
Not for nothing, baasskap is the foundation of Muskism, the ideology that Elon Musk epitomizes, even if he can't articulate it:
In "The Utopia of Rules," the late David Graeber described how neoliberal deregulation produced exactly the kind of state that we were warned we'd get under communism. Thanks to monopolies, all the stores were the same and they all sold the same goods. Thanks to the dismantling of labor protection and unions, no one had enough money to get by. Thanks to elite impunity, we were ruled by monsters who committed crimes in the open and thrived as a result. Thanks to unchecked greed, we paid everything we had for healthcare, only to be denied treatment when we needed it. Thanks to the dismantling of the welfare state, more and more of us had to wait in long lines to fill out absurdly long forms in triplicate. Thanks to the intrinsic instability of such a terrible system, more and more of us ended up in prison, and protest became more and more illegal:
Graeber pointed out that the rise of the web made it seductively easy for people in authority to force us to fill in forms. When analog bureaucracies impose paperwork costs on us, they also impose paperwork costs on themselves, because processing and filing those forms requires substantial effort, even if filling in those forms requires even more effort from us.
When it comes to virtual paperwork, the asymmetry is even more pronounced. Sure, it takes some admin to set up an online form and write the scripts to process its outputs, but that's a one-off. The form-giver can perform a very little admin and still impose a giant, repeated admin burden on the rest of us.
AI has only made this worse. Now, thanks to vibe coding, everyone can produce a form and its associated processing and analytics back-end with prompts, which creates a grave moral hazard. The kinds of activities that I used to fill in a single short form to accomplish now requires ten lengthy forms, created by different people in the same organization, all asking for variations on the same information. Through AI, we have democratized bureaucracy. It's Kafka-as-a-service.
What's more, when you're dealing with a monopoly, you have no choice but to complete whatever paperwork they throw at you. And when the vibe-coded back-end scripts shit the bed and lose or misinterpret your data, you have no choice but to endure an infinite telephone hold queue (if you're lucky) or get shunted to a customer service bot (if you're unlucky):
It's entirely possible to build webforms that are thoughtful, fast, respectful of our time, and well-processed. The problem is that fielding these forms requires that the form-giver undertake some intensive, moderately expensive work (once), while skipping this step merely requires that we all perform intensive, time-consuming work (over and over and over again):
https://mohkohn.co.uk/writing/html-first/
This is how we end up with government forms that require you to list every trip you have ever taken to the USA, since your infancy, with every flight number, which you can only get help with by talking to a chatbot that emails you an out-of-date PDF no matter what question you ask of it:
This is how we end up with massive customer service queues, long lines at tills, and no one at the gate to answer your questions when your flight is canceled. Understaffing is a form of enshittification, one that shifts value from shoppers to owners, and shifts consequences from owners to workers:
This is how we end up with broken machines that no one can fix. Firing workers and replacing them with chatbots or contractors means incinerating their process knowledge – the precious, inchoate, unrecorded understanding that keeps everything working:
This is how companies that make products we love suddenly decide to wreck those products: when the only consequences for shitty products is angry customers with nowhere to go and no one to vent their rage upon except workers who have no labor rights and can't afford to quit, why not do a mafia bust-out for every business?
The world has moved on. Nothing works. Everything costs too much. No one can help. No one knows how to fix anything. The beams were broken by the Crimson King and his economism-crazed minions. The Dark Tower might fall.
So what consumer advice do I have for people who are angry about this? I don't have any consumer advice, I'm afraid. You can't shop your way out of a monopoly. Once again, shopping is not politics.
What I have for you is political advice. To restore the beams and beat back entropy again, we need a better system, not more virtuous individuals. If you feel – as I do – that "the world has moved on," then to wrench it back, you will have to join a polity. Support activist groups like the Electronic Frontier Foundation, the digital rights group I've been at for the past 25 years:
https://supporters.eff.org/donate/join-eff
Join a union. If there's no union at your jobsite, start a union. If you work in tech, you start this process by talking to techsolidarity.org and the techworkerscoalition.org. In the UK, get in touch with United Tech and Allied Workers:
https://utaw.tech/
Get involved in party politics. Find a political party whose local organization supports your values (even if the national version of that party sucks) and then work with your fellow grassroots activists to drag or replace the party leaders. Get involved in local politics: if there's one thing Moms For Liberty has taught us, it's that unregarded, seemingly unimportant local offices have enormous potential to change facts on the ground for the people where you live. Those changes don't have to be change for the worse.
Doing politics is hard. Hell, after all, is other people. It would be great if we could make change by changing ourselves, but that's not how any of this works. The world has moved on, and you can't save it. But together, we can restore the beams and beat back entropy. Hell is other people, but only because other people are so great but it's so hard to figure out how to work together. We can do it, though. We did it with the post-war settlement, the 30 glorious years when we built the welfare state, regulated polluters and bosses, and kicked off the civil rights movement. We did it then, and we can do it again. We must. All things serve the beams.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
Google AI Overview court loss in Germany could spell doom for AI search industry.
The AI hype crowd really wants to have it all.
This is a unique tool we made, but we're not liable for its performance.
It's amazing and can do it all and replace search and replace all other tools, but also you should know it's not accurate or trustworthy.
This is a unique individual that is expressing speech and can think and act independently, but also this is a thing we made and we own it it's ours and WE are the ones providing value.
We're not at fault when it lies because it's just things it took from the internet where we scraped training data as well as live responses, but also this is totally not copyright infringement because we didn't STEAL the data we looked at it and made something novel!
This can replace developers and artists and analysts and scientists and writers and experts of all kinds...but also you need an expert to check everything it makes because the mistakes look plausible until you dig deeper.
The worst thing is that AI is genuinely interesting, including large generative models. It has real research applications, especially in places like drug discovery or protein analysis where there's a lot of educated guesses before you can even get something worth testing. I also love the philosophical questions about what it means to be conscious or alive or whatever and how we've had to completely re-evaluate the Turing test and the Chinese Room and dozens of other thought experiments.
The absolute bullshit companies are trying to pull here? Ruinous. It's likely to collapse and set AI research back for decades when the term becomes taboo for a proposal. If it doesn't it will continue to erode every field including AI research with this insistence on hype and exploitation, leaving us with literally everything set back decades because we can't produce anything new if we're statistically replicating the things that already exist.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
"Gish Gallop" is the debating term for an opponent who makes so many claims that "it's impossible to address them in the time available" (it's named for Creationist Duane Gish, who was notorious for this tactic):
https://en.wikipedia.org/wiki/Gish_gallop
I think about the Gish Gallop whenever I'm asked to comment on AI.
Here's a recent example: last week, I had a pre-interview call with a radio producer who wanted me to come on a 13-minute segment to discusses "whether there's a problem with AI governance?"
I asked what the show meant by that: was it whether regulation of AI in commercial or public sector decision-making needed more oversight? Was it that the siting and provisioning of data-centers needed more democratic accountability? Was it that workers deserved more of a say in AI's impact on labor markets? Was it that customers and/or audiences should be able to opt out of AI customer service and AI slop? Was it about whether we needed some kind of system to prevent "runaway AI," in the event that we teach so many words to the word-guessing program that it wakes up, becomes God, and turns us all into paperclips?
"Oh," the producer said, "all of that."
In 13 minutes.
You see the problem, right? The AI industry has made so many claims about its past, present and future that it's almost impossible to have a reasonable critical conversation about it:
Shortly after I did the radio show, a newspaper editor who'd heard my segment got in touch to ask me if I'd write an 800-word op-ed about the subject, and also, could I address claims that "AI is the next Industrial Revolution?"
I keep finding myself on stages or panels where an AI-struck person says something like, "AI is the next industrial revolution. It will change everything we do. It will let anyone create important works of art. It will cure cancer. It will take us to space. It will solve the climate crisis."
Or sometimes it's an AI critic, but that person's criticism is really more "criti-hype," which is when you accept tech industry hype claims at face value, and then criticize them rather than questioning them:
AI criti-hype might ask what we'll do once AI takes all our jobs, or what we'll do when AI replaces the government or teachers or doctors, or what we'll do when AI can bypass our critical faculties and brainwash us or drive us all mad.
What do you say to that? I usually start by talking about whether there's any economic basis for keeping the AI servers running. AI is – by far – the money-losingest venture in human history, and it's practically impossible to overstate just how bad the AI business is. Not only does AI have terrible unit economics, those unit economics are getting worse over time:
AI's happiest customers cite cost-benefit calculations that depend on truly unimaginable subsidies from the AI companies, who are basically selling $100 bills for $5 apiece. It would be pretty amazing if you couldn't find people who'd extol the virtues of this arrangement. But when AI companies try to raise the price of those $100 bills to, say, $20 apiece, those ecstatic customers fly into a rage and start loudly proclaiming that AI is so inefficient that they will lose money on this arrangement:
Now, it shouldn't fall to me, a card-carrying member of the Democratic Socialists of America, to point out that capitalist enterprises require profits to be sustainable. You can't keep a business afloat by selling $100 bills for $5, nor for $20. You can't even make a profit selling $100 bills for $100 apiece! For a company to succeed, it needs to take in more than it expends.
AI is a money-furnace, and AI hustlers are clearly on the hunt for a way to force all of us to feed every dime we've got to it. Elon Musk's (now scuttled) gambit to make every pension saver in America bail out Grok (and Twitter, but at a mere $44b, the losses from Twitter are dwarfed by the titanic losses from Grok) was the most ambitious and shameless population-scale bag-holder scheme, but it's not the only one:
So before we ask about the capabilities AI will acquire in the future, we should at least give some consideration to the question of whether anyone will be willing to fund the development of those capabilities, and if so, where the money would come from? Likewise, before we ask whether AI can perform adequately in a job, we should at least consider the possibility that the company that sells that AI tool will be bankrupt in a year or two. When we fight about data-center buildout, we mostly talk about the (considerable) environmental downsides to them – but what about the question of what we will do with these data-centers after their owners go bankrupt, possibly even before they can be provisioned with electricity? How many laser-tag arenas do we actually need?
This is just one example of the questions that you could spend days unpacking, which make many of the other questions about AI a little silly. Like, even if you think there are limitless returns to scale for creating new AI capabilities, which means that if we keep the money-furnace burning it's only a matter of time until it powers a cure for cancer and the end of the climate emergency, how much money do we need to shovel into the furnace before that happens, and where will it come from? There are plenty of cancer researchers who have promising approaches they haven't been able to pursue due to funding shortfalls.
Unless there's some way to estimate how much money we have to give to AI companies before they cure cancer, we should at least consider the possibility that the true sum is "more money than exists now and that will ever exist." We should also consider that whatever benefits to cancer research that AI might deliver could come with a higher price-tag than the promising cancer research we're dropping because we can't find far more modest sums.
Likewise, it may be that the amount of CO2 that AI will generate atmosphere before it "solves climate change" will render Earth permanently unfit for humans, consuming the only habitable planet capable of sustaining human life in the known universe. I mean, I suppose that's one way to "solve" climate change, but it's a pretty drastic solution.
My next book (out later this month) is The Reverse Centaur's Guide to Life After AI. I wrote it because I was frustrated by other people demanding that I talk to them about AI, and then handing me 800 words or 13 minutes to address fifty nebulous, poorly supported claims about AI:
Now that I'm about to go out on the road with the book, I find myself frustrated anew by the need to try and pull together a compact way to address the broad, incoherent claims the industry uses to keep its bubble inflated and the money furnaces roaring. The series of essays I've developed here on Pluralistic are part of that effort:
But it occurred to me that this whole enterprise of making sense of AI needs to be framed in the context of the messiness of AI itself, and AI boosters' overwhelming, promiscuous and disjointed Gish Gallop.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
imagine if you will, a fairly dry survival crafting game in which you live in a bunker and must periodically venture out to scavenge food, set up turrets for attacking monsters, etc
now, your computer inside the bunker has a game-inside-a-game on it which is a charming farming sim of undeniably greater quality and scope than the survival game you're playing. therefore, the object of the game becomes to keep your bunker secure so you can play the farming game more.
now, once you achieve the highest rating in the farming game, a secret shop inside it unlocks, and one of the novelty items you can purchase is a game console, giving you access to games-inside-a-game-inside-a-game. most of the games for it are typical mobile shovelware, but one of them is a highly polished, extremely brutal precision platformer with amazing level design and production values exceeding that of the survival game and farming sim combined.
it is only at this point that the purpose of this entire contrivance becomes clear: to create the most deranged speedrun community the world has ever seen.
what’s important to note and missing from the “headline” tweet is that they simultaneously constructed additional good public transit to the public transit already in the city (bus rapid transit, train stations). Just removing highway alone isn’t going to make traffic better, the bigger part of the story is that they improved public transportation. And the current mayor wants to do more - cyclist lanes and reinstate a tram system
“Because the truth is, tech doesn’t have an image problem. It doesn’t have a message problem. It has an intention problem. What’s wrong with the axe murderer who broke into my house is not that he hasn’t successfully persuaded me to buy into his narrative. What’s wrong is that he’s trying to kill me with an axe. Similarly, when you launch a product that’s designed to put millions of people out of work, block access to sources of verifiable truth, replace human creativity with slop, and lower the barriers to every sort of atrocity, the problem isn’t that you haven’t told the public a good story about those things. The problem is that you are trying to do them.”
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
Lawmakers push DoD to tighten smartphone controls after adversaries exploited commercial tracking data
Unfortunately, the US government seems to be taking the view "we need to restrict what our personnel can do more" and "we need to set up our devices differently" than the actual real answer which is to fucking stop ubiquitous non-consensual data harvesting. This is a problem for military sure but it is also a major issue for civilians. Literally since day 1 this has been an issue for everyone.
Maybe the EU will be smarter and regulate this shit.
Text of tweet under the cut because it is loooong.
But... Stochastic Parrots.
Timnit Gebru was fired from Google in December 2020 for refusing to retract a research paper, and every single warning that paper made about large language models has now happened at a scale the industry spent 4 years trying to make people forget about.
Her name is Timnit Gebru.
She co-led the Ethical AI team at Google. She co-wrote a paper called "On the Dangers of Stochastic Parrots" with Emily Bender at the University of Washington and two other researchers. The paper was 14 pages long. It was submitted to a top AI ethics conference. And it was the reason Google decided that one of the most senior Black women in AI research could no longer work there.
The story Google told publicly was that she resigned. The story she told, confirmed by 2,695 of her colleagues in an open letter, was that she was fired by email while on vacation because she refused to either retract the paper or remove her name from it.
The paper had not even been published yet.
Here is what she actually wrote, and why every prediction inside it has now come true.
The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. They called these systems stochastic parrots because they would repeat patterns from training data with statistical confidence and zero comprehension. The paper predicted that this apparent intelligence would fool both users and developers into trusting outputs that were structurally incapable of being reliable.
This was 2020. GPT-3 had just come out. The paper predicted the hallucination problem before anyone had a word for it.
The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it, because the optimization process rewards confident outputs, and confidence in language patterns tracks frequency in the training set.
The prediction was that hiring tools built on these models would discriminate against women. That healthcare triage tools would underperform on Black patients. That loan approval systems would entrench inequality while presenting their decisions as neutral algorithmic judgment.
Every one of those things has now been documented in deployment.
Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.
The third warning was about environmental cost. The paper calculated that training a single large language model produced emissions equivalent to the lifetime output of 5 cars. The prediction was that the race to scale would create an environmental footprint that would eventually rival entire industries.
In 2024, Google's emissions were up 48% from 2019, and the company explicitly blamed AI infrastructure. Microsoft's were up 29%, same reason. Both companies have now quietly abandoned the climate commitments they were publicly celebrating the year Gebru was fired.
The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit. Nobody at Google, OpenAI, Meta, or any other lab could tell you with confidence what was in the data their models were trained on. This was not a temporary problem to be solved later. It was a permanent feature of the approach.
In 2023, researchers discovered that the LAION-5B dataset, used to train Stable Diffusion and other major image models, contained thousands of images of child sexual abuse material. The companies that had trained on the dataset had no way of knowing. The paper predicted that category of failure 3 years before it was found.
The fifth warning was the one Google cared about most.
Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them. The internet would become a place where the dominant voice was a statistical average of dominant voices, presented as a neutral assistant. Languages underrepresented in the training data would degrade over time as more web content was generated by these systems and fed back into the next training run.
This is now happening in real time. A 2024 study found that 57% of new web content in English is AI-generated or AI-assisted. Researchers studying low-resource languages have documented active degradation in translation quality, because the synthetic content fed back into training is itself worse in those languages.
The paper Google fired her for predicted the model collapse problem before model collapse had a name.
The mechanism behind why this all happened is the part of her work that nobody quotes.
Gebru's argument was not that AI is dangerous in some abstract sci-fi sense. Her argument was that AI is dangerous in a very specific structural sense. The technology was being built by a small group of researchers who shared similar backgrounds, worked at similar companies, and were rewarded for shipping products faster than competitors. The incentive structure made it impossible for safety, ethics, and bias concerns to slow anything down. Anyone inside the system who raised those concerns was either ignored, sidelined, or removed.
She was making that argument from inside Google.
Then Google proved her right by removing her.
The team Google had built to make sure their AI was safe was dismantled in 90 days because they did the job they had been hired to do. Margaret Mitchell, the other co-lead of the Ethical AI team, was fired two months after Gebru for searching through her own emails for evidence of how Gebru had been treated.
Gebru did not stop. She founded DAIR, the Distributed AI Research Institute, in 2021. The mission is to do AI research outside the control of the companies that have a financial interest in not hearing the answers.
Every prediction in the Stochastic Parrots paper has now been validated by deployment. Hallucinations are an industry-wide problem the largest labs cannot solve. Bias amplification has been documented in hiring, healthcare, lending, and criminal justice. Environmental costs are larger than entire small countries. Training data audits remain impossible. Model collapse is an active research crisis at every major lab.
The question worth sitting with is the one almost no one in the industry will say out loud.
Every researcher with the technical credibility to call out these problems watched what happened to her in December 2020 and made a calculation about their own career. The number of people willing to speak publicly about safety and ethics issues inside the major AI labs collapsed after that firing and has not recovered.
The researcher Google fired for warning about exactly what is now happening was right.
The company that fired her is now the second-largest deployer of the technology she warned about.
And the people inside that company who agree with her are not allowed to say so.
I think this is just a trend everywhere but I've been very frustrated this week by how much admin work is being outsourced to me as the patient/customer.
My orthodontist tells me I can make an appointment with the surgeon. I call the surgeon. They tell me I need a new referral. I call the orthodontist. They do a referral. I call the surgeon. Referral didn't come through. They tell me about their special unique system we have to use. I call the ortho again and walk them through the referral. I call the surgeon. They say the referral was missing some details so they have to do it again. I call the ortho.
The insurance company calls me about repair shops. I give them the name of the repair shop which I already gave them yesterday. They say they're not in their system but I can use them, but I have to call the repair shop to ask them to contact the insurance company. I call the repair shop and they say the insurance company is supposed to email them.
I feel like at a certain point these constant fetch quests become unreasonable?? Is it too much to expect these groups to communicate with each other instead of making me run back and forth between them???
Made this post and then the new property manager (who started on Monday and only finally emailed us today because I sent a vaguely professionally hostile email to her boss because I hadn't heard anything and was not convinced she existed) asked for a list of open action items which her predecessor should have had but apparently wasn't keeping track of, which I learned when I met her boss and provided her with the list of open action items, which I guess tragically died in a fire in the last 2 weeks since she was sitting at my kitchen table, being menaced by the skull. How many people's jobs am I doing now
My next book is The Reverse Centaur's Guide to Life After AI, out next month. Pre-order it now, including as a DRM-free audiobook or ebook, at my Kickstarter, and help me continue to prove that DRM-free isn't just the right way to reach an audience, it's also the best way to reach them.
From the earliest days of technopolitics, the role of technology in resisting authoritarianism was unclear. On the one hand, there's the indisputable fact that modern cryptography, properly implemented, can deliver a degree of privacy that is proof against all technological attacks.
That is to say, if you pull out your distraction rectangle, fire up the camera, and tap the shutter button, in the ensuing eyeblink instant the image you've captured will be scrambled so thoroughly that it could never be unscrambled without the secret key unlocked by your passphrase or biometrics. Even if every hydrogen atom in the universe were converted into a computer, and even if all those computers spent all the time between now and the end of the universe trying to guess what the key was, we would run out of universe and time long before we ran out of possible keys.
What's more, this extremely robust form of scrambling and descrambling can be combined with other techniques to block tampering with the encrypted data, and to allow parties to reliably identify who scrambled the data and also to restrict who may unscramble it. These remarkable technological facts have inspired many excited debates about what they mean for our politics, most notably among a group of people who called themselves "cypherpunks":
One cypherpunk faction believed that modern cryptography could enable a kind of technological secession: by allowing ordinary people to communicate, transact and collaborate without the possibility of state interception or control, crypto could make states themselves obsolete.
But another faction pointed out that no amount of mathematics could help you if an agent of the state – or a criminal the state failed to protect you from – tortured you until you revealed the secret passphrase needed to unlock your secrets. This was (ironically) called "rubber hose cryptanalysis" (as in "Tell me your passphrase or I'll hit you with this rubber hose again"). Later, this became known as a "wrench attack" after a famous XKCD comic about $1m worth of security technology being defeated by hitting someone with a $5 wrench until they divulged the password:
https://xkcd.com/538/
Once you stipulate to the problem of wrench attacks and rubber-hose cryptanalysis, it becomes apparent that your cryptography is only as good as your physical defenses. What's more, the most effective physical defenses we have come from a strong rule of law, because even the thickest safe door benefits from the threat of prison for anyone who breaks into the safe, and the most effective tool for preventing a cop from hitting you with a rubber hose is the existence of a judge who can send that cop to prison for abusing your civil rights.
But what do you do if you already live under tyranny? The rule of law is a great defense, but cryptography alone can't bring about the rule of law. What is the role of technology in this foundational struggle?
My technopolitics faction – the faction associated with the Electronic Frontier Foundation, where I've worked for a quarter-century – has an answer: the role of encryption is to provide a measure of privacy and security that is best used to organize political struggles to demand the rule of law and respect for human rights. Encryption isn't proof against rubber hoses, but it is effective against many other forms of state repression, and it can provide a technical edge for those engaged in a political struggle.
Another faction – the faction most associated with bitcoin and subsequent cryptocurrency projects – rejects the role of the state altogether, and seeks to replace states (and state-regulated institutions like courts and banks) with mathematics. Rather than asking courts to interpret contracts, we can put our trust in self-executing "smart contracts," and rather than asking banks to safeguard our financial integrity, we can use cryptographic software to ensure that money only moves when the person it belongs to tells it to.
This has many problems. Smart contracts are slow, expensive, and unreliable. The number of people who understand contracts is small, the number of people who understand the software that embodies smart contracts is likewise small, and the Venn intersection of the two is more of a sphincter. What's more, there is irreducible ambiguity in all but the simplest of contracts, which means that even a "self-executing" contract ends up relying on a human adjudicator (an "oracle") who can be bribed or intimidated into cheating:
And when it comes to transactions, crypto proves to be unwieldy, expensive and complex, so that nearly all crypto users end up directing an intermediary (like Coinbase) to hold and move their cryptographic assets for them. The upshot is that cryptocurrency mostly replaces banks – imperfect, but heavily regulated and insured – with unregulated tech platforms with murky ownership and often defective security procedures, who may or may not be insured (or even locatable) in the event of a collapse or a breach. Consequently, cryptocurrency has become a scam magnet of unprecedented and unstoppable power, and hardly a day goes by without people being ripped off in the most ghastly ways imaginable:
https://www.web3isgoinggreat.com/
For bitcoin maxis and other anti-state cypherpunks, this is just a skill issue. Anyone who doesn't understand how to manage their own keys and turns to a platform to hold and move their crypto is getting what they deserve. As the maxim goes, "Not your keys, not your wallet," which is cypherpunkspeak for "caveat emptor."
That's where the wrench attacks come in. Because if you are in possession of keys that can be used to irreversibly and instantaneously steal large sums of money and move it to jurisdictions where the perpetrators are beyond any legal or physical recourse (e.g. North Korea), then there is a massive incentive for your adversaries to kidnap you and hit you with a wrench or a rubber hose.
That's precisely what's going on. People with substantial cryptocurrency holdings face grave personal danger, and the physical attacks on their person grow bolder, more violent, and more sadistic by the day:
As crypto critic David Rosenthal writes, this problem is even worse than it seems at first blush:
https://blog.dshr.org/2026/05/wrench-attacks.html
For one thing, cryptocurrencies depend on "public ledgers" that indelibly, publicly record every transaction in the network. Cryptocurrency is nothing without these ledgers, and they have to be immutable and public to work. This is very bad news for anyone who relies on anonymity as their defense against physical attacks.
That's because "reidentification attacks" (where an anonymous person in a dataset is positively identified) get easier to perform over time. You might be represented in a database of hospital prescribing activities by a random number, and that number might be hard to associate with your real identity…at first. But with every subsequent release of data – whether in the form of an anonymized data-set or a breach – it gets easier to cross-reference the facts associated with your record with other facts from other records, such that a detailed, identifying picture of you emerges one fact at a time.
For example, if the taxi company you use suffers a breach that reveals journeys associated with every doctor's appointment at the hospital, now an attacker can pick out the home or work address of the single person who visited the hospital just before you received your prescription. The longer an "anonymized" data-set sits around in public view, the easier it gets to de-anonymize it:
Combine the fact that permanent ledgers make it progressively easier to identify people whom you can torture into revealing their crypto keys with the irreversible, instantaneous nature of crypto transfers and you get some very juicy targets indeed. "Not your keys, not your wallet" means it's "not anyone else's problem" when you get robbed. You can't ask the bank to interdict or reverse the transaction.
Rosenthal provides a litany of the escalating security measures crypto holders are turning to as this problem goes progressively more dangerous and terrifying. There's the guy who splits his keys up in four physical vaults at four separate locations, whose management is instructed to make him wait a minimum of seven days when he asks to retrieve them. Despite all this, he keeps his identity secret:
Rosenthal quotes Nicholas Weaver, who asks what kind of "internet of money" bitcoin can be if it can't be safely stored on a computer connected to the actual internet:
https://doi.org/10.1145/3208095
But an equally valid question is, what kind of escape from tyranny is it that requires you to hide your identity at all times lest you be snatched off the street and brutally tortured? What kind of "liberty" requires you to spend $860,000 armoring your two top execs' personal vehicles to protect them from gunfire and light artillery?
It costs $6.2m/year to protect Coinbase's CEO – "more than the combined amount that JPMorgan Chase & Co., Goldman Sachs Group Inc. and Nvidia Corp. spent on their respective CEOs":
Crypto true believers exhort one another to "HODL" (hold on for dear life). Selling your crypto during downturns is considered a moral failing. But now, crypto holders – especially those who manage their own keys – are literally holding on for dear life, as they are hunted by crime syndicates and state actors alike.
It's a good reminder of how badly crypto has failed on its own terms, delivering its biggest users into an existence of fear and physical peril that rivals the plight of even the most hunted dissidents in the most repressive societies. Worse: as cryptocurrency lobbyists have fused crypto with the world's largest and most corrupt governments (especially the Trump regime), crypto now has all the exposure to state coercion that made banks so unsuitable, but without the (inconstant, insufficient) protections offered by traditional banking.
And that's before we talk about the energy consumption problems, the scams enabled by crypto, and the rampant human trafficking that those scams necessitate:
People in my technopolitical faction have a saying of our own: "'Crypto' means cryptography." Cryptography plays a hugely important role in protecting people from crime and state repression. It is no substitute for the rule of law and democracy, but it remains a key tool for securing and defending both:
Cryptocurrency, on the other hand? That's the worst of all worlds.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
Text of tweet under the cut because it is loooong.
But... Stochastic Parrots.
Timnit Gebru was fired from Google in December 2020 for refusing to retract a research paper, and every single warning that paper made about large language models has now happened at a scale the industry spent 4 years trying to make people forget about.
Her name is Timnit Gebru.
She co-led the Ethical AI team at Google. She co-wrote a paper called "On the Dangers of Stochastic Parrots" with Emily Bender at the University of Washington and two other researchers. The paper was 14 pages long. It was submitted to a top AI ethics conference. And it was the reason Google decided that one of the most senior Black women in AI research could no longer work there.
The story Google told publicly was that she resigned. The story she told, confirmed by 2,695 of her colleagues in an open letter, was that she was fired by email while on vacation because she refused to either retract the paper or remove her name from it.
The paper had not even been published yet.
Here is what she actually wrote, and why every prediction inside it has now come true.
The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language. They called these systems stochastic parrots because they would repeat patterns from training data with statistical confidence and zero comprehension. The paper predicted that this apparent intelligence would fool both users and developers into trusting outputs that were structurally incapable of being reliable.
This was 2020. GPT-3 had just come out. The paper predicted the hallucination problem before anyone had a word for it.
The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it, because the optimization process rewards confident outputs, and confidence in language patterns tracks frequency in the training set.
The prediction was that hiring tools built on these models would discriminate against women. That healthcare triage tools would underperform on Black patients. That loan approval systems would entrench inequality while presenting their decisions as neutral algorithmic judgment.
Every one of those things has now been documented in deployment.
Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.
The third warning was about environmental cost. The paper calculated that training a single large language model produced emissions equivalent to the lifetime output of 5 cars. The prediction was that the race to scale would create an environmental footprint that would eventually rival entire industries.
In 2024, Google's emissions were up 48% from 2019, and the company explicitly blamed AI infrastructure. Microsoft's were up 29%, same reason. Both companies have now quietly abandoned the climate commitments they were publicly celebrating the year Gebru was fired.
The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit. Nobody at Google, OpenAI, Meta, or any other lab could tell you with confidence what was in the data their models were trained on. This was not a temporary problem to be solved later. It was a permanent feature of the approach.
In 2023, researchers discovered that the LAION-5B dataset, used to train Stable Diffusion and other major image models, contained thousands of images of child sexual abuse material. The companies that had trained on the dataset had no way of knowing. The paper predicted that category of failure 3 years before it was found.
The fifth warning was the one Google cared about most.
Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them. The internet would become a place where the dominant voice was a statistical average of dominant voices, presented as a neutral assistant. Languages underrepresented in the training data would degrade over time as more web content was generated by these systems and fed back into the next training run.
This is now happening in real time. A 2024 study found that 57% of new web content in English is AI-generated or AI-assisted. Researchers studying low-resource languages have documented active degradation in translation quality, because the synthetic content fed back into training is itself worse in those languages.
The paper Google fired her for predicted the model collapse problem before model collapse had a name.
The mechanism behind why this all happened is the part of her work that nobody quotes.
Gebru's argument was not that AI is dangerous in some abstract sci-fi sense. Her argument was that AI is dangerous in a very specific structural sense. The technology was being built by a small group of researchers who shared similar backgrounds, worked at similar companies, and were rewarded for shipping products faster than competitors. The incentive structure made it impossible for safety, ethics, and bias concerns to slow anything down. Anyone inside the system who raised those concerns was either ignored, sidelined, or removed.
She was making that argument from inside Google.
Then Google proved her right by removing her.
The team Google had built to make sure their AI was safe was dismantled in 90 days because they did the job they had been hired to do. Margaret Mitchell, the other co-lead of the Ethical AI team, was fired two months after Gebru for searching through her own emails for evidence of how Gebru had been treated.
Gebru did not stop. She founded DAIR, the Distributed AI Research Institute, in 2021. The mission is to do AI research outside the control of the companies that have a financial interest in not hearing the answers.
Every prediction in the Stochastic Parrots paper has now been validated by deployment. Hallucinations are an industry-wide problem the largest labs cannot solve. Bias amplification has been documented in hiring, healthcare, lending, and criminal justice. Environmental costs are larger than entire small countries. Training data audits remain impossible. Model collapse is an active research crisis at every major lab.
The question worth sitting with is the one almost no one in the industry will say out loud.
Every researcher with the technical credibility to call out these problems watched what happened to her in December 2020 and made a calculation about their own career. The number of people willing to speak publicly about safety and ethics issues inside the major AI labs collapsed after that firing and has not recovered.
The researcher Google fired for warning about exactly what is now happening was right.
The company that fired her is now the second-largest deployer of the technology she warned about.
And the people inside that company who agree with her are not allowed to say so.
