Google’s new remote attestation scheme is every bit as terrible as its old remote attestation scheme
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Long before "agentic AI," we had the idea that software would act as your agent on the internet. That's why the old-fashioned technical term for a browser is a "user agent." Your browser acts on your behalf to retrieve information and then show it to you, in the format you choose. It's your agent:
This is a powerful and profound idea. It is because browsers are our "agents" that we expect them to accept our directives, say, by blocking pop-ups, or by turning off autoplay sound, or by blocking commercial surveillance trackers:
https://privacybadger.org/
Your browser does all that because your browser works for you. The reason your browser can work for you is that the web is an open, standardized technology. In theory, anyone who follows the standards published by the World Wide Web Consortium (W3C) can make a browser, and that web browser can connect to any web server. Browsers and servers are interoperable. It's the same force that means you can put anyone's gas in your gas-tank, or anyone's shoelaces in your shoes, or anyone's milk on your cereal.
But what if manufacturers could dictate those choices to you? What if your light socket refused to use a lightbulb unless it was officially blessed by the socket's manufacturer? What if your dishwasher refused to wash your dishes unless you bought them from one of the manufacturer's "dish partners"? What if your toaster refused to toast "unauthorized bread"?
It's hard to see how a company could win its market with this strategy. After all, if the dishes are really better than the competition's, you'd buy them voluntarily, without any need for law or technology to force the matter. The only reason to make a dishwasher that refuses a rival's dishes is if the manufacturer's own dishes are ugly, expensive, and/or badly made.
But once a company owns the market – once they've achieved dominance by buying out their rivals; by bribing potential competitors to stay out of their lane; and by engaging in deceptive conduct to trap key suppliers and customers – they could cement their dominance by blocking interoperability, keeping out rival dishes, milk, gas, lightbulbs, shoelaces and bread, capturing their whole market and squeezing it.
That's what Google has done, and that's what Google wants to do more of. Google's commercial behavior has been so unethical, deceptive and abusive that the company just lost three federal antitrust cases:
They cheated app vendors, ripping them off with sky-high junk fees and onerous conditions that raised prices while lowering the share of your spending that went to the companies whose products you were paying for:
They cheated advertisers, rigging the ad market to gouge businesses on ad prices and underinvesting to fight rampant ad-fraud, sucking hundreds of billions out of the productive economy for overpriced ads that no one saw:
Google wasn't always this way. The "don't be evil" company owes its very existence to the open web ecosystem. When the company started to index the web in 1998, it was playing on an open field, where any web server could talk to any "user agent," even one whose user was a startup like Google, that was making a copy of every page on the server.
For years, Google thrived on the open web, and built open technologies. Android – the mobile operating system that Google bought in 2005 – was presented as an "open" alternative to existing mobile offerings, and as the mobile market collapsed into two companies – Google and Apple – Google always presented Android as the open alternative to Apple's "walled garden."
There were always ways in which Google's "open" Android wasn't exactly open. The company engaged in illegal "tying" arrangements that forced hardware vendors and carriers to lock out versions of Android that were created by Google's competitors:
In other words, even though Google offered a mobile platform that was (mostly) technically open, they used commercial and legal strategies to choke off the market oxygen for alternative Android versions that tried to capitalize on that technical openness.
But life finds a way. The existence of an open, modifiable, tinkerer-friendly mobile operating system meant Android hackers could create alternatives to Google's (de facto) walled garden, which thrived in the cracks in that garden wall. Operating systems like CalyxOS, PureOS and Graphene offered a more private, more secure Android experience, one that was largely "de-Googled," blocking Google's relentless acquisition of your private data:
https://grapheneos.org/
And Google's data-hunger is relentless. Android exfiltrates a chunk of your personal and behavioral data every five minutes. The "resting heartbeat" of Android surveillance pulses and pulses, irrespective of whether you're using your device, and the instant you unlock your screen, that heartbeat quickens, sending even more data to the company:
All that data has proved irresistible to authoritarian governments. Donald Trump's enforcers have seized on Google data as a vital source of information about the identity of protesters and the location of migrants hunted by ICE:
So there are plenty of reasons why users would seek out these de-Googled alternatives to Android, finding them in spite of Google's illegal commercial tactics to block access to competing technologies. The worse it got, the better those alternatives looked.
Perhaps this explains Google's years-long effort to increase the technical barriers to using modified versions of Android, beefing these up to match the commercial restrictions that stand in the way of a de-Googled existence.
Back in 2023, Google floated the idea of "Web Environment Integrity" (WEI), a set of modifications to web standards that would force your computer to disclose its operating environment to the web servers it connected to, even if you objected to this disclosure:
WEI was a form of "remote attestation." That's when your device uses a sub-processor (sometimes called a "Technical Protection Module" or "TPM") or a walled off part of its main processor (sometimes called a "secure enclave") to produce a cryptographically signed description of your device and its configuration: which hardware, software, plug-ins and settings you're running.
When you connect to a server, it demands that your device send this "attestation" before it handles your request. If your device won't provide this data, or if the server doesn't like (or recognize) your device and its details, it can refuse to deal with you. And because the attestation is prepared by a TPM or a secure enclave that you can't modify or override, you don't get to decide which facts about your device it's allowed to see.
Practically speaking, this means that remote attestation lets a server refuse to deal with you until you turn off your ad-blocker and your tracker-blocker. It means that the server can discriminate against users who block auto-play sound and video, who block pop-ups, who put the tab in the background when it's playing a mandatory pre-roll ad.
WEI was especially disturbing in light of Google's efforts to kill ad-blockers and privacy blockers through updates to Chrome, an effort that continues to this day:
These blockers are an important part of the dynamic between web publishers and their users. In the real world, when you get an offer, you can make a counter-offer. That's all an ad-blocker is: a way for users to respond to a server whose opening bid is, "How about you give me all your data and let me take over your computer in exchange for showing you this page?" with "How about 'Nah?'"
We didn't get rid of pop-up ads by making them illegal, or by boycotting advertisers who used them. We got rid of pop-up ads when web users installed pop-up blockers, which made pop-up ads pointless. Take away our ability to block obnoxious digital content and you guarantee that we will be flooded with it.
These kinds of modifications aren't just used to block ads – they're also key to accessibility. People who have photosensitive epilepsy or who (like me) suffer from low-contrast vision problems use add-ons to reformat pages so that we can safely and legibly access them.
WEI's creators said they were only trying to put the web on a level playing field with apps, which routinely rat you out to the companies you connect to. Apps are a source of bottomless enshittification, not least because (unlike the web), they enjoy special, dangerous legal protections that make it very legally risky to modify them:
WEI wasn't an effort to level the playing field between apps and the web – it was a race to the bottom, an attempt to make the web as enshittogenic as the app hellscape.
Public outrage to WEI killed the project, but Google's commitment to augmenting its illegal commercial lockdown efforts with technical lockdowns never ended. Now, Google has rolled out an experimental "reCAPTCHA Mobile Verification" that uses an app, your camera, and your device's TPM or secure enclave to produce an attestation about your Android device:
This will make it much easier for the apps and other services you interact with to block your device if you run an Android alternative, or if you install a mod that overrides the actions of Google's stock Android:
This is a terrible idea – it's every bit as bad as WEI was. In an age in which Big Tech is ever-more tied to authoritarian governments, redesigning our devices to tell strangers things we don't want them to know isn't just shortsighted, it's inexcusable.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality✓ Free Actions
Free to watch • No registration required • HD streaming
Forget F1: the only car race that matters now is the race to turn your car into a digital extraction machine, a high-speed inkjet printer on wheels, stealing your private data as it picks your pocket. Your car’s digital infrastructure is a costly, dangerous nightmare — but for automakers in pursuit of postcapitalist utopia, it’s a dream they can’t give up on.
Your car is stuffed full of microchips, a fact the world came to appreciate after the pandemic struck and auto production ground to a halt due to chip shortages. Of course, that wasn’t the whole story: when the pandemic started, the automakers panicked and canceled their chip orders, only to immediately regret that decision and place new orders.
But it was too late: semiconductor production had taken a serious body-blow, and when Big Car placed its new chip orders, it went to the back of a long, slow-moving line. It was a catastrophic bungle: microchips are so integral to car production that a car is basically a computer network on wheels that you stick your fragile human body into and pray.
The car manufacturers got so desperate for chips that they started buying up washing machines for the microchips in them, extracting the chips and discarding the washing machines like some absurdo-dystopian cyberpunk walnut-shelling machine:
These digital systems are a huge problem for the car companies. They are the underlying cause of a precipitous decline in car quality. From touch-based digital door-locks to networked sensors and cameras, every digital system in your car is a source of endless repair nightmares, costly recalls and cybersecurity vulnerabilities:
What’s more, drivers hate all the digital bullshit, from the janky touchscreens to the shitty, wildly insecure apps. Digital systems are drivers’ most significant point of dissatisfaction with the automakers’ products:
Even the automakers sorta-kinda admit that this is a problem. Back in 2020 when Massachusetts was having a Right-to-Repair ballot initiative, Big Car ran these unfuckingbelievable scare ads that basically said, “Your car spies on you so comprehensively that giving anyone else access to its systems will let murderers stalk you to your home and kill you:
But even amid all the complaining about cars getting stuck in the Internet of Shit, there’s still not much discussion of why the car-makers are making their products less attractive, less reliable, less safe, and less resilient by stuffing them full of microchips. Are car execs just the latest generation of rubes who’ve been suckered by Silicon Valley bullshit and convinced that apps are a magic path to profitability?
Nope. Car execs are sophisticated businesspeople, and they’re surfing capitalism’s latest — and last — hot trend: dismantling capitalism itself.
Now, leftists have been predicting the death of capitalism since The Communist Manifesto, but even Marx and Engels warned us not to get too frisky: capitalism, they wrote, is endlessly creative, constantly reinventing itself, re-emerging from each crisis in a new form that is perfectly adapted to the post-crisis reality:
But capitalism has finally run out of gas. In his forthcoming book, Techno Feudalism: What Killed Capitalism, Yanis Varoufakis proposes that capitalism has died — but it wasn’t replaced by socialism. Rather, capitalism has given way to feudalism:
Under capitalism, capital is the prime mover. The people who own and mobilize capital — the capitalists — organize the economy and take the lion’s share of its returns. But it wasn’t always this way: for hundreds of years, European civilization was dominated by rents, not markets.
A “rent” is income that you get from owning something that other people need to produce value. Think of renting out a house you own: not only do you get paid when someone pays you to live there, you also get the benefit of rising property values, which are the result of the work that all the other homeowners, business owners, and residents do to make the neighborhood more valuable.
The first capitalists hated rent. They wanted to replace the “passive income” that landowners got from taxing their serfs’ harvest with active income from enclosing those lands and grazing sheep in order to get wool to feed to the new textile mills. They wanted active income — and lots of it.
Capitalist philosophers railed against rent. The “free market” of Adam Smith wasn’t a market that was free from regulation — it was a market free from rents. The reason Smith railed against monopolists is because he (correctly) understood that once a monopoly emerged, it would become a chokepoint through which a rentier could cream off the profits he considered the capitalist’s due:
Today, we live in a rentier’s paradise. People don’t aspire to create value — they aspire to capture it. In Survival of the Richest, Doug Rushkoff calls this “going meta”: don’t provide a service, just figure out a way to interpose yourself between the provider and the customer:
Don’t drive a cab, create Uber and extract value from every driver and rider. Better still: don’t found Uber, invest in Uber options and extract value from the people who invest in Uber. Even better, invest in derivatives of Uber options and extract value from people extracting value from people investing in Uber, who extract value from drivers and riders. Go meta.
This is your brain on the four-hour-work-week, passive income mind-virus. In Techno Feudalism, Varoufakis deftly describes how the new “Cloud Capital” has created a new generation of rentiers, and how they have become the richest, most powerful people in human history.
Shopping at Amazon is like visiting a bustling city center full of stores — but each of those stores’ owners has to pay the majority of every sale to a feudal landlord, Emperor Jeff Bezos, who also decides which goods they can sell and where they must appear on the shelves. Amazon is full of capitalists, but it is not a capitalist enterprise. It’s a feudal one:
This is the reason that automakers are willing to enshittify their products so comprehensively: they were one of the first industries to decouple rents from profits. Recall that the reason that Big Car needed billions in bailouts in 2008 is that they’d reinvented themselves as loan-sharks who incidentally made cars, lending money to car-buyers and then “securitizing” the loans so they could be traded in the capital markets.
Even though this strategy brought the car companies to the brink of ruin, it paid off in the long run. The car makers got billions in public money, paid their execs massive bonuses, gave billions to shareholders in buybacks and dividends, smashed their unions, fucked their pensioned workers, and shipped jobs anywhere they could pollute and murder their workforce with impunity.
Car companies are on the forefront of postcapitalism, and they understand that digital is the key to rent-extraction. Remember when BMW announced that it was going to rent you the seatwarmer in your own fucking car?
Not to be outdone, Mercedes announced that they were going to rent you your car’s accelerator pedal, charging an extra $1200/year to unlock a fully functional acceleration curve:
This is the urinary tract infection business model: without digitization, all your car’s value flowed in a healthy stream. But once the car-makers add semiconductors, each one of those features comes out in a painful, burning dribble, with every button on that fakakta touchscreen wired directly into your credit-card.
But it’s just for starters. Computers are malleable. The only computer we know how to make is the Turing Complete Von Neumann Machine, which can run every program we know how to write. Once they add networked computers to your car, the Car Lords can endlessly twiddle the knobs on the back end, finding new ways to extract value from you:
https://doctorow.medium.com/twiddler-1b5c9690cce6
That means that your car can track your every movement, and sell your location data to anyone and everyone, from marketers to bounty-hunters looking to collect fees for tracking down people who travel out of state for abortions to cops to foreign spies:
Digitization supercharges financialization. It lets car-makers offer subprime auto-loans to desperate, poor people and then killswitch their cars if they miss a payment:
https://www.youtube.com/watch?v=4U2eDJnwz_s
Subprime lending for cars would be a terrible business without computers, but digitization makes it a great source of feudal rents. Car dealers can originate loans to people with teaser rates that quickly blow up into payments the dealer knows their customer can’t afford. Then they repo the car and sell it to another desperate person, and another, and another:
Digitization also opens up more exotic options. Some subprime cars have secondary control systems wired into their entertainment system: miss a payment and your car radio flips to full volume and bellows an unstoppable, unmutable stream of threats. Tesla does one better: your car will lock and immobilize itself, then blare its horn and back out of its parking spot when the repo man arrives:
Digital feudalism hasn’t stopped innovating — it’s just stopped innovating good things. The digital device is an endless source of sadistic novelties, like the cellphones that disable your most-used app the first day you’re late on a payment, then work their way down the other apps you rely on for every day you’re late:
Usurers have always relied on this kind of imaginative intimidation. The loan-shark’s arm-breaker knows you’re never going to get off the hook; his goal is in intimidating you into paying his boss first, liquidating your house and your kid’s college fund and your wedding ring before you default and he throws you off a building.
Thanks to the malleability of computerized systems, digital arm-breakers have an endless array of options they can deploy to motivate you into paying them first, no matter what it costs you:
Car-makers are trailblazers in imaginative rent-extraction. Take VIN-locking: this is the practice of adding cheap microchips to engine components that communicate with the car’s overall network. After a new part is installed in your car, your car’s computer does a complex cryptographic handshake with the part that requires an unlock code provided by an authorized technician. If the code isn’t entered, the car refuses to use that part.
VIN-locking has exploded in popularity. It’s in your iPhone, preventing you from using refurb or third-party replacement parts:
It’s in fuckin’ ventilators, which was a nightmare during lockdown as hospital techs nursed their precious ventilators along by swapping parts from dead systems into serviceable ones:
And of course, it’s in tractors, along with other forms of remote killswitch. Remember that feelgood story about John Deere bricking the looted Ukrainian tractors whose snitch-chips showed they’d been relocated to Russia?
That wasn’t a happy story — it was a cautionary tale. After all, John Deere now controls the majority of the world’s agricultural future, and they’ve boobytrapped those ubiquitous tractors with killswitches that can be activated by anyone who hacks, takes over, or suborns Deere or its dealerships.
Control over repair isn’t limited to gouging customers on parts and service. When a company gets to decide whether your device can be fixed, it can fuck you over in all kinds of ways. Back in 2019, Tim Apple told his shareholders to expect lower revenues because people were opting to fix their phones rather than replace them:
By usurping your right to decide who fixes your phone, Apple gets to decide whether you can fix it, or whether you must replace it. Problem solved — and not just for Apple, but for car makers, tractor makers, ventilator makers and more. Apple leads on this, even ahead of Big Car, pioneering a “recycling” program that sees trade-in phones shredded so they can’t possibly be diverted from an e-waste dump and mined for parts:
John Deere isn’t sleeping on this. They’ve come up with a valuable treasure they extract when they win the Right-to-Repair: Deere singles out farmers who complain about its policies and refuses to repair their tractors, stranding them with six-figure, two-ton paperweight:
The repair wars are just a skirmish in a vast, invisible fight that’s been waged for decades: the War On General-Purpose Computing, where tech companies use the law to make it illegal for you to reconfigure your devices so they serve you, rather than their shareholders:
The force behind this army is vast and grows larger every day. General purpose computers are antithetical to technofeudalism — all the rents extracted by technofeudalists would go away if others (tinkereres, co-ops, even capitalists!) were allowed to reconfigure our devices so they serve us.
You’ve probably noticed the skirmishes with inkjet printer makers, who can only force you to buy their ink at 20,000% markups if they can stop you from deciding how your printer is configured:
https://pluralistic.net/2022/08/07/inky-wretches/#epson-salty
But we’re also fighting against insulin pump makers, who want to turn people with diabetes into walking inkjet printers:
These companies start with people who have the least agency and social power and wreck their lives, then work their way up the privilege gradient, coming for everyone else. It’s called the “shitty technology adoption curve”:
Technofeudalism is the public-private-partnership from hell, emerging from a combination of state and private action. On the one hand, bailing out bankers and big business (rather than workers) after the 2008 crash and the covid lockdown decoupled income from profits. Companies spent billions more than they earned were still wildly profitable, thanks to those public funds.
But there’s also a policy dimension here. Some of those rentiers’ billions were mobilized to both deconstruct antitrust law (allowing bigger and bigger companies and cartels) and to expand “IP” law, turning “IP” into a toolsuite for controlling the conduct of a firm’s competitors, critics and customers:
https://locusmag.com/2020/09/cory-doctorow-ip/
IP is key to understanding the rise of technofeudalism. The same malleability that allows companies to “twiddle” the knobs on their services and keep us on the hook as they reel us in would hypothetically allow us to countertwiddle, seizing the means of computation:
The thing that stands between you and an alternative app store, an interoperable social media network that you can escape to while continuing to message the friends you left behind, or a car that anyone can fix or unlock features for is IP, not technology. Under capitalism, that technology would already exist, because capitalists have no loyalty to one another and view each other’s margins as their own opportunities.
But under technofeudalism, control comes from rents (owning things), not profits (selling things). The capitalist who wants to participate in your iPhone’s “ecosystem” has to make apps and submit them to Apple, along with 30% of their lifetime revenues — they don’t get to sell you jailbreaking kit that lets you choose their app store.
Rent-seeking technology has a holy grail: control over “ring zero” — the ability to compel you to configure your computer to a feudalist’s specifications, and to verify that you haven’t altered your computer after it came into your possession:
For more than two decades, various would-be feudal lords and their court sorcerers have been pitching ways of doing this, of varying degrees of outlandishness.
At core, here’s what they envision: inside your computer, they will nest another computer, one that is designed to run a very simple set of programs, none of which can be altered once it leaves the factory. This computer — either a whole separate chip called a “Trusted Platform Module” or a region of your main processor called a secure enclave — can tally observations about your computer: which operating system, modules and programs it’s running.
Then it can cryptographically “sign” these observations, proving that they were made by a secure chip and not by something you could have modified. Then you can send this signed “attestation” to someone else, who can use it to determine how your computer is configured and thus whether to trust it. This is called “remote attestation.”
There are some cool things you can do with remote attestation: for example, two strangers playing a networked video game together can use attestations to make sure neither is running any cheat modules. Or you could require your cloud computing provider to use attestations that they aren’t stealing your data from the server you’re renting. Or if you suspect that your computer has been infected with malware, you can connect to someone else and send them an attestation that they can use to figure out whether you should trust it.
Today, there’s a cool remote attestation technology called “PrivacyPass” that replaces CAPTCHAs by having you prove to your own device that you are a human. When a server wants to make sure you’re a person, it sends a random number to your device, which signs that number along with its promise that it is acting on behalf of a human being, and sends it back. CAPTCHAs are all kinds of bad — bad for accessibility and privacy — and this is really great.
But the billions that have been thrown at remote attestation over the decades is only incidentally about solving CAPTCHAs or verifying your cloud server. The holy grail here is being able to make sure that you’re not running an ad-blocker. It’s being able to remotely verify that you haven’t disabled the bossware your employer requires. It’s the power to block someone from opening an Office365 doc with LibreOffice. It’s your boss’s ability to ensure that you haven’t modified your messaging client to disable disappearing messages before he sends you an auto-destructing memo ordering you to break the law.
And there’s a new remote attestation technology making the rounds: Google’s Web Environment Integrity, which will leverage Google’s dominance over browsers to allow websites to block users who run ad-blockers:
There’s plenty else WEI can do (it would make detecting ad-fraud much easier), but for every legitimate use, there are a hundred ways this could be abused. It’s a technology purpose-built to allow rent extraction by stripping us of our right to technological self-determination.
Releasing a technology like this into a world where companies are willing to make their products less reliable, less attractive, less safe and less resilient in pursuit of rents is incredibly reckless and shortsighted. You want unauthorized bread? This is how you get Unauthorized Bread:
If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
[Image ID: The interior of a luxury car. There is a dagger protruding from the steering wheel. The entertainment console has been replaced by the text 'You wouldn't download a car,' in MPAA scare-ad font. Outside of the windscreen looms the Matrix waterfall effect. Visible in the rear- and side-view mirror is the driver: the figure from Munch's 'Scream.' The screen behind the steering-wheel has been replaced by the menacing red eye of HAL9000 from Stanley Kubrick's '2001: A Space Odyssey.']
Powerful people imprisoned by the cluelessness of their own isolation, locked up with their own motivated reasoning: “It’s impossible to get a CEO to understand something when his quarterly earnings call depends on him not understanding it.”
Take Mark Zuckerberg. Zuckerberg insists that anyone who wanted to use a pseudonym online is “two-faced,” engaged in dishonest social behavior. The Zuckerberg Doctrine claims that forcing people to use their own names is a way to ensure civility. This is an idea so radioactively wrong, it can be spotted from orbit.
From the very beginning, social scientists (both inside and outside Facebook) told Zuckerberg that he was wrong. People have lots of reasons to hide their identities online, both good and bad, but a Real Names Policy affects different people differently:
For marginalized and at-risk people, there are plenty of reasons to want to have more than one online identity — say, because you are a #MeToo whistleblower hoping that Harvey Weinstein won’t sic his ex-Mossad mercenaries on you:
Or maybe you’re a Rohingya Muslim hoping to avoid the genocidal attentions of the troll army that used Facebook to organize — under their real, legal names — to rape and murder you and everyone you love:
But even if no one is looking to destroy your life or kill you and your family, there are plenty of good reasons to present different facets of your identity to different people. No one talks to their lover, their boss and their toddler in exactly the same way, or reveals the same facts about their lives to those people. Maintaining different facets to your identity is normal and healthy — and the opposite, presenting the same face to everyone in your life, is a wildly terrible way to live.
None of this is controversial among social scientists, nor is it hard to grasp. But Zuckerberg stubbornly stuck to this anonymity-breeds-incivility doctrine, even as dictators used the fact that Facebook forced dissidents to use their real names to retain power through the threat (and reality) of arrest and torture:
Why did Zuck cling to this dangerous and obvious fallacy? Because the more he could collapse your identity into one unitary whole, the better he could target you with ads. Truly, it is impossible to get a billionaire to understand something when his mega-yacht depends on his not understanding it.
This motivated reasoning ripples through all of Silicon Valley’s top brass, producing what Anil Dash calls “VC QAnon,” the collection of conspiratorial, debunked and absurd beliefs embraced by powerful people who hold the digital lives of billions of us in their quivering grasp:
https://www.anildash.com/2023/07/07/vc-qanon/
These fallacy-ridden autocrats like to disguise their demands as observations, as though wanting something to be true was the same as making it true. Think of when Eric Schmidt — then the CEO of Google — dismissed online privacy concerns, stating “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place”:
Both men knew better. Schmidt, in particular, is very jealous of his own privacy. When Cnet reporters used Google to uncover and publish public (but intimate and personal) facts about Schmidt, Schmidt ordered Google PR to ignore all future requests for comment from Cnet reporters:
(Like everything else he does, Elon Musk’s policy of responding to media questions about Twitter with a poop emoji is just him copying things other people thought up, making them worse, and taking credit for them:)
Schmidt’s actions do not reflect an attitude of “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.” Rather, they are the normal response that we all have to getting doxed.
When Schmidt and McNealy and Zuck tell us that we don’t have privacy, or we don’t want privacy, or that privacy is bad for us, they’re disguising a demand as an observation. “Privacy is dead” actually means, “When privacy is dead, I will be richer than you can imagine, so stop trying to save it, goddamnit.”
We are all prone to believing our own bullshit, but when a tech baron gets high on his own supply, his mental contortions have broad implications for all of us. A couple years after Schmidt’s anti-privacy manifesto, Google launched Google Plus, a social network where everyone was required to use their “real name.”
This decision — justified as a means of ensuring civility and a transparent ruse to improve ad targeting — kicked off the Nym Wars:
One of the best documents to come out of that ugly conflict is “Falsehoods Programmers Believe About Names,” a profound and surprising enumeration of all the ways that the experiences of tech bros in Silicon Valley are the real edge-cases, unreflective of the reality of billions of their users:
This, in turn, spawned a whole genre of programmer-fallacy catalogs, falsehoods programmers believe about time, currency, birthdays, timezones, email addresses, national borders, nations, biometrics, gender, language, alphabets, phone numbers, addresses, systems of measurement, and, of course, families:
https://github.com/kdeldycke/awesome-falsehood
But humility is in short supply in tech. It’s impossible to get a programmer to understand something when their boss requires them not to understand it. A programmer will happily insist that ordering you to remove your “mask” is for your own good — and not even notice that they’re taking your skin off with it.
There are so many ways that tech executives could improve their profits if only we would abandon our stubborn attachment to being so goddamned complicated. Think of Netflix and its anti-passsword-sharing holy war, which is really a demand that we redefine “family” to be legible and profitable for Netflix:
But despite the entreaties of tech companies to collapse our identities, our families, and our online lives into streamlined, computably hard-edged shapes that fit neatly into their database structures, we continue to live fuzzy, complicated lives that only glancingly resemble those of the executives seeking to shape them.
Now, the rich, powerful people making these demands don’t plan on being constrained by them. They are conservatives, in the tradition of #FrankWilhoit, believers in a system of “in-groups whom the law protects but does not bind, alongside out-groups whom the law binds but does not protect”:
As with Schmidt’s desire to spy on you from asshole to appetite for his own personal gain, and his violent aversion to having his own personal life made public, the tech millionaires and billionaires who made their fortune from the flexibility of general purpose computers would like to end that flexibility. They insist that the time for general purpose computers has passed, and that today, “consumers” crave the simplicity of appliances:
It is in the War On General Purpose Computing that we find the cheapest and flimsiest rhetoric. Companies like Apple — and their apologists — insist that no one wants to use third-party app stores, or seek out independent repair depots — and then spend millions to make sure that it’s illegal to jailbreak your phone or get it fixed outside of their own official channel:
The cognitive dissonance of “no one wants this,” and “we must make it illegal to get this” is powerful, but the motivated reasoning is more powerful still. It is impossible to get Tim Cook to understand something when his $49 million paycheck depends on him not understanding it.
The War on General Purpose Computing has been underway for decades. Computers, like the people who use them, stubbornly insist on being reality-based, and the reality of computers is that they are general purpose. Every computer is a Turing complete, universal Von Neumann machine, which means that it can run every valid program. There is no way to get a computer to be almost Turing Complete, only capable of running programs that don’t upset your shareholders’ fragile emotional state.
There is no such thing as a printer that will only run the “reject third-party ink” program. There is no such thing as a phone that will only run the “reject third-party apps” program. There are only laws, like the Section 1201 of the Digital Millennium Copyright Act, that make writing and distributing those programs a felony punishable by a five-year prison sentence and a $500,000 fine (for a first offense).
That is to say, the War On General Purpose Computing is only incidentally a technical fight: it is primarily a legal fight. When Apple says, “You can’t install a third party app store on your phone,” what they means is, “it’s illegal to install that third party app store.” It’s not a technical countermeasure that stands between you and technological self-determination, it’s a legal doctrine we can call “felony contempt of business model”:
https://locusmag.com/2020/09/cory-doctorow-ip/
But the mighty US government will not step in to protect a company’s business model unless it at least gestures towards the technical. To invoke DMCA 1201, a company must first add the thinnest skin of digital rights management to their product. Since 1201 makes removing DRM illegal, a company can use this molecule-thick scrim of DRM to felonize any activity that the DRM prevents.
More than 20 years ago, technologists started to tinker with ways to combine the legal and technical to tame the wild general purpose computer. Starting with Microsoft’s Palladium project, they theorized a new “Secure Computing” model for allowing companies to reach into your computer long after you had paid for it and brought it home, in order to discipline you for using it in ways that undermined its shareholders’ interest.
Secure Computing began with the idea of shipping every computer with two CPUs. The first one was the normal CPU, the one you interacted with when you booted it up, loaded your OS, and ran programs. The second CPU would be a Trusted Platform Module, a brute-simple system-on-a-chip designed to be off-limits to modification, even by its owner (that is, you).
The TPM would ship with a limited suite of simple programs it could run, each thoroughly audited for bugs, as well as secret cryptographic signing keys that you were not permitted to extract. The original plan called for some truly exotic physical security measures for that TPM, like an acid-filled cavity that would melt the chip if you tried to decap it or run it through an electron-tunneling microscope:
This second computer represented a crack in the otherwise perfectly smooth wall of a computer’s general purposeness; and Trusted Computing proposed to hammer a piton into that crack and use it to anchor a whole superstructure that could observe — and limited — the activity of your computer.
This would start with observation: the TPM would observe every step of your computer’s boot sequence, creating cryptographic hashes of each block of code as it loaded and executed. Each stage of the boot-up could be compared to “known good” versions of those programs. If your computer did something unexpected, the TPM could halt it in its tracks, blocking the boot cycle.
What kind of unexpected things do computers do during their boot cycle? Well, if your computer is infected with malware, it might load poisoned versions of its operating system. Once your OS is poisoned, it’s very hard to detect its malicious conduct, since normal antivirus programs rely on the OS to faithfully report what your computer is doing. When the AV program asks the OS to tell it which programs are running, or which files are on the drive, it has no choice but to trust the OS’s response. When the OS is compromised, it can feed a stream of lies to users’ programs, assuring these apps that everything is fine.
That’s a very beneficial use for a TPM, but there’s a sinister flipside: the TPM can also watch your boot sequence to make sure that there aren’t beneficial modifications present in your operating system. If you modify your OS to let you do things the manufacturer wants to prevent — like loading apps from a third-party app-store — the TPM can spot this and block it.
Now, these beneficial and sinister uses can be teased apart. When the Palladium team first presented its research, my colleague Seth Schoen proposed an “owner override”: a modification of Trusted Computing that would let the computer’s owner override the TPM:
This override would introduce its own risks, of course. A user who was tricked into overriding the TPM might expose themselves to malicious software, which could harm that user, as well as attacking other computers on the user’s network and the other users whose data were on the compromised computer’s drive.
But an override would also provide serious benefits: it would rule out the monopolistic abuse of a TPM to force users to run malicious code that the manufacturer insisted on — code that prevented the user from doing things that benefited the user, even if it harmed the manufacturer’s shareholders. For example, with owner override, Microsoft couldn’t force you to use its official MS Office programs rather than third-party compatible programs like Apple’s iWork or Google Docs or LibreOffice.
Owner override also completely changed the calculus for another, even more dangerous part of Trusted Computing: remote attestation.
Remote Attestation is a way for third parties to request a reliable, cryptographically secured assurances about which operating system and programs your computer is running. In Remote Attestation, the TPM in your computer observes every stage of your computer’s boot, gathers information about all the programs you’re running, and cryptographically signs them, using the signing keys the manufacturer installed during fabrication.
You can send this “attestation” to other people on the internet. If they trust that your computer’s TPM is truly secure, then they know that you have sent them a true picture of your computer’s working (the actual protocol is a little more complicated and involves the remote party sending you a random number to cryptographically hash with the attestation, to prevent out-of-date attestations).
Now, this is also potentially beneficial. If you want to make sure that your technologically unsophisticated friend is running an uncompromised computer before you transmit sensitive data to it, you can ask them for an attestation that will tell you whether they’ve been infected with malware.
But it’s also potentially very sinister. Your government can require all the computers in its borders to send a daily attestation to confirm that you’re still running the mandatory spyware. Your abusive spouse — or abusive boss — can do the same for their own disciplinary technologies. Such a tool could prevent you from connecting to a service using a VPN, and make it impossible to use Tor Browser to protect your privacy when interacting with someone who wishes you harm.
The thing is, it’s completely normal and good for computers to lie to other computers on behalf of their owners. Like, if your IoT ebike’s manufacturer goes out of business and all their bikes get bricked because they can no longer talk to their servers, you can run an app that tricks the bike into thinking that it’s still talking to the mothership:
Or if you’re connecting to a webserver that tries to track you by fingerprinting you based on your computer’s RAM, screen size, fonts, etc, you can order your browser to send random data about this stuff:
https://jshelter.org/fingerprinting/
Or if you’re connecting to a site that wants to track you and nonconsensually cram ads into your eyeballs, you can run an adblocker that doesn’t show you the ads, but tells the site that it did:
Owner override leaves some of the beneficial uses of remote attestation intact. If you’re asking a friend to remotely confirm that your computer is secure, you’re not going to use an override to send them bad data about about your computer’s configuration.
And owner override also sweeps all of the malicious uses of remote attestation off the board. With owner override, you can tell any lie about your computer to a webserver, a site, your boss, your abusive spouse, or your government, and they can’t spot the lie.
But owner override also eliminates some beneficial uses of remote attestation. For example, owner override rules out remote attestation as a way for strangers to play multiplayer video games while confirming that none of them are using cheat programs (like aimhack). It also means that you can’t use remote attestation to verify the configuration of a cloud server you’re renting in order to assure yourself that it’s not stealing your data or serving malware to your users.
This is a tradeoff, and it’s a tradeoff that’s similar to lots of other tradeoffs we make online, between the freedom to do something good and the freedom to do something bad. Participating anonymously, contributing to free software, distributing penetration testing tools, or providing a speech platform that’s open to the public all represent the same tradeoff.
We have lots of experience with making the tradeoff in favor of restrictions rather than freedom: powerful bad actors are happy to attach their names to their cruel speech and incitement to violence. Their victims are silenced for fear of that retaliation.
When we tell security researchers they can’t disclose defects in software without the manufacturer’s permission, the manufacturers use this as a club to silence their critics, not as a way to ensure orderly updates.
When we let corporations decide who is allowed to speak, they act with a mixture of carelessness and self-interest, becoming off-the-books deputies of authoritarian regimes and corrupt, powerful elites.
Alas, we made the wrong tradeoff with Trusted Computing. For the past twenty years, Trusted Computing has been creeping into our devices, albeit in somewhat denatured form. The original vision of acid-filled secondary processors has been replaced with less exotic (and expensive) alternatives, like “secure enclaves.” With a secure enclave, the manufacturer saves on the expense of installing a whole second computer, and instead, they draw a notional rectangle around a region of your computer’s main chip and try really hard to make sure that it can only perform a very constrained set of tasks.
This gives us the worst of all worlds. When secure enclaves are compromised, we not only lose the benefit of cryptographic certainty, knowing for sure that our computers are only booting up trusted, unalterted versions of the OS, but those compromised enclaves run malicious software that is essentially impossible to detect or remove:
But while Trusted Computing has wormed its way into boot-restrictions — preventing you from jailbreaking your computer so it will run the OS and apps of your choosing — there’s been very little work on remote attestation…until now.
Web Environment Integrity is Google’s proposal to integrate remote attestation into everyday web-browsing. The idea is to allow web-servers to verify what OS, extensions, browser, and add-ons your computer is using before the server will communicate with you:
Even by the thin standards of the remote attestation imaginaries, there are precious few beneficial uses for this. The googlers behind the proposal have a couple of laughable suggestions, like, maybe if ad-supported sites can comprehensively refuse to serve ad-blocking browsers, they will invest the extra profits in making things you like. Or: letting websites block scriptable browsers will make it harder for bad people to auto-post fake reviews and comments, giving users more assurances about the products they buy.
But foundationally, WEI is about compelling you to disclose true facts about yourself to people who you want to keep those facts from. It is a Real Names Policy for your browser. Google wants to add a new capability to the internet: the ability of people who have the power to force you to tell them things to know for sure that you’re not lying.
The fact that the authors assume this will be beneficial is just another “falsehood programmers believe”: there is no good reason to hide the truth from other people. Squint a little and we’re back to McNealy’s “Privacy is dead, get over it.” Or Schmidt’s “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.”
And like those men, the programmers behind this harebrained scheme don’t imagine that it will ever apply to them. As Chris Palmer — who worked on Chromium — points out, this is not compatible with normal developer tools or debuggers, which are “incalculably valuable and not really negotiable”:
As I wrote last week, giving manufacturers the power to decide how your computer is configured, overriding your own choices, is a bad tradeoff — the worst tradeoff, a greased slide into terminal enshittification:
All of which leads to the question: what now? What should be done about WEI and remote attestation?
Let me start by saying: I don’t think it should be illegal for programmers to design and release these tools. Code is speech, and we can’t understand how this stuff works if we can’t study it.
But programmers shouldn’t deploy it in production code, in the same way that programmers should be allowed to make pen-testing tools, but shouldn’t use them to attack production systems and harm their users. Programmers who do this should be criticized and excluded from the society of their ethical, user-respecting peers.
Corporations that use remote attestation should face legal restrictions: privacy law should prevent the use of remote attestation to compel the production of true facts about users or the exclusion of users who refuse to produce those facts. Unfair competition law should prevent companies from using remote attestation to block interoperability or tie their products to related products and services.
Finally, we must withdraw the laws that prevent users and programmers from overriding TPMs, secure enclaves and remote attestations. You should have the right to study and modify your computer to produce false attestations, or run any code of your choosing. Felony contempt of business model is an outrage. We should alter or strike down DMCA 1201, the Computer Fraud and Abuse Act, and other laws (like contract law’s “tortious interference”) that stand between you and “sole and despotic dominion” over your own computer. All of that applies not just to users who want to reconfigure their own computers, but also toolsmiths who want to help them do so, by offering information, code, products or services to jailbreak and alter your devices.
Tech giants will squeal at this, insisting that they serve your interests when they prevent rivals from opening up their products. After all, those rivals might be bad guys who want to hurt you. That’s 100% true. What is likewise true is that no tech giant will defend you from its own bad impulses, and if you can’t alter your device, you are powerless to stop them:
Companies should be stopped from harming you, but the right place to decide whether a business is doing something nefarious isn’t in the boardroom of that company’s chief competitor: it’s in the halls of democratically accountable governments:
So how do we get there? Well, that’s another matter. In my next book, The Internet Con: How to Seize the Means of Computation (Verso Books, Sept 5), I lay out a detailed program, describing which policies will disenshittify the internet, and how to get those policies:
Predictably, there are challenges getting this kind of book out into the world via our concentrated tech sector. Amazon refuses to carry the audio edition on its monopoly audiobook platform, Audible, unless it is locked to Amazon forever with mandatory DRM. That’s left me self-financing my own DRM-free audio edition, which is currently available for pre-order via this Kickstarter:
http://seizethemeansofcomputation.org
I’m kickstarting the audiobook for “The Internet Con: How To Seize the Means of Computation,” a Big Tech disassembly manual to disenshittify the web and bring back the old, good internet. It’s a DRM-free book, which means Audible won’t carry it, so this crowdfunder is essential. Back now to get the audio, Verso hardcover and ebook:
If you’d like an essay-formatted version of this post to read or share, here’s a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
[Image ID: An anatomical drawing of a flayed human head; it has been altered to give it a wide-stretched mouth revealing a gadget nestled in the back of the figure's throat, connected by a probe whose two coiled wires stretch to an old fashioned electronic box. The head's eyes have been replaced by the red, menacing eye of HAL 9000 from Stanley Kubrick's '2001: A Space Odyssey.' Behind the head is a code waterfall effect as seen in the credits of the Wachowskis' 'The Matrix.']
The disenshittified internet starts with loyal "user agents"
I'm in TARTU, ESTONIA! Overcoming the Enshittocene (TOMORROW, May 8, 6PM, Prima Vista Literary Festival keynote, University of Tartu Library, Struwe 1). AI, copyright and creative workers' labor rights (May 10, 8AM: Science Fiction Research Association talk, Institute of Foreign Languages and Cultures building, Lossi 3, lobby). A talk for hackers on seizing the means of computation (May 10, 3PM, University of Tartu Delta Centre, Narva 18, room 1037).
There's one overwhelmingly common mistake that people make about enshittification: assuming that the contagion is the result of the Great Forces of History, or that it is the inevitable end-point of any kind of for-profit online world.
In other words, they class enshittification as an ideological phenomenon, rather than as a material phenomenon. Corporate leaders have always felt the impulse to enshittify their offerings, shifting value from end users, business customers and their own workers to their shareholders. The decades of largely enshittification-free online services were not the product of corporate leaders with better ideas or purer hearts. Those years were the result of constraints on the mediocre sociopaths who would trade our wellbeing and happiness for their own, constraints that forced them to act better than they do today, even if the were not any better:
Corporate leaders' moments of good leadership didn't come from morals, they came from fear. Fear that a competitor would take away a disgruntled customer or worker. Fear that a regulator would punish the company so severely that all gains from cheating would be wiped out. Fear that a rival technology – alternative clients, tracker blockers, third-party mods and plugins – would emerge that permanently severed the company's relationship with their customers. Fears that key workers in their impossible-to-replace workforce would leave for a job somewhere else rather than participate in the enshittification of the services they worked so hard to build:
When those constraints melted away – thanks to decades of official tolerance for monopolies, which led to regulatory capture and victory over the tech workforce – the same mediocre sociopaths found themselves able to pursue their most enshittificatory impulses without fear.
The effects of this are all around us. In This Is Your Phone On Feminism, the great Maria Farrell describes how audiences at her lectures profess both love for their smartphones and mistrust for them. Farrell says, "We love our phones, but we do not trust them. And love without trust is the definition of an abusive relationship":
I (re)discovered this Farrell quote in a paper by Robin Berjon, who recently co-authored a magnificent paper with Farrell entitled "We Need to Rewild the Internet":
The new Berjon paper is narrower in scope, but still packed with material examples of the way the internet goes wrong and how it can be put right. It's called "The Fiduciary Duties of User Agents":
In "Fiduciary Duties," Berjon focuses on the technical term "user agent," which is how web browsers are described in formal standards documents. This notion of a "user agent" is a holdover from a more civilized age, when technologists tried to figure out how to build a new digital space where technology served users.
A web browser that's a "user agent" is a comforting thought. An agent's job is to serve you and your interests. When you tell it to fetch a web-page, your agent should figure out how to get that page, make sense of the code that's embedded in, and render the page in a way that represents its best guess of how you'd like the page seen.
For example, the user agent might judge that you'd like it to block ads. More than half of all web users have installed ad-blockers, constituting the largest consumer boycott in human history:
Your user agent might judge that the colors on the page are outside your visual range. Maybe you're colorblind, in which case, the user agent could shift the gamut of the colors away from the colors chosen by the page's creator and into a set that suits you better:
https://dankaminsky.com/dankam/
Or maybe you (like me) have a low-vision disability that makes low-contrast type difficult to impossible to read, and maybe the page's creator is a thoughtless dolt who's chosen light grey-on-white type, or maybe they've fallen prey to the absurd urban legend that not-quite-black type is somehow more legible than actual black type:
The user agent is loyal to you. Even when you want something the page's creator didn't consider – even when you want something the page's creator violently objects to – your user agent acts on your behalf and delivers your desires, as best as it can.
Now – as Berjon points out – you might not know exactly what you want. Like, you know that you want the privacy guarantees of TLS (the difference between "http" and "https") but not really understand the internal cryptographic mysteries involved. Your user agent might detect evidence of shenanigans indicating that your session isn't secure, and choose not to show you the web-page you requested.
This is only superficially paradoxical. Yes, you asked your browser for a web-page. Yes, the browser defied your request and declined to show you that page. But you also asked your browser to protect you from security defects, and your browser made a judgment call and decided that security trumped delivery of the page. No paradox needed.
But of course, the person who designed your user agent/browser can't anticipate all the ways this contradiction might arise. Like, maybe you're trying to access your own website, and you know that the security problem the browser has detected is the result of your own forgetful failure to renew your site's cryptographic certificate. At that point, you can tell your browser, "Thanks for having my back, pal, but actually this time it's fine. Stand down and show me that webpage."
That's your user agent serving you, too.
User agents can be well-designed or they can be poorly made. The fact that a user agent is designed to act in accord with your desires doesn't mean that it always will. A software agent, like a human agent, is not infallible.
However – and this is the key – if a user agent thwarts your desire due to a fault, that is fundamentally different from a user agent that thwarts your desires because it is designed to serve the interests of someone else, even when that is detrimental to your own interests.
A "faithless" user agent is utterly different from a "clumsy" user agent, and faithless user agents have become the norm. Indeed, as crude early internet clients progressed in sophistication, they grew increasingly treacherous. Most non-browser tools are designed for treachery.
A smart speaker or voice assistant routes all your requests through its manufacturer's servers and uses this to build a nonconsensual surveillance dossier on you. Smart speakers and voice assistants even secretly record your speech and route it to the manufacturer's subcontractors, whether or not you're explicitly interacting with them:
By design, apps and in-app browsers seek to thwart your preferences regarding surveillance and tracking. An app will even try to figure out if you're using a VPN to obscure your location from its maker, and snitch you out with its guess about your true location.
Mobile phones assign persistent tracking IDs to their owners and transmit them without permission (to its credit, Apple recently switch to an opt-in system for transmitting these IDs) (but to its detriment, Apple offers no opt-out from its own tracking, and actively lies about the very existence of this tracking):
An Android device running Chrome and sitting inert, with no user interaction, transmits location data to Google every five minutes. This is the "resting heartbeat" of surveillance for an Android device. Ask that device to do any work for you and its pulse quickens, until it is emitting a nearly continuous stream of information about your activities to Google:
These faithless user agents both reflect and enable enshittification. The locked-down nature of the hardware and operating systems for Android and Ios devices means that manufacturers – and their business partners – have an arsenal of legal weapons they can use to block anyone who gives you a tool to modify the device's behavior. These weapons are generically referred to as "IP rights" which are, broadly speaking, the right to control the conduct of a company's critics, customers and competitors:
https://locusmag.com/2020/09/cory-doctorow-ip/
A canny tech company can design their products so that any modification that puts the user's interests above its shareholders is illegal, a violation of its copyright, patent, trademark, trade secrets, contracts, terms of service, nondisclosure, noncompete, most favored nation, or anticircumvention rights. Wrap your product in the right mix of IP, and its faithless betrayals acquire the force of law.
This is – in Jay Freeman's memorable phrase – "felony contempt of business model." While more than half of all web users have installed an ad-blocker, thus overriding the manufacturer's defaults to make their browser a more loyal agent, no app users have modified their apps with ad-blockers.
The first step of making such a blocker, reverse-engineering the app, creates criminal liability under Section 1201 of the Digital Millennium Copyright Act, with a maximum penalty of five years in prison and a $500,000 fine. An app is just a web-page skinned in sufficient IP to make it a felony to add an ad-blocker to it (no wonder every company wants to coerce you into using its app, rather than its website).
If you know that increasing the invasiveness of the ads on your web-page could trigger mass installations of ad-blockers by your users, it becomes irrational and self-defeating to ramp up your ads' invasiveness. The possibility of interoperability acts as a constraint on tech bosses' impulse to enshittify their products.
The shift to platforms dominated by treacherous user agents – apps, mobile ecosystems, walled gardens – weakens or removes that constraint. As your ability to discipline your agent so that it serves you wanes, the temptation to turn your user agent against you grows, and enshittification follows.
This has been tacitly understood by technologists since the web's earliest days and has been reaffirmed even as enshittification increased. Berjon quotes extensively from "The Internet Is For End-Users," AKA Internet Architecture Board RFC 8890:
Defining the user agent role in standards also creates a virtuous cycle; it allows multiple implementations, allowing end users to switch between them with relatively low costs (…). This creates an incentive for implementers to consider the users' needs carefully, which are often reflected into the defining standards. The resulting ecosystem has many remaining problems, but a distinguished user agent role provides an opportunity to improve it.
And the W3C's Technical Architecture Group echoes these sentiments in "Web Platform Design Principles," which articulates a "Priority of Constituencies" that is supposed to be central to the W3C's mission:
User needs come before the needs of web page authors, which come before the needs of user agent implementors, which come before the needs of specification writers, which come before theoretical purity.
https://w3ctag.github.io/design-principles/
But the W3C's commitment to faithful agents is contingent on its own members' commitment to these principles. In 2017, the W3C finalized "EME," a standard for blocking mods that interact with streaming videos. Nominally aimed at preventing copyright infringement, EME also prevents users from choosing to add accessibility add-ons that beyond the ones the streaming service permits. These services may support closed captioning and additional narration of visual elements, but they block tools that adapt video for color-blind users or prevent strobe effects that trigger seizures in users with photosensitive epilepsy.
The fight over EME was the most contentious struggle in the W3C's history, in which the organization's leadership had to decide whether to honor the "priority of constituencies" and make a standard that allowed users to override manufacturers, or whether to facilitate the creation of faithless agents specifically designed to thwart users' desires on behalf of manufacturers:
This fight was settled in favor of a handful of extremely large and powerful companies, over the objections of a broad collection of smaller firms, nonprofits representing users, academics and other parties agitating for a web built on faithful agents. This coincided with the W3C's operating budget becoming entirely dependent on the very large sums its largest corporate members paid.
W3C membership is on a sliding scale, based on a member's size. Nominally, the W3C is a one-member, one-vote organization, but when a highly concentrated collection of very high-value members flex their muscles, W3C leadership seemingly perceived an existential risk to the organization, and opted to sacrifice the faithfulness of user agents in service to the anti-user priorities of its largest members.
For W3C's largest corporate members, the fight was absolutely worth it. The W3C's EME standard transformed the web, making it impossible to ship a fully featured web-browser without securing permission – and a paid license – from one of the cartel of companies that dominate the internet. In effect, Big Tech used the W3C to secure the right to decide who would compete with them in future, and how:
Enshittification arises when the everyday mediocre sociopaths who run tech companies are freed from the constraints that act against them. When the web – and its browsers – were a big, contented, diverse, competitive space, it was harder for tech companies to collude to capture standards bodies like the W3C to secure even more dominance. As the web turned into Tom Eastman's "five giant websites filled with screenshots of text from the other four," that kind of collusion became much easier:
In arguing for faithful agents, Berjon associates himself with the group of scholars, regulators and activists who call for user agents to serve as "information fiduciaries." Mostly, information fiduciaries come up in the context of user privacy, with the idea that entities that hold a user's data would have the obligation to put the user's interests ahead of their own. Think of a lawyer's fiduciary duty in respect of their clients, to give advice that reflects the client's best interests, even when that conflicts with the lawyer's own self-interest. For example, a lawyer who believes that settling a case is the best course of action for a client is required to tell them so, even if keeping the case going would generate more billings for the lawyer and their firm.
For a user agent to be faithful, it must be your fiduciary. It must put your interests ahead of the interests of the entity that made it or operates it. Browsers, email clients, and other internet software that served as a fiduciary would do things like automatically blocking tracking (which most email clients don't do, especially webmail clients made by companies like Google, who also sell advertising and tracking).
Berjon contemplates a legally mandated fiduciary duty, citing Lindsey Barrett's "Confiding in Con Men":
He describes a fiduciary duty as a remedy for the enforcement failures of EU's GDPR, a solidly written, and dismally enforced, privacy law. A legally backstopped duty for agents to be fiduciaries would also help us distinguish good and bad forms of "innovation" – innovation in ways of thwarting a user's will are always bad.
Now, the tech giants insist that they are already fiduciaries, and that when they thwart a user's request, that's more like blocking access to a page where the encryption has been compromised than like HAL9000's "I can't let you do that, Dave." For example, when Louis Barclay created "Unfollow Everything," he (and his enthusiastic users) found that automating the process of unfollowing every account on Facebook made their use of the service significantly better:
When Facebook shut the service down with blood-curdling legal threats, they insisted that they were simply protecting users from themselves. Sure, this browser automation tool – which just automatically clicked links on Facebook's own settings pages – seemed to do what the users wanted. But what if the user interface changed? What if so many users added this feature to Facebook without Facebook's permission that they overwhelmed Facebook's (presumably tiny and fragile) servers and crashed the system?
These arguments have lately resurfaced with Ethan Zuckerman and Knight First Amendment Institute's lawsuit to clarify that "Unfollow Everything 2.0" is legal and doesn't violate any of those "felony contempt of business model" laws:
https://pluralistic.net/2024/05/02/kaiju-v-kaiju/
Sure, Zuckerman seems like a good guy, but what if he makes a mistake and his automation tool does something you don't want? You, the Facebook user, are also a nice guy, but let's face it, you're also a naive dolt and you can't be trusted to make decisions for yourself. Those decisions can only be made by Facebook, whom we can rely upon to exercise its authority wisely.
Other versions of this argument surfaced in the debate over the EU's decision to mandate interoperability for end-to-end encrypted (E2EE) messaging through the Digital Markets Act (DMA), which would let you switch from, say, Whatsapp to Signal and still send messages to your Whatsapp contacts.
There are some good arguments that this could go horribly awry. If it is rushed, or internally sabotaged by the EU's state security services who loathe the privacy that comes from encrypted messaging, it could expose billions of people to serious risks.
But that's not the only argument that DMA opponents made: they also argued that even if interoperable messaging worked perfectly and had no security breaches, it would still be bad for users, because this would make it impossible for tech giants like Meta, Google and Apple to spy on message traffic (if not its content) and identify likely coordinated harassment campaigns. This is literally the identical argument the NSA made in support of its "metadata" mass-surveillance program: "Reading your messages might violate your privacy, but watching your messages doesn't."
This is obvious nonsense, so its proponents need an equally obviously intellectually dishonest way to defend it. When called on the absurdity of "protecting" users by spying on them against their will, they simply shake their heads and say, "You just can't understand the burdens of running a service with hundreds of millions or billions of users, and if I even tried to explain these issues to you, I would divulge secrets that I'm legally and ethically bound to keep. And even if I could tell you, you wouldn't understand, because anyone who doesn't work for a Big Tech company is a naive dolt who can't be trusted to understand how the world works (much like our users)."
Not coincidentally, this is also literally the same argument the NSA makes in support of mass surveillance, and there's a very useful name for it: scalesplaining.
Now, it's totally true that every one of us is capable of lapses in judgment that put us, and the people connected to us, at risk (my own parents gave their genome to the pseudoscience genetic surveillance company 23andme, which means they have my genome, too). A true information fiduciary shouldn't automatically deliver everything the user asks for. When the agent perceives that the user is about to put themselves in harm's way, it should throw up a roadblock and explain the risks to the user.
But the system should also let the user override it.
This is a contentious statement in information security circles. Users can be "socially engineered" (tricked), and even the most sophisticated users are vulnerable to this:
The only way to be certain a user won't be tricked into taking a course of action is to forbid that course of action under any circumstances. If there is any means by which a user can flip the "are you very sure?" circuit-breaker back on, then the user can be tricked into using that means.
This is absolutely true. As you read these words, all over the world, vulnerable people are being tricked into speaking the very specific set of directives that cause a suspicious bank-teller to authorize a transfer or cash withdrawal that will result in their life's savings being stolen by a scammer:
We keep making it harder for bank customers to make large transfers, but so long as it is possible to make such a transfer, the scammers have the means, motive and opportunity to discover how the process works, and they will go on to trick their victims into invoking that process.
Beyond a certain point, making it harder for bank depositors to harm themselves creates a world in which people who aren't being scammed find it nearly impossible to draw out a lot of cash for an emergency and where scam artists know exactly how to manage the trick. After all, non-scammers only rarely experience emergencies and thus have no opportunity to become practiced in navigating all the anti-fraud checks, while the fraudster gets to run through them several times per day, until they know them even better than the bank staff do.
This is broadly true of any system intended to control users at scale – beyond a certain point, additional security measures are trivially surmounted hurdles for dedicated bad actors and as nearly insurmountable hurdles for their victims:
At this point, we've had a couple of decades' worth of experience with technological "walled gardens" in which corporate executives get to override their users' decisions about how the system should work, even when that means reaching into the users' own computer and compelling it to thwart the user's desire. The record is inarguable: while companies often use those walls to lock bad guys out of the system, they also use the walls to lock their users in, so that they'll be easy pickings for the tech company that owns the system:
This is neatly predicted by enshittification's theory of constraints: when a company can override your choices, it will be irresistibly tempted to do so for its own benefit, and to your detriment.
What's more, the mere possibility that you can override the way the system works acts as a disciplining force on corporate executives, forcing them to reckon with your priorities even when these are counter to their shareholders' interests. If Facebook is genuinely worried that an "Unfollow Everything" script will break its servers, it can solve that by giving users an unfollow everything button of its own design. But so long as Facebook can sue anyone who makes an "Unfollow Everything" tool, they have no reason to give their users such a button, because it would give them more control over their Facebook experience, including the controls needed to use Facebook less.
It's been more than 20 years since Seth Schoen and I got a demo of Microsoft's first "trusted computing" system, with its "remote attestations," which would let remote servers demand and receive accurate information about what kind of computer you were using and what software was running on it.
This could be beneficial to the user – you could send a "remote attestation" to a third party you trusted and ask, "Hey, do you think my computer is infected with malicious software?" Since the trusted computing system produced its report on your computer using a sealed, separate processor that the user couldn't directly interact with, any malicious code you were infected with would not be able to forge this attestation.
But this remote attestation feature could also be used to allow Microsoft to block you from opening a Word document with Libreoffice, Apple Pages, or Google Docs, or it could be used to allow a website to refuse to send you pages if you were running an ad-blocker. In other words, it could transform your information fiduciary into a faithless agent.
Seth proposed an answer to this: "owner override," a hardware switch that would allow you to force your computer to lie on your behalf, when that was beneficial to you, for example, by insisting that you were using Microsoft Word to open a document when you were really using Apple Pages:
Seth wasn't naive. He knew that such a system could be exploited by scammers and used to harm users. But Seth calculated – correctly! – that the risks of having a key to let yourself out of the walled garden were less than being stuck in a walled garden where some corporate executive got to decide whether and when you could leave.
Tech executives never stopped questing after a way to turn your user agent from a fiduciary into a traitor. Last year, Google toyed with the idea of adding remote attestation to web browsers, which would let services refuse to interact with you if they thought you were using an ad blocker:
The reasoning for this was incredible: by adding remote attestation to browsers, they'd be creating "feature parity" with apps – that is, they'd be making it as practical for your browser to betray you as it is for your apps to do so (note that this is the same justification that the W3C gave for creating EME, the treacherous user agent in your browser – "streaming services won't allow you to access movies with your browser unless your browser is as enshittifiable and authoritarian as an app").
Technologists who work for giant tech companies can come up with endless scalesplaining explanations for why their bosses, and not you, should decide how your computer works. They're wrong. Your computer should do what you tell it to do:
These people can kid themselves that they're only taking away your power and handing it to their boss because they have your best interests at heart. As Upton Sinclair told us, it's impossible to get someone to understand something when their paycheck depends on them not understanding it.
The only way to get a tech boss to consistently treat you well is to ensure that if they stop, you can quit. Anything less is a one-way ticket to enshittification.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
