If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Lest anyone accuse me of bargaining in bad faith here, let me start with this admission: I don't think AI is intelligent; nor do I think that the current (admittedly impressive) statistical techniques will lead to intelligence. I think worrying about what we'll do if AI becomes intelligent is at best a distraction and at worst a cynical marketing ploy:
Now, that said: among some of the "AI doomers," I recognize kindred spirits. I, too, worry about technologies controlled by corporations that have grown so powerful that they defy regulation. I worry about how those technologies are used against us, and about how the corporations that make them are fusing with authoritarian states to create a totalitarian nightmare. I worry that technology is used to spy on and immiserate workers.
I just don't think we need AI to do those things. I think we should already be worried about those things.
Last week, I had a version of this discussion in front of several hundred people at the Bronfman Lecture in Montreal, where I appeared with Astra Taylor and Yoshua Bengio (co-winner of the Turing Prize for his work creating the "deep learning" techniques powering today's AI surge), on a panel moderated by CBC Ideas host Nahlah Ayed:
It's safe to say that Bengio and I mostly disagree about AI. He's running an initiative called "Lawzero," whose goal is to create an international AI consortium that produces AI as a "digital public good" that is designed to be open, auditable, transparent and safe:
http://lawzero.org
Bengio said he'd started Lawzero because he was convinced that AI was going to get a lot more powerful, and, in the absence of some public-spirited version of AI, we would be subject to all kinds of manipulation and surveillance, and that the resulting chaos would present a civilizational risk.
Now, as I've stated (and as I said onstage) I am not worried about any of this. I am worried about AI, though. I'm worried a fast-talking AI salesman will convince your boss to fire you and replace you with an AI that can't do your job (the salesman will be pushing on an open door, since if there's one thing bosses hate, it's paying workers).
I'm worried that the seven companies that comprise 35% of the S&P 500 are headed for bankruptcy, as soon as someone makes them stop passing around the same $100b IOU while pretending it's in all their bank accounts at once. I'm worried that when that happens, the chatbots that badly do the jobs of the people who were fired because of the AI salesman will go away, and nothing and no one will do those jobs. I'm worried that the chaos caused by vaporizing a third of the stock market will lead to austerity and thence to fascism:
I worry that the workers who did those jobs will be scattered to the four winds, retrained or "discouraged" or retired, and that the priceless process knowledge they developed over generations will be wiped out and we will have to rebuild it amidst the economic and political chaos of the burst AI bubble:
But Bengio disagrees. He's very smart, and very accomplished, and he's very certain that AI is about to become "superhuman" and do horrible things to us if we don't get a handle on it. Several times at our events, he insisted that the existence of this possibility made it wildly irresponsible not to take measures to mitigate this risk.
Though I didn't say so at the time, this struck me as an AI-inflected version of Pascal's wager:
A rational person should adopt a lifestyle consistent with the existence of God and should strive to believe in God… if God does not exist, the believer incurs only finite losses, potentially sacrificing certain pleasures and luxuries; if God does exist, the believer stands to gain immeasurably, as represented for example by an eternity in Heaven in Abrahamic tradition, while simultaneously avoiding boundless losses associated with an eternity in Hell.
https://en.wikipedia.org/wiki/Pascal%27s_wager
Smarter people than me have been poking holes in Pascal's wager for more than 350 years. But when it comes to this modern Pascal's AI Wager, I have my own objection: how do you know when you've lost?
As of this moment, the human race has lit more than $1.4t on fire to immanentize this eschaton, and it remains stubbornly disimmanentized. How much more do we need to spend before we're certain that god isn't lurking in the word-guessing program? Sam Altman says it'll take another $2-3t – call it six months' worth of all US federal spending. If we do that and we still haven't met god, are we done? Can we call it a day?
Not according to Elon Musk. Musk says we need to deconstruct the solar system and build a Dyson sphere out of all the planets to completely encase the sun, so we can harvest every photon it emits to power our word-guessing programs:
So let's say we do that and we still haven't met god – are we done? I don't see why we would be. After all, Musk's contention isn't that our sun emits one eschaton's worth of immanentizing particles. Musk just thinks that we need a lot of these sunbeams to coax god into our plane of existence. If one sun won't do it, perhaps two? Or two hundred? Or two thousand? Once we've committed the entire human species to this god-bothering project to the extent of putting two kilosuns into harness, wouldn't we be nuts to stop there? What if god is lurking in the two thousand and first sun? Making god out of algorithms is like spelling "banana" – easy to start, hard to stop.
But as Bengio and I got into it together on stage at the Montreal Centre, it occurred to me that maybe there was some common ground between us. After all, when someone starts talking about "humane technology" that respects our privacy and works for people rather than their bosses, my ears grow points. Throw in the phrase "international digital public goods" and you've got my undivided attention.
Because there's a sense in which Bengio and I are worried about exactly the same thing. I'm terrified that our planet has been colonized by artificial lifeforms that we constructed, but which have slipped our control. I'm terrified that these lifeforms corrupt our knowledge-creation process, making it impossible for us to know what's true and what isn't. I'm terrified that these lifeforms have conquered our apparatus of state – our legislatures, agencies and courts – and so that these public bodies work against the public and for our colonizing alien overlords.
The difference is, the artificial lifeforms that worry me aren't hypothetical – they're here today, amongst us, endangering the very survival of our species. These artificial lifeforms are called "limited liability corporations" and they are a concrete, imminent risk to the human race:
What's more, challenging these artificial lifeforms will require us to build massive, "international, digital public goods": a post-American internet of free/open, auditable, transparent, enshittification-resistant platforms and firmware for every purpose and device currently in service:
And even after we've built that massive, international, digital public good, we'll still face the challenge of migrating all of our systems and loved ones out of the enshitternet of defective, spying, controlling American tech exports:
Every moment that we remain stuck in the enshitternet is a moment of existential risk. At the click of a mouse, Trump could order John Deere to switch off all the tractors in your country:
He doesn't need tanks to steal Greenland. He can just shut off Denmark's access to American platforms like Office365, iOS and Android and brick the whole damned country. It would be another Strait of Hormuz, but instead of oil and fertilizer, he'd control the flow of Lego, Ozempic and deliciously strong black licorice:
These aren't risks that could develop in the future. They're the risks we're confronted with today and frankly, they're fucking terrifying.
So here's my side-bet on Pascal's Wager. If you think we need to build "international digital public goods" to head off the future risk of a colonizing, remorseless, malevolent artificial lifeform, then let us agree that the prototype for that project is the "international digital public goods" we need right now to usher in the post-American internet and save ourselves from the colonizing, remorseless, malevolent artificial lifeforms that have already got their blood-funnels jammed down our throats.
Once we defeat those alien invaders, we may find that all the people who are trying to summon the evil god have lost the wherewithal to do so, and your crisis will have been averted. But if that's not the case and the evil god still looms on our horizon, then I will make it my business to help you mobilize the legions of skilled international digital public goods producers who are still flush from their victory over the limited liability corporation, and together, we will fight the evil god you swear is in our future.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
Remember, when a blackpilled doomer calls you naive and tries to convince you that everything is hopeless, it's because they want you to be just as miserable as they are.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
People need to realize how the internet changes the context of things, because everyone acts like an authority or like they're an example of common public opinion when they're online. I recently met someone who had a lot of reddit environmentalist takes irl. And while those are things that would seem like serious climate takes online, when they say them irl in an actual conversation my internal reaction tends to be "do you need help, I don't think thinking about this is good for your mental health". And now I'm thinking of analyzing things I read online (or even say online) as if they were meatspace statements. '
Someone calling a kid's interests cringe. Imagine them doing that irl. Imagine how disgustingly unseemly an adult making fun of a teenager's interests to their face would be. Or if they're calling you out for not being good enough at something that's a hobby, imagine them saying that to you while you're showing off your hobby, imagine how terrible they look.
Someone has a hot take about how it's wrong to be attracted to adult women with short heights or flat chests, imagine someone telling that to a guy with a petite girlfriend. Someone tells you that enjoying a story with incest means you support incest, imagine someone telling that to someone reading an ASOIAF book on the train. Someone says you can't enjoy something with a problematic creator, imagine them saying that to someone reading Lovecraft on the train.
Someone has a hot take in activist space that seems really violent or somewhat facisty. Imagine them saying that irl, even with fellow activists. Imagine someone trying to defend Stalin in an actual human conversation, or trying to defend population control for environmentalist reasons.
I know I'm privileged to live in a large city and be pretty socially active, and even I can easily fall into overly online ways of thinking. But remember, even if you can't touch grass, you can imagine how things would be on the grass.
