I'm on a 20+ city book tour for my new novel PICKS AND SHOVELS. Catch me at NEW ZEALAND'S UNITY BOOKS in WELLINGTON TODAY (May 3). More tour dates (Pittsburgh, PDX, London, Manchester) here.
Have you noticed that all the buttons you click most frequently to invoke routine, useful functions in your device have been moved, and their former place is now taken up by a curiously butthole-esque icon that summons an unwanted AI?
These traps for the unwary aren't accidental, but neither are they placed there solely because tech companies think that if they can trick you into using their AI, you'll be so impressed that you'll become a regular user. To understand why you find yourself repeatedly fatfingering your way into an unwanted AI interaction ā and why those interactions are so hard to exit ā you have to understand something about both the macro- and microeconomics of high-growth tech companies.
Growth is a heady advantage for tech companies, and not because of an ideological commitment to "growth at all costs," but because companies with growth stocks enjoy substantial, material benefits. A growth stock trades at a higher "price to earnings ratio" ("P:E") than a "mature" stock. Because of this, there are a lot of actors in the economy who will accept shares in a growing company as though they were cash (indeed, some might prefer shares to cash). This means that a growing company can outbid their rivals when acquiring other companies and/or hiring key personnel, because they can bid with shares (which they get by typing zeroes into a spreadsheet), while their rivals need cash (which they can only get by selling things or borrowing money).
The problem is that all growth ends. Google has a 90% share of the search market. Google isn't going to appreciably increase the number of searchers, short of desperate gambits like raising a billion new humans to maturity and convincing them to become Google users (this is the strategy behind Google Classroom, of course). To continue posting growth, Google needs gimmicks. For example, in 2019, Google intentionally made Search less accurate so that users would have to run multiple queries (and see multiple rounds of ads) to find the answers to their questions:
Thanks to Google's monopoly, worsening search perversely resulted in increased earnings, and Wall Street rewarded Google by continuing to trade its stock with that prized high P:E. But for Google ā and other tech giants ā the most enduring and convincing growth stories comes from moving into adjacent lines of business, which is why we've lived through so many hype bubbles: metaverse, web3, cryptocurrency, and now, of course, AI.
For a company like Google, the promise of these bubbles is that it will be able to double or triple in size, by dominating an entirely new sector. With that promise comes peril: growth must eventually stop ("anything that can't go on forever eventually stops"). When that happens, the company's stock instantaneously goes from being a "growth stock" to being a "mature stock" which means that its P:E is way too high. Anyone holding growth stock knows that there will come a day when those stocks will transition, in an eyeblink, from being undervalued to being grossly overvalued, and that when that day comes, there will be a mass sell-off. If you're still holding the stock when that happens, you stand to lose bigtime:
So everyone holding a growth stock sleeps with one eye open and their fists poised over the "sell" button. Managers of growth companies know how jittery their investors are, and they do everything they can to keep the growth story alive, as a matter of life and death.
But mass sell-offs aren't just bad for the company ā it's also very bad for the company's key employees, that is, anyone who's been given stock in addition to their salary. Those people's portfolios are extremely heavy on their employer's shares, and they stand to disproportionately lose in the event of a selloff. So they are personally motivated to keep the growth story alive.
That's where these growth-at-all-stakes maneuvers bent on capturing an adjacent sector come from. If you remember the Google Plus days, you'll remember that every Google service you interacted with had some important functionality ripped out of it and replaced with a G+-based service. To make sure that happened, Google's bosses decreed that the company's bonuses would be tied to the amount of G+ activity each division generated. In companies where bonuses can amount to 90% of your annual salary or more, this was a powerful motivator. It meant that every product team at Google was fully aligned on a project to cram G+ buttons into their product design. Whether or not these made sense for users, they always made sense for the product team, whose ability to take a fancy Christmas holiday, buy a new car, or pay their kids' private school tuition depended on getting you to use G+.
Once you understand how corporate growth stories are converted to "key performance indicators" that drive product design, many of the annoyances of digital services suddenly make a great deal of sense. You know how it's almost impossible to watch a show on a streaming video service without accidentally tapping a part of the screen that whisks you to a completely different video?
The reason you have to handle your phone like a photonegative while watching a movie ā the reason every millimeter of screen real-estate has been boobytrapped with an icon that takes you somewhere else ā is that streaming services believe that their customers are apt to leave when they feel like there's nothing new to watch. These bosses have made their product teams' bonuses dependent on successfully "recommending" a show you've never seen or expressed any interest in to you:
Of course, bosses understand that their workers will be tempted to game this metric. They want to distinguish between "real" clicks that lead to interest in a new video, and fake fatfinger clicks that you instantaneously regret. The easiest way to distinguish between these two types of click is to measure how long you watch the new show before clicking away.
Of course, this is also entirely gameable: all the product manager has to do is take away the "back" button, so that an accidental click to a new video is extremely hard to cancel. The five seconds you spend figuring out how to get back to your show are enough to count as a successful recommendation, and the product team is that much closer to a luxury ski vacation next Christmas.
So this is why you keep invoking AI by accident, and why the AI that is so easy to invoke is so hard to dispel. Like a demon, a chatbot is much easier to summon than it is to rid yourself of.
Google is an especially grievous offender here. Familiar buttons in Gmail, Gdocs, and the Android message apps have been replaced with AI-summoning fatfinger traps. Android is filled with these pitfalls ā for example, the bottom-of-screen swipe gesture used to switch between open apps now summons an AI, while ridding yourself of that AI takes multiple clicks.
This is an entirely material phenomenon. Google doesn't necessarily believe that you will ever want to use AI, but they must convince investors that their AI offerings are "getting traction." Google ā like other tech companies ā gets to invent metrics to prove this proposition, like "how many times did a user click on the AI button" and "how long did the user spend with the AI after clicking?" The fact that your entire "AI use" consisted of hunting for a way to get rid of the AI doesn't matter ā at least, not for the purposes of maintaining Google's growth story.
Goodhart's Law holds that "When a measure becomes a target, it ceases to be a good measure." For Google and other AI narrative-pushers, every measure is designed to be a target, a line that can be made to go up, as managers and product teams align to sell the company's growth story, lest we all sell off the company's shares.
If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
ā Live Streamingā Interactive Chatā Private Showsā HD Quality
Anya is LIVE right now
FREE
Free to watch ā¢ No registration required ā¢ HD streaming
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
ā Live Streamingā Interactive Chatā Private Showsā HD Quality
Anya is LIVE right now
FREE
Free to watch ā¢ No registration required ā¢ HD streaming
