Librarian! Rafia!

Interview with Librarians on 'Rescuing Public Datasets the U.S. Gov't Deletes": 

Why is the American market so prized by Big Tech? Because it the only country in the world at the center of a Venn diagram with three overlapping circles. America is the only country in the world that is:

a) populous;

b) wealthy; and

c) totally lacking in legal privacy protections.

The US Congress last updated American consumer privacy law in 1988, when the Video Privacy Protection Act was passed to protect Americans from the high-tech threat of…video store clerks leaking your rental history to the newspapers. Despite the bewildering, obvious, serious privacy risks that have emerged since Die Hard was in theaters, Congress has done nothing to extend Americans' consumer privacy rights.

The case is the first federal antitrust case to center on an algorithm’s role in antitrust violations. “Using software as the sharing mechanism does not immunize this scheme from Sherman Act liability,” Attorney General Merrick Garland said in a statement. 

RealPage’s system collects and analyzes vast amounts of private data from its subscribers, including floor plans, lease agreements, discounts, and expiration dates, according to the Justice Department’s complaint. The private data is then supplemented by 50,000 monthly phone calls made by RealPage to 52,000 properties in America. 

The algorithm uses that information to generate a “smoothed” market minimum and maximum for each unit, creating a stable and uniform low and high rate across properties. The system will not suggest rent rates below the market minimum but will regularly offer above the maximum to inch the price upwards. The system is designed to push most landlords to auto-accept rent recommendations while requiring leasing agents to jump through several bureaucratic hoops to offer less than the algorithm suggests. According to the complaint, this design gives pricing authority to one central business: RealPage.

...“revenue protection" mode, which instructs landlords to remove vacant units from the market rather than lower prices. In one instance cited in the complaint, RealPage defended this approach to a landlord wondering why prices weren’t dropping in the face of mass vacancies, stating that "the model still sees the way to make more revenue is to lease fewer units at higher prices."

If you come to edtech through schooling and other venues of compulsory face-to-face learning, you understand quite well that just because a student’s body is at their desk that doesn’t mean their spirit or mind is present there as well. 

If you come to edtech through YouTube, it is easy to convince yourself that the people who are watching your videos are also learning from your videos, that they are also enjoying learning from your videos, that they would also enjoy learning from more of your videos. Certainly, all of that is true for some percentage of Karpathy’s two million views, but far less than 100%. I would anchor our predictions at roughly 5%.

Karpathy is now jumping into the top of a flume ride full of people who found early success explaining stuff on YouTube and later struggled to fulfill their greater ambitions for education, people like:

Sebastian Thrun, who created a very popular YouTube playlist on artificial intelligence, and later turned that playlist into Udacity, and later fell far short of his ambitions in K-16 schooling, and later pivoted to friendlier terrain in corporate education, and later sold his company altogether for what is widely assumed to be a steep discount.

Andrew Ng, who put videos of his Stanford lectures online, and later created Coursera with Daphne Koeller to surround those videos with a learning management system, and later saw course completion rates hover around 10%, and later saw the stock price of his company decline 85% from its initial public offering.

Sal Khan, a popular YouTuber whose videos were first watched by family members and then by millions, who later scaled his ideas about learning into Khan Academy, a platform that has demonstrated significant learning gains in studies where researchers first throw out 95% of the students in the study.

A large reason why these edtech startups do not work well for the majority of students is that the majority of students are not particularly interested in reading academic text or watching sequences of explanatory videos. If they were, we would have solved mass education many centuries ago with the printing press. Thomas Edison would have been correct in 1922 that “the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks.”

Whether generative media will over- or underperform static media here remains to be studied, but let’s just say I have my suspicions.

....

But the edtech graveyard is stacked several caskets deep with resources created by autodidacts—people who are highly self-motivated and highly self-directed, people who are just fine learning by themselves, thank you—for autodidacts. We might estimate the population of autodidacts at about (well look at that) 5% of the overall population.

Aug 2024)

Today's links

"Disenshittify or Die": My speech from Defcon 32.

Hey look at this: Delights to delectate.

This day in history: 2009, 2014, 2019, 2023

Upcoming appearances: Where to find me.

Recent appearances: Where I've been.

Latest books: You keep readin' em, I'll keep writin' 'em.

Upcoming books: Like I said, I'll keep writin' 'em.

Colophon: All the rest.

"Disenshittify or Die" (permalink)

Last weekend, I traveled to Las Vegas for Defcon 32, where I had the immense privilege of giving a solo talk on Track 1, entitled "Disenshittify or die! How hackers can seize the means of computation and build a new, good internet that is hardened against our asshole bosses' insatiable horniness for enshittification":

https://info.defcon.org/event/?id=54861

This was a followup to last year's talk, "An Audacious Plan to Halt the Internet's Enshittification," a talk that kicked off a lot of international interest in my analysis of platform decay ("enshittification"):

The Defcon organizers have earned a restful week or two, and that means that the video of my talk hasn't yet been posted to Defcon's Youtube channel, so in the meantime, I thought I'd post a lightly edited version of my speech crib. If you're headed to Burning Man, you can hear me reprise this talk at Palenque Norte (7&E); I'm kicking off their lecture series on Tuesday, Aug 27 at 1PM.

==

What the fuck happened to the old, good internet?

I mean, sure, our bosses were a little surveillance-happy, and they were usually up for sharing their data with the NSA, and whenever there was a tossup between user security and growth, it was always YOLO time.

But Google Search used to work. Facebook used to show you posts from people you followed. Uber used to be cheaper than a taxi and pay the driver more than a cabbie made. Amazon used to sell products, not Shein-grade self-destructing dropshipped garbage from all-consonant brands. Apple used to defend your privacy, rather than spying on you with your no-modifications-allowed Iphone.

There was a time when you searching for an album on Spotify would get you that album – not a playlist of insipid AI-generated covers with the same name and art.

Microsoft used to sell you software – sure, it was buggy – but now they just let you access apps in the cloud, so they can watch how you use those apps and strip the features you use the most out of the basic tier and turn them into an upcharge.

What – and I cannot stress this enough – the fuck happened?!

I’m talking about enshittification.

Here’s what enshittification looks like from the outside: First, you see a company that’s being good to its end users. Google puts the best search results at the top; Facebook shows you a feed of posts from people and groups you followl; Uber charges small dollars for a cab; Amazon subsidizes goods and returns and shipping and puts the best match for your product search at the top of the page.

That’s stage one, being good to end users. But there’s another part of this stage, call it stage 1a). That’s figuring out how to lock in those users.

There’s so many ways to lock in users.

If you’re Facebook, the users do it for you. You joined Facebook because there were people there you wanted to hang out with, and other people joined Facebook to hang out with you.

That’s the old “network effects” in action, and with network effects come “the collective action problem." Because you love your friends, but goddamn are they a pain in the ass! You all agree that FB sucks, sure, but can you all agree on when it’s time to leave?

No way.

Can you agree on where to go next?

Hell no.

You’re there because that’s where the support group for your rare disease hangs out, and your bestie is there because that’s where they talk with the people in the country they moved away from, then there’s that friend who coordinates their kid’s little league car pools on FB, and the best dungeon master you know isn’t gonna leave FB because that’s where her customers are.

So you’re stuck, because even though FB use comes at a high cost – your privacy, your dignity and your sanity – that’s still less than the switching cost you’d have to bear if you left: namely, all those friends who have taken you hostage, and whom you are holding hostage

Now, sometimes companies lock you in with money, like Amazon getting you to prepay for a year’s shipping with Prime, or to buy your Audible books on a monthly subscription, which virtually guarantees that every shopping search will start on Amazon, after all, you’ve already paid for it.

Sometimes, they lock you in with DRM, like HP selling you a printer with four ink cartridges filled with fluid that retails for more than $10,000/gallon, and using DRM to stop you from refilling any of those ink carts or using a third-party cartridge. So when one cart runs dry, you have to refill it or throw away your investment in the remaining three cartridges and the printer itself.

Sometimes, it’s a grab bag:

You can’t run your Ios apps without Apple hardware;

you can’t run your Apple music, books and movies on anything except an Ios app;

your iPhone uses parts pairing – DRM handshakes between replacement parts and the main system – so you can’t use third-party parts to fix it; and

every OEM iPhone part has a microscopic Apple logo engraved on it, so Apple can demand that the US Customs and Border Service seize any shipment of refurb Iphone parts as trademark violations.

Think Different, amirite?

Getting you locked in completes phase one of the enshittification cycle and signals the start of phase two: making things worse for you to make things better for business customers.

For example, a platform might poison its search results, like Google selling more and more of its results pages to ads that are identified with lighter and lighter tinier and tinier type.

Or Amazon selling off search results and calling it an “ad” business. They make $38b/year on this scam. The first result for your search is, on average, 29% more expensive than the best match for your search. The first row is 25% more expensive than the best match. On average, the best match for your search is likely to be found seventeen places down on the results page.

Other platforms sell off your feed, like Facebook, which started off showing you the things you asked to see, but now the quantum of content from the people you follow has dwindled to a homeopathic residue, leaving a void that Facebook fills with things that people pay to show you: boosted posts from publishers you haven’t subscribed to, and, of course, ads.

Now at this point you might be thinking ‘sure, if you’re not paying for the product, you’re the product.'

Bullshit!

Bull.

Shit.

The people who buy those Google ads? They pay more every year for worse ad-targeting and more ad-fraud

Those publishers paying to nonconsensually cram their content into your Facebook feed? They have to do that because FB suppresses their ability to reach the people who actually subscribed to them

The Amazon sellers with the best match for your query have to outbid everyone else just to show up on the first page of results. It costs so much to sell on Amazon that between 45-51% of every dollar an independent seller brings in has to be kicked up to Don Bezos and the Amazon crime family. Those sellers don’t have the kind of margins that let them pay 51% They have to raise prices in order to avoid losing money on every sale.

"But wait!" I hear you say!

[Come on, say it!]

"But wait! Things on Amazon aren’t more expensive that things at Target, or Walmart, or at a mom and pop store, or direct from the manufacturer.

"How can sellers be raising prices on Amazon if the price at Amazon is the same as at is everywhere else?"

[Any guesses?!]

That’s right, they charge more everywhere. They have to. Amazon binds its sellers to a policy called “most favored nation status,” which says they can’t charge more on Amazon than they charge elsewhere, including direct from their own factory store.

So every seller that wants to sell on Amazon has to raise their prices everywhere else.

Now, these sellers are Amazon’s best customers. They’re paying for the product, and they’re still getting screwed.

Paying for the product doesn’t fill your vapid boss’s shriveled heart with so much joy that he decides to stop trying to think of ways to fuck you over.

Look at Apple. Remember when Apple offered every Ios user a one-click opt out for app-based surveillance? And 96% of users clicked that box?

(The other four percent were either drunk or Facebook employees or drunk Facebook employees.)

That cost Facebook at least ten billion dollars per year in lost surveillance revenue?

I mean, you love to see it.

But did you know that at the same time Apple started spying on Ios users in the same way that Facebook had been, for surveillance data to use to target users for its competing advertising product?

Your Iphone isn’t an ad-supported gimme. You paid a thousand fucking dollars for that distraction rectangle in your pocket, and you’re still the product. What’s more, Apple has rigged Ios so that you can’t mod the OS to block its spying.

If you’re not not paying for the product, you’re the product, and if you are paying for the product, you’re still the product.

Just ask the farmers who are expected to swap parts into their own busted half-million dollar, mission-critical tractors, but can’t actually use those parts until a technician charges them $200 to drive out to the farm and type a parts pairing unlock code into their console.

John Deere’s not giving away tractors. Give John Deere a half mil for a tractor and you will be the product.

Please, my brothers and sisters in Christ. Please! Stop saying ‘if you’re not paying for the product, you’re the product.’

OK, OK, so that’s phase two of enshittification.

Phase one: be good to users while locking them in.

Phase two: screw the users a little to you can good to business customers while locking them in.

Phase three: screw everybody and take all the value for yourself. Leave behind the absolute bare minimum of utility so that everyone stays locked into your pile of shit.

Enshittification: a tragedy in three acts.

That’s what enshittification looks like from the outside, but what’s going on inside the company? What is the pathological mechanism? What sci-fi entropy ray converts the excellent and useful service into a pile of shit?

That mechanism is called twiddling. Twiddling is when someone alters the back end of a service to change how its business operates, changing prices, costs, search ranking, recommendation criteria and other foundational aspects of the system.

Digital platforms are a twiddler’s utopia. A grocer would need an army of teenagers with pricing guns on rollerblades to reprice everything in the building when someone arrives who’s extra hungry.

Whereas the McDonald’s Investments portfolio company Plexure advertises that it can use surveillance data to predict when an app user has just gotten paid so the seller can tack an extra couple bucks onto the price of their breakfast sandwich.

And of course, as the prophet William Gibson warned us, ‘cyberspace is everting.' With digital shelf tags, grocers can change prices whenever they feel like, like the grocers in Norway, whose e-ink shelf tags change the prices 2,000 times per day.

Every Uber driver is offered a different wage for every job. If a driver has been picky lately, the job pays more. But if the driver has been desperate enough to grab every ride the app offers, the pay goes down, and down, and down.

The law professor Veena Dubal calls this ‘algorithmic wage discrimination.' It’s a prime example of twiddling.

Every youtuber knows what it’s like to be twiddled. You work for weeks or months, spend thousands of dollars to make a video, then the algorithm decides that no one – not your own subscribers, not searchers who type in the exact name of your video – will see it.

Why? Who knows? The algorithm’s rules are not public.

Because content moderation is the last redoubt of security through obscurit: they can’t tell you what the como algorithm is downranking because then you’d cheat.

Youtube is the kind of shitty boss who docks every paycheck for all the rules you’ve broken, but won’t tell you what those rules were, lest you figure out how to break those rules next time without your boss catching you.

Twiddling can also work in some users’ favor, of course. Sometimes platforms twiddle to make things better for end users or business customers.

For example, Emily Baker-White from Forbes revealed the existence of a back-end feature that Tiktok’s management can access they call the “heating tool.”

When a manager applies the heating toll to a performer’s account, that performer’s videos are thrust into the feeds of millions of users, without regard to whether the recommendation algorithm predicts they will enjoy that video.

Why would they do this? Well, here’s an analogy from my boyhood I used to go to this traveling fair that would come to Toronto at the end of every summer, the Canadian National Exhibition. If you’ve been to a fair like the Ex, you know that you can always spot some guy lugging around a comedically huge teddy bear.

Nominally, you win that teddy bear by throwing five balls in a peach-basket, but to a first approximation, no one has ever gotten five balls to stay in that peach-basket.

That guy “won” the teddy bear when a carny on the midway singled him out and said, "fella, I like your face. Tell you what I’m gonna do: You get just one ball in the basket and I’ll give you this keychain, and if you amass two keychains, I’ll let you trade them in for one of these galactic-scale teddy-bears."

That’s how the guy got his teddy bear, which he now has to drag up and down the midway for the rest of the day.

Why the hell did that carny give away the teddy bear? Because it turns the guy into a walking billboard for the midway games. If that dopey-looking Judas Goat can get five balls into a peach basket, then so can you.

Except you can’t.

Tiktok’s heating tool is a way to give away tactical giant teddy bears. When someone in the TikTok brain trust decides they need more sports bros on the platform, they pick one bro out at random and make him king for the day, heating the shit out of his account.

That guy gets a bazillion views and he starts running around on all the sports bro forums trumpeting his success: *I am the Louis Pasteur of sports bro influencers!"

The other sports bros pile in and start retooling to make content that conforms to the idiosyncratic Tiktok format. When they fail to get giant teddy bears of their own, they assume that it’s because they’re doing Tiktok wrong, because they don’t know about the heating tool.

But then comes the day when the TikTok Star Chamber decides they need to lure in more astrologers, so they take the heat off that one lucky sports bro, and start heating up some lucky astrologer.

Giant teddy bears are all over the place: those Uber drivers who were boasting to the NYT ten years ago about earning $50/hour? The Substackers who were rolling in dough? Joe Rogan and his hundred million dollar Spotify payout? Those people are all the proud owners of giant teddy bears, and they’re a steal.

Because every dollar they get from the platform turns into five dollars worth of free labor from suckers who think they just internetting wrong.

Giant teddy bears are just one way of twiddling. Platforms can play games with every part of their business logic, in highly automated ways, that allows them to quickly and efficiently siphon value from end users to business customers and back again, hiding the pea in a shell game conducted at machine speeds, until they’ve got everyone so turned around that they take all the value for themselves.

That’s the how: How the platforms do the trick where they are good to users, then lock users in, then maltreat users to be good to business customers, then lock in those business customers, then take all the value for themselves.

So now we know what is happening, and how it is happening, all that’s left is why it’s happening.

Now, on the one hand, the why is pretty obvious. The less value that end-users and business customers capture, the more value there is left to divide up among the shareholders and the executives.

That’s why, but it doesn’t tell you why now. Companies could have done this shit at any time in the past 20 years, but they didn’t. Or at least, the successful ones didn’t. The ones that turned themselves into piles of shit got treated like piles of shit. We avoided them and they died.

Remember Myspace? Yahoo Search? Livejournal? Sure, they’re still serving some kind of AI slop or programmatic ad junk if you hit those domains, but they’re gone.

And there’s the clue: It used to be that if you enshittified your product, bad things happened to your company. Now, there are no consequences for enshittification, so everyone’s doing it.

Let’s break that down: What stops a company from enshittifying?

There are four forces that discipline tech companies. The first one is, obviously, competition.

If your customers find it easy to leave, then you have to worry about them leaving

Many factors can contribute to how hard or easy it is to depart a platform, like the network effects that Facebook has going for it. But the most important factor is whether there is anywhere to go.

Back in 2012, Facebook bought Insta for a billion dollars. That may seem like chump-change in these days of eleven-digit Big Tech acquisitions, but that was a big sum in those innocent days, and it was an especially big sum to pay for Insta. The company only had 13 employees, and a mere 25 million registered users.

But what mattered to Zuckerberg wasn’t how many users Insta had, it was where those users came from.

[Does anyone know where those Insta users came from?]

That’s right, they left Facebook and joined Insta. They were sick of FB, even though they liked the people there, they hated creepy Zuck, they hated the platform, so they left and they didn’t come back.

So Zuck spent a cool billion to recapture them, A fact he put in writing in a midnight email to CFO David Ebersman, explaining that he was paying over the odds for Insta because his users hated him, and loved Insta. So even if they quit Facebook (the platform), they would still be captured Facebook (the company).

Now, on paper, Zuck’s Instagram acquisition is illegal, but normally, that would be hard to stop, because you’d have to prove that he bought Insta with the intention of curtailing competition.

But in this case, Zuck tripped over his own dick: he put it in writing.

But Obama’s DoJ and FTC just let that one slide, following the pro-monopoly policies of Reagan, Bush I, Clinton and Bush II, and setting an example that Trump would follow, greenlighting gigamergers like the catastrophic, incestuous Warner-Discovery marriage.

Indeed, for 40 years, starting with Carter, and accelerating through Reagan, the US has encouraged monopoly formation, as an official policy, on the grounds that monopolies are “efficient.”

If everyone is using Google Search, that’s something we should celebrate. It means they’ve got the very best search and wouldn’t it be perverse to spend public funds to punish them for making the best product?

But as we all know, Google didn’t maintain search dominance by being best. They did it by paying bribes. More than 20 billion per year to Apple alone to be the default Ios search, plus billions more to Samsung, Mozilla, and anyone else making a product or service with a search-box on it, ensuring that you never stumble on a search engine that’s better than theirs.

Which, in turn, ensured that no one smart invested big in rival search engines, even if they were visibly, obviously superior. Why bother making something better if Google’s buying up all the market oxygen before it can kindle your product to life?

Facebook, Google, Microsoft, Amazon – they’re not “making things” companies, they’re “buying things” companies, taking advantage of official tolerance for anticompetitive acquisitions, predatory pricing, market distorting exclusivity deals and other acts specifically prohibited by existing antitrust law.

Their goal is to become too big to fail, because that makes them too big to jail, and that means they can be too big to care.

Which is why Google Search is a pile of shit and everything on Amazon is dropshipped garbage that instantly disintegrates in a cloud of offgassed volatile organic compounds when you open the box.

Once companies no longer fear losing your business to a competitor, it’s much easier for them to treat you badly, because what’re you gonna do?

Remember Lily Tomlin as Ernestine the AT&T operator in those old SNL sketches? “We don’t care. We don’t have to. We’re the phone company.”

Competition is the first force that serves to discipline companies and the enshittificatory impulses of their leadership, and we just stopped enforcing competition law.

It takes a special kind of smooth-brained asshole – that is, an establishment economist – to insist that the collapse of every industry from eyeglasses to vitamin C into a cartel of five or fewer companies has nothing to do with policies that officially encouraged monopolization.

It’s like we used to put down rat poison and we didn’t have a rat problem.Then these dickheads convinced us that rats were good for us and we stopped putting down rat poison, and now rats are gnawing our faces off and they’re all running around saying, "Who’s to say where all these rats came from? Maybe it was that we stopped putting down poison, but maybe it’s just the Time of the Rats. The Great Forces of History bearing down on this moment to multiply rats beyond all measure!"

Antitrust didn’t slip down that staircase and fall spine-first on that stiletto: they stabbed it in the back and then they pushed it.

And when they killed antitrust, they also killed regulation, the second force that disciplines companies. Regulation is possible, but only when the regulator is more powerful than the regulated entities. When a company is bigger than the government, it gets damned hard to credibly threaten to punish that company, no matter what its sins.

That’s what protected IBM for all those years when it had its boot on the throat of the American tech sector. Do you know, the DOJ fought to break up IBM in the courts from 1970-1982, and that every year, for 12 consecutive years, IBM spent more on lawyers to fight the USG than the DOJ Antitrust Division spent on all the lawyers fighting every antitrust case in the entire USA?

IBM outspent Uncle Sam for 12 years. People called it “Antitrust’s Vietnam.” All that money paid off, because by 1982, the president was Ronald Reagan, a man whose official policy was that monopolies were “efficient." So he dropped the case, and Big Blue wriggled off the hook.

It’s hard to regulate a monopolist, and it’s hard to regulate a cartel. When a sector is composed of hundreds of competing companies, they compete. They genuinely fight with one another, trying to poach each others’ customers and workers. They are at each others’ throats.

It’s hard enough for a couple hundred executives to agree on anything. But when they’re legitimately competing with one another, really obsessing about how to eat each others’ lunches, they can’t agree on anything.

The instant one of them goes to their regulator with some bullshit story, about how it’s impossible to have a decent search engine without fine-grained commercial surveillance; or how it’s impossible to have a secure and easy to use mobile device without a total veto over which software can run on it; or how it’s impossible to administer an ISP’s network unless you can slow down connections to servers whose owners aren’t paying bribes for “premium carriage"; there’s some *other company saying, “That’s bullshit”

“We’ve managed it! Here’s our server logs, our quarterly financials and our customer testimonials to prove it.”

100 companies are a rabble, they're a mob. They can’t agree on a lobbying position. They’re too busy eating each others’ lunch to agree on how to cater a meeting to discuss it.

But let those hundred companies merge to monopoly, absorb one another in an incestuous orgy, turn into five giant companies, so inbred they’ve got a corporate Habsburg jaw, and they become a cartel.

It’s easy for a cartel to agree on what bullshit they’re all going to feed their regulator, and to mobilize some of the excess billions they’ve reaped through consolidation, which freed them from “wasteful competition," so they can capture their regulators completely.

You know, Congress used to pass federal consumer privacy laws? Not anymore.

The last time Congress managed to pass a federal consumer privacy law was in 1988: The Video Privacy Protection Act. That’s a law that bans video-store clerks from telling newspapers what VHS cassettes you take home. In other words, it regulates three things that have effectively ceased to exist.

The threat of having your video rental history out there in the public eye was not the last or most urgent threat the American public faced, and yet, Congress is deadlocked on passing a privacy law.

Tech companies’ regulatory capture involves a risible and transparent gambit, that is so stupid, it’s an insult to all the good hardworking risible transparent ruses out there.

Namely, they claim that when they violate your consumer, privacy or labor rights, It’s not a crime, because they do it with an app.

Algorithmic wage discrimination isn’t illegal wage theft: we do it with an app.

Spying on you from asshole to appetite isn’t a privacy violation: we do it with an app.

"AI companies are generally secretive about their sources of training data, but an investigation by Proof News found some of the wealthiest AI companies in the world have used material from thousands of YouTube videos to train AI. Companies did so despite YouTube’s rules against harvesting materials from the platform without permission.

Our investigation found that subtitles from 173,536 YouTube videos, siphoned from more than 48,000 channels, were used by Silicon Valley heavyweights, including Anthropic, Nvidia, Apple, and Salesforce.

...

“It’s theft,” said Dave Wiskus, the CEO of Nebula, a streaming service partially owned by its creators, some of whom have had their work taken from YouTube to train AI.

Wiskus said it’s “disrespectful” to use creators’ work without their consent, especially since studios may use “generative AI to replace as many of the artists along the way as they can.”

“Will this be used to exploit and harm artists? Yes, absolutely,” Wiskus said. 

..The Times’ investigation also found OpenAI used YouTube videos without authorization.

Search the YouTube Videos Secretly Powering Generative AI

Annals of Artificial Intelligence

Will A.I. Become the New McKinsey?

As it’s currently imagined, the technology promises to concentrate wealth and disempower workers. Is an alternative possible?

By Ted Chiang

"It is easy to take for granted the value of data. It has come to seem self-evidently useful, as necessary and natural as water. It doesn’t even matter what has been measured and datafied; data in the abstract, as an idea, is taken to be a good thing, and of course there should be more of it, to enrich our knowledge of the world and to make anything that is “data-driven” work better. If data is being collected but not leveraged, why bother? Why have an archive of implosion images if not to simulate any implosion image imaginable?

But to accept that at face value would be to neglect the vast infrastructure involved not merely in collecting it and making it useful and tradable, but also establishing its reputation for objectivity. Measurement is an ideology; among its central tenets is that there is no such thing as datafication but just data itself, naturally given by the things in themselves. It presents itself as a form of representation that transcends representation: Data is no longer about the world but is instead taken to be the world itself, as though materiality were a matter not of atoms but of information. The image of an implosion is an implosion.

Likewise, this ideology would persuade us to ignore the market for data, which shapes what is measured and how, and have us believe it is more like a natural resource, a found material waiting for refinement rather than a structured informational good without any natural status at all. Implosions just happen.

Calls to measure everything and collect as much data as possible are offered as efficient strategies to better grasp the world as it is. But measurement is an act of power, not observation. Datafication always reifies an existing distribution of power that grants the measurers the ability to decide which aspects of the world count and which ones don’t. Having measurements taken as objective — having representations be treated as realities — requires power and recurrent processes of legitimation."

...

"When trying to understand systems, one really eye-opening and fundamental insight is to realize that the machine is never broken. What I mean by this is, when observing the outcomes of a particular system or institution, it’s very useful to start from the assumption that the outputs or impacts of that system are precisely what it was designed to do — whether we find those results to be good, bad or mixed.

The most effective and broadly-understand articulation of this idea is the phrase, “the purpose of a system is what it does”, often abbreviated as POSIWID.

....Through this lens, we can understand a lot about the world, and how we can be more effective in it. If we accept that the machine is never broken (except in the case of the McDonald’s ice cream machine), then we can recognize that driving change requires us to make the machine want something else. If the purpose of a system is what it does, and we don’t like what it does, then we have to change the system. And we change the system by making everyone involved, especially those in authority, feel urgency about changing the real-world impacts that a system has.

V.A. Uses a Suicide Prevention Algorithm To Decide Who Gets Extra Help. It Favors White Men.

An AI program designed to prevent suicide among U.S. military veterans prioritizes White men and ignores survivors of sexual violenceBy Aaron Glantz for The Fuller Project

"...it also revealed the emptiness of Google’s new approach to search. Without any knowledge base of its own, the company’s large language model simply summarizes and regurgitates what it finds on the web according to unknown criteria — an approach Today in Tabs’ Rusty Foster accurately calls automated plagiarism.

Google blamed all this on its users, Kylie Robison reported at The Verge:Google spokesperson Meghann Farnsworth said the mistakes came from “generally very uncommon queries, and aren’t representative of most people’s experiences.” The company has taken action against violations of its policies, she said, and are using these “isolated examples” to continue to refine the product.

… plenty of these queries were common enough. Asking about the race or religion of US presidents, or how to get cheese to stick to pizza, are straightforward uses of Google that the previous, non-AI-degraded version of the search engine handled just fine.

"But there is a yawning gap between "AI tools can be handy for some things" and the kinds of stories AI companies are telling (and the media is uncritically reprinting). And when it comes to the massively harmful ways in which large language models (LLMs) are being developed and trained, the feeble argument that "well, they can sometimes be handy..." doesn't offer much of a justification.

...

When I boil it down, I find my feelings about AI are actually pretty similar to my feelings about blockchains: they do a poor job of much of what people try to do with them, they can't do the things their creators claim they one day might, and many of the things they are well suited to do may not be altogether that beneficial. And while I do think that AI tools are more broadly useful than blockchains, they also come with similarly monstrous costs.

...

But I find one common thread among the things AI tools are particularly suited to doing: do we even want to be doing these things? If all you want out of a meeting is the AI-generated summary, maybe that meeting could've been an email. If you're using AI to write your emails, and your recipient is using AI to read them, could you maybe cut out the whole thing entirely? If mediocre, auto-generated reports are passing muster, is anyone actually reading them? Or is it just middle-management busywork?

...

Costs and benefits

Throughout all this exploration and experimentation I've felt a lingering guilt, and a question: is this even worth it? And is it ethical for me to be using these tools, even just to learn more about them in hopes of later criticizing them more effectively?

The costs of these AI models are huge, and not just in terms of the billions of dollars of VC funds they're burning through at incredible speed. These models are well known to require far more computing power (and thus electricity and water) than a traditional web search or spellcheck. Although AI company datacenters are not intentionally wasting electricity in the same way that bitcoin miners perform millions of useless computations, I'm also not sure that generating a picture of a person with twelve fingers on each hand or text that reads as though written by an endlessly smiling children's television star who's being held hostage is altogether that much more useful than a bitcoin.

There's a huge human cost as well. Artificial intelligence relies heavily upon "ghost labor": work that appears to be performed by a computer, but is actually delegated to often terribly underpaid contractors, working in horrible conditions, with few labor protections and no benefits. There is a huge amount of work that goes into compiling and labeling data to feed into these models, and each new model depends on ever-greater amounts of said data — training data which is well known to be scraped from just about any possible source, regardless of copyright or consent. And some of these workers suffer serious psychological harm as a result of exposure to deeply traumatizing material in the course of sanitizing datasets or training models to perform content moderation tasks.

Then there's the question of opportunity cost to those who are increasingly being edged out of jobs by LLMs,i despite the fact that AI often can't capably perform the work they were doing. Should I really be using AI tools to proofread my newsletters when I could otherwise pay a real person to do that proofreading? Even if I never intended to hire such a person?

Or, more accurately, by managers and executives who believe the marketing hype out of AI companies that proclaim that their tools can replace workers, without seeming to understand at all what those workers do.

What I'd love to see is a unversity that responds to the pressure and hype by saying something like this:

"These "hallucinations" are a stubbornly persistent feature of large language models, because these models only give the illusion of understanding; in reality, they are just sophisticated forms of autocomplete, drawing on huge databases to make shrewd (but reliably fallible) guesses about which word comes next:

Guessing the next word without understanding the meaning of the resulting sentence makes unsupervised LLMs unsuitable for high-stakes tasks. The whole AI bubble is based on convincing investors that one or more of the following is true:

I. There are low-stakes, high-value tasks that will recoup the massive costs of AI training and operation;

II. There are high-stakes, high-value tasks that can be made cheaper by adding an AI to a human operator;

III. Adding more training data to an AI will make it stop hallucinating, so that it can take over high-stakes, high-value tasks without a "human in the loop."

These are dubious propositions. There's a universe of low-stakes, low-value tasks – political disinformation, spam, fraud, academic cheating, nonconsensual porn, dialog for video-game NPCs – but none of them seem likely to generate enough revenue for AI companies to justify the billions spent on models, nor the trillions in valuation attributed to AI companies:

https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/

The proposition that increasing training data will decrease hallucinations is hotly contested among AI practitioners. I confess that I don't know enough about AI to evaluate opposing sides' claims, but even if you stipulate that adding lots of human-generated training data will make the software a better guesser, there's a serious problem. All those low-value, low-stakes applications are flooding the internet with botshit. After all, the one thing AI is unarguably very good at is producing bullshit at scale. As the web becomes an anaerobic lagoon for botshit, the quantum of human-generated "content" in any internet core sample is dwindling to homeopathic levels:

This means that adding another order of magnitude more training data to AI won't just add massive computational expense – the data will be many orders of magnitude more expensive to acquire, even without factoring in the additional liability arising from new legal theories about scraping:

"FORGING POLITICAL WEAPONS

By giving the President — any President — the power to designate which platforms are “foreign adversary controlled,” this bill provides the executive with a powerful new tool to coerce tech companies and exert control over the information ecosystem that platforms shape. While the “foreign adversaries” designation is narrow, and only applies to China, Iran, North Korea, and Russia for now, the long history of dubious fact patterns and malformed evidence that has sufficed to establish links between targeted organizations or individuals and malign actors should chill any optimism suggested by this limitation.

Consider, for instance, the case of the Holy Land Foundation. The Holy Land Foundation was, in the 1990s, the largest Muslim charity in the United States, and supported aid organizations in Palestine — many of which were also supported by the U.S. government. In the wake of 9/11, HLF was targeted by the Bush administration’s DOJ. Even though no direct or knowing link to terrorist activity was ever alleged, let alone concretely established, HLF was still designated a terrorist organization. Its assets were seized, and its leaders ultimately convicted of material support for Hamas “on the notion that the social programs they financed help win the ‘hearts and minds’ of Palestinian people for Hamas.” This was in a climate, not unlike our own, marked by strong Islamophobia.

Historically, such designations have proven flexible and highly conducive to political weaponization. From the McCarthy era, to the post-9/11 patriotic frenzy, to the recent wave of bans targeting pro-Palestinian student organizations, there’s a well-worn template that should give us pause before handing any given executive branch the power to force the divestiture of platforms so-designated, as this bill would. Indeed, the very idea that TikTok presents a threat to national security shows how such designations are often driven by ulterior motives — in this case, at least in part, hostility to support for Palestine. Many proponents of the ban harnessed the sinophobic narrative that TikTok was akin to “Chinese Opium,” enacting mind control to “brainwash” kids against Israel. Josh Hawley, for instance, alleged that the platform was a “purveyor of virulent antisemtic lies.” To be clear, there is no evidence for this claim, even as there is evidence that U.S.-based social media platforms suppressed pro-Palestinian speech, something we examine below.

"There’s no one thing called “AI”

The question of what AI can and can’t do is made very challenging to navigate by a frustrating tendency that I’ve observed among many commentators to blur the lines between hierarchical levels of AI technology. ....

AI is too broad and fuzzy to cleanly decompose into a proper hierarchy, but there are a few ways to impose a messy order on it. ...

Frequently, reporting on new technology will collapse this huge category into a single amorphous entity, ascribing any properties of its individual elements to AI at large. .... All of this really makes it seem like “an AI” is a discrete kind of thing that is manning chat bots, solving unsolved math problems, and beating high schoolers at geometry Olympiads. But this isn’t remotely the case. FunSearch, AlphaGeometry, and ChatGPT are three completely different kinds of technologies which do three completely different kinds of things are are not at all interchangeable or even interoperable. You can’t have a conversation with AlphaGeometry and and ChatGPT can’t solve geometry Olympiad problems.

... I believe that this property, where there are many ways to appear to have done it (by outputting a million random digits, for example), but only a very small number of ways to actually do it (by outputting the correct million digits), is characteristic of things that Generative AI systems will generally be bad at. ChatGPT works by making repeated guesses. At any given point in its attempt to generate the decimal digits of π, there are 10 digits to choose from, only one of which is the right one. The probability that it’s going to make a million correct guesses in a row is infinitesimally small, so small that we might as well call it zero. For this reason, this particular task is not one that’s well suited to this particular type of text generation.