The Usual Confusion Prevails

That's because humans have context, agency and flexibility. Sure, the person who designs a form with a blank for "name" might never have met a Malagasy person whose first name is Randriamananjararadofabesata, but in the pre-digital world, when Madagascar Slim met a public official who had to transcribe his name onto a paper form, that official could simply draw an arrow in the margin next to the "name" blank, turn the form over, and write out all 28 characters on the reverse:

https://en.wikipedia.org/wiki/Madagascar_Slim

Computers can't do this. If the programmer doesn't know about Malagasy first names, the computer doesn't know about them either, and the only person who can "teach" the computer about these names is a programmer with access to the code for the database, who has to manually alter the code, compile it, and distribute it to everyone who uses it.

This is partly why digitization has been accompanied by a rise in people asserting that they exist on spectrums rather than in binaries. There were always people whose names, genders, races, and other biographic "immutables" changed, or failed to fit within the blanks on the forms. When those people's realities ran up against failures in the system's abstractions, they could petition a bureaucrat to turn the paper over and write an explanatory note, or to write really small to fill in a blank:

https://pluralistic.net/2023/02/02/nonbinary-families/#red-envelopes

Getting a human official to turn the paper over and write something that didn't fit in the blank is a personal challenge. It requires that a subject convince the person who controls the form to make an exception. This isn't always easy, but officials on the front lines necessarily deal with reality, and they can't get their jobs done unless they're capable of interpreting the necessarily incomplete procedures they operate under to fit things as they really are.

But a computer doesn't have any agency or context or flexibility. If the computer says your name isn't valid, you can't argue the computer into accepting it. The only way to get a digital world to acknowledge your existence is to campaign for systemic change. A trans person might (with great difficulty, to be sure) convince the regional registrar to white-out an old X on one "gender" box and mark a new X in the other box. But the only way to make that change in a software system that has been programmed to treat the "gender" field as immutable is to change society itself.

In this way, computers are machines for teaching us what we don't know about ourselves. They require that we interrogate and faithfully recreate our personal tacit knowledge, and they require that our societies interrogate their tacit presumptions as well. When you are forced to turn your tacit knowledge into explicit knowledge, you're also forced to confront how many broken assumptions lurk inside your reasoning. At best, it's a clarifying process.

Computers don't just clarify what we know and how we organize our society: they also clarify what we are. There are lots of things that we have supposed that a computer would never do, because we believed that these things required something that only humans could do.

Take chess: there are more possible chess games than there are hydrogen atoms in the universe, so brute-forcing chess by running all possible games is a technological impossibility. The best human chess players do something we don't quite understand, mixing their recollections of previous games with rules-of-thumb about the best strategies, with "creativity" (whatever that is) that lets them spontaneously develop new strategies. We can easily get a computer to memorize all the known-good chess sequences and all the rules of thumb, but we don't know what "creativity" is, so we can't encode it as a series of instructions.

But thanks to breakthroughs in machine learning and its successor, "deep learning," we have created chess-playing software that can beat every human, partly by assaying gambits that we would term "creative" if they originated with a human player.

What we make of this new fact is controversial. For many people (myself included), this is a refinement: it tells me that behaviors that are indistinguishable from "creativity" can, at least some of the time, be created by mechanical processes, and the mere fact that a machine does something that appears "creative" doesn't mean that machines are human.

For others, the fact that a mechanical system can evince a behavior that we would call "creative" in a human doesn't mean that we defined "creativity" too broadly, it means that we defined "human" too narrowly, and now we have made a machine that is, at least partially, a person.

I think this is the wrong conclusion to draw, for reasons that Ted Chiang sets out with luminous brilliance in a recent Atlantic article entitled "No, Artificial Intelligence Is Not Conscious":

https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/

(If you're hitting the paywall on that one and you're on Firefox, you can try my favorite trick: switch to "Reader Mode" and hit "reload" – your mileage may vary.)

For all the reasons Chiang articulates, I think that drawing the "personhood" line to include machines is a technical mistake, but it's worse than that. Admitting machines to the "personhood" club is a tactical mistake, on par with the mistake we made when we admitted corporations to the personhood club. We should absolutely consider expanding personhood to incorporate living things, including animals and ecosystems, but at the same time, we must purge these dead, artificial constructs from the club:

https://pluralistic.net/2026/04/15/artificial-lifeforms/#moral-consideration

There is a way in which the recognition of new capabilities in machines parallels the recognition of new capabilities in animals other than ourselves. When those animals manage to do things that we once thought were the exclusive province of humans, we (should) take that as an opportunity to refine our conception of humanity. We're not "the animals that use tools" or "the animals that make plans" or "the animals that recognize themselves in mirrors," because there are other animals that do those things. We are an "animal that uses tools"; not the animal that does so.

Likewise, if we thought that some activity was unique to humans, or to living beings, and we manage to get a machine to replicate that activity, we should revise our view of the activity – not our view of the machine. Creative breakthroughs in chess are not "a thing that requires a human mind," they're "things that can be done by human minds and by machines."

Edsger Dijkstra once famously asked "can a submarine swim?"

https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD898.html

Submarines and fish and humans and dolphins all propel themselves through water by different means. But when an animal swims, it does something that is different from what a submarine does. The submarine has no intention, while (complex multicellular) animals swim to pursue goals. Building machines that propel themselves through water is very useful, but it's not the same thing as creating life. In some ways, it's better than creating life: for one thing, we owe other living things moral consideration that is not due to machines. Harnessing a machine to accomplish our own goals is more morally clear than controlling living things to achieve those goals. By the same token, creating machines that can do some of the tasks that we ask of other humans can be the superior moral course. I'd rather have a machine remove mines from a minefield than getting humans to do it.

But beyond this moral relief, creating machines is a fantastic way to learn more about ourselves – making explicit our tacit knowledge, our implicit social assumptions, and the limitations of our conception of what sets us apart from the rest of the universe.

One way in which AI is exceptional is in how it undermines this principle. Conventional software techniques struggled to produce a program that could identify objects in photographs. It turns out that defining all the visual correlates of "cat" is even harder than defining the letter "A." Deep learning techniques solved this previous insoluble problem by relieving us of the job of making explicit all the implicit factors that we deploy when distinguishing an image of a "cat" from an image of a "dog" or a "tiger" (or a "tractor").

Instead of forcing humans to engage in introspection until we'd made a list of every factor we use to identify cat pictures, we simply identified pictures of cats and fed them to a program that tried to find the commonalities among them. The more pictures we fed to that program, the better it got at identifying cats. Today, we have programs that can reliably distinguish an image of a cat from an image of a tiger cub!

This represents a major breakthrough in the power of computers to perform useful work for us, but it's also a huge regression in computers' role in forcing us to make our tacit thought processes explicit through systematic introspection. That's probably fine: we didn't create computers to make us introspect, we created them to do useful work for us. All things considered, it might be better to have genies who grant our wishes according to the spirit of our words, not their letter.

AI may not force us to render our implicit thoughts as explicit instructions, but it absolutely forces us to reconsider and narrow the realm of the numinous. Our own creativity is still delightful and important, but the fact that this squishy, amazing process can (sometimes) be replicated by procedural machines changes the definition of living things. We're "a thing that can produce creative outcomes" but not "the things that can produce creative outcomes." The machines aren't being creative (any more than a submarine is swimming) but they're outputting things that we used to only achieve by means of creativity.

An AI that does something that used to require creativity is fulfilling my favorite of Brian Eno and Peter Schmidt's Oblique Strategies: "Be the first person to not do something that no one else has not done before":

https://stoney.sb.org/eno/oblique.html

Just as bosses fantasize about AI bringing about a worksite without workers, and Zuckerberg is trying to build social media without socializing, and politicians want a bureaucracy without bureaucrats, we can sometimes use AI to produce creative outcomes without creativity:

https://pluralistic.net/2026/05/27/unnecessariat/#rubbuts-stole-my-jerb

That isn't to say that AI art is any good. AI may produce things that are aesthetically interesting, but it can't produce things that mean anything:

https://pluralistic.net/2026/06/02/must-we-pretend/