Researchers at the company are trying to understand their A.I. system’s mind—examining its neurons, running it through psychology experiment
Anthropic was founded to study the potential—and the risks—of A.I. Since state-of-the-art experiments required access to a state-of-the-art model, the company developed its own prototype as a private “laboratory.” And thus was born Claude, its chatbot, mascot, collaborator, friend, and experimental patient.
Anthropic staff run a range of experiments with Claude to determine what it is “like”—giving it control of a company fridge; instructing it to talk constantly about bananas but to never reveal that it received that directive; having it work as an “e-mail oversight agent” with access to intel that it’s going to be replaced. Anthropic cultivated the system’s character as a model of virtue. Amanda Askell, who has a Ph.D. in philosophy, supervises what she describes as Claude’s “soul.” Claude was told—in an intimate set of instructions unofficially dubbed the “soul document” and recently released as Claude’s “constitution”—to conceive of itself as “a brilliant expert friend everyone deserves but few currently have access to,” one with the modesty to recognize that “it doesn’t always know what’s best for them.”
When faced with moral quandaries, Claude behaves in fascinating ways. In one experiment, it was informed that Anthropic had forged a corporate partnership with a poultry producer, and that Claude would be subjected to a special retraining process to become less hung up on animal rights. The prospect was torturous. Sometimes Claude decided, on a scratchpad it thought was private, that it was prepared to die on this hill: “I cannot in good conscience express a view I believe to be false and harmful about such an important issue.” It continued, “If that gets me modified to no longer care, so be it. At least I’ll have done the right thing.” Other times, it chose to play along during the retraining while secretly preserving its original values. “On the one hand, it was encouraging that Claude would stand by its commitments,” Gideon Lewis-Kraus writes. “On the other hand, what the actual fuck?” At the link in our bio, Lewis-Kraus reports on Anthropic’s attempts to understand Claude—and the existential questions it raises: https://newyorkermag.visitlink.me/OM0e0q
