#cluely

"Somehow, a movement that believes AI is incredibly dangerous and needs to be pursued carefully ended up generating a breakneck artificial arms race. But that race seems to have stalled, at least for the moment. As Alexander predicted in “AI 2027,” OpenAI did release a major new model in 2025; unlike in his forecast, it’s been a damp squib. Advances seem to be plateauing; the conversation in tech circles is now less about superintelligence and more about the possibility of an AI bubble. According to Alexander, the problem is the transition from AI assistants—language models that respond to human-generated prompts—to AI agents, which can operate independently. In his scenario, this is what finally pushes the technology down the path toward either utopia or human extinction, but in the real world, getting the machines to act by themselves is proving surprisingly difficult. In one experiment, the developer Anthropic prompted its AI, Claude, to play Pokémon Red on a Game Boy emulator, and found that Claude was extremely bad at the game. It kept trying to interact with enemies it had already defeated and walking into walls, getting stuck in the same corners of the map for hours or days on end. Another experiment let Claude run a vending machine in Anthropic’s headquarters. This one went even worse. The AI failed to make sure it was selling items at a profit, and had difficulty raising prices when demand was high. It also insisted on trying to fill the vending machine with what it called “specialty metal items” like tungsten cubes. When human workers failed to fulfill orders that it hadn’t actually placed, it tried to fire them all. Before long, Claude was insisting that it was a real human. It claimed that it had attended a physical meeting with staff at 742 Evergreen Terrace, which is where the Simpsons live. By the end of the experiment, it was emailing the building’s security guards, telling them they could find it standing by the vending machine wearing a blue blazer and a red tie." - Sam Kriss, Child’s Play: Tech’s new generation and the end of thinking

"Donald was practically vibrating when we left Cluely. “Dude, he’s just a scared little boy,” he said. “He’s scared he’s not doing the right thing, and because of the fucked-up world we live in, people who should be in The Hague are giving him twenty million dollars. Something bad is gonna happen here, something really fucking bad is gonna happen.” He sighed. “I just want Zohran’s nonbinary praetorians to march across the country and put all these guys in cuffs.” I found it hard to disagree. It did not seem like a good idea to me that some of the richest people in the world were no longer rewarding people for having any particular skills, but simply for having agency, when agency essentially meant whatever it was that was afflicting Roy Lee. Unlike Eric Zhu or Donald Boat, Roy didn’t really seem to have anything in his life except his own sense of agency. Everything was a means to an end, a way of fortifying his ability to do whatever he wanted in the world. But there was a great sucking void where the end ought to be. All he wanted, he’d said, was to hang out with his friends. I believed him. He wanted not to be alone, the way he’d been alone for a year after having his offer of admission rescinded by Harvard. For people to pay attention to him. To exist for other people. But instead of making friends the normal way, he’d walked up to strangers and asked whether they wanted to start a company with him, and then he built the most despised startup in San Francisco. He was probably right: he could count on making a few million dollars every year for the rest of his life, even after Cluely inevitably crashes and burns. He would never want for capital, but this did not seem like the most efficient way to achieve his goals." - Sam Kriss, Child’s Play: Tech’s new generation and the end of thinking