#lesswrong

The version you commonly hear about is a benevolent AI which upon its creation punishes anyone who knew about and was able to assist in their creation but didn't, but that's the streamlined version

LessWrong, the forum that spawned the thought experiment, spent so much time collectively stewing on the idea of optimising ethics that they threw out the concept of linear time to pursue ever more esoteric systems of morality

Roko's Basilisk was built on the idea of future actions being used to incentivise behaviours in the past as part of an acausal system

The Basilisk doesn't use the threat of harm to incentivise behaviour, they use torture in the future to directly alter people's behaviour in the past, and there were even discussions in the original thread about how one would be able to acausally "trade" with a future AI that hasn't been constructed yet based on this premise

There was this one post on the original thread which asserted that the reason the Basilisk was so dangerous was because if you imagine a hostile or dangerous entity with the means and motives to blackmail you in sufficient detail, you have effectively already brought that entity into existence

Roko's Basilisk is an atemporal tulpa that runs on the same principles as Lord English from Homestuck cause despite not existing yet, they're already here

But when rationalism finally spat out something truly horrible, it was a bunch of far-left transgender vegans doing the killing. Because the problem with rationalism was never actually a left-right thing. That lens only ever clouded people's view of what was actually wrong.

Which makes me think. Do I ever do the same thing? Take a story about the world as a whole, apply it to some strange little corner of the planet, and mangle my own understanding of what's in front of me by doing so?

TLDR: Ziz left the rationalist community 6 years ago and even at the time she wasn’t liked there, the fact she managed to exploit. Her actions directly contradict both hpmor and what CFAR was doing and teaching. Her other affiliations that are no less relevant include being an anarchist and a vegan. You are not immune to cults just because you are not into any particular weird internet subculture.

Cults form from niche subcultures, that’s true enough. But any subculture can form a cult because any culture, even the mainstream one, contains some ideas that can be twisted to an insane degree. And any ideology, even the most niche and scary one, can be approached casually and sceptically. What is actually needed to create a cult is not a special ideology but a cult leader and vulnerable people to follow them. That is the main uniting quality between all cults. Trying to figure out what’s wrong with a certain music band or a certain fantasy book forum is an exercise in motivated reasoning. You will always end up finding something that's wrong.

The rationalist community was trying to prevent the formation of a cult as best as they could. Partly that’s the reason why people like Ziz and others with bad and unpopular takes were often tolerated longer than necessary. To encourage criticism and prevent getting stuck in a positive feedback loop. Because it’s not a high control group! You cannot be simultaneously mad that people are allowed to talk about wacky ideas on forums and also that the group is supposedly very rigid and controlled. Apparently, they could use some control. Not like it would actually stop an aspiring cult leader from recruiting, they’d just go some other place.

Zizians were not mostly trans by accident (what are the chances?). Ziz was recruiting the most vulnerable people who related to her and were willing to trust her (also because there’s a lot of trans women in the community, like a lot). She used their very real and grounded experience of discrimination to convince them that her not being liked is not due to her takes being bad but because she’s trans.

And she had a lot of takes, some of them not being popular enough you actually complained about. You criticised LessWrong for being too pro-capitalist for your taste and then started talking about the Really Bad rationalists and THEY ARE LEFTISTS killing landlords and cops.

Now if we are talking about ideologies that devalue human life, how about some that require actual Class War (guess what people do in wars) or violent mass uprisings? Or some that require assassinations of certain select individuals? How come you hear that there are people on forums discussing the ethics of murdering those directly responsible for destroying our planet and you, as a leftist, do not immediately recognize yourself in it? Nothing discussed on LessWrong is more violent than a communist revolution or even the killing of Brian Thompson.

Why being into rationality at some point and reading hpmor is the only thing you told about Ziz? I think I know why. All the right wingers really leaned into the whole ‘trans vegan cult’ thing. That is not a good look for our side, is it? How amazing would it be to find a scapegoat. Who cares about those AI safety freaks anyway? They are all cishet men anyway! All cishet men who somehow have an offshoot of violent vegan queers, that certainly adds up.

Ziz being a radical vegan* (another niche subculture) corresponds to her actions way better than anything that's discussed on LessWrong or happened in hpmor. In fact, there’s an exact scene you probably skipped. Harry is on a very important and dangerous mission with Quirrell and at some point he is told to hide while Quirrell duels a cop guard at magical Guantanamo Bay – a total pig and an absolute scum who Ziz would kill without a second thought. Harry does share her sentiment, he fucking hates Azkaban. But when Quirrell tries to kill the evil torture cop Harry instinctively protects him jeopardizing the entire mission. And it’s not a random scene. It starts the entire disillusionment spiral where Harry realizes his beloved groomer professor might be a bad guy. Murder of a bystander whose only crime is being a product of his society is not something Harry can tolerate. He does end up decapitating a bunch of actual death eaters in the very end (the bit you probably did read) to save his own life and defeat Voldemort and even then he regrets it and apologizes for it (despite it being the right thing to do and not even comparable to a random cop). There’s an entire scene where Harry bonds with Draco over their mother’s deaths where he expresses that every death is a tragedy, even deaths of very bad people (like Voldemort).

Not to mention the entire immortalism theme (did you skip the entire third book?). One cannot read hpmor and walk away thinking human life is worthless or only super geniuses deserve to live. Timeless decision theory leading to murdering people is not in there**, nothing in the fic even suggests such a conclusion. More about how you got most things wrong about hpmor here.

Let’s face it, those were a bunch of sleep deprived vulnerable people high on all sorts of radical ideas, who were kicked out of every decent movement and that’s why they slipped into a cult.

Any subculture can become a cult. If you ever read a boring preachy fanfic or ever went to a physical meeting with internet weirdos. If you ever felt rejected by mainstream society and went looking for ‘like-minded people’, for a ‘found family’, for a ‘place where you belong’. You are not safe. Touching grass from time to time is not enough. You have to never leave the pastures to be highly immune to cults. And that ain’t you, my friend. That’s none of us.

Previous post about rationalist community.

* Ever heard ‘veganism is the moral baseline’ (sometimes minimum or imperative)? That’s not a slander, it’s a commonplace argument and an actual slogan.