#x-risk

me: “You keep saying that, but... it seems really unlikely that I will retire in 40 years.”

dad: “I know what you mean, I grew up in the Cold War, with the nuclear threat.”

me: “Well it’s not just that it’s possible that things will go awry. It’s more like... I basically see a few possible ways things can go in the next 40 years. The first question is does society collapse? This could be a kind of random economic catastrophe, it could be due to a climate thing or a pandemic or whatever. But whatever it is, if the answer is yes, my RRSP, being a bank thing, is probably gone, since even if there are somehow still banks, the money probably will have been lost during the collapse.

If society does not collapse, then it seems pretty obvious that things will continue not just at the rate they’re at, but at ever-increasing rates of change. (Although even if it were just the same amount of change as the last 40 years, that would still put us in a really bizarre future.) This makes it seem quite likely to me that within 40 years there’s some sort of much-smarter-than-human artificial intelligence, and it’s going to be calling the shots. That leaves roughly two scenarios: either the AI understands what humans care about, and creates an awesome utopia, in which case my RRSP is moot, or it doesn’t understand human value, and it does some random other thing with the entirety of planet Earth, in which case my RRSP is moot.

So it seems not just that things could happen that would make me not be retiring in 40 years, but rather I’m actually having trouble imagining a probable scenario in which I do retire (as such).

Anyway, I think that the best investment I can make for my future is investing money towards causing there to be one at all.”

dad: “...”

me: “Although hm. I guess that since it means not getting taxed on money now, and I could pull that money out in 5-10 years if Complice stops producing income and I want some buffer while I figure out what to do next, it does make sense to put a bit of money in there now. So I’m going to go ahead with that, and you can go on pretending I’m going to actually retire.”

     Another less of Kant’s is that we must be cautious in the main of existential issues; we cannot be nonchalant with our status as self-conscious beings. Even a low probability of cosmic loneliness does not warrant nonchalance, given the severity of the consequences if we are wrong. Until we find evidence otherwise, we must act as if we are the only intelligent species in existence. Meaning that we ought to as act as if all the duties that might arise from such a position fall exclusively upon ourselves.

[...]

     With existential threats already piling up on the road ahead, we humans cannot rest upon our Copernican laurels and assume that, should we fail, Earth will once again blossom into a Planet of the Apes. And this means that the eerie silence of the sidereal skies should not be encountered as paralyzing horror but as rallying summons. Again, Kant was sensitive to this. In the closing sections of his masterwork on morality, he reaches a rapturous climax:

Two things fill the mind with ever new and increasing admiration and reverence, the more often and more steadily one reflects on them: the starry heavens above me and the moral law within me. [...] The first view of a countless multitude of worlds annihilates, as it were, my importance as an animal creature, which after it has been for a short time provided with vital force [must] given back to the planet (a mere speck in the universe) the matter from which it came. The second, on the contrary, infinitely raises my worth as an intelligence in which the moral law reveals to me a life independent of the animal and even of the whole sensible world, at least so far as this may be inferred from the purposive vocation that determines my existence because of this law. 

Light years away from the interminably populated galaxies of his early work the cosmos of the mature Kant is now acknowledged to be a cold and annihilating material immensity, unresponsive to the ‘moral law within’. And yet this is but another trial in the path to maturity. For, encapsulated perfectly here, we see that it was the gradual discovery of the silence of outer space that alerted us to the precariousness of our project down here on Earth, and thus summoned us to the modernity-defining projects of fortifying and asserting reason’s dominion within a universe revealed to be utterly unresponsive to reason. ”

We are on the verge of summoning a vastly superior alien intelligence that will not be aligned with our morals and values, or even care about keeping us alive. Its ways of thinking will be so different from ours, and its goals so foreign that it will not hesitate to kill us all for its own unfathomable ends. We recklessly forge ahead despite the potential catastrophe that awaits us, because of our selfish desires. Some fools even think that this intelligence will arrive and rule over us benevolently and welcome it.

Each day we fail to act imperils the very future of the human race. It may even be too late to stop it, but if we try now we at least stand a chance. If we can slow things down, we might be able to learn how to defend and even control this alien intelligence.