#intelligence explosion

By Jude B.

1. Introduction

The term artificial intelligence suffers from an embarrassment of definitions, which, among other reasons, is because the notions of intelligence and artificiality are themselves quite contentious.

  However, as I’m sure we all possess an adequate level of intelligence, I will assume an intuitive understanding on the reader’s part as to what these terms mean or entail.

  Having said that, a very general definition of A.I. could be given as any artificial device able to perform tasks that, if were accomplished by a person, would require them to utilize a measure of intelligence. The more such tasks this device can perform, and the greater level of intelligence involved, the more intelligent we would deem this device to be. 

  A less general definition specifies this device as a general purpose computer running a sophisticated program (or more generally, a finite Turing machine[1]).

  These A.I.’s can be classified into two major types, Type I and Type II,

A Type I intelligence (IT1) can be called a brute intelligence and is not conscious or self-aware, and it does not experience any sort of feelings or sensory experiences.

A Type II intelligence (IT2), on the other hand, is conscious[2] of its environment and can experience various qualia.

 In this essay, I will discuss why I suspect that there is more to achieving IT2 than simply running a sophisticated program on a (any) general purpose computer—i.e. any physical implementation of an FTM. I will also briefly discuss whether an IT1 can help us in the quest for constructing IT2, how we should go about it, and what might be our biggest hurdles.

  2. Down With Substrate

 There are numerous methods through which today’s experts hope to create IT2’s, among these are brain simulations, self-evolving programs, neural networks and more (some even think that the World Wide Web might spontaneously become self-aware).

 However, it is immaterial which of these methods we discuss, since they will all, if successful, result in the same thing: an IT2 manifested by a program running on a futuristic computer.[3]

 Does that compute?

 Before I give my personal position on the matter, we need to discuss the notion of substrate independence. Substrate independence is the position that (self-)consciousness is, basically, a computation[4] and that if we have two computing devices (FTMs) performing the same computation and one of them is self-consciousness then so will be the other. Thus, if an algorithm running on a futuristic desktop PC can achieve consciousness then so will an equivalent classical FTM containing a moving head and a read/write tape, or, for that matter, so will any computationally equivalent contraption made out of, say, billiard balls or rolls of toilette paper.

 I suspect that my next assessment will seem a bit controversial, but I think it would be fair to characterize this stance as Platonistic (the classical or the Gödelian flavors), because it can best be seen as couched in the belief that abstract objects exist and they can exert some sort of influence on our physical world (or even that our reality is, at its very core, such an abstract object).[5]

 Possible examples for such objects are our minds (according to Dualism), the laws of nature, karmic laws, or (according to Gödel) mathematical truths.

 Let me try and defend this assessment.

 There are four main ways by which we can frame the term computation,

It is an abstract, non-spatiotemporal object (Platonism).

It is a useful fiction (nominalism).

It is a set whose extension is all the (possible) physical processes that perform a computation (immanent realism).

It exists but only as an idea in our minds (conceptualism)

  First of all, allow me for the time being to simply state that the (abovementioned flavor of) Platonism presents the strongest case for consciousness-as-computation among all these approaches, and therefore I am not engaging in a straw man argument by describing substrate independence as a Platonistic idea. I will elaborate a bit later.

  O.k., first an example. Suppose one were to claim that all the universes containing more than a googolplex particles are self-conscious. I think you will agree that this claim is Platonistic in nature, since it states that self-consciousness is all about this number, and not about any other distinctive properties of the universe in question (except for containing more than googolplex particles).

  Let’s try and look at it from another angle. The expression “2+2=4” that you’re now looking at is simply a bunch of pixels on your screen, it means nothing without a mind that understands what those characters represent and what they “claim”.[6] The meaning (and the truth value we assign to it) are a result of the semantics of the above equation.

  So, what does happen in a physical process that generates self-consciousness? Well, it is fair to say that this process carries its own semantics, because it generates an observer who can think, for example, “this physical process is about generating me.” It is as if “2+2=4” could look at one of its tokens “from above” and understand what it means.

  But if you accept substrate independence, then this meaning/semantics cannot be attached to the actual physical process, and since it has a meaning, it must be attached to the Platonic object that is computation. Thus, it seems, substrate independents believe that the Platonic computation “recognizes” that some process in the universe is one of its tokens and endows it with meaning/semantics and even consciousness.

  Here is another example. Suppose we have a futuristic, self-conscious desktop PC, call it C1, and it is self-conscious due to it running a brain simulation, call it P1.

  Let us isolate a few seconds from the operation of C1 (when it’s running P1) where, say, the computer is thinking “this physical process is about generating me.” Now, if you examine everything relevant to the computation that happened in C1 during those few seconds, you can write another program, P2, that will simply tell C1 to repeat those operations.

  If you are not a dualist, then since C1 does the same (relevant) things when we run P2, then P2 should generate self-consciousness as well.

  Now take P1 and run it on a different computer, C2. According to substrate independents, since it is the same program (or because the semantics is of a brain simulation) it should also generate consciousness. However, now running P2 on C2 will not recreate the operations of P1 on C2 (say that C2’s hardware is quite different than C1’s).

  The question is whether P2 will still generate the above conscious thought or not?

  If it doesn’t, then substrate independence is false, since P2 generates it in C1.

  If it does, then the process somehow has to “know” that the semantics is really about simulating a brain for a few seconds, and not about causing some physical operations to occur within C1, which was the intended semantics of P2. This, I believe, calls for Platonism.

  “O.k.,” you might counter, “what prevents us from applying similar arguments to the physical object that is the human brain, and reach the same conclusions, i.e. that some Platonic object endows us with consciousness (and then why won’t this Platonic object give it also to a simulation?).”  I will give my answer in the next section.

  Let me now pay my debt and show why Platonism is the strongest approach to consciousness-as-computation amongst all the others.

  First, it would be quite difficult, and maybe even untenable, for a nominalist to argue that a non-existent entity can generate something that obviously does exist.

  This is basically the same difficulty we have with the contemporary-platonistic viewpoint because, according to it, the abstract notion of computation cannot casually influence our universe.

  Secondly, when it comes to immanent realism, certain versions of it are problematic and are rejected by most working philosophers while other versions can better be described as non-standard Platonism (see the discussion in http://Plato.stanford.edu/entries/Platonism),

  Finally, conceptualism creates an obvious chicken-and-egg problem; namely, how can an idea that only exists in our minds create our minds in the first place?

  So Platonism it is.

  To sum up this line of argumentation, it is somewhat strange the proponents of substrate independence, who often attach to their critics the titles “Dualists” or “Mysterians”—as opposed to their own position which they claim is derived from hard-nosed materialism—are actually the ones who offer a theory of consciousness not grounded in distinctive physical properties of our universe!

  3. What Really Matters?

Think about magnetism. Magnetism is a physical phenomenon that cannot be generated by just any material. Furthermore, a computer simulation of a magnet will not attract even one paperclip to the screen.

  Think about gravity. Again, a simulation of a black hole will not suck you in. And the same applies to phenomena such as flight, superposition and more.

  Now think about consciousness. The only example of a (self-)consciousness generating object (that we are confident about) is a human brain. Our brains are complex structures located in the physical universe, their operation relies on physical phenomena like conductivity, electromagnetism, temperature and so on, they are shaped in a certain way, they operate at a certain speed etc. It seems to me that the onus is on the substrate independents to explain why they think none of these physical phenomena[7] plays an imperative role in the generation of our self-consciousness, besides enabling some sort of a computation.

  Let me be clear that I don’t claim the only way (self-)consciousness can be generated is by a brain or that there is something supernatural about it. What I do believe is that consciousness is like gravity, flight, magnetism, heat etc., and if we ever manage to build an IT2, it won’t be just a better version of today’s desktop PC running a sophisticated program, or a galaxy-sized FTM built from empty beer cans.

  Put in another words, if one wishes to generate certain phenomena in the physical universe, then there are certain materials, certain physical forces, certain shapes, speeds, temperatures and so on, that one would have to utilize in order to achieve them.

  Consequently, I think that it’s very likely that consciousness relies on such phenomena, and might itself be—as some people believe—a fundamental element of the universe.[8]

  So let me now return to the whole syntax/semantics issue I talked about in the previous section.

  Some physical phenomena might be fundamental,[9] and thus, we may consider them to be like brute facts which simply lie at the bottom of reality. These fundamental phenomena could be seen as serving as a kind of a built-in semantics. It also could be that consciousness must rely on certain kinds (and/or combinations) of such fundamental phenomena for its operation (and it’s they that generate the needed semantics).

  For example, suppose that superposition is a phenomenon like that, and suppose that consciousness relies on superposition, then consciousness will not occur when we use a substrate that cannot generate it or operates too slow (because we’ll get destructive decoherence). In particular, it would mean that galactic-size Turing machines operating for trillions of years to generate one thought are out of the question.

  Another example is flying. The notion of flying is strictly rooted in our physical universe, and depends on combinations of phenomena like gravity, speed, shape, gas density, elevation forces etc. A simulation of all of the relevant phenomena will not cause the computer to fly. And I’m sure you can think of more examples.

  Let me now anticipate a possible counterargument which goes basically like this: The fact that a simulation of, say, a black hole will not generate gravity in our world means nothing because it will generate it inside the simulation (e.g. suck in simulated matter etc.); thus, simulated gravity is really gravity but in the simulated world, even though it is not real in ours. Therefore, a simulation, say, of all the relevant processes in the brain will generate real (self-)consciousness inside the simulation.

  Not so fast. First of all, self-consciousness is absolute—after all, it is the only thing we are sure about (cogito…). It doesn’t matter if you live inside a simulation within a simulation within a… to the nth degree, if you are self-conscious, you exist. Period. So it is meaningless to say you are self-conscious in a simulation but not in the higher reality.[10]

  But substrate independents know this, because they claim that the simulation will be self-conscious in this world, even though, they admit, the processes that are simulated (like conductivity, electromagnetism etc.) are not real in this very same world!

  This begs the question how can phenomena that are not real from the perspective of this world join together to create a phenomenon that is?

  And besides, all this talk about simulated matter in a simulated universe responding to simulated gravity is merely semantics imposed by us on some physical computation. For self-consciousness we need the semantics to be self-generated and, like I said, I’m doubtful that substrate is immaterial in that case.

   4. But If Platonism Is True…

“Wait,” some Platonists might protest, “Platonism is far from being an unreasonable position. It might even be expected that answers to many important questions about our existence will ultimately be traceable to the Platonic realm.”

True. The deeper we dig into the foundations of reality, it seems less physical or tangible, and more about information, relations, symmetry, geometry, transcendent laws and other (very) abstract notions. Some people even suspect that, at the very bottom (if there is one), the thing we call the physical universe dissolves away to reveal mathematics.[11]

  If that is true, then since everything might be, at its most fundamental level, mathematics, then so is consciousness! So there is nothing outrageous in assuming that the mathematical structure that is “consciousness” would turn out to be isomorphic to certain forms of computation, and from there substrate independence easily follows.

  Sure, but it might not.

  Even if we accept that everything is mathematics, it still is conceivable that the mathematical structure responsible for consciousness is not isomorphic to any form of computation and actually must rely on structures that are not embeddable in any computation-structure; furthermore, these non-embeddable structures might correspond to electromagnetism or superposition and so on.

Put differently, this flavor of Platonism entails that physical objects in our world correspond to structures in the Platonic realm, and therefore, creating certain constellations of matter here will correspond to some forms there. You can look at it as if the objects in our world are just projections or shadows of the "real" forms in the higher-dimensional Platonic realm.

  Now, a physical object that is a description of another projection (i.e. another object), is also a projection; but, like I said, these two projections might not be isomorphic. So the mathematical structure that corresponds to a computer running a brain simulation (i.e. a description of the brain) might not be isomorphic to the structure corresponding to consciousness.

  But wait! We just reframed our argument in the language of mathematical structures! This, at least for me, indicates that going down the rabbit hole into these kinds of speculations is not very productive.

    5. Why Do I Hold This Position

  Let me summarize the argument I am trying to make.

Substrate independence is not rooted in physicalism and is actually Platonistic in nature (at best).

It’s far from obvious that things are equal to their (mathematical/linguistic) description.

My position should be the default one.

Since I already touched on the first item, let me add a few words regarding the other two.

  Let me address the second item.

  There is a category of thought experiments involving a person who, for some reason, never had a certain conscious experience—like pain, hearing sounds, seeing a certain color etc. We also assume that this person managed to gain a complete (linguistic or mathematical) knowledge about this phenomenon and the physiology of its experience (e.g. knowledge of all the brain states related to this experience).

  Now, suppose this person finally manages to have this experience for the first time. The question is whether he gained new knowledge, and the answer seems to be “Yes,” since now he knows how this particular quale feels like.

  If you accept that he did gain new knowledge, then it does seem to suggest that a simulation of a brain will not be conscious. Why? The reason is that a computer program running a brain simulation can be thought of as a dynamic mathematical (or linguistic) description of our consciousness; however, the same way the knowledge this person has (which comes in linguistic or mathematical form) cannot generate this experience in his brain, then running a computer program will not generate consciousness either. 

  If you are the kind of Platonist I mentioned above, then you can think of the mathematical/linguistic knowledge that this person has as one form of platonic structure, and the knowledge of actually experiencing this sensation as another one, and these two are not isomorphic. Since a computer running a program encodes knowledge of the first kind then it doesn’t seem that it will generate consciousness.

  If you rather believe that physical reality is composed of fundamental building blocks, then these things might not be equal to their description; namely, they have the property of brute existence which might be non-descriptive, and they might also possess other properties which are not linguistic or mathematical either, like consciousness or beauty and so forth.

  Therefore, as I mentioned, it is far from obvious that things are equal to the sum of their mathematical-linguistic description.

  As for the third item, I think my position is more in accordance with the sweet spot principle. Let me elaborate.

The principle of mediocrity says that, in most aspects, we are likely not very special, which means we sometimes have to multiply entities; that is, to posit the existence of other beings, places, situations, phenomena and so on, in order to render ourselves “average.”

  Occam’s razor, on the other hand, declares that entities should not be multiplied beyond necessity. Thus, the sweet spot principle says that, unless you have good reasons not to, the number of entities you ought to posit should be large enough as to render your situation “non-special,” but not much larger than that.

  Back to our discussion.

  Assuming that only human brains can generate self-consciousness would make us too special,[12] but assuming that it can be generated by a bunch of billiard balls moving on a table would be, in my opinion, taking it to the other extreme. Therefore, I think, the sweet spot is somewhere in between.

  To sum up, I think that there is no good reason to suppose that (self-)consciousness will simply emerge whenever some computation takes place, and that our default assumption should be that there is more to it than we currently know.

  Let us now move to the second topic.

  6. Can an IT1 Help Us Build an IT2?

  As we know, there have been tremendous advancements made in the development of IT1s; programs like Deep Blue or Watson have beaten the best humans in games that were considered to be forever dominated by us. We have self-driving cars, we can converse with automatic assistants and the technology just keeps improving.

  Still, can such an IT1, or as we called it brute intelligence, eventually manage to perform all the tasks that require human intelligence (HI-tasks), even though “nobody’s at home”?

  The smart aleck answer is “yes,” for the same reason the proverbial monkey will eventually manage to type Hamlet; after all, everything that a human being can do in the physical universe can be done, in principle, by pure chance alone.

  But the more interesting question is whether we can actually build an IT1 able to (out)perform HI-tasks in a reasonable amount of time. Actually, we don’t even need that. A possibly more important question is whether can such an IT1 “figure out” consciousness and thus help us build it.

  It is quite possible for a mindless computer to “figure out” or devise artifacts that can perform tasks beyond its capabilities. For example, a mindless, flightless IT1 might be able to design a new airplane all by itself, which we will then build and fly. In the same vein, a very sophisticated IT1 able to process an ocean of data, look for patterns, make and test hypotheses and so on, might (mindlessly) find what “makes consciousness tick” or what can boost it, and so put us on the road to building an IT2.[13]

  So, is such an IT1 possible and can we build it? I don’t know. But it is my belief that if we want to construct an IT2, then we will have to go through this station.

  7. Getting On With The Program.

  The (somewhat) surprising reality regarding my position is that, operatively, it calls for basically the identical methods for achieving machine consciousness. Therefore, even though I think that the route of making IT1s better, hoping that they will somehow become conscious, is a dead-end, I do think that we will find a map somewhere along this route which will nevertheless allow us to reach our destination through another (as yet undreamt of) route.

  The fact is that both substrate independents and people like myself believe that our best chance to obtain an IT2 is by creating more and more sophisticated hardware (IT1s) and software; we only differ on whether this futuristic IT1 will actually be an IT2 or whether it will help us to construct one.

  For example, I don’t see any problem with the brain-mapping project, and think it should definitely be pursued. My personal belief is that such a simulation will not be itself conscious, but it will definitely help us and our sophisticated IT1 assistants to discover important facts about consciousness, and thus help us improve our own brains and/or to construct an IT2.

  I also don’t think that the singularity will be the event where an IT1 suddenly attains consciousness, but instead the event where an IT1 “figures it out,” that is, either discovers the principles underlying consciousness and how to construct one, or maybe how to dramatically boost it in us.

  Some might object and say that the singularity should be defined as the point where an IT1 figures out how to construct a better IT1; however, this definition is problematic since we don’t know that this event will generate a cascading effect of ever-improving IT1s, instead of coming to a screeching halt after, say, 2-3 iterations. And besides, since most versions of the singularity include ourselves being “raptured” into the cyber-heaven, this cascading event needs to end with the creation of an IT2 in order to fit the bill; so we might as well define the singularity as I did above.

  Therefore, I think our best shot at constructing an IT2 is to try and improve the hardware capabilities of our IT1s and develop more and more sophisticated algorithms that can do excel in many fields of human endeavor (scientific research in particular). Eventually, such “scientist” programs might be able to help us improve our own intelligence and to construct an IT2.

  8. Possible Hurdles

”So,” we may ask, “if we do just what we’re already doing anyway (with maybe putting now more emphasis on the IT1-to-IT2 route), are we guaranteed to live in a ‘compuradise’ in the not-so-far future?”

  Well, maybe, but there might be some difficult hurdles to overcome.

The first possible hurdle is that there might be things conscious beings can do much faster than an IT1. As mentioned above, it could be that consciousness allows us to efficiently perform certain tasks that would require a realistic non-conscious agent (one that we can actually built) considerably much more time and effort to complete. If “figuring out” consciousness is amongst these tasks then having a super-sophisticated IT1 at our disposal will not offer much help. So let us assume that this is not the case. There will still be more hurdles that might prevent us from building these desired IT1s (at least in the next few decades).

  For example, the creation of such IT1s in the near future relies on the continuation of Moore’s law which states that processing power doubles every ~18 months especially relative to cost or size. It will also require a concomitant improvement in software capabilities.

However, it is not certain that this “law” is going to hold true until such an IT1 is built.[14]  This also applies to software.

  Another major possible stumbling block is that we may face a steep rise in difficulty. Namely, it could be that achieving further advancements in both software and hardware will become increasingly difficult, akin to trying to reach the speed of light, where even a small increase in speed requires a steeper and steeper expense of energy.

  For example, the state of technology today is the fruit of the labor of millions of people working in the field. But maybe the next major advancements can only be obtained by the combined effort of billions or trillions of experts? This is a demand our civilization cannot meet. It could be also that the problems we need to solve in order to obtain the next major improvements are inherently difficult in a sense that they are NP-complete. That is, if NP ≠ P then there are many everyday problems where the best method to solve them is to simply search all the possible answers one by one. For these kind of problems being an intelligent agent does not give you any advantage. So we might be in a situation where the next advancement will require us to solve a problem whose difficulty is way above the capabilities of our best up-to-date IT1s (or any realistic IT1).[15]

  At this point some of you might raise the objection that since we are here, then the problems of building an IT2 cannot be that difficult.

  Well, not necessarily. If the universe or multiverse are infinite (or big enough), then even very unlikely events—like nature accomplishing something that is tantamount to solving a computationally very hard problem—will happen, and therefore we do not know how easy it is to create our brains. Also, the fact we have not encountered or received messages from alien civilizations might suggest that creating (self-)conscious brains is a very hard task.

  And of course, even if creating an IT2 is not that difficult, it does not mean that creating an IT2-facilitating IT1 is.

  “o.k.,” some might continue, “even if creating an IT2 is super difficult, we do possess the finished product, and therefore only need to reverse-engineer it.”

  True, but it can still be difficult enough to prevent us from making any headway.

  Currently, the main methods of achieving this reverse-engineering are via neural nets or, more ambitiously, creating a neuron-level simulation of the brain. However, as I opined before, I don’t think that these simulations will be conscious, and since there might be many processes in the brain that rely on consciousness, this simulation will be too far removed from the real McCoy to dramatically improve our chances of reverse-engineering the brain (and also to create super-advanced IT1s).[16]

  Another difficulty might be that we won’t have the required manufacturing know-how to create even the simplest IT2; after all, we cannot now, using only basic chemicals, create a beetle, or a heart, or a bird and so on. So while I think the simplest, human-level IT2 possible is probably less complex than our brains, it might be still extremely complex, rendering its creation well beyond our foreseeable technical abilities.

  9. Conclusion: When and If

  So what do I think about the chances of creating an IT2?

  I think it is likely that we will have a working IT2 at some time in the future, even though I am not too sure about it occurring before 2050 as many singularitarians believe.

  Nevertheless, the task of making better IT1s has its own obvious benefits and should be pursued regardless.

  Thank you.

[1] A more correct name would be a finite state machine. We will use the short FTM for these machines.

[2] Note that, in order not to be bogged down with too many cases, when we talk about Type II intelligences we will actually assume they’re self-conscious.

[3] This computer is posited to be just a (considerably) more powerful (speed- and memory- wise) version of today’s computers.

[4] This position is called Computationalism.

[5] The contemporary definition of Platonism is that there exist abstract (non-spatiotemporal) objects, they don’t exist in a “place” and there is no casual relationship between them and our physical reality.

[6] It is possible that in some alien language “2” actually means “3,” “4” means “6” and “+” is the same; thus, for them, this expression is also true but means something different.

[7] Some of which cannot be generated by just any simulation, like the aforementioned magnetism.

[8] For example, we might (wildly!!) speculate that some subatomic particles have a consciousness “charge,” and, like in a magnet, if they are put together in a particular manner, this effect is magnified to create more and more advanced forms of (self-)consciousness.

[9] The current main candidates are strings or branes, and various fields.

[10] I think, amongst other things, it stems from the fact that a phenomenon like gravity is relational in nature while self-consciousness is, by definition, reflexive.

[11] Or logic, or information, or properties, or computation, or sets, or pure consciousness, or something as yet unthought of.

[12] If it turns out to be true, then we will be forced to consider ideas akin to “intelligent design” more closely.

[13] Mind you that “figuring out” consciousness will not make the IT1 conscious, since it will be missing something essential (but the IT2 won’t).

  [14] The original Moore’s law which talks about the number of transistors in an integrated circuit is already believed to be, as of 2016, defunct. However, we don’t know that other ways of imporving performance will not be found.

[15] Note that quantum computers are not thought to be able to solve NP-complete problems much faster than classical computers.

What is the Intelligence Explosion?

C O N T E N T S:

KEY TOPICS

Machines that are smarter than humans will lead to an intelligence explosion.(More…)

It also poses substantial obstacles to the kind of “intelligence explosion” Bostrom fears, since it implies that machines could not become “superintelligent” by refining their inference algorithms.(More…)

In either case, the leap from intelligence to superintelligence (otherwise known…

Artificial intelligence is getting smarter by leaps and bounds -- within this century, research suggests, a computer AI could be as "smart" as a human being. And then, says Nick Bostrom, it will overtake us: "Machine intelligence is the last invention that humanity will ever need to make." A philosopher and technologist, Bostrom asks us to think hard about the world we're building right now, driven by thinking machines. Will our smart machines help to preserve humanity and our values -- or will they have values of their own?

A brilliant new TED talk on what some call the #Singularity, others the #IntelligenceExplosion: the point where machines (or at least one of them) become as smart as human beings. This point in time might be far in the future. Or it might not. Machine learning is accelerating so rapidly that it could happen sooner than thought. And when it happens, we had better make sure this all-powerful machine being has the right values embedded, so that it remains in our service, caring about our well-being, rather than discarding humanity as a hindrance in the pursuit of another goal. 

The great danger we face here is that we achieve a breakthrough and put a technology into the world with enormous power before having thought about how to harness that potential and make sure it does not become abused. Nuclear power and our ongoing struggle to contain its potentially life-ending power is a case in point. As Nick Bostrom puts it: “The risk is that if somebody figures out how to crack the first challenge without also having cracked the additional challenge of ensuring perfect safety.”

Imagine our caveman ancestors looking up at the birds soaring overhead. They could not have imagined that, thousands of years later, millions of humans would fly around the world in jets each day. Our ancestors looked up at the stars and planets, too—never dreaming we would one day walk on the moon. There was once a time when the average human couldn’t expect to live much past age thirty.

It’s easy to underestimate what the future will bring. When thinking about a post-intelligence explosion universe, it’s important to recognize that the benefits of a successful intelligence explosion could be much greater than we can currently imagine, for our imaginations are limited.

Another thing we have to realize is that these changes don’t have to take place over thousands of years. Many of the technologies upon which utopia could be built are in the process of being developed now, and future technologies will be developed much faster if enhanced whole brain emulations and superintelligent AIs are developing them. Economist Robin Hanson reminds us:

Though such growth may seem preposterous, consider that in the era of hunting and gathering, the economy doubled nine times; in the era of farming, it doubled seven times; and in the current era of industry, it has so far doubled 10 times. If, for some as yet unknown reason, the number of doublings is similar across these three eras, then we seem already overdue for another transition.1

What I’m about to say may sound like science fiction, but that’s no reason to dismiss it. We have a long history of turning science fiction into science fact. And it doesn’t stop now.

Real Utopia

“Utopia.” It’s an easy concept to comprehend once you realize that it’s what we have always been seeking. We’re changing the world around us to fit our needs and desires like no other creature before us. But, with machine superintelligence on our side, we could be vastly more successful at realizing utopia than ever before. Consider what a time traveler from a post-intelligence explosion universe might tell us:

My consciousness is wide and deep, my life long. I have read all your authors—and much more. I have experienced life in many forms and from many angles: jungle and desert, gutter and palace, heath and suburban creek and city back alley. I have sailed on the high seas of cultures, and swum, and dived. Quite some marvelous edifice builds up over a million years by the efforts of homunculi, just as the humble polyps amass a reef in time. And I’ve seen the shoals of colored biography fishes, each one a life story, scintillate under heaving ocean waters.2

Utopia is not a world in which humans are subjected to a never-ending source of empty pleasure. Citizens of utopia have the technology to simulate and experience thousands of environments and worlds. There is no shortage of novel and meaningful experiences.

Citizens of utopia are not mindless drones who never use their brains to solve anything. Just as making a video game easier does not always make it more fun, citizens of utopia still face challenges and experience the joy it is to overcome them.

Utopia is not meaningless, predictable, or boring. People living in utopia become stronger, not weaker, over time. Their lives grow more exciting and wonderful, their emotions more varied and exciting. They become more fulfilled as time goes on, not less so.

Utopia is a world in which our experience is limited only by our desire and imagination.

No Pain

Let’s start with something simple: Imagine a life without pain. Pain is not a law of physics. It is a consequence of (1) current human biology and (2) our ignorance about how to modify or transcend it. Pain is just a mechanism stumbled upon by evolution to inform us that “you should avoid that” or “you should take care of that injury.” But we could have systems that give us that information without also giving us pain. That’s how today’s AIs do it. In the future, we’ll be able to master pain, in human bodies and other substrates.3

And that’s nothing compared to the best of what could come, if we do AI right.

No Death or Aging

We wouldn’t just eliminate pain; we could do away with death and aging as well. Death and aging aren’t written in the laws of physics, either. They, too, are merely consequences of current human biology and our ignorance about how to modify or transcend it. Turritopsis nutricula, the “immortal jellyfish,” is biologically immortal because it didn’t evolve to die after a few decades like humans did. And if our minds are uploaded to computers, we’ll be able to make backup copies and achieve digital immortality.

Would you even want to live indefinitely? I think Patrick Hayden got it right when he said, “Personally, I’ve been hearing all my life about the Serious Philosophical Issues posed by life extension, and my attitude has always been that I’m willing to grapple with those issues for as many centuries as it takes.”

Eliminating death and aging isn’t just wishful thinking or science fiction. The more we understand how aging and death work, the more we will be able to control them.

Radical Abundance

In 1959, Richard Feynman gave a lecture called “Plenty of Room at the Bottom.” In it, Feynman talked about how nothing in physics prohibits us from constructing objects atom by atom. He considered the implications this would have for storing information, chemical synthesis, and manufacturing. Decades later, the field of nanotechnology emerged, inspired by Feynman’s lecture. Today we are able to construct some things atom by atom, just as Feynman predicted, and our powers to do so grow every year.

Molecular nanotechnology (MNT) was envisioned shortly thereafter. MNT is the technology we will use to build the future. With intelligently-guided nano-factories or self-replicating nano-botsand a stock of materials, we will be able to rapidly erect arbitrary structures. If we can rearrange atoms in whatever configuration we want (as long as they obey physical law), we can make a banana without having to grow it. We can make a car without needing large factories to assemble it. If it’s made of atoms, you can probably make it with MNT and sufficient intelligence.

Why is this important? Poverty exists, in great part, because of poor resource allocation. If food and housing were as abundant as, say, air, then nobody would be homeless or starving, and MNT would make it ludicrous to buy a car for tens of thousands of dollars. Things that we consider luxury items today would be, more or less, equally accessible to everyone. When you’re able to manipulate matter at the atomic level, making a block of diamond is just as easy as making a block of coal.

Endless Adventure

The implications of a safe and effective MNT would go beyond eliminating poverty and making our lives easier. MNT opens up possibilities for creating much more powerful computers, because data storage and processing structures could be arranged as perfectly and efficiently as is physically possible.

What could we do with such vast computational resources? Think of Pandora from James Cameron’s Avatar. Pandora, a fictional world, is magnificent, breathtaking, and very different from our own. Yet, for all its fantastical elements, people’s experience of that world felt real. A bunch of moving images on a screen created an experience so profound, that people left the theater depressed at the thought that they could not live in Pandora.

Imagine creating a computer simulation of Pandora and being able to enter that simulation. You could live as an inhabitant of Pandora for a whole lifetime. You could see, hear, taste, and feel this universe just as intensely as one would in reality. We could simulate thousands of different universes and experience them all.

Engineering Utopia

These examples suggest a bare minimum for how good a positive intelligence explosion could be, in the spirit of exploratory engineering. The actual outcome of a positive intelligence explosion will likely be completely different, for example much less anthropomorphic.

Once again: our current limitations aren’t fixed by physics, but by the limits of our intelligence and the resources that can be used at our current level of intelligence. With self-improving machine superintelligence acting on our behalf, our biological limits can be transcended.

Humans have always wanted more than what the universe gives us by default. We want to experience sounds that do not exist in nature, so we make music. We want to taste something more delicious than anything that would grow by itself, so we cook, and we develop cuisines. We want to explore worlds beyond the one in which we evolved, so we build ships, cars, planes, and spaceships to carry us off into distant lands. We invent literature, film, and art to experience the world from a different perspective.

We are, and have always been, engineering our own utopia. With superintelligence, we will have the opportunity to do it faster, better, and more completely than ever before.

But only if we try. Only if we decide that utopia tomorrow is more important than slight improvements to our already rich lives today.

I wrote before:

We find ourselves at a crucial moment in Earth’s history. Like a boulder perched upon a mountain’s peak, we stand at an unstable point. We cannot stay where we are: AI is coming provided that scientific progress continues. Soon we will tumble down one side of the mountain or another to a stable resting place.

In which direction will you push on the boulder?