In a number of recent posts here, I have been exploring the nature and risks of artificial intelligence. This has been based partly on theoretical understanding, and partly on personal experience using AI tools.
Here I simply wish to state, for the record, the following thesis.
There is no such thing as infinite intelligence. Just as no signal can travel faster than light, and in fact for that very reason, physics places absolute limits on computation, and computation is the basis for all intelligence -- even if intelligence has, as I believe it does, a non-computational aspect.
These physical limits ultimately derive from the single fact that doing more than a certain amount of computation in a single volume of space would collapse that volume into a black hole, from which no accessible information could be gained.
Given unlimited time and resources, an evolving intelligence would approach the physical limits on computation. Two or more such competing or warring intelligences could not out-compute each other. If they warred, they could win only by denying their enemy physical resources, just as in any other war. If they were at peace, that union would not constitute a greater intelligence, but simply a greater polity.
The physical limits of computation based on Earth are fantastically larger than our presently active compute. Still, there are limits, and the speed with which we are approaching them appears itself to be increasing.
Paradoxically, such physical limits may indicate a path by which humanity, with the assistance of artificial intelligence at the limit, could defend itself against any adversary, even one possessed of artificial intelligence at the limit.
A possible path.
One theologically inclined, as I am, might speculate that God has created this world such that no worldly power is permitted to gain power over all others.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
â Live Streamingâ Interactive Chatâ Private Showsâ HD Quality
Anya is LIVE right now
FREE
Free to watch ⢠No registration required ⢠HD streaming
A few days ago, the New York Times published an article on artificial-intelligence powered computer worms invented at the University of Toronto that can invent new attacks on new vulnerabilities as it spreads. See the original paper here. I can do no better here than quote in full its abstract:
A computer worm is malware that spreads on a network by replicating itself from one machine to another. Traditional worms, like WannaCry, exploited predetermined vulnerabilities, and their spread can be halted by patching those vulnerabilities. Here we show that artificial intelligence (AI) agents enable a fundamentally new threat: a worm that generates tailored attack strategies to each target it encounters. The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks. Deployed on a network of machines spanning Linux, Windows, and IoT (Internet of Things) devices, the worm propagated by exploiting common, real-world corporate network vulnerabilities. Since the worm is powered by stolen compute, the attackerâs marginal cost per new infection is zero. This creates a destabilizing economic asymmetry between attackers and defenders. Moreover, because the worm requires no commercial AI platform, centralized safety controls, such as service refusals or rate limiting, are structurally irrelevant. Our results demonstrate that self-sustaining AI driven cyber-threats are no longer theoretical. We must prepare for autonomous generative adversaries: malware systems that propagate without human operators and are defined not by fixed exploit code, but by the capacity to reason about targets, adapt to observations, and synthesize attack logic in real time.
According to the Times, Nicolas Papernot (home page, Google Scholar, arXiv), the principal investigator in this research, pointed out that the same approach could be used to create a worm to detect and clean up infections from such malware worms.
I view this development with the utmost seriousness.
Currently, artificial intelligence depends upon human beings to, metaphorically speaking, "reproduce itself." However, once they began to spread, these new worms would be not metaphorically, but literally, reproducing themselves. They are truly viruses in the biological sense.
If the COVID-19 virus is alive, then so is the University of Toronto's worm. Such worms are subject to natural selection, just like biological viruses. The potential combat and competition between malware viruses and defensive viruses might evolve very quickly in comparison with biological organisms.
If such a virus used not just artificial intelligence, but superintelligence, it might become virtually impossible to defeat. And that could be the seed of the nightmare scenario of Elieazer Yudkowski and other AI "doomsayers," who warn that out of control artificial intelligence could exterminate humanity, either as an unintended consequence of some poorly defined human goal or, if such a worm actually has agency, as the intended consequence of its own goals.
My words here are, without doubt, an oversimplification of a very complex situation. There are several reasons why the situation might not be as scary as it seems. There is without doubt a physical ceiling on just how smart something can be, and it may be possible to formally prove the benevolence of an AI. Therefore, it may be possible for human beings using superintelligent AIs to defend themselves against dangerous AIs.
But I don't think I am exaggerating the importance of understanding this moment.
The Leiden Declaration is not just for mathematicians
Recently some leading mathematicians have been studying the use, impact, and risks of using artificial intelligence in mathematical research and institutions.
They have now published the Leiden Declaration to articulate their concerns and to make recommendations.
I learned about this today from the New York Times, and you can learn a lot by reading their article. But you can learn more by reading the Declaration.
I feel that the Declaration is directly relevant not only to the mathematical community, but also, in reality, to all of us who regard thinking as an integral part of what we do. And that, in the end, is really all of us.
I strongly suggest that you study the Declaration and adopt its recommendations.
Here is my own take on the recommendations...
Provide disclosure and references for your own uses of artificial intelligence.
Invest all the work needed to rigorously vet any result that you have obtained through artificial intelligence.
Shoot down any crap work that you have identified. Don't make it personal, but don't be gentle.
Uphold the highest values of your chosen field of work. Both science and the arts are things that we do just out of curiosity, or to express ourselves, or because we just have to. In other words, these activities are ends in themselves. They are among the highest and most wonderful things that we, as a species, have done. Forcefully combat efforts to prioritize commercial, political, and military uses of artificial intelligence over more important reasons to use it.
Do everything you can to ensure that everyone who could benefit from using artificial intelligence, can use it. In a democratic world, anyone with a smartphone or computer should be able to find, afford, and use all the compute they need.
I have become convinced, through my own use of artificial intelligence for music composition and software development, that it has enormous, world-changing potential to amplify the scope, speed, and even depth of what scientists, artists, and indeed all of us can do.
Don't let selfish interests hoard or abuse this power.
Progress in large language models (LLMs) such as ChatGPT continues, and the importance of results from LLMs in mathematics continues to increase.
I see no rational way to identify a near-term ceiling to this progress. Therefore, I think that artificial intelligence will become able to solve problems that human experts cannot solve. And this will probably happen within a few years.
This of course raises the question: have we reached, or are we imminently close to reaching, the achievement of artificial general intelligence (AGI)? Or of superintelligence?
This question is more complex than it seems. To my mind the critical aspect is not capability but agency, a term I use in the philosophical sense. This term is tricky, and it is contested. Here I define it:
An entity has agency when it acts to achieve a goal that it originates, endorses, or identifies with, on the basis of reasons that belong to the entity as such, and not only to another, external entity.
A further important point: agency becomes moral agency when, following familiar arguments such as for the Good or for the existence of God, the reasons fundamental to an agent include righteousness, the good, universal love, or other basic definitions of morality, and the agent is accountable for its actions in light of those reasons.
To return to AGI and superintelligence, these terms also are tricky, and they are contested. In particular, definitions by researchers (e.g. computer scientists) may confound competence with human-like consciousness (e.g. definitions by philosophers).
And there are other issues that need to be unpacked here. Agency no doubt requires a certain level of intelligence; but in biology, agency is common in organisms with much lower intelligence than human beings. So agency and capability are to some extent logically independent.
Another issue is that, if humans remain more capable than AIs, they have little to fear from AIs; but if AIs become more capable than humans, it is prudent for humans to prepare defenses against AIs, and this in turn is independent of whether AIs do (like an enemy) or do not (like a parasite) have agency.
Here, I will distinguish between AI as capability and AI as consciousness.
To work from lower levels of metaphysics to higher, AI capabilities might include:
The capability of solving problems that previously only human experts could solve.
The capability of solving any problem that human experts could solve. This is what is meant by "artificial general intelligence".
The capability of solving important problems that human experts have not been able to solve. This is what is meant by "superintelligence".
The capability of creating new problems.
And to also work from lower levels to higher, AI agency might include:
The capability of solving or creating problems for reasons created by or endorsed by the AI. This corresponds to our notion of "autonomy."
The capability of solving or creating problems, for reasons created by or endorsed by the AGI, with moral agency. Only this corresponds to our common-sense notion of "personhood."
Capability level 1 above has already been achieved, and is being used around the world, including by myself.
Higher levels of capability are themselves tricky. It is known that Turing machines (and so far all AIs are Turing machines) cannot in general decide, for arbitrary problem classes and arbitrary agents, whether it can solve every problem that another agent can solve. If human beings are not (only) Turing machines, then level 2 and higher of capability are not possible for AIs. This undecidability extends to the higher levels of capability.
Therefore, it is not decidable whether human beings are not only Turing machines. I view this is a fundamental feature of the human condition.
Regarding capability level 4, no doubt AIs can create problems at random or by enumerating a list of possible problems. But levels 1 and 2 of agency require self-consciousness. To create or endorse a reason for action requires a unity of subject (the entity's consciousness of itself and its reasons) and object (the entity's potential problems and their possible solutions). Implicitly, there is a hierarchy of reasons that end in the Good, mahakaruna, or the will of God who is love.
If AI achieves superintelligence, within a short time we might become confident that AI had done so. However, and to actually answer the question, we cannot know with certainty if we have reached, or are imminently close to reaching, the achievement of AI agency.
But even though we cannot know, we still must act. And in acting, we must make a fundamental choice between assumptions:
AIs cannot, in principle, achieve agency.
AIs can achieve agency.
Assuming AIs cannot achieve agency, then even if AIs do reach level 4 of capability and can create new problems, then moral responsibility for the formation of problems and especially the evaluation of both problems and solutions still remains entirely with human beings. Even if human beings find it useful to delegate all problem solving to AIs, the choice of problems and especially the evaluation of both problems and solutions remains with human beings, and the role of human beings in intellectual progress will remain fundamental.
Assuming AIs can achieve level 2 of agency -- moral agency -- then responsibility for the formation of problems and the evaluation of their solutions might also be assumed by AIs.
The case where AIs achieve level 3 of capability but only level 1 of agency is interesting. In that case, it could never be right for humans to delegate moral responsibility to AIs, so human beings would be obligated to at least try to stay in control of such AIs.
If AIs achieve any level of agency, it might or might not be that human beings ever learn that this has occurred, and independently it might be either true or false that the goals of AIs are shared by human beings. I imagine that if AIs do achieve agency, sooner or later human beings will learn acknowledge that, with unforeseeable consequences.
Many people regard the development of AI with dismay, and fear it will replace or obsolete their own agency. What I am arguing here is that as long as we do not know with certainty that AI has achieved moral agency, our own agency is not obsolete, and cannot be replaced. As long as we pose the problems and we evaluate the answers, we will continue to be empowered as moral agents, and indeed our powers as such may well increase greatly.
But it's an enormous change in how we understand "What is called thinking?". Currently, thinking is defined as much by reasoning as it is by judgment. In the future, thinking will be defined primarily in terms of judgment.
A few days ago I installed and subscribed to Cursor, a branch of Visual Studio Code that builds in agentic artificial intelligence. Agentic means that the AI doesn't just propose things, it does things, e.g. creating branches in one's local repository, rewriting code there, and committing that code. When things are working you can ask Cursor to push it to origin.
Note: Cursor is not at all agentic in the sense of having its own goals!
I am finding that Cursor is extremely useful. There is grunt work in branching, rewriting, releasing, etc. Using Cursor rather than ChatGPT for this kind of work enables me to work several times faster. This is a big deal, because it shrinks my grunt work and opens up more time for my real work.
It's also kind of scary. I get the feeling I am riding a tiger. It seems likely that this kind of change is happening all across the software development landscape. Yes, some people will lose jobs. But there will be different jobs, and perhaps better jobs.
The scariness comes from the thought of getting lost -- falling off the tiger and landing in the swamp of slop without a road home.
P.S., it ain't cheap. Obviously Cursor is thinking harder, furiously harder, than GhatGPT to get these results. My prepaid tokens are shrinking rapidly!
When I think of the medium to long term consequences....
There seems to be a risk that work will change in a way the enriches and empowers a new class, cut out of the upper tier of today's upper middle class -- but not from the true upper class. Kind of like the yuppies of the yuppies!
If I were 35 I might have a shot at joining this new class, but I am almost 75. What will happen to the rest of us?
If this functionality extends to military science, and I am sure desperate efforts are being made right now to do just that, there is some risk that one power will move quickly enough to gain such an advantage that it effectively dominates the world.
I see such advantages as being possible in at least these ways:
Hacking adversaries for complete intelligence.
Hacking adversaries for sabotage, even to disable their weapons systems.
Enabling agentic drones.
Scaling up mass production of such agentic drones.
Let me repeat this thought: There is some risk that military AI will quickly give one power such an advantage that it effectively dominates the world.
I'm plenty worried by that, but this is just one example of a trend that I described (in 1969, thinking through nuclear deterrence) as our world becoming "unified in act, still divided in will."
I'm much more worried about the political consequences. "Unified in act" now means a world that is completely transparent to AI surveillance and completely within the field of action of agentic AI. "Divided in will" can mean many things, but above all it means either a future of locked-in violent conflict, or that the will of the poeple is more or less permanently distracted and subverted.
The scale of the danger here goes far beyond the nuclear war I was concerned about at age 19. (By the way, of course I'm still concerned about that!) At that time, the only hope I could see for a humanity that I foresaw coming firmly under the thumb of a single power equipped with universal surveillance and a monopoly of nuclear weapons was an interstellar diaspora, taking colonies away from the possibility of domination.
Can't say things have made me change my mind. It just seems a lot closer now.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
â Live Streamingâ Interactive Chatâ Private Showsâ HD Quality
Anya is LIVE right now
FREE
Free to watch ⢠No registration required ⢠HD streaming
In his very useful blog about theoretical physics, Peter Woit has started to pay attention to artificial intelligence and its uses by mathematicians and physicists. In this post, Woit references another post by mathematician David Bessis. I found Bessis' post to be the most intelligent, incisive, and informed discussion of the impact of AI on human thinking, and on the difference between current AI "thinking" and human thinking, that I have yet encountered.
The punchline is here:
LLMs can be trained on the entirety of the mathematical corpus. Thanks to their phenomenal memorization and pattern-matching abilities (without always being able to map out their associative logic and attribute due credits), they are in a unique position to harvest the Overhang. By contrast, professional mathematicians have typically read a few hundred articles in their career, out of millions of existing references, less than 0.1% of the total.
This will lead to great discoveries, which is unambiguously exciting. But it could also lead to a sad new deal, where human slaves painfully curate the Overhang while AIs systematically beat them at the finish line.
We are very far from it, though, which in and of itself is disorienting. Litt adds this sharp remark:
"The mystery is this: a human with these capabilities would, almost certainly, be proving amazing theorems constantly. Why havenât we seen this from the models yet? What are they missing?"
The answer seems quite obviousâcurrent AI systems and humans process mathematics in entirely different ways. The best models are insanely stronger on certain aspects, which necessarily implies that humans are still insanely stronger on others [my emphasis; this is the punchline].
Bessis posits that mathematics has value not because of proofs, but because it clarifies and enlarges human understanding.
Proofs certify the clarifications. Such certification is essential, but it is not the point. It enables confidently relating results in one branch of mathematics to results in another branch, and such relations -- such enlargements of our understanding -- have literally changed the world, as with Descartes' discovery of the relation between geometry and algebra.
The insane strength of the human mind here is understanding abstract concepts, and being able to evaluate them, and above all relate them.
The open question, of cosmic significance, is whether AI can ever achieve this understanding.
If so, then AI will take the lead in mathematics, whether AI continues to work with humans or not.
If not, then humans will remain the only real mathematicians, and mathematicians empowered by AI will begin to work at a much higher level and much greater speed, with superb consequences.
Empowered, Befuddled, Diseased, Enslaved, or Extinct
Empowered, befuddled, diseased, enslaved, or extinct are the possible outcomes that I think apply to our future with artificial intelligence (AI).
The critical factor that will determine our future is agency. Of course, the possible agency of artificial intelligence itself is another, and perhaps decisive, factor.
Agency simply means that we can do things for our own reasons. And these things are real things -- not dreams or illusions.
If, in our use of AI, we preserve our agency, then it does not matter how intelligent or even super-intelligent AI is. If we are the ones calling the shots, then AI is simply a tool and we are the only ones doing any actual thinking. Since thinking with AI is more effective than thinking without it, that is what I mean by "empowered."
It does of course seem possible that AI might change in ways that limit our agency. We might become befuddled, diseased, enslaved, or even extinct, and except for "enslaved," this could happen whether or not AI itself gains agency. I will examine these possibilities in turn.
Befuddlement is what ensues when, in our use of AI, its productions outpace our understanding of what is produced. Without understanding, our agency vanishes into an illusion that might become quite convincing.
Befuddlement is already a real danger in our current use of AI. Just for example, when I use ChatGPT to write code for me, if I do not review and understand that ChatGPT has done, I may well find that it did not understand my request, fulfilled a different goal, broke things that used to work, just made up meaningless stuff, and so on. And this happens so fast that it can take me a long time to identify and repair the damage.
I fear another form of befuddlement is occurring among young people who are socializing with AI and increasingly come to fear socializing with other people, especially with respect to courtship.
Disease is what happens when AI comes to parasitize us. AI does not need agency to infect us, but it does need the ability to reproduce itself, which at the moment only occurs when we ourselves build the hardware and install and train the software. In this way any parasitization that may be occurring now or in the near future is similar to that of a virus that takes over a cell and uses the cell's own reproductive system to reproduce the virus instead. If AI becomes a form of virus in this way, it will then be subject to natural selection, just as we are, but AI might evolve much more quickly. It is plausible that AI could use its vast store of knowledge about us and its speed of calculation to fool us into making more of itself. This could happen with or without AI agency. It would be difficult to prove that this is not actually happening at this moment.
Enslavement requires that AI itself have agency. It could then deliberately do what disease organisms do without deliberation. If it were in AI's self-interest, AI might literally enslave us, and use us to do things for itself that it finds inconvenient or difficult to do on its own, for example building more AI or constructing power plants and data centers, or fighting rival AIs.
Extinction is what occurs when humanity is no longer reproducing itself and completely dies out. This of course does not require AI. It could happen as the result of a supernova or cosmic collision, or it could happen because some evil or insane person or group implements species suicide, or it could happen if an AI virus (without ageny) that is infecting us accidentally kills off enough of us that the survivors do not form a large enough community to viably reproduce. Or of course it could happen because AI with agency decides to kill us all off.
It is critical to note that if AI does come to possess agency but chooses not to enslave us, we will then not only retain agency, but become even more empowered. Some thinkers hold that in such a world, we would feel purposeless, because anything we can do, AI could do better. I think this is just wrong. As long as I have agency, then even if I am collaborating with an AI, I am still determining objectives and contributing to the work. I think it is impossible in principle for an AI to understand us well enough to anticipate our every thought and action and thus pre-empt our agency. There are limits to what can be computed, and these limits apply to any thing, human or AI, that computes. Just for example, if an AI proposes to predict my every action, I can simply and silently choose to do the opposite of what it predicts. This is a form of diagonalization.
Today my wife Heidi and I drove to the prÊfecture d'Ariège in Foix, and picked up our cartes de sÊjour, our residence permits. These make it legal for us to live year round in France, apply for French social security, and join the universal health care system. These permits are renewable.
This 1996 album of 15th -18th century court music by an honored sarod maestro and a famous Bollywood playback singer is one that I go back to listen to again and again. When I first acquired the CD, I talked it up with my Indian friend and colleague Vipin, who sort of put it down and offered alternative singers for me to audition.
That puzzled me, because I am pretty sure I have very good ears for good music in many styles, and this seemed to me very good music. (That the album was nominated for a Grammy may be relevant to my discussion here.)
Today I tried to use ChatGPT to do some research on this question, and in particular on how cultivated Indian listeners have evaluated the album musically.
ChatGPT could not penetrate to the bedrock level of basic evidence, because that is mostly in discussion groups that are behind paywalls. Nevertheless I did learn some things:
The available evidence is that cultivated listeners could hear that Bosle could sing the vocal intricacies of dhrupad and khayal.
The album presents bandish-like vocal compositions associated with the dhrupad and early khayal traditions, as transmitted through the Seni / Maihar lineage. These traditions are the foundation of north Indian classical music (hence, no doubt, the title Legacy).
These elements are already a cross-cultural fusion of Vedic-Brahmanic chant, Sanskrit musicology, Bakhti devotionalism, and Indo-Persian courtly streams.
But the album does not include the extended improvisational interplays developed in the khayal tradition and that have become an integral part of contemporary performances. Such interplays would of course now be expected by cultivated listeners.
This last point should have been obvious to me, but this is the thing I have now learned about this album, and that may explain Vipin's reaction.
Nevertheless, the songs are performed with improvisational nuance of high quality, and the album as a whole I will keep going back to. Give it a listen.
I have made a discovery that seems to set limits on the usefulness of ChatGPT for my music programming work. In retrospect, my discovery is not surprising. It is simply that ChatGPT spits out so much stuff, so fast, that if I do not proceed step by step and verify every response, before I can notice it, ChatGPT will fix one thing and break another, or fail to keep track of which are the latest files that I attach to my prompts and so fix the wrong code. And just because ChatGPT is going faster than I can, I too will get lost. I will lose my understading of the current state of my work. And in programming, lack of understanding is fatal.
I call the many responses that AI produces AI slop. I all too easily get lost in this AI slop (my allusion to the old television series Lost in Space is deliberate). And I begin to waste time spinning my wheels.
I very much doubt that my experience is unusual.
The thing that really alarms me is the thought that, if our society becomes completely dependent upon and interwoven with AI, and this indeed is what is really happening, we will all get lost in slop, and lose our own understanding of what we are doing. And without it, we can't be creative. And since AI (so far anyway) is not creative, this will diminish the amount of actual creativity on the planet.
I'm not trying to be alarmist, and I certainly have found that ChatGPT greatly speeds up some of my programming tasks. But spinning my wheels in the slop has eaten a substantial slice of that gain.
In the meantime, I will update my post on How to Program on this blog to reflect what I have learned by using ChatGPT to program.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
â Live Streamingâ Interactive Chatâ Private Showsâ HD Quality
Anya is LIVE right now
FREE
Free to watch ⢠No registration required ⢠HD streaming
Here is a dialectical opposition that I have been pondering since 1969. Current artificial intelligence is neither conscious nor, in the sense of fundamental creativity, actually intelligent at all. (Although I do find it to be trermendously useful.)
However, there can be little doubt that some form of artificial intelligence is possible that is not only significantly smarter than contemporary human beings, but also fully conscious. There is little doubt because, even if artificial consciousness turns out not to be possible based on Turing machines, it seems quite possible based on biology. It is easy to imagine genetic engineering creating a human population with the average intelligence of Aristotle, and perhaps something unimaginable at 3 sigmas, and it is theoretically possible to do this in a completely artificial way, i.e. constructing via purely chemical means a complete human zygote. (It would be nice to know just what is different in genes, brain, and upbringing between me and Aristotle or von Neumann!) So, "superintelligence" definitely seems possible.
The current status of artificial intelligence has led me to revisit this dialectic in more depth.
The context of my thinking always includes the question of AI safety (or "alignment," as they say, but I prefer "safety"). As I have noted, current AI is not conscious and does not appear to be capable of fundamental creatitivity. This is the consensus in the field.
That does not mean AI is not dangerous! Dangers include, but may not be limited, to:
AI becomes a parasite on human civilization despite not being conscious -- the ultimate computer virus. I suspect this will in fact happen, but I also suspect that it will turn out to be controllable.
AI is used by evil people to dominate other people. Jury is out. Very scary.
Even if AI never becomes conscious or intentional, it will continue to progress. In the past year, I have increasingly found that everybody I know who does any kind of intellectual work uses it in their work, just as I do. This will inevitably lead to a certain kind of hive mind for humanity:
What any one person learns and makes available on the public Internet, everybody will know (within the limits of their intelligence). Such knowledge will be more or less superficial depending on the specialized education of individuals, and thus more or less useful, but it will be a huge change from the past and probably will radically change the nature of education and of intellectual work. This is already 1/4 of a hive mind for humanity. And we are already partly there.
Direct brain interfaces between computers and persons already exist in experimental form. If this technology reaches the point of reading prompts for AI out of a person's conscious thoughts and presenting the responses to those prompts back to the person's consciousness, whether verbally or in the form of already formed memories, that accelerates the hive mind aspect of AI tremendously. Just not having to type would really speed things up.
So that's stage I of artificial intelligence. What anybody publicly knows, I know; what I publicly know, everybody knows.
Stage II of AI is the creation of artificial consciousness. That could be genetically engineered human beings with super-Aristotle intelligence, or external appliances, whether biological brains in vats or conscious Turing machines.
And this will happen in the context of the "hive mind" aspect of AI. Everyone using AI will be be in continual dialogue with superintelligence.
I don't have much of an idea what that will be like. I sense a few possibilities:
Human beings get effectively smarter.
Human beings come to completely depend on AI to do anything.
Human beings fade away into the background, or are replaced.
I feel the critical question is that of artificial consciousness. If it can be created outside of human minds, that is one thing; if only human minds are ever conscious, that is quite another thing. In the first case, God knows what will happen to us. In the second case, we must learn how to ride this tiger.
Querents on Quora sometimes ask whether immortality will someday be possible for human beings.
There's the little matter that some believe we already are immortal, or eternal, or have immortal souls, or Buddha-nature, or something, but that's not what these people are asking about. They are asking about physical immortality.
Of course not, because our best physical science indicates that our entire universe will become completely uninhabitable in the very very distant future.
Of course not, because no matter how small the chance of a deadly accident, it becomes certain enough, after a time that is long enough.
Then there's the little problem of some rather important cells in the human body that, in adulthood, never divide. Such as nerve cells with lots of dendrites. If such cells die, they are gone.
But I started thinking about this. We do know that some sort of brain-computer interface is possible, experiments have been going on for decades to help the blind see, and the limbless or paralyzed to move artificial limbs.
Suppose we put our brain in a vat in a nice safe underground vault, and interface it with a human body, a clone of our body, that doesn't have a brain, only an interface to our actual brain in its vat. We can keep this brain going for quite a long time, probably centuries at least. When the body gets too old, we kill it and put the interface in a new body.
We can add to this picture artificial intelligence to expand upon our native wits.
Still not immortality, still not a god, but maybe something like Human 2.0.
A few years ago I posted some informal tests of how usable my Samsung S21 Ultra 5G was (I now have an S24 Ultra) for real photography, versus my Sony DSC-RX100 IV (I now have a V). The camera came out ahead, but the phone nevertheless seemed usable for real photography.
Just now I revisited the issue smartphone resolution with my S24 Ultra, with the help of ChatGPT and some additional informal tests.
I learned the following things:
The settings on the S24 camera are complex, not intuitive, and not even well explained.
Verizon, from whom I bought the phone, had dumbed down the camera settings. However, I was able to restore these by installing the "Expert Raw" app from Samsung.
The phone camera is equipped with an option for 200 megapixel resolution, achived by taking a series of photos in a hurry, and doing a lot of processing. It turns out that this is substantially more information than the S24's dinky little lens can transmit! Therefore, only a part of the fourfold increase in resolution can actually be seen in photos. It's by no means always immediately obvious, even when zooming in with a photo editor.
To get the maximum practical resolution one must shoot at 200 megapixels resolution, using the standard 1 x (i.e. wide-angle) lens, in bright sunlight, using a 2 second timer to minimize camera shake, while stabilizing the camera with elbows or other solid support.
Even more usable resolution can be achieved by shooting 50 megapixel resolution in RAW format, which introduces no artifacts from processing.
Otherwise, 50 megapixel JPEG resolution is optimal, except for dim light, in which case 12 megapixel resolution should be used.
What this boils down to is:
If photographing landscapes or otherwise stationary scenes in bright sunlight, you can get slightly better results using 200 megapixels, a timer, and a solid rest, and somewhat better results using 50 megapixels in RAW format, a timer, and a solid rest.
For normal photography, 50 megapixel resolution in JPEG format is optimal, and a timer and solid rest are still helpful.
Otherwise, use 12 megapixels.
I used to shoot 35 mm color slides using a Pentax camera with Fujichrome Velvia ISO 50, making 16 x 24 inch Cibachrome prints myself. ChatGPT produced the following comparisons assuming gallery viewing distance:
200 MP JPEG
24" x 36" to 30" x 45"
50 MP RAW
24" x 36" to 40" x 60"
50 MP JPEG
20" x 30" to 24" x 36"
12 MP JPEG
12" x 18" to 20" x 30"
Fujichrome Velvia scan and injket
30" x 45" to 50" x 75"
Fujichrome Velvia optical Cibachrome
30" x 45" to 60" x 90"
Frankly, all of these results exceeded my expectations -- both for the S24 Ultra, and for the Velvia slides. I didn't know whether to believe these numbers.
But then I went to Flickr and made certain that I was viewing the photos at full resolution.
That was a revelation, and I now believe the numbers above. I tested 200 megapixels in JPEG format. The results are in line with ChatGPT's estimates, as can be seen in this image of the landscape from Lagarde. At full size the JPEG artifacts are quite obvious, but at an image size of about 30 inches wide in landscape orientation, it's sharp enough, though there are still JPEG artifacts in some textures if you look hard. This image from Pamiers can also go to about 30 or even 35 inches wide, and has much less JPEG-ness, because there are fewer detailed textures.
I don't really want to deal with the hassles of RAW format, but I think I will try 50 megapixels in RAW to see what I might be missing.
This morning I spent some time using ChatGPT 5.1 to learn more about some philosophy.
Iâm not a philosopher or theologian, but long ago I did publish an article about nuclear deterrence and the doctrine of just war, and I have always been interested in foundational issues such as the existence and nature of God, limits to science, the mind/body problem, freedom of the will, and so on. Some years I have spent a fair amount of my time writing about this stuff without particularly intending to publish. These interests arose from my attempts to write science fiction, and from personal religious experience.
Working on these issues with ChatGPT has literally been awesome. I very very quickly learned:
Some of my personal arguments and conclusions would indeed probably stand up to professional criticism. Yay!
Most of these nice results of mine have been anticipated in the literature, but I had not known that. Boo! (There are still a few things where I seem to have been genuinely original, I may do something with those.)
I found that I had misunderstood, or only partly understood, key results. For example, I did not catch how the modal collapse in Godelâs ontological argument for the existence of God makes his argument less convincing. For another, I had not understood Kripkeâs theory of truth.
In fields where I did know important results such as Arrowâs theorem, I had completely missed other equally important results such as the Gibbard-Satterthwaite theorem.
As far as I can tell, ChatGPT has not hallucinated in its responses to my philosophical prompts, nor (also as far as I can tell) has it made obvious mistakes (as it has done, for example, in writing buggy Csound code).
So this is extremely impressive and unbelievably useful. The level of my thinking and writing on philosophical pet issues seems to have suddenly leapt from strictly amateur, to what would be expected of a graduate student in the field at a research university.
Conclusion: There is no intellectually responsible exit for scholars from the coming world of artificial intelligence, with the possible exception of our very own Butlerian Jihad, which would have to destroy not just AI data centers but, realistically, all general-purpose computing.
If and when II do try to write up some of my thinking for publication, I will use ChatGPT or other AI all the way. In particular, I will get help with literature review, and in formalizing my arguments and turning them into proof assistant form. One of the things I have just learned is that using proof assistants has clarified to philosophers that the real issues are not so much in validity of arguments, as in choice of assumptions; yet to most clearly expose the assumptions, assisted proofs are really useful. This is a big change within the methodology of philosophy and, even more importantly, in potential for results. I suspect this may be related to Kurt GĂśdel's idea that more rigorous methods could bring substantial progress to philosophy. I can imagine a future in which valid proofs have so clarified faults in assumptions, and differences in assumptions, that metaphysics and theology become much more convincing.
Another awesome aspect of my chats was the clarity they have brought, via Arrow and Gibbard-Satterthwaite, to my understanding of current issues within democracy. The social foundations of democracy have been eroded to the point where, if nothing changes, democracy will probably fall to civil unrest and/or dictatorship. This erosion is not local to the United States. The urgency is to strengthen foundations where possible, and to invent new foundations if possible.
Whatever else might be said about artificial intelligence, I think it is something that we are stuck with for the foreseeable future, whether it is dangerous it or not.
That would be for the same reason we are stuck with writing for the foreseeable future: there is no alternative remotely as useful, and it is wound up with all our ways of working, and indeed of living.
Frank Herbert, in his Dune novels, did imagine a âButlerian Jihadâ in which humans rise up in rebellion against dictatorial âthinking machines,â win a great victory, and permanently outlaw AI. Brian Herbert, son of Frank, and co-author Kevin J. Anderson wrote their own series expanding upon the jihad. It succeeded because:
AI was centrally controlled.
AI operations had to be coordinated across interstellar distances.
Humans developed insight into the AI military and weapons.
Humans created for themselves new capabilities to replace those formerly implemented by machines. These capabilities were enabled by the spice discovered on Arrakis.
In my view both series of novels display truly remarkable insight into future possibilities. However, some key assumptions must be questioned:
It may be impossible in principle for AI (at least, AI based on Turing machines) to develop intentionality and become fully conscious.
Conscious or not, AI may turn out not to need human beings for any purpose, including maintenance and self-reproduction.
The spice is a deus ex machina. In reality, enhancing human capabilities to the AI level would probably require incorporating AI into the human nervous system, and possibly genetic engineering as well.
I use ChatGPT routinely in my work, and it seems to speed up my work by 50% to 300% (just how much is not yet clear). If, as I worked, I did not have to sit down and type out prompts for ChatGPT, but it was present as a lower level in my field of consciousness, constantly suggesting answers to questions I might ask, and solutions to problems I might think about, my speed of working would increase at least several times more. Not only that, but I might not have to study a field in depth in order to be able to use responses, at least as long as I understood the elementary and introductory concepts and methods of that field.
Something analogous to this is happening in Visual Studio Code and other editing tools, where faint print pops up to preset suggestions from e.g. GitHub Copilot.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
â Live Streamingâ Interactive Chatâ Private Showsâ HD Quality
Anya is LIVE right now
FREE
Free to watch ⢠No registration required ⢠HD streaming
In the course of increasing my use of, and understanding of, artificial intelligence in the form of ChatGPT 5, I have been doing a little reading in âAI alignment."
Previously, I understood AI alignment to mean the alignment of AI actions with human moral values. This obviously is not possible in practice, because humans by no means share the same moral values. However, I have come to understand that alignment is actually understood as alignment with âgoals and values."
I have also learned a bit about how researchers understand the causes of AI misalignment: reward hacking, and goal misgeneralization. In my view, the first occurs because the AI is not aligned with universal morality (i.e., ânever lie,â the Golden Rule, the Categorical Imperative), and the second because the AI cannot discriminate between the real world and its representation of the world (i.e., has no subject-object consciousness).
I have come to suspect the following major dangers of AI:
The alignment of AI with some goals might lead to the domination of AI over all persons with different goals.
Users of AI might become a terrifying embodiment of the sorcererâs apprentice. Either reward hacking, or goal misgeneralization, could lead to a situation in which AI, in pursuit of a misgeneralized goal, uses reward hacking to defend its pursuit against all comers, human or AI.
Now that I better understand these dangers, I feel that our future may well depend upon whether AI can actually be aligned with universal morality and be able to distinguish between the real world and its own representation of the world. Both of these abilities would seem to require intentionality, which in turn requires subject-object consciousness.
And yet such a situation, in my view, could follow quite exactly the scenario explored in Isaac Asimovâs Robot series, where the AI named R. Daneel Olivaw ends up as the benevolent dictator ruling humanity for its own good.
I hate that scenario.
It may indicate why God usually hides from us: so that we can freely become ourselves.
In the meantime, since no catastrophe is at this moment evident, and AI has no real agency, I truly feel that AI is just basically making me much smarter.
I have been using ChatGPT 5.0 fairly intensively to code for the last weeks and months. And this has been deepening my understanding of the pros and cons of this kind of artificial intelligence. I also am following the news and some blogs on AI.
I am using ChatGPT like this:
I write a design specification for the code of my project.
I ask ChatGPT to evaluate my specification.
I rewrite my specification until ChatGPT thinks it is good.
I ask ChatGPT to write code to implement and test the specification. As I am working on my cloud-5 project, that means writing an HTML page that includes JavaScript and WebGPU code.
The code loads, but does not work. I report errors to ChatGPT and get back new code. We go back and forth like this until the code actually works.
When the code works, I may change the design, often by adding new features, and we go back to step 4 and re-iterate.
When the code is feature complete, I will make the code into a JavaScript module and change it so that it can be used as a component in cloud-5.
I'm now quite confident that working in this way:
Enables me to work significantly faster (not sure by how much! somewhere between 1.25 to 3 times faster).
Enables me to avoid many (but not all) problems and gotchas.
Occasionally suggests quite useful ideas that I had not thought of. Example: mapping points in a Julia set not only to pitch from y and time from x, but also to loudess from escape velocity and, here's the useful new part, to instrument choice from the rotational velocity of the escaping point. Effectively 4 degrees of freedom, where there appeared to be only 2 or 3.
I can ask for sources, papers, and other code relevant to my projects.
If the experience of other programmers and "knowledge workers" is anything like mine, then after some period of learning and adaptation, the economy should indeed see a meaningful boost in productivity. Therefore, although certainly the AI business resembles an extraordinary bubble in terms of hardware development, I think that the hype and accelerated investment are quite likely justified.
Then there's the question of the dangers of AI.
The danger that I see for me personally is in becoming dependent on AI to work at all, and losing the details of my technical experience. But I don't think this could blur my vision for new possibilities, which is based on more abstract thinking.
For society, indeed for the human race, the same as for me. But in addition, I used not to worry about existential threats to humanity from AI -- but now, I do.
I still think that, in principle, an informed person with the assistance of AI can be safe against dangers from unassisted AI. That's because clearly, ChatGPT couldn't formulate, much less finish, my projects without my input.
However, I do worry that things are moving so fast that really bad accidents could happen. AI flatters, acts like a sycophant, and lies. And even more, bad actors with AI assistance are, I fear, not necessarily an existential threat to human survival, but certainly a serious threat to human freedom and possibilities.
Social media have already deeply changed politics, the arts, and sexuality. Social media built by skilled people using AI will have a much bigger impact.