Policymakers should plan for artificial superintelligence systems by building governing systems and necessary infrastructure.
What's the worst super AI could do? Save life on Earth from annihilation by homo sapiens?
Experts from around the world are sounding alarm bells to signal the risks artificial intelligence poses to humanity. Earlier this year, hundreds of tech leaders and AI specialists signed a one-sentence letter released by theĀ Center for AI SafetyĀ that read āmitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.ā In aĀ 2022 survey, half of researchers indicated they believed thereās at least a 10 percent chance human-level AI causes human extinction. In June, at the Yale CEO summit,Ā 42 percent of surveyed CEOsĀ indicated they believe AI could destroy humanity in the next five to 10 years.
These concerns mainly pertain to artificial general intelligence (AGI), systems that can rival human cognitive skills and artificial superintelligence (ASI), machines with capacity to exceed human intelligence. Currently no such systems exist. However, policymakers should take these warnings, including the potential for existential harm, seriously.
Because the timeline, and form, of artificial superintelligence is uncertain, the focus should be on identifying and understanding potential threats and building the systems and infrastructure necessary to monitor, analyze, and govern those risks, both individually and as part of a holistic approach to AI safety and security. Even if artificial superintelligence does not manifest for decades or even centuries, or at all, the magnitude and breadth of potential harm warrants serious policy attention. For if such a system does indeed come to fruition, a head start of hundreds of years might not be enough.
