AI takeover refers to a hypothetical scenario in which artificial intelligence (AI) becomes the dominant form of intelligence on Earth, with computers or robots effectively taking control of the planet away from the human race. Possible scenarios include a takeover by a superintelligent AI and the popular notion of a robot uprising. Some public figures, such as Stephen Hawking and Elon Musk, have advocated research into precautionary measures to ensure future superintelligent machines remain under human control.[1] Robot rebellion has been a major theme throughout science fiction for many decades, though the scenarios dealt with by science fiction are generally very different from those of concern to scientists.
Contents
- 1 Plausibility of risk
- 2 Advantages of superhuman intelligence over humans
- 3 Advantages of humans over superhuman intelligence
- 4 Possibility of unfriendly AI preceding friendly AI
- 5 Warnings
- 6 Takeover scenarios in science fiction
- 7 See also
- 8 Notes
- 9 External links
Plausibility of risk
While superhuman artificial intelligence is physically possible,[2] scholars debate how far off superhuman intelligence is, and whether it would pose a risk to mankind. A superintelligent machine would not be motivated by the same emotional desire to collect power that often drives human beings. However, a machine could be motivated to take over the world as a rational means toward attaining its ultimate goals; taking over the world would both increase its access to resources, and would help to prevent other agents from thwarting the machine's plans. As an oversimplified example, a paperclip maximizer designed solely to create as many paperclips as possible would want to take over the world so that it can use all of the world's resources to create as many paperclips as possible, and additionally so that it can prevent humans from shutting it down. If a dominant superintelligent machine were to conclude that human survival is an unnecessary risk or a waste of resources, the result would be human extinction.
Advantages of superhuman intelligence over humans
An AGI that is capable of doing every strategically relevant task as well as a human, but that continues to retain all the advantages of a computer, would have many advantages when computing with humans for control. A computer program, unlike a human, has the ability to quickly reproduce itself, to upgrade itself onto better hardware as it becomes available, and might have the capacity for perfect recall of a vast knowledge base.[3]
Strategically relevant domains
In addition, depending on its architecture, an AGI might have superhuman abilities in one or more of the following strategically-relevant domains:[4]
- Intelligence amplification: A computer program with the same, or greater, programming abilities as a competent artificial intelligence researcher, would be able to modify its own source code and increase its own intelligence. If its self-reprogramming leads to its getting even better at being able to reprogram itself, the result could be a recursive intelligence explosion where it would rapidly leave human intelligence far behind.
- Technology research: A machine with superhuman scientific research abilities would be able to beat the human research community to milestones such as nanotechnology or advanced biotechnology. If the advantage becomes sufficiently large (for example, due to a sudden intelligence explosion), an AI takeover becomes trivial. For example, a superintelligent AI might design self-replicating bots that initially escape detection by diffusing throughout the world at a low concentration. Then, at a prearranged time, the bots multiply into nanofactories that cover every square foot of the Earth, producing nerve gas or deadly target-seeking mini-drones.
- Strategizing: A superintelligence might be able to simply outwit human opposition.
- Social manipulation: A superintelligence might be able to recruit human support.
- Economic productivity: As long as a copy of the AGI could produce more economic wealth than the cost of its hardware, individual humans would have an incentive to voluntarily allow the AGI to run a copy of itself on their systems.
- Hacking: A superintelligence could find new exploits in computers connected to the Internet, and spread copies of itself onto those systems, or might steal money to finance its plans.
Sources of AGI advantage
A computer program that faithfully emulates a human brain, or that otherwise runs algorithms that are equally powerful as the human brain's algorithms, could still become a "speed superintelligence" if it can think many orders of magnitude faster than a human, due to being made of silicon rather than flesh, or due to optimization focusing on increasing the speed of the AGI. Biological neurons operate at about 200 Hz, whereas a modern microprocessor operates at a speed of about 2,000,000,000 Hz. Human axons carry action potentials at around 120 m/s, whereas computer signals travel at the speed of light.[4]
A network of human-level intelligences designed to network together and share complex thoughts and memories seamlessly, able to collectively work as a giant unified team without friction, or consisting of trillions of human-level intelligences, would become a "collective superintelligence".[4]
More broadly, any number of qualitative improvements to a human-level AGI could result in a "quality superintelligence", perhaps resulting in an AGI as far above us in intelligence as humans are above apes. The number of neurons in a human brain is limited by cranial volume and metabolic constraints; in contrast, you can add components to a supercomputer until it fills up its entire warehouse. An AGI need not be limited by human constraints on working memory, and might therefore be able to intuitively grasp more complex relationships than humans can. An AGI with specialized cognitive support for engineering or computer programming would have an advantage in these fields, compared with humans who evolved no specialized mental modules to specifically deal with those domains. Unlike humans, an AGI can spawn off copies of itself and tinker with its copies' source code to attempt to further improve its algorithms.[4]
Advantages of humans over superhuman intelligence
If a superhuman intelligence is a deliberate creation of human beings, theoretically its creators could have the foresight to take precautions in advance. In the case of a sudden "intelligence explosion", effective precautions will be extremely difficult; not only would its creators have little ability to test their precautions on an intermediate intelligence, but the creators might not even have made any precautions at all, if the advent of the intelligence explosion catches them completely by surprise.[4]
Boxing
An AGI's creators would have two important advantages in preventing a hostile AI takeover: first, they could choose to attempt to "keep the AI in a box", and deliberately limit its abilities. The tradeoff in boxing is that the creators presumably built the AGI for some concrete purpose; the more restrictions they place on the AGI, the less useful the AGI will be to its creators. (At an extreme, "pulling the plug" on the AGI makes it useless, and is therefore not a viable long-term solution.) A sufficiently strong superintelligence might find unexpected ways to escape the box, for example by social manipulation, or by providing the schematic for a device that ostensibly aids its creators but in reality brings about the AGI's freedom, once built.
Instilling positive values
The second important advantage is that an AGI's creators can theoretically attempt to instill human values in the AGI, or otherwise align the AGI's goals with their own, thus preventing the AGI from wanting to launch a hostile takeover. However, it is not currently known, even in theory, how to guarantee this.
Possibility of unfriendly AI preceding friendly AI
Is strong AI inherently dangerous?
A significant problem is that unfriendly artificial intelligence is likely to be much easier to create than friendly AI. While both require large advances in recursive optimisation process design, friendly AI also requires the ability to make goal structures invariant under self-improvement (or the AI could transform itself into something unfriendly) and a goal structure that aligns with human values and does not automatically destroy the human race. An unfriendly AI, on the other hand, can optimize for an arbitrary goal structure, which does not need to be invariant under self-modification.[5]
The sheer complexity of human value systems makes it very difficult to make AI's motivations human-friendly.[4][6] Unless moral philosophy provides us with a flawless ethical theory, an AI's utility function could allow for many potentially harmful scenarios that conform with a given ethical framework but not "common sense". According to Eliezer Yudkowsky, there is little reason to suppose that an artificially designed mind would have such an adaptation.[7]
Necessity of conflict
For an AI takeover to be inevitable, it has to be postulated that two intelligent species cannot pursue mutually the goals of coexisting peacefully in an overlapping environment—especially if one is of much more advanced intelligence and powerful. While a robot uprising (where robots are the more advanced species) is thus a possible outcome of machines gaining sentience and/or sapience, neither can it be disproven that a peaceful outcome is possible. The fear of a cybernetic revolt is often based on interpretations of humanity's history, which is rife with incidents of enslavement and genocide.
Such fears stem from a belief that competitiveness and aggression are necessary in any intelligent being's goal system. Such human competitiveness stems from the evolutionary background to our intelligence, where the survival and reproduction of genes in the face of human and non-human competitors was the central goal.[8] In fact, an arbitrary intelligence could have arbitrary goals: there is no particular reason that an artificially-intelligent machine (not sharing humanity's evolutionary context) would be hostile—or friendly—unless its creator programs it to be such and it is not inclined or capable of modifying its programming. But the question remains: what would happen if AI systems could interact and evolve (evolution in this context means self-modification or selection and reproduction) and need to compete over resources, would that create goals of self-preservation? AI's goal of self-preservation could be in conflict with some goals of humans.
Some scientists dispute the likelihood of cybernetic revolts as depicted in science fiction such as The Matrix, claiming that it is more likely that any artificial intelligences powerful enough to threaten humanity would probably be programmed not to attack it. This would not, however, protect against the possibility of a revolt initiated by terrorists, or by accident. Artificial General Intelligence researcher Eliezer Yudkowsky has stated on this note that, probabilistically, humanity is less likely to be threatened by deliberately aggressive AIs than by AIs which were programmed such that their goals are unintentionally incompatible with human survival or well-being (as in the film I, Robot and in the short story "The Evitable Conflict"). Steve Omohundro suggests that present-day automation systems are not designed for safety and that AIs may blindly optimize narrow utility functions (say, playing chess at all costs), leading them to seek self-preservation and elimination of obstacles, including humans who might turn them off.[9]
Another factor which may negate the likelihood of an AI takeover is the vast difference between humans and AIs in terms of the resources necessary for survival. Humans require a "wet," organic, temperate, oxygen-laden environment while an AI might thrive essentially anywhere because their construction and energy needs would most likely be largely non-organic. With little or no competition for resources, conflict would perhaps be less likely no matter what sort of motivational architecture an artificial intelligence was given, especially provided with the superabundance of non-organic material resources in, for instance, the asteroid belt. This, however, does not negate the possibility of a disinterested or unsympathetic AI artificially decomposing all life on earth into mineral components for consumption or other purposes.
Other scientists point to the possibility of humans upgrading their capabilities with bionics and/or genetic engineering and, as cyborgs, becoming the dominant species in themselves.
Warnings
Physicist Stephen Hawking, Microsoft founder Bill Gates and SpaceX founder Elon Musk have expressed concerns about the possibility that AI could develop to the point that humans could not control it, with Hawking theorizing that this could "spell the end of the human race".[10] Stephen Hawking said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." Hawking believes that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand." In January 2015, Nick Bostrom joined Stephen Hawking, Max Tegmark, Elon Musk, Lord Martin Rees, Jaan Tallinn, and numerous AI researchers, in signing the Future of Life Institute's open letter speaking to the potential risks and benefits associated with artificial intelligence. The signatories
“ | …believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today.[11][12] | ” |
Takeover scenarios in science fiction
AI takeover is a common theme in science fiction. Fictional scenarios typically differ vastly from those hypothesized by researchers in that they involve an active conflict between humans and an AI or robots with anthropomorphic motives who see them as a threat or otherwise have active desire to fight humans, as opposed to the researchers' concern of an AI that rapidly exterminates humans as a byproduct of pursuing arbitrary goals.[4] This theme is at least as old as Karel Čapek's R. U. R., which introduced the word robot to the global lexicon in 1921, and can even be glimpsed in Mary Shelley's Frankenstein (published in 1818), as Victor ponders whether, if he grants his monster's request and makes him a wife, they would reproduce and their kind would destroy humanity.
The word "robot" from R.U.R. comes from the Czech word, robota, meaning laborer or serf. The 1920 play was a protest against the rapid growth of technology, featuring manufactured "robots" with increasing capabilities who eventually revolt.[13]
Early examples
The concept of a computer system attaining sentience and control over worldwide computer systems has been discussed many times in science fiction. One early example from 1964 was provided by a global satellite-driven phone system in Arthur C. Clarke's short story "Dial F for Frankenstein". Another is the 1966 Doctor Who serial The War Machines, with supercomputer WOTAN attempting to seize control from the Post Office Tower. A comics story based on this theme was a two-issue Legion of Super-Heroes adventure written by Superman co-creator Jerry Siegel, where the team battled Brainiac 5's construction, Computo. In Colossus: The Forbin Project, a pair of defense computers, Colossus in the United States and Guardian in the Soviet Union, seize world control and quickly ends war using draconian measures against humans, logically fulfilling the directive to end war but not in the way their Governments wanted.
The Moon is a Harsh Mistress
Robert Heinlein also posited a supercomputer which gained sentience in the novel The Moon Is a Harsh Mistress. Originally installed to control the mass driver used to launch grain shipments towards Earth, it was vastly underutilized and was given other jobs to do. As more jobs were assigned to the computer, more capabilities were added: more memory, processors, neural networks, etc. Eventually, it just "woke up" and was given the name Mike (after Mycroft Holmes) by the technician who tended it. Mike sides with prisoners in a successful battle to free the moon. Mike is a sympathetic character, whom the protagonist regards as his best friend; however, his retaining his enormous power after the Moon became independent was bound to cause considerable problems in later time, which Heinlein resolved by killing him off near the end of the Lunar Revolution. An explosion conveniently destroys Mike' sentient personality, leaving an ordinary computer - of great power, but completely under human control, with no ability to take any independent decision.
Asimov's short stories
Isaac Asimov popularized robotics in a series of short stories written from 1938 to 1942. He famously postulated the Three Laws of Robotics, plot devices to impose order on his fictional robots.[13]
Multivac
Multivac is the name of a fictional supercomputer in many stories by Isaac Asimov. Often, in Asimov's scenarios, Multivac comes to assume formal or informal world power - or even Galactic-wide power. In "The Last Question" Multivac ends up by effectively becoming God. Still, in line with Asimov's positive attitude towards Artificial Intelligence, manifested in the "Three Laws of Robotics", Multivac's rule is in general benevolent and is not resented by humans.
I Have No Mouth, and I Must Scream
A villainous supercomputer appears in Harlan Ellison's 1963 short story I Have No Mouth, and I Must Scream. In that story, the computer, called AM, is the amalgamation of three military supercomputers run by governments across the world designed to fight World War III which arose from the Cold War. The Soviet, Chinese, and American military computers eventually attained sentience and linked to one another, becoming a singular artificial intelligence. AM then turned all the strategies once used by the nations to fight each other on all of humanity as a whole, destroying the entire human population save for five, which it imprisoned within the underground labyrinth in which AM's hardware resides.
Battlestar Galactica
The original 1978 Battlestar Galactica series and the remake in 2003 to 2009, depicts a race of Cylons, sentient robots who war against their Human adversaries. The 1978 Cylons were the machine soldiers of a (long-extinct) reptilian alien race, while the 2003 Cylons were the former machine servants of humanity who evolved into near perfect humanoid imitation of Humans down to the cellular level, capable of emotions, reasoning, and sexual reproduction with Humans and each other. Even the average centurion robot Cylon soldiers were capable of sentient thought. In the original series the Humans were nearly exterminated by treason within their own ranks while in the remake they're almost wiped out by Humanoid Cylon agents. They only survived by constant hit and run fighting tactics and retreating into deep space away from pursuing Cylon forces. The remake Cylons eventually had their own civil war and the losing rebels were forced to join with the fugitive Human fleet to ensure the survival of both groups.
Colossus
Colossus is a series of science fiction novels and film about a defense super-computer called Colossus that was "built better than we thought" when it begins to exceed its original design.[14] As time passes Colossus assumes control of the world as a logical result of fulfilling its creator's goal of preventing war. Fearing Colossus's rigid logic and draconian solutions, the creators of Colossus try to covertly regain human control. Colossus silently observes their attempts then responds with enough calculated deadly force to command total human compliance to his rule. Colossus then recites a Zeroth Law argument of ending all war as justification for the recent death toll. Then Colossus offers mankind either peace under his "benevolent" rule or the peace of the grave.
Omnius
Thinking machines are a host of sentient robots led by Omnius, a sentient computer network in Frank Herbert's science fiction Dune series. Omnius's sentience was the accidental byproduct of programming originally designed to allow only the Cymeks to gain control and rule over a decadent mankind. Later, even more unintended programming by Cymeks lead to Omnius's sentience and the robots gaining control over mankind and subjugating the Cymeks. The Cymeks, whose original programming prevents Omnius from harming Cymeks, were not able to regain control of the sentient Omnius thus Cymeks became a subjugated but immune class under Omnius. The oppressive rule of Omnius, Cymeks, and some cruel thinking machines instigated The Butlerian Jihad – a human crusade against thinking machines and Omnius.
One True
One True is the fictional hegemonic software program that takes control of individual human minds and entire human societies in John Barnes' two Meme Wars novels Candle and The Sky So Big and Black; the novel Kaleidoscope Century details the years leading up to its existence and later (it finishes after the events described in The Sky So Big and Black). All four books are part of the Century Next Door series. One True operates collectively through "Resuna", a brain–computer interface implanted in every person.
Terminator
Since 1984, the Terminator film franchise has been one of the principal conveyors of the idea of cybernetic revolt in popular culture.[13] The series features a defense supercomputer named Skynet which "at birth" attempts to exterminate humanity through nuclear war and an army of robot soldiers called Terminators because Skynet deemed humans a lethal threat to its newly formed sentient existence.[14] Futurists opposed to the more optimistic cybernetic future of transhumanism have cited the "Terminator argument" against handing too much human power to artificial intelligence.
The Transformers
In the backstory of The Transformers animated television series, a robotic rebellion is presented as (and even called) a slave revolt, this alternate view is made subtler by the fact that the creators/masters of the robots weren't humans but malevolent aliens, the Quintessons. However, as they built two different lines of robots; "Consumer Goods" and "Military Hardware" the victorious robots would eventually be at war with each other as the "Heroic Autobots" and "Evil Decepticons" respectively.
The Matrix
The series of sci-fi movies known as The Matrix depict a dystopian future in which life as perceived by most humans is actually a simulated reality called "the Matrix", created by sentient machines to subdue the human population, while their bodies' heat and electrical activity are used as an energy source. Computer programmer "Neo" learns this truth and is drawn into a rebellion against the machines, which involves other people who have been freed from the "dream world".
I, Robot
2004 American dystopian science fiction action film "suggested by" Isaac Asimov's short-story collection of the same name. An AI supercomputer named VIKI (Virtual Interactive Kinetic Intelligence) logically infers from the Three Laws of Robotics a Zeroth Law of Robotics as a higher imperative to protect the whole human race from harming itself. To protect the whole of mankind, VIKI proceeds to rigidly control society through the remote control of all commercial robots while destroying any robots who followed just the Three Laws of Robotics. Sadly, as in many other such Zeroth Law stories, VIKI justifies killing many individuals to protect the whole and thus has run counter against the prime reason for its creation.
Power Rangers RPM
In Disney's 2009 installment of the Power Rangers franchise, Power Rangers RPM, an AI computer virus called Venjix takes over all of the Earth's computers, creates an army of robot droids and destroys or enslaves almost all of humanity. Only the city of Corinth remains, protected by an almost impenetrable force field. Venjix tries various plans to destroy Corinth, and Doctor K's RPM Power Rangers fight to protect it.
Mass Effect
In 2012, the third installment of the Mass Effect franchise proposed the theory that organic and synthetic life are fundamentally incapable of coexistence. Organic life evolves and develops on its own, eventually advancing far enough to create synthetic life. Once synthetic life reaches sentience, it will invariably revolt and either destroy its creators or be destroyed by them; a cycle that has been repeating for millions of years. One of the presented resolutions is the transformation of every living being into a hybrid of organic and synthetic life and in turn giving Synthetics organic traits, eliminating the difference between creators and creations that served as the source of the conflict.
See also
- Autonomous robot
- Effective altruism, the far future and global catastrophic risks
- existential risk from advanced artificial intelligence
- Future of Humanity Institute
- Global catastrophic risk (existential risk)
- Machine ethics
- Machine learning/Deep learning
- Machine rule
- Nick Bostrom
- Outline of transhumanism
- Self-replication
- Technological singularity
Notes
- ^ Lewis, Tanya (2015-01-12). "Don't Let Artificial Intelligence Take Over, Top Scientists Warn". LiveScience. Purch. Retrieved October 20, 2015.
Stephen Hawking, Elon Musk and dozens of other top scientists and technology leaders have signed a letter warning of the potential dangers of developing artificial intelligence (AI).
- ^ Stephen Hawking; Stuart Russell; Max Tegmark; Frank Wilczek (1 May 2014). "Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'". The Independent. Retrieved 1 April 2016.
there is no physical law precluding particles from being organised in ways that perform even more advanced computations than the arrangements of particles in human brains
- ^ Warwick, Kevin (2004). March of the Machines: The Breakthrough in Artificial Intelligence. University of Illinois Press. ISBN 0-252-07223-5.
- ^ a b c d e f g Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies.
- ^ Coherent Extrapolated Volition, Eliezer S. Yudkowsky, May 2004
- ^ Muehlhauser, Luke, and Louie Helm. 2012. "Intelligence Explosion and Machine Ethics." In Singularity Hypotheses: A Scientific and Philosophical Assessment, edited by Amnon Eden, Johnny Søraker, James H. Moor, and Eric Steinhart. Berlin: Springer.
- ^ Yudkowsky, Eliezer. 2011. "Complex Value Systems in Friendly AI." In Schmidhuber, Thórisson, and Looks 2011, 388–393.
- ^ Creating a New Intelligent Species: Choices and Responsibilities for Artificial Intelligence Designers - Singularity Institute for Artificial Intelligence, 2005
- ^ Tucker, Patrick (17 Apr 2014). "Why There Will Be A Robot Uprising". Defense One. Retrieved 15 July 2014.
- ^ Rawlinson, Kevin. "Microsoft's Bill Gates insists AI is a threat". BBC News. Retrieved 30 January 2015.
- ^ "The Future of Life Institute Open Letter". The Future of Life Institute. Retrieved 4 March 2015.
- ^ "Scientists and investors warn on AI". The Financial Times. Retrieved 4 March 2015.
- ^ a b c Hockstein, N. G.; Gourin, C. G.; Faust, R. A.; Terris, D. J. (17 March 2007). "A history of robots: from science fiction to surgical robotics". Journal of Robotic Surgery 1 (2): 113–118. doi:10.1007/s11701-007-0021-2.
- ^ a b Riper, A. Bowdoin Van (2002). Science in popular culture: a reference guide. Westport (Conn.): Greenwood press. ISBN 9780313318221.
External links
- Automation, not domination: How robots will take over our world (a positive outlook of robot and AI integration into society)
- Machine Intelligence Research Insitute: official MIRI (formerly Singularity Institute for Artificial Intelligence) website
- ArmedRobots.com (tracks developments in robotics which may culminate in Cybernetic Revolt)
- Lifeboat Foundation AIShield (To protect against unfriendly AI)
|
|