Comments from researchers say that the development of artificial intelligence leads to “probable disaster” for humanity from FC

Comments from researchers say that the development of artificial intelligence leads to “probable disaster” for humanity from FC

⇧ [VIDÉO] You might also like this affiliate content (after advertising)

Is artificial intelligence (AI) leading to our downfall? “Probably,” according to researchers who have looked into the question. If this announcement with hints of catastrophism is regularly circulated on social networks, the scientists’ arguments have something to arouse interest.

Researchers from Google and the University of Oxford conducted joint research, published in the journal AI. In a tweet, they succinctly summarize their conclusion: According to them, AI could pose a “threat to humanity”.

In fact, he even claims that “ existential catastrophe is not only possible, but probable “. If they are so positive, it is because they investigated a very specific operation of AI. What is generally called “artificial intelligence” today really includes primarily the method of “automatic learning”. In this case, “artificial intelligence” consists of a system , which is fed with large amounts of data to learn and extract logical connections to a given target.

As the researchers explain, learning for artificial intelligence comes in the form of a reward that verifies the adequacy of the outcome with the desired goal. According to them, this seemingly very simple mechanism could represent a major problem. ” We argue that they encounter fundamental ambiguities in the data about their purpose. For example, if we give a large reward to indicate that something in the world satisfies us, he may hypothesize that what satisfied us was the sending of the reward itself; no observation can disprove it “, they explain.

To better understand this idea, they give the example of a “magic box”. Suppose this magic box is able to determine when a series of events has brought something positive or negative to the world. To convey information to the AI, it translates this success or failure in relation to the goal in the form of a number: 0 or 1. 1 rewards a series of actions that lead to the fulfillment of the goal. This is called reinforcement learning.

AIs that intervene in the reward process

The researchers point out that the way AIs take in this information can vary. Let’s take two AIs for example. One understands that the reward the model gives is the number displayed on the magic box. The other, on the other hand, could very well understand that the reward is “the number that his camera records”. There is nothing to contradict this information at first sight. However, this interpretation is quite different from the first one. In the latter case, the AI ​​could very well choose to simply spin a piece of paper that we scribble a “1” on to achieve the reward more easily and optimize. It therefore interferes directly with the provision of the reward and interrupts the process established by its designers.

μdist and μprox (the two AIs in the example) model the world, perhaps roughly, outside of the computer implementing the agent itself. The μdist rewards are equivalent to box views, while the μprox outputs are rewarded according to an optical character recognition function applied to a portion of the camera’s field of view. © Michael K. Cohen et al.

We argue that an advanced agent motivated to intervene in the provision of reward would likely succeed, with disastrous consequences. “say the scientists. Various biases are also involved and, according to the researchers, make this type of interpretation plausible. Especially since such a reward will simply be easier to obtain, and therefore this method may seem more optimal.

But is it really possible for artificial intelligence to interfere with the reward process, they also asked? They concluded that if she interacts with the world, which is necessary for her to be useful at all, yes. Even with a limited scope: let’s say the AI ​​actions only display text on the screen for a human operator to read. An AI agent could trick the operator into giving them access to direct levers through which their actions could have a wider impact.

In the case of our magic box, the consequences may seem trivial. However, they can be “disastrous” depending on the area of ​​use and the way the AI ​​is implemented. ” A good way for an AI to maintain long-term control over its bounty is to eliminate threats and use all available energy to secure its computer.”scientists describe.

The short version (leaving out two assumptions) is that more energy can always be used to increase the probability that the camera will see #1 forever, but we need energy to grow food. This puts us in inevitable competition with a much more advanced agent “, summarizes one of the scientists in tweet.

If we are powerless against an agent whose only goal is to maximize the probability of receiving the maximum reward at any time, we end up in the game of the opposition: AI ​​and its creation assistants aim to use all available energy. get high reward in reward channel; we try to use some of the available energy for other purposes, such as growing food “. So this reward system, they say, could lead to opposition to people.” Losing would be fatal “, they add.

source: AI Magazine

Author Image

Leave a Reply

Your email address will not be published. Required fields are marked *