Despite well defined end-goals, subgoals such as getting more energy can produce unwanted results in the development of AI

Date created: 2023-07-11

The best way to reach many different goals is to follow these subgoals, and they may themselves be negative for humans.

“Well, here’s a Subgoal that almost always helps in biology: get more energy. So the first thing that could happen is these robots are going to say, ‘Let’s get more power. Let’s reroute all the electricity to my chips.’ Another great subgoal would be to make more copies of yourself. Does that sound good?” (View Highlight)