What happens when our computers get smarter than we are?


Example citation

Key takeaways

  • Because the differences between us and apes are due to small genetic alterations over time, yet have had enormous consequences for the way our two species live and thrive, it is possible to think that similarly small changes from now on can have a profound impact.
  • Once we arrive at machine superintelligence it’s likely that machine intelligence will take off rapidly.
  • Once there is superintelligence, the fate of humanity is decided by the superintelligence
  • We are at the mercy of this superintelligence, what is its preferences? Reminds me of the paper-clip essay on waitbutwhy. You better make sure that the algorithm optimizes for everything we care about.

Many AI-researchers think that we are fairly close to machine superintelligence

The train doesn’t stop at human-level intelligence