systematically thought through everything that could possibly go wrong when putting astronauts on top of a 110-meter rocket full of highly flammable fuel and launching them to a place where nobody could help them—and there were lots of things that could go wrong. Was this scaremongering? No, this was the safety engineering that ensured the mission’s success. Similarly, we should analyze what could go wrong with AI to ensure that it goes right. Outlook In summary, if our technology outpaces the wisdom with which we manage it, it can lead to our extinction. It’s already caused the extinction of from 20 to 50 percent of all species on Earth, by some estimates,”° and it would be ironic if we’re next in line. It would also be pathetic, given that the opportunities offered by AGI are literally astronomical, potentially enabling life to flourish for billions of years not only on Earth but also throughout much of our cosmos. Instead of squandering this opportunity through unscientific risk denial and poor planning, let’s be ambitious! Homo sapiens is inspiringly ambitious, as reflected in William Ernest Henley’s famous lines from /nvictus: “I am the master of my fate, / I am the captain of my soul.” Rather than drifting like a rudderless ship toward our own obsolescence, let’s take on and overcome the technical and societal challenges standing between us and a good high-tech future. What about the existential challenges related to morality, goals, and meaning? There’s no meaning encoded in the laws of physics, so instead of passively waiting for our Universe to give meaning to us, let’s acknowledge and celebrate that it’s we conscious beings who give meaning to our Universe. Let’s create our own meaning, based on something more profound than having jobs. AGI can enable us to finally become the masters of our own destiny. Let’s make that destiny a truly inspiring one! °° See Elizabeth Kolbert, The Sixth Extinction: An Unnatural History (New York: Henry Holt, 2014). 68 HO