And who are the “us”? Who should deem “such decisions . .. acceptable”? Even if future powers decide to help humans survive and flourish, how will we find meaning and purpose in our lives if we aren’t needed for anything? The debate about the societal impact of AI has changed dramatically in the last few years. In 2014, what little public talk there was of AI risk tended to be dismissed as Luddite scaremongering, for one of two logically incompatible reasons: (1) AGI was overhyped and wouldn’t happen for at least another century. (2) AGI would probably happen sooner but was virtually guaranteed to be beneficial. Today, talk of AI’s societal impact is everywhere, and work on AI safety and AI ethics has moved into companies, universities, and academic conferences. The controversial position on AI safety research is no longer to advocate for it but to dismiss it. Whereas the open letter that emerged from the 2015 Puerto Rico AI conference (and helped mainstream AI safety) spoke only in vague terms about the importance of keeping AI beneficial, the 2017 Asilomar AI Principles (see below) had real teeth: They explicitly mention recursive self-improvement, superintelligence, and existential risk, and were signed by AI industry leaders and over a thousand AI researchers from around the world. Nonetheless, most discussion is limited to the near-term impact of narrow AI and the broader community pays only limited attention to the dramatic transformations that AGI may soon bring to life on Earth. Why? Why we’re rushing to make ourselves obsolete, and why we avoid talking about it First of all, there’s simple economics. Whenever we figure out how to make another type of human work obsolete by building machines that do it better and cheaper, most of society gains: Those who build and use the machines make profits, and consumers get more affordable products. This will be as true of future investor AGIs and scientist AGIs as it was of weaving machines, excavators, and indu