12.9 AGI Ethics As Related to Various Future Scenarios 237 pragmatic underlying ethical pedagogy and experiential grounding may prove to be even more dangerous than our messy human mental ecologies. If one assumes a hard takeoff AGI, then all this must be codified in the system at launch, as once a potentially Singularitarian AGI is launched there is no way to know what time period constitutes “before the singularity point.” This means developing theory of mind empathy and logical ethics in code prior to giving the system unfettered access to hardware and self- modification code. However, though nobody can predict if or when a Singularity will occur after unrestricted launch, only a truly irresponsible AGI development team would attempt to create an AGI without first experimenting with ethical training of the system in an intelligence- capped form, by means of ethical instruction via human-AGI interaction both pedagogically and experientially. 12.9.4 Global Brain Mindplex Scenarios Another class of scenarios — overlapping some of the previous ones — involves the emergence of a “Global Brain,” an emergent intelligence formed from global communication networks in- corporating humans and software programs in a larger body of self-organizing dynamics. The notion of the Global Brain is reviewed in [Hey07, Tur77] and its connection with advanced AI is discussed in detail in Goertzel’s book Creating Internet Intelligence [Goe01], where three possible phases of “Global Brain” development are articulated: e Phase 1: computer and communication technologies as enhancers of human interactions. This is what we have today: science and culture progress in ways that would not be possible if not for the “digital nervous system” we’re spreading across the planet. The network of idea and feeling sharing can become much richer and more productive than it is today, just through incremental development, without any Metasystem transition. e Phase 2: the intelligent Internet. At this