KEsu Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes true today to some extent, but the above advances in ML/Al could make the situation substantially worse, in extreme cases perhaps even rendering useful mass communication on the internet untenable. The rising capabilities can be used in multiple ways in multiple settings with multiple goals. Some uses may be subtle and employed over time to do important but damaging biasing of sentiment about individuals and groups of people. The capabilities can be combined to enable identity theft or identify distortion for destroying the reputation of people and groups. As such, these abilities could enable small groups to wield great power in multiple arenas and for new forms of blackmail, threats, and control. SUMMARY Powerful personalized persuasion technologies are positioned to put massive power in the hands of a few and may even manipulate the owners of the technology. Powerful propaganda and persuasion machines threatens to undermine democracy, free availability of information about the state of the world, and, more generally, freedom of thought. Leaders may increasingly depend upon such propaganda optimization systems for attaining and holding power. Over time, even the potential initial owners of such systems might become unaware or unable to control these systems—and may believe the propaganda themselves. In the longer-term, there is the possibility that one or multiple systems, or distributed coalitions of systems communicating implicitly or explicitly could autonomously persuade, subjugate, and control populations. Pathways to such situations include the side effects of rise in the large-scale use by people of communicating personalized filters that interpret and pool information with the initial intention of grappling with widespread uses of manipulative information. SAMPLE TRAJE