Asu Origins C ARIZONA STATE UNIVERSITY February 24 — 26, 2017 An Origins Project Scientific Workshop Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes ARTIFICIAL INTELLIGENCE ADVERSE OUTCOMES & TEAMS 1) FINANCIAL MARKETS A!, Trading Systems, and Market Manipulation Chair: Michael Wellman Red Team: Miles Brundage, Randy Bryant, Gary Marchant, Jaan Tallinn Blue Team: Michael Littman, Greg Cooper, Yan Shoshitaishvili, Frank Wilczek 2) DEMOCRACY, INFORMATION, AND IDENTITY Al, Information, and Democracy Chair: Shahar Avin Red Team: Miles Brundage, Sean 0 hEigeartaigh, Andrew Maynard, Eric Horvitz Blue Team: Gary Marchant, Gireeja Ranade, Michael Littman, Subbarao Kambhampati, Jeremy Gillula 3) WAR & PEACE A!, Military Systems, and Stability Chair: Bart Selman Red Team: Richard Mallah, Eric Horvitz, Michael Wellman, Frank Wilczek Blue Team: Vinh Nguyen, Kathleen Fisher, Lawrence Krauss, John Launchbury, Rachel Bronson 4) AI, CYBERSECURITY, AND Al ATTACK SURFACES Al Attacks on Computing Systems, Devices, Infrastructure (focus) Manipulation & Disruption of Al Systems Chair: Kathleen Fisher Red Team: Jeffrey Coleman, John Launchbury, Vinh Nguyen, Mauno Pihelgas Blue Team: Ashish Kapoor, Randy Bryant, Yan Shoshitaishvili, Ben Zorn 5) Al, GOALS, AND INADVERTENT SIDE EFFECTS Runaway Resource Monopoly (focus) Self-Improvement, Shift of Objectives Chair: Sean 6 hEigeartaigh Red Team: Jaan Tallinn, Nate Soares, Jeff Coleman, Bart Selman Blue Team: Dario Amodei, Greg Cooper, Shahar Avin, Ben Zorn 6) DEEP LONG-TERM SOCIETAL INFLUENCES A!, Agency, and Disempowerment Chair: Gireeja Ranade Red Team: Richard Mallah, Andrew Maynard, Nate Soares, Mauno Pihelgas, Jeremy Gillula Blue Team: Subbarao Kambhampati, Lawrence Krauss, Dario Amodei, Frank Wilczek 1 EFTA00287327