Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes 6) DEEP SOCIETAL INFLUENCES Al, Agency, and Disempowerment (Contributions from Gireeja Ranade, Andrew Maynard, David McAllester, Stuart Russell and others) We will be benefitting from Al system that are competent at doing important tasks. People and organizations seek Al systems that bring new abilities to the table. We desire autonomous cars that drive without collisions, we medical assistants that can diagnose patients accurately and we would like to have household assistants that can infer our intentions and execute them flawlessly —and even proactively. The military wants Al systems that can help with strategy and tactics, and systems that outmaneuver human led troops, and anticipate and respond to threats either on timescales that humans cannot achieve, or over landscapes humans cannot cover. Today, there is still skepticism about performance of Al systems in a variety of domains. However, we expect that Al systems will become more central decision support, pattern recognition, autonomous decision making, and other types of problem solving. As such, we will become increasingly reliant on Al systems. This raises concerns in several areas, including personal decision support, healthcare, transportation, governance and the handling and operation of weapon systems. We shall consider example of healthcare from Gireeja Ranade. The scenario and trajectory applies to other areas as we consider the increasing role and power of Al in our lives and in society: As healthcare providers are increasingly stretched in providing consultations with patients, diagnosing conditions, and developing treatment and/or intervention plans, tech companies identify a market opportunity for Al-based digital assistants that are designed to augment healthcare providers by collecting data from consultations, c