Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes 4) Al, CYBERSECURITY, AND Al ATTACK SURFACES Al Attacks on Computing Systems, Devices, Infrastructure (focus) Manipulation & Disruption of Al Systems (Contributions by Kathleen Fisher, John Launchbury, Ashish Kapoor, Sean, Shahar, Jeff Coleman and others) Al will be used in new ways to enhance cyberwarfare. Targets could be either purely computational, aimed at the bringing down of computing systems, the stealing of stored information, of gaining access to monitoring activity and information streams. However, we are more likely to see potentially even more costly attacks involving a combination of cyber and physical systems, e.g., uranium enrichment plants, automated flight systems, weapon systems, automated driving systems, healthcare equipment, oil refineries, or the large swaths of the power grid of the US or other countries. Cyberwarfare is a domain in which the use of Al is inevitable. Attacks and/or responses are likely to happen at computing rather than human speeds. As soon as one side has autonomous cyber warriors systems (ACWs), other actors will have to adapt similar offensive or new defensive technologies. Given this context, imagine building an ACW designed to seek, disrupt, and destroy within high-value adversary networks and systems. The ACW has to be able to observe network behavior to build situational awareness, find places to hide, create exploits to pivot to new places, build a map and use it to navigate complex networks, find high-value information, and identify targets to disable or from which to extract information. Because high-value adversary networks are likely to be relatively isolated, the ACW will have very limited opportunities for external command and control communication, so it will need to make many decisions in isolation. It will read information it