Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes overall software security remains vulnerable: “vulnerabilities are dense” in production code, incentives for securing loT systems are low, key vulnerabilities are stockpiled rather than globally patched. Using machine learning the techniques for vulnerability detection are increasingly sophisticated but opaque. At some point adaptive cyber defense/offense systems become scalable so they can take over vulnerable systems. More aggressive actors combine these systems with botnet functionality and retaliatory responses (e.g. counter-hacking or DDoS attacks) to protect themselves. Since vulnerability discovery is scalable, as they spread and acquire more resources they become more effective. At this point an external cause (e.g. cyberattacks due to an international conflict) or just chance cause aggressive systems to begin large-scale cyberwarfare. This triggers other systems to join in. Some attacks disrupt command-and-control links, producing self-replicating independent systems. All together this leads to a massive degradation of the functionality of the Internet and modern society. Defeating the evolving cyberwarfare systems is hard without taking essential parts of society offline for an extended time - made doubly difficult due to the international stresses unleashed by the outbreak, which in some cases spill over into real-world conflicts and economic crashes. But without a decisive way of cleaning systems the problem will be persistent until entirely new secure infrastructure can be built at a great cost. HUMAN DIMENSION OF CYBERSECURITY: Al FOR SOCIAL ENGINEERING Beyond direct effects on computing systems, rising concerns include the use of Al methods for social engineering to gain access to system authentication information. For example, recent work demonstrated the use of an it