KEsu Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes work to degrade or destroy specific functionality (e.g., rewriting network routine tables, replacing plans, changing target information). Once the technology for ACWs exists for military targets, it seems likely there will be cross over into civilian use. Such technology could be deployed against law enforcement targets to disrupt criminal investigations, against banks to steal financial assets, or against companies to steal intellectual property. As they spread into these more general targets, the effects of ACWs might become less predictable. If an ACW incorrectly assesses the situation, it might end up taking down a flight control center or a stock exchange, for example. SOURCES The initial development of ACWs will likely be done by nation states with good intentions, i.e., securing the national interests. (Although what is in one country’s national interest may well not be in the national interest of other countries). The shared existence of such technology might serve as a deterrent against their use by anyone in much the same way that nuclear weapons have served as a deterrent, although ACWs would likely have to be used to devastating effect first to establish their efficacy and threat. However, once the technology exists, it would be very difficult to keep it out of the hands of people with malicious intent (criminals, terrorists, and rogue nation states). It is also the case that the technology has the potential to cause significant collateral damage even if its use was originally well intentioned because it can be difficult to distinguish civilian from military targets in cyberspace. PERSISTENCE Characteristics engineered into the ACW are likely to make it persistent and hard to find as it is designed to infiltrate adversary systems and hide from detection. Once release