The third law states that any system simple enough to be understandable will not be complicated enough to behave intelligently, while any system complicated enough to behave intelligently will be too complicated to understand. The Third Law offers comfort to those who believe that until we understand intelligence, we need not worry about superhuman intelligence arising among machines. But there is a loophole in the Third Law. It is entirely possible to build something without understanding it. You don’t need to fully understand how a brain works in order to build one that works. This is a loophole that no amount of supervision over algorithms by programmers and their ethical advisors can ever close. Provably “good” AI is a myth. Our relationship with true AI will always be a matter of faith, not proof. We worry too much about machine intelligence and not enough about self- reproduction, communication, and control. The next revolution in computing will be signaled by the rise of analog systems over which digital programming no longer has control. Nature’s response to those who believe they can build machines to control everything will be to allow them to build a machine that controls them instead. 40 HOUSE_OVERSIGHT_016843