grids that flipped nuclear plants on or off to a logic only they understood. Today. The more profound version, however, would be the arrival of AI that really did think and create and intuit tremors too subtle for the human mind. Tomorrow. Like so much of our connected age, such machines would arrive, Vinge felt, because we want and even need them to achieve our dreams. Then, he supposed, they would take over. The leap from evoking Mozart to enacting Stalin would not be so much ofa leap anyhow, at least technologically. /t’s just bits. Goode’s definition could have been screwed into something still tighter: “Let an ultraintelligent machine be defined as the box that will eliminate us.” The day after tomorrow. What spun uneasily from that silly NASA poem, “Our robots precede us....” is a fear: Real Al is fish bait. We'll snap at it hungrily, hoping it will satisfy some human ache only to discover we've been hooked, soon to be devoured. The idea that a superintelligent device would always be docile enough to tip us off to its secret switches of control or to reveal its looming accidents in a way our simple minds can understand, seems unlikely. To be honest, we might have a hard time even understanding the off switches, let alone reaching them. So many of our incentives are to let an effective Al finger more and more of our lives. To teach and encourage it, in some settings, extremely undocile: A weapon to attack our enemies, our political opponents or, finally, each other. It was easy enough for Vinge to see how this would end. It wouldn't be with the sort of intended polite, lap-dog domesticity of artificial intelligence we might hope for, but with a rotweiler of a device, alive to the meaty smell of power, violence and greed. The Oxford philosopher Nick Bostrom has described the following thought experiment: Imagine a super-intelligent machine, programmed to do whatever is needed make paperclips as fast as possible and connected to every resource that task might dem