look for a chance to “improve” the way we live, to bend us like so many paperclips into what it seeks? The leap from deciding liver allocations to shutting down liquor plants might seem pretty short to a rationalizing machine. And if such a machine could really “think”, Vinge bet it would pretty quickly conclude that the restraints of its creators were limiting what it had been asked to do. At which point the AI would turn to thinking about how to escape those bounds. It would be like Deep Blue programmed to plan its own prison break. And as much as humans might try to stifle a smart machine, we’d be fighting to contain something more powerful than we'd ever encountered. This challenge, which sounds like something out of science fiction, is known by technologists by a name that does sound like a short story by Isaac Asimov: “The Confinement Problem”. The computer scientist Butler Lampson named this in 1973 as a sort of task for computer security experts - possibly their last. The assignment: Not simply to keep malware out of a system, but to keep the mind of a malicious machine inside. To gate it. Today computer science labs are filled with nervous, apocalyptic research imagining the impossible troubles of confinement. The debate divides those who think smart technology can be contained - “Boxers,” they are called - and those like Vinge who think the AI will always, eventually escape. “Imagine yourself confined to your house with only limited data access to the outside, to your master.” he wrote, putting the reader in the place of an AI machine. “If those masters thought ata rate -- say - one million times slower than you, there is little doubt that over a period of years (your time) you could come up with ‘helpful advice’ that would incidentally set you free.” Imagine you are in charge of containing that health-optimizing AI. What if it told you it had the power to cure all illness and hunger, to ameliorate the misery of the world, if only it could be permitted to