This need to understand human actions and decisions applies to physical and nonphysical robots alike. If either sort bases its decision about how to act on the assumption that a human will do one thing but the human does something else, the resulting mismatch could be catastrophic. For cars, it can mean collisions. For an AI with, say, a financial or economic role, the mismatch between what it expects us to do and what we actually do could have even worse consequences. One alternative is for the robot not to predict human actions but instead just protect against the worst-case human action. Often when robots do that, though, they stop being all that useful. With cars, this results in being stuck, because it makes every move too risky. All this puts us, the AI community, into a bind. It suggests that robots will need accurate (or at least reasonable) predictive models of whatever people might decide to do. Our state definition can’t just include the physical position of humans in the world. Instead, we’ ll also need to estimate something internal to people. We’ll need to design robots that account for this human internal state, and that’s a tall order. Luckily, people tend to give robots hints as to what their internal state is: Their ongoing actions give the robot observations (in the Bayesian inference sense) about their intentions. If we start walking toward the right side of the hallway, we’re probably going to enter the next room on the right. What makes the problem more complicated is the fact that people don’t make decisions in isolation. It would be one thing if robots could predict the actions a person intends to take and simply figure out what to doin response. But unfortunately this can lead to ultra-defensive robots that confuse the heck out of people. (Think of human drivers stuck at four-way stops, for instance.) What the intent-prediction approach misses is that the moment the robot acts, that influences what actions the human starts taking. There