7.2 A Simple Formal Agents Model (SRAM) 131 7.2.1 Goals We define goals as mathematical functions (to be specified below) associated with symbols drawn from the alphabet G; and we consider the environment as sending goal-symbols to the agent along with regular observation-symbols. (Note however that the presentation of a goal- symbol to an agent does not necessarily entail the explicit communication to the agent of the contents of the goal function. This must be provided by other, correlated observations.) We also introduce a conditional distribution y(g, 4) that gives the weight of a goal g in the context of a particular environment ju. In this extended framework, an interaction sequence looks like @101 917142029292... or else a1Y142Yy2-.- where g; are symbols corresponding to goals, and y is introduced as a single symbol to denote the combination of an observation, a reward and a goal. Each goal function maps each finite interaction sequence Jg,.2 = ays: with gs to gy corre- sponding to g, into a value rg(Ig,s,4) € [0,1] indicating the value or “raw reward” of achieving the goal during that interaction sequence. The total reward 7; obtained by the agent is the sum of the raw rewards obtained at time ¢ from all goals whose symbols occur in the agent’s history before t. This formalism of goal-seeking agents allows us to formalize the notion of intelligence as “achieving complex goals in complex environments” — a direction that is pursued in Section 7.3 below. Note that this is an external perspective of system goals, which is natural from the perspective of formally defining system intelligence in terms of system behavior, but is not necessarily very natural in terms of system design. From the point of view of AGI design, one is generally more concerned with the (implicit or explicit) representation of goals inside an AGI system, as in CogPrime’s Goal Atoms to be reviewed in Chapter 22 below. Further, it is important to also consider the case where an AGI sys