70 4 Brief Survey of Cognitive Architectures application of supervised learning algorithms such as recurrent neural networks. These cluster- ing and prediction processes occur separately in each node, but the nodes are linked together via bidirectional dynamics: each node feeds input to its parents, and receives "advice" from its parents that is used to condition its probability calculations in a contextual way. These processes are executed formally by the following basic belief update rule, which governs the learning process and is identical for every node in the architecture. The belief state is a probability mass function over the sequences of stimuli that the nodes learns to represent. Consequently, each node is allocated a predefined number of state variables each denoting a dynamic pattern, or sequence, that is autonomously learned. The DeSTIN update rule maps the current observation (0), belief state (b), and the belief state of a higher-layer node or context (c), to a new (updated) belief state (8’), such that Pr(s’M on bNe) pb’ f — P . f b S._ Al () = Pr(s[o,bye) = SEO. (4.1) alternatively expressed as bf (a!) = Ptlels’, 6 ¢) Pr(s']b,¢) Pr(be)_ (4.2) Pr (06, c) Pr (6, ce) Under the assumption that observations depend only on the true state, or Pr(o|s’, b,c) = Pr(ols’), we can further simplify the expression such that b! (s!) = Pr(o|s’) Pr (s’|6, ce) . (4.3) Pr (o0|6, c) where Pr (s‘|b,c) = > Pr(s‘|s,c)b(s), yielding the belief update rule ses Pr (o|s’) S> Pr (s'|s, c) 6(s) b! (s') = s€S ee 4,4 (8) = $= Pr lols”) 2 Pr Ws, bs)” ie slES ses where S denotes the sequence set (i.e. belief dimension) such that the denominator term is a normalization factor. One interpretation of eq. (4.4) would be that the static pattern similarity metric, Pr (ols’), is modulated by a construct that reflects the system dynamics, Pr (s’|s,c). As such, the belief state inherently captures both spatial and temporal information. In our implementation, the belief state of the parent