256 13 Local, Global and Glocal Knowledge Representation 13.4 Knowledge Representation via Attractor Neural Networks Now we turn to global, implicit knowledge representation — beginning with formal neural net models, briefly discussing the brain, and then turning back to CogPrime. Firstly, this section reviews some relevant material from the literature regarding the representation of knowledge using attractor neural nets. It is a mix of well-established fact with more speculative material. 13.4.1 The Hopfield neural net model Hopfield networks [Hop82] are attractor neural networks often used as associative memories. A Hopfield network with N neurons can be trained to store a set of bipolar patterns P, where each pattern p has N bipolar (+1) values. A Hopfield net typically has symmetric weights with no selfconnections. The weight of the connection between neurons ¢ and j is denoted by wi;. In order to apply a Hopfield network to a given input pattern p, its activation state is set to the input pattern, and neurons are updated asynchronously, in random order, until the network converges to the closest fixed point. An often-used activation function for a neuron is: yi = sign(pi So wigs) JFi Training a Hopfield network, therefore, involves finding a set of weights w,; that stores the training patterns as attractors of its network dynamics, allowing future recall of these patterns from possibly noisy inputs. Originally, Hopfield used a Hebbian rule to determine weights: P Wig = S- piv; p=l1 Typically, Hopfield networks are fully connected. Experimental evidence, however, suggests that the majority of the connections can be removed without significantly impacting the net- work’s capacity or dynamics. Our experimental work uses sparse Hopfield networks. 13.4.1.1 Palimpsest Hopfield nets with a modified learning rule In [SV99] a new learning rule is presented, which both increases the Hopfield network capacity and turns it into a “palimpsest”, i.e., a network that ca