Cognitive Systems | Vision & Approach | Human Emulation | Projects | In The News

External Collaborations | Cognitive Systems Workshop | Publications

The Human Emulation cognitive architecture provides the underlying foundation for our Cognitive Systems solutions. This architecture, illustrated below, has been developed by Sandia National Laboratories for dual-use with simulator-based synthetic human and intelligent machine applications. It is believed to be unique in that the underlying design has a neurological basis in oscillating systems (Klimesch, 1996) offering a significant departure from typical rule-based systems.

Individual emulators consist of several key features. Emulators receive input from perceptual channels and may incorporate a variety of sensor and data processing capabilities (e.g., infrared, acoustic, etc.), with the specific makeup depending on the application.

Three types of knowledge are represented: associative, situational/contextual and episodic.

Associative Knowledge, consisting of concepts or cues, hereafter referred to as concepts, and the associative relationships between concepts, is represented in an associative network (Alba & Hasher, 1983; Anderson, 1983; Schvaneveldt, 1990). This network consists of a collection of nodes with each node representing a separate concept. Nodes are assigned associative links to other nodes. In implementation, nodes are modeled as oscillators with activation levels that exhibit frequency and amplitude characteristics. Nodes may be activated through either bottom-up (i.e., perceptual) or top-down (i.e., situational knowledge) processes. If sufficient, activation may spread to other associated concepts.

Situational/Contextual Knowledge is represented as a collection of situations or contexts (Johnson-Laird, 1983; Zwan & Radvansky, 1998), each associated with a pattern of activation in the associative network. A situation might be a “restaurant.” In this case, there would be a pattern of activation associated with the situation “restaurant” that would include activation of some combination of concepts (e.g., tables, menu, waitress, diners, food, etc.) When a pattern is recognized, there is an awareness that corresponds to activation of the situation, and associated knowledge, and a general, although sometimes implicit, comprehension of ongoing events (Klein, 1997). The situation recognition processes is modeled using a single oscillator with frequency and amplitude characteristics.

Based on knowledge assigned to situations, recognition of a situation also leads to top-down activation of concepts in the associative network (Cook, et.al., 2001; Gerrig & McKoon, 2001; Myers & O’Brien, 1998). This is particularly evident when some of the concepts associated with a situation are missing, but there are sufficient concepts present for recognition that the situation is relevant. In such a case, expectations arise. This occurs through priming of concepts in the associative network lowering the threshold for activation of the primed concepts by perceptual processes. Through this mechanism, perceptual processes may benefit from situational or contextual knowledge.

Episodic Memory provides a store of experiences or episodes organized as themes or storylines (Colcombe & Wyer, 2002; Conwa & Pleydell-Pearce, 2000; Eldridge, et.al., 1994). Episodes are place and time referenced with these facets of the experience, as well as objects, contexts or events, acting as retrieval cues for specific episodes (Magliano, et.al., in press; Shun, 1998; Zwan & Radvansky, 1998). The content of a stored episode consists of a compressed version of the sequential patterns of activation in the associative network during the episode. Thus, an episode may be recalled by replaying the associated patterns of activation in the original sequence enabling recollection and mental simulation (Burt, et.al., 1998).

A Comparator conducts an ongoing assessment of the current state as reflected by the pattern of activation in the associative network, and expected states based on expectations assigned to the situation or context that is currently activated. The Comparator operates in conjunction with situation recognition. When a mismatch is detected (e.g., a concept is detected that is not expected with the current situation or context), there is an emotional response corresponding to surprise with an accompanying arousal response, and activation of Selective Attention (Donchin & Coles, 1988; Milham, et.al., 2001). To illustrate this phenomenon, if in a restaurant and an elephant walks in the door, all attention would likely be focused on the elephant.

Emotional Processes respond to positive and negative experiences to enable learning and initiate drive mechanisms. In some applications, specific perceptual events may be assigned positive or negative emotional associations (LeDoux, 1998) leading to direct activation of emotional processes, with no intermediate cognitive processes. In these cases, there would be direct activation of emotional processes by perceptual components of the system. In other applications, either concepts in the associative network or situations may be assigned emotional associations. With these applications, activation of the concept or situation would activate emotional processes (DeHouwer & Hermans, 1994).

An immediate response to emotional activation involves initiation of Drive Mechanisms. Here, positive emotional responses give rise to an Approach Drive whereby there is a de-emphasis of perceptual input and bias toward continuation of current situation-based goal-action sequences (Mizuki, et.al., 1992). In contrast, negative emotional responses give rise to a Withdrawal Drive whereby there is emphasis placed on perceptual processes and updating or recalibration of the situational or contextual interpretation of ongoing events.

In practice, the above processes operate in parallel. The product is an ongoing representation of events underlying situation recognition, with the potential for generation of appropriate actions, based on a situation-based interpretation of events. Through these mechanisms, either a synthetic human or intelligent machine may be endowed with a sophisticated cognitive model that operates in real-time and in coordination with other simulation or system control processes.