spacer
Projects Sandia Home

Engineering a Transformation of Human-Machine Interaction to an Augmented Cognitive Relationship

Cognitive Systems | Vision & Approach | Human Emulation | Projects | Publications
Engineering a Transformation
 

This project is being conducted by Sandia National Laboratories in support of the DARPA Augmented Cognition program. Work commenced in April of 2002. The objective for the DARPA program is to "extend, by an order of magnitude or more, the information management capacity of the human-computer warfighter." Initially, emphasis has been placed on detection of an operator's cognitive state so that systems may adapt accordingly (e.g., adjust information throughput to the operator in response to workload). Work conducted by Sandia focuses on development of technologies to infer an operator's ongoing cognitive processes, with specific emphasis on detecting discrepancies between machine state and an operator’s ongoing interpretation of events.

Project Description
In Phase 1 of DARPA's Augmented Cognition program, Sandia has developed a capability referred to as Discrepancy Detection. The objective is to enable a machine to detect when an operator has committed a situation mode error (i.e., operator believes the system is in one state and behaves accordingly, when the system is actually in a different state).

As illustrated in the Figure below, we first develop a detailed cognitive model of an operator's knowledge of a task. Currently, our development has focused on an AWACS operator. The machine then uses the operator's cognitive model to infer its own state. Simultaneously, the machine uses data from the operator to infer the operator's beliefs concerning the state of the system. This data currently includes actions taken by the operator and eye-tracking to identify cues that have received attention from the operator. By comparing the machine's recognition of its own state with the operator's behavior, the machine may detect conditions in which a discrepancy exists between the actual state of the machine and the operator's beliefs concerning the state of the machine.

Discrepancy Detection graphic

Our immediate plans involve integration of physiological indices with our capability for Discrepancy Detection. We believe that real-time EEG measures will allow us to enhance our capability to infer cognitive processes, while output from our cognitive model will also enhance the ability to infer cognitive workload based on physiological measures. Thus, the two capabilities should combine synergistically to enhance the performance of each.

Collaboration
This project is being conducted in conjunction with three collaborating organizations:

Air Force Research Laboratory – developing a multi-modal human-computer interface to support Discrepancy Detection within the context of the AWACS operator task interface.
Notre Dame University – conducting basic research to gain an understanding of the mechanisms whereby Discrepancy Detection occurs within the context of human-to-human interactions.

Project Documents
Presentation July 2002 DARPA AugCog PIs Meeting – SNL [PDF file]

Contact Chris Forsythe (jcforsy@sandia.gov) for more information.


Back to top of page

Projects | Publications
Engineering a Transformation

© Sandia Corporation | Site Contact | Site Map | Privacy and Security