Publications

Adams, Susan S.; Hendrickson, Stacey M.; Murchison, Nicole M.

Abstract not provided.

Acquesta, Erin A.; Chen, Guenevere C.; Adams, Susan S.; Bryant, Ross D.; Haas, Jason J.; Johnson, Nicholas T.; Romanowich, Paul R.; Roy, Krishna C.; Shakamuri, Mayuri S.; Ting, Christina T.

The cybersecurity research community has focused primarily on the analysis and automation of intrusion detection systems by examining network traffic behaviors. Expanding on this expertise, advanced cyber defense analysis is turning to host-based data to use in research and development to produce the next generation network defense tools. The ability to perform deep packet inspection of network traffic is increasingly harder with most boundary network traffic moving to HTTPS. Additionally, network data alone does not provide a full picture of end-to-end activity. These are some of the reasons that necessitate looking at other data sources such as host data. We outline our investigation into the processing, formatting, and storing of the data along with the preliminary results from our exploratory data analysis. In writing this report, it is our goal to aid in guiding future research by providing foundational understanding for an area of cybersecurity that is rich with a variety of complex, categorical, and sparse data, with a strong human influence component. Including suggestions for guiding potential directions for future research.

Adams, Susan S.; Morris, Jason

Abstract not provided.

Adams, Susan S.; Murchison, Nicole M.; Hendrickson, Stacey M.

Abstract not provided.

Adams, Susan S.; Fleming Lindsley, Elizabeth S.; Heiden, Siobhan M.; Kittinger, Liza

Abstract not provided.

Chemical Engineering Transactions

Adams, Susan S.; Murchison, Nicole; Bruneau, Robert J.

Malicious cyber-attacks are becoming increasingly prominent due to the advance of technology and attack methods over the last decade. These attacks have the potential to bring down critical infrastructures, such as nuclear power plants (NPP’s), which are so vital to the country that their incapacitation would have debilitating effects on national security, public health, or safety. Despite the devastating effects a cyber-attack could have on NPP’s, it is unclear how control room operations would be affected in such a situation. In this project, the authors are collaborating with NPP operators to discern the impact of cyber-attacks on control room operations and lay out a framework to better understand the control room operators’ tasks and decision points. A cyber emulation of a digital control system was developed and coupled with a generic pressurized water reactor (GPWR) training simulator at Idaho National Laboratories. Licensed operators were asked to complete a series of scenarios on the simulator in which some of the scenarios were purposely obfuscated; that is, in which indicators were purposely displaying inaccurate information. Of interest is how this obfuscation impacts the ability to keep the plant safe and how it affects operators’ perceptions of workload and performance. Results, conclusions and lessons learned from this pilot experiment will be discussed. This research sheds light onto about how cyber events impact plant operations.

Adams, Susan S.; Fleming Lindsley, Elizabeth S.; Heiden, Siobhan M.; Kittinger, Liza

Abstract not provided.

Adams, Susan S.; Fleming Lindsley, Elizabeth S.; Heiden, Siobhan M.; Kittinger, Liza

Abstract not provided.

Adams, Susan S.; Bruneau, Robert J.; Jacobs, Nicholas J.; Murchison, Nicole M.; Sandoval, Daniel R.; Seng, Bibiana E.

There are differences in how cyber - attack, sabotage, or discrete component failure mechanisms manifest within power plants and what these events would look like within the control room from an operator's perspective. This research focuses on understanding how a cyber event would affect the operation of the plant, how an operator would perceive the event, and if the operator's actions based on those perceptions will allow him/her to maintain plant safety. This research is funded as part of Sandia's Laborator y Directed Research and Development (LDRD) program to develop scenarios with cyber induced failure of plant systems coupled with a generic pressurized water reactor plant training simulator. The cyber scenario s w ere developed separately and injected into the simulator operational state to simulate an attack. These scenarios will determine if Nuclear Power Plant (NPP) operators can 1) recognize that the control room indicators were presenting incorrect or erroneous i nformation and 2) take appropriate actions to keep the plant safe. This will also provide the opportunity to assess the operator cognitive workload during such events and identify where improvements might be made. This paper will review results of a pilot study run with NPP operators to investigate performance under various cyber scenarios. The d iscussion will provide an overview of the approach, scenario selection, metrics captured , resulting insights into operator actions and plant response to multiple sc enarios of the NPP system .

Adams, Susan S.; Murchison, Nicole M.; Bruneau, Robert J.

Abstract not provided.

Bruneau, Robert J.; Adams, Susan S.; Murchison, Nicole M.; Jacobs, Nicholas J.; Sandoval, Daniel R.; Seng, Bibiana E.

Abstract not provided.

Advances in Intelligent Systems and Computing

Adams, Susan S.; Hendrickson, Stacey M.; Turner, Phillip L.

Malicious cyber-attacks are becoming increasingly prominent due to the advance of technology and methods over the last decade. These attacks have the potential to bring down critical infrastructures, such as nuclear power plants (NPP’s), which are so vital to the country that their incapacitation would have debilitating effects on national security, public health, or safety. Despite the devastating effects a cyber-attack could have on NPP’s, there is a lack of understanding as to the effects on the plant from a discreet failure or surreptitious sabotage of components and a lack of knowledge in how the control room operators would react to such a situation. In this project, the authors are collaborating with NPP operators to discern the impact of cyber-attacks on control room operations and lay out a framework to better understand the control room operators’ tasks and decision points.

Adams, Susan S.; Hendrickson, Stacey M.; Murchison, Nicole M.; Turner, Phillip L.

Abstract not provided.

Turner, Phillip L.; Adams, Susan S.; Hendrickson, Stacey M.

Abstract not provided.

10th International Topical Meeting on Nuclear Plant Instrumentation, Control, and Human-Machine Interface Technologies, NPIC and HMIT 2017

Turner, Phillip L.; Adams, Susan S.; Hendrickson, Stacey M.

There are gaps in understanding how a cyber-attack would manifest itself within power plants and what these events would look like within the control room from an operator’s perspective. This is especially true for nuclear power plants where safety has much broader consequences than nonnuclear plants. The operating and emergency procedures that operators currently use are likely inadequate for targeted cyber-attacks. This research focuses on understanding how a cyber event would affect the operation of the plant, how an operator would perceive the event, and if the operator’s actions would keep the plant in a safe condition. This research is part of Sandia’s Laboratory Directed Research and Development program where a nuclear power plant cyber model of the control system digital architecture is coupled with a generic pressurized water reactor plant training simulator. Cyber event scenarios will be performed on the coupled system with plant operators. The scenarios simulate plant conditions that may exist during a cyber-attack, component failure, or insider sabotage, and provide an understanding of the displayed information and the actual plant conditions. These scenarios will determine if plant operators can 1) recognize that they are under cyber-attack and 2) take appropriate actions to keep the plant safe. This will also provide the opportunity to assess the operator cognitive workload during such events and identify where improvements might be made. Experiments with nuclear power plant operators will be carried out over FY 2018 and results of the research are expected by the end of FY 2018.

Advances in Intelligent Systems and Computing

Adams, Susan S.; Hannigan, Francis P.

Electric distribution utilities are on the brink of a paradigm shift to smart grids, which will incorporate new technologies and fundamentally change control room operations. Expertise in the control room, which has never been well defined, must be characterized in order to understand how this shift will impact control room operations and operator performance. In this study, the authors collaborated with a utility company in Vermont to define and understand expertise in distribution control room operations. The authors interviewed distribution control room operators, HR personnel, and managers and concluded that a control room expert is someone who has 7–9 years’ experience in the control room and possesses certain traits, such as the ability to remain calm under pressure, effectively multi-task and quickly synthesize large amounts of data. This work has implications for control room operator training and how expertise is defined in the control room domain.

Memory and Cognition

Matzen, Laura E.; Trumbo, Michael C.; Haass, Michael J.; Hunter, Michael A.; Silva, Austin R.; Adams, Susan S.; Bunting, Michael F.; O’Rourke, Polly

There is a great deal of debate concerning the benefits of working memory (WM) training and whether that training can transfer to other tasks. Although a consistent finding is that WM training programs elicit a short-term near-transfer effect (i.e., improvement in WM skills), results are inconsistent when considering persistence of such improvement and far transfer effects. In this study, we compared three groups of participants: a group that received WM training, a group that received training on how to use a mental imagery memory strategy, and a control group that received no training. Although the WM training group improved on the trained task, their posttraining performance on nontrained WM tasks did not differ from that of the other two groups. In addition, although the imagery training group’s performance on a recognition memory task increased after training, the WM training group’s performance on the task decreased after training. Participants’ descriptions of the strategies they used to remember the studied items indicated that WM training may lead people to adopt memory strategies that are less effective for other types of memory tasks. These results indicate that WM training may have unintended consequences for other types of memory performance.

Electricity Journal

Stamber, Kevin L.; Burnham, Laurie B.; Adams, Susan S.; Jeffers, Robert F.; Verzi, Stephen J.

Abstract not provided.

Burnham, Laurie B.; Stamber, Kevin L.; Jeffers, Robert F.; Adams, Susan S.; Verzi, Stephen J.; Sahakian, Meghan A.; Haass, Michael J.; Cauthen, Katherine R.

The transformation of the distribution grid from a centralized to decentralized architecture, with bi-directional power and data flows, is made possible by a surge in network intelligence and grid automation. While changes are largely beneficial, the interface between grid operator and automated technologies is not well understood, nor are the benefits and risks of automation. Quantifying and understanding the latter is an important facet of grid resilience that needs to be fully investigated. The work described in this document represents the first empirical study aimed at identifying and mitigating the vulnerabilities posed by automation for a grid that for the foreseeable future will remain a human-in-the-loop critical infrastructure. Our scenario-based methodology enabled us to conduct a series of experimental studies to identify causal relationships between grid-operator performance and automated technologies and to collect measurements of human performance as a function of automation. Our findings, though preliminary, suggest there are predictive patterns in the interplay between human operators and automation, patterns that can inform the rollout of distribution automation and the hiring and training of operators, and contribute in multiple and significant ways to the field of grid resilience.

Stamber, Kevin L.; Burnham, Laurie B.; Adams, Susan S.; Jeffers, Robert F.; Verzi, Stephen J.

Abstract not provided.

Adams, Susan S.; Hannigan, Frank H.

Abstract not provided.

Adams, Susan S.; Haass, Michael J.; Matzen, Laura E.; King, Saskia H.

Abstract not provided.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Adams, Susan S.; Haass, Michael J.; Matzen, Laura E.; King, Saskia H.

‘Big data’ is a phrase that has gained much traction recently. It has been defined as ‘a broad term for data sets so large or complex that traditional data processing applications are inadequate and there are challenges with analysis, searching and visualization’ [1]. Many domains struggle with providing experts accurate visualizations of massive data sets so that the experts can understand and make decisions about the data e.g., [2, 3, 4, 5]. Abductive reasoning is the process of forming a conclusion that best explains observed facts and this type of reasoning plays an important role in process and product engineering. Throughout a production lifecycle, engineers will test subsystems for critical functions and use the test results to diagnose and improve production processes. This paper describes a value-driven evaluation study [7] for expert analyst interactions with big data for a complex visual abductive reasoning task. Participants were asked to perform different tasks using a new tool, while eye tracking data of their interactions with the tool was collected. The participants were also asked to give their feedback and assessments regarding the usability of the tool. The results showed that the interactive nature of the new tool allowed the participants to gain new insights into their data sets, and all participants indicated that they would begin using the tool in its current state.

Harms, Gary A.; Knief, Ronald A.; Adams, Susan S.; Cole, Kerstan S.; Ford, John T.; Campbell, Rafe C.; Monahan, Shean P.

Abstract not provided.

Adams, Susan S.; Matzen, Laura E.; Haass, Michael J.; McNamara, Laura A.; McMichael, Stephanie N.

Abstract not provided.