Programmable logic controllers (PLCs) and other field devices are important components of many weapons platforms, including vehicles, ships, radar systems, etc. Many have significant cyber vulnerabilities that lead to unacceptable risk. Furthermore, common procedures used during Operational Test and Evaluation (OT&E) may unexpectedly lead to unsafe or severe impacts for the field devices or the underlying physical process. This document describes an assessment methodology that addresses vulnerabilities, mitigations, and safe OT&E.
This study describes a cyber security research & development (R&D) gap analysis and research plan to address cyber security for industrial control system (ICS) supporting critical energy systems (CES). The Sandia National Laboratories (SNL) team addressed a long-term perspective for the R&D planning and gap analysis. Investment will posture CES for sustained and resilient energy operations well into the future.
Distributed Energy Resources (DER) are being added to the nation's electric grid, and as penetration of these resources increases, they have the potential to displace or offset large-scale, capital-intensive, centralized generation. Integration of DER into operation of the traditional electric grid requires automated operational control and communication of DER elements, from system measurement to control hardware and software, in conjunction with a utility's existing automated and human-directed control of other portions of the system. Implementation of DER technologies suggests a number of gaps from both a security and a policy perspective.
Rigorous methods and models are needed to quantify, measure, and increase the cyber resilience of critical infrastructure. An adversary may exploit vulnerabilities in the vital networks such as industrial control systems (ICS) associated with critical infrastructure (e.g., energy, financial, transportation, security), in order to achieve harmful consequences. In cyber systems, the number of vulnerabilities may be large, the attack surface changes over time, and the problem consists of both technical and non-technical factors (e.g., errors in software and human error). Given this complex and dynamic landscape, strategically mitigating risk is important, where “risk” considers both the probability of an event and the consequences if that event occurs. One way to decrease risk is to address consequences by ensuring that critical infrastructure is resilient. In this context, resilience is characterized by the magnitude and duration of a deviation from targeted performance levels, given a disruption. Increasing resilience decreases the consequences of a successful attack.
Microgrids are a focus of localized energy production that support resiliency, security, local con- trol, and increased access to renewable resources (among other potential benefits). The Smart Power Infrastructure Demonstration for Energy Reliability and Security (SPIDERS) Joint Capa- bility Technology Demonstration (JCTD) program between the Department of Defense (DOD), Department of Energy (DOE), and Department of Homeland Security (DHS) resulted in the pre- liminary design and deployment of three microgrids at military installations. This paper is focused on the analysis process and supporting software used to determine optimal designs for energy surety microgrids (ESMs) in the SPIDERS project. There are two key pieces of software, an ex- isting software application developed by Sandia National Laboratories (SNL) called Technology Management Optimization (TMO) and a new simulation developed for SPIDERS called the per- formance reliability model (PRM). TMO is a decision support tool that performs multi-objective optimization over a mixed discrete/continuous search space for which the performance measures are unrestricted in form. The PRM is able to statistically quantify the performance and reliability of a microgrid operating in islanded mode (disconnected from any utility power source). Together, these two software applications were used as part of the ESM process to generate the preliminary designs presented by SNL-led DOE team to the DOD. Acknowledgements Sandia National Laboratories and the SPIDERS technical team would like to acknowledge the following for help in the project: * Mike Hightower, who has been the key driving force for Energy Surety Microgrids * Juan Torres and Abbas Akhil, who developed the concept of microgrids for military instal- lations * Merrill Smith, U.S. Department of Energy SPIDERS Program Manager * Ross Roley and Rich Trundy from U.S. Pacific Command * Bill Waugaman and Bill Beary from U.S. Northern Command * Tarek Abdallah, Melanie Johnson, and Harold Sanborn of the U.S. Army Corps of Engineers Construction Engineering Research Laboratory * Colleagues from Sandia National Laboratories (SNL) for their reviews, suggestions, and participation in the work.
Sandia National Laboratories has an existing capability for hybrid control systems testing called SCEPTRE. This article proposes an architecture to add dynamic simulation capability for the underlying physical process (e.g. the power grid). Dynamic simulation for SCEPTRE will enable very accurate simulation, and allow the full integration of analog control systems hardware.
This document describes a microgrid cyber security reference architecture leveraging defense-in-depth techniques that are executed by first describing actor communication using data exchange attributes, then segmenting the microgrid control system network into enclaves, and finally grouping enclaves into functional domains. To illustrate the design approach, two notional microgrid control implementations are presented. Both include a discussion on types of communication occurring on that network, data exchange attributes for the actors, and examples of segmentation via enclaves and functional domains. The second example includes results from Red Team analysis and quantitative scoring according to a novel system that derives naturally from the implementation of the cyber security architecture.
Many critical loads rely on simple backup generation to provide electricity in the event of a power outage. An Energy Surety Microgrid TM can protect against outages caused by single generator failures to improve reliability. An ESM will also provide a host of other benefits, including integration of renewable energy, fuel optimization, and maximizing the value of energy storage. The ESM concept includes a categorization for microgrid value proposi- tions, and quantifies how the investment can be justified during either grid-connected or utility outage conditions. In contrast with many approaches, the ESM approach explic- itly sets requirements based on unlikely extreme conditions, including the need to protect against determined cyber adversaries. During the United States (US) Department of Defense (DOD)/Department of Energy (DOE) Smart Power Infrastructure Demonstration for Energy Reliability and Security (SPIDERS) effort, the ESM methodology was successfully used to develop the preliminary designs, which direct supported the contracting, construction, and testing for three military bases. Acknowledgements Sandia National Laboratories and the SPIDERS technical team would like to acknowledge the following for help in the project: * Mike Hightower, who has been the key driving force for Energy Surety Microgrids * Juan Torres and Abbas Akhil, who developed the concept of microgrids for military installations * Merrill Smith, U.S. Department of Energy SPIDERS Program Manager * Ross Roley and Rich Trundy from U.S. Pacific Command * Bill Waugaman and Bill Beary from U.S. Northern Command * Melanie Johnson and Harold Sanborn of the U.S. Army Corps of Engineers Construc- tion Engineering Research Laboratory * Experts from the National Renewable Energy Laboratory, Idaho National Laboratory, Oak Ridge National Laboratory, and Pacific Northwest National Laboratory
In 2012, Hurricane Sandy devastated much of the U.S. northeast coastal areas. Among those hardest hit was the small community of Hoboken, New Jersey, located on the banks of the Hudson River across from Manhattan. This report describes a city-wide electrical infrastructure design that uses microgrids and other infrastructure to ensure the city retains functionality should such an event occur in the future. The designs ensure that up to 55 critical buildings will retain power during blackout or flooded conditions and include analysis for microgrid architectures, performance parameters, system control, renewable energy integration, and financial opportunities (while grid connected). The results presented here are not binding and are subject to change based on input from the Hoboken stakeholders, the integrator selected to manage and implement the microgrid, or other subject matter experts during the detailed (final) phase of the design effort.
This white paper focuses on "advanced microgrids," but sections do, out of necessity, reference today's commercially available systems and installations in order to clearly distinguish the differences and advances. Advanced microgrids have been identified as being a necessary part of the modern electrical grid through a two DOE microgrid workshops, the National Institute of Standards and Technology, Smart Grid Interoperability Panel and other related sources. With their grid-interconnectivity advantages, advanced microgrids will improve system energy efficiency and reliability and provide enabling technologies for grid-independence to end-user sites. One popular definition that has been evolved and is used in multiple references is that a microgrid is a group of interconnected loads and distributed-energy resources within clearly defined electrical boundaries that acts as a single controllable entity with respect to the grid. A microgrid can connect and disconnect from the grid to enable it to operate in both grid-connected or island-mode. Further, an advanced microgrid can then be loosely defined as a dynamic microgrid.
Design and operation of the electric power grid (EPG) relies heavily on computational models. High-fidelity, full-order models are used to study transient phenomena on only a small part of the network. Reduced-order dynamic and power flow models are used when analysis involving thousands of nodes are required due to the computational demands when simulating large numbers of nodes. The level of complexity of the future EPG will dramatically increase due to large-scale deployment of variable renewable generation, active load and distributed generation resources, adaptive protection and control systems, and price-responsive demand. High-fidelity modeling of this future grid will require significant advances in coupled, multi-scale tools and their use on high performance computing (HPC) platforms. This LDRD report demonstrates SNL's capability to apply HPC resources to these 3 tasks: (1) High-fidelity, large-scale modeling of power system dynamics; (2) Statistical assessment of grid security via Monte-Carlo simulations of cyber attacks; and (3) Development of models to predict variability of solar resources at locations where little or no ground-based measurements are available.
The development continues for Finite State Abstraction (FSA) methods to enable Impacts Analysis (IA) for cyber attack against power grid control systems. Building upon previous work, we successfully demonstrated the addition of Bounded Model Checking (BMC) to the FSA method, which constrains grid conditions to reasonable behavior. The new FSA feature was successfully implemented and tested. FSA is an important part of IA for the power grid, complementing steady-state approaches. It enables the simultaneous evaluation of myriad dynamic trajectories for the system, which in turn facilitates IA for whole ranges of system conditions simultaneously. Given the potentially wide range and subtle nature of potential control system attacks, this is a promising research approach. In this report, we will explain the addition of BMC to the previous FSA work and some testing/simulation upon the implemented code using a two-bus test system. The current FSA approach and code allow the calculation of the acceptability of power grid conditions post-cyber attack (over a given time horizon and for a specific grid topology). Future work will enable analysis spanning various topologies (to account for switching events), as well as an understanding of the cyber attack stimuli that can lead to undesirable grid conditions.
To analyze the risks due to cyber attack against control systems used in the United States electrical infrastructure, new algorithms are needed to determine the possible impacts. This research is studying the Reliability Impact of Cyber ttack (RICA) in a two-pronged approach. First, malevolent cyber actions are analyzed in terms of reduced grid reliability. Second, power system impacts are investigated using an abstraction of the grid's dynamic model. This second year of esearch extends the work done during the first year.
Process Control System (PCS) security is critical to our national security. Yet, there are a number of technological, economic, and educational impediments to PCS owners implementing effective security on their systems. OPSAID (Open PCS Security Architecture for Interoperable Design), a project sponsored by the US Department of Energy's Office of Electricity Delivery and Reliability, aims to address this issue through developing and testing an open source architecture for PCS security. Sandia National Laboratories, along with a team of PCS vendors and owners, have developed and tested this PCS security architecture. This report describes their progress to date.2 AcknowledgementsThe authors acknowledge and thank their colleagues for their assistance with the OPSAID project.Sandia National Laboratories: Alex Berry, Charles Perine, Regis Cassidy, Bryan Richardson, Laurence PhillipsTeumim Technical, LLC: Dave TeumimIn addition, the authors are greatly indebted to the invaluable help of the members of the OPSAID Core Team. Their assistance has been critical to the success and industry acceptance of the OPSAID project.Schweitzer Engineering Laboratory: Rhett Smith, Ryan Bradetich, Dennis GammelTelTone: Ori Artman Entergy: Dave Norton, Leonard Chamberlin, Mark AllenThe authors would like to acknowledge that the work that produced the results presented in this paper was funded by the U.S. Department of Energy/Office of Electricity Delivery and Energy Reliability (DOE/OE) as part of the National SCADA Test Bed (NSTB) Program. Executive SummaryProcess control systems (PCS) are very important for critical infrastructure and manufacturing operations, yet cyber security technology in PCS is generally poor. The OPSAID (Open PCS (Process Control System) Security Architecture for Interoperable Design) program is intended to address these security shortcomings by accelerating the availability and deployment of comprehensive security technology for PCS, both for existing PCS and inherently secure PCS in the future. All activities are closely linked to industry outreach and advisory efforts.Generally speaking, the OPSAID project is focused on providing comprehensive security functionality to PCS that communicate using IP. This is done through creating an interoperable PCS security architecture and developing a reference implementation, which is tested extensively for performance and reliability.This report first provides background on the PCS security problem and OPSAID, followed by goals and objectives of the project. The report also includes an overview of the results, including the OPSAID architecture and testing activities, along with results from industry outreach activities. Conclusion and recommendation sections follow. Finally, a series of appendices provide more detailed information regarding architecture and testing activities.Summarizing the project results, the OPSAID architecture was defined, which includes modular security functionality and corresponding component modules. The reference implementation, which includes the collection of component modules, was tested extensively and proved to provide more than acceptable performance in a variety of test scenarios. The primary challenge in implementation and testing was correcting initial configuration errors.OPSAID industry outreach efforts were very successful. A small group of industry partners were extensively involved in both the design and testing of OPSAID. Conference presentations resulted in creating a larger group of potential industry partners.Based upon experience implementing and testing OPSAID, as well as through collecting industry feedback, the OPSAID project has done well and is well received. Recommendations for future work include further development of advanced functionality, refinement of interoperability guidance, additional laboratory and field testing, and industry outreach that includes PCS owner education. 4 5 --This page intentionally left blank --
Supervisory Control and Data Acquisition (SCADA) systems for automation are very important for critical infrastructure and manufacturing operations. They have been implemented to work in a number of physical environments using a variety of hardware, software, networking protocols, and communications technologies, often before security issues became of paramount concern. To offer solutions to security shortcomings in the short/medium term, this project was to identify technologies used to secure "traditional" IT networks and systems, and then assess their efficacy with respect to SCADA systems. These proposed solutions must be relatively simple to implement, reliable, and acceptable to SCADA owners and operators. 4This page intentionally left blank.
This report presents a classification scheme for risk assessment methods. This scheme, like all classification schemes, provides meaning by imposing a structure that identifies relationships. Our scheme is based on two orthogonal aspects--level of detail, and approach. The resulting structure is shown in Table 1 and is explained in the body of the report. Each cell in the Table represent a different arrangement of strengths and weaknesses. Those arrangements shift gradually as one moves through the table, each cell optimal for a particular situation. The intention of this report is to enable informed use of the methods so that a method chosen is optimal for a situation given. This report imposes structure on the set of risk assessment methods in order to reveal their relationships and thus optimize their usage.We present a two-dimensional structure in the form of a matrix, using three abstraction levels for the rows and three approaches for the columns. For each of the nine cells in the matrix we identify the method type by name and example. The matrix helps the user understand: (1) what to expect from a given method, (2) how it relates to other methods, and (3) how best to use it. Each cell in the matrix represent a different arrangement of strengths and weaknesses. Those arrangements shift gradually as one moves through the table, each cell optimal for a particular situation. The intention of this report is to enable informed use of the methods so that a method chosen is optimal for a situation given. The matrix, with type names in the cells, is introduced in Table 2 on page 13 below. Unless otherwise stated we use the word 'method' in this report to refer to a 'risk assessment method', though often times we use the full phrase. The use of the terms 'risk assessment' and 'risk management' are close enough that we do not attempt to distinguish them in this report. The remainder of this report is organized as follows. In Section 2 we provide context for this report--what a 'method' is and where it fits. In Section 3 we present background for our classification scheme--what other schemes we have found, the fundamental nature of methods and their necessary incompleteness. In Section 4 we present our classification scheme in the form of a matrix, then we present an analogy that should provide an understanding of the scheme, concluding with an explanation of the two dimensions and the nine types in our scheme. In Section 5 we present examples of each of our classification types. In Section 6 we present conclusions.
This report documents work supporting the Sandia National Laboratories initiative in Distributed Energy Resources (DERs) and Supervisory Control and Data Acquisition (SCADA) systems. One approach for real-time control of power generation assets using feedback control, Quantitative feedback theory (QFT), has recently been applied to voltage, frequency, and phase-control of power systems at Sandia. QFT provided a simple yet powerful philosophy for designing the control systems--allowing the designer to optimize the system by making design tradeoffs without getting lost in complex mathematics. The feedback systems were effective in reducing sensitivity to large and sudden changes in the power grid system. Voltage, frequency, and phase were accurately controlled, even with large disturbances to the power grid system.