As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.
This document will detail a field demonstration test procedure for the Module OT device developed for the joint NREL-SNL DOE CEDS project titled "Modular Security Apparatus for Managing Distributed Cryptography for Command & Control Messages on Operational Technology (OT) Networks." The aim of this document is to create the testing and evaluation procedure for field demonstration of the device; this includes primarily functional testing and implementation testing at Public Service Company of New Mexico's (PNM's) Prosperity solar site environment. Specifically, the Module OT devices will be integrated into the Prosperity solar site system; traffic will be encrypted between several points of interest at the site (e.g., inverter micrologger and switch). The tests described in this document will be performed to assess the impact and effectiveness of the encryption capabilities provided by the Module OT device.
As the power grid incorporates increasing amounts of distributed energy resources (DER) that provide new generation sources, new opportunities are created for improving operation of the grid while large challenges also arise for preserving grid reliability and security. To improve grid performance, DERs can be utilized to provide important support functionality, such as supporting frequency and voltage levels, especially if they are assisted by communication schemes as part of an advanced distribution management system (ADMS). Unfortunately, such connectivity and grid support functionality also creates additional cyber security risk with the potential for degradation of grid services, especially under conditions with high amounts of distributed generation. This paper will first discuss the communications needed by DERs to support system and interoperability objectives, as well as the security requirements and impact of securing these communications. Some common security mechanisms are discussed in relation to DERs, and a simulated 15-bus model of a distribution feeder is used to demonstrate aspects of the DER communications and impact to grid performance. These results help to advance understanding of the benefits, requirements, and mechanisms for securely implementing DER communications while ensuring that grid reliability is maintained.
This document will detail a test procedure, involving bench and emulation testing, for the Module OT device developed for the joint NREL-SNL DOE CEDS project titled "Modular Security Apparatus for Managing Distributed Cryptography for Command & Control Messages on Operational Technology (OT) Networks." The aim of this document is to create the testing and evaluation protocol for the module for lab-level testing; this includes checklists and experiments for information gathering, functional testing, cryptographic implementation, public key infrastructure, key exchange/authentication, encryption, and implementation testing in the emulation environment.
In designing a security module for inverter communications in a DER environment, it is critical to consider the impact of the additional security on the environment as well as what types of security is required for the various messages that must pass from the inverter to and from a utility. Also, since cyber security is more than just preventing an unauthorized user from viewing data, mechanisms for proving identity and ensuring that data cannot be altered without such a modification being discovered are needed. This is where the security principles of confidentiality, integrity, and availability come into play. For different types of communications, these different security principles may be important or not needed at all. Furthermore, the cost and constraints for applying cryptography for securing DER communications must be considered to help determine what is feasible within this environment and what will be the impact and cost of applying common cryptographic protections to inverter communications.
Penetration of distributed energy resources (DERs) is rapidly increasing in the bulk power system (BPS); they are growing to be a significant portion of generation. As such, grid-support capabilities are being developed and implemented. However, as their presence increases, the impact of DERs on the BPS also increases. Therefore, if a disturbance occurs in the DER system, its effects could propagate throughout the BPS. These disturbances could range from equipment malfunctions to resource variability to cyber attacks.
Penetration of distributed energy resources (DERs) is rapidly increasing in the bulk power system (BPS); they are growing to be a significant portion of generation. As such, grid-support capabilities are being developed and implemented; IEEE Std. 1547 mandates new interconnection and interoperability standards to achieve these capabilities and allow remote users to change behaviors to many devices. However, as their presence increases, the impact of DERs on the BPS also increases. Therefore, if a disturbance occurs in the DER system, its effects could propagate throughout the BPS. These disturbances could range from equipment malfunctions to resource variability to cyber attacks. This document will discuss general requirements needed for developing a distributed cryptography module for implementation in DER systems. First, we will examine the composition of DER systems in Section 2. Procedures to identify critical assets and the security of DERs is discussed in Section 3. Section 4 introduces cryptography concepts and Section 5 discusses implementation needs and options. Finally, practical considerations are provided in Section 6 and conclusions are provided in Section 7.