The vertical borehole array at Farnsworth Unit, TX is used to monitor microseismic activity in the subsurface around the Carbon Capture and Sequestration (CCS) reservoir. The array consists of 16 3-component seismometers spaced vertically in a single borehole. Tube or borehole waves traveling up or down the borehole can corrupt signals of interest, such as microseismic events. A denoising convolutional neural network (DCNN) was trained to remove borehole waves from seismic waveforms of microseismic events for the purpose of reducing unwanted signal detections and better characterizing events of interest. This R&D leverages the work of Sandia colleague Rigo Tibi, who used a DCNN developed by Greg Beroza at Stanford University to improve the signal-to-noise ratio (SNR) of teleseismic events detected by the International Monitoring System.
The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. In geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.
Long-term seismic monitoring networks are well positioned to leverage advances in machine learning because of the abundance of labeled training data that curated event catalogs provide. We explore the use of convolutional and recurrent neural networks to accomplish discrimination of explosive and tectonic sources for local distances. Using a 5-year event catalog generated by the University of Utah Seismograph Stations, we train models to produce automated event labels using 90-s event spectrograms from three-component and single-channel sensors. Both network architectures are able to replicate analyst labels above 98%. Most commonly, model error is the result of label error (70% of cases). Accounting for mislabeled events (~1% of the catalog) model accuracy for both models increases to above 99%. Classification accuracy remains above 98% for shallow tectonic events, indicating that spectral characteristics controlled by event depth do not play a dominant role in event discrimination.
Seismic signals are composed of the seismic waves (phases) that reach a sensor, similar to the way speech signals are composed of phonemes that reach a listener's ear. Large/small seismic events near/far from a sensor are similar to loud/quiet speakers with high/low-pitched voices. We leverage ideas from speech recognition for the classification of seismic phases at a seismic sensor. Seismic Phase ID is challenging due to the varying paths and distances an event takes to reach a sensor, but there is consistent structure of the makeup (e.g. ordering) of the different phases arriving at the sensor.
Malware detection and remediation is an on-going task for computer security and IT professionals. Here, we examine the use of neural algorithms to detect malware using the system calls generated by executables-alleviating attempts at obfuscation as the behavior is monitored. We examine several deep learning techniques, and liquid state machines baselined against a random forest. The experiments examine the effects of concept drift to understand how well the algorithms generalize to novel malware samples by testing them on data that was collected after the training data. The results suggest that each of the examined machine learning algorithms is a viable solution to detect malware-achieving between 90% and 95% class-averaged accuracy (CAA). In real-world scenarios, the performance evaluation on an operational network may not match the performance achieved in training. Namely, the CAA may be about the same, but the values for precision and recall over the malware can change significantly. We structure experiments to highlight these caveats and offer insights into expected performance in operational environments. In addition, we use the induced models to better understand what differentiates malware samples from goodware, which can further be used as a forensics tool to provide directions for investigation and remediation.
The quality of automatic signal detections from sensor networks depends on individual detector trigger levels (TLs) from each sensor. The largely manual process of identifying effective TLs is painstaking and does not guarantee optimal configuration settings, yet achieving superior automatic detection of signals and ultimately, events, is closely related to these parameters. We present a Dynamic Detector Tuning (DDT) system that automatically adjusts effective TL settings for signal detectors to the current state of the environment by leveraging cooperation within a local neighborhood of network sensors. After a stabilization period, the DDT algorithm can adapt in near-real time to changing conditions and automatically tune a signal detector to identify (detect) signals from only events of interest. Our current work focuses on reducing false signal detections early in the seismic signal processing pipeline, which leads to fewer false events and has a significant impact on reducing analyst time and effort. This system provides an important new method to automatically tune detector TLs for a network of sensors and is applicable to both existing sensor performance boosting and new sensor deployment. With ground truth on detections from a local neighborhood of seismic sensors within a network monitoring the Mount Erebus volcano in Antarctica, we show that DDT reduces the number of false detections by 18% and the number of missed detections by 11% when compared with optimal fixed TLs for all sensors.
Rigorous characterization of the performance and generalization ability of cyber defense systems is extremely difficult, making it hard to gauge uncertainty, and thus, confidence. This difficulty largely stems from a lack of labeled attack data that fully explores the potential adversarial space. Currently, performance of cyber defense systems is typically evaluated in a qualitative manner by manually inspecting the results of the system on live data and adjusting as needed. Additionally, machine learning has shown promise in deriving models that automatically learn indicators of compromise that are more robust than analyst-derived detectors. However, to generate these models, most algorithms require large amounts of labeled data (i.e., examples of attacks). Algorithms that do not require annotated data to derive models are similarly at a disadvantage, because labeled data is still necessary when evaluating performance. In this work, we explore the use of temporal generative models to learn cyber attack graph representations and automatically generate data for experimentation and evaluation. Training and evaluating cyber systems and machine learning models requires significant, annotated data, which is typically collected and labeled by hand for one-off experiments. Automatically generating such data helps derive/evaluate detection models and ensures reproducibility of results. Experimentally, we demonstrate the efficacy of generative sequence analysis techniques on learning the structure of attack graphs, based on a realistic example. These derived models can then be used to generate more data. Additionally, we provide a roadmap for future research efforts in this area.
The quality of automatic detections from sensor networks depends on a large number of data processing parameters that interact in complex ways. The largely manual process of identifying effective parameters is painstaking and does not guarantee that the resulting controls are the optimal configuration settings, yet achieving superior automatic detection of events is closely related to these parameters. We present an automated sensor tuning (AST) system that tunes effective parameter settings for each sensor detector to the current state of the environment by leveraging cooperation within a neighborhood of sensors. After a stabilization period, the AST algorithm can adapt in near real-time to changing conditions and automatically self-tune a signal detector to identify (detect) only signals from events of interest. The overall goal is to reduce the number of missed legitimate event detections and the number of false event detections. Our current work focuses on reducing false signal detections early in the seismic signal processing pipeline, which leads to fewer false events and has a significant impact on reducing analyst time and effort. Applicable both for existing sensor performance boosting and new sensor deployment, this system provides an important new method to automatically tune complex remote sensing systems. Systems tuned in this way will achieve better performance than is currently possible by manual tuning, and with much less time and effort devoted to the tuning process. With ground truth on detections from a seismic sensor network monitoring the Mount Erebus Volcano in Antarctica, we show that AST increases the probability of detection while decreasing false alarms.