Publications

Marinella, Matthew J.; Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hughart, David R.; Richter, Isaac R.; Hsia, Alexander W.; Fuller, Elliot J.; Talin, A.A.; Goeke, Ronald S.; Plimpton, Steven J.; James, Conrad D.

Abstract not provided.

Marinella, Matthew J.; Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hughart, David R.; Richter, Isaac R.; Hsia, Alexander W.; Fuller, Elliot J.; Talin, A.A.; Goeke, Ronald S.; Plimpton, Steven J.; James, Conrad D.

Abstract not provided.

Marinella, Matthew J.; Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hsia, Alexander W.; Hughart, David R.; Richter, Isaac R.; Fuller, Elliot J.; Plimpton, Steven J.; Talin, A.A.; James, Conrad D.

Abstract not provided.

Marinella, Matthew J.; Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hsia, Alexander W.; Hughart, David R.; Richter, Isaac R.; Fuller, Elliot J.; Plimpton, Steven J.; Talin, A.A.; James, Conrad D.

Abstract not provided.

Hill, Aaron J.; Donaldson, Jonathon W.; Rothganger, Fredrick H.; Vineyard, Craig M.; Follett, David R.; Follett, Pamela L.; Smith, Michael R.; Verzi, Stephen J.; Severa, William M.; Wang, Felix W.; Aimone, James B.; Naegle, John H.; James, Conrad D.

Abstract not provided.

IEEE Access

Quach, Tu-Thach Q.; Agarwal, Sapan A.; James, Conrad D.; Marinella, Matthew J.; Aimone, James B.

Emerging memory devices, such as resistive crossbars, have the capacity to store large amounts of data in a single array. Acquiring the data stored in large-capacity crossbars in a sequential fashion can become a bottleneck. We present practical methods, based on sparse sampling, to quickly acquire sparse data stored on emerging memory devices that support the basic summation kernel, reducing the acquisition time from linear to sub-linear. The experimental results show that at least an order of magnitude improvement in acquisition time can be achieved when the data are sparse. In addition, we show that the energy cost associated with our approach is competitive to that of the sequential method.

Proceedings - 2017 International Conference on Computational Science and Computational Intelligence, CSCI 2017

Doak, Justin E.; Ingram, Joey; Mulder, Samuel A.; Naegle, John H.; Cox, Jonathan A.; Aimone, James B.; Dixon, Kevin R.; James, Conrad D.; Follett, David R.

A forensics investigation after a breach often uncovers network and host indicators of compromise (IOCs) that can be deployed to sensors to allow early detection of the adversary in the future. Over time, the adversary will change tactics, techniques, and procedures (TTPs), which will also change the data generated. If the IOCs are not kept up-to-date with the adversary's new TTPs, the adversary will no longer be detected once all of the IOCs become invalid. Tracking the Known (TTK) is the problem of keeping IOCs, in this case regular expression (regexes), up-to-date with a dynamic adversary. Our framework solves the TTK problem in an automated, cyclic fashion to bracket a previously discovered adversary. This tracking is accomplished through a data-driven approach of self-adapting a given model based on its own detection capabilities.In our initial experiments, we found that the true positive rate (TPR) of the adaptive solution degrades much less significantly over time than the naïve solution, suggesting that self-updating the model allows the continued detection of positives (i.e., adversaries). The cost for this performance is in the false positive rate (FPR), which increases over time for the adaptive solution, but remains constant for the naïve solution. However, the difference in overall detection performance, as measured by the area under the curve (AUC), between the two methods is negligible. This result suggests that self-updating the model over time should be done in practice to continue to detect known, evolving adversaries.

Neural Computation

Verzi, Stephen J.; Rothganger, Fredrick R.; Parekh, Ojas D.; Quach, Tu-Thach Q.; Miner, Nadine E.; Vineyard, Craig M.; James, Conrad D.; Aimone, James B.

Neural-inspired spike-based computing machines often claim to achieve considerable advantages in terms of energy and time efficiency by using spikes for computation and communication. However, fundamental questions about spike-based computation remain unanswered. For instance, how much advantage do spike-based approaches have over conventionalmethods, and underwhat circumstances does spike-based computing provide a comparative advantage? Simply implementing existing algorithms using spikes as the medium of computation and communication is not guaranteed to yield an advantage. Here, we demonstrate that spike-based communication and computation within algorithms can increase throughput, and they can decrease energy cost in some cases. We present several spiking algorithms, including sorting a set of numbers in ascending/descending order, as well as finding the maximum or minimum ormedian of a set of numbers.We also provide an example application: a spiking median-filtering approach for image processing providing a low-energy, parallel implementation. The algorithms and analyses presented here demonstrate that spiking algorithms can provide performance advantages and offer efficient computation of fundamental operations useful in more complex algorithms.

IEEE Transactions on Information Forensics and Security

Wang, Felix W.; Quach, Tu-Thach Q.; Wheeler, Jason W.; Aimone, James B.; James, Conrad D.

File fragment classification is an important step in the task of file carving in digital forensics. In file carving, files must be reconstructed based on their content as a result of their fragmented storage on disk or in memory. Existing methods for classification of file fragments typically use hand-engineered features, such as byte histograms or entropy measures. In this paper, we propose an approach using sparse coding that enables automated feature extraction. Sparse coding, or sparse dictionary learning, is an unsupervised learning algorithm, and is capable of extracting features based simply on how well those features can be used to reconstruct the original data. With respect to file fragments, we learn sparse dictionaries for n-grams, continuous sequences of bytes, of different sizes. These dictionaries may then be used to estimate n-gram frequencies for a given file fragment, but for significantly larger n-gram sizes than are typically found in existing methods which suffer from combinatorial explosion. To demonstrate the capability of our sparse coding approach, we used the resulting features to train standard classifiers, such as support vector machines over multiple file types. Experimentally, we achieved significantly better classification results with respect to existing methods, especially when the features were used in supplement to existing hand-engineered features.

Smith, Michael R.; Ingram, Joey; Lamb, Christopher L.; Draelos, Timothy J.; Doak, Justin E.; Aimone, James B.; James, Conrad D.

Abstract not provided.

Faust, Aleksandra F.; Aimone, James B.; James, Conrad D.; Tapia, Lydia T.

Abstract not provided.

Marinella, Matthew J.; Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hughart, David R.; Richter, Isaac R.; Hsia, Alexander W.; Fuller, Elliot J.; Talin, A.A.; Goeke, Ronald S.; Plimpton, Steven J.; James, Conrad D.

Abstract not provided.

Annual ACM Symposium on Parallelism in Algorithms and Architectures

Parekh, Ojas D.; James, Conrad D.; Phillips, Cynthia A.; Aimone, James B.

Boolean circuits of McCulloch-Pitts threshold gates are a classic model of neural computation studied heavily in the late 20th century as a model of general computation. Recent advances in large-scale neural computing hardware has made their practical implementation a near-term possibility. We describe a theoretical approach for multiplying two N by N matrices that integrates threshold gate logic with conventional fast matrix multiplication algorithms, that perform O(Nω) arithmetic operations for a positive constant ω < 3. Our approach converts such a fast matrix multiplication algorithm into a constant-depth threshold circuit with approximately O(Nω) gates. Prior to our work, it was not known whether the Θ(N3)-gate barrier for matrix multiplication was surmountable by constant-depth threshold circuits. Dense matrix multiplication is a core operation in convolutional neural network training. Performing this work on a neural architecture instead of off-loading it to a GPU may be an appealing option.

Parekh, Ojas D.; Phillips, Cynthia A.; James, Conrad D.; Aimone, James B.

Abstract not provided.

2017 IEEE International Conference on Rebooting Computing, ICRC 2017 - Proceedings

Hill, Aaron J.; Donaldson, Jonathon W.; Rothganger, Fredrick R.; Vineyard, Craig M.; Follett, David R.; Follett, Pamela L.; Smith, Michael R.; Verzi, Stephen J.; Severa, William M.; Wang, Felix W.; Aimone, James B.; Naegle, John H.; James, Conrad D.

Unlike general purpose computer architectures that are comprised of complex processor cores and sequential computation, the brain is innately parallel and contains highly complex connections between computational units (neurons). Key to the architecture of the brain is a functionality enabled by the combined effect of spiking communication and sparse connectivity with unique variable efficacies and temporal latencies. Utilizing these neuroscience principles, we have developed the Spiking Temporal Processing Unit (STPU) architecture which is well-suited for areas such as pattern recognition and natural language processing. In this paper, we formally describe the STPU, implement the STPU on a field programmable gate array, and show measured performance data.

Doak, Justin E.; Doak, Justin E.; Ingram, Joey; Ingram, Joey; Mulder, Samuel A.; Mulder, Samuel A.; Naegle, John H.; Naegle, John H.; Cox, Jonathan A.; Cox, Jonathan A.; Aimone, James B.; Aimone, James B.; Dixon, Kevin R.; Dixon, Kevin R.; James, Conrad D.; James, Conrad D.; Follet, David R.; Follet, David R.

Abstract not provided.

Severa, William M.; Timlin, Jerilyn A.; Kholwadwala, Suraj K.; James, Conrad D.; Aimone, James B.

Abstract not provided.

Agarwal, Sapan A.; Hsia, Alexander W.; Jacobs-Gedrim, Robin B.; Hughart, David R.; Plimpton, Steven J.; James, Conrad D.; Marinella, Matthew J.

Abstract not provided.

Verzi, Stephen J.; Dellana, Ryan A.; Severa, William M.; Smith, Michael R.; Rothganger, Fredrick R.; James, Conrad D.; Aimone, James B.; Vineyard, Craig M.

Abstract not provided.

Digest of Technical Papers - Symposium on VLSI Technology

Agarwal, Sapan A.; Jacobs-Gedrim, Robin B.; Hsia, Alexander W.; Hughart, David R.; Fuller, Elliot J.; Talin, A.A.; James, Conrad D.; Plimpton, Steven J.; Marinella, Matthew J.

Analog resistive memories promise to reduce the energy of neural networks by orders of magnitude. However, the write variability and write nonlinearity of current devices prevent neural networks from training to high accuracy. We present a novel periodic carry method that uses a positional number system to overcome this while maintaining the benefit of parallel analog matrix operations. We demonstrate how noisy, nonlinear TaOx devices that could only train to 80% accuracy on MNIST, can now reach 97% accuracy, only 1% away from an ideal numeric accuracy of 98%. On a file type dataset, the TaOx devices achieve ideal numeric accuracy. In addition, low noise, linear Li1-xCoO2 devices train to ideal numeric accuracies using periodic carry on both datasets.

ACM International Conference Proceeding Series

Follett, David R.; Karpman, Gabe D.; Townsend, Duncan; Naegle, John H.; Follett, Pamela L.; Suppona, Roger A.; Aimone, James B.; James, Conrad D.

In 2016, Lewis Rhodes Labs, (LRL), shipped the first commercially viable Neuromorphic Processing Unit, (NPU), branded as a Neuromorphic Data Microscope (NDM). This product leverages architectural mechanisms derived from the sensory cortex of the human brain to efficiently implement pattern matching. LRL and Sandia National Labs have optimized this product for streaming analytics, and demonstrated a 1,000x power per operation reduction in an FPGA format. When reduced to an ASIC, the efficiency will improve to 1,000,000x. Additionally, the neuromorphic nature of the device gives it powerful computational attributes that are counterintuitive to those schooled in traditional von Neumann architectures. The Neuromorphic Data Microscope is the first of a broad class of brain-inspired, time domain processors that will profoundly alter the functionality and economics of data processing.

Draelos, Timothy J.; Miner, Nadine E.; Cox, Jonathan A.; Lamb, Christopher L.; Vineyard, Craig M.; Carlson, Kristofor C.; Severa, William M.; James, Conrad D.; Aimone, James B.

Abstract not provided.

Proceedings of the International Joint Conference on Neural Networks

Draelos, Timothy J.; Miner, Nadine E.; Lamb, Christopher L.; Cox, Jonathan A.; Vineyard, Craig M.; Carlson, Kristofor D.; Severa, William M.; James, Conrad D.; Aimone, James B.

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.

Proceedings of the International Joint Conference on Neural Networks

Smith, Michael R.; Hill, Aaron J.; Carlson, Kristofor D.; Vineyard, Craig M.; Donaldson, Jonathon W.; Follett, David R.; Follett, Pamela L.; Naegle, John H.; James, Conrad D.; Aimone, James B.

Information in neural networks is represented as weighted connections, or synapses, between neurons. This poses a problem as the primary computational bottleneck for neural networks is the vector-matrix multiply when inputs are multiplied by the neural network weights. Conventional processing architectures are not well suited for simulating neural networks, often requiring large amounts of energy and time. Additionally, synapses in biological neural networks are not binary connections, but exhibit a nonlinear response function as neurotransmitters are emitted and diffuse between neurons. Inspired by neuroscience principles, we present a digital neuromorphic architecture, the Spiking Temporal Processing Unit (STPU), capable of modeling arbitrary complex synaptic response functions without requiring additional hardware components. We consider the paradigm of spiking neurons with temporally coded information as opposed to non-spiking rate coded neurons used in most neural networks. In this paradigm we examine liquid state machines applied to speech recognition and show how a liquid state machine with temporal dynamics maps onto the STPU - demonstrating the flexibility and efficiency of the STPU for instantiating neural algorithms.

Proceedings of the International Joint Conference on Neural Networks

Verzi, Stephen J.; Vineyard, Craig M.; Vugrin, Eric D.; Galiardi, Meghan; James, Conrad D.; Aimone, James B.

Considerable effort is currently being spent designing neuromorphic hardware for addressing challenging problems in a variety of pattern-matching applications. These neuromorphic systems offer low power architectures with intrinsically parallel and simple spiking neuron processing elements. Unfortunately, these new hardware architectures have been largely developed without a clear justification for using spiking neurons to compute quantities for problems of interest. Specifically, the use of spiking for encoding information in time has not been explored theoretically with complexity analysis to examine the operating conditions under which neuromorphic computing provides a computational advantage (time, space, power, etc.) In this paper, we present and formally analyze the use of temporal coding in a neural-inspired algorithm for optimization-based computation in neural spiking architectures.