Publications

Poster for CCN

Internal LDRD virtual poster session

Poster for CCN

Abstract not provided.

Abstract not provided.

Abstract not provided.

Abstract not provided.

Abstract not provided.

As deep learning networks increase in size and performance, so do associated computational costs, approaching prohibitive levels. Dendrites offer powerful nonlinear "on-The-wire"computational capabilities, increasing the expressivity of the point neuron while preserving many of the advantages of SNNs. We seek to demonstrate the potential of dendritic computations by combining them with the low-power event-driven computation of Spiking Neural Networks (SNNs) for deep learning applications. To this end, we have developed a library that adds dendritic computation to SNNs within the PyTorch framework, enabling complex deep learning networks that still retain the low power advantages of SNNs. Our library leverages a dendrite CMOS hardware model to inform the software model, which enables nonlinear computation integrated with snnTorch at scale. By leveraging dendrites in a deep learning framework, we examine the capabilities of dendrites via coincidence detection and comparison in a machine learning task with a SNN. Finally, we discuss potential deep learning applications in the context of current state-of-The-Art deep learning methods and energy-efficient neuromorphic hardware.

Dendrites enable neurons to perform nonlinear operations. Existing silicon dendrite circuits sufficiently model passive and active characteristics, but do not exploit shunting inhibition as an active mechanism. We present a dendrite circuit implemented on a reconfigurable analog platform that uses active inhibitory conductance signals to modulate the circuit's membrane potential. We explore the potential use of this circuit for direction selectivity by emulating recent observations demonstrating a role for shunting inhibition in a directionally-selective Drosophila (Fruit Fly) neuron.

Spatial navigation involves the formation of coherent representations of a map-like space, while simultaneously tracking current location in a primarily unsupervised manner. Despite a plethora of neurophysiological experiments revealing spatially-tuned neurons across the mammalian neocortex and subcortical structures, it remains unclear how such representations are acquired in the absence of explicit allocentric targets. Drawing upon the concept of predictive learning, we utilize a biologically plausible learning rule which utilizes sensory-driven observations with internally-driven expectations and learns through a contrastive manner to better predict sensory information. The local and online nature of this approach is ideal for deployment to neuromorphic hardware for edge-applications. We implement this learning rule in a network with the feedforward and feedback pathways known to be necessary for spatial navigation. After training, we find that the receptive fields of the modeled units resemble experimental findings, with allocentric and egocentric representations in the expected order along processing streams. These findings illustrate how a local and self-supervised learning method for predicting sensory information can extract latent structure from the environment.

Neuromorphic computing uses brain-inspired concepts to accelerate and efficiently execute a wide range of applications, such as mimicking biological circuits, solving NPhard optimization problems and accelerating machine learning at the edge. In particular, neuromorphic architectures to efficiently execute Spiking Neural Networks (SNNs) have gained popularity. SNNs extend artificial neural networks (ANNs) by encoding information in time as either rates or delays between spiking events, shared between neurons via their weighted connections. SNN-based platforms are event-driven, resulting in naturally sparse, noise-tolerant and power-efficient computation. In this tutorial, we present the state-of-the-art in scalable digital and analog spiking neuromorphic system architectures, and discuss current research trends within the neuromorphic architecture field at the system level. We further introduce our SANA-FE tool for Simulation of Advanced Neuromorphic Architectures for Fast Exploration, which has been developed as part of a collaboration between the University of Texas at Austin and Sandia National Laboratories. SANA-FE allows for modeling and performance-power prediction of different spiking hardware architectures executing SNN applications to support rapid, early system-level design-space exploration, hardware-aware application development and system architecture co-design. The tutorial includes a hands-on component in which SANA-FE's capabilities are demonstrated and used to perform system design and application mapping case studies.

Dendrites enable neurons to perform nonlinear operations. Existing silicon dendrite circuits sufficiently model passive and active characteristics, but do not exploit shunting inhibition as an active mechanism. We present a dendrite circuit implemented on a reconfigurable analog platform that uses active inhibitory conductance signals to modulate the circuit's membrane potential. We explore the potential use of this circuit for direction selectivity by emulating recent observations demonstrating a role for shunting inhibition in a directionally-selective Drosophila (Fruit Fly) neuron.

As Moore’s Law and Dennard Scaling come to an end, it is becoming increasingly important to develop non-von Neumann computing architectures that can perform low-power computing in the domains of scientific computing, artificial intelligence, embedded systems, and edge computing. Next-generation computing technologies, such as neuromorphic computing and quantum computing, have the potential to revolutionize computing. However, in order to make progress in these fields, it is necessary to fundamentally change the current computing paradigm by codesigning systems across all system level, from materials to software. Because skilled labor is limited in the field of next-generation computing, we are developing artificial intelligence-enhanced tools to automate the codesign and co-discovery of next-generation computers. Here, we develop a method called Modular and Multi-level MAchine Learning (MAMMAL) which is able to perform analog codesign and co-discovery across multiple system levels, spanning devices to circuits. We prototype MAMMAL by using it to design simple passive analog low-pass filters. We also explore methods to incorporate uncertainty quantification into MAMMAL and to accelerate MAMMAL by using emerging technologies, such as crossbar arrays. Ultimately, we believe that MAMMAL will enable rapid progress in developing next-generation computers by automating the codesign and co-discovery of electronic systems.

It is essential to Sandia National Laboratory’s continued success in scientific and technological advances and mission delivery to embrace a hybrid workforce culture under which current and future employees can thrive. This report focuses on the findings of the Hybrid Work Team for the Center for Computing Research, which met weekly from March to June 2023 and conducted a survey across the Center at Sandia. Conclusions in this report are drawn from the 9 authors of this report, which comprises the Hybrid Work Team, and 15 responses to a center-wide survey, as well as numerous conversations with colleagues. A major finding was widespread dissatisfaction with the quantity, execution, and tooling surrounding formal meetings with remote participants. While there was consensus that remote work enables people to produce high quality individual and technical work, there was also consensus that there was widespread social disconnect, with particular concern about hires that were made after the onset of the Covid-19 pandemic. There were many concerns about tooling and policy to facilitate remote collaboration both within Sandia and with its external collaborators. This report includes recommendations for mitigating these problems. For problems for which obvious recommendations cannot be made, ideas of what a successful solution might look like are presented.

Abstract not provided.

Recent work in neuromorphic computing has proposed a range of new architectures for Spiking Neural Network (SNN)-based systems. However, neuromorphic design lacks a framework to facilitate exploration of different SNN-based architectures and aid with early design decisions. While there are various SNN simulators, none can be used to rapidly estimate latency and energy of different spiking architectures. We show that while current spiking designs differ in implementation, they have common features which can be represented as a generic architecture template. We describe an initial version of a framework that simulates a range of neuromorphic architectures at an abstract time-step granularity. We demonstrate our simulator by modeling Intel's Loihi platform, estimating time-varying energy and latency with less than 10% mean error for various sizes of a two-layer SNN.

In this paper, we highlight how computational properties of biological dendrites can be leveraged for neuromorphic applications. Specifically, we demonstrate analog silicon dendrites that support multiplication mediated by conductance-based input in an interception model inspired by the biological dragonfly. We also demonstrate spatiotemporal pattern recognition and direction selectivity using dendrites on the Loihi neuromorphic platform. These dendritic circuits can be assembled hierarchically as building blocks for classifying complex spatiotemporal patterns.

Recent work in neuromorphic computing has proposed a range of new architectures for Spiking Neural Network (SNN)-based systems. However, neuromorphic design lacks a framework to facilitate exploration of different SNN-based architectures and aid with early design decisions. While there are various SNN simulators, none can be used to rapidly estimate latency and energy of different spiking architectures. We show that while current spiking designs differ in implementation, they have common features which can be represented as a generic architecture template. We describe an initial version of a framework that simulates a range of neuromorphic architectures at an abstract time-step granularity. We demonstrate our simulator by modeling Intel's Loihi platform, estimating time-varying energy and latency with less than 10% mean error for various sizes of a two-layer SNN.

Coordinate transformations are a fundamental operation that must be performed by any animal relying upon sensory information to interact with the external world. We present a neural network model that performs a coordinate transformation from the dragonfly eye's frame of reference to the body's frame of reference while hunting. We demonstrate that the model successfully calculates turns required for interception, and discuss how future work will compare our model with biological dragonfly neural circuitry and guide neural-inspired neuromorphic implementations of coordinate transformations.

Shunting inhibition is a potential mechanism by which biological systems multiply two time-varying signals, most recently proposed in single neurons of the fly visual system. Our work demonstrates this effect in a biological neuron model and the equivalent circuit in neuromorphic hardware modeling dendrites. We present a multi-compartment neuromorphic dendritic model that produces a multiplication-like effect using the shunting inhibition mechanism by varying leakage along the dendritic cable. Dendritic computation in neuromorphic architectures has the potential to increase complexity in single neurons and reduce the energy footprint for neural networks by enabling computation in the interconnect.

Coordinate transformations are a fundamental operation that must be performed by any animal relying upon sensory information to interact with the external world. We present a neural network model that performs a coordinate transformation from the dragonfly eye's frame of reference to the body's frame of reference while hunting. We demonstrate that the model successfully calculates turns required for interception, and discuss how future work will compare our model with biological dragonfly neural circuitry and guide neural-inspired neuromorphic implementations of coordinate transformations.

Abstract not provided.

Abstract not provided.

Abstract not provided.