In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.
We present a deep learning image reconstruction method called AirNet-SNL for sparse view computed tomography. It combines iterative reconstruction and convolutional neural networks with end-to-end training. Our model reduces streak artifacts from filtered back-projection with limited data, and it trains on randomly generated shapes. This work shows promise to generalize learning image reconstruction.
X-ray phase contrast imaging (XPCI) is a nondestructive evaluation technique that enables high-contrast detection of low-attenuation materials that are largely transparent in traditional radiography. Extending a grating-based Talbot-Lau XPCI system to three-dimensional imaging with computed tomography (CT) imposes two motion requirements: the analyzer grating must translate transverse to the optical axis to capture image sets for XPCI reconstruction, and the sample must rotate to capture angular data for CT reconstruction. The acquisition algorithm choice determines the order of movement and positioning of the two stages. The choice of the image acquisition algorithm for XPCI CT is instrumental to collecting high fidelity data for reconstruction. We investigate how data acquisition influences XPCI CT by comparing two simple data acquisition algorithms and determine that capturing a full phase-stepping image set for a CT projection before rotating the sample results in higher quality data.
Many optical systems are used for specific tasks such as classification. Of these systems, the majority are designed to maximize image quality for human observers. However, machine learning classification algorithms do not require the same data representation used by humans. We investigate the compressive optical systems optimized for a specific machine sensing task. Two compressive optical architectures are examined: an array of prisms and neutral density filters where each prism and neutral density filter pair realizes one datum from an optimized compressive sensing matrix, and another architecture using conventional optics to image the aperture onto the detector, a prism array to divide the aperture, and a pixelated attenuation mask in the intermediate image plane. We discuss the design, simulation, and trade-offs of these systems built for compressed classification of the Modified National Institute of Standards and Technology dataset. Both architectures achieve classification accuracies within 3% of the optimized sensing matrix for compression ranging from 98.85% to 99.87%. The performance of the systems with 98.85% compression were between an F / 2 and F / 4 imaging system in the presence of noise.
Images are often not the optimal data form to perform machine learning tasks such as scene classification. Compressive classification can reduce the size, weight, and power of a system by selecting the minimum information while maximizing classification accuracy.In this work we present designs and simulations of prism arrays which realize sensing matrices using a monolithic element. The sensing matrix is optimized using a neural network architecture to maximize classification accuracy of the MNIST dataset while considering the blurring caused by the size of each prism. Simulated optical hardware performance for a range of prism sizes are reported.
Sandia National Laboratories is developing a laboratory-based x-ray phase contrast imaging (XPCI) computed tomography (CT) system. This system utilizes a Talbot-Lau interferometer based on in-house fabricated gratings and a conventional x-ray system. Initial work has focused on adding CT capabilities to a 28 keV XPCI system. A new set of gratings tuned for an x-ray energy of 100 keV is being developed. This new grating set will facilitate imaging denser components. System configuration details will be presented as well as a discussion of the challenges associated with building an XPCI CT system. Additionally, initial imaging results will be presented.
