Publications

Chen, Maximillian G.; Stracuzzi, David J.; Darling, Michael C.

Abstract not provided.

Chen, Maximillian G.; Stracuzzi, David J.; Darling, Michael C.

Abstract not provided.

Darling, Michael C.; Hush, Don R.; Stracuzzi, David J.

Abstract not provided.

Stracuzzi, David J.

Abstract not provided.

Stracuzzi, David J.

Abstract not provided.

Stracuzzi, David J.

Abstract not provided.

Corley, Courtney C.; Stracuzzi, David J.

Abstract not provided.

Descour, Michael R.; Tsao, Jeffrey Y.; Stracuzzi, David J.; Wakeland, Anna K.; Schultz, David R.; Smith, William S.; Weeks, Jacquilyn A.

On April 6-8, 2021, Sandia National Laboratories hosted a virtual workshop to explore the potential for developing AI-Enhanced Co-Design for Next-Generation Microelectronics (AICoM). The workshop brought together two themes. The first theme was articulated in the 2018 Department of Energy Office of Science (DOE SC) “Basic Research Needs for Microelectronics” (BRN) report, which called for a “fundamental rethinking” of the traditional design approach to microelectronics, in which subject matter experts (SMEs) in each microelectronics discipline (materials, devices, circuits, algorithms, etc.) work near-independently. Instead, the BRN called for a non-hierarchical, egalitarian vision of co-design, wherein “each scientific discipline informs and engages the others” in “parallel but intimately networked efforts to create radically new capabilities.” The second theme was the recognition of the continuing breakthroughs in artificial intelligence (AI) that are currently enhancing and accelerating the solution of traditional design problems in materials science, circuit design, and electronic design automation (EDA).

Peterson, Kara J.; Parks, Michael L.; Ackerman, Eric A.; Bambha, Ray B.; Bull, Diana L.; Frederick, Jennifer M.; Hardesty, Jasper O.; Ilgen, Anastasia G.; Jakeman, John D.; Powell, Amy J.; Peterson, Matthew G.; Roesler, Erika L.; Safta, Cosmin S.; Stracuzzi, David J.; Kalashnikova, Irina

Abstract not provided.

McNamara, Laura A.; Stracuzzi, David J.; Czuchlewski, Kristina R.

Abstract not provided.

Peterson, Matthew G.; Stracuzzi, David J.; Vollmer, Charles V.

Abstract not provided.

Stracuzzi, David J.

Abstract not provided.

Nichol, Jeffrey N.; Peterson, Matthew G.; Peterson, Kara J.; Stracuzzi, David J.

Abstract not provided.

Statistical Analysis and Data Mining

Stracuzzi, David J.; Brost, Randolph B.; Phillips, Cynthia A.; Robinson, David G.; Wilson, Alyson G.; Woodbridge, Diane W.

Geospatial semantic graphs provide a robust foundation for representing and analyzing remote sensor data. In particular, they support a variety of pattern search operations that capture the spatial and temporal relationships among the objects and events in the data. However, in the presence of large data corpora, even a carefully constructed search query may return a large number of unintended matches. This work considers the problem of calculating a quality score for each match to the query, given that the underlying data are uncertain. We present a preliminary evaluation of three methods for determining both match quality scores and associated uncertainty bounds, illustrated in the context of an example based on overhead imagery data.

Stracuzzi, David J.; Darling, Michael C.; Chen, Maximillian G.; Peterson, Matthew G.

Abstract not provided.

Proceedings of SPIE - The International Society for Optical Engineering

Stracuzzi, David J.; Darling, Michael C.; Chen, Maximillian G.; Peterson, Matthew G.

We discuss uncertainty quantification in multisensor data integration and analysis, including estimation methods and the role of uncertainty in decision making and trust in automated analytics. The challenges associated with automatically aggregating information across multiple images, identifying subtle contextual cues, and detecting small changes in noisy activity patterns are well-established in the intelligence, surveillance, and reconnaissance (ISR) community. In practice, such questions cannot be adequately addressed with discrete counting, hard classifications, or yes/no answers. For a variety of reasons ranging from data quality to modeling assumptions to inadequate definitions of what constitutes "interesting" activity, variability is inherent in the output of automated analytics, yet it is rarely reported. Consideration of these uncertainties can provide nuance to automated analyses and engender trust in their results. In this work, we assert the importance of uncertainty quantification for automated data analytics and outline a research agenda. We begin by defining uncertainty in the context of machine learning and statistical data analysis, identify its sources, and motivate the importance and impact of its quantification. We then illustrate these issues and discuss methods for data-driven uncertainty quantification in the context of a multi-source image analysis example. We conclude by identifying several specific research issues and by discussing the potential long-term implications of uncertainty quantification for data analytics, including sensor tasking and analyst trust in automated analytics.

Stracuzzi, David J.; Chen, Maximillian G.; Darling, Michael C.; Dauphin, Stephen M.; Peterson, Matthew G.; Vollmer, Charles V.

Abstract not provided.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Speed, Ann S.; Silva, Austin R.; Trumbo, Derek T.; Stracuzzi, David J.; Warrender, Christina E.; Trumbo, Michael; Divis, Kristin

The Transportation Security Administration has a large workforce of Transportation Security Officers, most of whom perform interrogation of x-ray images at the passenger checkpoint. To date, TSOs on the x-ray have been limited to a 30-min session at a time, however, it is unclear where this limit originated. The current paper outlines methods for empirically determining if that 30-min duty cycle is optimal and if there are differences between individual TSOs. This work can inform scheduling TSOs at the checkpoint and can also inform whether TSOs should continue to be cross-trained (i.e., performing all 6 checkpoint duties) or whether specialization makes more sense.

Speed, Ann S.; Silva, Austin R.; Trumbo, Derek T.; Trumbo, Michael C.; Stracuzzi, David J.; Warrender, Christina E.; Divis, Kristin

Abstract not provided.

Chen, Maximillian G.; Stracuzzi, David J.; Darling, Michael C.; Peterson, Matthew G.

Abstract not provided.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Stracuzzi, David J.; Speed, Ann S.; Silva, Austin R.; Haass, Michael J.; Trumbo, Derek T.

Visual search data describe people’s performance on the common perceptual problem of identifying target objects in a complex scene. Technological advances in areas such as eye tracking now provide researchers with a wealth of data not previously available. The goal of this work is to support researchers in analyzing this complex and multimodal data and in developing new insights into visual search techniques. We discuss several methods drawn from the statistics and machine learning literature for integrating visual search data derived from multiple sources and performing exploratory data analysis. We ground our discussion in a specific task performed by officers at the Transportation Security Administration and consider the applicability, likely issues, and possible adaptations of several candidate analysis methods.

Stracuzzi, David J.; Speed, Ann S.; Haass, Michael J.; Silva, Austin R.; Trumbo, Derek T.

Abstract not provided.

Leger, Michelle A.; Darling, Michael C.; Jones, Stephen T.; Matzen, Laura E.; Stracuzzi, David J.; Wilson, Andrew T.; Bueno, Denis B.; Christentsen, Matthew C.; Ginaldi, Melissa J.; Hannasch, David A.; Heidbrink, Scott H.; Howell, Breannan C.; Leger, Chris; Reedy, Geoffrey E.; Rogers, Alisa N.; Williams, Jack A.

Reverse engineering (RE) analysts struggle to address critical questions about the safety of binary code accurately and promptly, and their supporting program analysis tools are simply wrong sometimes. The analysis tools have to approximate in order to provide any information at all, but this means that they introduce uncertainty into their results. And those uncertainties chain from analysis to analysis. We hypothesize that exposing sources, impacts, and control of uncertainty to human binary analysts will allow the analysts to approach their hardest problems with high-powered analytic techniques that they know when to trust. Combining expertise in binary analysis algorithms, human cognition, uncertainty quantification, verification and validation, and visualization, we pursue research that should benefit binary software analysis efforts across the board. We find a strong analogy between RE and exploratory data analysis (EDA); we begin to characterize sources and types of uncertainty found in practice in RE (both in the process and in supporting analyses); we explore a domain-specific focus on uncertainty in pointer analysis, showing that more precise models do help analysts answer small information flow questions faster and more accurately; and we test a general population with domain-general sudoku problems, showing that adding "knobs" to an analysis does not significantly slow down performance. This document describes our explorations in uncertainty in binary analysis.

Nichol, Jeffrey N.; Peterson, Matthew G.; Peterson, Kara J.; Stracuzzi, David J.; Fricke, Matthew F.; Moses, Melanie M.

Abstract not provided.

Stracuzzi, David J.; Chen, Maximillian G.; Darling, Michael C.; Peterson, Matthew G.

Abstract not provided.