This report summarizes the work performed under the Sandia LDRD project "Eyes on the Ground: Visual Verification for On-Site Inspection." The goal of the project was to develop methods and tools to assist an IAEA inspector in assessing visual and other information encountered during an inspection. Effective IAEA inspections are key to verifying states' compliance with nuclear non-proliferation treaties. In the course of this work we developed a taxonomy of candidate inspector assistance tasks, selected key tasks to focus on, identified hardware and software solution approaches, and made progress in implementing them. In particular, we demonstrated the use of multiple types of 3-d scanning technology applied to simulated inspection environments, and implemented a preliminary prototype of a novel inspector assistance tool. This report summarizes the project's major accomplishments, and gathers the abstracts and references for the publication and reports that were prepared as part of this work. We then describe work in progress that is not yet ready for publication. Approved for public release; further dissemination unlimited.
The goal of the Eyes On the Ground project is to develop tools to aid IAEA inspectors. Our original vision was to produce a tool that would take three-dimensional measurements of an unknown piece of equipment, construct a semantic representation of the measured object, and then use the resulting data to infer possible explanations of equipment function. We report our tests of a 3-d laser scanner to obtain 3-d point cloud data, and subsequent tests of software to convert the resulting point clouds into primitive geometric objects such as planes and cylinders. These tests successfully identified pipes of moderate diameter and planar surfaces, but also incurred significant noise. We also investigated the IAEA inspector task context, and learned that task constraints may present significant obstacles to using 3-d laser scanners. We further learned that equipment scale and enclosing cases may confound our original goal of equipment diagnosis. Meanwhile, we also surveyed the rapidly evolving field of 3-d measurement technology, and identified alternative sensor modalities that may prove more suitable for inspector use in a safeguards context. We conclude with a detailed discussion of lessons learned and the resulting implications for project goals. Approved for public release; further dissemination unlimited.
A previous report assesses our progress to date on the Eyes On the Ground project, and reviews lessons learned. In this report, we address the implications of those lessons in defining the most productive path forward for the remainder of the project. We propose two main concepts: Interactive Diagnosis and Model-Driven Assistance. Among these, the Model-Driven Assistance concept appears the most promising. The Model-Driven Assistance concept is based on an approximate but useful model of a facility, which provides a unified representation for storing, viewing, and analyzing data that is known about the facility. This representation provides value to both inspectors and IAEA headquarters, and facilitates communication between the two. The concept further includes a lightweight, portable field tool to aid the inspector in executing a variety of inspection tasks, including capture of images and 3-d scan data. We develop a detailed description of this concept, including its system components, functionality, and example use cases. The envisioned tool would provide value by reducing inspector cognitive load, streamlining inspection tasks, and facilitating communication between the inspector and teams at IAEA headquarters. We conclude by enumerating the top implementation priorities to pursue in the remaining limited time of the project. Approved for public release; further dissemination unlimited.
We address the problem of wide-area search of overhead imagery. Given a time sequence of overhead images, we construct a geospatial-temporal semantic graph, which expresses the complex continuous information in the overhead images in a discrete searchable form, including explicit modeling of changes seen from one image to the next. We can then express desired search goals as a template graph, and search for matches using simple and efficient graph search algorithms. This produces a set of potential matches which provide cues for where to examine the imagery in detail, applying human expertise to determine which matches are correct. We include a match quality metric that scores the matches according to how well they match the stated search goal. This enables matches to be presented in sorted order with the best matches first, similar to the results returned by a web search engine. We present an evaluation of the method applied to several examples and data sets, and show that it can be used successfully for some problems. We also remark on several limitations of the method and note additional work needed to improve its scope and robustness. Approved for public release; further dissemination unlimited.
We assess how geospatial-temporal semantic graphs and our GeoGraphy code implementation might contribute to induced seismicity analysis. We focus on evaluating strengths and weaknesses of both 1) the fundamental concept of semantic graphs and 2) our current code implementation. With extensions and research effort, code implementation limitations can be overcome. The paper also describes relevance including possible data input types, expected analytical outcomes and how it can pair with other approaches and fit into a workflow.
Remote sensing systems produce large volumes of high-resolution images that are difficult to search. The GeoGraphy (pronounced Geo-Graph-y) framework [2, 20] encodes remote sensing imagery into a geospatial-temporal semantic graph representation to enable high level semantic searches to be performed. Typically scene objects such as buildings and trees tend to be shaped like blocks with few holes, but other shapes generated from path networks tend to have a large number of holes and can span a large geographic region due to their connectedness. For example, we have a dataset covering the city of Philadelphia in which there is a single road network node spanning a 6 mile x 8 mile region. Even a simple question such as "find two houses near the same street" might give unexpected results. More generally, nodes arising from networks of paths (roads, sidewalks, trails, etc.) require additional processing to make them useful for searches in GeoGraphy. We have assigned the term Path Network Recovery to this process. Path Network Recovery is a three-step process involving (1) partitioning the network node into segments, (2) repairing broken path segments interrupted by occlusions or sensor noise, and (3) adding path-aware search semantics into GeoQuestions. This report covers the path network recovery process, how it is used, and some example use cases of the current capabilities.
Geospatial semantic graphs provide a robust foundation for representing and analyzing remote sensor data. In particular, they support a variety of pattern search operations that capture the spatial and temporal relationships among the objects and events in the data. However, in the presence of large data corpora, even a carefully constructed search query may return a large number of unintended matches. This work considers the problem of calculating a quality score for each match to the query, given that the underlying data are uncertain. We present a preliminary evaluation of three methods for determining both match quality scores and associated uncertainty bounds, illustrated in the context of an example based on overhead imagery data.
This report summarizes preliminary research into uncertainty quantification for pattern ana- lytics within the context of the Pattern Analytics to Support High-Performance Exploitation and Reasoning (PANTHER) project. The primary focus of PANTHER was to make large quantities of remote sensing data searchable by analysts. The work described in this re- port adds nuance to both the initial data preparation steps and the search process. Search queries are transformed from does the specified pattern exist in the data? to how certain is the system that the returned results match the query? We show example results for both data processing and search, and discuss a number of possible improvements for each.
