Publications

Matzen, Laura E.; Divis, Kristin; Haass, Michael J.; Cronin, Deborah C.

Abstract not provided.

Matzen, Laura E.; Divis, Kristin; Cronin, Deborah C.; Haass, Michael J.

Abstract not provided.

Butler, Karin B.; Leger, Michelle A.; Bueno, Denis B.; Cueller, Christopher R.; Haass, Michael J.; Loffredo, Timothy; Reedy, Geoffrey E.; Tuminaro, Julian T.

Abstract not provided.

IEEE Transactions on Visualization and Computer Graphics

Wall, Emily; Agnihotri, Meeshu; Matzen, Laura E.; Divis, Kristin; Haass, Michael J.; Endert, Alex; Stasko, John

Recently, an approach for determining the value of a visualization was proposed, one moving beyond simple measurements of task accuracy and speed. The value equation contains components for the time savings a visualization provides, the insights and insightful questions it spurs, the overall essence of the data it conveys, and the confidence about the data and its domain it inspires. This articulation of value is purely descriptive, however, providing no actionable method of assessing a visualization's value. In this work, we create a heuristic-based evaluation methodology to accompany the value equation for assessing interactive visualizations. We refer to the methodology colloquially as ICE-T, based on an anagram of the four value components. Our approach breaks the four components down into guidelines, each of which is made up of a small set of low-level heuristics. Evaluators who have knowledge of visualization design principles then assess the visualization with respect to the heuristics. We conducted an initial trial of the methodology on three interactive visualizations of the same data set, each evaluated by 15 visualization experts. We found that the methodology showed promise, obtaining consistent ratings across the three visualizations and mirroring judgments of the utility of the visualizations by instructors of the course in which they were developed.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Butler, Karin B.; Leger, Michelle A.; Bueno, Denis B.; Cueller, Christopher R.; Haass, Michael J.; Loffredo, Timothy; Reedy, Geoffrey E.; Tuminaro, Julian T.

Vulnerability analysts protecting software lack adequate tools for understanding data flow in binaries. We present a case study in which we used human factors methods to develop a taxonomy for understanding data flow and the visual representations needed to support decision making for binary vulnerability analysis. Using an iterative process, we refined and evaluated the taxonomy by generating three different data flow visualizations for small binaries, trained an analyst to use these visualizations, and tested the utility of the visualizations for answering data flow questions. Throughout the process and with minimal training, analysts were able to use the visualizations to understand data flow related to security assessment. Our results indicate that the data flow taxonomy is promising as a mechanism for improving analyst understanding of data flow in binaries and for supporting efficient decision making during analysis.

Stevenson, Sarah R.; Stevenson, Sarah R.; Delker, Collin J.; Haass, Michael J.

Abstract not provided.

IEEE Transactions on Visualization and Computer Graphics

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin; Wang, Zhiyuan; Wilson, Andrew T.

Evaluating the effectiveness of data visualizations is a challenging undertaking and often relies on one-off studies that test a visualization in the context of one specific task. Researchers across the fields of data science, visualization, and human-computer interaction are calling for foundational tools and principles that could be applied to assessing the effectiveness of data visualizations in a more rapid and generalizable manner. One possibility for such a tool is a model of visual saliency for data visualizations. Visual saliency models are typically based on the properties of the human visual cortex and predict which areas of a scene have visual features (e.g. color, luminance, edges) that are likely to draw a viewer's attention. While these models can accurately predict where viewers will look in a natural scene, they typically do not perform well for abstract data visualizations. In this paper, we discuss the reasons for the poor performance of existing saliency models when applied to data visualizations. We introduce the Data Visualization Saliency (DVS) model, a saliency model tailored to address some of these weaknesses, and we test the performance of the DVS model and existing saliency models by comparing the saliency maps produced by the models to eye tracking data obtained from human viewers. Finally, we describe how modified saliency models could be used as general tools for assessing the effectiveness of visualizations, including the strengths and weaknesses of this approach.

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin; Wang, Zhiyuan W.; Wilson, Andrew T.

Abstract not provided.

Wilson, Andrew T.; Divis, Kristin; Matzen, Laura E.; Haass, Michael J.

Abstract not provided.

Haass, Michael J.; Matzen, Laura E.; Divis, Kristin; Wilson, Andrew T.

Abstract not provided.

Matzen, Laura E.; Divis, Kristin; Haass, Michael J.

Abstract not provided.

Smartt, Heidi A.; Gastelum, Zoe N.; Matzen, Laura E.; Smartt, Heidi A.; Horak, Karl E.; Solodov, Alexander A.; Haass, Michael J.

Abstract not provided.

Haass, Michael J.; Matzen, Laura E.; Wilson, Andrew T.; Divis, Kristin

Abstract not provided.

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin; Stites, Mallory C.

Abstract not provided.

McNamara, Laura A.; Bauer, Travis L.; Haass, Michael J.; Matzen, Laura E.

Abstract not provided.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Matzen, Laura E.; Haass, Michael J.; Divis, Kristin; Stites, Mallory C.

Data visualizations are used to communicate information to people in a wide variety of contexts, but few tools are available to help visualization designers evaluate the effectiveness of their designs. Visual saliency maps that predict which regions of an image are likely to draw the viewer’s attention could be a useful evaluation tool, but existing models of visual saliency often make poor predictions for abstract data visualizations. These models do not take into account the importance of features like text in visualizations, which may lead to inaccurate saliency maps. In this paper we use data from two eye tracking experiments to investigate attention to text in data visualizations. The data sets were collected under two different task conditions: a memory task and a free viewing task. Across both tasks, the text elements in the visualizations consistently drew attention, especially during early stages of viewing. These findings highlight the need to incorporate additional features into saliency models that will be applied to visualizations.

Memory and Cognition

Matzen, Laura E.; Trumbo, Michael C.; Haass, Michael J.; Hunter, Michael A.; Silva, Austin R.; Adams, Susan S.; Bunting, Michael F.; O’Rourke, Polly

There is a great deal of debate concerning the benefits of working memory (WM) training and whether that training can transfer to other tasks. Although a consistent finding is that WM training programs elicit a short-term near-transfer effect (i.e., improvement in WM skills), results are inconsistent when considering persistence of such improvement and far transfer effects. In this study, we compared three groups of participants: a group that received WM training, a group that received training on how to use a mental imagery memory strategy, and a control group that received no training. Although the WM training group improved on the trained task, their posttraining performance on nontrained WM tasks did not differ from that of the other two groups. In addition, although the imagery training group’s performance on a recognition memory task increased after training, the WM training group’s performance on the task decreased after training. Participants’ descriptions of the strategies they used to remember the studied items indicated that WM training may lead people to adopt memory strategies that are less effective for other types of memory tasks. These results indicate that WM training may have unintended consequences for other types of memory performance.

McNamara, Laura A.; Bauer, Travis L.; Haass, Michael J.; Matzen, Laura E.

Abstract not provided.

McNamara, Laura A.; Bauer, Travis L.; Haass, Michael J.; Matzen, Laura E.

In the context of text-based analysis workflows, we propose that an effective analytic tool facilitates triage by a) enabling users to identify and set aside irrelevant content (i.e., reduce the complexity of information in a dataset) and b) develop a working mental model of which items are most relevant to the question at hand. This LDRD funded research developed a dataset that is enabling this team to evaluate propose normalized compression distance (NCD) as a task, user, and context-insensitive measure of categorization outcomes (Shannon entropy is reduced as order is imposed). Effective analytics tools help people impose order, reducing complexity in measurable ways. Our concept and research was documented in a paper accepted to the ACM conference Beyond Time and Error: Novel Methods in Information Visualization Evaluation , part of the IEEE VisWeek Conference, Baltimore, MD, October 16-21, 2016. The paper is included as an appendix to this report.

Burnham, Laurie B.; Stamber, Kevin L.; Jeffers, Robert F.; Adams, Susan S.; Verzi, Stephen J.; Sahakian, Meghan A.; Haass, Michael J.; Cauthen, Katherine R.

The transformation of the distribution grid from a centralized to decentralized architecture, with bi-directional power and data flows, is made possible by a surge in network intelligence and grid automation. While changes are largely beneficial, the interface between grid operator and automated technologies is not well understood, nor are the benefits and risks of automation. Quantifying and understanding the latter is an important facet of grid resilience that needs to be fully investigated. The work described in this document represents the first empirical study aimed at identifying and mitigating the vulnerabilities posed by automation for a grid that for the foreseeable future will remain a human-in-the-loop critical infrastructure. Our scenario-based methodology enabled us to conduct a series of experimental studies to identify causal relationships between grid-operator performance and automated technologies and to collect measurements of human performance as a function of automation. Our findings, though preliminary, suggest there are predictive patterns in the interplay between human operators and automation, patterns that can inform the rollout of distribution automation and the hiring and training of operators, and contribute in multiple and significant ways to the field of grid resilience.

Trumbo, Michael C.; Armenta, Mika; Haass, Michael J.; Butler, Karin B.; Jones, Aaron P.; Robinson, Charles S.

Abstract not provided.

Haass, Michael J.; Matzen, Laura E.; Wilson, Andrew T.; Divis, Kristin

Abstract not provided.

Haass, Michael J.; Matzen, Laura E.; Wilson, Andrew T.; Divis, Kristin; Armenta, Mika

Abstract not provided.

Eye Tracking Research and Applications Symposium (ETRA)

Haass, Michael J.; Matzen, Laura E.; Butler, Karin B.; Armenta, Mika

From the seminal work of Yarbus [1967] on the relationship of eye movements to vision, scanpath analysis has been recognized as a window into the mind. Computationally, characterizing the scanpath, the sequential and spatial dependencies between eye positions, has been demanding. We sought a method that could extract scanpath trajectory information from raw eye movement data without assumptions defining fixations and regions of interest. We adapted a set of libraries that perform multidimensional clustering on geometric features derived from large volumes of spatiotemporal data to eye movement data in an approach we call GazeAppraise. To validate the capabilities of GazeAppraise for scanpath analysis, we collected eye tracking data from 41 participants while they completed four smooth pursuit tracking tasks. Unsupervised cluster analysis on the features revealed that 162 of 164 recorded scanpaths were categorized into one of four clusters and the remaining two scanpaths were not categorized (recall/sensitivity=98.8%). All of the categorized scanpaths were grouped only with other scanpaths elicited by the same task (precision=100%). GazeAppraise offers a unique approach to the categorization of scanpaths that may be particularly useful in dynamic environments and in visual search tasks requiring systematic search strategies.

Haass, Michael J.; Matzen, Laura E.; Butler, Karin B.; Armenta, Mika

Abstract not provided.