Publications

Haass, Michael J.; Matzen, Laura E.; Wilson, Andrew T.; Divis, Kristin; Armenta, Mika; McNamara, Laura A.

Abstract not provided.

McNamara, Laura A.; Stracuzzi, David J.; Czuchlewski, Kristina R.

Abstract not provided.

Matzen, Laura E.; Haass, Michael J.; Silva, Austin R.; Trumbo, Michael C.; Adams, Susan S.; McNamara, Laura A.; Speed, Ann S.

Abstract not provided.

Brost, Randolph B.; Chow, James G.; Rintoul, Mark D.; McNamara, Laura A.; Stracuzzi, David J.; Wilson, Andrew T.; Moya, Mary M.

Abstract not provided.

McNamara, Laura A.; Phillips, Cynthia A.; Wilson, Andrew T.

Abstract not provided.

McNamara, Laura A.; Stubblefield, W.A.

Abstract not provided.

Stubblefield, W.A.; McNamara, Laura A.; Shepherd, Jason F.

Abstract not provided.

Stubblefield, W.A.; McNamara, Laura A.; Trucano, Timothy G.; Shepherd, Jason F.

Abstract not provided.

McNamara, Laura A.; Trucano, Timothy G.; Backus, George A.; Mitchell, Scott A.

Sandia National Laboratories is investing in projects that aim to develop computational modeling and simulation applications that explore human cognitive and social phenomena. While some of these modeling and simulation projects are explicitly research oriented, others are intended to support or provide insight for people involved in high consequence decision-making. This raises the issue of how to evaluate computational modeling and simulation applications in both research and applied settings where human behavior is the focus of the model: when is a simulation 'good enough' for the goals its designers want to achieve? In this report, we discuss two years' worth of review and assessment of the ASC program's approach to computational model verification and validation, uncertainty quantification, and decision making. We present a framework that extends the principles of the ASC approach into the area of computational social and cognitive modeling and simulation. In doing so, we argue that the potential for evaluation is a function of how the modeling and simulation software will be used in a particular setting. In making this argument, we move from strict, engineering and physics oriented approaches to V&V to a broader project of model evaluation, which asserts that the systematic, rigorous, and transparent accumulation of evidence about a model's performance under conditions of uncertainty is a reasonable and necessary goal for model evaluation, regardless of discipline. How to achieve the accumulation of evidence in areas outside physics and engineering is a significant research challenge, but one that requires addressing as modeling and simulation tools move out of research laboratories and into the hands of decision makers. This report provides an assessment of our thinking on ASC Verification and Validation, and argues for further extending V&V research in the physical and engineering sciences toward a broader program of model evaluation in situations of high consequence decision-making.

McNamara, Laura A.; Trucano, Timothy G.; Backus, George A.

Since 1998, the Department of Energy/NNSA National Laboratories have invested millions in strategies for assessing the credibility of computational science and engineering (CSE) models used in high consequence decision making. The answer? There is no answer. There's a process--and a lot of politics. The importance of model evaluation (verification, validation, uncertainty quantification, and assessment) increases in direct proportion to the significance of the model as input to a decision. Other fields, including computational social science, can learn from the experience of the national laboratories. Some implications for evaluating 'low cognition agents'. Epistemology considers the question, How do we know what we [think we] know? What makes Western science special in producing reliable, predictive knowledge about the world? V&V takes epistemology out of the realm of thought and puts it into practice. What is the role of modeling and simulation in the production of reliable, credible scientific knowledge about the world? What steps, investments, practices do I pursue to convince myself that the model I have developed is producing credible knowledge?

Boslough, Mark B.; Sprigg, James A.; Backus, George A.; Taylor, Mark A.; McNamara, Laura A.; Murphy, Kathryn M.; Malczynski, Leonard A.

This white paper represents a summary of work intended to lay the foundation for development of a climatological/agent model of climate-induced conflict. The paper combines several loosely-coupled efforts and is the final report for a four-month late-start Laboratory Directed Research and Development (LDRD) project funded by the Advanced Concepts Group (ACG). The project involved contributions by many participants having diverse areas of expertise, with the common goal of learning how to tie together the physical and human causes and consequences of climate change. We performed a review of relevant literature on conflict arising from environmental scarcity. Rather than simply reviewing the previous work, we actively collected data from the referenced sources, reproduced some of the work, and explored alternative models. We used the unfolding crisis in Darfur (western Sudan) as a case study of conflict related to or triggered by climate change, and as an exercise for developing a preliminary concept map. We also outlined a plan for implementing agents in a climate model and defined a logical progression toward the ultimate goal of running both types of models simultaneously in a two-way feedback mode, where the behavior of agents influences the climate and climate change affects the agents. Finally, we offer some ''lessons learned'' in attempting to keep a diverse and geographically dispersed group working together by using Web-based collaborative tools.