Uncertainty Quantification

Public policy seeks to influence complex natural, social, and engineered systems to achieve desired outcomes. Effective public policies are those which combine good outcomes with high reliability such that their choice is robust to a wide range of possible uncertainties. Modeling these complex systems and their potential response to proposed policies can provide decision-makers with an objective basis for policy design.

• Goal /Aspiration for Capability Development
• Develop rigorous methods to evaluate and rank modeled policy effectiveness in context of model uncertainty
• Provide metrics and information to bridge the different levels of focus that characterize complexity modelers and policy makers
• Provide the information needed to classify potential public policy options based on both their outcomes and their risk of failure
• Approach/Methods/Models
• Combination of space-filling experimental designs, modeling policy options as continuous rather than categorical variables, and treed Gaussian process and polynomial chaos expansion-based sensitivity analysis are projected to yield a straightforward ranking of policy options that is robust to the identified aleatory and epistemic model uncertainties
• Define policy inputs as numerical ranges rather than categorical choices whenever possible
• Run simple parameter scans to get a feel for effects
• Run near-orthogonal Latin hypercube space-filling design on small sets of runs (n = ~200)
• Document sensitivity and interactions with meta-models
• Trace uncertainty from sources to results
• Apply uncertainty to rank policy options
• Look for interesting peaks and troughs in state space and distributions
• Use uncertainty to guide further refinement
• Status, Accomplishments and Next Steps
• Phase 2 work underway now addresses the integration of rigorous uncertainty quantification to complex adaptive system models to enable robust decision making for public health policy questions
• Acknowledgements
• This development was funded in part by the Office of Public Health and Environmental Hazards (OPHEH) in the Veterans Health Administration (VHA) of the Department of Veterans Affairs (VA)
• Capability Documentation
• 8th International Conference on Complex Systems, June 2011, Quincy, MA (ICCS 2011 Proceedings available for download)
• Integrating Uncertainty Analysis into Complex-System Modeling to Design Effective Public Policy I: Preliminary Findings, Patrick D. Finley, Robert J. Glass, Thomas W. Moore, Arlo L. Ames, Leland B. Evans, Daniel C. Cannon, Victoria J. Davey (paper)
• Integrating Uncertainty Analysis into Complex-System Modeling for Public Policy Design, Patrick D. Finley, Robert J. Glass, Victoria J. Davey, Thomas W. Moore, Walter E. Beyeler (presentation)