The Method of Manufactured Universes for Testing Uncertainty Quantification Methods
Hayes Stripling, Texas A&M University
This poster will present the Method of Manufactured Universes (MMU) as a framework for testing the predictive capability of a given Uncertainty Quantification (UQ) technique. The framework calls for a manufactured reality and attempts to approximate this reality with an uncertain computational model. A UQ method is then applied to the data and, because the modeler manufactured the universe, the predictions made by the UQ model can be directly tested amidst a number of imposed uncertainties. This project is motivated by the need to understand the statistical assumptions and models that are embedded in most UQ methods and how well these methods are able to adapt to a certain set of physics. The first universe is a particle transport problem in which we measure partial currents at the edge of a 1D slab. Two UQ models were applied to this initial universe. The first was a Gaussian Process Model from LANL, which is a multivariate implementation of the Kennedy-O'Hagan (2001) method. The second method is a Bayesian Multivariate Adaptive Regression Splines (BMARS) algorithm, which we combined with a filtering/weighting technique to refine the uncertain input space. We found that both methods were able to calibrate the approximate model to more accurately predict `experimental results'. However, we also discovered that the assumptions within the Gaussian Process code limited its predictive capability, especially in simulated `black box' implementations. The overall conclusion of the presentation is that MMU is a useful framework for developing a UQ strategy around a given problem. The method allows for improved understanding of statistical techniques and insight into the strengths, weaknesses, and embedded assumptions in the myriad methods and techniques often employed in predictive science.
Abstract Author(s): Stripling, H.F. and Adams, M.L. and McClarren, R.G. and Mallick, B.K.