Uncertainty quantification

Computer models (also known as process models, mechanistic models, simulation models) are used widely throughout science and engineering for making predictions, and for conducting ‘virtual experiments’ when physical experiments would be too costly or impractical.

There will almost always be uncertainty in any model prediction, caused by uncertainty about what input values to use, and/or uncertainty about how well the model represents reality. We cannot trust a computer model prediction until we have quantified the uncertainty properly.

My interest in this topic began with my PhD, which was on propagating input uncertainty through computationally expensive models, using Gaussian process emulators. My current interests are at the interface of UQ and expert elicitation: using elicitation to quantify model input uncertainty, and understanding model input sensitivities to inform elicitation exercises.

Papers on uncertainty quantification

Coveney, S., Corrado, C., Oakley, J. E., Wilkinson, R. D., Niederer, S. A. and Clayton, R. H. (2021) Bayesian calibration of electrophysiology models using restitution curve emulators. Frontiers in Physiology.

Coveney, S., Corrado, C., Roney, C. H., O’Hare, D., Williams, S. E., O’Neill, M. D., Niederer, S. A., Clayton R. H., Oakley, J. E. and Wilkinson, R. D. (2020) Gaussian process manifold interpolation for probabilistic atrial activation maps and uncertain conduction velocity. Phil. Trans. R. Soc. A. 378.
DOI: http://doi.org/10.1098/rsta.2019.0345

Coveney, S., Corrado, C., Roney, C., Wilkinson, R., Oakley, J., Lindgren, F., Williams, S., O’Neill, M., Niederer, S. and Clayton, R. (2020) Probabilistic interpolation of uncertain local activation times on human atrial manifolds. IEEE Transactions on Biomedical Engineering, 67(1), pp. 99-109.
DOI: 10.1109/TBME.2019.2908486

McKinley, T. J., Vernon, I, Andrianakis, I, McCreesh, N, Oakley, J. E., Nsubuga, R. N., Goldstein, M. and White, R. G. (2018) Approximate Bayesian computation and simulation-based inference for complex stochastic epidemic models. Statist. Sci. 33(1), 4–18.

Andrianakis, I., McCreesh, N., Vernon, I., McKinley, T. J., Oakley, J. E., Nsubuga, R. N., Goldstein, M. and White, R. G. (2017) Efficient history matching of a high dimensional individual-based HIV transmission model. SIAM/ASA J. Uncertainty Quantification, 5(1), 694–719.

Oakley, J. E. and Youngman, B. D. (2017) Calibration of stochastic computer simulators using likelihood emulation. Technometrics, 59(1), 80-92.

Andrianakis, I. , Vernon, I. , McCreesh, N. , McKinley, T. J., Oakley, J. E., Nsubuga, R. N., Goldstein, M. and White, R. G. (2017) History matching of a complex epidemiological model of human immunodeficiency virus transmission by using variance emulation. J. R. Stat. Soc. C, 66, 717-740.
DOI: 10.1111/rssc.12198

Strong, M., Oakley, J. E., Brennan, A. and Breeze, P. (2015) Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: A fast nonparametric regression-based method. Medical Decision Making, 35(5), 570-83.

Andrianakis, I., Vernon, I. R., McCreesh, N., McKinley, T. J., Oakley, J. E. et al. (2015) Bayesian history matching of complex infectious disease models using emulation: A tutorial and a case study on HIV in Uganda. PLoS Comput. Biol., 11(1): e1003968.
DOI: 10.1371/journal.pcbi.1003968

Strong, M. and Oakley, J. E. (2014) When is a model good enough? Deriving the expected value of model improvement via specifying internal model discrepancies. SIAM/ASA Journal on Uncertainty Quantification, 2(1), 106-125.

Strong, M., Oakley, J. E. and Brennan, A. (2014) Estimating multi-parameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: A non-parametric regression approach. Medical Decision Making, 34(3), 311-26.

Strong, M. and Oakley, J. E. (2013) An efficient method for computing single parameter partial expected value of perfect information. Medical Decision Making, 33, 755-766.

Fricker, T. E., Oakley, J. E. and Urban, N. M. (2013) Multivariate Gaussian process emulators with non-separable covariance structures. Technometrics, 55(1), 47-56.

Becker, W., Oakley, J. E., Surace, C., Gili, P., Rowson, J. and Worden, K. (2012). Bayesian sensitivity analysis of a nonlinear finite element model. Mechanical Systems and Signal Processing, 32, 18-31.

Strong, M., Oakley J. E. and Chilcott, J. (2012) Managing structural uncertainty in health economic decision models: A discrepancy approach. Journal of the Royal Statistical Society, Series C, 61(1), 25-45.

Wilkinson, R. D., Vrettas, M., Cornford, D. and Oakley, J. E. (2011) Quantifying simulator discrepancy in discrete-time dynamical simulators. Journal of Agricultural, Biological, and Environmental Statistics,16(4), 554-570.

Fricker, T. E., Oakley J. E., Sims, N. D., Worden, K. and Chilcott, J. (2011) Probabilistic uncertainty analysis of an FRF of a structure using a Gaussian process emulator. Mechanical Systems and Signal Processing, 25(8), 2962-2975.

Oakley, J. E. (2011) Modelling with deterministic computer models. In Simplicity, complexity and modelling, Christie, M., Cliffe, A., Dawid, P. and Senn, S. (eds.). Chichester: Wiley.

Becker, W., Rowson, J., Oakley J. E., Yoxall, A., Manson, G. and Worden K. (2011) Bayesian sensitivity analysis of a model of the aortic valve. Journal of Biomechanics, 44(8), 1499-1506.

Oakley, J. E. and Clough, H. E. (2010) Sensitivity analysis in microbial risk assessment: Vero-cytotoxigenic E.coli O157 in farm-pasteurised milk. Handbook of Applied Bayesian Analysis, O’Hagan, A. and West, M. (eds.). Oxford University Press.

Conti, S., Gosling, J. P., Oakley, J. E. and O’Hagan, A. (2009) Gaussian process emulation of dynamic computer codes. Biometrika, 96, 663-676.

Oakley, J. E. (2009) Decision-theoretic sensitivity analysis for complex computer models. Technometrics, 51, 121-129.

Oakley, J. (2004). Estimating percentiles of computer code outputs. Journal of the Royal Statistical Society, Series C, 53, 83-93.

Oakley, J. and O’Hagan, A. (2004) Probabilistic sensitivity analysis of complex models: A Bayesian approach. Journal of the Royal Statistical Society, Series B, 66, 751-769.
Download example data (TXT, 4KB)

Oakley, J. and O’Hagan, A. (2002) Bayesian inference for the uncertainty distribution of computer model outputs. Biometrika, 89, 769-784.

Oakley, J. (2002) Eliciting Gaussian process priors for complex computer codes. The Statistician, 51, 81-97.

O’Hagan, A., Kennedy, M. C. and Oakley, J. E. (1999) Uncertainty analysis and other inference tools for complex computer codes (with discussion). In Bayesian Statistics 6, Bernardo, J. M. et al. (eds.). Oxford University Press, 503-524.

PhD thesis

Bayesian uncertainty analysis for complex computer codes (PDF, 818KB)