Uncertainty Quantification for Emulators

Consider approximating a function $f$ by an emulator $f$ based on $n$ observations of $f$. This problem is a common when observing $f$ requires a computationally demanding simulation or an actual experiment. Let $w$ be a point in the domain of $f$. The potential error of $f$ at $w$ is the largest value of $|f(w) - g(w)|$ among functions $g$ that satisfy all constraints $f$ is known to satisfy. The supremum over $w$ of the potential error is the maximum potential error of $f$. Suppose that $f$ is in a known class of regular functions. The observations provide a lower bound for the (global) regularity of $f$ as an element of that class. Consider the set $\mathcal F$ of all functions in the regularity class that agree with the $n$ observations and are globally no less regular than $f$ has been observed to be. Among all emulators that produce functions in $\mathcal F$, we find a lower bound on the potential error for $f ∈ \mathcal F$; its maximum over $w$ is a lower bound on the maximum potential error of any $f$. If this lower bound is large, every emulator based on these observations is potentially substantially incorrect. To guarantee higher accuracy would require stronger assumptions about the regularity of $f$. We find a lower bound on the number of observations required to ensure that some emulator based on those observations approximates all $f ∈ \mathcal F$ to within $ε$. For the Community Atmosphere Model, the maximum potential error of any emulator trained on a particular set of 1154 observations of $f$ is no smaller than the potential error based on a single observation of $f$ at the centroid of the 21-dimensional parameter space.