Maximum entropy sampling and optimal Bayesian experimental design

@article{Sebastiani:MaximumEntropySampling, abstract = {When Shannon entropy is used as a criterion in the optimal design of experiments, advantage can be taken of the classical identity representing the joint entropy of parameters and observations as the sum of the marginal entropy of the observations and the preposterior conditional entropy of the parameters. Following previous work in which this idea was used in spatial sampling, the method is applied to standard parameterized Bayesian optimal experimental design. Under suitable conditions, which include non-linear as well as linear regression models, it is shown in a few steps that maximizing the marginal entropy of the sample is equivalent to minimizing the preposterior entropy, the usual Bayesian criterion, thus avoiding the use of conditional distributions. It is shown using this marginal formulation that under normality assumptions every standard model which has a two-point prior distribution on the parameters gives an optimal design supported on a single point. Other results include a new asymptotic formula which applies as the error variance is large and bounds on support size.}, address = {The Open University, Milton Keynes, UK, ; University of Warwick, Coventry, UK}, author = {Sebastiani, P. and Wynn, H. P.}, citeulike-article-id = {3615937}, doi = {10.1111/1467-9868.00225}, journal = {Journal of the Royal Statistical Society: Series B (Methodological)}, keywords = {active\_learning, experimental\_design}, number = {1}, pages = {145--157}, posted-at = {2008-11-20 18:34:42}, priority = {4}, title = {Maximum entropy sampling and optimal Bayesian experimental design}, url = {http://dx.doi.org/10.1111/1467-9868.00225}, volume = {62}, year = {2000} }

TY - JOUR ID - Sebastiani:MaximumEntropySampling L3 - citeulike-article-id:3615937 N2 - When Shannon entropy is used as a criterion in the optimal design of experiments, advantage can be taken of the classical identity representing the joint entropy of parameters and observations as the sum of the marginal entropy of the observations and the preposterior conditional entropy of the parameters. Following previous work in which this idea was used in spatial sampling, the method is applied to standard parameterized Bayesian optimal experimental design. Under suitable conditions, which include non-linear as well as linear regression models, it is shown in a few steps that maximizing the marginal entropy of the sample is equivalent to minimizing the preposterior entropy, the usual Bayesian criterion, thus avoiding the use of conditional distributions. It is shown using this marginal formulation that under normality assumptions every standard model which has a two-point prior distribution on the parameters gives an optimal design supported on a single point. Other results include a new asymptotic formula which applies as the error variance is large and bounds on support size. IS - 1 JF - Journal of the Royal Statistical Society: Series B (Methodological) AD - The Open University, Milton Keynes, UK, ; University of Warwick, Coventry, UK EP - 157 TI - Maximum entropy sampling and optimal Bayesian experimental design VL - 62 SP - 145 KW - active_learning KW - experimental_design AU - Sebastiani, P AU - Wynn, HP PY - 2000/// UR - http://dx.doi.org/10.1111/1467-9868.00225 ER -