Confidence Sets in Sparse Regression

The problem of constructing confidence sets in the high dimensional linear model with $n$ response variables and $p$ parameters, possibly $p ≥ n$, is considered. Necessary and sufficient conditions for the existence of confidence sets that adapt to the unknown sparsity of the parameter vector are given in terms of $\ell^2$-separation conditions. These are derived from a minimax analysis of closely related composite testing problems. The design conditions cover common coherence assumptions used in models for sparse inference, such as Gaussian and sub-Gaussian designs. The results imply in particular that sparse confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Qualitative differences between the highly and moderately sparse case are shown to exist, and the case of $p ≤ n$ is analysed separately, where a transition to the theory of adaptive confidence sets in standard nonparametric and parametric models is exhibited. Concrete inferential procedures that can be used over maximal parameter spaces are discussed.