High Dimensional Statistical Modelling with Limited Information

Abstract

Modern scientific experiments often rely on different statistical tools, regularisation
being one of them. Regularisation methods are usually used to avoid overfitting but
we may also want use regularisation methods for variable selection,
especially when the number of modelling parameters are higher than the total number
of observations. However, performing variable selection can often be difficult under limited
information and we may get a misspecified model. To overcome this issue, we propose a robust
variable selection routine using a Bayesian hierarchical model.

We adapt the framework of Narisetty and He to propose a novel spike and slab prior specification
for the regression coefficients. We take inspiration from the imprecise beta model and
use a set of beta distributions to specify the prior expectation of the selection probability.
We perform a robust Bayesian analysis over this set of distributions in order to
incorporate expert opinion in an efficient manner.

We also discuss novel results on likelihood-based approaches for variable selection.
We exploit the framework of the adaptive LASSO to propose sensitivity analyses
of LASSO-type problems. The sensitivity analysis also gives us a novel non-deterministic classifier
for high dimensional problems, which we illustrate using real datasets.

Finally, we illustrate our novel robust Bayesian variable selection using synthetic and real-world data.
We show the importance of prior elicitation in variable selection as well as model fitting and compare
our method with other Bayesian approaches for variable selection.