The effects of selection by regularization in high-dimensions - composite estimation, model averaging and dimension reduction

There is a recent awareness of requiring additional efforts for inference when a selection of variables or models has taken place. This can be placed into the larger framework of correct inference and research integrity.

We focus mainly on the high-dimensional setting and consider these valid post-selection issues for composite estimation, model averaging and dimension reduction. Composite estimation involves using a linear combination of loss functions where the coefficients, i.e. the weights, might be datadriven. In high dimensions, variable selection is achieved by regularization. We investigate its influence on the estimator's asymptotic distribution and use this to obtain a proper choice of the weights.

In model averaging a linear combination of different estimators (all based on the same dataset) is constructed. We study how to do this in high dimensions and relate model averaging estimators to composite estimators.

Sparse sufficient dimension reduction also uses regularization. Its effect will be studied theoretically to arrive at correct inference.

A correct way of including the selection uncertainty will have high impact on the many domains where such methods are used. Freely available software will be developed.