Length modified ridge regression

Biased regression methods may improve considerably on ordinary least squares regression with few or noisy data, or when the predictor variables are highly collinear. In the present work, I present a new, biased method that modifies the ordinary least squares estimate by adjusting each element of the estimated coefficient vector. The adjusting factors are found by minimizing a measure of prediction error. However, the optimal adjusting factors depend on the unknown coefficient vector as well as the variance of the noise, so in practice these are replaced by preliminary estimates. The final estimate of the coefficient vector has the same direction as the preliminary estimate, but the length is modified. Ridge regression is used as the principal method to find the preliminary estimate, and the method is called length modified ridge regression. In addition, length modified principal components regression is considered. The prediction performance of the methods are compared to other regression methods (ridge, James-Stein, partial least squares, principal components and variable subset selection) in a simulation study. Of all methods considered, length modified ridge regression shows the overall best behaviour. The improvement over ridge regression is moderate, but significant, especially when the data are few and noisy.