On the choice of the predictive performance of penalized ridge, lasso and elastic net estimators using simulated data
Penalized regression methods have been developed to overcome the drawbacks of ordinary least squares (OLS) regression model of its inability to perform well with respect to both prediction accuracy and model complexity, especially when the data suffer from multicollinearity problem and large sample size. Penalized model performs variable selection to reduce the number of parameters in the model in order to increase interpretability and reduce the model complexity. This study investigates the predictive performance of Ridge, LASSO and Elastic net penalized (or regularized) regression models. Monte Carlo simulation study was performed to obtain three sample sizes of the data (25, 50 and 100) with three predictors (12, 20 and 25). Root Mean Square Error (RMSE) was used to evaluate the performance of the three estimators. LASSO estimator outperformed Ridge and Elastic net in terms of prediction accuracy, since it exhibits a least RMSE from the overall results and can be recommended to practitioners.
Keywords: Penalized Regression, Multicollinearity, MSE, MLE, Simulation Study
Download Paper