The Impact of Regularization on Linear Regression Based Model

Abstract

This paper aims to analyse the effects that regularisation has on linear regression models while concentration on Lasso (L1) and Ridge (L2) methods. Academic Rigor definition is one of the areas in which Regularisation helps in regression modelling since it minimises the overfitting and multicollinearity issue by putting tight constraints on the coefficients values to make the models more balanced and easier to understand. In order to optimise the hyperparameters of the models, we employed cross-validation, and to apply the suggested forms of regularisation, we used the Materialise Python scikit-learn library. Finally, the results show that Lasso and Ridge possess a great potential to increase the performance of models, which may be indicated by improvements in such indicators as MSE, MAE, and R². Diagnostic plots and curves like scatter plots, residual diagnostic plots as well as learning curves were used to analyse the operational performance of the models. It showed that in terms of feature selection Lasso performs better than Ridge with correlated features managed by Ridge Regression. The F-test of the Statistical analysis provided supporting evidences for the regularisation effects. The guidelines drawn suggest the importance of feature extraction or construction, data pre-processing, and cross-validation during the tuning of parameters. Further, it is important for scholars to consider other more complex regression models such as Elastic Net and their broad practical applicability at a higher level. This work therefore draws attention to the issue of regularisation as a useful and significant tool to ameliorate the precision, robustness and transferability of linear regression models.