Research Paper
Year: 2019 | Month: January-March | Volume: 3 | Issue: 1 | Pages: 1-7
Comparison of Performance of the Least Square Regression, Principal Component Regression and Ridge Regression on Handling Multicollinearity Problem in Linear Models
Oguagbaka S. K1, Osuji G. A2, Aronu C. O3
1Department of Statistics, Federal Polytechnic, Oko, Anambra State, Nigeria
2Department of Statistics, Nnamdi Azikiwe University, Awka, Nigeria
3Department of Statistics, Chukwuemeka Odumegwu Ojukwu University, Anambra State
ABSTRACT
This study compared the performance of three methods on multicollinearity situation. The methods include the linear regression, the principal component regression and the ridge regression method. The methods were compared using 50 simulations and for number of independent variables p=5 and number of observation 6, 10, 20, 30, 40, 50, 60, and 100 respectively. The objectives of the study is to compare the performance of Least Squares Regression, Principal Component Regression (PCR) and Ridge Regression for handling multicollinearity problem and to determine the method that ranked best in terms of the degree of relative efficiency in overcoming the multicollinearity problem using simulated data sets. Findings of the study showed that as p is closer to n, (p=5 and n=6) the multicollinearity is very presence and evident on the R-square value for the linear regression method with is 100% and the standard error of the predicted value being zero (0). It was found that as the sample size increases, the R-square value tends to normalize. Further result showed that the Ridge regression method recorded the least R-square value while the linear regression method recorded the highest R-square value. In addition, it was found that the PCR method has the least standard deviation value across the observed sample size followed by the linear model and then the ridge regression method. This result implies that the principal component regression methods is relatively efficient for solving multicollinearity problems in linear models that the ridge regression.
Key words: Linear Model; Multicollinearity; Standard Deviation; Relative Efficiency
[PDF Full Text]