Kybernetika 53 no. 1, 1-25, 2017

Instrumental weighted variables under heteroscedasticity, Part I -- Consistency

Jan Ámos VíšekDOI: 10.14736/kyb-2017-1-0001

Abstract:

The proof of consistency instrumental weighted variables, the robust version of the classical instrumental variables is given. It is proved that all solutions of the corresponding normal equations are contained, with high probability, in a ball, the radius of which can be selected - asymptotically - arbitrarily small. Then also $\sqrt{n}$-consistency is proved. An extended numerical study (the Part II of the paper) offers a picture of behavior of the estimator for finite samples under various types and levels of contamination as well as various extent of heteroscedasticity. The estimator in question is compared with two other estimators of the type of "robust instrumental variables'' and the results indicate that our estimator gives comparatively good results and for some situations it is better. The discussion on a way of selecting the weights is also offered. The conclusions show the resemblance of our estimator with the $M$-estimator with Hampel's $\psi$-function. The difference is that our estimator does not need the studentization of residuals (which is not a simple task) to be scale- and regression-equivariant while the $M$-estimator does. So the paper demonstrates that we can directly compute - moreover by a quick algorithm (reliable and reasonably quick even for tens of thousands of observations) - the scale- and the regression-equivariant estimate of regression coefficients.

Keywords:

consistency of the instrumental weighted variables, heteroscedasticity of disturbances, numerical study, weighting order statistics of the squared residuals

Classification:

62J02, 62F35

References:

  1. T. Amemiya: Two stage least absolute deviation estimators. Econometrica 50 (1982), 689-711.   DOI:10.2307/1912608
  2. A. C. Atkinson, M. Riani and A. Cerioli: Exploring Multivariate Data with the Forward Search. Springer Series in Statistics 2004, 31-88.   DOI:10.1007/978-0-387-21840-3_2
  3. R. Beran: An efficient and robust adaptive estimator of location. Ann. Statist. 6 (1978), 292-313.   DOI:10.1214/aos/1176344125
  4. R. J. Bowden and D. A. Turkington: Instrumental variables. Cambridge Univ. Press, Cambridge 1984.   DOI:10.1017/ccol0521262410
  5. C. M. Bramati and C. Croux: Robust estimators for the fixed effects panel data model. The Econometr. J. 10 (2077), 521-540.   DOI:10.1111/j.1368-423x.2007.00220.x
  6. L. Breiman: Probability. Addison-Wesley Publishing Company, London 1968.   CrossRef
  7. P. Čížek: Generalized method of trimmed moments. J. Statist. Planning Inference 171 (2009), 63-78.   DOI:10.1016/j.jspi.2015.11.004
  8. D. Cochrane and G. H. Orcutt: Application of least squares regression to relationhips containing autocorrelated error terms. J. Amer. Statist. Assoc. 44 (1949), 32-61.   DOI:10.1080/01621459.1949.10483290
  9. G. V. Cohen-Freue, H. Ortiz-Molina and R. H. Zamar: A New Robust Instrumental Variables Estimator.    CrossRef
  10. G. V. Cohen-Freue, H. Ortiz-Molina and R. H. Zamar: Natural robustification of the ordinary instrumental variables estimator. Biometrics 69 (2013), 641-650.   DOI:10.1111/biom.12043
  11. C. Croux, S. Van Aelst and C. Dehon: Bounded influence regression using high breakdown scatter matrices. Ann. Inst. Statist. Math. 55 (2013), 265-285.   DOI:10.1007/bf02530499
  12. P. L. Davies: Asymptotic behavior of $S$-estimates of multivariate location parameters and dispersion matrices. Ann. Statist. 15 (1987), 1269-1292.   DOI:10.1214/aos/1176350505
  13. M. Donsker (1952): Justification and extension of Doob's heuristic approach to the Kolmogorov-Smirnov theorems. Ann. Math. Statist. 23 (1952), 277-281.   DOI:10.1214/aoms/1177729445
  14. F. Eicker: Asymptotic normality and consistency of the least squares estimators for families of linear regressions. Ann. Math. Stat. 34 (1963), 447-456.   DOI:10.1214/aoms/1177704156
  15. F. Eicker: Limit theorems for regression with unequal and dependent errors. In: Proc. Fifth Berkeley Symposium on Mathematical Statistics and Probability (L. Le Cam and J. Neyman, eds.), University of California Press, Berkeley 1967.   CrossRef
  16. Z. Fabián: Induced cores and their use in robust parametric estimation. Comm. Statist. - Theory and Methods 30 (2001), 537-555.   DOI:10.1214/aoms/1177704156
  17. Z. Fabián: New measures of central tendency and variability of continuous distributions. Comm. Statist. - Theory and Methods 37 (2008), 159-174.   DOI:10.1080/03610920701648987
  18. C. A. Field and E. M. Ronchetti (1990): Small Sample Asymptotics. Institute of Mathematical Statistics Monograph Series, Hayward 1990.   CrossRef
  19. R. A. Fisher: A mathematical examination of the methods of determining the accuracy of an observation by the mean error and by the mean squares error. Monthly Notes Royal Astronomical Society 80 (1920), 758-770.   DOI:10.1093/mnras/80.8.758
  20. R. A. Fisher: Statistical Methods for Research Workers. Second edition (1928).   CrossRef
  21. F. Galton: Regression towards mediocrity in hereditary stature. J. Anthropol. Inst. 15 (1886), 246-263.   DOI:10.2307/2841583
  22. W. H. Greene: Econometric Analysis. Macmillam Press, New York 1993.   CrossRef
  23. J. Hájek and Z. Šidák: Theory of Rank Test. Academic Press, New York 1967.   CrossRef
  24. P. R. Halmos: Applied mathematics is a bad mathematics. In: Mathematics Tomorrow (L. Steen, ed.), Springer Verlag, New York 1981, pp. 9-20.   DOI:10.1007/978-1-4613-8127-3_2
  25. F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw and W. A. Stahel: Robust Statistics - The Approach Based on Influence Functions. J. Wiley and Sons, New York 1986.    DOI:10.1002/9781118186435
  26. L. P. Hansen: Large sample properties of generalized method of moments estimators. Econometrica 50 (1982), 1029-1054.   DOI:10.2307/1912775
  27. A. C. Harvey: Estimating regression models with multiplicative heteroscedasticity. Econometrica 44 (1976), 461-465.   DOI:10.2307/1913974
  28. J. Hausman, W. Newey, T. Voutersen, J. Chao and N. Swanson: Instrumental variable estimation with heteroscedasticity and many instruments. Quantitative Economics 3 (2012), 211-255.   DOI:10.3982/qe89
  29. I. S. Helland: Partial Least Squares Regression and Statistical Models. Scand. J. Statist. 17 (1990), 97-114.   CrossRef
  30. T. P. Hettmansperger and S. J. Sheather: A cautionary note on the method of least median squares. Amer. Statist. 46 (1992), 79-83.   DOI:10.2307/2684169
  31. G. G. Judge, W. E. Griffiths, R. C. Hill, H. Lutkepohl and T. C. Lee: The Theory and Practice of Econometrics. Second edition. J. Wiley and Sons, New York 1985.   CrossRef
  32. J. Jurečková: Regression quantiles and trimmed least squares estimator under a general design. Kybernetika 20 (1984), 345-357.   CrossRef
  33. J. Kmenta: Elements of Econometrics. Macmillan Publishing Company, New York 1986.   DOI:10.3998/mpub.15701
  34. W. S. Krasker: Two-stage bounded-influence estimators for simultaneous equations models. J. Business Econom. Statist. 4 (1986), 437-444.   DOI:10.2307/1391499
  35. W. S. Krasker and R. E. Welsch: Resistant estimation for simultaneous - equations models using weighted instrumental variables. Econometrica 53 (1985), 1475-1488.   DOI:10.2307/1913223
  36. J. Krishnakumar and E. Ronchetti: Robust-estimators for simultaneous equations models. J. Econometr. 78 (1997), 295-314.   DOI:10.1016/s0304-4076(97)80014-0
  37. H. P. Lopuhaa: On the relations between $S$-estimators and $M$-estimatros of multivariate location and covariance. Ann. Statist. 17 (1989), 1662-1683.   DOI:10.1214/aos/1176347386
  38. R. A. Maronna and S. Morgenthaler: Robust regression through robust covariances. Comm. Statist. - Theory and Methods 15 (1986), 1347-1365.   DOI:10.1080/03610928608829187
  39. R. A. Maronna and V. J. Yohai: Asymptotic behaviour of general $M$-estimates for regression and scale with random carriers. Zeitschrift fűr Wahrscheinlichkeitstheorie und verwandte Gebiete 58 (1981), 7-20.   DOI:10.1007/bf00536192
  40. R. A. Maronna and V. J. Yohai: Robust estimation in simultaneous equations models. J. Statist. Planning Inference 57 (1997), 233-244.   DOI:10.1016/s0378-3758(96)00046-8
  41. L. Mašíček: Optimality of the least weighted squares estimator. Kybernetika 40 (2004), 715-734.   CrossRef
  42. G. E. Mizon: A simple message for autocorrelation correctors: Don't. J. Econometr. 69 (1995), 267-288.   DOI:10.1016/0304-4076(94)01671-l
  43. C. C. Paige and Z. Strakoš: Scaled total least squares fundamentals. Numer. Math. 91 (2002), 117-146.   DOI:10.1007/s002110100314
  44. P. C. B. Phillips and V. Solo: Asymptotics for linear processes. Ann. Statist. 20 (1992), 971-1001.   DOI:10.1214/aos/1176348666
  45. K. R. Popper: The Logic of Scientific Discovery. (Logik der Forscung, Springer, Vienna 1935). Hutchinson and co., New York 1952.   CrossRef
  46. S. Portnoy: Tightness of the sequence of empiric c.d.f. processes defined from regression fractiles. In: Robust and Nonlinear Time - Series Analysis (J. Franke, W. H\H{a}rdle, and D. Martin, eds.), Springer Verlag, New York 1983, pp. 231-246.   DOI:10.1007/978-1-4615-7821-5_13
  47. R. C. Rao: Estimation of heteroscedastic variances in linear models. J. Amer. Statist. Assoc. 65 (1970), 161-172.   DOI:10.2307/2283583
  48. R. C. Rao: Linear Statistical Inference and Its Applications. J. Wiley and Sons, New York 1973.   DOI:10.1002/9780470316436
  49. P. M. Robinson: Asymptotically efficient estimation in the presence of heteroskedasticity of unknown form. Econometrica 55 (1987), 875-891.   DOI:10.2307/1911033
  50. E. Ronchetti and F. Trojani: Robust inference with GMM estimators. J. Econometrics 101 (2001), 37-69.   DOI:10.1016/s0304-4076(00)00073-7
  51. P. J. Rousseeuw: Least median of square regression. J. Amer. Statist. Assoc. 79 (1984), 871-880.   DOI:10.1080/01621459.1984.10477105
  52. P. J. Rousseeuw and A. M. Leroy: Robust Regression and Outlier Detection. J. Wiley and Sons, New York 1987.   DOI:10.1002/0471725382
  53. P. J. Rousseeuw and V. Yohai: Robust regressiom by means of $S$-estimators. In: Robust and Nonlinear Time Series Analysis (J. Franke, W. H\H{a}rdle, and R. D. Martin, eds.), Lecture Notes in Statistics 26 Springer Verlag, New York 1984, pp. 256-272.   DOI:10.1007/978-1-4615-7821-5_15
  54. J. Štěpán: Teorie pravděpodobnosti (Probability Theory). Academia, Praha 1987.   CrossRef
  55. S. Van Huffel: Total least squares and error-in-variables modelling: Bridging the gap between statistics, computational mathematics and enginnering. In: Proc. Computational Statistics, COMPSTAT 2004 (J. Antoch, ed.), Physica Verlag/Springer, Heidelberg 2004, pp. 539-555.   DOI:10.1007/978-3-7908-2656-2_44
  56. J. Á. Víšek: A cautionary note on the method of Least Median of Squares reconsidered. In: Trans. Twelfth Prague Conference on Information Theory, Statistical Decision Functions and Random Processes (P. Lachout, ed.), Academy of Sciences of the Czech Republic, Praha 1994, pp. 254-259.   CrossRef
  57. J. Á. Víšek: Robust instruments. In: Robust'98 (J. Antoch and G. Dohnal, eds.), Union of Czech Mathematicians and Physicists, Matfyzpress, Praha 1998, pp. 195-224.   CrossRef
  58. J. Á. Víšek: The robust regression and the experiences from its application on estimation of parameters in a dual economy. In: Proc. Conference Macromodels'99, Wroclaw University 1999, pp. 424-445.   CrossRef
  59. J. Á. Víšek: Regression with high breakdown point. In: Robust 2000 (J. Antoch and G. Dohnal, eds.), Union of Czech Mathematicians and Physicists, Matfyzpress, Praha 2000, pp. 324-356.   CrossRef
  60. J. Á. Víšek: The least weighted squares I. The asymptotic linearity of normal equations. Bull. Czech Econometr. Soc. 9 (2002), 31-58.   CrossRef
  61. J. Á. Víšek: The least weighted squares II. Consistency and asymptotic normality. Bull. Czech Econometr. Soc. 9 (2002), 1-28.   CrossRef
  62. J. Á. Víšek: Development of the Czech export in nineties. In: Konsolidace vládnutí a podnikání v České republice a v Evropské unii I. Umění vládnout, ekonomika, politika, Matfyzpress, Praha 2003, pp. 193-220.   CrossRef
  63. J. Á. Víšek: Robustifying instrumental variables. In: Proc. COMPSTAT'2004 (J. Antoch, ed.), Physica Verlag/Springer, pp. 1947-1954.   DOI:10.1007/978-3-7908-2656-2
  64. J. Á. Víšek: Instrumental weighted variables - algorithm. In: Proc. COMPSTAT 2006 (A. Rizzi and M. Vichi, eds.), Physica Verlag/Springer, Heidelberg 2006, pp. 777-786.   DOI:10.1007/978-3-7908-1709-6
  65. J. Á. Víšek: Kolmogorov-Smirnov statistics in multiple regression. In: Proc. ROBUST 2006 (J. Antoch and G. Dohnal, eds.), pp. 367-374.   CrossRef
  66. J. Á. Víšek: Consistency of the instrumental weighted variables. Ann. Inst. Statist. Math. 61 (2009), 543-578.   CrossRef
  67. J. Á. Víšek: Robust error - term - scale estimate. In: IMS Collections. Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: Festschrift for Jana Jurečková, 2010, pp. 254-267.   DOI:10.1007/s10463-007-0159-8
  68. J. Á. Víšek: Heteroscedasticity resistant robust covariance matrix estimator. Bull. Czech Econometric Society 17 (2010), 33-49.   CrossRef
  69. J. Á. Víšek: Consistency of the least weighted squares under heteroscedasticity. Kybernetika 47 (2011), 179-206.   CrossRef
  70. J. Á. Víšek: Weak $\sqrt{n}$ - consistency of the least weighted squares under heteroscedasticity. Acta Universitatis Carolinae, Mathematica et Physica 2 (2011), 51, 71-82.   CrossRef
  71. J. Á. Víšek: Empirical distribution function under heteroscedasticity. Statistics 45 (2011), 497-508.   DOI:10.1080/02331881003768891
  72. J. Á. Víšek: Robustifying estimation of the model with fixed and random effects. Part I - Theoretical considerations. Part II - Numerical study. Workshop on Algorithm for Outliers/regressors Selection organized by Bent Nielsen, Nuffield College, Oxford 2013. Methodology and Computing in Applied Probability 17 (2014), 4, 999-1014.   DOI:10.1007/s11009-014-9432-5
  73. R. Wagenvoort and R. Waldmann: On $B$-robust instrumental variable estimation of the linear model with panel data. J. Econometr. 106 (2002), 297-324.   DOI:10.1016/s0304-4076(01)00102-6
  74. H. White: A heteroskedasticity - consistent covariance matrix estimator and a direct test for heteroscedasticity. Econometrica 48 (1980), 817-838.   DOI:10.2307/1912934
  75. J. M. Wooldridge: Econometric Analysis of Cross Section and Panel Data. MIT Press, Cambridge 2001. (Second edition 2008.)   CrossRef
  76. J. M. Wooldridge: Introductory Econometrics. A Modern Approach. MIT Press, Cambridge 2006. (Second edition 2009.)   CrossRef