Kybernetika 55 no. 4, 740-754, 2019

An alternating minimization algorithm for Factor Analysis

Valentina Ciccone, Augusto Ferrante and Mattia ZorziDOI: 10.14736/kyb-2019-4-0740

Abstract:

The problem of decomposing a given covariance matrix as the sum of a positive semi-definite matrix of given rank and a positive semi-definite diagonal matrix, is considered. We present a projection-type algorithm to address this problem. This algorithm appears to perform extremely well and is extremely fast even when the given covariance matrix has a very large dimension. The effectiveness of the algorithm is assessed through simulation studies and by applications to three real benchmark datasets that are considered. A local convergence analysis of the algorithm is also presented.

Keywords:

matrix decomposition, factor analysis, covariance matrices, low rank matrices, projections

Classification:

62H25

paper.pdf

References:

  1. A. Agarwal, S. Negahban and M. J. Wainwright: Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions. Ann. Statist. 40 (2012), 2, 1171-1197.   DOI:10.1214/12-aos1000
  2. J. Bai, K. Li and et al.: Statistical analysis of factor models of high dimension. Ann. Statist. 40 (2012), 1, 436-465.   CrossRef
  3. J. Bai and S. Ng: Determining the number of factors in approximate factor models. Econometrica 70 (2002), 1, 191-221.   DOI:10.1111/1468-0262.00273
  4. J. Bai and S. Ng: Large dimensional factor analysis. Found. Trends Econometr. 3 (2008), 2, 89-163.   CrossRef
  5. D. Bertsimas, M. S. Copenhaver and R. Mazumder: Certifiably optimal low rank factor analysis. J. Machine Learning Res. 18 (2017), 29, 1-53.   CrossRef
  6. V. Ciccone, A. Ferrante and M. Zorzi: Factor analysis with finite data. In: Proc. 56th Annual Conference on Decision and Control (CDC), VIC, Melbourne 2017, pp. 4046-4051.   DOI:10.1109/cdc.2017.8264253
  7. V. Ciccone, A. Ferrante and M. Zorzi: Robust identification of "Sparse Plus Low-rank" graphical models: An optimization approach. In: Proc. 2018 IEEE Conference on Decision and Control (CDC), Miami Beach 2018, pp. 2241-2246.   DOI:10.1109/cdc.2018.8619796
  8. V. Ciccone, A. Ferrante and M. Zorzi: Factor models with real data: A robust estimation of the number of factors. IEEE Trans. Automat. Control 64 (2019), 6, 2412-2425.   DOI:10.1109/tac.2018.2867372
  9. M. Deistler and C. Zinner: Modelling high-dimensional time series by generalized linear dynamic factor models: An introductory survey. Comm. Inform. Systems 7 (2007), 2, 153-166.   DOI:10.4310/cis.2007.v7.n2.a3
  10. G. Della Riccia and A. Shapiro: Minimum rank and minimum trace of covariance matrices. Psychometrika 47 (1982), 443-448.   DOI:10.1007/bf02293708
  11. M. Fazel: Matrix rank minimization with applications. Elec. Eng. Dept. Stanford University 54 (2002), 1-130.   CrossRef
  12. M. Fazel, H. Hindi and S. Boyd: Rank minimization and applications in system theory. In: Proc. American Control Conference 4 (2004), pp. 3273-3278.   DOI:10.23919/acc.2004.1384521
  13. J. Geweke: The dynamic factor analysis of economic time series models. In: Latent Variables in Socio-Economic Models, SSRI workshop series, North-Holland 1977, pp. 365-383.   CrossRef
  14. L. Guttman: Some necessary conditions for common-factor analysis. Psychometrika 19 (1954), 2, 149-161.   DOI:10.1007/bf02289162
  15. H. H. Harman and W. H. Jones: Factor analysis by minimizing residuals (minres). Psychometrika 31 (1066), 3, 351-368.   DOI:10.1007/bf02289468
  16. C. Heij, W. Scherrer and M. Deistler: System identification by dynamic factor models. SIAM J. Control Optim. 35 (1997), 6, 1924-1951.   DOI:10.1137/s0363012995282127
  17. U. Helmke and M. A. Shayman: Critical points of matrix least squares distance functions. Linear Algebra Appl. 215 (1995), 1-19.   DOI:10.1016/0024-3795(93)00070-g
  18. P. D. Lax: Linear Algebra and Its Applications. Second edition. Wiley-Interscience, 2007.   CrossRef
  19. A. S. Lewis and J. Malick: Alternating projections on manifolds. Math. Oper. Res. 33 (2008), 1, 216-234.   DOI:10.1287/moor.1070.0291
  20. L. Ning, T. T. Georgiou, A. Tannenbaum and S. P. Boyd: Linear models based on noisy data and the {F}risch scheme. SIAM Rev. 57 (2015), 2, 167-197.   DOI:10.1137/130921179
  21. O. Reiersøl: Identifiability of a linear relation between variables which are subject to error. Econometrica: J. Economet. Soc. 18 (1950), 4, 375-389.   DOI:10.2307/1907835
  22. W. Scherrer and M. Deistler: A structure theory for linear dynamic errors-in-variables models. SIAM J. Control Optim. 36 (1998), 6, 2148-2175.   DOI:10.1137/s0363012994262464
  23. A. Shapiro: Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis. Psychometrika 47 (1982), 2, 187-199.   DOI:10.1007/bf02296274
  24. A. Shapiro and J. M. F. Ten Berge: Statistical inference of minimum rank factor analysis. Psychometrika 67 (2002), 1, 79-94.   DOI:10.1007/bf02294710
  25. M. Zorzi and R. Sepulchre: AR identification of latent-variable graphical models. IEEE Trans. Automat. Control 61 (2016), 9, 2327-2340.   DOI:10.1109/tac.2015.2491678