Kybernetika 62 no. 2, 286-304, 2026

An efficient sparse identification algorithm for stochastic systems with general observation sequences

Ziming Wang, Mingkun Li, Yiming Xing and Xinghua ZhuDOI: 10.14736/kyb-2026-2-0286

Abstract:

This paper studies the sparse identification problem of unknown sparse parameter vectors in stochastic dynamic systems. Firstly, a novel sparse identification algorithm is proposed, which can generate sparse estimates based on least squares estimation by adaptively adjusting the threshold. Secondly, under a possibly weakest non-persistent excited condition, we prove that the proposed algorithm can correctly identify the zero and nonzero elements of the sparse parameter vector using a finite number of observations, and further estimates of the nonzero elements almost surely converge to the true values. Compared with the related works, e.\,g., LASSO, our method only requires the weakest assumptions and does not require solving additional optimization problems. Thirdly, the number of finite observations that guarantee the convergence of the zero-element set of unknown sparse parameters of the Hammerstein system is derived for the first time. Finally, numerical simulations are provided, demonstrating the effectiveness of the proposed method. Since there is no additional optimization problem, i.\,e., no additional numerical error, the proposed algorithm performs much better than other related algorithms.

Keywords:

strong consistency, stochastic dynamic system, sparse parameter identification, the weakest non-persistent excited condition, feedback control system

Classification:

93A10, 93E12, 93E24

paper.pdf

References:

  1. O. A. Alzubi, J. A. Alzubi, M. Alweshah, I. Qiqieh, S. Al-Shami and M. Ramachandran: An optimal pruning algorithm of classifier ensembles: dynamic programming approach. Neural Computing Appl. 32 (2020), 20, 16091-16107.   DOI:10.1007/s00521-020-04761-6
  2. E. J. Candès et al.: Compressive sampling. In: Proc. International congress of mathematicians, volume 3, Citeseer 2006, pp. 1433-1452.   CrossRef
  3. E. J. Candès, J. K. Romberg and T. Tao: Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 59 (2006), 8, 1207-1223.   CrossRef
  4. E. J. Candes and T. Tao: Decoding by linear programming. IEEE Trans. Inform. Theory 51 (2005), 12, 4203-4215.   DOI:10.1109/TIT.2005.858979
  5. Y. Chen, Y. Gu and A. O. Hero: Sparse lms for system identification. In: IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE 2009, pp. 3125-3128.   CrossRef
  6. E. Eskinat, S. H. Johnson and W. L. Luyben: Use of hammerstein models in identification of nonlinear systems. AIChE J. 37 (1991), 2, 255-268.   DOI:10.1002/aic.690370211
  7. Y. Gu, J. Jin and S. Mei: $l_0$ norm constraint LMS algorithm for sparse system identification. IEEE Signal Process. Lett. 16 (2009), 9, 774-777.   DOI:10.1109/LSP.2009.2024736
  8. L. Guo: Convergence and logarithm laws of self-tuning regulators. Automatica 31 (1995), 3, 435-450.   DOI:10.1016/0005-1098(94)00127-5
  9. L. Guo: Time-varying Stochastic Systems, Stability and Adaptive Theory. Second edition. Science Press, Beijing 2020.   CrossRef
  10. L. Guo and H. F. Chen: Identification and Stochastic Adaptive Control. Springer Science, Boston 1991.   CrossRef
  11. S. Haykin: Radar signal processing. IEEE Signal Process. Magazine 2 (1985), 2, 2-18.   DOI:10.1109/MASSP.1985.1163737
  12. N. Kalouptsidis, G. Mileounis, B. Babadi and V. Tarokh: Adaptive algorithms for sparse system identification. Signal Process. 91 (2011), 8, 1910-1919.   DOI:10.1016/j.sigpro.2011.02.013
  13. Y. Kopsinis, K. Slavakis and S. Theodoridis: Online sparse system identification and signal reconstruction using projections onto weighted $l_1$ balls. IEEE Trans. Signal Process. 59 (2010), 3, 936-952.   DOI:10.1109/TSP.2010.2090874
  14. T. L. Lai and Ch. Z. Wei: Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. Annals Statist. 10 (1982), 1, 154-166.   CrossRef
  15. S. K. Perepu and A. K. Tangirala: Identification of equation error models from small samples using compressed sensing techniques. In: 9th IFAC Symposium on Advanced Control of Chemical Processes ADCHEM 2015, IFAC-PapersOnLine 48 (2015), 8, 795-800.   DOI:10.1016/j.ifacol.2015.09.066
  16. J. G. Smith, S. Kamat and K. P. Madhavan: Modeling of ph process using wavenet based hammerstein model. J. Process Control 17 (2007), 6, 551-561.   DOI:10.1016/j.jprocont.2006.11.001
  17. R. Tibshirani: Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc.: Series B: Methodological 58 (1996), 1, 267-288.   DOI:10.1111/j.2517-6161.1996.tb02080.x
  18. R. Tóth, B. M. Sanandaji, K. Poolla and T. L. Vincent: Compressive system identification in the linear time-invariant framework. In: 50th IEEE Conference on Decision and Control and European Control Conference 2011, pp. 783-790.   CrossRef
  19. M. Vaezi and A. Izadian: Piecewise affine system identification of a hydraulic wind power transfer system. IEEE Trans. Control Systems Technol. 23 (2015), 6, 2077-2086.   DOI:10.1109/TCST.2015.2398311
  20. S. Xie and L. Guo: Analysis of compressed distributed adaptive filters. Automatica 112 (2020), 108707.   DOI:10.1016/j.automatica.2019.108707
  21. X. Xu, B. Pan, Z. Chen, Z. Shi and T. Li: Simultaneously multiobjective sparse unmixing and library pruning for hyperspectral imagery. IEEE Trans. Geosci. Remote Sensing 59 (2020), 4, 3383-3395.   DOI:10.1109/TGRS.2020.3016941
  22. P. Zhao and B. Yu: On model selection consistency of lasso. J. Machine Learn. Res. 7 (2006), 2541-2563.   CrossRef
  23. W.-X. Zhao: Parametric identification of hammerstein systems with consistency results using stochastic inputs. IEEE Trans. Automat. Control 55 (2010), 2, 474-480, 2010.   DOI:10.1109/TAC.2009.2036380
  24. W. Zhao, G. Yin and E.-W. Bai: Sparse system identification for stochastic systems with general observation sequences. Automatica 121 (2020), 109162.   DOI:10.1016/j.automatica.2020.109162
  25. H. Zou: The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 (2006), 476, 1418-1429.   DOI:10.1198/016214506000000735