Analytical Approach to the Study of Heterogeneous Markov Chains with Piecewise Constant Changes in Transition Probabilities
https://doi.org/10.17586/0021-3454-2024-67-8-657-669
Abstract
An analytical approach to the study of countable non-homogeneous Markov chains based on the z-transformation is proposed. Using two variants of the initial data for the transition probabilities of a Markov chain as an example, it is shown that estimates of the steady-state mode can significantly distort the understanding of the system behavior. Analytical procedures for obtaining probability functions for real and complex-conjugate eigenvalues of the transition probability matrix are described for the case when its elements change abruptly. Estimates are given for the boundaries of the onset of a steady-state mode in the clock time. The main calculations are illustrated by an assessment of the characteristics of Markov chains taking into account the influence of transition dynamics under variance of the state probabilities and the piecewise constant change in transition probabilities over the operating interval in clock time.
About the Author
A. V. Zemskov
Admiral Makarov State University of Maritime and Inland Shipping
Russian Federation
Russian Federation
Alexey V. Zemskov – Dr. Sci., Professor; Department of Mathematical Simulation and Applied Informatics; Professor
St. Petersburg
References
1. Tikhonov V.I., Mironov M.A. Markovskiye protsessy (Markov Processes), Moscow, 1977, 485 р. (in Russ.)
2. Ventzel E.S. Issledovaniye operatsiy: zadachi, printsipy, metodologiya (Operations Research: Objectives, Principles, Methodology), Moscow, 2010, 190 р. (in Russ.)
3. Nummelin E. General Irreducible Markov Chains and Non-Negative Operators, Cambridge etc., 1984, 156 p.
4. Furman Ya.A., Yuryev A.N., Yanshin V.V. Tsifrovyye metody obrabotki i raspoznavaniya binarnykh izobrazheniy (Digital Methods of Processing and Recognition of Binary Images), Krasnoyarsk, 1992, 245 р. (in Russ.)
5. Zemskov A.V. Journal of Instrument Engineering, 2023, no. 12(66), pp. 1035–1049, DOI: 10.17586/0021-3454-2023-66-12-1035-1049.
6. Vytovtov K.A. and Barabanova E.A. Automation and Remote Control, 2021, no. 12, рр. 2112–2124, DOI: 10.31857/S0005231021120060.
7. Harrison P.G. J. Appl. Prob., 1981, no. 2(18), pp. 482–490.
8. Dudin A.N., Karolik A.V. Performance Evaluat., 2001, no. 1(45), pp. 19–32.
9. Dharmaraja S., Rakesh Kumar, OPSEARCH, 2015, no. 4(52), pp. 810–826.
10. Kumar B. Krishna, Madheshwari S. Pavai, Venkatakrishanan K.S. Int. J. Inform. Management Sci., 2017, no. 1(18), pp. 63–80.
11. Clarke A.B. The time-dependent waiting line problem, Umv Michigan Rept M720-1RS9, 1953.
12. Clarke A.B. Ann. Math. Statist., 1953, vol. 24, рр. 491–492.
13. Kogan B.Ya., Menn A.A. Automation and Remote Control, 1976, no. 12, pp. 1908–1919.
14. Dudin A.N. Automatic Control and Computer Sciences, 1985, no. 2, pp. 27–29. (in Russ.)
15. Golovko N.I., Karetnik V.O., Peleshok O.V. Automatic Control and Computer Sciences, 2009, no. 10, pp. 75–96. (in Russ.)
16. Bondrova O.V., Krylova D.S., Golovko N.I., Zhuk T.A. Proceedings of Voronezh State University. Series: Physics. Mathematics, 2015, no. 4, pp. 89–100. (in Russ.)
17. Miller A.B., Miller B.M., Stepanyan K.V. Automation and Remote Control, 2020, no. 3, pp. 469–482, DOI: 10.31857/S0005231020030071.
Review
For citations:
Zemskov A.V. Analytical Approach to the Study of Heterogeneous Markov Chains with Piecewise Constant Changes in Transition Probabilities. Journal of Instrument Engineering. 2024;67(8):657-669. (In Russ.) https://doi.org/10.17586/0021-3454-2024-67-8-657-669
Views:
21
Email this article 