Perturbed Semi-Markov Type Processes I: Limit Theorems for Rare-Event Times and Processes 3030924025, 9783030924027

Citation preview

Dmitrii Silvestrov

Perturbed Semi-Markov Type Processes I Limit Theorems for Rare-Event Times and Processes

Perturbed Semi-Markov Type Processes I

Dmitrii Silvestrov

Perturbed Semi-Markov Type Processes I Limit Theorems for Rare-Event Times and Processes

123

Dmitrii Silvestrov Department of Mathematics Stockholm University Stockholm, Sweden

ISBN 978-3-030-92402-7 ISBN 978-3-030-92403-4 (eBook) https://doi.org/10.1007/978-3-030-92403-4 Mathematics Subject Classification: 60J10, 60J22, 60J27, 60K05, 60K15, 60K20, 65C40 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

This book is the first volume of a two-volume monograph devoted to the study of limit and ergodic theorems for regularly and singularly perturbed Markov chains, semi-Markov processes, and alternating regenerative processes with semi-Markov modulation. The first volume presents necessary and sufficient conditions for weak convergence for first-rare-event times and convergence in the topology J for first-rare-event processes defined on regularly perturbed finite Markov chains and semi-Markov processes and also new asymptotic recurrent algorithms of phase space reduction and effective conditions of weak convergence for distributions of hitting times and convergence of expectations of hitting times for regularly and singularly perturbed finite Markov chains and semi-Markov processes. The second volume presents new super-long, long, and short time ergodic theorems for perturbed multi-alternating regenerative processes modulated by regularly or singularly perturbed finite semi-Markov processes. Models of perturbed Markov chains and semi-Markov processes, in particular for the most complex cases of the so-called singularly perturbed processes, attracted the attention of researchers in the middle of the twentieth century. Interest to these models has been stimulated by applications to control and queuing systems, information networks, epidemic models, and models of mathematical genetics and population dynamics. As a rule, Markov-type processes with singular perturbations arise as a natural tool for the mathematical analysis of multi-component systems with weakly interacting components. Semi-Markov processes are a natural generalisation of discrete and continuous time Markov chains. These jump processes possess the Markov property at the moments of jumps and can have arbitrary distributions concentrated on a positive half-line for the times between jumps. In fact, this combination of basic properties makes semi-Markov processes a very flexible and effective tool for constructing new classes of stochastic processes, for example, alternating regenerative processes with semi-Markov modulation, and describing various applied stochastic models such as queuing, reliability and bio-stochastic systems, stochastic networks, financial and insurance processes, etc. v

vi

Preface

As for Markov chains, hitting times play a very important role in the theory of semi-Markov processes and their applications. These random functionals are also known by names such as first-rare-event times, first passage times, absorption times, in theoretical studies, and also lifetimes, failure times, extinction times, etc., in applications. Hitting times and their expectations play a key role in ergodic theorems because of the dual relations connecting the corresponding stationary distributions and expectations of the return times, the laws of large numbers, central limit theorems, and more general limit theorems for additive, extremal, and other types of functionals for Markov type processes. The most deeply investigated are cases of Markov chains and semi-Markov processes with finite phase spaces. There is a huge bibliography of works that contain limit theorems for hitting times and related functionals for perturbed finite Markov chains and semi-Markov type processes. However, the theory of such limit and ergodic theorems is still far from completion. The results presented in the book, I hope, confirm this point of view well. The first volume includes an introduction, 11 chapters grouped in two parts, and two appendices. In Introduction (Chap. 1), simple examples, models of stochastic processes, conditions, and results are presented in an informal way. It also provides chapter-by-chapter content and additional information for potential readers. Part I (Chaps. 2–7) presents necessary and sufficient conditions for weak convergence for the first-rare-event times and convergence in the topology J for the first-rare-event processes defined on regularly perturbed finite Markov chains and semi-Markov processes. Similar results are also obtained for counting processes generated by flows of rare events defined on regularly perturbed finite Markov chains and semi-Markov processes. These results are extended to some more general models, such as vector rare-event times and processes, first-rare-event times for Markov renewal process with transition period and extending phase spaces, and others. The above results are illustrated by applications to risk processes and perturbed queuing systems. Part II (Chaps. 8–12) presents new asymptotic recurrent algorithms of phase space reduction and effective conditions of weak convergence for distributions of hitting times and convergence of expectations for hitting times for regularly and singularly perturbed semi-finite Markov processes. In such models, the phase space is one class of communicative states for embedded Markov chains of pre-limiting perturbed semi-Markov processes, but it can have an arbitrary communicative structure, i.e., it can consist of one (regular perturbations) or several (singular perturbations) closed classes of communicative states and, possibly, the class of transient states for the corresponding limiting embedded Markov chain. Efficient asymptotic forward and backward recurrent algorithms are presented that allow one to find suitable normalisation functions, obtain convergence relations for distributions and expectations of hitting times, and compute limiting Laplace transforms for distributions and limits for expectations of hitting times. Appendix A contains some limit theorems for randomly stopped stochastic processes, which are essential in the book.

Preface

vii

Appendix B contains some methodological and bibliographical notes, as well as comments on the new results presented in the book and some new problems for future research. I hope that the publication of this new book on asymptotic problems for perturbed stochastic processes will be a useful contribution to the ongoing intensive research in this area. In addition to being used for research and reference purposes, the book can also be used in special courses on this topic and as additional reading to general courses on stochastic processes. In this regard, it can be useful for both specialists as well as doctoral and senior students. I would also like to thank my colleagues at the Department of Mathematics, Stockholm University and at the Division of Mathematics and Physics, School of Education, Culture and Communication, Mälardalen University for creating an inspiring research environment and a friendly atmosphere, which stimulated my work. Stockholm, Sweden October 2021

Dmitrii Silvestrov

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v

List of symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xv

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Part I: First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Part I: Contents, Examples, Models, Results . . . . . . . . 1.1.2 Part I: Contents by Chapters . . . . . . . . . . . . . . . . . . . . . . 1.2 Part II: Hitting Times and Phase Space Reduction for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Part II: Contents, Examples, Models, Results . . . . . . . . 1.2.2 Part II: Contents by Chapters . . . . . . . . . . . . . . . . . . . . . 1.3 Appendices and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Appendix A: Limit Theorems for Randomly Stopped Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Appendix B: Methodological and Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

1

Part I

2

1 1 9 10 10 23 24 24 24 24

First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes

Asymptotics of First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 First-Rare-Event Times for Perturbed Semi-Markov Processes . 2.1.1 First-Rare-Event Times . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Asymptotically Uniformly Ergodic Markov Chains . . . 2.1.3 Necessary and Sufficient Conditions for Convergence in Distribution of First-Rare-Event Times . . . . . . . . . . .

29 30 30 32 34

ix

x

Contents

2.2

2.3

2.4

3

4

Asymptotics of Step-Sum Reward Processes . . . . . . . . . . . . . . . . 2.2.1 Necessary and Sufficient Conditions for Convergence in Distribution for Step-Sum Reward Processes . . . . . . 2.2.2 Examples of Step-Sum Reward Processes . . . . . . . . . . Asymptotics of First-Rare-Event Times for Perturbed Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Convergence in Distribution for First-Rare-Event Times for Perturbed Markov Chains . . . . . . . . . . . . . . . J-Convergence for First-Rare-Event Time Processes 2.3.2 for Perturbed Markov Chains . . . . . . . . . . . . . . . . . . . . . Asymptotics of First-Rare-Event Times for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Convergence in Distribution for First-Rare-Event Times for Perturbed Semi-Markov Processes . . . . . . . . 2.4.2 J-Convergence for First-Rare-Event Processes for Perturbed Semi-Markov Process . . . . . . . . . . . . . . . . . .

Flows of Rare Events for Regularly Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Counting Processes Generated by Flows of Rare Events . . . . . . . 3.1.1 Counting Processes for Rare Events . . . . . . . . . . . . . . . 3.1.2 Necessary and Sufficient Conditions of Convergence for Counting Processes Generated by Flows of Rare Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Markov Renewal Processes Generated by Flows of Rare Events 3.2.1 Return Times and Rare Events . . . . . . . . . . . . . . . . . . . . 3.2.2 Necessary and Sufficient Conditions of Convergence for Markov Renewal Processes Generated by Flows of Rare Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Vector Counting Processes Generated by Flows of Rare Events . 3.3.1 Vector Counting Processes for Rare Events . . . . . . . . . 3.3.2 Necessary and Sufficient Conditions of Convergence for Vector Counting Process Generated by Flows of Rare Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generalisations of Limit Theorems for First-Rare-Event Times . . . . 4.1 Modifications of First-Rare-Event Times and Rare-Event Time Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 Vector First-Rare-Event Times and Rewards . . . . . . . . 4.1.2 Upper and Lower First-Rare-Event Times . . . . . . . . . . . 4.1.3 First-Rare-Event Times for Markov Renewal Processes with Transition Periods . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 First-Rare-Event Times for Markov Renewal Processes with Extending Phase Spaces . . . . . . . . . . . . . . . . . . . . .

39 39 49 52 52 54 56 56 62 67 67 68

69 72 72

81 85 86

88 93 93 94 98 102 111

Contents

4.2

5

6

7

xi

First-Rare-Event Times and Hitting Times . . . . . . . . . . . . . . . . . . 4.2.1 Necessary and Sufficient Conditions of Convergence in Distribution for Standard Hitting Times . . . . . . . . . . 4.2.2 Necessary and Sufficient Conditions of Convergence in Distribution for Directed Hitting Times . . . . . . . . . .

119 119 126

First-Rare-Event Times for Perturbed Risk Processes . . . . . . . . . . . . 5.1 Stable Asymptotics of First-Rare-Event Times . . . . . . . . . . . . . . . 5.2 Necessary and Sufficient Conditions for Stable Approximation of Non-ruin Distribution Functions . . . . . . . . . . . . . . . . . . . . . . . .

133 133

First-Rare-Event Times for Perturbed Closed Queuing Systems . . . 6.1 Queuing Systems with Rare Events Caused by Too Long Service Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Asymptotic Uniform Ergodicity for Birth–Death Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 M/M-Type Queuing System with Rare Events Caused by Too Long Service Times . . . . . . . . . . . . . . . . . . . . . . 6.2 Queuing Systems with Highly Reliable Servers . . . . . . . . . . . . . .

145

First-Rare-Event Times for Perturbed M/M-Type Queuing Systems 7.1 Rare Events for Queuing Systems with Bounded Buffers . . . . . . 7.1.1 Queuing Systems with a Buffer of Fixed Size . . . . . . . 7.1.2 Queuing Systems with Asymptotically Unbounded Buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Rare Events for Queuing Systems with Unbounded Buffers . . . .

165 165 166

141

145 145 146 150

177 188

Part II Hitting Times and Phase Space Reduction for Perturbed Semi-Markov Processes 8

Asymptotically Comparable Functions . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Complete Families of Asymptotically Comparable Functions . . 8.1.1 Definitions for Families of Asymptotically Comparable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Operating Rules for Asymptotically Comparable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Examples of Complete Families of Asymptotically Comparable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 Asymptotically Comparable Power-Type Functions . . . 8.2.2 Asymptotically Comparable Power-Exponential-Type Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.3 Asymptotically Comparable Power-Logarithmic-Type Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

203 203 203 206 207 208 211 214

xii

9

10

Contents

Perturbed Semi-Markov Processes and Reduction of Phase Space . 9.1 Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.1 Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . 9.1.2 Perturbation Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.3 Additional Asymptotic Comparability Conditions . . . . 9.1.4 Perturbed Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Removing of Virtual Transitions for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Semi-Markov Processes with Removed Virtual Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 Perturbation Conditions for Semi-Markov Processes with Removed Virtual Transitions . . . . . . . . . . . . . . . . . 9.3 One-State Reduction of Phase Space for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 A Procedure of One-Step Phase Space Reduction for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . 9.3.2 Perturbation Conditions for Reduced Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Recurrent Reduction of Phase Space for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 The Recurrent Phase Space Reduction Algorithm . . . . 9.4.2 Summary of Recurrent Phase Space Reduction Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

219 220 220 221 225 228

Asymptotics of Hitting Times for Perturbed Semi-Markov Processes 10.1 Hitting Times for Perturbed Semi-Markov Processes . . . . . . . . . 10.1.1 Hitting Times and Related Asymptotic Problems . . . . 10.1.2 Hitting Times and Reduced Perturbation Conditions for Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . 10.1.3 Hitting Times for Semi-Markov Processes with Reduced Phase Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.4 Recurrent Relations for Distributions and Laplace Transforms of Hitting Times . . . . . . . . . . . . . . . . . . . . . 10.2 Weak Asymptotics for Distributions of Hitting Times . . . . . . . . . 10.2.1 Asymptotics for Hitting Probabilities . . . . . . . . . . . . . . 10.2.2 Weak Asymptotic for Distributions of Hitting Times for the Case Where the Initial State Belongs to Domain ¯ ............................................ D 10.2.3 Admissible Normalisation Functions and Phase Types of Limiting Distributions . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Weak Asymptotics for Distributions of Return Times . . . . . . . . . 10.3.1 Hitting and Return Times . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 Weak Asymptotics for Distributions of Hitting and Return Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

261 262 262

228 229 231 239 239 242 250 250 255

266 269 274 277 277

280 292 294 294 296

Contents

11

Asymptotics for Expectations of Hitting Times for Perturbed Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Expectations of Hitting Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Recurrent Relations for Expectations of Hitting Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Recurrent Relations for Limits of Expectations of Hitting Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Asymptotics of Expectations of Hitting Times . . . . . . . . . . . . . . . 11.2.1 Asymptotics for Expectations of Hitting Times for the ¯ ... Case Where an Initial State Belongs to Domain D 11.2.2 Conditions of Simultaneous Convergence for Distributions and Expectations of Hitting Times . . . . . 11.3 Asymptotics for Expectations of Return Times . . . . . . . . . . . . . . 11.3.1 Asymptotic for Expectations of Hitting Times in the Case Where an Initial State Belongs to Domain D . . . 11.3.2 Asymptotics of Expectations for Return Times to Domain D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiii

307 307 308 309 311 311 317 322 322 324

Generalisations and Examples of Limit Theorems for Hitting Times 12.1 Generalisations of Limit Theorems for Hitting Times . . . . . . . . . 12.1.1 More General Perturbation Conditions . . . . . . . . . . . . . 12.1.2 More General Hitting Reward Functionals . . . . . . . . . . 12.2 Hitting Times for Perturbed Birth–Death-Type Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Perturbed Birth–Death-Type Semi-Markov Processes . 12.2.2 Phase Space Reduction for Perturbed Birth–Death-Type Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 An Example of Singularly Perturbed Semi-Markov ¯ ............... Processes with Two-State Domain D 12.3.2 A One-Step Asymptotic Reduction of Phase Space . . . 12.3.3 Asymptotics for Distributions and Expectations of Hitting Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

327 327 328 330

Limit Theorems for Randomly Stopped Stochastic Processes . . . . . . . . . . A.1 Functional Limit Theorems for Càdlàg Stochastic Processes . . . A.1.1 Convergence of Càdlàg Stochastic Processes in Topology U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1.2 Convergence of Càdlàg Stochastic Processes in Topology J . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1.3 Convergence of Step-Sum Processes with Independent Increments in Topology J . . . . . . . . . . . . . . . . . . . . . . . .

355 355

12

333 333 337 343 343 345 350

355 356 357

xiv

Contents

A.2

Limit Theorems for Randomly Stopped Stochastic Processes and Superpositions of Stochastic Processes . . . . . . . . . . . . . . . . . A.2.1 Convergence in Distribution for Randomly Stopped Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2.2 Convergence in Distribution for Superpositions of Càdlàg Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2.3 Convergence of Superpositions of Càdlàg Processes in Topology J . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Supplementary Asymptotic Results . . . . . . . . . . . . . . . . . . . . . . . . A.3.1 Slutsky Theorem and Related Results . . . . . . . . . . . . . . A.3.2 Functional Analogues of Slutsky Theorem . . . . . . . . . .

359 360 360 360

Methodological and Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1 Methodological Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 General Bibliographical Remarks . . . . . . . . . . . . . . . . . . . . . . . . .

363 363 370

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

375

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

399

A.3

358 358 359

List of Symbols

⇒ d

−→ P

−→ a.s.

−→ J

−→ U

−→ ∼ ε ε→0 d

=

Symbol of weak convergence Symbol of convergence in distribution Symbol of convergence in probability Symbol of almost sure convergence Symbol of convergence in topology J Symbol of convergence in topology U Symbol of asymptotic equivalence Perturbation parameter A shorter variant of symbol 0 < ε → 0 Symbol of equality for distributions

a.s.

= Symbol of almost sure equality ∗ Symbol of convolution (∗n) Symbol of n-fold convolution [u] Integer part of a real number u R+ Interval [0, ∞) B+ Borel σ-algebra of subsets of R+ m(A) Lebesgue measure on σ-algebra B+ X A finite phase space {1, . . . , m} BX The σ-algebra of all subsets of the space X Z A phase space of a stochastic process (an arbitrary set) BZ A σ-algebra of measurable subsets of space Z E xpa (·) Exponential distribution function with parameter a

xv

xvi

List of Symbols

Geop (·) Cona (·) A, B, C, . . . ηε,n k ηε,n, k¯ n ηε,n, . . . κε,n ρε,n ζε,n ηε (t) k ηε (t),

k¯ n ηε (t), . . .

ξε (t) Nε (t), N¯ ε (t), . . . νε ξε νε,D k νε , k¯ n νε,D τε,D k τε,D, τε,D

k¯ n τε,D, . . .

Geometrical distribution function with parameter p Distribution function with the unit jump in point a Conditions Embedded Markov chain Reduced embedded Markov chains Transition time Indicator of rare event Moment of jumps Semi-Markov process Reduced semi-Markov processes Rare-event-time process Counting processes generated by flows of rare events First-rare-event time for embedded Markov chain First-rare-event time for semi-Markov process Hitting time for embedded Markov chain Hitting times for reduced embedded Markov chains Hitting time for semi-Markov process Hitting times for reduced semi-Markov processes Return time for semi-Markov process

Q ε,i j (t) ei j (ε)

Transition probability of semi-Markov process Expectation of transition time for semi-Markov processes ψε,i j (s) Laplace transform of transition probability of semi-Markov process k Q ε,i j (t), k¯ n Q ε,i j (t), . . . Transition probabilities for reduced k eε,i j ,

k¯ n eε,i j , . . .

semi-Markov processes Expectations of transition time for reduced

semi-Markov processes k ψε,i j (s), k¯ n ψε,i j (s), . . . Laplace transform of transition probability pε,i j k pε,i j , Fε,i j (t) fε,i j

k¯ n pε,i j , . . .

for reduced semi-Markov processes Transition probability for embedded Markov chain Transition probabilities for reduced embedded Markov chains Conditional distribution function of transition time Conditional expectation of transition time

List of Symbols

φε,i j (s)

xvii

k Fε,i j (t), k¯ n Fε,i j (t), . . .

Laplace transform for conditional distribution function of transition time Conditional distribution functions of transition

k fε,i j ,

times for reduced semi-Markov processes Conditional expectations of transition times

k¯ n fε,i j , . . .

k φε,i j (s), k¯ n φε,i j (s), . . .

Pε,D,i j G ε,D,i j (t) Eε,D,i j Ψε,D,i j (s) k Pε,D,i j , k¯ n Pε,D,i j

for reduced semi-Markov processes Laplace transform for conditional distribution functions of transition times for reduced semi-Markov processes Hitting probability for semi-Markov process Distribution function of hitting time Expectation of hitting time Laplace transform for distribution function of hitting time Hitting probabilities for reduced

semi-Markov processes k G ε,D,i j (t), k¯ n G ε,D,i j (t), . . . Distribution functions of hitting times k Eε,D,i j ,

k¯ n Eε,D,i j

for reduced semi-Markov processes Expectations of hitting times for reduced

semi-Markov processes k Ψε,D,i j (s), k¯ n Ψε,D,i j (s), . . . Laplace transforms for distribution functions of MC MRP SMP

hitting times for reduced semi-Markov processes Markov chain Markov renewal process Semi-Markov process

Chapter 1

Introduction

This book is devoted to the study of limit theorems for first-rare-event times and hitting times for regularly and singularly perturbed semi-Markov processes, which is the main objects of study in the book. The basic fact concerned these processes can be found in the books listed in Sect. B.2.7. The introduction aims to informally present the main problems, methods, and algorithms that make up the content of the book. We give simple examples, illustrated by figures, and try to show the logic and ideas underlying the methods of asymptotic analysis of perturbed semi-Markov type processes developed in the book, as well as explain the meaning of the results presented in the book. We also describe the content of the book in parts and chapters and provide additional information for potential readers of the book.

1.1 Part I: First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes In Sect. 1.1, we outline the circle of problems and illustrate results presented in Part I.

1.1.1 Part I: Contents, Examples, Models, Results 1.1.1.1 Content of Part I. In this part, which includes Chaps. 2–7, we give necessary and sufficient condition of convergence in distribution (equivalent to weak convergence of distributions) for first-rare-event times and J-convergence for first-rare-event time processes defined on regularly perturbed Markov chains and semi-Markov processes and counting processes generated by the corresponding flows of rare events. These results are also illustrated by some applications to be perturbed risk processes and perturbed queuing systems.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_1

1

2

1 Introduction

Limit theorems for random functionals of first-rare-event time type for Markov type processes have been studied by many researchers. The case of Markov chains and semi-Markov processes with finite phase spaces is the most deeply investigated. The main features for the most previous results are that they give sufficient conditions of convergence for such functionals. As a rule, these conditions involve assumptions, which imply weak convergence for sums of i.i.d. random variables distributed as sojourn times for the semi-Markov process (for every state) to some infinitely divisible laws, plus some ergodicity condition for the embedded Markov chain, plus condition of vanishing probabilities of occurring a rare event during one transition step for the semi-Markov process. The results of Part I relate to the model of regularly perturbed semi-Markov processes with finite phase space. Instead of conditions based on “individual” distributions of sojourn times, we use more general and weaker conditions imposed on distributions sojourn times averaged over the stationary distributions of the corresponding embedded Markov chains. Moreover, we show that these conditions are not only sufficient but also necessary conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event time processes. These results give some kind of a “final solution” for limit theorems for first-rare-event times and first-rare-event time processes for regularly perturbed semi-Markov process with a finite phase space. 1.1.1.2 A Model Example. Let (ηε,n, κε,n, ρε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1],1 a Markov renewal process, i.e., a discrete time, homogenous Markov chain with a phase space Z = X × [0, ∞) × {0, 1}, where X = {1, 2}, and transition probabilities defined for i, j ∈ X, s, t ≥ 0, ı, j = 0, 1, Q ε,i j (t, j) = I(t ≥ eε,i, j )pε,i j pε,i ( j) = P{ηε,1 = j, κε,1 ≤ t, ρε,1 = j/ηε,0 = i, κε,0 = s, ρε,0 = ı},

(1.1)

where: (a) eε,i, j > 0, i ∈ X, j = 0, 1, (b) pε,i j ≥ 0, i, j ∈ X, pε,i1 + pε,i2 = 1, i ∈ X, (c) pε,i ( j) ∈ [0, 1], i ∈ X, j = 0, 1, pε,i (0) + pε,i (1) = 1, i ∈ X. Let Sε be a queuing system, the functioning of which is described by the Markov renewal process (ηε,n, κε,n, ρε,n ). The system Sε can operate in two modes say 1 and 2 (which, for example, can be interpreted as “low” and “high” intensity modes) switching modes at moments ζε,n = κε,1 + · · · + κε,n, n = 0, 1, . . .. If ηε,n = i ∈ X and ρε,n+1 = 0, then the working period [ζε,n, ζε,n+1 ) is “successful” and its duration is κε,n+1 = eε,i,0 . If ηε,n = i ∈ X and ρε,n+1 = 1, then the working period [ζε,n, ζε,n+1 ) is “unsuccessful,” its duration is κε,n+1 = eε,i,1 , and a failure of the system Sε is registered at the end moment of this working period ζε,n+1 . According to the above remarks, the random variable ηε,n is the indicator of mode in the (n + 1)-th working period [ζε,n, ζε,n+1 ). The random variable κε,n+1 is the duration of this working period. The binary random variable ρε,n+1 indicates successful/unsuccessful type of the (n + 1)-th working period. It can be referred as a failure indicator. 1

Here and below, ε plays the role of a perturbation parameter.

1.1 Part I

3

According to the above description, eε,i, j is a duration of working period with the work regime i and successful/unsuccessful type of working period j, pε,i j is the conditional probability of switching the type i of working period to the type j for the next working period, and pε,i (1) is the failure probability for the working period of the type i. The object of our interest is the random functional, ξε =

νε 

κε,n,

(1.2)

n=1

where νε = min(n ≥ 1 : ρε,n = 1).

(1.3)

The random variable ξε can be referred to as the first failure time for the queuing system Sε . The characteristic property of the Markov renewal process (ηε,n, κε,n, ρε,n ) is that its transition probabilities depend only on the current state of the first component and do not depend on states of other components. In this case, the first component ηε,n is itself a homogeneous Markov chain with the phase space X and transition probabilities pε,i j , i, j ∈ X. Also, the Markov renewal process (ηε,n, κε,n, ρε,n ) can be used for construction of the semi-Markov process defined as ηε (t) = ηε,n , for ζε,n ≤ t < ζε,n+1, n = 0, 1, . . ., where ζε,n = κε,1 + · · · + κε,n are moments of jumps for the process ηε (t). As far as the binary random variables ρε,n are concerned, they are used as indicators of “rare” events {ρε,n = 1}. The random variable ξε can be referred to as the first-rare-event time for the semi-Markov process ηε (t). The possibility to interpret events {ρε,n = 1} as rare is supported by the “rare-event” condition: (1) The failure probabilities 0 < pε,i (1) → 0 as ε → 0,2 for i ∈ X. We also assume that the Markov chains ηε,n are asymptotically uniformly ergodic. This means that the following condition is satisfied: (2) The regime switching probabilities pε,12, pε,21 are asymptotically separated from 0, i.e., limε→0 pε,12 , limε→0 pε,21 > 0. The condition (2) implies that the Markov chain ηε,n is ergodic for ε small enough and its stationary probabilities take the following forms: πε,1 =

pε,21 pε,12 , πε,2 = . pε,12 + pε,21 pε,12 + pε,21

(1.4)

Moreover, the above stationary probabilities are asymptotically separated from 0 and 1, i.e., (1.5) 0 < lim πε,i ≤ lim πε,i < 1, for i ∈ X. ε→0

2

ε→0

In what follows, ε → 0 is a shorten version of the symbol 0 < ε → 0.

4

1 Introduction

Let us define, for ε ∈ (0, 1], the averaged (over the above stationary distribution) failure probability, (1.6) pε = pε,1 (1) πε,1 + pε,2 (1) πε,2, and the averaged duration of successful working period, eε = eε,1,0 πε,1 + eε,2,0 πε,2 .

(1.7)

Finally, we also assume that the duration of unsuccessful working period makes an asymptotically negligible contribution in the first failure time. This means that the following condition is satisfied: (3) pε eε,i,1 → 0 as ε → 0, for i ∈ X. The asymptotic results presented in Part I let us state that, under the assumption that conditions (1)–(3) are satisfied, the asymptotic relation, eε → e0 ∈ (0, ∞) as ε → 0,

(1.8)

is the necessary and sufficient condition for the fulfilment of the following asymptotic relation, d (1.9) pε ξε −→ ξ0 as ε → 0, where ξ0 is a proper non-negative random variable, which has distribution not concentrated in 0. Moreover, in this case, Ee−sξ0 = (1 + e0 s)−1, s ≥ 0, i.e., the random variable ξ0 is exponentially distributed with parameter e0−1 . Importantly, neither the convergence of the switching probabilities pε,12 and pε,21 nor the convergence of the parameters eε,1,0 , as ε → 0, is not required. These parameters can fluctuate as functions of ε → 0. The only convergence of one “leading ” parameter eε , as ε → 0, is required. It is also worth commenting on the role of the condition (3). If the duration of working period does not depend on the value of the failure indicator, i.e., eε,i,1 = eε,i,0, ε ∈ 0, 1], for i ∈ X, then the first failure time can be represented in the form, ξε =

νε  k=1

κε,k =

νε 

eε,ηε, k−1,0 = ξε,0 .

(1.10)

k=1

Under the assumption that the conditions (1)–(2) are satisfied, any of the relations (1.8) or (1.9) implies that the condition (3) is satisfied, and, thus, this condition can be omitted in the above statement about the equivalence of relations (1.8) and (1.9). If the duration of working period depends on the value of the failure indicator, then the condition (3) cannot be omitted. For example, let us assume that eε,1 (1) = eε,2 (1) = eε (1), ε ∈ (0, 1]. In this case, the first failure time can be represented in the form,

1.1 Part I

5

ξε =

νε 

κε,k =

k=1

ν ε −1

eε,ηε, k−1,0 + eε (1)

k=1

= ξε,0 − eε,ηε,νε −1,0 + eε (1).

(1.11)

The normalised first failure times pε ξε may not converge in distribution to a proper random variable. Indeed, according to the above remarks, the conditions (1)–(2) and the relation d

(1.8) imply that (a) pε ξε,0 −→ ξ0 as ε → 0. Also, the relations (1.5) and (1.8) imply that (b) pε eε,ηε,νε −1,0 ≤ pε max(eε,1,0, eε,2,0 ) → 0 as ε → 0. The asymptotic relations (a) and (b) imply that the random variables pε ξε do not converge in distribution to a proper random variable if the quantities pε eε (1) do not converge to P

a finite limit, as ε → 0. Also, pε ξε −→ ∞ as ε → 0, if pε eε (1) → ∞ as ε → 0. In conclusion, let us mention that it is possible to interpret the first-rare-event time ξε as the first hitting time for some semi-Markov process. Indeed, the two-component random sequence (ηε,n, ρε,n ), n = 0, 1, . . . is also a ˆ = {(1, 0), (2, 0), (1, 1), (2, 1)} and homogeneous Markov chain with the phase space X ˆ The corresponding transition probabilities pε,(i,ı),(j, j) = pε,i j pε,i ( j), (i, ı), ( j, j) ∈ X. two-component semi-Markov process can be defined as (ηε (t), ρε (t)) = (ηε,n, ρε,n ), for ζε,n ≤ t < ζε,n+1, n = 0, 1, . . ., where ζε,n = κε,1 + · · · + κε,n are moments of jumps for process (ηε (t), ρε (t)). The first-rare-event time ξε is, in fact, the first hitting time in the domain D = {(1, 1), (2, 1)} for the semi-Markov process (ηε (t), ρε (t)), i.e., ξε =

ν ε,D

κε,k ,

(1.12)

k=1

where νε,D = min(n ≥ 1 : (ηε,n, ρε,n ) ∈ D).

(1.13)

The result presented in the above example is a very particular case of the results given in Part I. In this part, we consider the first-rare-event times (or, equivalently, first hitting times) for perturbed finite semi-Markov processes. Transition times for semi-Markov processes have arbitrary distributions concentrated of positive halfline. The concept of asymptotically uniformly ergodic embedded Markov chains is introduced. In this case, neither the convergence of its transition probabilities nor the convergence of the corresponding stationary probabilities is not required. We give necessary and sufficient conditions for convergence in distribution of first-rare-event times for perturbed semi-Markov processes (based on one-step probabilities of rare events and distributions of sojourn times averaged over the stationary distributions of asymptotically uniformly ergodic embedded Markov chains) and show that the corresponding limiting distribution has the Laplace transform of the form (1 + A(s))−1 , where A(s) is a cumulant of some infinitely divisible distribution concentrated on the positive half-line. Also, we give necessary and sufficient conditions for convergence in J-topology for first-rare-event processes and related counting processes as

6

1 Introduction

well as generalise the above results to more general modifications of first-rare-event times and processes. These results are illustrated by applications to perturbed risk processes and perturbed queuing systems. 1.1.1.3 First-Rare-Event Times and Related Processes for Regularly Perturbed Semi-Markov Processes. Let (ηε,n, κε,n, ρε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, i.e., a discrete time, homogenous Markov chain with a phase space Z = X × [0, ∞) × {0, 1}, where X = {1, 2, . . . , m} is a finite set, and transition probabilities, defined for i, j ∈ X, s, t ≥ 0, ı, j = 0, 1, Q ε,i j (t, j) = P{ηε,1 = j, κε,1 ≤ t, ρε,1 = j/ηε,0 = i, κε,0 = s, ρε,0 = ı}.

(1.14)

The characteristic property of the Markov renewal process is that its transition probabilities depend only on the current state of the first component and do not depend on states of other components. In this case, the first component ηε,n is itself a homogeneous Markov chain with transition probabilities pε,i j = Q ε,i j (∞, 0) + Q ε,i j (∞, 1), i, j ∈ X. Also, the above Markov renewal process can be used for construction of semi-Markov process defined as ηε (t) = ηε,n , for ζε,n ≤ t < ζε,n+1, n = 0, 1, . . ., where ζε,n = κε,1 + · · · + κε,n are moments of jumps for the semi-Markov process ηε (t). As far as random variables ρε,n are concerned, they are used as indicators of “rare” events {ρε,n = 1}. The object of studies in Part I is the random functional, ξε =

νε 

κε,n,

(1.15)

n=1

where νε = min(n ≥ 1 : ρε,n = 1).

(1.16)

The random variable ξε can be referred to as the first-rare-event time for the semi-Markov process ηε (t). The possibility to interpret events {ρε,n = 1} as rare is supported by the “rare-event” condition:  (A): 0 < maxi ∈X pε,i → 0 as ε → 0, where pε,i = j ∈X Q ε,i j (∞, 1), i ∈ X. Figure 1.1 illustrates the construction described above. In the example, illustrated by this figure, the first two indicator random variables ρε,1, ρε,2 = 0, while the indicator random variables ρε,3 = 1. Thus, νε = 3 and the first-rare-event time ξε = κε,1 + κε,2 + κε,3 . Other objects of research in Part I are the first-rare-event processes,  ξε (t) = κε,k , t ≥ 0 (1.17) n ≤tνε

and the counting processes generated by flows of rare events such as Nε (t) = max(n :

ν ε (n) k=1

κε,k ≤ t), t ≥ 0,

(1.18)

1.1 Part I

7

Fig. 1.1 First-rare-event time ξε

where νε (n) = min(k :

k 

ρε,r = n).

(1.19)

r=1

1.1.1.4 Asymptotically Uniformly Ergodic Markov Chains. Let ηε,n, n = 0, 1, . . ., be, for every ε ∈ (0, 1], a finite Markov chain, with phase space X = {1, . . . , m} and the matrix of transition probabilities Pε = pε,i j . The standard perturbation model is based on the assumption that the transition probabilities pε,i j → p0,i j as ε → 0, for i, j ∈ X. In this case, the matrix P0 = p0,i j  is also stochastic. Let η0,n, n = 0, 1, . . ., be a Markov chain with the phase space X and the matrix of transition probabilities P0 . The above convergence relation for the transition probabilities pε,i j allows us to consider the Markov chain ηε,n as a perturbed version of the Markov chain η0,n . In the regular perturbation model, it is assumed that the phase space X is one class of communicative states for the limiting Markov chain η0,n . Thus, the phase space X also is one class of communicative states for the pre-limiting Markov chain ηε,n , for ε ∈ (0, ε0 ], for some ε0 ∈ (0, 1]. In this case, there exists the unique stationary distribution π¯ ε = πε,i, i ∈ X for the Markov chain ηε,n , for ε ∈ [0, ε0 ] and πε,i → π0,i > 0 as ε → 0, for i ∈ X. The concept of asymptotically uniformly ergodic Markov chains let us generalise the regular perturbation model to the model, where it is only assumed that: (B): There exists a chain of states i0, i1, . . . , i N = i0 , which contains all states from the phase space X and such that limε→0 pε,ik−1 ik > 0, for k = 1, . . . , N. As in the regular perturbation model, the phase space X is one class of communicative states for the pre-limiting Markov chain ηε,n , for ε ∈ (0, ε0 ], for some ε0 ∈ (0, 1]. In this case, there exists the unique stationary distribution π¯ ε = πε,i, i ∈ X for Markov chain ηε,n , for ε ∈ (0, ε0 ]. Instead of the convergence relation, the following

8

1 Introduction

relation holds: 0 < limε→0 πε,i ≤ limε→0 πε,i < 1, for i ∈ X. However, the convergence of transition probabilities pε,i j and stationary probabilities πε,i may not take place. Figure 1.2 illustrates the concept of asymptotically uniformly ergodic Markov chains. In this figure, transitions along the corresponding chain of states, with transition probabilities separated from 0, as ε → 0, are indicated by red arrows, while other transitions with positive probabilities are indicated by black arrows.

Fig. 1.2 Transition graph for asymptotically uniformly ergodic Markov chains

Obviously, the standard model of regularly perturbed Markov chains is a particular case of the model of asymptotically uniformly ergodic Markov chains. 1.1.1.5 Averaged Characteristics and Necessary and Sufficient Conditions  of Convergence. Let Fε,i (t) = j ∈X, j=0,1 Q ε,i j (t, j), t ≥ 0 be, for ε ∈ (0, 1], the distribution function of sojourn time in state i ∈ X for the semi-Markov process ηε (t). Let also θ ε,i,n, n = 1, 2, . . . be, for every ε ∈ (0, 1] and i ∈ X, a sequence of i.i.d. random variables with the distribution function Fε,i (·). As mentioned above, standard conditions of convergence for random functionals such as first-rare-event times, hitting times, etc., are usually based on the assumption  d that sums n ≤uε θ ε,i,n −→ θ 0,i as ε → 0,3 for i ∈ X. Here, θ 0,i, i ∈ X are some limiting random variables and 0 < uε → ∞ as ε → 0 is some specially chosen function connected with stopping probabilities pε,i defined above in the condition (A). We, however, use more general and weaker conditions imposed on distributions sojourn times averaged over the stationary distributions of the corresponding embedded Markov chains.  Let Fε (t) = i ∈X Fε,i (t)πε,i, t ≥ 0 be, for ε ∈ (0, 1], the distribution function of sojourn time Fε,i (t) averaged over the stationary distribution π¯ ε , and θ ε,n, n = 1, 2, . . . 3

d

−→ is the symbol of convergence in distribution for random variables.

1.1 Part I

9

be, for ε ∈ (0, 1], a sequence of i.i.d. random variables with the distribution function  , where p = p Fε (·). Also, let uε = p−1 ε ε,i πε,i . i ∈X ε Necessary and sufficient conditions of convergence in distributions for first-rareevent times ξε presented in Part I are based on three model assumptions (which are analogues of the conditions (1)–(3) used in the example considered in Sect. 1.1.1.2). The first is the “rare-event” condition (A). The second is the condition (B) of asymptotically uniform ergodicity of Markov chains ηε,n . The third is the following condition, which guarantees that the last summand κε,νε in the random sum ξε is asymptotically negligible: (C): Pi {κε,1 > δ/ρε,1 = 1} → 0 as ε → 0, for δ > 0 and i ∈ X. The role of necessary and sufficient condition of convergence in distribution for the first hitting times is played by the following condition:  d (D): n≤uε θ ε,n −→ θ 0 as ε → 0, where θ 0 is a proper non-negative random variable with distribution not concentrated at zero. It is well known that, in this case, the random variable θ 0 has an infinitely divisible distribution. Let A(s) be its cumulant, i.e., Ee−sθ0 = e−A(s), s ≥ 0. The central criteria of convergence (see, for example, Loève (1977)) also make it possible to reformulate the condition (D) in the ∫ uequivalent form of convergence relations for characteristics uε (1 − Fε (u)) and uε 0 sFε (ds). One of the main results presented in Part I is Theorem 2.1, which states that, under the assumption that the conditions (A), (B), and (C) are satisfied, the condition (D) is the necessary and sufficient condition for the fulfilment of the asymptotic relation, d

ξε (1) −→ ξ0 as ε → 0,

(1.20)

where ξ0 is a proper non-negative random variable with distribution not concentrated at zero. Moreover, the limiting random variable ξ0 has, in this case, the Laplace transform Ee−sξ0 = (1 + A(s))−1, s ≥ 0. J

Moreover, in this case, the rare-event processes ξε (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 ), t ≥ 0 as ε → 0,4 where (a) ν0 is a random variable, which has the exponential distribution with parameter 1, (b) θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0, and (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent.

1.1.2 Part I: Contents by Chapters Part I includes Chaps. 2–7. Chapter 2. In this chapter, we introduce first-rare-event times for asymptotically uniformly ergodic perturbed semi-Markov processes and give necessary and suf4

J

−→ is the symbol of convergence in Skorokhod J-topology for càdlàg stochastic processes.

10

1 Introduction

ficient conditions of convergence in distribution for first-rare-event times ξε and J-convergence for first-rare-event time processes ξε (t), t ≥ 0. Chapter 3. In this chapter, we introduce process Nε (t), t ≥ 0, which counts the number of rare events occurring on trajectory of semi-Markov process ηε (s) in time interval [0, t]. Necessary and sufficient conditions of convergence in distribution for counting processes Nε (t) are given. Analogous results are also obtained for some more general multivariate counting processes related to rare-event times. Chapter 4. In this chapter, we generalise results of Chap. 2 to some modifications of first-rare-event times and first-rare-event time processes, in particular, to vector, upper and lower first-rare-event times and first-rare-event time processes, first-rareevent times for Markov renewal process with transition period and extending phase spaces. Also, connection between first-rare-event times and standard and directed hitting times is clarified, and necessary and sufficient conditions of convergence in distribution for first hitting times for asymptotically uniformly ergodic perturbed semi-Markov processes are given. Chapter 5. In this chapter, we formulate conditions of convergence in distribution for rare-event times represented by geometric type random sums and give necessary and sufficient conditions for weak convergence of non-ruin distribution functions for perturbed risk processes in the models of stable and diffusion approximations for non-ruin distribution functions. Chapter 6. In this chapter we present necessary and sufficient conditions for convergence in distribution for first-rare-event times for a number of models of perturbed closed M/M-type queuing systems. Chapter 7. In this chapter we present necessary and sufficient conditions for weak convergence in distribution for first-rare-event times for a number of models of perturbed M/M-type queuing systems with bounded and unbounded buffers.

1.2 Part II: Hitting Times and Phase Space Reduction for Perturbed Semi-Markov Processes In Sect. 1.2, we outline the circle of problems and illustrate results presented in Part II.

1.2.1 Part II: Contents, Examples, Models, Results 1.2.1.1 Content of Part II. In this part, which includes Chaps. 8–12, we present new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed finite semi-Markov processes. Using these algorithms, we formulate effective conditions of weak convergence for distributions and convergence of expectations for hitting times and get effective recurrent formulas for computing

1.2 Part II

11

the corresponding normalisation functions, Laplace transforms for limiting distributions, and limits for expectations. The phase space reduction algorithms presented in Part II are universal. They can be applied to perturbed semi-Markov processes with an arbitrary asymptotic communicative structure and are computationally effective due to recurrent character of computational procedures. What is also important, for some models, for example, for perturbed birth–death-type semi-Markov processes, the above-mentioned algorithms of phase space reduction preserve birth–death structure of the original semi-Markov processes. 1.2.1.2 A Model Example. Let Sε be, for every ε ∈ (0, 1], a queuing system, the functioning of which is described by a continuous time, homogeneous Markov chain ηε (t), t ≥ 0, with the phase space X = {0, 1, 2} and continuous from the right trajectories. The system can operate in two “work” modes or in the “failure” mode. If ηε (t) = 1 or ηε (t) = 2, then the system is, respectively, in the operating mode 1 or 2 at the moment t. If ηε (t) = 0, then the system is in the failure mode at the moment t. The corresponding generator matrix has the following form, for ε ∈ (0, 1],    −λε,0 λε,01 λε,02  ⇐= 0   ⇐= 1 (1.21) Qε =  λε,10 −λε,1 λε,12   λε,20 λε,21 −λε,2  ⇐= 2  where: (a) λε,i j > 0, for i  j, i, j ∈ X, (b) λε,i = ji λε,i j , for i ∈ X. Figure 1.3 shows the transition graph, the transition intensities, and the parameters of exponentially distributed transition times for the Markov chain ηε (t).

Fig. 1.3 The transition graph for the Markov chain ηε (t)

Suppose that the initial state of the system ηε (0)  0 and describe asymptotic behaviour of the random variables that can be interpreted as the first failure times, ξε = inf(t > 0 : ηε (t) = 0).

(1.22)

12

1 Introduction

Note that, in the case where the initial state ηε (0)  0, the distribution of random variable ξε does not depend on the intensities λε,0i, i = 1, 2. Therefore, for simplicity, we can assume that λε,0i = 1, i = 1, 2, for ε ∈ (0, 1]. n κε,k , n = 0, 1, . . . Let κε,n, n = 1, 2, . . . be sequential transition times, ζε,n = k=1 be sequential moments of jumps, and ηε,n = ηε (ζε,n ), n = 0, 1, . . . be states of the Markov chain ηε (t) at moments of jumps. As is known, the random sequence ηε,n is a discrete time (embedded) Markov chain, with transition probabilities,  λ ε, i j for j  i, i, j ∈ X, (1.23) pε,i j = λε, i 0 for j = i, i ∈ X. Note that the probabilities pε,i j > 0, for j  i, i, j ∈ X. In what follows, we use limits for different sums of products of the intensities λε,i j and for quotients of such sums and should guarantee the existence of such limits. Using the concept of a complete family of asymptotically comparable functions let us solve this problem. A family H of positive functions h(ε) defined on interval (0, 1] is a complete family of asymptotically comparable functions if: (1) it is closed with respect to operations of summation, multiplication, and division and (2) the functions h(·) ∈ H have limits taking values in the interval [0, ∞], as ε → 0. The corresponding comments and examples of complete families of asymptotically comparable functions and the corresponding operating rules and formulas are given in Chap. 8. Let us assume that the following condition is satisfied: (4) The intensities λε,i j , j  i, i = 1, 2 belong to a complete family of comparable functions H and limε→0 λε,i j = λ0,i j ∈ [0, ∞), for j  i, i = 1, 2. The condition (4) implies that pε,i j → p0,i j ∈ [0, 1] as ε → 0, for j  i, i, j = 1, 2.

(1.24)

This creates a variety of possible variants for asymptotics of the first failure times ξε .

The Markov chain ηε (t) can be considered as a semi-Markov process with transition probabilities Q ε,i j (t) = pε,i j (1 − e−λε, i t ), t ≥ 0, i, j ∈ X. The distribution functions of transition times take the following form, Fε,i j (t) = −λ ε, i t , t ≥ 0, j  i, i, j ∈ X, where intensities λ Fε,i (t) = p−1 ε,i are ε,i j Q ε,i j (t) = 1 − e given by the relation (1.21). The corresponding Laplace transforms for the above λ i exponential distribution functions take the form, φε,i j (s) = φε,i (s) = λε,ε,i +s ,s ≥ 0, j  i, i, j ∈ X. −1 , for i ∈ X. Let us introduce the normalisation functions uε,i = λε,i Obviously, the following relation holds, for ε ∈ (0, 1] and j  i, i, j ∈ X,

1.2 Part II

13

φε,i j (s/uε,i ) = φε,i (s/uε,i ) = φ0 (s) =

1 , s ≥ 0. 1+s

(1.25)

The states 1 and 2 can be compared by their absorption rates. Suppose, for example, that the state 1 is less absorbing than the state 2. This means that the following limit (which exists according to the condition (4)) uε,1 = w0,1,2 ∈ [0, ∞). ε→0 uε,2 lim

(1.26)

We now exclude state 1 from the phase space X = {0, 1, 2} and construct a new reduced semi-Markov process 1 ηε (t) using the following time-space screening procedure. Let 0 = 1 ζε,0 < 1 ζε,1 < · · · be successive moments of hitting (resulted by jumps) in the reduced phase space 1 X = {0, 2} by the process ηε (t). Then, the reduced semi-Markov process 1 ηε (t) is defined as 1 ηε (t) = ηε (1 ζε,n ) for 1 ζε,n ≤ t < 1 ζε,n+1, n = 0, 1, . . .. Figure 1.4 illustrates the above phase space reduction procedure. It is important that the first hitting times in state 0 coincides for the semi-Markov processes ηε (t) and 1 ηε (t), i.e., ξε = 1 ξε = inf(t > 0 : 1 ηε (t) = 0).

(1.27)

The semi-Markov process 1 ηε (t) has the phase space 1 X = {0, 2} and the transition probabilities 1 Q ε,i j (t) = Q ε,i j (t) + Q ε,i1 (t) ∗ Q ε,1j (t), t ≥ 0, i, j ∈ 1 X. The corresponding embedded Markov chain 1 ηε,n has the transition probabilities, 1 pε,i j = pε,i j + pε,i1 pε,1j , i, j ∈ 1 X. Note that 1 pε,i j > 0, i, j ∈ 1 X, for ε ∈ (0, 1] and (1.28) 1 pε,i j → 1 p0,i j = p0,i j + p0,i1 p0,1j as ε → 0, for i, j ∈ 1 X. The distribution functions of transition times take the following form, 1 Fε,i j (t) = −1 Q p 1 ε,i j 1 ε,i j (t), t ≥ 0, i, j ∈ 1 X, and the Laplace transforms of the above distribution functions take the following form, for i, j ∈ 1 X, 1 φε,i j (s)

= 1 p−1 ε,i j (φε,i j (s)pε,i j + φε,i1 (s)φε,1j (s)pε,i1 pε,1j ), s ≥ 0.

(1.29)

The condition (4) implies that the probabilities pε,i j , i, j ∈ X and 1 pε,i j , i, j ∈ 1 X also belong to the family of asymptotically comparable functions H, and thus, there exist limits, for i, j ∈ 1 X, lim

pε,i j

ε→0 1 pε,i j

= 1 qˆ0,i j ∈ [0, 1].

(1.30)

The relations (1.25), (1.26), and (1.28)–(1.30) imply that the following relation holds, for j ∈ 1 X and s ≥ 0, 1 φε,2j (s/uε,2 )

→ 1 φ0,2j (s) = 1 qˆ0,2j φ0 (s) + φ0 (s)φ0 (sw0,1,2 )(1 − 1 qˆ0,2j ) as ε → 0.

(1.31)

14

1 Introduction

Fig. 1.4 Phase space reduction and removing of virtual transitions for semi-Markov process η(t)

The next step is to remove virtual transitions from trajectories of the semi-Markov process 1 ηε (t) and construct a new reduced semi-Markov process 1 η˜ε (t) using the following time-space screening procedure. Let 0 = 1 ζ˜ε,0 < 1 ζ˜ε,1 < · · · be sequential instants of change of state (resulted by jumps) by the process 1 ηε (t). Then the reduced semi-Markov process 1 η˜ε (t) is defined as 1 η˜ε (t) = 1 ηε (1 ζ˜ε,n ) for 1 ζ˜ε,n ≤ t < 1 ζ˜ε,n+1, n = 0, 1, . . .. Figure 1.4 also illustrates the above procedure of removing virtual transitions. It is important that the first hitting times in the state 0 coincide for the semi-Markov processes ηε (t), 1 ηε (t), and 1 η˜ε (t), i.e., ξε = 1 ξε = 1 ξ˜ε = inf(t > 0 : 1 η˜ε (t) = 0).

(1.32)

1.2 Part II

15

Moreover, the first hitting time in state 0 for the process 1 η˜ε (t) obviously coincides with the moment of its first jump, i.e., 1 ξ˜ε

= 1 ζ˜ε,1 .

(1.33)

The semi-Markov process 1 η˜ε (t) has the phase space 1 X = {0, 2} and the transition probabilities, 1 Q˜ ε,i j (t) = 0, t ≥ 0, for t ≥ 0, j = i, i ∈ 1 X, and 1 Q˜ ε,i j (t) = ∞ (∗n) n=0 1 Q ε,ii (t) ∗ 1 Q ε,i j (t), t ≥ 0, for j  i, i ∈ 1 X. The corresponding embedded Markov chain 1 η˜ε,n has the transition probabilities, 1 p˜ε,i j = I( j  i), i, j ∈ 1 X. The distribution functions of transition times take the following form, 1 F˜ε,i j (t) = ˜ Q 1 ε,i j (t), t ≥ 0, for j  i, i ∈ 1 X, and the Laplace transforms of the above distribution functions take the following form, for j  i, i ∈ 1 X, 1 φ˜ε,i j (s)

=

1 φε,i j (s) 1 pε,i j

1 − 1 pε,ii 1 φε,ii (s)

, s ≥ 0.

(1.34)

Let us introduce the Laplace transforms Ψε,i (s) = Ei e−sξε , s ≥ 0, for i = 1, 2. Two cases should be considered. (I) 1 p0,20 > 0. In this case, the normalisation function uε,2 should be used. The relations (1.28), (1.31), and (1.34) imply that the following relation holds, for s ≥ 0, Ψε,2 (s/uε,2 ) = 1 φ˜ε,20 (s/uε,2 )  (s) = → Ψ0,2

1 φ0,20 (s) 1 p0,20

1 − 1 p0,22 1 φ0,22 (s)

as ε → 0,

(1.35)

where the Laplace transforms 1 φ0,2i (s), i = 0, 2 are given by the relation (1.31). The above asymptotic relation implies that the following relation holds, for the case where ηε (0) = 2, d

 ξε /uε,2 −→ ξ0,2 as ε → 0,

 ξ0,2

(1.36)  (s). Ψ0,2

has the Laplace transform where the random variable (II) 1 p0,20 = 0. In this case, the normalisation function u˜ε,2 = uε,2 1 p−1 ε,20 should be used. The relations (1.25), (1.31), and (1.34) imply that the following relation holds, for s ≥ 0, Ψε,2 (s/u˜ε,2 ) = 1 φ˜ε,20 (1 pε,20 s/uε,2 ) 1 φε,20 (1 pε,20 s/uε,2 ) = ( pε,20 s/uε,2 ) 1− φ 1 + 1 pε,22 1 ε,221 p1 ε,20  → Ψ0,2 (s) = (1 + (1 + w0,1,2 )s)−1 as ε → 0.

(1.37)

Here, it is taken into account that, by the relations (1.23) and (1.28), the quantity = 0. The above asymptotic relation implies that the following relation holds, for the case where ηε (0) = 2, 1 qˆ0,22

16

1 Introduction d

 ξε /u˜ε,2 −→ ξ0,2 as ε → 0,  ξ0,2

(1.38)

 (s), i.e., Laplace transform Ψ0,2 with parameter (1 + w0,1,2 )−1 .

where is a random variable with the it is an exponentially distributed random variable, The following backward relation connects the Laplace transforms Ψε,i (s) for i = 1 and i = 2, (1.39) Ψε,1 (s) = φε,10 (s)pε,10 + Ψε,2 (s)φε,12 (s)pε,12, s ≥ 0. Three cases should be considered. (III) 1 p0,20 > 0 and p0,12 > 0. In this case, the relations (1.25), (1.26), (1.35), and (1.39) imply that the following relation relation holds, for s ≥ 0,  Ψε,1 (s/uε,2 ) → Ψ0,1 (s)

 (s)φ0 (sw0,1,2 )p0,12 as ε → 0, = φ0 (sw0,1,2 )p0,10 + Ψ0,2

(1.40)

 (s) is given by the relation (1.35). where the Laplace transform Ψ0,2 The relation (1.40) implies that the following relation holds, for the case where ηε (0) = 1, d

 ξε /uε,2 −→ ξ0,1 as ε → 0,  ξ0,1

(1.41)  (s) Ψ0,1

where is a random variable with the Laplace transform given by the relation (1.40). (IV) 1 p0,20 = 0 and p0,12 > 0. In this case, the relations (1.25), (1.26), (1.37), and (1.39) imply that the following relation holds, for s ≥ 0,   (s) = p0,10 + Ψ0,2 (s)p0,12 as ε → 0, Ψε,1 (s/u˜ε,2 ) → Ψ0,1

(1.42)

 (s) is given by relation (1.37). where the Laplace transform Ψ0,2 Relation (1.42) implies that the following relation holds, for the case where ηε (0) = 1, d

 ξε /u˜ε,2 −→ ξ0,1 as ε → 0,  ξ0,1

(1.43)  (s) Ψ0,1

where is a random variable with the Laplace transform given by the relation (1.42). (V) p0,12 = 0. In this case, the relations (1.35), (1.37), and (1.39) imply that the following relation relation holds, for s ≥ 0,  (s) = Ψε,1 (s/uε,1 ) → Ψ0,1

1 as ε → 0. 1+s

(1.44)

The relation (1.44) implies that the following relation holds, for the case where ηε (0) = 1, d

 ξε /uε,1 −→ ξ0,1 as ε → 0,

(1.45)

1.2 Part II

17

 is a random variable exponentially distributed with parameter 1. where ξ0,1 Having in mind applications to ergodic theorems for semi-Markov type process, we are also interested in conditions, under which expectations of first hitting times normalised by the same functions (as in relations of convergence in distribution) converge to the first moments of the corresponding limiting random variables. In the example considered above, such “simultaneous” convergence takes place in the cases (I)–(IV). While, in the case (V), the answer is determined by the limit, p p uε,2 u˜ ε,2   = limε→0 ε,12 ˆ 0,1,2 = limε→0 ε,12 wˆ 0,1,2 uε,1 , if 1 p0,20 > 0, or by the limit, w uε,1 , if 1 p0,20 = 0 (these limits exist due to the condition (4)). The above-mentioned simultaneous convergence takes place, if the corresponding limit is 0, or does not take place, if the corresponding limit is in the range (0, ∞]. The results presented in Part II generalise and essentially expand the above asymptotic results in several directions. First, we consider hitting times for perturbed finite semi-Markov processes. In this case, transition times have arbitrary distributions concentrated on a positive half-line. Second, we consider regularly and singularly perturbed semi-Markov processes with an arbitrary communicative structure of the phase space for the corresponding limiting process. Third, we present effective asymptotic recurrent algorithms (of phase space reduction for perturbed semi-Markov processes) for finding normalisation functions, Laplace transforms for limiting distributions, and limit for expectations of hitting times. 1.2.1.3 Hitting Times for Perturbed Semi-Markov Processes. Let (ηε,n, κε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, i.e., a homogenous Markov chain with a phase space Z = X × [0, ∞), where X = {1, . . . , m} is a finite set, and transition probabilities, defined for i, j ∈ X, s, t ≥ 0,

Q ε,i j (t) = Fε,i j (t)pε,i j = P{ηε,1 = j, κε,1 ≤ t/ηε,0 = i, κε,0 = s},

(1.46)

where: (a) pε,i j = Q ε,i j (∞), i, j ∈ X and (b) Fε,i j (·), i, j ∈ X are proper distribution functions, which are not concentrated in zero. The characteristic property of the Markov renewal process is that its transition probabilities depend only on the current state of the first component and do not depend on the current state of the second component. In this case, the first component ηε,n is itself a homogeneous Markov chain with transition probabilities pε,i j , i, j ∈ X. Also, the above Markov renewal process can be used for construction of a semiMarkov process defined as ηε (t) = ηε,n , for ζε,n ≤ t < ζε,n+1, n = 0, 1, . . ., where ζε,n = κε,1 + · · · + κε,n are moments of jumps for process ηε (t). In this case, Fε,i j (·) are conditional distribution functions of transition (inter-jump) times for the semiMarkov process ηε (t). It is worth to mention two important cases. The semi-Markov process ηε (t) is a continuous time Markov chain, if its transition probabilities have the following form, Q ε,i j (t) = (1 − e−λε, i t )pε,i j , t ≥ 0, i, j ∈ X, where λε,i > 0, i ∈ X.

(1.47)

18

1 Introduction

The semi-Markov process ηε (t) is a discrete time Markov chain embedded in continuous time, if its transition probabilities have the following form: Q ε,i j (t) = I(t ≥ 1)pε,i j , t ≥ 0, i, j ∈ X.

(1.48)

The main difference between the models of Markov chains and semi-Markov processes can be described informally as follows. The Markov property (the future, ηε (s), s ≥ t, depends on the past, ηε (s), s ≤ t, only through the present, ηε (t)) holds for Markov chains at any time moment t ≥ 0 and, moreover, at any moment of jump, ζε,n, n = 0, 1, . . ., while for general semi-Markov processes, the Markov property is guaranteed only at the moments of jumps ζε,n, n = 0, 1, . . .. The basic fact concerned semi-Markov processes can be found in the books listed in Sect. B.2.7. The object of studies in Part II is the random functional, which is the hitting time of semi-Markov process ηε (t) in a non-empty domain D ⊆ X, τε,D =

ν ε,D

κε,n,

(1.49)

n=1

where

νε,D = min(n ≥ 1 : ηε,n ∈ D).

(1.50)

Figure 1.5 illustrates the definition given in relation (1.49).

Fig. 1.5 Hitting time τε,D

We carry out a detailed asymptotic analysis of the distributions G ε,D,i j (t) =

Pi {τε,D ≤ t, ηε (τε,D ) = j} and the expectations Eε,D,i j = Ei τε,D I(ηε (τε,D ) = j), for

regularly and singularly perturbed semi-Markov processes. 1.2.1.4 Regularly and Singularly Perturbed Markov Chains and SemiMarkov Processes. Let ηε,n, n = 0, 1, . . ., be, for every ε ∈ [0, 1], a homogeneous Markov chain, with the phase space X = {1, . . . , m} and the matrix of transition probabilities Pε = pε,i j .

1.2 Part II

19

In Part II, the standard model of perturbed Markov chains is considered, where the transition probabilities pε,i j → p0,i j as ε → 0, for i, j ∈ X. It is assumed that the phase space X is one class of communicative states for the pre-limiting Markov chains ηε,n , while X can possess an arbitrary communicative structure for the limiting Markov chain η0,n (with the matrix of transition probabilities P0 = p0,i j ), i.e., it can consist of one class of communicative states (models with regular perturbations), or one class of communicative states plus a class of transient states (models with semi-regular perturbations), or several closed classes of communicative states plus possibly a class of transient states (models with singular perturbations). Figure 1.6 illustrates the most complex model with singular perturbations. In this figure, black and white circles indicate, respectively, positive and zero elements in the corresponding matrices of transition probabilities. Here, the phase space X = {1, . . . , 7} is, for every ε ∈ (0, 1], one class of communicative states, for the pre-limiting Markov chains ηε,n , while the phase space X is divided into two closed classes of communicative states X1 = {1, 2} and X2 = {3, 4, 5}, and the class of transient states X0 = {6, 7}, for the limiting Markov chain η0,n .

Fig. 1.6 Singularly perturbed Markov chains

Perturbed semi-Markov processes are classified according to the types of perturbations for their embedded Markov chains. 1.2.1.5 Asymptotic Algorithms of Phase Space Reduction for Perturbed SemiMarkov Processes. Our approach to limit theorems for hitting times is based on natural conditions imposed on original perturbed semi-Markov processes. First, it is the condition of convergence for the transition probabilities of embedded Markov chains: (E) pε,i j → p0,i j as ε → 0, for i, j ∈ X Second, it is the condition of weak convergence for the distribution functions of transition times: (F) Fε,i j (· uε,i ) ⇒ F0,i j (·) as ε → 0,5 for i, j ∈ X, where: (a) F0,i j (·) are distribution functions, which are not concentrated in zero; (b) uε,i ∈ (0, ∞), i ∈ X are some normalisation functions such that uε,i → u0,i ∈ (0, ∞] as ε → 0. Third, this is the condition of convergence for the first moments of transition times: 5

⇒ is the symbol of weak convergence for distribution functions.

20

(G) fε,i j /uε,i = i, j ∈ X.

1 Introduction

∫∞ 0

uFε,i j (du)/uε,i → f0,i j =

∫∞ 0

uF0,i j (du) ∈ (0, ∞) as ε → 0, for

Note that condition (G) is implied by condition (F) in the example considered in Sect. 1.2.1.2, since distribution functions Fε,i j (·) are exponential in this example. The asymptotic algorithm of phase space reduction presented in Part II is based on recurrent application of two procedures. First, all virtual transitions (of the form i → i) are excluded from trajectories of the semi-Markov process ηε (t), and this semi-Markov process is transformed into the semi-Markov process η˜ ε (t) without virtual transitions. The inter-jump times for the new semi-Markov process η˜ ε (t) are times between sequential moments of state change (transitions of the form i → j  i), for the semi-Markov process ηε (t), and states of the semi-Markov process η˜ ε (t) at instants of jumps coincide with the corresponding states for the semi-Markov process ¯ is chosen, and the semi-Markov process η˜ε (t) is ηε (t). Second, some state k ∈ D transformed into the semi-Markov process k ηε (t), with the new reduced phase space k X = X \ {k}. The initial state of semi-Markov process k ηε (t) coincides with the state of semi-Markov process η˜ε (t) at the time of its first hitting in the space k X, the inter-jump times for the semi-Markov process k ηε (t) are times between further sequential hitting times of the semi-Markov process η˜ ε (t) in the space k X, and states of the semi-Markov process k ηε (t) at moments of jumps coincide with the corresponding states for the semi-Markov process η˜ ε (t). We show that under some additional conditions described below, the basic conditions (E)–(G) are also satisfied for new semi-Markov processes η˜ε (t) and k ηε (t), and we give explicit formulas for calculating the transition characteristic for these semi-Markov processes, normalisation functions, and the corresponding limits of transition probabilities, distribution functions of transition times, and their first moments. This allows us to apply the above-described algorithm of phase space reduction to the reduced semi-Markov processes k ηε (t), etc. Using this technique, we construct the asymptotic recurrent algorithm of phase space reduction for perturbed semiMarkov processes. To be able to compute all the limits that appear in the above procedures, two additional conditions are also assumed. The first condition is based on the concept of a complete family of asymptotically comparable functions introduced in Sect. 1.2.1.2. Typical examples of such families, including the family of asymptotically comparable power-type functions (satisfying relations h(ε)/ah ε bh → a[h(·)] ∈ [0, ∞] as ε → 0, where constants ah > 0 and bh ∈ (−∞, ∞)), are given in Chap. 8. The first additional condition mentioned above: (H) There exists a complete family of asymptotically comparable functions H such that: (a) pε,i j either equal to zero for ε ∈ (0, 1] or belongs to family H, for i, j ∈ X, (b) uε,i belongs to family H, for i ∈ X. The main case when studying the asymptotics of the hitting times τε,D is, where ¯ the initial state ηε (0) ∈ D. The second additional condition imposes some natural restriction on the choice ¯ (here m¯ is the number of states in the domain D), ¯ which ∈D of states k1, k2, . . . k m−1 ¯

1.2 Part II

21

¯ At each step of the recurrent should be sequentially excluded from the domain D. algorithm described above, the reduced semi-Markov process k¯ n ηε (t) (with states from the sequence k¯ n = k1, . . . , k n excluded from the phase space X) must be constructed in such a way that the state k n is the least absorbing state in the reduced phase space k¯ n−1 X = X \ {k1, . . . , k n−1 } of the semi-Markov process k¯ n−1 ηε (t) (here, k¯0 ηε (t) = ηε (t) is the original semi-Markov process). The corresponding normalisation functions used in the analogue of condition (E) for the semi-Markov process k¯ n−1 ηε (t) are given by formulas, k¯ n−1 uε,i =

n−1 

(1 −

k¯r pε,ii )

−1

uε,i, i ∈

k¯ n−1 X,

(1.51)

r=0

where k¯r pε,i j , i, j ∈ k¯r X are the transition probabilities of the embedded Markov chain for the semi-Markov process k¯r ηε (t). The second additional condition mentioned above: (I): limε→0

k¯ n−1 u ε, k n k¯ n−1 u ε, i

= w0,kn,i ∈ [0, ∞), for i ∈k¯ n−1 X, n = 1, . . . , m¯ − 1.

The condition (H) implies that all normalisation functions k¯ n−1 uε,i belong to the family H and guarantees the existence of limits appearing in the condition (I), as = k1, . . . , k m−1 well as the existence of a sequence k m−1 ¯ ¯ such that the condition (I) is satisfied. Moreover, these limits can be computed using operational rules for asymptotically comparable functions given in Chap. 8. What is important is that, in the case where the initial state ηε (0) ∈ k¯ n X, the hitting times in the domain D coincide for the semi-Markov processes ηε (t) and k¯ n ηε (t), as well as for the semi-Markov process k¯ n η˜ε (t) constructed by removing virtual transitions from trajectories of the semi-Markov process k¯ n ηε (t). ¯ \ {k1, . . . , k m−1 However, the domain D ¯ } = {k m ¯ } contains only one state. The η ˜ (t) hits domain D immediately after the first jump. Thus, semi-Markov process k¯ m−1 ε ¯ η˜ε (t) coincides the hitting time in the domain D for the semi-Markov process k¯ m−1 ¯ with the first transition time for this process. The normalisation function and the weak limit for the distribution function of the normalised first transition time for the η˜ε (t) (calculated using the asymptotic recurrent phase space reduction process k¯ m−1 ¯ algorithm described above) coincide with the normalisation function and the weak limit for the distribution of the normalised hitting time τε,D for the original semiMarkov processes ηε (t), for the case where the initial state ηε (0) = k m¯ . We also build the backward recurrent algorithm (based on relations connecting the Laplace transforms Ei e−sτε,D for the states i = k m¯ , k m−1 ¯ , . . . , k 1 ), which allows us to sequentially find normalisation functions and weak limits for the distribution functions of normalised hitting times for the cases, where the initial state ηε (0) = ki , for i = m¯ − 1, . . . , 1. Figure 1.7 illustrates the transformation of trajectories for perturbed semi-Markov processes ηε (t) into the above-described above recurrent algorithm of phase space reduction.

22

1 Introduction

Fig. 1.7 Removing virtual transitions and phase space reduction for the semi-Markov process η(t)

1.2 Part II

23

In this example, the original semi-Markov process ηε (t) has the phase space ¯ X = {0, 1, 2}. The domain D = {0}. The initial state ηε (0) = 2 ∈ D. At the first sub-step, the virtual transitions are excluded from trajectories of the semi-Markov process ηε (t) = k¯0 ηε (t) and this process is transformed into the semiMarkov process k¯0 η˜ε (t). At the second sub-step, state k1 = 1 is excluded from the phase space X, and the semi-Markov process k¯0 η˜ε (t) is transformed into the semi-Markov process k¯1 ηε (t), with the phase space k¯1 X = {0, 2}. Finally, the virtual transitions are excluded from trajectories of the semi-Markov process k¯1 ηε (t), and this process is transformed into the semi-Markov process k¯1 η˜ε (t). The hitting time τε, {0} is invariant with the respect to the above procedures, that is, the hitting time in the domain D = {0} coincides for the original semi-Markov process ηε (t) and the resulting semi-Markov process k¯1 η˜ε (t). It is useful to comment the difference between examples illustrated by Figs. 1.4 and 1.7. In the former example, the initial semi-Markov process ηε (t) = k¯0 ηε (t) is, in fact, a continuous time Markov chain, which does not make virtual transitions. In this case, the semi-Markov process k¯0 ηε (t) coincides with the semi-Markov process k¯0 η˜ε (t). In the latter example, the initial semi-Markov process ηε (t) = k¯0 ηε (t) admits virtual transitions. Keeping in mind the applications of limit theorems for hitting times to ergodic theorems for regularly and singularly perturbed semi-Markov processes and alternating regenerating processes modulated by perturbed semi-Markov processes, we are also interested in conditions under which distribution functions of normalised hitting times weakly converge to some limiting distribution functions not concentrated in 0 and, also, their first moments normalised by the same functions converge to the first moments of the corresponding limiting distribution functions.

1.2.2 Part II: Contents by Chapters Part II includes Chaps. 8–12. Chapter 8. In this chapter, we introduce the notion of a complete family of asymptotically comparable functions and present operational rules and formulas for such families. We also consider typical examples of such families, namely, families of asymptotically comparable power-type, power-exponential-type, and powerlogarithmic-type functions. Chapter 9. In this chapter, we introduce models of regularly and singularly perturbed finite semi-Markov processes, formulate basic perturbation conditions, and describe procedures of removing virtual transitions and one-state reduction of phase space as well as the asymptotic recurrent algorithm of sequential phase space reduction for perturbed semi-Markov processes. Chapter 10. In this chapter, we introduce hitting times τε,D and some related random functionals, present forward and backward recurrent relations for distributions, Laplace transforms of hitting times, describe the variant of asymptotic recurrent al-

24

1 Introduction

gorithm of sequential phase space reduction, with respect to which the hitting times are invariant in distribution, and get weak limit theorems for hitting times provided by explicit recurrent formulas for computing normalisation functions and limiting Laplace transforms. Chapter 11. In this chapter, we present forward and backward recurrent relations for expectations of hitting times, get limit theorems for expectation of hitting times, as well as present conditions, which guarantee that pre-limiting expectations normalised by functions used in weak limit theorems for distributions of hitting times converge to the first moments of the corresponding limiting distributions. Chapter 12. In this chapter, we present some generalisations of limit theorems for hitting times for regularly and singularly perturbed semi-Markov processes to vector and real-valued hitting times and rewards, present the variant of the asymptotic recurrent algorithm of phase space reduction for birth–death-type semi-Markov processes, and consider some numerical examples.

1.3 Appendices and Conclusion 1.3.1 Appendix A: Limit Theorems for Randomly Stopped Stochastic Processes In this appendix, we present basic functional limit theorems for stochastic processes and some limit theorems for randomly stopped stochastic processes, which play an important role in obtaining necessary and sufficient conditions in limit theorems for first-rare-event times and rare-event time processes.

1.3.2 Appendix B: Methodological and Bibliographical Notes This appendix includes methodological and historical notes related to the results presented in the book, comments on new results presented in the book and related new problems for future research, as well as bibliographic notes to an extensive bibliography of works in the field of asymptotic problems for perturbed stochastic processes and systems of Markov type.

1.3.3 Conclusion The asymptotic analysis for perturbed processes of Markov type and related problems are subjects of intensive studies during several decades. However, the development of the theory, in particular, its parts connected with necessary and sufficient conditions of convergence for regularly and singularly perturbed models, and computational

1.3 Appendices and conclusion

25

algorithms for singularly perturbed models are still far from the completion. The book is concentrated in this area. Conditions in limit theorems for first-rare-event times and processes for regularly perturbed finite semi-Markov processes presented in the book have the final form, without gaps between its necessary and sufficient parts. New computational asymptotic algorithms of phase space reduction presented in the book have a universal character. They can be applied to perturbed semi-Markov processes with an arbitrary asymptotic communicative structure of phase spaces. The corresponding algorithms are computationally effective due to recurrent character of computational procedures and give effective conditions of weak convergence for distributions and convergence of expectations for hitting times. The results presented in the book will be interesting to specialists, who work in such areas of the theory of stochastic processes such as ergodic, limit, and large deviation theorems, analytical and computational methods for Markov chains, Markov, semi-Markov, regenerative, and other classes of Markov type stochastic processes and their queuing, network, bio-stochastic and other applications. I hope that the book will also attract attention of those researchers, who are interested in new analytical methods of analysis for perturbed stochastic processes and systems. It can also be useful for doctoral and senior students. This gives me hope that the book will find a sufficient number of readers interested in stochastic processes and their applications.

Part I

First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes

Chapter 2

Asymptotics of First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes

Chapter 2 plays the key role in Part I. In this chapter, we present necessary and sufficient conditions for convergence in distribution for first-rare-event times and convergence in Skorokhod J-topology for first-rare-event processes for regularly perturbed Markov chains and semi-Markov processes with a finite phase space. This chapter includes four sections. In Sect. 2.1, we define first-rare-event times and first-rare-event processes defined on perturbed semi-Markov processes. The notion of asymptotically uniformly ergodic Markov chains is introduced. The basic model assumptions and the main Theorem 2.1, which gives necessary and sufficient conditions of convergence in distribution for first-rare-event times and convergence in topology J for first-rare-event processes defined on asymptotically uniformly ergodic semi-Markov processes, are formulated. In Sect. 2.2, we formulate and prove Theorem 2.2, which gives necessary and sufficient conditions of convergence in distribution and convergence in J-topology for step-sum reward processes defined on asymptotically uniformly ergodic Markov chains. This theorem plays the key role in the proof of Theorem 2.1 but also has its own value. In Sect. 2.3, we formulate and prove Lemmas 2.7 and 2.8, which give necessary and sufficient conditions of convergence in distribution for first-rare-event times and convergence in topology J for first-rare-event reward processes defined on asymptotically uniformly ergodic Markov chains. In Sect. 2.4, we complete the proof of Theorem 2.1. The integral part of this proof is Lemma 2.9, which establishes the asymptotic independence of the first-rareevent times and the step-sum reward processes defined on asymptotically uniformly ergodic semi-Markov processes, Theorem 2.2, and Lemmas 2.7–2.8.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_2

29

30

2 First-rare-event times for regularly perturbed SMP

2.1 First-Rare-Event Times for Perturbed Semi-Markov Processes In this section, we introduce first-rare-event times and processes defined on perturbed semi-Markov processes, formulate the basic model assumptions, and give necessary and sufficient conditions of convergence in distribution for first-rare-event times and convergence in J-topology for rare-event processes defined on asymptotically regularly perturbed semi-Markov processes.

2.1.1 First-Rare-Event Times Let (ηε,n, κε,n, ρε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, i.e., a homogenous Markov chain with a phase space Z = X × [0, ∞) × {0, 1}, where X = {1, 2, . . . , m} is a finite set, an initial distribution qˆε = qε,i,s,ı = P{ηε,0 = i, κε,0 ≤ s, ρε,0 = ı}, (i, s, ı) ∈ Z , and transition probabilities defined by the following relation, for (i, s, ı), ( j, t, j) ∈ Z, P{ηε,1 = j, κε,1 ≤ t, ρε,1 = j/ηε,0 = i, κε,0 = s, ρε,0 = ı}

= P{ηε,1 = j, κε,1 ≤ t, ρε,1 = j/ηε,0 = i} = Q ε,i j (t, j).

(2.1)

The characteristic property of the Markov renewal process (ηε,n, κε,n, ρε,n ) is that its transition probabilities depend only on the current state i of the first component ηε,n but do not depend on the current states s and ı of the second and third components κε,n and ρε,n . As is known, the first component ηε,n of the above Markov renewal process is also a homogenous Markov chain, with the phase space X, the initial distribution q¯ε = qε,i = P{ηε,0 = i}, i ∈ X , and the transition probabilities defined by the following relation, for i, j ∈ X, pε,i j = Q ε,i j (+∞, 0) + Q ε,i j (+∞, 1).

(2.2)

Also, the random sequence (ηε,n, ρε,n ), n = 0, 1, . . . is a Markov renewal process with the phase space X × {0, 1}, the initial distribution qε = qε,iı = P{ηε,0 = i, ρε,0 = ı}, (i, ı) ∈ X × {0, 1} , and the transition probabilities defined by the following relation, for (i, ı), ( j, j) ∈ X × {0, 1}, pε,(i,ı),(j, j) = Q ε,i j (+∞, j).

(2.3)

The random variables κε,n, n = 1, 2, . . . can be interpreted as transition times and the random variables ζε,n = κε,1 + · · · + κε,n, n = 1, 2, . . . , ζε,0 = 0 as moments of jumps for a semi-Markov process ηε (t), t ≥ 0, which is defined by the following relation:

2.1 First-rare-event times

31

ηε (t) = ηε,n for ζε,n ≤ t < ζε,n+1, n = 0, 1, . . .

(2.4)

This semi-Markov process has the phase space X and the transition probabilities, which are defined by the following relation for i, j ∈ X, s, t ≥ 0, P{ηε,n+1 = j, κε,n+1 ≤ t/ηε,n = i, κε,n = s}

= P{ηε,n+1 = j, κε,n+1 ≤ t/ηε,n = i} = Q ε,i j (t) = Q ε,i j (t, 0) + Q ε,i j (t, 1).

(2.5)

As far as the random variables ρε,n, n = 1, 2, . . . are concerned, they can be interpreted as the so-called indicator variables and used to record events {ρε,n = 1}, which can be interpreted as “rare” events. Let us introduce random variables, for ε ∈ (0, ε0 ], ξε =

νε 

κε,n,

(2.6)

n=1

where νε = min(n ≥ 1 : ρε,n = 1).

(2.7)

The random variable νε counts the number of transitions for the embedded Markov chain ηε,n until the first “rare” event occurs. Accordingly, the random variable ξε can be interpreted as the first-rare-event time for the semi-Markov process ηε (t). We also consider the first-rare-event processes, ξε (t) =

[tν ε] 

κε,n, t ≥ 0.

(2.8)

n=1 d

We use the symbol −→ to denote convergence in distribution for random variables (equivalent to weak convergence of their distribution functions, indicated usually by the symbol ⇒) or stochastic processes (equivalent to weak convergence of their P

finitely dimensional distributions), the symbol −→ to denote convergence of random U

J

variables in probability, and the symbols −→ and −→ to denote convergence, respectively, in the uniform U-topology and Skorokhod J-topology for càdlàg stochastic processes defined on the time interval [0, ∞) and taking values in Rk . We refer to Appendix A, where some basic definitions and theorems related to conditions of convergence in U- and J-topologies for càdlàg stochastic processes are formulated, and references to some books, where readers can find the detailed presentation of functional limit theorems for càdlàg stochastic processes. The main objective of this chapter is to describe the class G of all possible random variables ξ0 , which can appear in the corresponding limit theorem given in the form d

of the asymptotic relation, ξε −→ ξ0 as ε → 0, and give the necessary and sufficient conditions for the fulfilment of the indicated asymptotic relation with a specific (with respect to the distribution function) limiting random variable ξ0 from the class G.

32

2 First-rare-event times for regularly perturbed SMP

We also describe the class G∞ of all possible càdlàg processes ξ0 (t), t ≥ 0, which can appear in the corresponding functional limit theorem given in the form of the J

asymptotic relation, ξε (t), t ≥ 0 −→ ξ0 (t), t ≥ 0 as ε → 0, and give necessary and sufficient conditions the fulfilment of the above asymptotic relation with a specific (by its finite-dimensional distributions) limiting stochastic process ξ0 (t), t ≥ 0 from the class G∞ . The problems formulated above are solved under three general model assumptions. We introduce the probabilities of occurrence of a rare event at one transient step of the semi-Markov process ηε (t), pε,i = Pi {ρε,1 = 1}, i ∈ X.

(2.9)

Hereinafter, Pi and Ei denote, respectively, the conditional probability and expectation under the condition ηε,0 = i. The first model assumption, imposed on the probabilities pε,i , refines the interpretation of the events {ρε,n = 1} as “rare” events: A1 : (a) There exist ε0 ∈ (0, 1] such that maxi ∈X pε,i > 0, for ε ∈ (0, ε0 ], (b) pε,i → 0 as ε → 0, for i ∈ X.

2.1.2 Asymptotically Uniformly Ergodic Markov Chains Let us introduce random variables, με,i (n) =

n 

I(ηε,k−1 = i), n = 0, 1, . . . , i ∈ X.

(2.10)

k=1

If the phase space X of the Markov chain ηε,n is one class of communicative states for this Markov chain, then this Markov chain is ergodic and its stationary distribution is given by the following ergodic relation, με,i (n) P −→ πε,i as n → ∞, for i ∈ X. n

(2.11)

The ergodic relation (2.11) holds for any initial distribution q¯ε , and the stationary distribution πε,i, i ∈ X does not depend on the initial distribution. Also, all stationary probabilities are positive, i.e., πε,i > 0, i ∈ X. As is known, the stationary probabilities πε,i, i ∈ X are the unique solution for the following system of linear equations,   πε,i = πε, j pε, ji, i ∈ X, πε,i = 1. (2.12) j ∈X

i ∈X

2.1 First-rare-event times

33

The second model assumption is a condition of asymptotically uniform ergodicity of the embedded Markov chains ηε,n : B1 : There exists a ring chain of states i0, i1, . . . , i N = i0 , which contains all states from the phase space X and such that limε→0 pε,ik−1 ik > 0, for k = 1, . . . , N. Let η˜ε,n be, for every ε ∈ (0, 1], a Markov chain with the phase space X and a matrix of transition probabilities  p˜ε,i j . We shall use the following condition: B2 : pε,i j − p˜ε,i j → 0 as ε → 0, for i, j ∈ X. If the transition probabilities p˜ε,i j ≡ p0,i j , i, j ∈ X do not depend on ε, then the condition B2 reduces to the following condition: B3 : pε,i j → p0,i j as ε → 0, for i, j ∈ X. Lemma 2.1 Let the condition B1 be satisfied for the Markov chains ηε,n . Then: (i) There exists ε0 ∈ (0, 1] such that the Markov chain ηε,n is ergodic, for every ε ∈ (0, ε0] and 0 < limε→0 πε,i ≤ limε→0 πε,i < 1, for i ∈ X. (ii) If, in addition, the condition B2 is satisfied, then there exists ε˜0 ∈ (0, ε0] such that the Markov chain η˜ε,n is ergodic, for every ε ∈ (0, ε˜0 ], and its stationary distribution π˜ ε,i, i ∈ X satisfies the asymptotic relation, πε,i − π˜ ε,i → 0 as ε → 0, for i ∈ X. (iii) If condition B3 is satisfied, then matrix p0,i j  is stochastic, condition B1 is equivalent to the assumption that the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic, and the following asymptotic relation holds, πε,i → π0,i as ε → 0, for i ∈ X, where π0,i, i ∈ X is the stationary distribution of the Markov chain η0,n . Proof First, we prove the statement (iii). The condition B3 obviously implies that the matrix p0,i j  is stochastic. The conditions B1 and B3 imply that limε→0 pε,ik−1 ik = p0,ik−1 ik > 0, k = 1, . . . , N, for the ring chain appearing in the condition B1 . Thus, the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic. Vice versa, the assumption that the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic implies that there exists a ring chain of states i0, . . . , i N = i0 , which contains all states from the phase space X and such that p0,ik−1 ik > 0, k = 1, . . . , N. In this case, this condition B3 implies that limε→0 pε,ik−1 ik = p0,ik−1 ik > 0, k = 1, . . . , N, and, thus, the condition B1 is satisfied. Let us assume that the convergence relation for stationary distributions penetrating the statement (iii) does not hold. In this case, there exist δ > 0 and a sequence 0 < εn → 0 as n → ∞ such that limn→∞ |πεn,i − π0,i | ≥ δ, for some i  ∈ X. Since the sequences πεn,i, n = 1, 2, . . . , i ∈ X are bounded, there exists a subsequence  as k → ∞, for i ∈ X. This relation, 0 < εnk → 0 as k → 0 such that πεn k ,i → π0,i  , i ∈ X satisfy the the condition B3 , and the relation (2.12) imply that the limits π0,i system of linear equations given in (2.12). This is impossible, since the inequality  − π  | ≥ δ should hold, while the stationary distribution π , i ∈ X is the |π0,i  0,i 0,i unique solution of the system (2.12).

34

2 First-rare-event times for regularly perturbed SMP

Let us now prove the statement (i). The condition B1 obviously implies that there exists ε0 ∈ (0, 1] such that pε,ik−1 ik > 0, k = 1, . . . , N for the ring chain penetrating condition B1 and ε ∈ (0, ε0]. Thus, the Markov chain ηε,n is ergodic, for every ε ∈ (0, ε0]. Assume that limε→0 πε,i = 0, for some i  ∈ X. In this case, there exists a sequence 0 < εn → 0 as n → ∞ such that πεn,i → 0 as n → ∞. Since the sequences pεn,i j , n = 1, 2, . . . , i, j ∈ X are bounded, there exists a subsequence 0 < εnk → 0 as k → 0 such that pεn k ,i j → p0,i j as k → ∞, for i, j ∈ X. By the statement (iii), the matrix p0,i j  is stochastic, the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic, and its stationary distribution π0,i, i ∈ X satisfies the asymptotic relation, πεn k ,i → π0,i as k → ∞, for i ∈ X. This is impossible since the equality π0,i = 0 should hold, while all stationary probabilities π0,i, i ∈ X are positive. Thus, limε→0 πε,i > 0, for i ∈ X. This implies also that limε→0 πε,i < 1,  for i ∈ X, since i ∈X πε,i = 1, for ε ∈ (0, ε˜0 ]. Finally, let us prove the statement (ii). The conditions B1 and B2 obviously imply that limε→0 p˜ε,ik−1 ik = limε→0 pε,ik−1 ik > 0, k = 1, . . . , N, for the ring chain penetrating condition B1 . Thus, the condition B1 is also satisfied for the Markov chains η˜ε,n , and there exist ε˜0 ∈ (0, ε0 ] such that Markov chain η˜ε,n is ergodic, for every ε ∈ (0, ε˜0 ]. Let us assume that the convergence relation for stationary distributions penetrating the statement (ii) does not hold. In this case, there exist δ > 0 and a sequence 0 < εn → 0 as n → ∞ such that limn→∞ |πεn,i − π˜ εn,i | ≥ δ, for some i  ∈ X. Since the sequences pεn,i j , n = 1, 2, . . . , i, j ∈ X are bounded, there exists a subsequence 0 < εnk → 0 as k → 0 such that pεn k ,i j → p0,i j as k → ∞, for i, j ∈ X. These relations and the condition B2 imply that, also, p˜εn k ,i j → p0,i j as k → ∞, for i, j ∈ X. By the statement (iii), the matrix p0,i j  is stochastic, the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic, and its stationary distribution π0,i, i ∈ X satisfies the asymptotic relations, πεn k ,i → π0,i as k → ∞, for i ∈ X and π˜ εn k ,i → π0,i as k → ∞, for i ∈ X. This is impossible,  since the relation limk→∞ |πεn k ,i − π˜ εn k ,i | ≥ δ should hold. Remark 2.1 The statement (iii) of Lemma 2.1 implies that, in the case, where the transition probabilities pε,i j = p0,i j , i, j ∈ X do not depend on parameter ε or pε,i j → p0,i j as ε → 0, for i, j ∈ X, the condition B1 reduces to the standard assumption that the Markov chain η0,n , with the matrix of transition probabilities p0,i j , is ergodic.

2.1.3 Necessary and Sufficient Conditions for Convergence in Distribution of First-Rare-Event Times It should be noted that the conditions A1 and B1 obviously imply that Pi {νε < ∞} = 1, for i ∈ X and ε ∈ (0, ε0 ], where ε0 = ε0 ∧ ε0, and, thus, the first-rare-event time ξε and the first-rare-event process ξε (t), t ≥ 0 are well defined, for ε ∈ (0, ε0 ]. The third model assumption is the following condition that guarantees that the last summand κε,νε in the random sum ξε is asymptotically negligible:

2.1 First-rare-event times

35

C1 : Pi {κε,1 > δ/ρε,1 = 1} → 0 as ε → 0, for δ > 0 and i ∈ X. Note that we define the probability Pi {κε,1 > δ/ρε,1 = 1} = 0 in the cases where Pi {ρε,1 = 1} = 0. In fact, both probabilities appear together in calculations only in the form of the product Pi {κε,1 > δ/ρε,1 = 1}Pi {ρε,1 = 1}. Let us introduce the step-sum stochastic process, for ε ∈ (0, ε0 ], κε (t) =

[tv ε] 

κε,n, t ≥ 0.

(2.13)

n=1

The random variables κε (t) can be interpreted as rewards accumulated on trajectories of the Markov chain ηε,n . Respectively, the random variable ξε can be interpreted as the reward accumulated on the trajectories of the Markov chain ηε,n before the first occurrence of a “rare” event. Asymptotics of the step-sum reward processes κε (t), t ≥ 0 have its own value. At the same, the corresponding result formulated below in Theorem 2.2 plays an important role in the proof of Theorem 2.1. Let us define the probability that is the result of averaging the probabilities of occurrence of a rare event (in one transition step of the semi-Markov process ηε (t)) pε,i over the stationary distribution of the embedded Markov chain ηε,n ,  pε = πε,i pε,i and uε = p−1 (2.14) ε . i ∈X

We introduce the distribution function of the sojourn time κε,1 for the semiMarkov process ηε (t), Fε,i (t) = Pi {κε,1 ≤ t}, t ≥ 0, i ∈ X.

(2.15)

Let us define the distribution function Fε (t), which is a result of averaging the distribution functions of sojourn times Fε,i (t) over the stationary distribution of the embedded Markov chain ηε,n ,  Fε (t) = πε,i Fε,i (t), t ≥ 0. (2.16) i ∈X

Let also θ ε,n, n = 1, 2, . . . be i.i.d. random variables with the distribution function Fε (t). We can now formulate a necessary and sufficient condition for convergence in distribution for first-rare-event times:  ε] d D1 : θ ε = [u n=1 θ ε,n −→ θ 0 as ε → 0, where θ 0 is a non-negative random variable with distribution function not concentrated at zero. As well known, (d1 ), the limiting random variable θ 0 penetrating the condition D1 should be infinitely divisible and, ∫ ∞ thus, its Laplace transform has the form, Ee−sθ0 = e−A(s) , where A(s) = πs + 0 (1 − e−sv )π(dv), s ≥ 0, π is a non-negative

36

2 First-rare-event times for regularly perturbed SMP

constant σ-algebra of the interval (0, ∞) such ∫ Borel ∫ andv π(dv) is a measure on the v π(dv) < ∞; (d2 ) π + (0,∞) 1+v π(dv) > 0 (this is equivalent to the that (0,∞) 1+v assumption that P{θ 0 = 0} < 1). Let us also define a homogeneous step-sum process with independent increments (the summands are i.i.d. random variables), θ ε (t) =

[tu ε] 

θ ε,n, t ≥ 0.

(2.17)

n=1

According to Remark A.3 and Theorem A.3, the condition D1 is necessary and sufficient for the following asymptotic relation to hold, θ ε (t) =

[tu ε] 

J

θ ε,n, t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0,

(2.18)

n=1

where θ 0 (t), t ≥ 0 is a non-negative Lévy process (a càdlàg homogeneous process with independent increments) with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0. Let us define the Laplace transforms, ∫ ∞ e−st Fε,i (dt), s ≥ 0, i ∈ X, (2.19) φε,i (s) = Ei e−sκε,1 = 0

and φε (s) = Ee−sθε,1 =

∫ 0

∞

e−st Fε (dt) =



πε,i φε,i (s), s ≥ 0.

(2.20)

∈X

Condition D1 can be reformulated (see, for example, Feller (1971)) in the following equivalent form in terms of Laplace transforms: D2 : uε (1 − φε (s)) → A(s) as ε → 0, for s > 0, where the limiting function A(s) > 0, for s > 0 and A(s) → 0 as s → 0. In this case, (d3 ) A(s) is the cumulant of a non-negative random variable with a distribution function not concentrated at zero. Moreover, (d4 ) A(s) must be the cumulant of the infinitely divisible distribution function of the form specified in the above conditions (d1 ) and (d2 ). The following condition, which is a variant of the so-called central criterion of convergence (see, for example, Loève (1977)), is equivalent to the condition D1 , with the Laplace transform of the limiting random variable θ 0 given in the above conditions (d1 ) and (d2 ): D3 : (a) uε (1 − Fε (u)) → π(u) as ε → 0 for all continuity points u > 0 of the limiting function π(u), which is non-negative, non-increasing, and right continuous function defined on the interval (0, ∞), having a limiting value π(+∞) = 0, and is connected with the measure π(dv) by the relation π((u , u ]) = π(u ) − π(u ),

2.1 First-rare-event times

37

∫ ∫ 0 < u  ≤ u  < ∞; (b) uε (0,u] vFε (dv) → π + (0,u] vπ(dv) as ε → 0 for some u > 0 that is a continuity point of the function π(u). It is useful to note that (d5 ) the asymptotic relation appearing in the condition D3 (b) is satisfied, under the condition D2 (a), for any u > 0 that is a continuity point of the function π(u). The main result of this chapter is the following theorem. Theorem 2.1 Let the conditions A1 , B1 , and C1 are satisfied. Then: (i) The condition D1 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) d

of the asymptotic relation, ξε = ξε (1) −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution function not concentrated at zero. (ii) The distribution function G(u) = P{ξ0∫ ≤ u}, u ≥ 0 of the limiting random ∞ 1 , s ≥ 0, where variable ξ0 has the Laplace transform φ(s) = 0 e−su G(du) = 1+A(s) A(s) is the cumulant of infinitely divisible distribution appearing in the condition D1 (D2 ). J

(iii) The stochastic processes ξε (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 ), t ≥ 0 as ε → 0, where (a) ν0 is a random variable, which has the exponential distribution with parameter 1, (b) θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0, and (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent. Remark 2.2 According to Theorem 2.1, the class G of all possible non-negative random variables ξ0 with distribution functions not concentrated in zero, and such d

that the asymptotic relation, ξε −→ ξ0 as ε → 0, holds, coincides with the class of limiting random variables described in the statement (ii). The condition D1 is the necessary and sufficient condition for holding of the asymptotic relation given in the statements (i)–(ii). Also, the class G∞ of all possible non-negative, nondecreasing, càdlàg, stochastically continuous processes ξ0 (t), t ≥ 0 with distribution functions of random variables ξ0 (t), t > 0 not concentrated at zero, and such that the J

asymptotic relation, ξε (t), t ≥ 0 −→ ξ0 (t), t ≥ 0 as ε → 0, holds, coincides with the class of limiting processes described in the statement (iii). The condition D1 is the necessary and sufficient condition for the fulfilment not only of the asymptotic relation appearing in the statements (i)–(ii) but also of the stronger asymptotic relation appearing in the statement (iii). Remark 2.3 The note “for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency” used in the formulation of Theorem 2.1 should be understood in the sense that the asymptotic relation penetrating the statement (i) must hold for at least one family of initial distributions q¯ε, ε ∈ (0, ε0 ], in the statement of necessity, and for any family of initial distributions q¯ε, ε ∈ (0, ε0 ], in the statement of sufficiency.

38

2 First-rare-event times for regularly perturbed SMP

Remark 2.4 It is possible to modify the condition D1 and to assume that the random variable θ 0 appearing in this condition ∫ ∞ has no atom at zero. In terms of the corresponding cumulant A(s) = πs + 0 (1 − e−sv )π(dv) this additional assumption is satisfied if and only if A(s) → ∞ as s → ∞ or, equivalently, π > 0 or π = 0 but π((0, ∞)) = ∞. In this case, the random variable ξ0 appearing in the statement (i) of Theorem 2.1 must also have a distribution function without an atom at zero. In what follows, the relation a(ε) ∼ b(ε) as ε → 0 means that a(ε)/b(ε) → 1 as ε → 0. Remark 2.5 Due to Lemma 2.1, the asymptotic relation appearing in the condition D2 can, under the conditions A1 , B1 and B2 , be rewritten in an equivalent form, where the stationary probabilities πε,i, i ∈ X are replaced by the stationary probabilities π˜ ε,i, i ∈ X. Indeed, in this case, πε,i − π˜ ε,i → 0 as ε → 0, for i ∈ X, and, thus,  πε,i uε (1 − φε,i (s)) uε (1 − φε (s)) = i ∈X

∼



π˜ ε,i uε (1 − φε,i (s))

i ∈X

→ A(s) as ε → 0, for s > 0.

(2.21)

Remark 2.6 It is worth noting that the condition C1 cannot be omitted. If this condition is not satisfied, the last summand κε,νε may be a dominating contribution in the first-rare-event time ξε . As shown in the example considered in Sect. 1.1.1.2, in this case it is possible that the conditions A1 , B1 are satisfied, but the random variables ξε either converge in distribution to ∞ as ε → 0 or do not converge at all in distribution. However, the condition C1 can be omitted in Theorem 2.1, if the sojourn times κε,n and the indicator random variables ρε,n are conditionally independent, that is: C2 : Pi {κε,1 ≤ t, ρε,1 = 1} = Pi {κε,1 ≤ t}Pi {ρε,1 = 1}, for t ≥ 0, i ∈ X. In this case, the condition C1 is equivalent to the assumption that the following asymptotic relation holds, for δ > 0, i ∈ X, Pi {κε,1 > δ} = 1 − Fε,i (δ) → 0 as ε → 0.

(2.22)

The conditions A1 , B1 , and D1 imply the fulfilment of the above asymptotic relation. Indeed,  the condition D3 (equivalent to the condition D1 ) implies that 1 − Fε (δ) = i ∈X πε,i (1 − Fε,i (δ)) → 0 as ε → 0, for δ > 0. Also, the condition B1 implies that limε→0 πε,i > 0, for i ∈ X. Thus, the relation (2.22) holds. At the same time, the proof of necessity statement of Theorem 2.2 is based on the conditions A1 , B1 but does not use the condition C1 . Once the condition D1 is proved, the above arguments again lead to the fulfilment of the relation (2.22).

2.2 Asymptotics of step-sum reward processes

39

2.2 Asymptotics of Step-Sum Reward Processes In this section, we give necessary and sufficient conditions of convergence in distribution and convergence in the J-topology for step-sum reward processes defined on asymptotically uniformly ergodic Markov chains.

2.2.1 Necessary and Sufficient Conditions for Convergence in Distribution for Step-Sum Reward Processes In what follows, we always assume that asymptotic relations for random variables and processes, defined on trajectories of the Markov renewal processes (ηε,n, κε,n, ρε,n ), hold for any initial distributions q¯ε , if such distributions are not specified. It is useful to note that the indicator variables ρε,n are not involved in the definition of the processes κε (t). This let us replace function uε = p−1 ε by an arbitrary function 0 < uε → ∞ as ε → 0, in the condition D1 , Theorem 2.2 and Lemmas 2.3– 2.6 formulated below and their proofs, which are integral parts of the proof of Theorem 2.1. Theorem 2.2 Let the condition B1 be satisfied. Then: (i) The condition D1 is the necessary and sufficient condition for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity d

and sufficiency) of the asymptotic relation, κε (1) −→ θ 0 as ε → 0, where θ 0 is a non-negative random variable with distribution function not concentrated at zero. (ii) The limiting random variable θ 0 has the infinitely divisible distribution with the Laplace transform Ee−sθ0 = e−A(s), s ≥ 0, where A(s) is the cumulant of infinitely divisible distribution appearing in the condition D1 . J

(iii) The stochastic processes κε (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0, where θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0. Remark 2.7 According to Theorem 2.2, the class of non-negative, non-decreasing, càdlàg, stochastically continuous processes θ 0 (t), t ≥ 0 with distributions of random variables θ 0 (t), t > 0 not concentrated in zero, and such that the asymptotic relation, J

κε (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0, holds, coincides with the class of non-negative Lévy processes. The condition D1 is the necessary and sufficient condition for the fulfilment of the asymptotic relation appearing in the statements (i)–(ii) as well as for the stronger asymptotic relation appearing in the statement (iii). The proof of Theorem 2.2 is based on three lemmas formulated below. According to Lemma 2.1, the condition B1 implies that there exists ε0 ∈ (0, 1] such that the phase space X is one class of communicative states for the Markov chain ηε,n , for every ε ∈ (0, ε0 ]. In what follows, we assume that ε ∈ (0, ε0 ].

40

2 First-rare-event times for regularly perturbed SMP

Let αε,i,0 = 0 and αε,i,n = min(k > αε,i,n−1 : ηε,k = i), n = 1, 2, . . . be the sequential moments of hitting in state i ∈ X for the Markov chain ηε,n . Also, let us recall the random variables με,i (n) introduced in the relation (2.10). Lemma 2.2 Let the condition B1 be satisfied. Then, for any initial distributions q¯ε , ∗ αε,i (t) =

and μ∗ε,i (t) =

αε,i,[πε, i tuε ]

U

, t ≥ 0 −→ t, t ≥ 0 as ε → 0,

(2.23)

με,i ([tuε ]) U , t ≥ 0 −→ t, t ≥ 0 as ε → 0. πε,i uε

(2.24)

uε

Proof Let αε, j = min(n > 0 : ηε,n = j) be the moment of first hitting into the state j ∈ X for the Markov chain ηε,n . The condition B1 implies that there exist N pε,ik−1 ik > p, for ε ∈ (0, ε0 ]. The following p ∈ (0, 1) and ε p ∈ (0, 1] such that k=1 inequalities obviously hold, for ε ∈ (0, ε p ] and k ≥ 1, i, j ∈ X, Pi {αε, j > k N } ≤ (1 − p)k .

(2.25)

These inequalities imply that there exists K p ∈ (0, ∞) such that, for ε ∈ (0, ε p ] and i, j ∈ X, 2 Ei αε, (2.26) j ≤ K p < ∞. −1 , i ∈ X, for ε ∈ (0, ε ]. Also, as well known, Ei αε,i = πε,i p Let βε,i,n = αε,i,n − αε,i,n−1, n = 1, 2, . . ., where αε,i,0 = 0. The random variables βε,i,n, n ≥ 1 are independent and identically distributed for n ≥ 2. The relations (2.25) and (2.26) imply that, for i ∈ X, P

αε,i,1 /uε −→ 0 as ε → 0,

(2.27)

and, for ε ∈ (0, ε p ], i ∈ X, and δ > 0, t ≥ 0, −1 2 Pi {u−1 ε |αε,i,[tuε ] − πε,i [tuε ]| > δ} ≤ tK p /δ uε .

(2.28)

The relations (2.27) and (2.28) imply that, for i ∈ X and t ≥ 0, P

−1 αε,i,[tuε ] /πε,i uε −→ t as ε → 0

and

P

αε,i,[πε, i tuε ] /uε −→ t as ε → 0.

(2.29)

(2.30)

The dual identities P{με,i (r) ≥ k} = P{αε,i,k ≤ r }, r, k = 0, 1, . . . allow converting the asymptotic relation (2.30) into the following equivalent relation, for i ∈ X and t ≥ 0, P (2.31) μ∗ε,i (t) = με,i ([tuε ])/πε,i uε −→ t as ε → 0. ∗ (t) and μ∗ (t), t ≥ 0 are non-decreasing and the correSince the processes αε,i ε,i sponding limiting function is continuous, the asymptotic relations (2.29) and (2.31)

2.2 Asymptotics of step-sum reward processes

41

are, according to Remark A.1, equivalent to the asymptotic relations (2.23) and (2.24) given in Lemma 2.2.  Let us introduce random variables, which are successive moments of hitting in the state i ∈ X by the Markov chain ηε,n ,

for n = 1, min(k ≥ 0, ηε,k = i) τε,i,n = (2.32) min(k > τε,i,n−1, ηε,k = i) for n ≥ 2. Let us also define random variables, κε,i,n = κε,τε, i, n +1, n = 1, 2, . . . , i ∈ X.

(2.33)

The following lemma describes useful properties of the above family of random variables. Lemma 2.3 Let the condition B1 be satisfied. Then, for every ε ∈ (0, ε0 ]: (i) The random variables κε,i,n, n = 1, 2, . . . , i ∈ X are mutually independent. (ii) P{κε,i,n ≤ t} = Fε,i (t), t ≥ 0, for n = 1, 2, . . . , i ∈ X. Proof Let us take an arbitrary sequence of different pairs (i1, k1 ), (i2, k2 ), . . . taking values in the space X×{1, 2 . . .}. Please note that the random variables τε,i1,k1 , τε,i2,k2 , . . ., by definition, take different values. The hitting times τε,i,k are Markov moments for the embedded Markov chain ηε,n as, therefore, also for the Markov renewal process (ηε,n, κε,n ). Also, any event {τε,i1,k1 < · · · < τε,ir ,kr < minr 3 is similar. The statement (ii) follows from the above remarks.  The following representation takes place for the process κε (t), κε (t) =

[tu ε] 

κε,n =

([tuε ])  με, i i ∈X

n=1

κε,i,n, t ≥ 0.

(2.39)

n=1

The relation (2.39) is obvious because it only represents two alternative ways of grouping summands in the same random sum. It is useful to note that the families of random variables με,i (n), n = 0, 1, . . . , i ∈ X and κε,i,n, n = 1, 2, . . . , i ∈ X are not independent. Let now introduce step-sum processes with independent increments, κ˜ε (t) =

ε, i u ε ]  [t π

i ∈X

κε,i,n, t ≥ 0.

(2.40)

n=1

Lemmas 2.2 and 2.3 let us presume that processes κ˜ε (t) can be good approximations for processes κε (t). Lemma 2.4 Let the condition B1 be satisfied. Then: (i) The condition D1 is satisfied if and only if the following relation holds, d

κ˜ε (1) −→ θ 0 as ε → 0, where θ 0 is a non-negative random variable with distribution not concentrated at zero.

2.2 Asymptotics of step-sum reward processes

43

(ii) The random variable θ 0 has an infinitely divisible distribution with the Laplace transform Ee−sθ0 = e−A(s), s ≥ 0 with the cumulant A(s) defined in the condition D1 . J

(iii) The stochastic processes κ˜ε (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0, where θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0. Proof (Of Lemma 2.4 and Theorem 2.2) First, let us prove that the condition D1 implies the fulfilment of the asymptotic relations given in Lemma 2.4 and Theorem 2.2. Let ηˆε,n, n = 1, 2, . . . be, for every ε ∈ (0, 1], a sequence of random variables such that: (a) it is independent of the Markov chain (ηε,n, κε,n ), n = 0, 1, . . . and (b) it is a sequence of i.i.d. random variables taking value i with probability πε,i , for i ∈ X. Note that, in this case, the sequence of random variables ηˆε,n, n = 1, 2, . . . is also independent of the families of random variables με,i (n), n = 0, 1, . . . , i ∈ X and κε,i,n, n = 1, 2, . . . , i ∈ X . Let us define random variables, μˆ ε,i (n) =

n 

I(ηˆε,n = i), n = 0, 1, . . . , i ∈ X,

(2.41)

k=1

and stochastic processes, κˆε (t) =

([tuε ])  μˆ ε, i i ∈X

κε,i,n, t ≥ 0.

(2.42)

n=1

Let us also consider the sequence of random variables θ ε,n = κε, ηˆ ε, n,n, n = 1, 2, . . .. This is the sequence of i.i.d. random variables, which follows from the above definition of the sequence of random variables ηˆε,n, n = 1, 2, . . . and Lemma 2.3 describing the family of random variables κε,i,n, n = 1, 2, . . . , i ∈ X. Also,  P{θ ε,1 ≤ t} = πε,i Fε,i (t) = Fε (t), t ≥ 0. (2.43) i ∈X

We also define for ε ∈ (0, 1] a homogeneous step-sum càdlàg processes with independent increments (the relation (2.43) allows us to use the same notation as for the process introduced in the relation (2.17)), θ ε (t) =

[tu ε] 

θ ε,n, t ≥ 0.

(2.44)

n=1

As well known, the condition D1 is equivalent to the following relation, d

θ ε (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0.

(2.45)

44

2 First-rare-event times for regularly perturbed SMP

From the definitions of the sequence of random variables ηˆε,n, n = 1, 2, . . . and the family of random variables κε,i,n, n = 1, 2, . . . , i ∈ X, Lemma 2.3, and the independence of the above sequence and family, it follows that the following relation holds, d (2.46) κˆε (t), t ≥ 0 = θ ε (t), t ≥ 0. The relation (2.46) implies that κˆε (t), t ≥ 0 is a homogeneous step-sum càdlàg process with independent increments and that the condition D1 is equivalent to the following relation: d

κˆε (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0.

(2.47)

The random variables I(ηˆε,n = i), n = 1, 2, . . . are, for every i ∈ X, i.i.d. random variables taking values 1 and 0 with probabilities, respectively, πε,i and 1 − πε,i . According to the statement (i) of Lemma 2.1, 0 < limε→0 πε,i ≤ limε→0 πε,i < 1, for every i ∈ X. Taking into account the above remarks, we get the following relations, E | μˆ ∗ε,i (t) − [tuε ]/uε | ≤ [tuε ]/(πε,i uε )2 → 0 as ε → 0 and [tuε ]/uε | − t → 0 as ε → 0. These P

relations obviously imply that μˆ ∗ε,i (t) −→ t as ε → 0, for t ≥ 0. This relation and Theorem A.7 imply that the following relation holds, for i ∈ X, μˆ ∗ε,i (t) =

μˆ ε,i ([tuε ]) d , t ≥ 0 −→ μ0,i (t) = t, t ≥ 0 as ε → 0. πε,i uε

(2.48)

Since the processes μˆ ∗ε,i (t), t ≥ 0 are monotonic, relation (2.48) implies, by Remark A.1, that the following relation holds, for i ∈ X, μˆ ∗ε,i (t) =

μˆ ε,i ([tuε ]) U , t ≥ 0 −→ μ0,i (t) = t, t ≥ 0 as ε → 0. πε,i uε

(2.49)

Let us choose some 0 < u < 1. By the definition, the processes κ˜ε (t), κˆε (t), and μˆ ∗ε,i (t), i ∈ X are non-negative and non-decreasing. Taking this into account, we obtain, for x > 0, P{ κ˜ε (u) > x} ≤ P{ κ˜ε (u) > x, μˆ ∗ε,i (1) > u, i ∈ X}

+

 i ∈X

P{ κ˜ε (u) > x, μˆ ∗ε,i (1) ≤ u}

≤ P{ κˆε (1) > x} +

 i ∈X

P{ μˆ ∗ε,i (1) ≤ u}.

(2.50)

The relations (2.47) and (2.49) imply the distributions of random variables κ˜ε (u) are tight, as ε → 0. Indeed,

2.2 Asymptotics of step-sum reward processes

45

lim lim P{ κ˜ε (u) > x} ≤ lim lim (P{ κˆε (1) > x} x→∞ ε→0  ∗ + P{ μˆ ε,i (1) ≤ u}) ≤ lim P{θ 0 (1) > x/2} = 0.

x→∞ ε→0

x→∞

i ∈X

(2.51)

Let us also introduce homogeneous step-sum càdlàg processes with independent increments, for i ∈ X, [t π ε, i u ε ] κε,i,n, t ≥ 0. (2.52) κ˜ε,i (t) = n=1

Lemma 2.3 implies that, for every ε ∈ (0, ε0 ], the processes κ˜ε,i (t), t ≥ 0 , i ∈ X are independent. Since κ˜ε,i (u) ≤ κ˜ε (u), for i ∈ X, the relation (2.51) implies that distributions of random variables κ˜ε,i (1) are also tight, as ε → 0, for every i ∈ X, that is, lim lim P{ κ˜ε,i (u) > x} ≤ lim lim P{ κ˜ε (u) > x} = 0.

x→∞ ε→0

x→∞ ε→0

(2.53)

This implies that the above distributions are relatively compact and, thus, any sequence 0 < εn → 0 as n → ∞ contains a subsequence 0 < εnk → 0 as k → ∞ such that, for i ∈ X, d (2.54) κ˜εn k ,i (u) −→ θ 0,i,u as k → ∞, where θ 0,i,u, i ∈ X are proper non-negative random variables with distributions possibly depending on the choice of subsequence εnk . Moreover, by the central criterion of convergence (see, for example, Loève (1977)), the random variables θ 0,i,u, i ∈ X have infinitely divisible distributions. Let Ee−sθ0, i, u = e−Ai, u (s), s ≥ 0, i ∈ X be their Laplace transforms. The relation (2.54) implies, by Remark A.3 and Theorem A.3, that J

κ˜εn k ,i (t), t ≥ 0 −→ θ 0,i (t), t ≥ 0 as k → ∞, for i ∈ X,

(2.55)

where θ 0,i (t), t ≥ 0, i ∈ X are non-negative Lévy processes with Laplace transforms

Ee−sθ0, i (t) = e−t Ai (s), s, t ≥ 0, i ∈ X, possibly depending on the choice of subse-

quence εnk . Moreover, in this case, the cumulants Ai,u (s) and Ai (s) are connected by the following relation, Ai,u (s) = uAi (s), s ≥ 0. Since the processes κ˜ε,i (t), t ≥ 0, , i ∈ X are independent, the weak convergence of the vector processes ( κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t ≥ 0 also takes place, ( κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t ≥ 0 d

−→ (θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞,

(2.56)

where θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes with Laplace transforms Ee−sθ0, i (t) = e−t Ai (s), s, t ≥ 0, i ∈ X, possibly depending on the choice of subsequence εnk .

46

2 First-rare-event times for regularly perturbed SMP

Also (see, for example, Whitt (1980) or Silvestrov (2004)) J-compactness of the vector processes ( κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)) follows from J-compactness of their components κ˜εn k ,i (t), i ∈ X, since the corresponding limiting processes θ 0,i (t), t ≥ 0 , i ∈ X are stochastically continuous and independent and, therefore, their trajectories do not have joint discontinuity points with probability 1. Therefore, the J-convergence of the vector processes ( κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t ≥ 0 also takes place, ( κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t ≥ 0 J

−→ (θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞,

(2.57)

where θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes with Laplace transforms Ee−sθ0, i (t) = e−t Ai (s), s, t ≥ 0, i ∈ X, possibly depending on the choice of subsequence εnk . The relation (2.57) and the above remarks obviously imply the fulfilment of the following relation: κ˜εn k (t) =

 i ∈X

J

κ˜εn k ,i (t), t ≥ 0 −→ θ 0 (t) =



θ 0,i (t), t ≥ 0 as k → ∞,

(2.58)

i ∈X

where θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes described in the relation (2.56). Since the limiting processes in (2.24) and (2.49) are nonrandom functions, the relations (2.24), (2.49), and (2.58) imply, by Theorem A.9, that (μ∗εn

k

,1 (t), . . . ,

μ∗εn

k

,m (t), κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t

≥0

d

−→ (μ0,1 (t), . . . , μ0,m (t), θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞

(2.59)

and ( μˆ ∗εn

k

,1 (t), . . . ,

μˆ ∗εn

k

,m (t), κ˜εn k ,1 (t), . . . , κ˜εn k ,m (t)), t

≥0

d

−→ (μ0,1 (t), . . . , μ0,m (t), θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞,

(2.60)

where μ0,i (t) = t, t ≥ 0, i ∈ X and θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes defined in the relation (2.56). We can now apply Theorem A.6, which gives conditions of J-convergence for vector compositions of càdlàg stochastic processes, and get the following asymptotic relations, ( κ˜εn k ,1 (μ∗εn

k

∗ ,1 (t)), . . . , κ˜εn k ,m (μεn k ,m (t))), t

≥0

J

−→ (θ 0,1 (μ0,1 (t)), . . . , θ 0,m (μ0,m (t))) = (θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞,

(2.61)

2.2 Asymptotics of step-sum reward processes

47

and ( κ˜εn k ,1 ( μˆ ∗εn

k

ˆ ∗εn ,m (t))), t ,1 (t)), . . . , κ˜εn k ,m ( μ k

≥0

J

−→ (θ 0,1 (μ0,1 (t)), . . . , θ 0,m (μ0,m (t))) = (θ 0,1 (t), . . . , θ 0,m (t)), t ≥ 0 as k → ∞,

(2.62)

where θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes defined in relation (2.56). The relations (2.61) and (2.62) obviously imply J-convergence for the sums of components of these processes, i.e., that the following relations hold,  κεn k (t) = κ˜εn k ,i (μ∗εn ,i (t)), t ≥ 0 k

i ∈X

J

−→ θ 0 (t) =



θ 0,i (t), t ≥ 0 as k → ∞,

(2.63)

i ∈X

and κˆεn k (t) = J

 i ∈X

κ˜εn k ,i ( μˆ ∗εn

−→ θ 0 (t) =



k

,i (t)), t

≥0

θ 0,i (t), t ≥ 0 as k → ∞,

(2.64)

i ∈X

where θ 0,i (t), t ≥ 0, i ∈ X are independent non-negative Lévy processes defined in the relation (2.56). The relation (2.47) implies that, (2.65) θ 0 (t), t ≥ 0 = θ 0 (t), t ≥ 0.  Thus, the limiting process θ 0 (t) = i ∈X θ 0,i (t), t ≥ 0 has the same finitedimensional distributions for all subsequences εnk described above. Moreover, the cumulant A(s) of the limiting Lévy process θ 0 (t) is connected with the cumulants  Ai (s), i ∈ X of the Lévy processes θ 0,i (t) by the relation, A(s) = i ∈X Ai (s), s ≥ 0. Therefore, the relations (2.58), (2.63), and (2.64) imply that, respectively, the following relations hold: d

κ˜ε (t) =



J

κ˜ε,i (t), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0,

(2.66)

i ∈X

and κε (t) =

 i ∈X

as well as,

J

κ˜ε,i (μ∗ε,i (t)), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0,

(2.67)

48

2 First-rare-event times for regularly perturbed SMP

κˆε (t) =

 i ∈X

J

κ˜ε,i ( μˆ ∗ε,i (t)), t ≥ 0 −→ θ 0 (t), t ≥ 0 as ε → 0.

(2.68)

It is useful to note that the relation (2.68) for the homogeneous step-sum processes κˆε (t) follows directly from the relation (2.47). The relation (2.68) was obtained in the way described above only to prove that the limiting process in the relations (2.58), (2.63), and (2.64) is the same and does not depend on the choice of subsequences εnk described above. This allows us to write the relations (2.66) and (2.67). Let us now prove that the asymptotic relation appearing in the statement (i) of Theorem 2.2 or in the statement (i) of Lemma 2.4 implies that the condition D1 is satisfied. In both cases, the first step is to prove that distributions of random variables κ˜ε (u) are tight, as ε → 0, for some u > 0. Let us choose some 0 < u < 1. By the definition, the processes κε (t), κ˜ε (t), and μ∗ε,i (t), i ∈ X are non-negative and non-decreasing. Taking this into account, we obtain, for any x > 0, P{ κ˜ε (u) > x} ≤ P{ κ˜ε (u) > x, μ∗ε,i (1) > u, i ∈ X}

+

 i ∈X

P{ κ˜ε (u) > x, μ∗ε,i (1) ≤ u}

≤ P{κε (1) > x} +

 i ∈X

P{μ∗ε,i (1) ≤ u}.

(2.69)

The asymptotic relation appearing in the statement (i) of Theorem 2.2, the relation (2.24), and the inequality (2.69) imply that lim lim P{ κ˜ε (u) > x} ≤ lim lim (P{κε (1) > x} x→∞ ε→0  + P{μ∗ε,i (1) ≤ u}) ≤ lim P{θ 0 > x/2} = 0. (2.70)

x→∞ ε→0

x→∞

i ∈X

Note that the necessity statement requires the fulfilment of the asymptotic relation appearing in the statement (i) of Theorem 2.2 only for at least one family of initial distributions q¯ε, ε ∈ (0, ε0 ]. The asymptotic relation appearing in the statement (i) of Lemma 2.4 implies that lim lim P{ κ˜ε (u) > x} ≤ lim lim P{ κ˜ε (1) > x}

x→∞ ε→0

x→∞ ε→0

≤ lim P{θ 0 > x/2} = 0. x→∞

(2.71)

The relation (2.70), as well as the relation (2.71), implies that the distributions of random variables κ˜ε (u) are tight and, thus, relatively compact as ε → 0. Now, we can repeat the part of the above proof related to the relations (2.52)– (2.64).

2.2 Asymptotics of step-sum reward processes

49

The relation (2.63) and the asymptotic relation appearing in the statement (i) of Theorem 2.2, as well as the relation (2.58) and the asymptotic relation appearing in the statement (i) of Lemma 2.4, imply that the random variables θ (1) and θ 0 , which appear in the above asymptotic relations, have the same distribution, θ (1) = θ 0 . d

(2.72)

Moreover, the cumulant A(s) of the limiting Lévy process θ 0 (t) coincides with the cumulant of the random variable θ 0 , which, therefore, has an infinitely divisible distribution. Moreover, the relation (2.64) implies that the cumulant A(s) is connected  with  the cumulants Ai (s), i ∈ X of Lévy processes θ 0,i (t) by the relation A(s) = i ∈X Ai (s), s ≥ 0.  Thus, the limiting process θ 0 (t), t ≥ 0 = i ∈X θ 0,i (t), t ≥ 0 has the same finitedimensional distributions for all subsequences εnk described above. This allows us again to write the relations (2.66)–(2.68). The relation (2.68) proves, in this case, that the condition D1 is satisfied. The relation (2.66) proves the statement (iii) of Lemma 2.4. The relation (2.67) proves the statement (iii) of Theorem 2.2. 

2.2.2 Examples of Step-Sum Reward Processes Let us consider a particular case of the model, with random variables κε,n = fε,ηε, n−1 , n = 1, 2, . . . , i ∈ X, where fε,i ≥ 0, i ∈ X are nonrandom Non-negative numbers. In this case, the stochastic process κε (t) takes the following form: κε (t) =

[tu ε] 

fε,ηε, n−1 , t ≥ 0.

(2.73)

n=1

Also, the Laplace transforms φε,i (s) and φε (s) take the following forms: φε,i (s) = Ei e−s fε, η ε,0 = e−s fε, i , s ≥ 0, for i ∈ X and φε (s) =



πε,i e−s fε, i , s ≥ 0.

i ∈X

The condition D2 takes, in this case, the form of the following relation:  uε (1 − φε (s)) = πε,i uε (1 − e−s fε (i) ) i ∈X

→ A(s) as ε → 0, for s > 0, where the limiting function A(s) > 0, for s > 0 and A(s) → 0 as s → 0.

(2.74)

50

2 First-rare-event times for regularly perturbed SMP

This condition obviously implies that 1 − φε,i (s) → 0 as ε → 0, for s > 0, i ∈ X that is equivalent to the relation, fε,i → 0 as ε → 0, for i ∈ X. In this case, 1 − φε,i (s) = s fε,i + o(s fε,i ) as ε → 0, for every s > 0, i ∈ X. These relations allow us to reformulate the condition D2 for the above model in terms of functions,  fε = uε πε,i fε,i . i ∈X

The condition D2 is equivalent to the following condition: D4 : fε → f0 ∈ (0, ∞) as ε → 0. Moreover, in this case the cumulant A(s) = f0 s, s ≥ 0. Theorem 2.2 takes the following form. Lemma 2.5 Let the condition B1 holds. Then: (i) The condition D4 is the necessary and sufficient condition for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity d

and sufficiency) of the asymptotic relation, κε (1) −→ θ 0 as ε → 0, where θ 0 is a non-negative random variable with distribution function not concentrated at zero. d

(ii) The limiting random variable θ 0 = f0 , i.e., it is a constant. J

(iii) The stochastic processes κε (t), t ≥ 0 −→ f0 t, t ≥ 0 as ε → 0. Let us assume that function fε satisfies the following natural assumption: D5 : There exists ε0 ∈ (0, ε0 ] such that fε > 0 for ε ∈ (0, ε0 ]. In this case, we can describe the asymptotic behaviour of the step-sum processes κε (t) under a condition weaker than D4 : D6 : fε → f0 ∈ [0, ∞] as ε → 0. The following lemma generalises and complements Lemma 2.5. Lemma 2.6 Let the conditions B1 and D5 hold. Then: J

(i) fε−1 κε (t), t ≥ 0 −→ g0 (t) = t, t ≥ 0 as ε → 0. (ii) The condition D6 is the necessary and sufficient condition for the fulfilment (for some or any initial distributions q¯ε , respectively, in statements of necessity d

and sufficiency) of the asymptotic relation, κε (1) −→ θ 0 as ε → 0, where θ 0 is a non-negative proper or improper random variable. d

(iii) The limiting random variable θ 0 = f0 , i.e., it is a constant, and, moreover: P

(iv) κε (t) −→ f0 t as ε → 0, for every t > 0. J

(v) If f0 ∈ [0, ∞), then κε (t), t ≥ 0 −→ f0 t, t ≥ 0 as ε → 0. J

(vi) If f0 = ∞, then min(T, κε (t)), t > 0 −→ hT (t) = T, t > 0 as ε → 0, for every T > 0.

2.2 Asymptotics of step-sum reward processes

51

Proof We can use the following representation,  κε (t) = μ∗ε,i (t)uε πε,i fε,i, t ≥ 0,

(2.75)

i ∈X

where processes μ∗ε,i (t), t ≥ 0, i ∈ X are defined in relation (2.24). For any sequence 0 < εn → 0 as n → ∞, there exists a subsequence 0 < εnk → 0 as k → ∞ such that uεn k πεn k ,i fεn k ,i fεn k

→ gi ∈ [0, 1] as k → ∞, for i ∈ X.

(2.76)

The constants gi, i ∈ X can depend on the choice of subsequence εnk but, obviously, satisfy the following relation:  gi = 1. (2.77) i ∈X

Since the limiting processes in relations (2.24) are nonrandom functions, the relations (2.24) and (2.76) obviously imply that d

κεn k (t), t ≥ 0 −→ fε−1 n k



tgi = t, t ≥ 0 as k → ∞.

(2.78)

i ∈X

Moreover, since the processes on the left hand side of the above relation are nondecreasing and the limiting function is continuous, the following relation (according to Remarks A.1 and A.2) holds: J

κεn k (t), t ≥ 0 −→ fε−1 n k



tgi = t, t ≥ 0 as k → ∞.

(2.79)

i ∈X

Since the limiting process is the same for all subsequences εnk described above, relation (2.79) implies that the following relation holds: J

fε−1 κε (t), t ≥ 0 −→ g0 (t) = t, t ≥ 0 as ε → 0.

(2.80) d

The relation (2.80) implies that the random variables fε−1 κε (t) −→ t as ε → 0, for every t ≥ 0. This implies that the random variables κε (1) = fε · ( fε−1 κε (1)) can converge in distribution if and only if fε → f0 ∈ [0, ∞] as ε → 0. Moreover, in this case, the limiting (possibly improper) random variable is the constant f0 , and P

κε (t) −→ f0 t as ε → 0, for every t ≥ 0. If f0 ∈ [0, ∞), then the asymptotic relation appearing in the statement (v) can be obtained by applying Theorem A.6 to processes κε (t) = gε ( fε−1 κε (t)), t ≥ 0, which are compositions of processes fε−1 κε (t), t ≥ 0 and functions gε (t) = fε t, t ≥ 0. If f0 = ∞, then the asymptotic relation appearing in the statement (vi) can be obtained by applying Theorem A.6 to processes min(T, κε (t)) = hε,T ( fε−1 κε (t)), t >

52

2 First-rare-event times for regularly perturbed SMP

0, which are composition processes fε−1 κε (t), t > 0 and functions hε,T (t) = min(T, fε t), t > 0. Let us now assume that the asymptotic relation appearing in the statement (ii) holds, but the condition D6 is not satisfied. The relation fε → f0 ∈ [0, ∞] as ε → 0 holds if and only if there exist at least two subsequences 0 < εn , εn → 0 as n → ∞ such that (a) fεn → f0 ∈ [0, ∞] as n → ∞, P

(b) fεn → f0 ∈ [0, ∞] as n → ∞, and (c) f0  f0. In this case, κεn (1) −→ f0 as P

n → ∞ and κεn (1) −→ f0 as n → ∞. Therefore, the random variables κε (1) do not converge in distribution. 

2.3 Asymptotics of First-Rare-Event Times for Perturbed Markov Chains In this section, we give necessary and sufficient conditions of convergence in distribution and convergence in J-topology for first-rare-event times and processes defined on asymptotically uniformly ergodic Markov chains.

2.3.1 Convergence in Distribution for First-Rare-Event Times for Perturbed Markov Chains The following lemma describes the asymptotics for the first-rare-event times νε for the Markov chains ηε,n . Note that in this section, we use the function uε = p−1 ε . Lemma 2.7 Let the conditions A1 and B1 be satisfied. Then, the random variables, d

νε∗ = pε νε −→ ν0 as ε → 0,

(2.81)

where ν0 is a random variable exponentially distributed with parameter 1. Proof Let us define probabilities, for ε ∈ (0, ε0 ], Pε,i j Pε,i j = , i, j ∈ X. P 1 − pε,i r ∈X ε,ir

Pε,i j = Pi {ηε,1 = j, ρε,1 = 0}, p˜ε,i j = 

Let also η˜ε,n, n = 0, 1, . . . be a homogeneous Markov chain with phase space X, initial distribution q¯ε = qε,i, i ∈ X , and the matrix of transition probabilities  p˜ε,i j . The following relation holds, for t ≥ 0,

2.3 Asymptotics of first-rare-event times for perturbed MC

P{νε∗ > t} =

=

 i ∈X





qε,i

[tu ε] 

53

Pε,ik−1 ik

i=i0,i1,...,i[t u ε ] ∈X k=1



qε,i

i ∈X

[tu ε] 

p˜ε,ik−1 ik (1 − pε,ik−1 )

i=i0,i1,...,i[t u ε ] ∈X k=1

= E exp{−

[tu ε] 

− ln(1 − pε, η˜ ε, k−1 )}.

(2.82)

k=1

The conditions A1 and B1 imply that the condition B2 holds for the transition probabilities of the Markov chains η˜ε,n , since the following relation holds, for i, j ∈ X, |pε,i j (1 − pε,i ) − Pε,i j | 1 − pε,i | Pi {ηε,1 = j, ρε,1 = 0} − pε,i j pε,i | = 1 − pε,i 2pε,i ≤ → 0 as ε → 0. 1 − pε,i

|pε,i j − p˜ε,i j | =

(2.83)

Thus, by Lemma 2.1, there exists ε˜0 ∈ (0, ε0 ] such that the Markov chain η˜ε,n is ergodic, for every ε0 ∈ (0, ε˜0 ], and its stationary probabilities π˜ ε,i, i ∈ X satisfy the following relation: π˜ ε,i − πε,i → 0 as ε → 0, for i ∈ X.

(2.84)

We can apply Lemma 2.5, which is a special case of Theorem 2.2, to non-negative step-sum processes, κε∗ (t) =

[tu ε] 

− ln(1 − pε, η˜ ε, n−1 ), t ≥ 0.

(2.85)

n=1

To do this, we check that the condition D4 is satisfied for the functions fε (i) = − ln(1 − pε,i ), i ∈ X. Indeed, using the condition A1 , B1 , Lemma 2.1, and the relation (2.84), we obtain   π˜ ε,i ln(1 − pε,i ) ∼ uε π˜ ε,i pε,i fε = −uε i ∈X

∼ uε



i ∈X

πε,i pε,i = uε pε = 1 as ε → 0.

(2.86)

i ∈X

This relation is a variant of the condition D4 . In this case, the corresponding limiting constant f0 = 1. By applying the sufficiency statement of Lemma 2.5 to the step-sum processes κε∗ (t), we obtain the following relation, for t ≥ 0,

54

2 First-rare-event times for regularly perturbed SMP d

κε∗ (t) −→ t as ε → 0.

(2.87)

The expression on the right hand side of the relation (2.82) is the Laplace transform of the non-negative random variable κε∗ (t) at point 1. Thus, the relation (2.87) implies, by the continuity theorem for Laplace transforms, that the following relation holds, for t ≥ 0, ∗ P{νε∗ > t} = Ee−κε (t) → e−t as ε → 0. (2.88) 

The proof is complete.

2.3.2 J-Convergence for First-Rare-Event Time Processes for Perturbed Markov Chains Let, as in Lemmas 2.5 and 2.6, fε,i, i ∈ X be nonrandom non-negative numbers and   πε,i fε,i = p−1 πε,i fε,i . (2.89) fε = uε ε i ∈X

i ∈X

Let us introduce stochastic processes, νε (t) =

[tν ε] 

fε,ηε, n−1 , t ≥ 0.

(2.90)

n=1

The following lemma generalises Lemma 2.7 and is used in what follows. Lemma 2.8 Let the conditions A1 , B1 , and D5 be satisfied. Then: J

(i) fε−1 νε (t), t ≥ 0 −→ tν0, t ≥ 0 as ε → 0, where ν0 is a random variable exponentially distributed with parameter 1. (ii) The condition I is the necessary and sufficient condition for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity d

and sufficiency) of the asymptotic relation, νε (1) −→ ν as ε → 0, where ν is a non-negative random variable with distribution not concentrated at zero. d

(iii) The limiting random variable ν = f0 ν0 . J

(iv) If f0 ∈ [0, ∞), then νε (t), t ≥ 0 −→ f0 ν0 t, t ≥ 0 as ε → 0. J

(v) If f0 = ∞, then min(T, νε (t)), t > 0 −→ hT (t) = T, t > 0 as ε → 0, for every P

T > 0 and, thus, νε (t) −→ ∞ as ε → 0, for t > 0. Proof The following representation takes place, νε (t) = κε (tνε∗ ), t ≥ 0, where κε (t) are the processes defined in the relation (2.73).

(2.91)

2.3 Asymptotics of first-rare-event times for perturbed MC

55

The relations appearing in the statement (i) of Lemma 2.6 and in Lemma 2.7 imply, by Theorem A.9, that the following relation holds, d

(tνε∗ , fε−1 κε (t)), t ≥ 0 −→ (tν0, t), t ≥ 0 as ε → 0.

(2.92)

The components of the processes on the left hand side of the relation (2.92) are non-decreasing processes, and the process on the right hand side of the relation (2.92) is continuous. This let us apply Theorem A.6 to the processes fε−1 νε (t) = fε−1 κε (tνε∗ ), t ≥ 0 and to obtain the asymptotic relation appearing in the statement (i) of Lemma 2.8. The asymptotic relation appearing in the statement (i) of Lemma 2.8 implies that the random variables d (2.93) fε−1 νε (1) −→ ν0 as ε → 0. This relation implies that the random variables νε (1) = fε · ( fε−1 νε (1)) can converge in distribution if and only if fε → f0 ∈ [0, ∞] as ε → 0. Moreover, in this d case, the limiting (possibly improper) random variable ν = f0 ν0 . If f0 ∈ [0, ∞), then relations appearing in the statement (iv) of Lemma 2.6 and in Lemma 2.7 imply, by Theorem A.9, that the following relation holds: d

(tνε∗ , κε (t)), t ≥ 0 −→ (tν0, f0 t), t ≥ 0 as ε → 0.

(2.94)

The components of the processes on the left hand side of the relation (2.94) are non-decreasing processes and the process on the right hand side of the relation (2.92) is continuous. This let us apply Theorem A.6 to processes νε (t) = κε (tνε∗ ), t ≥ 0 and to obtain the asymptotic relation appearing in the statement (iv) of Lemma 2.8. If f0 = ∞, then relations appearing in the statement (v) of Lemma 2.6 and in Lemma 2.7 imply, by Theorem A.9, that the following relation holds: d

(tνε∗ , min(T, κε (t))), t > 0 −→ (tν0, T), t > 0 as ε → 0, for T > 0.

(2.95)

The components of the processes on the left hand side of the relation (2.95) are non-decreasing processes and the process on the right hand side of the relation (2.92) is continuous. Also the limiting random variable tν0 > 0 with probability 1, for every t > 0. This let us apply Theorem A.6 to processes min(T, νε (t)) = min(T, κε (tνε∗ )), t > 0 and to obtain the asymptotic relation given in appearing in the statement (v) of Lemma 2.8. The statement (vi) of this lemma is a direct corollary of the statement (v). Let us now assume that the asymptotic relation appearing in the statement (ii) holds, but the condition D6 is not satisfied. The relation fε → f0 ∈ [0, ∞] as ε → 0 holds if and only if there exist at least two subsequences 0 < εn , εn → 0 as n → ∞ such that (a) fεn → f0 ∈ [0, ∞] as n → ∞, d

(b) fεn → f0 ∈ [0, ∞] as n → ∞ and (c) f0  f0. In this case, νεn (1) −→ f0 ν0 as

56

2 First-rare-event times for regularly perturbed SMP d

n → ∞ and νεn (1) −→ f0 ν0 as n → ∞ and, thus, the random variables νε (1) do not converge in distribution. 

2.4 Asymptotics of First-Rare-Event Times for Perturbed Semi-Markov Processes In this section, we give the necessary and sufficient conditions of convergence in distribution and convergence in the topology J for step-sum reward processes defined on semi-Markov processes with asymptotically uniformly ergodic embedded Markov chains. Also, we complete the proof of Theorem 2.1.

2.4.1 Convergence in Distribution for First-Rare-Event Times for Perturbed Semi-Markov Processes We are now ready to complete the proof of Theorem 2.1. First of all, we focus on the statements (i) and (ii) of Theorem 2.1. Let us introduce Laplace transforms, φε,i j (ı, s) = Ei I(ηε,1 = j, ρε,1 = ı)e−sκε,1 , s ≥ 0, for i, j ∈ X, ı = 0, 1, φε,i (ı, s) = Ei I(ρε,1 = ı)e−sκε,1 , s ≥ 0, for i ∈ X, ı = 0, 1, φε,i j (ı, s) = Ei {I(ηε,1 = j)e−sκε,1 /ρε,1 = ı}, s ≥ 0, for i, j ∈ X, ı = 0, 1, and φε,i (ı, s) = Ei {e−sκε,1 /ρε,1 = ı}, s ≥ 0, for i ∈ X, ı = 0, 1. Now, let us define probabilities, for s ≥ 0, φε,i j (0, s) φε,i j (0, s) = , i, j ∈ X. φ (0, s) φε,i (0, s) r ∈X ε,i j

pε,s,i j = 

Let (ηε,s,n, ρε,s,n ), n = 0, 1, . . . be, for every s ≥ 0, a Markov renewal process, with the phase space X × {0, 1}, an initial distribution q¯ε = qε,i = P{ηε,s,0 = i, ρε,s,0 = 0} = P{ηε,s,0 = i}, i ∈ X , and transition probabilities, P{ηε,s,n+1 = j, ρε,s,n+1 = j/ηε,s,n = i, ρε,s,n = ı}

= P{ηε,s,n+1 = j, ρε,s,n+1 = j/ηε,s,n = i} = pε,s,i j (pε,i j + (1 − pε,i )(1 − j)), i, j ∈ X, ı, j = 0, 1.

(2.96)

2.4 Asymptotics of first-rare-event times for perturbed SMP

57

Note that the first component of the Markov renewal process, ηε,s,n, n = 0, 1, . . ., is a homogeneous Markov chain with phase space X, initial distribution q¯ε = qε,i, i ∈ X , and the matrix of transition probabilities pε,s,i j . Let us also introduce random variables, νε,s = min(n ≥ 1 : ρε,s,n = 1).

(2.97)

Let us prove that the condition D1 or the conditions A1 , B1 and the asymptotic relation appearing in the statement (i) of Theorem 2.1 imply that the condition B1 holds for the transition probabilities of the Markov chain ηε,s,n , for every s ≥ 0. The condition D2 , which is equivalent to the condition D1 , obviously, implies that, for i ∈ X, (2.98) φε,i (s) → 1 as ε → 0, for s ≥ 0. Conditions A1 , B1 and the asymptotic relation penetrating proposition (i) of Theorem 2.1 also imply that relation (2.98) holds. Indeed, the following representation takes place, ξε =

i (νε )  με,

i ∈X

∗

κε,i,n =

ε )πε, i u ε ]  [με, i (ν

i ∈X

n=1

κε,i,n .

(2.99)

n=1

Suppose that relation (2.98) does not hold. This means that there exists i ∈ X such that for some δ, p > 0 and εδ, p ∈ (0, ε0 ], the probability P{κε,i,1 ≥ δ} ≥ p, for ε ∈ (0, εδ, p ]. This obviously implies that, for t > 0, κ˜ε,i (t) =

[t π ε, i u ε ]

P

κε,i,n −→ ∞ as ε → 0.

(2.100)

n=1

The relation (2.100) implies that d

min(T, κ˜ε,i (t)), t > 0 −→ hT (t) = T, t > 0 as ε → 0.

(2.101)

Since the processes κ˜ε,i (t), t > 0 are non-decreasing and the limiting function hT (t) = T, t > 0, is continuous, the relation (2.101) implies (according to Remark A.1) that J

min(T, κ˜ε,i (t)), t > 0 −→ hT (t) = T, t ≥ 0 as ε → 0.

(2.102)

From Lemma 2.8 (applied for the model with functions fε, j = I( j = i)(πε,i uε )−1 , j ∈ X), it follows that the following relation holds: d

μ∗ε,i (νε∗ ) −→ ν0 as ε → 0, where ν0 is a random variable exponentially distributed with parameter 1. The relations (2.102) and (2.103) imply, by Theorem A.9, that

(2.103)

58

2 First-rare-event times for regularly perturbed SMP d

(μ∗ε,i (νε∗ ), min(T, κ˜ε,i (t))), t > 0 −→ (ν0, hT (t)), t > 0 as ε → 0.

(2.104)

Now we can apply Theorem A.4 to the stochastic processes min(T, κ˜ε,i (t))), t > 0 stopped at the random moments μ∗ε,i (νε∗ ). This gives the following relation, for any T > 0, d min(T, κ˜ε,i (μ∗ε,i (νε∗ ))) −→ T as ε → 0. (2.105) This is only possible if P

κ˜ε,i (μ∗ε,i (νε∗ ))) −→ ∞ as ε → 0.

(2.106)

The relation (2.106) implies that κ˜ε,i (μ∗ε,i (νε∗ ))) =

με, i (νε )

P

κε,i,n ≤ ξε −→ ∞ as ε → 0.

(2.107)

n=1

This relation contradicts to the asymptotic relation appearing in the statement (i) of Theorem 2.1. The relation (2.98) and the condition A1 imply that, for s ≥ 0, i, j ∈ X, φε,i (s, 0) = Ei I(ρε,1 = 0)e−sκε,1 → 1 as ε → 0

(2.108)

and, thus, for s ≥ 0, i, j ∈ X, pε,i j φε,i (0, s) − φε,i j (0, s) | φε,i (0, s) |pε,i j φε,i (0, s) − pε,i j | + |pε,i j − φε,i j (0, s)| ≤ φε,i (0, s) pε,i j |φε,i (0, s) − 1| + Ei I(ηε,1 = j)|1 − I(ρε,1 = 0)e−κε,1 )| ≤ φε,i (0, s) 2(1 − φε,i (0, s)) → 0 as ε → 0. (2.109) ≤ φε,i (0, s)

|pε,i j − pε,s,i j | = |

Relation (2.109) and Lemma 2.1 imply that, for every s ≥ 0, there exists ε˜0,s ∈ (0, ε0 ] such that the Markov chain η˜ε,n,s is ergodic, for every ε ∈ (0, ε˜0,s ], and its stationary probabilities πε,s,i, i ∈ X satisfy the following relation, πε,s,i − πε,i → 0 as ε → 0, for i ∈ X.

(2.110)

Let us assume that the Markov chains ηε,n and ηε,n,s have the same initial distribution q¯ε . The following representation takes place for the Laplace transform of the random variables ξε , for s ≥ 0,

2.4 Asymptotics of first-rare-event times for perturbed SMP

Ee−sξε =



qε,i

i ∈X

=





n−1 

φε,ik−1 ik (0, s)φε,in−1 in (1, s)

n=1 i=i0,i1,...,in ∈X k=1

qε,i

i ∈X

=

∞ 

59



∞ 



n−1 

φε,ik−1 ik (0, s)

n=1 i=i0,i1,...,in−1 ∈X k=1

qε,i

i ∈X

∞ 



n−1 



φε,in−1 in (1, s)

in ∈X

pε,s,ik−1 ik

n=1 i=i0,i1,...,in−1 ∈X k=1

× (1 − pε,ik−1 )φε,ik−1 (0, s)pε,in−1

 φε,i ,i (1, s) n−1 n pε,in−1 i ∈X n

=

 i ∈X

qε,i

∞ 



n−1 

pε,s,ik−1 ik

n=1 i=i0,i1,...,in−1 ∈X k=1

× (1 − pε,ik−1 )φε,ik−1 (0, s)pε,in−1 φε,in−1 (1, s) ν ε, s = E exp{− − ln φε,ηε, s, k−1 (0, s) k=1

− ln φε,ηε, s,νε, s −1 (0, s) + ln φε,ηε, s,νε, s −1 (1, s)}.

(2.111)

The relation (2.108) and the condition A1 imply that the following relation holds: φε,i (0, s) =

φε,i (0, s) → 1 as ε → 0, for s ≥ 0, i ∈ X. 1 − pε,i

(2.112)

Also, the condition C1 is equivalent to the following relation: φε,i (1, s) = Ei {e−sκε,1 /ρε,1 = 1} → 1 as ε → 0, for s ≥ 0, i ∈ X.

(2.113)

The relations (2.112) and (2.113) imply that, for s ≥ 0, P

| ln φε,ηε, s,νε, s −1 (0, s)| + | ln φε,ηε, s,νε, s −1 (1, s)| −→ 0 as ε → 0.

(2.114)

Let us introduce random variables, for s > 0, ν˜ε,s =

ν ε, s

− ln φε,ηε, s, k−1 (0, s).

(2.115)

n=1

The representation (2.111) and the relation (2.114) imply that, for s > 0, Ee−sξε ∼ Ee−ν˜ ε, s as ε → 0.

(2.116)

The relations (2.110), (2.112) and the statement (i) of Lemma 2.1 imply that, for s > 0,

60

2 First-rare-event times for regularly perturbed SMP

Aε (s) = −uε



πε,s,i ln φε,i (0, s)

i ∈X

∼ uε



πε,s,i (1 − φε,i (0, s))

i ∈X

∼ uε



πε,i (1 − φε,i (0, s)) as ε → 0.

(2.117)

i ∈X

Suppose that in addition to the conditions A1 –C1 , the condition D1 is satisfied. The condition D1 is equivalent to the condition D2 . Thus, due to the relations (2.112) and (2.113), the condition A1 , and the statement (i) of Lemma 2.1, the condition D2 is equivalent to the following relation:  uε (1 − φε (s)) = uε πε,i (1 − φε,i (s)) i ∈X

= vε



πε,i (1 − (1 − pε,i )φε,i (0, s) − pε,i φε,i (1, s))

i ∈X

= uε



πε,i ((1 − pε,i )(1 − φε,i (0, s)) + pε,i (1 − φε,i (1, s))

i ∈X

∼ uε



πε,i (1 − pε,i )(1 − φε,i (0, s))

i ∈X

∼ uε



πε,i (1 − φε,i (0, s)) → A(s) as ε → 0, for s > 0,

(2.118)

i ∈X

where A(s) > 0, for s > 0, and A(s) → 0 as s → 0. The relations (2.117) and (2.118) imply that, in this case, for s > 0,  Aε (s) = −uε πε,s,i ln φε,i (0, s) → A(s) as ε → 0.

(2.119)

i ∈X

Now, we can apply the sufficiency part of the statement (iv) of Lemma 2.8 to the random variables ν˜ε,s , for every s > 0, This gives the following relation: d

ν˜ε,s −→ A(s)ν0 as ε → 0, for s > 0,

(2.120)

where ν0 is an exponentially distributed random variable with parameter 1. This relation implies, by the continuity theorem for Laplace transforms, the following relation, Ee−sξε ∼ Ee−ν˜ ε, s → Ee−A(s)ν0 =

1 as ε → 0, for s > 0. 1 + A(s)

(2.121)

2.4 Asymptotics of first-rare-event times for perturbed SMP

61

The relation (2.121) proves sufficiency parts of the statements (i) and (ii) of Theorem 2.1. Suppose now that the conditions A1 –C1 are satisfied plus the statement (i) of Theorem 2.1 takes place. The asymptotic relation appearing in the statement (i) of Theorem 2.1 can be expressed in terms of Laplace transforms. It takes the form of the following relation (which should be assumed to hold for some initial distributions q¯ε ): Ee−sξε → e−A0 (s) as ε → 0, for s > 0,

(2.122)

where A0 (s) > 0 for s > 0 and A0 (s) → 0 as s → 0. Let us assume that the conditions A1 –C1 are satisfied, but the condition D1 is not satisfied. This means, due to the relation (2.122), that either (a) Aε (s) → A(s) ∈ (0, ∞) as s → 0, for every s > 0, but A(s) → 0 as ε → 0, or (b) Aε (s∗ ) → A(s∗ ) ∈ (0, ∞) as ε → 0, for some s∗ > 0. The latter relation holds if and only if there exist at least two subsequences 0 < εn , εn → 0 as n → ∞ such that (b1 ) Aεn (s∗ ) → A(s∗ ) ∈ [0, ∞] as n → ∞, (b2 ) Aεn (s∗ ) → A(s∗ ) ∈ [0, ∞] as n → ∞ and (b3 ) A(s∗ ) < A(s∗ ). In the case (a), we can repeat the part of the above proof presented in the relations (2.117)–(2.126) and, taking into account the relation (2.122), obtain the relation, 1 Ee−sξε ∼ Ee−ν˜ ε, s → 1+A(s) = e−A0 (s) as ε → 0, for s > 0. This relation implies that A(s) → 0 as ε → 0, and thus, the case (a) is impossible. In the case (b), the first sub-case, A(s∗ ) = ∞, is impossible. Indeed, as was shown in the proof of Lemma 2.8 (applied to the random variables ν˜ε,s∗ ), in this ∗

case, ν˜εn ,s∗ −→ ∞ as n → ∞. Thus, Ee−s ξε n ∼ Ee−ν˜ ε n , s ∗ → 0 as n → ∞. This relation contradicts to the relation (2.122). The second sub-case, A(s∗ ) = 0, is also impossible. Indeed, as was shown in the P

P

proof of Lemma 2.8 (applied to the random variables ν˜ε,s∗ ), in this case, ν˜εn,s∗ −→ 0 ∗ as n → ∞, and, thus, Ee−s ξε n ∼ Ee−ν˜ ε n , s ∗ → 1 as n → ∞. This relation also contradicts to the relation (2.122). Finally, the third sub-case, 0 < A(s∗ ) < A(s∗ ) < ∞, is also impossible. Indeed, the sufficiency statement of Lemma 2.7 applied to the random variables ν˜ε,s∗ yields, d

d

in this case, two relations ν˜εn ,s∗ −→ A(s∗ )ν0 as n → ∞ and ν˜εn,s∗ −→ A(s∗ )ν0 as n → ∞, where ν0 is exponentially distributed random variable with parameter ∗ 1. These relations imply that Ee−s ξε n ∼ Ee−ν˜ ε n , s ∗ → 1+A1 (s∗ ) as n → ∞ and −s ∗ ξε n

∼ Ee−ν˜ ε n , s ∗ → 1+A1 (s∗ ) as n → ∞. These relations contradict to the relation (2.122), since 1+A1 (s∗ )  1+A1 (s∗ ) . Therefore, the condition D1 should hold. This completes the proof of the statements (i) and (ii) of Theorem 2.1.

Ee

62

2 First-rare-event times for regularly perturbed SMP

2.4.2 J-Convergence for First-Rare-Event Processes for Perturbed Semi-Markov Process The following lemma combines the asymptotic relations given in Theorem 2.1 and Lemma 2.7. This lemma gives the last intermediate result necessary to complete the proof of the statement (iii) in Theorem 2.1. Lemma 2.9 Let the conditions A1 , B1 , C1 , and D1 be satisfied. Then, the following asymptotic relation holds: d

(νε∗ , κε (t)), t ≥ 0 −→ (ν0, θ 0 (t)), t ≥ 0 as ε → 0,

(2.123)

where: (a) ν0 is a random variable that has the exponential distribution with parameter 1, (b) θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0 and cumulant A(s) defined in the condition D1 , and (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent. Proof The following representation takes place, for s, t ≥ 0, EI(νε∗ > t)e−sκε (t) =

=

 i ∈X

 i ∈X



qε,i

[tv ε] 

φε,ik−1 ik (0, s)

i=i0,i1,...,i[t v ε ] ∈X k=1



qε,i

[tv ε] 

pε,s,ik−1 ik

i=i0,i1,...,i[t v ε ] ∈X k=1

× (1 − pε,ik−1 )φε,ik−1 (0, s) = E exp{−

[tv ε] 

(− ln(1 − pε, η˜ ε, k−1 )

k=1

− ln φε, η˜ ε, k−1 (0, s))}.

(2.124)

Using the conditions A1 , B1 , Lemma 2.1, and the relation (2.110), we obtain the following analogue of the relation (2.86), for s ≥ 0,  fε,s = −uε π˜ ε,s,i ln(1 − pε,i ) i ∈X

∼ uε



π˜ ε,s,i pε,i

i ∈X

∼ uε



πε,i pε,i = uε pε = 1 as ε → 0.

(2.125)

i ∈X

The relations (2.119) and (2.125) imply that Lemma 2.5 can be applied to the processes, for every s > 0,

2.4 Asymptotics of first-rare-event times for perturbed SMP

κε,s (t) =

63

[tu ε] 

(− ln(1 − pε, η˜ ε, k−1 ) − ln φε, η˜ ε, k−1 (0, s)), t ≥ 0.

(2.126)

k=1

This gives the following relation, for every s > 0, d

κε,s (t), t ≥ 0 −→ t + A(s)t, t ≥ 0 as ε → 0.

(2.127)

Denote, for i, j ∈ X, n = 0, 1, . . . , s ≥ 0, Ψε,i j (n, s) = Ei I(νε > n, ηε,n = j)e−s and

Ψε,i (n, s) = Ei I(νε > n)e−s

n

k=1 κ ε, k

=

n



k=1 κ ε, k

,

Ψε,i j (n, s).

(2.128) (2.129)

j ∈X

The relation (2.127) implies, by the continuity theorem for Laplace transforms, the following relation, for t ≥ 0, EI(νε∗ > t)e−sκε (t) = Ψε,i ([tvε ], s)

= Ee−κε, s (t) → e−t−A(s)t = e−t e−A(s)t as ε → 0, for s > 0.

(2.130)

We also denote, for i, j ∈ X, n = 0, 1, . . . , s ≥ 0, ψε,i j (n, s) = Ei I(ηε,n = j)e−s and

ψε,i (n, s) = Ei e−s

n

k=1 κ ε, k

=

n



k=1 κ ε, k

,

ψε,i j (n, s).

(2.131) (2.132)

j ∈X

The relation (2.130) easily implies that, for s > 0 and 0 ≤ t  ≤ t  < ∞, Ψε,i ([t  vε ] − [t  vε ], s) ∼ Ψε,i ([(t  − t )vε ], s) 







→ e−(t −t ) e−A(s)(t −t ) as ε → 0. ∞.

(2.133)

Also, the statement (iii) of Theorem 2.2 implies that, for s > 0 and 0 ≤ t  ≤ t  < ψε,i ([t  vε ] − [t  vε ], s) ∼ ψε,i ([(t  − t )vε ], s) 



→ e−A(s)(t −t ) as ε → 0.

(2.134)

The relations (2.133) and (2.134) imply that, for s > 0 and 0 ≤ t  ≤ t  < ∞,

64

2 First-rare-event times for regularly perturbed SMP



Ψε,i j ([t  vε ] − [t  vε ], s)

j ∈X 







= Ψε,i ([t  vε ] − [t  vε ], s) → e−(t −t ) e−A(s)(t −t ) as ε → 0

(2.135)

and 

ψε,i j ([t  vε ] − [t  vε ], s)

j ∈X 



= ψε,i ([t  vε ] − [t  vε ], s) → e−A(s)(t −t ) as ε → 0.

(2.136)

The following representation for multivariate joint distributions of the random variable νε∗ and the increments of stochastic process κε (t) for 0 = t0 ≤ t1 < · · · tk = t ≤ tk+1 ≤ · · · ≤ tn < ∞, 1 ≤ k < n < ∞ and s1, . . . , sn ≥ 0 take place, n 

EI(νε∗ > tk ) exp{−

sr (κε (tr ) − κε (tr−1 )}

r=1



=

qε,i0

i0,...,in ∈X n 

×

Ψε,ir −1 ir ([tr vε ] − [tr−1 vε ], sr )

r=1

ψε,ir −1 ir ([tr vε ] − [tr−1 vε ], sr )

r=k+1

=

k 





qε,i0

i0 ∈X

···×

Ψε,i0 i1 ([t1 vε ] − [t0 vε ], s1 )

i1 ∈X



Ψε,ik−1 ik ([tk vε ] − [tk−1 vε ], sk )

ik ∈X

×



ψε,ik ik+1 ([tk+1 vε ] − [tk vε ], sk+1 )

ik+1 ∈X

···×



ψε,in−1 in ([tn vε ] − [tn−1 vε ], sn ).

(2.137)

in ∈X

Using the relations (2.135), (2.136), and the representation (2.137), we obtain recurrently, for 0 = t0 ≤ t1 < · · · tk = t ≤ tk+1 ≤ · · · ≤ tn < ∞, 1 ≤ k < n < ∞ and s1, . . . , sn > 0, EI(νε∗ > tk ) exp{−

n 

sr (κε (tr ) − κε (tr−1 ))}

r=1

∼ EI(νε∗ > tk ) exp{−

n−1 

sr (κε (tr ) − κε (tr−1 ))}e−A(sn )(tn −tn−1 )

r=1

· · · ∼ EI(νε∗ > tk ) exp{−

k  r=1

sr (κε (tr ) − κε (tr−1 ))}

2.4 Asymptotics of first-rare-event times for perturbed SMP n 

× exp{

65

−A(sr )(tr − tr−1 )}

r=k+1

∼ EI(νε∗ > tk−1 ) exp{−

k−1 

sr (κε (tr ) − κε (tr−1 ))}

r=1

× exp{−(tk − tk−1 )} exp{

n 

−A(sr )(tr − tr−1 )}

r=k

· · · ∼ exp{−

k n   (tr − tr−1 )} exp{ −A(sr )(tr − tr−1 )} r=1

= exp{−t} exp{

n 

r=1

−A(sr )(tr − tr−1 )} as ε → 0.

(2.138)

r=1

The relation (2.138) is equivalent to the asymptotic relation (2.123) given in Lemma 2.9.  Now, we can complete the proof of Theorem 2.1. The asymptotic relation given in Lemma 2.9 can, obviously, be rewritten in the following equivalent form: d

(tνε∗ , κε (t)), t ≥ 0 −→ (tν0, θ 0 (t)), t ≥ 0 as ε → 0,

(2.139)

where the random variable ν0 and the stochastic process θ 0 (t), t ≥ 0 are described in Lemma 2.9. The asymptotic relation appearing in statement (iii) of Theorem 2.2 and the relation (2.139) let us apply Theorem A.6 to the compositions of stochastic processes κε (t), t ≥ 0 and tνε∗ , t ≥ 0. This gives the following relation: J

κε (tνε∗ ), t ≥ 0 −→ θ 0 (tν0 ), t ≥ 0 as ε → 0. The proof of Theorem 2.1 is complete.

(2.140)

Chapter 3

Flows of Rare Events for Regularly Perturbed Semi-Markov Processes

In this chapter, we introduce counting processes generated by flows of rare events defined on asymptotically uniformly ergodic semi-Markov processes. We present necessary and sufficient conditions of convergence in distribution for these counting processes. This chapter includes three sections. In Sect. 3.1, we introduce process Nε (t) counting the number of rare events in interval [0, t], for t ≥ 0. In Theorem 3.1, we give necessary and sufficient conditions of convergence in distribution for such processes. In Sect. 3.2, we introduce Markov renewal processes generated by flow of rare events defined on asymptotically uniformly ergodic semi-Markov processes. In Theorem 3.2, we give necessary and sufficient conditions for weak convergence of transition probabilities for such processes. In Sect. 3.3, we introduce vectors counting processes, in which components count the numbers of rare events in interval [0, t] occurring at moments of hitting different states by the corresponding semi-Markov processes. In Theorem 3.3, we give necessary and sufficient conditions of convergence in distribution for such processes.

3.1 Counting Processes Generated by Flows of Rare Events In this section, we introduce counting processes generated by flows of rare events defined on asymptotically uniformly perturbed semi-Markov processes and give necessary and sufficient conditions of convergence in distribution and J-convergence for these processes.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_3

67

68

3 Flows of rare events for regularly perturbed SMP

3.1.1 Counting Processes for Rare Events Let us define recurrently random variables, νε (k) = min(n ≥ νε (k − 1) : ρε,n = 1), k = 1, 2, . . . , νε (0) = 0.

(3.1)

The random variable νε (k) counts the number of transitions of the embedded Markov chain ηε,n up to the k-th appearance of the “rare” event {ρε, · = 1}. Let us also define inter-rare-event times, for k = 1, 2, . . ., ν ε (k)

κε (k) =

κε,n .

(3.2)

n=νε (k−1)+1

We also introduce random variables showing the states of the embedded Markov chain ηε,n at moments νε (k), k = 0, 1, . . ., ηε (k) = ηε,νε (k) .

(3.3)

Obviously, the random sequence (ηε (k), κε (k)), k = 0, 1, . . . (here κε (k) = 0) is a Markov renewal process, that is, a homogeneous Markov chain with phase space X × [0, ∞) and transition probabilities, for i, j ∈ X, s, t ≥ 0, P{ηε (k + 1) = j, κε (k + 1) ≤ t/ηε (k) = i, κε (k) = s}

= P{ηε (k + 1) = j, κε (k + 1) ≤ t/ηε (k) = i} = Pi {ηε,νε = j, ξε ≤ t} = Qi(ε) j (t).

(3.4)

Let us now define random variables, ξε (k) =

ν ε (k) n=1

κε,n =

k 

κε (n), k = 0, 1, . . . , ξε (0) = 0.

(3.5)

n=1

The random variable ξε (k) can be interpreted as the time of occurrence of the k-th rare-event time for the semi-Markov process η(t). Obviously, ξε (1) = ξε , where ξε is the first-rare-event time for the semi-Markov process ηε (t). Now, we can define a counting stochastic process generated by a flow of rare events, (3.6) Nε (t) = max(k ≥ 0 : ξε (k) ≤ t), t ≥ 0. We denote by N the class of integer-valued, non-negative, non-decreasing, stepwise with a finite number of jumps in every finite interval, continuous from the right processes defined on the interval [0, ∞). Obviously, any process from the class N is a càdlàg process. The counting process Nε (t) belongs to the class N.

3.1 Counting processes generated by flows of rare events

69

3.1.2 Necessary and Sufficient Conditions of Convergence for Counting Processes Generated by Flows of Rare Events Let κ(k), k = 1, 2, . . ., be a sequence of i.i.d. non-negative random variables with distribution function G(u) = P{κ(1) ≤ t}, t ≥ 0 not concentrated at zero. We also define a sequence of random variables, commonly referred to as the renewal process, k  κ(n), k = 0, 1, . . . , ξ(0) = 0. ξ(k) = n=1

Let us also define the corresponding standard renewal counting process, N(t) = max(k ≥ 0 : ξ(k) ≤ t), t ≥ 0.

(3.7)

Any standard renewal counting process N(t), t ≥ 0 belongs to the class N. We denote by N the set of points of stochastic continuity t > 0 for the process N(t). The set N is at most countable. Moreover, N ⊆ K = ∪n ≥1 Kn , where Kn is the set of discontinuity points u > 0 for the distribution function G(∗n) (u) of the random variable ξ(n), for n = 1, 2, . . .. The following theorem takes place. Theorem 3.1 Let the conditions A1 , B1 , and C1 be satisfied. Then: (i) The condition D1 is necessary and sufficient for the fulfilment (for some or any initial distribution q¯ε , respectively, in the statements of necessity and sufficiency) of d

the asymptotic relation Nε (t), t ∈ N −→ N(t), t ∈ N as ε → 0, where N(t), t ≥ 0 is some process from class N such that P{N(t) ≥ 1} = G(t), t ≥ 0 is the distribution function on [0, ∞) not concentrated at zero, and N is the set of stochastic continuity points t > 0 for this process. In this case: (ii) The limiting process N(t), t ≥ 0 is a standard renewal counting process defined by the relation (3.7) through some sequence of non-negative i.i.d. random variables κ(k), k = 1, 2, . . ., with the distribution function∫ G(u) = P{κ(1) ≤ t}, t ≥ 0 not ∞ 1 ,s ≥ concentrated at zero and the Laplace transform 0 e−st G(dt) = φ(s) = 1+A(s) 0, where A(s) is the cumulant of infinitely divisible distribution appearing in the condition D1 . Proof Obviously, the random variable κε (1) has the following conditional distribution function, for i ∈ X,  Gi(ε) (u) = Pi {κε (1) ≤ u} = Pi {ξε ≤ u} = Qi(ε) (3.8) j (u), u ≥ 0. j ∈X

According to Theorem 2.1, the conditions A1 –D1 imply that, for i ∈ X, Gi(ε) (·) = P{κε (1) ≤ ·} ⇒ G(·) as ε → 0,

(3.9)

70

3 Flows of rare events for regularly perturbed SMP

where G(u) is the distribution function on [0, ∞) not concentrated at zero, with the ∫∞ 1 appearing in the condition D1 . Laplace transform 0 e−st G(dt) = φ(s) = 1+A(s) Using the Markov property of the Markov renewal process (ηε (k), κε (k)), we obtain the following formula for the joint conditional distribution functions of random variables κε (k), k = 1, . . . , n, for u1, . . . , un ≥ 0, and i0 ∈ X, n = 1, 2, . . ., Pi0 {κε (k) ≤ uk , k = 1, . . . , n}

=



Pi0 {κε (k) ≤ uk , k = 1, . . . , n − 1,

in−1 ∈X

ηε (n − 1) = in−1 }G(ε) in−1 (un ).

(3.10)

The conditions A1 –D1 and the relations (3.10) and (3.9) imply that the following convergence relation holds for the joint conditional distribution functions of random variables κε (k), k = 1, 2, for all points of continuity for the corresponding limiting two-dimensional distribution function, and i0 ∈ X, | Pi0 {κε (k) ≤ uk , k = 1, 2} − G(u1 )G(u2 )| ≤ | Pi0 {κε (k) ≤ uk , k = 1, 2} − Pi0 {κε (1) ≤ u1 }G(u2 )| + | Pi0 {κε (1) ≤ u1 }G(u2 )| − G(u1 )G(u2 )|  (ε) Pi0 {κε (k) ≤ u1, ηε (1) = i1 }|Gi (u2 ) − G(u2 )| ≤ 1 i1 ∈X

+ | Pi0 {κε (1) ≤ u1 } − G(u1 )|G(u2 ) → 0 as ε → 0.

(3.11)

Similarly, the conditions A1 –D1 and the relations (3.9) and (3.10) imply that the following convergence relation holds for the joint conditional distribution functions of random variables κε (k), k = 1, . . . , n, for all points of continuity for the corresponding limiting multi-dimensional distribution functions, and i0 ∈ X, n = 1, 2, . . ., | Pi0 {κε (k) ≤ uk , k = 1, . . . , n} −

n 

G(uk )| → 0 as ε → 0.

(3.12)

k=1

The relation (3.11) means that the inter-renewal times κε (k), k = 1, 2, . . ., are asymptotically independent. Let κ(k), n k = 1, 2, . . ., be i.i.d. random variables with distribution function G(u), κ(k), n = 1, 2, . . ., and H(u1, . . . , un ) be the joint distribution function ξ(n) = k=1 of random variables ξ(k), k = 1, . . . , n, for n = 1, 2, . . .. The conditions A1 –D1 and the relation (3.11) imply that the following convergence relation holds k for the joint conditional distribution functions of random variables κε (r), k = 1, . . . , n, for all points of continuity for the corresponding ξε (k) = r=1 limiting multi-dimensional distribution functions, and i ∈ X, n = 1, 2, . . . Pi {ξε (k) ≤ uk , k = 1, . . . , n} → P{ξ(k) ≤ uk , k = 1, . . . , n}

= H(u1, . . . , un ) as ε → 0.

(3.13)

3.1 Counting processes generated by flows of rare events

71

The conditions A1 –D1 and the relation (3.13) imply that the following convergence relation holds for (a) integers 0 ≤ r1 ≤ r2 ≤ · · · < ∞, (b) 0 < t1 ≤ t2 ≤ · · · < ∞, which are points of continuity for the corresponding limiting distribution functions of random variables ξ(k), k = 1, 2, . . ., and (c) i ∈ X, n = 1, 2, . . ., Pi {Nε (tk ) ≥ rk , k = 1, . . . , n} = Pi {ξε (rk ) ≤ tk , k = 1, . . . , n} → P{ξ(rk ) ≤ tk , k = 1, . . . , n}

= P{N(tk ) ≥ rk , k = 1, . . . , n} as ε → 0.

(3.14)

The relation (3.14) implies that the following asymptotic relation holds, for any initial distributions q¯ε , d

Nε (t), t ∈ K −→ N(t), t ∈ K as ε → 0.

(3.15)

Note that the process N(t), t ≥ 0 is non-decreasing. Also, the set K is at most countable and, thus, the set K ⊆ N is dense in the interval [0, ∞). Therefore, the relation (3.15) can be extended in the standard way to the asymptotic relation appearing in the statement (i), where the limiting process N(t) is the standard counting process described in the statement (ii) of Theorem 3.1. Suppose now that the conditions A1 –C1 are satisfied and the asymptotic relation, d

Nε (t), t ∈ N −→ N(t), t ∈ N as ε → 0, takes place, for some initial distributions q¯ε , where N(t) is some stochastic process described in proposition (i) of Theorem 3.1, and N is the set of points of stochastic continuity t > 0 for this process. It follows from the above assumptions that the following relation holds, for t ∈ N, Pq¯ ε {ξε ≤ t} = Pq¯ ε {Nε (t) ≥ 1}

→ P{N(t) ≥ 1} = G(t) as ε → 0.

(3.16)

Since G(t) is the distribution function on [0, ∞) not concentrated at zero and the set N is dense in [0, ∞), the relation (3.16) implies that Pq¯ ε {ξε ≤ ·} ⇒ G(·) as ε → 0.

(3.17)

The relation (3.17) implies, by Theorem 2.1, that the condition D1 is satisfied (with the distribution function G(·) playing the role of the distribution function of random variable ξ0 ). Thus, the statement (ii) of Theorem 3.1 takes place and N(t) is the standard renewal counting process described in this statement.  Remark 3.1 If the condition D1 is modified, as described in Remark 2.5, that is, it is assumed that the distribution function of the random variable ξ0 has no atom at zero, and, thus, N(0) = 0 with probability 1, the point 0 can be additionally included in the set N in the sufficiency statement. In addition, if the distribution function P{N(t) ≥ 1} = G(t), which appears in the statement (i) of Theorem 3.1, has no atom at zero, then the above-mentioned modified condition D1 is satisfied.

72

3 Flows of rare events for regularly perturbed SMP

Remark 3.2 In the case described in Remark 3.1, the conditions, A1 –D1 , imply (by Theorem 4.4.1 from Silvestrov (2004)) that a more general asymptotic relation, J

Nε (t), t ≥ 0 −→ N(t), t ≥ 0 as ε → 0, takes place. Remark 3.3 The conditions A1 –D1 of Theorem 2.1 guarantee that the distribution functions of the first-rare-event times G(ε) (·) = P{ξε ≤ ·} ⇒ G(·) as ε → 0, where G(·) is the distribution function of the limiting random variable appearing in the condition D1 . However, the above weak convergence relation does not guarantee that the probabilities G(ε) (0) converge to probability G(0) if G(0) > 0. This is due to the fact that in this case, 0 is a discontinuity point of the distribution function G(·). Since P{Nε (0) = r } = G(ε) (0)r (1 − G(ε) (0)), r = 0, 1, . . ., the point 0 cannot be included in the set of weak convergence N appearing in the asymptotic relation appearing in the statement (i) of Theorem 3.2. Also, in the above case, the J-convergence of the processes Nε (t) is not guaranteed.

3.2 Markov Renewal Processes Generated by Flows of Rare Events In this section, we present Markov renewal processes generated by flows of rare events defined on asymptotically uniformly perturbed semi-Markov processes and give necessary and sufficient conditions for weak convergence of transition probabilities for such process.

3.2.1 Return Times and Rare Events According to Lemma 2.1, the condition B1 implies that the Markov chain ηε,n is ergodic for ε ∈ (0, ε0 ], for some ε0 ∈ (0, 1]. In what follows, we assume that ε ∈ (0, ε0 ]. It is interesting that the conditions A1 –D1 do not guarantee weak convergence of the transition probabilities for the Markov renewal processes (ηε (k), κε (k)). Some additional condition must be included in the necessary and sufficient conditions for the weak convergence of the transition probabilities for the above Markov renewal processes. Let us formulate and prove several useful lemmas. Let us define the successive return moments of the Markov chain ηε,n in the state i ∈ X, αε,i (k) = min(n > αε,i (k − 1), ηε,n = i), k = 1, 2, . . . , αε,0 = 0

(3.18)

and define the following probabilities, for j, i, r ∈ X, qε, ji (r) = P j {νε ≤ αε,i (1), ηε,νε = r },

(3.19)

3.2 MRP generated by flows of rare events

73

and qε, ji = P j {νε ≤ αε,i (1)} =



qε, ji (r).

(3.20)

r ∈X

Let us also introduce the following probabilities, for i, r ∈ X. pε,i (r) = Pi {ηε,1 = r, ρε,1 = 1}, and pε (r) =

m 

(3.21)

πε,i pε,i (r).

(3.22)

i=1

Obviously, for i ∈ X, pε,i = Pi {ρε,1 = 1} =



pε,i (r),

(3.23)

r ∈X

and pε =



πε,i pε,i =

i ∈X



πε,i

i ∈X



pε,i (r) =

r ∈X



pε (r).

(3.24)

r ∈X

Lemma 3.1 Let the conditions A1 –C1 be satisfied. Then, for i ∈ X, πε,i qε,ii → 1 as ε → 0. pε

(3.25)

Let δε,ik be the number of visits of the embedded Markov chain ηε,n to the state k before the first visit to the state i. Proof Let δε,ik be the number of visits of the embedded Markov chain ηε,n to the state k before the first visit to the state i, for i, k ∈ X, δε,ik =

α ε, i (1)

I(ηε,n−1 = k).

(3.26)

n=1

We also define matrices, for i ∈ X, ⎡ pε,11 . . . pε,1 i−1 ⎢ ⎢ .. .. i Pε = ⎢ . ⎢ . ⎢ pε,m1 . . . pε,m i−1 ⎣

0 pε,1 i+1 . . . pε,1 m ⎤⎥ .. .. .. ⎥ . . . . ⎥⎥ 0 pε,m i+1 . . . pε,m m ⎥⎦

(3.27)

As is known, due to the ergodicity of the Markov chain ηε,n , for j, i, k ∈ X, E j δε,ik < ∞.

Moreover, there exists the inverse matrix, for i ∈ X,   [I − i Pε ]−1 =  E j δε,ik  . We also define matrices, for i ∈ X,

(3.28)

(3.29)

74

3 Flows of rare events for regularly perturbed SMP

⎡ p˜ε,1 1 . . . p˜ε,1 i−1 ⎢ ⎢ .. .. ˜ i Pε = ⎢ . ⎢ . ⎢ p˜ε,m 1 . . . p˜ε,m i−1 ⎣ where, for j, k ∈ X,

0 p˜ε,1 i+1 . . . p˜ε,1 m ⎤⎥ .. .. .. ⎥ , . . . ⎥⎥ 0 p˜ε,m i+1 . . . p˜ε,m m ⎥⎦

p˜ε, jk = P j {ηε,1 = k, ρε,1 = 0}.

(3.30)

(3.31)

Let δ˜ε,ik be the number of visits of the embedded Markov chain ηε,n to the state k before the first visit to the state i or the occurrence of the first-rare-event, for i, k ∈ X, δ˜ε,ik =

αε, i (1)∧νε

I(ηε,n−1 = k).

(3.32)

n=1

Obviously, δ˜ε,ik ≤ δε,ik and, thus, for j, i, k ∈ X, E j δ˜ε,ik ≤ E j δε,ik < ∞.

(3.33)

The matrix i P˜ εn can be represented in the following form, for i ∈ X and n ≥ 1,    ˜n  (3.34) i Pε = P j {ηε,n = k, αε,i (1) ∧ νε > n} , and, therefore, there exists the inverse matrix, 

I − i P˜ ε

 −1

  = I + i P˜ ε + i P˜ 2ε + · · · = E j δ˜ε,ik  .

(3.35)

The probabilities pε, jk ∈ [0, 1], j, k ∈ X, for ε ∈ (0, 1]. Therefore, any sequence εn ∈ (0, 1], n = 1, 2, . . . such that εn → 0 as n → ∞ contains a subsequence εnr , r = 1, 2, . . . such that, for j, k ∈ X, pεnr , jk → p0, jk ∈ [0, 1] as r → ∞,

(3.36)

where the limits p0, jk , j, k ∈ X can depend on the choice of the subsequence εnr . According to the above remarks, there exists, for every i ∈ X, the inverse matrix [I − i Pεnr ]−1 , for all r such that εnr ∈ (0, ε0 ]. Obviously, the matrix P0 = p0, jk  is stochastic. Moreover, the condition B1 implies that the phase space X of a Markov chain η0,n with the matrix of transition probabilities P0 is one class of communicative states, i.e., this Markov chain is ergodic. This means that for reasons similar to those used for the relations (3.28) and (3.29), there exists the inverse matrix [I − i P0 ]−1 , for i ∈ X. Thus, det(I − i P0 )  0. Elements of the matrices [I − i Pεnr ]−1 and [I − i P0 ]−1 are continuous rational functions of elements, respectively, of the matrices [I − i Pεnr ] and [I − i P0 ]. That is why, the relation (3.36) implies that, for i ∈ X, [I − i Pεnr ]−1 → [I − i P0 ]−1 as ε → 0.

(3.37)

3.2 MRP generated by flows of rare events

75

The condition A1 obviously implies that, for j, k ∈ X, | p˜ε, jk − pε, jk | ≤ pε, j → 0 as ε → 0,

(3.38)

and, therefore, for j, k ∈ X, p˜εnr , jk → p0, jk ∈ [0, 1] as r → ∞.

(3.39)

The relation (3.39) implies that for reasons similar to those used for the relation (3.37), the following relation holds, for i ∈ X, [I − i P˜ εnr ]−1 → [I − i P0 ]−1 as ε → 0.

(3.40)

The matrices i P0, i ∈ X depend on the choice of subsequence εnr . However, the relations (3.37) and (3.40) imply that, for i ∈ X, [I − i P˜ εnr ]−1 − [I − i Pεnr ]−1 → 0 as ε → 0,

(3.41)

where 0 is a m × m matrix with all zero elements. Since an arbitrary choice of sequence 0 < εn → 0 and independence of limit 0 in relation (3.41) on the choice of subsequence εnr , the relation (3.41) implies that [I − i P˜ ε ]−1 − [I − i Pε ]−1 → 0 as ε → 0

(3.42)

or, equivalently, that, for j, i, k ∈ X, E j δ˜ε,ik − E j δε,ik → 0 as ε → 0.

(3.43)

The probabilities qε, ji, j ∈ X satisfy, for every i ∈ X, the following system of linear equations,  p˜ε, jk qε,ki, j ∈ X. (3.44) qε, ji = pε, j + ki

The system (3.44) can be rewritten, for every i ∈ X, in the following matrix form, qε,i = pε + i Pε qε,i, where qε,i

⎡ qε,1i ⎤ ⎡ pε,1 ⎤ ⎢ ⎢ ⎥ ⎥ ⎢ .. ⎥ ⎢ ⎥ = ⎢ . ⎥ , pε = ⎢ ... ⎥ . ⎢ ⎢ ⎥ ⎥ ⎢ qε,mi ⎥ ⎢ pε,m ⎥ ⎣ ⎣ ⎦ ⎦

(3.45)

(3.46)

The matrix I − i P˜ ε has the inverse matrix, for ε ∈ (0, ε0 ] and i ∈ X. Therefore, the solution of system (3.44) has the following form, for every i ∈ X,  −1  qε,i = I − i P˜ ε pε . The relations (3.35) and (3.47) imply that, for i ∈ X,

(3.47)

76

3 Flows of rare events for regularly perturbed SMP

qε,ii =



Ei δ˜ε,ik pε,k .

(3.48)

k ∈X

Since the Markov chain ηε,n is ergodic, the following well-known formula takes place, for i, k ∈ X, πε,k Ei δε,ik = . (3.49) πε,i Using the relations (3.43), (3.48), (3.49), and the statement (i) of Lemma 2.1, according to which limε→0 πε,k > 0, we obtain, for i ∈ X, |qε,ii −

pε πε, i

pε πε, i

|

   πε,k  πε,i pε,k  ˜ ≤ Ei δε,ik − πε,i   j ∈X πε, j pε, j k ∈X   πε,i Ei δ˜ε,ik − Ei δε,ik  ≤ → 0 as ε → 0. πε,k k ∈X

The relation (3.50) implies that the relation (3.25) holds.

(3.50) 

Let us define probabilities, for i, j ∈ X, Qi(ε) j (∞) = Pi {ηε,νε = j}.

(3.51)

Lemma 3.2 Let the conditions A1 –C1 be satisfied. Then, the following relation holds, for i, r ∈ X, pε (r) (ε) (∞) − → 0 as ε → 0. (3.52) Qir pε Proof Taking into account that (a) the Markov renewal process (ηε,n, κε,n, ρε,n ) regenerates at the moments of return of the component ηε,n into any state i ∈ X and (b) νε is a Markov moment for this process, we can get the following cyclic (ε) representation for probabilities Qir (∞), for i, r ∈ X, (ε) Qir (∞) =

=

∞ 

Pi {αε,i (n) < νε ≤ αε,i (n + 1), ηε,νε = r }

n=0 ∞ 

(1 − qε,ii )n qε,ii (r) =

n=0

qε,ii (r) . qε,ii

(3.53)

The probabilities qε, ji (r), j ∈ X satisfy, for every i, r ∈ X, the following system of linear equations,  qε, ji (r) = pε (r) + p˜ε, jk qε,ki (r), j ∈ X. (3.54) ki

This system has the matrix of coefficients i P˜ ε (the same as the system of linear equations (3.44)) and differs from this system only by the corresponding free terms. Thus, by repeating reasoning used in the proof of Lemma 3.1, we can get the

3.2 MRP generated by flows of rare events

77

following formula similar to formula (3.35), for i, r ∈ X, qiiε (r) =

m 

Ei δ˜ε,ik pε (r).

(3.55)

k=1

Using relations (3.43), (3.49), (3.55), the statement (i) of Lemma 2.1 (according to which limε→0 πε,k > 0), and relation (3.25), we get the following relation, for i ∈ X,      qε,ii (r) πε,k  πε,i pε,k (r) pε (r)    ˜ pε  ·  qε,i − πε,i qε,i  ≤ Ei δε,ik − πε,i  ·  π p π ε,i qε,i j ∈X ε, j ε, j k ∈X   Ei δ˜ε,ik − Ei δε,ik  · πε,i · pε ≤ πε,k πε,i qε,i k ∈X → 0 as ε → 0.

(3.56)

Also, we get, using relation (3.24) and relation (3.25) given in Lemma 3.1, for i ∈ X,      pε (r)  pε (r)  pε (r)  pε    πε,i qε,i − pε  = pε  πε,i qε,i − 1    pε   (3.57) ≤ − 1 → 0 as ε → 0. πε,i qε,i Relations (3.53), (3.56), and (3.57) imply that relation (3.52) holds.



Let us introduce the following balancing condition: A2 :

pε (r) pε

→ Qr as ε → 0, for r ∈ X.

Constants Qr automatically satisfy the following relations,  Qr ≥ 0, r ∈ X, Qr = 1.

(3.58)

r ∈X

The following lemma is a direct corollary of Lemma 3.2. Lemma 3.3 Let the conditions A1 –C1 hold. Then, condition A2 is necessary and sufficient for holding (for some or every i ∈ X, respectively, in the statements of necessity and sufficiency) the following relation: (ε) (∞) → Qr as ε → 0, for r ∈ X. Qir

(3.59)

Let us define Laplace transforms, for j, i, r ∈ X, ψε, jir (s) = E{e−sξε I(ηε,νε = r)/ηε,0 = j, νε ≤ αε,i (1)), s ≥ 0 and

(3.60)

78

3 Flows of rare events for regularly perturbed SMP

ψ˜ ε, jir (s) = E j e−sξε I(ηε,νε = r, νε ≤ αε,i (1)), s ≥ 0.

(3.61)

The following lemma takes place. Lemma 3.4 Let the conditions A1 –D1 be satisfied. Then, for i, r ∈ X and s ≥ 0, pε (r) − ψε,iir (s) → 0 as ε → 0. pε

(3.62)

Proof We define Laplace transforms, for j, r ∈ X, p˜ε, jr (s) = E j e−sκε,1 I(ρε,1 = 0, ηε,1 = r) = φˆε, j (s)pε, j (r), s ≥ 0,

(3.63)

p˜ε, j (s) = E j e−sκε,1 I(ρε,1 = 1) = ϕˆε, j (s)pε, j , s ≥ 0,

(3.64)

and

where

φˆε, j (s) = E{e−sκε,1 /ηε,0 = j, ρε,1 = 1}, s ≥ 0.

(3.65)

The functions ψε, jir (s), j ∈ X satisfy, for every s ≥ 0 and i, r ∈ X, the following system of linear equations:  ψε, jir (s) = pε, jr (s) + pε, jk (s)ψε,kir (s), j ∈ X. (3.66) ki

The system (3.66) can be rewritten in the following equivalent matrix form: ˜ ε,ir (s) = p˜ ε,r (s) + i P˜ ε (s)Ψ ˜ ε,ir (s) Ψ where

⎡ ψ˜ ε,1ir (s) ⎤ ⎢ ⎥ ⎥ .. ˜ ε,ir (s) = ⎢⎢ Ψ ⎥ , p˜ ε,r (s) = . ⎢ ⎥ ⎢ ψ˜ ε,mir (s) ⎥ ⎣ ⎦

⎡ p˜ε,1r (s) ⎤ ⎢ ⎥ ⎢ ⎥ .. ⎢ ⎥, . ⎢ ⎥ ⎢ pε,mr (s) ⎥ ⎣ ⎦

(3.67)

(3.68)

and ⎡ p˜ε,11 (s) . . . p˜ε,1 i−1 (s) ⎢ ⎢ .. .. ˜ i Pε (s) = ⎢ . . ⎢ ⎢ p˜ε,m 1 (s) . . . p˜ε,m i−1 ⎣

0 p˜ε,1 i+1 (s) . . . p˜ε,1 m (s) ⎤⎥ ⎥ .. .. .. ⎥. . . . ⎥ 0 p˜ε,m i+1 (s) . . . pε,m m (s) ⎥⎦

(3.69)

Let us introduce random variables, for s ≥ 0, i, k ∈ X, δ˜ε,ik (s) =

αε, i (1)∧νε

e−sτε, n I(ηε,n−1 = k).

n=1

Obviously, δ˜ε,ik (s) ≤ δ˜ε,ik and, therefore, for s ≥ 0 and j, i, k ∈ X,

(3.70)

3.2 MRP generated by flows of rare events

E j δ˜ε,ik (s) ≤ E j δ˜ε,ik < ∞.

79

(3.71)

Moreover, the matrix i P˜ εn (s) can be represented in the following form, for s ≥ 0, i ∈ X and n ≥ 1,  −sτ  ε, n  ˜n I(ηε,n = k, αε,i (1) ∧ νε > n)} (3.72) i Pε (s) = E j e and, thus, there exists the inverse matrix, 

I − i P˜ ε (s)

 −1

  = I + i P˜ ε (s) + i P˜ 2ε (s) + · · · = E j δ˜ε,ik (s) .

(3.73)

Therefore, the solution of the system (3.67) has the following form:   ˜ ε,ir (s) = I − i P˜ ε,i (s) −1 p˜ ε,r (s). Ψ

(3.74)

The following part of the proof is similar to those given in the relations (3.36)– (3.37). Choose a sequence εn → 0 as n → ∞ and a subsequence εnr → 0 as r → ∞ in the same way as it was done in the relation (3.36). In this case, det(I − i P0 )  0, i.e., there exists the inverse matrix [I − i P0 ]−1 and the relation (3.37) holds. The condition D1 implies that, for δ > 0,  1 − Fε (δ) = πε,i (1 − Fε, j (δ)) → 0 as ε → 0. (3.75) j ∈X

Since, according to condition B1 and Lemma 2.1, limε→0 πε,i > 0, for i ∈ X, the relation (3.75) implies that, for j ∈ X and δ > 0, P j {κε,1 > δ} = (1 − Fε, j (δ)) → 0 as ε → 0.

(3.76)

The relation (3.76) and the condition B1 imply that, for any s ≥ 0, j, k ∈ X and δ > 0, |pε, jk − p˜ε, jk (s)| = |pε, jk − p˜ε, jk | + | E j (1 − e−sκε, j )I(ηε,1 = k, ρε,1 = 0}| ≤ pε, j + (1 − e−δ ) + P j {κε, j > δ} → 1 − e−δ as ε → 0. (3.77) Due to the arbitrary choice of δ > 0 in the relation (3.77), this relation implies that, for s ≥ 0 and j, k ∈ X, p˜ε, jk (s) − pε, jk ∈ [0, 1] as ε → 0.

(3.78)

The relations (3.36) and (3.78) imply that, for s ≥ 0 and j, k ∈ X, p˜εnr , jk (s) → p0, jk ∈ [0, 1] as ε → 0.

(3.79)

80

3 Flows of rare events for regularly perturbed SMP

The relation (3.79) implies that det(I − i P˜ εnr (s)) → det(I − i P0 )  0 as r → ∞, for s ≥ 0, i ∈ X. Thus, for every s ≥ 0, i ∈ X, there exists the inverse matrix [I − i P˜ εnr (s)]−1 for r large enough. Elements of the matrices [I − i P˜ εnr (s)]−1 and [I − i P0 ]−1 are continuous rational functions of elements, respectively, of the matrices [I − i P˜ εnr (s)] and [I − i P0 ]. Therefore, the relation (3.79) implies that, for s ≥ 0, i ∈ X, [I − i P˜ εnr (s)]−1 → [I − i P0 ]−1 as ε → 0.

(3.80)

The matrices i P0, i ∈ X depend on the choice of subsequence εnr . However, the relations (3.37) and (3.80) imply that, for s ≥ 0 and i ∈ X, [I − i Pεnr (s)]−1 − [I − i Pεnr ]−1 → 0 as ε → 0,

(3.81)

where 0 is a m × m matrix with all zero elements. Since an arbitrary choice of sequence 0 < εn → 0 and independence of the limit 0 in the relation (3.41) on the choice of sequence εn and subsequence εnr , the relation (3.81) implies that, for s ≥ 0 and i ∈ X, [I − i P˜ ε (s)]−1 − [I − i Pε ]−1 → 0 as ε → 0

(3.82)

or equivalently that, for s ≥ 0, j, i, k ∈ X, E j δ˜ε,ik (s) − E j δε,ik → 0 as ε → 0.

(3.83)

Taking into account the relations (3.63), (3.64), (3.73), and (3.74), we obtain the following relation, for s ≥ 0 and i, r ∈ X,  Ei δ˜ε,ik (s) ϕˆ ε,k (s)pε,k (r) (3.84) ψ˜ ε,iir (s) = k ∈X

The condition A2 implies that, for every s ≥ 0 and k ∈ X, 0 ≤ lim (1 − φˆε,k (s)) ≤ 1 − e−sδ + lim Pk {κε,1 > δ/ρε,1 = 1} ε→0

ε→0

→ 1 − exp{−sδ} as ε → 0.

(3.85)

Due to the arbitrary choice of δ > 0 in the relation (3.85), the relation (3.85) implies that, for s ≥ 0 and j, k ∈ X, φˆε,k (s) → 1 as ε → 0.

(3.86)

Obviously, for s ≥ 0 and i, r ∈ X, ψε,iir (s) =

ψ˜ ε,iir (s) . qε,ii

(3.87)

3.2 MRP generated by flows of rare events

81

Lemma 3.1, the statement (i) of Lemma 2.1 (according to which limε→0 πε,k > 0), and the relations (3.49), (3.83), (3.86) imply that, for every s ≥ 0 and i, k ∈ X, |ψε,iir (s) −

pε (r) pε (r) pε (r) | ≤| − | pε πε,i qε,ii pε  pε,k (r) πε,k pε,k (r) + | Eδε,iik (s)φˆε,k (s) − | qε,ii πε,i qε,ii k ∈X  πε,k pε pε,k (r)πε,i ≤ | Eδε,iik (s)φˆε,k (s) − | π π qε,ii pε ε,i ε,i k ∈X  πε,i πε,k pε ≤ | Eδε,iik (s)φˆε,k (s) − | π π q π ε,i ε,i ε,ii ε,k k ∈X → 0 as ε → 0.

(3.88) 

The proof is complete. The following lemma is a direct corollary of Lemma 3.4.

Lemma 3.5 Let the conditions A1 –D1 hold. Then, the condition A2 is necessary and sufficient for the fulfilment (for some or every i ∈ X, respectively, in the statements of necessity and sufficiency) of the following relation: ψε,iir (s) → Qr as ε → 0, for s ≥ 0, r ∈ X.

(3.89)

3.2.2 Necessary and Sufficient Conditions of Convergence for Markov Renewal Processes Generated by Flows of Rare Events The following theorem shows that the first-rare-event time ξε and the random functional ηε,νε are asymptotically independent and completes the description of the asymptotic behaviour of the transition probabilities Q(ε) i j (t) for the Markov renewal process (ηε (k), κε (k)). Theorem 3.2 Let the conditions A1 , B1 , and C1 be satisfied. Then: (i) The conditions D1 and A2 are necessary and sufficient for the fulfilment (for some or every initial distributions q¯ε , respectively, in the statements of necessity and d

sufficiency) of the asymptotic relation (ηε (1), ξε (1)) −→ (η0 (1), ξ0 (1)) as ε → 0, where (η0 (1), ξ0 (1)) is a random vector that takes values in the space X × [0, ∞) and has the joint distribution P{η0 (1) = r, ξ0 (1) ≤ u, } = G(r, u), r ∈ X, u ≥ 0 such that  the distribution function G(u) = r ∈X G(r, u) is not concentrated at zero. (ii) The distribution function G(r, u) = Qr G(u), r ∫∈ X, u ≥ 0, where G(u), u ≥ 0 ∞ 1 ,s ≥ 0 is the distribution function with Laplace transform 0 e−st G(dt) = 1+A(s) and A(s) is the cumulant appearing in the condition D1 , and Qr , r ∈ X are the limits appearing in the condition A2 .

82

3 Flows of rare events for regularly perturbed SMP

(iii) The following relation holds, for any rk ∈ X, k = 1, . . . , n, points uk ≥ 0, k = 1, . . . , n, which are points of continuity for the distribution function G(u), and any i ∈ X, n = 1, 2, . . ., Pi {ηε (k)) = rk , ξε (k) ≤ uk , k = 1, . . . , n}

→ Q(rk , uk , k = 1, . . . , n) =

n 

Qrk G(uk ) as ε → 0.

(3.90)

k=1

Proof The necessity part in the statement (i) of Theorem 3.2 follows from Theorem 2.1 and Lemma 3.3, since the asymptotic relation appearing in this statement d

implies that ξε (1) −→ ξ0 (1) as ε → 0, where ξ0 (1) is a non-negative random vari able with the distribution function P{ξ0 (1) ≤ u} = G(u) = r ∈X G(u, r), u ≥ 0, and d

ηε (1) −→ η0 (1) as ε → 0, where η0 (1) is a random variable taking values in space X such that P{η0 (1) = r } = Qr = G(∞, r), r ∈ X. Let us prove that the conditions A1 –C1 and D1 , A2 imply the fulfilment of the asymptotic relation appearing in the statement (i) and the relation appearing in the statement (ii) of Theorem 3.2. We define random variables, for i ∈ X, α ε, i (k)

βε,i (k) =

κε,n, k = 1, 2, . . . ,

(3.91)

n=αε, i (k−1)+1

and Laplace transforms, for j, i ∈ X, ψε, ji (s) = E{e−sβε, i (1) /ηε,0 = j, νε > αε,i (1)}, s ≥ 0.

(3.92)

The following representation, similar to the representation (3.53), takes place, for s ≥ 0, i ∈ X, Ei exp{−s

νε, i 

βε,i (k)}

k=1

= =

∞  n=0 ∞ 

Ei exp{−s

n 

βε,i (k)}I(αε,i (n) < νε ≤ αε,i (n + 1))

k=1

(1 − qε,ii )n ψε,ii (s)n qε,ii

n=0

=

qε,ii . 1 − (1 − qε,ii )ψε,ii (s)

Let us introduce random variables, for i ∈ X,

(3.93)

3.2 MRP generated by flows of rare events

83 νε 

βε,i =

κε,n,

(3.94)

n=αε, i (νε, i )+1

where νε,i = max{n ≥ 0 : αε,i (n) ≤ νε } = με,i (νε ).

(3.95)

Also, the following representation, similar to the representation (3.53), take place, for s ≥ 0, i, r ∈ X, Ei exp{−sβε,i }I(ηε,νε = r)

=

∞ 

νe 

Ei {exp{−s

κε,k }I(ηε,νε = r)/αε,i (n) < νε ≤ αε,i (n + 1)}

k=αε, i (n)+1

n=0

× P{αε,i (n) < νε ≤ αε,i (n + 1)} ∞  = ψε,iir (s)(1 − qε,ii )n qε,ii = ψε,iir (s).

(3.96)

n=0

Let us introduce Laplace transforms, for i, r ∈ X, Φε,ir (s) = Ei e−sξε I(ηε,νε = r), s ≥ 0.

(3.97)

Finally, the following representation, similar to the representation (3.53), takes place, for s ≥ 0, i, r ∈ X, Φε,ir (s) = Ei exp{−s =

∞ 

νε, i  k=1

Ei exp{−s

n=0

βε,i (k) + βε,i }I(ηε,νε = r) n 

βε,i (k)}

k=1

× exp{−sβε,i }I(ηε,νε = r, αε,i (n) < νε ≤ αε,i (n + 1)) ∞  = (1 − qε,ii )n ψε,ii (s)n qε,ii ψε,iir (s) n=0

=

qε,ii · ψε,iir (s). 1 − (1 − qε,ii )ψε,ii (s)

(3.98)

The following relation holds, for s ≥ 0, i ∈ X, Ei exp{−s

νε, i  n=1

βε,i (n)} = Ei exp{−s

αε, i (νε, i )

κε,n }.

(3.99)

n=1

The random variable αε,i (νε,i ) can be represented, for i ∈ X, in the following form, (3.100) αε,i (νε,i ) = αε,i (με,i (νε )).

84

3 Flows of rare events for regularly perturbed SMP

Lemma 2.1 allows us to apply Theorem A.6 to the random variables αε,i (με,i (|tuε ])) and get the following asymptotic relation, for i ∈ X and the function uε = p−1 ε , αε,i (με,i (|t p−1 U ε ])) ∗ = αε,i (μ∗ε,i (t), t ≥ 0 −→ t, t ≥ 0 as ε → 0, −1 pε

(3.101)

or, equivalently, αε,i (με,i (|t p−1 U ε ])) ∗ − t = αε,i (μ∗ε,i (t) − t, t ≥ 0 −→ 0(t), t ≥ 0 as ε → 0 −1 pε

(3.102)

where 0(t) ≡ 0, t ≥ 0. The relation (3.102), Lemma 2.9, and Theorem A.9 imply that, for i ∈ X, d

∗ (μ∗ε,i (t) − t), t ≥ 0 −→ (ν0, 0(t)), t ≥ 0 as ε → 0. (νε∗ , αε,i

(3.103)

The relations (3.102), (3.103) and Theorem A.4 imply that, for i ∈ X, d

∗ αε,i (μ∗ε,i (νε∗ )) − νε∗ −→ 0 as ε → 0.

(3.104)

The relation (3.104), Lemma 2.9, Theorem A.9 imply that, for i ∈ X, ∗ ∗ (αε,i (μ∗ε,i (νε∗ )), κε (t)) = (αε,i (μ∗ε,i (νε∗ )) − νε∗ + νε∗ , κε (t)), t ≥ 0 d

−→ (ν0, θ 0 (t)), t ≥ 0 as ε → 0.

(3.105)

The asymptotic relation appearing in the statement (iii) of Theorem 2.2 and the relation (3.105) allows us to apply Theorem A.4 to the càdlàgprocesses κε (t), t ≥ 0 randomly stopped at the moment νε∗ = p−1 ε νε . This gives the following relation: d

∗ κε (αε,i (μ∗ε,i (νε∗ ))) −→ ξ0 = θ 0 (ν0 ) as ε → 0,

(3.106)

where θ 0 (ν0 ) is the random variable appearing in the condition D1 . Lemma 3.5 and the relations (3.98), (3.99), and (3.106) imply that the following relation holds, i, r ∈ X and s ≥ 0, Φε,ir (s) = →

1 1 + (1 − qε,ii )

(1−ψε, ii (s)) qε, i

· ψε,iir (s)

1 · Qr as ε → 0. 1 + A(s)

(3.107) (3.108)

The relation (3.107) is equivalent to the sufficiency part of the statement (i) and the statement (ii) of Theorem 3.2. Suppose now that the conditions A1 –D1 and A2 are satisfied. Obviously, the random vector (ηε (1), κε (1)) has the following conditional distribution, for i, r ∈ X, u ≥ 0,

3.3 Vector counting processes generated by flows of rare events (ε) Qir (u) = Pi {ηε (1) = r, κε (1) ≤ u} = Pi {ηε,νε = r, ξε ≤ u}.

85

(3.109)

Using the Markov property of the Markov renewal process (ηε (k), κε (k)), we get the following relation for the joint conditional distribution function of the random variables ηε (k), κε (k), k = 1, . . . , n, for ik ∈ X, uk ≥ 0, k = 1, . . . , n, and i0 ∈ X, n = 1, 2, . . ., Pi0 {ηε (k) = ik , κε (k) ≤ uk , k = 1, . . . , n}

= Pi0 {ηε (k) = ik , κε (k) ≤ uk , k = 1, . . . , n − 1}Q(ε) in−1,in (un ) = ··· = n  Qi(ε) (uk ). = k−1,ik

(3.110)

k=1

According to the statements (i) and (ii) of Theorem 3.2, the conditions A1 –D1 and A2 imply that, for i, r ∈ X, (ε) Qir (·) ⇒ G(·)Qr as ε → 0,

(3.111)

where G(u), u ≥ 0 is the distribution function on [0, ∞) not concentrated at zero, which appears in the condition D1 and Qr , r ∈ X are limits appearing in the condition A2 . The conditions A1 –D1 and A2 and the relations (3.110) and (3.111) imply that the following convergence relation holds for the joint conditional distribution function of random variables ηε (k), κε (k), k = 1, . . . , n, for all rk ∈ X, k = 1, . . . , n and uk ≥ 0, k = 1, . . . , n, which are points of continuity for the distribution function F(u), and i ∈ X, n = 1, 2, . . ., Pi {ηε (k) = rk , κε (k) ≤ uk , k = 1, . . . , K }

→ P{η(k) = rk , κ(k) ≤ uk , k = 1, . . . , n} = Q(rk , uk , k = 1, . . . , n) as ε → 0.

(3.112)

The relation (3.112) completes the proof of the statement (iii) of Theorem 3.2.

3.3 Vector Counting Processes Generated by Flows of Rare Events In this section, we introduce vectors counting processes, in which components count the numbers of rare events in interval [0, t] occurring at moments of hitting different states by the corresponding semi-Markov processes, and give necessary and sufficient condition of convergence in distribution for such processes.

86

3 Flows of rare events for regularly perturbed SMP

3.3.1 Vector Counting Processes for Rare Events Let us consider the following vector counting processes: M¯ ε (t) = (Mε,r (t), r ∈ X), t ≥ 0, where, for r ∈ X, Mε,r (t) =

[t] 

I(ηε (k) = r), t ≥ 0,

(3.113)

(3.114)

k=1

and where, for r ∈ X,

N¯ ε (t) = (Nε,r (t), r ∈ X), t ≥ 0,

(3.115)

Nε,r (t) = Mε,r (Nε (t)), t ≥ 0.

(3.116)

Obviously, the following relation takes place:  Nε (t) = Nε,r (t), t ≥ 0.

(3.117)

r ∈X

Let η(k), κ(k), k = 1, 2, . . ., be mutually independent random variables such that: (a) the random variable η(k), k = 1, 2, . . ., takes values r with probabilities Qr , for r ∈ X, (b) κ(k), k = 1, 2, . . ., are non-negative random variables with distribution function G(u) not concentrated at zero. We also define random variables ξ(n) = n k=1 κ(k), n = 1, 2, . . . , ξ(0) = 0. ¯ Let us now define a vector frequency process, M(t) = (Mr (t), r ∈ X), t ≥ 0, where, for r ∈ X, [t]  Mr (t) = I(η(k) = r), t ≥ 0 (3.118) k=1

¯ = (Nr (t), r ∈ X), t ≥ 0, where and a counting process N(t) Nr (t) = Mr (N(t)), t ≥ 0.

(3.119)

Obviously, the following relation takes place:  N(t) = Nr (t), t ≥ 0.

(3.120)

r ∈X

Since N(t), t ≥ 0 and Mr (t), t ≥ 0, r ∈ X are integer-valued, non-negative, step-wise, non-decreasing, càdlàg processes, Mr (N(t)), t ≥ 0, r ∈ X also are integervalued, non-negative, step-wise, non-decreasing, càdlàg processes. ¯ Note also that processes N(t) and M(N(t)) have the same set N of points of stochastic continuity t > 0. The role of finite-dimensional distributions for the vector process N(t), t ≥ 0 is played by the following probabilities (which can be expressed via the joint distribu-

3.3 Vector counting processes generated by flows of rare events

87

n tions of random sums ξ(n) = k=1 κ(k) of i.i.d. random variables κ(k), k = 1, 2, . . ., with the distribution function G(·)), for (a) integers 0 ≤ m[1] ≤ m[2] ≤ · · · < ∞, and (b) 0 ≤ t1 ≤ t2 ≤ · · · < ∞, P{N(t1 ) = m[1]}

= P{N(t1 ) ≥ m[1]} − P{N(t1 ) ≥ m[1] + 1} = P{ξ(m[1]) ≤ t1 } − P{ξ(m[1] + 1) ≤ t1 }, P{N(tl ) = m[l], l = 1, 2} = P{N(t1 ) = m[1], N(t2 ) ≥ m[2]} − P{N(t1 ) = m[1], N(t2 ) ≥ m[2] + 1} = P{N(t1 ) ≥ m[1], N(t2 ) ≥ m[2]} − P{N(t1 ) ≥ m[1] + 1, N(t2 ) ≥ m[2]} + P{N(1) ≥ m[1], N(t2 ) ≥ m[2] + 1} − P{N(t1 ) ≥ m[1] + 1, N(t2 ) ≥ m[2] + 1} = P{ξ(m[1]) ≤ t1, ξ(m[2]) ≤ t2 } − P{ξ(m[1] + 1) ≤ t1, ξ(m[2]) ≤ t2 } + P{ξ(m[1]) ≤ t1, ξ(m[2] + 1) ≤ t2 } − P{ξ(m[1] + 1) ≤ t1, ξ(m[2] + 1) < t2 }, etc.

(3.121)

In what follows, l¯k = (lk,r , r ∈ X), k = 1, 2, . . ., be vectors with integer components lk,r , r ∈ X satisfying the following relation:  lk,r = k, k = 1, 2, . . . (3.122) lk,r ≥ 0, r ∈ X, r ∈X

Also, p ¯ = (pr , r ∈ X) be discrete distributions with probabilities pr ≥ 0, r ∈ X such that r ∈X pr = 1. ¯ be, for k = 1, 2, . . ., the multinomial probabilities, defined by the Let G(k, l¯k , p) following formula, for vectors l¯k = (lk,r , r ∈ X) and discrete distributions p¯ = (pr , r ∈ X) satisfying the above assumptions, G(k, l¯k , p) ¯ =

k!



r ∈X lk,r ! r ∈X

l

prk, r .

(3.123)

¯ The process M(t), t ≥ 0 is a vector process with independent increments, whose ¯ − M(s) ¯ increments M(t) have multinomial joint distributions of the components, i.e., for vectors Q¯ = (Qr , r ∈ X) and l¯m[s,t] , where m[s, t] = [t] − [s], and 0 ≤ s ≤ t < ∞, ¯ − M(s) ¯ ¯ P{ M(t) = l¯m[s,t] } = G(m[s, t], l¯m[s,t], Q).

(3.124)

¯ Note also that the counting process N(t), t ≥ 0 and the process M(t), t ≥ 0 are independent.

88

3 Flows of rare events for regularly perturbed SMP

¯ Therefore, the finite-dimensional distributions for the vector process N(t), t≥0 ¯ take the following form, for (a) any vectors lk , k = 1, 2, . . ., satisfying the relation (3.122), (b) integer 0 ≤ m[1] ≤ m[2] ≤ · · · < ∞, (c) 0 ≤ t1 ≤ t2 ≤ · · · < ∞, and (d) n = 1, 2, . . ., ¯ k ) = l¯m[k], k = 1, . . . , n} P{ N(t = P{N(tk ) = m[k], k = 1, . . . , n} ¯ × P{ M(m[k]) = l¯m[k], k = 1, . . . , n}.

(3.125)

Here, the relation (3.120) is taken into account.

3.3.2 Necessary and Sufficient Conditions of Convergence for Vector Counting Process Generated by Flows of Rare Events The following theorem generalises Theorem 3.2 and describes the asymptotics in distribution for the vector rare-event counting processes N¯ ε (t). Theorem 3.3 Let the conditions A1 , B1 , and C1 hold. Then: (i) The conditions D1 and A2 are necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and d ¯ sufficiency) of the asymptotic relation N¯ ε (t), t ∈ N −→ N(t), t ∈ N as ε → 0, where ¯ t ≥ 0 is a stochastic process with integer-valued, non-negative, non-decreasing, N(t),  step-wise, càdlàg components such that P{N(t) = r ∈X Nr (t) ≥ 1} = G(t) is the distribution function on [0, ∞) not concentrated at zero, and N is the set of stochastic continuity points t > 0 for this process. ¯ t ≥ 0 is defined by the relations (3.7), (3.118), and (ii) The limiting process N(t), (3.116) via some mutually independent random variables η(k), κ(k), k = 1, 2, . . ., where: (a) κ(k), k = 1, 2, . . . are non-negative random variables with the distribution function G(u) not concentrated at zero and with the Laplace transform ∫∞ 1 −st G(dt) = φ(s) = e 1+A(s) defined via the cumulant A(s) of infinitely divisi0 ble distribution appearing in the condition D1 , (b) η(k), k = 1, 2, . . . are random variables taking values r ∈ X with probabilities Qr , r ∈ X appearing in the condition A2 , and (c) N is the set of stochastic continuity points t > 0 for the process ¯ N(t). Proof The conditions A1 –D1 and A2 , and the relation (3.90) appearing in the statement (iii) of Theorem 3.2 imply that the following convergence relation holds for: (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) 0 < u1 ≤ u2 ≤ · · · < ∞, which are points of continuity for the distribution functions of random variables ξ(k), k = 1, 2, . . ., and (c) i ∈ X, n = 1, 2, . . ., ¯ ε (k) = l¯k , ξε (k) ≤ uk , k = 1, . . . , n} Pi { M ¯ → P{ M(k) = l¯k , k = 1, . . . , n}P{ξ(k) ≤ uk , k = 1, . . . , n} as ε → 0. (3.126)

3.3 Vector counting processes generated by flows of rare events

89

The conditions A1 –D1 and the relation (3.126) imply that the following convergence relation holds for (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) 0 < u1 ≤ u2 ≤ · · · < ∞, which are points of continuity for the distribution functions of random variables ξ(k), k = 1, 2, . . ., and (c) i ∈ X, n = 1, 2, . . ., ¯ ε (mk ) = l¯mk , Nε (tk ) ≥ rk , k = 1, . . . , n} Pi { M = Pi { M¯ ε (mk ) = l¯mk , ξε (rk ) ≤ tk , k = 1, . . . , n} ¯ k ) = l¯mk , k = 1, . . . , n}P{ξ(rk ) ≤ tk , k = 1, . . . , n} → P{ M(m ¯ k ) = l¯mk , k = 1, . . . , n}P{N(tk ) ≥ rk , k = 1, . . . , n} = P{ M(m as ε → 0. (3.127) The following relations, similar to (3.121), take place, for (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) integers 0 ≤ m1 ≤ m2 ≤ · · · < ∞ and 0 ≤ m[1] ≤ m[2] ≤ · · · , (c) 0 < t1 ≤ t2 ≤ · · · < ∞, and (d) i ∈ X, ¯ ε (m1 ) = l¯m1 , Nε (t1 ) = m[1]} Pi { M = Pi { M¯ ε (m1 ) = l¯m1 , Nε (t1 ) ≥ m[1]} − Pi { M¯ ε (m1 ) = l¯m1 , Nε (t1 ) ≥ m[1] + 1} = Pi { M¯ ε (m1 ) = l¯m1 , ξε (m[1]) ≤ t1 } − Pi { M¯ ε (m1 ) = l¯m1 , ξε (m[1] + 1) ≤ t1 }, ¯ ε (ml ) = l¯ml , Nε (tl ) = m[l], l = 1, 2} Pi { M = Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) = m[1], Nε (t2 ) ≥ m[2]} − Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) = m[1], Nε (t2 ) ≥ m[2] + 1} = Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) ≥ m[1], Nε (t2 ) ≥ m[2]} − Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) ≥ m[1] + 1, Nε (t2 ) ≥ m[2]} + Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) ≥ m[1], Nε (t2 ) ≥ m[2] + 1} − Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, Nε (t1 ) ≥ m[1] + 1, Nε (t2 ) ≥ m[2] + 1} = Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, ξε (m[1]) ≤ t1, ξε (m[2]) ≤ t2 } − Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, ξε (m[1] + 1) ≤ t1, ξε (m[2]) ≤ t2 } + Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, ξε (m[1]) ≤ t1, ξε (m[2] + 1) ≤ t2 } − Pi { M¯ ε (ml ) = l¯ml , l = 1, 2, ξε (m[1] + 1) ≤ t1, ξε (m[2] + 1) < t2 }, etc.

(3.128)

The relations (3.127 and (3.128) imply that, for (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) integers 0 ≤ m1 ≤ m2 ≤ · · · < ∞ and 0 ≤ m[1] ≤ m[2] ≤ · · · , (c) 0 < t1 ≤ · · · ≤ tn < ∞, which are the points of continuity for the corresponding limiting distribution functions of random variables ξ(k), k = 1, . . ., and (d) i ∈ X, n = 1, 2, . . .,

90

3 Flows of rare events for regularly perturbed SMP

¯ ε (ml ) = l¯ml , Nε (tl ) = m[l], l = 1, . . . , n} Pi { M ¯ l ) = l¯ml , l = 1, . . . , n} → P{ M(m × P{N(tl ) = m[l], l = 1, . . . , n} as ε → 0.

(3.129)

Finally, the conditions A1 –D1 and A2 , and the relations (3.117) and (3.129), imply that, for (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) integers 0 ≤ m[1] ≤ m[2] ≤ · · · , (c) 0 < t1 ≤ · · · ≤ tn < ∞, which are the points of continuity for the corresponding limiting distribution functions of random variables ξ(k), k = 1, . . ., and (d) i ∈ X, n = 1, 2, . . ., Pi { N¯ ε (tk ) = l¯m[k], k = 1, . . . , n}

= Pi {Nε (tk ) = m[k], M¯ ε (m[k]) = l¯m[k], k = 1, . . . , n} → P{N(tk ) = m[k], k = 1, . . . , n} ¯ × P{ M(m[k]) = l¯m[k], k = 1, . . . , n} ¯ k ) = l¯m[k], k = 1, . . . , n} as ε → 0. = P{ N(t

(3.130)

The relation (3.14) implies that the following asymptotic relation holds, for any initial distributions q¯ε , d ¯ t ∈ K as ε → 0. N¯ ε (t), t ∈ K −→ N(t),

(3.131)

¯ t ≥ 0 is a càdlàgprocess with non-decreasing components. The The process N(t), set K is at most countable and, thus, the set K ⊆ N is dense in the interval [0, ∞). Therefore, the relation (3.15) can be extended in a standard way to the asymptotic ¯ relation appearing in the statement (i), with the limiting counting process N(t) described in the statement (ii) of Theorem 3.3. Suppose now that the conditions A1 –C1 are satisfied and the following asymptotic relation takes place for some initial distributions q¯ε , d ¯ t ∈ N as ε → 0, N¯ ε (t), t ∈ N −→ N(t),

(3.132)

¯ is some stochastic process described in the statement (i) of Theorem 3.3, where N(t) and N is the set of points of stochastic continuity t > 0 for this process. Let us prove that, in this case, the conditions D1 and A2 are satisfied. The relation (3.132) implies that the following relation holds, for the above initial distributions q¯ε and t ∈ N, Nε (t) =

 r ∈X

d

Nε,r (t), t ∈ N −→ N(t) =



Nr (t), t ∈ N as ε → 0.

(3.133)

r ∈X

Obviously, N(t) is an integer-valued, non-negative, non-decreasing, step-wise, càdlàgstochastic process such that P{N(t) ≥ 1} = G(t) is the distribution function on [0, ∞) not concentrated at zero, and N is the set of stochastic continuity points t > 0 for this process.

3.3 Vector counting processes generated by flows of rare events

91

Therefore, according to the statement (i) of Theorem 3.1, the condition D1 is satisfied, and, according to the statement (ii) of Theorem 3.1, G(u) is the distribution 1 , where A(s) is the cumulant appearing in function with the Laplace transform 1+A(s) the condition D1 . Since the functions pε (r)/pε ∈ [0, 1] for ε ∈ (0, 1], r ∈ X, any sequence 0 < εn → 0 as n → ∞ contains a subsequence 0 < εnl → 0 as l → ∞ such that the following relation holds: pεnl (r) pε n l

→ Qr as l → ∞, for r ∈ X.

(3.134)

Obviously, the conditions A1 –C1 are satisfied for the Markov renewal processes (ξεnl (k), ηεnl (k)). According to the above remarks, the condition D1 also holds for the Markov renewal processes (ξεnl (k), ηεnl (k)). The relation (3.134) plays the role of the condition A2 for the Markov renewal processes (ξεnl (k), ηεnl (k)). Therefore, the statement (ii) of Theorem 3.3 takes place for the Markov renewal processes (ξεnl (k), ηεnl (k)). According to this statement, the following asymptotic relation holds, for (a) any vectors l¯k , k = 1, 2, . . ., satisfying the relation (3.122), (b) t ∈ N, and (c) i ∈ X, ¯ εn (k) = l¯k } Pi { N¯ εnl (t) = l¯k } = Pi {Nεnl (t) = k, M l ¯ = l¯k } → P{ N(t) ¯ = P{N(t) = k}P{ M(1) = l¯k } ¯ as l → ∞. = P{N(t) = k}G(k, l¯k , Q)

(3.135)

Note that, according to the relations (3.132) and (3.133), the probabilities ¯ P{ N(t) = l¯k } and P{N(t) = k} do not depend on the choice of subsequence εnl ,

¯ may depend on the while vectors Q¯ = (Qr , r ∈ X) and, thus, probabilities G(k, l¯k , Q) choice of subsequence εnl .  P Obviously, N(t) −→ ∞ as t → ∞. Therefore, P{N(t) ≥ 1} = k ≥1 P{N(t) = k} > 0 for t large enough. Thus, there exists t ∈ N such that P{N(t) = k} > 0 for some k ≥ 1. Let us choose an arbitrary r ∈ X and, then, a vector l¯k = (kI(li = lr ), i ∈ X). In this case, the equality in relation (3.135) takes the following form: ¯ = l¯k } = P{N(t) = k}Qrk . P{ N(t)

(3.136)

This equality implies that probability Qr is the same for any sequence εn and subsequence εnl chosen such that the asymptotic relation (3.134) holds. It is so for any r ∈ X. Thus, the following asymptotic relation takes place:

where Qr ≥ 0, r ∈ X,



pε (r) → Qr as ε → 0, for r ∈ X, pε

r ∈X Qr

= 1.

(3.137)

92

3 Flows of rare events for regularly perturbed SMP

Therefore, not only the condition D1 , but also the condition A2 holds.



Remark 3.4 In the case, where condition D1 is modified in the way described in Remark 3.3, Nr (0) = 0 with probability 1, for r ∈ X, and, thus, the point 0 can be additionally included in the set N. Remark 3.5 In the case described in Remark 3.4, the relation (3.11) implies (by Theorem 4.4.1 from Silvestrov (2004)) that, under the conditions, A1 –D1 and A2 , J ¯ t ≥ 0 as ε → 0, takes the more general asymptotic relation, N¯ ε (t), t ≥ 0 −→ N(t), place.

Chapter 4

Generalisations of Limit Theorems for First-Rare-Event Times

In this chapter, we generalise the results of Chap. 2 to some modifications of firstrare-event times and processes. The relationship between first-rare-event times and hitting times is also clarified. This chapter includes two sections. In Sect. 4.1, we present results generalising Theorem 2.1. Theorem 4.1 presents necessary and sufficient conditions of convergence in distribution for vector first-rareevent times and convergence in the topology J for vector first-rare-event processes defined on asymptotically uniformly ergodic semi-Markov processes. Lemmas 4.2 and 4.3 present necessary and sufficient conditions of convergence in distribution for low and upper first-rare-event times and processes. Theorems 4.2 and 4.3 present necessary and sufficient conditions of convergence in distribution for first-rare-event times and convergence in the topology J for first-rare-event processes for models with transition periods. Finally, Theorem 4.4 presents necessary and sufficient conditions of convergence in distribution for first-rare-event times for models with extending phase spaces of the corresponding semi-Markov processes. In Sect. 4.2, we clarify connection between first-rare-event times and standard and directed hitting times for semi-Markov processes. In Theorems 4.5 and 4.6 we present necessary and sufficient conditions of convergence in distribution, respectively, for standard and directed hitting times defined on asymptotically uniformly ergodic semi-Markov processes.

4.1 Modifications of First-Rare-Event Times and Rare-Event Time Processes In this section, we present several generalisations of asymptotic results given in Chap. 2 on vector first-rare-event times and processes, low and upper first-rare-event times and processes for models with transition periods, and first-rare-event times for models with extending phase spaces of the corresponding semi-Markov processes.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_4

93

94

4 Generalisations of limit theorems for first-rare-event times

4.1.1 Vector First-Rare-Event Times and Rewards 4.1.1.1 Necessary and Sufficient Conditions of Convergence for Vector FirstRare-Event Times. Let (ηε,n, κ¯ε,n = (κε,1,n, . . . , κε, L,n ), ρε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, i.e., a homogenous Markov chain with a phase space Z = X × R+L × {0, 1}, where X = {1, 2, . . . , m} is a finite set and R+L = { x¯ = (x1, . . . , x L ) : x1, . . . , xl ∈ [0, ∞)}, an initial distribution qˆε = qε,i,s,ı = P{ηε,0 = i, κ¯ε,0 ≤ s¯, ρε,0 = ı}, (i, s, ı) ∈ Z , and transition probabilities defined by the following relation, for (i, s¯, ı), ( j, t¯, j) ∈ Z, P{ηε,1 = j, κ¯ε,1 ≤ t¯, ρε,1 = j/ηε,0 = i, κ¯ε,0 = s¯, ρε,0 = ı} = P{ηε,1 = j, κ¯ε,1 ≤ t¯, ρε,1 = j/ηε,0 = i}

= Q ε,i j (t¯, j).

(4.1)

In this case, we are interested in vector first-rare-event times, ξ¯ε = (ξε,1, . . . , ξε, L ) νε νε νε    κε,1,n, . . . , κε, L,n ) = κ¯ε,n =( n=1

n=1

(4.2)

n=1

and vector first-rare-event time processes, ξ¯ε (t) = (ξε,1 (t), . . . , ξε, L (t)) =(

[tν ε] 

κε,1,n, . . . ,

n=1

[tν ε]  n=1

κε, L,n ) =

[tν ε] 

κ¯ε,n, t ≥ 0,

(4.3)

n=1

where νε = min(n ≥ 1 : ρε,n = 1).

(4.4)

In this model, the components of the random vector ξ¯ε can be interpreted in different ways. For example, the first component ξε,1 can be interpreted as the firstrare-event time, while the other components ξε,l, l > 1 as rewards accumulated on trajectories of Markov chain ηε,n until the first-rare-event time. Let us recall the stationary distribution π¯ ε = πε,i, i ∈ X for the imbedded Markov chain ηε,n . Let us also recall the one-step probabilities of occurrence of the first-rare-event, pε,i = Pi {ρε,1 = 1}, i ∈ X, and pε =

 i ∈X

πε,i pε,i and uε = p−1 ε .

The conditions A1 and B1 do not change. The condition C1 takes, in this case, the following form:

(4.5) (4.6)

4.1 Modifications of first-rare-event times

95

C3 : Pi {κε,l,1 > δ/ρε,1 = 1} = P{κε,l,1 > δ/ρε,1 = 1, ηε,0 = i} → 0 as ε → 0, for δ > 0, l = 1, . . . , L, i ∈ X. Note that we define the probability Pi {κε,l,1 > δ/ρε,1 = 1} = 0 in the cases where

Pi {ρε,1 = 1} = 0.

Let us introduce multi-dimensional distribution functions, for i ∈ X and ε ∈ (0, 1], Fε,i (t¯) = Pi { κ¯ε,1 ≤ t¯}, t¯ ∈ R+L .

(4.7)

Let θ¯ε,n, n = 1, 2, . . . be i.i.d. random vectors with the distribution function Fε (t¯), which is the result of averaging the distribution functions Fε,i (t¯) over the stationary distribution of the imbedded Markov chain ηε,n ,  πε,i Fε,i (t¯), t ∈ R+L . (4.8) Fε (t¯) = i ∈X

Now we can formulate a necessary and sufficient condition for the convergence in the distribution of the vectors of the first-rare-event times:  ε] ¯ d ¯ ¯ D7 : θ¯ε = (θ ε,1, . . . , θ ε, L ) = [u n=1 θ ε,n −→ θ 0 = (θ 0,1, . . . , θ 0, L ) as ε → 0, where θ 0 is a random vector, with non-negative components not concentrated at zero. As well known, the limiting random vector θ¯0 penetrating condition D7should be L ¯ infinitely divisible, i.e., its Laplace transform has the form, Ee−(s,¯ θ0 ) = Ee− l=1 sl θ0, l = + −A( s) ¯ , s¯ = (s1, . . . , s L ) ∈ R L, t ≥ 0, where A(s¯) is the cumulant of the corresponding e multi-dimensional infinitely divisible distribution. Let us also consider a homogeneous step-sum process with independent increments (summands are i.i.d. random vectors), θ¯ε (t) =

[tu ε] 

θ¯ε,n, t ≥ 0.

(4.9)

n=1

According to Theorem A.3, the condition D7 is necessary and sufficient for holding of the asymptotic relation, θ¯ε (t) =

[tu ε] 

J θ¯ε,n, t ≥ 0 −→ θ¯0 (t), t ≥ 0 as ε → 0,

(4.10)

n=1

where θ 0 (t) = (θ 0,1 (t), . . . , θ 0, L (t)), t ≥ 0 is the vector Lévy process (a càdlàg homogeneous process with independent increments) with non-negativecomponents L ¯ not concentrated at zero and with Laplace transforms Ee−(s,¯ θ0 (t)) = Ee− l=1 sl θ0, l (t)) = ¯ , s¯ = (s , . . . , s ) ∈ R+ , t ≥ 0, where A( s¯) is the cumulant of infinitely divisible e−t A(s) 1 L L random vector with non-negative components not concentrated at zero appearing in condition D7 . Let us recall the initial distribution q¯ε = qε,i = P{ηε,0 = 0}, i ∈ X for the imbedded Markov chain ηε,n . The following analogue of Theorem 2.1 takes place.

96

4 Generalisations of limit theorems for first-rare-event times

Theorem 4.1 Let the conditions A1 , B1 , and C3 be satisfied. Then: (i) The condition D7 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) d of the asymptotic relation ξ¯ε = ξ¯ε (1) −→ ξ¯0 as ε → 0, where ξ¯0 is a random vector with non-negative components not concentrated at zero. (ii) The distribution function G(u) ¯ = P{ ξ¯0 ≤∫ u}, ¯ u¯ ∈ R+L of the limiting random ∞ −(s, 1 + ¯ G(d u) ¯ = 1+A( vector ξ¯0 has the Laplace transform φ(s¯) = 0 e ¯ u) s) ¯ , s ∈ RL , where A(s¯) is the cumulant of multivariate infinitely divisible random vector with non-negative components not concentrated at zero appearing in the condition D7 . J

(iii) The stochastic processes ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t) = θ¯0 (tν0 ), t ≥ 0 as ε → 0, where: (a) ν0 is a random variable, which has the exponential distribution with parameter 1, (b) θ¯0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace ¯ ¯ , s¯ ∈ R+ , t ≥ 0, and (c) the random variable ν and transforms Ee−(s,¯ θ0 (t)) = e−t A(s) 0 L the process θ¯0 (t), t ≥ 0 are independent. The proof can be reduced to the scalar case using the method known as the Wold–Cramér device (see, for example, Silvestrov (2004) or Kallenberg (2021)). The following “scalar” random variables can be defined, for s¯ = (s1, . . . , s L ) ∈ R+L , θ ε [s¯] =

L 

sl θ ε,l .

(4.11)

l=1

The condition D7 can, according to the Wold–Cramér device, be reformulated in the following equivalent form: L d sl θ 0,l as ε → 0, for s¯ = (s1, . . . , s L ) ∈ R+L , where D8 : θ ε [s¯] −→ θ¯0 [s¯] = l=1 θ¯0 = (θ 0,1, . . . , θ 0, L ) is a random vector, with non-negative components not concentrated at zero. Then, Theorem 2.1 can be applied to the following random variables, for s¯ = (s1, . . . , s L ) ∈ R+L , νε L   sl ξε,l = κε,n [s¯], (4.12) ξε [s¯] = n=1

l=1

where κε,n [s¯] =

L 

sl κε,l,n, n = 1, 2, . . . .

(4.13)

l=1

In this way, we shall prove that, under the conditions A1 , B1 , and C5 , the condition D7 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in statements of necessity and sufficiency) of the following relation, for s¯ = (s1, . . . , s L ) ∈ R+L , d

ξε [s¯] −→ ξ0 [s¯] =

L  l=1

sl ξ0,l as ε → 0,

(4.14)

4.1 Modifications of first-rare-event times

97

where ξ¯0 = (ξ0,1, · · · , ξ0, L ) is a random vector with non-negative components not concentrated at zero. The relation (4.14) is, according to the Wold–Cramér device, equivalent to the following asymptotic relation: d ξ¯ε −→ ξ¯0 as ε → 0.

(4.15)

The proof of the statement (iii) can be accomplished in the following way. First, the Wold–Cramér device can be used in the manner described above to obtain the following relation of convergence in distribution for stochastic processes  (tνε∗ , κ¯ε (t)) = (t pε νε, k ≤tuε κ¯ε,k ), t ≥ 0, similar to the relation (2.123) given in Lemma 2.9, d (tνε∗ , κ¯ε (t)), t ≥ 0 −→ (tν0, θ¯0 (t)), t ≥ 0 as ε → 0, (4.16) where: (a) ν0 is a random variable, which has the exponential distribution with parameter 1, (b) θ¯0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s¯ ∈ R+L, t ≥ 0, and (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent. Also, the relation of J-convergence, J

( κ¯ε (t)), t ≥ 0 −→ θ¯0 (t), t ≥ 0 as ε → 0,

(4.17)

can be proved for the vector case (L > 1) in the way analogous to those used in the proof of Theorem 2.2 for the scalar case (L = 1). Finally, the asymptotic relations (4.16) and (4.17) let us apply Theorem A.6 to the compositions of stochastic processes κε (t), t ≥ 0 and tνε∗ . This gives the following relation: J (4.18) κ¯ε (tνε∗ ), t ≥ 0 −→ θ¯0 (tν0 ), t ≥ 0 as ε → 0. 4.1.1.2 Sufficient Conditions of Convergence for Real-Valued First-RareEvent Rewards. Another natural generalisation concerns the model with real-valued rewards. In this case, we consider a Markov renewal process (ηε,n, κε,n, ρε,n ), n = 0, 1, . . . with a phase space X × R1 × {0, 1} (X = {1, . . . , m} is a finite set and R1 = (−∞, ∞) is a real line). The corresponding real-valued first-rare-event process can be defined in the standard way and represented as the difference of two non-negative first-rare-event processes, νε t  κε,n = ξε+ (t) − ξε− (t), t ≥ 0, (4.19) ξε (t) = n=1

where ξε± (t) =

νε t 

± ± κε,n and κε,n = ±κε,n I(±κε,n ≥ 0), n ≥ 1.

(4.20)

n=1

One can consider the vector first-rare-event process ξ¯ε (t) = (ξε+ (t), ξε− (t)), t ≥ 0 and apply to these processes the sufficiency statements of Theorem 4.1. The relations

98

4 Generalisations of limit theorems for first-rare-event times

of convergence in distribution and J-convergence for the processes ξε (t), t ≥ 0 obviously imply the corresponding convergence relations for the processes ξε (t) = ξε+ (t) − ξε− (t)), t ≥ 0, but not vice versa. The question about necessary and sufficient convergence conditions for processes ξε (t), t ≥ 0 (based on the characteristics of the Markov renewal processes (ηε,n, κε,n ) averaged over the stationary distributions of the embedded Markov chains ηε,n ) is open for the case where the corresponding “external” limiting process is a general Lévy process with a Gaussian component. Our conjecture is that this can be done by using methods similar to those used in the proofs of Theorems 2.1 and 2.2 but combined with a more sophisticated approach to proving the tightness for distributions of the corresponding non-monotone, in this case, step-sum reward processes and the asymptotic independence of these processes and the first-rare-event times. However, this is outside the scope of this book.

4.1.2 Upper and Lower First-Rare-Event Times Let us define the random variables ξε,+ and ξε,− , which can be interpreted, respectively, as the upper first-rare-event time and the lower first-rare-event time, ξε,+ = ξε =

νε 

κε,n, and ξε,− =

n=1

ν ε −1

κε,n .

(4.21)

n=1

Lemma 4.1 Let the condition C1 be satisfied. Then, the following relation holds, for any initial distribution q¯ε , P

κνε = ξε,+ − ξε,− −→ 0 as ε → 0.

(4.22)

Proof The Markov property of the Markov renewal process (ηε,n, κε,n , ρε,n ) and the condition C1 imply that, for δ > 0 and i ∈ X, Pi {κε,νε > δ} =

= =

∞ 

Pi {κε,n n=1 ∞  

> δ, νε = n}

Pi {κε,n > δ, ηε,n−1 = j, νε > n − 1, ρε,n = 1}

n=1 j ∈X ∞  

P{κε,n > δ, ρε,n = 1/ηε,n−1 = j, νε > n − 1}

n=1 j ∈X

× Pi {ηε,n−1 = j, νε > n − 1}

4.1 Modifications of first-rare-event times

=

∞  

99

P{κε,n > δ, ρε,n = 1/ηε,n−1 = j}

n=1 j ∈X

× Pi {ηε,n−1 = j, νε > n − 1} =

∞  

P{κε,n > δ/ρε,n = 1, ηε,n−1 = j}

n=1 j ∈X

× P{ρε,n = 1/ηε,n−1 = j}Pi {ηε,n−1 = j, νε > n − 1} ∞   = P{κε,1 > δ/ρε,1 = 1, ηε,0 = j} n=1 j ∈X

× Pi {ηε,n−1 = j, νε = n} ≤ max P j {κε,1 > δ/ρε,1 = 1} j ∈X

×

∞  

Pi {ηε,n−1 = j, νε = n}

n=1 j ∈X

= max P j {κε,1 > δ/ρε,1 = 1} → 0 as ε → 0.

(4.23)

j ∈X



The proof is complete. The condition C1 can be reformulated in the following equivalent form: C1 :

Pi {κε,1 > δ, ρε,1 = 1} ≤ oi,δ (ε)Pi {ρε,1 = 1}, where 0 ≤ oi,δ (ε) → 0 as ε → 0, for δ > 0, i ∈ X.

The Markov inequality Pi {κε,1 > δ/ρε,1 = 1} ≤ δ−1 Ei {κε,1 /ρε,1 = 1}, δ > 0 implies that the condition C1 can also be replaced by the following sufficient condition: C4 : Ei κε,1 I(ρε,1 = 1) ≤ oi (ε)Pi {ρε,1 = 1}, where 0 ≤ oi (ε) → 0 as ε → 0, for i ∈ X. 1

α ) α P {ρ β Also, the Hölder inequality Ei κε,1 I(ρε,1 = 1) ≤ (Ei κε,1 i ε,1 = 1}) (where 1 1 α, β > 1, α + β = 1) implies that the following condition is sufficient for the fulfilment of the condition C1 : 1

α

α P {ρ α β ≤ o C5 : Ei κε,1 i ε,1 = 1} i,α,β (ε)Pi {ρε,1 = 1} , where 0 ≤ oi,α,β (ε) → 0 as ε → 0, for i ∈ X.

It is useful to note that the inequality in the condition C5 automatically holds, α < ∞ and P {ρ if Ei κε,1 i ε,1 = 1} = 0. In the case, where Pi {ρε,1 = 1} > 0, this inequality reduces to the inequality, α Ei κε,1 ≤ oi,α,β (ε)Pi {ρε,1 = 1}

α(β−1) β

.

(4.24)

The relation (4.23) implies that, under the condition C1 , the following relation holds, for any initial distributions q¯ε ,

100

4 Generalisations of limit theorems for first-rare-event times P

κνε = ξε,+ − ξε,− → 0 −→ 0 as ε → 0.

(4.25)

Relation (4.25)1 implies that the upper first-rare-event times ξε,+ = ξε can be replaced in Theorem 2.1 by the low first-rare-event times ξε,− . Moreover, let ξε,◦ be random variables for which there exists ε◦ ∈ (0, 1] such that the following relation holds, for ε ∈ (0, ε◦ ], ξε,− ≤ ξε,◦ ≤ ξε,+ .

(4.26)

The relations (4.25) and (4.26) imply that the following lemma takes place. Lemma 4.2 Let ξε,◦ be random variables, for which the relation (4.26) holds. Then, under the conditions A1 , B1 , and C1 of Theorem 2.1, the condition D1 is necessary and sufficient (for some or any initial distributions q¯ε , respectively, in the statements d

of necessity and sufficiency) for holding of the following relation, ξε,◦ −→ ξ0 as ε → 0, where the limiting random variable ξ0 is described in Theorem 2.1. A similar result can be obtained for the first-rare-event processes. We define the stochastic processes, which can be interpreted, respectively, as the upper rare-event time process and the lower rare-event time process, ξε,+ (t) = ξε (t) =

[tν ε] 

κε,n, t ≥ 0 and ξε,− (t) =

n=1

[t(ν ε −1)]

κε,n, t ≥ 0.

(4.27)

n=1

Let us also prove that, under the conditions of A1 –D1 of Theorem 2.1, the following relation holds: d

ξε,+ (t) − ξε,− (t), t ≥ 0 −→ 0(t) ≡ 0, t ≥ 0 as ε → 0.

(4.28)

Indeed, the weak convergence relation (2.139) given in the proof of Lemma 2.9 can be rewritten in the following equivalent doubled form: (tνε∗ , tνε∗ , κε (t), κε (t)) = (t

[tuε ] [tu ε]  νε νε  ,t , κε,k , κε,k ), t ≥ 0 uε uε k=1 k=1

d

−→ (tν0, tν0, θ 0 (t), θ 0 (t)), t ≥ 0 as ε → 0,

(4.29)

where the limiting random variable ν0 and the stochastic process θ 0 (t), t ≥ 0 are described in Lemma 2.9. Obviously, for t ≥ 0, −1 tνε∗ − t(νε∗ − u−1 ε ) = tuε → 0 as ε → 0.

(4.30)

The relations (4.29), (4.30), and Theorem A.10 imply that the following relation holds:

4.1 Modifications of first-rare-event times ∗ (t(νε∗ − u−1 ε ), tνε ,

101

[tu ε] 

κε,n,

[tu ε] 

n=1

κε,n ), t ≥ 0

n=1

d

−→ (tν0, tν0, θ 0 (t), θ 0 (t)), t ≥ 0 as ε → 0.

(4.31)

The relation of J-convergence for processes κε (t), t ≥ 0 appearing in the statement (iii) of Theorem 2.2 can obviously be rewritten in the following doubled form: (κε (t), κε (t)) = (

[tu ε] 

κε,n,

n=1

[tu ε] 

κε,n )), t ≥ 0

n=1

J

−→ (θ 0 (t), θ 0 (t)) as ε → 0.

(4.32)

The relations (4.31) and (4.32) imply, by Theorem A.6, that the following relation holds: ∗ (ξε,− (t), ξε,+ (t)) = (κε (t(νε∗ − u−1 ε )), κε (tνε ))

=(

[t(ν ε −1)]

κε,n,

n=1

[tν ε] 

κε,n )), t ≥ 0

n=1

d

−→ (θ 0 (tν0 ), θ 0 (tν0 )) as ε → 0.

(4.33)

The relation (4.33) obviously implies the fulfilment of the relation (4.28). Let ξε,◦ (t), t ≥ be some càdlàg stochastic processes, for which the following relation takes place, for 0 < ε ≤ ε◦ ≤ 1, ξε,− (t) ≤ ξε,◦ (t) ≤ ξε,+ (t), for t ≥ 0.

(4.34)

The relations (4.33) and (4.34) imply that the following lemma takes place. Lemma 4.3 Let ξε,◦ (t), t ≥ 0 be càdlàg stochastic processes, for which the relation (4.34) holds. Then, under the conditions A1 , B1 , and C1 of Theorem 2.1, the condition D1 is necessary and sufficient (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) for the fulfilment of the relation, d

ξε,◦ (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 )), t ≥ 0 as ε → 0, where the limiting process ξ0 (t), t ≥ 0 is described in Theorem 2.1. It is useful to note that the relations (4.31) and (4.32) imply, by Theorem A.6, holding of the following relation, J

ξε,± (t), t ≥ 0 −→ θ 0 (tν0 ), t ≥ 0 as ε → 0.

(4.35)

However, the vector processes (ξε,− (t), ξε,+ (t)), t ≥ 0 may not converge in the topology J. Indeed, the processes ξε,+ (t) and ξε,− (t) are connected by the relation ε ), t ≥ 0, i.e., the process ξε,− (t) is a shifted in time version of the ξε,+ (t) = ξε.− (t νεν−1 process ξε,+ (t). This can lead to the fact that the vector processes (ξε,− (t), ξε,+ (t))

102

4 Generalisations of limit theorems for first-rare-event times

can have large increments at close moments τ − c ≤ τ  ≤ τ ≤ τ  ≤ τ + c and, therefore, the J-compactness may not take place for these vector processes. As a consequence, the J-convergence of the processes ξε,◦ (t), t ≥ 0 also cannot be guaranteed. The vector processes (ξε,− (t), ξε,+ (t)), t ≥ 0 converge in the topology J if the limiting process θ 0 (tν0 ), t ≥ 0 is continuous (in this case, J-convergence is equivalent to U-convergence). This is so if the limiting process θ 0 (t), t ≥ 0 is a nonrandom linear function, i.e., θ 0 (t) = a0 t, t ≥ 0, where a0 > 0 is a positive constant. In this case, the following relation takes place, U

ξε,◦ (t), t ≥ 0 −→ a0 ν0 t, t ≥ 0 as ε → 0.

(4.36)

4.1.3 First-Rare-Event Times for Markov Renewal Processes with Transition Periods 4.1.3.1 First-Rare-Event Times and Processes for Markov Renewal Process with ˜ = {1, . . . , m} Transition Period. Let X = {1, . . . , m} and X ˜ are two finite sets such ˜ that X ⊆ X. We define a Markov renewal process (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), n = 0, 1, . . ., with transition ˜ × [0, ∞) × {0, 1} for the initial period as a Markov chain, with the phase space Z˜ = X random vector (η˜ε,0, κ˜ε,0, ρ˜ ε,0 ) and the phase space Z = X×[0, ∞)×{0, 1} for random vectors (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), n ≥ 1, initial distribution q˜ε = qε,i,s,ı = P{η˜ε,0 = i, κ˜ε,0 ≤ ˜ and transition probabilities of the following form: s, ρ˜ ε,0 = ı}, (i, s, ı) ∈ Z , P{η˜ε,n = j, κ˜ε,n ≤ t, ρ˜ ε,n = j/η˜ε,n−1 = i, κ˜ε,n−1 = s, ρ˜ ε,n−1 = ı}

= P{η˜ε,n = j, κ˜ε,n ≤ t, ρ˜ ε,n = j/η˜ε,n−1 = i}

=

˜ j ∈ X, s, t ≥ 0, ı, j = 0, 1, n = 1, Q˜ ε,i j (t, j) for i ∈ X, Q ε,i j (t, j) for i, j ∈ X, s, t ≥ 0, ı, j = 0, 1, n > 1.

(4.37)

The differences from the standard definition of the Markov renewal process are, first, that the phase space of the initial state of the first component η˜ε,0 differs from the phase space for the random sequence ηε,n, n ≥ 1 and, secondly, that the transition probabilities of this Markov renewal process at the first transition step differ from its transition probabilities at subsequent steps. As in the standard case, the first component ηε,n of the above Markov renewal ˜ for the initial state η˜ε,0 and process is also a Markov chain, with the phase space X the phase space X for random variables η˜ε,n, n ≥ 1, initial distribution q˜ε = q˜ε,i = ˜ and transition probabilities of the following form, P{η˜ε,0 = i}, i ∈ X , P{η˜ε,n = j/η˜ε,n−1 = i}

=

˜ j ∈ X, n = 1, p˜ε,i j = Q˜ ε,i j (∞, 0) + Q˜ ε,i j (∞, 1) for i ∈ X, pε,i j = Q ε,i j (∞, 0) + Q ε,i j (∞, 1) for i, j ∈ X, n > 1.

(4.38)

4.1 Modifications of first-rare-event times

103

The corresponding first-rare-event time and process are defined for this Markov renewal process as in Sect. 2.1.1, i.e., ξ˜ε =

ν˜ ε 

κ˜ε,n,

(4.39)

n=1

and ξ˜ε (t) =

[t ν˜ ε ] 

κ˜ε,n, t ≥ 0,

(4.40)

n=1

where ν˜ε = min(n ≥ 1 : ρ˜ ε,n = 1).

(4.41)

Let us also consider the shifted Markov renewal process, (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . .

(4.42)

This is a standard Markov renewal process (no transition period) with phase space Z and transition probabilities Q ε,i j (t, j). The first-rare-event time and process are defined for this Markov renewal process as in Sect. 2.1.1, i.e., νε  ξε = κε,n, (4.43) n=1

and ξε (t) =

[tν ε] 

κε,n, t ≥ 0,

(4.44)

n=1

where νε = min(n ≥ 1 : ρε,n = 1).

(4.45)

The following two relations play a key role in the subsequent analysis, ξ˜ε = κ˜ε,1 + (1 − ρ˜ ε,1 ) · = κ˜ε,1 + (1 − ρ˜ ε,1 ) ·

ν ε +1 n=2 νε 

κ˜ε,n

κε,n

n=1

= κ˜ε,1 + (1 − ρ˜ ε,1 )ξε

(4.46)

and ξ˜ε (t) = κ˜ε,1 + ρ˜ ε,1 ·

[t]  n=2

κ˜ε,n + (1 − ρ˜ ε,1 ) ·

[t(ν ε +1)] n=2

κ˜ε,n

104

4 Generalisations of limit theorems for first-rare-event times

= κ˜ε,1 + ρ˜ ε,1 ·

[t]−1 

κε,n + (1 − ρ˜ ε,1 ) ·

[t(νε +1)]−1

n=1

= κ˜ε,1 + ρ˜ ε,1 ·

[t]−1 

κε,n

n=1

κε,n − ρ˜ ε,1 · ξˆε (t) + ξˆε (t), t ≥ 0,

(4.47)

n=1

where ξˆε (t) =

[t(νε +1)]−1

κε,n, t ≥ 0.

(4.48)

n=1

4.1.3.2 First-Rare-Event Times and Processes for Markov Renewal Process with Degenerating Transition Period. According to Theorem 2.1, if the conditions A1 , B1 , and C1 are satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), then the condition D1 (for these Markov renewal processes) is necessary and sufficient for the fulfilment of the following relation (for some or any initial distributions q¯ε of the random variable ηε,0 = η˜ε,1 , respectively, in the statements of necessity and sufficiency), d

ξε −→ ξ0 as ε → 0,

(4.49)

where the limiting random variable ξ0 is described in Theorem 2.1. Also, the following relation holds (for any initial distributions q¯ε of the random variable ηε,0 = η˜ε,1 ), d

ξε (t), t ≥ 0 −→ ξ0 (t) = κ0 (tθ 0 ), t ≥ 0 as ε → 0,

(4.50)

where the limiting stochastic process ξ0 (t), t ≥ 0 is also described in Theorem 2.1. In addition, let us assume that the following conditions are satisfied: ˜ A3 : Pi { ρ˜ ε,1 = 1} → 0 as ε → 0, for i ∈ X and ˜ C6 : Pi { κ˜ε,1 > δ} → 0 as ε → 0, for δ > 0 and i ∈ X. The relation (4.46) and the conditions A3 and C6 imply that (for any initial distributions q˜ε of the random variable η˜ε,0 ) P

κ˜ε,1 − ρ˜ ε,1 ξε −→ 0 as ε → 0.

(4.51)

Note that the initial distribution q¯ = qε, j = P{ηε,0 = j}, j ∈ X , for the random q˜ = q˜ε,i = P{η˜ε,0 = variable ηε,0 = η˜ε,1 , is determined by the initial distribution ˜ for the random variable η˜ε,0 . Namely, qε, j = i ∈X˜ q˜ε,i P j {η˜ε,1 = j}, j ∈ i}, i ∈ X , X. The relations (4.49) and (4.51) imply, by Theorem A.9, that, under the conditions A3 and C6 , the relation (4.49) holds if and only if the following relation holds d ξ˜ε −→ ξ0 as ε → 0.

(4.52)

4.1 Modifications of first-rare-event times

105

An analogue of the relation (4.52) also takes place for the first-rare-event processes ξ˜ε (t), t ≥ 0.  ˆ The processes κ˜ε,1 + I( ρ˜ ε,1 = 1) · [t]−1 n=1 κε,n, t ≥ 0 and ρ˜ ε,1 ξε (t), t ≥ 0 are monotonic. Therefore, the conditions A3 and C6 imply that (for any initial distributions q˜ε of the random variable η˜ε,0 ) κ˜ε,1 + ρ˜ ε,1 ·

[t]−1 

U

κε,n, t ≥ 0 −→ 0(t), t ≥ 0 as ε → 0

(4.53)

n=1

and

U

ρ˜ ε,1 ξˆε (t), t ≥ 0 −→ 0(t), t ≥ 0 as ε → 0

(4.54)

where 0(t) = 0, t ≥ 0.  ε] νε The process ξε (t) = [tν n=1 κε,n, t ≥ 0 is a superposition of two processes, t uε , t ≥ [tuε ] 0, and n=1 κε,n , t ≥ 0. The conditions A1 –D1 (imposed on the Markov renewal processes (ηε,n, κε,n, ρε,n )) and Lemma 2.9 imply that the following relation of J-convergence takes place, [tν ε]  J κε,n, t ≥ 0 −→ θ 0 (tν0 ), t ≥ 0 as ε → 0. (4.55) ξε (t) = n=1

This follows from the statement (iii) of Theorem 2.2 and the relation (2.139) taking, in this case, the following form, ε νε  d , κε,n ), t ≥ 0 −→ (tν0, θ 0 (t)), t ≥ 0 as ε → 0, uε n=1

[tν ]

(t

(4.56)

where the random variable ν0 and the stochastic process θ 0 (t), t ≥ 0 are described in Lemma 2.9. Obviously, for t ≥ 0, |

[t(νε + 1)] − 1 tνε 1+t − |≤ → 0 as ε → 0. uε uε uε

(4.57)

The relations (4.56) and (4.57) imply that (

[tuε ] [t(νε + 1)] − 1  d , κε,n+1 ), t ≥ 0 −→ (tν0, θ 0 (t)), t ≥ 0 as ε → 0. uε n=1

(4.58)

The relations (4.55) and (4.58) imply, by Theorem A.6, that, under the conditions A1 –D1 (imposed on the Markov renewal processes (ηε,n, κε,n, ρε,n )), ξˆε (t) =

[t(νε +1)]−1 n=1

J

κε,n, t ≥ 0 −→ θ 0 (tν0 ), t ≥ 0 as ε → 0.

(4.59)

106

4 Generalisations of limit theorems for first-rare-event times

Finally, the relations (4.47), (4.53), (4.54), and (4.59) imply by Theorem A.11, that, under the conditions A1 –D1 (imposed on the Markov renewal processes (ηε,n, κε,n, ρε,n )) and A3 , C6 , the following relation holds, ξ˜ε (t) = κ˜ε,1 + ρ˜ ε,1 ·

[t]−1 

κε,n − ρ˜ ε,1 ξˆε (t) + ξˆε (t), t ≥ 0

n=1 J

−→ θ 0 (tν0 ), t ≥ 0 as ε → 0.

(4.60)

This relation is an analog of the asymptotic relation appearing in the statement (iii) of Theorem 2.1. It follows from the above remarks that the following modified version of Theorem 2.1 takes place. The remarks made above allow us to formulate a variant of Theorem 2.1 for the first-rare-event times and processes ξ˜ε and ξ˜ε (t) based on the Markov renewal processes with transition period (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) and the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ) defined by the relation (4.42). Note that in the theorem formulated below, the conditions A1 , B1 , C1 , and D1 refer to processes (ηε,n, κε,n, ρε,n ), while the conditions A3 and C6 refer to processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n ). Theorem 4.2 Let the conditions A1 , B1 , C1 and A3 , C6 be satisfied; then: (i) The condition D1 is necessary and sufficient for the fulfilment (for some or any initial distributions q˜ε , respectively, in the statements of necessity and sufficiency) d of the asymptotic relation, ξ˜ε = ξ˜ε (1) −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated in zero. (ii) The distribution function G(u) = P{ξ0∫ ≤ u}, u ≥ 0 of the limiting random ∞ 1 , s ≥ 0, where variable ξ0 has the Laplace transform φ(s) = 0 e−su G(du) = 1+A(s) A(s) is the cumulant of infinitely divisible distribution appearing in the condition D1 . J (iii) The stochastic processes ξ˜ε (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 ), t ≥ 0 as ε → 0, where (a) ν0 is a random variable, which has the exponential distribution with parameter 1, (b) θ 0 (t), t ≥ 0 is a non-negative Lévy process with the Laplace transforms Ee−sθ0 (t) = e−t A(s), s, t ≥ 0, and (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent.

It is useful to note that the statement similar to Lemma 4.2 takes place for random variables ξε,◦ such that, for some 0 < ε ≤ ε◦ ≤ 1, ξ˜ε,− ≤ ξ˜ε,◦ ≤ ξ˜ε,+, where ξ˜ε,+ = ξ˜ε =

[t ν˜ ε ]  n=1

κ˜ε,n, ξ˜ε,− =

(4.61)

[t(ν ˜ ε −1)] n=1

κ˜ε,n

(4.62)

4.1 Modifications of first-rare-event times

107

Lemma 4.4 Let ξ˜ε,◦ be random variables, for which relation (4.61) holds. Then, under conditions A1 , B1 , C1 , and A3 , C6 of Theorem 4.2, condition D1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) is necessary and sufficient (for some or any initial distributions q¯ε of the random variables η˜ε,0 , respectively, in statements of d necessity and sufficiency) for holding the asymptotic relation, ξ˜ε,◦ −→ ξ0 as ε → 0, where the limiting random variable ξ0 is described in Theorem 4.2. Also, it is useful to note that an analogue of Lemma 4.3 takes place for càdlàg stochastic processes ξε,◦ (t), t ≥ 0 such that, for some 0 < ε ≤ ε◦ ≤ 1, ξ˜ε,− (t) ≤ ξε,◦ (t) ≤ ξ˜ε,+ (t), t ≥ 0, where ξ˜ε,+ (t) = ξ˜ε (t) =

[t ν˜ ε ] 

κ˜ε,n, ξ˜ε,− (t) =

n=1

[t(ν ˜ ε −1)]

(4.63)

κ˜ε,n, t ≥ 0.

(4.64)

n=1

Lemma 4.5 Let ξ˜ε,◦ (t), t ≥ 0 be càdlàg stochastic processes, for which relation (4.63) holds. Then, under the conditions A1 , B1 , C1 , and A3 C6 of Theorem 4.2, the condition D1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) is necessary and sufficient (for some or any initial distributions q¯ε of the random variable η˜ε,0 , respectively, in the statements of necessity and sufficiency) for holding the following d relation, ξ˜ε,◦ (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 )), t ≥ 0 as ε → 0, where the limiting process ξ0 (t) is described in Theorem 4.2. 4.1.3.3 First-Rare-Event Times and Processes for Markov Renewal Process with Non-degenerating Transition Periods. Suppose now that instead of the conditions A3 and C4 , the following conditions are satisfied: ˜ where A4 : (a) P˜ε,i, j = Pi { ρ˜ ε,1 = j} → P˜0,i, j as ε → 0, for j = 0, 1, i ∈ X, P0,i, j ≥ 0, l = 0, 1, P0,i,0 + P0,i,1 = 1, and ˜ where C7 : F˜ε,i, j (·) = Pi { κ˜ε,1 ≤ ·/ ρ˜ ε,1 = j} ⇒ F˜0,i, j (·) as ε → 0, for j = 0, 1, i ∈ X, ˜ are some proper distribution functions concentrated on F˜0,i, j (·), j = 0, 1, i ∈ X [0, ∞). We also use the following convergence condition imposed on the initial distribu˜ tions q˜ε = q˜ε,i = P{η˜ε,0 = i}, i ∈ X : ˜ where q˜0,i ≥ 0, i ∈ X, ˜ i ∈X˜ q˜0,i = 1. B4 : (a) q˜ε,i → q˜0,i as ε → 0, for i ∈ X, The Markov property of the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) implies that the following relation takes place, for u, t ≥ 0, i ∈ X, Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, ξε ≤ t}

=



j ∈X

Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j}P j {ξε ≤ t}.

(4.65)

108

4 Generalisations of limit theorems for first-rare-event times

The conditions A4 and C7 imply that the following relation holds, for any u ≥ 0, ˜ j = 0, 1 which is a point of continuity of the distribution function F0,i, j (·), i  ∈ X, ˜ and i ∈ X, j = 0, 1, Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j} ⇒ F0,i, j (u)P0,i, j as ε → 0.

(4.66)

The relations (4.49) and (4.66) imply that, under the conditions A1 –C1 assumed to be satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), and the conditions A4 , B4 , and C7 , the following relation takes place, for any u ≥ 0 and v ≥ 0, which are points of continuity for distribution functions, respectively, F0,i, j (u), i  ∈ ˜ j = 0, 1, ˜ j = 0, 1 and G(v) = P{ξ0 ≤ v}, and i ∈ X, X, Pq˜ ε { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, ξε ≤ v}

=



q˜ε,i



Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j}P j {ξε ≤ v}

j ∈X

˜ i ∈X

=



q˜ε,i

˜ i ∈X



Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j}

j ∈X

 × (P j {ξε ≤ t} − P{ξ0 ≤ v}) + Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j}P{ξ0 ≤ v}  q˜0,i F˜0,i, j (u)P˜0,i, j G(v) as ε → 0. (4.67) → ˜ i ∈X

The relations (4.46) and (4.67) imply that, under the conditions A1 –C1 assumed to be satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), and the conditions A4 , B4 , and C7 , the condition D1 is satisfied (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) and the following relation takes place: Pq˜ ε { ξ˜ε ≤ ·} = Pq˜ ε { κ˜ε,1 + (1 − ρ˜ ε,1 )ξε ≤ ·}

⇒ Pq˜0 { κ˜0,1 + (1 − ρ˜0,1 )ξ0 ≤ ·} = Pq˜0 { ξ˜0 ≤ ·} as ε → 0,

(4.68)

where: (a) the limiting random vector ( κ˜0,1, ρ˜0,1 ) and the limiting random variable  ξ0 are independent, (b) Pq˜0 { κ˜0,1 ≤ u, ρ˜0,1 = j} = i ∈X˜ q˜0,i F˜0,i, j (u)P˜0,i, j, u ≥ 0, j = 0, 1, (c) P{ξ0 ≤ v} = G(v), v ≥ 0. Let us consider processes, ξˆε (t) =

[tk (ν ε +1)]−1

κε,n, t ≥ 0.

(4.69)

n=1

The Markov property of the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) implies that the following relation takes place, for u, uk , tk ≥ 0, k = 1, 2, . . . , n, n ≥ 1, i ∈ X,

4.1 Modifications of first-rare-event times

Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j,

=



109

[tk (ν ε +1)]−1

κε,n ≤ uk , k = 1, . . . , n}

n=1

Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j}

j ∈X

× P j { ξˆε (tk ) ≤ uk , k = 1, . . . , n}.

(4.70)

The relations (4.50), (4.65), and (4.70) imply that, under the conditions A1 –C1 assumed to be satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), and the conditions A4 , B4 , and C7 , the following relation takes place, for any u ≥ 0 and uk ≥ 0, k ≥ 1, which are points of continuity for distribution functions, ˜ j = 0, 1 and P{ξ0 (tk ) ≤ uk , k = 1, . . . , n}, n ≥ 1, and respectively, F0,i, j (u), i  ∈ X, ˜ j = 0, 1, i ∈ X, Pq˜ ε { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, ξε (tk ) ≤ uk , k = 1, . . . , n}

=



q˜ε,i

˜ i ∈X



Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j}

j ∈X

× P j { ξˆε (tk ) ≤ uk , k = 1, . . . , n}   q˜ε,i Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j, η˜ε,1 = j} = ˜ i ∈X



j ∈X

P j { ξˆε (tk ) ≤ uk , k = 1, . . . , n} − P{ξ0 (tk ) ≤ uk , k = 1, . . . , n})

 + Pi { κ˜ε,1 ≤ u, ρ˜ ε,1 = j}P{ξ0 (tk ) ≤ uk , k = 1, . . . , n}  q˜0,i F˜0,i, j (u)P˜0,i, j P{ξ0 (tk ) ≤ uk , k = 1, . . . , n} as ε → 0. →

(4.71)

˜ i ∈X

The relations (4.50) and (4.71) imply that, under the conditions A1 –C1 assumed to be satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), and the conditions A4 , B4 , and C7 , the condition D1 is satisfied (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) and the following relation takes place, κ˜ε,1 + (1 − ρ˜ ε,1 )

[t(νε +1)]−1

κε,n, t ≥ 0

n=1 d

−→ κ˜0,1 + (1 − ρ˜0,1 )ξ0 (t), t ≥ 0 as ε → 0,

(4.72)

where: (a) the limiting random vector ( κ˜0, ρ˜0 ) and the limiting stochastic process  ξ0 (t), t ≥ 0 are independent, (b) Pq˜0 { κ˜0,1 ≤ u, ρ˜0,1 = j} = i ∈X˜ q˜0,i F˜0,i, j (u)P˜0,i, j, u ≥ 0, j = 0, 1, and (c) the process ξ0 (t) = θ(tν0 ), t ≥ 0 is described in Theorem 2.1. The relation (4.59) implies that the processes ξˆε (t), t ≥ 0 are J-compact, as ε → 0, i.e., limc→0 limε→0 P{ΔJ (ξˆε (·), c, T) > δ} = 0, δ, T > 0 (see, relation A.3 for the formula defining the modulus of J-compactness). Since ΔJ ( κ˜ε,1 + (1 − ρ˜ ε,1 )ξˆε (·), c, T) ≤

110

4 Generalisations of limit theorems for first-rare-event times

ΔJ (ξˆε (·), c, T), T ≥ 0, the processes κ˜ε,1 + (1 − ρ˜ ε,1 )ξˆε (t), t ≥ 0 are also J-compact, as ε → 0. This fact and the relation (4.72) imply that κ˜ε,1 + (1 − ρ˜ ε,1 )

[t(νε +1)]−1

κε,n, t ≥ 0

n=1 J

−→ κ˜0,1 + (1 − ρ˜0,1 )ξ0 (t), t ≥ 0 as ε → 0.

(4.73)

Theorem 2.2 implies that, for any s, t > 0, and δ > 0 Pq˜ ε { ρ˜ ε,1 ·

[t]−1 

κε,n > δ} ≤ lim Pq˜ ε { ε→0

n=1

[su ε ]

κε,n > δ}

n=1

≤ P{θ 0 (s) > δ/2} → 0 as s → 0.

(4.74)

 Since the processes ρ˜ ε,1 · [t]−1 n=1 κε,n, t ≥ 0 are monotonic, the relation (4.74) implies, by Theorem A.3 and Remark A.1, that, under the conditions A1 –C1 assumed to be satisfied for the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ), and the conditions A4 , B4 , and C7 , ρ˜ ε,1 ·

[t]−1 

U

κε,n, t ≥ 0 −→ 0(t), t ≥ 0 as ε → 0.

(4.75)

n=1

Finally, the relations (4.47), (4.73), (4.75) and Theorem A.11 imply that ξ˜ε (t), t ≥ 0 = ρ˜ ε,1 ·

[t]−1 

κε,n + κ˜ε,1 + (1 − ρ˜ ε,1 )ξˆε (t), t ≥ 0

n=1 J

−→ κ˜0,1 + (1 − ρ˜0,1 )ξ0 (t), t ≥ 0 as ε → 0.

(4.76)

The following theorem complements Theorem 4.2. Theorem 4.3 Let (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) be, for every ε ∈ (0, 1], a Markov renewal pro˜ and X, and cess with transition period and the corresponding phase spaces X (ηε,n, κε,n, ρε,n ) be the corresponding shifted Markov renewal process. If the conditions A1 –C1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) and the conditions A4 , B4 , and C7 are satisfied, then: d

(i) The first-rare-event times ξ˜ε −→ ξ˜0 = κ˜0,1 + (1 − ρ˜0,1 )ξ0 as ε → 0, where the limiting random vector ( κ˜0,1, ρ˜0,1, ξ0 ) has the distribution given by the relation (4.68). J (ii) The first-rare-event processes ξ˜ε (t), t ≥ 0 −→ ξ˜0 (t) = κ˜0,1 +(1− ρ˜0,1 )ξ0 (t), t ≥ 0 as ε → 0, where the limiting process is described in the relation (4.72).

4.1 Modifications of first-rare-event times

111

4.1.4 First-Rare-Event Times for Markov Renewal Processes with Extending Phase Spaces 4.1.4.1 First-Rare-Event Times and the Phase Space Reduction for Markov Renewal Processes with Extending Phase Spaces. Let (ηˇε,n, κˇε,n, ρˇ ε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, with a phase space Zε = Xε × [0, ∞) × {0, 1}, where Xε = {1, 2, . . .} is a non-empty finite or countable set, and transition probabilities defined for i, j ∈ Xε, s, t ≥ 0, ı, j = 0, 1, P{ηˇε,1 = j, κˇε,1 ≤ t, ρˇ ε,1 = j/ηˇε,0 = i, ξˇε,0 = s, ρˇ ε,0 = ı}

= P{ηˇε,1 = j, κˇε,1 ≤ t, ρˇ ε,1 = j/ηˇε,0 = i} = Qˇ ε,i j (t, j).

(4.77)

Let us assume that the phase spaces Xε, ε ∈ (0, 1] have the following property, Xε ⊆ Xε, for 0 < ε  ≤ ε  ≤ 1.

(4.78)

From the relation (4.78) it follows that there exists a set limit, lim Xε = X0 = ∪ε ∈(0,1] Xε as ε → 0.

ε→0

(4.79)

It is useful to note that the case, where sets Xε ≡ X0 for ε ∈ (0, 1], is also admitted. ˜ = {1, . . . , m} ˜ ⊆ X0 . Let X = {1, . . . , m} ⊆ X ˜ be two finite sets such that X ˜ ⊆ Xε , for The relation (4.79) implies that there exists ε˜ ∈ (0, 1] such that X ε ∈ (0, ε]. ˜ Let νˇε,X be the first hitting time in the set X for the Markov chain ηˇε,n , i.e., νˇε,X = min(n ≥ 1 : ηˇε,n ∈ X).

(4.80)

In what follows, we assume that the following condition is satisfied: B5 : There exists εˇ ∈ (0, ε] ˜ such that Pi { νˇε,X < ∞} = 1, i ∈ Xε , for ε ∈ (0, ε]. ˇ We are interested in asymptotic relations for distributions of various functionals defined on the trajectories of Markov chains ηˇε,n , as ε → 0. Therefore, we can assume that ε ∈ (0, ε]. ˇ ˜ for ε ∈ (0, ε]. ˇ Let us also assume that the initial state ηˇε,0 ∈ X, Condition B5 allows us to recursively define, for each ε ∈ (0, ε], ˇ successive hitting times in the set X by the Markov chain ηˇε,n , νˇε,X,n = min(k > νˇε,X,n−1, ηˇε,k ∈ X), n = 1, 2, . . . , where νˇε,X,0 = 0.

(4.81)

Now, we can construct the new Markov renewal process (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), n = ˜ for the initial state η˜ε,0 , 0, 1, . . ., with transition period, the reduced phase spaces X

112

4 Generalisations of limit theorems for first-rare-event times

and the phase space X for the random variables η˜ε,n, n ≥ 1, using the following recurrent relations, ⎧ η˜ε,n = ηˇε,νˇ ε,X, n , n = 1, 2, . . . , η˜ε,0 = ηˇε,0, ⎪ ⎪ νˇ ε,X, n ⎨ ⎪ κ˜ε,n = k= κˇ , n = 1, 2, . . . , κ˜ε,0 = κˇε,0, νˇ +1 ε,k ⎪ ε,X, n−1 ⎪ ⎪ ρ˜ ε,n = 1 − νˇ ε,X, n (1 − ρˇ ε,k ), n = 1, 2, . . . , ρ˜ ε,0 = ρˇ ε,0 . k=νˇ ε,X, n−1 +1 ⎩

(4.82)

The Markov renewal process (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), n = 0, 1, . . . has the phase space ˜ × [0, ∞) × {0, 1} for the initial random vector (η˜ε,0, κ˜ε,0, ρ˜ ε,0 ) and the phase Z˜ = X space Z = X × [0, ∞) × {0, 1} for random vectors (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), n ≥ 1, the initial ˜ and transition distribution q˜ε = qε,i,s,ı = P{η˜ε,0 = i, κ˜ε,0 ≤ s, ρ˜ ε,0 = ı}, (i, s, ı) ∈ Z , probabilities of the following form, P{η˜ε,n = j, κ˜ε,n ≤ t, ρ˜ ε,n = j/η˜ε,n−1 = i, κ˜ε,n−1 = s, ρ˜ ε,n−1 = ı}

= P{η˜ε,n = j, κ˜ε,n ≤ t, ρ˜ ε,n = j/η˜ε,n−1 = i}

=

˜ j ∈ X, s, t ≥ 0, ı, j = 0, 1, n = 1, Q˜ ε,i j (t, j) for i ∈ X, Q ε,i j (t, j) for i, j ∈ X, s, t ≥ 0, ı, j = 0, 1, n > 1.

(4.83)

˜ j ∈ X, t ≥ 0, j = 0, 1, where, for i ∈ X, Q˜ ε,i j (t, j) = P{η˜ε,1 = j, κ˜ε,1 ≤ t, ρ˜ ε,1 = j/η˜ε,0 = i} = Pi {ηˇε,νˇ ε,X,1 = j,

νˇ ε,X,1

κˇε,k ≤ t, 1 −

k=1

νˇ ε,X,1

(1 − ρˇ ε,k ) = j},

(4.84)

k=1

and, for i, j ∈ X, t ≥ 0, j = 0, 1, Q ε,i j (t, j) = Q˜ ε,i j (t, j).

(4.85)

It is useful to note that the Markov renewal process (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) has no ˜ = X. Indeed, the transition probabilities transition period, if the phase spaces X ˜ Q ε,i j (t, j) in this case coincide with the transition probabilities Q ε,i j (t, j). The above embedding procedure allows us to formulate a version of Theorem 4.2 for the first-rare-event times ξ˜ε based on the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) and the corresponding shifted Markov renewal processes, (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . . .

(4.86)

It is also worth noting that Theorem 4.2 allows us to obtain the corresponding asymptotic relations for the first-rare-event times based on the original Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ), ξˇε =

νˇ ε  n=1

κˇε,n,

(4.87)

4.1 Modifications of first-rare-event times

113

where νˇε = min(n ≥ 1 : ρˇ ε,n = 1).

(4.88)

The point is that the following inequality holds for ε ∈ (0, ε], ˇ ξ˜ε,− ≤ ξˇε ≤ ξ˜ε,+, where ξ˜ε,− =

ν˜ ε −1 n=1

κ˜ε,n, ξ˜ε,+ = ξ˜ε =

(4.89) ν˜ ε 

κ˜ε,n .

(4.90)

n=1

The remarks made above and in Sect. 4.1.3.2 allow us to formulate a variant of Theorem 4.2 for the first-rare-event times ξˇε based on the Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ), the embedded Markov renewal process with transition period (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) constructed using the recurrent relations (4.82), the shifted Markov renewal processes (ηε,n, κε,n, ρε,n ) defined by the relation (4.87). Note that in the theorem formulated below, the conditions A1 , B1 , C1 , and D1 refer to the Markov renewal processes (ηε,n, κε,n, ρε,n ), the conditions A3 and C6 refer to the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n ), and the condition B5 refers to the Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ). Theorem 4.4 Let the conditions A1 , B1 , C1 and A3 , B5 , C6 be satisfied; then: (i) The condition D1 is necessary and sufficient for the fulfilment (for some or ˜ respectively, in the any initial distributions q˜ε concentrated of the phase space X, d statements of necessity and sufficiency) of the asymptotic relation, ξˇε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated at zero. (ii) The distribution function G(u) = P{ξ0∫ ≤ u}, u ≥ 0 of the limiting random ∞ 1 variable ξ0 has the Laplace transform φ(s) = 0 e−su G(du) = 1+A(s) , s ≥ 0, where A(s) is the cumulant of infinitely divisible distribution appearing in the condition D1 . It is also useful to mention the case, where the conditions A3 and C6 are satisfied ˜ = {1, . . . , m} for any finite set X ˜ ⊆ X0 . In this case, Theorem 4.4 is obviously true for the above-mentioned Markov renewal processes with initial distributions q˜ε ˜ of X0 . concentrated on any finite subset X 4.1.4.2 First-Rare-Event Times and the Phase Space Reduction for Markov Renewal Processes with a Finite Phase Space. Additional useful analysis of the conditions A1 –C1 and A3 , B5 , and C6 can be done for a model, in which X0 = {1, . . . , m0 } is a finite set and Xε = X0 , for ε ∈ (0, 1]. ˜ = X0 and, thus, ε˜ = 1. In this case, it is natural to choose the set X ˜ The case, where the set X = X, is covered by Theorem 2.1. Thus, we assume that ˜ the set X ⊂ X. Suppose now that the following ergodicity condition is satisfied: ˜ of the Markov chain ηˇε,n is B6 : There exists ε  ∈ (0, 1] such that the phase space X  one class of communicative states, for ε ∈ (0, ε ].

114

4 Generalisations of limit theorems for first-rare-event times

The condition B6 obviously implies that the condition B5 is satisfied. However, the condition B6 does not guarantee the fulfilment of the condition B1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )), i.e., that the Markov chains ηε,n are asymptotically uniformly ergodic. Let pˇε,i j be the transition probabilities for the Markov chain ηˇε,n , i.e., ˜ pˇε,i j = Pi {ηˇε,1 = j}, i, j ∈ X.

(4.91)

Suppose that the following condition of asymptotically uniform attainability of the set X is satisfied: ˜ j ∈ X, a chain of states ri j,0, ri j,1, . . . , ri j,ni j ∈ X ˜ B7 : There exists, for every i ∈ X, such that ri j,0 = i, ri j,ni j = j and limε→0 pˇε,ri j, k−1 ri j, k > 0, for k = 1, . . . , ni j . Let us show that the condition B7 implies that the condition B1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) is satisfied. Let pε,i j be the transition probabilities for the Markov chain ηε,n = η˜ε,n+1 , i.e., pε,i j = Pi {ηε,1 = j}, i, j ∈ X.

(4.92)

Let us rii+1,0, rii+1,1, . . . , rii+1,nii+1 be the sequences of states appearing in the condition B7 , for i = 1, . . . , m (here, we assume that m + 1 denotes the state 1 in the set X = {1, . . . , m}). Let also 0 = li,0 < li,1 < · · · < li,ni = nii+1 be all indices such that the states rii+1,li, k ∈ X, for k = 0, . . . , ni, i = 1, . . . , m. Then, for k = 0, . . . , ni − 1, i = 1, . . . , m, pε,rii+1, li, k rii+1, li, k+1 ≥

li, k+1 −1

pˇε,rii+1, l rii+1, l+1 .

(4.93)

l=li, k

The condition B7 implies that, for k = 0, . . . , ni − 1, i = 1, . . . , m, lim pε,rii+1, li, k rii+1, li, k+1 ≥ lim

ε→0

li, k+1 −1

ε→0 l=li, k

pˇε,rii+1, l rii+1, l+1 > 0.

(4.94)

Due to the relation (4.93), the role of ring of states appearing in the condition B1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) can be played by the sequence of states, 1 = r12,l1,0 , . . . , r12,l1, n1 = 2 = r23,l2,0 , . . . , r23,l2, n2 = 3  = . . . = m = rmm+1,lm,0 , . . . , rmm+1,l mn

Denote p(ε) = min

˜ j ∈X i ∈X,

ni j  k=1

m

= 1.

pˇε,ri j, k−1 ri j, k , N = max ni j . ˜ j ∈X i ∈X,

(4.95)

(4.96)

4.1 Modifications of first-rare-event times

115

The condition B7 implies that there exist ε  ∈ (0, ε ] and p ∈ (0, 1) such that p(ε) ≥ p, for ε ∈ (0, ε ]. ˜ n ≥ 1, It is easy to see that, for ε ∈ (0, ε ] and i ∈ X, Pi { νˇε,X > nN } ≤ (1 − p)n → 0 as n → ∞.

(4.97)

Let us also assume that the following condition is satisfied: A5 : (a) There exists εˇ  ∈ (0, 1] such that maxi ∈X˜ Pi { ρˇ ε,1 = 1} > 0, for ε ∈ (0, εˇ ], ˜ (b) Pi { ρˇ ε,1 = 1} → 0 as ε → 0, for i ∈ X. Let us prove that the conditions A5 and B7 imply that the condition A3 is satisfied. In this case, the condition A3 takes the form of the following relation, which ˜ should hold for i ∈ X, Pi { ρ˜ ε,1 = 1} → 0 as ε → 0. (4.98) Using the relations (4.82) and (4.97), we obtain the following relation, for ε ∈ ˜ n ≥ 1, (0, ε ] and i ∈ X, νˇ ε,X,1

Pi { ρ˜ ε,1 = 1} = Pi {

(1 − ρˇ ε,k ) = 0}

k=1

≤ Pi {νε,X,1 > nN } +

nN 

Pi {

r=1

≤ (1 − p)n + ≤ (1 − p)n +

nN  r 

Pi { ρˇ ε,k r=1 k=1 nN  r  

r  (1 − ρˇ ε,k ) = 0, νε,X,1 = r } k=1

= 1, νε,X,1 = r }

Pi { ρˇ ε,k = 1, ηˇε,k−1 = j}

˜ r=1 k=1 j ∈X

≤ (1 − p)n +

nN(nN + 1) m˜ max P j { ρˇ ε,1 = 1}. ˜ 2 j ∈X

(4.99)

˜ The relation (4.99) and the condition A5 (b) imply that, for i ∈ X, lim Pi { ρ˜ ε,1 = 1} ≤ (1 − p)n → 0 as n → ∞.

ε→0

(4.100)

The condition A1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) consists of two sub-conditions. The first sub-condition, A1 (a), requires there exists some ε˜  ∈ (0, 1] such that for ε ∈ (0, ε˜ ] and i ∈ X, max Pi {ρε,1 = 1} = Pi { ρ˜ ε,1 = 1} > 0. i ∈X

(4.101)

The second sub-condition, A1 (b), requires the fulfilment of the asymptotic relation (4.98), for i ∈ X.

116

4 Generalisations of limit theorems for first-rare-event times

˜ The latter sub-condition is a special case of the relation (4.98), since X ⊂ X. Let us prove that the conditions B6 and A5 imply that the relation (4.101) holds. Let j be some state such that P j { ρˇ ε,1 = 1} > 0, for ε ∈ (0, εˇ ]. First, suppose j ∈ X. In this case, the relation (4.102) implies that, for ε ∈ (0, εˇ ], P j {ρε,1 = 1} = P j {1 −

ν ˇ ε,X

(1 − ρˇ ε,k ) = 1}

k=1 ν ˇ ε,X

= Pj {

(1 − ρˇ ε,k ) = 0}

k=1 ν ˇ ε,X

= Pj {

ρˇ ε,k ≥ 1} ≥ P j { ρˇ ε,k = 1} > 0.

(4.102)

k=1

¯ =X ˜ \ X. Second, suppose j ∈ X In this case, condition B6 implies that there exists, for ε ∈ (0, ε ], a chain of states ¯ . . . , ri j,ni j −1 ∈ X, ¯ ri j,ni j = j such that i = ri j,0, ri j,1 ∈ X, ni j 

pε,ri j, k−1,ri j, k > 0.

(4.103)

k=1

˜ \ X, then, by the condition Indeed, let us choose an arbitrary state l ∈ X. If j ∈ X B6 , there exists, for ε ∈ (0, ε ], a chain of states l = rlj,0, rlj,1, . . . , rlj,n = j such lj that 1≤k=≤n pε,rl j, k−1,rl j, k > 0. Define, kl j = max(1 ≤ k ≤ nl j − 1 : rij,k ∈ X). lj Obviously, the state i = kl j ∈ X and the chain of states ri j,0 = rlj,kl j , ri j,1 = rlj,kl j +1, . . . , ri j,ni j = rij,nl j , where ni j = nl j − kl j , satisfies the relation (4.103). Denote by Ar,X,n the set of all chains l¯n = l0, ll, . . . , ln such that: (a) l0 = r, ln ∈ X, (b) ∪l¯n ∈Ar,X n {ηε,k = lk , k = 0, . . . , n} = {ηε,0 = r, νε,X = n}, for n ≥ 1 and ε ∈ (0, ε ]. The chain of states i = ri j,0, . . . , ri j,ni j , j = l0, l1, . . . , ln ∈ Ai,X,ni j +n , for any chain of states l¯ = l1, . . . , ln ∈ A j,X,n . The following relation takes place, for ε ∈ (0, ε ] and i ∈ X, ν ˇ ε,X

Pi {ρε,1 = 1} = Pi {

=



ρˇ ε,k ≥ 1}

k=1



n ≥1 l¯n ∈Ai,X, n

≥





n ≥1 l¯n ∈A j,X, n

Pi {ηε,k = lk , k = 0, . . . , n,

n  k=1

Pi {ηε,k = ri j,k , k = . . . , ni j ,

ρˇ ε,k ≥ 1}

4.1 Modifications of first-rare-event times

117

ηε,ni j +r = lr , r = 0, . . . , n, ρˇ ε,ni j +1 = 1}   × P j {ηε,ni j +r = lr , r = 0, . . . , n, ρˇ ε,ni j +1 = 1} n ≥1 l¯n ∈A j,X, n

= =

ni j  k=1 ni j 

pε,ri j, k−1,ri j, k P j {ηε,ni j = j, ρˇ ε,ni j +1 = 1} pε,ri j, k−1,ri j, k P j { ρˇ ε,1 = 1} > 0.

(4.104)

k=1

The relations (4.102) and (4.104) imply that the relation (4.101) holds. Thus, the conditions A5 and B7 imply that the condition A1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) is satisfied. Let us assume that the following condition is satisfied: ˜ C8 : Pi { κˇε,1 > δ} → 0 as ε → 0, for δ > 0 and i ∈ X. The conditions B7 and C8 imply that the condition C6 is satisfied. Indeed, using the relations (4.82) and (4.97), we obtain the following relation, for ¯ n ≥ 1, ε ∈ (0, ε ] and i ∈ X, νˇ ε,X,1

Pi { κ˜ε,1 > δ} = Pi {

κˇε,k > δ}

k=1

≤ Pi {νε,X,1 > nN } +

nN  r=1

≤ (1 − p)n +

nN  r   ˜ r=1 k=1 j ∈X

≤ (1 − p)n +

Pi {

r 

κˇε,k > δ, νˇε,X,1 = r }

k=1

Pi { κˇε,k >

δ , ηˇε,k−1 = j} r

nN(nN + 1) δ m˜ max P j { κˇε,1 > }. ˜ 2 nN j ∈X

(4.105)

˜ The relation (4.105) and the condition C8 imply that, for i ∈ X, lim Pi { κ˜ε,1 > δ} ≤ (1 − p)n → 0 as n → ∞.

ε→0

(4.106)

We summarise the results of the above analysis in the following lemma. Lemma 4.6 Let (ηˇε,n, κˇε,n, ρˇ ε,n ) be a Markov renewal process defined above, with a finite phase space of the first component X0 = {1, . . . , m0 }, and (η˜ε,n, κ˜ε,n, ρ˜ ε,n ) ˜ = X0 and X = be the Markov renewal process with the reduced phase spaces X ˜ defined by the recurrent relations (4.82) and (ηε,n, κε,n, ρε,n ) be the {1, . . . , m} ⊆ X shifted Markov renewal process defined by the relation (4.86). Then: (i) The conditions A5 , B6 , B7 , C8 assumed to be satisfied for the processes (ηˇε,n, κˇε,n, ρˇ ε,n ) imply that: (a) the condition B5 is satisfied for the processes

118

4 Generalisations of limit theorems for first-rare-event times

(ηˇε,n, κˇε,n, ρˇ ε,n ); (b) the conditions A5 , C6 are satisfied for the processes (η˜ε,n, κ˜ε,n , ρ˜ ε,n ); (c) the conditions A1 , B1 are satisfied for the processes (ηε,n, κε,n, ρε,n ). (ii) The version of Theorem 4.4 takes place, where the condition of this theorem is replaced by the conditions A5 , B6 , B7 , C8 assumed to be satisfied for the processes (ηˇε,n, κˇε,n, ρˇ ε,n ) and the condition C1 assumed to be satisfied for the processes (ηε,n, κε,n, ρε,n ). Section 4.1.2 contains the condition C5 , which entails the fulfilment of the condiα , i ∈ X, for some tion C1 . The condition C5 is formulated in terms of moments Ei κε,1 α > 1. α ,i ∈ X ˜ Below, we give upper bounds for moments of random variables Ei κ˜ε,1 αβ ˜ for some α, β > 1. Since distribution functions in terms of moments Ei κˇε,1, i ∈ X Pi { κ˜ε,1 ≤ ·} = Pi {κε,1 ≤ ·}, for i ∈ X, such upper bounds, in combination with the above condition C5 , give useful conditions (formulated in terms of processes (ηˇε,n, κˇε,n, ρˇ ε,n )) entailing the fulfilment of the condition C1 . Let us assume that the following condition is satisfied: ˜ C9 : There exist ε  ∈ (0, 1] and α, β > 1 such that, for ε ∈ (0, ε ] and i ∈ X, αβ

αβ

1

eˇε,i,α,β = (Ei κˇε,1 ∨ max E j κˇε,1 ) β < ∞. ˜ j ∈X\X

Lemma 4.7 Let the conditions B7 and C9 be satisfied. Then, the following inequality takes place for ε ∈ (0, ε ] (where ε  = ε  ∧ε  and ε  is defined in relation (4.97)), β ˜ , and i ∈ X, γ = 1−β α Ei κ˜ε,1 ≤ eˇε,i,α,β · a p, N,α,γ, (4.107) where 

a p, N,α,γ =

1 n−1   nα−1 1 + (n − 1)(m˜ − m) (1 − p) γ [ N ] < ∞.

(4.108)

n ≥1

Proof The condition C9 and the relation (4.97) imply that the following relation ˜ holds, for ε ∈ (0, ε  ∧ ε ] and i ∈ X, α Ei κ˜ε,1

νˇ

= Ei

ε,X 

κˇε,k

α

k=1

= Ei

n   n ≥1

=



Ei



n ≥1

≤

 n ≥1

k=1 n 

κˇε,k κˇε,k

k=1



Ei nα−1

α

α

n  k=1

I(νˇε,X = n)

I(νˇε,X = n)

α κˇε,k I(νˇε,X = n)



4.2 First-rare-event times and hitting times

=



119

 α nα−1 Ei κˇε,1 · I(νˇε,X = n)

n≥1 n 

+



α Ei κˇε,k I(ηε,k−1 = j) · I(νˇε,X = n)



˜ k=2 j ∈X\X

≤



1  αβ 1 nα−1 (Ei κˇε,1 ) β Pi { νˇε,X = n} γ

n≥1 n 

+



αβ

1

1

(Ei κˇε,k I(ηε,k−1 = j)) β Pi { νˇε,X = n} γ



˜ k=2 j ∈X\X

=



 αβ 1 nα−1 (Ei κˇε,1 ) β

n≥1 n 

+



1  1 αβ (Ei { κˇε,k /ηε,k−1 = j}Pi {ηε,k−1 = j}) β Pi { νˇε,X = n} γ

˜ k=2 j ∈X\X

≤ eˇε,i,α,β



1   nα−1 1 + (n − 1)(m˜ − m) Pi { νˇε,X = n} γ

n ≥1

≤ eˇε,i,α,β



  n − 1 γ1 ]} nα−1 1 + (n − 1)(m˜ − m) Pi { νˇε,X > N[ N n ≥1

≤ eˇε,i,α,β · a p, N,α,γ . The proof is complete.

(4.109) 

4.2 First-Rare-Event Times and Hitting Times In this section, we define standard and directed hitting times for semi-Markov processes, clarify their relationship with the first-rare-event times, and present necessary and sufficient conditions of convergence in distribution for standard and directed hitting times defined on asymptotically uniformly ergodic semi-Markov processes.

4.2.1 Necessary and Sufficient Conditions of Convergence in Distribution for Standard Hitting Times 4.2.1.1 Limit Theorems for Standard Hitting Times – I. Let (ηε,n, κε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, with a phase space X × [0, ∞), where X = {1, 2, . . . , m} is a finite set, and transition probabilities defined, for (i, s), ( j, t) ∈ X × [0, ∞),

120

4 Generalisations of limit theorems for first-rare-event times

Q ε,i j (t) = P{ηε,1 = j, κε,1 ≤ t/ηε,0 = i, κε,0 = s} = P{ηε,1 = j, κε,1 ≤ t/ηε,0 = i}.

(4.110)

Let us now introduce the Markov renewal process (ηε,n, κε,n, ρε,n ), n = 0, 1, . . ., with the additional third component ρε,n that has the following form, ρε,n = I(ηε,n ∈ D), for n = 1, 2, . . . , ρε,0 = 0,

(4.111)

where D is a subset of the phase space X, such that ∅ ⊂ D ⊂ X. In this case, the transition probabilities Q ε,i j (u, j) for the above Markov renewal process take the following form, for u ≥ 0, i, j ∈ X, j = 0, 1, Q ε,i j (t, j) = P{ηε,1 = j, κε,1 ≤ t, ρε,1 = j/ηε,0 = i}

¯ or j = 1, j ∈ D, Q ε,i j (u) for j = 0, j ∈ D = ¯ 0 for j = 0, j ∈ D or j = 1, j ∈ D.

(4.112)

In this model, the first-rare-event time, ξε = τε,D =

ν ε,D

κε,n,

(4.113)

n=1

where νε,D = min(n ≥ 1 : ηε,n ∈ D).

(4.114)

Let ηε (t), t ≥ 0 be the semi-Markov process, defined by the following relation: ηε (t) = ηε,n for ζε,n−1 ≤ t < ζε,n, n = 1, 2, . . . , where ζε,n =

n 

κε,k , n = 0, 1, . . . .

(4.115)

(4.116)

k=1

The relation (4.113) means that, in this case, the first-rare-event time ξε = τε,D is the first hitting time in the domain D for the semi-Markov process ηε (t). In this case, the one-step probabilities of occurrence of rare event take the following form:  pε,i j , i ∈ X, (4.117) pε,i = Pi {ηε,1 ∈ D} = j ∈D

where pε,i j = Pi {ηε,1 = j}, i, j ∈ X.

(4.118)

The condition A1 requires that 0 < maxi ∈X pε,i → 0 as ε → 0. It is useful to note that, in this case, the distribution functions G ε,D,i (u) = ¯ are determined by the transition probabilities Pi {τε,D ≤ u}, u ≥ 0, i ∈ D ¯ ¯ Q ε,kr (v), v ≥ 0, k ∈ D, r ∈ X. Thus, the distribution functions G ε,D,i (·), i ∈ D do not change if one changes probabilities Q ε,kr (v), v ≥ 0, k ∈ D, r ∈ X.

4.2 First-rare-event times and hitting times

121

This allows us to replace the condition A1 with a weaker condition: A6 : (a) There exists εˆ  ∈ (0, 1] such that maxi ∈D¯ pε,i > 0, for ε ∈ (0, εˆ ], (b) pε,i → 0 ¯ as ε → 0, for i ∈ D.

X,

¯ for ε ∈ (0, 1]. First, we analyse the case, where the initial state ηε,0 ∈ D, Let us introduce the following conditional distribution functions, for u ≥ 0, i, j ∈ Fε,i j (u) = P{κε,1 ≤ u/ηε,0 = i, ηε,1 = j}.

(4.119)

Obviously, for u ≥ 0, i, j ∈ X,  Fε,i j (u) =

Q ε, i j (u) pε, i j

if pε,i j > 0, I(u ≥ 0) if pε,i j = 0.

(4.120)

Here, we take into account that arbitrary distribution functions Fε,i j (u) can play the role of the above conditional distribution functions for i, j ∈ X such that pε,i j = 0, and choose, in these cases, the simplest distribution function I(u ≥ 0), u ≥ 0. Let us also introduce the following distribution functions, for u ≥ 0, i ∈ X,  Fε,i (u) = P{κε,1 ≤ u/ηε,0 = i} = Fε,i j (u)pε,i j . (4.121) j ∈X

In what follows we denote by F1 (u) ∗ · · · ∗ Fn (u) the convolution of n proper or improper distribution functions F1 (u), . . . , Fn (u). The following representation takes place for the distribution of the first hitting ¯ time τε,D , for u ≥ 0, i0 ∈ D, Pi0 {τε,D ≤ u} =

= =

∞ 

∞ 

Pi {τε,D ≤ u, νε,D = n}

n=1



Fε,i0 i1 (u) ∗ · · · ∗ Fε,in−1 in (u) ∗

¯ n ∈D n=1 i0,...,in−1 ∈D,i ∞  

n 

pε,ik−1 ik

k=1

Fε,i0 i1 (u) ∗ · · · ∗ Fε,in−2 in−1 (u)

¯ n=1 i0,...,in−1 ∈D

∗ F˜ε,in−1 (u) =

∞ 



n−1 

(1 − pε,ik−1 )pε,in−1

k=1

p˜ε,ik−1 ik

k=1

Fε,i0 i1 (u)(1 − pε,i0 ) ∗ · · · ∗ Fε,in−2 in−1 (u)(1 − pε,in−2 )

¯ n=1 i0,...,in−1 ∈D

∗ F˜ε,in−1 (u)pε,in−1

n  k=1

where

n 

p˜ε,ik−1 ik ,

(4.122)

122

4 Generalisations of limit theorems for first-rare-event times

p˜ε,i j =

pε, i j 1−pε, i

¯ such that pε,i < 1, for i ∈ D ¯ such that pε,i = 1, I( j = i) for i ∈ D

and F˜ε,i (u) =



j ∈D

Fε, i j (u)pε, i j pε, i

I(u ≥ 0)

¯ if pε,i > 0, for u ≥ 0, i ∈ D, ¯ if pε,i = 0. for u ≥ 0, i ∈ D,

(4.123)

(4.124)

ˆ × Let (ηˆε,n, κˆε,n, ρˆ ε,n ) be a Markov renewal process, with phase space Zˆ = X ˆ [0, ∞) × {0, 1} and transition probabilities Qˆ ε,i j (u, j), u ≥ 0, j = 0, 1, i, j ∈ X. Let also ξˆε be the corresponding first-rare-event time, i.e., ξˆε =

νˆ ε 

κˆε,n,

(4.125)

n=1

where νˆε = max(n ≥ 1 : ρˆ ε,n ). ˆ We define, for u ≥ 0, j = 0, 1, i, j ∈ X,

Pi { κˆε,1 ≤ u, ρˆ ε,1 = j/ηˆε,1 = j} if pˆε,i j > 0, ˆ Fε,i j, j (u) = I(u ≥ 0) if pˆε,i j = 0,

(4.126)

(4.127)

ˆ are the transition probabilities of the Markov chain ηˆε,n , where pˆε,i j , i, j ∈ X pˆε,i j = Pi {ηˆε,1 = j} = Qˆ ε,i j (∞, 0) + Qˆ ε,i j (∞, 1), and Fˆε,i,1 (u) = Pi { κˆε,1 ≤ u , ρˆ ε,1 = 1} =



Fˆε,i j,1 (u) pˆε,i j .

(4.128) (4.129)

ˆ j ∈X

Using the Markov property of the Markov renewal processes (ηˆε,n, κˆε,n, ρˆ ε,n ), we obtain the following useful representation for the distribution of the first-rare-event ˆ time ξˆε , for u ≥ 0, i0 ∈ X, Pi0 { ξˆε ≤ u} = Pi0 {

= =

∞  n=1 ∞ 

Pi0 {

n 

νˆ ε 

κˆε,n ≤ u}

n=1

κˆε,k ≤ u, νˆε = n}

k=1



ˆ n=1 i0,...,in ∈X

Pi0 {

n 

κˆε,k ≤ u, ρˆ ε,k = 0,

k=1

ηˆεk = ik , k = 1, . . . , n − 1, ρˆ ε,n = 1, ηˆε,n = in }

4.2 First-rare-event times and hitting times

=

∞ 



Fˆε,i0 i1,0 (u) ∗ · · · ∗ Fˆε,in−2 in−1,0 (u)

ˆ n=1 i1,...,in ∈X

∗ Fˆε,in−1 in,1 (u) · =

∞ 

123



n 

pˆε,ik−1 ik

k=1

Fˆε,i0 i1,0 (u) ∗ · · · ∗ Fˆε,in−2 in−1,0 (u)

ˆ n=1 i1,...,in−1 ∈X

∗ Fˆε,in−1,1 (u) ·

n−1 

pˆε,ik−1 ik .

(4.130)

k=1

Comparison of the relations (4.122) and (4.130) allows us to conclude that, for ¯ i ∈ D, Pi {τε,D ≤ ·} = Pi { ξˆε ≤ ·}, (4.131) if the Markov renewal process (ηˆε,n, κˆε,n, ρˆ ε,n ) is constructed in such a way that the following relations hold: ⎧ ˆ = D, ¯ X ⎪ ⎪ ⎪ ⎨ pˆε,i j = p˜ε,i j , i, j ∈ D, ⎪ ¯ ¯ ˆ Fε,i j,0 (·) = Fε,i j (·)(1 − pε,i ), i, i ∈ D, ⎪ ⎪ ⎪ ⎪ Fˆε,i,1 (·) = F˜ε,i (·)pε,i, i ∈ D. ¯ ⎩

(4.132)

The above relations hold for the Markov renewal process (ηˆε,n, κˆε,n, ρˆ ε,n ), with ¯ × [0, ∞) × {0, 1} and the transition probabilities given by the following phase space D relation:

¯ F (u)(1 − pε,i ) p˜ε,i j for u ≥ 0, j = 0, i, j ∈ D, ˆ (4.133) Q ε,i j (·, j) = ˜ε,i j ¯ Fε,i (u)pε,i p˜ε,i j for u ≥ 0, j = 1, i, j ∈ D. d We are going to analyse the asymptotic behaviour of the hitting times τε,D = ξˆε , applying Theorem 2.1 to the first-rare-event times ξˆε . The condition A1 takes, in this case, the form of the condition A6 . The condition B1 takes in this case, the following form:

B8 : There exists a ring chain of states i0, i1, . . . , i N = i0 , which contains all states ¯ and such that lim from set D ε→0 p˜ε,ik−1 ik > 0, for k = 1, . . . , N. It is useful to note that, due to the condition A6 , the following relation holds, for ¯ i, j ∈ D, (4.134) p˜ε,i j − pε,i j → 0 as ε → 0. Therefore, the condition B8 can also be reformulated in the following equivalent form: B◦8 : There exists a ring chain of states i0, i1, . . . , i N = i0 , which contains all states ¯ and such that lim from set D ε→0 pε,ik−1 ik > 0, for k = 1, . . . , N.

124

4 Generalisations of limit theorems for first-rare-event times

The condition C1 takes, in this case, the following form: ¯ C10 : 1 − F˜ε,i (δ) → 0 as ε → 0, for δ > 0 and i ∈ D. The condition B7 implies that there exists εˆ ∈ (0, 1] such that the Markov chain ˆ ηˆε,n is ergodic, for every ε ∈ (0, ε]. ¯ be its stationary distribution. Let πˆ ε = πˆ ε,i, i ∈ D ¯ In this case, for i ∈ D, Fˆε,i (·) = Pi { κˆε,i ≤ ·} = Fε,i (·)(1 − pε,i ) + F˜ε,i (·)pε,i .

(4.135)

Let θˆε,n, n = 1, 2, . . . be a sequence of i.i.d. random variables with the distribution function Fˆε (·), where  πˆ ε,i Fˆε,i (·). (4.136) Fˆε (·) = ¯ i ∈D

The function uε takes in this case the following form,  uε = p−1 πˆ ε,i pε,i . ε , where pε =

(4.137)

¯ i ∈D

The condition D1 takes, in this case, the following form:  ε] ˆ d ˆ ˆ D9 : [u n=1 θ ε,n −→ θ 0 as ε → 0, where θ 0 is a non-negative random variable with distribution not concentrated at zero. In this case, θˆ0 is infinitely divisible random variable and its Laplace transform ˆ has the form Ee−s θ0 = e−A(s), s ≥ 0, where A(s) is the cumulant of corresponding infinitely divisible distribution. The following theorem, which presents analogues of the statements (i) and (ii) of Theorem 2.1, takes place. Theorem 4.5 Let the conditions A6 , B8 , and C10 be satisfied. Then: (i) The condition D9 is necessary and sufficient for the fulfilment (for some or ¯ any initial distributions q¯ε of random variable ηε,0 concentrated on domain D, respectively, in the statements of necessity and sufficiency) of the asymptotic relation d

τε,D −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated at zero. (ii) The distribution function G(u) = P{ξ0∫ ≤ u}, u ≥ 0 of the limiting random ∞ 1 , s ≥ 0, where variable ξ0 has the Laplace transform φ(s) = 0 e−su G(du) = 1+A(s) A(s) is a cumulant of infinitely divisible distribution appearing in the condition D9 . It is worth to note that Theorem 4.5 can also be reformulated in terms of first-rareevent times ξˆε based on the defined above Markov renewal processes (ηˆε,n, κˆε,n, ρˆ ε,n ), ¯ and transition probabilities given by the relation (4.133). with the phase space D Moreover, an analogue of the statement (iii) of Theorem 2.1 can also be formulated  νˆ ε ] κˆε,n, t ≥ 0. for the first-rare-event processes ξˆε (t) = [tn=1

4.2 First-rare-event times and hitting times

125

However, it should be borne in mind that the Markov renewal process (ηˆε,n, κˆε,n , ρˆ ε,n ), differs from the original Markov renewal processes (ηε,n, κε,n, ρε,n ) used in the relation (4.113) to define the hitting time τε,D . 4.2.1.2 Limit Theorems for Standard Hitting Times – II. Theorem 4.5 describes the asymptotics in distribution for the hitting times τε,D for the case, where their ¯ = 1, for ¯ i.e., P{ηε,0 ∈ D} initial distributions are concentrated in the domain D, ε ∈ (0, 1]. Suppose now that the initial state ηε,0 ∈ D, for ε ∈ (0, 1]. In this case, we can use the following representation, which takes place for ε ∈ (0, 1],  d τε,i,D = κε,i,1 I(ηε,i,1 = j) j ∈D

+



(κε,i,1 + τε, j,D )I(ηε,i,1 = j)

¯ j ∈D



= κε,i,1 +

I(ηε,i,1 = j)τε, j,D, for i ∈ D,

(4.138)

¯ j ∈D

where: (a) τε,i,D is a non-negative random variable, which has the distribution function P{τε, j,D ≤ u} = P j {τε,D ≤ u}, u ≥ 0, for i ∈ X; (b) (κε,i,1, ηε,i,1 ) is a random vector, which takes values in the set [0, ∞) × X and has the distribution P{κε,i,1 ≤ u, ηε,i,1 = j} = Q ε,i j (u), u ≥ 0, j ∈ X, for i ∈ X; (c) the random vector (κε,i,1, ηε,i,1 ) and the random variable τε, j,D are independent for every i, j ∈ X. Let us assume that the following condition is satisfied:  C11 : (a) pε,i j → p0,i j as ε → 0, for i ∈ D, j ∈ X, where p0,i j ≥ 0, j ∈X0 p0,i j = 1, i ∈ D, j ∈ X, (b) Fε,i j (·) ⇒ F0,i j (·) as ε → 0, for i ∈ D, j ∈ X, where F0,i j (·), i ∈ D, j ∈ X are some distribution functions concentrated on the interval [0, ∞). Let us introduce Laplace transforms, for ε ∈ [0, 1] and i ∈ X, ∫ ∞ φε,i j (s) = e−su Fε,i j (du), s ≥ 0

(4.139)

0

and, for ε ∈ (0, 1] and i ∈ X, ∫ Φε,i,D (s) =

0

∞

e−su Pi {τε,D ∈ du}, s ≥ 0.

(4.140)

The relation (4.138) can be rewritten in the following equivalent form, for ε ∈ (0, 1],  φε,i j (s)pε,i j Φε,i,D (s) = j ∈D

126

4 Generalisations of limit theorems for first-rare-event times

+



φε,i j (s)Φε, j,D (s)pε,i j for s ≥ 0, i ∈ D.

(4.141)

¯ j ∈D

Let us assume that the conditions of Theorem 4.5 and the condition C11 are satisfied. In this case, the following relation takes place, for i ∈ D,   Φε,i,D (s) = φε,i j (s)pε,i j + φε,i j (s)Φε, j,D (s)pε,i j j ∈D

→



¯ j ∈D

φ0,i j (s)p0,i j + φ(s)

j ∈D



φ0,i j (s)p0,i j

¯ j ∈D

= Φ0,i,D (s) for s ≥ 0.

(4.142)

The relation (4.142) is equivalent to the following relation, for i ∈ D, G ε,i,D (·) = Pi {τε,D < ·} ⇒ G0,i,D (·) as ε → 0,

(4.143)

where G0,i,D (·) is, for i ∈ D, the distribution function with the Laplace transform Φ0,i,D (s), s ≥ 0 given by the relation (4.142).

4.2.2 Necessary and Sufficient Conditions of Convergence in Distribution for Directed Hitting Times 4.2.2.1 Directed Hitting Times. Let (ηε,n, κε,n, ρε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], the Markov renewal process, introduced in Sect. 4.2.1.1. Let us consider the model, where the third component ρε,n of this process has the following form, ¯ ηε,n ∈ D), for n = 1, 2, . . . , ρε,0 = 0, ρε,n = I(ηε,n−1 ∈ D,

(4.144)

where D is some non-empty subset of the phase space X. This form differs from the form of the indicators ρε,n specified in the relation (4.111). The random variables ρε,n , defined in the relation (4.144), indicate directed ¯ hits of the Markov chain ηε,n in the domain D from the domain D. Accordingly, the directed hitting time is defined as follows: τε,→D =

ν ε,→D

κε,n,

(4.145)

n=1

where ¯ ηε,n ∈ D). νε,→D = max(n ≥ 1 : ηε,n−1 ∈ D,

(4.146)

¯ the standard and directed hitting times coincide. Thus, If the initial state ηε,0 ∈ D, the following relation takes place,

4.2 First-rare-event times and hitting times

127 d

¯ τε, j,→D = τε, j,D, j ∈ D,

(4.147)

where the random variables τε, j,→D and τε, j,D have the distribution function

¯ P{τε, j,→D ≤ u} = P{τε, j,D ≤ u} = P j {τε,D ≤ u}, u ≥ 0, for j ∈ D.

If the initial state ηε,0 ∈ D, the standard and directed hitting times do not coincide. In this case, the following relation takes place,  d I(ηε,i, D¯ = j)(τε,i, D¯ + τε, j,D ), i ∈ D, (4.148) τε,i,→D = ¯ j ∈D

¯ and has where: (a) the random vector (τε,i, D¯ , ηε,i, D¯ ) takes values in the set [0, ∞) × D ¯ the distribution P{τε, D¯ ≤ u, ηε,i, D¯ = j} = Pi {τε, D¯ ≤ u, ηε,νε, D¯ = j}, u ≥ 0, j ∈ D, for i ∈ D; (b) the random vector (τε,i, D¯ , ηε,i, D¯ ) and the random variable τε, j,D are ¯ independent, for i ∈ D, j ∈ D. The only case, where ηε,0 ∈ D, should be analysed. Two sub-cases should be considered. ¯ and D ¯ →D In the first sub-case, the probabilities of one-step transitions D → D tend to zero as ε → 0, i.e., the following condition is satisfied: ¯ and i ∈ D, ¯ j ∈ D. A7 : pε,i j → 0 as ε → 0, for i ∈ D, j ∈ D ¯ → D tend to In the second sub-case, the probabilities of one-step transitions D zero as ε → 0, i.e., the following condition is satisfied: ¯ j ∈ D. A8 : pε,i j → 0 as ε → 0, for i ∈ D, 4.2.2.2 Hitting Times for the Case of Asymptotically Small Probabilities ¯ Let us consider the case, where the of Transitions Between Domains D and D. condition A7 is satisfied. In this case, Theorem 4.5 can be applied, both to the hitting times τε,D , for the ¯ and to the hitting times τε, D¯ , for the case where case where the initial state ηε,0 ∈ D, the initial state ηε,0 ∈ D.  , and D  , respectively, the conditions A , B , Let us denote by A6 , B8 , C10 6 8 9 ¯ and the transition probabilities C10 , and D9 , for the case, where the domain D ¯ j ∈ X are replaced by the domain D and the transition Q ε,i j (u), u ≥ 0, i ∈ D, probabilities Q ε,i j (u), u ≥ 0, i ∈ D, j ∈ X. Analogously, we denote by ξ0 and φ (s) = (1+ A(s))−1, s ≥ 0 the limiting random d

variable and its Laplace transform, in the asymptotic relation τε, D¯ −→ ξ0 as ε → 0,  , and D  are satisfied which holds, under the assumption that conditions A8 , B8 , C10 9  (for some or any initial distributions q¯ε of the random variables ηε,0 concentrated on the domain D, respectively, in the statements of necessity and sufficiency), according to the corresponding variant of Theorem 4.5. The relation (4.148), rewritten in terms of Laplace transforms, takes the following form, for any initial distribution q¯ε of the random variable ηε,0 concentrated on the domain D,  Eq¯ ε e−sτε,→D = Eq¯ ε e−sτε, D¯ I(ηε,νε, D¯ = j)E j e−sτε, D¯ , s ≥ 0. (4.149) ¯ j ∈D

128

4 Generalisations of limit theorems for first-rare-event times

 , D  are satisfied, then (a) for If the conditions A6 , B8 , C10 , D9 and A6 , B8 , C10 9 ¯ any initial distributions q¯ε of the random variables ηε,0 concentrated on domain D, d

the hitting times τε,D −→ ξ0 as ε → 0 and (b) for any initial distributions q¯ε of d

the random variables ηε,0 concentrated on domain D, the hitting times τε, D¯ −→ ξ0 as ε → 0, where (c) ξ0 and ξ0 are non-negative random variables with the Laplace transforms, respectively, φ(s) = (1 + A(s))−1, s ≥ 0 and φ (s) = (1 + A(s))−1, s ≥ 0. Therefore, the following relation holds  Eq¯ ε e−sτε,→D = Eq¯ ε e−sτε, D¯ I(ηε,νε, D¯ = j)E j e−sτε, D¯ ¯ j ∈D

=



Eq¯ ε e−sτε, D¯ I(ηε,νε, D¯ = j)(E j e−sτε, D¯ − φ(s))

¯ j ∈D

+ Eq¯ ε e−sτε, D¯ φ(s)

→ φ (s)φ(s) as ε → 0, for s ≥ 0.

(4.150)

The relation (4.150) is equivalent to the following relation (which holds for some or any initial distributions q¯ε of the random variables ηε,0 concentrated on domain D, respectively, in the statements of necessity and sufficiency), Pq¯ ε {τε,→D < ·} ⇒ G0,→D (·) as ε → 0,

(4.151)

where G0,→D (·) is, for i ∈ D, the distribution function with the Laplace transform φ→ (s) = φ(s)φ (s) = (1 + A(s))−1 (1 + A(s))−1, s ≥ 0 given by the relation (4.150).  are satisfied, and, (a) for some If the conditions A6 , B8 , C10 and A6 , B8 , C10 initial distributions q¯ε of the random variables ηε,0 concentrated on the domain d ¯ the hitting times τε,D −→ ξ0 as ε → 0 and, (b) for some initial distributions D,  q¯ε of the random variables ηε,0 concentrated on the domain D, the hitting times d

τε, D¯ −→ ξ0 as ε → 0, where (c) ξ0 and ξ0 are some non-negative random variables with distribution functions not concentrated at zero, then the conditions D9 and D9 are satisfied, and the random variables ξ0 and ξ0 have the Laplace transforms, respectively, φ(s) = (1 + A(s))−1, s ≥ 0 and φ (s) = (1 + A(s))−1, s ≥ 0 appearing in the conditions D9 and D9 , respectively. Therefore, the relations (4.150) and (4.151) take place. 4.2.2.3 Hitting Times for the Case of Asymptotically Small Probabilities of ¯ to Domain D. Let us consider the case, where the Transitions from Domain D condition A8 is satisfied. Let us first assume that the following condition is satisfied: B9 : For every i ∈ D, there exists a chain of states i = j0, j1, . . . , j Ni,D −1 ∈ D, j Ni,D ∈ ¯ such that lim D ε→0 pε, jk−1 jk > 0, for k = 1, . . . , Ni,D . The condition B9 implies that there exists εD ∈ (0, 1] such that pi,D = Ni,D p > 0, for ε ∈ (0, εD ]. k=1 ε, jk−1 jk Denote ND = maxi ∈D Ni,D < ∞ and pD = mini ∈D pi,D > 0.

4.2 First-rare-event times and hitting times

129

It is easy to see that, for ε ∈ (0, εD ] and i ∈ D, n ≥ 1, Pi {νε,D > nND } ≤ (1 − pD )n .

(4.152)

It should be noted that the conditions of Theorem 4.5 (applied to the hitting times ¯ τε,D ) imply that Pi {κε,1 > δ} → 0, as ε → 0, for δ > 0 and i ∈ D. Therefore, for the states i ∈ D, it is natural to make a similar assumption, i.e., assume that the following condition is satisfied: C12 : Pi {κε,1 > δ} → 0, as ε → 0, for δ > 0 and i ∈ D. The condition B10 implies that, for ε ∈ (0, εD ] and δ > 0, νε, D¯

Pi {τε, D¯ > δ} = Pi {



κε,k > δ}

k=1

≤ Pi {νε, D¯ } > nND } +

nN D r=1

≤ (1 − pD )n +

nN r  D 

Pi {

r 

κε,k > δ, νε, D¯ = r }

k=1

Pi {κε,k >

r=1 k=1 j ∈D

δ , ηε,k−1 = j} r

nND (nND + 1) δ mD max P j {κε,1 > }, ≤ (1 − pD )n + j ∈D 2 nND

(4.153)

where mD is the number of states in the domain D. The conditions B9 and C12 imply that, for any initial distributions q¯ε of the random variables ηε,0 concentrated on the domain D, and δ > 0, lim Pq¯ ε {τε, D¯ > δ} ≤ (1 − pD )n → 0 as n → ∞.

ε→0

(4.154)

Therefore, the conditions B9 and C12 imply that, for any initial distributions q¯ε of the random variables ηε,0 concentrated on the domain D, P

τε, D¯ −→ 0 as ε → 0.

(4.155)

¯ The above relation implies that the following relation holds, for j ∈ D, Eq¯ ε (1 − e−sτε, D¯ )I(ηε,νε, D¯ = j)E j e−sτε, D¯

≤ Eq¯ ε (1 − e−sτε, D¯ )

= 1 − Eq¯ ε e−sτε, D¯ → 0 as ε → 0, for s ≥ 0.

(4.156)

If the conditions A6 , B8 , C10 , D9 and B9 , C12 are satisfied, then: (a) for any initial ¯ the distributions q¯ε of the random variables ηε,0 concentrated on the domain D, 1 −sτ ε,D → φ(s) = 1+A(s) as ε → 0, for s ≥ 0, and (b) the Laplace transforms Eq¯ ε e asymptotic relation (4.156) holds. Let us denote

130

4 Generalisations of limit theorems for first-rare-event times

¯ q¯ε = qε, j = Pq¯ ε {ηε,νε, D¯ = j}, j ∈ D .

(4.157)

The above remarks and the relations (4.148) and (4.156) imply that, for any initial distributions q¯ε of the random variables ηε,0 concentrated on the domain D,  Eq¯ ε e−sτε,→D = Eq¯ ε e−sτε, D¯ I(ηε,νε, D¯ = j)E j e−sτε, D¯ ¯ j ∈D

=−



Eq¯ ε (1 − e−sτε, D¯ )I(ηε,νε, D¯ = j)E j e−sτε, D¯

¯ j ∈D

+



Eq¯ ε I(ηε,νε, D¯ = j)E j e−sτε, D¯

¯ j ∈D

=−



Eq¯ ε (1 − e−sτε, D¯ )I(ηε,νε, D¯ = j)E j e−sτε, D¯

¯ j ∈D

+ Eq¯ ε e−sτε, D¯ → φ(s) as ε → 0, for s ≥ 0,

(4.158)

or, equivalently, d

τε,→D −→ ξ0 as ε → 0,

(4.159)

where ξ0 is a non-negative random variable such that Ee−sξ0 = φ(s) = (1 + A(s))−1, s ≥ 0. Let us now assume that: (a) the conditions A6 , B8 , C10 , and B9 , C12 are satisfied, (b) for some initial distributions q¯ε of the random variable ηε,0 concentrated on the d

domain D, the directed hitting times τε,→D −→ ξ→,0 as ε → 0, where ξ→,0 is a non-negative random variable with distribution function not concentrated at zero. The relations (4.148) and (4.156) imply that, for distributions q¯ε given by the relation (4.157), there exists a limit   lim Eq¯ ε e−sτε, D¯ = lim − Eq¯ ε (1 − e−sτε, D¯ )I(ηε,νε, D¯ = j)E j e−sτε, D¯ ε→0

ε→0

¯ j ∈D

 + Eq¯ ε e−sτε, D¯   Eq¯ ε (1 − e−sτε, D¯ )I(ηε,νε, D¯ = j)E j e−sτε, D¯ = lim − ε→0

+



¯ j ∈D

Eq¯ ε I(ηε,νε, D¯ = j)E j e−sτε, D¯



¯ j ∈D

= lim

ε→0



Eq¯ ε e−sτε, D¯ I(ηε,νε, D¯ = j)E j e−sτε, D¯

¯ j ∈D

= lim Eq¯ ε e−sτε,→D = φ→ (s) as ε → 0, for s ≥ 0, ε→0

(4.160)

or, equivalently, d

τε,D −→ ξ→,0 as ε → 0,

(4.161)

4.2 First-rare-event times and hitting times

131

where ξ→,0 is a random variable with the Laplace transform Ee−sξ→,0 = φ→ (s), s ≥ 0. Therefore, by Theorem 4.5, the condition D9 is satisfied, and thus, the limiting random variable ξ→,0 (in the relation (4.159)) has the Laplace transform Ee−sξ0 = φ→ (s) = (1 + A(s))−1, s ≥ 0, where A(s) is the cumulant of the infinitely divisible distribution appearing in the condition D9 . The following theorem, which is an analogue of Theorem 4.5 for directed hitting times, summarises the above remarks. Theorem 4.6 Let the conditions A6 , B8 , C10 , and B9 , C12 be satisfied. Then: (i) The condition D9 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε of the random variables ηε,0 concentrated on the domain D, respectively, in the statements of necessity and sufficiency) of the asymptotic relation d

τε,→D −→ ξ→,0 as ε → 0, where ξ→,0 is a non-negative random variable with distribution not concentrated at zero. (ii) The distribution function G→ (u) = P{ξ→,0 ≤∫ u}, u ≥ 0 of the limiting random ∞ 1 , s ≥ 0, variable ξ→,0 has the Laplace transform φ→ (s) = 0 e−su G→ (du) = 1+A(s) where A(s) is a cumulant of infinitely divisible distribution appearing in the condition D9 .

Chapter 5

First-Rare-Event Times for Perturbed Risk Processes

In this chapter, the results obtained in previous chapters are illustrated by applications to geometric type random sums. Necessary and sufficient conditions of convergence in distribution for first-rare-event times represented by geometric type random sums are formulated. Also, necessary and sufficient conditions of weak convergence for non-ruin distribution functions in the models of stable and diffusion approximations for perturbed risk processes are given. This chapter includes two sections. In Sect. 5.1, we give alternative definitions for first-rare-event times and first-rareevent processes, which are more natural for the model, in which the basic Markov renewal process does not depend on perturbation parameter. Theorem 5.1 gives necessary and sufficient conditions of convergence in distribution for first-rare-event times and convergence in topology J for first-rare-event processes. In Sect. 5.2, we consider perturbed risk processes. Theorem 5.2 gives necessary and sufficient conditions of weak convergence for non-ruin distribution functions in the models of stable and diffusion approximations for perturbed risk processes.

5.1 Stable Asymptotics of First-Rare-Event Times In this section, we give necessary and sufficient condition of stable asymptotics for first-rare-event times and processes defined on a finite ergodic Markov chain. Let (ηn, κn, ρn ), n = 0, 1, . . ., be a Markov renewal process, i.e., a homogenous Markov chain with phase space Z = X × [0, +∞) × Y (here, X = {1, 2, . . . , m} and Y is a measurable space with σ-algebra of measurable subsets BY ) and transition probabilities, P{ηn+1 = j, κn+1 ≤ t, ρn+1 ∈ A/ηn = i, κn = s, ρn = y} = P{ηn+1 = j, κn+1 ≤ t, ρn+1 ∈ A/ηn = i}

= Qi j (t, A), i, j ∈ X, s, t ≥ 0, y ∈ Y, A ∈ BY .

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_5

(5.1)

133

134

5 First-rare-event times for perturbed risk processes

The characteristic property, which specifies Markov renewal processes in the class of general multivariate Markov chains (ηn, κn, ρn ), is that transition probabilities do depend only on the current position of the first component ηn . As is known, the first component ηn of the Markov renewal process is also a homogenous Markov chain with the phase space X and transition probabilities pi j = Qi j (+∞, Y), i, j ∈ X. Also, the first two components of the above Markov renewal process ηn and κn can be associated with the semi-Markov process η(t), t ≥ 0 defined as η(t) = ηn for ζn ≤ t < ζn+1, n = 0, 1, . . . ,

(5.2)

where ζ0 = 0 and ζn = κ1 + · · · + κn , n ≥ 1. The random variables κn are times between jumps for the semi-Markov η(t). As for the random variables ρn , they are used to register “rare” events and can be referred as “rare-event indicators.” Let Dε , ε ∈ (0, 1] be a family of measurable “small” in some sense subsets of Y. Then events {ρn ∈ Dε } can be interpreted as “rare” events. Let us introduce random variables, ξε =

νε 

κε,n,

(5.3)

n=1

where: (a) νε = min(n ≥ 1 : ρn ∈ Dε ), (b) κε,n = κn /wε, n = 1, 2, . . ., (c) wε, ε ∈ (0, 1] is a normalisation function such that 0 < wε → ∞ as ε → 0. The random variable νε counts the number of transitions of the embedded Markov chain ηn before the first-rare-event occurs, and the random variable ξε can be interpreted as the first-rare-event time for the semi-Markov process η(t). We also consider rare-event time processes, ξε (t) =

[tν ε] 

κε,n, t ≥ 0.

(5.4)

n=1

The rare-event time model presented above is a special case of the model considered in Sect. 2.1. The corresponding Markov renewal processes (ηε,n, κε,n, ρε,n ), n = 0, 1, . . ., and (ηn, κn, ρn ), n = 0, 1, . . ., are connected by the following relation, for ε ∈ (0, 1], ηε,n = ηn, κε,n = κn /wε, ρε,n = I(ρn ∈ Dε ), n = 0, 1, . . . .

(5.5)

The probabilities pε,i of occurrence of a rare event at one transient step of the semi-Markov process ηε (t) take, in this case, the following form: pε,i = Pi {ρn ∈ Dε }, i ∈ X. The condition A1 takes in this case the following form:

(5.6)

5.1 Stable asymptotics of first-rare-event times

135

 (b) pε,i → 0 A9 : (a) There exists ε ∈ (0, 1] such that maxi ∈X pε,i > 0, for ε ∈ (0, ε], as ε → 0, for i ∈ X. The condition B1 reduces to the following standard condition of ergodicity for the Markov chain ηn : B10 : ηn, n = 0, 1, . . ., is an ergodic Markov chain (i.e., its phase space X is one class of communicative states). Let πi, i ∈ X be the stationary distribution of the Markov chain ηn . This stationary distribution does not depend on ε. The condition C1 , which guaranties that the last summand in the random sum ξε is negligible, takes the following form: C13 : Pi { wκ1ε > δ/ρ1 ∈ Dε } → 0 as ε → 0, for δ > 0, i ∈ X. The relation (2.14) defining probability pε (which is the result of averaging of the probabilities of occurrence of rare event in one transition step over the stationary distribution of the embedded Markov chain ηn ) and its reciprocal uε takes in this case the following forms:  pε = πi pε,i and uε = p−1 (5.7) ε . i ∈X

Let us introduce the distribution functions of the sojourn times for semi-Markov processes η(t), (5.8) Fi (t) = P{κ1 ≤ t}, t ≥ 0, i ∈ X, and F(t) =



πi Fi (t), t ≥ 0, i ∈ X.

(5.9)

i ∈X

Then, the distribution functions Fε,i (t) and Fε (t), introduced in the relations (2.15) and (2.16), take the following forms, Fε,i (t) = Pi {κ1 /wε ≤ t} = Fi (twε ), t ≥ 0, i ∈ X and Fε (t) =



πi Fε,i (t) = F(twε ), t ≥ 0.

(5.10) (5.11)

i ∈X

We will say that a positive function eε , ε ∈ (0, 1] belongs to the class E if: (a) eε → ∞ as ε → 0, (b) there exists a sequence 0 < εn → 0 such that eεn+1 /eεn → 1 as n → ∞. The fourth condition is a kind of assumption about the regularity of the corresponding normalisation functions: E1 : uε = p−1 ε , wε ∈ E. The condition E1 is not restrictive. For example, it is satisfied if uε and wε are continuous functions of ε.

136

5 First-rare-event times for perturbed risk processes

Now we are now to formulate necessary and sufficient conditions for the weak convergence of distribution functions for the first-rare-event times ξε . ∫∞ Let 0 < γ ≤ 1 and a > 0. Let also Γ(α) = 0 t α−1 e−t dt, t ≥ 0 be a Gamma function. The above-mentioned necessary and sufficient conditions take the following forms: D10 :

t[1−F(t)] ∫t sF(ds)

→

1−γ γ

as t → ∞.

0

D11 :

uε

∫ wε 0

sF(ds) wε

γ → a Γ(2−γ) as ε → 0.

The conditions D10 and D11 replace and are, in this case, equivalent to the condition D3 (equivalent to the condition D1 ). Moreover, in this case, the cumulant A(s) = asγ, s ≥ 0. It is the cumulant of a stable law. Theorem 2.1 takes the following form. Theorem 5.1 Let the conditions A9 , B10 , C13 , and E1 be satisfied. Then: (i) The conditions D10 and D11 are necessary and sufficient for the fulfilment (for some or any initial distributions q, ¯ respectively, in the statements of necessity and d

sufficiency) of the asymptotic relation, ξε = ξε (1) −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution function not concentrated at zero. (ii) The distribution function random variable ξ0 has the ∫ ∞ G(u) of the limiting 1 Laplace transform φ(s) = 0 e−su G(du) = 1+as γ , s ≥ 0, where a and γ are parameters appearing in the conditions D10 and D11 . J

(iii) The stochastic processes ξε (t), t ≥ 0 −→ ξ0 (t) = θ 0 (tν0 ), t ≥ 0 as ε → 0, where (a) ν0 is a random variable that has the exponential distribution with parameter 1, (b) θ 0 (t), t ≥ 0 is a non-negative stable Lévy process with the Laplace transforms γ Ee−sθ0 (t) = e−as t , s, t ≥ 0, (c) the random variable ν0 and the process θ 0 (t), t ≥ 0 are independent. Proof Let us introduce Laplace transforms, ∫ ∞ −sκ1 = e−st Fi (dt), s ≥ 0, i ∈ X φi (s) = Ee

(5.12)

0

∫

and

∞

φ(s) = 0

e−st F(dt) =



πi φi (s), s ≥ 0, i ∈ X.

(5.13)

i ∈X

In this case, the Laplace transforms introduced in relations (2.19) and (2.20) take the following forms: φε,i (t) = φi (s/wε ), s ≥ 0, i ∈ X, and

(5.14)

5.1 Stable asymptotics of first-rare-event times

∫ φε (s) =

∞

0

137

e−st Fε (dt) = φ(s/wε ), s ≥ 0, i ∈ X.

(5.15)

As mentioned in Sect. 2.1.3, the condition D3 is equivalent to the condition D2 , which, in this case, takes the form of the following relation, uε (1 − ϕ(s/wε ) → A(s) as ε → 0, for s ≥ 0,

(5.16)

where the limiting function A(s) > 0, for s > 0 and A(s) → 0 as s → 0. The relation (5.16) holds if and only if the following two asymptotic relations hold, (5.17) 1 − ϕ(s) ∼ sγ L(1/s) as 0 < s → 0, and

−γ

uε wε L(wε ) → a as ε → 0,

(5.18)

where: 0 < γ ≤ 1, a > 0, and L(t) is a slowly varying function. This statement is well known, and we present its proof here only for the text to be self-sufficient. If the relations (5.17) and (5.18) hold, then, for s > 0, −γ

uε (1 − ϕ(s/wε )) ∼ uε wε L(wε ) · sγ ·

L(wε /s) → asγ as ε → 0. L(wε )

(5.19)

On the other hand, let us assume that the relation (5.16) holds. ˜ ˜ Let us define an auxiliary function φ(s) = 1 − φ(1/s). The function φ(s) is monotonically decreasing. Due to the condition E1 , there exists εn → 0 as n → ∞ ˜ εn s) → such that uεn+1 /uεn → 1 as n → ∞. Relation (5.16) implies that uεn φ(w A(1/s) > 0 as n → ∞, for s > 0. Thus, according to the well-known criterion (see, ˜ regularly varies, i.e., φ(s) ˜ = s ρ L(s), for example, Feller (1966)), the function φ(s) ρ where L(s) is a slowly varying function, and A(1/s) = as , where −∞ < ρ < +∞ and a > 0. These representations can be rewritten in the following equivalent forms: 1 − φ(s) = sγ L(1/s), and A(s) = asγ, s > 0,

(5.20)

where −∞ < γ = ρ−1 < ∞ and a > 0. γ Since function e−A(s) = e−as should be the Laplace transform of some nonnegative and non-zero random variable, the only values 0 < γ ≤ 1 are admissible. γ In this case, e−as is the Laplace transform of a non-negative stable random variable with parameter γ. The case γ ≤ 0 must obviously be excluded. The case γ > 1 must also be excluded, since, for any non-negative and non-zero random variable ξ, the corresponding Laplace transform Ee−sξ ≥ e−sδ P{ξ ≤ δ}. Therefore, Ee−sξ cannot γ decline in s with the super-exponential rate e−as . The relation (5.17) follows from the relation (5.20) and the above remarks. To verify the relation (5.18), we simply have to repeat the calculations given in the relation (5.19), assuming that the relations (5.16) and representation (5.20) hold. Indeed, the relations (5.16) and (5.20) imply that, for s > 0,

138

5 First-rare-event times for perturbed risk processes −γ

uε (1 − ϕ(s/wε )) = uε wε · sγ L(wε /s) L(wε /s) −γ = uε wε L(wε )sγ L(wε ) γ → A(s) = as as ε → 0.

(5.21)

The relation (5.21), where one should take s = 1, implies that the relation (5.18) holds. Let us now prove that the conditions D10 and D11 are equivalent to the assumption that the asymptotic relations (5.17) and (5.18) hold. First, let us consider the case γ = 1, which corresponds to the situation where the limiting process θ 0 (t) = at, t ≥ 0 is a nonrandom linear function. According to the central criterium of convergence for the sums of i.i.d. random variables (see, for example, Loève (1977)), the necessary and sufficient conditions for the weak convergence of such sums given in the condition D3 (which is equivalent to the relation (5.16) or, according to the above proof, to the relations (5.17) and (5.18)) take the form of the following two relations: uε (1 − Fε (u)) = uε (1 − F(uwε )) → 0 as ε → 0, for u > 0, ∫

and uε

v

0

sFε (ds) → a as ε → 0, for some v > 0.

(5.22)

(5.23)

Note that the relation (5.22) implies that the relation (5.23) either holds or does not hold simultaneously for all v > 0. Indeed, the relation (5.22) implies that, for any 0 < v  < v  < ∞, ∫ uε

v

v

sFε (ds) ≤ uε v (1 − Fε (v ) → 0 as ε → 0.

(5.24)

Taking into account the relation (5.24), we can transform the relations (5.22) and (5.23) into the following equivalent forms: (1 − Fε (u)) ∫u → 0 as ε → 0, for u > 0, sFε (ds) 0 and

∫ uε

0

1

sFε (ds) → a as ε → 0.

Since 1 − Fε (u) = 1 − F(uwε ) and (5.25) and (5.26) can be rewritten as

∫u 0

sFε (ds) =

∫ uwε 0

(5.26) sF(ds)/wε , the relations

wε (1 − F(uwε )) ∫ uwε → 0 as ε → 0, for u > 0, tF(dt) 0 and

(5.25)

(5.27)

5.1 Stable asymptotics of first-rare-event times

uε

∫ wε 0

139

sF(ds)

→ a as ε → 0.

wε

(5.28)

The relation (5.28) coincides with the condition D11 . The condition D10 (with parameter γ = 1) implies that the relation (5.27) holds. This is easy to see by setting t = uuε in the asymptotic relation given in the condition D10 (with the parameter γ = 1). It remains to show that the relation (5.27) implies the fulfilment of the condition D10 (with the parameter γ = 1). According to the condition E1 , the function wε ∈ E. Thus, there exists sequence 0 < εn → 0 as n → ∞ such that wεn+1 /wεn → 1 as n → ∞. For any t, u > 0, we define nu (t) = max(n : uwεn ≤ t). The definition of nu (t) implies that uwεn(t ) ≤ t < uwεn(t )+1 , for u, t > 0, and nu (t) → ∞ as t → ∞, for u > 0. Thus, wεnu (t )+1 /wεnu (t ) → 1 as t → ∞, for u > 0. Using the above asymptotic relation and the relation (5.27), we obtain, for u > 0, t(1 − F(t)) uwεnu (t )+1 (1 − F(uwεnu (t ) )) ≤ ∫t ∫ uwε n (t ) u sF(ds) sF(ds) 0 0 =

wεnu (t )+1 wεnu (t )

·

uwεnu (t ) (1 − F(uwεnu (t ) )) ∫ uwε n (t ) u sF(ds) 0

→ 0 as t → ∞.

(5.29)

Now consider the case 0 < γ < 1. Due to the corresponding Tauberian theorem (see, for example, Feller (1971)), the relation (5.17) is equivalent to the following relation: 1 − F(t) ∼

t −γ L(t) as t → ∞. Γ(1 − γ)

(5.30)

Due to the corresponding theorem about regularly varying functions (see, for example, Feller (1971)), the relation (5.30) is equivalent to the following relation: t(1 − F(t)) → 1 − γ as t → ∞. ∫t (1 − F(s))ds 0

(5.31)

∫t ∫t Since [1 − F(s)]ds = t[1 − F(t)] + sF(ds), t ≥ 0, the relation (5.31) can be 0

0

rewritten in the following equivalent form: t(1 − F(t)) 1−γ as t → ∞. → ∫t γ sF(ds) 0

(5.32)

The relation (5.32) coincides with the condition D10 (with the parameter γ ∈ (0, 1)) and, as mentioned above, is equivalent to the relation (5.17).

140

5 First-rare-event times for perturbed risk processes

Let us show that, under the condition D10 (with parameter γ ∈ (0, 1)), the condition D11 (with parameters γ ∈ (0, 1) and a > 0) and the relation (5.18) are equivalent. Indeed, the relations (5.30) and (5.32), which are equivalent to the condition D10 , imply that the following asymptotic relation holds: ∫u vε 0 ε sF(ds) vε uε (1 − F(uε )) γ ∼ · uε uε 1−γ L(uε ) γ ∼ · γ pε uε Γ(1 − γ) 1 − γ γ L(uε ) as ε → 0. (5.33) = γ · pε uε Γ(2 − γ) This relation completes the proof of Theorem 5.1.



Remark 5.1 As follows from the proof presented above, the assumption wε ∈ E can be omitted in the statements of necessity of Theorem 5.1; the assumption uε = p−1 ε ∈ E in the statement of sufficiency of Theorem 5.1, for the case γ = 1; and the assumption wε, uε ∈ E, i.e., the condition E1 , in the statement of sufficiency of Theorem 5.1, for the case 0 < γ < 1. Remark 5.2 The simplest variant for normalisation functions is where wε = ε −1 . In aε −γ this case, due to the relation (5.18), the function uε = p−1 ε = L(ε −1 ) . In this case, both functions, wε and uε , belong to the class E. In this case, the condition E1 can be omitted in Theorem 5.1. Remark 5.3 As follows from the proof of Theorem 5.1, the conditions D10 and D11 can be replaced by the equivalent conditions given in the relations (5.17) and (5.18). Remark 5.4 In the case 0 < γ < 1, the condition D10 is equivalent to the relation (5.30), which means that the distribution function F(t) belongs to the domain of attraction of a stable law with the parameter γ. In the case γ = 1, the condition D10 is the necessary and sufficient condition for the distribution G(t) to belong to the domain of attraction of the degenerate law (see, for example, Feller (1971)). In Theorem 5.1, these conditions are presented in a convenient unified form. As for the condition D11 , this is the condition that balances the normalisation function wε and the function uε = p−1 ε that determines the number of terms in the sum, representing the first-rare-event time ξε (t). Remark 5.5 It follows from the remarks made in Sect. 4.1.1.2 that, under the condition P C12 , the random variables κνε /wε −→ 0 as ε → 0. This relation implies that the νε first-rare-event times ξε = n=1 κn can be replaced in Theorem 5.1 by the modified  ε −1 κn and, moreover, by any random variable ξε such first-rare-event times ξε = νn=1   that ξε ≤ ξε ≤ ξε .

5.2 Stable approximation of non-ruin distribution functions

141

5.2 Necessary and Sufficient Conditions for Stable Approximation of Non-ruin Distribution Functions Let us apply the results given in Theorem 5.1 to the so-called geometric random sums. This is the reduction of our model for the case when the embedded Markov {1}. chain ηn has the degenerate set of states X =  ε κn is a geometric random sum. In this case, the first-rare-event time ξε = νn=1 Indeed, (κn, ρn ), n = 1, 2, . . ., is a sequence of i.i.d. random vectors. Therefore, the random variable νε = min(n ≥ 1 : ρn ∈ Dε ) has a geometric distribution with the success probability pε = P{ρn ∈ Dε }. It is worth noting that ξε is not a standard geometric sum, since in the above model the random summands κn, n = 1, 2, . . ., and the geometric random index νε = min(n ≥ 1 : ρε,n = 1) are dependent random variables. They are rare-event indicators ρε,n = I(ρn ∈ Dε ), n = 1, 2, . . .. The random vectors (κn, ρn ), n = 1, 2, . . ., are independent and identically distributed. This implies that (κn, ρε,n ), n = 1, 2, . . ., is, for every ε ∈ (0, 1], a sequence of i.i.d. random vectors. However, the components κn and ρn of the random vector (κn, ρn ) and, thus, the components κn and ρε,n of the random vector (κn, ρε,n ) may be dependent random variables. In this case, the condition B10 obviously holds. The conditions A9 and C13 take, in this case, the following forms:  (b) pε = P{ρ1 ∈ Dε } → A10 : There exists ε ∈ (0, 1] such that pε > 0, for ε ∈ (0, ε], 0 as ε → 0. C14 : P{κ1 > δwε /ρ1 ∈ Dε } → 0 as ε → 0. The condition E1 does not change. It should be imposed on the function uε = p−1 ε appearing in the condition A10 and the normalisation function wε . The conditions D10 and D11 also do not change. They should be imposed on the distribution function G(t) = P{κ1 ≤ t} (no averaging is involved) and the functions uε = p−1 ε and wε . The standard geometric sum is a special case of the model described above, in which κn, n = 1, 2, . . ., and ρε,n = I(ρn ∈ Dε }, n = 1, 2, . . ., are independent sequences of i.i.d. random variables, for each ε ∈ (0, 1]. Note that the standard geometric random sum, with any distribution functions index G(t) of the summands κn and parameter pε ∈ (0, 1] of the geometric  random ε κn , where: νε , can be modelled as follows. Consider the geometric sum ξε = νn=1 (a) κn, n = 1, 2, . . ., is a sequence of i.i.d. random variables with the distribution function G(t); (b) νε = max(n ≥ 1 : ρn ∈ Dε ), where ρn, n = 1, 2, . . ., is a sequence of i.i.d. random variables uniformly distributed in the interval [0, 1] and the domains Dε = [0, pε ]; (b) the sequences of random variables κn, n = 1, 2, . . ., and ρn, n = 1, 2, . . ., are independent. In the case of standard geometric sums, the propositions (i) and (ii) of Theorem 5.1 reduce to the result equivalent to those obtained by Kovalenko (1965). The difference is in the form of necessary and sufficient conditions. In the above paper, conditions

142

5 First-rare-event times for perturbed risk processes

based on the Laplace transforms φ(s) and expressed in the form of the relations (5.17) and (5.18) were used. We use the conditions D10 and D11 , which are based on the distribution function G(t) and, as we think, have a more transparent form. Let us illustrate applications of Theorem 5.1, indicating the necessary and sufficient conditions for stable approximation of non-ruin probabilities. Let us consider a classical risk process, Σε (t) = cε t −

N λ (t)

ρn, t ≥ 0.

(5.34)

n=1

Here, cε is a positive constant (depending on parameter ε ∈ (0, 1]) known as gross premium rate, Nλ (t), t ≥ 0 is a Poisson process with parameter λ counting the number of claims against an insurance company for a period of time [0, t], and ρn, n = 1, 2, . . ., is a sequence of non-negative i.i.d. random variables independent on the process Nλ (t), t ≥ 0. The random variable ρk is the amount of the k-th claim. An important object for the study in this model is the non-ruin probabilities on infinite time interval for a company with an initial capital u ≥ 0, G ε (u) = P{u + inf Σε (t) ≥ 0}, u ≥ 0. t ≥0

Let H(u) = P{ρ1 ≤ u} be the claim distribution function. We assume that the following standard condition is satisfied: ∫∞ F1 : μ = 0 sH(ds) < ∞. The crucial role is played by the so-called safety loading coefficient αε = λμ/cε . If αε ≥ 1, then G(u) = 0, u ≥ 0. The only non-trivial case is where αε < 1. We assume that the following condition is satisfied: F2 : αε < 1 for ε ∈ (0, 1] and αε → 1 as ε → 0. Let us introduce the normalised non-ruin distribution function F(uwε ), u ≥ 0, where 0 < wε → ∞ as ε → 0 is some normalisation function. According to the Pollaczek–Khinchine formula (see, for example, Asmussen (2000) or Asmussen and Albrecher (2010)), the non-ruin distribution function coincides with the distribution function of a geometric random sum, which is slightly different from the standard geometric sums discussed above. Namely, G ε (uwε ) = P{ξε,− ≤ u}, u ≥ 0, where ξε,− =

ν ε −1

κn /wε .

(5.35)

(5.36)

n=1

Here: (a) κn, n = 1, 2, . . ., is a sequence ∫ u of non-negative i.i.d. random variables ¯ with the distribution function H(u) = μ1 0 (1 − H(s))ds, u ≥ 0 (the so-called steady claim distribution); (b) νε = min(n ≥ 1, ρε,n = 1); (c) ρε,n, n = 1, 2, . . ., is a

5.2 Stable approximation of non-ruin distribution functions

143

sequence of i.i.d. random variables taking values 1 and 0 with probabilities pε = 1 − αε and 1 − pε ; (d) the random sequences κn, n = 1, 2, . . ., and ρnε, n = 1, 2, . . ., are independent. The condition A10 implies the fulfilment of the condition A9 . The condition B10 can be omitted. The condition C14 is obviously satisfied, since in this case, P{κ1 > δwε /ρε,1 = 1} = P{κ1 > δwε } → as ε → 0 for δ > 0. The condition E1 takes the following form: −1 ∈ E. E2 : wε, uε = p−1 ε = (1 − αε )

According to Remark 5.5, Theorem 5.1 (which is specified for geometric sums as described above) can be applied to the geometric random sums ξε . The conditions D10 and D11 (a > 0 and 0 < γ ≤ 1) take, in this case, the following forms: D12 : and D13 :

∫∞ t t (1−H(s))ds ∫t s(1−H(s))ds 0 ∫ wε 0

→

s(1−H(s))ds (1−αε )μwε

1−γ γ

as t → ∞,

γ → a Γ(2−γ) as ε → 0.

We will summarise the above discussion in the form of the following theorem that gives necessary and sufficient conditions for a stable approximation of non-ruin distribution functions. Theorem 5.2 Let the conditions E2 , F1 , and F2 be satisfied. Then: (i) The conditions D12 and D13 are necessary and sufficient for the fulfilment of the asymptotic relation, G ε (·wε ) ⇒ G0 (·) as ε → 0, where G0 (·) is a distribution function on [0, ∞) not concentrated at zero. ∫∞ (ii) The distribution function G0 (u) has the Laplace transform 0 e−su G0 (du) = 1 1+asγ , s ≥ 0. The case γ = 1 corresponds to the so-called diffusion approximations for risk processes. The traditional method for obtaining the diffusion type asymptotics is based on the approximation of risk processes by Wiener processes with a shift. Typical conditions assume finiteness of the second moment of the claim distribution H(t) (this is equivalent to the assumption about finiteness of the expectation for the ∫ 1 t ¯ steady claim distribution H(t) = μ 0 (1 − H(s))ds): ∫∞ F3 : μ2 = 0 s2 H(ds) < ∞. Obviously, the condition F3 is sufficient for the fulfilment of the condition D12 (with the parameter γ = 1). Note that this condition does not require the finiteness of the second moment of the claim distribution. In this case, the condition D13 takes the following simple equivalent form: D14 : (1 − αε )wε → b = μ2 /2μa as ε → 0.

144

5 First-rare-event times for perturbed risk processes

The corresponding limiting distribution G0 (·) is in this case the exponential with the parameter a. This is consistent with the classical form of diffusion approximation for ruin probabilities. The case γ ∈ (0, 1) corresponds to the so-called stable approximation for risk processes. Recall that, according to the relation (5.30), the condition D12 (with the parameter γ ∈ (0, 1)) is equivalent to the following condition, which requires a∫ regular variation ¯ = 1 t (1 − H(s))ds: for the tail probabilities of the steady claim distribution H(t) μ 0 ∫ −γ L(t) ∞ t D15 : μ1 t (1 − H(s))ds ∼ Γ(1−γ) as t → ∞. As follows from theorems on regularly varying functions (see, for example, Feller (1971)), the following condition imposing requirement of regular variation for the tail probabilities of the claim distribution H(t) is sufficient for the fulfilment of the condition D15 : D16 : 1 − H(t) ∼

t −(γ+1) L(t)γμ Γ(1−γ)

as t → ∞.

As for the condition D13 , in this case it can be formulated in the following form, which is equivalent to the relation (5.18): D17 :

L(wε ) γ (1−αε )wε

→ a as ε → 0.

Chapter 6

First-Rare-Event Times for Perturbed Closed Queuing Systems

In this chapter, the results obtained in Chaps. 2–4 are illustrated by applications to perturbed closed M/M-type queuing systems. We present necessary and sufficient conditions for convergence in distribution for first-rare-event times for a number of models of perturbed closed M/M-type queuing systems. This chapter includes two sections. In Sect. 6.1, we present, in Theorem 6.1, necessary and sufficient conditions of convergence in distribution for the first-rare-event times for perturbed M/M-type queuing systems with a finite number of servers and rare failure events caused by too long service times. In Sect. 6.2, we present, in Theorem 6.2, necessary and sufficient conditions of convergence in distribution for the first-rare-event times for perturbed M/M-type queuing systems with a finite number of servers and rare failure events caused by the absence of working servers in the system with small failure intensities for all servers.

6.1 Queuing Systems with Rare Events Caused by Too Long Service Times In this section, we present necessary and sufficient conditions of convergence in distribution for the first failure event times for perturbed M/M-type queuing system with a finite number of servers and rare failure events caused by too long service times in some states with a small number of working servers.

6.1.1 Asymptotic Uniform Ergodicity for Birth–Death Markov Chains Let ηε,n, n = 0, 1, . . ., be, for every ε ∈ (0, 1], a birth–death finite Markov chain with the phase space X = {0, 1, . . . , m} and transition probabilities, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_6

145

146

6 Perturbed closed queuing systems

pε,i j

⎧ pε,i,± if j = 1±1 for i = 0, ⎪ ⎪ 2 , ⎪ ⎨ pε,i,± if j = i ± 1, ⎪ for 1 < i < m, = ⎪ pε,m,± if j = m + −1±1 ⎪ 2 , for i = m, ⎪ ⎪0 otherwise, ⎩

(6.1)

where pε,i,± ≥ 0, pε,i,− + pε,i,+ = 1, for i ∈ X and ε ∈ (0, 1]. In this case, the condition of asymptotically uniform ergodicity B1 takes the following form: B11 : (a) limε→0 pε,i,+ > 0, for i = 0, . . . , m − 1, (b) limε→0 pε,i,− > 0, for i = 1, . . . , m. The condition B11 implies that there exists ε0 ∈ (0, 1] such that the Markov chain ηε,n is ergodic, or every ε ∈ (0, ε0 ]. The stationary probabilities πε,i, i ∈ X are the only positive solution for the following system of linear equations, ⎧ πε,0 = πε,0 pε,0,− + πε,1 pε,1,−, ⎪ ⎪ ⎪ ⎨ πε,i = πε,i−1 pε,i−1,+ + πε,i+1 pε,i+1,−, 1 < i < m, ⎪ πε,m = πε,m−1 pε,m−1,+ + πε,m pε,m,+, ⎪ ⎪ ⎪ ⎪  j ∈X πε, j = 1. ⎩

(6.2)

The solution of this system is well known and is given by the following relation: πε,i =

i−1 m j−1  pε,r,+    pε,k,+  −1 , i ∈ X. p p r=0 ε,r+1,− j=0 k=0 ε,k+1,−

(6.3)

6.1.2 M/M-Type Queuing System with Rare Events Caused by Too Long Service Times Let Sε be, for every ε ∈ (0, 1], a closed M/M-type queuing system, which consists of N servers functioning independently, with exponentially distributed life and repairing times. The failure and repairing intensities, respectively, με,− > 0 and με,+ > 0, are the same for all servers. The state of system at the moment t is represented by the random variable η˜ε (t), which is the number of working servers at this moment. The process η˜ε (t) is a birth– death continuous time Markov chain, with phase space X = {0, . . . , N }, continuous from the right step-wise trajectories and generator matrix, ⎡ −λε,0 ⎢ ⎢λ p Gε = ⎢⎢ ε,1 ε,1,− ⎢ ··· ⎢ 0 ⎣ where, for k = 0, . . . , N,

λε,0 0 −λε,1 λε,1 pε,1,+ ··· ··· ··· 0···

0 0 ··· 0

··· 0 ⎤⎥ ··· 0 ⎥⎥ , · · · · · · ⎥⎥ λε, N −λε, N ⎥⎦

(6.4)

6.1 Rare events caused by too long service times

pε,k,+ =

147

με,+ · (N − k) με,− · k , pε,k,− = , λε,k = με,+ · (N − k) + με,− · k. (6.5) λε,k λε,k

We also assume that the initial state of the system ηε (0) is a random variable independent of service processes and q¯ε = qε,0, . . . qε, N is its distribution. Let 0 = τ˜ε,0 < τ˜ε,1 < · · · be successive moments of jumps for the Markov chain η˜ε (t), ν˜ε (t) = max(n ≥ 1 : τ˜ε,n ≤ t) be the number of jumps in the interval [0, t], and κ˜ε (t) = t − τ˜ε,ν˜ ε (t) be the time of continuous stay of the system Sε in the state η˜ε (t). ˜ τ˜ε,n ), n = 0, 1, . . ., be the corresponding discrete time embedded Let also η˜ε,n = η( Markov chain and κ˜ε,n = τ˜ε,n − τ˜ε,n−1, n = 1, . . . be the inter-jump times for the Markov chain η˜ε (t). The Markov chain η˜ε,n has the transition probabilities determined in an obvious way by the matrix Gε . The following condition is, in this case, equivalent to the condition B11 : B12 : 0 < limε→0 με,± ≤ limε→0 με,± < ∞. The corresponding stationary probabilities πε,i, i ∈ X are given by the explicit formulas (6.3), which take the following form: πε,i

i−1 m j−1  pε,r,+    pε,k,+  −1 = , i ∈ X, p p r=0 ε,r+1,− j=0 k=0 ε,k+1,−

(6.6)

where the transition probabilities pε,i,±, i ∈ X are given by the relation (6.5). We assume that there exist 0 ≤ L ≤ N such that a rare failure event occurs in the system at instant t, if η˜ε (t) = i ≤ L and κ˜ε (t) = Tε,i . Here, 0 < Tε,i → ∞ as ε → 0, for i = 0, . . . , L. For example, if L = 0, then the first-rare-event occurs at the moment when the time of continuous absence of working servers reaches the level Tε,0 for the first time. We assume that Tε,i → ∞ as ε → 0 with comparable rates, i.e., that the following condition is satisfied: A11 : Tε,i = aε,i Tε, i = 0, . . . , L, where: (a) 0 < limε→0 aε,i ≤ limε→0 aε,i < ∞, for i = 0, . . . , L, (b) 0 < Tε → ∞ as ε → 0. In this case, the corresponding Markov renewal process (ηε,n, κε,n, ρε,n ), n = 0, 1, . . ., defined in Sect. 2.1.1 has components, ⎧ ηε,n ⎪ ⎪ ⎪ ⎨ κε,n ⎪ ρε,n ⎪ ⎪ ⎪ ⎪ ⎩

= η˜ε,n, = κ˜ε,n /wε, = I(η˜ε,n−1 ≤ L)I( κ˜ε,n ≥ Tε,ηε, n−1 ) = I(ηε,n−1 ≤ L)I(κε,n ≥ Tε,ηε, n−1 /wε ),

(6.7)

where 0 < wε → ∞ as ε → 0 is some normalisation function, which we will define below. It is natural to define the normalised first failure time for the system Sε as follows:

148

6 Perturbed closed queuing systems

ξSε =

1 inf(t ≥ 0 : η˜ε (t) ≤ L, κ˜ε (t) = Tηε (t) ). wε

(6.8)

The first-rare-event time for the above Markov renewal process is defined as ξε =

νε 

κε,k ,

(6.9)

k=1

where νε = max(n ≥ 1 : ρε,n = 1) = max(n ≥ 1 : ηε,n−1 ≤ L, κε,n ≥ Tηε, n−1 /wε ).

(6.10)

Obviously, the following inequality holds, for ε ∈ (0, 1], ξε,− ≤ ξSε ≤ ξε,+, where ξε,− =

ν ε −1

κε,k , ξε,+ =

k=1

νε 

(6.11)

κε,k .

(6.12)

k=1

Taking into account remarks made in Sect. 4.1.2, we can apply Lemma 4.1 to the first-rare-event times ξSε . The distribution functions Fε,i (t) are, in this case, exponential, i.e., for i = 0, . . . , N, Fε,i (t) = Pi {κε,1 ≤ t} = Pi { κ˜ε,1 /wε ≤ t} = 1 − exp{−λε,i wε t}, t ≥ 0,

(6.13)

and, thus, the averaged distribution function Fε (t) has the following form: Fε (t) =

N 

πε,i (1 − exp{−λε,i wε t}), t ≥ 0.

(6.14)

i=0

The probabilities pε,i take, in this case, the following form, for i = 0, . . . , N, pε,i = I(i ≤ L)Pi {κε,1 ≥ Tε,i /wε } = I(i ≤ L)Pi { κ˜ε,1 ≥ Tε,i } = I(i ≤ L) exp{−λε,i Tε,i } = I(i ≤ L) exp{−λε,i aε,i Tε }.

(6.15)

The conditions A11 and B12 imply that there exists ε  ∈ (0, 1] such that pε,i > 0, for ε ∈ (0, ε ] and pε,i → 0 as ε → 0, for i ≤ L. Thus, the condition A1 holds. The probability pε = u−1 ε takes, in this case, the following form:   pε = πε,i exp{−λε,i Tε,i } = πε,i exp{−λε,i aε,i Tε }. (6.16) i ≤L

i ≤L

6.1 Rare events caused by too long service times

149

In this case, the Laplace transform φε (s) takes the following form: ∫ φε (s) =

0

∞

e−st Fε (dt) =

N 

πε,i

i=0

1 , s ≥ 0. 1 + s/λε,i wε

(6.17)

Let us denote, for ε ∈ (0, ε0 ], Bε =

N  πε,i . λ i=0 ε,i

(6.18)

The condition B12 implies that 0 < lim Bε ≤ lim Bε < ∞. ε→0

ε→0

(6.19)

1 Let 1+s = 1 − s + o(s) be the asymptotic Taylor expansion of the first order for 1 the function 1+s at the point 0. The relation (6.17) and the condition B12 imply that the following asymptotic relation takes place, for s > 0,

uε (1 − φε (s)) = uε

N 

 πε,i 1 −

i=0

= uε

N 

 1 1 + s/λε,i wε

  πε,i s/λε,i wε + o(s/λε,i wε )

i=0 N 

= uε s

πε,i /λε,i wε +

i=0

∼ sBε

uε as ε → 0. wε

N 

πε,i uε o(s/λε,i wε )

i=0

(6.20)

The relation (6.20) implies that the condition D2 is, in this case, equivalent to the following condition: D18 : Aε = Bε wuεε → A ∈ (0, ∞) as ε → 0. Moreover, in this case, the corresponding limiting cumulant A(s) = As, and thus the corresponding limiting distribution F(u), is exponential with parameter A−1 . It is useful also noting that condition D18 automatically holds if to choose the normalisation function wε = Bε uε . In this case, A = 1. The condition D18 and the relation (6.19) imply that lim

ε→0

and, thus, for i ≤ L,

uε 0, Pi {κε,1 > δ/ρε,1 = 1} = Pi {κε,1 > δ/κε,1 ≥ Tε,i /wε } Pi {κε,1 > δ, κε,1 ≥ Tε,i /uε }

=

= =

P{κε,1 ≥ Tε,i /wε }} Pi { κ˜ε,1 > δwε, κ˜ε,1 ≥ Tε,i } P{ κ˜ε,1 ≥ Tε,i }}

exp{−λε,i (δwε ∨ Tε,i )} → 0 as ε → 0. exp{−λε,i Tε,i }

(6.23)

Therefore, the conditions A11 , B12 , and D18 imply that the condition C1 is satisfied. Therefore, the condition C1 is not required in the sufficiency statement of the corresponding variant of Theorem 2.1. At the same time, the proof of the necessity statement of Theorem 2.2 is based on the conditions A1 , B1 but does not use the condition C1 . Once the condition D1 is proved, the above arguments again lead to the fact that the condition C1 is satisfied. The following theorem is a variant of Lemma 4.2 for the above model. Theorem 6.1 Let the conditions A11 and B12 be satisfied. Then: (i) The condition D18 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) d

of the asymptotic relation ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated in zero. (ii) The limiting random variable ξ0 has an exponential distribution with the parameter A−1 . In conclusion, note that analogues of the statement (iii) of Theorem 2.1 and Theorems 3.1–3.3 can also be formulated.

6.2 Queuing Systems with Highly Reliable Servers In this section, we present necessary and sufficient conditions of convergence in distribution for the first-rare-event times for perturbed closed M/M-type queuing systems with a finite number of servers and rare failure events caused by the absence of working servers.

6.2 Queuing systems with highly reliable servers

151

Let us Sε be, for every ε ∈ (0, 1], a closed M/M-type queuing system, which consists of N servers functioning independently of each other, with independent life and repairing times. The failure and repairing times are exponentially distributed, with intensities, respectively, qε,i,− and qε,i,+ , for i = 1, . . . , N. The state of a server i at instant t is given by the random indicator variable ηˆε,i (t) that takes the value 1 if the server i is functioning at this instant, and the value 0 if the server i is being repaired at this moment. The state of the system at the moment t is given by the random vector ηˆε (t) = (ηˆε,1 (t), . . . , ηˆε, N (t)), t ≥ 0. The phase space of the stochastic process ηˆε (t) is the ˆ = { x¯ = (x1, . . . , x N ) : xi = 0, 1, i = 1, . . . , N }. m-dimensional binary hypercube X ˆ contains m = 2 N states, which is a large It should be noted that the space X number even for relatively small values of N. ˆ In It is also useful to note that there is a natural way to order states in the space X. fact, there is a one-to-one correspondence between the states x¯ = (x1, . . . , xm ) and binary fractions 0.x1 x2 · · · x N , which can be naturally ordered to be their values, (1, . . . , 1, 1, 1) (1, . . . , 1, 1, 0) (1, . . . , 1, 0, 1) (1, . . . , 1, 0, 0) ··· (0, . . . , 0, 0, 0)

↔ ↔ ↔ ↔

0.1 · · · 111 0.1 · · · 110 0.1 · · · 101 0.1 · · · 100 ··· ↔ 0.0 · · · 000

↔ ↔ ↔ ↔

1, 2, 3, 4, ··· ↔ m.

(6.24)

ˆ continThe process ηˆε (t) is a continuous time Markov chain with phase space X, uous from the right step-wise trajectories and transition intensities, qε, x,¯ y¯ =

⎧ ⎪ ⎨ qε,i,− if yi = 0, xi = 1, y j = x j , j  i, for some 1 ≤ i ≤ N, ⎪ qε,i,+ if yi = 1, xi = 0, y j = x j , j  i, for some 1 ≤ i ≤ N, ⎪ ⎪ 0 otherwise. ⎩

(6.25)

We also assume that the initial state of the system ηˆε (0) is a random variable ˆ is its distribution. ¯ x¯ ∈ X independent of service processes and q¯ε = qε ( x), Let τˆε (n), n = 0, 1, . . ., be successive moments of jumps of the Markov chain ηˆε (t), t ≥ 0, κˆε,n = τˆε,n − τˆε,n−1, n = 1, . . . be inter-jump times for the Markov chain ηˆε (t), and ηˆε,n = ηˆε (τˆε (n)), n = 0, 1, . . ., be the states of the Markov chain ηˆε (t) at successive moments of jumps. The random sequence ηˆε,n, n = 0, 1, . . ., is an embedded discrete time Markov chain for the continuous time Markov chain ηˆε (t). Consider also the Markov renewal process (ηˆε,n, κˆε,n, ρˆ ε,n ), n = 0, 1, . . ., which has the phase space X × [0, ∞) × {0, 1} and the third component ρˆ ε,n of the following form: ¯ for n = 0, 1, . . . , ρˆ ε,0 = 0, (6.26) ρˆ ε,n = I(ηˆε,n = 0), where: 0¯ = (0, . . . , 0). Let us assume that the following condition is satisfied:

152

6 Perturbed closed queuing systems

B13 : (a) 0 < limε→0 qε,i,+ ≤ limε→0 qε,i,+ < ∞, for i = 1, . . . , N, (b) there exists ε  ∈ (0, 1] such that qε,i,− > 0, for ε ∈ (0, ε ] and qε,i,− → 0 as ε → 0, for i = 1, . . . , N. The condition B13 implies that there exist ε  ∈ (0, ε ] such that all intensities με,i,± > 0, i = 1, . . . , N, for ε ∈ (0, ε ]. In what follows, we assume that ε ∈ (0, ε ]. ˇ It is easy to see that, for x¯ = (x1, . . . , x N ) ∈ X, P x¯ { κˆε,1 ≤ u} = 1 − e−qε, x¯ u, u ≥ 0,

where qε, x¯ =



qε, x,¯ y¯ =



qε,i,− +

i:xi =1

y ¯ x¯



qε,i,+ .

(6.27) (6.28)

i:xi =0

ˇ Also, for x¯ = (x1, . . . , x N ), y¯ = (y1, . . . , y N ) ∈ X, P x¯ {ηˆε,1 = y¯ /ηˆε,0 = x} ¯ = pε, x,¯ y¯ ,

where pε, x,¯ y¯ =

qε, x,¯ y¯ . qε, x¯

(6.29)

(6.30)

The condition B13 (b) means that all servers are asymptotically highly reliable. The first hitting of the process η˜ε (t) in the state 0¯ = (0, . . . , 0) occurs in the above system at the first moment when all servers are down (are in repair mode). Such event can be interpreted as a rare, due to the condition B13 . Let us consider the Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ), n = 0, 1, . . ., which ˇ × [0, ∞) × {0, 1}, where X ˇ = X, ˆ and components, has the phase space X ⎧ ⎪ ⎨ ηˇε,n = ηˆε,n, ⎪ κˇε,n = κˆε,n /wε, ⎪ ⎪ ρˇ ε,n = I(ηˆε,n−1 = 0), ¯ ⎩

(6.31)

where 0 < wε → ∞ as ε → 0 is some normalisation function, which we will define below. It is natural to define the normalised first-rare-event time for the system Sε , which can also be called the first failure time, as follows: ξSε =

νˇ ε  n=1

κˇε,n =

νˆ ε 

κˆε,n /wε,

(6.32)

n=1

where ¯ νˇε = min(n ≥ 1 : ηˇε,n = 0).

(6.33)

In this case, the condition B1 is not satisfied for the Markov chains ηˇε,n , since the condition B13 (b) implies that, for y¯  1¯ = (1, . . . , 1), pε, 1,¯ y¯ → 0 as ε → 0.

(6.34)

6.2 Queuing systems with highly reliable servers

153

However, we can use the results presented in Sect. 4.1.4.2. ˇ is one class of communicative The condition B13 implies that the phase space X  states for the Markov chain ηˇε,n , for ε ∈ (0, ε ]. ˇ and Therefore, the condition B6 is satisfied and, thus, for any subset ∅  D ⊆ X ˇ i ∈ X, Pi { νˇε,D < ∞} = 1, (6.35) where νˇε,D = min(n ≥ 1 : ηˇε,n ∈ D).

(6.36)

Let us use the simplest variant, where X is the following one-state set, X = {1¯ = (1, . . . , 1)}.

(6.37)

The relation (6.35) makes it possible to define the successive times of hitting the Markov chain ηˇε,n in the set X, ¯ n = 1, 2, . . . , where νˇε,X,0 = 0. νˇε,X,n = min(k > νˇε,X,n−1, ηˇε,k = 1),

(6.38)

The corresponding Markov renewal process with transition period (η˜ε,n, κ˜ε,n , ˜ = X, ˇ for the initial state η˜ε,0 , and the ρ˜ ε,n ), n = 0, 1, . . . (with the phase space X phase space X, for the random variables η˜ε,n, n ≥ 1) can be constructed using the following recurrent relations: ⎧ η˜ε,n = ηˇε,νˇ ε,X, n , n = 1, 2, . . . , η˜ε,0 = ηˇε,0, ⎪ ⎪ νˇ ε,X, n ⎨ ⎪ κ˜ε,n = k= κˇε,k , n = 1, 2, . . . , κ˜ε,0 = κˇε,0, νˇ n−1 +1 ⎪ ε,X, νˇ ε,X, n ⎪ ⎪ ρ˜ ε,n = 1 − (1 − ρˇ ε,k ), n = 1, 2, . . . , ρ˜ ε,0 = ρˇ ε,0 . k=νˇ ε,X, n−1 +1 ⎩

(6.39)

Also, as in Sect. 4.1.4.2, the corresponding shifted Markov renewal process can be defined by the following relation: (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . . .

(6.40)

¯ n = 0, 1, . . ., i.e., it is a trivial It is obvious that the first component ηε,n = 1, nonrandom sequence and, therefore, (κε,n, ρε,n ), n = 0, 1, . . ., is the sequence i.i.d. random vectors. Obviously, in this case, the stationary probabilities πε, 1¯ ≡ 1. Obviously, the following inequality holds, for ε ∈ (0, ε ], ξ˜ε,− ≤ ξSε ≤ ξ˜ε,+, where ξ˜ε,− =

ν˜ ε −1 k=1

κ˜ε,k , ξ˜ε,+ =

ν˜ ε 

(6.41)

κ˜ε,k ,

(6.42)

k=1

and ν˜ε = min(n ≥ 1 : ρ˜ ε,n = 1).

(6.43)

154

6 Perturbed closed queuing systems

Let us show that the conditions C1 (for the Markov renewal processes (ηε,n, κε,n , ρε,n )) and A5 , B6 , B7 , C8 are satisfied and, thus, Lemma 4.6 can be applied. In this case, the condition B1 is obviously satisfied for Markov chains ηε,n , which are asymptotically uniformly ergodic. The condition B13 obviously implies that the conditions B6 and B7 are satisfied. Let us introduce sets, for n = 0, . . . , N, Yn = { x¯ = (x1, . . . , x N ) :

N 

xk = N − n}.

(6.44)

k=1

  Obviously, the number of states in the set Yn is mn = Nn , for n = 0, 1, . . . , N. In this case, the probabilities,

p ¯ for x¯ ∈ Y N −1, pˇε, x¯ = Px¯ { ρˇ ε,1 = 1} = ε, x,¯ 0 (6.45) 0 for x¯  Y N −1 . If x¯ = (x1, . . . , x N ), where xi = I(i = j), i = 1, . . . , N, for some 1 ≤ j ≤ N, then  qε, j,− pε, x,¯ 0¯ = , qε, x¯ = qε, j,− + qε,i,+ . (6.46) qε, x¯ ij The condition B13 and the relations (6.45)–(6.46) imply that the condition A5 is satisfied. Thus, the condition A1 is also satisfied, i.e., there exists ε  ∈ (0, ε ] such that, for ε ∈ (0, ε ], νˇ ε,X,1

pε, 1¯ = P{ ρ˜ ε,1 = 1} = P{

ρˇ ε,k ≥ 1} > 0,

(6.47)

k=1

and Thus, the function

pε, 1¯ → 0 as ε → 0.

(6.48)

uε = p−1 → ∞ as ε → 0. ε, 1¯

(6.49)

Finding more explicit asymptotic approximations for the functions pε, 1¯ and uε is a rather difficult task. ˆ must be renumbered according to the relation First of all, note that the states x¯ ∈ X (6.24) in order to get the standard representations for the matrices defined below. Let us introduce the 1 × m1 matrix, Pε,0,− = pε, 1,¯ y¯ 1∈Y ¯ 0, y¯ ∈Y1 .

(6.50)

If y¯ = (y1, . . . , y N ), where yi = I(i  j), i = 1, . . . , N, for some 1 ≤ j ≤ N, then

6.2 Queuing systems with highly reliable servers

pε, 1,¯ y¯ =

155

N  qε, j,− , qε, 1¯ = qε,i,− . qε, 1¯ i=1

(6.51)

Let us also introduce the mn × mn±1 matrices, for n = 1, . . . , N − 1, Pε,n,± = pε, x,¯ y¯ x¯ ∈Yn, y¯ ∈Yn∓1 . If x¯ = (x1, . . . , x N ) ∈ Yn, y¯ = (y1, . . . , y N ) ∈ Yn∓1 , and y j = i  j for some 1 ≤ j ≤ N, then   qε, j,± pε, x,¯ y¯ = , qε, x¯ = qε,i,− + qε,i,+ . qε, x¯ x =1 x =0 i

(6.52) 1±1 2

 x j , yi = xi,

(6.53)

i

The relations (6.52), (6.53) and the condition B13 imply that, for n = 1, . . . , N − 1,   qε,i,± |Pε,n,± | = max |pε, x,¯ y¯ | = max < 1. (6.54) x¯ ∈Y n x¯ ∈Y n qε, x¯ 1∓1 y¯ ∈Y xi =

n∓1

2

Moreover, the relation (6.53) and the condition B13 imply that, for n = 1, . . . , N −1,  1≤i ≤ N qε,i,− |Pε,n,− | ≤ qε = → 0 as ε → 0, (6.55) min1≤i ≤ N qε,i,+ while |Pε,n,+ | → 1 as ε → 0.

(6.56)

Let us also define, for n = 1, . . . , N − 1, the matrices, Qε,n = Q ε,n, x,¯ y¯ x¯ ∈Yn, y¯ ∈Yn+1 ,

(6.57)

where, for x¯ ∈ Yn, y¯ ∈ Yn+1 , Q ε,n, x,¯ y¯ = Px¯ { τˇε,Yn+1 < τˇε,Y0 , ηˇε, τˇ ε,Y n+1 = y¯ }.

(6.58)

Qε,1 = Pε,1,− .

(6.59)

Obviously, The condition B6 implies that, for n = 1, . . . , N − 1,  |Qε,n | = max Q ε,n, x,¯ y¯ x¯ ∈Y n

y¯ ∈Y n+1

= max Px¯ { τˇε,Yn+1 < τˇε,Y0 } < 1. x¯ ∈Y n

(6.60)

Let us also define, for n = 1, . . . , N − 1, the matrices, Rε,n = Rε, x,¯ y¯ x¯ ∈Yn, y¯ ∈Yn ,

(6.61)

156

6 Perturbed closed queuing systems

where, for x¯ ∈ Yn, y¯ ∈ Yn , Rε,n, x,¯ y¯ = Px¯ { τˇε,Yn < τˇε,Y0 ∧ τˇε,Yn+1 , ηˇε, τˇ ε,Y n = y¯ }.

(6.62)

Obviously, for x¯ ∈ Y1, y¯ ∈ Y1 , Rε,1, x,¯ y¯ = 0.

(6.63)

The following relation takes place, for n = 2, . . . , N − 1, Rε,n = Pε,n,+ Qε,n−1,

(6.64)

and, thus, by the relations (6.54) and (6.60), |Rε,n | ≤ |Pε,n,+ ||Qε,n−1 | < 1.

(6.65)

The above relation implies that, for n = 2, . . . , N − 1, there exists the inverse matrix, [In − Rε,n ]−1 , where In = I( x¯ = y¯ x¯ ∈Yn, y¯ ∈Yn is the mn × mn unit matrix, for n = 1, . . . , N − 1. The following relation takes place, for n = 1, . . . , N − 1, Qε,n = Pε,n,− + Rε,n Pε,n,− + R2ε,n Pε,n,− + · · ·   = In + Rε,n + R2ε,n + · · · Pε,n,− = [In − Rε,n ]−1 Pε,n,− = [In + Rε,n [In − Rε,n ]−1 ]Pε,n,− .

(6.66)

Let us choose an arbitrary 0 < q < 1. The relations (6.54), (6.55), and (6.59) imply that there exists εq ∈ (0, ε ] such that for ε ∈ (0, εq ], (6.67) |Qε,1 | = qε ≤ q, and |Qε,1 | → 0 as ε → 0.

(6.68)

The relations (6.60), (6.64), (6.67), and (6.68) imply that, for ε ∈ (0, εq ], |Rε,2 | ≤ |Qε,1 | = qε ≤ q,

(6.69)

and |Rε,2 | → 0 as ε → 0. The relations (6.54), (6.55), and (6.69) imply that there exist that, for ε ∈ (0, εq],

(6.70) εq

|Qε,2 | ≤ |Pε,2,− | + |Rε,2 ||Pε,2,− | + |Rε,2 | 2 |Pε,2,− | + · · ·   ≤ 1 + q + q2 + · · · qε = (1 − q)−1 qε ≤ q and

∈

(0, εq ]

such

(6.71)

6.2 Queuing systems with highly reliable servers

157

|Qε,2 | → 0 as ε → 0.

(6.72)

The relations (6.60), (6.64), (6.71), and (6.72) imply that, for ε ∈ (0, εq], |Rε,3 | ≤ |Qε,2 | ≤ q,

(6.73)

|Rε,3 | → 0 as ε → 0.

(6.74)

and The relations (6.54), (6.55), and (6.73) imply that, for ε ∈ (0, εq], |Qε,3 | ≤ |Pε,3,− | + |Rε,2 ||Pε,3,− | + |Rε,3 | 2 |Pε,3,− | + · · ·   ≤ 1 + q + q2 + · · · qε = (1 − q)−1 qε ≤ q

(6.75)

|Qε,3 | → 0 as ε → 0.

(6.76)

and The above recurrent calculations can be continued and give two series of relations. First, the following relations hold, for n = 1, . . . , N − 1 and ε ∈ (0, εq], |Qε,n | ≤ q, and, for n = 1, . . . , N − 1,

(6.77)

|Qε,n | → 0 as ε → 0.

(6.78)

Second, the following relation holds, for n = 2, . . . , N − 1 and ε ∈

and, for n = 1, . . . , N − 1,

(0, εq],

|Rε,n | ≤ q,

(6.79)

|Rε,n | → 0 as ε → 0.

(6.80)

The following representation takes place: pε, 1¯ = Pε,0,− Qε,1 · · · Qε, N −1 = Pε,0,− Pε,1,− [I2 + Rε,2 [I2 − Rε,2 ]−1 ]Pε,2,− · · · [I N −1 + Rε, N −1 [I N −1 − Rε, N −1 ]−1 ]Pε, N −1,− .

(6.81)

The relations (6.79) and (6.80) imply that, for n = 2, . . . , N − 1 and ε ∈ (0, εq], |Rε,n [I N −1 − Rε, N −1 ]−1 ]| ≤ and

q , 1−q

|Rε,n [I N −1 − Rε, N −1 ]−1 ]| → 0 as ε → 0.

(6.82)

(6.83)

Finally, the relations (6.81), (6.82), and (6.83) imply that pε, 1¯ ∼ pε, 1¯ = Pε,0,− Pε,1,− · · · Pε, N −1,− as ε → 0.

(6.84)

158

6 Perturbed closed queuing systems

We can use the explicit formulas for elements of the matrices Pε,n,− , n = 0, . . . , N − 1 given by the relation (6.53) and extract from the product of these matrices all quantities qε, j,− and qε, 1¯ , which tends to 0 as ε → 0. The resulting matrix takes the following form: pε, 1¯ = Pε,0,− Pε,1,− · · · Pε, N −1,− N j=1 qε, j,− = N P ε,0,− P ε,1,− · · · P ε, N −1,− , q ε, j,− j=1 where, for n = 0, . . . , N − 1,  I(qε, 1,¯ y¯ > 0)1∈Y ¯ 0, y¯ ∈Y1 for n = 0, P ε,n,− = I(qε, x, ¯ y¯ >0)  qε, x¯ x¯ ∈Yn, y¯ ∈Yn+1 for n = 1, . . . , N − 1.

(6.85)

(6.86)

The matrix P ε,0,− P ε,1,− · · · P ε, N −1,− = Cε = Cε  is a 1 × 1 matrix. The condition B13 implies that 0 < lim Cε ≤ lim Cε < ∞. ε→0

ε→0

Let us now define the new normalisation function, N j=1 qε, j,− uε = N Cε−1 . q ε, j,− j=1

(6.87)

(6.88)

The relations (6.84), (6.85), and (6.87) imply that uε ∼ uε as ε → 0.

(6.89)

ˇ and δ ≥ 0, In this case, for x¯ ∈ X P x¯ { κˇε,1 > δ} = e−δwε qε, x¯ .

(6.90)

¯ The condition B13 and the relation (6.28) imply that, for x¯  1, 0 < lim qε, x¯ ≤ lim qε, x¯ < ∞, ε→0

while qε, 1¯ =

N 

ε→0

qε,i,− → 0 as ε → 0.

i=1

Let us, also, introduce, for n = 1, . . . , N − 1, the random variables,

(6.91)

(6.92)

6.2 Queuing systems with highly reliable servers

159

τˇ ε,Y0 ∪Y n+1

νˇε,n =



I(ηˇε,k ∈ Yn ).

(6.93)

k=1

Obviously, for n = 1, . . . , N − 1 and k = 0, 1, . . ., Sε,n,k =  Px¯ { νˇε,n = k, ηˇε, τˇ ε,Y0 ∪Yn+1 = y¯ }x¯ ∈Yn, y¯ ∈Yn+1 = Rkε,n Pε,n,−

(6.94)

The condition B13 implies that, for x¯ ∈ Yn, n = 1, . . . , N − 1, P x¯ { νˇε,n < ∞} = 1.

(6.95)

Also, the condition B13 obviously implies that the condition C8 holds, if the ˇ function 0 < wε → ∞ as ε → 0 is chosen in such a way that, for x¯ ∈ X, wε qε, 1¯ → ∞ as ε → 0.

(6.96)

ˇ The relation (6.31) implies that, for u ≥ 0 and x¯ ∈ X, νˇ ε,X,1

F˜ε, x¯ (u) = Px¯ { κ˜ε,1 ≤ u} = P1¯ {

κˇε,k ≤ u} = Fˆε (uwε ),

(6.97)

k=1

where

νˆ ε,X,1

Fˆε (u) = P1¯ {

κˆε,k ≤ u}.

(6.98)

k=1

The following stochastic representation takes place and plays a key role in subsequent asymptotic analysis,  d τε, 1¯ = ζε + I(ιε = y¯ )τε, y¯ , (6.99) y¯ ∈Y1

where: (a) the random variable τε, x¯ has the distribution function Px¯ { κ˜ε,1 ≤ ·}, for ˜ (b) the random variable ζε is exponentially distributed with parameter qε, 1¯ wε , x¯ ∈ X, (c) the random variable ιε takes values y¯ with probabilities, respectively, pε, 1,¯ y¯ , for ˜ are mutually independent. y¯ ∈ Y1 , and (d) the random variables ζε , ιε , τε, x¯ , x¯ ∈ X ˇ and, thus, for x¯ ∈ X ˇ In this case, Px¯ { κˇε,1 ≤ u} = 1 − e−wε qε, x¯ u, u ≥ 0, for x¯ ∈ X, and n ≥ 1, n! n E x¯ κˇε,1 = . (6.100) (wε qε, x¯ )n Taking into account the relations (6.99) and (6.100) and calculating expectations and second moments for the random variable τε, 1¯ , we get the following relations: E1¯ κ˜ε,1 =

 1 + p ¯ Ey¯ κ˜ε,1, wε qε, 1¯ y¯ ∈Y ε, 1,− 1

(6.101)

160

6 Perturbed closed queuing systems

and 2 E1¯ κ˜ε,1 =

 2 1 +2 p ¯ Ey¯ κ˜ε,1 2 wε qε, 1¯ y¯ ∈Y ε, 1,− (wε qε, 1¯ ) 1  2 + pε, 1,− ¯ Ey¯ κ˜ε,1 .

(6.102)

y¯ ∈Y1

˜ \ X, α = The relation (6.100) implies that the condition C9 is satisfied, for x¯ ∈ X ¯ β = 2 and, thus, by Lemma 4.7, for x¯  1, 2 E x¯ κ˜ε,1 ≤

Bε , wε2

where a p, N,2,2 = max Bε = eˇε, x,2,2 ¯ y ¯ 1¯

(6.103)

2 a p, N,2,2 2 qε, y¯

(6.104)

and the constant a p, N,2,2 is given by the relation (4.108). The condition B13 implies that 0 < lim Bε ≤ lim Bε < ∞. ε→0

ε→0

(6.105)

Let θˆε,n, n = 1, 2, . . ., be, for ε ∈ (0, εq], i.i.d. random variables with the distribution function Fˆε,X (·) = Fˆε (·). Let us also consider the sequence of i.i.d. random (w) variables θ ε,n = θˆε,n /wε, n = 1, 2, . . .. Obviously, these random variables have the distribution function Fε (·) = Fˆε (uwε ). ¯ The relation (6.103) also implies that, for x¯  1, √  Bε 2 E x¯ κ˜ε,1 ≤ E x¯ κ˜ε,1 ≤ . (6.106) wε The relations (6.92), (6.103), and (6.106) imply that 1

(w)

Eθ ε,1 = E1¯ κ˜ε,1 ∼ as ε → 0 wε qε, 1¯

and (w)

2 E(θ ε,1 )2 = E1¯ κ˜ε,1 ≤

  1 2  2 + 2 Bε qε, 1¯ + Bε qε, . 1¯ 2 (wε qε, 1¯ )

(6.107)

(6.108)

Let us consider the random variables, θ ε(w) =

[u ε]  n=1

(w) θ ε,n =

[u ε]  n=1

θˆε,n /wε .

(6.109)

6.2 Queuing systems with highly reliable servers

161

Let us, first, choose the normalisation function, w◦,ε =

uε . qε, 1¯

(6.110)

The relations (6.49), (6.89), (6.92), and (6.107) imply that (w◦ )

Eθ ε

(w◦ ) = [uε ]Eθ ε,1 = [uε ]

=

1 w◦,ε qε, 1¯

[uε ] uε qε, 1¯ → 1 as ε → 0. uε uε qε, 1¯

(6.111)

Also, the relations (6.49), (6.92), and (6.108) imply that (w◦ ) (w◦ ) 2 Varθ ε(w◦ ) = [uε ]Varθ ε,1 ≤ [uε ]E(θ ε,1 )   1 2  2 + 2 Bε qε, 1¯ + Bε qε, ≤ [uε ] 1¯ (w◦,ε qε, 1¯ )2  [uε ]  2  → 0 as ε → 0. ≤ 2 2 + 2 Bε qε, 1¯ + Bε qε, 1¯ uε

(6.112)

The relations (6.108) and (6.112) imply that the following asymptotic relation takes place: [u ε]  (w◦ ) P θ ε(w◦ ) = θ ε,n −→ 1 as ε → 0. (6.113) n=1

Let us now consider the case, where the function 0 < wε → ∞ as ε → 0 satisfies the following condition: D19 : Aε =

u ε qε, 1¯ wε

→ A ∈ (0, ∞) as ε → 0.

Under the condition B13 , the condition D19 is necessary and sufficient for the fulfilment of the condition D1 , which in this case takes the form of the following asymptotic relation: [u ε]  (w (w (w) θ ε,n ⇒ θ 0 ) as ε → 0, (6.114) θε ) = n=1 (w

where θ 0 ) is a non-negative random variable with distribution function not concentrated at zero. (w Moreover, in this case, the limiting random variable θ 0 ) = A with probability 1. Indeed, assume that the conditions B13 and D19 are satisfied. In this case, the relation (6.113) and the asymptotic relation appearing in the condition D19 imply that

162

6 Perturbed closed queuing systems

θ ε(w) =

[u ε] 

(w) θ ε,n =

n=1

=

[u ε] 

(w◦ ) θ ε,n

n=1

θ ε(w◦ )

uε qε, 1¯ wε

uε P −→ A as ε → 0. qε, 1¯ wε

(6.115)

Thus, the above sufficiency statement is true. Let us assume that the condition B13 and the relation (6.114) hold, but the condition D19 is not satisfied. In this case, one can always select two subsequences 0 < εk , εk ≤ 1 such that: u ε 

(a) εk , εk → 0 as n → ∞, (b)

qε  , 1¯ wε  → k k (d) A  A.

A

k

A ∈ [0, ∞] as k → ∞, (c)

[uε  ]

=

k n=1

=

[uε  ]

θ ε(w)  ,n k

k

=

◦) θ ε(w ,n k

n=1

uεk θ ε(w ◦ ) k q  ¯ wε  εk , 1 k

k

k

∈ [0, ∞] as k → ∞, and In this case, the relation (6.113) implies that θ ε(w)  k

u ε 

qε  , 1¯ wε 

→

k

uεk qε , 1¯ wεk k

P

−→ A as k → ∞

(6.116)

and [uε  ]

θ ε(w)  k

=

k n=1

= θ ε(w◦ ) k

[uε  ]

θ ε(w) ,n k

=

uεk

k n=1

qε, 1¯ wεk

P

◦) θ ε(w,n k

uεk qε, 1¯ wεk k

−→ A as k → ∞.

(6.117)

k

The above two relations contradict to the relation (6.114). Thus, the above necessity statement is also true, i.e., the condition D19 is satisfied. (w In this case, due to the relation (6.115), the limiting random variable θ 0 ) in relation (6.114) equals to the constant A with probability 1. The relation appearing in the condition D19 obviously implies that the relation (6.96) holds and, thus, the condition C8 also holds. The condition D19 plays, in this case, the role of the condition D1 . Finally, the relation (6.108) implies that √ 2 2 + 2 Bε qε, 1¯ + Bε qε, 1¯ 2 E1¯ κε,1 ≤ 2 (wε qε, 1¯ ) √ 2 2 + 2 Bε qε, 1¯ + Bε qε, uε uε 1¯ = p ¯ = o1¯ (ε)pε, 1¯ , (6.118) qε, 1¯ wε qε, 1¯ wε uε ε, 1

6.2 Queuing systems with highly reliable servers

163

where o1¯ (ε) =

√ 2 2 + 2 Bε qε, 1¯ + Bε qε, 1¯ qε, 1¯ wε

uε uε → 0 as ε → 0. qε, 1¯ wε uε

(6.119)

The relations (6.118) and (6.119 ) imply that the condition C5 is satisfied (for α, β = 2), and, thus, the condition C1 also holds for the Markov renewal processes (ηε,n, κε,n, ρε,n ). Therefore, according to Lemma 4.6, all conditions of Theorem 4.4 are satisfied for the Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ). In this case, Theorem 4.4 takes the following form. Theorem 6.2 Let the condition B13 be satisfied. Then: (i) The condition D19 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) d

of the asymptotic relation ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution function not concentrated at zero. (ii) The limiting random variable ξ0 has an exponential distribution with the parameter A−1 . In conclusion, we would like to note that typical sufficient conditions for convergence in distribution for random functionals, such as normalised first failure times ξSε , usually require convergence of all intensities με,i,+, i = 1, . . . , N (as functions of ε → 0) to some limiting values. The conditions B13 and D19 do not require the convergence of these intensities. They can fluctuate almost arbitrary in the interval (0, ∞). Convergence of only one “leading” parameter Aε appearing in the condition D19 is required. Moreover, this convergence is not only sufficient but also a necessary condition for convergence in distribution for the normalised first failure times ξSε .

Chapter 7

First-Rare-Event Times for Perturbed M/M-Type Queuing Systems

In this chapter, the results obtained in Chaps. 2–4 are illustrated by applications to perturbed M/M-type queuing systems with bounded and unbounded buffers. We are especially interested in the applications of the results of Chap. 4 related to the first-rare-event times, based on Markov renewal processes with transition periods and extending phase spaces for modulating semi-Markov processes. We present necessary and sufficient conditions for convergence in distribution of first-rare-event times for a number of models of perturbed M/M-type queuing systems with bounded and unbounded buffers. This chapter includes two sections. In Sect. 7.1, we present necessary and sufficient conditions for the convergence in the distribution for the first-rare-event times for perturbed M/M queuing systems with limited queue buffers and rare failures caused by losses of customers trying to enter the system with a full buffer. Two cases are considered when the buffer size is fixed and the input stream intensity tends to zero, and when the buffer size can tend to infinity. The ergodic asymptotics is given in Theorems 7.1 and 7.2. In Sect. 7.2, we present necessary and sufficient conditions of convergence in distribution for the first-rare-event times for perturbed M/M-type queuing systems with infinite queue buffer and rare failure events caused by too large number of customers appearing in the queue. The corresponding asymptotic result is given in Theorem 7.3.

7.1 Rare Events for Queuing Systems with Bounded Buffers In this section, we give necessary and sufficient conditions for the convergence in distribution for first failure event times for perturbed M/M-type queuing system with bounded queue buffers.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_7

165

166

7 Perturbed M/M-type queuing systems

7.1.1 Queuing Systems with a Buffer of Fixed Size Let Sε for each ε ∈ (0, 1] be a standard M/M queuing system with bounded queue buffer. This system has one server and a bounded queue buffer of a fixed size N ≥ 1 (including the place in the server). It is assumed that: (a) The input flow of customers is a standard Poisson flow with parameter με,+ > 0; (b) customers are placed in the queue buffer according to the order they arrive in the system; (c) a new customer coming in the system is immediately served if the queue buffer is empty, is placed in the queue buffer if the number of customers in the system is less than N, or leaves the system if the queue buffer is full, i.e., the number of customers in the system equals to N; (d) the service times for different customers are independent random variables, which have an exponential distribution with parameter με,− > 0; (f) the input flow and all service processes are mutually independent. The state of the system at the moment t is represented by the random variable ηˆε (t), which is the number of customers in the buffer at this moment. The process ηˆε (t) is ˆ = {0, . . . , N } a birth–death-type semi-Markov process, which has a phase space X and transition probabilities, ˆ Qˆ ε,i j (t) = pˆε,i j Fˆε,i (t), t ≥ 0, i, j ∈ X.

(7.1)

The (N + 1) × (N + 1) matrix of transition probabilities for the corresponding embedded Markov chain ηˆε,n has the following form: ⎡ 0 ⎢ ⎢ pε,− ⎢ Pˆ ε =  pˆε,i j  = ⎢⎢ · · · ⎢ 0 ⎢ ⎢ 0 ⎣ where, pε,+ =

1 0 0 0 pε,+ 0 ··· ··· ··· · · · 0 pε,− ··· 0··· 0

··· ··· ··· 0 pε,−

0 ⎤⎥ 0 ⎥⎥ · · · ⎥⎥ , pε,+ ⎥⎥ pε,+ ⎥⎦

με,+ με,− , pε,− = , λε = με,+ + με,− λε λε

and the distribution functions of sojourn times have the following forms:

1 − e−με,+ t for t ≥ 0, i = 0, Fˆε,i (t) = 1 − e−λε t for t ≥ 0, i = 1, . . . , N.

(7.2)

(7.3)

(7.4)

We also assume that the initial state of the system η˜ε (0) is a random variable independent of service processes and q¯ε = qε,0, . . . qε, N is its distribution. It is useful to note that the semi-Markov process η˜ε (t) is, in fact, a homogeneous Markov process. However, we do prefer to consider it as a semi-Markov process, since its trajectories can include jump transitions of the form N → N. In the standard model of continuous time Markov chains, jump transitions of the form i → i are usually excluded. Let us consider the case, where the following condition is satisfied:

7.1 Rare events for queuing systems with bounded buffers

167

B14 : (a) 0 < limε→0 με,+ ≤ limε→0 με,+ < ∞, (b) με,− → ∞ as ε → 0. The jump transition of the form N → N occurs in the above system, when a new customer tries to enter the system when the buffer is full. According to the above description this customer will be lost for the system. Such an event can be interpreted as rare due to the condition B14 . Let 0 = τˆε,0 < τˆε,1 < · · · be successive moments of jumps for the semiˆ τˆε,n ), n = 0, 1, . . . be the corresponding discrete time Markov process ηˆε (t), ηˆε,n = η( embedded Markov chain, κˆε,n = τˆε,n − τˆε,n−1, n = 1, . . . be inter-jump times for the semi-Markov process ηˆε (t), and νˆε (t) = max(n ≥ 1 : τˆε,n ≤ t) be the number of jumps for the semi-Markov process η(s) ˆ in the time interval [0, t]. Let us also consider the Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ), n = 0, 1, . . . ˇ × [0, ∞) × {0, 1}, where X ˇ = X, ˆ and components, which has the phase space X ⎧ ⎪ ⎨ ηˇε,n = ηˆε,n, ⎪ κˇε,n = κˆε,n /wε, ⎪ ⎪ ρˇ ε,n = I(ηˆε,n−1 = N, ηˆε,n = N), ⎩

(7.5)

where 0 < wε → ∞ as ε → 0 is some normalisation function, which we will define below. It is natural to define the normalised first-rare-event time for the system Sε , which can also be called the first failure time, as follows: ξSε =

νˇ ε 

κˇε,n =

νˆ ε 

n=1

κˆε,n /wε,

(7.6)

n=1

where νˇε = min(n ≥ 1 : ηˇε,n−1 = N, ηˇε,n = N).

(7.7)

The Markov chain ηˇε,n has the matrix of transition probabilities, Pˇ ε = Pˆ ε .

(7.8)

The condition B14 obviously implies that there exists ε  ∈ (0, 1] such that pε,± > 0 for ε ∈ (0, ε ] and, (7.9) pε,+ → 0 as ε → 0. This relation implies that, Pˇ ε → Pˇ 0 as ε → 0,

(7.10)

where the limiting matrix Pˇ 0 has the following form: ⎡ 0 ⎢ ⎢ 1 ⎢ ˇP0 = ⎢ · · · ⎢ ⎢ 0 ⎢ ⎢ 0 ⎣

1 0 0 0 ··· ··· ··· 0 ··· 0···

0 0 ··· 1 0

··· ··· ··· 0 1

0 ⎤⎥ 0 ⎥⎥ · · · ⎥⎥ . 0 ⎥⎥ 0 ⎥⎦

(7.11)

168

7 Perturbed M/M-type queuing systems

ˇ and the matrix of Let ηˇ0,n, n = 0, 1, . . . be a Markov chain with the phase space X ˇ transition probabilities P0 . ˇ of the limiting Markov chain ηˇ0,n is one class of If N = 1, the phase space X communicative states. ˇ = X ∪ X, ¯ where X = {0, 1} is a closed class of If N > 1, the phase space X ¯ communicative states and X = {2, . . . , N } is a class of transient states. First, consider the simplest case, where N = 1. In this case, we can directly apply Theorem 2.1 to the Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ). The matrices Pˇ ε and Pˇ 0 take the following forms:     0 1 ˇPε = 0 1 ˇ . (7.12) and P0 = pε,− pε,+ 1 0 The condition B1 is obviously satisfied, i.e., the Markov chains ηˇε,n are asymptotically uniformly ergodic. The stationary probabilities for the Markov chain ηˇε,n , πε,0 =

με,− με,− + με,+ pε,− 1 = = , πε,1 = . pε,− + 1 2με,− + με,+ pε,− + 1 2με,− + με,+

(7.13)

The condition B14 implies that, πε,0, πε,1 →

1 as ε → 0. 2

(7.14)

In this case, the probabilities,

pˇε,i = Pi { ρˇ ε,1 = 1} =

0 for i = 0, pε,+ for i = 1.

(7.15)

The condition B14 and the relation (7.14) imply that pˇε,1 > 0, for ε ∈ (0, ε ], and, for i = 0, 1, (7.16) pˇε,i → 0 as ε → 0. Thus, the condition A1 is satisfied. Also, the corresponding averaged probability and the function uε are given by the following relations: pˇε = πε,0 pˇε,0 + πε,1 pˇε,1 =

pε,+ , pε,− + 1

(7.17)

and pε,− + 1 pε,+ 2με,− + με,+ 2 = ∼ με,− as ε → 0. με,+ με,+

uε = pˇ−1 ε =

(7.18)

7.1 Rare events for queuing systems with bounded buffers

169

Also, the distribution functions,

1 − e−με,+ wε u for u ≥ 0, i = 0, ˇ Fε,i (u) = Pi { κˇε,1 ≤ u} = 1 − e−λε wε u for u ≥ 0, i = 1

(7.19)

and, thus, the averaged distribution function, Fˇε (u) = πε,0 Fˇε,0 (u) + πε,1 Fˇε,1 (u) pε,− 1 (1 − e−με,+ wε u ) + (1 − e−λε wε u ), u ≥ 0. = pε,− + 1 pε,− + 1

(7.20)

In this case, for u ≥ 0, Pi { κˇε,1 ≤ u, ρˇ ε,1

−με,+ wε u )I( j = 0) for j = 0, 1, i = 0, ⎧ ⎪ ⎨ (1 − e ⎪ for j = 0, i = 1, = 1} = (1 − e−λε wε u )pε,− ⎪ ⎪ (1 − e−λε wε u )pε,+ for j = 1, i = 1. ⎩

(7.21)

Thus, the condition C2 is satisfied, and, according to Remark 2.6, the condition C1 reduces to the assumption that the following asymptotic relation holds, for i = 0, 1 and δ > 0, Pi { κˇε,1 > δ} = 1 − Fˇε,i (δ) → 0 as ε → 0. (7.22) The condition B14 and the relation (7.21) imply that the relation (7.22) holds for any function 0 < wε → ∞ as ε → 0. In this case the averaged Laplace transforms, φˇε (s) =

pε,− 1 1 1 + , s ≥ 0. pε,− + 1 1 + s/με,+ wε pε,− + 1 1 + s/με,+ wε

(7.23)

1 Let, as in Sect. 5.2.2, 1+s = 1 − s + o(s) be the asymptotic Taylor expansion of the 1 first order for the function 1+s at the point 0. The relation (7.23) and the condition B13 imply that the following asymptotic relation takes place, for s > 0,

  1 1 ) + πε,1 (1 − ) uε (1 − φε (s)) = uε πε,0 (1 − 1 + s/με,+ wε 1 + s/λε wε  = uε πε,0 (s/με,+ wε + o(s/με,+ wε ))  + πε,1 (s/λε wε + o(s/λε wε )) με,− + με,+ ∼ uε πε,0 s/με,+ wε ∼ s μ2ε,+ wε με,− ∼ s 2 as ε → 0. (7.24) με,+ wε From the relation (7.24) it follows that condition D2 , in this case, is equivalent to the following condition: D20 : Aε =

πε,0 uε με,+ wε

=

με,− 2 w με,+ ε

→ A ∈ (0, ∞) as ε → 0.

170

7 Perturbed M/M-type queuing systems

Moreover, in this case, the corresponding limiting cumulant A(s) = As, and, thus, the corresponding limiting distribution G(u) is exponential with parameter A−1 . It is also useful to note that the condition D20 is automatically satisfied if to choose the normalisation function wε = με,− /μ2ε,+ . In this case, A = 1. The variant of Theorem 2.1, which corresponds to the statements (i) and (ii) of this theorem, takes the following form. Lemma 7.1 Let N = 1 and the condition B14 is satisfied. Then: (i) The condition D20 is necessary and sufficient for the fulfilment (for some or any initial distributions q¯ε , respectively, in the statements of necessity and sufficiency) d

of the asymptotic relation, ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution function not concentrated at zero. In this case: (ii) The limiting random variable ξ0 has an exponential distribution with the parameter A−1 . Let us now consider the case, where N > 1. In this case, the condition B1 is not satisfied for the Markov chains ηˇε,n . However, we can use the results presented in Sect. 4.1.4.2. We can choose the phases spaces, ˜ =X ˇ =X ˆ and X = {0, 1}. X

(7.25)

As mentioned above, the condition B14 implies that there exist ε  ∈ (0, 1] such ˇ is one class of communicative states for that, for every ε ∈ (0, ε ], the phase space X the Markov chain ηˇε,n . Obviously, we can assume that ε  = 1. ˇ and Therefore, the condition B6 is satisfied and, thus, for any subset ∅  D ⊆ X ˇ i ∈ X, Pi { νˇε,D < ∞} = 1, (7.26) where νˇε,D = min(n ≥ 1 : ηˇε,n ∈ D).

(7.27)

The relation (7.26) makes it possible to define the successive times of hitting the Markov chain ηˇε,n in the set X, νˇε,X,n = min(k > νˇε,X,n−1, ηˇε,k ∈ X), n = 1, 2, . . . , where νˇε,X,0 = 0.

(7.28)

The corresponding Markov renewal process with transition period (η˜ε,n, κ˜ε,n , ˜ for the initial state η˜ε,0 , and the phase ρ˜ ε,n ), n = 0, 1, . . . (with the phase space X, space X, for the random variables η˜ε,n, n ≥ 1) can be constructed using the following recurrent relations: ⎧ η˜ε,n = ηˇε,νˇ ε,X, n , n = 1, 2, . . . , η˜ε,0 = ηˇε,0, ⎪ ⎪ νˇ ε,X, n ⎨ ⎪ κ˜ε,n = k= κˇ , n = 1, 2, . . . , κ˜ε,0 = κˇε,0, νˇ +1 ε,k ⎪ ε,X, n−1 ⎪ ⎪ ρ˜ ε,n = 1 − νˇ ε,X, n (1 − ρˇ ε,k ), n = 1, 2, . . . , ρ˜ ε,0 = ρˇ ε,0 . k=νˇ ε,X, n−1 +1 ⎩

(7.29)

7.1 Rare events for queuing systems with bounded buffers

171

Also, as in Sect. 4.1.3.1, the corresponding shifted Markov renewal process can be defined by the following relation: (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . . .

(7.30)

Obviously, the following inequality holds, for every ε ∈ (0, ε ]: ξ˜ε,− ≤ ξSε ≤ ξ˜ε,+ . where ξ˜ε,− =

ν˜ ε −1

κ˜ε,k , ξ˜ε,+ =

k=1

ν˜ ε 

(7.31)

κ˜ε,k ,

(7.32)

k=1

and ν˜ε = min(n ≥ 1 : ρ˜ ε,n = 1).

(7.33)

Let us show that the conditions C1 (for the Markov renewal processes (ηε,n, κε,n , ρε,n )) and A5 , B6 , B7 , C8 (for the Markov renewal processes (ηˇε,n, κˇε,n , ρˇ ε,n )) are satisfied and, thus, Lemma 4.6 can be applied. The relation (7.9) implies that the condition B7 holds. Thus, the condition B1 holds for the Markov chains ηε,n , which are asymptotically uniformly ergodic. The Markov chain ηε,n has the phase space X = {0, 1} and the matrix of transition probabilities,   0 1 . (7.34) Pε = pε,− pε,+ Obviously,

 Pε → P0 =

0 1 1 0

 as ε → 0.

(7.35)

The Markov chain ηε,n has the stationary probabilities, πε,0 =

με,− με,− + με,+ pε,− 1 = = , πε,1 = , pε,− + 1 2με,− + με,+ pε,− + 1 2με,− + με,+

(7.36)

and, thus, πε,0, πε,1 → ˇ In this case, for i ∈ X, pˇε,i = Pi { ρˇ ε,1 = 1} =



1 as ε → 0. 2

0 for i = 0, . . . , N − 1, pε,+ for i = N.

The relations (7.9) and (7.38) imply that the condition A5 is satisfied. ˜ In this case, for u ≥ 0 and i ∈ X,

1 − e−με,+ wε u for i = 0, Pi { κˇε,1 ≤ u} = 1 − e−λε wε u for i = 1, . . . , N.

(7.37)

(7.38)

(7.39)

172

7 Perturbed M/M-type queuing systems

Therefore, the condition B14 implies that the condition C8 is satisfied, for any function 0 ≤ wε → 0 as ε → 0. ˇ Let us introduce the hitting times, for j ∈ X, νˇε, j = min(n ≥ 1 : ηˇε,n = j),

(7.40)

νε, N = min(n ≥ 1 : ηˇε,n−1 = N, ηˇε,n = N).

(7.41)

and ˜ take the following form: In this case, the probabilities pε,i = Pi { ρ˜ ε,1 = 1}, i ∈ X pε,i = Pi { ρ˜ ε,1 = 1} = where, for i = 2, . . . , N,

⎧ ⎪ ⎨ ⎪

0 for i = 0, pε,+ Pε,2, N for i = 1, ⎪ ⎪ Pε,i, N for i = 2, . . . m, ˜ ⎩

Pε,i, N = Pi { νε, N < νˇε,1 }.

(7.42)

(7.43)

The condition B6 implies that, for ε ∈ (0, ε ], the probabilities Pε,i N , i = 2, . . . , N are the only positive solution for the following system of linear equations: Pε,i, N = pε,− Pε,i−1, N + pε,+ Pε,i+1, N , 2 ≤ i ≤ N,

(7.44)

where Pε,1, N = 0, Pε, N +1, N = 1. This system can be rewritten in the equivalent form of the recurrent relations, Pε,i+1, N − Pε,i, N = (Pε,i, N − Pε,i−1, N )

or Pε,i+1, N − Pε,i, N = Pε,2, N (

pε,− , 2 ≤ i ≤ N, pε,+

pε,− i−1 ) , 2 ≤ i ≤ N. pε,+

(7.45)

(7.46)

Summing from i = 2 to i = N in the relation (7.46), we get the following relation: 1 − Pε,2, N = Pε,2, N

N  pε,− i−1 ( ) , p ε,+ i=2

(7.47)

which gives the following explicit formula: Pε,2, N =

pε,− pε,+ pε,− N ( pε,+ )

1− 1−

pε,+

pε,+ N −1 1 − pε,− =( ) . p pε,− 1 − ( pε,+ )N ε,−

(7.48)

Also, summing from i = 2 to i = j − 1 in the relation (7.46), we obtain the following formula, for 2 ≤ j ≤ N: p

Pε, j, N = Pε,2, N ·

1 − ( pε,− ) j−1 ε,+ 1−

pε,− pε,+

p

= Pε,2, N

ε,+ j−1 pε,+ 2−j 1 − ( pε,− ) ·( ) . p pε,− 1 − pε,+ ε,−

(7.49)

7.1 Rare events for queuing systems with bounded buffers

173

Therefore, the averaged rare-event probability pε for the Markov chain ηε,n , and the function uε = p−1 ε take the following forms: pε = πε,0 pε,0 + πε,1 pε,1 = πε,1 pε,1 pε,+

pε,+ N −1 1 − pε,− 1 = · pε,+ ( ) , p 1 + pε,− pε,− 1 − ( pε,+ )N ε,− and −1 ∼ uε = p−1 ε = (πε,1 pε,1 )

2 N με,− → ∞ as ε → 0. N με,+

(7.50)

(7.51)

Consider the random variables, κ˜ε,1 =

νˇ ε,X,1

κˇε,k =

k=1

νˇ ε,X,1

κˆε,k /wε

(7.52)

k=1

˜ and the distribution functions, for j ∈ X, νˇ ε,X,1

Fˆε, j (u) = P j {

κˆε,k ≤ u}, u ≥ 0.

(7.53)

k=1

It is useful to note that, P0 { νˇε,X,1 = 1} = 1.

(7.54)

In this case, the distribution functions,

1 − e−με,+ wε u, u ≥ 0 for u ≥ 0, i = 0, Fε,i (u) = Pi {κε,1 ≤ u} = ˆ for u ≥ 0, i = 1. Fε,1 (uwε ) The distribution function Fε,0 (·) is exponential with parameter με,+ wε , and, thus, its two first moments are, E0 κε,1 =

1 2 2 , E0 κε,1 = . με,+ wε (με,+ wε )2

(7.55)

In this case, the quantities eˇε,i,α,β appearing in the condition C9 and the relation (4.109) take the following form, for i = 1 and α = β = 2, √ 4! 4! 1 4 4 12 eˇε,1,2,2 = (E1 κˇε,1 ∨ max E j κˇε,1 ) = ( 4 4 ) 2 = 2 2 (7.56) ˜ λε wε λε wε j ∈X\X and, thus, by Lemma 4.7, 2 E1 κε,1 ≤ eˇε,1,2,2 a p, N,2,2 ≤

and

√ 4!a p, N,2,2

1 λε2 wε2

(7.57)

174

7 Perturbed M/M-type queuing systems

E1 κε,1 ≤



2 ≤ E1 κε,1

 5a p, N,2,2

1 , λε wε

(7.58)

where the constant a p, N,2,2 is given by relation (4.108). Let θˆε,n, n = 1, 2, . . . be a sequence of i.i.d. random variables with the distribution function Fˆε (·) = πε,0 Fˆε,0 (·) + πε,1 Fˆε,1 (·). (w) = θˆε,n /wε, n = 1, 2, . . . is also a sequence of i.i.d. random In this case, θ ε,n variables with the distribution function Fε (·) = πε,0 Fε,0 (·) + πε,1 Fε,1 (·). Using relations (7.36), (7.37), (7.55), and (7.58) and taking into account that, by condition B13 , λε → ∞ as ε → 0, we get, (w)

Eθ ε,1 = πε,0 E0 κ˜ε,1 + πε,1 E1 κ˜ε,1

=

pε,− 1 1 Cε + E1 κε,1 = , pε,− + 1 με,+ wε pε,− + 1 wε

(7.59)

where pε,− 1 1 + E1 κε,1 wε pε,− + 1 με,+ pε,− + 1 1 ∼ Cε,◦ = as ε → 0. 2με,+

Cε =

(7.60)

The condition B14 implies that, 0 < lim Cε,◦ ≤ lim Cε◦ < ∞. ε→0

ε→0

(7.61)

Also, using relations (7.36), (7.37), and (7.57), we get, (w)

2 2 E(θ ε,1 )2 = πε,0 E0 κε,1 + πε,1 E1 κε,1

=

pε,− 2 1 Bε 2 + E1 κε,1 = 2, 2 pε,− + 1 (με,+ wε ) pε,− + 1 wε

(7.62)

where pε,− 2 1 + E1 κε,1 wε2 2 pε,− + 1 με,+ pε,− + 1 √ pε,− 2 1 1 ≤ Bε,◦ = + 4!a p, N,2,2 2 2 pε,− + 1 με,+ pε,− + 1 λε

Bε =

(7.63)

The condition B14 implies that, 0 < lim Bε,◦ ≤ lim Bε,◦ < ∞. ε→0

Consider the random variables,

ε→0

(7.64)

7.1 Rare events for queuing systems with bounded buffers

θ ε(w) =

[u ε] 

175

(w) θ ε,n .

(7.65)

w◦,ε = Cε,◦ uε .

(7.66)

n=1

Let us choose the normalisation function,

The relations (7.59), (7.60), and (7.61) imply that, (w◦ )

Eθ ε

(w◦ ) = [uε ]Eθ ε,1 =

[uε ]Cε → 1 as ε → 0. Cε,◦ uε

(7.67)

Also, the relations (7.61), (7.62), (7.63), and (7.64) imply that, (w◦ ) Varθ ε(w◦ ) = [uε ]Varθ ε,1 (w◦ ) 2 ≤ [uε ]E(θ ε,1 ) ≤

[uε ]Bε,◦ → 0 as ε → 0. 2 u2 Cε,◦ ε

(7.68)

The relations (7.67) and (7.68) imply that the following asymptotic relation takes place: [u ε]  (w◦ ) P θ ε(w◦ ) = θ ε,n −→ 1 as ε → 0. (7.69) n=1

Let us now consider the case, where the function 0 < wε → ∞ as ε → 0 satisfies the following condition: D21 : Aε =

Cε,◦ uε wε

=

N με,− N +1 w με,+ ε

→ A ∈ (0, ∞) as ε → 0.

Under the condition B14 , the condition D21 is necessary and sufficient for the fulfilment of the condition D1 , which in this case takes the form of the following asymptotic relation, [u ε]  (w (w (w) θ ε,n ⇒ θ 0 ) as ε → 0, (7.70) θε ) = n=1 (w θ0 )

where is a non-negative random variable with distribution function not concentrated at zero. (w Moreover, in this case the limiting random variable θ 0 ) = A with probability 1. Indeed, let the conditions B14 and D21 be satisfied. In this case, the relation (7.69) and the asymptotic relation appearing in the condition D21 imply that, θ ε(w) =

[u ε] 

(w) θ ε,n =

n=1

=

[u ε]  n=1

[u ε] 

θˆε,n /wε

n=1 (w◦ ) θ ε,n

Cε,◦ uε Cε,◦ uε P = θ ε(w◦ ) −→ A as ε → 0. wε wε

(7.71)

176

7 Perturbed M/M-type queuing systems

Thus, the above sufficiency statement is true. Let us assume that the condition B14 is satisfied and the relation (7.70) holds, but the condition D20 is not satisfied. In this case, two subsequences 0 < εk , εk ≤ 1 can be selected such that: (a) εk , εk → 0 as n → ∞, (b)

Cε  ,◦ uε  k

k

wε  k  A .

→ A ∈ [0, ∞] as k → ∞, (c)

[uε  ]

=

[uε  ]

k

θ ε(w)  ,n k

n=1

= θ ε(w ◦ ) k

k

=

◦) θ ε(w ,n k

n=1

Cεk ,◦ uεk wεk

k

wε 

k

→ A ∈

k

[0, ∞] as k → ∞, (d) A  In this case, the relation (7.69) implies that θ ε(w)  k

Cε  ,◦ uε 

Cεk ,◦ uεk wεk

d

−→ A as k → ∞

(7.72)

and [uε  ]

θ ε(w)  k

=

k n=1

= θ ε(w◦ ) k

[uε  ]

θ ε(w) ,n k

=

k n=1

Cεk,◦ uεk w

εk

◦) θ ε(w,n k

Aεk,◦ uεk wεk

d

−→ A as k → ∞.

(7.73)

The above two relations contradict to the relation (7.70). Thus, the above necessity statement is also true, i.e., the condition D20 is satisfied. (w In this case, according to the relation (7.71), the limiting random variable θ 0 ) = A with probability 1. The condition D21 plays, in this case, the role of the condition D1 . Finally, by Lemma 4.7, the following relation takes place, for ε ∈ (0, ε ]: √ 4! 2 E1 κε,1 ≤ eˇε,1,2,2 a p, N,2,2 = ( 2 2 )a p, N,2,2 λε wε √ 4! uε = ( 2 )a p, N,2,2 πε,1 pε,1 = o1 (ε)pε,1, (7.74) wε λε wε where

√ 4! uε o1 (ε) = ( 2 )a p, N,2,2 πε,1 → 0 as ε → 0. w λε wε ε

(7.75)

The relations (7.74) and (7.75) imply that the condition C5 is satisfied (for α, β = 2), and, thus, condition C1 holds for the Markov renewal processes (ηε,n, κε,n, ρε,n ).

7.1 Rare events for queuing systems with bounded buffers

177

Therefore, according to Lemma 4.6, all conditions of Theorem 4.4 are satisfied for the Markov renewal processes (ηˇε,n, κˇε,n, ρˇ ε,n ). In this case, Theorem 4.4 takes the following form. Theorem 7.1 Let N > 1 and the condition B14 is satisfied. Then: (i) Condition D21 is necessary and sufficient for holding (for some or any initial distributions q¯ε , respectively, in statements of necessity and sufficiency) of the d

asymptotic relation ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated at zero. (ii) The limiting random variable ξ0 has an exponential distribution with the parameter A−1 .

7.1.2 Queuing Systems with Asymptotically Unbounded Buffers Let now Sε be, for every ε ∈ (0, 1], a standard M/M queuing system described in Sect. 4.3.1.1, but with a buffer size Nε , which depend on ε in such way that Nε → ∞ as ε → 0. The state of system at the moment t is represented by the random variable ηˆε (t), which is the number of customers in the buffer at this moment. The process ηˆε (t) is a ˆ ε = {0, . . . , Nε } birth–death-type semi-Markov process, which has the phase space X and transition probabilities, ˆ ε, Qˆ ε,i j (t) = pˆε,i j Fˆε,i (t), t ≥ 0, i, j ∈ X

(7.76)

The (Nε + 1) × (Nε + 1) matrix of transition probabilities for the corresponding embedded Markov chain ηˆε,n has the following form: ⎡ 0 ⎢ ⎢ pε,− ⎢ Pε =  pˆε,i j  = ⎢⎢ · · · ⎢ 0 ⎢ ⎢ 0 ⎣ where, pε,+ =

1 0 0 0 pε,+ 0 ··· ··· ··· · · · 0 pε,− ··· 0··· 0

··· ··· ··· 0 pε,−

0 ⎤⎥ 0 ⎥⎥ · · · ⎥⎥ , pε,+ ⎥⎥ pε,+ ⎥⎦

με,+ με,− , pε,− = , λε = με,+ + με,− λε λε

and the distribution functions of sojourn times have the following form:

1 − e−με,+ t for t ≥ 0, i = 0, ˆ Fε,i (t) = 1 − e−λε t for t ≥ 0, i = 1, . . . , Nε .

(7.77)

(7.78)

(7.79)

178

7 Perturbed M/M-type queuing systems

It is useful to note that the semi-Markov process η˜ε (t) is, in fact, a homogeneous Markov process. However, we do prefer consider it as a semi-Markov process, since its trajectories can include jump transitions of the form Nε → Nε . In the standard model of continuous time Markov chain jump transitions of the form i → i are usually excluded. Let us consider the case, where the following condition is satisfied: B15 : (a) 0 < limε→0 με,± ≤ limε→0 με,± < ∞, (b) limε→0 ∞ as ε → 0.

με,+ με,−

< 1, (c) 1 ≤ Nε →

The jump transition of the form Nε → Nε occurs in the above system, when a new customer tries to enter the system in the situation where the buffer is full. According to the above description this customer will be lost for the system. Such an event can be interpreted as rare due to the condition B15 . Let 0 = ζˆε,0 < ζˆε,1 < · · · be sequential moments of jumps for the semi-Markov ˆ ζˆε,n ), n = 0, 1, . . . be the corresponding discrete time emprocess ηˆε (t), ηˆε,n = η( bedded Markov chain, κˆε,n = ζˆε,n − ζˆε,n−1, n = 1, . . . be inter-jump times for the semi-Markov process ηˆε (t), and νˆε (t) = max(n ≥ 1 : ζˆε,n ≤ t) be the number of jumps for the semi-Markov process η(s) ˆ in the time interval [0, t]. Let us consider the Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ), n = 0, 1, . . . which ˇε = X ˆ ε , and components, ˇ ε × [0, ∞) × {0, 1}, where X has the phase space X ⎧ ⎪ ⎨ ηˇε,n = ηˆε,n, ⎪ κˇε,n = κˆε,n /wε, ⎪ ⎪ ρˇ ε,n = I(ηˆε,n−1 = Nε, ηˆε,n = Nε ), ⎩

(7.80)

where 0 < wε → ∞ as ε → 0 is some normalisation function, which we will define below. As in Sect. 7.1.1, let us define the normalised first-rare-event time for the system Sε , which can also be called the first failure time, as follows: ξSε =

νˇ ε  n=1

κˇε,n =

νˆ ε 

κˆε,n /wε,

(7.81)

n=1

where νˇε = min(n ≥ 1 : ηˇε,n−1 = Nε, ηˇε,n = Nε ).

(7.82)

In this case, we can choose the phases spaces, ˜ = {1, . . . , m}, X ˜ where m˜ ≥ 2 and X = {0, 1}.

(7.83)

˜ ⊆X ˇ ε , for Since, Nε → ∞ as ε → 0, it can be assumed that Nε ≥ m˜ and, thus, X ε ∈ (0, 1]. ˜ Let us assume that the initial state ηˇε,0 ∈ X. The condition B15 obviously implies that, 0 < lim pε,+ ≤ lim pε,+ < ε→0

ε→0

1 . 2

(7.84)

7.1 Rare events for queuing systems with bounded buffers

179

In this case, there exist ε  ∈ (0, 1] such that, for every ε ∈ (0, ε ], the phase space ˇ X is one class of communicative states for the Markov chain ηˇε,n . Obviously, we can assume that ε  = 1. ˇ ε, In this case, the condition B5 holds and, thus, for i ∈ X Pi { νˇε,X < ∞} = 1,

(7.85)

νˇε,X = min(n ≥ 1 : ηˇε,n ∈ X).

(7.86)

where The relation (7.85) makes it possible to define the successive times of hitting the Markov chain ηˇε,n in the set X, νˇε,X,n = min(k > νˇε,X,n−1, ηˇε,k ∈ X), n = 1, 2, . . . , where νˇε,X,0 = 0.

(7.87)

The corresponding Markov renewal process with transition period (η˜ε,n, κ˜ε,n , ˜ for the initial state η˜ε,0 , and the phase ρ˜ ε,n ), n = 0, 1, . . . (with the phase space X, space X, for the random variables η˜ε,n, n ≥ 1) can be constructed using the following recurrent relations: ⎧ η˜ε,n = ηˇε,νˇ ε,X, n , n = 1, 2, . . . , η˜ε,0 = ηˇε,0, ⎪ ⎪ νˇ ε,X, n ⎨ ⎪ κ˜ε,n = k= κˇ , n = 1, 2, . . . , κ˜ε,0 = κˇε,0, νˇ +1 ε,k ⎪ ε,X, n−1 ⎪ ⎪ ρ˜ ε,n = 1 − νˇ ε,X, n (1 − ρˇ ε,k ), n = 1, 2, . . . , ρ˜ ε,0 = ρˇ ε,0 . k=νˇ ε,X, n−1 +1 ⎩

(7.88)

Also, as in Sect. 4.1.3.1, the corresponding shifted Markov renewal process can be defined by the following relation: (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . . .

(7.89)

Obviously, the following inequality holds, for every ε ∈ (0, 1]: ξ˜ε,− ≤ ξSε ≤ ξ˜ε,+, where ξ˜ε,− =

ν˜ ε −1 k=1

κ˜ε,k , ξ˜ε,+ =

ν˜ ε 

(7.90)

κ˜ε,k ,

(7.91)

k=1

and ν˜ε = min(n ≥ 1 : ρ˜ ε,n = 1).

(7.92)

Let us show that conditions of Theorem 4.4 are satisfied. The Markov chain ηε,n has the phase space X = {0, 1} and the matrix of transition probabilities,   0 1 . (7.93) Pε = pε,− pε,+

180

7 Perturbed M/M-type queuing systems

The relation (7.84) implies that condition B1 holds for Markov chains ηε,n , which are asymptotically uniformly ergodic. The stationary probabilities for the Markov chain ηε,n , πε,0 =

με,− με,− + με,+ pε,− 1 = = , πε,1 = pε,− + 1 2με,− + με,+ pε,− + 1 2με,− + με,+

and, thus, for i = 0, 1,

0 < lim πε,i ≤ lim πε,i < 1.

(7.95)

ε→0

ε→0

(7.94)

In this case,

pˇε,i = Pi { ρˇ ε,1 = 1} =

0 for i = 0, . . . , Nε − 1, pε,+ for i = Nε .

(7.96)

˜ coincide for It is easy to see that the probabilities pε,i = Pi { ρ˜ ε,1 = 1}, i ∈ X the above queuing system and the queue system (with a bounded buffer of size Nε ), which was the subject of research in Sect. 7.1.1. ˜ take the following forms: Thus, the probabilities pε,i, i ∈ X pε,i = Pi { ρ˜ ε,1 = 1} =

⎧ ⎪ ⎨ ⎪

0 for i = 0, pε,+ Pε,2, Nε for i = 1, ⎪ ⎪ Pε,i, N for i = 2, . . . , m, ¯ ε ⎩

(7.97)

where, by the relation (7.48), Pε,2, Nε =

pε,− pε,+

1− p

p

1 − ( pε,− ) Nε ε,+

ε,+ pε,+ Nε −1 1 − pε,− =( ) , p pε,− 1 − ( pε,+ ) Nε ε,−

(7.98)

and, by the relation (7.49), for j = 2, . . . , Nε , p

Pε, j, Nε = Pε,2, Nε ·

1 − ( pε,− ) j−1 ε,+ 1−

pε,− pε,+

p

= Pε,2, Nε

ε,+ j−1 pε,+ 2−j 1 − ( pε,− ) ·( ) . p pε,− 1 − pε,+ ε,−

(7.99)

The condition B15 and the relations (7.84), (7.98), (7.99) imply that pε,i > 0, i ∈ ˜ for ε ∈ (0, ε ] and, for every i ∈ X, ˜ X, pε,i → 0 as ε → 0.

(7.100)

Therefore the condition A1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n ) and the condition A3 (for the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n )) are satisfied. In this case, the averaged rare-event probability pε for the Markov chain ηε,n and function uε = p−1 ε take the following forms:

7.1 Rare events for queuing systems with bounded buffers

181

pε = πε,0 pε,0 + πε,1 pε,1 = πε,1 pε,1 p

=

ε,+ pε,+ Nε −1 1 − pε,− 1 · pε,+ ( ) , p 1 + pε,− pε,− 1 − ( pε,+ ) Nε ε,−

and uε = p−1 ε ∼

1 + pε,− pε,− Nε ( ) → ∞ as ε → 0. pε,− − pε,+ pε,+

(7.101)

(7.102)

Let us consider the random variables, κ˜ε,1 =

νˇ ε,X,1

κˇε,k .

(7.103)

k=1

ˇ ε, Let us also introduce the distribution functions, for j ∈ X νˇ ε,X,1

Fˆε, j (u) = P j {

κˆε,k ≤ u}, u ≥ 0.

(7.104)

k=1

It is useful to note that, P0 { νˇε,X = 1} = 1.

In this case, the distribution functions,

1 − e−με,+ wε u for u ≥ 0, j = 0, Fε,i (u) = Pi { κ˜ε,1 ≤ u} = Fˆε,i (uwε ) for u ≥ 0, i = 1, . . . , Nε .

(7.105)

(7.106)

In what follows, we use the following stochastic representation: τε,i

⎧ for i = 0, ⎪ ⎨ ζε,+ ⎪ for i = 1, 2, = ζε + I(ιε = +1)τε,i+1 ⎪ ⎪ ζε + I(ιε = −1)τε,i−1 + I(ιε = +1)τε,i+1 for 2 < i ≤ Nε, ⎩ d

(7.107)

where: (a) the random variable τε,i has the distribution function Pi { κ˜ε,1 ≤ ·}, for d

i = 0, 1, . . . , Nε , (b) the random variable τε, Nε +1 = τε, Nε , (c) the random variable ζε is exponentially distributed with parameter λε wε , (d) the random variable ζε,+ is exponentially distributed with parameter με,+ wε , (d) the random variable ιε takes values −1 and +1 with probabilities, respectively, pε,− and pε,+ , (e) the random variables ζε , ιε , τε,i−1 , and τε,i+1 are mutually independent, for every i = 1, . . . , Nε . The condition B15 implies that there exists εn ∈ (0, ε ] such that, for ε ∈ (0, εn ], moments E(n) ε,i, Nε < ∞, i = 1, . . . , Nε for n ≥ 1 (see, for example Silvestrov (1980a) or Gyllenberg and Silvestrov (2008)). The representation (7.107) implies that, E0 κ˜ε,1 =

n! , n ≥ 1. (με,+ wε )n

(7.108)

182

7 Perturbed M/M-type queuing systems

The representation (7.107) also implies that the following recurrent relations hold: n E1 κ˜ε,1

! n−1  n k! n! n−k = + · pε,+ E2 κ˜ε,1 , n ≥ 1. (λε wε )n k=0 k (λε wε )k

Let us denote,

(n)

n Eε,i, N = Ei κ˜ε,1 , n ≥ 1, 2 ≤ i ≤ Nε . ε

(7.109)

(7.110)

The recurrent relations (7.109) can be rewritten in the following form: (n)

(n)

(n)

Eε,i, N = Fε,i, N + pε,− Eε,i−1, N ε ε ε

+ pε,+ E(n) ε,i+1, Nε , 2 ≤ i ≤ Nε , n ≥ 1, where,

(k)

(k)

(k)

Eε,1, N = 0, Eε, N +1, N = Eε, N , N , n ≥ 1, ε ε ε ε ε

(7.111)

(7.112)

and (n) Fε,i, N ε

! n  n (n−k) (n−k) = E ρkε (pε,− Eε,i−1, N + pε,+ Eε,i+1, N ) ε ε k k=1 ! n  n k! (n−k) = (E(n−k) ε,i, Nε − Fε,i, Nε ), 2 ≤ i ≤ Nε , n ≥ 1. k k (λ w ) ε ε k=1

(7.113)

In particular, the recurrent relations (7.111) for expectations (n = 1) take the following form: (1)

Eε,i, N = ε

1 (1) + pε,− E(1) ε,i−1, Nε + pε,+ Eε,i+1, Nε , 2 ≤ i ≤ Nε , λε wε

(7.114)

(1) (1) where E(1) ε,1, Nε = 0, Eε, Nε +1, Nε = Eε, Nε , Nε . These recurrent relations be rewritten in the following equivalent form: (1)

1

(1)

Eε,i+1, N − Eε,i, N = − ε ε λε wε pε,+ (1) + (E(1) ε,i, Nε − Eε,i−1, Nε )

pε,− , 2 ≤ i ≤ Nε, pε,+

(7.115)

or (1)

(1)

1

Eε,i+1, N − Eε,i, N = − ε ε λε wε pε,+

+ E(1) ε,2, Nε (

i−1  pε,− k−1 ( ) p k=1 ε,+

pε,− i−1 ) , 2 ≤ i ≤ Nε . pε,+

Taking i = Nε in the relation (7.116), we get the following relation:

(7.116)

7.1 Rare events for queuing systems with bounded buffers ε,− N ε −1 1 − ( pε,+ ) pε,− Nε −1 1 0=− + E(1) ( ) , pε,− ε,2, N ε λε wε pε,+ pε,+ 1− p

183

p

(7.117)

ε,+

which gives the following explicit formula: (1)

E2 κ˜ε,1 = Eε,2, N

ε

=

ε,− N ε −1 1 − ( pε,+ ) pε,+ Nε −1 1 ·( ) pε,− λε wε pε,+ pε,− 1− p

=

pε,+ Nε −1 − ( pε,− ) . pε,− pε,+ − 1

p

ε,+

1 1 λε wε pε,+

(7.118)

In addition, summing from i = 2 to i = j − 1 in the relation (7.116), we obtain the following formula, for 2 ≤ j ≤ Nε : (1)

E j κ˜ε,1 = Eε, j, N ε

=

(1) Eε,2, N ε

j−1 1 − ( pε,− )i−1 j−1    pε,− i−1  1 pε,+ (1) + ( ) + Eε,2, Nε · λε wε pε,+ i=2 ppε,− − 1 pε,+ i=2 ε,+

=

1 · λε wε pε,+

j−1 (1 − ( pε,+ ) Nε −i )  pε,− i=1

pε,− pε,+

−1

.

(7.119)

Also, the relation (7.109), taken for n = 1, and the relation (7.118) imply that, 1 + pε,+ E2 κ˜ε,1 λε wε p ) Nε −1 1 − ( pε,+ 1 ε,− = (1 + ). pε,− λε wε pε,+ − 1

E1 κ˜ε,1 = E1 κε,1 =

(7.120)

The condition B15 and the relations (7.108), (7.118), and (7.119), and (7.120) ˜ imply that, for δ > 0 and j ∈ X, P j { κ˜ε,1 > δ} ≤

Eε, j, Nε

δ

→ 0 as ε → 0,

(7.121)

for any function 0 < wε → 0 as ε → 0. The relation (7.121) implies that the condition C6 (for the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n )) is satisfied. The recurrent relations (7.111) for second moments (for n = 2) take the following form: (2)

(2)

(2)

Eε,i, N = Fε,i, N + pε,− Eε,i−1, N ε ε ε

+ pε,+ E(2) ε,i+1, Nε , 2 ≤ i ≤ Nε ,

(7.122)

184

7 Perturbed M/M-type queuing systems

(2) (2) where E(2) ε,1, Nε = 0, Eε, Nε +1, Nε = Eε, Nε , Nε and, (2)

2 1 (1) +2 (pε,− E(n−k) ε,i−1, Nε + pε,+ Eε,i+1, Nε ) λε wε (λε wε )2 1 1 2 +2 (E(1) − ) = ε,i, N 2 ε λε wε λε wε (λε wε ) 2 (1) E , 1 ≤ i ≤ Nε . = λε wε ε,i, Nε

Fε,i, N = ε

(7.123)

The relations (7.122) can be rewritten in the equivalent form, (2)

2

(2)

(1)

Eε,i+1, N − Eε,i, N = − E ε ε λε wε pε,+ ε,i, Nε (2) + (E(2) ε,i, Nε − Eε,i−1, Nε )

pε,− , 2 ≤ i ≤ Nε, pε,+

(7.124)

or (2)

(2)

2

i−1 

(1)

pε,−

Eε,i+1, N − Eε,i, N = − E ( )k−1 ε ε λε wε pε,+ k=1 ε,i−k+1, Nε pε,+

+ E(2) ε,2, Nε (

pε,− i−1 ) , 2 ≤ i ≤ Nε . pε,+

(7.125)

Taking i = Nε in the relation (7.125), we get the following relation: N ε −1 pε,− k−1 2 (1) Eε, N −k+1, N ( ) ε ε λε wε pε,+ k=1 pε,+ pε,− Nε −1 + E(2) ) , ε,2, Nε ( p ε,+

0=−

(7.126)

which gives the following explicit formula: (2)

2 E2 κ˜ε,1 = Eε,2, Nε N ε −1 pε,− k−1 pε,+ Nε −1 2 (1) = E ( ) ·( ) λε wε pε,+ k=1 ε, Nε −k+1, Nε pε,+ pε,−

=

Nε  pε,+ r−1 2 (1) E ( ) λε wε pε,+ r=2 ε,r, Nε pε,−

Nε  r−1 (1 − ( pε,+ ) Nε −i )  pε,+ r−1 2 pε,− = ) . ( pε,− 2 pε,− (λε wε pε,+ ) r=2 i=1 −1 p

(7.127)

ε,+

Also, the relation (7.109), taken for n = 2, and the relations (7.118) and (7.127) imply that,

7.1 Rare events for queuing systems with bounded buffers

2

185

1

2 2 2 E1 κ˜ε,1 = E1 κε,1 = +2 pε,+ E2 κ˜ε,1 + pε,+ E2 κ˜ε,1 λε wε (λε wε )2

1 − ( pε,+ ) Nε −1  2 ε,− 1+ = pε,− (λε wε )2 −1 p p

ε,+

Nε  r−1 (1 − ( pε,+ ) Nε −i ) pε,+ r−1 1  pε,− + ) . ( pε,− pε,+ r=2 i=1 p ε,− pε,+ − 1

(7.128)

The relation (7.128) implies that, 2 E1 κ˜ε,1 ≤

where 2  Bε = 2 1 + λε

1 pε,− pε,+

Bε . wε2

(7.129)

Nε (r − 1)( pε,+ )r 1  pε,−  + . pε,− p −1 ε,+ r=2 pε,+ − 1

(7.130)

The condition B15 obviously implies that, 0 < lim Bε ≤ lim Bε < ∞. ε→0

(7.131)

ε→0

Let θˆε,n, n = 1, 2, . . . be, for ε ∈ (0, ε2 ], i.i.d. random variables with the distribution function Fˆε,X (·) = πε,0 Fˆε,0,X (·) + πε,1 Fˆε,1,X (·). (w) Consider also the sequence of i.i.d. random variables θ ε,n = θˆε,n /wε, n = 1, 2, . . .. Obviously, these random variables have the distribution function Fε (·) = πε,0 Fˆε,0,X (·wε ) + πε,1 Fˆε,1,X (·wε ) = πε,0 Fε,0 (·) + πε,1 Fε,1 (·). The relations (7.94), (7.109), and (7.120) imply that, (w)

Eθ ε,1 = πε,0 E0 κ˜ε,1 + πε,1 E1 κ˜ε,1 =

Cε,◦ , wε

(7.132)

where 1 − ( pε,+ ) Nε −1 pε,− 1 1 1 ε,− + (1 + ). pε,− pε,− + 1 με,+ pε,− + 1 λε −1 p p

Cε,◦ =

(7.133)

ε,+

The condition B15 implies that, 0 < lim Cε,◦ ≤ lim Cε,◦ < ∞. ε→0

(7.134)

ε→0

Also, the relations (7.94), (7.109), and (7.128) imply that, (w)

2 2 E(θ ε,1 )2 = πε,0 E0 κ˜ε,1 + πε,1 E0 κ˜ε,1 ≤

Bε,◦ , wε2

(7.135)

186

7 Perturbed M/M-type queuing systems

where Bε,◦ =

pε,− 2 1 Bε . + pε,− + 1 μ2ε,+ pε,− + 1

(7.136)

The relation (7.131) implies that, 0 < lim Bε,◦ ≤ lim Bε,◦ < ∞. ε→0

ε→0

(7.137)

Let us consider the random variables, θ ε(w) =

[u ε] 

(w) θ ε,n =

n=1

[u ε] 

θˆε,n /wε .

(7.138)

n=1

First, we choose the normalisation function, w◦,ε = Cε,◦ uε .

(7.139)

The relations (7.132), (7.133), and (7.134) imply that, (w◦ )

Eθ ε

(w◦ ) = [uε ]Eθ ε,1 = [uε ]

Cε,◦ → 1 as ε → 0. Cε,◦ uε

(7.140)

Also, the relations (7.134), (7.135), (7.136), and (7.137) imply that, (w◦ ) Varθ ε(w◦ ) = [uε ]Varθ ε,1 (w◦ ) 2 ≤ [uε ]E(θ ε,1 ) ≤ [uε ]

Bε,◦ → 0 as ε → 0. 2 u2 Cε,◦ ε

(7.141)

The relations (7.140) and (7.141) obviously imply that the following asymptotic relation takes place: [u ε]  (w◦ ) P θ ε(w◦ ) = θ ε,n −→ 1 as ε → 0. (7.142) n=1

Let us now consider the case, where the function 0 < wε → ∞ as ε → 0 satisfies the following condition: uε D22 : Aε = uε Eθ ε,1 = Cε,◦ w → A ∈ (0, ∞) as ε → 0. ε

Under the condition B15 , the condition D21 is necessary and sufficient for the fulfilment of the condition D1 , which in this case takes the form of asymptotic relation, [u ε]  (w (w (w) θε ) = θ ε,n ⇒ θ 0 ) as ε → 0, (7.143) n=1 (w θ0 )

where is a non-negative random variable with distribution function not concentrated at zero.

7.1 Rare events for queuing systems with bounded buffers

187 (w

Moreover, in this case, the limiting random variable θ 0 ) = A with probability 1. Indeed, let the conditions B15 and D22 be satisfied. In this case, relations (7.142) and the asymptotic relation appearing in the condition D22 imply that, θ ε(w) =

[u ε] 

(w) θ ε,n =

n=1

[u ε] 

(w◦ ) θ ε,n

n=1

Cε,◦ uε wε

Cε,◦ uε P = θ ε(w◦ ) −→ A as ε → 0. wε

(7.144)

Thus, the above sufficiency statement is true. Let us assume that the condition B15 is satisfied and the relation (7.143) holds, but the condition D22 is not satisfied. In this case, two subsequences 0 < εk , εk ≤ 1 can be selected such that: (a) εk , εk → 0 as n → ∞, (b)

A ε  uε  k

k

wε  k A.

→ A ∈ [0, ∞] as k → ∞, (c)

[uε  ]

=

[uε  ]

k

θ ε(w)  ,n k

n=1

= θ ε(w ◦ ) k

k

=

w

◦) θ ε(w ,n k

n=1

Cεk ,◦ uεk εk

k

wε 

k

→ A ∈

k

[0, ∞] as k → ∞, (d) A  In this case, relation (7.142) implies that θ ε(w)  k

A ε  uε 

Cεk ,◦ uεk wεk

d

−→ A as k → ∞

(7.145)

and [uε  ]

θ ε(w)  k

=

k n=1

= θ ε(w◦ ) k

[uε  ]

θ ε(w) ,n k

=

k n=1

Cεk,◦ uεk w

εk

◦) θ ε(w,n k

Cεk,◦ uεk wεk

d

−→ A as k → ∞.

(7.146)

The above two relations contradict to the relation noplutaop. Thus, the above necessity statement is also true, i.e., the condition D22 is satisfied. (w In this case, according to the relation (7.71), the limiting random variable θ 0 ) = A with probability 1. The condition D22 plays, in this case, the role of the condition D1 . Finally, the relations (7.129)–(7.131) imply, by Lemma 4.7, that the following relation takes place, for ε ∈ (0, ε ]: 2 E1 κε,1 ≤

Bε,1 Bε,1 uε = πε,1 pε,1 = o1 (ε)pε,1, 2 wε wε wε

(7.147)

188

7 Perturbed M/M-type queuing systems

where o1 (ε) =

Bε,1 uε πε,1 → 0 as ε → 0. wε wε

(7.148)

The relation (7.147) implies that condition C5 is satisfied (for α, β = 2), and, thus, the condition C1 is satisfied for the Markov renewal processes (ηε,n, κε,n, ρε,n ). Therefore, all conditions of Theorem 4.4 hold for Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ). In this case, Theorem 4.4 takes the following form. Theorem 7.2 Let the condition B15 be satisfied. Then: (i) The condition D22 is necessary and sufficient for the fulfilment (for some or ˜ respectively, in statements any initial distributions q¯ε concentrated on the space X, d

of necessity and sufficiency) of the asymptotic relation ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution not concentrated in zero. (ii) The limiting random variable ξ0 has the exponential distribution with the parameter A−1 . ˜ can be played by any finite subset X ˜ = It is useful to note that the role of X ˇ {1, . . . , m} ˜ of the space Xε .

7.2 Rare Events for Queuing Systems with Unbounded Buffers In this section, we give necessary and sufficient conditions of convergence in distribution for the first failure event times for perturbed M/M queuing systems with an unbounded (infinite) queue buffer. Let Sε be, for every ε ∈ (0, 1], a standard M/M queuing system with an unbounded queue buffer. This system has one server and an unbounded queue buffer (including the place in the server). It is assumed that: (a) the input flow of customers is a standard Poisson flow with parameter με,+ > 0; (b) customers are placed in the queue buffer according to the order they arrive in the system; (c) a new customer coming in the system is immediately served if the queue buffer is empty, otherwise it is placed in the queue buffer; (d) the service times for different customers are independent random variables, which have an exponential distribution with parameter με,− > 0; (e) the input flow and all service processes are mutually independent. The state of system at the moment t is represented by the random variable ηˆε (t), which is the number of customers in the buffer at this instant. The process ηˆε (t) is ˆ = {0, 1, . . .} a birth–death-type continuous Markov chain. It has the phase space X and transition probabilities of the corresponding embedded Markov chain, pˆε,i j where,

⎧ for i = 0, j = 1, ⎪ ⎨1 ⎪ = pε,± for i > 1, j = i + ⎪ ⎪0 otherwise, ⎩

1±1 2 ,

(7.149)

7.2 Rare events for queuing systems with unbounded buffers

pε,+ =

189

με,+ με,− , pε,− = , λε = με,+ + με,− λε λε

and distribution functions of sojourn times,

1 − e−με,+ t for t ≥ 0, i = 0, ˆ Fε,i (t) = 1 − e−λε t for t ≥ 0, i ≥ 1.

(7.150)

(7.151)

We consider two cases: subcritical and critical. In the subcritical case, the following condition is assumed to be satisfied: B16 : (a) 0 < limε→0 με,± ≤ limε→0 με,± < ∞, (b) limε→0

με,+ με,−

< 1.

In the critical case, the following condition is assumed to be satisfied: B17 : (a) 0 < limε→0 με,± ≤ limε→0 με,± < ∞, (b) με,+ με,− → 1 as ε → 0.

με,+ με,−

< 1, for ε ∈ (0, 1] and

Let Nε be an integer threshold parameter such that, 1 ≤ Nε → ∞ as ε → 0.

(7.152)

The event {ηε,n ≥ Nε + 1} occurs in the above system, when the number of customers in the system exceeds the threshold Nε . Such event can be interpreted as a rare one, due to condition B16 , or B17 , and the assumption (7.152). Let 0 = τˆε,0 < τˆε,1 < · · · be successive moments of jumps for the semiˆ τˆε,n ), n = 0, 1, . . . be the corresponding discrete time Markov process ηˆε (t), ηˆε,n = η( embedded Markov chain, κˆε,n = τˆε,n − τˆε,n−1, n = 1, . . . be the inter-jump times for the semi-Markov process ηˆε (t), and νˆε (t) = max(n ≥ 1 : τˆε,n ≤ t) be the number of jumps for the Markov chain η(s) ˆ in the time interval [0, t]. Let us consider the Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ), n = 0, 1, . . . which ˇ × [0, ∞) × {0, 1}, where X ˇ =X ˆ and components, has the phase space X ⎧ ⎪ ⎨ ηˇε,n = ηˆε,n, ⎪ κˇε,n = κˆε,n /wε, ⎪ ⎪ ρˇ ε,n = I(ηˆε,n ≥ Nε + 1), ⎩

(7.153)

where 0 < wε → ∞ as ε → 0 is some normalisation function, which we will define below. As in Sect. 7.1.2, let us define the normalised first-rare-event time for the system Sε , which can also be called the first failure time, as follows: ξSε =

νˇ ε  n=1

κˇε,n =

νˆ ε 

κˆε,n /wε,

(7.154)

n=1

where νˇε = min(n ≥ 1 : ηˇε,n ≥ Nε + 1).

(7.155)

190

7 Perturbed M/M-type queuing systems

In this case, we choose the phases spaces, ˜ = {1, . . . , m} ˇ where m˜ ≥ 2 and X = {0, 1}. X ˜ ⊂ X,

(7.156)

Since, Nε → ∞ as ε → 0, it can be assumed that Nε ≥ m, ˜ for ε ∈ (0, 1]. ˜ Let us assume that the initial state ηˇε,0 ∈ X. The condition B16 implies that, 1 0 < lim pε,+ ≤ lim pε,+ < , ε→0 2 ε→0

(7.157)

while the condition B17 implies that, 1 1 pε,+ < , ε ∈ (0, 1] and lim pε,+ = . ε→0 2 2

(7.158)

Therefore, there exist ε  ∈ (0, 1] such that, for every ε ∈ (0, ε ], the phase space ˇ X is one class of communicative states for the Markov chain ηˇε,n . Obviously, we can assume that ε  = 1. ˇ In this case, the condition B5 is satisfied and, thus, for i ∈ X, Pi { νˇε,X < ∞} = 1,

(7.159)

νˇε,X = min(n ≥ 1 : ηˇε,n ∈ X).

(7.160)

where The relation (7.159) makes it possible to define the successive times of hitting the Markov chain ηˇε,n in the set X, νˇε,X,n = min(k > νˇε,X,n−1, ηˇε,k ∈ X), n = 1, 2, . . . , where νˇε,X,0 = 0.

(7.161)

The corresponding Markov renewal process with transition period (η˜ε,n, κ˜ε,n , ˜ for the initial state η˜ε,0 , and the phase ρ˜ ε,n ), n = 0, 1, . . . (with the phase space X, space X, for the random variables η˜ε,n, n ≥ 1) can be constructed using the following recurrent relations: ⎧ η˜ε,n = ηˇε,νˇ ε,X, n , n = 1, 2, . . . , η˜ε,0 = ηˇε,0, ⎪ ⎪ νˇ ε,X, n ⎨ ⎪ κ˜ε,n = k= κˇε,k , n = 1, 2, . . . , κ˜ε,0 = κˇε,0, νˇ n−1 +1 ⎪ ε,X, νˇ ε,X, n ⎪ ⎪ ρ˜ ε,n = 1 − (1 − ρˇ ε,k ), n = 1, 2, . . . , ρ˜ ε,0 = ρˇ ε,0 . k=νˇ ε,X, n−1 +1 ⎩

(7.162)

Also, as in Sect. 4.1.3.1, the corresponding shifted Markov renewal process can be defined by the following relation: (ηε,n, κε,n, ρε,n ) = (η˜ε,n+1, κ˜ε,n+1, ρ˜ ε,n+1 ), n = 0, 1, . . . . Obviously, the following inequality holds, for every ε ∈ (0, 1]:

(7.163)

7.2 Rare events for queuing systems with unbounded buffers

191

ξ˜ε,− ≤ ξSε ≤ ξ˜ε,+, where ξ˜ε,− =

ν˜ ε −1

κ˜ε,k , ξ˜ε,+ =

k=1

ν˜ ε 

(7.164)

κ˜ε,k ,

(7.165)

k=1

and ν˜ε = min(n ≥ 1 : ρ˜ ε,n = 1).

(7.166)

Let us show that conditions of Theorem 4.4 hold. In the subcritical case, the subsequent analysis and results are partially similar to those presented in Sect. 7.1.2, but in the critical case, the results are different. The Markov chain ηε,n has the phase space X = {0, 1} and the matrix of transition probabilities,   0 1 . (7.167) Pε = pε,− pε,+ The relation (7.157) implies that the condition B1 is satisfied for Markov chains ηε,n , which are asymptotically uniformly ergodic. The stationary probabilities for the Markov chain ηε,n , πε,0 =

με,− με,− + με,+ pε,− 1 = = , πε,1 = pε,− + 1 2με,− + με,+ pε,− + 1 2με,− + με,+

and, thus, for i = 0, 1,

0 < lim πε,i ≤ lim πε,i < 1. ε→0

ε→0

(7.168)

(7.169)

In this case, pˇε,i = Pi { ρˇ ε,1 = 1} = and, for u ≥ 0,

Pi { κˇε,1 ≤ u} =

⎧ for i = 0, . . . , Nε − 1, ⎪ ⎨0 ⎪ pε,+ for i = Nε, Nε + 1, ⎪ ⎪1 for i > Nε + 1, ⎩

(7.170)

1 − e−με,+ wε u for i = 0, 1 − e−λε wε u for i ≥ 1.

(7.171)

˜ coincides for the It is easy to see that the probabilities pε,i = Pi { ρ˜ ε,1 = 1}, i ∈ X above model of M/M queuing system and the model of M/M queuing system (with bounded buffer of size Nε ), which was the subject of research in Sect. 7.1.2. Therefore, according to the relations (7.172), (7.98), and (7.99) the probabilities ˜ take the following form: pε,i = Pi { ρ˜ ε,1 = 1}, i ∈ X pε,i = Pi { ρ˜ ε,1 = 1} = where,

⎧ ⎪ ⎨ ⎪

0 for i = 0, pε,+ Pε,2, Nε for i = 1, ⎪ ⎪ Pε,i, N for i = 2, . . . , m, ¯ ε ⎩

(7.172)

192

7 Perturbed M/M-type queuing systems

Pε,2, Nε =

pε,− pε,+

1−

p

ε,+ pε,+ Nε −1 1 − pε,− =( ) , p pε,− 1 − ( pε,+ ) Nε ε,−

p

1 − ( pε,− ) Nε ε,+

(7.173)

and, for j = 2, . . . , Nε , p

Pε, j, Nε = Pε,2, Nε ·

ε,− j−1 1 − ( pε,+ )

1−

pε,− pε,+

p

= Pε,2, Nε

ε,+ j−1 pε,+ 2−j 1 − ( pε,− ) ·( ) . p pε,− 1 − pε,+ ε,−

(7.174)

The condition B16 , or B16 , and the relations (7.172)–(7.174) imply that pε,i > ˜ for ε ∈ (0, ε ]. 0, i ∈ X, ˜ The condition B16 obviously implies that, for every i ∈ X, pε,i → 0 as ε → 0.

(7.175)

This relation also holds in the case, where the condition B17 is assumed to be satisfied. Indeed, the relation (7.173), the condition B17 , and the assumption Nε → ∞ as ε → 0, imply that, for j = 2, . . . , m¯ and N ≥ 1, lim Pε, j, Nε = lim

ε→0

ε→0

≤

pε,− pε,+

p

−1

·

p

ε,− N ε ( pε,+ ) −1

pε,− pε,+ − 1 lim pε,− N ε→0 ( pε,+ ) − 1

·

ε,− j−1 ) 1 − ( pε,+

pε,− pε,+ pε,− j−1 ) − ( pε,+ p 1 − pε,− ε,+

1−

1

=

j −1 → 0 as N → ∞. (7.176) N

Thus, the condition B16 , or B17 , implies that the conditions A1 (for the Markov renewal processes (ηε,n, κε,n, ρε,n )) and A3 (for the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n )) are satisfied. In this case, the averaged rare-event probability pε for the Markov chain ηε,n , and function uε = p−1 ε take the following forms: pε = πε,0 pε,0 + πε,1 pε,1 = πε,1 pε,1 p

ε,− 1 pε,+ − 1 = · pε,+ pε,− N , 1 + pε,− (p ) ε − 1

(7.177)

ε,+

and

p

uε =

p−1 ε

ε,− N ε 1 + pε,− ( pε,+ ) − 1 = → ∞ as ε → 0. pε,− pε,+ −1 p

(7.178)

ε,+

Let us consider the random variables: κ˜ε,1 =

νˇ ε,X,1 k=1

κˇε,k .

(7.179)

7.2 Rare events for queuing systems with unbounded buffers

193

Let us also introduce the distribution functions, for j = 0, 1, . . ., νˇ ε,X,1

Fˆε, j,X (u) = P j {

κˆε,k ≤ u}, u ≥ 0.

(7.180)

k=1

It is useful to note that, P0 { νˇε,X = 1} = 1.

(7.181)

In this case, distribution functions, Fε, j (u) = Pi { κ˜ε,1 ≤ u}

1 − e−με,+ wε u for u ≥ 0, j = 0, = Gˆ ε,1,X (uwε ) for u ≥ 0, i = 1, 2, . . . .

(7.182)

The following stochastic representation takes place,

τε,i

⎧ ζε ⎪ ⎪ ⎪  ⎨ ζε + I(ιε = +1)τε,2 d ⎪ =  + τ ) ζε + I(ιε = +1)(τε,3 ⎪ ε,2 ⎪ ⎪   ⎪τ +···+τ ⎩ ε,i ε,2

for i for i for i for i

= 0, = 1, = 2, = 3, 4, . . . ,

(7.183)

where: (a) the random variable τε,i has the distribution function Pi { κ˜ε,1 ≤ ·}, for d

 i = 0, 1, . . ., (b) the random variable τε,k = τε,2 , for k = 2, 3, . . ., (c) the random variable ζε is exponentially distributed with parameter λε wε , (d) the random variable ζε is exponentially distributed with parameter με,+ wε , (e) the random variable ιε takes values −1 and +1 with probabilities, respectively, pε,− and pε,+ , and (f)  , k = 2, 3, . . . are mutually independent, for every the random variables ζε , ιε , τε,k k = 2, 3, . . .. The conditions B16 , or B17 , imply that there exists ε  ∈ (0, ε ] such that pε,+ < pε,− , for ε ∈ (0, ε ]. As follows from the well-known results related to random walks that, for ε ∈ (0, ε ] and n ≥ 1, i = 0, 1, . . ., (n)

n Eε,i = Ei κ˜ε,1 < ∞.

(7.184)

The stochastic representation (7.183) implies that the following relation holds for the expectations E(1) ε,i = Ei κ˜ε,1, i = 0, 1, . . .,

(1)

Eε,i =

⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪

1 με,+ wε 1 λ ε wε + 1 λ ε wε +

⎪ ⎪ ⎪ ⎪ ⎪ (i − ⎩

pε,+ E(1) ε,2 2pε,+ E(1) ε,2 1)E(1) ε,2

for i = 0, for i = 1, for i = 2, for i = 3, 4, . . . .

The relation (7.185) takes the form of the following relation, for i = 2:

(7.185)

194

7 Perturbed M/M-type queuing systems (1)

Eε,2 =

1 + 2pε,+ E(1) ε,2, λε wε

(7.186)

which gives the following explicit formula: (1)

Eε,2 =

1 1 . λε wε pε,− − pε,+

(7.187)

The relation (7.187) allows us to rewrite the relation (7.185) in the following explicit form: 1 ⎧ for i = 0, ⎪ με,+ wε ⎪ ⎪ pε,− ⎪ 1 ⎨ ⎪ for i = 1, (1) ε,+ (7.188) Eε,i = λε1wε pε,− −p 1 ⎪ ⎪ λ ε wε pε,− −pε,+ for i = 2, ⎪ ⎪ i−1 ⎪ 1 ⎩ λε wε pε,− −pε,+ for i = 3, 4, . . . . The condition B16 , or B17 , and the relation (7.188) imply that, for δ > 0 and ˇ i ∈ X, (1)

Pi { κ˜ε,1 > δ} ≤

Eε,i

δ

→ 0 as ε → 0,

(7.189)

for any function 0 < wε → 0 as ε → 0. The relation (7.189) implies that the condition C6 (for the Markov renewal processes (η˜ε,n, κ˜ε,n, ρ˜ ε,n )) is satisfied. The stochastic representations (7.183) imply that the following relation holds for second moments E(2) ε,i = Ei κ˜ε,1, i = 0, 1, . . .,

(2)

Eε,i =

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪

2 (με,+ wε )2 (2) 2 + 2 λε1wε pε,+ E(1) ε,2 + pε,+ Eε,2 (λ ε wε )2  (2) 4p (1) (1) 2  2 ⎪ + λε ε,+ ⎪ wε Eε,2 + 2pε,+ Eε,2 + (Eε,2 ) (λ ε wε )2 ⎪ ⎪ ⎪ ⎪ (i − 1)E(2) + (i − 1)(i − 2)(E(1) )2 ε,2 ε,2 ⎩

for i = 0, for i = 1, for i = 2,

(7.190)

for i = 3, 4, . . . .

Relation (7.190) takes the form of the following relation, for i = 2: (2)

Eε,2 =

4pε,+ 1 1 2 + 2 λε wε λε wε pε,− − pε,+ (λε wε ) 1 1 + 2pε,+ E(2) ε,2 + 2pε,+ (λ w )2 (p 2 ε ε ε,− − pε,+ )

(7.191)

which gives the following explicit formula: (2)

Eε,2 =

1 2 2 2 λε wε (pε,− − pε,+ ) 4pε,+ 2pε,+ 1 1 + 2 + 2 . 2 2 2 λε wε (pε,− − pε,+ ) λε wε (pε,− − pε,+ )3

(7.192)

7.2 Rare events for queuing systems with unbounded buffers

195

Also, the relation (7.190), for i = 1, and the relation (7.192) imply that, (2)

2 1 (2) +2 pε,+ E(1) ε,2 + pε,+ Eε,2 λε wε (λε wε )2 4pε,+ 1 2 1 = 2 2 + 2 λε wε λε wε2 (pε,− − pε,+ )

Eε,1 =

+

4p2ε,+ λε2

2p2ε,+ 1 1 + . 2 2 2 2 wε (pε,− − pε,+ ) λε wε (pε,− − pε,+ )3

(7.193)

The relation (7.193) implies that, (2)

Bε

1

2 E κ˜ε,1 = Eε,1 = , (pε,− − pε,+ )3 wε2

(7.194)

where Bε =

2(pε,− − pε,+ )3 4pε,+ (pε,− − pε,+ )2 + λε2 λε2 4p2ε,+ (pε,− − pε,+ ) 2p2ε,+ + + 2 . λε2 λε

(7.195)

The condition B16 , or B17 , implies that, 0 < lim Bε ≤ lim Bε < ∞. ε→0

ε→0

(7.196)

Let θˆε,n, n = 1, 2, . . . be, for ε ∈ (0, ε ], i.i.d. random variables with the distribution function Fˆε,X (·) = πε,0 Fˆε,0,X (·) + πε,1 Fˆε,1,X (·). (w) Let us also consider the sequence of i.i.d. random variables θ ε,n = θˆε,n /wε, n = 1, 2, . . .. These random variables have the distribution function Fε (·) = πε,0 Fˆε,0,X (·wε )+ πε,1 Fˆε,1,X (·wε ) = πε,0 Fε,0 (·) + πε,1 Fε,1 (·). The relations (7.94), (7.109), and (7.120) imply that, Aε,◦ 1 , (pε,− − pε,+ ) wε

(7.197)

pε,− 1 1 1 (pε,− − pε,+ ) + pε,− . pε,− + 1 με,+ pε,− + 1 λε

(7.198)

(w)

Eθ ε,1 = πε,0 E0 κ˜ε,1 + πε,1 E1 κ˜ε,1 =

where Aε,◦ =

The condition B16 , or B17 , implies that, 0 < lim Aε,◦ ≤ lim Aε,◦ < ∞. ε→0

ε→0

Also, the relations (7.94), (7.190), and (7.193) imply that,

(7.199)

196

7 Perturbed M/M-type queuing systems (w)

2 2 E(θ ε,1 )2 = πε,0 E0 κε,1 + πε,1 E0 κε,1 =

where Bε,◦ =

Bε,◦ 1 , 3 (pε,− − pε,+ ) wε2

pε,− 2(pε,− − pε,+ )3 1 Bε . + 2 pε,− + 1 pε,− + 1 με,+

(7.200)

(7.201)

The condition B16 , or B17 , implies that, 0 < lim Bε,◦ ≤ lim Bε,◦ < ∞. ε→0

ε→0

(7.202)

Let us consider the random variables, θ ε(w)

=

[u ε] 

(w) θ ε,n

n=1

=

[u ε] 

θˆε,n /wε .

(7.203)

n=1

Let us, first, choose the normalisation function, w◦,ε = Aε,◦

uε . pε,− − pε,+

(7.204)

The relations (7.197), (7.198), and (7.199) imply that, (w◦ )

Eθ ε

(w◦ ) = [uε ]Eθ ε,1 = [uε ]

Aε,◦ /(pε,− − pε,+ ) → 1 as ε → 0. Aε,◦ uε /(pε,− − pε,+ )

(7.205)

Also, the relations (7.197), (7.198), (7.200), and (7.201) imply that, (w◦ ) (w◦ ) 2 (w◦ ) 2 Varθ ε(w◦ ) = [uε ]Varθ ε,1 = [uε ](E(θ ε,1 ) − (Eθ ε,1 ) )

 Bε,◦ /(pε,− − pε,+ )3 Aε,◦ /(pε,− − pε,+ ) 2  ) − ( 2 2 Aε,◦ uε /(pε,− − pε,+ ) Aε,◦ uε /(pε,− − pε,+ )2  Bε,◦ [uε ]  = −1 . (7.206) uε A2ε,◦ uε /(pε,− − pε,+ ) = [uε ]

At this stage, the subsequent analysis differs for the subcritical case, where the condition B16 is satisfied and the critical case, where the condition B17 is satisfied. If the condition B16 is satisfied, limε→0 (pε,− − pε,+ ) > 0, and, thus, the relation (7.206) implies that, (7.207) Var θ ε(w◦ ) → 0 as ε → 0. If the condition B17 holds, limε→0 (pε,− − pε,+ ) = 0. In this case, the relation (7.178) implies that,

7.2 Rare events for queuing systems with unbounded buffers

197

pε,− Nε −1

1 + pε,− ( pε,+ ) (pε,− − pε,+ ) uε (pε,− − pε,+ ) = pε,− pε,+ pε,+ − 1 pε,− Nε −1 = (1 + pε,− )(( ) − 1). pε,+

(7.208)

The relation (7.208) implies that, uε (pε,− − pε,+ ) → ∞ as ε → 0,

(7.209)

and, thus, the relation (7.207) holds, if and only if the following condition is satisfied, which is in addition to the condition B17 : μ

B18 : ( με,− − 1)Nε → ∞ as ε → 0. ε,+ According to the above remarks, if the condition B16 holds, or the conditions B17 and B18 hold, then the relations (7.205) and (7.207) imply that the following asymptotic relation takes place: θ ε(w◦ ) =

[u ε] 

(w◦ ) θ ε,n −→ 1 as ε → 0. P

(7.210)

n=1

Let us now consider the case, where the function 0 < wε → ∞ as ε → 0 satisfies the following condition: uε D23 : Aε = uε Eθ ε,1 = Aε,◦ wε (pε,− −pε,+ ) → A ∈ (0, ∞) as ε → 0.

Under condition B16 , or the conditions B17 and B18 , the condition D23 is necessary and sufficient for holding of condition D1 , which in this case takes the form of the following asymptotic relation: (w

θε ) =

[u ε] 

(w

(w) θ ε,n ⇒ θ 0 ) as ε → 0,

(7.211)

n=1 (w

where θ 0 ) is a non-negative random variable with distribution function not concentrated at zero. (w Moreover, in this case, the limiting random variable θ 0 ) = A with probability 1. Indeed, let the condition B16 and D23 or the conditions B17 , B18 and D23 be satisfied. In this case, the relation (7.210) and the asymptotic relation appearing in the condition D23 imply that, θ ε(w) =

[u ε]  n=1

(w) θ ε,n =

[u ε]  n=1

(w◦ ) θ ε,n

Aε,◦ uε (pε,− − pε,+ )wε

Aε,◦ uε P = θ ε(w◦ ) −→ A as ε → 0. (pε,− − pε,+ )wε

(7.212)

198

7 Perturbed M/M-type queuing systems

Thus, the above sufficiency statement is true. Let us assume that the condition B16 is satisfied and the relation (7.211) holds, or the conditions B17 and B18 are satisfied and the relation (7.211) holds, but the condition D23 is not satisfied. In this case, two subsequences 0 < εk , εk ≤ 1 can be selected such that: (a) εk , εk → 0 as n → ∞, (b) A ε  ,◦ uε  k

k

(pε  ,− −pε  ,+ )wε  k

k

A ε  ,◦ uε  k

→ A ∈ [0, ∞] as k → ∞, (c)

k

(pε  ,− −pε  ,+ )wε  k

k

k

→ A ∈ [0, ∞] as k → ∞, (d) A  A.

k

In this case, relation (7.211) implies that [uε  ]

θ ε(w)  k

=

[uε  ]

k

θ ε(w)  ,n k

n=1

= θ ε(w ◦ ) k

=

k

◦) θ ε(w ,n k

n=1

Aεk ,◦ uεk (pεk ,− − pεk ,+ )wεk

Aεk ,◦ uεk

d

(pεk ,− − pεk ,+ )wεk

−→ A as k → ∞

(7.213)

and [uε  ]

θ ε(w)  k

=

k n=1

= θ ε(w◦ ) k

[uε  ]

θ ε(w) ,n k

=

k

◦) θ ε(w,n

n=1

k

Aεk,◦ uεk (pεk,− − pεk,+ )wεk

Aεk,◦ uεk (pεk,− − pεk,+ )wεk

d

−→ A as k → ∞.

(7.214)

The above two relations contradict to the relation (7.213). Thus, the above necessity statement is also true, i.e. the condition D23 is satisfied. (w In this case, due to relation (7.212), the limiting random variable θ 0 ) in relation (7.211) should be constant A. Condition D23 plays, in this case, the role of condition D1 . Finally, by Lemma 4.7, relations (7.194)–(7.196) imply that the following relation takes place, for ε ∈ (0, ε ]: 2 E1 κε,1 ≤

Bε,1 Bε,1 uε = πε,1 pε,1 = o1 (ε)pε,1 . 2 wε wε wε

(7.215)

B

uε πε,1 → 0 as ε → 0, the relation (7.215) implies that Since, o1 (ε) = wε,1 ε wε the condition C5 holds, and, thus, the condition C1 is satisfied for Markov renewal processes (ηε,n, κε,n, ρε,n ). Therefore, all conditions of Theorem 4.4 hold for Markov renewal process (ηˇε,n, κˇε,n, ρˇ ε,n ). In this case, Theorem 4.4 takes the following form.

Theorem 7.3 Let the condition B16 be satisfied, or the conditions B17 and B18 be satisfied. Then:

7.2 Rare events for queuing systems with unbounded buffers

199

(i) The condition D23 is necessary and sufficient for the fulfilment (for some or any ˜ respectively, in the statements initial distributions q¯ε concentrated on the space X, d

of necessity and sufficiency) of the asymptotic relation, ξSε −→ ξ0 as ε → 0, where ξ0 is a non-negative random variable with distribution function not concentrated at zero. (ii) The limiting random variable ξ0 has an exponential distribution with the parameter A−1 . ˜ can be played by any finite subset X ˜ = It is useful to note that the role of X ˆ {1, . . . , m} ˜ of the space X.

Part II

Hitting Times and Phase Space Reduction for Perturbed Semi-Markov Processes

Chapter 8

Asymptotically Comparable Functions

In this chapter, we introduce the notion of a complete family of asymptotically comparable functions and present operating rules and formulas for such families. This concept plays a key role in perturbation conditions for semi-Markov type processes. We also consider typical examples of such families, namely, families of asymptotically comparable power type, power-exponential type, and power-logarithmic type functions. This chapter includes two sections. In Sect. 8.1 we give a definition of complete family of asymptotically comparable functions. In Lemmas 8.2 and 8.3, we present operating rules (summation, multiplication, and division) for such families. In Sect. 8.2, we consider the complete families of asymptotically comparable power-type, power-exponential-type, and power-logarithmic-type functions. Operating rules for these complete families are presented for asymptotically comparable power-type functions in Lemmas 8.4 and 8.5, for power-exponential-type functions in Lemmas 8.6 and 8.7, and for power-logarithmic-type functions in Lemmas 8.8 and 8.9.

8.1 Complete Families of Asymptotically Comparable Functions In this section, the notion of a complete family of asymptotically comparable functions is introduced and some general properties of such families are described.

8.1.1 Definitions for Families of Asymptotically Comparable Functions We consider functions h(ε) defined on the interval (0, 1] and taking values in the interval (0, ∞). Let H = {h(ε)} be a non-empty family of such functions.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_8

203

204

8 Asymptotically comparable functions

Definition 8.1 H = {h(·)} is a complete family of asymptotically comparable functions if: (1) it is closed with respect to operations of summation, multiplication, and division, (2) there exists limε→0 h(ε) = aH [h(·)] ∈ [0, ∞], for any function h(·) ∈ H. Here, aH [h(·)] is some functional acting from the functional space H to the interval [0, ∞]. We call aH [h(·)] the comparability limit for the function h(·). Definition 8.2 H  is a family of asymptotically comparable functions if it is a subfamily of some complete family of asymptotically comparable functions H. Let functions hk (ε) ∈ H , k = 1, 2, . . ., indices jk = ±1, k = 1, 2, . . .. Then, obviously, the following limit exists, for any n ≥ 1: lim

ε→0

n 

hk (ε) jk = aH [

k=1

n 

hk (·) jk ] ∈ [0, ∞].

(8.1)

k=1

n hk (ε) jk belong to Indeed, according to the obvious induction, functions k=1 family H, for n ≥ 1. Let H  be a family of functions defined on interval (0, 1], taking values in the interval (0, ∞), for which the asymptotic relation (8.1) holds. Let us now define the family [H ] of all functions given by the following relation: n

h(ε) =

nr 

j h  (ε) r , k  k  =1 r ,k  , n nr   jr, k  h (ε)     r =1 k =1 r ,k r  =1

(8.2)

where, (a) functions hr ,k  (ε) ∈ H, for 1 ≤ k  ≤ nr  < ∞, 1 ≤ r  ≤ n  < ∞; (b) functions hr,k  (ε) ∈ H, for 1 ≤ k  ≤ nr < ∞, 1 ≤ r  ≤ n  < ∞; (c) indices jr ,k  = ±1, for 1 ≤ k  ≤ nr  < ∞, 1 ≤ r  ≤ n  < ∞, (d) indices jr,k  = ±1, for 1 ≤ k  ≤ nr < ∞, 1 ≤ r  ≤ n  < ∞. Obviously, H  ⊆ [H ]. Indeed, any function h(ε) ∈ H  can be represented in the form, h(ε) = h(ε)2 /h(ε) ∈ [H ]. Also, the constant 1 = 1(ε), ε ∈ (0, 1] belongs to the family [H ]. Indeed, 1 = 1(ε) can be represented in the form, 1(ε) = h(ε)/h(ε) ∈ [H ]. We call the family [H ] the closure of the family H . The following lemma presents the basic properties of the family [H ]. Lemma 8.1 Let H  be a family of functions, for which the asymptotic relation (8.1) holds. Then its closure [H ] is a complete family of asymptotically comparable functions. Proof The fulfilment of the condition (1) given in the Definition 8.1 for the family [H ] and the corresponding operating formulas for calculating sums, products, and quotients for functions from the family [H ] are obvious. The following relation, which holds for any function h(ε) ∈ [H ] given by the relation (8.2), implies that the family [H ] satisfies the condition (2) given in the Definition 8.1,

8.1 Complete families of asymptotically comparable functions

n

lim h(ε) =

ε→0

nr 

j h  (ε) r , k  k  =1 r ,k  lim  j ε→0 n nr  h  (ε) r , k  r  =1 k  =1 r ,k  r  =1



= lim

ε→0

nr  n   n   r  =1

=

r  =1 k  =1

 n

r  =1

205

n

aH [

r  =1

nr 

 k  =1



j hr,k  (ε) r , k 

nr   k  =1

hr ,k  (ε)

− jr , k 



j hr,k  (·) r , k 

nr   k  =1

hr ,k  (·)

− jr , k 

]

 −1  −1

∈ [0, ∞].

(8.3)

Thus, [H ] is, indeed, a complete family of asymptotically comparable functions.  Lemma 8.1 allows us to give alternative variants of Definitions 8.1 and 8.2. Since H  ⊆ [H ], we can say that H  is a family of asymptotically comparable functions if the asymptotic relation (8.1) holds for functions from the family H . Moreover, the family of asymptotically comparable functions H  is complete if  H = [H ]. The complete family of asymptotically comparable functions H can be called trivial if comparability limits aH [h(·)] ∈ (0, ∞) for all functions h(·) ∈ H. In such case, there always exists a family of asymptotically comparable func˜ such that: (a) H ⊂ H, ˜ (b) there exists a function h(·) ˜ such that its ˜ ∈ H tions H ˜ comparability limit aH [ h(·)] = ∞. Indeed, take an arbitrary function u(ε), defined on interval (0, 1], taking values in interval (0, ∞) and such that limε→0 u(ε) = ∞. Let us consider the family of functions Hu(·) = H ∪ {u(·)}. Obviously, the asymptotic relation (8.1) holds for functions from this family, and, moreover, for any sequence of functions hk (ε) ∈ Hu(·), k = 1, 2, . . ., indices jk = ±1, k = 1, 2, . . ., and n ≥ 1, the following limit exists, for any n ≥ 1:

lim

ε→0

where

n 

hk (ε) jk

k=1

(n) aH =

⎧ 0 if b(n) < 0, ⎪ ⎪ u(·) ⎨ ⎪ (n) (n) jk = aHu(·) [ hk (·) ] = aH if bu(·) = 0, ⎪ ⎪ k=1 ⎪ ∞ if b(n) > 0, ⎩ u(·)

 1≤k ≤n:hk (·)∈H

n 

aH [hk (·) jk ], b(n) = u(·)



jk .

(8.4)

(8.5)

1≤k ≤n:hk (·)=u(·)

According to Lemma 8.1, we can define the complete family of asymptotically ˜ = [Hu(·) ], as the closure of the family Hu(·) . comparable H Since, H is itself a complete family of comparable functions, the family [Hu(·) ] consists of all functions of the form, n+

h(ε) =

 r r  =n− hr  (ε)u(ε) , n+  r  r  =n− hr  (ε)u(ε)

(8.6)

206

8 Asymptotically comparable functions

where, (a) functions hr  (ε) ∈ H, for −∞ < n− ≤ r  ≤ n+ < ∞; (b) functions hr (ε) ∈ H, for −∞ < n− ≤ r  ≤ n+ < ∞. Moreover, for any function h(·) ∈ [Hu(·) ], ⎧ if n+ < n+, ⎪ ⎨0 ⎪   a[Hu(·) ] [h(·)] = lim h(ε) = aH [hn+ (·)]/aH [hn+ (·)] if n+ = n+, ⎪ ε→0 ⎪∞ if n+ > n+ . ⎩

(8.7)

8.1.2 Operating Rules for Asymptotically Comparable Functions The following lemmas, which are obvious consequences of the operating rules for limits of sums, products, and quotients of functions, present operating rules for functions from a complete family of asymptotically comparable functions. Lemma 8.2 Let H be a complete family of asymptotically comparable functions, hi (·) ∈ H, i = 1, 2, and limε→0 hi (ε) = aH [hi (·)], for i = 1, 2. Then, the following operating sum, product, and quotient formulas for calculating comparability limits take place: (a) If h (·) = h1 (·) + h2 (·). Then, h (·) ∈ H and, aH [h (·)] = lim h (ε) = aH [h1 (·)] + aH [h2 (·)]. ε→0

(8.8)

(b) If h (·) = h1 (·) · h2 (·). Then, h (·) ∈ H and, aH [h (·)] = lim h (ε) ε→0

=

⎧ ⎪ ⎨ aH [h1 (·) · h2 (·)] ⎪

if aH [h1 (·)] = 0, aH [h2 (·)] = ∞, or aH [h1 (·)] = ∞, aH [h2 (·)] = 0, ⎪ ⎪ aH [h1 (·)] · aH [h2 (·)] otherwise. ⎩

(8.9)

(c) If h  (·) = h1 (·)/h2 (·). Then, h (·) ∈ H and, aH [h (·)] = lim h  (ε) ε→0

=

⎧ ⎪ ⎨ aH [h1 (·)/h2 (·)] ⎪

if aH [h1 (·)], aH [h2 (·)] = 0, or aH [h1 (·)], aH [h2 (·)] = ∞, ⎪ ⎪ aH [h1 (·)]/aH [h2 (·)] otherwise. ⎩

(8.10)

The operating rules presented in Lemma 8.1 admit the following multivariate generalisations. Lemma 8.3 Let H be a complete family of asymptotically comparable functions, hi (·) ∈ H, i = 1, 2, . . ., and limε→0 hi (ε) = aH [hi (·)], for i = 1, 2, . . .. Then,

8.2 Examples of complete families of asymptotically comparable functions

207

the following operating multiple sum and multiple product/quotient formulas for computing comparability limits take place: n (a) If h(n) (·) = i=1 hi (·), for n ≥ 1. Then, h(n) (·) ∈ H, for n ≥ 1, and, aH [h(n) (·)] = lim h(n) (ε) ε→0

=

n 

aH [hi (·)]

i=1

= aH [h(n−1) (·)] + aH [hn (·)],

(8.11)

where aH [h(0) (·)] ≡ 0. n hi (·) ji , for n ≥ 1, where ji = ±1, i = 1, 2, . . .. Then, (b) If h◦(n) (·) = i=1 (n) h◦ (·) ∈ H, for n ≥ 1, and, aH [h◦(n) (·)] = lim h◦(n) (ε) ε→0

n

ji ji ⎧ ⎪ ⎨ aH [ i=1 hi (·) ] if min1≤i ≤n aH [hi (·)] = 0, ⎪ max1≤i ≤n aH [hi (·)] ji = ∞, = ⎪ ⎪ n aH [hi (·) ji ] otherwise, ⎩ i=1 (n−1) ⎧ jn ⎪ if aH [h◦(n−1) (·)] ∧ aH [hn (·)] jn = 0, ⎪ ⎨ aH [h◦ (·)hn (·) ] ⎪ = aH [h◦(n−1) (·)] ∨ aH [hn (·)] ji = ∞, ⎪ ⎪ (n−1) j ⎪ aH [h◦ (·)]aH [hn (·)] n otherwise, ⎩

(8.12)

(8.13)

where aH [h◦(0) (·)] ≡ 1. The relations, given in Lemmas 8.2 and 8.3 allow expressing the comparability limits for asymptotically comparable functions, obtained as results of operations of summation, multiplication, or division, in terms of the corresponding comparability limits for functions involved in the above operations, except for the asymptotic cases of the forms 0 · ∞, 0/0, or ∞/∞. In such cases, some additional information about the asymptotic behaviour of the corresponding functions is required so that the limits can be calculated more explicitly. The complete families of asymptotically comparable functions considered in the next section represent models, in which the corresponding parameters and limits for asymptotically comparable functions can be calculated explicitly.

8.2 Examples of Complete Families of Asymptotically Comparable Functions In this section, we present three typical examples of complete families of asymptotically comparable functions, namely, families of asymptotically comparable functions of power, power exponential, and power-logarithmic types.

208

8 Asymptotically comparable functions

8.2.1 Asymptotically Comparable Power-Type Functions Let us consider the family of asymptotically comparable power-type functions H1 = {h(·)}, which includes all functions h(·) defined on the interval (0, 1], taking values in the interval (0, ∞) and such that, for any function h(·) ∈ H1 , there exist constants ah > 0 and bh ∈ (−∞, ∞) such that the following asymptotic relation holds: h(ε) → 1 as ε → 0. ah ε bh

(8.14)

We call ah and bh comparability parameters for the function h(·). Lemma 8.4 The following statements are true for the family of asymptotically comparable power-type functions H1 : (i) If the functions hi (·) ∈ H1, i = 1, 2, i.e., hi (ε)/ai ε bi → 1 as ε → 0, for i = 1, 2, where ai > 0, bi ∈ (−∞, ∞), i = 1, 2, then the following operating formulas for calculating the comparability parameters and limits for the sums, products, and quotients of these functions take place: (a) The function h (ε) = h1 (ε) + h2 (ε) ∈ H1 , and the relation (8.14) holds for this function, with comparability parameters, a  = a1 I(b1 < b2 ) + (a1 + a2 )I(b1 = b2 ) + a2 I(b1 > b2 ), b = b1 ∧ b2, i.e.,

h (ε)  → 1 as ε → 0. aεb

(8.15)

(8.16)

(b) The function h (ε) = h1 (ε) · h2 (ε) ∈ H1 , and the relation (8.14) holds for this function, with comparability parameters,

i.e.,

a  = a1 a2, b = b1 + b2,

(8.17)

h (ε) → 1 as ε → 0. a  ε b

(8.18)

(c) The function h  (ε) = h1 (ε) · h2 (ε)−1 ∈ H1 , and the relation (8.14) holds for this function, with comparability parameters

i.e.,

a  = a1 /a2, b = b1 − b2,

(8.19)

h (ε) → 1 as ε → 0. a  ε b

(8.20)

8.2 Examples of complete families of asymptotically comparable functions

209

(ii) If the function h(·) ∈ H1 , i.e., h(ε)/ah ε bh → 1 as ε → 0, for some ah > 0, bh ∈ (−∞, ∞), then there exists the corresponding comparability limit, lim h(ε) = aH1 [h(·)] =

ε→0

⎧ ⎪ ⎨ 0 if bh > 0, ⎪ ah if bh = 0, ⎪ ⎪ ∞ if bh < 0. ⎩

(8.21)

(iii) H1 is a complete family of asymptotically comparable functions. Proof First, suppose b1 < b2 . Then, h (ε) h1 (ε) a1 b1 −b h2 (ε) a2 b2 −b ε = ε + a  ε b a1 ε b1 a  a2 ε b2 a h1 (ε) h2 (ε) a2 b2 −b1 = + ε → 1 as ε → 0. a1 ε b1 a2 ε b2 a1

(8.22)

Second, suppose b1 = b2 . Then, h (ε) h1 (ε) a1 b1 −b h2 (ε) a2 b2 −b ε ε +  =  b aε a1 ε b1 a a2 ε b2 a h1 (ε) a1 h2 (ε) a2 = + → 1 as ε → 0. a1 ε b1 a1 + a2 a2 ε b2 a1 + a2

(8.23)

The third case, where b1 > b2 , is similar to the first considered above. Thus, the family H1 is closed with respect to the operation of summation. Also, the relation (8.14) holds for the function h( j) (ε) = h1 (ε)h2 (ε) j , with the j corresponding comparability parameters h j = a1 a2 and b j = b1 + b2 j, for j = ±1. Indeed, h1 (ε)h2 (ε) j h1 (ε)  h2 (ε)  j h( j) (ε) = → 1 as ε → 0. j b1 +b2 j = b j a1 ε b1 a2 ε b2 a1 a2 ε ajε

(8.24)

Thus, the family H1 is closed with respect to the operations of multiplication and division. Finally, the relation (8.14) obviously implies that, for any function h(·) ∈ H1 , for which the asymptotic relation (8.14) takes the form, h(ε)/ah ε bh → 1 as ε → 0, for some ah > 0, bh ∈ (−∞, ∞), there exists the following limit: lim h(ε) = lim ah ε bh

ε→0

ε→0

⎧ ⎪ ⎨ 0 if bh > 0, ⎪ = a[h(·)] = ah if bh = 0, ⎪ ⎪ ∞ if bh < 0. ⎩

(8.25)

Therefore, H1 is a complete family of asymptotically comparable functions. 

210

8 Asymptotically comparable functions

The following lemma generalises Lemma 8.4 to the case of multiple operations of summation, multiplication and division. Lemma 8.5 Let hi (·) ∈ H1, i = 1, 2, . . . and the asymptotic relation (8.14) takes for these functions the following forms, hi (ε)/ai ε bi → 1 as ε → 0, for i = 1, 2, . . ., where ai > 0, bi ∈ (−∞, ∞), i = 1, 2, . . .. Then the following operating formulas for calculating comparability parameters and limits for the multiple sums and products/quotients of these functions take place: n (a) The function h(n) (·) = i=1 hi (·) ∈ H1 , for n ≥ 1, and the relation (8.14) holds for this function with comparability parameters,  a(n) = ai, b(n) = min bi, (8.26) 1≤i ≤n: bi =b (n)

i.e.,

h(n) (ε) a(n) ε b(n)

1≤i ≤n

→ 1 as ε → 0,

(8.27)

and, thus, ⎧ if b(n) > 0, ⎪ ⎨0 ⎪ (n) lim h (ε) = aH1 [h (·)] = a if b(n) = 0, ⎪ ε→0 ⎪ ∞ if b(n) < 0. ⎩ (n)

(n)

(8.28)

n (b) The function h◦(n) (·) = i=1 hi (·) ji ∈ H1 , for ji = ±1, i = 1, 2, . . . , n and n ≥ 1, and the relation (8.14) holds for this function with comparability parameters,   j a◦(n) = ai i , b(n) bi ji, (8.29) ◦ = 1≤i ≤n

i.e.,

h◦(n) (ε) a◦(n) ε b◦

(n)

1≤i ≤n

→ 1 as ε → 0,

(8.30)

and, thus,

lim

ε→0

h◦(n) (ε)

=

aH1 [h◦(n) (·)]

(n) ⎧ ⎪ ⎪ ◦ > 0, ⎨ 0(n) if b(n) ⎪ = a◦ if b◦ = 0, ⎪ ⎪ ⎪ ∞ if b(n) ◦ < 0. ⎩

(8.31)

It is also worth mentioning the family of asymptotically comparable integerˆ 1 = {h(·)}, which includes all functions h(·) defined on the power-type functions H interval (0, 1], taking values in the interval (0, ∞) and such that, for any function ˆ 1 , there exist a constant ah > 0 and an integer bh ∈ (−∞, ∞) such that the h(·) ∈ H following asymptotic relation holds:

8.2 Examples of complete families of asymptotically comparable functions

h(ε) → 1 as ε → 0. ah ε bh

211

(8.32)

ˆ 1 is a complete family of asymptotically comparable functions, and Obviously, H ˆ 1 ⊂ H1 . H All relations appearing in Lemmas 8.4 and 8.5 hold for functions from the family ˆ 1 . The only difference is that the parameters b, b, b, b(n) , and b(n) H ◦ , which appears in the relations (8.15)–(8.31), are integers in this case. In applications, characteristics of perturbed stochastic models f (ε) are often linearly perturbed positive functions, which means that they admit the following asymptotic expansions, f (ε) = f0 + f1 ε + o f (ε) as ε → 0, where, (a) f0 > 0 or f0 = 0, f1 > 0, (b) o f (ε)/ε → 0 as ε → 0. ˆ 1 . Indeed, the following asymptotic Any such function f (ε) belongs to the family H b f relation holds, f (ε)/a f ε → 1 as ε → 0, where a f = f0, b f = 0, if f0 > 0, or a f = f1, b f = 1, if f0 = 0, f1 > 0. A case similar to the one above is where f (ε) are assumed to be nonlinearly perturbed functions, which means that they admit the following asymptotic expansions, f (ε) = f0 + f1 ε + · · · + fk ε k + o f (ε) as ε → 0, where: (a) fn = 0, n < r < k, fr > 0, for some 0 ≤ r ≤ k, (b) o f (ε)/ε k → 0 as ε → 0. ˆ 1 . Indeed, the following asymptotic Any such function f (ε) belongs to the family H b f relation holds, f (ε)/a f ε → 1 as ε → 0, where a f = fr , b f = r.

8.2.2 Asymptotically Comparable Power-Exponential-Type Functions Let us consider the family of asymptotically comparable power-exponential-type functions H2 = {h(·)}, which includes all functions h(·) defined on the interval (0, 1], taking values in the interval (0, ∞) and such that, for any function h(·) ∈ H2 , there exist constants ah > 0 and bh, ch ∈ (−∞, ∞) such that the following asymptotic relation holds: h(ε) → 1 as ε → 0. (8.33) −1 ah ε bh e−ch ε We call ah, bh , and ch comparability parameters for the function h(·). Lemma 8.6 The following statements are true for the family of asymptotically comparable power-exponential-type functions H2 : −1

(i) If the functions hi (·) ∈ H2, i = 1, 2, i.e., hi (ε)/ai ε bi e−ci ε → 1 as ε → 0, for i = 1, 2, where ai > 0, bi, ci ∈ (−∞, ∞), i = 1, 2, then the following operating formulas for calculating the comparability parameters and limits for the sums, products, and quotients of these functions take place: (a) The function

212

8 Asymptotically comparable functions

h (ε) = h1 (ε) + h2 (ε) ∈ H2 , and the relation (8.33) holds for this function, with comparability parameters, a  = a1 (I(c1 < c2 ) + I(c1 = c2, b1 < b2 )) + (a1 + a2 )I(c1 = c2, b1 = b2 ) + a2 (I(c1 > c2 ) + I(c1 = c2, b1 > b2 )), b = b1 I(c1 < c2 ) + (b1 ∧ b2 )I(c1 = c2 ) + b2 I(c1 > c2 ), 

c  = c1 ∧ c2,

i.e.,

(8.34) h (ε) a  ε b e−c ε −1

→ 1 as ε → 0.

(8.35)

(b) The function h (ε) = h1 (ε)h2 (ε) ∈ H2 , and the relation (8.33) holds for this function, with comparability parameters,

i.e.,

a  = a1 a,2 b = b1 + b2, c  = c1 + c2,

(8.36)

h (ε) → 1 as ε → 0.   −1 a  ε b e−c ε

(8.37)

(c) The function h (ε) = h1 (ε)/h2 (ε) ∈ H2 , and the relation (8.33) holds for this function, with comparability parameters,

i.e.,

a  = a1 /a,2 b = b1 − b2, c  = c1 − c2,

(8.38)

h (ε) → 1 as ε → 0. a  ε b e−c ε −1

(8.39)

−1

(ii) If the function h(·) ∈ H2 , i.e., h(ε)/ah ε bh e−ch ε → 1 as ε → 0, for some ah > 0, bh, ch ∈ (−∞, ∞), then there exists the corresponding comparability limit, ⎧ ⎪ ⎨ 0 if ch > 0 or ch = 0, bh > 0, ⎪ lim h(ε) = aH2 [h(·)] = ah if ch = 0, bh = 0, ⎪ ε→0 ⎪ ∞ if ch < 0 or ch = 0, bh < 0. ⎩ (iii) H2 is a complete family of asymptotically comparable functions. Proof First, suppose c1 < c2 or c1 = c2, b1 < b2 . Then,

(8.40)

8.2 Examples of complete families of asymptotically comparable functions

213

h (ε) a1 b1 −b −(c1 −c )ε −1 h1 (ε) = ε e  −c  ε −1 −1  b b −c ε a aε e a1 ε 1 e 1 +

h2 (ε) b ε 2 e−c2 ε −1

a2 b2 −b −(c2 −c )ε −1 ε e a a2 b2 −b1 −(c2 −c1 )ε −1 h2 (ε) + ε e −1 a2 ε b2 e−c2 ε a1

a2 h1 (ε) = −1 a1 ε b1 e−c1 ε

→ 1 as ε → 0.

(8.41)

Second, suppose c1 = c2, b1 = b2 . Then, h (ε) a1 b1 −b −(c1 −c )ε −1 h1 (ε) = ε e  ε −1 −1  b −c b −c ε 1 1 a aε e a1 ε e +

a2

h2 (ε) b ε 2 e−c2 ε −1

h1 (ε) −1 b ε 1 e−c1 ε

=

a1 → 1 as ε → 0.

a2 b2 −b −(c2 −c )ε −1 ε e a

a1 a2 h2 (ε) + −1 b −c ε a1 + a2 a2 ε 2 e 2 a1 + a2 (8.42)

The third case, where c1 > c2 or c1 = c2, b1 > b2 , is similar to the first case considered above. Thus, the family H2 is closed with respect to the operation of summation. Also, the relation (8.33) holds for the function h( j) (ε) = h1 (ε)h2 (ε) j , with the j corresponding comparability parameters h j = a1 a2 , b j = b1 + b2 j, and c j = c1 + c2 j, for j = ±1. Indeed, h( j) (ε) aj

−1 ε b j e−c j ε

= =

h1 (ε)h2 (ε) j j b1 +b2 j −(c1 +c2 j)ε −1 a1 a2 ε e a1

h1 (ε) b ε 1 e−c1 ε −1



a2

j h2 (ε) −1 b −c ε 2 2 ε e

→ 1 as ε → 0.

(8.43)

Thus, the family H2 is closed with respect to the operations of multiplication and division. Finally, the relation (8.33) obviously implies that, for any function h(·) ∈ H2 , −1 for which the asymptotic relation (8.33) takes the form, h(ε)/ah ε bh e−ch ε → 1 as ε → 0, for some ah > 0, bh, ch ∈ (−∞, ∞), there exists the following limit: lim h(ε) = lim ah ε bh e−ch ε

ε→0

−1

ε→0

= aH2 [h(·)] =

⎧ ⎪ ⎨ 0 if ch > 0 or ch = 0, bh > 0, ⎪ ah if ch = 0, bh = 0, ⎪ ⎪ ∞ if ch < 0 or ch = 0, bh < 0. ⎩

(8.44)

Therefore, H2 is a complete family of asymptotically comparable functions. 

214

8 Asymptotically comparable functions

The following lemma generalises Lemma 8.6 to the case of multiple operations of summation, multiplication and division. Lemma 8.7 Let hi (·) ∈ H2, i = 1, 2, . . . and the asymptotic relation (8.33) takes −1 for these functions the following forms, hi (ε)/ai ε bi e−ci ε → 1 as ε → 0, for i = 1, 2, . . ., where ai > 0, bi, ci ∈ (−∞, ∞), i = 1, 2, . . .. Then the following operating formulas for calculating comparability parameters and limits for the multiple sums and products/quotients of these functions take place: n (a) The function h(n) (·) = i=1 hi (·) ∈ H1 , for n ≥ 1, and the relation (8.14) holds for this function with comparability parameters,  a(n) = ai, b(n) = min bi, c(n) = min ci, (8.45) i.e.,

1≤i ≤n

1≤i ≤n:ci =c (n)

1≤i ≤n: bi =b (n),ci =c (n)

h(n) (ε) a(n) ε b(n) e−c(n) ε −1

→ 1 as ε → 0,

(8.46)

and, thus, ⎧ if c(n) > 0 or c(n) = 0, b(n) > 0, ⎪ ⎨0 ⎪ (n) lim h (ε) = aH2 [h (·)] = a if c(n) = 0, b(n) = 0, ⎪ ε→0 ⎪ ∞ if c(n) < 0 or c(n) = 0, b(n) < 0. ⎩ (n)

(n)

(8.47)

n (b) The function h◦(n) (·) = i=1 hi (·) ji ∈ H1 , for ji = ±1, i = 1, 2, . . . , n and n ≥ 1, and the relation (8.33) holds for this function with comparability parameters,    j a◦(n) = ai i , b(n) bi ji, c◦(n) = ci ji, (8.48) ◦ = 1≤i ≤n

i.e.,

1≤i ≤n

h◦(n) (ε) a◦(n) ε b◦ e−c◦ (n)

(n) −1 ε

1≤i ≤n

→ 1 as ε → 0,

(8.49)

and, thus,

lim

ε→0

h◦(n) (ε)

=

aH2 [h◦(n) (·)]

(n) ⎧ ⎪ c◦(n) = 0, b(n) ⎪ ◦ > 0, ⎨ 0(n) if c◦(n) > 0 or(n) ⎪ = a◦ if c◦ = 0, b◦ = 0, ⎪ ⎪ ⎪ ∞ if c◦(n) < 0 or c◦(n) = 0, b(n) ◦ < 0. ⎩

(8.50)

8.2.3 Asymptotically Comparable Power-Logarithmic-Type Functions Let us consider the family of asymptotically comparable power-logarithmic-type functions H3 = {h(·)}, which includes all functions h(·) defined on the interval (0, 1], taking values in the interval (0, ∞) and such that, for any function h(·) ∈ H2 ,

8.2 Examples of complete families of asymptotically comparable functions

215

there exist constants ah > 0 and bh, dh ∈ (−∞, ∞) such that the following asymptotic relation holds: h(ε) → 1 as ε → 0. (8.51) b h ah ε (1 + ln ε −1 )−dh We call ah, bh and dh comparability parameters for the function h(·). Lemma 8.8 The following statements are true for the family of asymptotically comparable power-logarithmic-type functions H3 : (i) If the functions hi (·) ∈ H3, i = 1, 2, i.e., hi (ε)/ai ε bi (1 + ln ε −1 )−di → 1 as ε → 0, for i = 1, 2, where ai > 0, bi, di ∈ (−∞, ∞), i = 1, 2, then the following operating formulas for calculating the comparability parameters and limits for the sums, products, and quotients of these functions take place: (a) The function h (ε) = h1 (ε) + h2 (ε) ∈ H3 , and the relation (8.51) holds for this function, with comparability parameters, a  = a1 (I(b1 < b2 ) + I(b1 = b2, d1 < d2 )) + (a1 + a2 )I(b1 = b2, d1 = d2 ) + a2 (I(b1 > b2 ) + I(b1 = b2, d1 > d2 )),

b = b1 ∧ b2, d  = d1 I(b1 < b2 ) + (d1 ∧ d2 )I(b1 = b2 ) + d2 I(b1 > b2 ). i.e., a  ε b (1

h (ε) → 1 as ε → 0. + ln ε −1 )−d

(8.52)

(8.53)

(b) The function h (ε) = h1 (ε)h2 (ε) ∈ H3 , and the relation (8.51) holds for this function, with comparability parameters,

i.e.,

a  = a1 a,2 b = b1 + b2, d  = d1 + d2,

(8.54)

h (ε)  → 1 as ε → 0. a  ε (1 + ln ε −1 )−d

(8.55)

b

(c) The function h (ε) = h1 (ε)/h2 (ε) ∈ H3 , and the relation (8.33) holds for this function, with comparability parameters, a  = a1 /a,2 b = b1 − b2, d  = d1 − d2, i.e., a  ε

b

h (ε) → 1 as ε → 0. (1 + ln ε −1 )−d

(8.56)

(8.57)

(ii) If the function h(·) ∈ H3 , i.e., h(ε)/ah ε bh (1 + ln ε −1 )−dh → 1 as ε → 0, for some ah > 0, bh, dh ∈ (−∞, ∞), then there exists the following limit:

216

8 Asymptotically comparable functions

⎧ ⎪ ⎨ 0 if bh > 0 or bh = 0, dh > 0, ⎪ lim h(ε) = aH3 [h(·)] = ah if bh = 0, dh = 0, ⎪ ε→0 ⎪ ∞ if bh < 0 or bh = 0, dh < 0. ⎩

(8.58)

(iii) H3 is a complete family of asymptotically comparable functions. Proof First, suppose b1 < b2 or b1 = b2, d1 < d2 . Then,  h (ε) a1 b1 −b h1 (ε) ε (1 + ln ε −1 )−(d1 −d )  =   b −1 −d b −1 −d 1 1 a a ε (1 + ln ε ) a1 ε (1 + ln ε )  a2 b2 −b h2 (ε) + ε (1 + ln ε −1 )−(d2 −d )  b −1 −d 2 2 a a2 ε (1 + ln ε ) h1 (ε) = a1 ε b1 (1 + ln ε −1 )−d1 a2 b2 −b1 h2 (ε) + ε (1 + ln ε −1 )−(d2 −d1 ) a2 ε b2 (1 + ln ε −1 )−d2 a1 → 1 as ε → 0. (8.59)

Second, suppose b1 = b2, d1 = d2 . Then,  h (ε) a1 b1 −b h1 (ε) ε (1 + ln ε −1 )−(d1 −d )  =   b −1 −d b −1 −d 1 1 a a ε (1 + ln ε ) a1 ε (1 + ln ε )  a2 b2 −b h2 (ε) + ε (1 + ln ε −1 )−(d2 −d )  b −1 −d 2 2 a a2 ε (1 + ln ε ) a1 h1 (ε) = a1 ε b1 (1 + ln ε −1 )−d1 a1 + a2 a2 h2 (ε) + a2 ε b2 (1 + ln ε −1 )−d2 a1 + a2 → 1 as ε → 0. (8.60)

Thus, the family H3 is closed with respect to the operation of summation. Also, the relation (8.51) holds for the function h( j) (ε) = h1 (ε)h2 (ε) j , with the j comparability parameters h j = a1 a2 , b j = b1 + b2 j, and d j = d1 + d2 j for j = ±1. Indeed,

aj

h1 (ε)h2 (ε) j h( j) (ε) = j b1 +b2 j −d −1 j a1 a2 ε (1 + ln ε −1 )−(d1 +d2 j) + ln ε )  j h2 (ε) h1 (ε) = b −1 −d b −1 −d a1 ε 1 (1 + ln ε ) 1 a2 ε 2 (1 + ln ε ) 2 → 1 as ε → 0. (8.61)

ε b j (1

Thus, the family H3 is closed with respect to the operations of multiplication and division.

8.2 Examples of complete families of asymptotically comparable functions

217

Finally, the relation (8.51) obviously implies that, for any function h(·) ∈ H3 , for which the asymptotic relation (8.51) holds, i.e., h(ε)/ah ε bh (1 + ln ε −1 )−dh → 1 as ε → 0, for some ah > 0, bh, dh ∈ (−∞, ∞), there exists the following limit: lim h(ε) = lim ah ε bh (1 + ln ε −1 )−dh

ε→0

ε→0

= aH3 [h(·)] =

⎧ ⎪ ⎨ 0 if bh > 0 or bh = 0, dh > 0, ⎪ ah if bh = 0, dh = 0, ⎪ ⎪ ∞ if bh < 0 or bh = 0, dh < 0. ⎩

(8.62)

Therefore, H3 is a complete family of asymptotically comparable functions.  The following lemma generalises Lemma 8.8 to the case of multiple operations of summation, multiplication, and division. Lemma 8.9 Let hi (·) ∈ H3, i = 1, 2, . . . and the asymptotic relation (8.51) takes for these functions the following forms, hi (ε)/ai ε bi (1+ln ε −1 )−di → 1 as ε → 0, for i = 1, 2, . . ., where ai > 0, bi, di ∈ (−∞, ∞), i = 1, 2, . . .. Then the following operating formulas for calculating comparability parameters and limits for the multiple sums and products/quotients of these functions take place: n (a) The function h(n) (·) = i=1 hi (·) ∈ H3 , for n ≥ 1, and the relation (8.14) holds for this function with comparability parameters,  a(n) = ai, b(n) = min bi, d (n) = min di, (8.63) 1≤i ≤n

1≤i ≤n: di =d (n),bi =b (n)

i.e.,

h(n) (ε) (n)

a(n) ε b (1 + ln ε −1 )−d

(n)

1≤i ≤n:bi =b (n)

→ 1 as ε → 0,

(8.64)

and, thus, ⎧ if b(n) > 0 or b(n) = 0, d (n) > 0, ⎪ ⎨0 ⎪ (n) lim h (ε) = aH3 [h (·)] = a if b(n) = 0, d (n) = 0, ⎪ ε→0 ⎪ ∞ if b(n) < 0 or b(n) = 0, d (n) < 0. ⎩ (n)

(n)

(8.65)

n (b) The function h◦(n) (·) = i=1 hi (·) ji ∈ H1 , for ji = ±1, i = 1, 2, . . . , n and n ≥ 1, and the relation (8.33) holds for this function with comparability parameters,    j a◦(n) = ai i , b(n) bi ji, d◦(n) = di ji, (8.66) ◦ = 1≤i ≤n

1≤i ≤n

i.e.,

h◦(n) (ε) a◦(n) ε b◦ (1 + ln ε −1 )−d◦ (n)

and, thus,

(n)

1≤i ≤n

→ 1 as ε → 0,

(8.67)

218

8 Asymptotically comparable functions

lim

ε→0

h◦(n) (ε)

=

aH2 [h◦(n) (·)]

(n) (n) (n) ⎧ ⎪ ⎪ ◦ > 0 or b◦ = 0, d◦ > 0, ⎨ 0(n) if b(n) ⎪ (n) = a◦ if b◦ = 0, d◦ = 0, ⎪ ⎪ (n) (n) ⎪ ∞ if b(n) ◦ < 0 or b◦ = 0, d◦ < 0. ⎩

(8.68)

In conclusion, we note that many other examples of complete families of asymptotically comparable functions based on power, exponential, logarithmic, iterated logarithmic, and other types of functions can be constructed in a similar way.

Chapter 9

Perturbed Semi-Markov Processes and Reduction of Phase Space

In this chapter, we present models of regularly and singularly perturbed semi-Markov processes and formulate basic perturbation conditions and asymptotic comparability conditions based on the notion of a complete family of asymptotically comparable functions. We also describe asymptotic procedures of removing virtual transitions and one-state reducing the phase space, and an asymptotic recurrent phase space reduction algorithm for perturbed semi-Markov processes. These procedures and algorithm are provided by explicit formulas for recalculating normalisation functions, Laplace transforms for limit distributions, and limit expectations appearing in perturbation conditions for semi-Markov processes with reduced phase space. This chapter includes four sections. In Sect. 9.1, models of regularly and singularly perturbed semi-Markov processes are introduced and basic perturbation conditions are formulated. In Sect. 9.2, the asymptotic procedure of removing virtual transition for perturbed semi-Markov processes is described. The corresponding recurrent formulas for re-calculating normalisation functions, limiting distributions and expectations in the corresponding perturbation conditions are given in Lemmas 9.3–9.13 and summarised in Theorem 9.1. In Sect. 9.3, the asymptotic procedure of one-state reduction of phase space for perturbed semi-Markov processes is described. The corresponding recurrent formulas for re-calculating normalisation functions, limiting distributions and expectations in the corresponding perturbation conditions are given in Lemmas 9.14–9.25 and summarised in Theorem 9.2. In Sect. 9.4, the asymptotic recurrent algorithm of phase space reduction for perturbed semi-Markov processes is described. The invariance properties of the perturbation conditions with respect to the above asymptotic recurrent phase space reduction algorithm are summarised in Theorem 9.3.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_9

219

220

9 Perturbed SMP and reduction of phase space

9.1 Perturbed Semi-Markov Processes In Sect. 9.1, models of regularly and singularly perturbed semi-Markov processes are introduced and basic perturbation conditions are formulated.

9.1.1 Perturbed Semi-Markov Processes Let X = {1, . . . , m} be a finite set. Let also (ηε,n, κε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1], a Markov renewal process, i.e., a homogeneous Markov chain, with the phase space X × [0, ∞) and transition probabilities, for (i, s), ( j, t) ∈ X × [0, ∞), Q ε,i j (t) = P{ηε,1 = j, κε,1 ≤ t/ηε,0 = i, κε,0 = s}.

(9.1)

The first component of the Markov renewal process ηε,n, n = 0, 1, . . . is itself a homogeneous, so-called embedded, Markov chain, with the phase space X and transition probabilities, for i, j ∈ X, pε,i j = Q ε,i j (∞) = P{ηε,1 = j, /ηε,0 = i}.

(9.2)

The above Markov renewal process is used to define a semi-Markov process, ηε (t) = ηε,νε (t), t ≥ 0,

(9.3)

where ζε,n = κε,1 + · · · + κε,n, n = 1, 2, . . . , ζε,0 = 0, are the corresponding moments of jumps, and νε (t) = max(n ≥ 1 : ζε,n ≤ t) is the number of jumps in the interval [0, t] for the above semi-Markov process. Perturbed semi-Markov processes are the main objects of study in the book. The basic fact concerning these processes can be found in the books listed in Sect. B.2.7. Here we would like to mention again two important cases. The semi-Markov process ηε (t) is a continuous time Markov chain, if its transition probabilities, Q ε,i j (t) = (1−e−λε, i t )pε,i j , t ≥ 0, i, j ∈ X or a discrete time Markov chain embedded in continuous time, if its transition probabilities, Q ε,i j (t) = I(t ≥ 1)pε,i j , t ≥ 0, i, j ∈ X. Let us introduce the distribution function of sojourn time κε,n for the semi-Markov process ηε (t), for i ∈ X,  Fε,i (t) = Pi {κε,1 ≤ t} = Q ε,i j (t), t ≥ 0. (9.4) j ∈X

We assume that the following regularity condition is satisfied: G: (a) pε,i j > 0, ε ∈ (0, 1] or pε,i j = 0, ε ∈ (0, 1], for every i, j ∈ X, (b) Fε,i (0) < 1, for i ∈ X, ε ∈ (0, 1]. Let us introduce sets, for i ∈ X and ε ∈ (0, 1],

9.1 Perturbed SMP

221

Yε,i = { j ∈ X : pε,i j > 0}.

(9.5)

The following obvious lemma presents an equivalent form of the condition G (a). Lemma 9.1 The condition G (a) is satisfied if and only if sets Yε,i = Y1,i, ε ∈ (0, 1], for i ∈ X. The condition G (b) guarantees that Pi {limn→∞ ζε,n = ∞} = 1, for any i ∈ X and ε ∈ (0, 1]. Therefore, the process ηε (t), t ≥ 0 is well defined on the time interval [0, ∞), for each ε ∈ (0, 1]. We also assume that the following ergodicity condition is satisfied: H: For any i, j ∈ X, there exist ni j ≥ 1 and a chain of states i = r0, r1, . . . , rni j = j such that 1≤l ≤ni j p1,rl−1 rl > 0. The condition G (a) implies that the product 1≤l ≤ni j pε,rl−1 rl is a positive number or equals zero simultaneously for all ε ∈ (0, 1]. Therefore, the condition H (b) implies that the phase space X is one class of communicative states for the embedded Markov chain ηε,n , and, thus, this Markov chain is ergodic, for every ε ∈ (0, 1].

9.1.2 Perturbation Conditions In what follows, we assume that the conditions G and H are satisfied. We also use that the following perturbation condition: I: pε,i j → p0,i j as ε → 0, for i, j ∈ X. Since the matrix Pε = pε,i j  is stochastic, for ε ∈ (0, 1], the condition I implies that the matrix P0 = p0,i j  is also stochastic. Let η0,n, n = 0, 1, . . . be a Markov chain with the phase space X and the matrix of transition probabilities P0 . Condition I allows us to interpret the Markov chain ηε,n , for ε ∈ (0, 1], as a perturbed version of the Markov chain η0,n . The case, where the phase space X is one class of communicative states plus, possibly, the class of transient states for the Markov chain η0,n , refers to the model with regular perturbations. The case, where the phase space X is split into several closed classes of communicative states plus, possibly, the class of transient states for the Markov chain η0,n , refers to the model with singular perturbations. Let us introduce sets, for i ∈ X, Y0,i = { j ∈ X : p0,i j > 0}.

(9.6)

222

9 Perturbed SMP and reduction of phase space

Lemma 9.2 The conditions G and I imply that Y0,i ⊆ Y1,i , for i ∈ X. The transition probabilities Q ε,i j (t) can, for every ε ∈ (0, 1], be represented in the following form, for t ≥ 0, i, j ∈ X: Q ε,i j (t) = Fε,i j (t)pε,i j ,

(9.7)

Fε,i j (t) = P{κε,1 ≤ t/ηε,0 = i, ηε,1 = j}.

(9.8)

where If j ∈ Y1,i, i ∈ X, then, Fε,i j (t) = p−1 ε,i j Q ε,i j (t), t ≥ 0.

(9.9)

Let us recall the distribution function of the sojourn time in state i ∈ X for the semi-Markov process ηε (t), Fε,i (t) = P{κε,1 ≤ t/ηε,0 = i}    Q ε,i j (t) = Q ε,i j (t) = Fε,i j (t)pε,i j . = j ∈X

j ∈Y1, i

(9.10)

j ∈Y1, i

¯ 1,i, i ∈ X, an arbitrary distribution function concentrated on [0, ∞) can If j ∈ Y play the role of Fε,i j (t). ¯ 1,i, i ∈ X does not affect the transition probabilities The choice of Fε,i j (t) for j ∈ Y Q ε,i j (t). In any case, the following relations hold, for t ≥ 0, i, j ∈ X and ε ∈ (0, 1]: Q ε,i j (t) = Fε,i j (t)pε,i j .

(9.11)

In what follows, we use the standard variant and choose, Fε,i j (t) = Fε,i (t), t ≥ 0.

(9.12)

In the case of continuous and discrete time Markov chains, Fε,i j (t) = Fε,i (t) = 1 − e−λε, i t , t ≥ 0, i ∈ X and Fε,i j (t) = Fε,i (t) = I(t ≥ 1), t ≥ 0, i ∈ X, respectively. We also use the following perturbation condition: J: (a) Fε,i j (· uε,i ) ⇒ F0,i j (·) as ε → 0, for j ∈ Y1,i, i ∈ X, (b) F0,i j (·), j ∈ Y1,i, i ∈ X are proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ X, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X. We call the functions uε,i “initial local” normalisation functions. The standard case is, where all initial normalisation functions uε,i = 1, ε ∈ (0, 1]. The conditions G, I, and J imply that the following relation of weak convergence holds, for i ∈ X:

9.1 Perturbed SMP

223



Fε,i (·uε,i ) =

Fε,i j (·uε,i )pε,i j

j ∈Y1, i

⇒



F0,i j (·)p0,i j

j ∈Y1, i



=

F0,i j (·)p0,i j = F0,i (·) as ε → 0.

(9.13)

j ∈Y0, i

The condition J and Lemma 9.2 imply that F0,i (t) is a proper distribution function such that F0,i (0) < 1, for i ∈ X. The relations (9.12) and (9.13) imply that, in fact, the weak convergence relation appearing in the condition J is satisfied for all i, j ∈ X. The condition J (b) guarantees that the initial local normalisation functions uε,i do not deliver excessive levels of local normalisation. Let us introduce Laplace transforms, for i, j ∈ X and ε ∈ (0, 1], ∫ ∞ φε,i j (s) = e−st Fε,i j (dt), s ≥ 0 (9.14) 0

∫

and φε,i (s) =

0

∞

e−st Fε,i (dt), s ≥ 0.

(9.15)

¯ 1,i, i ∈ X and ε ∈ (0, 1], According to the relation (9.12), for j ∈ Y φε,i j (s) = φε,i (s), s ≥ 0.

(9.16)

The condition J can also be re-formulated in the following equivalent form: J◦ :

(a) φε,i j (s/u ∫ ∞ε,i ) → φ0,i j (s) as ε → 0, for s ≥ 0 and j ∈ Y1,i, i ∈ X, (b) φ0,i j (s) = 0 e−st F0,i j (dt), s ≥ 0, j ∈ Y1,i, i ∈ X are Laplace transforms of proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ X, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X.

It is useful to note that the relation, F0,i j (0) < 1 holds if and only if φ0,i j (s) < 1, s > 0. Note also that all these inequalities hold if φ0,i j (s0 ) < 1, for some s0 > 0. The conditions G, I, and J◦ imply that the following convergence relation holds, for i ∈ X and s ≥ 0:  φε,i (suε,i ) = φε,i j (suε,i )pε,i j j ∈Y1, i

⇒



φ0,i j (s)p0,i j

j ∈Y1, i

=



φ0,i j (s)p0,i j = φ0,i (s) as ε → 0.

(9.17)

j ∈Y0, i

The relations (9.15) and (9.17) imply that, in fact, the convergence relation appearing in the condition J◦ is satisfied for all i, j ∈ X.

224

9 Perturbed SMP and reduction of phase space

Let us now introduce the semi-Markov transition probabilities, for t ≥ 0, i, j ∈ X,

F0,i j (t)p0,i j for j ∈ Y0,i, Q0,i j (t) = (9.18) ¯ 0,i . 0 for j ∈ Y The probabilities Q0,i j (t), t ≥ 0, i, j ∈ X can serve as transition probabilities for some Markov renewal process (η0,n, κ0,n ), n = 0, 1, . . ., with the phase space X × [0, ∞). We also can define the corresponding semi-Markov process η0 (t) = η0,ν0 (t), t ≥ 0, where ζ0,n = κ0,1 + · · · + κ0,n, n = 1, 2, . . . , ζ0,0 = 0, are the corresponding instants of jumps, and ν0 (t) = max(n ≥ 1 : ζ0,n ≤ t) is the number of jumps in the interval [0, t] for the above semi-Markov process. As mentioned above, the probabilities F0,i (0) < 1, for i ∈ X. This implies that Pi {limn→∞ ζ0,n = ∞} = 1, for i ∈ X. Thus, the process η0 (t) is well defined on the time interval [0, ∞). The conditions I, J and Lemma 9.2 imply that, for i, j ∈ X, Q ε,i j (·) ⇒ Q0,i j (·) as ε → 0.

(9.19)

Also, according to the condition I, for i, j ∈ X, Q ε,i j (∞) = pε,i j → p0,i j = Q0,i j (∞) as ε → 0.

(9.20)

The relations (9.19) and (9.20) allow us to interpret the semi-Markov process ηε (t), for ε ∈ (0, 1] as a perturbed version of the semi-Markov process η0 (t). Let us also introduce the expectations of inter-jump times, for i, j ∈ X and ε ∈ (0, 1], ∫ fε,i j =

∞

0

∫

and fε,i =

0

∞

tFε,i j (dt),

(9.21)

tFε,i (dt).

(9.22)

¯ 1,i, i ∈ X and ε ∈ (0, 1], According to the relation (9.12), for j ∈ Y fε,i j = fε,i .

(9.23)

We also use the following perturbation condition: K: (a) ∫ ∞ fε,i j < ∞, j ∈ Y1,i, i ∈ X, for ε ∈ (0, 1], (b) fε,i j /uε,i → f0,i j = tF0,i j (dt) < ∞ as ε → 0, for j ∈ Y1,i, i ∈ X. 0 Note that the perturbation conditions J and K assume not only weak convergence of the distribution functions Fε,i j (·uε,i ) to the distribution function F0,i j (·), as ε → 0, but also the convergence of their first moments fε,i j /uε,i as ε → 0 to the corresponding first moments f0,i j of the distribution function F0,i j (·), for j ∈ Y1,i, i ∈ X. Note also that the conditions J and K imply that, for j ∈ Y1,i, i ∈ X, f0,i j ∈ (0, ∞).

(9.24)

9.1 Perturbed SMP

225

The conditions G, I, J, and K also imply that the following asymptotic relation holds, for i ∈ X:  fε,i j fε,i = pε,i j uε,i j ∈Y uε,i 1, i   → f0,i j p0,i j = f0,i j p0,i j j ∈Y1, i

j ∈Y0, i

= f0,i as ε → 0.

(9.25)

It is useful to note that the expectation f0,i ∈ (0, ∞), for i ∈ X. The relation (9.25) implies that, in fact, the convergence relation given in the condition K holds for all i, j ∈ X. Let us denote, for i, j ∈ X and ε ∈ [0, 1], ∫ ∞ sQ ε,i j (ds). (9.26) eε,i j = 0

It also follows from the conditions G, I, J, and K that the following asymptotic relation complementing the relations (9.19) and (9.20) holds, for i, j ∈ X: u−1 ε,i eε,i j → e0,i j as ε → 0.

(9.27)

In what follows, we also impose on the initial normalisation functions uε,i the following natural asymptotic stability condition: L: uε,i → u0,i ∈ (0, ∞] as ε → 0, for i ∈ X. In the standard case, where all initial normalisation functions uε,i = 1, ε ∈ (0, 1], for i ∈ X, the condition L automatically holds, with limits u0,i = 1, i ∈ X.

9.1.3 Additional Asymptotic Comparability Conditions The recurrent asymptotic analysis of perturbed semi-Markov processes requires to impose some stronger conditions on transition probabilities pε,i j and normalisation functions vε,i . Let us introduce the following condition of asymptotic comparability for transition probabilities based on some complete family of asymptotically comparable functions H: IH : The functions p ·,i j , j ∈ Y1,i, i ∈ X belong to a complete family of asymptotically comparable functions H. Note that the condition IH implies the fulfilment of the condition I. Since, the matrix Pε = pε,i j  is stochastic, for ε ∈ [0, 1], the corresponding comparability limits aH [p ·,i j ] satisfy the following relation:

226

9 Perturbed SMP and reduction of phase space

aH [p ·,i j ] = p0,i j ∈ [0, 1], i, j ∈ X,



aH [p ·,i j ] = 1, i ∈ X.

(9.28)

j ∈X

The perturbation analysis and related recurrent algorithms of phase space reduction presented in this chapter also involve other probabilities such as, p¯ε,ii = 1 − pε,ii , p˜ε,i j = pε,i j / p¯ε,ii , k pε,i j = p˜ε,i j + p˜ε,ik p˜ε,k j , k qε,i j = p˜ε,i j / k pε,i j , k p¯ε,ii = 1 − k p¯ε,ii , etc. In all cases, the condition IH and Lemmas 8.2 and 8.3 imply that the above functions belong to the family H and, thus, they converge to some limits, as ε → 0. Moreover, if the role of H is played by one of the families H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated explicitly as rational functions of the comparability parameters and limits for transition probabilities pε,i j using the operating rules and formulas given in Lemmas 8.4–8.9. Since, the matrix Pε = pε,i j  is stochastic, for ε ∈ [0, 1], the conditions IH1 take the following form: IH1 :

pε, i j ai j ε b i j

→ 1 as ε → 0, where ai j > 0, bi j ∈ [0, ∞), j ∈ Y1,i, i ∈ X.

The condition IH1 implies that, for j ∈ Y1,i, i ∈ X,

0 if bi j > 0, lim pε,i j = p0,i j = ai j if bi j = 0. ε→0 Moreover, the following relation holds:  ai j = 1, i ∈ X.

(9.29)

(9.30)

j ∈Y1, i : bi j =0

Similarly, the condition IH2 takes the following form: IH2 :

pε, i j ai j ε b i j e−c i j ε

−1

→ 1 as ε → 0, where ai j > 0, bi j , ci j ∈ [0, ∞), j ∈ Y1,i, i ∈ X.

The condition IH2 implies that, for j ∈ Y1,i, i ∈ X,

0 if ci j > 0 or ci j = 0, bi j > 0, lim pε,i j = p0,i j = ai j if bi j , ci j = 0. ε→0 Moreover, the following relation holds:  ai j = 1, i ∈ X.

(9.31)

(9.32)

j ∈Y1, i : bi j ,ci j =0

Finally, the condition IH3 takes the following form: IH3 :

pε, i j

ai j ε b i j (1+ln ε −1 )−d i j

Y1,i, i ∈ X.

→ 1 as ε → 0, where ai j > 0, bi j , di j ∈ [0, ∞), j ∈

9.1 Perturbed SMP

227

The condition IH3 implies that, for j ∈ Y1,i, i ∈ X,

0 if bi j > 0 or bi j = 0, di j > 0, lim pε,i j = p0,i j = ai j if bi j , di j = 0. ε→0 Moreover, the following relation holds,  ai j = 1, i ∈ X.

(9.33)

(9.34)

j ∈Y1, i : bi j ,di j =0

Let us also use the following condition of asymptotic comparability for normalisation functions based on the same complete family of asymptotically comparable functions H as in the condition IH : LH : The functions u ·,i, i ∈ X belong to the complete family of asymptotically comparable functions H appearing in the condition IH . The conditions L and LH imply that the corresponding comparability limits aH [u ·,i ] satisfy the following relation: aH [u ·,i ] = u0,i ∈ (0, ∞], i ∈ X.

(9.35)

Note also that in the case of standard initial normalisation functions u ·,i ≡ 1, i ∈ X, condition LH is automatically satisfied for any family of asymptotically comparable functions H. The perturbation analysis and related recurrent algorithms of phase space reduction presented in this chapter also involve other functions such as, u˜ε,i = p¯−1 ε,ii uε,i , w˜ ε,i j = u˜ε,i /u˜ε, j , k u˜ε,i = k p¯−1 u ˜ , etc. ε,ii ε,i In all cases, the conditions IH , LH and Lemmas 8.2 and 8.3 imply that the above functions belong to the family H and, thus, they converge to some limits, as ε → 0. Moreover, if the role of H is played by one of the families H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated explicitly as rational functions of the comparability parameters and limits for transition probabilities pε,i j and the initial normalisation functions uε,i , using the operating rules and formulas given in Lemmas 8.4–8.9. The condition LH1 takes the following form: LH1 :

uε, i ai ε b i

→ 1 as ε → 0, where ai > 0, bi ∈ (−∞, 0], i ∈ X.

The condition LH1 implies that, for i ∈ X,

a if bi = 0, lim uε,i = u0,i = i ∞ if bi < 0. ε→0 Similarly, the condition LH2 takes the following form: LH2 :

uε, i ai ε b i e−c i ε

−1

→ 1 as ε → 0, where ai > 0, bi, ci ∈ (−∞, 0], i ∈ X.

The condition LH2 implies that, for i ∈ X,

(9.36)

228

9 Perturbed SMP and reduction of phase space

lim uε,i = u0,i =

ε→0

ai if bi j , ci j = 0, ∞ if ci j < 0 or ci j = 0, bi j < 0.

(9.37)

Finally, the condition LH3 takes the following form: LH3 :

uε, i ai ε b i (1+ln ε −1 )−d i

→ 1 as ε → 0, where ai > 0, bi, di ∈ (−∞, 0], i ∈ X.

The condition LH3 implies that, for i ∈ X,

a if bi, di = 0, lim uε,i = u0,i = i ∞ if bi < 0 or bi = 0, di < 0. ε→0

(9.38)

9.1.4 Perturbed Markov Chains In the case of perturbed continuous and discrete Markov chains conditions G, H, I, IH , L, and LH do not change. In the case of continuous time Markov chains (see, the relation (1.47)), the condition J takes the form of the asymptotic relation 0 < λε,i → [0, ∞) as ε → 0, for i ∈ X. In this case the role of initial normalisation functions is played by functions u ·,i = λ−1 ·,i , i ∈ X. In this case the condition J is obviously satisfied and the corresponding limiting distribution functions F0,i j (t) = 1 − e−t , t ≥ 0, j ∈ Y1,i, i ∈ X are exponential with parameter 1. The condition K is also satisfied, with the corresponding limiting expectations f0,i j = 1, j ∈ Y1,i, i ∈ X. In the case of discrete time Markov chains (see, the relation (1.48)), the condition J is automatically satisfied, with the corresponding initial normalisation functions u ·,i ≡ 1, i ∈ X and the limiting distribution functions F0,i j (t) = I(t ≥ 1), t ≥ 0, j ∈ Y1,i, i ∈ X. The condition K is also satisfied, with the corresponding limiting expectations f0,i j = 1, j ∈ Y1,i, i ∈ X.

9.2 Removing of Virtual Transitions for Perturbed Semi-Markov Processes In Sect. 9.2, we describe the procedure of removing virtual transitions of the form i → i from the trajectories of perturbed semi-Markov processes. It is shown that the basic perturbation conditions imposed on the original semi-Markov processes are also satisfied for the semi-Markov processes with removed virtual transitions. We also give explicit formulas for recalculating normalisation functions, limiting distributions and expectations in the corresponding perturbation conditions for semiMarkov processes with removed virtual transitions.

9.2 Removing of virtual transitions

229

9.2.1 Semi-Markov Processes with Removed Virtual Transitions 9.2.1.1 Definition of Semi-Markov Processes with Removed Virtual Transitions. As above, we assume that conditions G and H are satisfied. In what follows in this section, we also assume that ε ∈ (0, 1]. Let us define the stopping times for the Markov chain ηε,n , for r = 0, 1, . . ., θ ε [r] = min(n > r : ηε,n  ηε,r ).

(9.39)

By the definition, θ ε [r] is the first moment after r, when the Markov chain ηε,n changes its state. We also define the following successive stopping times: ρε,n = θ ε [ρε,n−1 ], n = 1, 2, . . . , where ρε,0 = 0.

(9.40)

We now construct a new Markov renewal process (η˜ε,n, κ˜ε,n ), n = 0, 1, . . . with the phase space X × [0, ∞),

(ηε,0, 0), for n = 0, ρε, n (η˜ε,n, κ˜ε,n ) = (η (9.41) , κ ), for n = 1, 2, . . . . ε,ρε, n l=ρε, n−1 +1 ε,l We can also define the corresponding semi-Markov process, η˜ε (t) = η˜ε,ν˜ ε (t), t ≥ 0,

(9.42)

ζ˜ε,n = κ˜ε,1 + · · · + κ˜ε,n, n = 1, 2, . . . , ζ˜ε,0 = 0,

(9.43)

where are the successive moments of jumps, and ν˜ε (t) = max(n ≥ 1 : ζ˜ε,n ≤ t),

(9.44)

is the number of jumps in the interval [0, t] for the above semi-Markov process. Below, we use the symbol ∗ and the superscript (∗n) to denote convolution operation and n-fold convolution for a proper or improper distribution function. The definition of stopping times ρε,n implies that the transition probabilities for the above Markov renewal process are determined by the following relation: Q˜ ε,i j (t) = P{η˜ε,1 = j, κ˜ε,1 ≤ t/η˜ε,0 = i} ∞  = I( j  i) Q(∗n) ε,ii (t) ∗ Q ε,i j (t), t ≥ 0, i, j ∈ X,

(9.45)

n=0

where, as usual, Q(∗0) ε,ii (t) = I(t ≥ 0), t ≥ 0. Accordingly, the transition probabilities of the embedded Markov chain η˜ε,n are given by the following relation:

230

9 Perturbed SMP and reduction of phase space

p˜ε,i j = P{η˜ε,1 = j/η˜ε,0 = i} pε,i j = I( j  i) , i, j ∈ X, p¯ε,ii where p¯ε,ii = 1 − pε,ii =



pε,i j , i ∈ X.

(9.46)

(9.47)

ji

Note that the conditions G and H guarantee that the probabilities p¯ε,ii > 0, ε ∈ (0, 1], for i ∈ X. ˜ For i ∈ X and ε ∈ (0, 1], we introduce the distribution 9.2.1.2 Condition G. functions of the sojourn times for the semi-Markov process η˜ε (t),  (9.48) Q˜ ε,i j (t), t ≥ 0. F˜ε,i (t) = Pi { κ˜ε,i ≤ t} = j ∈X

The following condition plays the role of the condition G for the semi-Markov processes η˜ε (t): ˜ (a) p˜ε,i j > 0, ε ∈ (0, 1] or p˜ε,i j = 0, ε ∈ (0, 1], for every i, j ∈ X, (b) F˜ε,i (0) < G: 1, i ∈ X, ε ∈ (0, 1]. Lemma 9.3 The conditions G and H, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition G for the semi-Markov processes ˜ η˜ε (t) in the form of condition G. Proof The condition G (a) and the relation (9.46) obviously imply that the condition ˜ (a) is satisfied. G Since, Q ε,ii (0) ≤ Fε,i (0) < 1, for i ∈ X, the condition G (b) and the relations (9.45) imply that, for i ∈ X and ε ∈ (0, 1], F˜ε,i (0) =

∞  n=0

n Q ε,ii (0)



Q ε,i j (0) =

ji

Fε,i (0) − Q ε,ii (0) < 1. 1 − Q ε,ii (0)

(9.49)

Thus, the regularity condition G (b) is also satisfied for the semi-Markov processes η˜ε (t).  ˜ implies that the process η˜ε (t) is well defined on the time interval The condition G [0, ∞), for each ε ∈ (0, 1]. Let us introduce sets, for i ∈ X and ε ∈ (0, 1, ˜ ε,i = { j ∈ X : p˜ε,i j > 0}. Y

(9.50)

The following obvious lemma takes place. ˜ (a) is satisfied if and only if the sets Y ˜ ε,i = Y ˜ 1,i, ε ∈ Lemma 9.4 The condition G (0, 1], for i ∈ X, and the sets, ˜ 1,i = Y1,i \ {i}, i ∈ X. Y

(9.51)

9.2 Removing of virtual transitions

231

˜ The following condition plays the role of the condition H 9.2.1.3 Condition H. for the semi-Markov processes η˜ε (t): ˜ For any i, j ∈ X, there exist n˜ i j ≥ 1 a chain of states i = r˜0, r˜1, . . . , r˜n˜ i j = j such H: that 1≤l ≤ n˜ i j p˜1, r˜l−1 r˜l > 0. Lemma 9.5 The conditions G and H, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition H for the semi-Markov processes ˜ η˜ε (t) in the form of condition H. Proof It can always be assumed that a chain of states r0, . . . , rni j appearing in the condition H satisfies that rl−1  rl, l = 1, . . . , ni j . For nassumption ni j the additional ij p˜ε,rl−1 rl ≥ l=1 pε,rl−1 rl > 0. Therefore, the condition H is any such chain, l=1  satisfied for the semi-Markov processes η˜ε (t).

9.2.2 Perturbation Conditions for Semi-Markov Processes with Removed Virtual Transitions We are going to describe assumptions, under which basic conditions I–L and IH , LH hold for the semi-Markov processes η˜ε (t) constructed with the use of the described above procedure of removing virtual transitions for the semi-Markov processes ηε (t). 9.2.2.1 Conditions I˜ and I˜ H . The following conditions play the roles of the conditions I and IH for the semi-Markov processes η˜ε (t): ˜ p˜ε,i j = I( j  i) pε, i j → p˜0,i j ∈ [0, 1] as ε → 0, for i, j ∈ X I: p¯ ε, ii and ˜ 1,i, i ∈ X belong to the complete family of asymptotically I˜ H : Functions p˜ ·,i j , j ∈ Y comparable functions H appearing in the condition IH . ˜ implies that the probabilities p¯ε,ii > 0, i ∈ X, for Recall that the condition H every ε ∈ (0, 1]. Since, in this case, the matrix P˜ ε =  p˜ε,i j  is stochastic, for ε ∈ (0, 1], the condition I˜ implies that the matrix P˜ 0 =  p˜0,i j  is also stochastic. Let η˜0,n, n = 0, 1, . . . be a Markov chain with the phase space X and the matrix of transition probabilities P˜ 0 . The condition I˜ allows us to interpret the Markov chain η˜ε,n , for ε ∈ (0, 1], as a perturbed version of the Markov chain η˜0,n . Let us introduce, for i ∈ X, sets, ˜ 0,i = { j ∈ X : p˜0,i j > 0}. Y

(9.52)

˜ and I˜ imply that Y ˜ 0,i ⊆ Y ˜ 1,i , for i ∈ X. Lemma 9.6 The conditions G Lemma 9.7 The conditions G, H, I, IH , assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the conditions I, IH for the semi-Markov ˜ I˜ H . processes η˜ε (t) in the form of conditions I,

232

9 Perturbed SMP and reduction of phase space

Proof It follows from the relation (9.47) and Lemma 8.2 according to which the ˜ 1,i, i ∈ X belong to the family H.  functions p˜ ·,i j , j ∈ Y The following two lemmas present operating formulas which  express the comparability parameters and limits for the functions p¯ ·,ii = ji p ·,i j , i ∈ X and ˜ 1,i, i ∈ X appearing in the conditions I˜ and I˜ H in terms p˜ ·,i j = p ·,i j / p¯ ·,ii, j ∈ Y of the comparability parameters and limits for the functions appearing in the conditions I and IH . Lemma 9.8 Let the conditions G, H, I, IH be satisfied. Then:  (i) The functions p¯ ·,ii = ji p ·,i j , i ∈ X belong to the family H and the corresponding comparability limits can be calculated by applying to these functions the operating multiple sum formulagiven in Lemma  8.3, which, in this case, gives the relation limε→0 p¯ε,ii = limε→0 ji pε,i j = ji p0,i j = p¯0,ii , for i ∈ X. (ii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated by applying the operating multiple sum formulas given, respectively, in Lemmas 8.5, 8.7, or 8.9 to the functions  p¯ ·,ii = ji p ·,i j , i ∈ X. Lemma 9.9 Let the conditions G, H, I, IH hold. Then: ˜ 1,i, i ∈ X belong to the family H and the (i) The functions p˜ ·,i j = p ·,i j / p¯ ·,ii, j ∈ Y corresponding comparability limits can be calculated by applying to these functions the operating quotient formula given in Lemma 8.2 to these functions. (ii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated by applying the operating quotient formulas given, respectively, in Lemmas 8.4, 8.6, or 8.8 to the functions ˜ 1,i, i ∈ X. p˜ ·,i j = p ·,i j / p¯ ·,ii, j ∈ Y ˜ Let us assume that the conditions G, H, I, IH , J◦ (equivalent 9.2.2.2 Condition J. to the condition J) and K are satisfied. The distribution function F˜ε,i j (t), t ≥ 0, which is an analogue of the distribution function Fε,i j (t), t ≥ 0 is defined (for ε ∈ (0, 1]) by the following relation, for ˜ 1,i, i ∈ X, j∈Y F˜ε,i j (t) = P{ κ˜ε,1 ≤ t/η˜ε,0 = i, η˜ε,1 = j} ∞ 1  (∗n) n = F (t) ∗ Fε,i j (t)pε,ii pε,i j , t ≥ 0, p˜ε,i j n=0 ε,ii

(9.53)

˜ 1,i, i ∈ X, and, for j  Y F˜ε,i j (t) = F˜ε,i (t), t ≥ 0, where, for i ∈ X,

(9.54)

9.2 Removing of virtual transitions

233

F˜ε,i (t) = P{ κ˜ε,1 ≤ t/η˜ε,0 = i}   Q˜ ε,i j (t) = F˜ε,i j (t) p˜ε,i j , t ≥ 0. = j ∈X

(9.55)

˜ 1, i j ∈Y

The corresponding Laplace transform φ˜ε,i j (s) takes the following form, for j ∈ ˜ 1,i, i ∈ X: Y ∫ ∞ e−st F˜ε,i j (dt) φ˜ε,i j (s) = E{e−s κ˜ε,1 /η˜ε,0 = i, η˜ε,1 = j} = 0

φε,i j (s)pε,i j φε,i j (s) p¯ε,ii = · = p˜ε,i j 1 − φε,ii (s)pε,ii 1 − φε,ii (s)pε,ii φε,i j (s) = , s ≥ 0, −1 (1 − φ 1 + pε,ii p¯ε,ii ε,ii (s)) 1

(9.56)

˜ 1,i, i ∈ X, and, for j  Y φ˜ε,i j (s) = φ˜ε,i (s), s ≥ 0, where

φ˜ε,i (s) = E{e−s κ˜ε,1 /η˜ε,0 = i} =

(9.57) 

φ˜ε,i j (s) p˜ε,i j , s ≥ 0.

(9.58)

˜ 1, i j ∈Y

¯ 1,i, i ∈ X has no effect on the Note that the above choice of F˜ε,i j (t) for j  Y ˜ transition probabilities Q ε,i j (t). The following relations hold for t ≥ 0, i, j ∈ X and ε ∈ (0, 1], (9.59) Q˜ ε,i j (t) = F˜ε,i j (t) p˜ε,i j . The following condition plays the role of condition J for the semi-Markov processes η˜ε (t): ˜ 1,i, i ∈ X, (b) F˜0,i j (·), j ∈ Y ˜ 1,i, i ∈ X ˜ (a) F˜ε,i j (· u˜ε,i ) ⇒ F˜0,i j (·) as ε → 0, for j ∈ Y J: ˜ ˜ are proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ X, (c) u˜ε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X. The role of condition J˜ ◦ (equivalent to condition J), is played by the following ˜ condition equivalent to the condition J: ˜ 1,i, i ∈ X, (b) J˜ ◦ : (a) φ˜ε,i j (s/ ˜ε,i ) → φ˜0,i j (s) as ε → 0, for s ≥ 0 and j ∈ Y ∫ u∞ −st ˜ ˜ ˜ φ0,i j (s) = 0 e F0,i j (dt), s ≥ 0, j ∈ Y1,i, i ∈ X are Laplace transforms of ˜ 1,i, i ∈ X, (c) u˜ε,i ∈ proper distribution functions such that F˜0,i j (0) < 1, j ∈ Y (0, ∞), ε ∈ (0, 1], for i ∈ X, The operation of removing virtual transitions aggregates the sojourn times for the n−1 (1 − p semi-Markov process ηε (t). Obviously, Pi {θ ε [0] = n} = pε,ii ε,ii ), n = 1, 2, . . . −1 and, thus, Ei θ ε [0] = p¯ε,ii , for i ∈ X and ε ∈ (0, 1]. This note provides a hint for compensating the aggregation of sojourn times using the new local normalisation functions u˜ε,i ∈ (0, ∞), ε ∈ (0, 1], defined by the

234

9 Perturbed SMP and reduction of phase space

following relation, for i ∈ X,

−1 uε,i . u˜ε,i = p¯ε,ii

(9.60)

Recall that the conditions G and H imply that the probability p¯ε,ii ∈ (0, 1], ε ∈ −1 ∈ [1, ∞), ε ∈ (0, 1], for i ∈ X. Therefore, the normalisation (0, 1], and thus p¯ε,ii functions u˜ε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X. Let us define the set of asymptotically absorbing states, Y0 = {i ∈ X : p0,ii = 1}.

(9.61)

The asymptotic relations and the corresponding limiting Laplace transforms and ˜ take two different forms for asympdistributions appearing in the condition J˜ ◦ (J) ¯ 0 and absorbing states i ∈ Y0 . totically non-absorbing states i ∈ Y ¯ 0. First, let us consider the non-absorbing case, where i ∈ Y Using the conditions I, IH , J◦ and the relations (9.17) and (9.56), we get the ¯ 0: ˜ 1,i, i ∈ Y following relation, for j ∈ Y φε,i j ( p¯ε,ii s/uε,i ) p¯ε,ii 1 − pε,ii φε,ii ( p¯ε,ii s/uε,i ) → φ˜0,i j (s) as ε → 0, for s ≥ 0,

φ˜ε,i j (s/u˜ε,i ) =

where φ˜0,i j (s) =

φ0,i j ( p¯0,ii s) p¯0,ii , s ≥ 0. 1 − p0,ii φ0,ii ( p¯0,ii s)

(9.62)

(9.63)

∫∞ Obviously, φ˜0,i j (s) = 0 e−st F˜0,i j (dt) = Ee−s κ˜0, i j , s ≥ 0, is the Laplace transform for the distribution function F˜0,i j (·) of some non-negative random variable κ˜0,i j . In fact, the random variable κ˜0,i j is a geometric type sum of random variables, that is, κ˜0,i j =

ρ 0, i −1

p¯0,ii κ0,ii,n + p¯0,ii κ0,i j ,

(9.64)

n=1

where: (a) ρ0,i is a geometrically distributed random variable with parameter p¯0,ii = n−1 , for n = 1, 2, . . .); (b) 1 − p0,ii (which takes value n with probability p¯0,ii p0,ii κ0,ii,n, n = 1, 2, . . . are i.i.d. random variables with the Laplace transform φ0,ii (s), (d) κ0,i j is a random variable with the Laplace transform φ0,i j (s); (c) the random variables ρ0,i, κ0,i j , κ0,ii,n, n = 1, 2, . . . are mutually independent; ˜ 1,i, i ∈ Y ¯ 0 , i.e., the part of condition J˜ ◦ related Obviously, F˜0,i j (0) < 1, for j ∈ Y ¯ ˜ to states j ∈ Y1,i, i ∈ Y0 is satisfied. Second, let us consider the absorbing case, where i ∈ Y0 . Let: (a) κˆε,i j,n, n = 1, 2, . . . be i.i.d. random variables with distribution function Fˆε,i j (·) = Fε,i j (· uε,i ), for every j ∈ Y1,i, i ∈ Y0 and ε ∈ (0, 1]; (b) κˆ0,i j be a random variable with the distribution function Fˆ0,i j (·) = F0,i j (·), for every j ∈ Y1,i, i ∈ Y0 ; ∫∞ (c) fˆ = fε,i j = 0 t Fˆε,i j (dt), for j ∈ Y1,i, i ∈ X, ε ∈ (0, 1], and fˆ0,i j = f0,i j = ∫ ∞ ε,i j t Fˆ0,i j (dt), for j ∈ Y1,i, i ∈ X. 0

9.2 Removing of virtual transitions

235

By the central criterium of convergence for sums of i.i.d random variables (see, for example, Loève (1977)), the conditions J and K imply that the following weak LLN (law of large numbers) type relation holds, for any 0 < uε → ∞ as ε → 0 and j ∈ Y1,i, i ∈ Y0 ,  n ≤uε κˆε,i j,n d −→ fˆ0,i j as ε → 0. (9.65) uε Indeed, the above conditions imply that (a) Fˆε,i j (·) ⇒ Fˆ0,i j (·) as ε → 0, and (b) ˆfε,i j → fˆ0,i j = f0,i j < ∞ as ε → 0, for j ∈ Y1,i, i ∈ Y0 . Let 0 < sk → ∞ as k → ∞ be a sequence of continuity points for the distribution function Fˆ0,i j (·). The above asymptotic relations (a) and (b) obviously imply that, for any t > 0, ∫ ∞ ∫ ∞ ˆ lim s Fε,i j (ds) ≤ lim s Fˆε,i j (ds) ε→0 tuε ε→0 sk ∫ sk ˆ = lim ( fε,i j − s Fˆε,i j (ds)) ε→0 0 ∫ sk s Fˆ0,i j (ds) → 0 as k → ∞, (9.66) = fˆ0,i j − 0

and, thus, the following relation holds, for any t > 0: ∫ ∞ lim s Fˆε,i j (ds) = 0. ε→0

(9.67)

tuε

The relation (9.67) implies that, for any t > 0, ˆ uε P1 {u−1 ε κˆε,i j,1 > t} = uε (1 − Fε,i j (tuε )) ∫ ∞ ≤ t −1 s Fˆε,i j (ds) → 0 as ε → 0.

(9.68)

tuε

Also, the relation (9.67) implies that, for any t > 0, ∫ tuε −1 uε E1 u−1 κ ˆ I(u κ ˆ ≤ t) = s Fˆε,i j (ds) ε ε,i j,1 ε ε,i j,1 0

→ fˆ0,i j as ε → 0.

(9.69)

The relations (9.68) and (9.69) imply, by the above-mentioned central criterion of convergence, that the relation (9.65) holds. Let us introduce Laplace transforms, for j ∈ Y1,i, i ∈ X and ε ∈ (0, 1], ∫ ∞ e−st Fˆε,i j (dt) = φε,i j (s/uε,i ), s ≥ 0. (9.70) φˆε,i j (s) = 0

As well known (see, for example, Feller (1971)), the relation (9.65) is equivalent to the following relation:

236

9 Perturbed SMP and reduction of phase space

uε (1 − φˆε,i j (s/uε )) = uε (1 − φε,i j (s/uε uε,i )) → f0,i j s as ε → 0, for s ≥ 0.

(9.71)

Since, 0 < p¯ε,ii → 0 as ε → 0, for i ∈ Y0 , we can, in this case, choose, −1 uε = p¯ε,ii .

(9.72)

Using the conditions I, IH , J◦ , K and the relations (9.17), (9.25), (9.56), (9.71), (9.72) and taking into account that the state i ∈ Y1,i if i ∈ Y0 , we get the following ˜ 1,i, i ∈ Y0 : relation, for j ∈ Y φ˜ε,i j (s/u˜ε,i ) =

−1 u ) φε,i j (s/ p¯ε,ii ε,i −1 (1 − φ −1 1 + pε,ii p¯ε,ii ε,ii (s/ p¯ε,ii uε,i ))

1 = φ˜0,i j (s) as ε → 0, for s ≥ 0. (9.73) 1 + f0,ii s ∫∞ In this case, φ˜0,i j (s) = φ˜0,i (s) = 0 e−st F˜0,i (dt) is the Laplace transform of an −1 . exponentially distributed random variable κ˜0,i j = κ0,i , with parameter f0,ii −1 t} has no Note that f0,ii > 0 and the distribution function F˜0,i (t) = 1 − exp{− f0,ii atom in zero. ˜ 1,i, i ∈ Y0 is also Therefore, the part of condition J˜ ◦ related to states j ∈ Y satisfied. ˜ I˜ H and the relations (9.58), (9.62) and (9.73) imply that Also, the conditions I, the following relation takes place, for i ∈ X:  φ˜ε,i j (s) p˜ε,i j φ˜ε,i (s) = →

˜ 1, i j ∈Y

→



φ˜0,i j (s) p˜0,i j =

˜ 1, i j ∈Y

=





φ˜0,i j (s) p˜0,i j

˜ 1, i j ∈Y

φ˜0,i j (s) p˜0,i j = φ˜0,i (s) as ε → 0, for s ≥ 0.

(9.74)

˜ 0, i j ∈Y

The above remarks can be summarised in the following lemma. Lemma 9.10 The conditions G, H, I, IH , J◦ (J), K, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition J◦ (J) for the ˜ where: semi-Markov processes η˜ε (t) in the form of condition J˜ ◦ (J), (i) The asymptotic relation appearing in the condition J◦ takes two different forms, given in the relations (9.62) or (9.73), for the cases where the initial state i is, respectively, asymptotically non-absorbing or asymptotically absorbing state, that is, i ∈ Y0 or i ∈ Y0 . (ii) The normalisation functions u˜ ·,i, i ∈ X appearing in the condition J◦ are given by the relation (9.60).

9.2 Removing of virtual transitions

237

˜ Let us assume that the conditions G, H, I, IH , J, and K 9.2.2.3 Condition K. hold. The expectation f˜ε,i j , which is an analogue of the expectation fε,i j , is defined (for ˜ 1,i, i ∈ X: ε ∈ (0, 1]) by the following relation, for j ∈ Y f˜ε,i j = E{ κ˜ε,1 /η˜ε,0 = i, η˜ε,1 = j} −1 = fε,i j + pε,ii p¯ε,ii fε,ii .

Also, for i ∈ X,



f˜ε,i = E{ κ˜ε,1 /η˜ε,0 = i} =

f˜ε,i j p˜ε,i j .

(9.75)

(9.76)

˜ 1, i j ∈Y

The following condition plays the role of condition K for the semi-Markov processes η˜ε (t): ˜ (a) f˜ε,i j < ∞, j ∈ Y1,i, i ∈ X, for every ε ∈ (0, 1], (b) f˜ε,i j /u˜ε,i → f˜0,i j = K: ∫∞ t F˜0,i j (dt) < ∞ as ε → 0, for j ∈ Y1,i, i ∈ X. 0 X,

˜ 1,i, i ∈ The conditions G, H, IH , J, K and the relation (9.75) imply that, for j ∈ Y fε,i j f˜ε,i j fε,ii = p¯ε,ii + pε,ii u˜ε,i uε,i uε,i → p¯0,ii f0,i j + p0,ii f0,ii = f˜0,i j as ε → 0.

(9.77)

˜ 1,i, i ∈ Y ¯ 0 or Note that the above relation works for both cases, where j ∈ Y ˜ j ∈ Y1,i, i ∈ Y0 . Also, the following relation takes place, for i ∈ X:  f˜ε,i j f˜ε,i = p˜ε,i j u˜ε,i u ˜ 1, i ε,i j ∈Y   → f˜0,i j p˜0,i j = f˜0,i j p˜0,i j ˜ 1, i j ∈Y

= f˜0,i =

∫ 0

˜ 0, i j ∈Y ∞

t F˜0,i (dt) as ε → 0.

(9.78)

The above remarks can be summarised in the following lemma. Lemma 9.11 The conditions G, H, I, IH , J, K assumed to be satisfied for the semiMarkov processes ηε (t), entail the fulfilment of the condition K for the semi-Markov ˜ where: processes η˜ε (t) in the form of condition K, (i) The asymptotic relation appearing in condition K takes the form of relation (9.77).

238

9 Perturbed SMP and reduction of phase space

(ii) The normalisation functions u˜ ·,i, i ∈ X appearing in the condition K are given by relation (9.60). Remark 9.1 It is useful noting that the conditions G, H, and K (a), assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition K ˜ (a). This follows (a) for the semi-Markov processes η˜ε (t) in the form of condition K from the relation (9.75). ˜ and L ˜ H . The following conditions play the roles of 9.2.2.4 Conditions L condition L and LH for the semi-Markov processes η˜ε (t): ˜ u˜ε,i → u˜0,i ∈ (0, ∞] as ε → 0, for i ∈ X L: and ˜ H : The functions u˜ ·,i, i ∈ X belong to the complete family of asymptotically L comparable functions H appearing in the condition LH . Lemma 9.12 The conditions G, H and I, IH , L, LH , assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the conditions L, LH for the ˜ L ˜ H. semi-Markov processes η˜ε (t) in the form of conditions L, Proof It follows from the relation (9.60) and Lemma 8.2 that the functions u˜ ·,i = −1 u , i ∈ X belong to the family H. Moreover, the limit u˜ −1 p¯ ·,ii ·,i 0,i = p¯0,ii u0,i ∈ (0, ∞], for i ∈ X.  The following lemma presents operating formulas which express the compara˜ and L ˜ H in bility parameters and limits for the functions appearing in conditions L terms of the comparability parameters and limits for the functions appearing in the conditions I, L and IH , LH . Lemma 9.13 Let the conditions G, H, I, IH , L, LH be satisfied. Then: −1 u , i ∈ X belong to the family H and the corre(i) The functions u˜ ·,i = p¯ ·,ii ·,i sponding comparability limits can be calculated by applying to these functions the operating quotient formula given in Lemma 8.2. (ii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated by applying to the functions −1 u , i ∈ X the operating quotient formulas given, respectively, in Lemu˜ ·,i = p¯ ·,ii ·,i mas 8.5, 8.7, or 8.9. 9.2.2.5 Summary of Perturbation Conditions for Semi-Markov Process η˜ε (t). The following theorem summarises the remarks made in Sect. 9.2. Theorem 9.1 The conditions G, H, I, IH , L, LH , J, K, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of these conditions for the ˜ H, ˜ I, ˜ I˜ H , L, ˜ L ˜ H , J, ˜ K ˜ semi-Markov processes η˜ε (t) in the form of conditions G, presented in Lemmas 9.1–9.13.

9.3 One-step reduction of phase space

239

9.3 One-State Reduction of Phase Space for Perturbed Semi-Markov Processes In Sect. 9.3, we describe the procedure of one-state phase space reduction for perturbed semi-Markov processes. It is shown that the basic perturbation conditions imposed on the original semi-Markov processes are also satisfied for the semiMarkov processes with the reduced phase space. We also give explicit formulas for recalculating normalisation functions, limiting distributions and expectations in the corresponding perturbation conditions for semi-Markov processes with the reduced phase space.

9.3.1 A Procedure of One-Step Phase Space Reduction for Perturbed Semi-Markov Processes 9.3.1.1 Definition of Semi-Markov Process with Reduced Phase Space. We assume now that the number of states m in the phase space X is greater than 1. We also assume that the conditions G and H are satisfied for the semi-Markov processes ηε (t) and, therefore, these condition are also satisfied for the semi-Markov ˜ and H. ˜ processes η˜ε (t) in the form of conditions G In what follows in this section, we assume that ε ∈ (0, 1]. Let us choose some state k ∈ X and consider the reduced phase space k X = X \ {k}. Let us define the stopping times for the Markov chain η˜ε,n , for r = 0, 1, . . ., k αε [r]

= min(n > r : η˜ε,n ∈ k X).

(9.79)

By the definition, k αε [r] is the first after r time of hitting in the reduced phase space k X by the Markov chain η˜ε,n . Since the Markov chain η˜ε,n does not make virtual transitions (of the form i → i), the following relation takes place, for r = 0, 1, . . .,

r + 1 if η˜ε,r+1 ∈ k X, (9.80) k αε [r] = r + 2 if η˜ε,r+1 = k. We also define the following successive stopping times: k βε,n

= k αε [k βε,n−1 ], n = 1, 2, . . . , where k βε,0 = I(η˜ε,0 = k).

(9.81)

Let us now construct a new Markov renewal process (k ηε,n, k κε,n ), n = 0, 1, . . ., with the phase space k X × [0, ∞),  (η˜ε, k βε,0 , 0) for n = 0, k βε, n (k ηε,n, k κε,n ) = (9.82) (η˜ε, k βε, n , l= k βε, n−1 +1 κ˜ε,l ) for n = 1, 2, . . . .

240

9 Perturbed SMP and reduction of phase space

We also can define the corresponding reduced semi-Markov process, k ηε (t)

= k ηε, k νε (t), t ≥ 0,

(9.83)

= k κε,1 + · · · + k κε,n, n = 1, 2, . . . , k ζε,0 = 0,

(9.84)

where, k ζε,n

are the successive moments of jumps, and, k νε (t)

= max(n ≥ 1 : k ζε,n ≤ t),

(9.85)

is the number of jumps in the interval [0, t] for the above semi-Markov process. The definition of stopping times k βε,n implies that the transition probabilities of the above Markov renewal process are determined by the following relation, for t ≥ 0, i, j ∈ k X: k Q ε,i j (t)

= P{k ηε,1 = j, k κε,1 ≤ t/ k ηε,0 = i} = Q˜ ε,i j (t) + Q˜ ε,ik (t) ∗ Q˜ ε,k j (t).

(9.86)

Accordingly, the transition probabilities of the embedded Markov chain k ηε,n are given by the following relation, for i, j ∈ k X: k pε,i j

= P{k ηε,1 = j/ k ηε,0 = i} = p˜ε,i j + p˜ε,ik p˜ε,k j .

(9.87)

Note that the transition probabilities Q˜ ε,ii (t) = 0, t ≥ 0, i ∈ X and p˜ε,ii = 0, i ∈ X, since the semi-Markov process η˜ε (t) does not make virtual transitions of the form i → i. The semi-Markov process k ηε (t) can be considered as the result of some two-stage procedure applied to the original semi-Markov process ηε (t). At the first stage, the semi-Markov process ηε (t) is transformed in the semi-Markov process η˜ε (t), with the use of the procedure of removing virtual transitions. At the second stage, the semi-Markov process η˜ε (t) is transformed in the semi-Markov k ηε (t), with the use of the described above procedure of phase space reduction. The following relation holds, for i ∈ k X: Pi {k η(t) ∈ k X, t ≥ 0} = 1.

(9.88)

The relation (9.88) allows us to consider k η(t), as a semi-Markov process with the reduced phase space k X. We call k η(t) a reduced semi-Markov process. 9.3.1.2 Condition k G. Let us assume that the conditions G and H are satisfied. Let us introduce the distribution functions of sojourn times for the reduced semiMarkov process k ηε (t), for i ∈ k X and ε ∈ (0, 1],

9.3 One-step reduction of phase space k Fε,i (t)

241

= P{k κε,i ≤ t/ k ηε,0 = i}  = k Q ε,i j (t), t ≥ 0.

(9.89)

j∈ kX

The following condition plays the role of the condition G for the reduced semiMarkov process k ηε (t): (a) k pε,i j > 0, ε ∈ (0, 1] or k pε,i j = 0, ε ∈ (0, 1], for i, j ∈ k X, (b) k Fε,i (0) < 1, i ∈ k X, ε ∈ (0, 1].

k G:

Lemma 9.14 The conditions G and H, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition G for the reduced semi-Markov processes k ηε (t) in the form of condition k G. ˜ is satisfied. The condition G ˜ (a) and Proof Lemma 9.3 implies that the condition G the relation (9.87) obviously imply that the condition k G (a) is satisfied. ˜ (b) and the relation (9.89) obviously imply, for i ∈ k X and The condition G ε ∈ (0, 1],  Q˜ ε,i j (0) + Q˜ ε,ik (0)F˜ε,k (0) k Fε,i (0) = ji,k

≤



Q˜ ε,i j (0) + Q˜ ε,ik (0) = F˜ε,i (0) < 1.

(9.90)

ji,k

Thus, the regularity condition k G is satisfied for the semi-Markov process η  k ε (t). Therefore, the reduced semi-Markov process k ηε (t) is well defined on the time interval [0, ∞), for every ε ∈ (0, 1]. Let us introduce sets, for i ∈ k X and ε ∈ (0, 1], k Yε,i

= { j ∈ k X : k pε,i j > 0}.

(9.91)

The following lemma takes place. Lemma 9.15 The condition k G (a) is satisfied if and only if the sets k Yε,i = k Y1,i, ε ∈ (0, 1], for i ∈ k X, and sets,

˜ 1,k ) \ {k} if k ∈ Y ˜ 1,i, i ∈ k X, ˜ 1,i ∪ Y (Y Y = (9.92) k 1,i ˜ ˜ Y1,i if k  Y1,i, i ∈ k X. Proof The proof obviously follows from the relation (9.87).



9.3.1.3 Condition k H. The following conditions plays the role of condition H for the reduced semi-Markov processes k ηε (t): k H:

For any i, j ∈

k r0, k r1, . . . , k rk ni j

k X, there exist k ni j ≥ 1 and a chain of states i = = j such that 1≤l ≤ k ni j k p1, k rl−1 k rl > 0.

242

9 Perturbed SMP and reduction of phase space

Lemma 9.16 The conditions G and H, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition H for the reduced semi-Markov processes k ηε (t) in the form of condition k H. Proof The condition H for the semi-Markov processes η˜ε (t), is, under the condition ˜ (a), equivalent to the assumption that, for any states i, j ∈ X, there exist an integer G number n˜ i j and a chain of states i = r˜0, r˜1, . . . , r˜n˜ i j = j such that r˜l−1  r˜l, l = n˜ i j p˜1, r˜l−1 r˜l > 0. Suppose that the states i = r˜0, j = r˜n˜ i j ∈ k X. 1, . . . , n˜ i j and l=1 If the states r˜l−1, r˜l ∈ k X, for some 1 ≤ l ≤ ni , then the probability k p1, r˜l−1 r˜l ≥ p˜1, r˜l−1 r˜l > 0. If the state r˜l = k, for some 1 ≤ l ≤ n, then the states r˜l−1, r˜l+1 ∈ k X and the probability k p1, r˜l−1 r˜l+1 ≥ p˜1, r˜l−1 k p˜ε,k r˜l+1 > 0. Let i = k r0, k r1, . . . , k rk ni j = j be the new chain of states obtained from the chain i = r˜0, r˜1, . . . , r˜n˜ i j by excluding all states r˜l, 1 ≤ l ≤ n˜ i j − 1 such that r˜l = k. The above inequalities imply that k ni j p > 0 for this new chain. Therefore, the condition k H is satisfied l=1 k 1, k rl−1 k rl  for the reduced semi-Markov processes k ηε (t).

9.3.2 Perturbation Conditions for Reduced Semi-Markov Processes We are going to describe the assumptions, under which the basic conditions I–L and IH , LH are valid for the reduced semi-Markov processes k ηε (t), constructed by using the described above procedure of one-state reduction of the phase space for semi-Markov processes η˜ε (t). 9.3.2.1 Conditions k I and IkH . The following conditions play the roles of the conditions I and IH for the reduced semi-Markov processes k ηε (t): k I: k pε,i j

= p˜ε,i j + p˜ε,ik p˜ε,k j → k p0,i j = p˜0,i j + p˜0,ik p˜0,k j as ε → 0, for i, j ∈ k X,

and The functions k p ·,i j , j ∈ k Y1,i, i ∈ k X belong to the complete family of asymptotically comparable functions H appearing in the condition IH .

k IH :

Recall that the condition k H implies that the probabilities k pε,ii > 0, i ∈ k X, for ε ∈ (0, 1]. Since, the matrix k Pε = k pε,i j  is stochastic, for ε ∈ (0, 1], the condition k I implies that the matrix k P0 = k p0,i j  is also stochastic. Let k η0,n, n = 0, 1, . . . be a Markov chain with the phase space k X and the matrix of transition probabilities k P0 . The condition k I allows us to interpret the Markov chains k ηε,n , for ε ∈ (0, 1], as a perturbed version of the Markov chain k η0,n . Lemma 9.17 The conditions G, H, I, IH , assumed to be satisfied for the semiMarkov processes ηε (t), entail the fulfilment of the conditions I, IH for the reduced semi-Markov processes k ηε (t) in the form of conditions k I, k IH . Proof The conditions G, H, I, IH assumed to be satisfied for the semi-Markov processes ηε (t) imply that these conditions are also satisfied for the semi-Markov

9.3 One-step reduction of phase space

243

˜ H, ˜ I, ˜ I˜ H . Due to this fact, the statement processes η˜ε (t) in the form of conditions G, of Lemma 9.17 follows from the relation (9.87) and Lemmas 8.2 and 9.7, according  to which, the functions k p ·,i j , j ∈ k Y1,i, i ∈ k X belong to the family H. The following lemma presents operating formulas, which express the comparability parameters and limits for functions appearing in the conditions k I and k IH in terms of the comparability parameters and limits for the functions appearing in the conditions I˜ and I˜ H . Lemma 9.18 Let conditions G, H, I, IH be satisfied. Then: (i) The functions k p ·,i j = p˜ ·,i j + p˜ ·,ik p˜ ·,k j , j ∈ k Y1,i, i ∈ k X belong to the family H and the corresponding comparability limits can be computed by applying the operating product formula given in Lemma 8.2 to the functions p˜ ·,ik p˜ ·,k j and then the operating sum formula given in Lemma 8.2 to the functions k p ·,i j = p˜ ·,i j + p˜ ·,ik p˜ ·,k j , j ∈ k Y1,i, i ∈ k X. (ii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be computed by applying the operating product formulas given, respectively, in Lemmas 8.4, 8.6, or 8.8 to the functions p˜ ·,ik p˜ ·,k j and then the operating sum formulas given, respectively, in Lemmas 8.4, 8.6, or 8.8 to the functions k p ·,i j = p˜ ·,i j + p˜ ·,ik p˜ ·,k j , j ∈ k Y1,i, i ∈ k X. 9.3.2.2 Condition k J. Let us assume that the conditions G, H, I, IH , L, LH , J◦ (equivalent to the condition J), and K, are satisfied for the semi-Markov processes ηε (t). Then, according to Lemmas 9.3, 9.5, 9.10, and 9.11, these conditions are also ˜ H, ˜ I, ˜ I˜ H , satisfied for the semi-Markov processes η˜ε (t) in the form of conditions G, ˜ and K. ˜ ˜L, L ˜ H , J˜ ◦ (equivalent to the condition J), The distribution function k Fε,i j (t), t ≥ 0, which is an analogue of the distribution function F˜ε,i j (t), t ≥ 0 is defined (for ε ∈ (0, 1]) by the following relation, for j ∈ k Y1,i, i ∈ k X: k Fε,i j (t)

= P{k ηε,1 = j, k κε,1 ≤ t/ k ηε,0 = i, k ηε,1 = j} p˜ε,i j p˜ε,ik p˜ε,k j + F˜ε,ik (t) ∗ F˜ε,k j (t) ,t ≥ 0 = F˜ε,i j (t) k pε,i j k pε,i j

(9.93)

and, for j  k Y1,i, i ∈ k X, k Fε,i j (t)

= k Fε,i (t), t ≥ 0,

(9.94)

where, for i ∈ k X, k Fε,i (t)

= P{k κε,1 ≤ t/ k ηε,0 = i}   = k Q ε,i j (t) = j∈ kX

k Fε,i j (t) k pε,i j , t

≥ 0.

(9.95)

j ∈ k Y1, i

The relations (9.93)–(9.95) imply that the corresponding Laplace transforms φ k ε,i j (s), s ≥ 0 take the following form, for j ∈ k Y1,i, i ∈ k X:

244

9 Perturbed SMP and reduction of phase space k φε,i j (s)

= E{e−s k κε,1 / k ηε,0 = i, k ηε,1 = j} p˜ε,i j p˜ε,ik p˜ε,k j = φ˜ε,i j (s) + φ˜ε,ik (s)φ˜ε,k j (s) , s ≥ 0, k pε,i j k pε,i j

(9.96)

and, for j  k Y1,i, i ∈ k X, k φε,i j (a)

= k φε,i (s), s ≥ 0,

(9.97)

where, for i ∈ k X, k φε,i (s)

= E{e−s k κε,1 / k ηε,0 = i} =



k φε,i j (s) k pε,i j , s

≥ 0.

(9.98)

j ∈ k Y1, i

The following condition plays the role of the condition J for the reduced semiMarkov processes k ηε (t): (a) k Fε,i j (· k uε,i ) = P{ k κε,1 / k uε,i ≤ ·/ηε,0 = i, ηε,1 = j} ⇒ k F0,i j (·) as ε → 0, for j ∈ k Y1,i, i ∈ k X, (b) k F0,i j (·), j ∈ k Y1,i, i ∈ k X are proper distribution functions such that k F0,i j (0) < 1, j ∈ k Y1,i, i ∈ k X, (c) k uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ k X,

k J:

The role of the condition J◦ (equivalent to the condition J) is played by the following condition equivalent to the condition k J: ◦ k J : (a) k φε,i j (s/ ∫ ∞k uε,i ) → k φ0,i j (s) as ε → 0, for s ≥ 0 and j ∈ k Y1,i, i ∈ k X, (b) φ (s) = e−st k F0,i j (dt), s ≥ 0, j ∈ k Y1,i, i ∈ k X are Laplace transforms k 0,i j 0 of proper distribution functions such that k F0,i j (0) < 1, j ∈ k Y1,i, i ∈ k X, (c) k uε,i

∈ (0, ∞), ε ∈ (0, 1], for i ∈ k X,

The above operation of excluding the state k does not aggregate the sojourn times for the semi-Markov process η˜ε (t). That is why, the local normalisation functions k uε,i ∈ [1, ∞), ε ∈ (0, 1] are defined by the following relation, for i ∈ k X: k uε,i

−1 = u˜ε,i = p¯ε,ii uε,i .

(9.99)

Recall that the conditions G and H imply that the probability p¯ε,ii ∈ (0, 1], ε ∈ −1 ∈ [1, ∞), ε ∈ (0, 1], for i ∈ X. That is why, the normalisation (0, 1], and, thus, p¯ε,ii functions k uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ k X. The relation (9.99) and Lemma 8.2 imply that the functions k u ·,i, i ∈ X belong to −1 u the family H. Moreover, the limit u˜0,i = p¯0,ii 0,i ∈ (0, ∞], for i ∈ X. To obtain the corresponding asymptotic relations and limiting Laplace transforms appearing in the condition k J◦ (k J), it is necessary to use some additional conditions imposed on the transition probabilities of the semi-Markov processes η˜ε (t). The first is the following condition: ˆ k qε,i j k I: k X.

=

p˜ ε, i j p˜ ε, i j + p˜ ε, i k p˜ ε, k j

=

p˜ ε, i j k p ε, i j

→ k q0,i j ∈ [0, 1] as ε → 0, for j ∈ k Y1,i, i ∈

9.3 One-step reduction of phase space

245

Lemma 9.19 Let the conditions G, H, I, and IH be satisfied for the semi-Markov processes ηε (t). Then: (i) The condition k Iˆ is satisfied. ˜ 1,i, j ∈ k Y1,i, i ∈ k X, then k qε,i j = 0, ε ∈ (0, 1], and thus, the (ii) If j  Y asymptotic relation appearing in the condition k Iˆ holds, with the limit k q0,i j = 0. ˜ 1,i, j ∈ k Y1,i, i ∈ k X, then the function k qε,i j ∈ H and the corre(iii) If j ∈ Y sponding comparability limits can be calculated by applying the operating quotient formula given in Lemma 8.2 to the functions p˜ ·,i j and k p ·,i j . (iv) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be calculated by applying the quotient formulas given, respectively, in Lemmas 8.4, 8.6, or 8.8 to the functions p˜ ·,i j and k p ·,i j . Proof Conditions G, H, I, IH , assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of these conditions for the semi-Markov processes η˜ε (t) ˜ H, ˜ I, ˜ I˜ H . in the form of conditions G, ˜ 1,i, j ∈ k Y1,i, i ∈ k X, then p˜ε,i j = 0, k pε,i j > 0, ε ∈ (0, 1]. Thus, If j  Y k qε,i j = 0, ε ∈ (0, 1]. Therefore, k qε,i j → k q0,i j = 0 as ε → 0. ˜ 1,i, j ∈ k Y1,i, i ∈ k X, then p˜ε,i j , k pε,i j > 0, ε ∈ (0, 1] and, thus, the If j ∈ Y function k q ·,i j ∈ H, as the quotient of the functions p˜ ·,i j , k p ·,i j ∈ H. Therefore, there exists limε→0 k qε,i j = k q0,i j . Since, k qε,i j ∈ (0, 1], ε ∈ (0, 1], the limit k q0,i j ∈ [0, 1].  The second is the following condition of asymptotic comparability for normalisation functions: ˆ wε, j,i = L:

u˜ ε, j u˜ ε, i

→ w0, j,i ∈ [0, ∞] as ε → 0, for i, j ∈ X.

Lemma 9.20 Let the conditions G, H, I, IH , L, LH be satisfied for the semi-Markov processes ηε (t). Then: ˆ is satisfied. (i) The condition L (ii) The functions w ·, j,i, i, j ∈ X belong to the family H, and the corresponding comparability limits can be calculated by applying the operating quotient formula given in Lemma 8.2 to the functions u ·, j and u ·,i , for i, j ∈ X. (iii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be computed by applying the operating quotient formulas given, respectively, in Lemmas 8.4, 8.6, or 8.8 to the functions u˜ ·, j and u˜ ·,i , for i, j ∈ X. Proof Conditions G, H, I, IH , L, LH assumed to be satisfied for the semi-Markov processes ηε (t) imply that these conditions are also satisfied for the semi-Markov ˜ H, ˜ I, ˜ I˜ H , L, ˜ L ˜ H. processes η˜ε (t) in the form of conditions G, Since u˜ε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X, the functions wε, j,i ∈ (0, ∞), ε ∈ (0, 1], for i, j ∈ X. Moreover, the functions w˜ ·, ji ∈ H as the quotients of the functions u˜ ·, j , u˜ ·,i ∈ H. Therefore, there exists limε→0 wε, j,i = w0, j,i ∈ [0, ∞], for i, j ∈ X. 

246

9 Perturbed SMP and reduction of phase space

ˆ it follows that there exist sets Wl, l = 0, . . . , L such that, From the condition L (a) Wl ∩ Wl = ∅, l   l  . L Wl = X. (b) ∪l=0 (c) w0, j,i ∈ (0, ∞), j, i ∈ Wl, for 0 ≤ l ≤ L.

(d) w0, j,i = 0, j ∈ Wl, i ∈ Wl, for 0 ≤ l  < l  ≤ L.

(9.100)

We are especially interested in the set W0 , which includes the so-called least absorbing states in the phase space X. In Sect. 9.3.2.4, we present a simple algorithm for finding the set W0 . The third is the following absorbing rate condition: k M:

The state k ∈ W0 , i.e., w0,k,i ∈ [0, ∞), for i ∈ X.

We assume that the conditions G, H, I, IH , L, LH , J◦ (equivalent to the condition J), and K are satisfied for the semi-Markov processes ηε (t). Thus, according to ˜ H, ˜ L ˜ H J˜ ◦ ˜ I, ˜ I˜ H , L, Lemmas 9.3, 9.5, 9.10, 9.11, 9.19, and 9.20, the conditions G, ˜ and k I, ˆ L ˆ are satisfied for the semi-Markov processes (equivalent to condition J), η˜ε (t). Suppose also that the condition k M is satisfied. Using the above conditions and the relation (9.96), we obtain the following relation, which plays the role of asymptotic relation appearing in the condition k J◦ , for j ∈ k Y1,i, i ∈ k X, k φε,i j (s/ k uε,i )

= φ˜ε,i j (s/u˜ε,i )

p˜ε,i j k pε,i j

+ φ˜ε,ik (s/u˜ε,i )φ˜ε,k j (

p˜ε,ik p˜ε,k j u˜ε,k s/u˜ε,k ) u˜ε,i k pε,i j

→ k φ0,i j (s) as ε → 0, for s ≥ 0,

(9.101)

where k φ0,i j (s)

= φ˜0,i j (s) k q0,i j + φ˜0,ik (s)φ˜0,k j (w0,k,i s)(1 − k q0,i j ), s ≥ 0.

(9.102)

Also, the conditions k IH and the relation (9.101) imply that the following asymptotic relation holds, for i ∈ k X:  k φε,i (s/ k uε,i ) = k φε,i j (s/ k uε,i ) k pε,i j j ∈ k Y1, i

→



j ∈ k Y1, i

k φ0,i j (s) k p0,i j

=



k φ0,i j (s) k p0,i j

j ∈ k Y0, i

= k φ0,i (s) as ε → 0, for s ≥ 0.

(9.103)

9.3 One-step reduction of phase space

247

In the case, where the condition k M is satisfied, the condition k J◦ (b) is satisfied for any k q0,i j ∈ [0, 1], even, if k q0,i j = 0 or k q0,i j = 1. ˜ 1,i and Indeed, if j ∈ k Y1,i and k q0,i j = 0, then p˜0,ik , p˜0,k j > 0, and, thus, k ∈ Y ˜ ˜ ˜ j ∈ Y1,k . In this case, the limiting functions φ0,ik (s) and φ0,k j (w0,k,i s) are the Laplace transforms of some proper distribution functions which are not concentrated at zero, and, thus, the limiting function k φ0,i j (s) = φ˜0,ik (s)φ˜0,k j (w0,k,i s) is the Laplace transform of some proper distribution function which is not concentrated in 0. ˜ 1,i If j ∈ k Y1,i and k q0,i j ∈ (0, 1), then p˜0,i j , p˜0,ik , p˜0,k j > 0 and, thus, j, k ∈ Y ˜ ˜ ˜ ˜ and j ∈ Y1,k . In this case, the limiting functions φ0,i j (s), φ0,ik (s) and φ0,k j (w0,k,i s) are the Laplace transforms of some proper distribution functions which are not concentrated at zero, and, thus, the limiting function k φ0,i j (s) = φ˜0,i j (s) k q0,i j + φ˜0,ik (s)φ˜0,k j (w0,k,i s)(1 − k q0,i j ) is the Laplace transform of some proper distribution function which is not concentrated at zero. ˜ 1,i . In this case, Finally, if j ∈ k Y1,i and k q0,i j = 1, then p˜0,i j > 0 and, thus, j ∈ Y ˜ the limiting function k φ0,i j (s) = φ0,i j (s) is the Laplace transform of some proper distribution function which is not concentrated at zero. The above remarks can be summarised in the following lemma. Lemma 9.21 The conditions G, H, I, IH , L, LH , J◦ (J), K, and k M, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition J◦ (J) for the reduced semi-Markov processes k ηε (t) in the form of condition k J◦ (k J), where: (i) The asymptotic relation appearing in the condition k J◦ takes the form given in relation (9.101). (ii) The normalisation functions k u ·,i, i ∈ k X appearing in the condition k J◦ are given by the relation (9.99). It is worth commenting on the need to assume the fulfilment of the condition k M. If the condition k M is not satisfied, it is possible that the coefficient w0,ki = ∞ for some i ∈ k X. In this case, it follows from the relation (9.96) that, for j ∈ k Y1,i and s ≥ 0, lim k φε,i j (s/ k uε,i ) ≤ φ˜0,i j (s) k q0,i j + φ˜0,ik (s)I(s = 0)

ε→0

+ F˜0,k j (0)I(s > 0))(1 − k q0,i j ).

(9.104)

In this case, the relation (9.104) implies that the condition k J1 (b) is not satisfied, ˜ 1,k . if also k q0,i j < 1 and j ∈ Y ˜ 1,k . Therefore, the function on the right-hand side of Indeed, F˜0,k j (0) < 1 if j ∈ Y the relation (9.104) converges to limit F˜0,k j (0)(1− k q0,i j ) < 1 as s → 0. This fact and the relation (9.104) imply that the functions k φε,i j (s/ k uε,i ) cannot converge pointwise to the Laplace transform of some proper distribution function concentrated on the interval [0, ∞). 9.3.2.3 Condition k K. We assume that the conditions G, H, I, IH , L, LH , J◦ (equivalent to the condition J), K are satisfied for the semi-Markov processes ηε (t).

248

9 Perturbed SMP and reduction of phase space

˜ H, ˜ I, ˜ I˜ H , L, ˜ L ˜ H , J˜ ◦ (equivalent As mentioned above, this implies that conditions G, ˜ ˆ ˆ to condition J), k I, and L are satisfied for the semi-Markov processes η˜ε (t). We also assume that the condition k M is satisfied. The expectation k fε,i j , which is an analogue of the expectation fε,i j is defined, for ε ∈ (0, 1] by the following relation, for j ∈ k Y1,i, i ∈ k X, k fε,i j

= E{k κε,1 / k ηε,0 = i, k ηε,1 = j} = f˜ε,i j p˜ε,i j + ( f˜ε,ik + f˜ε,k j ) p˜ε,ik p˜ε,k j .

Also, for i ∈ k X, k fε,i



= E{k κε,1 / k ηε,0 = i} =

(9.105)

k fε,i j k pε,i j .

(9.106)

j ∈ k Y1, i

The following condition plays the role of condition K for the reduced semi-Markov processes k ηε (t): (a) k f∫ε,i j < ∞, j ∈ k Y1,i, i ∈ k X, for every ε ∈ (0, 1], (b) ∞ f k 0,i j = 0 s k F0,i j (ds) < ∞ as ε → 0, for j ∈ k Y1,i, i ∈ k X.

k K:

k fε,i j / k uε,i

→

The conditions G, H, I, IH , L, LH , J, K, k M and the relation (9.75) imply that, for j ∈ k Y1,i, i ∈ k X, k fε,i j k uε,i

 f˜ε,ik f˜ε,i j p˜ε,i j f˜ε,k j u˜ε,k  p˜ε,ik p˜ε,k j + + u˜ε,i k pε,i j u˜ε,i u˜ε,k u˜ε,i k pε,i j ˜ ˜ ˜ → k f0,i j = f0,i j k q0,i j + ( f0,ik + f0,k j w0,k,i )(1 − k q0,i j ) ∫ ∞ = t k F0,i j (dt) as ε → 0, for s ≥ 0. =

(9.107)

0

Also, the following relation takes place, for i ∈ k X: k fε,i k uε,i

=



k fε,i j

u ˜ 1, i k ε,i j ∈Y

→



k f0,i j k p0,i j

˜ 1, i j ∈Y

=

k pε,i j

k f0,i =

∫ 0

=



k f0,i j k p0,i j

˜ 0, i j ∈Y ∞

t k F0,i (dt) as ε → 0.

(9.108)

The above remarks can be summarised in the following lemma. Lemma 9.22 The conditions G, H, I, IH , L, LH , J (J◦ ), K, and k M, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the condition K for the reduced semi-Markov processes k ηε (t) in the form of condition k K, where: (i) The asymptotic relation appearing in the condition k K takes the form of the relation (9.107).

9.3 One-step reduction of phase space

249

(ii) The normalisation functions k u ·,i, i ∈ k X appearing in the condition k K are given by the relation (9.99). Remark 9.2 It is useful noting that the conditions G, H, and K (a), assumed to be satisfied for the semi-Markov processes ηε (t), entail that, for any state k ∈ X, the condition K (a) is satisfied for the reduced semi-Markov processes k ηε (t) in the form of condition k K (a). This follows from the relation (9.105). ˆ implies that the set W0  ∅. 9.3.2.4 Set W0 and condition k M. The condition L Obviously, w0,k,i ∈ (0, ∞), for k, i ∈ W0 , while w0,k,i = 0, for k ∈ W0 , i ∈ X \ W0 . Also, W0 = X, if w0,k,i ∈ (0, ∞), for any k, i ∈ X. The set W0 can be found using the following simple algorithm. Let us order by an arbitrary way states in space X, i.e., represent this space in the form X = {i1, . . . , im }. Let us now define the states in∗ , n = 1, . . . , m using the following recurrent procedure: i1∗ = i1, i2∗ = i1∗ I(w0,i1∗,i2 = 0) + i2 I(w0,i1∗,i2 ∈ (0, ∞]), ... ∗ ∗ ∗ ∗ = im−1 I(w0,im−1 im ,im = 0) + im I(w0,im−1 ,im ∈ (0, ∞]).

(9.109)

The following simple lemma takes place. ∗ ∈ W and the set, ˆ be satisfied. Then, the state im Lemma 9.23 Let the condition L 0 ∗ ,i ∈ (0, ∞)}. W0 = {in ∈ X : w0,im n

(9.110)

9.3.2.5 Conditions L and LH . The following conditions play the roles of the conditions L and LH for the reduced semi-Markov processes k ηε (t): k L: k uε,i

→

k u0,i

∈ (0, ∞] as ε → 0, for i ∈ k X

and functions k u ·,i, i ∈ k X belong to the complete family of asymptotically comparable functions H appearing in the condition LH .

k LH : The

The following two lemmas are direct corollaries of Lemmas 9.12 and 9.13, since the normalisation functions k u ·,i = u˜ ·,i, i ∈ k X. Lemma 9.24 The conditions G, H, I, IH , L, LH , assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the conditions L, LH for the reduced semi-Markov processes k ηε (t) in the form of conditions k L, k LH . Lemma 9.25 Let conditions G, H, I, IH , L, LH be satisfied. Then: (i) The functions k u ·,i = u˜ ·,i, i ∈ k X belong to the family H and the corresponding comparability limits can be calculated by applying to these functions the operating quotient formula given in Lemma 9.13. (ii) If H is one of the families, H1 , H2 , or H3 , then the corresponding comparability parameters and limits can be computed by applying the operating quotient formulas given in Lemma 9.13 to the functions k u ·,i = u˜ ·,i, i ∈ k X.

250

9 Perturbed SMP and reduction of phase space

9.3.2.6 Summary of Perturbation Conditions for Semi-Markov Process The following theorem summarises remarks made in Sect. 9.3.2.

k ηε (t).

Theorem 9.2 The conditions G, H, I, IH , L, LH , J (J◦ ), K, and k M, assumed to be satisfied for the semi-Markov processes ηε (t), entail the fulfilment of the conditions G, H, I, IH , L, LH , J (J◦ ), K, and hold for reduced semi-Markov processes k ηε (t) in the form of conditions k G, k H, k I, k IH , k L, k LH , k J (k J◦ ), k K, presented in Lemmas 9.14–9.25. Remark 9.3 The relations (9.101) and (9.107) imply that condition k M can be weakened in Theorem 9.2 and replaced by the following local absorbing rate condition: ˆ k M:

w0,k,i ∈ [0, ∞), for i ∈ X such that k ∈ Y1,i .

9.4 Recurrent Reduction of Phase Space for Perturbed Semi-Markov Processes In this section, we describe an asymptotic multi-step algorithm for recurrent phase space reduction for perturbed semi-Markov processes.

9.4.1 The Recurrent Phase Space Reduction Algorithm 9.4.1.1 Notation for Semi-Markov Processes with Reduced Phase Spaces. An important element of the recurrent phase space reduction algorithm presented in this section is the procedure of successive exclusion of a sequence of states k¯ n = k1, k2, . . . , kh , for some 1 ≤ h ≤ m − 1 from the phase space X of the semi-Markov process ηε (t). This procedure results in reduction of the phase space X to the space X \ {k1, k2, . . . , kh }. We use the abbreviated notation k¯h ηε (t) for the reduced semi-Markov process obtained by successive exclusion of the sequence of states k¯ n = k1, k2, . . . , kh from the phase space X of semi-Markov processes ηε (t). We also use the abbreviated notation k¯h η˜ε (t) for the semi-Markov process obtained by the removal of virtual transitions from trajectories of the reduced semi-Markov process k¯ n ηε (t). The semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) have the phase space k¯h X = X \ {k1, k2, . . . , kh }. In particular, we change the notation from k1 ηε (t) and k1 η˜ε (t), respectively, to ¯k1 ηε (t) and k¯1 η˜ε (t), for the reduced semi-Markov process k1 ηε (t) obtained by the exclusion of the state k = k1 from the initial phase space X and the reduced semiMarkov process k1 η˜ε (t) by removal of virtual transitions from trajectories of the reduced semi-Markov process k1 ηε (t).

9.4 Recurrent reduction of phase space

251

In the similar way, we index random variables, in particular hitting times, phase spaces and other related sets, conditions, probabilities, expectations and other quantities and objects related to the semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) by the left lower index k¯h (in such a way as k¯h f ). This is done in order to distinguish these random variables, phase spaces and other sets, probabilities, expectations and other quantities and objects, for h = 1, . . . , m − 1. To obtain similar forms for recurrent relations connecting random variables, probabilities and expectations and other quantities and objects resulted by excluding the sequences of states k¯ h−1 and k¯ h from the phase space X, we also use the left lower index k¯0 (for the “empty” sequence k¯ 0 = ) for h = 1. Thus, we change the notation from ηε (t) and η˜ε (t), respectively, to k¯0 ηε (t) and k¯0 η˜ε (t). In similar way, we index k¯0 X = X and other related sets, conditions, probabilities, expectations and other quantities and objects related to the semi-Markov processes k¯0 ηε (t) = ηε (t) and k¯0 η˜ε (t) = η˜ε (t). Let us describe the recurrent algorithm described above in more detail. 9.4.1.2 The First Step of the Phase Space Reduction Algorithm. Initially, it should be assumed that the conditions k¯ 0 G = G and k¯ 0 H = H are satisfied for the semi-Markov processes k¯0 ηε (t) = ηε (t). Other conditions are indicated below. The first step of the phase space reduction algorithm includes two sub-steps. At the first sub-step, the original semi-Markov process k¯0 ηε (t) = ηε (t) is transformed in the semi-Markov process k¯0 η˜ε (t) = η˜ε (t) by applying the procedure of removal of virtual transitions described in Sect. 9.2. The resulting semi-Markov process k¯0 η˜ε (t) has the phase space k¯0 X = X. At the second sub-step, the semi-Markov process k¯0 η˜ε (t) = η˜ε (t) is transformed in the reduced semi-Markov process k¯1 ηε (t) = k1 ηε (t) by applying the procedure of one-state reduction of phase space described in Sect. 9.3. The resulting reduced semi-Markov process k¯1 ηε (t) has the reduced phase space k¯1 X = k1 X. There are five variations of the procedures described above that use different sets of conditions. In the variant (i), it is initially assumed that the only conditions k¯ 0 G and k¯ 0 H are satisfied. In this case, Lemmas 9.3–9.5 imply that the conditions k¯ 0 G and k¯ 0 H are satisfied ˜ = G ˜ and for the semi-Markov processes k¯0 η˜ε (t) in the form of conditions k¯ 0 G ˜ ˜ k¯ 0 H = H. Also, Lemmas 9.14–9.16 imply that the conditions k¯ 0 G and k¯ 0 H are satisfied for the reduced semi-Markov processes k¯1 ηε (t) in the form of conditions k¯ 1 G = k1 G and k¯ 1 H = k1 H. In the variant (ii), it is initially assumed that the conditions k¯ 0 G, k¯ 0 H, and k¯ 0 K (a) = k0 K (a) (the assumption about finiteness of the expectations fε,i j , j ∈ Y1,i, i ∈ X, for ε ∈ (0, 1]) are satisfied. In this case, Lemmas 9.3–9.5 and Remark 9.1 imply that the conditions k¯0 G, H, and k¯ 0 K (a) are satisfied for the semi-Markov processes k¯0 η˜ε (t) in the form of k¯ 0 ˜ k¯ H, ˜ and k¯ K ˜ (a) = k0 K ˜ (a). Also, Lemmas 9.14–9.16 and Remark 9.1 conditions k¯ 0 G, 0 0 imply that the conditions k¯ 0 G, k¯ 0 H, and k¯ 0 K (a) are satisfied for the reduced semi-

252

9 Perturbed SMP and reduction of phase space

Markov processes k¯1 ηε (t) in the form of conditions k¯ 1 G, k¯ 1 H and k¯ 1 K (a) = k1 K (a). In the variant (iii), it is initially assumed that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I = k0 I, and k¯ 0 IH = k0 IH are satisfied. In this case, Lemmas 9.6–9.9 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, and k¯ 0 IH ˜ are satisfied for the semi-Markov processes k¯0 η˜ε (t) in the form of conditions k¯ 0 G, ˜ ˜ ˜ ˜ ˜ k¯ 0 H, k¯ 0 I = k0 I, and k¯ 0 IH = k0 IH . Also, Lemmas 9.17–9.18 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, and k¯ 0 IH are satisfied for the reduced semi-Markov processes k¯1 ηε (t) in the form of conditions k¯ 1 G, k¯ 1 H, k¯ 1 I = k1 I, and k¯ 1 IH = k1 IH . In the variant (iv), it is initially assumed that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L = k0 L, and k¯ 0 LH = k0 LH are satisfied. In this case, Lemmas 9.12–9.13 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L, and k¯ 0 LH are satisfied for the semi-Markov processes k¯0 η˜ε (t) in the form of ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ , k¯ L ˜ = k0 L, ˜ and k¯ L ˜ = k0 L ˜ . Also, Lemmas 9.24– conditions k¯ 0 G, H 0 0 0 H 0 0 H 9.25 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L, and k¯ 0 LH are satisfied for the reduced semi-Markov processes k¯1 ηε (t) in the form of conditions k¯ 1 G, k¯ 1 H, k¯ 1 I, k¯ 1 IH , k¯ 0 L = k0 L, and k¯ 1 LH = k1 LH . Finally, in the variant (v), it is initially assumed that the conditions k¯ 0 G, k¯ 0 H, I, k¯ 0 k¯ 0 IH , k¯ 0 L, k¯ 0 LH , k¯ 0 J = k0 J, k¯ 0 K = k0 K are satisfied. In addition, it should be assumed that the condition k¯ 1 M = k1 M (state k1 ∈ k¯0 W0 = W0 ), which is required to implement the second sub-algorithm of transforming the semi-Markov process k¯0 η˜ε (t) in the reduced semi-Markov process k¯1 ηε (t), is satisfied. In this case, Lemmas 9.10–9.11 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , L, k¯ 0 k¯ 0 LH , k¯ 0 J, and k¯ 0 K are satisfied for the semi-Markov processes k¯0 η˜ε (t) in the ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ , k¯ J˜ = k0 J, ˜ and k¯ K ˜ = k0 K. ˜ form of conditions k¯ 0 G, 0 0 0 H 0 0 0 H 0 Also, Lemmas 9.21–9.23 imply that the conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L, k¯ 0 LH , k¯ 0 J, and k¯ 0 K are satisfied for the reduced semi-Markov processes k¯1 ηε (t) in the form of conditions k¯ 1 G, k¯ 1 H, k¯ 1 I, k¯ 1 IH , k¯ 1 L, k¯ 1 LH , k¯ 1 J = k1 J, and k¯ 1 K = k1 K. What is important is that in the variants (i)–(iv), the state k1 can be an arbitrary state in the phase space X, while, in the variant (v), the state k1 should be chosen such that k1 ∈ k¯0 W0 = W0 , that is, the condition k¯ 1 M = k1 M is assumed to be satisfied. 9.4.1.3 The Recurrent Phase Space Reduction Algorithm. As mentioned above, initially it should assumed that the conditions k¯ 0 G = G and k¯ 0 H = H are satisfied for the semi-Markov processes k¯0 ηε (t) = ηε (t). The first step of phase space reduction algorithm described in Sect. 9.4.1.2 can be recurrently repeated h times for some 1 ≤ h ≤ m − 1. The h-th step of the phase space reduction algorithm includes two sub-steps similar to those described above for the first step. The only difference is that the corresponding sub-algorithms must be applied to the reduced semi-Markov process k¯ h−1 ηε (t) instead of the semi-Markov process k¯0 ηε (t). First, the reduced semi-Markov process k¯h−1 ηε (t) is transformed in the semiMarkov process k¯h−1 η˜ε (t) by applying the procedure of removal of virtual transi-

9.4 Recurrent reduction of phase space

253

tions described above in Sects. 9.2 and 9.4.1.2. The resulting semi-Markov process k¯ h−1 η˜ε (t) has the phase space k¯ h−1 X = X \ {k 1, . . . , k h−1 }. Second, the semi-Markov process k¯h−1 η˜ε (t) is transformed in the reduced semiMarkov process k¯h ηε (t) by applying the procedure of one-state reduction of phase space described in Sects. 9.3 and 9.4.1.2. The resulting semi-Markov process k¯h ηε (t) has the reduced phase space k¯h X = k¯h−1 X \ {kh } = X \ {k1, . . . , kh }. Again, there are five variants of the procedures described above that use different sets of conditions. In the variant (i), the conditions k¯ h−1 G and k¯ h−1 H replace the initial conditions k¯ 0 G and k¯ 0 H. The fulfilment of the conditions k¯ h−1 G and k¯ h−1 H was the result of the h − 1 steps in the variant (i) of the recurrent phase space reduction algorithm. In this case, Lemmas 9.3–9.5 imply that the conditions k¯ h−1 G and k¯ h−1 H are ˜ satisfied for the semi-Markov processes k¯h−1 η˜ε (t) in the form of conditions k¯ h−1 G ˜ and k¯ h−1 H. Also, Lemmas 9.14–9.16 imply that the conditions k¯h−1 G and k¯ h−1 H are satisfied for the reduced semi-Markov processes k¯h ηε (t) in the form of conditions k¯ h G and k¯ h H. In the variant (ii), the conditions k¯ h−1 G, k¯ h−1 H, and k¯ h−1 K (a) replace the initial conditions k¯ 0 G, k¯ 0 H, and k¯ 0 K (a). The fulfilment of the conditions k¯ h−1 G, k¯ h−1 H, and k¯ h−1 K (a) was the result of the h − 1 steps in the variant (ii) of the recurrent phase space reduction algorithm. In this case, Lemmas 9.3–9.5 and Remark 9.1 imply that the conditions k¯ h−1 G, k¯ h−1 H, and k¯ h−1 K (a) are satisfied for the semi-Markov processes k¯ h−1 η˜ε (t) in the form ˜ k¯ H, ˜ and k¯ K ˜ (a). Also, Lemmas 9.14–9.16 and Remark 9.1 of conditions k¯ h−1 G, h−1 h−1 imply that the conditions k¯ h−1 G, k¯ h−1 H, and k¯ h−1 K are satisfied for the reduced semiMarkov processes k¯h ηε (t) in the form of conditions k¯ h G, k¯ h H and k¯ h K (a). In the variant (iii), the conditions k¯ h−1 G, k¯ h−1 H, k¯ h−1 I, and k¯ h−1 IH replace the initial conditions k¯ 0 G, k¯ 0 H, k¯ 0 I, and k¯ 0 IH , The fulfilment of the conditions k¯ h−1 G, k¯ h−1 H, k¯ h−1 I, and k¯ h−1 IH was the result of the h − 1 steps in the variant (iii) of the recurrent phase space reduction algorithm. In this case, Lemmas 9.6–9.9 imply that the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, and ¯kh−1 IH are satisfied for the semi-Markov processes k¯ h−1 η˜ε (t) in the form of conditions ˜ ˜ ˜ ˜ k¯ h−1 G, k¯ h−1 H, k¯ h−1 I, and k¯ h−1 IH . Also, Lemmas 9.17–9.18 imply that the conditions G, H, I, and I k¯ h−1 k¯ h−1 k¯ h−1 H are satisfied for the reduced semi-Markov processes k¯ h−1 η (t) in the form of conditions k¯ h G, k¯ h H, k¯ h I, and k¯ h IH . k¯ h ε In the variant (iv), the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, and k¯ h−1 LH replace the initial conditions k¯0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L, and k¯ 0 LH . The fulfilment of the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, and k¯ h−1 LH was the result of the h − 1 step in the variant (iv) of the recurrent phase space reduction algorithm. In this case, Lemmas 9.12–9.13 imply that the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, and k¯ h−1 LH are satisfied for the semi-Markov processes k¯ h−1 η˜ε (t) ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ , k¯ L, ˜ and k¯ L ˜ . Also, in the form of conditions k¯ h−1 G, h−1 h−1 h−1 H h−1 h−1 H Lemmas 9.24–9.25 imply that the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, and k¯ h−1 LH hold for the reduced semi-Markov processes k¯ h ηε (t) in the form of conditions k¯ h G, k¯ h H, k¯ h I, k¯ h IH , k¯ h L, and k¯ h LH .

254

9 Perturbed SMP and reduction of phase space

Finally, in the variant (v), the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, k¯ h−1 LH , ¯kh−1 J, k¯ h−1 K, and k¯ h M (state k l ∈ k¯l−1 W0 , for l = 1, . . . , h) replace the conditions k¯0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH , k¯ 0 L, k¯ 0 LH , k¯ 0 J, k¯ 0 K and k¯ 1 M (state k 1 ∈ k¯0 W0 = W0 ). The

fulfilment of the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, k¯ h−1 LH , k¯ h−1 J, k¯ h−1 K was the result of the h − 1 step in the variant (v) of the recurrent phase space reduction algorithm. At the (h − 1)-th step, the fulfilment of the condition k¯ h−1 M was additionally assumed. At the h-th step, this assumption should be extended to the assumption that the condition k¯ h M, which is required to implement the second sub-algorithm of transforming the semi-Markov process k¯h−1 η˜ε (t) in the reduced semi-Markov process k¯h ηε (t), is satisfied. In this case, Lemmas 9.10–9.11 imply that the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, k¯ h−1 L, k¯ h−1 J, and k¯ h−1 K are satisfied for the semi-Markov processes ˜ ˜ ˜ ˜ ˜ ˜ ˜ k¯ h−1 η˜ε (t) in the form of conditions k¯ h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, k¯ h−1 LH , k¯ h−1 J, ˜ and k¯ h−1 K. Also, Lemmas 9.21–9.23 imply that the conditions k¯h−1 G, k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH , k¯ h−1 L, k¯ h−1 LH , k¯ h−1 J, k¯ h−1 K are satisfied for the reduced semi-Markov processes k¯h ηε (t) in the form of conditions k¯ h G, k¯ h H, k¯ h I, k¯ h IH , k¯ h L, k¯ h LH , k¯ h J, and k¯ h K. What is important is that in the variants (i)–(iv), the states k1, . . . , kh can be arbitrary different states in the phase space X. In the variant (v), the state k1 should be chosen such that k1 ∈ k¯0 W0 , . . ., the state kh should be chosen such that kh ∈ k¯ n−1 W0 , that is, the condition k¯ h M is assumed to be satisfied. Also, the algorithm described in Sects. 9.2 and 9.4.1.2 can additionally be applied to the reduced semi-Markov process k¯h ηε (t) and this semi-Markov process be transformed in the reduced semi-Markov process k¯h η˜ε (t). By Theorems 9.1 and 9.2, ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ , ¯ J, ˜ and k¯ K ˜ are satisfied for the the conditions k¯ h G, h h h H h h H k h h semi-Markov processes k¯h η˜ε (t). It is useful to note that the normalisation functions k¯h−1 u˜ ·,i, i ∈ k¯h−1 X, h = 1, . . . , m used for the perturbed reduced semi-Markov processes k¯h−1 η˜ε (t) take the following forms: h−1  −1 (9.111) ¯kh−1 u˜ ε,i = k¯r p¯ε,ii uε,i r=0

and the normalisation functions k¯h u ·,i, i ∈ k¯h X, n = 1, . . . , m − 1 used for the perturbed reduced semi-Markov processes k¯h ηε (t) take the following forms: k¯ h uε,i

=

k¯ h−1 u˜ ε,i .

(9.112)

It is also worth noting that the normalisation functions k¯h−1 u˜ ·,i, i ∈ k¯h−1 X, h = 1, . . . , m and k¯h u ·,i, i ∈ k¯h X, h = 1, . . . , m − 1 belong to the space of asymptotically comparable functions H. The absorbing rates condition k¯ h M mentioned above takes the following form, for h = 1, . . . , m − 1: k¯ h M:

kl ∈

k¯l−1 W0 ,

l = 1, . . . , h.

i.e.,

k¯l−1 w0,kl ,i

= limε→0

˜ ε, k l k¯ l−1 u ˜ ε, i k¯ l−1 u

∈ [0, ∞), i ∈

k¯l−1 X,

for

9.4 Recurrent reduction of phase space

255

This condition means that at each step of successive exclusion of the states k1, . . . , kh from the phase space X, the state kl is chosen from the set of the least absorbing states k¯l−1 W0 (this set is a subset of the phase space k¯l−1 X) for the semiMarkov process k¯l−1 η˜ε (t), sequentially for l = 1, . . . , h. The conditions G, H, IH , and LH imply that the limits in the asymptotic relations appearing in the condition k¯ h M exist and can be recurrently calculated using Lemmas 8.2–8.9. We denote by Mh the set of sequences of states k¯ h , such that all asymptotic relations appearing in the condition k¯ h M hold (with the limits indicated in this condition). According to Lemmas 9.20 and 9.23, the conditions G, H, IH , and LH imply that the set Mh is not empty. Thus, the condition k¯ h M, in fact, means only that the sequence k¯ h is selected from the set Mh .

9.4.2 Summary of Recurrent Phase Space Reduction Algorithm The above phase space reduction algorithm is realised in sequential recurrent steps of construction the sequence of semi-Markov processes with removed virtual transitions and reduced phase spaces, for some 0 ≤ h ≤ m−1. In addition, virtual transitions can be removed from trajectories of the resulting semi-Markov process. This sequence of recurrent transformations can be represented by the following symbolic diagram: ηε (t) =

k¯0 ηε (t)

→

k¯0 η˜ε (t)

→

k¯1 ηε (t)

··· →

k¯ h−1 η˜ε (t)

→

k¯ h η˜ε (t).

→

k¯ h ηε (t)

(9.113)

The corresponding recurrent formulas for finding transition characteristics of the above perturbed semi-Markov process with removed virtual transitions and reduced phase spaces are given in Sects. 9.2–9.4. As mentioned above, five variants of recurrent phase space reduction algorithm can be realised. In the variant (i), the initial set of conditions G, H is imposed on the semiMarkov processes ηε (t). This variant of the recurrent phase space reduction algorithm is described in Sect. 9.4.3.3. It insures the fulfilment of the similar set of conditions for the corresponding transformed semi-Markov processes. This can be represented by the following symbolic diagram: G, H = k¯ 0 G, k¯ 0 H ⇓

256

9 Perturbed SMP and reduction of phase space

˜ k¯ 0 G,

˜ k¯ 0 H

⇓ k¯ 1 G, k¯ 1 H ⇓ ··· ⇓ ˜ G,

k¯ h−1

˜ k¯ h−1 H

⇓ k¯ h G,

k¯ h H

⇓ ˜ k¯ h G,

˜ k¯ h H .

(9.114)

The corresponding recurrent relations for “re-calculation” of the above sets of conditions are given in Lemmas 9.3–9.5 and 9.14–9.16. In the variant (ii), the initial set of condition G, H, K(a) is imposed on the semi-Markov processes ηε (t). This variant of the recurrent phase space reduction algorithm is described in Sect. 9.4.1. It insures the fulfilment of the similar set of conditions for the corresponding transformed semi-Markov processes. This can be represented by the following symbolic diagram: k¯ 0 H, k¯ 0 K

(a)

⇓ ˜ k¯ H,

˜ k¯ 0 K

(a)

⇓ k¯ 1 G, k¯ 1 H,

k¯ 1 K

(a)

G, H, K (a) = k¯ 0 G, ˜ k¯ 0 G,

0

⇓ ··· ⇓ ˜ k¯ h−1 G,

˜ ˜ k¯ h−1 H, k¯ h−1 K

⇓ k¯ h G, k¯ h H,

k¯ h K

(a)

⇓ ˜ k¯ H, ˜ k¯ h G, h

˜ k¯ h K

(a) .

(a)

(9.115)

The corresponding recurrent relations for “re-calculation” of the above set of conditions are given in Lemmas 9.3–9.5 and 9.14–9.16 and Remarks 9.1 and 9.2. In the variant (iii), the initial set of conditions G, H, I, IH are imposed on the semi-Markov processes ηε (t). This variant of the recurrent phase space reduction algorithm is also described in Sect. 9.4.1. It insures the fulfilment of the similar set of conditions for the corresponding transformed semi-Markov processes. This can be represented by the following symbolic diagram:

9.4 Recurrent reduction of phase space

257

G, H, I, IH = k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH ⇓ ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ k¯ 0 G, 0 0 0 H ⇓ k¯ 1 G, k¯ 1 H, k¯ 1 I, k¯ 1 IH ⇓ ··· ⇓ ˜ k¯ h−1 G,

˜ ˜ ˜ k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH

⇓ k¯ h G, k¯ h H, k¯ h I, k¯ h IH ˜ k¯ h G,

⇓ ˜ ˜ ˜ k¯ h H, k¯ h I, k¯ h IH .

(9.116)

The corresponding recurrent relations for “re-calculation” of the above set of conditions are given in Lemmas 9.6–9.9 and 9.17–9.18. In the variant (iv), the initial set of conditions G, H, I, IH, L, LH are imposed on the semi-Markov processes ηε (t). This variant of the recurrent phase space reduction algorithm is also described in Sect. 9.4.1. It insures the fulfilment of the similar set of conditions for the corresponding transformed semi-Markov processes. This can be represented by the following symbolic diagram: G, H, I, IH, L, LH = k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH, k¯ 0 L, k¯ 0 LH ⇓ ˜ k¯ H, ˜ k¯ I, ˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ k¯ 0 G, 0 0 0 H 0 0 H ⇓ k¯ 1 G, k¯ 1 H, k¯ 1 I, k¯ 1 IH, k¯ 1 L, k¯ 1 LH

˜ k¯ h−1 G,

⇓ ··· ⇓ ˜ k¯ H,

k¯ h−1

h−1

˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ I, h−1 H h−1 h−1 H

⇓ k¯ h G, k¯ h H, k¯ h I, k¯ h IH, k¯ h L, k¯ h LH ˜ k¯ h G,

⇓ ˜ ˜ ˜ ˜ ˜ k¯ h H, k¯ h I, k¯ h IH, k¯ h L, k¯ h LH .

(9.117)

258

9 Perturbed SMP and reduction of phase space

The corresponding recurrent relations for “re-calculation” of the above set of conditions are given in Lemmas 9.12–9.13 and 9.24–9.25. In the variant (v), the initial set of conditions G, H, I, IH, L, LH, J, K and the condition k¯ h M are imposed on the semi-Markov processes ηε (t) (at the second substep of each step of the algorithm). This variant of the recurrent phase space reduction algorithm is also described in Sect. 9.4.1. It insures the fulfilment of the similar set of conditions for the corresponding transformed semi-Markov processes. This can be presented by the following symbolic diagram: G, H, I, IH, L, LH, J, K = k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH, k¯ 0 L, k¯ 0 LH, k¯ 0 J, k¯ 0 K ⇓ ˜ ˜ ˜ ˜ ˜ k¯ L ˜ , k¯ J, ˜ K ˜ k¯ 0 G, k¯ 0 H, k¯ 0 I, k¯ 0 IH, k¯ 0 L, 0 H 0 k¯ 0 ⇓ k¯ 1 M

⇒ k¯ 1 G,

k¯ 1 H, k¯ 1 I, k¯ 1 IH, k¯ 1 L, k¯ 1 LH, k¯ 1 J, k¯ 1 K

⇓ ··· ⇓ ˜ k¯ h−1 G,

˜ ˜ ˜ k¯ h−1 H, k¯ h−1 I, k¯ h−1 IH, k¯ h−1

˜ k¯ L ˜ , k¯ J, ˜ L, h−1 H h−1

⇓ k¯ h M ⇒ k¯ h G, k¯ h H, k¯ h I, k¯ h IH, k¯ h L, k¯ h LH, k¯ h J,

˜ k¯ h−1 K

k¯ h K

⇓ ˜ k¯ h G,

˜ ˜ ˜ k¯ h H, k¯ h I, k¯ h IH, k¯ h

˜ k¯ L ˜ , ¯ J, ˜ L, h H k h

˜ k¯ h K .

(9.118)

The corresponding recurrent relations for “re-calculation” of the above set of conditions are given in Lemmas 9.10–9.11 and 9.21–9.23. What is important is that, in the variants (i)–(iv), the sequence k¯ h = k1, . . . , kh can (for every 1 ≤ h ≤ m − 1) be an arbitrary sequence of different states from the phase space X. In the variant (v), the sequence k¯ h = k1, . . . , kh should (for every 1 ≤ h ≤ m−1) satisfy the condition k¯ h M (at the second sub-step of each step of the algorithm). Note also that k¯ 0 M is an “empty” condition, which is always satisfied. This explains the absence of its symbol in the first line of the above diagram. The following theorem summarises the remarks made above. Theorem 9.3 Let the conditions G, H be satisfied for the semi-Markov processes ηε (t). Then, for h = 0, . . . , m − 1: (i) The conditions G, H are satisfied for the reduced semi-Markov processes ˜ ˜ k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, k¯ h G, k¯ h H and k¯ h G, k¯ h H. (ii) If, in addition to the conditions G, H, the condition K (a) is satisfied for the semi-Markov processes ηε (t), then the condition K (a) is satisfied for the reduced

9.4 Recurrent reduction of phase space

259

semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) in the form of conditions, respectively, ˜ k¯ h K (a) and k¯ h K, (a). (iii) If, in addition to the conditions G, H, the conditions I, IH are satisfied for the semi-Markov processes ηε (t), then the conditions I, IH are satisfied for the reduced semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) in the form of conditions, respectively, ˜ ˜ k¯ h I, k¯ h IH , and k¯ h I, k¯ h IH . (iv) If, in addition to the conditions G, H, the conditions I, IH , L, LH are satisfied for the semi-Markov processes ηε (t), then the conditions I, IH , L, LH are satisfied for the reduced semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) in the form of conditions, ˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ . respectively, k¯ h I, k¯ h IH , k¯ h L, k¯ h LH , and k¯ h I, h H h h H (v) If, in addition to the conditions G, H, the conditions I, IH , L, LH , J, K and k¯ h M are satisfied for the semi-Markov processes ηε (t), then the conditions J, K are satisfied for the reduced semi-Markov processes k¯h ηε (t) and k¯h η˜ε (t) in the forms of ˜ k¯ I˜ , k¯ L, ˜ k¯ L ˜ , conditions, respectively, k¯ h I, k¯ h IH , k¯ h L, k¯ h LH , k¯ h J, k¯ h K and k¯ h I, h H h h H ˜ ˜ k¯ J, k¯ K. h

h

Chapter 10

Asymptotics of Hitting Times for Perturbed Semi-Markov Processes

This chapter plays a key role in Part II. Here we apply asymptotic recurrent phase space reduction algorithms for regularly and singularly perturbed finite semi-Markov processes to an asymptotic analysis of distributions of hitting times. It is important that the hitting times are asymptotically invariant in distribution with respect to the proposed procedures for reducing the phase space. We formulate conditions that ensure that the basic perturbation conditions imposed on the original semiMarkov processes are also satisfied for semi-Markov processes with reduced phase spaces. We also give recurrent formulas for recalculating normalisation functions, limiting distributions and expectations appearing in the corresponding perturbation conditions for semi-Markov processes with reduced phase spaces. Acting in this way, we recursively reduce the asymptotic analysis to the case, where the hitting time coincides with the first transition time for the corresponding reduced semi-Markov processes and obtain theorems on weak convergence of hitting times for regularly and singularly perturbed finite semi-Markov processes. This chapter includes three sections. In Sect. 10.1, we introduce hitting times, present variants of asymptotic recurrent algorithms of phase space reduction suitable for asymptotic analysis of hitting times for perturbed semi-Markov processes, and summarise the corresponding results in Theorem 10.1. In Lemmas 10.1–10.12, we describe the invariance properties of hitting times with respect to the above algorithms of phase space reduction and present recurrent formulas connecting distributions and Laplace transforms of hitting times for reduced semi-Markov processes. In Sect. 10.2, we present the results of an asymptotic analysis of distributions of hitting times for regularly and singularly perturbed semi-Markov processes. Theorems 10.2–10.4 present results on the weak convergence of the distributions of hitting times τε,D in a domain D for the case, where the initial state of the indicated ¯ semi-Markov processes belongs to the domain D. In Sect. 10.3, we present results of an asymptotic analysis of distributions of return times for regularly and singularly perturbed semi-Markov processes. Theorems 10.5 and 10.6 present results on the weak convergence of the distributions of return © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_10

261

262

10 Asymptotics of hitting times

times, that is, hitting times τε,D for the case, where the initial state of the indicated semi-Markov processes belongs to the domain D.

10.1 Hitting Times for Perturbed Semi-Markov Processes In this section, we introduce hitting times and present variants of asymptotic recurrent algorithms of phase space reduction adapted to the asymptotic analysis of hitting times for perturbed semi-Markov processes.

10.1.1 Hitting Times and Related Asymptotic Problems 10.1.1.1 Hitting Times. Let D be a non-empty subset of X and ε ∈ (0, 1]. The object of our interest are random functionals, τε,D =

ν ε,D

κε,n,

(10.1)

n=1

where νε,D = min(n ≥ 1 : ηε,n ∈ D).

(10.2)

The random functionals νε,D and τε,D are the first hitting times in the domain D, respectively, for the embedded Markov chain ηε,n and the semi-Markov process ηε (t). We assume that the basic regularity conditions G and H are satisfied for the semi-Markov processes ηε (t). Let us introduce, for ε ∈ (0, 1] and i ∈ X, j ∈ D, distributions, G ε,D,i j (t) = Pi {τε,D ≤ t, ηε (τε,D ) = j}, t ≥ 0, for j ∈ D,

(10.3)

Laplace transforms, ∫ Ψε.D,i j (s) =

0

∞

e−st G ε,D,i j (dt), s ≥ 0,

(10.4)

expectations, Eε,D,i j = Ei τε,D I(ηε (τε,D ) = j),

(10.5)

Pε,D,i j = Pi {ηε (τε,D ) = j} = G ε,D,i j (∞).

(10.6)

and hitting probabilities,

Let us also introduce conditional distribution functions,

10.1 Hitting times for perturbed SMP

263

Fε,D,i j (t) = Pi {τε,D ≤ t/ηε (τε,D ) = j}, t ≥ 0.

(10.7)

Yε,D,i = { j ∈ D : Pε,D,i j > 0}.

(10.8)

We define sets, As usual,

 Fε,D,i j (·) =

−1 Pε,D,i j G ε,D,i j (·) if j ∈ Yε,D,i,

Fε,D,i (·)

if j  Yε,D,i,

(10.9)

where, 

Fε,D,i (·) =

G ε,D,i j (·)

j ∈D



=

Fε,D,ir (·)Pε,D,ir , t ≥ 0.

(10.10)

r ∈Y ε,D, i

Let us also introduce Laplace transforms, ∫ ∞ e−st Fε,D,i j (dt), s ≥ 0, for j ∈ D. Φε,D,i j (s) =

(10.11)

0

Obviously,

Φε,D,i j (·) =

−1 Pε,D,i j Ψε,D,i j (·) if j ∈ Yε,D,i, if j  Yε,D,i, Φε,D,i (·)

(10.12)

where, Φε,D,i (·) =



Ψε,D,i j (·)

j ∈D



Φε,D,ir (·)Pε,D,ir , s ≥ 0.

(10.13)

Finally, let us introduce conditional expectations, ∫ ∞ tFε,D,i j (dt), s ≥ 0, for j ∈ D. Fε,D,i j =

(10.14)

=

r ∈Y ε,D, i

0

Obviously,

Fε,D,i j =

where, Fε,D,i =

−1 Pε,D,i j Eε,D,i j if j ∈ Yε,D,i, if j  Yε,D,i, Fε,D,i

 j ∈D

Eε,D,i j =

 r ∈Y ε,D, i

Fε,D,ir Pε,D,ir .

(10.15)

(10.16)

264

10 Asymptotics of hitting times

10.1.1.2 Asymptotic Problems for Hitting Times and Related Functionals. We are interested in finding conditions that entail the fulfilment of the following asymptotic relations, for i ∈ X, j ∈ D, Pε,D,i j = G ε,D,i j (∞) → G0,D,i j (∞) = P0,D,i j as ε → 0

(10.17)

and G ε,D,i j (· uˇε,i ) = Fε,D,i j (· uˇε,i )Pε,D,i j ⇒ G0,D,i j (·) = F0,D,i j (·)P0,D,i j as ε → 0,

(10.18)

where: (a) P0,D,i j , j ∈ D is a discrete distribution, i.e. P0,D,i j ≥ 0, j ∈ D and  j ∈D P0,D,i j = 1, for i ∈ X, (b) F0,D,i j (·) is a proper distribution function, i.e., F0,D,i j (∞) = 1, for i ∈ X, j ∈ D, (c) uˇ ·,i >, i ∈ X are normalisation functions such that uˇε,i > 0, ε ∈ (0, 1], for i ∈ X. The relations (10.17) and (10.18) imply that the following weak convergence relation holds, for i ∈ X:  Fε,D,i (· uˇε,i ) = G ε,D,i j (· uˇε,i ) j ∈D

⇒



G0,D,i j (·) = F0,D,i (·) as ε → 0,

(10.19)

j ∈D

where F0,D,i (·) =



F0,D,i j (·)P0,D,i j .

(10.20)

j ∈D

We are especially interested to clarify conditions, under which the corresponding limiting distributions F0,D,i (·), i ∈ X are not concentrated at zero, i.e., for i ∈ X, F0,D,i (0) < 1.

(10.21)

The relation (10.21) means, in fact, that the normalisation functions uˇ ·,i, i ∈ X are chosen correctly in the sense that the hitting times normalised by these functions do not converge in probability to 0. It is useful noting that the relations (10.17) and (10.18) are equivalent to the following relation for Laplace transforms, for i ∈ X, j ∈ D: Ψε,D,i j (s/uˇε,i ) → Ψ0,D,i j (s) = Φ0,D,i j (s)P0,D,i j as ε → 0, for s ≥ 0,

(10.22)

∫∞ where: (a) Φ0,D,i j (s) = 0 e−st F0,D,i j (dt), s ≥ 0 are Laplace transforms of proper distribution functions F0,D,i j (·) (such that F0,D,i j (∞) = 1), for i ∈ X, j ∈ D, (b)  P0,D,i j , j ∈ D is a discrete distribution, i.e., P0,D,i j ≥ 0, j ∈ D and j ∈D P0,D,i j = 1, for i ∈ X.

10.1 Hitting times for perturbed SMP

265

The relation (10.22) implies that the following relation holds, for i ∈ X:  Φε,D,i (s/uˇε,i ) = Ψε,D,i j (s/uˇε,i ) j ∈D

→ Φ0,D,i (s) = =





Ψ0,D,i j (s)

j ∈D

Φ0,D,i j (s)P0,D,i j as ε → 0, for s ≥ 0.

(10.23)

j ∈D

It is also useful noting that the relation (10.21) can be expressed in the equivalent form in terms of the corresponding Laplace transforms, for i ∈ X, Φ0,D,i (s) < 1, for s > 0.

(10.24)

We also would like to find conditions, which would imply that there exist normalisation functions u¯ ·,i > 0, i ∈ X such that the following convergence relation holds, for i ∈ X, j ∈ D: ¯ (10.25) u¯−1 ε,i Eε,D,i j → E0,D,i j < ∞ as ε → 0. We are going to present conditions insuring holding of the asymptotic relations (10.18), (10.22), and (10.25) and to construct effective recurrent algorithms, which would allow ∫us to find the above normalisation functions, the Laplace transforms ∞ Ψ0,D,i j (s) = 0 e−st G0,D,i j (dt), s ≥ 0, the limits of expectations E¯0,D,i j , and the limiting hitting probabilities P0,D,i j , for i ∈ X, j ∈ D. We are also interested to find additional conditions, under which the following asymptotic relation holds, for i ∈ X, j ∈ D: u¯ε,i /uˇε,i → Ci ∈ (0, ∞) as ε → 0,

(10.26)

and the limits for expectations satisfy the following equalities, for i ∈ X, j ∈ D: ∫ ∞ Ci E¯0,D,i j = E0,D,i j = tG0,D,i j (dt). (10.27) 0

The relations (10.25), (10.26), (10.27) imply that, for i ∈ X, j ∈ D, uˇ−1 ε,i Eε,D,i j → E0,D,i j < ∞ as ε → 0.

(10.28)

The relation (10.28) means that the expectations Eε,D,i j , normalised by the function uˇε,i (which appears in the weak convergence relation (10.18) for the distributions G ε,D,i j (· uˇε,km¯ )), converge to the first moment E0,D,i j of the corresponding limiting distribution G0,D,i j (·), as ε → 0, for i ∈ X, j ∈ D. There exist cases, where the weak convergence relation (10.18) and the convergence relation (10.25) hold, while the asymptotic relations (10.26) and (10.28) do not hold, and, for example, uˇ−1 ε,i Eε,D,i j → ∞ as ε → 0. The conditions G and H guarantee that, for any non-empty domain D ⊆ X, the initial state i ∈ X and ε ∈ (0, 1], Pi {νε,D < ∞} = Pi {τε,D < ∞} = 1.

(10.29)

266

10 Asymptotics of hitting times

The case where D = X is trivial, since νε,X = 1 and τε,X = κε,1 . In this case, the asymptotic relations given in the conditions formulated in Sect. 9.1 solve the asymptotic problems formulated above. Thus, we assume that D ⊂ X, and, thus, m ≥ 2 and 1 ≤ m¯ < m, where m¯ is the ¯ number of states in the domain D.

10.1.2 Hitting Times and Reduced Perturbation Conditions for Semi-Markov Processes We study the asymptotics of hitting times for some non-empty domain D ⊂ X. The asymptotic problems formulated above have different solutions for two cases where ¯ or i ∈ D. The main is the former case. the initial state i ∈ D The following obvious lemma plays an important role in what follows. ¯ are, for every ε ∈ (0, 1], Lemma 10.1 The distributions G ε,D,i j (t), t ≥ 0, j ∈ D, i ∈ D ¯ completely determined by the transition probabilities Q ε,kr (t), t ≥ 0, r ∈ X, k ∈ D ¯ and, thus, the distributions G ε,D,i j (t), t ≥ 0, j ∈ D, i ∈ D coincide for semi-Markov processes with any form of transition probabilities Q ε,kr (t), t ≥ 0, r ∈ X, k ∈ D. This lemma lets us simplify the model in the case, when we are interested in studying the asymptotics of hitting times for some fixed domain D and only for the ¯ This case is the subject of our study in Sects. 10.1 and 10.2 initial states i ∈ D. In what follows, we assume that the transition probabilities of the semi-Markov processes ηε (t) satisfy the following condition: ND : Q ε,kr (t) =

1 m I(t

≥ uε ), for t ≥ 0, r ∈ X, k ∈ D and ε ∈ (0, 1].

Here uε, ε ∈ (0, 1] is some function taking values in interval (0, ∞) such that uε → ∞ as ε → 0. We specify this function later. The condition ND implies that the transition probabilities pε,i j ≡ m1 , j ∈ X, for i ∈ D and, thus, the set Y1,i = X, for i ∈ D. Also, the distribution function Fε,i j (t) = Fε,i (t) = I(t ≥ uε ), t ≥ 0, for j ∈ X, i ∈ D. Obviously, the relations appearing in the condition G hold for the above transition probabilities pε,i j and distribution functions Fε,i (t), for i ∈ D. The condition G can, in fact, be reduced to the following form: ¯ (b) GD : (a) pε,i j > 0, ε ∈ (0, 1] or pε,i j = 0, ε ∈ (0, 1], for every j ∈ X, i ∈ D, ¯ Fε,i (0) < 1, i ∈ D, ε ∈ (0, 1]. Thus, the conditions GD and ND imply that the condition G is satisfied. The conditions GD and ND make it possible to reduce the condition H to the following form: ¯ there exists a chain of states i = j0, j1 ∈ D, ¯ . . . , jni −1 ∈ D, ¯ j ni ∈ D HD : For any i ∈ D, such that 1≤l ≤ni p1, jl−1 jl > 0.

10.1 Hitting times for perturbed SMP

267

Indeed, conditions GD and ND imply that, p1,i j = m1 , for any i ∈ D, j ∈ X, ¯ j ∈ X and the and 1≤l ≤ni p1, jl−1 jl p1, jni , j = 1≤l ≤ni p1, jl−1 jl m1 > 0 for any i ∈ D, chains of states appearing in the condition HD . Thus, the condition H is satisfied. The transition probabilities pε,i j ≡ m1 , j ∈ X, i ∈ D belong to any family of asymptotically comparable functions. The conditions I and IH can be reduced to the following forms: ¯ ID : pε,i j → p0,i j as ε → 0, for j ∈ X, i ∈ D and ¯ belong to a complete family of asymptotiID,H : The functions p ·,i j , j ∈ Y1,i, i ∈ D cally comparable functions H. Obviously, the distribution functions Fε,i j (tuε ) = I(t ≥ 1), t ≥ 0, for ε ∈ (0, 1], j ∈ X, i ∈ D and, thus, the asymptotic relation appearing in the condition J holds, with the normalisation function uε,i = uε, ε ∈ (0, 1], for i ∈ D and the limiting distribution function F0,i j (t) = I(t ≥ 1), t ≥ 0, for j ∈ X, i ∈ D. The condition J can be reduced to the following form: JD : (a) Fε,i j (· uε,i ) = P{κε,1 /uε,i ≤ ·/ηε,0 = i, ηε,1 = j} ⇒ F0,i j (·) as ε → 0, for ¯ (b) F0,i j (·), j ∈ Y1,i, i ∈ D ¯ are proper distribution functions j ∈ Y1,i, i ∈ D, ¯ (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ D ¯ such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ D, or equivalently: ¯ J◦D : (a) φε,i j (s/u ∫ ∞ε,i ) → φ0,i j (s) as ε → 0, for s ≥ 0 and j ∈ Y1,i, i ∈ D, (b) −st ¯ φ0,i j (s) = 0 e F0,i j (dt), s ≥ 0, j ∈ Y1,i, i ∈ D are Laplace transforms of ¯ (c) uε,i proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ D, ¯ ∈ (0, ∞), ε ∈ (0, 1], for i ∈ D. Also, the expectations fε,i j = uε , for ε ∈ (0, 1], j ∈ X, i ∈ D and, thus, the asymptotic relation appearing in the condition K holds, with the normalisation function uε,i = uε, ε ∈ (0, 1], for i ∈ D and the limiting expectation f0,i j = 1, for j ∈ X, i ∈ D. The condition K can be reduced to the following form: ¯ KD : (a) ∫ ∞ fε,i j < ∞, j ∈ Y1,i, i ∈ D, for every ε ∈ (0, 1], (b) fε,i j /uε,i → f0,i j = ¯ tF (dt) < ∞ as ε → 0, for j ∈ Y1,i, i ∈ D. 0,i j 0 Finally, let us formulate the reduced forms of conditions L and LH : ¯ LD : uε,i → u0,i ∈ (0, ∞] as ε → 0, for i ∈ D and ¯ belong to the complete family of asymptotically compaLD,H :Functions u ·,i, i ∈ D rable functions H appearing in the condition ID,H . ¯ = {k1, . . . , k m }. Recall that we denoted by m¯ the number of states in domain D We would like to realise the algorithm of sequential reduction of phase space X ¯ so in the ∈ D, for the semi-Markov process ηε (t) by excluding states k1, . . . , k m−1 ¯

268

10 Asymptotics of hitting times

end we would get the semi-Markov process k¯ m− ηε (t) and k¯ m−1 η˜ε (t), with the phase ¯ ¯ X = D ∪ {k }. space k¯ m−1 m ¯ ¯ The normalisation function uε appearing in the condition ND can always be ¯ r = 1, . . . , m, ¯ chosen in such a way that, for i ∈ k¯r −1 D, lim

k¯r −1 u˜ ε,i

ε→0

uε

= 0,

(10.30)

where ¯ k¯r −1 D

¯ \ {k1, . . . , kr−1 }. =D

(10.31)

¯ for l = 1, . . . , m¯ and, thus, the conditions k¯ M, In this case, the sets k¯l−1 W0 ⊆ D, n n = 1, . . . , m¯ − 1 will take the following reduced form: k¯ n MD :

kl ∈

k¯l−1 WD,0 ,

i.e.,

l = 1, . . . , n.

k¯l−1 w0,kl ,i

= limε→0

˜ ε, k l k¯ l−1 u ˜ ε, i k¯ l−1 u

∈ [0, ∞), i ∈

¯ k¯l−1 D,

for

In the above condition, we use the sets of the least absorbing states, k¯l−1 WD,0

= {k ∈

¯ k¯l−1 D

:

k¯l−1 w0,kl ,i

∈ [0, ∞), i ∈

¯ k¯l−1 D},

l = 1, . . . , n.

(10.32)

Three cases should be considered, in order to prove the statement formulated in the relation (10.30). m¯  (1) The function u˜ε = r=1 i ∈ k¯ X k¯r −1 u˜ ε,i → u˜0 = ∞ as ε → 0. In this case, r −1

we can choose uε = u˜2ε . In this case, the relation (10.30) obviously holds. (2) The function u˜ε → u˜0 ∈ (0, ∞) as ε → 0, but the family H is non-trivial, i.e., it includes a function h(ε) such that the corresponding comparability limit limε→0 h(ε) = aH [h(·)] = ∞. In this case, we can choose uε = h(·). Again, the relation (10.30) holds. (3) The family H is trivial, i.e., the corresponding comparability limit, aH [h(·)] = limε→0 h(ε) ∈ (0, ∞), for any function h(·) ∈ H. In this case, we can choose any function uε → ∞ as ε → 0, and then replace the family H appearing in the conditions ID and LD by the new family of asymptotically comparable functions ˜ = [H ∪ {u · }] (see Sect. 8.1.1). In this case, the relation (10.30) also holds. H The semi-Markov processes k¯r ηε (t) and k¯r η˜ε (t) have the phase space k¯r X = ¯ for r = 0, . . . , m¯ − 1. D ∪ k¯r D, ¯ and Importantly, their transition probabilities k¯r Q ε,i j (t), t ≥ 0, j ∈ k¯r X, i ∈ k¯r D ¯ ˜ ¯ are determined by the initial k¯r Q ε,i j (t), t ≥ 0, j ∈ k¯r X, i ∈k¯r D, for r = 0, . . . , m, ¯ transition probabilities Q ε,i j (t), t ≥ 0, j ∈ X, i ∈ D. ¯ and Therefore, the only transition probabilities Q ε,i j (t), t ≥ 0, j ∈ X, i ∈ D ¯ the normalisation functions uε,i, i ∈ D are used in the relations appearing in the ˜ , k¯ H ˜ , I˜ , conditions k¯ r GD , k¯ r HD , k¯ r ID,H , k¯ r ID , k¯ r JD , k¯ r KD , k¯ r LD,H , k¯ r LD , k¯ r G D r D k¯ r D,H ˜ ˜ ˜ ˜ ˜ ¯ − 1. k¯ r ID , k¯ r JD , k¯ r KD , k¯ r LD,H , k¯ r LD , and k¯ r MD , for r = 0, 1, . . . , m Therefore, it is possible to use the family of asymptotically comparable functions ˜ in the formulations of the above conditions, in particular, in the H instead of H relations appearing in these conditions and the formulas for the quantities appearing

10.1 Hitting times for perturbed SMP

269

¯ Also, for the initial states i ∈ D, ¯ it is in these relations, for the initial states i ∈ D. possible to use the corresponding analogues of Lemmas 9.1–9.25. As far as the initial states i ∈ D are concerned, the family of asymptotically ˜ and the normalisation function uε should be used. As it is comparable functions H pointed above, the condition ND implies that the corresponding relations appearing in the conditions G, H, ID , I, J, K, L, and LD are automatically satisfied, for i ∈ D. Therefore, the corresponding relations appearing in Lemmas 9.1–9.25 hold, for i ∈ D. However, these relations are not used, in the case, when we are interested in ¯ the asymptotics of the hitting times, for the initial states i ∈ D. The following theorem, which is an analogue of Theorem 9.3, summarises the remarks made in Sects. 9.4 and 10.1.2. Theorem 10.1 Let the conditions GD , HD be satisfied for the semi-Markov processes ηε (t). Then, for n = 0, . . . , m¯ − 1: (i) The conditions GD , HD are satisfied for the reduced semi-Markov processes ˜ k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, k¯ n GD , k¯ n HD and k¯ n GD , ˜ k¯ n HD . (ii) If, in addition to the conditions GD , HD , the condition KD (a) is satisfied for the semi-Markov processes ηε (t), then the condition KD (a) is satisfied for the reduced semi-Markov processes k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, ˜ k¯ n KD (a) and k¯ n KD (a). (iii) If, in addition to the conditions GD , HD , the conditions ID , ID,H are satisfied for the semi-Markov processes ηε (t), then the conditions ID , ID,H are satisfied for the reduced semi-Markov processes k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, k¯ n ID , k¯ n ID,H and k¯ n I˜ D , k¯ n I˜ D,H . (iv) If, in addition to the conditions GD , HD , the conditions ID , ID,H , LD , LD,H are satisfied for the semi-Markov processes ηε (t), then conditions ID,H , ID , LD,H , LD are satisfied for the reduced semi-Markov processes k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, k¯ n ID , k¯ n ID,H , k¯ n LD , k¯ n LD,H , and k¯ n I˜ D , k¯ n I˜ D,H , ˜ ˜ k¯ n LD , k¯ n LD,H . (v) If, in addition to the conditions GD , HD , the conditions ID , ID,H , LD , LD,H , JD , KD and k¯ n MD are satisfied for the semi-Markov processes ηε (t), then the conditions ID , ID,H , LD , LD,H , JD , KD are satisfied for the reduced semi-Markov processes k¯ n ηε (t) and k¯ n η˜ε (t) in the form of conditions, respectively, k¯ n ID , k¯ n ID,H , k¯ n LD , ˜ ˜ ˜ ˜ ˜ ˜ k¯ n LD,H , k¯ n JD , k¯ n KD and k¯ n ID , k¯ n ID,H , k¯ n LD , k¯ n LD,H , k¯ n JD , k¯ n KD .

10.1.3 Hitting Times for Semi-Markov Processes with Reduced Phase Spaces In this subsection, we present forward and backward recurrent relations for hitting times, their distributions, and Laplace transforms. These relations create the base

270

10 Asymptotics of hitting times

for asymptotic recurrent algorithms of phase space reductions for regularly and singularly perturbed semi-Markov processes used for obtaining limit theorems on weak convergence of distributions and convergence of expectations for hitting times. ¯ and introduce the hitting times for the Let us choose an arbitrary state k ∈ D semi-Markov processes η˜ε (t) and k ηε (t), for ε ∈ (0, 1], τ˜ε,D =

ν ˜ ε,D

κ˜ε,n, where ν˜ε,D = min(n ≥ 1 : η˜ε,n ∈ D)

(10.33)

n=1

and k νε,D

k τε,D

=



k κε,n,

where k νε,D = min(n ≥ 1 : k ηε,n ∈ D).

(10.34)

n=1

The definitions of the semi-Markov processes ηε (t), η˜ε (t), and the reduced semiMarkov processes k ηε (t) imply that the following lemma takes place. ˜ D, H ˜ D, Lemma 10.2 Let the conditions GD , HD be satisfied. Then the conditions G k GD , k HD are also satisfied, and the following relations take place, for ε ∈ (0, 1]: Pi {τε,D = τ˜ε,D = k τε,D,

¯ ηε (τεD ) = η˜ε (τ˜εD ) = k ηε (k τε,D )} = 1, i ∈ k D,

(10.35)

and ¯ Pk {τε,D = τ˜ε,D = κ˜ε,1 I(η˜ε,1 ∈ D) + ( κ˜ε,1 + k τε,D )I(η˜ε,1 ∈ k D), ηε (τε,D ) = η˜ε (τ˜ε,D ) = η˜ε,1 I(η˜ε,1 ∈ D) ¯ = 1. + k ηε (k τε,D )I(η˜ε,1 ∈ k D)}

(10.36)

Proof The following obvious relation takes place for the semi-Markov processes ηε (t), t ≥ 0 and η˜ε (t), t ≥ 0, for i ∈ X and ε ∈ (0, 1]: Pi {ηε (t) = η˜ε (t), t ≥ 0} = 1.

(10.37)

Let us define the following variants of the hitting times, for s ≥ 0: τε [s] = inf(t ≥ s : ηε (t) ∈ D),

(10.38)

τ˜ε [s] = inf(t ≥ s : η˜ε (t) ∈ D).

(10.39)

and ¯ and ε ∈ (0, 1], The relation (10.37) obviously implies that, for s ≥ 0, i ∈ D Pi {τε [s] = τ˜ε [s], ηε (τε [s]) = η˜ε (τ˜ε [s])} = 1.

(10.40)

¯ and ε ∈ (0, 1], Also, the relations (10.37) and (10.40) imply that, for i ∈ D

10.1 Hitting times for perturbed SMP

271

Pi {τε,D = τε [0] = τ˜ε [0] = τ˜ε,D,

ηε (τε,D ) = ηε (τε [0]) = η˜ε (τ˜ε [0]) = η˜ε (τ˜ε,D )} = 1.

(10.41)

¯ and ηε,1 ∈ D, then The relations (9.39)–(9.41) imply that: (a) if ηε,0 = i ∈ D the random functional θ ε [0] = νε,D = ν˜ε,D = 1, and, thus, τε,D = τ˜ε,D = κε,1 , ¯ and ηε,1 ∈ D, ¯ then τε,D = and ηε (τε,D ) = η˜ε (τ˜ε,D ) = ηε,1 ; (b) if ηε,0 = i ∈ D κε,1 + τε [κε,1 ] and τ˜ε,D = κε,1 + τ˜ε [κε,1 ]. Using the above remarks, the Markov property of the Markov renewal process (ηε,n, κε,n ), and the relations (10.40), (10.41) we obtain, for i ∈ D and ε ∈ (0, 1], Pi {τε,D = τ˜ε,D, ηε (τε,D ) = η˜ε (τ˜ε,D )}

= Pi {τε,D = τ˜ε,D, ηε (τε,D ) = η˜ε (τ˜ε,D ), ηε,1 ∈ D} ¯ + Pi {τε,D = τ˜ε,D, ηε (τε,D ) = η˜ε (τ˜ε,D ), ηε,1 ∈ D} = Pi {ηε,1 ∈ D} ∫ ∞ + Pk {s + τε [s] = s + τ˜ε [s], ηε (τε [s]) = η˜ε (τ˜ε [s])}Q ε,ik (ds) ¯ k ∈D

=



r ∈D

0

pε,ir +

∫ ¯ k ∈D

0

∞

Q ε,ik (ds) = 1.

(10.42)

¯ and k D ¯ =D ¯ \ {k}, k X ¯ = kD ¯ ∪ D. Recall that the state k ∈ D The relations (9.79)–(9.82) imply that k βε,0 = 0, if η˜ε,0 = i ∈ k X. In this case, the hitting times ν˜ε,D and k νε,D are connected by the following relation, for ε ∈ (0, 1]: Pi { ν˜ε,D = k βε, k νε,D } = 1, i ∈ k X.

(10.43)

Thus, the hitting times τ˜ε,D and k τε,D are connected by the following relation, for ε ∈ (0, 1]: Pi { τ˜ε,D =

=

ν ˜ ε,D

k β ε, k ν ε,D

κ˜ε,n =

n=1 νε,D k



κ˜ε,n

n=1 k κε,n

¯ = k τε,D } = 1, i ∈ k D,

(10.44)

n=1

and the random variables η˜ε (τ˜ε,D ) and k ηε (k τε,D ) are connected by the following relation, for ε ∈ (0, 1]: Pi {η˜ε (τ˜ε,D ) = η˜ε,ν˜ ε,D = η˜ε, k βε, k νε,D

= k ηε, k νε,D = k ηε (k τε,D )} = 1, i ∈ k X.

(10.45)

The relations (10.42), (10.44), and (10.45) imply that the relation (10.35) holds. The relations (9.79)–(9.82) also imply that βε,0 = 1 and η˜ε,1 ∈ k X, if η˜ε,0 = k. In this case, the following relation similar to (10.43) takes place, for ε ∈ (0, 1]:

272

10 Asymptotics of hitting times

¯ = 1. Pk { ν˜ε,D = I(η˜ε,1 ∈ D) + k βε, k νε,D I(η˜ε,1 ∈ k D)}

(10.46)

Thus, the hitting times τ˜ε,D and k τε,D are connected by the following relation, for ε ∈ (0, 1]: Pk { τ˜ε,D =

ν ˜ ε,D

κ˜ε,n

n=1 k β ε, k ν ε,D

= κ˜ε,1 I(η˜ε,1 ∈ D) + ( κ˜ε,1 +



¯ κ˜ε,n )I(η˜ε,1 ∈ k D)

n=2

k νε,D

= κ˜ε,1 I(η˜ε,1 ∈ D) + ( κ˜ε,1 +



k κε,n )I(η˜ε,1

¯ ∈ k D)

n=1

¯ = 1, = κ˜ε,1 I(η˜ε,1 ∈ D) + ( κ˜ε,1 + k τε,D )I(η˜ε,1 ∈ k D)}

(10.47)

and the random variables η˜ε (τ˜ε,D ) and k ηε (k τε,D ) are connected by the following relation, for ε ∈ (0, 1]: ¯ = 1. Pk {η˜ε (τ˜ε,D ) = η˜ε,1 I(η˜ε,1 ∈ D) + k ηε (k τε,D )I(η˜ε,1 ∈ k D)} The relations (10.47) and (10.48) imply that the relation (10.36) holds.

(10.48) 

¯ and ε ∈ (0, 1]: Let us also introduce the following distributions, for i ∈ k X, k ∈ D G˜ ε,D,i j (t) = Pi { τ˜ε,D ≤ t, η˜ε (τ˜εD ) = j}, t ≥ 0, j ∈ D,

(10.49)

k G ε,D,i j (t)

(10.50)

and = Pi {k τε,D ≤ t, k ηε (k τεD ) = j}, t ≥ 0, j ∈ D.

The following lemma is a corollary of Lemma 10.2. ˜ D, H ˜ D, Lemma 10.3 Let the conditions GD , HD be satisfied. Then the conditions G G , H are also satisfied, and the following relations take place, for ε ∈ (0, 1]: k D k D G ε,D,i j (t) = G˜ ε,D,i j (t) = k G ε,D,i j (t), t ≥ 0, j ∈ D, i ∈ k X,

(10.51)

and G ε,D,k j (t) = G˜ ε,D,k j (t) = F˜ε,k j (t) p˜ε,k j +



(F˜ε,kr (t) ∗ k G ε,D,r j (t) p˜ε,kr

¯ r∈ kD

= F˜ε,k j (t) p˜ε,k j +



(F˜ε,kr (t) ∗ G ε,D,r j (t) p˜ε,kr , t ≥ 0, j ∈ D.

(10.52)

¯ r∈ kD

Proof The equalities given in the relations (10.51) are obvious corollaries of the relation (10.35).

10.1 Hitting times for perturbed SMP

273

The first equality given in the relation (10.52) follows from the relation (10.36). The relations (9.79)–(9.82) imply that, in the case where η˜ε,0 = k, the random variable βε,0 = 1 and, thus, the random functional (k τε,D, k ηε (k τε,D )) is determined by the trajectory of the Markov renewal process (η˜ε,n, κ˜ε,n ) for n ≥ 1. This allows us to use the relation (10.36) and the Markov property of Markov renewal process (η˜ε,n, κ˜ε,n ), and to obtain the following relation, for t ≥ 0, j ∈ D and ε ∈ (0, 1]: G˜ ε,D,k j (t) = Pk { κ˜ε,1 ≤ t, η˜ε,1 = j}  Pk { κ˜ε,1 + k τε,D ≤ t, k ηε (k τε,D ) = j, η˜ε,1 = r } + ¯ r∈ kD

= F˜ε,k j (t) p˜ε,k j +



(F˜ε,kr (t) ∗ k G ε,D,r j (t) p˜ε,kr .

(10.53)

¯ r∈ kD

Thus, the second equality given in the relation (10.52) holds. The third equality given in the relation (10.52) follows from the second equality given in this relation and the relation (10.51).  Let us also introduce the following Laplace transforms, for i ∈ k X and ε ∈ (0, 1]: ˜ ε,D,i j (s) = Ei exp{−s τ˜ε,D }I(η˜ε (τ˜ε,D ) = j)}, s ≥ 0, j ∈ D, Ψ

(10.54)

and k Ψε,D,i j (s)

= Ei exp{−s k τε,D }I(k ηε (k τε,D ) = j)}, s ≥ 0, j ∈ D.

(10.55)

The next lemma reformulates the statements of Lemma 9.3 in an equivalent form of relations for the Laplace transforms of the hitting times. ˜ D, H ˜ D, Lemma 10.4 Let the conditions GD , HD be satisfied. Then the conditions G k GD , k HD are also satisfied, and the following relations take place, for ε ∈ (0, 1]: ˜ ε,D,i j (s) = k Ψε,D,i j (s), s ≥ 0, j ∈ D, i ∈ k X Ψε,D,i j (s) = Ψ

(10.56)

and ˜ ε,D,k j (s) Ψε,D,k j (s) = Ψ = φ˜ε,k j (s) p˜ε,k j +



k Ψε,D,r j (s) φ˜ε,kr (s) p˜ε,kr

¯ r∈ kD

= φ˜ε,k j (s) p˜ε,k j +



Ψε,D,r j (s)φ˜ε,kr (s) p˜ε,kr , s ≥ 0, j ∈ D.

(10.57)

¯ r∈ kD

Lemmas 10.2–10.4 allow us to reduce the study of the asymptotics for distributions of the hitting times from the case of semi-Markov processes ηε (t) to the simpler case of reduced semi-Markov processes η˜ε (t) and k ηε (t).

274

10 Asymptotics of hitting times

10.1.4 Recurrent Relations for Distributions and Laplace Transforms of Hitting Times 10.1.4.1 Recurrent Relations for Distributions and Laplace Transforms. Let ¯ an arbitrary sequence of different states from k¯ n = k1, . . . , k n be, for 1 ≤ n ≤ m, ¯ As is Sect. 9.4.1, we use the notation k¯ 0 = for an “empty” sequence the domain D. of states. Let us introduce the hitting times for the semi-Markov processes k¯ n η˜ε (t) and ¯ − 1 and ε ∈ (0, 1], ¯k n ηε (t), for n = 0, . . . , m k¯ n τ˜ε,D

=

k¯ n ν˜ ε,D



k¯ n κ˜ε,n,

where k¯ n ν˜ε,D = min(n ≥ 1 :

k¯ n η˜ε,n

∈ D)

(10.58)

k¯ n κε,n,

where k¯ n νε,D = min(n ≥ 1 :

k¯ n ηε,n

∈ D).

(10.59)

n=1

and k¯ n τε,D

=

k¯ n νε,D

 n=1

Let us also introduce the following distributions, for i ∈ and ε ∈ (0, 1]:

k¯ n X, n

= 0, . . . , m¯ − 1

˜ k¯ n G ε,D,i j (t)

= Pi {k¯ n τ˜ε,D ≤ t,

k¯ n η˜ε (k¯ n τ˜ε,D )

= j}, t ≥ 0, j ∈ D,

(10.60)

k¯ n G ε,D,i j (t)

= Pi {k¯ n τε,D ≤ t,

k¯ n ηε (k¯ n τε,D )

= j}, t ≥ 0, j ∈ D.

(10.61)

and

Finally, let us introduce the following Laplace transforms, for i ∈ 0, . . . , m¯ − 1 and ε ∈ (0, 1]:

k¯ n X, n

=

˜ k¯ n Ψε,D,i j (s)

= Ei exp{−s k¯ n τ˜ε,D }I(k¯ n η˜ε (k¯ n τ˜ε,D ) = j)}, s ≥ 0, j ∈ D,

(10.62)

k¯ n Ψε,D,i j (s)

= Ei exp{−s k¯ n τε,D }I(k¯ n ηε (k¯ n τε,D ) = j)}, s ≥ 0, j ∈ D.

(10.63)

and

Lemmas 10.5–10.10 are similar to Lemmas 10.2–10.4. They play a key role in phase space reduction algorithms and obtaining recurrent weak convergence relations for hitting times. Lemma 10.5 Let conditions GD = k¯0 GD , HD = k¯0 HD hold. Then, for every n = ˜ D , k¯ H ˜ D , k¯ GD , k¯ HD are also satisfied, and the 1, . . . , m¯ − 1, the conditions k¯ n−1 G n n n−1 following relations take place, for ε ∈ (0, 1]:

10.1 Hitting times for perturbed SMP

275

Pi {τε,D = k¯ n−1 τ˜ε,D = k¯ n τε,D,

ηε (τε,D ) =

k¯ n−1 η˜ε (k¯ n−1 τ˜ε,D )

=

k¯ n ηε (k¯ n τε,D )}

= 1, j ∈ D, i ∈

¯ k¯ n D,

(10.64)

and Pk n {τε,D = k¯ n−1 τ˜ε,D = k¯ n−1 κ˜ε,1 I(k¯ n−1 η˜ε,1 ∈ D)

+ (k¯ n−1 κ˜ε,1 + ηε (τε,D ) = +

k¯ n τε,D )I(k¯ n−1 η˜ε,1

∈

¯ k¯ n D),

k¯ n−1 η˜ε (k¯ n−1 τ˜ε,D )

=

k¯ n−1 η˜ε,1 I(k¯ n−1 η˜ε,1

k¯ n ηε (k¯ n τε,D )I(k¯ n−1 η˜ε,1

∈

¯ k¯ n D)}

∈ D)

= 1, j ∈ D.

(10.65)

Lemma 10.6 Let the conditions GD = k¯0 GD , HD = k¯0 HD be satisfied. Then, for ˜ D , k¯ H ˜ D , k¯ GD , k¯ HD are also satisfied, every n = 1, . . . , m−1, ¯ the conditions k¯ n−1 G n n n−1 and the following relations take place, for ε ∈ (0, 1]: G ε,D,i j (t) = =

˜ k¯ n−1 G ε,D,i j (t) k¯ n G ε,D,i j (t),

t ≥ 0, j ∈ D, i ∈

¯ k¯ n D,

(10.66)

and G ε,D,kn j (t) =  +

˜ k¯ n−1 G ε,D,k n j (t)

=

˜ k¯ n−1 Fε,k n r (t)

k¯ n G ε,D,r j (t) k¯ n−1 p˜ε,k n r

∗

˜ k¯ n−1 Fε,k n j (t) k¯ n−1 p˜ε,k n j

¯ r ∈ k¯ n D

=

˜ k¯ n−1 Fε,k n j (t) k¯ n−1 p˜ε,k n j +



˜ k¯ n−1 Fε,k n r (t)

∗ G ε,D,r j (t) k¯ n−1 p˜ε,kn r , t ≥ 0, j ∈ D.

(10.67)

¯ r ∈ k¯ n D

Lemma 10.7 Let the conditions GD = k¯0 GD , HD = k¯0 HD be satisfied. Then, for ˜ D , k¯ H ˜ D , k¯ GD , k¯ HD are also satisfied, every n = 1, . . . , m−1, ¯ the conditions k¯ n−1 G n n n−1 and the following relations take place, for ε ∈ (0, 1]: Ψε,D,i j (s) = =

˜ k¯ n−1 Ψε,D,i j (s) k¯ n Ψε,D,i j (s),

s ≥ 0, j ∈ D, i ∈

k¯ n X,

and ˜ ε,D,kn j (s) = k¯ φ˜ε,kn j (s) k¯ p˜ε,kn j Ψε,D,kn j (s) = k¯ n−1 Ψ n−1 n−1  ˜ + k¯ n Ψε,D,r j (s) k¯ n−1 φε,k n r (s) k¯ n−1 p˜ε,k n r ¯ r ∈ k¯ n D

(10.68)

276

10 Asymptotics of hitting times

=

˜ k¯ n−1 φε,k n j (s) k¯ n−1 p˜ε,k n j +



Ψε,D,r j (s) k¯ n−1 φ˜ε,kn r (s) k¯ n−1 p˜ε,kn r , s ≥ 0, j ∈ D.

(10.69)

¯ r ∈ k¯ n D

Lemma 10.8 Let the conditions GD = k¯0 GD , HD = k¯0 HD be satisfied. Then, for ˜ D , k¯ H ˜ D are also satisfied, every n = 0, . . . , m¯ − 1, the conditions k¯ n GD , k¯ n HD , k¯ n G n and the following relations take place, for ε ∈ (0, 1]: Pi {τε,D = k¯ n τε,D = k¯ n τ˜ε,D,

ηε (τε,D ) =

k¯ n ηε (k¯ n τε,D )

=

k¯ n η˜ε (k¯ n τ˜ε,D )}

= 1, i ∈

k¯ n X.

(10.70)

Lemma 10.9 Let the conditions GD = k¯0 GD , HD = k¯0 HD hold. Then, for every ˜ D , k¯ H ˜ D are also satisfied, and n = 0, . . . , m¯ − 1, the conditions k¯ n GD , k¯ n HD , k¯ n G n the following relations take place, for ε ∈ (0, 1]: G ε,D,i j (t) =

k¯ n G ε,D,i j (t)

=

˜ k¯ n G ε,D,i j (t),

t ≥ 0, j ∈ D, i ∈

¯ k¯ n D.

(10.71)

Lemma 10.10 Let the conditions GD = k¯0 GD , HD = k¯0 HD be satisfied. Then, for ˜ D , k¯ H ˜ D are also satisfied, every n = 0, . . . , m¯ − 1, the conditions k¯ n GD , k¯ n HD , k¯ n G n and the following relations take place, for ε ∈ (0, 1]: Ψε,D,i j (s) =

k¯ n Ψε,D,i j (s)

=

˜ k¯ n Ψε,D,i j (s),

s ≥ 0, j ∈ D, i ∈

¯ k¯ n D.

(10.72)

Proof Lemmas 10.5–10.7 can be proved by repeated application of Lemmas 10.2– 10.4. For r = 1, the statements of Lemmas 10.5–10.7 coincide with the statements of Lemmas 10.2–10.4, as applied to the semi-Markov processes k¯0 η˜ε (t) and k¯1 ηε (t), for k = k1 . For r = 2, the statements of Lemmas 10.5–10.7 coincide with the statements of Lemmas 10.2–10.4, as applied to the semi-Markov processes k¯1 η˜ε (t) and k¯2 ηε (t), for k = k2 , etc. Finally, for r = n, the statements of Lemmas 10.5–10.7 coincide with the statements of Lemmas 10.2–10.4, as applied to the semi-Markov processes k¯ n−1 η˜ε (t) and k¯ n ηε (t), for k = k n . For example, the recurrent procedure described above gives the following relation, which hold, for r = 1, . . . , n, 1 ≤ n ≤ m¯ − 1: Pi {k¯r −1 τε,D = k¯r −1 τ˜ε,D = k¯r τε,D, k¯r −1 ηε (k¯r −1 τε,D )

=

k¯r −1 η˜ε (k¯r −1 τ˜ε,D )

=

k¯r ηε (k¯r τε,D )}

= 1, j ∈ D, i ∈

¯ k¯r D.

(10.73)

The relation (10.73) obviously entails the fulfilment of the relation (10.64) given in Lemma 10.5. Lemmas 10.8–10.10 can be proved by additional application of Lemmas 10.2–  10.4 to the semi-Markov processes k¯ n ηε (t) and k¯ n η˜ε (t). 10.1.4.2 Distributions of Hitting Times and Permutation of Excluding Sequences of States. We assume now that the number of states m in the phase space X is larger than 1.

10.2 Weak asymptotics for distributions of hitting times

277

¯ = {k1, . . . , k m¯ }. Also, let k¯ n = k1, . . . , k n be a subsequence Let the domain D  = k  , . . . , k  be some permutation of ¯ of the sequence k m¯ = k1, . . . , k m¯ and k¯ m n ¯ 1 sequence k¯ n , for n = 0, 1, . . . , m¯ − 1. Here, as in Sect. 9.4.1.1, k¯ 0 = is the “empty” subsequence. The next two lemmas are direct corollaries of Lemmas 9.8 and 9.9. ¯ = {k1, . . . , k m¯ } and the conditions G and H be Lemma 10.11 Let the domain D satisfied for the semi-Markov processes ηε (t). Then, for n = 0, 1, . . . , m¯ − 1 and any permutation k¯ n = k1 , . . . , k n of the subsequence k¯ n = k1, . . . , k n , the following relation takes place, for ε ∈ (0, 1]: ¯ Pi { k¯ n τε,D = k¯ n τε,D , ηε (k¯ n τε,D ) = ηε (k¯ n τε,D )} = 1, i ∈ k¯ n D.

(10.74)

¯ = {k1, . . . , k m¯ } and the conditions G and H be Lemma 10.12 Let the domain D satisfied for the semi-Markov processes ηε (t). Then, for n = 0, 1, . . . , m¯ − 1 and any permutation k¯ n = k1 , . . . , k n of the subsequence k¯ n = k1, . . . , k n , the following relation takes place, for ε ∈ (0, 1]: k¯ n G ε,i j (t)

=

k¯ n G ε,i j (t),

t ≥ 0, j ∈ D, i ∈

¯ k¯ n D.

(10.75)

10.2 Weak Asymptotics for Distributions of Hitting Times In this section, we present limit theorems on the weak convergence of distributions of hitting times for regularly and singularly perturbed finite semi-Markov processes.

10.2.1 Asymptotics for Hitting Probabilities ¯ = {k1, . . . , k m¯ } and k¯ n = k1, . . . , k n , 1 ≤ Further, we assume that the domain D n ≤ m¯ are subsequences of the sequence k¯ m¯ = k1, . . . , k m¯ . ¯ there exists the unique 1 ≤ N( k¯ m, i) ≤ m¯ such that, Obviously, for any i ∈ D, k N (k¯ m,i) = i.

(10.76)

Note that, N( k¯ m, k n ) = n, and, thus, k N (k¯ m,kn ) = k n , for n = 1, . . . , m. ¯ First, let us give conditions of convergence for the hitting probabilities defined, ¯ j ∈ D and ε ∈ [0, 1], for i ∈ D, Pε,D,i j = Pi {ηε (τε,D ) = j} = G ε,D,i j (∞) = Ψε,D,i j (0).

(10.77)

¯ and ε ∈ (0, 1], Note that the conditions GD and HD imply that, for any i ∈ D

278

10 Asymptotics of hitting times

Pε,D,i j ≥ 0, j ∈ D,



Pε,D,i j = 1.

(10.78)

j ∈D

Taking s = 0 in the relation (10.69), we get the following backward recurrent relations for the hitting probabilities Pε,D,kn j , n = m, ¯ . . . , 1, for every j ∈ D and ε ∈ (0, 1]:  Pε,D,kl j k¯ n−1 p˜ε,kn kl , n = m, ¯ . . . , 1. (10.79) Pε,D,kn j = k¯ n−1 p˜ε,kn j + n+1≤l ≤ m ¯

¯ and ε ∈ [0, 1]: Let us introduce the following sets, for i ∈ D Yε,D,i = { j ∈ D : Pε,D,i j > 0}.

(10.80)

¯ Lemma 10.13 Let the conditions GD and HD be satisfied. Then, the sets, Yε,D,i, i ∈ D ¯ do not depend on ε ∈ (0, 1], i.e., for i ∈ D, Yε,D,i = Y1,D,i, ε ∈ (0, 1].

(10.81)

Proof According to the statement (i) of Theorem 10.1, either k¯ n−1 p˜ε,kn j = 0, ε ∈ ¯ (0, 1], or k¯ n−1 p˜ε,kn j > 0, ε ∈ (0, 1], for j ∈ X, n = 1, . . . , m. p˜ε,km¯ j , for The relation (10.79) taken for n = m, ¯ implies that Pε,D,km¯ , j = k¯ m−1 ¯ j ∈ X. The above remarks imply that the sets Yε,D,km¯ = Y1,D,km¯ , ε ∈ (0, 1]. Then, the above remarks and the relation (10.79) taken for n = m¯ − 1, imply that = Y1,D,km−1 , ε ∈ (0, 1]. the sets Yε,D,km−1 ¯ ¯ ¯ . . . , 1 and Continuing in this way, we get that, Yε,D,kn = Y1,D,kn , for any n = m, ε ∈ (0, 1]. ¯ j ∈ D and It remains noting that the functions P·,D,i j = P·,D,k N (k¯ m , i), j , for i ∈ D, ¯ and ε ∈ (0, 1]. the sets Yε,D,i = Yε,D,k N (k¯ m , i) , for i ∈ D  The following useful lemma takes place. Lemma 10.14 Let the conditions GD , HD , and ID be satisfied. Then: (i) The functions P·,D,kn j ∈ H, for j ∈ D, n = 1, . . . , m, ¯ and the following relation takes place, for j ∈ D, n = 1, . . . , m: ¯ Pε,D,kn j → P0,D,kn j as ε → 0,

(10.82)

 where limits P0,D,kn j satisfy the following relations, P0,D,kn j ≥ 0, j ∈ D, j ∈D P0,D,kn j = 1, for n = m, ¯ . . . , 1, (ii) The limits P0,D,kn j , j ∈ D, n = 1, . . . , m¯ are given by the relations (10.84), (10.85), and (10.86). Proof The statement (ii) of Theorem 10.1 implies that, for every j ∈ ¯ ∪ D, n = 1, . . . , m, D ¯ k¯

k¯ n−1 X

=

n−1

k¯ n−1 p˜ε,k n j

→

k¯ n−1 p˜0,k n j

as ε → 0.

(10.83)

10.2 Weak asymptotics for distributions of hitting times

279

The above remarks and the relation (10.79), taken for n = m, ¯ imply that the functions P·,D,km¯ j ∈ H, j ∈ D and that the following relation takes place, for j ∈ D: Pε,D,km¯ j =

p˜ε,km¯ j k¯ m−1 ¯

→

p˜0,km¯ j k¯ m−1 ¯

= P0,D,km¯ j as ε → 0.

(10.84)

Then, the above remarks, the relation (10.79), taken for n = m¯ −1, and the relation (10.84) imply that the functions P·,D,km−1 j ∈ H, j ∈ D and that the following relation ¯ takes place, for j ∈ D: Pε,D,km−1 j = ¯ →

p˜ε,km−1 j ¯ k¯ m−1 ¯ p˜0,km−1 j ¯ k¯ m−1 ¯

+ Pε,D,km j k¯ m−1 p˜ε,km−1 km ¯ ¯ + P0,D,km j k¯ m−1 p˜0,km−1 km ¯ ¯

= P0,D,km−1 ¯ , j as ε → 0.

(10.85)

Then, the above remarks and the relation (10.79), taken recurrently for n = m, ¯ . . . , 1 imply that, the functions P·,D,kn j ∈ H, j ∈ D and that the following relation takes place, for j ∈ D:  Pε,D,kn j = k¯ n−1 p˜ε,kn j + Pε,D,kl j k¯ n−1 p˜ε,kn kl n+1≤l ≤ m ¯

→

k¯ n−1 p˜0,k n j +



n+1≤l ≤ m ¯

P0,D,kl j k¯ n−1 p˜0,kn kl

= P0,D,kn j as ε → 0.

(10.86)

The relation (10.78) obviously  implies that the limits P0,D,kn j satisfy the additional ¯ . . . , 1. relations P0,D,kn j ≥ 0, j ∈ D, j ∈D P0,D,kn j = 1, for n = m, The proof is complete.  ¯ j ∈ D, makes it possible to rewrite The relation, P·,D,i j = P·,D,k N (k¯ m , i), j , for i ∈ D, ¯ relation (10.82) in an alternative equivalent form, for j ∈ D, i ∈ D, Pε,D,i j → P0,D,i j as ε → 0.

(10.87)

¯ Note that relations (10.81) and (10.82) imply that, for i ∈ D, Y0,D,i = { j ∈ D : P0,D,i j > 0} ⊆ Y1,D,i .

(10.88)

¯ j ∈ D may not coincide It should be also mentioned that the limits P0,D,i j , i ∈ D, with the corresponding hitting probabilities for the Markov chain η0,n , with the matrix of transition probabilities P0,D = p0,i j i ∈D, ¯ j ∈D given by the basic perturbation condition ID . In fact, it is possible that the random variable ν0,D = min(n ≥ 1, η0,n ∈ D) is  improper, and, thus, the probability Pi {ν0,D < ∞} = j ∈D Pi {ν0,D < ∞, η0,ν0,D = j} ¯ takes a value less than 1, for some i ∈ D.

280

10 Asymptotics of hitting times

10.2.2 Weak Asymptotic for Distributions of Hitting Times for the ¯ Case Where the Initial State Belongs to Domain D ¯ = {k1, . . . , k m¯ } and k¯ n = k1, . . . , k n , In what follows, we assume that the domain D for 1 ≤ n ≤ m¯ are subsequences of the sequence k¯ m¯ = k1, . . . , k m¯ . In this section, we consider the case, where the conditions GD , HD , ID,H , JD , MD are satisfied for KD , LD,H (and, thus, also the conditions ID and LD ), and k¯ m−1 ¯ the semi-Markov processes ηε (t). ¯ First, 10.2.2.1 The Case of the Most Absorbing Initial State in the Domain D. let us consider the case, where the initial state is k m¯ , which is the most absorbing ¯ state in domain D. ¯ =D ¯ \ {k1, . . . , k m−1 D In this case, domain k¯ m−1 ¯ } = {k m ¯ } is a one-state set. ¯ In what follows, we use the following normalisation function: uˇε,km¯ =

u˜ε,km¯ k¯ m−1 ¯

=

m−1 ¯ 

(1 −

k¯l pε,k m¯ k m¯ )

−1

uε,km¯ .

(10.89)

l=0

The following theorem takes place. Theorem 10.2 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for j ∈ D: G ε,D,km¯ j (· uˇε,km¯ ) = Fε,D,km¯ j (·)Pε,D,km¯ j ⇒ G0,D,km¯ j (·) = F0,D,km¯ j (·)P0,D,km¯ j as ε → 0,

(10.90)

where: (a) uˇ ·,km¯ is a normalisation function, which belongs to the family of asymptotically comparable functions H, (b) F0,D,km¯ j (·), j ∈ D is a proper distribution function, i.e., F0,D,km¯ j (∞) = 1, for j ∈ D, (c) P0,D,km¯ j , j ∈ D is a discrete distribution, i.e.  P0,D,km¯ j ≥ 0, j ∈ D and j ∈D P0,D,km¯ j = 1. (ii) The normalisation function uˇ ·,km¯ is given by the relation (10.89). (iii) The Laplace transforms Ψ0,D,km¯ j (·), j ∈ D of distributions G0,D,km¯ j (·), j ∈ D are given by the relations (10.95). (iv) The limiting distribution functions F0,D,km¯ j (·) = k¯ m−1 F˜0,km¯ j (·), j ∈ D are not ¯ concentrated at zero, i.e., F0,D,km¯ j (0) < 1, j ∈ D and, thus, the distribution function  F0,D,km¯ (·) = j ∈D F0,D,km¯ j (·)P0,D,km¯ j is also not concentrated at zero, i.e., F0,D,km¯ (0) < 1. (v) The limits of hitting probabilities P0,D,km¯ j = k¯ m−1 p˜0,km¯ j , j ∈ D are given by ¯ the relation (10.84). Proof The reduced semi-Markov processes k¯ m−1 ηε (t) and k¯ m−1 η˜ε (t) have the phase ¯ ¯ X = X \ {k , . . . , k } = D ∪ {k }. Therefore, space k¯ m−1 1 m−1 ¯ m ¯ ¯ Pk m¯ { k¯ m−1 τ˜ε,D = k¯ m−1 κ˜ε,1, k¯ m−1 η˜ε (k¯ m−1 τ˜ε,D ) = k¯ m−1 η˜ε,1 } = 1. ¯ ¯ ¯ ¯ ¯

(10.91)

10.2 Weak asymptotics for distributions of hitting times

281

Lemma 10.10 and the relation (10.91) imply that the following relation takes place, t ≥ 0, j ∈ D and ε ∈ (0, 1]: G ε,D,km¯ j (t) = Fε,D,km¯ j (t)Pε,D,km¯ j = k¯ Q˜ ε,km¯ j (t) = k¯ m−1 ¯

m−1 ¯

p˜ε,km¯ j . F˜ε,km¯ j (t) k¯ m−1 ¯

(10.92)

where as usual, for t ≥ 0, j ∈ D such that Pε,D,km¯ j = 0,  Fε,D,km¯ j (t)Pε,D,km¯ j G ε,D,km¯ j (t) = Fε,D,km¯ (t) = =

 j ∈D

j ∈D

p˜ε,km¯ j = Fε,km¯ (t). F˜ε,km¯ j (t) k¯ m−1 k¯ m−1 ¯ ¯

(10.93)

˜ , k¯ H ˜ , k¯ I˜ , k¯ I˜ G According to Theorem 10.1, the conditions k¯ m−1 , ¯ ¯ ¯ ¯ D m−1 D m−1 D m−1 D,H ˜ , k¯ L ˜ ˜ are satisfied for semi-Markov processes k¯ η˜ε (t). , ¯ J˜ D , k¯ m−1 L K k¯ m−1 m−1 ¯ ¯ ¯ ¯ D m−1 D,H k¯ m−1 D ¯ the probabilities k¯ n−1 p˜ε,km¯ km¯ ∈ According to the conditions k¯ n−1 HD , n = 1, . . . , m, [0, 1), for n = 1, . . . , m. ¯ Also, according to the conditions JD the initial normalisation function uε,km ∈ (0, ∞), ε ∈ (0, 1]. Therefore, the function uˇε,km¯ ∈ (0, ∞), ε ∈ (0, 1]. The conditions ID , ID,H , LD , LD,H and Lemma 8.3 imply the function uˇ ·,km¯ ∈ H. Moreover, the corresponding comparability limit, limε→0 uˇε,km¯ ∈ (0, ∞]. Lemma 10.12 implies that the asymptotic relation (10.84) holds. Theorem 10.1 implies that the following asymptotic relation, which coincides J˜ D , holds, for j ∈ D, with the asymptotic relation appearing in the condition k¯ m−1 ¯ Fε,D,km¯ j (· uˇε,km¯ ) k¯ m−1 ¯

⇒

F0,D,km¯ j (·) k¯ m−1 ¯

as ε → 0.

(10.94)

The relations (10.84) and (10.94) obviously imply that the asymptotic relation (10.90) holds, for j ∈ D. In this case, the limiting distribution G0,D,km¯ j (·) has the following Laplace transform, for j ∈ D: ∫ ∞ e−st G0,D,km¯ j (dt) Ψ0,D,km¯ j (s) = 0

=

p˜0,km¯ j , φ˜0,km¯ j (s) k¯ m−1 k¯ m−1 ¯ ¯

s ≥ 0.

(10.95)

According to the condition k¯ m−1 J˜ D , the limiting distribution functions k¯ m−1 F˜0,km¯ j (·), ¯ ¯ ˜ j ∈ D are not concentrated at zero, i.e., k¯ m−1 F0,km¯ j (0) < 1, j ∈ D. ¯ The relation (10.90) entails the fulfilment of the following asymptotic relation for ¯ distribution of hitting times, for i ∈ D:  Fε,D,km¯ (· uˇε,km¯ ) = G ε,D,km¯ j (· uˇε,km¯ ) j ∈D

⇒ F0,D,km¯ (·) =

 j ∈D

F0,D,km¯ j (·)P0,D,km¯ j

282

10 Asymptotics of hitting times

=

 j ∈D

p˜0,km¯ j F˜0,km¯ j (·) k¯ m−1 k¯ m−1 ¯ ¯

as ε → 0.

(10.96)

The above remarks and the relation (10.96) obviously imply that, F0,D,km¯ (0) < 1.

(10.97) 

The proof is complete.

¯ 10.2.2.2 The Case of the Second Most Absorbing Initial State in Domain D. Let us now consider the case, where the initial state is k m−1 ¯ , which can be referred ¯ as the second most absorbing state in the domain D. ηε (t) and k¯ m−2 η˜ε (t) have the phase The reduced semi-Markov processes k¯ m−2 ¯ ¯ ¯ = space k¯ m−2 X = X \ {k , . . . , k } = D ∪ {k , k }, while domain k¯ m−2 D 1 m−2 ¯ m−1 ¯ m ¯ ¯ ¯ ¯ \ {k1, . . . , k m−2 D } = {k , k } is a two-state set. ¯ m−1 ¯ m ¯ In what follows, we use the following normalisation function:  u˜ε,km¯ if k¯ m−2 p˜0,km−1 k m¯ > 0, ¯ k¯ m−1 ¯ ¯ = uˇε,km−1 ¯ u˜ε,km−1 if k¯ m−2 p˜0,km−1 k m¯ = 0, ¯ ¯ k¯ m−2 ¯ ¯  m−1 ¯ =

l=0

m−2 ¯ l=0

(1 − (1 −

k¯l pε,k m¯ k m¯ )

−1 u

k m−1 ¯ ¯ k¯l pε,k m−1

ε,k m¯

)−1 u

ε,k m−1 ¯

if k¯ m−2 p˜0,km−1 k m¯ > 0, ¯ ¯ if k¯ m−2 p˜0,km−1 k m¯ = 0. ¯ ¯

(10.98)

The following theorem takes place. Theorem 10.3 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for j ∈ D, G ε,D,km−1 ) ⇒ G0,D,km−1 j (· uˇ ε,k m−1 j (·) ¯ ¯ ¯ = F0,D,km−1 (·)P j 0,D,k m−1 j as ε → 0, ¯ ¯

(10.99)

where: (a) uˇ ·,km−1 is a normalisation function, which belongs to the family of asymp¯ totically comparable functions H, (b) F0,D,km−1 j (·), j ∈ D is a proper distribution ¯ function, i.e., F0,D,km¯ j (∞) = 1, for j ∈ D, (c) P0,D,km¯ j , j ∈ D is a discrete distribution,  i.e. P0,D,km¯ j ≥ 0, j ∈ D and j ∈D P0,D,km¯ j = 1. (ii) The normalisation function uˇ ·,km−1 is given by relation (10.98). ¯ (iii) The Laplace transforms Ψ0,D,km−1 (·), j ∈ D of distributions G0,D,km−1 j j (·), j ∈ ¯ ¯ p ˜ > 0, or by the D are given by the relation (10.103), in the case where k¯ m−2 0,k m−1 k m¯ ¯ ¯ p ˜ = 0. relation (10.112), in the case where k¯ m−2 0,k m−1 k m¯ ¯ ¯  (·) = j ∈D F0,D,km−1 (iv) The limiting distribution function G0,D,km−1 j (·)P0,D,k m−1 j ¯ ¯ ¯ is not concentrated at zero, i.e., G0,D,km−1 (0) < 1. ¯ (v) The limits of hitting probabilities P0,D,km−1 j , j ∈ D are given by the relation ¯ (10.85).

10.2 Weak asymptotics for distributions of hitting times

283

Proof Lemmas 10.9 and 10.10 imply that the following relation takes place, for s ≥ 0, j ∈ D and ε ∈ (0, 1]: Ψε,D,km−1 j (s) = ¯

p˜ε,km−1 φ˜ε,km−1 j (s) k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯ + Ψε,D,km¯ j (s) k¯ m−2 p˜ε,km−1 φ˜ε,km−1 k m¯ (s) k¯ m−2 k m¯ . ¯ ¯ ¯ ¯

(10.100)

Theorems 10.1 imply that the conditions k¯ m−2 GD , k¯ m−2 HD , k¯ m−2 ID , k¯ m−2 ID,H , ¯ ¯ ¯ ¯ LD , k¯ m−2 LD,H , k¯ m−2 JD , k¯ m−2 KD are satisfied for the semi-Markov processes k¯ m−2 ¯ ¯ ¯ ¯ ˜ ˜ , k¯ I˜ , k¯ I˜ ˜ , k¯ L ˜ G η (t), and conditions , k¯ H , k¯ L , ε k¯ k¯ m−2 ¯

D

¯ m−1

¯ m−1

D

¯ m−1

D

¯ m−1

D,H

¯ m−1

D

¯ m−1

D,H

hold for the semi-Markov processes k¯ m−1 η˜ε (t). ¯ ¯ the probabilities k¯ n−1 pε,km¯ km¯ , According to the conditions k¯ n−1 HD, n = 1, . . . , m, , n = 1, . . . , m¯ − 1 take values in the interval [0, 1). n = 1, . . . , m¯ and k¯ n−1 pε,km−1 k m−1 ¯ ¯ Also, according to the condition JD , the initial normalisation functions uε,km¯ , uε,km−1 ¯ take values in the interval (0, ∞). Thus, the normalisation function uˇε,km−1 given by ¯ the relation (10.98) also takes values in the interval (0, ∞). The conditions ID , ID,H , ∈ H. Moreover, the LD , LD,H , and Lemma 8.3 imply that the function uˇε,km−1 ¯ ∈ (0, ∞]. corresponding comparability limit, limε→0 uˇε,km−1 ¯ First, let us consider the case, where the limiting probability,

˜ J˜ D , k¯ m−1 K k¯ m−1 ¯ ¯ D

p˜0,km−1 k m¯ ¯ k¯ m−2 ¯

> 0.

(10.101)

In this case, the normalisation function uˇε,km−1 takes the following form: ¯ uˇε,km−1 = ¯

u˜ε,km¯ . k¯ m−1 ¯

(10.102)

The above remarks, Theorem 10.1, the relation (10.90) given in Theorem 10.2, and the relation (10.100) imply that the following relation takes place, for j ∈ D: Ψε,D,km−1 ) j (s/uˇ ε,k m−1 ¯ ¯ =

φ˜ε,km−1 j ((1 ¯ k¯ m−2 ¯ ×

−

pε,km¯ km¯ ) k¯ m−1 ¯

m ¯

s/ k¯ m−2 u˜ε,km−1 ) ¯ ¯

u˜ε,km¯ ) j (s/ k¯ m−1 ¯

×

φ˜ε,km−1 k m¯ ((1 ¯ k¯ m−2 ¯

×

p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

φ˜0,km−1 j ((1 ¯ k¯ m−2 ¯

−

−

p˜0,km−1 k m¯ ¯ k¯ m−2 ¯

pε,km¯ km¯ ) k¯ m−1 ¯

u˜ε,km−1 ¯ k¯ m−2 ¯ u˜ε,km¯ k¯ m−2 ¯

s/ k¯ m−2 u˜ε,km−1 ) ¯ ¯

p0,km¯ km¯ ) km−2 w0,km−1 p˜0,km−1 j ¯ s) k¯ m−2 ¯ ¯ ,k m ¯ k¯ m−1 ¯ ¯

+ Ψ0,D,km¯ j (s) × k¯ m−2 φ˜0,km−1 k m¯ ((1 − ¯ ¯ ×

u˜ε,km¯ k¯ m−2 ¯

p˜ε,km−1 j ¯ k¯ m−2 ¯

+ Ψε,D, k¯

→

u˜ε,km−1 ¯ k¯ m−2 ¯

p0,km¯ km¯ ) km−2 w0,km−1 ¯ s) ¯ ¯ ,k m k¯ m−1 ¯

284

10 Asymptotics of hitting times

=

wˇ 0,km−1 p˜0,km−1 φ˜0,km−1 j (k m−2 j ¯ s) k¯ m−2 ¯ ¯ ¯ ,k m ¯ k¯ m−2 ¯ ¯ + Ψ0,D,km¯ j (s) k¯ m−2 wˇ 0,km−1 p˜0,km−1 φ˜0,km−1 k m¯ (k m−2 k m¯ ¯ s) k¯ m−2 ¯ ¯ ¯ ,k m ¯ ¯ ¯

= Ψ0,D,km−1 j (s) as ε → 0, for s ≥ 0, ¯

(10.103)

where wˇ 0,km−1 k m−2 ¯ ¯ ¯ ,k m

= (1 −

p0,km¯ km¯ ) km−2 w0,km−1 ¯ . ¯ ¯ ,k m k¯ m−1 ¯

(10.104)

In this case, the limiting distribution G0,D,k j (·) has, for every j ∈ D, the ¯ ∫ ∞ m−1 −st G following Laplace transform Ψ0,D,km−1 (s) = e j 0,D,k m−1 j (dt), s ≥ 0 given by ¯ ¯ 0 the relation (10.103). The relation (10.103) entails the fulfilment of the following asymptotic relation for the Laplace transforms of hitting times:  Ψε,D,km−1 (s/ u ˇ ) = Ψε,D,km−1 (s/uˇε,km−1 ε,k j) ¯ m−1 ¯ ¯ ¯ →

 j ∈D

j ∈D

wˇ 0,km−1 p˜0,km−1 φ˜0,km−1 j (k m−2 j ¯ s) k¯ m−2 ¯ ¯ ¯ ,k m ¯ k¯ m−2 ¯ ¯

+ Ψ0,D,km¯ (s) k¯ m−2 wˇ 0,km−1 p˜0,km−1 φ˜0,km−1 k m¯ (k m−2 k m¯ ¯ s) k¯ m−2 ¯ ¯ ¯ ,k m ¯ ¯ ¯ = Ψ0,D,km−1 (s) as ε → 0, for s ≥ 0. ¯

(10.105)

According to Theorem 10.2, the probability G0,km¯ (0) < 1, and, thus, the Laplace transform, (10.106) Ψ0,D, k¯ m¯ (s) < 1, for s > 0. The above assumption (10.101) and the relations (10.105) and (10.106) imply that the following inequality holds, for s > 0:  Ψ0,D,km−1 (s) ≤ p˜0,km−1 p˜0,km−1 j + Ψ0,D, k¯ (s) · k¯ m−2 k m¯ ¯ ¯ ¯ k¯ m−2 ¯ ¯ m ¯

j ∈D


0.

(10.114)

The assumption (10.109) and the relations (10.113) and (10.114) imply that the following inequality holds, for s > 0:  Ψ0,D,km−1 (s) ≤ p˜0,km−1 φ˜0,km−1 j (s) k¯ m−2 j ¯ ¯ ¯ k¯ m−2 ¯ ¯ j ∈D


0, k¯ n−1 p˜0,k n kl = 0, n ⎪ ⎪ n if ¯ p˜0,k k = 0, n + 1 ≤ l ≤ m. ¯ n l k n−1 ⎩

(10.118)

¯ Let us denote, for 1 ≤ n , n  ≤ m, n¯  = n( ¯ k¯ m¯ , n ), n¯  = n( ¯ k¯ m¯ , n  ).

(10.119)

In what follows, we use the following lemma.  D,H , and k¯ M be satisfied Lemma 10.15 Let the conditions GD , HD , ID,H , L ¯ m−1 D for the semi-Markov processes ηε (t). Then, the following asymptotic relation takes ¯ place, for any 1 ≤ n , n  ≤ m: uˇε,kn → uˇε,kn

ˇ 0,kn ,kn k¯ m¯ w

n¯  −1 ⎧ ⎪ ⎪ ⎨ q=n¯  (1 − ⎪ = 1 n¯  −1 ⎪ ⎪ ⎪ q= n¯  (1 − ⎩

k¯ q p0,k n¯  k n¯  ) k¯ n¯  −1 w0, k¯ n¯  , k¯ n¯  k¯ q p0,k n¯  k n¯  ) k¯ n¯  −1 w0, k¯ n¯  , k¯ n¯ 

if n¯  < n¯ , if n¯  = n¯ , if n¯  < n¯  .

(10.120)

Proof The relation (10.117), (10.118), and the conditions of the lemma imply that ¯ the following relation holds, for 1 ≤ n¯  < n¯  ≤ m:

10.2 Weak asymptotics for distributions of hitting times 

n¯ −1 uˇε,kn = (1 − uˇε,kn q=n¯ 

k¯ q pε,k n¯  k n¯  )

287

k¯ n¯  −1 u˜ ε, k¯ n¯  k¯ n¯  −1 u˜ ε, k¯ n¯ 

n¯  −1

→



(1 −

q=n¯ 

k¯ q p0,k n¯  k n¯  ) k¯ n¯  −1 w0, k¯ n¯  , k¯ n¯ 

as ε → 0.

(10.121)

The proof for the case, 1 ≤ n¯  < n¯  ≤ m, ¯ is similar. The case, 1 ≤ n¯  = n¯  ≤ m¯ is trivial.  The following theorem takes place. Theorem 10.4 Let the conditions GD , HD , ID,H , JD , KD , LD,H (and, thus, also MD be satisfied for the semi-Markov processes ηε (t). conditions ID and LD ) and k¯ m−1 ¯ Then: (i) The following asymptotic relation takes place, for j ∈ D, n = m, ¯ . . . , 1: G ε,D,kn j (· uˇε,kn ) ⇒ G0,D,kn j (·) = F0,D,kn j (·)P0,D,kn j as ε → 0,

(10.122)

where: (a) uˇ ·,kn , n = m, ¯ . . . , 1 are normalisation functions, which belong to the family ¯ . . . , 1 are of asymptotically comparable functions H, (b) F0,D,kn j (·), j ∈ D, n = m, ¯ . . . , 1, (c) proper distribution functions, i.e., F0,D,kn j (∞) = 1, for j ∈ D, n = m, ¯ . . . , 1 are discrete distributions, i.e., P0,D,kn j ≥ 0, j ∈ D and P0,D,kn j , j ∈ D, n = m,  P = 1, for n = m, ¯ . . . , 1. 0,D,k j j ∈D m ¯ (ii) The normalisation functions uˇε,kn , n = m, ¯ . . . , 1 are given by the relations (10.117) and (10.118). (iii) The Laplace transforms Ψ0,D,kn j (·), j ∈ D, n = m, ¯ . . . , 1 of distributions ¯ . . . , 1 are given by the relations (10.127), if n( ¯ k¯ m¯ , n) = n, ¯ G0,D,kn j (·), j ∈ D, n = m, where n + 1 ≤ n¯ ≤ m, ¯ or by the relation (10.133), if n( ¯ k¯ m¯ , n) = n.  (iv) The limiting distribution functions G0,D,kn (·) = j ∈D F0,D,kn j (·)P0,D,kn j , n = ¯ . . . , 1. m, ¯ . . . , 1 are not concentrated at zero, i.e., G0,D,kn (0) < 1, n = m, (v) The limits of hitting probabilities P0,D,kn j , j ∈ D are given by the relation (10.86). Proof Lemma 10.7 implies that the following relation takes place, for s ≥ 0, j ∈ D and ε ∈ (0, 1]:  Ψε,D,kn j (s) = k¯ n−1 φ˜ε,kn j (s) k¯ n−1 p˜ε,kn j + Ψε,D,r j (s) ¯ r ∈ k¯ n D

× =

˜ k¯ n−1 φε,k n r (s) k¯ n−1 p˜ε,k n r

˜ k¯ n−1 φε,k n j (s) k¯ n−1 p˜ε,k n j ×

+



Ψε,D,kl j (s)

n+1≤l ≤ m ¯

˜ k¯ n−1 φε,k n kl (s) k¯ n−1 p˜ε,k n kl .

(10.123)

288

10 Asymptotics of hitting times

Theorems 10.1 implies that the conditions k¯ n GD , k¯ n HD , k¯ n ID,H , k¯ n JD , k¯ n KD , (and, thus, the conditions k¯ n ID and k¯ n LD ) are satisfied for the semi-Markov ˜ , k¯ H ˜ , k¯ I˜ ˜ , , k¯ J˜ , k¯ K processes k¯ ηε (t), and the conditions k¯ G

k¯ n LD,H

m−2 ¯

¯ m−1

D

¯ m−1

D

¯ m−1

D,H

¯ m−1

D

¯ m−1

D

˜ ˜ ) are satisfied for the semi(and, thus, the conditions k¯ m−1 L I˜ D and k¯ m−1 L ¯ ¯ D,H D Markov processes k¯ m−1 η˜ε (t). ¯ The probabilities k¯q pε,kn(¯ k¯ , n) kn(¯ k¯ , n) , q = 0, . . . , n( ¯ k¯ m¯ , n) − 1, n = m, ¯ . . . , 1 take m ¯ m ¯ values in the interval [0, 1), since all conditions k¯ n−1 HD, n = 1, . . . , m¯ are satisfied. The condition JD implies that the initial normalisation functions uε,kn , n = m, ¯ . . ., 1 take values in the interval (0, ∞). Therefore, the normalisation functions uˇε,kn , n = m, ¯ . . . , 1 given by the relations (10.117) and (10.118) also take values in the interval ¯ . . . , 1 belong to the family H due to the conditions (0, ∞). The functions uˇ ·,kn , n = m, ID , ID,H , LD , LD,H and Lemma 8.3. Moreover, limε→0 uˇε,kn ∈ (0, ∞], for n = m, ¯ . . . , 1. Let us assume that, we have already realised m¯ − n steps in the announced above asymptotic backward algorithm and, thus: (a) the relation (10.122) holds, for j ∈ D, l = m, ¯ . . . , n+1, (b) the statements (ii)–(v) are true, for j ∈ D, l = m, ¯ . . . , n+1,  in particular, the distribution function G0,D,kl (·) = j ∈D F0,D,kn j (·)P0,D,kn j is not ¯ . . . , n + 1. concentrated at zero, i.e., G0,D,kl (0) < 1, for j ∈ D, l = m, The above assumption (a) which assumes that the relation (10.122) holds can be also expressed in the equivalent form of the following relation for the corresponding Laplace transforms, for j ∈ D, l = m, ¯ . . . , n + 1: k¯ m−1 ¯

Ψε,D,kl j (s/uˇε,kl ) → Ψ0,D,kl j (s) as ε → 0, for s ≥ 0.

(10.124)

According to Theorem 10.1, the probabilities, k¯ n−1 p˜ε,k n kl

→ 0 as ε → 0, if

k¯ n−1 p˜0,k n kl

= 0.

(10.125)

First, suppose that, n( ¯ k¯ m¯ , n) = n, ¯ where n + 1 ≤ n¯ ≤ m. ¯

(10.126)

The above remarks, Theorems 10.2–10.3, the relations (10.124)–(10.125), and Lemma 10.15 imply that the following relation takes place, for j ∈ D: Ψε,D,kn j (s/uˇε,kn ) =

˜ k¯ n−1 φε,k n j ( +

 n+1≤l ¯ ≤m ¯

n−1 ¯ 

(1 −

k¯ q pε,k n¯ k n¯ )

q=n

k¯ n−1 u˜ ε,k n k¯ n−1 u˜ ε,k n¯

s/k¯ n−1 u˜ε,kn ) k¯ n−1 p˜ε,kn j

Ψε,D,kl j (s/ k¯ m¯ uˇε,kn ) k¯ n−1 φ˜ε,kn kl (s/ k¯ m¯ uˇε,kn ) k¯ n−1 p˜ε,kn kl .

+ Ψε,D,kn¯ j (s/k¯ n−1 u˜ε,kn¯ ) ¯

10.2 Weak asymptotics for distributions of hitting times

×

˜ k¯ n−1 φε,k n k n¯ (

n−1 ¯ 

(1 −

k¯ q pε,k n¯ k n¯ )

q=n

+



Ψε,D,kl j (

n+1≤l< n¯

×

n−1 ¯  (1 −

k¯ n−1 u˜ ε,k n¯

k¯ q pε,k n¯ k n¯ )

q=l

˜ k¯ n−1 φε,k n kl (

n−1 ¯ 

(1 −

k¯ q pε,k n¯ k n¯ )

q=n

→

k¯ n−1 u˜ ε,k n

˜ k¯ n−1 φ0,k n j (

n−1 ¯ 

(1 −

289

s/k¯ n−1 u˜ε,kn ) k¯ n−1 p˜ε,kn kn¯ .

k¯l−1 u˜ ε,kl k¯l−1 u˜ ε,k n¯

k¯ n−1 u˜ ε,k n k¯ n−1 u˜ ε,k n¯

s/k¯ m−l u˜ε,kl ) ¯

s/k¯ n−1 u˜ε,kn ) k¯ n−1 p˜ε,kn kl

k¯ q p0,k n¯ k n¯ ) k¯ n−1 w0,k n ,k n¯ s) k¯ n−1 p˜0,k n j

q=n

+ Ψ0,D,kn¯ j (s) ×

˜ k¯ n−1 φ0,k n k n¯ (

n−1 ¯ 

(1 −

k¯ q p0,k n¯ k n¯ ) k¯ n−1 w0,k n ,k n¯ s) k¯ n−1 p˜0,k n k n¯

q=n

+



Ψ0,D,kl j (

n+1≤l< n¯

×

n−1 ¯ 

(1 −

k¯ q p0,k n¯ k n¯ ) k¯l−1 w0,kl ,k n¯ s)

q=l

˜ k¯ n−1 φ0,k n kl (

n−1 ¯ 

(1 −

k¯ q p0,k n¯ k n¯ ) k¯ n−1 w0,k n,k n¯ s) k¯ n−1 p˜0,k n kl

q=n

=

˜ ˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn j k¯ n−1 φ0,k n j ( k¯ n−1 w + Ψ0,D,kn¯ j (s) k¯ n−1 φ˜0,kn kn¯ ( k¯ n−1 wˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn kn¯  + Ψ0,D,kl j ( k¯l−1 wˇ 0,kl ,kn¯ s) k¯ n−1 φ˜0,kn kl ( k¯ n−1 wˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn kl n+1≤l< n¯

= Ψ0,D,kn j (s) as ε → 0, for s ≥ 0.

(10.127)

In this case, the limiting ∫ ∞ distribution G0,D,kn j (·) has, for every j ∈ D, the Laplace transform Ψ0,D,kn j (s) = 0 e−st G0,D,kn j (dt), s ≥ 0 given by the relation (10.127). The relation (10.127) entails the fulfilment of the following asymptotic relation for the Laplace transforms of hitting times:  Ψε,D,kn (s/uˇε,kn ) = Ψε,D,kn, j (s/uˇε,kn ) →

 j ∈D

j ∈D

˜ ˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn j k¯ n−1 φ0,k n j ( k¯ n−1 w

+ Ψ0,D,kn¯ (s) k¯ n−1 φ˜0,kn kn¯ ( k¯ n−1 wˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn kn¯  + Ψ0,D,kl ( k¯l−1 wˇ 0,kl ,kn¯ s) n+1≤l< n¯

×

˜ ˇ 0,kn,kn¯ s) k¯ n−1 p˜0,kn kl k¯ n−1 φ0,k n kl (k¯ n−1 w

= Ψ0,D,kn (s) as ε → 0, for s ≥ 0.

(10.128)

290

10 Asymptotics of hitting times

According to Theorem 10.2 and the above induction assumptions, the probability ¯ . . . , n + 1, and, thus, for l = m, ¯ . . . , n + 1, the Laplace transforms, G0,kl (0) < 1, l = m, Ψ0,D, k¯l (s) < 1, for s > 0.

(10.129)

The assumption (10.126) and the relations (10.128) and (10.129) imply that the following inequality holds, for s > 0:  Ψ0,D,kn (s) ≤ k¯ n−1 p˜0,k n j + Ψ0,D,k n¯ (s) k¯ n−1 p˜0,k n k n¯ j ∈D

+



k¯ n−1 p˜0,k n kl

n+1≤l< n¯


0.

(10.135)

The assumption (10.132) and the relations (10.134) and (10.135) imply that the following inequality holds, for s > 0:  ˜ Ψ0,D,kn (s) = k¯ n−1 φ0,k n j (s) k¯ n−1 p˜0,k n j j ∈D


0, ε ∈ (0, 1] or pε,i j = 0, ε ∈ (0, 1], for every j ∈ X, i ∈ D, (b) G Fε,i (0) < 1, i ∈ D, ε ∈ (0, 1]. Recall the sets Yε,i = { j ∈ X : pε,i j > 0} defined in the relation (9.5). Lemma 9.1 implies that, under the condition GD , the set Yε,i = Y1,i, ε ∈ (0, 1],  D implies that the set Yε,i = Y1,i, ε ∈ (0, 1], for i ∈ D. ¯ The condition G for i ∈ D. The following condition is an analogue of the condition HD . It is required only in the case, where the return times τε,D are objects of interest:  D : For any i ∈ D, there exists a chain of states i = j0, j1 ∈ D, . . . , jn i −1 ∈ D, jn i ∈ D ¯ H such that 1≤l ≤ n i p1, jl−1 jl > 0. Analogues of the conditions ID and ID,H take the following forms: ID : pε,i j → p0,i j as ε → 0, for i ∈ D, j ∈ X. and ID,H : The functions p ·,i j , j ∈ Y1,i, i ∈ D belong to the complete family of asymptotically comparable functions H appearing in the condition ID,H . Analogues of the equivalent conditions JD and JD,1 take the following forms: J D : (a) Fε,i j (· uε,i ) ⇒ F0,i j (·) as ε → 0, for j ∈ Y1,i, i ∈ D, (b) F0,i j (·), j ∈ Y1,i, i ∈ D are proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ D, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ D. and J ◦D : (a) φε,i j (s/u ∫ ∞ε,i ) → φ0,i j (s) as ε → 0, for s ≥ 0 and j ∈ Y1,i, i ∈ D, (b) φ0,i j (s) = 0 e−st F0,i j (dt), s ≥ 0, j ∈ Y1,i, i ∈ D are Laplace transforms of proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ D, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ D. The following condition is an analogue of the condition KD . It is required only in the case where the return times τε,D are objects of interest:  D : (a) fε,i j < ∞, j ∈ Y1,i, i ∈ D, for every ε ∈ (0, 1], (b) fε,i j /uε,i → f0,i j = K ∫∞ tF0,i j (dt < ∞ as ε → 0, for j ∈ Y1,i, i ∈ D. 0

296

10 Asymptotics of hitting times

Analogues of the conditions L and LH takes the following forms:  D : uε,i → u0,i ∈ (0, ∞] as ε → 0, for i ∈ D. L and  D,H : Functions u ·,i, i ∈ D belong to the complete family of asymptotically comL parable functions H appearing in the condition LD,H . Let m be the number of states in the domain D, and let r¯m = r1, . . . , rm be a sequence of different states from the domain D. Also, let r¯n = r1, . . . , rn , n = 0, 1, . . . , m be the corresponding subsequences of the sequence r¯n = r1, . . . , rm , As in Sect. 9.4.1, r¯0 = is an “empty” sequence. Analogues of conditions k¯ n MD , n = 1, . . . , m¯ − 1 take the following forms, for n = 1, . . . , m − 1  r¯ n MD :

 D,0 , i.e., rl ∈ r¯l−1 W l = 1, . . . , n.

r¯l−1 w0,rl i

= limε→0

˜ ε, rl r¯ l−1 u ˜ ε, i r¯ l−1 u

∈ [0, ∞), i ∈

r¯l−1 D,

for

Here, the corresponding normalisation functions r¯n−1 u˜ε,i take the following forms, for i ∈ r¯l−1 D = D \ {r1, . . . , rl−1 }, l = 1, . . . , m − 1, r¯l−1 u˜ ε,i

=

n−1 

−1 r¯l p¯ε,ii

uε,i .

(10.150)

l=0

 D,0 take the following forms, for l = 1, . . . , m − 1, and the sets r¯l−1 W 

r¯l−1 WD,0

= {k ∈

r¯l−1 D

:

r¯l−1 w0,rl ,i

∈ [0, ∞), i ∈

r¯l−1 D}.

(10.151)

 D is equivalent to the condition It worth noting that the pair of conditions GD and G  G, the pair of conditions ID and ID is equivalent to the condition I, the pair of conditions ID,H and ID,H is equivalent to condition IH , the pair of conditions JD  D is equivalent and J D is equivalent to the condition J, the pair of conditions KD and K  to the condition K, the pair of conditions LD and LD is equivalent to the condition  D,H is equivalent to the condition LH . L, and the pair of conditions LD,H and L  However, the pair of conditions HD and HD is weaker than the condition H.

10.3.2 Weak Asymptotics for Distributions of Hitting and Return Times 10.3.2.1 Convergence of Hitting and Return Probabilities. The relation (10.147) implies that the following relation holds, for r, j ∈ D and ε ∈ (0, 1]: Pε,D,r j = Pr {ηε (τε,D ) = j} = Ψε,D,r j (0)  Pε,D,i j pε,ri . = pε,r j + ¯ i ∈D

(10.152)

10.3 Weak asymptotic for distributions of return times

297

Note that the condition HD and the relation (10.152) imply that, for r ∈ D and ε ∈ (0, 1],  Pε,D,r j ≥ 0, j ∈ D, Pε,D,r j = 1. (10.153) j ∈D

 D , Lemma 10.11, and the relation (10.152) also The conditions GD , HD and G imply that the sets Yε,D,r = { j ∈ D : Pε,D,r j > 0}, r ∈ D do not depend on ε ∈ (0, 1], i.e., for r ∈ D, (10.154) Yε,D,r = Y1,D,r , ε ∈ (0, 1].  D , ID , ID,H , Lemma 10.12, and the The conditions GD , HD , ID , ID,H and G relation (10.152) imply that the functions P·,D,r j ∈ H, r, j ∈ D and the following relation holds, for r, j ∈ D:  Pε,D,i j pε,ri Pε,D,r j = pε,r j + ¯ i ∈D

→ p0,r j +



P0,D,i j p0,ri = P0,D,r j as ε → 0,

(10.155)

¯ i ∈D

where the limits appearing in the above relation satisfy the following relations,  P0,D,r j ≥ 0, r, j ∈ D, j ∈D P0,D,r j = 1, for r ∈ D. The relation (10.149) implies that the following relation holds for the return probabilities, for r, j ∈ D and ε ∈ (0, 1]:  ε,D,r j (0) Pε,D,r j = Pr {ηε (τε,D ) = j} = Ψ  Pε,D,i j Pε, D,ri = ¯ .

(10.156)

¯ i ∈D

 D, H  D , ID , ID,H , Lemma 10.14, and the The conditions GD , HD , ID , ID,H and G  relation (10.156) imply that the functions P·,D,r j ∈ H, r, j ∈ D and the following relation holds, for r, j ∈ D:  Pε,D,i j Pε, D,ri Pε,D,r j = ¯ ¯ i ∈D

→



P0,D,i j P0, D,ri = P0,D,r j as ε → 0, ¯

(10.157)

¯ i ∈D

where the limits appearing in the above relation satisfy the following relations:  P0,D,r j ≥ 0, r, j ∈ D, P0,D,r j = 1, r ∈ D. (10.158) j ∈D

10.3.2.2 Weak Asymptotics for Distributions of Hitting Times. Let us assume that the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 MD and the conditions ¯       GD , ID , ID,H , LD , LD,H , JD are satisfied for the semi-Markov processes ηε (t).

298

10 Asymptotics of hitting times

The conditions GD , HD , ID,H , LD,H , following condition holds: ˆ : L D,1

uˇ ε, k n uε, r

→

 0,kn,r k¯ m¯ w

MD k¯ m−1 ¯

 D,H imply that the and ID,H , L

¯ ∈ [0, ∞] as ε → 0, for r ∈ D, k n ∈ D.

¯ and u ·,r , r ∈ D belong to the class of asympSince the functions uˇ ·,kn , k n ∈ D ˆ can be totically comparable functions H, the limits appearing in the condition L D calculated using the quotient operating formulas given in Lemma 8.2. In the case, where the family H is one of the families, H1 , H2 , or H3 , the quotient operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used. Let us introduce sets, for r ∈ D, ¯ k¯ m¯ Dr

¯ : p0,r kn > 0} = {k n ∈ D

(10.159)

and denote,  k¯ m¯ , r) = n(

max n( ¯ k¯ m¯ , n),

¯r k n ∈ k¯ m¯ D

where n( ¯ k¯ m¯ , n) are the numbers defined in the relation (10.118). Let us define the normalisation functions, for r ∈ D,  uε,r if k¯ m¯ w 0,kn( k¯ , r ),r ∈ [0, ∞), m ¯ uˇε,r = if k¯ m¯ w 0,kn( k¯ , r ),r = ∞. uˇε,kn(k  m ¯ ,r)

(10.160)

(10.161)

m ¯

The following theorem takes place. Theorem 10.5 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 MD ¯  D , ID , ID,H , L  D, L  D,H , J D be satisfied for the semi-Markov processes ηε (t). and G Then: (i) The following asymptotic relation takes place, for j, r ∈ D, G ε,D,r j (· uˇε,r ) ⇒ G0,D,r j (·) = F0,D,r j (·)P0,D,r j as ε → 0,

(10.162)

where: (a) uˇ ·,r , r ∈ D are normalisation functions, which belong to the family of asymptotically comparable functions H, (b) F0,D,r j (·), j, r ∈ D are proper distribution functions, i.e., F0,D,r j (∞) = 1, for j, r ∈ D,, (c) P0,D,r j , j, r ∈ D are discrete  distributions, i.e., P0,D,r j ≥ 0, j, r ∈ D and j ∈D P0,D,r j = 1, for r ∈ D. (ii) The normalisation functions uˇ ·,r , r ∈ D are given by the relation (10.161). (iii) The Laplace transforms Ψ0,D,r j (·), j, r ∈ D of distributions G0,D,r j (·), j, r ∈ D are given by the relations (10.165) and (10.171).  (iv) The limiting distribution functions G0,D,r (·) = j ∈D F0,D,r j (·)P0,D,r j , r ∈ D are not concentrated at zero, i.e., G0,D,r (0) < 1, r ∈ D. (v) The limits of hitting probabilities P0,D,r j , j, r ∈ D are given by the relation (10.155).

10.3 Weak asymptotic for distributions of return times

299

Proof Let us choose an arbitrary state r ∈ D. First, let us consider case, where the following assumption holds true:  0,kn( k¯ , r ),r k¯ m¯ w m ¯

∈ [0, ∞).

(10.163)

In this case, the normalisation function, uˇε,r = uε,r .

(10.164)

The conditions of Theorem 10.5, the relation (10.147), and Theorem 10.4 imply that the following relation holds, for j ∈ D: Ψε,D,r j (s/uˇε,r ) = φε,r j (s/uε,r )pε,r j + Ψε,D,kn( k¯

× φε,r kn( k¯ +

m ¯

(s

uˇε,kn( k¯

,r)

m ¯ /uˇε,kn( k¯ , r ) ) m ¯ uε,r (s/uε,r )pε,r kn( k¯ , r ) ,r)

,j m ¯ ,r)



m ¯

Ψε,D,kn j (s

uˇε,kn( k¯

m ¯ ,r)

uε,r

¯ r ,nn( k n ∈ k¯ m¯ D  k¯ m¯ ,r)

uˇε,kn /uˇε,kn ) uˇε,kn( k¯ , r ) m ¯

× φε,r,kn (s/uε,r )pε,r kn  uˇε,kn( k¯ , r ) uˇε,k m ¯ n + Ψε,D,kn j (s /uˇε,kn ) u u ˇ ε,r ε,k ¯ n(  k ,r) ¯ ¯ k n ∈D\ k¯ m¯ Dr

m ¯

× φε,r kn (s/uε,r )pε,r kn → φ0,r j (s)p0,r j + Ψ0,D,kn( k¯ , r ), j (s k¯ m¯ w 0,kn( k¯ m ¯

m ¯ ,r)

,r )

× φ0,r kn( k¯ , r ) (s)p0,r kn( k¯ , r ) m ¯ m¯ + Ψε,D,kn j (s k¯ m¯ w 0,kn( k¯ ¯ r ,nn( k n ∈ k¯ m¯ D  k¯ m¯ ,r)

m ¯ ,r)

ˇ 0,kn,kn( k¯ , r ) ) ,r k¯ m¯ w m ¯

× φ0,r,kn (s)p0,r kn = Ψ0,D,r j (s) as ε → 0, for s ≥ 0.

(10.165)

In this case, the limiting ∫ ∞ distribution G0,D,r j (·) has, for every j ∈ D, the Laplace transform Ψ0,D,r j (s) = 0 e−st G0,D,r j (dt), s ≥ 0 given by the relation (10.165). Theorem 10.4 and the relation (10.165) also imply that the following relation holds:

300

10 Asymptotics of hitting times

Ψε,D,r (s/uˇε,r ) = →





Ψε,D,r j (s/uˇε,r )

j ∈D

φ0,r j (s)p0,r j

j ∈D

+ Ψ0,D,kn( k¯

m ¯ ,r)

(s k¯ m¯ w 0,kn( k¯

m ¯ ,r)

,r )

× φ0,r kn( k¯ , r ) (s)p0,r kn( k¯ , r ) m ¯ m¯ + Ψε,D,kn (s k¯ m¯ w 0,kn( k¯ ¯ r ,nn( k n ∈ k¯ m¯ D  k¯ m¯ ,r)

m ¯ ,r)

ˇ 0,kn,kn( k¯ , r ) ) ,r k¯ m¯ w m ¯

× φ0,r,kn (s)p0,r kn = Ψ0,D,r (s) as ε → 0, for s ≥ 0.

(10.166)

The conditions JD,1 and J D,1 imply that, φ0,r j (s) < 1, j ∈ D and φ0,kn j (s) < ¯ for s > 0. These inequalities and the relation (10.166) imply that the 1, k n ∈ D, following inequalities holds, for s > 0:  Ψ0,D,r (s) ≤ φ0,r j (s)p0,r j j ∈D

+ φ0,r kn( k¯ , r ) (s)p0,r kn( k¯ , r ) m ¯ m ¯  + φ0,r,kn (s)p0,r kn ¯ r ,nn( k n ∈ k¯ m¯ D  k¯ m¯ ,r)


0. These inequalities Theorem 10.4 implies that, Ψ0,D,kn, j (s) < 1, k n ∈ D, and relation (10.172) imply that the following inequality holds, for s > 0:

302

10 Asymptotics of hitting times

Ψ0,D,r (s) ≤ p0,r j + Ψ0,D,kn( k¯ , r ) (s)p0,r kn( k¯ , r ) m ¯  m¯ + p0,r kn ¯ r ,nn( k n ∈ k¯ m¯ D  k¯ m¯ ,r)


0}

(10.175)

and denote,  k¯ m¯ , r¯m , rl ) = n(

max

¯r k n ∈ k¯ m¯ , r¯ m D l

n( ¯ k¯ m¯ , n).

Let us define the normalisation functions, for rl ∈ D,  uˇε,rl if k¯ m¯ , r¯m w 0,kn( k¯ , r¯ , r ),rl ∈ [0, ∞),  l m ¯ m uε,rl =  uˇε,kn(k w if = ∞. ¯ 0,k ¯ k , r ¯

 m , r ) n(  k , r¯ , r ),rl m ¯ m  ¯ m ¯

m 

(10.176)

(10.177)

l

The following theorem takes place.  D , IH , LH (and, thus, also conditions I Theorem 10.6 Let the conditions G, HD , H  MD be satisfied for the semi-Markov processes ηε (t). MD , r¯ m−1 and L), J, K, and k¯ m−1  ¯ Then:

10.3 Weak asymptotic for distributions of return times

303

(i) The following asymptotic relation takes place, for j, rl ∈ D: G ε,D,rl j (· uε,r ) ⇒ G 0,D,rl j (·) = F0,D,rl j (·)P0,D,rl j as ε → 0,

(10.178)

where: (a) u ·,rl , rl ∈ D are normalisation functions, which belong to the family of asymptotically comparable functions H, (b) F0,D,rl j (·), j, rl ∈ D are proper distribution functions, i.e., F0,D,rl j (∞) = 1, for j, rl ∈ D, (c) P0,D,rl j , j, r ∈ D are discrete  distributions, i.e., P0,D,rl j ≥ 0, j, r ∈ D and j ∈D P0,D,rl j = 1, for rl ∈ D. (ii) The normalisation functions uε,r , r ∈ D are given by the relation (10.177).  0,D,r j (·), j, rl ∈ D of distributions G 0,D,r j (·), j, rl ∈ (iii) The Laplace transforms Ψ l l D are given by the relations (10.181) and (10.187).  (iv) The limiting distribution functions G 0,D,rl (·) = j ∈D F0,D,rl j (·)P0,D,rl j , rl ∈ D are not concentrated at zero, i.e., G 0,D,rl (0) < 1, rl ∈ D. (v) The limits of the return probabilities P0,D,rl j , j, rl ∈ D are given by the relations (10.155) and (10.157). Proof Let us now choose an arbitrary state rl ∈ D. First, let us consider case, where the following assumption holds true:  0,kn,rl k¯ m¯ , r¯m w

∈ [0, ∞).

(10.179)

In this case, the normalisation function, uε,rl = uˇε,rl .

(10.180)

The conditions of Theorem 10.6, the relation (10.149), and Theorem 10.4 imply that the following relation holds, for j ∈ D:  ε,D,r j (s/uε,r ) Ψ l l = Ψε,D,kn( k¯

, j (s

uˇε,kn( k¯

 , rl ) m ¯ , r¯ m

× Ψε, D,r ¯ lk ¯ n(  k

 , rl ) m ¯ , r¯ m

 , rl ) m ¯ , r¯ m

uˇε,rl

 , rl ) m ¯ , r¯ m

Ψε,D,kn j (s

uˇε,kn( k¯

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

× +

uˇε,kn uˇε,kn( k¯

 , rl ) m ¯ , r¯ m



¯ ¯ ¯ k n ∈D\ D k m  rl ¯ , r¯ m

)

(s/uˇε,rl )



+

/uˇε,kn( k¯

/uˇε,kn )Ψε, D,r ¯ l ,k n (s/uˇ ε,rl )

Ψε,D,kn j (s

uˇε,kn( k¯

 , rl ) m ¯ , r¯ m

uε,rl

 , rl ) m ¯ , r¯ m

uˇε,rl

304

10 Asymptotics of hitting times

×

uˇε,kn uˇε,kn( k¯

→ Ψ0,D,kn( k¯ +

 , rl ) m ¯ , r¯ m

/uˇε,kn )Ψε, D,r ¯ l k n (s/uˇ ε,rl )

 0,kn( k¯ , r ), r¯m ,rl ) , j (s k¯ m¯ , r¯m w m ¯

 , rl ) m ¯ , r¯ m

× Ψ0, D,r (s) ¯ lk ¯ n(  km , r¯ , r ) ¯ m l

Ψε,D,kn j (s k¯ m¯ , r¯m w 0,kn( k¯

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

×

,rl

m ¯ , rl )

ˇ 0,kn,kn( k¯ , r ) )Ψ0, D,r ¯ l ,k n (s) k¯ m¯ w m ¯ l

 0,D,r j (s) as ε → 0, for s ≥ 0. =Ψ l

(10.181)

 In this case, the limiting ∫ ∞ distribution G0,D,rl j (·) has, for every j ∈ D, the Laplace −st   transform Ψ0,D,rl j (s) = 0 e G0,D,rl j (dt), s ≥ 0 given by the relation (10.181). Theorem 10.4 and relation (10.181) also imply that the following relation holds:   ε,D,r j (s/uε,r )  ε,D,r (s/uε,r ) = Ψ Ψ l l l l j ∈D

→ Ψ0,D,kn( k¯ +

 , rl ) m ¯ , r¯ m

(s k¯ m¯ , r¯m w 0,kn( k¯

× Ψ0, D,r (s) ¯ lk ¯ n(  km , r¯ , r ) ¯ m l

, r¯m ,rl )

m ¯ ,r)

Ψε,D,kn (s k¯ m¯ , r¯m w 0,kn( k¯

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

×

,rl

m ¯ , rl )

ˇ 0,kn,kn( k¯ , r ) )Ψ0, D,r ¯ l ,k n (s) k¯ m¯ w m ¯ l

 0,D,r (s) as ε → 0, for s ≥ 0. =Ψ l

(10.182)

Theorem 10.4 implies that, Ψ0, D,r ¯ l (s) < 1, for s > 0. These inequalities and the relation (10.172) imply that the following inequality holds, for s > 0:  0,D,r (s) ≤ Ψ0, D,r Ψ ¯ lk ¯ l n(  k +

 , rl ) m ¯ , r¯ m



(s) Ψ0, D,r ¯ l ,k n (s)

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

= Ψ0, D,r ¯ l (s) < 1, for s > 0.

(10.183)

The relation (10.183) implies that the following relation holds: G 0,D,rl (0) < 1.

(10.184)

Second, let us consider case, where the following assumption holds true:  0,kn,rl k¯ m¯ , r¯m w

= ∞.

(10.185)

10.3 Weak asymptotic for distributions of return times

305

In this case, the normalisation function, uε,rl = uˇε,kn(k .  m ¯ ,r)

(10.186)

The conditions of Theorem 10.6, the relation (10.149) and Theorem 10.4 imply that the following relation holds, for j ∈ D:  ε,D,r j (s/uε,r ) Ψ l l = Ψε,D,kn( k¯ , r¯ m ¯

, j (s/uˇ ε,k n(  k¯ m  ,r ¯ , r¯ m

m  , rl )

× Ψε, D,r ¯ lk ¯ n(  k

 , rl ) m ¯ , r¯ m

(s



+

l)

)

uˇε,rl uˇε,kn( k¯

/uˇε,rl )

 , rl ) m ¯ , r¯ m

Ψε,D,kn j (s

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

× Ψε, D,r ¯ l ,k n (s

m ¯



+

uˇε,rl uˇε,kn( k¯ , r¯

+

uˇε,kn uˇε,kn( k¯

uˇε,rl uˇε,kn( k¯

/uˇε,kn ) m  , rl )

/uˇε,rl )

¯ ¯ ¯ k n ∈D\ D k m  rl ¯ , r¯ m

→ Ψ0,D,kn( k¯

m ¯

m  , rl )

Ψε,D,kn j (s

× Ψε, D,r ¯ l k n (s

uˇε,kn uˇε,kn( k¯ , r¯

/uˇε,kn )

 , rl ) m ¯ , r¯ m

/uˇε,rl )

 , rl ) m ¯ , r¯ m

, j (s)

 , rl ) m ¯ , r¯ m



Ψ0,D,kn j (s k¯ m¯ wˇ 0,kn,kn( k¯

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

m ¯ ,r)

)

× P0, D,r ¯ l ,k n

 0,D,r j (s) as ε → 0, for s ≥ 0. =Ψ l

(10.187)

 In this case, the limiting ∫ ∞ distribution G0,D,rl j (·) has, for every j ∈ D, the Laplace −st   transform Ψ0,D,rl j (s) = 0 e G0,D,rl j (dt), s ≥ 0 given by the relation (10.187). Theorem 10.4 and the relation (10.181) also imply that the following relation holds:   ε,D,r j (s/uε,r )  ε,D,r (s/uε,r ) = Ψ Ψ l l l l j ∈D

→ Ψ0,D,kn( k¯ +

(s)P0, D,r ¯ lk ¯ n(  km  , rl ) ¯ , r¯ m  Ψ0,D,kn (s k¯ m¯ wˇ 0,kn,kn( k¯

 , rl ) m ¯ , r¯ m

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

 0,D,r (s) as ε → 0, for s ≥ 0. =Ψ l

m ¯ ,r)

)P0, D,r ¯ l ,k n (10.188)

306

10 Asymptotics of hitting times

¯ for s > 0. These inequalities Theorem 10.4 implies that, Ψ0,D,kn (s) < 1, k n ∈ D, and the relation (10.172) imply that the following inequality holds, for s > 0:  0,D,r (s) ≤ Ψ0,D,k ¯ Ψ l n(  k

(s)P0, D,r ¯ lk ¯ n(  km  , rl ) ¯ , r¯ m  P0, D,r ¯ l ,k n

 , rl ) m ¯ , r¯ m

+

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

< P0, D,r ¯ lk ¯ n(  k +

 , rl ) m ¯ , r¯ m



P0, D,r ¯ l ,k n = 1.

(10.189)

¯ r ,nn( k n ∈ k¯ m¯ , r¯ m D  k¯ m¯ , r¯m ,rl ) l

The relation (10.189) implies that the following relation holds: G 0,D,rl (0) < 1. The proof is complete.

(10.190) 

Chapter 11

Asymptotics for Expectations of Hitting Times for Perturbed Semi-Markov Processes

In this chapter, we apply asymptotic recurrent phase space reduction algorithms for regularly and singularly perturbed finite semi-Markov processes to an asymptotic analysis of expectations of hitting times and obtain theorems about convergence of expectations of hitting times for regularly and singularly perturbed finite semi-Markov processes. Special attention is paid to the conditions under which the expectations of hitting times, normalised by the functions used in the weak limit theorems for the distributions of the hitting times, converge to the first moments of the corresponding limiting distributions. This may not be the case for singularly perturbed semi-Markov processes. Such simultaneous convergence plays an important role in ergodic theorems for perturbed Markov type processes. This chapter includes three sections. In Sect. 11.1, we present variants of the asymptotic recurrent algorithms of phase space reduction suitable for asymptotic analysis of expectations of hitting times for regularly and singularly perturbed semi-Markov processes. The corresponding recurrent relations for expectations of hitting times are presented, in Lemmas 11.1– 11.3. In Sect. 11.2, the asymptotics for expectations of hitting times is presented in Theorems 11.1–11.3. In addition, in Theorem 11.4, conditions are given that guarantee the simultaneous convergence of distributions and expectations of the hit time for regularly and singularly perturbed semi-Markov processes. In Sect. 11.3, limit theorems for expectations of return times and related asymptotic results are given in Theorems 11.5 and 11.6.

11.1 Expectations of Hitting Times In this section, we present results concerning the convergence of expected response times for regularly and singularly perturbed semi-Markov processes. These results supplement the results on the weak asymptotics of the distributions of the impact moments for regularly and singularly perturbed semi-Markov processes. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_11

307

308

11 Asymptotics for expectations of hitting times for perturbed SMP

11.1.1 Recurrent Relations for Expectations of Hitting Times Let us introduce the expectations of hitting times for the original semi-Markov processes ηε (t), for i ∈ X, j ∈ D and ε ∈ [0, 1], Eε,D,i j = Ei τε,D I(ηε (τε,D ) = j) ∫ ∞ = tG ε,D,i j (dt),

(11.1)

0

and Eε,D,i = Ei τε,D ∫  = Eε,D,i j =

0

j ∈D

∞

tG ε,D,i (dt).

(11.2)

Let us, also, introduce the expectations, of hitting times for the reduced semi¯ j ∈ D, n = 0, . . . , m¯ − 1 and Markov processes k¯ n ηε (t) and k¯ n η˜ε (t), for i ∈ D, ε ∈ [0, 1], k¯ n Eε,D,i j

= Ei k¯ n τε,D I( k¯ n ηε ( k¯ n τε,D = j) ∫ ∞ = t k¯ n G ε,D,i j (dt)

(11.3)

= Ei k¯ n τ˜ε,D I( k¯ n η˜ε ( k¯ n τ˜ε,D = j) ∫ ∞ = t k¯ n G˜ ε,D,i j (dt).

(11.4)

0

and ˜ k¯ n Eε,D,i j

0

The following lemmas supplement Lemmas 10.5–10.10. As in these lemmas, let ¯ k¯ m¯ = k1, k2, . . . , k m¯ be an arbitrary sequence of different states from the domain D and k¯ n = k1, k2, . . . , k n , n = 0, 1, . . . , m¯ be the subsequences of the sequence k¯ m¯ . As in Sect. 9.4.1, r¯0 = is an “empty” sequence. Lemma 11.1 Let the conditions GD = k¯0 GD , HD = k¯0 HD , and KD (a) = k¯0 KD (a) ˜ D , k¯ H ˜ D , k¯ GD , be satisfied. Then, for every n = 1, . . . , m¯ − 1, the conditions k¯ n−1 G n n−1 ˜ k¯ n HD and k¯ n−1 KD , k¯ n KD are satisfied, and the following relations take place, for ε ∈ (0, 1]: Eε,D,i j =

˜ k¯ n−1 Eε,D,i j

=

k¯ n Eε,D,i j

< ∞, j ∈ D, i ∈

and Eε,D,kn j =

˜ k¯ n−1 Eε,D,k n j

=

˜ k¯ n−1 fε,k n j k¯ n−1 p˜ε,k n j

¯ k¯ n D,

(11.5)

11.1 Expectations of hitting times



+

309

(k¯ n−1 f˜ε,kn r k¯ n−1 Pε,D,r j +

k¯ n Eε,D,r j ) k¯ n−1 p˜ε,k n r

¯ r ∈ k¯ n D

=

˜ k¯ n−1 fε,k n j k¯ n−1 p˜ε,k n j 

+

(k¯ n−1 f˜ε,kn r Pε,D,r j + Eε,D,r j ) k¯ n−1 p˜ε,kn r < ∞, j ∈ D.

(11.6)

¯ r ∈ k¯ n D

Lemma 11.2 Let the conditions GD = k¯0 GD , HD = k¯0 HD and KD = k¯0 KD (a) be ˜ D , k¯ H ˜ D, satisfied. Then, for every n = 0, . . . , m¯ − 1, the conditions k¯ n GD , k¯ n HD , k¯ n G n and k¯ n KD are satisfied, and the following relation takes place, for ε ∈ (0, 1]: Eε,D,i j =

k¯ n Eε,D,i j

=

˜ k¯ n Eε,D,i j

< ∞, j ∈ D, i ∈

¯ k¯ n D.

(11.7)

Proof The equalities given in the relations (11.5), (11.6), and (11.7) are direct corollaries of the equalities given, respectively, in the relations (10.66), (10.67), and (10.71), for both cases, where the corresponding expectations Eε,D,i j are finite or infinite. In the latter case, the product Eε,D,r j k¯ n−1 p˜ε,kn r should be counted as 0, if the probability p˜ε,kn r = 0. K is However, the statement (ii) of Theorem 10.1 implies that the condition k¯ m−1 ¯ satisfied, and, thus, for j ∈ D and ε ∈ (0, 1], Eε,D,km¯ j =

p˜ε,km¯ j f˜ε,km¯ j k¯ m−1 k¯ m−1 ¯ ¯

< ∞.

(11.8)

The relation (11.6), taken for n = m¯ − 1, implies that, for j ∈ D and ε ∈ (0, 1], Eε,D,km−1 j = ¯

p˜ε,km−1 f˜ε,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯ + (k¯ m−2 p˜ε,km−1 f˜ε,km−1 k m¯ Pε,D,k m¯ j + Eε,D,k m¯ j ) k¯ m−2 k m¯ < ∞. ¯ ¯ ¯ ¯

(11.9)

Continuing the recurrent use of the relation (11.6), for n = m¯ − 2, . . . , 1, we prove that, for n = m, ¯ . . . , 1, j ∈ D and ε ∈ (0, 1], Eε,D,kn j < ∞. The proof of Lemmas 11.1 and 11.2 is complete.

(11.10) 

11.1.2 Recurrent Relations for Limits of Expectations of Hitting Times In what follows, we assume that the conditions of Theorem 10.4, GD , HD , ID , ID,H , MD are satisfied for the semi-Markov processes ηε (t). LD , LD,H , JD , KD , and k¯ m−1 ¯ According to Theorem 10.4, the weak convergence relation G ε,D,i j (·uˇε,i ) ⇒ ¯ j ∈ D. This relation implies that, for G0,D,kn j (·) as ε → 0, takes place, for i ∈ D, ¯ j ∈ D and any sequence of 0 < Tn → ∞ as n → ∞, which are continuity i ∈ D, points for the distribution function G0,D,i j (·),

310

11 Asymptotics for expectations of hitting times for perturbed SMP

∫ lim Eε,D,kn j /uˇε,i = lim

ε→0

ε→0

∞

0

∫

≥ lim

ε→0

0

Tn

tdG ε,D,i j (t uˇε,i ) ∫ tdG ε,D,i j (t uˇε,i ) =

0

Tn

tdG0,D,i j (t).

(11.11)

By passing n → ∞ in the relation (11.11), we get the following relation: ∫ ∞ lim Eε,D,i j /uˇε,i ≥ E0,D,i j = tdG0,D,i j (t). (11.12) ε→0

0

We are going to show that lower limit can be replaced by the usual limit for normalised expectations Eε,D,i j /uˇε,i on the left hand side of the relation (11.12) and clarify conditions under which this limit coincides with the expectation E0,D,i j . We also show that, in some cases, Eε,D,i j /uˇε,i can converge to finite limits exceeding E0,D,i j , or converge to ∞, as ε → 0. We find normalisation functions u¯ε,i such that the normalised expectations Eε,D,i j /u¯ε,i → E¯ε,D,i j ∈ (0, ∞) as ε → 0 and show that these functions are connected with normalisation functions uˇε,i by asymptotic relations u¯ε,i /uˇε,i → Ci ∈ (0, ∞] as ε → 0. From the two above asymptotic relations it follows that Eε,D,i j /uˇε,i → Ci E¯0,D,i j ∈ (0, ∞] as ε → 0. Finally, we clarify conditions under which the limits Ci E¯0,D,i j = E0,D,i j . The following lemma takes place. Lemma 11.3 Let the conditions of Theorem 10.4 be satisfied for the semi-Markov processes ηε (t). Then: ∫∞ ¯ j ∈ D and given by (i) The expectation E0,D,i j = 0 tG0,D,i j (dt) < ∞, for i ∈ D, the relation (11.13) ∫∞  ¯ (ii) The expectation E0,D,i = j ∈D E0,D,i j = 0 tG0,D,i (dt) > 0, for i ∈ D. Proof The recurrent relations (10.127) for the Laplace transforms Ψ0,D,kn j (s) of the limiting distributions G0,D,kn j (t) let us (by computing the values of derivatives at zero for the corresponding Laplace transforms) get the following explicit recurrent ¯ . . . , 1, j ∈ D: relations for the expectations E0,D,kn j , for n = m, E0,D,kn j =

ˇ 0,kn,kn¯ k¯ n−1 f˜0,kn j k¯ n−1 p˜0,kn j k¯ n−1 w +

ˇ 0,kn,kn¯ k¯ n−1 f˜0,kn kn¯ k¯ n−1 w

P0,D,kn¯ j k¯ n−1 p˜0,kn kn¯

+ E0,D,kn¯ j k¯ n−1 p˜0,kn kn¯  + ˇ 0,kn,kn¯ k¯ n−1 f˜0,kn kl P0,D,kl j k¯ n−1 w

k¯ n−1 p˜0,k n kl

n+1≤l< n¯

+



ˇ 0,kl ,kn¯ k¯l−1 w

E0,D,kl j

k¯ n−1 p˜0,k n kl ,

(11.13)

n+1≤l< n¯

where the parameter n ≤ n¯ ≤ m¯ is defined by the relation (9.28). The corresponding expectations E0,D,kn j are finite or infinite. In the latter case, the product Eε,D,kn j k¯ n−1 p˜ε,kn r should be counted as 0, if the probability p˜ε,kn r = 0.

11.2 Asymptotics of expectations of hitting times

311

However, the statement (ii) of Theorem 10.1 implies that the condition k¯ m−1 K is ¯ satisfied. Thus, the relation (11.13) taken, for n = m, ¯ j ∈ D, gives the following relation: E0,D,km¯ j =

p˜0,km¯ j f˜0,km¯ j k¯ m−1 k¯ m−1 ¯ ¯

< ∞.

(11.14)

Then, the relations (11.13) taken, for n = m¯ − 1, j ∈ D and the relation (11.14) give the following relation: E0,D,km−1 j = ¯

wˇ 0,km−1 p˜0,km−1 f˜0,km−1 k m−2 j k¯ m−2 j ¯ k¯ m−2 ¯ ¯ ,k m ¯ ¯ ¯ ¯ +

wˇ 0,km−1 p˜0,km−1 f˜0,km−1 k m−2 ,k m¯ k¯ m−2 k m¯ P0,D,k m j k¯ m−2 k m¯ ¯ ¯ ¯ ¯ ¯ ¯

+ E0,D, km¯ j k¯ m−2 p˜0,km−1 k m¯ < ∞. ¯ ¯

(11.15)

Continuing the recurrent use of the relation (11.13), for n = m¯ − 2, . . . , 1, we prove that, for n = m, ¯ . . . , 1, j ∈ D, E0,D,kn j < ∞.

(11.16)

The relation (11.16) completes the proof of the statement (i). The statement (ii) is obviously implied by the statement (iv) of Theorem 10.4 ¯ according to which G0,D,i (0) < 1, for i ∈ D. 

11.2 Asymptotics of Expectations of Hitting Times In this section, we present theorems on the asymptotics of expectations of hitting ¯ times, in the case where the initial state belongs to domain D.

11.2.1 Asymptotics for Expectations of Hitting Times for the Case ¯ Where an Initial State Belongs to Domain D 11.2.1.1 The Case with the Most Absorbing Initial State. Let us consider the case, MD and the initial state is the most absorbing state k m . where the condition k¯ m−1 ¯ In this case, we can use the normalisation function, given in the relation (10.89), i.e., the same as in the corresponding weak convergence relation for hitting times given in Theorem 10.2, u¯ε,km¯ = uˇε,km¯ =

u˜ε,km¯ . k¯ m−1 ¯

(11.17)

The following theorem takes place. Theorem 11.1 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then:

312

11 Asymptotics for expectations of hitting times for perturbed SMP

(i) The following asymptotic relation takes place, for j ∈ D, Eε,D,km¯ j /u¯ε,km¯ → E¯0,D,km¯ j = E0,D,km¯ j = k¯ m−1 p˜0,km¯ j < ∞ as ε → 0. f˜0,km¯ j k¯ m−1 ¯ ¯

(11.18)

(ii) The normalisation function u¯ ·,km¯ ∈ (0, ∞) is given by the relation (11.17) and belongs to the family of asymptotically comparable functions H. (iii) The limit E¯0,D,km¯ j = E0,D,km¯ j is, for j ∈ D, the first moment for the limiting distribution G0,D,km¯ j (·) given in Theorem 9.2.  (iv) E¯0,D,km¯ = j ∈D E¯0,D,km¯ , j ∈ (0, ∞). Proof Lemma 10.14 and the relation (10.91), given in the proof of Theorem 10.2, imply, that the following relation takes place, for j ∈ D, Eε,D,km¯ j =

Eε,D,km¯ j k¯ m−1 ¯

= Ei k¯ n τ˜ε,D I( k¯ n η˜ε ( k¯ n τ˜ε,D = j) = Ei k¯ n κε,1 I(k¯ n ηε,1 = j) = ¯ f˜ε,km¯ j ¯ p˜ε,km¯ j . k m−1 ¯

(11.19)

k m−1 ¯

According to Theorem 10.1, the conditions thus, the following relation holds, for j ∈ D:

K k¯ m−1 ¯

are satisfied and,

= E0,D,km¯ j = E¯0,D,km¯ j < ∞ as ε → 0.

(11.20)

Eε,D,km¯ j = u¯ε,km¯ →

f˜ε,km¯ j k¯ m−1 ¯ u˜ε,km¯ k¯ m−1 ¯

J k¯ m−1 ¯

and

p˜ε,km¯ j k¯ m−1 ¯

p˜0,km¯ j f˜0,km¯ j k¯ m−1 k¯ m−1 ¯ ¯

The statements (i)–(iii) follow from the relation (11.20). The relation (11.20) also implies that the following relation holds:  k¯ f˜ε,km¯ j Eε,D,km¯ m−1 ¯ = ¯ ¯ p˜ε,k m¯ j u¯ε,km¯ u˜ε,km¯ km−1 ¯k m−1 j ∈D ¯  → f˜0,km¯ j ¯ p˜0,km¯ j ¯ j ∈D

k m−1 ¯

k m−1 ¯

= E0,D,km¯ = E¯0,D,km¯ < ∞ as ε → 0.

(11.21)

According to the statement (v) of Theorem 10.2, the distribution function G0,D,km¯ (·) is not concentrated at zero, and, thus, its first moment, E0,D,km¯ = E¯0,D,km¯ > 0, i.e., the statement (iv) is also true.

(11.22) 

11.2 Asymptotics of expectations of hitting times

313

11.2.1.2 The Case with the Second Most Absorbing Initial State. In what follows in this section, we assume that the conditions of Theorem 11.1 are satisfied. Let us consider the case, where the initial state is the second most absorbing state k m−1 . Let us define the following normalisation function: u¯ε,km−1 = ¯

u˜ε,km−1 ¯ k¯ m−2 ¯

+ u¯ε,km¯

=

u˜ε,km−1 ¯ k¯ m−2 ¯

+

p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

u˜ε,km¯ k¯ m−2 p˜ε,km−1 k m¯ . ¯ k¯ m−1 ¯ ¯

(11.23)

Obviously, u¯ε,km−1 ∈ (0, ∞), ε ∈ (0, 1]. ¯ ∈ H since the functions k¯ m−2 u˜ε,km−1 , u¯ε,km¯ , The normalisation function u¯ε,km−1 ¯ ¯ ¯ p ˜ belong to the family of asymptotically comparable functions H. and k¯ m−2 ε,k k m ¯ m−1 ¯ ¯ M entail the fulfilment of The conditions GD , HD , ID , ID,H , LD , LD,H , and k¯ m−1 ¯ D the following condition: ¯ D,1 : L

u¯ ε, k m¯

p˜ ε, k m−1 km k¯ m−2 ¯ ¯ ¯ u¯ ε, k m−1 ¯

→

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

∈ [0, 1] as ε → 0.

p˜ε,km−1 , and u¯ε,km−1 belong to the class Since the functions u¯ε,km¯ , k¯ m−2 k m¯ , u¯ ε,k m−1 ¯ ¯ ¯ ¯ of asymptotically comparable functions H, the limits appearing in the condition ¯ D,1 can be calculated using the operating formulas given in Lemma 8.2. In the L case, where the family H is one of the families, H1 , H2 , or H3 , the operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used. The following theorem takes place. Theorem 11.2 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for j ∈ D, Eε,D,km−1 → E¯0,D,km−1 j /u¯ ε,k m−1 j < ∞ as ε → 0. ¯ ¯ ¯

(11.24)

(ii) The normalisation function u¯ε,km−1 ∈ (0, ∞) is given by the relation (11.23) ¯ and belongs to the family of asymptotically comparable functions H. (iii) The limits E¯0,D,km−1 j , j ∈ D are given by the relation (11.25). ¯  ¯ (iv) E¯0,D,km−1 = E j ∈ (0, ∞). j ∈D 0,D,k m−1 ¯ ¯ Proof The relation (11.9), (11.18), and the condition LD,1 imply that the following relation takes place, for j ∈ D: Eε,D,km−1 j ¯ = u¯ε,km−1 ¯

u˜ε,km−1 f˜ε,km−1 j ¯ ¯ k¯ m−2 k¯ m−2 ¯ ¯ ¯ k¯ m¯ u¯ ε,k m−1

+

u˜ε,km−1 ¯ k¯ m−2 ¯

p˜ε,km−1 j ¯ k¯ m−2 ¯

u˜ε,km−1 f˜ε,km−1 k m¯ ¯ ¯ k¯ m−2 k¯ m−2 ¯ ¯ ¯ k¯ m¯ u¯ ε,k m−1

u˜ε,km−1 ¯ k¯ m−2 ¯

Pε,D,km¯ j k¯ m−2 p˜ε,km−1 k m¯ ¯ ¯

314

11 Asymptotics for expectations of hitting times for perturbed SMP

+

Eε,D,km¯ j k¯ m¯ u¯ ε,k m¯

→ (1 −

p˜ε,km−1 k m¯ ¯ k¯ m¯ u¯ ε,k m¯ k¯ m−2 ¯ ¯ k¯ m¯ u¯ ε,k m−1

¯ 0,km−1 p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ) k¯ m−2 ¯ ,k m ¯ ¯ k¯ m¯ w ¯ ¯

+ (1 −

¯ 0,km−1 p˜0,km−1 f˜0,km−1 k m¯ P0,D,k m¯ j k¯ m−2 k m¯ ¯ ) k¯ m−2 ¯ ,k m ¯ ¯ k¯ m¯ w ¯ ¯

¯ + k¯ m¯ w¯ 0,km−1 ¯ E0,D,k m ¯ j ¯ ,k m = E¯0,D,km−1 < ∞ as ε → 0. j ¯

(11.25)

The relation (11.25) also implies that the following relation holds:  Eε,D,k j Eε,D,km−1 m−1 ¯ ¯ = u¯ε,km−1 u ¯ ε,k m−1 ¯ ¯ j ∈D → (1 −

¯ 0,km−1 ¯ ) ¯ ,k m k¯ m¯ w

+ (1 − +

 j ∈D

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

¯ 0,km−1 p˜0,km−1 f˜0,km−1 k m¯ k¯ m−2 k m¯ ¯ ) k¯ m−2 ¯ ,k m ¯ ¯ k¯ m¯ w ¯ ¯

¯ 0 [k m−1 ¯ , km ¯ ] E¯0,D,k m¯ k¯ m¯ w

= (1 −

¯ 0,km−1 ¯ ) ¯ ,k m k¯ m¯ w



j ∈D

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

 + k¯ m−2 p˜0,km−1 f˜0,km−1 k m¯ k¯ m−2 k m¯ + ¯ ¯ ¯ ¯ = E¯0,D,km−1 < ∞ as ε → 0. ¯

¯ ¯ 0,km−1 ¯ E0,D,k m ¯ ¯ ,k m k¯ m¯ w (11.26)

Theorem 10.1 implies that the conditions k¯ m−1 J and k¯ m−1 K are satisfied and, thus, ¯ ¯ k¯ m¯

f˜ = ∧ j ∈D k¯ m−2 f˜0,km−1 j ∧ ¯ ¯

f˜0,km−1 k m¯ ¯ k¯ m−2 ¯

∈ (0, ∞).

(11.27)

Also, according to Theorem 10.1, E¯0,D,km¯ > 0. The above remarks and the relation (11.26) imply that the following inequality holds:  E¯0,D,km−1 = (1 − k¯ m¯ w¯ 0,km−1 p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ )( ¯ ¯ ,k m ¯ ¯ k¯ m−2 ¯ ¯ j ∈D

¯ + k¯ m−2 p˜0,km−1 ¯ 0,km−1 f˜0,km−1 k m¯ k¯ m−2 k m¯ ) + k¯ m¯ w ¯ E0,D,k m ¯ ¯ ¯ ¯ ,k m ¯ ¯  ≥ k¯ m¯ f˜ · (1 − k¯ m¯ w¯ 0 [k m−1 p˜0,km−1 p˜0,km−1 ¯ , km ¯ ])( j + k¯ m−2 k m¯ ) ¯ ¯ k¯ m−2 ¯ ¯ j ∈D

¯ + k¯ m¯ w¯ 0,km−1 ¯ E0,D,k m ¯ ¯ ,k m = k¯ m¯ f˜ · (1 − k¯ m¯ w¯ 0,km−1 ,k m ¯ ) + ¯ The proof is complete.

¯ ¯ 0,km−1 ¯ E0,D,k m ¯ ¯ ,k m k¯ m¯ w

> 0.

(11.28) 

¯ \ {k m }. 11.2.1.3 The Case with an Arbitrary Initial State from the Domain D Let us now consider the general case, with the initial state being k n , for some n = 1, . . . , m¯ − 1.

11.2 Asymptotics of expectations of hitting times

315

We can use, in this case, the relation (11.6) given in Lemma 11.1. This relation can be written in the following form, for j ∈ D and n = 1, . . . , m¯ − 1: Eε,D,kn j =

˜ k¯ n−1 fε,k n j k¯ n−1 p˜ε,k n j 

+

n+1≤l ≤ m ¯

( k¯ n−1 f˜ε,kn kl Pε,D,kl j

+ Eε,D,kl j ) k¯ n−1 p˜ε,kn kl .

(11.29)

Let us define recurrently the following normalisation functions:  u¯ε,kn = k¯ n−1 u ˜ + u¯ε,kl k¯ n−1 p˜ε,kn kl , n = 1, . . . , m¯ − 1. ε,k n ¯

(11.30)

n+1≤l ≤ m ¯

Obviously, k¯ m¯ u¯ε,kn ∈ (0, ∞), ε ∈ (0, 1], for n = 1, . . . , m¯ − 1. The normalisation function u¯ε,kn can be expressed in terms of the normalisation ¯ for n = 1, . . . , m. ¯ functions k¯l−1 u˜ε,kl , l = n, . . . , m, In particular, according to the relations (11.17) and (11.23), the normalisation u˜ε,km¯ and k¯ m¯ u¯ε,km−1 = k¯ m−2 u˜ε,km−1 + k¯ m−1 u˜ε,km¯ k¯ m−2 p˜ε,km−1 functions k¯ m¯ u¯ε,km¯ = k¯ m−1 k m¯ . ¯ ¯ ¯ ¯ ¯ ¯ ¯ Continuing the recurrent substitution in the relation (11.30), we get, ¯ k¯ m¯ u¯ ε,k m−2

=

u˜ε,km−2 ¯ k¯ m−3 ¯

+ (k¯ m−2 u˜ε,km−1 + ¯ ¯

u˜ε,km¯ k¯ m−2 p˜ε,km−1 p˜ε,km−2 k m¯ ) k¯ m−3 k m−1 ¯ ¯ ¯ k¯ m−1 ¯ ¯ ¯

+

u˜ε,km¯ k¯ m−3 p˜ε,km−2 k m¯ ¯ k¯ m−1 ¯ ¯

=

u˜ε,km−2 ¯ k¯ m−3 ¯

+

u˜ε,km−1 p˜ε,km−1 k m¯ ¯ ¯ k¯ m−2 k¯ m−2 ¯ ¯

+

u˜ε,km¯ (k¯ m−3 p˜ε,km−2 p˜ε,km−1 k m−1 k m¯ ¯ ¯ ¯ k¯ m−1 k¯ m−2 ¯ ¯ ¯

+

p˜ε,km−2 k m¯ ). ¯ k¯ m−3 ¯

(11.31)

Similar relations can be obtained for n = m¯ − 3, . . . , 1. The conditions GD , HD , ID , ID,H , LD , LD,H , and k¯ m−1 MD imply the fulfilment of ¯ the following condition: ¯ D,2 : L

u¯ ε, k l

˜ ε, k n k l k¯ n−1 p

→ u¯ ε, k n n = 1, . . . , m¯ − 1.

¯ 0,kn,kl k¯ m¯ w

∈ [0, 1] as ε → 0, for l = n + 1, . . . , m, ¯

Since the functions u¯ ·,kl , k¯ m−2 p˜ ·,kn kl , u¯ ·,km−1 , u¯ ·,kn , l = n+1, . . . , m, ¯ n = 1, . . . , m−1 ¯ ¯ ¯ belong to the class of asymptotically comparable functions H, the limits appear¯ D,2 can be calculated using the operating formulas given in ing in the condition L Lemma 8.2. In the case, where the family H is one of the families, H1 , H2 , or H3 , the operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used. The following theorem takes place. Theorem 11.3 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for j ∈ D, n = 1, . . . , m¯ − 1:

316

11 Asymptotics for expectations of hitting times for perturbed SMP

Eε,D,kn j /u¯ε,kn → E¯0,D,kn j < ∞ as ε → 0.

(11.32)

(ii) The normalisation functions u¯ ·,kn ∈ (0, ∞), n = 1, . . . , m¯ − 1 are given by the relation (11.30) and belong to the family of asymptotically comparable functions H. (iii) The limits E¯0,D,kn j , j ∈ D, n = 1, . . . , m¯ − 1 are given by the recurrent relation (11.35).  (iv) E¯0,D,kn = j ∈D E¯0,D,kn j ∈ (0, ∞), for n = 1, . . . , m¯ − 1. ¯ D,2 obviously implies that, for n = 1, . . . , m¯ − 1, Proof The condition k¯ m¯ L u˜ε,kn k¯ n−1 ¯

→

u¯ε,kn

¯ 0,kn,kn k¯ m¯ w

=1−

m ¯ 

¯ 0,kn,kl k¯ m¯ w

∈ [0, 1] as ε → 0.

(11.33)

l=n+1

The asymptotic relation (11.24) given in Theorem 11.2 can be considered as a result of the first step in some backward asymptotic recurrent algorithm for computing limits for expectations of hitting times Eε,D,kn j /u¯ε,kn , for n = m, ¯ . . . , 1, where the corresponding normalisation functions are given by the relations (11.23) and (11.30). Suppose we have already implemented m¯ − n steps in this backward algorithm, which gave the following asymptotic relations, for j ∈ D, l = m, ¯ . . . , n + 1, for some 1 ≤ n ≤ m¯ − 1: (11.34) Eε,D,kl , j /u¯ε,kl → E¯0,D,kl j < ∞ as ε → 0 with some limits E¯0,D,kl j < ∞, j ∈ D, l = m, ¯ . . . , n + 1. Then, using the relation (11.29), we obtain the following relations, for j ∈ D: u˜ε,kn k¯ n−1 f˜ε,kn j k¯ n−1 ¯

Eε,D,kn j /u¯ε,kn =

+

u¯ε,kn 

u˜ε,kn k¯ n−1 ¯

u˜ε,kn k¯ n−1 f˜ε,kn l k¯ n−1 ¯ u¯ε,kn

n+1≤l ≤ m ¯

+



u¯ε,kl

¯ 0,kn,kn k¯ m¯ w

+



n+1≤l ≤ m ¯

+



n+1≤l ≤ m ¯

u˜ε,kn k¯ n−1 ¯

k¯ n−1 p˜ε,k n kl

u¯ε,kn

n+1≤l ≤ m ¯

→

k¯ n−1 p˜ε,k n j

Pε,D,kl j

k¯ n−1 p˜ε,k n kl

Eε,D,kl j u¯ε,kl

( k¯ n−1 f˜0,kn j k¯ n−1 p˜0,kn j

˜ k¯ n−1 f0,k n l P0,D,kl j k¯ n−1 p˜0,k n l ) ¯ 0,kn,kl E¯0,D,kl j k¯ m¯ w

= E¯0,D,kn j < ∞ as ε → 0.

(11.35)

11.2 Asymptotics of expectations of hitting times

317

By induction, the relation (11.35) holds for any j ∈ D, n = m¯ − 1, . . . , 1. In addition, (11.35) gives the explicit recurrent formulas for calculating the limits  E¯0,D,kn j < ∞, j ∈ D, n = m¯ − 1, . . . , 1.

11.2.2 Conditions of Simultaneous Convergence for Distributions and Expectations of Hitting Times 11.2.2.1 Conditions of Simultaneous Convergence for Distributions and Expectations of Hitting Times. Theorem 11.2 implies that the expectations Eε,D,km¯ j , normalised by the function uˇε,km¯ (which is used in Theorem 11.2 as the normalisation function in the weak convergence relation (11.18) for the distributions G ε,D,km¯ j (· uˇε,km¯ )) converge to the first moment of the corresponding limiting distribution G0,D,km¯ j (·), as ε → 0. The question arises about conditions, under which expectations Eε,D,km−1 j , nor¯ , which is used in Theorem 11.3 as the normalisation malised by the function uˇε,km−1 ¯ function in the weak convergence relation for distributions G ε,D,km−1 ), conj (· uˇ ε,k m−1 ¯ ¯ verge to the first moment of the corresponding limiting distribution G0,D,km−1 j (·), as ¯ ε → 0. The following theorem gives the answer to this question. Theorem 11.4 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , and k¯ m−1 MD ¯ be satisfied for the semi-Markov processes ηε (t). Then: ˜ 1,km−1 (i) If one of the following is true: (a) k m¯ ∈ k¯ m−2 and k¯ m−2 p˜0,km−1 Y k m¯ > ¯ ¯ ¯ ¯ ˜ ˜ 0, or, (b) k m¯  k¯ m−2 , or (c) k ∈ , p ˜ = 0, and Y Y ¯ ¯ 1,k m ¯ 1,k 0,k k m ¯ m−1 ¯ m−1 ¯ m−1 ¯ k m−2 k m−2 ¯ ¯ ¯ ¯ 0,km−1 = 0, then the following asymptotic relation takes place, for j ∈ D: ¯k m¯ w ,k m ¯ ¯ Eε,D,km−1 → E0,D,km−1 j /uˇ ε,k m−1 j as ε → 0. ¯ ¯ ¯

(11.36)

˜ 1,km−1 , k¯ m−2 p˜0,km−1 Y (ii) If the following is true: (d) k m¯ ∈ k¯ m−2 k m¯ = 0, and ¯ ¯ ¯ ¯ ¯ 0,km−1 ¯ ∈ (0, 1), then the following asymptotic relation takes place, for j ∈ D: ¯ ,k m k¯ m¯ w → E¯0,D,km−1 Eε,D,km−1 j /uˇ ε,k m−1 j ¯ ¯ ¯ = E0,D,km−1 j + ¯

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w 1−

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

E0,D,km¯ j as ε → 0.

(11.37)

˜ 1,km−1 , k¯ m−2 p˜0,km−1 Y (iii) If the following is true: (e) k m¯ ∈ k¯ m−2 k m¯ = 0, and ¯ ¯ ¯ ¯ the following asymptotic relation takes place, for j ∈ D such

¯ 0,km−1 ¯ = 1, then ¯ ,k m k¯ m¯ w p ˜ that k¯ m−1 0,k m ¯ j > 0: ¯

Eε,D,km−1 → ∞ as ε → 0. j /uˇ ε,k m−1 ¯ ¯

(11.38)

318

11 Asymptotics for expectations of hitting times for perturbed SMP

Proof First, let us consider the case, where k m¯ ∈ > 0, i.e., the following relation holds: p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

˜ 1,km−1 Y ¯ k¯ m−2 ¯

and

p˜0,km−1 k m¯ ¯ k¯ m−2 ¯

> 0, for ε ∈ [0, 1].

(11.39)

In this case, the asymptotic relation appearing in the condition LD,1 holds, with the limit k¯ m¯ w¯ 0,km−1 ¯ ∈ (0, 1]. Indeed, ¯ ,k m u¯ε,km¯

p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯ u¯ε,km−1 ¯ (1 −  = 1+

 = 1+

 −1

u˜ε,km−1 ¯ k¯ m−2 ¯ u˜ε,km¯ k¯ m−2 p˜ε,km−1 k m¯ ¯ k¯ m−1 ¯ ¯

pε,km¯ km¯ ) k¯ m−2 u˜ε,km−1  −1 ¯ k¯ m−1 ¯ ¯

u˜ε,km¯ k¯ m−2 p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯ ¯

 → 1+ = =

(1 −

p0,km¯ km¯ ) k¯ m−2 w0,km−1 ¯  −1 ¯ ,k m k¯ m−1 ¯ ¯ p˜0,km−1 k m¯ ¯ k¯ m−2 ¯

¯ 0,km−1 k m¯ ¯ k¯ m¯ w

∈ (0, 1] as ε → 0.

(11.40)

In the case where the relation (11.39) holds, the normalisation function k¯ m¯ uˇε,km−1 ¯ is used in the weak convergence relation given in Theorem 9.3. Obviously,

u˜ε,km¯ k¯ m−1 ¯

u¯ε,km−1 u¯ε,km−1 ¯ ¯ = uˇε,km−1 u ˜ ¯ ε,k m ¯ ¯ k m−1 ¯ p˜ε,km−1 u¯ε,km−1 k m¯ ¯ ¯ k¯ m−2 ¯ = u ˜ p ˜ ε,k m¯ k¯ m−2 ε,k m−1 k m¯ ¯ k¯ m−1 ¯ ¯ p ˜ ¯k m−2 0,k k m ¯ m−1 ¯ → ¯ ∈ (0, ∞) as ε → 0. ¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

(11.41)

Also, the relation (11.40) implies that, 1−

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

=

(1 −

p0,km¯ km¯ ) k¯ m−2 w0,km−1 k m¯ ¯ k¯ m−1 ¯ ¯ p˜0,km−1 k m¯ ¯ k¯ m−2 ¯

.

(11.42)

Finally, the relations (11.15), (11.18), (11.25), (11.41), and (11.42) imply that, in the case where the relation (11.39) holds, the following relation takes place, for j ∈ D: Eε,D,km−1 Eε,D,km−1 j j u¯ ε,k m−1 ¯ ¯ ¯ = uˇε,km−1 u¯ε,km−1 uˇε,km−1 ¯ ¯ ¯ p ˜ ¯k m−2 0,k k m ¯ m−1 ¯ ¯ → E¯0,D,km−1 j ¯ ¯ 0,km−1 ¯k m¯ w ,k m ¯ ¯ = (1 −

p0,km¯ km¯ ) k¯ m−2 w0,km−1 p˜0,km−1 f˜0,km−1 k m¯ k¯ m−2 j k¯ m−2 j ¯ ¯ ¯ k¯ m−1 ¯ ¯ ¯ ¯

11.2 Asymptotics of expectations of hitting times

+ (1 −

319

p0,km¯ km¯ ) k¯ m−2 w0,km−1 f˜0,km−1 k m¯ k¯ m−2 k m¯ ¯ ¯ k¯ m−1 ¯ ¯ ¯

P0,D,km¯ j k¯ m−2 p˜0,km−1 k m¯ ¯ ¯

+ E0,D,km¯ j k¯ m−2 p˜0,km−1 k m¯ ¯ ¯ = k¯ m−2 wˇ 0,km−1 p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ k¯ m−2 ¯ ,k m ¯ ¯ ¯ ¯ ¯ ˜ + ¯ wˇ 0,km−1 P f k m¯ ¯ 0,k m−1 k m¯ 0,D,k m¯ j ¯ ¯ ¯ k m−2 ¯

p˜0,km−1 k m¯ ¯ k m−2 ¯

k m−2 ¯

+ E0,D,km¯ j k¯ m−2 p˜0,km−1 k m¯ ¯ ¯ = E0,D,km−1 j as ε → 0. ¯

(11.43)

Second, let us assume that state k m¯ 

˜ 1,km−1 , Y ¯ k¯ m−2 ¯

that is,

= 0, ε ∈ [0, 1].

p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

(11.44)

In this case, the normalisation function, u¯ε,km−1 = uˇε,km−1 = ¯ ¯

u˜ε,km−1 . ¯ k¯ m−2 ¯

(11.45)

The asymptotic relation appearing in the condition LD,1 obviously holds, with the limit, k¯ m¯ w¯ 0,km−1 ¯ = 0. ¯ ,k m The relation (11.25) takes, in this case, the following form, for j ∈ D: Eε,D,km−1 j ¯ = uˇε,km−1 ¯ →

f˜ε,km−1 j ¯ k¯ m−2 ¯ u˜ε,km−1 ¯ k¯ m−2 ¯

p˜ε,km−1 j ¯ k¯ m−2 ¯

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

= E¯0,D,km−1 j = E0,D,k m−1 j < ∞ as ε → 0. ¯ ¯ Third, let us assume that state k m¯ ∈ p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

˜ 1,km−1 Y ¯ k¯ m−2 ¯

(11.46)

and k¯ m−2 p˜0,km−1 k m¯ = 0, that is, ¯ ¯

> 0, ε ∈ (0, 1], while k¯ m−2 p˜0,km−1 k m¯ = 0. ¯ ¯

(11.47)

Note that, in this case, p˜ε,km−1 k m¯ ¯ k¯ m−2 ¯

→ 0 as ε → 0

(11.48)

and the normalisation function, uˇε,km−1 = ¯

u˜ε,km−1 . ¯ k¯ m−2 ¯

(11.49)

In this case, the asymptotic relation appearing in the condition LD,1 holds, with the limit k¯ m¯ w¯ 0,km−1 ¯ , which can take any value in interval [0, 1]. Three cases should ¯ ,k m be considered, when limit k¯ m¯ w¯ 0,km−1 ¯ equals 0, or takes value in interval (0, 1), or ¯ ,k m equals 1. (1) If k¯ m¯ w¯ 0,km−1 ¯ = 0, then the asymptotic relation (11.25) takes the following ¯ ,k m form, for j ∈ D:

320

11 Asymptotics for expectations of hitting times for perturbed SMP

Eε,D,km−1 j ¯ → E¯0,D,km−1 j ¯ u¯ε,km−1 ¯ = ¯ f˜0,km−1 j ¯

p˜0,km−1 j ¯ k¯ m−2 ¯

k m−2 ¯

In this case,

u¯ε,km−1 ¯ = uˇε,km−1 ¯

= E0,D,km−1 j < ∞ as ε → 0. ¯

u¯ε,km−1 ¯ → 1 as ε → 0. u ˜ ε,k m−1 ¯ k¯ m−2 ¯

(11.50)

(11.51)

The relations (11.50) and (11.51) imply that, for j ∈ D, Eε,D,km−1 j ¯ → uˇε,km−1 ¯

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

= E¯0,D,km−1 j = E0,D,k m−1 j < ∞ as ε → 0. ¯ ¯

(11.52)

(2) If k¯ m¯ w¯ 0,km−1 ¯ ∈ (0, 1), then the asymptotic relation (11.25) takes the follow¯ ,k m ing form, for j ∈ D: Eε,D,km−1 j ¯ → E¯0,D,km−1 j ¯ u¯ε,km−1 ¯ = (1 − +

¯ 0,km−1 p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ) k¯ m−2 ¯ ,k m ¯ ¯ k¯ m¯ w ¯ ¯

¯ ¯ 0,km−1 ¯ E0,D,k m ¯ j ¯ ,k m k¯ m¯ w

< ∞ as ε → 0.

(11.53)

In this case, u¯ε,km−1 ¯ = uˇε,km−1 ¯

¯ k¯ m¯ u¯ ε,k m−1

u˜ε,km−1 ¯ k¯ m−2 ¯

→ (1 −

¯ 0,km−1 ¯ ) ¯ ,k m k¯ m¯ w

−1

as ε → 0.

(11.54)

Therefore, the relation (11.53) implies that, for j ∈ D, Eε,D,km−1 j ¯ → uˇε,km−1 ¯

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

+

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w 1−

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

E0,D,km¯ j as ε → 0.

(11.55)

KD , the expectation k¯ m−2 According to the condition k¯ m−2 f˜0,km−1 j ∈ (0, ∞) and, ¯ ¯ ¯ ˜0,km¯ j ¯ p˜0,km¯ j ∈ (0, ∞) if and only if ¯ p˜0,km¯ j > 0. In this thus, E0,D,km¯ j = k¯ m−1 f k m−1 k m−1 ¯ ¯ ¯ case, p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯ >

+

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w 1−

¯ 0,km−1 ¯ ¯ ,k m k¯ m¯ w

p˜0,km−1 f˜0,km−1 j k¯ m−2 j ¯ ¯ k¯ m−2 ¯ ¯

E0,D,km¯ j

= E0,D,km−1 j. ¯

(11.56)

(3) If k¯ m¯ w¯ 0,km−1 ¯ = 1, then the asymptotic relation (11.25) takes the following ¯ ,k m form, for j ∈ D:

11.2 Asymptotics of expectations of hitting times

321

Eε,D,km−1 j ¯ → E¯0,D,km−1 j ¯ u¯ε,km−1 ¯ = E¯0,D,km¯ j = E0,D,km¯ j < ∞ as ε → 0. In this case,

u¯ε,km−1 ¯ = uˇε,km−1 ¯

u¯ε,km−1 ¯ → ∞ as ε → 0. u ˜ ε,k m−1 ¯ k¯ m−2 ¯

(11.57)

(11.58)

According to the condition k¯ m−2 KD , the expectation k¯ m−2 f˜0,km−1 j ∈ (0, ∞) ¯ ¯ ¯ ˜ ¯ p˜0,km¯ j ∈ (0, ∞) if and only if and, thus, E0,D,km¯ j = E0,D,km¯ j = k¯ m−1 f0,km¯ j k¯ m−1 ¯ ¯ p ˜ > 0. In this case, 0,k j m ¯ k¯ m−1 ¯ Eε,D,km−1 j ¯ → ∞ as ε → 0. uˇε,km−1 ¯

(11.59)

In the case, E0,D,km−1 p˜0,km−1 f˜0,km−1 j = k¯ m−2 j k¯ m−2 j < ∞, while the expectations ¯ ¯ ¯ ¯ ¯ normalised by the function u ˇ (the normalisation function used in the Eε,D,km−1 j ε,k m−1 ¯ ¯ (·) given in Theorem 10.3, in weak convergence relation for distributions G ε,D,km−1 j ¯ p ˜ = 0), converge to ∞ as ε → 0.  the case where the probability k¯ m−2 0,k m−1 k m¯ ¯ ¯ In conclusion, let us mention that, under the conditions of the statement (iii) of as ε → 0 Theorem 11.4, the limit of normalised expectations Eε,D,km−1 j /uˇ ε,k m−1 ¯ ¯ p ˜ = 0. Indeed, the relations (11.8) and (11.9) exists also for j ∈ D such that k¯ m−1 0,k m¯ j ¯ imply, in this case, that, Eε,D,km−1 j ¯ = uˇε,km−1 ¯

f˜ε,km−1 j ¯ k¯ m−2 ¯ u˜ε,km−1 ¯ k¯ m−2 ¯ + +

p˜ε,km−1 j ¯ k¯ m−2 ¯

f˜ε,km−1 k m¯ ¯ k¯ m−2 ¯ u˜ε,km−1 ¯ k¯ m−2 ¯

p˜ε,km¯ j k¯ m−2 p˜ε,km−1 k m¯ ¯ k¯ m−1 ¯ ¯

u˜ε,km¯ f˜ε,km¯ j k¯ m−1 k¯ m−1 ¯ ¯ u˜ε,km¯ k¯ m−2 u˜ε,km−1 ¯ k¯ m−1 ¯ ¯

p˜ε,km¯ j k¯ m−2 p˜ε,km−1 k m¯ . ¯ k¯ m−1 ¯ ¯

(11.60)

p˜ ·,km−1 p˜ ·,km−1 u˜ε,km¯ , and k¯ m−2 u˜ε,km−1 belong Since the functions k¯ m−2 j , k¯ m−2 k m¯ , k¯ m−1 ¯ ¯ ¯ ¯ ¯ ¯ ¯ to the class of asymptotically comparable functions H, there exist limits, lim

u˜ε,km¯ k¯ m−1 ¯

ε→0 k¯ u˜ε,km−1 ¯ m−2 ¯

p˜ε,km¯ j k¯ m−2 p˜ε,km−1 k m¯ ¯ k¯ m−1 ¯ ¯

=

¯ 0,km−1, ¯ m, ¯ j k¯ m¯ w

∈ [0, ∞].

(11.61)

The above limits can be calculated using the operating formulas given in Lemma 8.2. In the case, where the family H is one of the families, H1 , H2 , or H3 , the operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used. e˜ε,km¯ j / k¯ m−1 u˜ε,km¯ → k¯ m−1 Since, k¯ m−1 f˜0,km¯ j ∈ (0, ∞) as ε → 0, there also exist ¯ ¯ ¯ limits, Eε,D,km−1 j ¯ = k¯ m−1 (11.62) lim f˜0,km¯ j k¯ m¯ w¯ 0,km−1, ¯ m, ¯ j ∈ [0, ∞]. ¯ ε→0 uˇ ε,k m−1 ¯

322

11 Asymptotics for expectations of hitting times for perturbed SMP

A similar analysis regarding additional conditions, under which the expectations Eε,D,kn j normalised by function uˇε,kn (the normalisation function used in the weak convergence relation for the distributions G ε,D,kn j (·uˇε,kn ) given in Theorem 10.4) converge to the first moment of the corresponding limiting distribution G0,D,kn j (·), ¯ can be conducted for the general case with an arbitrary initial state in the domain D.

11.3 Asymptotics for Expectations of Return Times In this section, we present conditions of convergence for normalised expectations of hitting and return times for the case, where an state i ∈ D.

11.3.1 Asymptotic for Expectations of Hitting Times in the Case Where an Initial State Belongs to Domain D We assume that conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 MD and GD , ¯ HD , ID , ID,H , LD , LD,H , JD , KD are satisfied for the semi-Markov processes ηε (t). The key role is played by the following relation, which takes place, for r, j ∈ D and ε ∈ (0, 1]:  ( fε,r kn Pε,D,kn j + Eε,D,kn j )pε,r kn Eε,D,r j = fε,r j pε,r j + ¯ k n ∈D

= fε,r j pε,r j +



fε,r kn Pε,D,kn j pε,r kn +

¯ k n ∈D



Eε,D,kn j pε,r kn .

(11.63)

¯ k n ∈D

Note, first of all, that the above assumptions and relation (11.63) imply that Eε,D,r j < ∞, for r, j ∈ D and ε ∈ (0, 1]. Let us introduce the following normalising functions, for r ∈ D:  u¯ε,r = uε,r + u¯ε,kn pε,r kn . (11.64) ¯ k n ∈D

MD imply that the following condition The conditions G, HD , IH , LH , and k¯ m−1 ¯ is satisfied: LD,1 :

u¯ ε, k n uε, r

→

 0,kn,r k¯ m¯ w

¯ r ∈ D. ∈ [0, ∞] as ε → 0, for k n ∈ D,

¯ r ∈ D belong to the class of Since the functions u¯ ·,kn , uε,r , p ·,r kn , k n ∈ D, asymptotically comparable functions H, the limits appearing in the condition LD,1 can be computed with the use of the operating formulas given in Lemma 8.2. In the case, where the family H is one of the families H1 , H2 , or H3 , the operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used.

11.3 Asymptotics for expectations of return times

323

Note also that the normalisation functions u¯ ·,r , r ∈ D belong to the family of asymptotically comparable functions H. The following theorem takes place. Theorem 11.5 Let conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 MD and ¯ GD , HD , ID , ID,H , LD , LD,H , JD , KD are satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for r, j ∈ D: Eε,D,r j /u¯ε,r → E¯0,D,r j < ∞ as ε → 0.

(11.65)

(ii) The normalisation functions u¯ ·,r ∈ (0, ∞), r ∈ D are given by relation (11.64) and belong to the family of asymptotically comparable functions H. (iii) The limits E¯0,D,r j , r, j ∈ D are given by the recurrent relation (11.68).  (iv) E¯0,D,r = j ∈D E¯0,D,r j ∈ (0, ∞), for r ∈ D. Proof The condition k¯ m¯ LD implies that the following relations holds, for r ∈ D:  u¯ε,k pε,r k uε,r n n =1− u¯ε,r u ¯ ε,r ¯ k n ∈D   0,kn,r = →1− k¯ m¯ w

 0,r,r k¯ m¯ w

∈ [0, 1] as ε → 0.

(11.66)

¯ k n ∈D

Obviously, the following relation holds, for r ∈ D:   0,r,r +  0,kn,r = 1. k¯ m¯ w k¯ m¯ w

(11.67)

¯ k n ∈D

Theorems 10.1–10.3, the condition LD,1 , and the relations (11.63) and (11.66) imply that the following relation holds, for r, j ∈ D, Eε,D,r j /u¯ε,r =

 uε,r fε,r k uε,r fε,r j n pε,r j + Pε,D,kn j pε,r kn u¯ε,r uε,r u ¯ u ε,r ε,r ¯ k n ∈D

 u¯ε,k pε,r k Eε,D,k j n n n + u ¯ u ¯ ε,r ε,k n ¯ k n ∈D  → k¯ m¯ w 0,r,r ( f0,r j p0,r j + f0,r kn P0,D,kn j p0,r kn ) +



¯ k n ∈D

 0,kn,r E¯0,D,kn j k¯ m¯ w

¯ k n ∈D

= E¯0,D,r j as ε → 0. The relation (11.68) implies that the following relation holds, for r ∈ D:

(11.68)

324

11 Asymptotics for expectations of hitting times for perturbed SMP

Eε,D,r /u¯ε,r =



Eε,D,r j /u¯ε,r

j ∈D

→

 0,r,r ( f0,r j p0,r j k¯ m¯ w

+



f0,r kn p0,r kn )

¯ k n ∈D

 0,kn,r E¯0,D,kn k¯ m¯ w

¯ k n ∈D

=



+

 0,r,r f0,r k¯ m¯ w

+



 0,kn,r E¯0,D,kn k¯ m¯ w

¯ k n ∈D

= E¯0,D,r as ε → 0, where f0,r =



(11.69)

f0,ri p0,r j .

(11.70)

i ∈X

The conditions of Theorem 11.5 and Theorems 11.1–11.3 imply that, for r ∈ D, f¯ = f0,r ∧ min E¯0,D,kn > 0.

(11.71)

¯ k n ∈D

Finally, the relations (11.69) and (11.71) imply that the following inequality holds, for r ∈ D:   0,kn,r E¯0,D,kn E¯0,D,r = k¯ w 0,r,r f0,r + k¯ w m ¯

m ¯

¯ k n ∈D

≥ f¯ · (k¯ m¯ w 0,r,r +



 0,kn,r ) k¯ m¯ w

= f¯ > 0.

(11.72)

¯ k n ∈D



The proof is complete.

11.3.2 Asymptotics of Expectations for Return Times to Domain D The return times τε,D have been defined in Sect. 10.3.1, which also contains results concerning weak asymptotics for distributions of return times. Here, we present results concerning asymptotics for expectations of return times, Eε,D,i j = Ei τε,D, i, j ∈ D.

(11.73)

The relation (10.149) for the Laplace transforms of return times implies that the following relation holds for their expectations, for ε ∈ (0, 1] and r, j ∈ D,  Eε,D,r j = (Eε, D,ri (11.74) ¯ Pε,D,i j + Eε,D,i j Pε, D,ri ¯ ). ¯ i ∈D

Let us introduce the following normalising functions, for rl ∈ D:

11.3 Asymptotics for expectations of return times

uε,rl = u¯ε,rl +



325

u¯ε,kn Pε,D,rl kn .

(11.75)

¯ k n ∈D

MD and We assume that the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 ¯ GD , HD , ID , ID,H , LD , LD,H , JD , KD , r¯ m−1 MD are satisfied for the semi-Markov processes ηε (t). MD and GD , HD , ID , ID,H , LD , The conditions GD , HD , ID , ID,H , LD , LD,H , k¯ m−1 ¯ LD,H , r¯ m−1 MD imply the fulfilment of following condition: LD,2 :

u¯ ε, k n u¯ ε, rl

→

 0,kn,rl k¯ m¯ , r¯m w

¯ ∈ [0, ∞] as ε → 0, for rl ∈ D, k n ∈ D.

¯ rl ∈ D belong to the class of asymptotically Since the functions u¯ ·,kn , u¯ ·,rl , k n ∈ D, comparable functions H, the limits appearing in the condition LD,2 can be calculated using the operating formulas given in Lemma 8.2. In the case, where the family H is one of the families, H1 , H2 , or H3 , the operating formulas given in Lemma 8.4, 8.6, or 8.8 should be used. Note also that the normalisation functions u ·,rl , rl ∈ D belong to the family of asymptotically comparable functions H. The following theorem takes place. Theorem 11.6 Let the conditions GD , HD , ID , ID,H , LD , LD,H , JD , KD , k¯ m−1 MD ¯ and GD , HD , ID , ID,H , LD , LD,H , JD , KD , r¯ m−1 MD be satisfied for the semi-Markov processes ηε (t). Then: (i) The following asymptotic relation takes place, for rl, j ∈ D, Eε,D,rl j /uε,rl → E0,D,rl j < ∞ as ε → 0.

(11.76)

(ii) The normalisation functions u ·,rl ∈ (0, ∞), rl ∈ D are given by relation (11.75) and belong to the family of asymptotically comparable functions H. (iii) The limits E0,D,r j , r, j ∈ D are given by the recurrent relation (11.79).  (iv) E0,D,rl = j ∈D E0,D,rl j ∈ (0, ∞), for rl ∈ D. Proof The condition LD,3 implies that the following relation holds, for rl ∈ D:  u¯ε,k Pε,D,r k u¯ε,rl n l n =1−  uε,rl u ε,r l ¯ k n ∈D   0,kn,rl →1− k¯ m¯ , r¯m w ¯ k n ∈D

=

 0,rl ,rl k¯ m¯ , r¯m w

∈ [0, 1] as ε → 0.

(11.77)

Obviously, the following relation holds, for rl ∈ D:    0,kn,rl = 1. w + ¯ 0,r ,r l l k¯ m¯ , r¯m w k m¯ , r¯m

(11.78)

¯ k n ∈D

326

11 Asymptotics for expectations of hitting times for perturbed SMP

Theorems 11.1–11.3, the condition LD,2 , and the relations (10.149) and (11.77) imply that the following relation holds, for rl, j ∈ D: Eε,D,rl j /uε,rl =

 u¯ε,r Eε, D,r ¯ l kn l Pε,D,kn j u u¯ε,rl ¯ ε,rl

k n ∈D

 u¯ε,kn Pε, D,r ¯ l k n Eε,D,k n j uε,rl u¯ε,kn ¯ k n ∈D  → k¯ m¯ , r¯m w 0,rl ,rl E¯0, D,r ¯ l k n P0,D,k n j +

+



¯ k n ∈D

 0,kn,rl E0,D,kn j k¯ m¯ , r¯m w ¯

¯ k n ∈D

= E0,D,rl j as ε → 0.

(11.79)

The relation (11.79) implies that the following relation holds, for rl ∈ D:  Eε,D,rl /uε,rl = Eε,D,rl j /uε,rl j ∈D

→



 0,rl ,rl k¯ m¯ , r¯m w

+



E¯0, D,r ¯ l kn

¯ k n ∈D

 0,kn,rl E0,D,kn k¯ m¯ , r¯m w ¯

¯ k n ∈D

=

 0,rl ,rl E0, D,r ¯ l k¯ m¯ , r¯m w ¯

+



 0,kn,rl E¯0,D,kn k¯ m¯ w

¯ k n ∈D

= E0,D,rl as ε → 0.

(11.80)

The conditions of Theorem 11.6 and Theorems 11.1–11.3 imply that, for rl ∈ D, f = E¯0, D,r ¯ l ∧ min E¯0,D,k n > 0.

(11.81)

¯ k n ∈D

Finally, the relations (11.69) and (11.71) imply that the following inequality holds, for r ∈ D:   0,kn,rl E¯0,D,kn E¯0,D,r = k¯ , r¯ w 0,rl ,rl E¯0, D,r + ¯ k¯ , r¯ w m ¯

m 

l

m ¯

m 

¯ k n ∈D

≥ f · ( k¯ m¯ , r¯m w 0,r,r +



 0,kn,r ) k¯ m¯ , r¯m w

= f > 0.

(11.82)

¯ k n ∈D

The proof is complete.



Chapter 12

Generalisations and Examples of Limit Theorems for Hitting Times

In this chapter, we discuss some natural generalisations of the asymptotic results presented in Chaps. 8–11. We also describe the features of the asymptotic recurrent algorithms of phase space reduction for perturbed semi-Markov processes of the birth–death type. Finally, we give numerical examples illustrating the above algorithms and their applications to the asymptotic analysis of distributions and expectations of hitting times for singularly perturbed semi-Markov processes. This chapter includes three sections. In Sect. 12.1, we discuss some generalisations of the perturbation conditions and asymptotic results for hitting times, in particular, the possibilities of obtaining analogous asymptotic results for vector- and real-valued hitting type functionals and obtaining some more general asymptotic results for perturbed semi-Markov processes with distributions of transition times belonging to domains of attraction for infinitely divisible laws. In Sect. 12.2, the specific features of the asymptotic recurrent algorithms of phase space reductions for perturbed birth–death-type semi-Markov processes are commented. In this case, these algorithms preserve the structure of birth–death of reduced semi-Markov processes. This greatly simplifies the asymptotic analysis of hitting time. In Sect. 12.3, we present some numerical examples illustrating particularities of the asymptotic recurrent algorithms of phase space reduction for distributions and expectations of hitting times for singularly perturbed semi-Markov processes.

12.1 Generalisations of Limit Theorems for Hitting Times In this section, we give some additional comments and discuss some natural generalisations of the asymptotic results presented in Chaps. 8–11.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4_12

327

328

12 Generalisations of limit theorems for hitting times

12.1.1 More General Perturbation Conditions 12.1.1.1. Regularity Conditions and Atoms at Zero for Limiting Distributions. Let us begin from the remark that the regularity condition GD (b) can be, in fact, omitted. In this case, it is possible that the random variable νε (t) can be improper random variables, i.e., take value ∞ with positive probabilities, and thus, the semi-Markov process ηε (t) is not well defined. However, the condition H still guarantees the fulfilment of the relation (10.29), i.e., that the hitting times νε,D and τε,D are proper random variables. The random variable τε,D can be interpreted as the Markov reward accumulated on the trajectory of the Markov chain ηε,n until the first hitting of the Markov chain ηε,n in the domain D. The recurrent algorithms of phase space reduction presented in Chap. 9 and the asymptotic results for distributions and expectations of hitting times formulated in Chaps. 10 and 11 remain to be valid. Another related generalisation is concerned the condition JD (b), which requires that the corresponding limiting distributions for transition times are not concentrated at zero. This condition, together with other basic conditions, guarantees that the corresponding limiting distribution functions of hitting times k¯ n G0,D,i (·) appearing in the theorems presented in Chap. 10 are also not concentrated at zero. The recurrent algorithms of phase space reduction presented in Chap. 9 and the asymptotic results formulated in Chaps. 10 and 11 remain to be valid without the condition JD (b). The only difference is that, in this case, there is no guarantee that the corresponding limiting distribution functions of hitting times k¯ n G0,D,i (·) are not concentrated at zero. An important example is connected with the functional, which can be defined for ¯ any domain C ⊆ D, ν ε,D κε,n I(ηε,n−1 ∈ C), (12.1) τε,C,D = n=1

where νε,D = min(n ≥ 1 : ηε,n ∈ D).

(12.2)

The random variable τε,C,D can be interpreted as the accumulated reward in states from the domain C on the trajectory of the Markov chain ηε,n until the first hitting of this Markov chain in the domain D. In this case, one can consider the new Markov renewal process (ηε,C,n , κε,C,n , where ηε,C,n = ηε,n and κε,C,n = κε,n I(ηε,n−1 ∈ C). Obviously, the basic conditions HD , IH,D , LH,D (and, thus, also ID and LD ), JD , and KD remain to be satisfied for the new Markov renewal processes, with the only change that the distribution functions ¯ j ∈ X and ε ∈ (0, 1]. Fε,i j (t) = I(t ≥ 0), t ≥ 0 and expectations fε,i j = 0, for i ∈ C, ¯ In this case, the One can choose the initial normalisation functions uε,i ≡ 1, for i ∈ C. limiting distribution function F0,i j (t) = I(t ≥ 0), t ≥ 0 and the expectation f0,i j = 0, ¯ j ∈ X. for i ∈ C,

12.1 Generalisations

329

12.1.1.2 More General Perturbation Conditions. The question arises whether the use of the same normalisation function uε,i in the asymptotic relations appearing in the condition J for the distribution functions of transition times Fε,i j (·uε,i ) is a restrictive assumption. In fact, the original model of the Markov renewal process (ηε,n, κε,n ), n = 0, 1, . . ., with the phase space X and the transition probabilities Q ε,i j (t) = pε,i j Fε,i j (t), t ≥ 0, i, j ∈ X can be replaced by its modification (ηˆε,n = (ηε,n, ηε,n+1 ), κε,n ), n = 0, 1, . . .. The modified Markov renewal process (ηˆε,n, κε,n ) has the phase ˆ × [0, ∞), where X ˆ = {(i, j) : i ∈ X, j ∈ Y1,i }, and the transition probabilities space X ˆ Q ε,(i, j)(k,r) (t) = pε,(i, j)(k,r) Fε,(i, j)(k,r) (t) = I(k = j)pε, jr Fε,i j (t), t ≥ 0, (i, j), (k, r) ∈ ˆ In this case, the distribution function Fε,(i, j)(k,r) (t) = Fε,i j (t) depends on the state X. (i, j) but does not depend on the state (k, r). The hitting times νε,D = min(n ≥ 1 : ηε,n ∈ D) and τε,D for the original Markov renewal process (ηε,n, κε,n ) coincide with the hitting times, respectively, νˆε, Dˆ = ˆ (here, D ˆ = {(i, j) : i ∈ D, j ∈ Y1,i }) and τˆ ˆ = νˆ ε, Dˆ κε,n for min(n ≥ 1 : ηˆε,n ∈ D) ε, D n=1 the modified Markov renewal process (ηˆε,n, κε,n ). The conditions GD (a), HD , and ID , IH,D , assumed to be satisfied for the original Markov renewal processes (ηε,n, κε,n ), are also satisfied for the modified Markov renewal processes (ηˆε,n, κε,n ). As far as the conditions JD and KD are concerned, they will take the following modified forms: Jˆ D : (a) Fε,i j (· uε,i j ) ⇒ F0,i j (·) as ε → 0, for j ∈ Y1,i, i ∈ X, (b) F0,i j (·), j ∈ Y1,i, i ∈ X are proper distribution functions such that F0,i j (0) < 1, j ∈ Y1,i, i ∈ X, (c) ¯ uε,i j ∈ (0, ∞), ε ∈ (0, 1], for j ∈ Y1,i, i ∈ D. and

∫ ˆ D : (a) fε,i j = ∞ tFε,i j (dt) < ∞, for j ∈ Y1,i, i ∈ X, (b) fε,i j /uε,i j → f0,i j = K 0 ∫∞ ¯ tF0,i j (dt) < ∞ as ε → 0, for j ∈ Y1,i, i ∈ D. 0 Also, the conditions L and LH take the following modified forms: ¯ ˆ D : uε,i j → u0,i j ∈ (0, ∞] as ε → 0, for j ∈ Y1,i, i ∈ D L and ¯ belong to the complete family of asymptoˆ H,D :The functions u ·,i j , j ∈ Y1,i, i ∈ D L tically comparable functions H appearing in the condition IH,D . The asymptotic recurrent algorithms of phase space reduction can be applied to the modified Markov renewal process (ηˆε,n, κε,n ). The corresponding asymptotic results will generalise asymptotic results presented in Chaps. 10 and 11 to the case, ˆ D, L ˆ H,D take the above more general forms. where conditions JD , KD , and L Moreover, in this case, the corresponding asymptotic relations can be obtained for the distributions Gˆ ε,D,i(j,r) (t) = Pi { τˆε, Dˆ ≤ t, ηˆνˆ ε, Dˆ = ( j, r)}, which are more general than the distributions G ε,D,i j (t). 12.1.1.3 Perturbation Conditions Working for an Arbitrary Domain D. It is easy to see that the conditions G, H, IH , LH (and, thus, also I and L), J, and K

330

12 Generalisations of limit theorems for hitting times

entail the fulfilment of the conditions GD , HD , IH,D , LH,D (and, thus, also ID and LD ), JD and KD as well as the conditions GD , HD , IH,D , LH,D (and, thus, also ID and LD ), JD , and KD for any non-empty domain D ⊂ X.

12.1.2 More General Hitting Reward Functionals 12.1.2.1 Vector Hitting Rewards. In this case, we consider Markov renewal processes (ηε,n, κ¯ε,n ) = (ηε,n, (κε,1,n, . . . , κε, L,n )), n = 0, 1, . . ., with a phase space X × R+L , where X = {1, . . . , M } is a finite set and R+L = [0, ∞) × · · · × [0, ∞) is the L-product of the interval [0, ∞). The corresponding vector hitting reward functional is defined in the following way, for D ⊆ X, ν ε,D κ¯ε,n = (τε,1,D, . . . , τε,l,D ), (12.3) τ¯ε,D = n=1

where τε,l,D =

ν ε,D

κε,l,n, l = 1, . . . , L.

(12.4)

n=1

In this case, one can reduce the problem to scalar case by using the method known as the Wold–Cramér device (see, for example, Silvestrov (2004) or Kallenberg (2021)). Let us introduce the following scalar hitting reward functional, for z¯ = (z1, . . . , zl ) ∈ R+L , ν ε,D κε, z,n (12.5) τε, z,D ¯ = ¯ , n=1

where κε, z,n ¯ =

L 

zl κε,l,n, n ≥ 1.

(12.6)

l=1 L

The L-dimensional Laplace transforms φε,i j ( z¯) = E{e− l=1 zl κε, l,1 /ηε,0 = i, ηε,1 = −sκε, z,1 ¯ /η j} and the scalar Laplace transforms φε, z,i ¯ j (s) = E {e ε,0 = i, ηε,1 = j) are + connected by the relation, φε, z,i ¯ j (s) = φε,i j (s z¯), for s ≥ 0, z¯ ∈ R L . This property of Laplace transforms for transition times is invariant with respect to procedures of removing virtual transitions and one-step reduction of phase space. This makes it possible to prove that the Laplace transforms of hitting times also have this property. This means that the L-dimensional Laplace transforms Ψε,D,i j ( z¯) L = Ei e− l=1 zl τε, l,D I(ηε,νε,D = j) and the scalar Laplace transforms Ψε, z,D,i ¯ j (s) = ¯ I(η Ei e−sτε, z,D = j) are also connected by the relation Ψ (s) = Ψ ε,νε,D ε, z,D,i ¯ j ε,D,i j (s z¯), for s ≥ 0, z¯ ∈ R+L . The condition J◦ (a) takes the form of convergence relations for the L-dimensional Laplace transforms, φε,i j ( z¯/uε,i ), assumed to hold for z¯ ∈ R+L . According to

12.1 Generalisations

331

the Wold–Cramér device, this condition can be reformulated in the equivalent form of the corresponding convergence relations for the scalar Laplace transforms, + φε, z,i ¯ j (s/uε,i ), assumed to hold for s ≥ 0, z¯ ∈ R L . Similar remarks can be made with regard to expectations of transition and hitting times. This makes it possible to apply results presented in Chaps. 8–10 and to get the corresponding convergence relations for the Laplace transforms Ψε, z,D,i ¯ j (s/uˇ ε,i ), s ≥ 0, for z¯ ∈ R+L , and then, using the Wold–Cramér device, rewrite them in the equivalent form of the corresponding convergence relations for the L-dimensional Laplace transforms Ψε,D,i j ( z¯/uˇε,i ), z¯ ∈ R+L and the corresponding weak convergence relations for the L-dimensional distributions of hitting times G ε,D,i j (t¯ uˇε,i ), t¯ = (t1, . . . , t L ) ∈ R+L . 12.1.2.2 Real-Valued Hitting Rewards. Another natural generalisation is concerned the model with real-valued rewards. In this case, we consider Markov renewal processes (ηε,n, κε,n ), n = 0, 1, . . . with a phase space X × R1 , where X = {1, . . . , m} is a finite set, and R1 = (−∞, ∞) is a real line. The corresponding real-valued hitting reward functional can be defined in the standard way and represented as the difference of two non-negative hitting reward functionals, for D ⊆ X, ν ε,D

+ − κε,n = τε,D − τε,D ,

(12.7)

± ± κε,n and κε,n = ±κε,n I(±κε,n ≥ 0), n ≥ 1.

(12.8)

τε,D =

n=1

where ± τε,D =

ν ε,D n=1

+ , τ − ) and apWe can consider the vector hitting reward functional τ¯ε,D = (τε,D ε,D ply to this vector hitting reward functional the Wold–Cramér device, as described in Sect. 12.1.2.1, for obtaining weak convergence relations for the distributions + + −1 − ¯ G ε,D,i j (t¯ uˇε,i ) = Pi {uˇ−1 ε,i τε,D ≤ t+, uˇ ε,i τε,D ≤ t−, ηε,νε,D = j}, t = (t+, t− ) ∈ R2 . Such relations imply, in an obvious way, the corresponding weak convergence rela+ − τ− . tions for the real-valued hitting reward functionals τε,D = τε,D ε,D In order to escape some possible side effects, which can be caused by discontinuity of distribution functions of transition times in point 0, one can also use the more general splitting procedure for hitting times, which is based on random variables ± = ±(κε,n − c)I(±κε,n ≥ ±c), n ≥ 1, for some c ≥ 0. κε,c,n 12.1.2.3 General Reward Functionals. Generalisations concerned vector hitting reward functionals (presented in Sect. 12.1.2.1) make it possible to obtain weak asymptotic relations for hitting frequencies. In this case, we consider the Markov renewal process (ηε,n, κ¯ε,n ) = (ηε,n, (I(ηε,n−1 = k, ηε,n = r), k, r ∈ X), n = 0, 1, . . . with the phase space X × R+L , where L = M 2 . The corresponding vector hitting reward functional is defined in the following way,

332

12 Generalisations of limit theorems for hitting times

τ¯ε,D =

ν ε,D

κ¯ε,n = (τε,kr,D, k, r ∈ X),

(12.9)

n=1

where τε,kr,D =

ν ε,D

I(ηε,n−1 = k, ηε,n = r).

(12.10)

n=1

In this case, the random variable τε,kr,D is the number of transitions of the form k → r made by the Markov chain ηε,n before hitting to domain D. Let us now consider the Markov renewal process (ηε,n, γε,n ), n = 0, 1, . . ., with a phase space X × R1 and define the hitting reward functional, τε,D =

ν ε,D

γε,n .

(12.11)

n=1

The following relation, based on conditional independence of random variables γε,n, n = 0, 1, . . ., with respect to the Markov chain ηε,n, n = 0, 1, . . ., represents the above functional in the form of a random sum, k r,D  τε,

d

τε,D =

γε,kr,n,

(12.12)

k,r ∈X n=1

where: (a) the random vector τ¯ε,D = (τε,kr,D , k, r ∈ X) and the random variables γε,kr,n , n ≥ 1, k, r ∈ X are mutually independent, (b) the random variable γε,kr,n has the distribution function Fε,kr (·) = P{γε,kr,n ≤ t} = P{γε,1 ≤ ·/ηε,0 = k, ηε,1 = r }, for n ≥ 1 and k, r ∈ X. The representation (12.12) let us formulate some natural conditions of weak convergence for the hitting reward functionals τε,D using the weak convergence theorems for randomly stopped stochastic processes given in Appendix A. Let, for example: (c) ηε,0 = i ∈ X, and the following asymptotic relation takes d

place τ¯ε,D /uˇε,i −→ τ¯0,i,D as ε → 0, where τ¯0,i,D = (τ0,i,kr,D, k, r ∈ X) is a random vector with non-negative components, (d) uˇε,i > 0 is some normalisation function  d such that uˇε,i → ∞ as ε → 0, and (e) 1≤n ≤t uˇ ε, i γε,kr,n /uˆε,i, t ≥ 0 −→ γ0,kr (t), t ≥ 0 as ε → 0, for every k, r ∈ X, where γ0,i,kr (t), t ≥ 0 is a Lévy process, (f) uˆε,i > 0 is some normalisation function such that uˆε,i → ∞ as ε → 0. Then, relation (12.12) and Theorem A.4 imply that the following weak convergence relation takes place: d

τε,D /uˆε,i −→ τ0,D =



γ0,i,kr (τ0,i,kr,D ) as ε → 0,

(12.13)

k,r ∈X

where: (g) the random vector τ¯0,i,D and the Lévy processes γ0,i,kr (·), k, r ∈ X are mutually independent. The problem that should be commented on is caused by the fact that the normalisation functions uˇε,i , appearing in the above asymptotic relations (e), may differ for

12.2 Perturbed birth-death-type SMP

333

different i ∈ X. This imposes some additional restrictions on distribution functions Fε,kr (·), k, r ∈ X. The conditions J and K give a hint about one version of the model, in which the asymptotic relations of type (e) hold for any normalisation functions 0 < uˇε,i → ∞ and uˆε,i = uˇε,i , with the corresponding limiting processes γ0,i,kr (t) = f0,kr t, t ≥ 0, which are nonrandom linear functions. This is the case, where it is assumed that the distribution functions Fˇε,kr (·) = Fε,kr (·uˇε,i ) converge weakly to some limiting distribution function F0,kr (·) and the first moments fˇε,kr of the distribution functions Fˇε,kr (·) converge to the first moment f0,kr < ∞ of the distribution function F0,kr (·), as ε → 0, for k, r ∈ X. Another example is where the random variables γε,kr,n = γ0,kr,n do not depend on the perturbation parameter ε and their distribution functions Fε,kr (·) = F0,kr (·) (which, in this case, also do not depend on ε) belong to the domain of attraction of some stable law. This means that the asymptotic relations of type (e) hold for α h(uˇ ) (here, α ∈ (0, 1] any normalisation functions 0 < uˇε,i → ∞ and uˆε,i = uˇε,i ε,i and h(·) is some slowly varying function), with some limiting stable Lévy processes γ0,kr (·), k, r ∈ X (which, in this case, do not depend on i).

12.2 Hitting Times for Perturbed Birth–Death-Type Semi-Markov Processes In this section, we describe procedures of removing virtual transition and phase space reduction for birth–death-type semi-Markov processes.

12.2.1 Perturbed Birth–Death-Type Semi-Markov Processes 12.2.1.1 Perturbation Conditions for Birth–Death-Type Semi-Markov Processes. Let X = {0, . . . , m} be a finite set. Let also, (ηε,n, κε,n ), n = 0, 1, . . . be, for every ε ∈ (0, 1] a birth–death-type Markov renewal process, i.e., a homogeneous Markov chain, with a phase space X × [0, ∞) and transition probabilities, for (i, s), ( j, t) ∈ X × [0, ∞), ⎧ Fε,00 (t)pε,00 ⎪ ⎪ ⎪ ⎪ F ⎪ ε,01 (t)pε,01 ⎪ ⎪ ⎪ F ⎪ ε,ii+1 (t)pε,ii+1 ⎪ ⎪ ⎨ Fε,ii (t)pε,ii ⎪ Q ε,i j (t) = Fε,ii−1 (t)pε,ii−1 ⎪ ⎪ ⎪ ⎪ F ⎪ ε,mm (t)pε,mm ⎪ ⎪ ⎪ F ⎪ ε,mm−1 (t)pε,mm−1 ⎪ ⎪ ⎪ 0 ⎩

for i = 0, j = 0, for i = 0, j = 1, for 0 < i < m, j = i + 1, for 0 < i < m, j = i, for 0 < i < m, j = i − 1, for i = m, j = m, for i = m, j = m − 1, otherwise,

(12.14)

334

12 Generalisations of limit theorems for hitting times

where: (a) Fε,i j (·) are distribution functions concentrated on [0, ∞), for 0 ≤ i, j ≤ m, | j − i| ≤ 1, (b) pε,i j ≥ 0, for 0 ≤ i, j ≤ m, | j − i| ≤ 1 and pε,00 + pε,01 = 1, pε,ii−1 + pε,ii + pε,ii+1 = 1, for 0 < i < m, pε,mm−1 + pε,mm = 1. The first component of the above Markov renewal process, ηε,n, n = 0, 1, . . ., is itself a homogeneous, so-called embedded, birth–death-type Markov chain, with the phase space X and the transition probabilities, for i, j ∈ X,

for 0 ≤ i, j ≤ m, | j − i| ≤ 1, p (12.15) Q ε,i j (∞) = ε,i j 0 otherwise. The above Markov renewal process is used to define the birth–death-type semiMarkov process, (12.16) ηε (t) = ηε,νε (t), t ≥ 0, where ζε,n = κε,1 + · · · + κε,n, n = 1, 2, . . . , ζε,0 = 0, are the corresponding moments of jumps, and νε (t) = max(n ≥ 1 : ζε,n ≤ t) is the number of jumps in the interval [0, t] for the above semi-Markov process. The conditions G–L and IH , LH take, in this case, forms presented below. Let us recall the notation introduced in Sect. 9.1. The distribution functions of sojourn times take the following form, for i ∈ X and ε ∈ (0, 1], and t ≥ 0, Fε,i (t) = Pi {κε,1 ≤ t} ⎧ if i = 0, ⎪ ⎨ Fε,00 (t)pε,00 + Fε,01 (t)pε,01 ⎪ = Fε,ii−1 (t)pε,ii−1 + Fε,ii (t)pε,ii + Fε,ii+1 (t)pε,ii+1 if 0 < i < m, ⎪ ⎪ Fε,mm (t)pε,mm + Fε,mn−1 (t)pε,mm−1 if i = m ⎩

(12.17)

The conditions G and H take the following forms: ˇ (a) pε,i j > 0, ε ∈ (0, 1] or pε,i j = 0, ε ∈ (0, 1], for 0 ≤ i, j ≤ m, | j − i| ≤ 1, (b) G: Fε,i (0) < 1, 0 ≤ i ≤ m, ε ∈ (0, 1] and ˇ p1,ii+1 > 0, for i = 0, . . . , m − 1 and p1,ii−1 > 0, for i = 1, . . . , m. H: ˇ implies that sets Y1,i = Condition H forms: ⎧ {1} ⎪ ⎪ ⎪ ⎪ {0, 1} ⎪ ⎪ ⎪ ⎨ {i − 1, i + 1} ⎪ Y1,i = {i − 1, i, i + 1} ⎪ ⎪ ⎪ ⎪ {m − 1} ⎪ ⎪ ⎪ ⎪ {m − 1, m} ⎩

{i ∈ X : p1,i j > 0}, i ∈ X take the following for i = 0, if p1,00 = 0, for i = 0, if p1,00 > 0, for 0 < i < m, if p1,ii = 0, for 0 < i < m, if p1,ii > 0, for i = m, if p1,mm = 0, for i = m, if p1,mm > 0.

The perturbation conditions I and IˇH take the following forms: ˇ pε,i j → p0,i j as ε → 0, for i, j ∈ X. I:

(12.18)

12.2 Perturbed birth-death-type SMP

335

IˇH : The functions p ·,i j , j ∈ Y1,i, i ∈ X belong to a complete family of asymptotically comparable functions H. The perturbation condition J takes the following form: ˇ (a) Fε,i j (· uε,i ) ⇒ F0,i j (·) as ε → 0, for j ∈ Y1,i, i ∈ X, (b) F0,i j (·) is a proper J: distribution function such that F0,i j (0) < 1, for j ∈ Y1,i, i ∈ X, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X. Let us recall the corresponding Laplace transforms, for i, j ∈ X, ∫ ∞ φε,i j (s) = e−st Fε,i j (dt), s ≥ 0.

(12.19)

0

The condition Jˇ can be reformulated in the equivalent form in terms of the above Laplace transforms: Jˇ ◦ : (a) φε,i j (suε,i ) → φ0,i j (s) as ε → 0, for s ≥ 0, j ∈ Y1,i, i ∈ X, (b) φ0,i j (s) is the Laplace transform of a proper distribution function F0,i j (·) such that F0,i j (0) < 1, for j ∈ Y1,i, i ∈ X, (c) uε,i ∈ (0, ∞), ε ∈ (0, 1], for i ∈ X. Let us also recall the expectations of transition times, for i, j ∈ X and ε ∈ (0, 1], ∫ ∞ fε,i j = tFε,i j (dt). (12.20) 0

The perturbation condition K takes the following form: ˇ (a) fε,i j < ∞, j ∈ Y1,i, i ∈ X, for every ε ∈ (0, 1], (b) fε,i j /uε,i → f0,i j = K: ∫∞ tF0,i j (dt) < ∞ as ε → 0, for j ∈ Y1,i, i ∈ X. 0 The conditions IH , L, and LH take the following forms: IˇH : The functions p ·,i j , j ∈ Y1,i, i ∈ X belong to a complete family of asymptotically comparable functions H. ˇ uε,i → u0,i ∈ (0, ∞] as ε → 0, for i ∈ X. L: ˇ H : The functions u ·,i, i ∈ X belong to the complete family of asymptotically L comparable functions H. 12.2.1.2 Examples of Perturbed Birth–Death-Type Semi-Markov Processes Let us briefly give a number of examples of transition probabilities for stochastic systems of the birth–death type. For simplicity, we restrict ourselves to models of the Markov type, the transient characteristics of which are as follows: ⎧ Fε,0,− (t)pε,0,− ⎪ ⎪ ⎪ ⎪ F ⎪ ⎨ ε,0,+ (t)pε,0,+ ⎪ Q ε,i j (t) = Fε,i,± (t)pε,i,± ⎪ ⎪ Fε,m,− (t)pε,m,− ⎪ ⎪ ⎪ ⎪ Fε,m,+ (t)pε,m,+ ⎩

for for for for for

j j j j j

= 0, i = 0, = 1, i = 0, = i ± 1, 1 < i < m, = m − 1, i = m, = m, i = m,

(12.21)

336

12 Generalisations of limit theorems for hitting times

where, for i = 1, . . . , m, pε,i,± =

λε,i,± , Fε,i,± (t) = 1 − eλε, i t , t ≥ 0, λε,i

(12.22)

and λε,i,± ∈ (0, ∞), λε,i = λε,i,+ + λε,i,− .

(12.23)

First, let us mention some examples of perturbed bio-stochastic systems. In perturbed population dynamics models, the following options for parameters are typical,

λ i + νε if ◦ = +, (12.24) λε,i,◦ = ε if ◦ = −, με i where λε and με are, respectively, birth and death intensities and νε is an immigration rate. In perturbed epidemic models, one of the possible options for parameters is as follows and has the following form:

λ i(N − i) + νε (N − i) if ◦ = +, (12.25) λε,i,◦ = ε if ◦ = −, με i where λε and με are, respectively, contamination and recovering intensities, and νε is an external contamination intensity. In perturbed population genetics models, parameters can, for example, take the following form:

λ i + νε (N − i) if ◦ = +, (12.26) λε,i,◦ = ε νε i + με (N − i) if ◦ = −, where λε and με are birth intensities of individual with genes of two different types, and νε is a mutation intensity. Let us also list some examples of perturbed queuing systems. In the standard M/M(m/n) queuing systems, with the number of servers m and the number of repairing units n ≤ m, the parameters take the following form:

με min(i, n) if ◦ = +, (12.27) λε,i,◦ = if ◦ = −, λε i where με and λε are, respectively, the recovery and failure rates. 12.2.1.3 Hitting Times for Birth–Death-Type Semi-Markov Processes. The study of the hitting times τε,D for birth–death-type semi-Markov processes can be limited in either the cases, where the domain D = {0}, D = {m}, or D = {0, m}. All other cases can easily be reduced to these three cases. Moreover, the second case is similar to the first and, therefore, does not require separate consideration.

12.2 Perturbed birth-death-type SMP

337

12.2.2 Phase Space Reduction for Perturbed Birth–Death-Type Semi-Markov Processes 12.2.2.1 Removing of Virtual Transitions for Original Perturbed Birth–DeathType Semi-Markov Processes. At this step, we transform the original semi-Markov process ηε (t) into a new semi-Markov process η˜ε (t) with virtual transitions removed, using the procedure described in Sect. 9.2. In fact, the description of the corresponding algorithm and proofs are given in Sect. 9.2. We should only give the corresponding formulas for characteristics ˜ L ˜ H for the semi-Markov process η˜ε (t), ˜ and I˜ H , L appearing in the conditions G– and the corresponding characteristics for the limiting semi-Markov process η˜0 (t). The transition probabilities p˜ε,i j , i, j ∈ X for the embedded Markov chain η˜ε,n are given by the following relation, for ε ∈ (0, 1],

p˜ε,i j

⎧ 1 for j = 1, i = 0, ⎪ ⎪ ⎪ ⎨ pε, ii±1 for j = i ± 1, 0 < i < m, ⎪ p¯ ε, ii = ⎪1 for j = m − 1, i = m, ⎪ ⎪ ⎪0 otherwise. ⎩

(12.28)

˜ 1,i = {i ∈ X : p˜1,i j > 0}, i ∈ X, take the following forms: In this case, the sets Y ˜ 1,i = Y

⎧ for i = 0, ⎪ ⎨ {1} ⎪ {i − 1, i + 1} for 1 < i < m, ⎪ ⎪ {m − 1} for i = m. ⎩

(12.29)

˜ 1,i, i ∈ X for the distribution functions The Laplace transforms φ˜ε,i j (s), j ∈ Y ˜ Fε,i j (t) take the following forms, for ε ∈ (0, 1] and s ≥ 0,

φ˜ε,i j (s) =

⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

φ ε,01 (s) p¯ ε,00 1−φ ε,00 (s)pε,00 φ ε, ii±1 (s) p¯ ε, ii 1−φ ε, ii (s)pε, ii φ ε, mm−1 (s) p¯ ε, mm 1−φ ε, mm (s)pε, mm

for j = 1, i = 0, for j = i ± 1, 1 < i < m,

(12.30)

for j = m − 1, i = m,

where p¯ε,ii = 1 − pε,ii, i ∈ X. ˜ 1,i, i ∈ X take the following forms, for ε ∈ (0, 1], The expectations f˜ε,i j , j ∈ Y

f˜ε,i j

p fε,00 ⎧ ⎪ fε,01 + ε,00 for j = 1, i = 0, ⎪ p¯ ε,00 ⎪ ⎨ ⎪ pε, ii fε, ii for j = i ± 1, 0 < i < m, = fε,ii±1 + p¯ ε, ii ⎪ ⎪ pε, mm fε, mm ⎪ ⎪ fε,mm−1 + p¯ for j = m − 1, i = m. ε, mm ⎩

(12.31)

The normalisation functions u˜ε,i, ε ∈ (0, 1] are given by the following relation: u˜ε,i = p¯−1 ε,ii uε,i for i ∈ X.

(12.32)

338

12 Generalisations of limit theorems for hitting times

The transition probabilities p˜0,i j for the limiting embedded Markov chain η˜0,n are given by the following relation, for i, j ∈ X,

p˜ε,i j =

⎧ 1 ⎪ ⎪ ⎪ ⎪ ⎨ p˜ ⎪

for j = 1, i = 0,

0,ii±1

⎪ 1 ⎪ ⎪ ⎪ ⎪0 ⎩

for j = i ± 1, 0 < i < m, for j = m − 1, i = m, otherwise.

(12.33)

The limiting Laplace transforms φ˜0,i j (s) for the distribution functions F˜0,i j (t) are ˜ 1,i, i ∈ X and s ≥ 0, given by the following relation, for j ∈ Y

φ˜0,i j (s) =

X,

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

φ ε,01 (s) p¯ ε,00 1−φ ε,00 (s)pε,00 1 1+ f0,00 s φ0, ii±1 (s) p¯ 0, ii 1−φ0, ii (s)p0, ii 1 1+ f0, ii s φ0, mm−1 (s) p¯ 0, mm 1−φ0, mm (s)p0, mm 1 1+ f0, mm s

for j = 1, i = 0, if p¯0,00 > 0, for j = 1, i = 0, if p¯0,00 = 0, for j = i ± 1, 0 < i < m, if p¯0,ii > 0, for j = i ± 1, 0 < i < m, if p¯0,ii = 0,

(12.34)

for j = m − 1, i = m, if p¯0,mm > 0, for j = m − 1, i = m, if p¯0,mm = 0.

˜ 1,i, i ∈ The limiting expectations f˜0,i j are given by the following relation, for j ∈ Y

f˜0,i j

⎧ p¯ f + p0,00 f0,00 for j = 1, i = 0, ⎪ ⎪ ⎨ 0,00 0,01 ⎪ for j = i ± 1, 0 < i < m, = p¯0,ii f0,ii±1 + p0,ii f0,ii ⎪ ⎪ ⎪ p¯0,mm f0,mm−1 + p0,mm f0,mm for j = m − 1, i = m. ⎩

(12.35)

12.2.2.2 One-State Reduction of Phase Space for Perturbed Birth–DeathType Semi-Markov Processes. At this step, we transform the semi-Markov process η˜ε (t) into a new semi-Markov process k ηε (t) with the reduced phase space k X = X \ {k}, using the procedure described in Sect. 9.3. It should be noted that in the case where 0 ≤ k ≤ m, the reduced phase space X = {0, . . . , k −1, k +1, . . . , m}. By obvious re-numeration of states, 0 → 0, . . . , k − k 1 → k − 1, k + 1 → k, . . . , m → m − 1, the space k X can be transformed into the standard form k X = {0, . . . , m − 1}. Two cases should be considered. The first case, where one of end states 0 or m is to be excluded from the phase space X. These cases are similar, so we can restrict consideration, for example, to the latter case. The second case, where some internal state 0 < k < m is to be excluded from the phase space X. In fact, the description of the corresponding algorithm and proofs are given in Sect. 9.3. We should only give the corresponding formulas for characteristics appearing in the conditions k G–k L and k IH , k LH for the semi-Markov processes

12.2 Perturbed birth-death-type SMP

339

η˜ε (t) and the corresponding characteristics for the limiting semi-Markov process k η0 (t). The transition probabilities m pε,i j , i, j ∈ m X for the embedded Markov chain m ηε,n are given by the following relation, for ε ∈ (0, 1],

m pε,i j

⎧ 1 ⎪ ⎪ ⎪ ⎪ ⎪ p˜ ⎪ ⎪ ⎨ ε,ii±1 ⎪ = p˜ε,m−1m ⎪ ⎪ ⎪ ⎪ p˜ε,m−1m−2 ⎪ ⎪ ⎪ ⎪0 ⎩

for j = 1, i = 0, for j = i ± 1, 0 < i < m − 1, for j = m − 1, i = m − 1,

(12.36)

for j = m − 2, i = m − 1, otherwise.

The relation (12.36) shows that the embedded Markov chain m ηε,n has the same birth–death type as the original embedded Markov chain ηε,n . The Markov chain m ηε,n has the reduced phase space m X = {0, . . . , m − 1}. Note also that this Markov chain admits virtual transitions of the form m − 1 → m − 1. The situation is slightly different in the case, where 0 < k < m. The transition probabilities k pε,i j , i, j ∈ k X for the embedded Markov chain k ηε,n are given by the following relation, for ε ∈ (0, 1],

k pε,i j

⎧ 1 ⎪ ⎪ ⎪ ⎪ ⎪ p˜ε,ii±1 ⎪ ⎪ ⎪ ⎪ ⎪ p ˜ε,k±1,k p˜ε,kk±1 ⎪ ⎪ ⎪ ⎪ ⎨ p˜ε,k±1k±2 ⎪ = ⎪ p˜ε,k±1,k p˜ε,kk∓1 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ p˜ε,ii±1 ⎪ ⎪ ⎪ ⎪ 1 ⎪ ⎪ ⎪ ⎪0 ⎩

for j = 1, i = 0, for j = i ± 1, 0 < i < k − 1, for j = k ± 1, i = k ± 1, for j = k ± 2, i = k ± 1, for j = k ∓ 1, i = k ± 1,

(12.37)

for j = i ± 1, k + 1 < i < m, for j = m − 1, i = m, otherwise.

The relation (12.37) shows that the embedded Markov chain k ηε,n has the same birth–death type as the initial embedded Markov chain ηε,n . The Markov chain k ηε,n has the reduced phase space m X = {0, . . . , k − 1, k + 1, . . . m}. Note also that this Markov chain admits virtual transitions of the forms k +1 → k +1 and k −1 → k −1. The sets m Y1,i = {i ∈ X : m p1,i j > 0}, i ∈ m X take the following forms, m Y1,i

⎧ {1} for i = 0, ⎪ ⎪ ⎨ ⎪ = {i − 1, i + 1} for 1 ≤ i < m − 1, ⎪ ⎪ ⎪ {m − 2, m − 1} for i = m − 1. ⎩

The sets k Y1,i = {i ∈ X : 0 < k < m,

k p1,i j

> 0}, i ∈

kX

(12.38)

take, the following forms, for

340

12 Generalisations of limit theorems for hitting times

k Y1,i

⎧ {1} ⎪ ⎪ ⎪ ⎪ ⎪ {i − 1, i + 1} ⎨ ⎪ = {k ± 2, k ± 1, k ∓ 1} ⎪ ⎪ ⎪ {i − 1, i + 1} ⎪ ⎪ ⎪ {m} ⎩

for i = 0, if 0 < k − 1, for 1 ≤ i < k − 1, for i = k ± 1, for k + 1 < i ≤ m − 1, for i = m, if m > k − 1.

(12.39)

The Laplace transforms m φε,i j (s), j ∈ m Y1,i, i ∈ m X for the distribution functions take the following forms, for ε ∈ (0, 1] and s ≥ 0,

m Fε,i j (t)

m φε,i j (s)

=

⎧ φ˜ε,10 (s) ⎪ ⎪ ⎪ ⎪ ⎨ φ˜ ⎪ (s)

for j = 1, i = 0,

for j = i ± 1, 1 < i < m − 1, ε,ii±1 ˜ ⎪ φε,m−1m−2 (s) for j = m − 2, i = m − 1, ⎪ ⎪ ⎪ ⎪ φ˜ ˜ ⎩ ε,m−1m (s)φε,mm−1 (s) for j = m − 1, i = m − 1.

(12.40)

The Laplace transforms k φε,i j (s), j ∈ k Y1,i, i ∈ k X for the distribution functions take the following forms, for 0 < k < m, ε ∈ (0, 1], and s ≥ 0,

k Fε,i j (t)

⎧ φ˜ε,10 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜ε,ii±1 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜ (s) ⎪ ⎪ ⎨ ε,k±1k±2 ⎪ ˜ φ (s) φ˜ε,kk±1 (s) ε,k±1k k φε,i j (s) = ⎪ ⎪ ⎪ φ˜ε,k±1k (s)φ˜ε,kk∓1 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜ε,ii±1 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜ε,mm−1 (s) ⎩

for j = 1, i = 0, if 0 < k − 1, for j = i ± 1, 1 ≤ i < k − 1, for j = k ± 2, i = k ± 1, for j = k ± 1, i = k ± 1, for j = k ∓ 1, i = k ± 1,

(12.41)

for j = i ± 1, k + 1 < i ≤ m − 1, for j = m − 1, i = m, if m > k − 1.

The expectations m fε,i j , j ∈ k Y1,i, i ∈ X take the following forms, for ε ∈ (0, 1],

m fε,i j

⎧ f˜ε,10 ⎪ ⎪ ⎪ ⎪ ⎨ f˜ε,ii±1 ⎪ = ⎪ ⎪ f˜ε,m−1m−2 ⎪ ⎪ ⎪ f˜ ⎩ ε,m−1m + f˜ε,mm−1

for j = 1, i = 0, for j = i ± 1, 1 < i < m − 1, for j = m − 2, i = m − 1,

(12.42)

for j = m − 1, i = m − 1.

The expectations k fε,i j , j ∈ k Y1,i, i ∈ X take the following forms, for 0 < k < m, ε ∈ (0, 1],

k fε,i j

=

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

f˜ε,10 f˜ε,ii±1 f˜ε,k±1k±2 f˜ε,k±1k + f˜ε,kk±1 f˜ε,k±1k + f˜ε,kk∓1 f˜ε,ii±1 f˜ε,mm−1

for for for for for for for

j j j j j j j

= 1, i = 0, if 0 < k − 1, = i ± 1, 1 ≤ i < k − 1, = k ± 2, i = k ± 1, = k ± 1, i = k ± 1, = k ∓ 1, i = k ± 1, = i ± 1, k + 1 < i ≤ m − 1, = m − 1, i = m, if m > k − 1.

(12.43)

12.2 Perturbed birth-death-type SMP

341

The normalisation function k uε,i, ε ∈ (0, 1] is given by the following relation, for i ∈ k X, (12.44) k uε,i = u˜ ε,i . The limiting transition probabilities m p0,i j for the embedded Markov chain m η0,n are given by the following relation:

m p0,i j

⎧ 1 ⎪ ⎪ ⎪ ⎪ ⎪ p˜ ⎪ ⎨ 0,ii±1 ⎪ = p˜0,m−1m ⎪ ⎪ ⎪ p˜0,m−1m−2 ⎪ ⎪ ⎪ ⎪0 ⎩

for j = 1, i = 0, for j = i ± 1, 1 < i < m − 1, for j = m − 1, i = m − 1, for j = m − 2, i = m − 1, otherwise.

(12.45)

The limiting transition probabilities k p0,i j for the embedded Markov chain k η0,n are given by the following relation, for 0 < k < m,

k p0,i j

⎧ 1 ⎪ ⎪ ⎪ ⎪ ⎪ p˜0,ii±1 ⎪ ⎪ ⎪ ⎪ p˜0,k±1,k p˜ε,kk±1 ⎪ ⎪ ⎪ ⎪ ⎨ p˜0,k±1k±2 ⎪ = ⎪ p˜0,k±1,k p˜0,kk∓1 ⎪ ⎪ ⎪ ⎪ ⎪ p˜0,ii±1 ⎪ ⎪ ⎪ ⎪1 ⎪ ⎪ ⎪ ⎪0 ⎩

for j = 1, i = 0, for j = i ± 1, 0 < i < k − 1, for j = k ± 1, i = k ± 1, for j = k ± 2, i = k ± 1, for j = k ∓ 1, i = k ± 1, for j = i ± 1, k + 1 < i < m, for j = m − 1, i = m, otherwise.

(12.46)

As it was pointed out in Lemma 9.20, the conditions G, H, IH , and LH imply ˆ is satisfied. This condition takes the following form: that the condition L ˇˆ w L: ε, j,i =

u˜ ε, j u˜ ε, i

→ w0, j,i ∈ [0, ∞] as ε → 0, for i, j ∈ X.

Also, the absorbing rate condition k M should be satisfied for the state k, in order the conditions k J1 would be satisfied for the semi-Markov processes k ηε (t). By Remark 9.3, this condition can be replaced by the weaker local absorbing rate ˆ which takes, in this case, the following form: condition k M, ˇ

ˆ k M:

w0,k,i ∈ [0, ∞), for i ∈ X, |i − k | ≤ 1.

In this case, the corresponding limiting Laplace transforms k φ0,i j (s), j ∈ k Y1,i, i ∈ X for the distribution functions k F0,i j (t) are given by the following relations, for the case k = m and s ≥ 0, m φ0,i j (s)

=

⎧ φ˜0,10 (s) ⎪ ⎪ ⎪ ⎪ ⎨ φ˜ ⎪ (s)

for j = 1, i = 0,

for j = i ± 1, 1 < i < m − 1, 0,ii±1 ˜ ⎪ φ (s) for j = m − 2, i = m − 1, 0,m−1m−2 ⎪ ⎪ ⎪ ⎪ φ˜ ⎩ 0,m−1m (s)φ˜0,mm−1 (w0,m,m−1 s) for j = m − 1, i = m − 1

(12.47)

342

12 Generalisations of limit theorems for hitting times

and, for the case 0 < k < m and s ≥ 0, k φ0,i j (s)

⎧ φ˜0,10 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜0,ii±1 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜ (s) ⎪ ⎪ ⎨ 0,k±1k±2 ⎪ = φ˜0,k±1k (s)φ˜ε,kk±1 (w0,k,±1 s) ⎪ ⎪ ⎪ φ˜0,k±1k (s)φ˜ε,kk∓1 (w0,k,±1 s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜0,ii±1 (s) ⎪ ⎪ ⎪ ⎪ ⎪ φ˜0,mm−1 (s) ⎩

for j = 1, i = 0, if 0 < k − 1, for j = i ± 1, 1 ≤ i < k − 1, for j = k ± 2, i = k ± 1, for j = k ± 1, i = k ± 1, for j = k ∓ 1, i = k ± 1,

(12.48)

for j = i ± 1, k + 1 < i ≤ m − 1, for j = m − 1, i = m, if m > k − 1.

˜ 1,i, i ∈ X are given by the following relaThe limiting expectations k f0,i j , j ∈ Y tions, for the case k = m,

m f0,i j

⎧ f˜0,10 ⎪ ⎪ ⎪ ⎪ ⎨ f˜0,ii±1 ⎪ = ⎪ f˜0,m−1m−2 ⎪ ⎪ ⎪ ⎪ f˜ ⎩ 0,m−1m + f˜ε,mm−1 w0,m,m−1

for j = 1, i = 0, for j = i ± 1, 1 < i < m − 1, for j = m − 2, i = m − 1,

(12.49)

for j = m − 1, i = m − 1

and, for the case 0 < k < m,

k f0,i j

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪

f˜0,10 f˜0,ii±1

for j = 1, i = 0, if 0 < k − 1,

for j = i ± 1, 1 ≤ i < k − 1, ˜f0,k±1k±2 for j = k ± 2, i = k ± 1, = f˜0,k±1k + f˜0,kk±1 w0,k,k±1 for j = k ± 1, i = k ± 1, ⎪ ⎪ ⎪ ⎪ f˜0,k±1k + f˜0,kk∓1 w0,k,k∓1 for j = k ∓ 1, i = k ± 1, ⎪ ⎪ ⎪ ⎪ ⎪ for j = i ± 1, k + 1 < i ≤ m − 1, f˜0,ii±1 ⎪ ⎪ ⎪ ⎪ f˜ for j = m − 1, i = m, if m > k − 1. ⎩ 0,mm−1

(12.50)

12.2.2.3 The Asymptotic Recurrent Algorithm of Phase Space Reduction for Perturbed Birth–Death-Type Semi-Markov Processes. In Sects. 12.2.2.1 and 12.2.2.2, the first step of the asymptotic recurrent algorithm of phase space reduction for perturbed birth–death-type semi-Markov processes connected with the transformations of the original semi-Markov process was presented. These transformations can be represented by the symbolic diagram, ηε (t) → η˜ε (t) → k ηε (t). Importantly, both processes η˜ε (t) and k ηε (t) are also birth–death-type semiMarkov processes. The resulting process k η˜ε (t) is a birth–death-type semi-Markov process with the reduced phase space Xk . The following steps of the asymptotic recurrent algorithm of phase space reduction are described in Sect. 9.4. In the case of birth–death-type semi-Markov processes

12.3 Numerical examples

343

these steps should be implemented using the relations given above in Sects. 12.2.2.1 and 12.2.2.2. It should be noted that in the case of semi-Markov birth–death processes, the asymptotic algorithms for removing virtual transitions and reducing the phase space preserve the birth–death structure for reduced semi-Markov processes. This makes these algorithms computationally more efficient. 12.2.2.4 Limit Theorems for Hitting Times for Perturbed Birth–Death-Type Semi-Markov Processes. As mentioned in Sect. 12.2.1.3, hitting times τε,D are subject of studies for two cases, where domain D = {0} or D = {0, m}. The corresponding limit theorems based on the use of asymptotic recurrent algorithms of phase space reduction are presented in Chaps. 10 and 11.

12.3 Numerical Examples In this section, we present numerical examples illustrating asymptotic recurrent algorithms for singularly perturbed semi-Markov processes and asymptotic results for hitting times, which are presented in Chaps. 10 and 11.

12.3.1 An Example of Singularly Perturbed Semi-Markov Processes ¯ with Two-State Domain D 12.3.1.1 Definition of the Model. Let us consider an example, which let us illustrate the asymptotic results presented in Chaps. 10 and 11. In this example, the phase space X = {1, 2, 3}, and the matrix Pε = pε,i j  of transition probabilities for the embedded Markov chain ηε,n , has the following form, for ε ∈ (0, 1],    1 − 1 εα − 1 εβ 1 εα 1 εβ  2 2 2 2     (12.51) Pε =  21 ε 1 − ε 12 ε  .   1 1  1 3

3

3

Here, parameters α, β ∈ [0, ∞). Also, we assume that the distribution functions of transition times take the following forms, for ε ∈ (0, 1],  Conε −γ (·) for i = 1, j = 1, 2, 3, Fε,i j (·) = (12.52) Con1 (·) for i = 2, 3, j = 1, 2, 3, where the parameter γ ∈ [0, ∞). This means that the transition times from the state 1 take the value ε −γ , while transition times from the states 2 and 3 take the value 1. The semi-Markov process ηε (t) is constructed as described in Sect. 9.1. Note that, in the case γ = 0, the semi-Markov process ηε (t) is a discrete time Markov chain embedded in continuous time.

344

12 Generalisations of limit theorems for hitting times

12.3.1.2 Hitting Times. Let us consider the case, where domain D = {3} and, ¯ = {1, 2}, i.e., it is a two-state set. Since D is a one-state set, the thus, domain D ¯ indicator I(ηε (τε,D ) = 3) = 1, and, thus, for i ∈ D, G ε,D,i3 (·) = Pi {τε,D ≤ ·}.

(12.53)

Let us describe the weak convergence asymptotics for the distribution functions ¯ that is, find the correct normalisation functions of the hitting times G ε,D,i3 (·), i ∈ D, ¯ ¯ not uˇε,i, i ∈ D, the use of which leads to the limiting distributions G0,D,i3 (·), i ∈ D concentrated in 0, and describe the corresponding asymptotics for the expectations of hitting times Eε,D,i3 . It is useful to recall that, according to Lemma 10.1, the distribution functions ¯ are completely determined by the transition probabilities Q ε,i j (·), i ∈ G ε,D,i3 (·), i ∈ D ¯ D, j ∈ X for the semi-Markov process ηε (t) and, thus, the above distribution functions of hitting times coincide for semi-Markov processes with any form of transition probabilities Q ε,3r (·), r ∈ X. 12.3.1.3 Perturbation Conditions. All basic perturbation conditions are satisfied for the semi-Markov processes ηε (t). The condition G is satisfied. Moreover, the sets Y1,i = X, for i ∈ X. The condition H is also satisfied. The corresponding limiting matrix of transition probabilities P0 of the embedded Markov chain η0,n takes different forms for the following six cases: (1) α = β = 0, (2) α > β = 0, (3) β > α = 0, (4) α = β > 0, (5) α > β > 0, (6) β > α > 0.

(12.54)

Let us denote the matrix P0 for these six cases, respectively, as P(h) 0 , for h = 1, . . . , 6. Obviously,      1  1 1 1   0 1 1    2 2  2 0 2  2 2 0      0 1 0  , P(2) =  0 1 0  , P(3) =  0 1 0  , (12.55) = P(1)  1 1 1  0  1 1 1  0  1 1 1  0       3

3

3

3

and P(h) 0

3

3

3

3

  1 0 0   =  0 1 0  , h = 4, 5, 6.  1 1 1  3

3

3

(12.56)

3

In this model, the state 2 is an asymptotically absorbing state, while the state 1 is an asymptotically non-absorbing state, for the cases h = 1, 2, 3, or an asymptotically absorbing state, for the cases h = 4, 5, 6. Obviously, the transition probabilities pε,i j , i, j ∈ X belong to the family of asymptotically comparable power-type functions H1 introduced in Sect. 8.2.1 and, thus, condition IH1 is satisfied. In this case, it is natural to choose the initial normalisation functions, uε,1 = ε −γ, uε,2 = uε,3 = 1, ε ∈ (0, 1].

(12.57)

12.3 Numerical examples

345

Note, first of all that, in this case, the condition of asymptotic comparability L is satisfied for the normalisation functions uε,i, i ∈ X, which belong to the complete family of asymptotically comparable functions H1 defined by the relation (8.14). Thus, the condition LH1 is satisfied. The conditions J and J◦ are satisfied, with the corresponding limiting distribution functions F0,i j (t) and their Laplace transforms φ0,i j (s), given by the following relations, for i, j ∈ X, (12.58) F0,i j (·) = Con1 (·), and

φ0,i j (s) = e−s, s ≥ 0.

(12.59)

The condition K is also satisfied, and the corresponding limiting expectations take the following form, for i, j ∈ X, f0,i j = 1.

(12.60)

12.3.2 A One-Step Asymptotic Reduction of Phase Space ¯ is a two-state set. Therefore, the asymptotic recurrent algorithm The domain D described in Chap. 9 includes only the chain of transformations ηε (t) → η˜ε (t) → k1 ηε (t) → k1 η˜ε (t). As we will see, the state k 1 = 1 belongs to the set W0 of the least absorbing states and, thus, can be chosen for exclusion from the initial phase space X. 12.3.2.1 Removing of Virtual Transitions for Semi-Markov Processes ηε (t). The procedure of asymptotic removing of virtual transitions described in Sect. 9.2 should be applied to the semi-Markov processes ηε (t). The embedded Markov chain η˜ε,n has the transition probabilities p˜ε,i j = I( j  p i) 1−pε,ε,i jii , i, j ∈ X given by the relation (9.46). The matrix of transition probabilities for this Markov chain takes the following form, for ε ∈ (0, 1],   εβ   0 εα  ε α +e β ε α +e β    1 . (12.61) P˜ ε =  12 0  2  1 1  0  2 2 The corresponding limiting matrix of transition probabilities P˜ 0 for the Markov chain η˜0,n takes various forms for the cases (1)–(6). We denote the matrix P˜ 0 for these cases, respectively, as P˜ (h) 0 , for h = 1, . . . , 6. Obviously, P˜ (1) 0

  0 12 21   =  12 0 12   1 1 0 2 2

   0 0 1    ˜ (2)  1  , P0 =  2 0 12     1 1 0 2 2

   0 1 0    ˜ (3)  1  , P0 =  2 0 12     1 1 0 2 2

    ,  

(12.62)

346

12 Generalisations of limit theorems for hitting times

and P˜ (4) 0

  0 12 12   =  21 0 12   1 1 0 2 2

   0 0 1    ˜ (5)  1  , P0 =  2 0 12     1 1 0 2 2

  0 1 0     ˜ (6)  1  , P0 =  2 0 12     1 1 0 2 2

    .  

(12.63)

In this case, the normalisation functions u˜ε,i = (1 − pε,ii )−1 uε,i, i ∈ X take, according to the relation (9.60), the following forms: u˜ε,1 = 2ε −γ (ε α + ε β )−1, u˜ε,2 = ε −1, u˜ε,3 = 3/2, ε ∈ (0, 1].

(12.64)

Let us define the parameter, δ = min(α, β) + γ.

(12.65)

If δ = 0, then: u˜ε,1 ≡ 1, in the case (1), while u˜ε,1 → 2 as ε → 0, in the cases (2) and (3). If δ > 0, then u˜ε,1 → ∞ as ε → 0, in the cases (3)–(6). Also, u˜ε,2 → ∞ as ε → 0, while u˜ε,3 ≡ 3/2, in the cases (1)–(6). The probability p0,11 < 1 in the cases (1)–(3), i.e., if α ∧ β = 0. While p0,11 = 1 in the cases (4)–(6), i.e., if α ∧ β > 0. According to Theorem 9.1, the condition J˜ ◦ is satisfied for the semi-Markov processes with removed virtual transitions η(t). ˜ The Laplace transform φ˜0,1j (·), j ∈ X for the corresponding limiting distribution function F˜0,1j (·) appearing in the condition J˜ ◦ should be calculated using the relations (9.62) and (9.63), if p0,11 < 1, or the relation (9.73), if p0,11 = 1. Thus, the relations (12.52), (12.55), and (12.59) imply that, for j ∈ X and s ≥ 0, φ˜0,1j (s) = =

φ0,1j ((1 − p0,11 )s)(1 − p0,11 ) 1 − p0,11 φ0,1j ((1 − p0,11 )s))  e−s if α = β = 0, 1 − s2 /(1 2e

− 12 e− 2 ) if α > 0, β = 0 or α = 0, β > 0. s

(12.66)

Note that the distribution functions corresponding to the Laplace transforms appearing in the relation (12.66) are, respectively, Con1 (·), for the case α = β = 0, or Geo 1 (2·), for the cases α > 0, β = 0 and α = 0, β > 0. 2 If α ∧ β > 0, the relations (12.52), (12.55), and (12.60) imply that, for j ∈ X and s ≥ 0, (12.67) φ˜0,1j (s) = (1 + f0,11 s)−1 = (1 + s)−1 . The probability p0,22 = 1, in the cases (1)–(6). Thus, the Laplace transform φ˜0,2j (s), s ≥ 0 for the corresponding limiting distribution function F˜0,2j (·), j = 1, 2, 3 appearing in the condition J˜ ◦ should be calculated using the relations (9.73) and (9.74). In the case, the relations (12.52), (12.55), and (12.60) imply that, for j ∈ X and s ≥ 0,

12.3 Numerical examples

347

φ˜0,2j (s) = (1 + f0,22 s)−1 = (1 + s)−1 .

(12.68)

The distribution function corresponding to the Laplace transform appearing in the relations (12.67) and (12.68) is E xp1 (·). ˜ is satisfied for the semi-Markov According to Theorem 9.1, the condition K processes η(t). ˜ ˜ is the first Therefore, the limiting expectation f˜0,i j appearing in the condition K ¯ j ∈ X, moment of the distribution function F˜0,i j (·) and, thus, for i ∈ D, f˜0,i j = 1.

(12.69)

As noted in Sect. 12.3.1.2, computing of the Laplace transforms φ˜0,3j (·), j ∈ X and the expectations f˜0,3j , j ∈ X is not required. 12.3.2.2 Reduction of the Phase Space for Semi-Markov Processes η˜ε (t). The procedure of asymptotic one-step reduction of phase space, described in Sect. 9.3, should be applied to the semi-Markov processes η˜ε (t). The relations (12.64) and (12.65) imply that, u˜ε,1 = 2ε −γ (ε α + ε β )−1 ∼

2 ε −δ as ε → 0. 1 + I(α = β)

(12.70)

There are three cases to consider: (i) δ ∈ [0, 1), (ii) δ = 1, (iii) δ ∈ (1, ∞).

(12.71)

Obviously, wε,1,2 = where w0,1,2 =

u˜ε,1 → w0,1,2 as ε → 0, u˜ε,2 ⎧ ⎪ ⎨0 ⎪

2 1+I(α=β)

⎪ ⎪∞ ⎩

if δ ∈ [0, 1), if δ = 1, if δ ∈ (1, ∞).

Let us consider the case (i), where δ ∈ [0, 1). Note that u˜ε,3 wε,3,2 = → 0 as ε → 0. u˜ε,2

(12.72)

(12.73)

(12.74)

ˆ and 1 M are satisfied, i.e., the states 1 and 3 are In this case, the conditions L asymptotically less absorbing than the state 2. That is why, the state 1 should be chosen for exclusion from the phase space X, according to the procedure described in Sect. 9.3. In this case, the reduced phase ¯ = {2} is a one-state set. space is 1 X = {2, 3} and the domain 1 D The Markov chain 1 ηε,n has the transition probabilities 1 pε,i j = p˜ε,i j + p˜ε,i1 p˜ε,1j , i, j = 2, 3 given by the relation (9.87). The matrix of transition probabilities for this

348

12 Generalisations of limit theorems for hitting times

Markov chain takes the following form, for ε ∈ (0, 1],   ε α +2ε β   εα  2(ε α +ε β ) 2(ε α +ε β )  . 1 Pε =  2ε α +ε β εβ  α β  α β

(12.75)

2(ε +ε ) 2(ε +ε )

The corresponding limiting matrix of transition probabilities 1 P0 for the Markov chain 1 η0,n takes three various forms in the cases: (1), (4), i.e., if α = β; (2), (5), i.e., if α > β; and (3), (6), i.e., if β > α. We denote the matrix 1 P0 for these cases, respectively, as 1 P(h), h = 1, . . . , 6. Obviously,   1 3    1 1     0 1  (1) (3) 4 4  , P(2) =     2 2 . , P = P = 1 0  3 1  1 0  1 1  1 0 1 0 2 2 4 4 (4) 1 P0

  = 

3 4 1 4

1 4 3 4

    1 1      , 1 P(5) =  01 11  , 1 P(6) =  2 2    1 0 0 0 2 2

  . 

(12.76)

In this case, the normalisation functions, 1 uε,2 and 1 uε,3 , take, according to the relation (9.112), the following form: 1 uε,2

= u˜ε,2 = ε −1, 1 uε,3 = u˜ε,3 = 3/2, ε ∈ (0, 1].

(12.77)

According to Theorem 9.2, the condition 1 J◦ is satisfied. By Lemma 9.19, the condition 1 Iˆ is satisfied and, for j = 2, 3,

1 qˆ0,2j

p˜ε,2j = lim ε→0 1 pε,2j

⎧ 0 if j ⎪ ⎪ ⎪ ⎨ 2 if j ⎪ = 31 ⎪ ⎪ 2 if j ⎪ ⎪ 1 if j ⎩

= 2, = 3, α = β, = 3, α > β, = 3, β > α.

(12.78)

The Laplace transform 1 φ0,2j (s), s ≥ 0 for the corresponding limiting distribution function 1 F0,2j (·) appearing in the condition 1 J◦ should be calculated using the relations (9.101) and (9.102), for j = 2, 3. Using the relations (9.101), (9.102), (12.68), (12.72), (12.73), and (12.78) we get, taking into account that w0,1,2 = 0, that, for j = 3 and s ≥ 0, 1 φ0,23 (s)

= φ˜0,23 (s) 1 q0,23 + φ˜0,21 (s)φ˜0,13 (w0,1,2 s)(1 − 1 q0,23 ) = (1 + s)−1 1 q0,23 + (1 + s)−1 (1 − 1 q0,23 ) = (1 + s)−1,

(12.79)

and, for j = 2 and s ≥ 0, 1 φ0,22 (s)

= φ˜0,21 (s)φ˜0,12 (w0,1,2 s) = (1 + s)−1 .

(12.80)

Thus, the distribution function 1 F0,2j (·) = E xp1 (·), for j = 2, 3. According to Theorem 9.2, the condition 1 K is also satisfied. Therefore, the limiting expectation 1 f0,2j appearing in the condition 1 K is the first moment of the distribution function 1 F0,2j (·) and, thus, for j = 2, 3,

12.3 Numerical examples

349 1 f0,2j

= 1.

(12.81)

According to the remarks made in Sect. 12.3.1.2, computing of the Laplace transforms 1 φ0,3j (·), j ∈ X and the expectations 1 f0,3j , j ∈ X is not required. 12.3.2.3 Removing of Virtual Transitions for the Reduced Semi-Markov Processes 1 ηε (t). The procedure of asymptotic removing of virtual transitions described in Sect. 9.2 should be applied to the reduced semi-Markov process 1 ηε (t). p The Markov chain 1 η˜ε,n has the transition probabilities 1 pε,i j = I( j  i) 1−1 1 ε,pε,i jii = I( j  i), i, j ∈ 2, 3, and, thus, the matrix of its transition probabilities takes the following form, for ε ∈ (0, 1],   0 1 .  ˜ (12.82) 1 Pε =  0 1 In this case, the normalisation function 1 u˜ε,2 takes, according to the relations (9.111), (12.51), and (12.75), the following form: 1 u˜ ε,2

= (1 − 1 pε,22 )−1 (1 − pε,22 )−1 uε,2 = (1 −

εα 2(ε α + ε β ) −1 −1 ε = ) 2(ε α + eβ ) ε(ε α + 2ε β )

−1 ∼ 1 p−1 as ε → 0, 0,23 ε

(12.83)

where the coefficient 1 p−1 0,23 takes, according to the relation (12.76), the following value, 4 −1 · I(α = β) + 1 · I(α > β) + 2 · I(β > α). (12.84) 1 p0,23 = 3 ◦ According to Theorem 9.1, the condition 1 J˜ is satisfied for the semi-Markov ˜ with removed virtual transitions. processes 1 η(t) The Laplace transform 1 φ˜0,23 (s), s ≥ 0 for the corresponding limiting distribution ◦ function 1 F˜0,23 (·) appearing in the condition 1 J˜ should be calculated using the relations (9.62) and (9.63), if 1 p0,22 < 1, or the relation (9.73), if 1 p0,22 = 1. These relations should be applied to the semi-Markov process 1 ηε (t). The relation (12.76) shows that the first variant takes place. Thus, using the relations (12.76), (12.79), and (12.80), we get, for s ≥ 0, 1 φ˜0,23 (s)

− 1 p0,22 )s)(1 − 1 p0,22 ) 1 − 1 p0,22 1 φ0,23 ((1 − 1 p0,22 )s))   −1 1 − 1 p0,22 1 1 p0,22 1− . = = 1 + (1 − 1 p0,22 )s 1 + (1 − 1 p0,22 )s 1+s =

1 φ0,23 ((1

(12.85)

˜ is also satisfied for the semi-Markov According to Theorem 9.1, the condition 1 K ˜ with removed virtual transitions. processes 1 η(t) Also, according to the relations (12.76) and (12.81), ˜

1 f0,23

= (1 − 1 p0,22 ) 1 f0,23 + 1 p0,22 1 f0,22 = 1.

(12.86)

350

12 Generalisations of limit theorems for hitting times

12.3.3 Asymptotics for Distributions and Expectations of Hitting Times Now we are ready to present asymptotic relations for the distribution functions ¯ and the expectations Eε,D,i3, i ∈ D. ¯ G ε,D,i3 (·), i ∈ D 12.3.3.1 Weak Asymptotics for Distribution Functions of Hitting Times. First, let us describe the weak asymptotics for distribution functions G ε,D,23 (·). According to Theorem 10.2, the normalisation function uˇε,2 = 1 u˜ε,2 , given by the relation (12.83), should be used in this case. Also, Theorem 10.2 and the relation (12.85) imply that the limiting Laplace transform Ψ0,D,23 (s), s ≥ 0 takes the following form: 1 , for s ≥ 0. (12.87) Ψ0,D,23 (s) = 1 φ˜0,23 (s) = 1+s Thus, Theorem 10.2 gives, in accordance with the relation (12.87), the following asymptotic relation: G ε,D,23 (· 1 u˜ε,2 ) ⇒ G0,D,23 (·) = E xp1 (·) as ε → 0.

(12.88)

Second, let us describe the weak asymptotics for distribution functions G ε,D,13 (·). According to Theorem 10.3, the form of the corresponding weak asymptotics depends on the value of the probability p˜0,12 . It takes, by the relations (12.62) and (12.63), the following value: p˜0,12 =

1 · I(α = β) + 0 · I(α > β) + 1 · I(β > α). 2

(12.89)

In the cases α = β and β > α, the probability p˜0,12 takes positive values. According to Theorem 10.3, the normalisation function uˇε,1 = 1 u˜ε,2 , given by relation (12.83), should be used in this case. Also, Theorem 10.3 and the relations (12.62), (12.63), (12.66), (12.67), (12.72), (12.73), (12.76), (12.87), and (12.89) imply that the limiting Laplace transform Ψ0,D,13 (s), s ≥ 0 takes the following form, for s ≥ 0, Ψ0,D,13 (s) = φ˜0,13 ((1 − 1 p0,22 )w0,1,2 s)) p˜0,13 + Ψ0,D,23 (s)φ˜0,12 ((1 − 1 p0,22 )w0,1,2 s) p˜0,12 1 p˜0,12 = = p˜0,13 + 1+s

1 2

+

1 1+s

1 1 2 1+s

if α = β, if β > α.

(12.90)

Thus, Theorem 10.3 gives, in accordance with the relation (12.90), the following asymptotic relation: G ε,D,13 (· 1 u˜ε,2 ) ⇒ G0,D,13 (·) as ε → 0, where G0,D,13 (·) =

1

2 Con0 (·) E xp1 (·)

+ 12 E xp1 (·) if α = β, if β > α.

(12.91)

(12.92)

12.3 Numerical examples

351

In the case α > β, the probability p˜0,12 = 0. According to Theorem 10.3, the normalisation function uˇε,1 = u˜ε,1 , given by relation (12.70), should be used in this case. Also, Theorem 9.3 and the relations (12.62), (12.63), (12.66), (12.67), and (12.89) imply that, in this case, the limiting Laplace transform Ψ0,D,13 (s) takes the following form, for s ≥ 0, Ψ0,D,13 (s) = φ˜0,13 (s) p˜0,13

1 −s s e 2 /(1 − 12 e− 2 ) if α > β = 0, = 21 if α > β > 0. 1+s

(12.93)

Thus, Theorem 10.3 gives, in accordance with the relation (12.93), the following asymptotic relation: G ε,D,13 (· u˜ε,1 ) ⇒ G0,D,13 (·) as ε → 0, where

G0,D,13 (·) =

Geo 1 (2·) if α > β = 0, 2 E xp1 (·) if α > β > 0.

(12.94)

(12.95)

12.3.3.2 Asymptotics for Expectations of Hitting Times. Remind that we consider the case (i), where δ = min(α, β) + γ ∈ [0, 1). In this case, the normalisation function 1 u˜ε,2 , given by the relation (12.83) and used in the weak convergence relations (12.88), (12.91), and the normalisation function u˜ε,1 , given by the relation (12.70) and used in the weak convergence relation (12.94), are connected by the following relation: u˜ε,1 = o(1 u˜ε,2 ) as ε → 0.

(12.96)

According to Theorem 11.1, the following asymptotic relation takes place: Eε,D,23 / 1 u˜ε,2 → E¯0,D,23 = E0,D,23 = 1 as ε → 0.

(12.97)

The situation is more complex in the case, where the initial state is 1. In this case, the normalisation function used in Theorem 11.2 takes the following form: u¯ε,1 = u˜ε,1 + 1 u˜ε,2 p˜ε,12 =

εα 2 2(ε α + ε β ) . + εγ (ε α + ε β ) ε(ε α + 2ε β ) ε α + eβ

(12.98)

¯ D,1 is satisfied and takes the following The asymptotic comparability condition L form: 1 u˜ ε,2

p˜ε,12

u¯ε,1 where

 ε(ε α + 2ε β )  −1 = 1 + α+γ α → u¯α,β,γ as ε → 0, ε (ε + ε β )

(12.99)

352

12 Generalisations of limit theorems for hitting times

u¯α,β,γ =

⎧ ⎪ 1 ⎪ ⎨1 ⎪

if α + γ < 1, 2 1 I(α > β) + I(α = β) + I(α < β) if α + γ = 1, 3 5 2

⎪ ⎪ ⎪0 ⎩

(12.100)

if α + γ > 1.

Recall that we are considering the case (i), where the parameter δ = min(α, β) + γ ∈ [0, 1). If α ≤ β, then δ = α + γ < 1, and, thus, u¯α,β,γ = 1. In this case, the probability p˜0,12 = 12 I(α = β) + 1I(α < β) > 0 and u¯ε,1 / 1 u˜ε,2 → p˜0,12 as ε → 0. Therefore, according to Theorems 11.2 and 10.4, the following equivalent relations take place: Eε,D,13 / u¯ε,1 → E¯0,D,13 = E¯0,D,23 = 1 as ε → 0,

(12.101)

Eε,D,13 / 1 u˜ε,2 → E0,D,13 as ε → 0

(12.102)

and where E0,D,13 =

1

2 if α = β, 1 if α < β.

(12.103)

According to the relation (12.102), the expectations of hitting times Eε,D,13 normalised by the function 1 u˜ε,2 (used in the corresponding weak convergence relations for distributions of hitting times, for the case where p˜0,12 > 0) converge to the first moment E0,D,13 of the corresponding limiting distribution G0,D,13 (·). If α > β, then δ = β + γ < 1, and thus, u¯α,β,γ = 1I(α + γ < 1) + 13 I(α + γ = 1)+0I(α+γ > 1). In this case, the probability p˜0,12 = 0, and u¯ε,1 /u˜ε,1 → (1−u¯α,β,γ )−1 as ε → 0. Therefore, according to Theorems 11.2 and 11.4, the following relations take place: Eε,D,13 /u¯ε,1 → E¯0,D,13 = (1 − u¯α,β,γ ) f˜0,13 p˜0,13 + u¯α,β,γ E¯0,D,23 = (1 − u¯α,β,γ ) + u¯α,β,γ = 1 as ε → 0,

(12.104)

and Eε,D,13 /u˜ε,1 → E¯0,D,13 (1 − u¯α,β,γ )−1 = (1 − u¯α,β,γ )−1 as ε → 0.

(12.105)

Three sub-cases should be considered. (a) If β + γ < 1 < α + γ, then u¯α,β,γ = 0. In this case, E¯0,D,13 = f˜0,13 p˜0,13 = E0,D,13 = 1 and E¯0,D,13 (1 − u¯α,β,γ )−1 = E0,D,13 = 1. Therefore, according to the relation (12.105), the expectations of hitting times Eε,D,13 normalised by the function u˜ε,1 (used in the corresponding weak convergence relation for distributions of hitting times, for the case where p˜0,12 = 0) converge to the first moment E0,D,13 for the corresponding limiting distribution G0,D,13 (·).

12.3 Numerical examples

353

(b) If β + γ < α + γ = 1, then u¯α,β,γ = 13 . In this case, E¯0,D,13 = 23 e˜0,13 p˜0,13 + 13 = + 13 = 1, while E¯0,D,13 (1 − u¯α,β,γ )−1 = f˜0,13 p˜0,13 + 12 = E0,D,13 + 12 = 32 . Therefore, according to the relation (12.105), the expectations of hitting times Eε,D,13 normalised by the function u˜ε,1 (used in the corresponding weak convergence relation for distributions of hitting times, for the case where p˜0,12 = 0) converge to the constant, which differs from the first moment E0,D,13 for the corresponding limiting distribution G0,D,13 (·). (c) If β + γ < α + γ < 1, then u¯α,β,γ = 1. In this case, E¯0,D,13 = 1, while E¯0,D,13 (1 − u¯α,β,γ )−1 = ∞. Therefore, according to the relation (12.105), the expectations of hitting times Eε,D,13 normalised by the function u˜ε,1 (used in the corresponding weak convergence relation for distributions of hitting times for the case where p˜0,12 = 0) converge to ∞. Similar calculations can be performed in the cases (ii) δ = 1 and (iii) δ ∈ (1, ∞). In the case δ = 1, the states 1 and 2 are equivalently asymptotically absorbing. Each state 1 or 2 can be used (in the procedure of one-step reduction of the phase space) in the above algorithm. In the case of δ ∈ (1, ∞), the state 2 is asymptotically less absorbing than the state 1. Thus, the state 2 should be used (in the procedure of one-step reduction of phase space) in the above algorithm. 2 3 E0,D,13

Appendix A

Limit Theorems for Randomly Stopped Stochastic Processes

In this appendix, we present some classical functional limit theorems for stochastic processes and limit theorems for randomly stopped stochastic processes, which are one of the main tools used to derive limit theorems for the first-rare-event times and the hitting times presented in the book.

A.1 Functional Limit Theorems for Càdlàg Stochastic Processes We refer to the books by Gikhman and Skorokhod (1971), Whitt (2002), Jacod and Shiryaev (2002), Silvestrov (2004), Billingsley (2014), and and Kallenberg (2021), where readers can find a detailed exposition of the theory of functional limit theorems for stochastic processes.

A.1.1 Convergence of Càdlàg Stochastic Processes in Topology U Let Dk be the space of càdlàg (possessing limits from the lefts and continuous from the right at any point) functions x(t) ¯ = (x1 (t), . . . , xk (t)), t ≥ 0, defined on the interval [0, ∞) and taking values in the space Rk . Let ξ¯ε (t) = (ξε,1 (t), . . . , ξε,k (t)), t ≥ 0, be, for each ε ∈ [0, 1], a càdlàg stochastic process (its trajectories belong to the space Dk ) defined on a probability space Ωε, Fε, Pε . Prokhorov (1956) deeply researched the natural U-topology in the space of continuous functions and gave necessary and sufficient conditions of U-convergence for continuous stochastic processes. Let us formulate them in a slightly extended form, for càdlàg processes defined on a semi-infinite interval. This extension is known due to works by Skorokhod (1956), Stone (1963), and Lindwall (1973): © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4

355

356

A Limit theorems for randomly stopped stochastic processes

d Z1 : (a) ξ¯ε (t), t ∈ U −→ ξ¯0 (t), t ∈ U as ε → 0, where U is some set dense in [0, ∞) and containing point 0, and ξ¯0 (t), t ≥ 0, is a continuous process, (b) limc→0 limε→0 P{ΔU (ξ¯ε (·), c, T) ≥ δ} = 0, δ > 0, for T > 0.

¯ c, T) is the modulus of U-compactness defined by the following Here, ΔU ( x(·), relation: ¯ c, T) = sup | x(t ¯ ) − x(t ¯ )|. (A.1) ΔU ( x(·), 0∨(t  −c)≤t  ≤t  ≤(t  +c)∧T

The symbol of convergence in distribution used above means that the random d vectors (ξ¯ε (tr ), r = 1, . . . , n) −→ (ξ¯0 (tr ), r = 1, . . . , n) as ε → 0, for any t1, · · · , tn ∈ U, n ≥ 1. Theorem A.1 The condition Z1 is necessary and sufficient for the fulfilment of the U following U-convergence relation, ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t), t ≥ 0, as ε → 0. Remark A.1 It is useful noting that the condition Z1 (b) is implied by the condition Z1 (a) in the case, where all components of processes ξ¯ε (t) are monotonic. In this case, the condition Z1 (a) is necessary and sufficient for the fulfilment of the U-convergence relation given in Theorem A.1. Here we just note that the main meaning of the U-convergence of càdlàg stochastic processes is that it is equivalent to the following relation of convergence in distribution: d (A.2) f (ξ¯ε (·)) −→ f (ξ¯0 (·)) as ε → 0, for any functional f (·) acting from the space Dk to R1 and U-continuous a.s. with respect to the measure generated by the limiting continuous process ξ¯0 (·) on the Borel σ-algebra Bk of the space Dk . We again refer to the books pointed out in the preamble to this section for the detailed presentation of the theory.

A.1.2 Convergence of Càdlàg Stochastic Processes in Topology J Skorokhod (1956) proposed a natural J-topology of convergence of càdlàg stochastic processes and gave necessary and sufficient condition for the J-convergence of càdlàgstochastic processes. This extension on the case of càdlàg stochastic processes defined on the interval [0, ∞) is known due to works by Stone (1963) and Lindvall (1973): d Z2 : (a) ξ¯ε (t), t ∈ U −→ ξ¯0 (t), t ∈ U as ε → 0, where U is some set dense in [0, ∞) and containing point 0, (b) limc→0 limε→0 P{ΔJ (ξ¯ε (·), c, T) ≥ δ} = 0, δ > 0, for T > 0.

A.1 Functional limit theorems

357

Here, ΔJ ( x(·), ¯ c, T) is the modulus of J-compactness defined by the following relation: ΔJ ( x(·), ¯ c, T) =

sup

0∨(t−c)≤t  ≤t ≤t  ≤(t+c)∧T

min(| x(t ¯ ) − x(t)|, ¯ | x(t) ¯ − x(t ¯ )|).

(A.3)

Theorem A.2 The condition Z2 is necessary and sufficient for the fulfilment of the J following J-convergence relation, ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t), t ≥ 0, as ε → 0. Remark A.2 J-convergence and U-convergence of càdlàg processes are equivalent, in the case where the limiting process ξ¯0 (t), t ≥ 0, is continuous. Here we just note that the main meaning of the J-convergence of càdlàg stochastic processes is that it is equivalent to the relation of convergence in distribution: d

f (ξ¯ε (·)) −→ f (ξ¯0 (·)) as ε → 0,

(A.4)

for any functional f (·) acting from the space Dk to R1 and J-continuous a.s. with respect to the measure generated by the limiting càdlàg process ξ¯0 (·) on the Borel σ-algebra Bk of the space Dk . The conditions of a.s. J-continuity for various functionals such as the value of a càdlàg function and the value of jump at a point; the time moment and the value of a large jump; the maximal jump, the number and the sum of large jumps in a time interval; the maximum and the minimum of càdlàg process on a time interval; the exceeding time and the over-jump for a given level; the integral, and other functionals can be found, for example, in Silvestrov (2004). We again refer to the books pointed out in the preamble to this section for the detailed presentation of the theory.

A.1.3 Convergence of Step-Sum Processes with Independent Increments in Topology J Here, we would like to formulate one important result for step-sum processes with independent increments:  (A.5) ξ¯ε (t) = ξ¯ε,r , t ≥ 0, r ≤tvε

where: (a) ξ¯ε,r , r = 1, 2, . . ., are i.i.d. random vectors and (b) 0 ≤ vε → ∞ as ε → 0. Let us assume that the following condition holds: d Z3 : ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t), t ≥ 0, as ε → 0.

Remark A.3 Since ξ¯ε,r = (ξε,r,1, . . . , ξε,r,k ), r = 1, 2, . . ., are i.i.d. random vectors, the condition Z3 is implied by the following simpler relation of convergence in d distribution, ξ¯ε (u) −→ ξ¯0 (u) as ε → 0, assumed to hold for some u > 0.

358

A Limit theorems for randomly stopped stochastic processes

As is well known, the limiting process ξ¯0 (t), t ≥ 0, is, in this case, a Lévy process (càdlàg homogeneous in time process with independent increments). The central criterion of convergence, which can be found, for example, in Loève (1977) and Skorokhod (1964, 1986) gives necessary and sufficient conditions for the fulfilment of the condition Z3 . The following theorem is also known, thanks to the works of Skorokhod (1957, 1964, 1986). J Theorem A.3 Let the condition Z3 be satisfied. Then, ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t), t ≥ 0, as ε → 0.

According to Theorem A.3, the convergence of step-sum processes ξ¯ε (t), t ≥ 0, in distribution implies their J-convergence without any additional assumptions.

A.2 Limit Theorems for Randomly Stopped Stochastic Processes and Superpositions of Stochastic Processes We refer to the book by Silvestrov (2004), where reader can find the detailed presentation of theory of limit theorems for randomly stopped stochastic processes.

A.2.1 Convergence in Distribution for Randomly Stopped Stochastic Processes Let ν¯ε = (νε,1, . . . , νε,k ) be, for each ε ∈ [0, 1], a random vector with non-negative components, which is defined on the same probability space Ωε, Fε, Pε as the càdlàg process ξ¯ε (t), t ≥ 0. The object of our interest are conditions of convergence in distribution for the random vectors: ζ¯ε = (ζε,1, . . . , ζε,k ) = (ξε,1 (νε,1 ), . . . , ξε,k (νε,k )).

(A.6)

We extend the condition Z2 (a) and assume that the following condition is satisfied: d Z4 : (ν¯ε, ξ¯ε (t)), t ∈ U −→ (ν¯0, ξ¯0 (t)), t ∈ U as ε → 0, where U is some set dense in [0, ∞) and containing point 0.

Unfortunately, the conditions Z2 (b) and Z4 are not enough to ensure the convergence in distribution for the random vectors ζ¯ε . Let us assume that the following continuity condition holds: Z5 : P{ξ0,r (ν0,r ) = ξ0,r (ν0,r − 0)} = 1, for r = 1, . . . , k.

A.2 Limit theorems for randomly stopped stochastic processes

359

The condition Z5 means that the stopping moment ν0,r is a point of continuity for the càdlàg process ξ0,r (t) with probability 1, for r = 1, . . . , k. It is worth noting that condition Z5 is automatically satisfied, if ξ¯0 (t), t ≥ 0, is a continuous process. Also, the condition Z5 holds, if ξ¯0 (t), t ≥ 0, is a stochastically continuous càdlàg process independent of the random vector ν¯0 . The following theorem is the simplified variant of results obtained in Silvestrov (1971c, 1972d, 1974, 2004). Theorem A.4 Let the conditions Z2 (b), Z4 , and Z5 be satisfied. Then, d ζ¯ε −→ ζ¯0 as ε → 0.

(A.7)

A.2.2 Convergence in Distribution for Superpositions of Càdlàg Processes Let ν¯ε (t) = (νε,1 (t), . . . , νε,k (t)), t ≥ 0, be a vector càdlàg stochastic process with non-negative non-decreasing components, which is defined on the same probability space Ωε, Fε, Pε as the càdlàg process ξ¯ε (t), t ≥ 0, for every ε ∈ [0, 1]. Consider also for ε ∈ [0, 1] a càdlàg stochastic process, which is a vector superposition of stochastic processes ξ¯ε (t), t ≥ 0, and ν¯ε (t), t ≥ 0: ζ¯ε (t) = (ζε,1 (t), . . . , ζε,k (t)) = (ξε,1 (νε,1 (t)), . . . , ξε,k (νε,k (t))), t ≥ 0.

(A.8)

The following condition is an analogue of the condition Z4 for stochastic processes: d Z6 : (ν¯ε (t), ξ¯ε (s)), (t, s) ∈ U × V −→ (ν¯0 (t), ξ¯0 (s)), U × V as ε → 0, where U and V are some sets dense in [0, ∞) and containing point 0.

Also, the following condition is an analogue of the condition Z5 for stochastic processes: Z7 : P{ξ0,r (ν0,r (t)) = ξ0,r (ν0,r (t) − 0)} = 1, for r = 1, . . . , k, t ∈ U. The following theorem is the simplified variant of results obtained in Silvestrov (1972d, 1974, 2004). Theorem A.5 Let the conditions Z2 (b), Z6 , and Z7 be satisfied. Then, d ζ¯ε (t), t ∈ U −→ ζ¯0 (t), t ∈ U as ε → 0.

(A.9)

A.2.3 Convergence of Superpositions of Càdlàg Processes in Topology J Let us also assume that the following condition holds:

360

A Limit theorems for randomly stopped stochastic processes

Z8 : ν¯0 (t), t ≥ 0, is a continuous process. The following theorem is the simplified variant of results obtained in Silvestrov (1972e, 1973, 1974, 2004). Theorem A.6 Let the conditions Z2 (b), Z6 , Z7 , and Z8 be satisfied. Then, J ζ¯ε (t), t ≥ 0 −→ ζ¯0 (t), t ≥ 0 as ε → 0.

(A.10)

A.3 Supplementary Asymptotic Results In this section we present some useful supplementary convergence results used in the book.

A.3.1 Slutsky Theorem and Related Results  , . . . , ξ  ) and ξ¯ = (ξ  , . . . , ξ  ) be, for every ε ∈ [0, 1], random Let ξ¯ε = (ξε,1 ε ε,1 ε,k ε,k vectors defined on a probability space Ωε, Fε, Pε and taking values in space Rk . Let us assume that the following condition holds: d d a.s Z9 : (a) ξ¯ε −→ ξ¯0 as ε → 0, (b) ξ¯ε −→ ξ¯0 as ε → 0, where ξ¯0 = const ∈ Rk .

The following useful theorem takes place. Theorem A.7 Let condition Z9 hold. Then, d (ξ¯ε , ξ¯ε) −→ (ξ¯0, ξ¯0) as ε → 0.

(A.11)

d The relation (A.11) obviously implies that random variables f (ξ¯ε , ξ¯ε) −→   ¯ y¯ ) acting from Rk × Rk to f (ξ¯0, ξ¯0 ) as ε → 0, for any continuous function f ( x, ¯ y¯ ) = x¯ + y¯ . Rk . For example, this relation holds for functions f ( x, Let, for ε ∈ [0, 1], the random vectors ξ¯ε+ = ξ¯ε + ξ¯ε. The following theorem follows from Theorem A.7 and the above remark.

Theorem A.8 Let the condition Z9 be satisfied. Then, the following relation holds: d a.s ξ¯ε+ −→ ξ¯0+ as ε → 0. In particular, if the random vector ξ¯0 = 0¯ = (0, . . . , 0), then +  ξ¯0 = ξ¯0 .

A.3.2 Functional Analogues of Slutsky Theorem Theorem A.7 has also an analogue for stochastic processes.

A.3 Supplementary asymptotic results

361

 (t), . . . , ξ  (t)), t ≥ 0, and ξ¯ (t) = (ξ  (t), . . . , ξ  (t)) be, Let ξ¯ε (t) = (ξε,1 ε ε,1 ε,k ε,k  for every ε ∈ [0, 1], vector stochastic processes defined on a probability space Ωε, Fε, Pε , whose components are real-valued processes defined on the interval [0, ∞). The following condition is an analogue of the condition Z9 for stochastic processes. d d Z10 : (a) ξ¯ε (t), t ≥ 0, −→ ξ¯0 (t), t ≥ 0, as ε → 0, (b) ξ¯ε(t), t ≥ 0 −→ ξ¯0(t), t ≥ 0,  as ε → 0, where ξ¯0 (t), t ≥ 0, is a.s. a non-random function.

The following theorem is an analogue of Theorem A.7. Theorem A.9 Let the condition Z10 be satisfied. Then, d (ξ¯ε (t), ξ¯ε(t)), t ≥ 0 −→ (ξ¯0 (t), ξ¯0(t)), t ≥ 0 as ε → 0.

(A.12)

Let, for ε ∈ [0, 1], the stochastic process ξ¯ε+ (t) = ξ¯ε (t) + ξ¯ε(t), t ≥ 0. The following useful theorem is a direct corollary of Theorem A.9. Theorem A.10 Let the condition Z10 be satisfied. Then, the following relations d a.s. ¯ holds: ξ¯ε+ (t), t ≥ 0 −→ ξ¯0+ (t), t ≥ 0, as ε → 0. In particular, if ξ¯0(t) = 0, t ≥ 0, +  then ξ¯ (t) = ξ¯ (t), t ≥ 0. 0

0

 (t), . . . , ξ  (t)), t ≥ 0, and ξ¯ (t) = (ξ  (t), . . . , ξ  (t)) be, for Let ξ¯ε (t) = (ξε,1 ε ε,1 ε,k ε,k  every ε ∈ [0, 1], càdlàg stochastic processes. Let as also assume that the following condition holds: J U Z11 : (a) ξ¯ε (t), t ≥ 0 −→ ξ¯0 (t), t ≥ 0, as ε → 0, (b) ξ¯ε(t), t ≥ 0 −→ ξ¯0(t), t ≥ 0, as ε → 0, where ξ¯0(t), t ≥ 0, is a non-random continuous function.

The following useful theorem is a functional analogue of Theorem A.10. J Theorem A.11 Let the condition Z11 be satisfied. Then, ξ¯ε+ (t), t ≥ 0 −→ ξ¯0+ (t), t ≥ 0, ¯ t ≥ 0, then ξ¯+ (t) = ξ¯ (t), t ≥ 0. as ε → 0. In particular, if ξ¯0(t) = 0, 0 0

The proofs of Theorems A.7–A.11 as well as more general propositions can be found, for example, in Silvestrov (2004).

Appendix B

Methodological and Bibliographical Notes

This appendix contains methodological and bibliographic notes and comments on the new results presented in Volume 1. Some promising issues for future research in this area are also commented on.

B.1 Methodological Notes B.1.1 A Survey of Works Related to Results Presented in the Book. In this subsection, we briefly comment on works related to the results presented in Volume 1. Volume 1 is devoted to limit theorems for first-rare-event and hitting times for perturbed finite semi-Markov processes. Random functionals similar to first-rare-event times and hitting times are also known by names such as first passage times, absorption times, in theoretical studies, as well as lifetimes, first failure times, extinction times, etc., in applications. Limit theorems for such functionals for perturbed Markov-type processes have been studied by many researchers. The most deeply studied is the case of perturbed Markov chains and semi-Markov processes with finite phase spaces. The relevant works can be divided into two groups. Regularly perturbed models with one class of communicative states for the corresponding limiting embedded Markov chains were studied in the works of Harris (1952), Belyaev (1963), Keilson (1966b, 1974, 1979, 1998), Darroch and Morris (1967), Korolyuk (1969), Silvestrov (1969b, 1970a, b , 1971a, b, 1972c, 1974, 1980a, 2016, 2019), Anisimov (1971a), Holst (1971, 1977, 1979), Koroljuk, Penev, and Turbin (1972, 1973), Masol and Silvestrov (1972), Zakusilo (1972a, b), Kovalenko (1973), Zubkov, and Miha˘ılov (1974), Masol (1974, 1976), Courtois (1975), Aldous (1982), Anisimov and Chernyak (1982), Korolyuk, D. (1983), Brown and Guang Ping (1984), Gut and Holst (1984), Aldous and Brown (1992, 1993), Asmussen (1994), Gyllenberg and Silvestrov (1994, 1998, 1999, 2008), Silvestrov and Drozdenko (2005, 2006a, b), Drozdenko (2007a, b ,2009), Benois, Landim, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4

363

364

B Methodological and bibliographical notes

and Mourragui (2013), Griffiths, Kang, Oliveira, and Patel (2014), Peres and Sousi (2015), Nardi, Zocca, and Borst (2016), and Mao Yong-Hua and Zhang Chi (2017). Singularly perturbed models with several closed classes of communicative states for the corresponding limiting embedded Markov chains were studied in the works of Meshalkin (1958), Simon and Ando (1961), Hanen (1963d), Anisimov (1971b, 1988, 2008), Korolyuk and Turbin (1970, 1976a), Gusak and Korolyuk (1971), Kovalenko (1975, 1976), Ga˘ıtsgori and Pervozvanskiy(1975), Louchard and Latouche (1982), Coderch,Willsky, Sastry, and Castañon (1983), Delebecque, Quadrat, and Kokotović (1984), Cao and Stewart (1985), Rohlichek (1987), Rohlicek and Willsky (1988), Latouche (1991), Schweitzer (1991), Stewart (1994, 1998, 2001, 2003, and Yin and Zhang (2003, 2005, 2013). Random functionals such as first-rare-event times and hitting times are natural generalisations of geometric type random sums. That is why, limit theorems for such random functionals are closely related to limit theorems for geometric type random sums and first-rare-event times for regenerative processes and related counting processes, which were the subject of research in works listed below in Sect. B.2.3. There exists also an extensive bibliography of works listed below in Sect. B.2.4, where limit theorems for hitting type functionals have been studied for perturbed Markov chains and semi-Markov type processes with countable and arbitrary phase spaces. In Part I we present necessary and sufficient conditions for convergence in distribution for first-rare-event times and J-convergence for first-rare-event processes and the corresponding rare-event counting processes for regularly perturbed semiMarkov processes. The main feature of most of the previous results is that they provide sufficient conditions of convergence for such functionals. As a rule, these conditions include assumptions that imply weak convergence for the distributions of sums of i.i.d. random variables distributed as the sojourn times for the semi-Markov processes (for each state) to some infinitely divisible laws plus some ergodicity conditions for the embedded Markov chains plus the condition of vanishing probabilities of a rare event occurring during one transition step for the semi-Markov processes. The originating works containing such limit theorems for hitting times and similar stopping times for Markov chains and semi-Markov processes are Harris (1952), Meshalkin (1958), Gnedenko and Kovalenko (1964), Keilson (1966b), Solov’ev (1966), and and Korolyuk (1969). The most general sufficient conditions of this kind for convergence in distribution and convergence in topologies U and J for sum processes of real-valued random variables defined on regularly perturbed finite Markov chains stopped at hitting, renewal, and more general Markov-type moments can be found in Silvestrov (1970a, b, 1974). The results presented in Part I relate to the model of regularly perturbed semiMarkov processes with a finite phase space. Instead of conditions based on “individual” distributions of sojourn times, we use more general and weaker conditions imposed on distributions of sojourn times averaged over the stationary distributions of the corresponding embedded Markov chains. Moreover, we show that these conditions are not only sufficient but also necessary conditions for convergence in

B.1 Methodological notes

365

distribution of first-rare-event times and convergence in the Skorokhod J-topology of rare-event processes and the corresponding rare-event counting processes. These results are also illustrated by necessary and sufficient conditions of weak convergence for non-ruin distribution functions for perturbed risk processes in the models of diffusion and stable approximations as well as necessary and sufficient conditions of convergence in distribution of first-rare-event times for some perturbed M/M-type queuing systems. The approach used in the book combines traditional methods based on cyclic representations of first-rare-event times in the form of geometric type random sums and general limit theorems for randomly stopped stochastic processes, developed in the works of Silvestrov (1971c, 1972c, d, e, 1973, 1974, 2004). In the context of necessary and sufficient conditions of convergence in distribution for first-rare-event times, we would like also to mention the books Gnedenko and Korolev (1996) and Bening and Korolev (2002), where one can find some related results for geometric sums of random variables as well as the articles Korolyuk, D. and Silvestrov (1983, 1984) and Silvestrov and Veliki˘ı (1988), where one can find some related results for similar functionals defined on Markov chains with arbitrary phase spaces. In Part II we present new asymptotic recurrent algorithms of phase space reduction for regularly and singularly perturbed finite semi-Markov processes, efficient conditions for weak convergence of distributions and convergence of expectations for hitting times for regularly and singularly perturbed finite semi-Markov processes, and recurrent formulas for finding the corresponding normalisation functions, Laplace transforms for limiting distributions, and limits for expectations of hitting times. Aggregation/disaggregation is one of the most effective and widely used approaches in the study of asymptotics for hitting times and similar functionals, especially for singularly perturbed models of Markov chains and semi-Markov processes. Selected works containing relevant asymptotic results for finite Markov chains and semi-Markov processes are Gambin, Krzyżanowski, and Pokarowski (2008), Marek, Mayer, and Pultarová (2009), Simon and Ando (1961), Anisimov (1971a, b, 1988, 2008), Courtois (1975), Ga˘ıtsgori and Pervozvanskiy (1975, 1983), Korolyuk and Turbin (1978), Vo˘ına (1979), Louchard and Latouche (1982), Coderch, Willsky, Sastry, and Castañon (1983), Chatelin and Miranker (1984), Delebecque, Quadrat, and Kokotović (1984), Cao and Stewart (1985), Haviv (1987, 1992, 1999), Rohlichek (1987), Rohlicek and Willsky (1988), Latouche (1991), Schweitzer (1984, 1991), Stewart (1994, 1998, 2001, 2003), Marek and Mayer (1998), Yin, Zhang, and Badowski (2000c, 2003), Yin and Zhang (2003, 2005, 2013), Korolyuk and Limnios (2005), Marek and Pultarová (2006), Gambin, Krzyżanowski, and Pokarowski (2008), and Marek, Mayer, and Pultarová (2009). This approach is also widely used in limit theorems for hitting type functionals for perturbed Markov chains and semi-Markov type processes with countable and general phase spaces listed below in Sect. B.2.4, works containing results on rates of convergence and asymptotic expansions for distributions of hitting type functionals listed below in Sect. B.2.5, as well as in works containing asymptotic results for other functionals such as moments of hitting times, stationary and quasi-stationary

366

B Methodological and bibliographical notes

distributions, eigenvalues, eigenvectors, Perron roots, coefficients of ergodicity, etc., which are listed below in Sect. B.2.6. In Part II, we use the aggregation/disaggregation approach in a stochastic semiMarkov setting. Instead of algebraic reduction of the corresponding systems of linear equations, we use algorithms of phase space reduction for the corresponding semiMarkov processes based on the aggregation of their transition times. In such an algorithm, successive moments of hitting into new reduced phase spaces play the role of successive moments of jumps for new reduced semi-Markov processes. We also found natural perturbation conditions for the initial transition characteristics of the corresponding perturbed semi-Markov processes, which are inherited by the reduced semi-Markov processes. The above asymptotic recurrent phase space reduction algorithm is based on the recurrent alternating application of two procedures: the removal of virtual transitions and one-step reduction of phase space for singularly perturbed semi-Markov processes. The first procedure provides a kind of asymptotic compression of the transition times and allows one to explicitly transform the initial normalisation functions and effectively compute the corresponding weak limits for distribution functions of new transition times for semi-Markov processes with removed virtual transitions. The second procedure allows one to reduce the phase space of the original semi-Markov processes and effectively compute the corresponding weak limits for distribution functions of transition times and the limiting transition probabilities of embedded Markov chains for the reduced semi-Markov processes. The above procedures preserve hitting times between states from the reduced phase spaces, that is, these times and, therefore, their distributions are the same for the original and the reduced semiMarkov processes. They allow one to effectively compute normalisation functions, weak limits for distributions of hitting times, and limits for their expectations for singularly perturbed semi-Markov processes as well as to track “switching” parameters determining the forms of these normalisation functions, weak limits for distributions of hitting times, and limits for their expectations. Semi-Markov processes independently introduced by Lévy (1954), Smith (1955), and Takács (1954) naturally generalise discrete and continuous time Markov chains. These processes are a very flexible and efficient tool for analysing queuing, reliability and control systems, bio-stochastic systems, information networks, financial and insurance processes, and many other stochastic systems and processes. The algorithms of phase space reduction presented in the book give one more example of effectiveness of the semi-Markov setting, which is an adequate and, moreover, a necessary element of the proposed method. Even in the case, where the initial process is a discrete or continuous time Markov chain, the procedure of phase space reduction leads to a semi-Markov process, since the times between successive hitting into the reduced phase space by the original process have distributions that can be neither geometric nor exponential. Particular attention is paid to finding conditions under which the distribution functions of properly normalised hitting times for perturbed semi-Markov processes converge weakly to some distribution functions that are not concentrated at zero, and at the same time, expectations of hitting times normalised by the same functions con-

B.1 Methodological notes

367

verge to the first moments of the corresponding limiting distribution functions. The point is that this is not always true for singularly perturbed semi-Markov processes. At the same time, such simultaneous convergence plays an important role in applications to ergodic theorems for perturbed semi-Markov-type processes, in particular, to ergodic theorems for perturbed alternating regenerative processes modulating semi-Markov processes, which are the subject of research in Volume 2. In this context, we would like to mention works of Hatori (1959, 1960), Leadbetter (1963), Silvestrov (1980a, b, 1996a, b, 2000a, 2007, 2010), Kohlas (1983), Abadov (1984), Gaver, Jacobs and Latouche (1984), Alsmeyer and Irle (1986), Silvestrov and Abadov (1984, 1991, 1993), Gyllenberg and Silvestrov (1998, 1999, 2008), Avrachenkov and Lasserre (1999), Avrachenkov and Haviv (2004), Hunter (2005), Gambin, Krzyżanowski, and Pokarowski (2008), Avrachenkov, Filar and Howlett (2013), Griffiths, Kang, Oliveira, and Patel (2014), Silvestrov,Manca, and Silvestrova (2014), Petersson (2016a, c), Silvestrov, D. and Silvestrov, S. (2016, 2017a, b, c, d), and Silvestrov and Manca (2017a, b) devoted to the study of moments of hitting times and related functionals, in particular stationary probabilities, for Markov-type processes including the corresponding asymptotic problems for perturbed processes. Here, we have the opportunity to pay special attention to some of the latest works, where a deep asymptotic analysis for such functionals was carried out for regularly and singularly perturbed Markov chains and semi-Markov processes, based on asymptotically comparable power-type functions, in Gambin, Krzyżanowski, and Pokarowski (2008), Laurent series, in Avrachenkov, Filar and Howlett (2013), and Laurent asymptotic expansions without and with explicit upper bound for reminders, in Silvestrov, D. and Silvestrov, S. (2017c). B.1.2 New Results Presented in the Book. In this section, we comment on the new results presented in Volume 1. The main achievement of Part I is the necessary and sufficient conditions for convergence in distribution for first-rare-event times and processes and the corresponding rare-event counting processes. These conditions do not have a gap between their necessary and sufficient parts and give a kind of “final solution” for the limit theorems for the first-rare-event times and processes for regularly perturbed semiMarkov processes with finite phase spaces. These results generalise and improve the results concerning necessary and sufficient conditions for convergence in distribution for first-rare-event times for semiMarkov processes obtained in papers by Silvestrov and Drozdenko (2005, 2006a, 2006b), and Drozdenko (2007a, 2007b, 2009). First, weaker ergodic conditions for the embedded Markov chains are proposed. Instead of the traditional assumption about the convergence of transition probabilities to transition probabilities of some ergodic Markov chain, a new condition of asymptotically uniform ergodicity is used. Second, new proofs based on general limit theorems for stopping random processes, developed and presented in detail in Silvestrov (2004), are used instead of more traditional proofs based on cyclical representations of the first-rare-event times in the form of geometric-type random sums. This made it possible to formulate the results in a more general form of the

368

B Methodological and bibliographical notes

corresponding functional limit theorems as well as to clarify the “random stopping” stricture of limiting random variables for the first-rare-event times. The main limit theorems for the first-rare-event times and processes are presented in Chap. 2, which is an extended version of article Silvestrov (2016), and for rareevent counting processes in Chap. 3. Applications to perturbed risk processes given in Chap. 5 represent the corresponding results from Silvestrov and Drozdenko (2006b). Generalisations of the limit theorems given in Chap. 2 to the vector first-rare-event times and rewards, the upper and low first-rare-event times, first-rare-event times for Markov renewal processes with transition period and extending phase spaces, as well as standard and directed hitting times given in Chap. 4, and the examples of applications to perturbed queuing systems given in Chaps. 6 and 7, are new results. I would like to add comments related to the applications presented in Chaps. 5–7. The idea was to illustrate the theoretical results presented in the previous chapters by applying them to some classical models and to show that even for such models new results can be obtained. Namely, the necessary and sufficient conditions in their final form without gaps between their necessary and sufficient parts can be obtained in the corresponding limit theorems. Possible generalisations for more complex models are beyond the scope of this book. In Part II, the aforementioned aggregation/disaggregation approach is used for constructing new asymptotic recurrent algorithms of phase space reduction and applying them to study of asymptotics for the hitting times for regularly and singularly perturbed finite semi-Markov processes. The main achievement of Part II is the construction of new asymptotic recurrent algorithms of phase space reductions based on stochastic aggregation of transition times for perturbed semi-Markov processes. The key role here is played by new balanced forms of perturbation conditions imposed on transition characteristics of the original semi-Markov processes. These conditions are inherited by the reduced semi-Markov processes and are supplemented by effective recurrent formulas for finding transition characteristics of these reduced semi-Markov processes. The proposed asymptotic recurrent algorithms are computationally efficient for models with regular and singular perturbations. They have a clear recurrent form well-prepared, for example, for efficient programming. In this book, we continue the direction of research based on recurrent phase space reduction algorithms for semi-Markov processes, proposed in the article Silvestrov and Manca (2017), where a forward recurrent algorithm for computing the Laplace transforms of hitting times was presented, papers Silvestrov, D. and Silvestrov, S. (2016, 2017a, b, d) and book Silvestrov, D. and Silvestrov, S. (2017c), where the recurrent algorithms of phase space reduction combined with the techniques of Laurent asymptotic expansions are used for obtaining asymptotic expansions for moments of hitting times and stationary distributions for regularly and singularly perturbed semi-Markov processes. In this book, the asymptotic recurrent algorithms for singularly perturbed semiMarkov processes are applied for solving a different problem, namely, obtaining weak asymptotics for distributions of hitting times. In the above works, the polynomial type perturbation models were researched. In this book, new type of perturbation

B.1 Methodological notes

369

conditions, based on the notion of complete families of asymptotically comparable functions, is proposed and thoroughly investigated. In particular, perturbation models can be based on general power functions and their natural combinations with exponential and logarithmic functions. Also, the forward recurrent algorithms of phase space reduction are supplemented by new backward recurrent algorithms, which allow one recurrently compute normalisation functions and Laplace transforms for limiting distributions of hitting times for arbitrary initial states. Finally, the analogous asymptotic results, based on more general perturbation conditions connected with the notion of complete families of asymptotically comparable functions, are obtained for expectations of hitting times. Part II is a significantly expanded version of the article Silvestrov (2019). New asymptotic recurrent algorithms of phase space reduction for singularly perturbed semi-Markov processes and limit theorems for hitting times and their expectations presented in Part II play a key role in obtaining ergodic theorems for perturbed regenerative type processes modulated by regularly and singularly perturbed semi-Markov processes. Such ergodic theorems are the subject of research in Volume 2 of this monograph. The key role in such theorems is played by the asymptotics of the distributions and expectations of the hitting times (in particular, the return times) for the modulating semi-Markov processes. It is important that it was possible to obtain convergence conditions for the hitting times, which are based on the minimal convergence conditions for the transition characteristics of perturbed semi-Markov processes typical for ergodic theorems, and allows one to effectively compute normalisation functions, limiting distributions, expectations, and other characteristics determining the corresponding stationary distributions. In conclusion it worth mentioning that continuous and discrete time Markov chains are particular cases of semi-Markov processes. That is why all limit theorems for perturbed semi-Markov processes presented in the book have obvious simplified analogues for regularly and singularly perturbed continuous and discrete time Markov chains. B.1.3 New Problems. In this subsection, we formulate some new open problems related to the results presented in the book. Part I presents necessary and sufficient conditions (without a gap between the necessary and sufficient parts) in limit theorems on convergence in distribution for the first-rare-event times and convergence in the Skorokhod J-topology for the rareevent processes and the counting processes generated by flows of rare events for the regularly perturbed finite semi-Markov processes. It should be noted that obtaining necessary and sufficient conditions for convergence without a gap between their necessary and sufficient parts is usually a difficult task. It would be natural to try to generalise these results to perturbed semi-Markov-type processes with countable and general phase spaces. It would also be interesting to try to obtain necessary and sufficient conditions for convergence similar to those presented in Part I (based on the averaged characteristics of the corresponding semi-Markov processes) for the model of the first-rare-event reward functionals and processes with real-valued random summands for the case

370

B Methodological and bibliographical notes

where the corresponding “external” limiting process is a general Lévy process with a Gaussian component. A more ambitious open problem is related to finding similar necessary and sufficient convergence conditions in limit theorems for hitting times for singularly perturbed semi-Markov processes. There is no doubt that the necessary and sufficient conditions for weak convergence of distribution functions without ruin, given in Chap. 5, can be generalised to more general models of risk processes, for example, risk processes with Markov-type modulation of input claim flows, risk processes with renewal type input claim flows, etc. In Chaps. 6 and 7, some examples of necessary and sufficient conditions for the convergence in distribution for the first-rare-event times for perturbed M/M-type queuing systems are given. These results can certainly be generalised to perturbed M/G queuing systems and other models of perturbed queuing systems. Part II presents phase space reduction algorithms and limit theorems for hitting times for singularly perturbed finite semi-Markov processes for the case when the time of reaching has finite expectations. In Chap. 12, we present some generalisation of the model in which the transition time distributions belong to the domains of attraction of stable laws. Our hypothesis is that these results can be generalised to the case where the transition times belong to the regions of attraction of general infinitely divisible laws. It would be natural to try to generalise the limit theorems for the hitting times for singularly perturbed semi-Markov processes, presented in Part II, to a more general form of the functional limit theorems. Recurrent algorithms for obtaining asymptotic expansions with explicit estimates of the remainder terms for the hitting times for regularly and singularly perturbed semi-Markov processes, presented in the articles and the book of Silvestrov D. and Silvestrov S. (2017a, b, c, d), give hope that similar results with explicit estimates of the convergence rates can be obtained for distributions of the hitting times.

B.2 General Bibliographical Remarks In this section, we briefly present a general bibliography of papers in the field of asymptotic problems for hitting times and related functionals, as well as for perturbed Markov, semi-Markov, and regenerative processes. These works can be divided into several groups. B.2.1. The first works related to asymptotic problems for perturbed Markov and semi-Markov processes are Harris (1952), Meshalkin (1958), Simon, and Ando (1961), Belyaev (1963), Hanen (1963a, b, c, d), Kingman (1963), Kovalenko (1965, 1973, 1975), Keilson (1966a, b, 1974), Seneta (1967, 1968, 1973), Korolyuk (1969), Silvestrov (1969a, b, c, 1970a, b, 1971a, b, d, 1972a, b, c, 1974), Korolyuk and Turbin (1970), Anisimov (1971a, b, 1973a, b), Gusak and Korolyuk (1971), Turbin

B.2 General bibliographical remarks

371

(1971), Korolyuk, Penev, and Turbin (1972), Poliščuk and Turbin (1973), Korolyuk, Brodi, and Turbin (1974), Courtois (1975), and Ga˘ıtsgori and Pervozvanskiy (1975). B.2.2. Limit theorems for distributions of hitting times and related problems for regularly and singularly perturbed finite Markov chains and semi-Markovtype processes were studied in Harris (1952), Meshalkin (1958), Simon and Ando (1961), Belyaev (1963), Hanen (1963d), Keilson (1966b, 1974, 1979, 1998), Darroch and Morris(1967), Korolyuk (1969), Silvestrov (1969b, 1970a, b, 1971a, b, d, 1974, 1980a, 2000a, 2016, 2019), Anisimov (1971a, b, 1988, 2008), Korolyuk and Turbin (1970, 1976a), Gusak and Korolyuk (1971), Holst (1971, 1977, 1979), Turbin (1971), Koroljuk, Penev, and Turbin (1972, 1973), Masol and Silvestrov (1972), Zakusilo (1972a, b), Kovalenko (1973, 1975, 1976), Zubkov, and Miha˘ılov (1974), Masol (1974, 1976), Courtois (1975), Ga˘ıtsgori and Pervozvanskiy (1975), Aldous (1982), Anisimov and Chernyak (1982), Louchard and Latouche (1982), Coderch, Willsky, Sastry, and Castañon (1983), Korolyuk, D. (1983), Brown and Guang Ping (1984), Delebecque, Quadrat, and Kokotović (1984), Gut and Holst (1984), Cao and Stewart (1985), Rohlichek (1987), Rohlicek and Willsky (1988), Latouche (1991), Schweitzer (1991), Aldous and Brown (1992, 1993), Asmussen (1994), Gyllenberg and Silvestrov (1994, 1998, 1999a, 2008), Stewart (1994, 1998, 2001, 2003), Silvestrov and Drozdenko (2005, 2006a, b), Drozdenko (2007a, b, 2009), Benois, Landim, and Mourragui (2013), Kartashov (2013a, 2015), Griffiths, Kang, Oliveira, and Patel (2014), Peres and Sousi (2015), Nardi, Zocca, and Borst (2016), Butler (2017), Mao Yong-Hua and Zhang Chi (2017), Petersson (2017), Clancy and Tjia (2018), Hössjer, Bechly, and Gauger (2018), Kumar and Gupta (2020), Lopes and Luczak (2020), and Cator and Don (2021). B.2.3. Random functionals such as the first-rare-event times and the hitting times can be considered as natural generalisations of geometric-type random sums. That is why, limit theorems for such random functionals are closely connected with limit theorems for geometric-type random sums and the first-rare-event times for regenerative processes and related counting processes that were a subject of thorough studies in Rényi (1956), Belyaev (1963), Kovalenko (1965), Keilson (1966b, 1979), Gnedenko and Kovalenko (1964), Gnedenko and Frayer (1969), Silvestrov (1969a, b, 1972c, 1974, 1980a, 1995, 2000b, 2004, 2014a), Masol (1973, 1974), Solov’ev (1966, 1971), Jagers and Lindvall (1974), Kovalenko and Kuznetsov (1981, Korolyuk, D. (1982, 1983), Kalashnikov (1983, 1986, 1990, 1994b, 1997), Khusanbaev (1983, 1984), Kalashnikov and Vsekhsvyatskii (1985), Vsekhsvyatskii and Kalashnikov (1988), Hsu Guanghui and He Qiming (1990), Kruglov and Korolev (1990), Kartashov (1991), Gnedenko and Korolev (1996), Englund and Silvestrov (1997), Gyllenberg and Silvestrov (1999b, 2000a, b, 2008). Englund (2001), Bening and Korolev (2002), Silvestrov and Teugels (2004), Silvestrov and Drozdenko (2005, 2006a, b), Blanchet and Glynn (2007), Drozdenko (2007a, b, 2009), Glynn (2011), Ni (2011, 2014), Asmussen and Kortschak (2013), Asmussen and Foss (2014), Petersson (2014, 2016c), Korolev (2016), Clément and Landy (2017), Kushnir, O. and Kushnir, V. (2017), and Slepov (2021). B.2.4. Similar asymptotic problems for Markov chains and processes of Markov type with a countable and general phase space were studied in Gaver (1964, 1976),

372

B Methodological and bibliographical notes

Keilson (1966a), Silvestrov (1969a, b, c, 1972a, b, c, 1974, 1979, 1980a, 1981), Gusak and Korolyuk (1971), Basu (1972), Anisimov (1973a, b, 1980, 1986, 1988, 2008), Solov’ev (1972), Gut (1974a, b, 1975, 1988), Mileshina (1975), Chow and Hsiung (1976), Korolyuk and Turbin (1976b, 1978), Kalashnikov (1978a, b, 1981, 1986, 1994a), Vo˘ına (1978, 1979), Kaplan (1979, 1980), Chow, Hsiung, and Yu (1980), Wang Zi Kun (1980), Kalashnikov and Anichkin (1981), Janson (1983), Bobrova (1983), Korolyuk, D. and Silvestrov (1983, 1984), Abadov (1984), Lalley (1984), Kartashov (1986, 1987, 1996, 2013b), Martin-Löf (1986), Silvestrov and Veliki˘ı (1988), Aldous (1989), Korolyuk, D. (1990), Silvestrov and Abadov (1991, 1993), Korolyuk and Swishchuk (1995a, b), Sevast’yanov (1995, 1998, 2000, 2006), Motsa and Silvestrov (1996), Korolyuk (1997), Borovkov (1998a, b, 2015, 2016), Gasanenko (1999, 2001), Kesten and Maller (1999), Korolyuk, V.S. and Korolyuk, V.V. (1999), Asmussen (1998, 2000), Korolyuk and Limnios (2000, 2002, 2005, 2010), Alsmeyer and Hoefs (2001, 2004), Kupsa and Lacroix (2005), Jagers, Klebaner, and Sagitov (2007), Goldsheid (2008), Asmussen and Albrecher (2010), Aliyev, Rahimov, and Navidi (2011), Löpker and Stadje (2011), Glynn (2011), Bulinskaya (2012), Limnios (2012), Asmussen and Kortschak (2013), Serlet (2013, 2018), Fernandez, Manzo, Nardi, and Scoppola (2015), Vakhtel’ and Denisov (2015), Vysotsky (2015), Afanasyev (2016, 2018), Alsmeyer and Marynych (2016), Bertoin and Kortchemski (2016), Fernandez, Manzo, Nardi, Scoppola, and Sohier (2016), Sun, Feuillet, and Robert (2016), Mao Yong-Hua and Zhang Chi (2017), Pène, Saussol, and Zweimüller (2017), Schulte-Geers and Stadje (2017), van Doorn (2017), Zakusilo and Matsak (2017), Denisov, Sakhanenko, and Wachtel (2018), D’Onofrio, Macci, and Pirozzi (2018), Ragimov, Farhadova, and Khalilov (2018), Aliev, Ragimov, Farhadova, and Khalilov (2019), Sloothaak,Wachtel, and Zwart (2018), Helali and Löwe (2019), Kuntz, Thomas, Stan, and Barahona (2019), Rahimov, Ibadova, and Farhadova (2019), Zweimüller (2019), Ascione, Pirozzi, and Toaldo (2020), Chatrabgoun, Daneshkhah, and Parham (2020), Labbé and Lacoin (2020), Grama, Lauvergnat, and Le Page (2020), and Buck (2021). B.2.5. Convergence rates, asymptotic expansions in weak and large deviations limit theorems for distributions of hitting type functionals for Markov-type processes with finite, countable, and general phase spaces have been studied in Silvestrov (1969a, 1969b, 1970a, 1970b, 1971b, 1972b, 1995, 2000, 2007, 2010, 2014a), Korolyuk and Turbin (1970, 1976a, b, 1978), Gusak and Korolyuk (1971), Turbin (1971), Koroljuk, Penev, and Turbin (1972, 1973, 1981), Poliščuk and Turbin (1973, Courtois (1975), Kalashnikov (1981, 1983, 1986, 1990, 1994a, b, 1997), Kalashnikov and Anichkin (1981), Aldous (1982), Ahmad and Basu (1983), Bobrova (1983), Abadov (1984), Cao and Stewart (1985), Kalashnikov and Vsekhsvyatskii (1985), Kartashov (1986, 1987, 1991, 1996, 2013b), Vsekhsvyatskii and Kalashnikov (1988), Stewart and Sun (1990), Sudakova (1990), Silvestrov and Abadov (1991, 1993), Schweitzer (1991), Aldous and Brown (1992, 1993), Stewart (1994, 1998, 2001, 2003), Englund and Silvestrov (1997), Korolyuk (1997), Gyllenberg and Silvestrov (1998, 1999, 2000, 2008), Roberts, Rosenthal, and Schwartz (1998), Gasanenko (1999, 2001), Korolyuk, V.S. and Korolyuk, V.V. (1999), Bon and Kalashnikov (2001), Englund (2001), Aliyev and Abadov (2007), Ni (2011, 2014),

B.2 General bibliographical remarks

373

Ferré, Hervé, and Ledoux (2013), Griffiths, Kang, Oliveira, and Patel (2014), Petersson (2014, 2016c), Peres and Sousi (2015), Gast and Gaujal (2017), and Matsak (2021). B.2.6. Asymptotic results including convergence, estimates for convergence rates, and asymptotic expansions for other related functionals such as moments of hitting times, stationary and quasi-stationary distributions, eigenvalues, eigenvectors, Perron roots, coefficients of ergodicity, etc., were studied in Hanen (1963a, b, c, d), Seneta (1967, 1968a, b, 1973, 1980, 1988a, b, 1991, 1993, 2006), Schweitzer (1968, 1984, 1986, 1987, 1991), Silvestrov (1969b, 2000, 2007, 2010), Stewart (1969, 1973, 1979, 1983, 1984a, b, 1990, 1991, 1993a, b, 1994, 1998, 2001, 2003), Turbin (1972), Golub and Seneta (1973, 1974), Gut (1974a, b, 1988), Courtois (1975, 1982), Ga˘ıtsgori and Pervozvanskiy (1975, 1983), Kovalenko (1975, 1976), Courtois and Louchard (1976), Korolyuk and Turbin (1976b, 1978), Allen, Anderssen and Seneta (1977), Kalashnikov (1978b, 1986, 1990, 1994a), Latouche and Louchard (1978), Meyer (1980, 1994, 2015), Tweedie (1980, 1998), Kalashnikov and Anichkin (1981), Delebecque (1983), Kohlas (1983), Chatelin and Miranker (1984), Courtois and Semal (1984a, b, c, 1991), Gaver, Jacobs and Latouche (1984), Haviv and Rothblum (1984), Haviv and Van der Heyden (1984), Koury, McAllister and Stewart (1984), McAllister, Stewart, G., and Stewart, W. (1984), Silvestrov and Abadov (1984, 1991, 1993), Cao and Stewart (1985), Funderlic and Meyer (1985), Kartashov (1985a, b, c, 1996), Vantilborgh (1985), Alsmeyer and Irle (1986), Hunter (1986, 1991a, b, 2005), Haviv (1986, 1987, 1988, 1992, 1999, 2006), Haviv and Ritov (1986, 1993), Burnley (1987), Gibson and Seneta (1987), Haviv, Ritov and Rothblum (1987); Haviv, Ritov, and Rothblum (1992), Rohlichek (1987), Meyer and Stewart (1988), Rohlicek and Willsky (1988), Sumita and Reiders (1988), Louchard and Latouche (1990), Stewart and Sun (1990), Stewart and Zhang (1991), Borovkov, Fayolle, and Korshunov (1992), Hassin and Haviv (1992), Meyn and Tweedie (1992, 1993a, b, 2009), Barlow (1993a, b), O’Cinneide (1993), Lasserre (1994), Pollett and Stewart (1994), Stewart, G., Stewart, W., and McAllister (1994), Ele˘ıko and Shurenkov (1996), Hoppensteadt, Salehi, and Skorokhod (1996), Cao (1998), Gyllenberg and Silvestrov (1998, 1999, 2000, 2008), Kovalenko, Birolini, Kuznetsov, and Shenbukher (1998), Marek and Mayer (1998), Yin and Zhang (1998, 2003, 2005, 2013), Roberts, Rosenthal, and Schwartz (1998), Avrachenkov (1999, 2000), Avrachenkov and Lasserre (1999), Yin, Zhang, and Badowski (2000a, b, c, 2003), Yin, G., Zhang, Yang, and Yin, K. (2001), Avrachenkov, Filar, and Haviv (2002), Craven (2003), Mitrophanov (2003, 2005a, b, 2006), Altman, Avrachenkov, and Núñez-Queija (2004), Avrachenkov and Haviv (2004), Zhang and Yin (2004), Mitrophanov, Lomsadze, and Borodovsky (2005), Guo (2006), Marek and Pultarová (2006), Sirl, Zhang and Pollett (2007), Gambin, Krzyżanowski, and Pokarowski (2008), Marek, Mayer, and Pultarová (2009), Barbour and Pollett (2010, 2012), Nonaka, Ono, Sadakane, and Yamashita (2010), Avrachenkov, Filar, and Haviv (2013), Petersson (2013, 2016a, b, c), Silvestrov, Manca, and Silvestrova (2014), Masuyama (2015), Betz and Le Roux (2016), Boxma, Mandjes, and Reed (2016), Hunter (2016, 2018), Silvestrov, D. and Silvestrov, S. (2016, 2017a, b, c, d), Silvestrov, Petersson, and Hössjer (2018), Abola, Biganda, Silvestrov, D., Silvestrov, S., Engström, Mango, and Kakuba (2019),

374

B Methodological and bibliographical notes

Silvestrov, D., Silvestrov, S., Abola, Biganda, Engström, Mango, and Kakuba(2019, 2020a,b), Hervé and Ledoux (2020), and and Issaadi (2020). B.2.7. In conclusion, we would like to refer to books and review articles devoted to models of stochastic processes and asymptotic problems, which are the objects of study in the book. First, we would like to point out several books and review articles that contain materials on semi-Markov-type processes, regenerative processes, and related modes of random processes. These are books Smith (1955), Cox (1962), Kemperman (1962), Feller (1968, 1971), Kingman (1972), Korolyuk, Brodi and Turbin (1974), Çinlar (1974, 1975), Gikhman and Skorokhod (1973), Cohen (1976), Silvestrov (1980a), Volker (1980), Revuz (1984), Kalashnikov (1994b), Korolyuk and Swishchuk (1995a, b), Kovalenko, Kuznetsov, and Shurenkov (1996), Kijima (1997), Rolski, Schmidli, Schmidt and Teugels (1999), Thorisson (2000), Limnios and Oprişan (2001), Janssen and Manca (2006, 2007), Barbu and Limnios (2008), Gyllenberg and Silvestrov (2008), Harlamov (2008), Serfozo (2009), Stewart, W. (2009), Collet, Martínez, and San Martín (2013), Ross (2014), Grabski (2015), and Kallenberg (2021). We would like to mention books and review articles that include material on perturbed Markov, semi-Markov, and regenerative processes and related asymptotic problems. These are books Seneta (1973, 2006), Silvestrov (1974, 2004, 2014b, 2015), Korolyuk and Turbin (1976a, 1978), Courtois (1977), Kalashnikov (1978a, 1994b, 1997), Anisimov (1988, 2008), Gut (1988), Kruglov and Korolev (1990), Stewart and Sun (1990), Kovalenko(1994), Stewart (1994, 1998, 2001), Korolyuk and Swishchuk (1995a, b), Gnedenko and Korolev (1996), Kovalenko, Kuznetsov, and Shurenkov (1996), Kartashov (1996), Borovkov (1998c), Yin and Zhang (1998, 2005, 2013), Korolyuk, V.S. and Korolyuk, V.V. (1999), Bening and Korolev (2002), Bini, Latouche and Meini (2005), Korolyuk and Limnios (2005), Barbu and Limnios (2008), Gyllenberg and Silvestrov (2008), Meyn and Tweedie (2009), Avrachenkov, Filar and Howlett (2013), and Silvestrov, D. and Silvestrov, S. (2017c). We would also like to mention some books and reviews in which results on perturbed Markov-type processes are applied to control theory, decision-making processes, the Internet, queuing theory, reliability theory, mathematical genetics, models of population dynamics and epidemics, insurance, and financial mathematics. These are books Gnedenko and Kovalenko (1964), Kovalenko (1975, 1994), Courtois (1977), Kalashnikov (1978a, 1994b, 1997), Iglehart and Shedler (1980), Silvestrov (1980a, 2014b, 2015), Woodroofe (1982), Anisimov, Zakusilo and Donchenko 1987), Shedler (1987, 1993), Abbad and Filar (1995), Kovalenko, Kuznetsov, and Shurenkov 1996), Kijima (1997), Kovalenko, Kuznetsov, and Pegg 1997), Asmussen (2000, 2003), Anisimov (2008), Barbu and Limnios (2008), Gyllenberg and Silvestrov (2008), Asmussen and Albrecher (2010), Avrachenkov, Filar and Howlett 2013), and Obzherin and Boyko (2015).

References

1. Abadov, Z.A. (1984). Asymptotical Expansions with Explicit Estimation of Constants for Exponential Moments of Sums of Random Variables Defined on a Markov Chain and their Applications to Limit Theorems for First Hitting Times. Candidate of Science dissertation, Kiev State University. 2. Abbad, M., Filar, J.A. (1995). Algorithms for singularly perturbed Markov control problems: A survey. In: Leondes, C.T. (Ed.) Techniques in Discrete-Time Stochastic Control Systems. Control and Dynamic Systems, 73, Academic Press, New York, 257–289. 3. Abola, B., Biganda, P.S., Silvestrov, D., Silvestrov, S., Engstrröm, C., Mango, J.M., Kakuba, G. (2019). Nonlinearly perturbed Markov chains and information networks. In: Skiadas, C.H. (Ed.) Proceedings of the 18th Applied Stochastic Models and Data Analysis International Conference. Florence, Italy, 2019, ISAST: International Society for the Advancement of Science and Technology, 51–79. 4. Afanasyev, V.I. (2016). On the time of attaining a high level by a transient random walk in a random environment. Teor. Verojatnost. Primen., 61, no. 2, 234–267 (English translation in Theory Probab. Appl., 61, no. 2, 178–207). 5. Afanasyev, V.I. (2018). Two-boundary problem for a random walk in a random environment. Teor. Verojatnost. Primen., 63, no. 3, 417–430 (English translation in Theory Probab. Appl., 63, no. 3 339–350). 6. Ahmad, I.A., Basu, A.K. (1983). Equivalence of rates of convergence in the CLT between maximum partial sums and first passage times of random walks. Scand. Actuar. J., no. 2, 89–105. 7. Aldous, D.J. (1982). Markov chains with almost exponential hitting times. Stoch. Proces. Appl., 13, 305–310. 8. Aldous, D. (1989). Hitting times for random walks on vertex-transitive graphs. Math. Proc. Cambridge Philos. Soc., 106, no. 1, 179–191. 9. Aldous, D.J., Brown, M. (1992). Inequalities for rare events in time-reversible Markov chains. I. In: Shaked, M, Tong. Y.L (Eds.) Stochastic Inequalities. AMS-IMS-SIAM Joint Summer Research Conference held in Seattle, Washington, 1991. Institute of Mathematical Statistics Lecture Notes, Monograph Series, 22, Hayward, CA, 1–16. 10. Aldous, D.J., Brown, M. (1993). Inequalities for rare events in time-reversible Markov chains. II. Stoch. Process. Appl., 44, no. 1, 15–25. 11. Aliyev, S.A., Abadov, Z.A. (2007). A new approach for studying asymptotics of exponential moments of sums of random variables, connected in Markov chain. I. Trans. Natl. Acad. Sci. Azerb., Ser. Phys.-Tech. Math. Sci. 27, no. 7, 17–32. 12. Aliev, S.A., Ragimov, F.H., Farhadova, A.D., Khalilov, V.S. (2019). Limit theorems for the family of the first passage time of the level by random walk described by a nonlinear function of the autoregressive process of order one. Uzbek Math. J., no. 1, 4–13. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4

375

376

References

13. Aliyev, S., Rahimov, F., Navidi, M. (2011). On asymptotic behavior of conditional probability of crossing the nonlinear boundary by a perturbed random walk. Theory Stoch. Process., 17, no. 1, 5–11. 14. Allen, B., Anderssen, R.S., Seneta, E. (1977). Computation of stationary measures for infinite Markov chains. In: Neuts, M.F. (Ed.) Algorithmic Methods in Probability. Studies in the Management Sciences, 7, North-Holland, Amsterdam, 13–23. 15. Alsmeyer, G., Hoefs, V. (2001). Markov renewal theory for stationary m-block factors. Markov Process. Related Fields, 7, no. 2, 325–348. 16. Alsmeyer, G., Hoefs, V. (2004). Markov renewal theory for stationary (m + 1)-block factors: first passage time and overshoot. In: Semi-Markov Processes: Theory and Applications. Comm. Statist. Theory Methods, 33, no. 3, 545–568. 17. Alsmeyer, G., Irle, A. (1986). Asymptotic expansions for the variance of stopping times in nonlinear renewal theory. Stoch. Process. Appl., 23, 235–258. 18. Alsmeyer, G. Marynych, A. (2016). Renewal approximation for the absorption time of a decreasing Markov chain. J. Appl. Probab.. 53, no. 3, 765–782. 19. Altman, E., Avrachenkov, K.E., Núñez-Queija, R. (2004). Perturbation analysis for denumerable Markov chains with application to queueing models. Adv. Appl. Probab., 36, no. 3, 839–853. 20. Anisimov, V.V. (1971a). Limit theorems for sums of random variables on a Markov chain, connected with the exit from a set that forms a single class in the limit. Teor. Verojatnost. Mat. Stat., 4, 3–17 (English translation in Theory Probab. Math. Statist., 4, 1–13). 21. Anisimov, V.V. (1971b). Limit theorems for sums of random variables that are given on a subset of states of a Markov chain up to the moment of exit, in a series scheme. Teor. Verojatnost. Mat. Stat., 4, 18–26 (English translation in Theory Probab. Math. Statist., 4, 15–22). 22. Anisimov, V.V. (1973a). Limit theorems for sums of random variables that are given on a countable subset of the states of a Markov chain up to the first exit time. Teor. Verojatnost. Mat. Stat., 8, 3–13 (English translation in Theory Probab. Math. Statist., 8, 1–10). 23. Anisimov, V.V. (1973b). Asymptotical consolidation of states for random processes. Kibernetika, no. 3, 109–117. 24. Anisimov, V.V. (1980). Limit theorems for processes that allow the asymptotic enlargement of a component. Theor. Verojatnost. Mat. Stat., 22, 3–15 (English translation in Theory Probab. Math. Statist., 22, 1–13). 25. Anisimov, V.V. (1986). Approximation of Markov processes that can be asymptotically lumped. Teor. Verojatnost. Mat. Stat., 34, 1–12 (English translation in Theory Probab. Math. Statist., 34, 1–11). 26. Anisimov, V.V. (1988). Random Processes with Discrete Components. Vysshaya Shkola and Izdatel’stvo Kievskogo Universiteta, Kiev, 183 pp. 27. Anisimov, V.V. (2008). Switching Processes in Queueing Models. Applied Stochastic Methods Series. ISTE, London and Wiley, Hoboken, NJ, 345 pp. 28. Anisimov, V.V., Chernyak, A.V. (1982). Limit theorems for certain rare functionals on Markov chains and semi-Markov processes. Teor. Verojatnost. Mat. Stat., 26, 3–8 (English translation in Theory Probab. Math. Statist., 26, 1–6). 29. Anisimov, V.V., Zakusilo, O.K., Donchenko, V.S. (1987). Elements of Queueing and Asymptotical Analysis of Systems. Lybid’, Kiev, 248 pp. 30. Ascione, G., Pirozzi, E., Toaldo, B. (2020). On the exit time from open sets of some semiMarkov processes. Ann. Appl. Probab., 30, no. 3, 1130–1163. 31. Asmussen, S. (1994). Busy period analysis, rare events and transient behaviour in fluid flow models. J. Appl. Math. Stoch. Anal., 7, no. 3, 269–299. 32. Asmussen, S. (1997). Phase-type distributions and related point processes: fitting and recent advances. In: Chakravarthy, S., Attahiru S. Alfa (Eds.) Matrix-analytic methods in stochastic models. Lecture Notes in Pure and Applied Mathematics, 183, Dekker, New York, 137–149. 33. Asmussen, S. (1998). Subexponential asymptotics for stochastic processes: extremal behavior, stationary distributions and first passage probabilities. Ann. Appl. Probab., 8, no. 2, 354–374.

References

377

34. Asmussen, S. (2000). Ruin probabilities. Advanced Series on Statistical Science & Applied Probability, 2. World Scientific, River Edge, NJ, xii+385 pp. 35. Asmussen, S. (2003). Applied Probability and Queues. Second edition, Applications of Mathematics, 51, Stochastic Modelling and Applied Probability. Springer, New York, xii+438 pp. 36. Asmussen, S., Albrecher, H. (2010). Ruin Probabilities. Second edition, Advanced Series on Statistical Science & Applied Probability, 14, World Scientific, Hackensack, NJ, xviii+602 pp. 37. Asmussen, S., Foss, S. (2014). On exceedance times for some processes with dependent increments. J. Appl. Probab., 51, no. 1, 136–151. 38. Asmussen, S. Kortschak, D. (2013). Second order corrections for the limits of normalized ruin times in the presence of heavy tails. Stoch. Syst., 3, no. 2, 591–633. 39. Avrachenkov, K.E. (1999). Analytic Perturbation Theory and Its Applications. PhD Thesis, University of South Australia, 1999. 40. Avrachenkov, K.E. (2000). Singularly perturbed finite Markov chains with general ergodic structure. In: R. Boel, R., Stremersch, G. (Eds.) Discrete Event Systems. Analysis and Control. Kluwer International Series in Engineering and Computer Science, 569, Kluwer, Boston, 429–432. 41. Avrachenkov, K.E., Filar, J., Haviv, M. (2002). Singular perturbations of Markov chains and decision processes. In: Feinberg, E.A., Shwartz, A. (Eds.) Handbook of Markov Decision Processes. Methods and Applications. International Series in Operations Research & Management Science, 40, Kluwer, Boston, 113–150. 42. Avrachenkov, K.E., Filar, J.A., Howlett, P.G. (2013). Analytic Perturbation Theory and Its Applications. SIAM, Philadelphia, PA, xii+372 pp. 43. Avrachenkov, K.E., Haviv, M. (2004). The first Laurent series coefficients for singularly perturbed stochastic matrices. Linear Algebra Appl., 386, 243–259. 44. Avrachenkov, K.E., Lasserre, J.B. (1999). The fundamental matrix of singularly perturbed Markov chains. Adv. Appl. Probab., 31, no. 3, 679–697. 45. Barbour, A.D., Pollett, P.K. (2010). Total variation approximation for quasi-stationary distributions. J. Appl. Probab., 47, no. 4, 934–946. 46. Barbour, A.D., Pollett, P.K. (2012). Total variation approximation for quasi-equilibrium distributions, II. Stoch. Process. Appl., 122, no. 11, 3740–3756. 47. Barbu, V.S., Limnios, N. (2008). Semi-Markov Chains and Hidden Semi-Markov Models Toward Applications. Their Use in Reliability and DNA Analysis. Lecture Notes in Statistics, 191, Springer, New York, xiv+224 pp. 48. Barlow, J. (1993a). Perturbation results for nearly uncoupled Markov chains with applications to iterative methods. Numer. Math., 65, no. 1, 51–62. 49. Barlow, J.L. (1993b). Error bounds for the computation of null vectors with applications to Markov chains. SIAM J. Matrix Anal. Appl., 14, no. 3, 798–812. 50. Basu, A.K. (1972). Invariance theorems for first passage time variables. Canad. Math. Bull., 15, 171–176. 51. Belyaev, Yu.K. (1963). Limit theorems for flows subject to rarefaction. Teor. Verojatnost. Primen., 8, 175–184 (English translation in Theory Probab. Appl., 8, 165–173). 52. Bening, V.E., Korolev, V.Yu. (2002). Generalized Poisson Models and their Applications in Insurance and Finance. Modern Probability and Statistics, VSP, Utrecht, 432 pp. 53. Benois, O., Landim, C., Mourragui, M. (2013). Hitting times of rare events in Markov chains. J. Stat. Phys., 153, no. 6, 967–990. 54. Bertoin, J., Kortchemski, I. (2016). Self-similar scaling limits of Markov chains on the positive integers. Ann. Appl. Probab., 26, no. 4, 2556–2595. 55. Betz, V., Le Roux, S. (2016). Multi-scale metastable dynamics and the asymptotic stationary distribution of perturbed Markov chains. Stochastic Process. Appl., 126, no. 11, 3499–3526. 56. Billingsley, P. (2014). Convergence of Probability Measures. Second edition. Wiley Series in Probability and Statistics, Wiley, New York, 296 pp. (First edition in 1968). 57. Bini, D.A., Latouche, G., Meini, B. (2005). Numerical Methods for Structured Markov Chains. Numerical Mathematics and Scientific Computation, Oxford Science Publications. Oxford University Press, New York, xii+327 pp.

378

References

58. Blanchet, J., Glynn, P. (2007). Uniform renewal theory with applications to expansions of random geometric sums. Adv. Appl. Probab., 39, no. 4, 1070–1097. 59. Bobrova, A.F. (1983). Estimates of accuracy of an asymptotic consolidation of countable Markov chains. In Stability Problems for Stochastic Models. VNIISI, Moscow, 16–24. 60. Bon, J.L., Kalashnikov, V. (2001). Some estimates of geometric sums. Statist. Probab. Lett., 55, no. 1, 89–97. 61. Borovkov, A.A. (1998a). On the asymptotics of distributions of first-passage times. I. Mat. Zametki, 75, no. 1, 24–39 (English translation in Math. Notes, 75, no. 1–2, 23–37). 62. Borovkov, A.A. (1998b). On the asymptotics of distributions of first-passage times. II. Mat. Zametki, 75, no. 3, 350–359 (English translation in Math. Notes, 75, no. 3–4, 322–330). 63. Borovkov, A.A. (1998c). Ergodicity and Stability of Stochastic Processes. Wiley Series in Probability and Statistics, 314, Wiley, Chichester, xxiv+585 pp. (Translation from the 1994 Russian original). 64. Borovkov, A.A. (2015). Integral theorems for the first passage time of an arbitrary boundary by a compound renewal process. Sibirsk. Mat. Zh., 56, no. 5, 961–981 (English translation in Sib. Math. J., 56, no. 5, 765–782). 65. Borovkov, A.A. (2016). On the distribution of the first passage time of an arbitrary remote boundary by a random walk. Teor. Verojatnost. Primen., 61, no. 2, 210–233 (English translation in Theory Probab. Appl., 61, no. 2, 235–254). 66. Borovkov, A.A., Fayolle, G., Korshunov, D.A. (1992). Transient phenomena for Markov chains and applications. Adv. Appl. Probab., 24, 322–342. 67. Boxma, O., Mandjes, M., Reed, J. (2016). On a class of reflected AR(1) processes. J. Appl. Probab., 53, no. 3, 818–832. 68. Brown, M.G., Guang Ping (1984). On the waiting time for the first occurrence of a pattern. In: Reliability Theory and Models, Charlotte, N.C., 1983. Notes Rep. Comput. Sci. Appl. Math., 10, Academic Press, Orlando, FL, 267–272. 69. Buck, M. (2021). Limit theorems for random walks with absorption. J. Theor. Probab., 34, 241–263. 70. Bulinskaya, E.V. (2012). Hitting times in presence of taboo points for a random walk. Mat. Tr., 15, no. 1, 3–26 (English translation in Siberian Adv. Math., 22 (2012), no. 4, 227–242. 71. Burnley, C. (1987). Perturbation of Markov chains. Math. Mag., 60, no. 1, 21–30. 72. Butler, R.W. (2017). Asymptotic expansions and hazard rates for compound and first-passage distributions. Bernoulli, 23, no. 4B, 3508–3536. 73. Cao, W.L., Stewart, W.J. (1985). Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. J. Ass. Comp. Mach., 32, 702–719. 74. Cao, X.R. (1998). The Maclaurin series for performance functions of Markov chains. Adv. Appl. Prob., 30, 676–692. 75. Cator, E., Don, H. (2021). Explicit bounds for critical infection rates and expected extinction times of the contact process on finite random graphs. Bernoulli, 27, no. 3, 1556–1582. 76. 2020) Chatelin, F., Miranker, W.L. (1984). Aggregation/disaggregation for eigenvalue problems. SIAM J. Numer. Anal., 21, no. 3, 567–582. 77. Chatrabgoun, O., Daneshkhah, A., Parham, G. (2020). On the functional central limit theorem for first passage time of nonlinear semi-Markov reward processes. Comm. Statist. Theory, Meth., 49, no. 19, 4737–4750. 78. Chow, Y.S., Hsiung, C.A. (1976). Limiting behaviour of max j ≤n S j /j −α and first passage time in random walk with positive drift. Bull. Inst. Math. Acad. Sinica, 4, 35–44. 79. Chow, Y.S., Hsiung, C.A., Yu, K.F. (1980). Limit theorems for a positively drifting process and its related first passage times. Bull. Inst. Math. Acad. Sinica, 8, 141–172. 80. Çinlar, E. (1974). Markov renewal theory: a survey. Manag. Sci., 21, no. 7, 727–752. 81. Çinlar, E. (1975). Introduction to Stochastic Processes. Prentice-Hall, Englewood Cliffs, N.J., x+402 pp. 82. Clancy, D., Tjia, E. (2018). Approximating time to extinction for endemic infection models. Methodol. Comput. Appl. Probab., 20, no. 4, 1043–1067.

References

379

83. Clément, D., Landy, R. (2017). High order expansions for renewal functions and applications to ruin theory. Ann. Appl. Probab., 27, no. 4, 2342–2382. 84. Coderch, M., Willsky, A.S., Sastry, S.S., Castañon, D.A. (1983). Hierarchical aggregation of singularly perturbed finite state Markov processes. Stochastics, 8, 259–289. 85. Cohen, J.W. (1976). On Regenerative Processes in Queueing Theory. Lecture Notes in Economics and Mathematical Systems, 121, Springer, Berlin, ix+93 pp. 86. Collet, P., Martínez, S., San Martín, J. (2013). Quasi-Stationary Distributions. Markov Chains, Diffusions and Dynamical Systems. Probability and its Applications, Springer, Heidelberg, xvi+280 pp. 87. Courtois, P.J. (1975). Error analysis in nearly-completely decomposable stochastic systems. Econometrica, 43, no. 4, 691–709. 88. Courtois, P.J. (1977). Decomposability: Queueing and Computer System Applications. ACM Monograph Series, Academic Press, New York, xiii+201 pp. 89. Courtois, P.J. (1982). Error minimization in decomposable stochastic models. In: Ralph L.D., Teunis J.O. (Eds). Applied Probability – Computer Science: the Interface, Vol. I. Progress in Computer Science, 2, Birkhäuser, Boston, 189–210. 90. Courtois, P.J., Louchard, G. (1976). Approximation of eigen characteristics in nearlycompletely decomposable stochastic systems. Stoch. Process. Appl., 4, 283–296. 91. Courtois, P.J., Semal, P. (1984a). Error bounds for the analysis by decomposition of nonnegative matrices. In: Iazeolla, G., Courtois, P.J., Hordijk, A. (Eds). Mathematical Computer Performance and Reliability. North-Holland, Amsterdam, 209–224. 92. Courtois, P.J., Semal, P. (1984b). Block decomposition and iteration in stochastic matrices. Philips J. Res., 39, no. 4–5, 178–194. 93. Courtois, P.J., Semal, P. (1984c). Bounds for the positive eigenvectors of nonnegative matrices and for their approximations by decomposition. J. Assoc. Comput. Mach.. 31, no. 4, 804–825. 94. Courtois, P.J., Semal, P. (1991). Bounds for transient characteristics of large or infinite Markov chains. In: Stewart, W.J. (Ed.) Numerical Solution of Markov Chains. Probability: Pure and Applied, 8, Marcel Dekker, New York, 413–434. 95. Cox, D.R. (1962). Renewal Theory. Methuen, London and Wiley, New York, ix+142 pp. 96. Craven, B.D. (2003). Perturbed Markov processes. Stoch. Models, 19, no. 2, 269–285. 97. Darroch, J.N., Morris, K.W. (1967). Some passage-time generating functions for discrete-time and continuous-time finite Markov chains. J. Appl. Probab., 4, 496–507. 98. Delebecque, F. (1983). A reduction process for perturbed Markov chains. SIAM J. Appl. Math., 43, 325–350. 99. Delebecque, F., Quadrat, J.P., Kokotović, P.V. (1984). A unified view of aggregation and coherency in networks and Markov chains. Inter. J. Control, 40, 939–952. 100. Denisov, D., Sakhanenko, A., Wachtel, V. (2018). First-passage times over moving boundaries for asymptotically stable walks. Teor. Verojatnost. Primen. 63, no. 4, 755–778 (Also in Theory Probab. Appl., 63, no. 4, 613–633). 101. D’Onofrio, G., Macci, C., Pirozzi, E. (2018). Asymptotic results for first-passage times of some exponential processes. Methodol. Comput. Appl. Probab., 20, no. 4, 1453–1476. 102. Drozdenko, M. (2007a). Weak convergence of first-rare-event times for semi-Markov processes. I. Theory Stoch. Process., 13, no. 4, 29–63. 103. Drozdenko, M. (2007b). Weak Convergence of First-Rare-Event Times for Semi-Markov Processes. Doctoral dissertation 49, Mälardalen University, Västerås. 104. Drozdenko, M. (2009). Weak convergence of first-rare-event times for semi-Markov processes. II. Theory Stoch. Process., 15, no. 2, 99–118. 105. Ele˘ıko, Ya.I., Shurenkov, V.M. (1996) On the asymptotic representation of the Perron root of a matrix-valued stochastic evolution. Ukr. Mat. Zh., 48, 35–43 (English translation in Ukr. Math. J., 48, 38–47). 106. Englund, E. (2001). Nonlinearly Perturbed Renewal Equations with Applications. Doctoral dissertation, Umeå University. 107. Englund, E., Silvestrov, D.S. (1997). Mixed large deviation and ergodic theorems for regenerative processes with discrete time. In: Jagers, P., Kulldorff, G., Portenko, N., Silvestrov,

380

108. 109.

110.

111.

112. 113. 114.

115.

116.

117.

118.

119. 120. 121. 122. 123. 124.

125.

126.

127.

References D. (Eds.) Proceedings of the Second Scandinavian–Ukrainian Conference in Mathematical Statistics, Vol. I, Umeå, 1997. Theory Stoch. Process., 3(19), no. 1–2, 164–176. Feller, W. (1968). An Introduction to Probability Theory and Its Applications. Vol. I. Third edition, Wiley, New York, xviii+509 pp. (First edition in 1950). Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. II. (Second edition, Wiley Series in Probability and Statistics, Wiley, New York, xxiv+669 pp. (First edition in 1966). Fernandez, R., Manzo, F., Nardi, F.R., Scoppola, E. (2015). Asymptotically exponential hitting times and metastability: a pathwise approach without reversibility. Electron. J. Probab., 20, Paper No. 122, 37 pp. Fernandez, R., Manzo, F., Nardi, F.R., Scoppola, E., Sohier, J. (2016). Conditioned, quasistationary, restricted measures and escape from metastable states. Ann. Appl. Probab., 26, no. 2, 760–793. Ferré, D., Hervé, L., Ledoux, J. (2013). Regular perturbation of V-geometrically ergodic Markov chains. J. Appl. Probab., 50, no. 1, 184–194. Funderlic, R.E., Meyer, C.D.(1985). Sensitivity of the stationary distribution vector for an ergodic Markov chain. Linear Algebra Appl., 76, 1–17. Ga˘ıtsgori, V.G., Pervozvanskiy, A.A. (1975). Aggregation of states in a Markov chain with weak interaction. Kibernetika, no. 3, 91–98 (English translation in Cybernetics, 11, no. 3, 441–450). Ga˘ıtsgori, V.G., Pervozvanskiy, A. A. (1983). Decomposition and aggregation in problems with a small parameter. Izv. Akad. Nauk SSSR, Tekhn. Kibernet., no. 1, 33–46 (English translation in Engineer. Cybernetics, 2, no. 1, 26–38). Gambin, A., Krzyżanowski, P., Pokarowski, P. (2008). Aggregation algorithms for perturbed Markov chains with applications to networks modeling. SIAM J. Sci. Comput. 31, no. 1, 45–73. Gasanenko, V.A. (1999). Normed Poisson process in one-sided curvelinear boundary domain. In: Proceedings of the Third Ukrainian-Scandinavian Conference in Probability Theory and Mathematical Statistics, Kiev, 1999. Theory Stoch. Process., 5(21), no. 3–4, 64–71. Gasanenko, V.A. (2001). On the asymptotics of the sojourn probability of a Poisson process between two divergent nonlinear boundaries. Ukr. Mat. Zh., 53, 14–22 (English translation in Ukr. Math. J., 53, 15–25). Gast, N., Gaujal, B. (2017). Computing absorbing times via fluid approximations. Adv. Appl. Probab., 49, no. 3, 768–790. Gaver, D.P., Jr. (1964). An absorption probability problem. J. Math. Anal. Appl., 9, 384–393. Gaver, D.P. (1976). Random record models. J. Appl. Probab., 13, 538–547. Gaver, D.P., Jacobs, P.A., Latouche, G. (1984). Finite birth-and-death models in randomly changing environments. Adv. Appl. Probab., 16, 715–731. Gibson, D., Seneta, E. (1987). Augmented truncations of infinite stochastic matrices. J. Appl. Probab., 24, no. 3, 600–608. Gikhman, I.I., Skorokhod, A.V. (1971). Theory of Random Processes. 1. Probability Theory and Mathematical Statistics, Nauka, Moscow, 664 pp. (English edition: The Theory of Stochastic Processes. 1. Fundamental Principles of Mathematical Sciences, 210, Springer, New York (1974) and Berlin (1980)). Gikhman, I.I., Skorokhod, A.V. (1973). Theory of Random Processes. 2. Probability Theory and Mathematical Statistics, Nauka, Moscow, 639 pp. (English edition: The Theory of Stochastic Processes. II. Classics in Mathematics, Springer, Berlin (2004) and Fundamental Principles of Mathematical Sciences, 218, Springer, Berlin (1975)). Glynn, P. (2011). On exponential limit laws for hitting times of rare sets for Harris chains and processes. In: Glynn, P. Mikosch, T. and Rolski, T. (Eds.) New Frontiers in Applied Probability: a Festschrift for Søren Asmussen. J. Appl. Probab., Spec. Vol. 48A, 319–326. Gnedenko, B.V., Frayer, B. (1969). A few remarks on a result by I.N. Kovalenko. Litov. Mat. Sbornik, 9, 463–470.

References

381

128. Gnedenko, B.V., Korolev, V.Yu. (1996). Random Summation. Limit Theorems and Applications. CRC Press, Boca Raton, FL, 288 pp. 129. Gnedenko, B.V., Kovalenko, I.N. (1964). Introduction to Queueing Theory. Nauka, Moscow (English edition: Introduction to Queuing Theory. Mathematical Modeling, 5, Birkhäuser, Boston (1989), xi+315 pp.). 130. Goldsheid, I.Ya. (2008). Linear and sub-linear growth and the CLT for hitting times of a random walk in random environment on a strip. Probab. Theory Related Fields, 141„ no. 3–4, 471–511. 131. Golub, G.H., Seneta, E. (1973). Computation of the stationary distribution of an infinite Markov matrix. Bull. Austral. Math. Soc., 8, 333–341. 132. Golub, G. H., Seneta, E. (1974). Computation of the stationary distribution of an infinite stochastic matrix of special form. Bull. Austral. Math. Soc., 10, 255–261. 133. Grabski, F. (2015). Semi-Markov Processes: Applications in System Reliability and Maintenance. Elsevier, Amsterdam, xiv+255 pp. 134. Grama, I., Lauvergnat, R., Le Page, É. (2020). Conditioned local limit theorems for random walks defined on finite Markov chains. Probab. Theory Related Fields, 176, no. 1–2, 669–735. 135. Griffiths, S., Kang, R.J., Oliveira, R.I., Patel, V. (2014). Tight inequalities among set hitting times in Markov chains. Proc. Amer. Math. Soc., 142, no. 9, 3285–3298. 136. Guo, D.Z. (2006). On the sensitivity of the solution of nearly uncoupled Markov chains. SIAM J. Matrix Anal. Appl., 14, no. 4, 1112–1123. 137. Gusak, D.V., Korolyuk, V.S. (1971). Asymptotic behaviour of semi-Markov processes with a decomposable set of states. Teor. Verojatnost. Mat. Stat., 5, 43–50 (English translation in Theory Probab. Math. Statist., 5, 43–51). 138. Gut, A. (1974a). On the moments and limit distributions of some first passage times. Ann. Probab., 2, 277–308. 139. Gut, A. (1974b). On convergence in r-mean of some first passage times and randomly indexed partial sums. Ann. Probab., 2, 321–323. 140. Gut, A. (1975). Weak convergence and first passage times. J. Appl. Probab., 12, 324–332. 141. Gut, A. (1988). Stopped Random Walks. Limit Theorems and Applications. Applied Probability. A Series of the Applied Probability Trust, 5, Springer, New York, xiv+263 pp. 142. Gut, A., Holst, L. (1984). On the waiting time in a generalized roulette game. Statist. Probab. Lett., 2, no. 4, 229–239. 143. Gyllenberg, M., Silvestrov, D.S. (1994). Quasi-stationary distributions of a stochastic metapopulation model. J. Math. Biol., 33, 35–70. 144. Gyllenberg, M., Silvestrov, D.S. (1998). Quasi-stationary phenomena in semi-Markov models. In: Janssen, J., Limnios, N. (Eds.) Proceedings of the Second International Symposium on Semi-Markov Models: Theory and Applications, Compiègne, 1998, 87–93. 145. Gyllenberg, M., Silvestrov, D.S. (1999a). Quasi-stationary phenomena for semi-Markov processes. In: Janssen, J., Limnios, N. (Eds.) Semi-Markov Models and Applications. Kluwer, Dordrecht, 33–60. 146. Gyllenberg, M., Silvestrov, D.S. (1999b). Cramér-Lundberg and diffusion approximations for nonlinearly perturbed risk processes including numerical computation of ruin probabilities. In: Silvestrov, D., Yadrenko, M., Borisenko, O., Zinchenko, N. (Eds.) Proceedings of the Second International School on Actuarial and Financial Mathematics, Kiev, 1999. Theory Stoch. Process., 5(21), no. 1–2, 6–21. 147. Gyllenberg, M., Silvestrov, D.S. (2000a). Nonlinearly perturbed regenerative processes and pseudo-stationary phenomena for stochastic systems. Stoch. Process. Appl., 86, 1–27. 148. Gyllenberg, M., Silvestrov, D.S. (2000b). Cramér–Lundberg approximation for nonlinearly perturbed risk processes. Insur. Math. Econom., 26, 75–90. 149. Gyllenberg, M., Silvestrov, D.S. (2008). Quasi-Stationary Phenomena in Nonlinearly Perturbed Stochastic Systems. De Gruyter Expositions in Mathematics, 44, Walter de Gruyter, Berlin, ix+579 pp. 150. Hanen, A. (1963a). Probème central limite dans le cas Markovien fini. La matrice limite n’a qu’une seule classe ergodique et pas d’état transitoire. C. R. Acad. Sci. Paris, 256, 68–70.

382

References

151. Hanen, A. (1963b). Problème central limite dans le cas Markovien fini. II. La matrice limite a plusieurs classes ergodiques et pas détats transitoires. C. R. Acad. Sci. Paris, 256, 362–364. 152. Hanen, A. (1963c). Problème central limite dans le cas Markovien fini. Cas général. C. R. Acad. Sci. Paris, 256, 575–577. 153. Hanen, A. (1963d). Théorèmes limites pour une suite de chaînes de Markov. Ann. Inst. H. Poincaré, 18, 197–301. 154. Harlamov, B. (2008). Continuous Semi-Markov Processes. Applied Stochastic Methods Series, ISTE, London and Wiley, Hoboken, NJ, 375 pp. 155. Harris, T.E. (1952). First passage and recurrence distributions. Trans. Amer. Math. Soc., 73, 471–486. 156. Hassin, R., Haviv, M. (1992). Mean passage times and nearly uncoupled Markov chains. SIAM J. Disc. Math., 5, 386–397. 157. Hatori, H. (1959). Some theorems in an extended renewal theory. I. K¯odai Math. Semin. Repts., 11, no. 3, 139–146. 158. Hatori, H. (1960). Some theorems in an extended renewal theory. II. K¯odai Math. Semin. Repts., 12, no. 1, 21–27. 159. Haviv, M. (1986). An approximation to the stationary distribution of a nearly completely decomposable Markov chain and its error analysis. SIAM J. Algebr. Discr. Meth., 7, no. 4, 589–593. 160. Haviv, M. (1987). Aggregation/disaggregation methods for computing the stationary distribution of a Markov chain. SIAM J. Numer. Anal., 24, no. 4, 952–966. 161. Haviv, M. (1988). Error bounds on an approximation to the dominant eigenvector of a nonnegative matrix. Linear and Multilinear Algebra, 23, no. 2, 159–163. 162. Haviv, M. (1992). An aggregation/disaggregation algorithm for computing the stationary distribution of a large Markov chain. Comm. Statist. Stoch. Models, 8, no. 3, 565–575. 163. Haviv, M. (1999). On censored Markov chains, best augmentations and aggregation/disaggregation procedures. Aggregation and disaggregation in operations research. Comput. Oper. Res., 26, no. 10–11, 1125–1132. 164. Haviv, M. (2006). More on Rayleigh–Ritz refinement technique for nearly uncoupled stochastic matrices. SIAM J. Matrix Anal. Appl., 10, no. 3, 287–293. 165. Haviv, M., Ritov, Y. (1986). An approximation to the stationary distribution of a nearly completely decomposable Markov chain and its error bound. SIAM J. Algebr. Discr. Methods, 7, no. 4, 583–588. 166. Haviv, M., Ritov, Y. (1993). On series expansions and stochastic matrices. SIAM J. Matrix Anal. Appl., 14, no. 3, 670–676. 167. Haviv, M., Ritov, Y., Rothblum, U.G. (1987). Iterative methods for approximating the subdominant modulus of an eigenvalue of a nonnegative matrix. Linear Algebra Appl., 87, 61–75. 168. Haviv, M., Ritov, Y., Rothblum, U.G. (1992). Taylor expansions of eigenvalues of perturbed matrices with applications to spectral radii of nonnegative matrices. Linear Algebra Appl., 168, 159–188. 169. Haviv, M., Rothblum, U.G. (1984). Bounds on distances between eigenvalues. Linear Algebra Appl., 63, 101–118. 170. Haviv, M., Van der Heyden, L. (1984). Perturbation bounds for the stationary probabilities of a finite Markov chain. Adv. Appl. Probab., 16, 804–818. 171. Helali, A., Löwe, M. (2019). Hitting times, commute times, and cover times for random walks on random hypergraphs. Statist. Probab. Lett., 154, 108535, 6 pp. 172. Hervé, L., Ledoux, J. (2020). State-discretization of V-geometrically ergodic Markov chains and convergence to the stationary distribution. Methodol. Comput. Appl. Probab., 22, 905– 925. 173. Holst, L. (1971). Limit theorems for some occupancy and sequential occupancy problems. Ann. Math. Statist., 42, 1671–1680. 174. Holst, L. (1977). Some asymptotic results for occupancy problems. Ann. Probab., 5, 1028– 1035.

References

383

175. Holst, L. (1979). A unified approach to limit theorems for urn models. J. Appl. Probab., 16, 154–162. 176. Hoppensteadt, F., Salehi, H., Skorokhod, A. (1996). On the asymptotic behavior of Markov chains with small random perturbations of transition probabilities. In: Gupta, A.K. (Ed.) Multidimensional Statistical Analysis and Theory of Random Matrices: Proceedings of the Sixth Eugene Lukacs Symposium. Bowling Green, OH, 1996. VSP, Utrecht, 93–100. 177. Hössjer, O., Bechly, G., Gauger, A. (2018). Phase-Type Distribution Approximations of the Waiting Time Until Coordinated Mutations Get Fixed in a Population. In: Silvestrov, S., Ran˘cić, M., Malyarenko, A. (Eds.) Stochastic Processes and Applications, Springer Proceedings in Mathematics & Statistics, 271, Springer, Cham, Chapter 12, 245–314. 178. Hsu, Guanghui; He, Qiming (1990). The residual life for alternating renewal processes. Asia-Pac. J. Oper. Res., 7, 76–81. 179. Hunter, J.J. (1986). Stationary distributions of perturbed Markov chains. Linear Algebra Appl., 82, 201–214. 180. Hunter, J.J. (1991a). The computation of stationary distributions of Markov chains through perturbations. J. Appl. Math. Stoch. Anal., 4, no. 1, 29–46. 181. Hunter, J.J. (1991b). A survey of generalized inverses and their use in applied probability. Math. Chronicle, 20, 13–26. 182. Hunter, J.J. (2005). Stationary distributions and mean first passage times of perturbed Markov chains. Linear Algebra Appl., 410, 217–243. 183. Hunter, J.J. (2016). The computation of key properties of Markov chains via perturbations. Linear Algebra Appl., 511, 176–202. 184. Hunter, J.J. (2018). The computation of the mean first passage times for Markov chains. Linear Algebra Appl., 549, 100–122. 185. Iglehart, D.L., Shedler, G.S. (1980). Regenerative Simulation of Response Times in Networks of Queues. Lecture Notes in Control and Information Sciences, 26, Springer, Berlin, xii+204 pp. 186. Issaadi, B. (2020). Weak stability bounds for approximations of invariant measures with applications to queueing. Methodol. Comput. Appl. Probab., 22, 371–400. 187. Jacod, J., Shiryaev, A.N. (2002). Limit Theorems for Stochastic Processes. Second edition. Fundamental Principles of Mathematical Sciences, 288, Springer, Berlin, 684 pp. 188. Jagers, P., Lindvall, T. (1974). Thinning and rare events in point processes. Z. Wahrsch. Verw. Gebiete, 28, 89–98. 189. Jagers, P, Klebaner, F.C., Sagitov, S. (2007). Markovian paths to extinction. Adv. in Appl. Probab., 39, no. 2, 569–587. 190. Janson, S. (1983). Renewal theory for M-dependent variables. Ann. Probab. 11, 558–568. 191. Janssen, J., Manca, R. (2006). Applied Semi-Markov Processes. Springer, New York, xii+309 pp. 192. Janssen, J., Manca, R. (2007). Semi-Markov Risk Models for Finance, Insurance and Reliability. Springer, New York, xvii+429 pp. 193. Jarner, S.F., Tweedie, R.L. (2003). Necessary conditions for geometric and polynomial ergodicity of random-walk-type Markov chains. Bernoulli, 9, no. 4, 559–578. 194. Kalashnikov, V.V. (1978a). Qualitative Analysis of the Behaviour of Complex Systems by the Method of Test Functions. Series in Theory and Methods of Systems Analysis, Nauka, Moscow, 247 pp. 195. Kalashnikov, V.V. (1978b). Solution of the problem of approximating a denumerable Markov chain. Izv. Akad. Nauk SSSR Tekhn. Kibernet., no. 3, 92–95. 196. Kalashnikov, V.V. (1981). Methods of estimation of continuity in the problem of consolidation of states. In: Problems of Stability of Stochastic Models, VNIISI, Moscow, 48–53. 197. Kalashnikov, V.V. (1983). Estimation of the rate of convergence in Rénei’s theorem. In: Stability Problems for Stochastic Models. VNIISI, Moscow, 48–57. 198. Kalashnikov, V.V. (1986). Approximation of some stochastic models. In: Semi-Markov Models, Plenum, New York, 319–335.

384

References

199. Kalashnikov, V.V. (1990). Regenerative queueing processes and their qualitative and quantitative analysis. Queueing Systems Theory Appl., 6, no. 2, 113–136. 200. Kalashnikov, V.V. (1994a). Regeneration and general Markov chains. J. Appl. Math. Stoch. Anal., 7, no. 3, 357–371. 201. Kalashnikov, V.V. (1994b). Topics on Regenerative Processes. CRC Press, Boca Raton, FL, 240 pp. 202. Kalashnikov, V.V. (1997). Geometric Sums: Bounds for Rare Events with Applications. Mathematics and its Applications, 413, Kluwer, Dordrecht, ix+265 pp. 203. Kalashnikov, V.V., Anichkin, S.A. (1981). Continuity of random sequences and approximation of Markov chains. Adv. Appl. Probab., 13, no. 2, 402–414. 204. Kalashnikov, V.V., Vsekhsvyatskii, S.Yu. (1985). Metric estimates of the first occurrence time in regenerative processes. In Stability Problems for Stochastic Models, Uzhgorod, 1984, Lecture Notes in Math., 1155, Springer, Berlin, 102–130. 205. Kallenberg, O. (2021). Foundations of Modern Probability. Third Edition. Probability Theory and Stochastic Modelling, 99, Springer, xii+946 pp. 206. Kaplan, E.I. (1979). Limit theorems for exit times of random sequences with mixing. Teor. Verojatnost. Mat. Stat., 21, 53–59 (English translation in Theory Probab. Math. Statist., 21, 59–65). 207. Kaplan, E.I. (1980). Limit Theorems for Sum of Switching Random Variables with an Arbitrary Phase Space of Switching Component. Candidate of Science dissertation, Kiev State University. 208. Kartashov, N.V. (1985a). Inequalities in stability and ergodicity theorems for Markov chains with a general phase space. I. Teor. Verojatnost. Primen., 30, 230–240 (English translation in Theory Probab. Appl., 30, 247–259). 209. Kartashov, N.V. (1985b). Inequalities in stability and ergodicity theorems for Markov chains with a general phase space. II. Teor. Verojatnost. Primen., 30, 478–485 (English translation in Theory Probab. Appl., 30, 507–515). 210. Kartashov, N.V. (1985c). Asymptotic expansions and inequalities in stability theorems for general Markov chains under relatively bounded perturbations. In: Stability Problems for Stochastic Models, Varna, 1985. VNIISI, Moscow, 75–85 (English translation in J. Soviet Math., 40, no. 4, 509–518). 211. Kartashov, N.V. (1986). Inequalities in theorems of consolidation of Markov chains. Theor. Verojatnost. Mat. Stat., 34, 62–73 (English translation in Theory Probab. Math. Statist., 34, 67–80). 212. Kartashov, N.V. (1987). Estimates for the geometric asymptotics of Markov times on homogeneous chains. Teor. Verojatnost. Mat. Stat., 37, 66–77 (English translation in Theory Probab. Math. Statist., 37, 75–88). 213. Kartashov, N.V. (1991). Inequalities in Rénei’s theorem. Teor. ˇImovirn. Mat. Stat., 45, 27–33 (English translation in Theory Probab. Math. Statist., 45, 23–28). 214. Kartashov, M.V. (1996). Strong Stable Markov Chains. VSP, Utrecht and TBiMC, Kiev, 138 pp. 215. Kartashov, M.V. (2013a). The asymptotic behavior of rare Markov moments defined on time inhomogeneous Markov chains, Teor. ˇImovirn. Mat. Stat., 88, 83–94 (English translation in Theor. Probab. Math. Statist., 88, 109–121). 216. Kartashov, M.V. (2013b). Quantitative and qualitative limits for exponential asymptotics of hitting times for birth-and-death chains in a scheme of series. Teor. ˇImovirn. Mat. Stat., 89, 40–50 (English translation in Theory Probab. Math. Statist, 89, (2014), 45–56). 217. Kartashov, M.V. (2015). The asymptotic behavior of the distribution of Markov moments in time-inhomogeneous Markov chains and its application to a discrete Cramér-Lundberg model. Teor. ˇImovirn. Mat. Stat., 92, 41–60 (English translation in Theory Probab. Math. Statist, 92, 37–58). 218. Keilson, J. (1966a). A technique for discussing the passage time distribution for stable systems. J. Royal Statist. Soc., Ser. B 28, 477–486.

References

385

219. Keilson, J. (1966b). A limit theorem for passage times in ergodic regenerative processes. Ann. Math. Statist., 37, 866–870. 220. Keilson, J. (1974). Sojourn times, exit times and jitter in multivariate Markov processes. Adv. Appl. Probab., 6, 747–756. 221. Keilson, J. (1979). Markov Chain Models – Rarity and Exponentiality. Applied Mathematical Sciences, 28, Springer, New York, xiii+184 pp. 222. Keilson, J. (1998). Covariance and relaxation time in finite Markov chains. J. Appl. Math. Stoch. Anal., 11, 391–396. 223. Kemperman, J.H.B. (1962). The Passage Problem for a Stationary Markov Chain. Statistical Research Monographs, 1, Univ. Chicago Press, 127 pp. 224. Kesten, H., Maller, R.A. (1999). Stability and other limit laws for exit times of random walks from a strip or a halfplane. Ann. Inst. H. Poincaré Probab. Statist., 35, no. 6, 685–734. 225. Khusanbaev, Ya.M. (1983). Limit Theorems for Processes with Conditionally Independent Increments and Functionals of Additive Type for Regenerative Processes. Candidate of Science dissertation, Kiev State University. 226. Khusanbaev, Ya.M. (1984). Conditions for the convergence of random processes stopped at the time of underjump. Ukr. Mat. Zh., 36, no. 1, 126–129 (English translation in Ukr. Math. J., 36, no. 1, 116–119). 227. Kijima, M. (1997). Markov Processes for Stochastic Modelling. Stochastic Modeling Series. Chapman & Hall, London, x+341 pp. 228. Kingman, J.F. (1963). The exponential decay of Markovian transition probabilities. Proc. London Math. Soc., 13, 337–358. 229. Kingman, J.F.C. (1972). Regenerative Phenomena. Wiley Series in Probability and Mathematical Statistics. Wiley, London, xii+190 pp. 230. Kohlas, J. (1983). Numerical computation of mean passage times and absorption probabilities in Markov and semi-Markov models, Zeitschrift Oper. Res., 30, A197 – A207. 231. Korolev, V.Yu. (2016). Limit distributions for doubly stochastically rarefied renewal processes and their properties. Teor. Verojatnost. Primen., 61, no. 4, 753–773 (English translation in Theory Probab. Appl., 61, no. 4, 649–664). 232. Korolyuk, D.V. (1982). Limit distributions of additive functionals of regenerative processes up to the time of reaching asymptotically receding regions. Dokl. Akad. Nauk Ukr. SSR, Ser. A, no. 8, 16–18. 233. Korolyuk, D.V. (1983). Limit Theorems for Hitting Time Type Functionals Defined on Processes with Semi-Markov Switchings. Candidate of Science dissertation, Kiev State University. 234. Korolyuk, D.V. (1990). Asymptotics of the time of attaining a distant level by a random walk with delay. Teor. Verojatnost. Mat. Stat., 42, 64–67 (English translation in Theory Probab. Math. Statist., 42, 73–76). 235. Korolyuk, D.V., Silvestrov D.S. (1983). Time of reaching asymptotically receding domains for ergodic Markov chains. Teor. Verojatnost. Primen., 28, 410–420 (English translation in Theory Probab. Appl., 28, 432–442). 236. Korolyuk, D.V., Silvestrov D.S. (1984). Entry times into asymptotically receding regions for processes with semi-Markov switchings. Teor. Verojatnost. Primen., 29, 539–544 (English translation in Theory Probab. Appl., 29, 558–563). 237. Korolyuk, V.S. (1969). On asymptotical estimate for time of a semi-Markov process being in the set of states. Ukr. Mat. Zh., 21, 842–845 (English translation in Ukr. Math. J., 21, 705–710). 238. Korolyuk, V.S. (1997). Singularly perturbed stochastic systems. Ukr. Mat. Zh., 49), 25–34 (English translation in Ukr. Math. J., 49, 25–35). 239. Korolyuk, V.S., Brodi, S.M., Turbin, A.F. (1974). Semi-Markov processes and their application. Probability Theory. Mathematical Statistics. Theoretical Cybernetics, Vol. 11, VINTI, Moscow, 1974, 47–97. 240. Korolyuk, V.S., Korolyuk, V.V. (1999). Stochastic Models of Systems. Mathematics and its Applications, 469, Kluwer, Dordrecht, xii+185 pp.

386

References

241. Korolyuk, V.S., Limnios, N. (2000). Evolutionary systems in an asymptotic split phase space. In: Limnios, N., Nikulin, M. (Eds.) Recent Advances in Reliability Theory: Methodology, Practice and Inference. Birkhäuser, Boston, 145–161. 242. Korolyuk, V.S., Limnios, N. (2002). Markov additive processes in a phase merging scheme. In: Korolyuk, V., Prokhorov, Yu., Khokhlov, V., Klesov, O. (Eds.) Proceedings of the Conference Dedicated to the 90th Anniversary of Boris Vladimirovich Gnedenko, Kiev, 2002. Theory Stoch. Process., 8, no. 3–4, 213–225. 243. Koroliuk, V.S., Limnios, N. (2005). Stochastic Systems in Merging Phase Space. World Scientific, Singapore, xv+331 pp. 244. Koroliuk, V.S., Limnios, N. (2010). Reliability of semi-Markov systems with asymptotic merging phase space. In: Rykov, V.V, Balakrishnan, N., Nikulin, M.S. (Eds.) Mathematical and Statistical Models and Methods in Reliability. Birkhäuser, Boston, 3–18. 245. Korolyuk, V.S., Penev, I.P., Turbin, A.F. (1972). The asymptotic behavior of the distribution of the absorption time of a Markov chain. Kibernetika, no. 2, 20–22. 246. Koroljuk, V.S., Penev, I.P., Turbin, A.F. (1973). Asymptotic expansion for the distribution of the absorption time of a Markov chain. Kibernetika, no. 4, 133–135. 247. Korolyuk, V.S., Penev, I.P., Turbin, A.F. (1981). Asymptotic expansion for the distribution of the absorption time of a weakly inhomogeneous Markov chain. In: Korolyuk, V.S. (Ed.) Analytic Methods of Investigation in Probability Theory. Akad. Nauk Ukr. SSR, Inst. Mat., Kiev, 97–105. 248. Korolyuk, V., Swishchuk, A. (1995a). Semi-Markov Random Evolutions. Mathematics and its Applications, 308, Springer, Dordrecht, x+310 pp. (Revised English edition of Semi-Markov Random Evolutions. Naukova Dumka, Kiev, (1992). 254 pp.). 249. Korolyuk, V.S., Swishchuk, A.V. (1995b). Evolution of Systems in Random Media. CRC Press, Boca Raton, FL, 368 pp. 250. Korolyuk, V.S., Turbin, A.F. (1970). On the asymptotic behaviour of the occupation time of a semi-Markov process in a reducible subset of states. Teor. Verojatnost. Mat. Stat., 2, 133–143 (English translation in Theory Probab. Math. Statist., 2, 133–143). 251. Korolyuk, V.S., Turbin, A.F. (1976a). Semi-Markov Processes and its Applications. Naukova Dumka, Kiev, 184 pp. 252. Korolyuk, V.S., Turbin, A.F. (1976b). Asymptotic enlarging of semi-Markov processes with an arbitrary state space. In: Proceedings of the Third Japan-USSR Symposium on Probability Theory, Tashkent, 1975. Lecture Notes in Mathematics, 550, Springer, Berlin, 297–315. 253. Korolyuk, V.S., Turbin, A.F. (1978). Mathematical Foundations of the State Lumping of Large Systems. Naukova Dumka, Kiev, 218 pp. (English edition: Mathematical Foundations of the State Lumping of Large Systems. Mathematics and its Applications, 264, Kluwer, Dordrecht, 1993, x+278 pp.). 254. Koury, J.R., McAllister, D.F., Stewart, W.J. (1984). Iterative methods for computing stationary distributions of nearly completely decomposable Markov chains. SIAM J. Alg. Disc. Melh., 5, 164–186. 255. Kovalenko, I.N. (1965). On the class of limit distributions for thinning flows of homogeneous events. Litov. Mat. Sbornik, 5, 569–573. 256. Kovalenko, I.N. (1973). An algorithm of asymptotic analysis of a sojourn time of Markov chain in a set of states. Dokl. Acad. Nauk Ukr. SSR, Ser. A, no. 6, 422–426. 257. Kovalenko, I.N. (1975). Studies in the Reliability Analysis of Complex Systems. Naukova Dumka, Kiev, 210 pp. 258. Kovalenko, I.N. (1976). An analytic-statistical method for computing the characteristics of systems with high reliability. Kibernetika, no. 6, 82–92 (English translation in Cybernetics 12, no. 6, 895–907). 259. Kovalenko, I.N. (1994). Rare events in queuing theory – a survey. Queuing Syst. Theory Appl., 16, no. 1–2, 1–49. 260. Kovalenko, I.N., Birolini, A., Kuznetsov, N.Yu., Shenbukher, M. (1998). On the reduction of the phase space of a random process in analyzing the reliability of systems. Kibernet. Sistem. Anal., no. 4, 134–144 (English translation in Cybernet. Systems Anal., 34, no. 4, 588–595).

References

387

261. Kovalenko, I.N., Kuznetsov, M.Ju. (1981). Renewal process and rare events limit theorems for essentially multidimensional queueing processes. Math. Operat. Statist., Ser. Statist., 12, no. 2, 211–224. 262. Kovalenko, I.N., Kuznetsov, N.Yu., Pegg, P.A. (1997). Mathematical Theory of Reliability of Time Dependent Systems with Practical Applications. Wiley Series in Probability and Statistics, Wiley, New York, 316 pp. 263. Kovalenko, I.N., Kuznetsov, N.Yu., Shurenkov, V.M. (1996). Models of Random Processes. A Handbook for Mathematicians and Engineers. CRC Press, Boca Raton, FL, 446 pp. (A revised edition of the 1983 Russian original). 264. Kruglov, V.M., Korolev, V.Yu. (1990). Limit Theorems for Random Sums. Izdatel’stvo Moskovskogo Universiteta, Moscow, 269 pp. 265. Kumar, N., Gupta, U.C. (2020). A renewal generated geometric catastrophe model with discrete-time Markovian arrival process. Methodol. Comput. Appl. Probab.. 22, 1293–1324. 266. Kuntz, J., Thomas, P., Stan, G., Barahona, M. (2019).The exit time finite state projection scheme: bounding exit distributions and occupation measures of continuous-time Markov chains. SIAM J. Sci. Comput., 41, no. 2, A748–A769. 267. Kupsa, M., Lacroix, Y. (2005). Asymptotics for hitting times. Ann. Probab., 33, no. 2, 610– 619. 268. Kushnir, O.O., Kushnir, V.P. (2017). A study of characteristics of a highly reliable system with protection assuming that the renewal process is Poisson, Theor. ˘Imovirn. Mat. Stat. 96, 125–130 (English translation in Theory Probab. Math. Statist. 96, 127–132). 269. Labbé, C., Lacoin, H. (2020). Mixing time and cutoff for the weakly asymmetric simple exclusion process Ann. Appl. Probab., 30, no. 4, 1847–1883. 270. Lalley, S.P. (1984). Limit theorems for first-passage times in linear and nonlinear renewal theory. Adv. Appl. Probab., 16, 766–803. 271. Lasserre, J.B. (1994). A formula for singular perturbations of Markov chains. J. Appl. Probab., 31, 829–833. 272. Latouche, G. (1991). First passage times in nearly decomposable Markov chains. In: Stewart, W.J. (Ed). Numerical Solution of Markov Chains. Probability: Pure and Applied, 8, Marcel Dekker, New York, 401–411. 273. Latouche, G., Louchard, G. (1978). Return times in nearly decomposable stochastic processes. J. Appl. Probab., 15, 251–267. 274. Leadbetter, M.R. (1963). On series expansion for the renewal moments. Biometrika, 50, no. 1–2, 75–80. 275. Lévy, P. (1954/56). Processus semi-markoviens. In: Erven, P., Noordhoff, N.V. (Eds.) Proceedings of the International Congress of Mathematicians, Vol. III. Amsterdam, 1954. NorthHolland, Amsterdam, 416–426. 276. Limnios, N. (2012). Reliability measures of semi-Markov systems with general state space. Methodol. Comput. Appl. Probab., 14, no. 4, 895–917. 277. Limnios, N., Oprişan, G. (2001). Semi-Markov Processes and Reliability. Statistics for Industry and Technology, Birkhäuser, Boston, xii+222 pp. 278. Lindvall, T. (1973). Weak convergence of probability measures and random functions in the function space D[0, ∞). J. Appl. Probab., 10, 109–121. 279. Loève, M. (1977). Probability Theory. I. Fourth edition. Graduate Texts in Mathematics, 45, Springer, New York, xvii+425 pp. (First edition in 1955). 280. Lopes, F., Luczak, M. (2020). Extinction time for the weaker of two competing SIS epidemics. Ann. Appl. Probab., 30, no. 6, 2880–2922. 281. Löpker, A., Stadje, W. (2011). Hitting times and the running maximum of Markovian growthcollapse processes. J. Appl. Probab., 48, no. 2, 295–312. 282. Louchard, G., Latouche, G. (1982). Random times in nearly-completely decomposable, transient Markov chains. Cahiers Centre Études Rech. Opér., 24, no. 2–4, 321–352. 283. Louchard, G., Latouche, G. (1990). Geometric bounds on iterative approximations for nearly completely decomposable Markov chains. J. Appl. Probab., 27, no. 3, 521–529.

388

References

284. Mao Yong-Hua, Zhang Chi (2017). Hitting time distributions for birth-death processes with bilateral absorbing boundaries. Probab. Engrg. Inform. Sci., 31, no. 3, 345–356. 285. Marek, I., Mayer, P. (1998). Convergence analysis of an iterative aggregation/disaggregation method for computing stationary probability vectors of stochastic matrices. Numer. Linear Algebra Appl., 5, no. 4, 253–274. 286. Marek, I., Mayer, P., Pultarová, I. (2009). Convergence issues in the theory and practice of iterative aggregation/disaggregation methods. Electron. Trans. Numer. Anal., 35, 185–200. 287. Marek, I., Pultarová, I. (2006). A note on local and global convergence analysis of iterative aggregation-disaggregation methods. Linear Algebra Appl., 413, no. 2–3, 327–341. 288. Martin-Löf, A. (1986). Entropy estimates for the first passage time of a random walk to a time dependent barrier. Scand. J. Statist., 13, no. 3, 221–229. 289. Masol, V.I. (1973). On the asymptotic behaviour of the time of arrival at a high level by a regenerating stable process with independent increments. Teor. Verojatnost. Mat. Stat., 9, 131–136 (English translation in Theory Probab. Math. Statist., 9, 139–144). 290. Masol, V.I. (1974). Limit Distributions for Functionals of Additive Type of Regenerative Processes. Candidate of Science dissertation, Kiev State University. 291. Masol, V.I. (1976). Limit theorems for regenerating random processes that are controlled by a finite Markov chain. Dokl. Akad. Nauk Ukr. SSR, Ser. A, no. 3, 206–209. 292. Masol, V.I., Silvestrov, D.S. (1972). Record values of the occupation time of a semi-Markov process. Visnik Kiev. Univ., Ser. Mat. Meh., 14, 81–89. 293. Masuyama, H. (2015). Error bounds for augmented truncations of discrete-time blockmonotone Markov chains under geometric drift conditions. Adv. Appl. Probab., 47, no. 1, 83–105. 294. McAllister, D.F., Stewart, G.W., Stewart, W.J. (1984). On a Rayleigh-Ritz refinement technique for nearly uncoupled stochastic matrices. Linear Algebra Appl., 60, 1–25. 295. Meshalkin, L.D. (1958). Limit theorems for Markov chains with a finite number of states. Teor. Verojatnost. Primen., 3, 361–385 (English translation in Theory Probab. Appl., 3, 335–357). 296. Meyer, C.D., (1980). The condition of a finite Markov chain and perturbation bounds for the limiting probabilities. SIAM J. Algebr. Discr. Methods, 1, no. 3, 273–283. 297. Meyer, C.D. (1994). Sensitivity of the stationary distribution of a Markov chain. SIAM J. Matrix Anal. Appl., 15, no. 3, 715–728. 298. Meyer, C.D. (2015). Continuity of the Perron root. Linear Multilinear Algebra, 63, no. 7, 1332–1336. 299. Meyer, C.D., Stewart, G.W. (1988). Derivatives and perturbations of eigenvectors. SIAM J. Numer. Anal., 25, no. 3, 679–691. 300. Meyn, S.P., Tweedie, R.L. (1992). Stability of Markovian processes. I. Criteria for discretetime chains. Adv. Appl. Probab., 24, 542–574. 301. Meyn, S.P., Tweedie, R.L. (1993a). Stability of Markovian processes. II. Continuous-time processes and sampled chains. Adv. Appl. Probab., 25, 487–517. 302. Meyn, S.P., Tweedie, R.L. (1993b). Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes. Adv. Appl. Probab., 25, 518–548. 303. Meyn, S.P., Tweedie, R.L. (2009). Markov Chains and Stochastic Stability. Cambridge University Press, xxviii+594 pp. (2nd edition of Markov Chains and Stochastic Stability. Communications and Control Engineering Series, Springer, London, 1993, xvi+ 548 pp.). 304. Mileshina, R.I. (1975). Limit Theorems for Controlled Random Walks. Candidate of Science dissertation, Kiev State University. 305. Mitrophanov, A.Yu. (2003). Stability and exponential convergence of continuous-time Markov chains. J. Appl. Probab., 40, 970–979. 306. Mitrophanov, A.Yu. (2005a). Sensitivity and convergence of uniformly ergodic Markov chains. J. Appl. Probab., 42, 1003–1014. 307. Mitrophanov, A.Yu. (2005b). Ergodicity coefficient and perturbation bounds for continuoustime Markov chains. Math. Ineq. Appl., 8, no 1, 159–168. 308. Mitrophanov, A.Yu. (2006). Stability estimates for finite homogeneous continuous-time Markov chains. Theory Probab. Appl., 50, no 2, 319–326.

References

389

309. Mitrophanov, A.Yu., Lomsadze, A., Borodovsky, M. (2005). Sensitivity of hidden Markov models. J. Appl. Probab., 42, 632–642. 310. Motsa, A.I., Silvestrov, D.S. (1996). Asymptotics of extremal statistics and functionals of additive type for Markov chains. In: Klesov, O., Korolyuk, V., Kulldorff, G., Silvestrov, D. (Eds.) Proceedings of the First Ukrainian–Scandinavian Conference on Stochastic Dynamical Systems, Uzhgorod, 1995. Theory Stoch. Proces., 2(18), no. 1–2, 217–224. 311. Nardi, F.R., Zocca, A., Borst, S.C. (2016). Hitting time asymptotics for hard-core interactions on grids. J. Stat. Phys., 162, no. 2, 522–576. 312. Ni, Y. (2011). Nonlinearly Perturbed Renewal Equations: Asymptotic Results and Applications. Doctoral dissertation, 106, Mälardalen University, Västerås. 313. Ni, Y. (2014). Exponential asymptotical expansions for ruin probability in a classical risk process with non-polynomial perturbations. In: Silvestrov, D., Martin-Löf, A. (Eds.) Modern Problems in Insurance Mathematics. Chapter 6, EAA series, Springer, Cham, 67–91. 314. Nonaka, Y., Ono, H., Sadakane, K., Yamashita, M. (2010). The hitting and cover times of Metropolis walks. Theoret. Comput. Sci., 411, no. 16–18, 1889–1894. 315. Obzherin, Y.E., Boyko, E.G. (2015). Semi-Markov Models. Control of Restorable Systems with Latent Failures. Elsevier/Academic Press, Amsterdam, xiv+199 pp. 316. O’Cinneide, C.A. (1993). Entrywise perturbation theory and error analysis for Markov chains. Numer. Math., 65, 109–120. 317. Pène, F., Saussol, B., Zweimüller, R. (2017). Return- and hitting-time limits for rare events of null-recurrent Markov maps. Ergodic Theory Dynam. Syst., 37, no. 1, 244–276. 318. Peres, Y., Sousi, P. (2015). Mixing times are hitting times of large sets. J. Theoret. Probab., 28, no. 2, 488–519. 319. Petersson, M. (2013). Quasi-stationary distributions for perturbed discrete time regenerative processes. Teor. ˇImovirn. Mat. Stat., 89, 140–155 (Also in Theor. Probab. Math. Statist., 89, 153–168). 320. Petersson, M. (2014). Asymptotics of ruin probabilities for perturbed discrete time risk processes. In: Silvestrov, D., Martin-Löf, A. (Eds). Modern Problems in Insurance Mathematics. Chapter 7, EAA series, Springer, Cham, 93–110. 321. Petersson, M. (2016a). Asymptotic expansions for moment functionals of perturbed discrete time semi-Markov processes. In: Silvestrov, S., Ran˘cić, M. (Eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, Chapter 8, Springer Proceedings in Mathematics & Statistics 179, Springer, Cham, 109—130. 322. Petersson, M. (2016b). Asymptotics for quasi-stationary distributions of perturbed discrete time semi-Markov processes. In: Silvestrov, S., Ran˘cić, M. (Eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, Chapter 8, Springer Proceedings in Mathematics & Statistics 179, Springer, Cham, 131—150. 323. Petersson, M. (2017). Quasi-stationary asymptotics for perturbed semi-Markov processes in discrete time. Methodol. Comput. Appl. Probab., 19, 1047–1074. 324. Poliščuk, L.I., Turbin, A.F. (1973). Asymptotic expansions for certain characteristics of semiMarkov processes. Teor. Verojatnost. Mat. Stat., 8, 122–127 (English translation in Theory Probab. Math. Statist., 8, 121–126). 325. Pollett, P.K., Stewart, D.E. (1994). An efficient procedure for computing quasi-stationary distributions of Markov chains with sparse transition structure. Adv. Appl. Probab., 26, 68– 79. 326. Prokhorov, Yu.V. (1956). Convergence of random processes and limit theorems of probability theory. Teor. Verojatnost. Primen., 1, 117–238 (English translation in Theory Probab. Appl., 1, 157–214). 327. Ragimov, F.H., Farhadova, A.D., Khalilov, V.S. (2018). Limit theorem for the first passage time of the level by the random walk described by nonlinear function of the autoregression process of order one AR(1). Uzbek Math. J., no. 2, 127–135.

390

References

328. Rahimov, F H., Ibadova, I.A., Farhadova, A.D. (2019). Limit theorems for a family of the first passage times of a parabola by the sums of the squares autoregression process of order one (AR(1)). Uzbek Math. J. 2019, no. 2, 81–88. 329. Rényi, A. (1956). A characterization of Poisson processes. Magyar. Tud. Akad. Mat. Kutató Int. K˘ozl., 1, 519–527. 330. Revuz, D. (1984). Markov Chains. North-Holland Mathematical Library, 11, North-Holland, Amsterdam, Elsevier, New York, 374 pp. (Revised version of 1st 1975 edition). 331. Roberts, G.O., Rosenthal, J.S., Schwartz, P.O. (1998). Convergence properties of perturbed Markov chains. J. Appl. Probab., 35, 1–11. 332. Rohlichek, J.R., (1987). Aggregation and Time Scale Analysis of Perturbed Markov Systems. Ph.D. Thesis, Massachusetts Inst. Tech., Cambridge, MA. 333. Rohlicek, J.R., Willsky, A.S. (1988). Multiple time scale decomposition of discrete time Markov chains. Systems Control Lett., 11, no. 4, 309–314. 334. Rolski, T., Schmidli, H., Schmidt, V., Teugels, J. (1999). Stochastic Processes for Insurance and Finance. Wiley Series in Probability and Statistics, Wiley, New York, xviii+654 pp. 335. Ross, S.M. (2014). Introduction to Probability Models. Eleventh edition. Elsevier/Academic Press, Amsterdam, xvi+767 pp. 336. Saas, D. (1972). Classes of limit distributions for sums of a random number of independent identically distributed random variables. Teor. Verojatnost. i Primenen., 17, no. 3, 424–439 (English translation in Theory Probab. Appl., 17, no. 3, 401–415). 337. Saas, D., Frayer, B. (1971). A certain problem of the theory of summing with a random index. Litov. Mat. Sb., 11, 181–187. 338. Schulte-Geers, E., Stadje, W. (2017). Small drift limit theorems for random walks. J. Appl. Probab., 54, no. 1, 199–212. 339. Schweitzer, P.J. (1968). Perturbation theory and finite Markov chains. J. Appl. Probab., 5, 401–413. 340. Schweitzer, P.J. (1984). Aggregation methods for large Markov chains. In: Iazeolla, G., Courtois, P.J., Hordijk, A. (Eds.) Mathematical Computer Performance and Reliability. NorthHolland, Amsterdam, 275–286. 341. Schweitzer, P.J. (1986). Posterior bounds on the equilibrium distribution of a finite Markov chain. Comm. Statist. Stoch. Models, 2, no. 3, 323–338. 342. Schweitzer, P.J. (1987). Dual bounds on the equilibrium distribution of a finite Markov chain. J. Math. Anal. Appl., 126, no. 2, 478–482. 343. Schweitzer, P.J. (1991). A survey of aggregation-disaggregation in large Markov chains. In: Stewart, W.J. (Ed). Numerical solution of Markov chains. Probability: Pure and Applied, 8. Marcel Dekker, New York, 63–88. 344. Seneta, E. (1967). Finite approximations to infinite non-negative matrices. Proc. Cambridge Philos. Soc., 63, 983–992. 345. Seneta, E. (1968a). Finite approximations to infinite non-negative matrices. II. Refinements and applications. Proc. Cambridge Philos. Soc., 64, 465–470. 346. Seneta, E. (1968b). The principle of truncations in applied probability. Comment. Math. Univ. Carolinae, 9, 237–242. 347. Seneta, E. (1973). Nonnegative Matrices. An Introduction to Theory and Applications. Wiley, New York, x+214 pp. 348. Seneta, E. (1980). Computing the stationary distribution for infinite Markov chains. Linear Algebra Appl. 34, 259–267. 349. Seneta, E. (1988a). Sensitivity to perturbation of the stationary distribution: some refinements. Linear Algebra Appl., 108, 121–126. 350. Seneta, E. (1988b). Perturbation of the stationary distribution measured by ergodicity coefficients. Adv. Appl. Probab., 20, 228–230. 351. Seneta, E. (1991). Sensitivity analysis, ergodicity coefficients, and rank-one updates for finite Markov chains. In: Stewart, W.J. (Ed.) Numerical Solution of Markov Chains. Probability: Pure and Applied, 8, Marcel Dekker, New York, 121–129.

References

391

352. Seneta, E. (1993). Sensitivity of finite Markov chains under perturbation. Statist. Probab. Lett., 17, no. 2, 163–168. 353. Seneta, E. (2006). Nonnegative Matrices and Markov chains. Springer Series in Statistics, Springer, New York, 2006, xvi+287 pp. (A revised reprint of 2nd edition of Nonnegative Matrices and Markov Chains. Springer Series in Statistics. Springer, New York, 1981, xiii+279 pp.). 354. Serfozo, R. (2009). Basics of Applied Stochastic Processes. Probability and Its Applications. Springer, Berlin, xiv+443 pp. 355. Serlet, L. (2013). Hitting times for the perturbed reflecting random walk. Stoch. Process. Appl., 123, no. 1, 110–130. 356. Serlet, L. (2018). Laurent Explicit laws for the records of the perturbed random walk on Z. Sém. Probab. XLIX, Lecture Notes in Math., 2215, Springer, Cham, 495–519. 357. Sevast’yanov, B.A. (1995). Extinction probabilities of branching processes bounded from below. Teor. Verojatnost. Primen., 40, 556–564 (English translation in Theory Probab. Appl., 40, 495–502). 358. Sevast’yanov, B.A. (1998) Asymptotic behavior of extinction probabilities of stopped branching processes. Teor. Verojatnost. Primen., 43, 390–397 (English translation in Theory Probab. Appl., 43, 315–322). 359. Sevast’yanov, B.A. (2000). Sojourn times in a finite set of states of Markov branching processes and the probabilities of extinction of a modified Galton-Watson process. Diskret. Mat., 12, no. 4, 39–45 (English translation in Discrete Math. Appl., 10, no. 6, 535–541). 360. Sevast’yanov, B.A. (2006). Asymptotics of the survival probability of a bounded from below Markov critical process with continuous time and infinite variance. Diskret. Mat., 18, no. 1, 3–8 (English translation in Discrete Math. Appl., 16, no. 1, 1–5). 361. Shedler, G.S. (1987). Regeneration and Networks of Queues. Applied Probability. A Series of the Applied Probability Trust. Springer, New York, viii+224 pp. 362. Shedler, G.S. (1993). Regenerative Stochastic Simulation. Statistical Modeling and Decision Science. Academic Press, Boston, MA, x+400 pp. 363. Silvestrov, D.S. (1969a). Limit theorems for a non-recurrent walk connected with a Markov chain. Ukr. Mat. Ž., 21, 790–804 (English translation in Ukr. Math. J., 21, 657–669). 364. Silvestrov, D.S. (1969b). Limit Theorems for Semi-Markov Processes and their Applications to Random Walks. Candidate of Science dissertation, Kiev State University. 365. Silvestrov, D.S. (1969c). Asymptotic behaviour of the first passage time for sums of random variables controlled by a regular semi-Markov process. Dokl. Akad. Nauk SSSR, 189, 949– 951 (English translation in Soviet Math. Dokl., 10, 1541–1544). 366. Silvestrov, D.S. (1970a). Limit theorems for semi-Markov processes and their applications.1. Teor. Verojatnost. Mat. Stat., 3, 155–172 (English translation in Theory Probab. Math. Statist., 3, 159–176). 367. Silvestrov, D.S. (1970b). Limit theorems for semi-Markov processes and their applications.2. Teor. Verojatnost. Mat. Stat., 3, 173–194 (English translation in Theory Probab. Math. Statist., 3, 177–198). 368. Silvestrov, D.S. (1971a). Limit theorems for semi-Markov summation schemes. 1. Teor. Verojatnost. Mat. Stat., 4, 153–170 (English translation in Theory Probab. Math. Statist., 4, 141–157). 369. Silvestrov, D. S. (1971b). Uniform estimates of the rate of convergence for sums of random variables defined on a finite homogeneous Markov chain with absorption. Theor. Verojatnost. Mat. Stat., 5, 116–127 (English translation in Theory Probab. Math. Statist., 5, 123–135). 370. Silvestrov, D.S. (1971c). Limit distributions for compositions of random functions. Dokl. Akad. Nauk SSSR, 199, 1251–1252 (English translation in Soviet Math. Dokl., 12, 1282– 1285). 371. Silvestrov, D.S. (1971d). Limit theorems for semi-Markov processes. Dokl. Acad. Nauk USSR. Ser. A, no. 11, 987–989.

392

References

372. Silvestrov, D.S. (1972a). Limit distributions for sums of random variables that are defined on a countable Markov chain with absorption. Dokl. Acad. Nauk Ukr. SSR, Ser. A, no. 4, 337–340. 373. Silvestrov, D.S. (1972b). Estimation of the rate of convergence for sums of random variables that are defined on a countable Markov chain with absorption. Dokl. Acad. Nauk Ukr. SSR, Ser. A, no. 5, 436–438. 374. Silvestrov, D.S. (1972c). Limit Theorems for Composite Random Functions. Doctor of Science dissertation, Kiev State University. 375. Silvestrov, D.S. (1972d). Remarks on the limit of composite random function. Teor. Verojatnost. Primen., 17, no. 4, 707–715 (English translation in Theory Probab. Appl., 17, no. 4, 669–677). 376. Silvestrov, D.S. (1972e). On convergence of complex random function in the J topology. Dokl. Akad. Nauk SSSR, 202, 539–540 (English translation in Soviet Math. Dokl., 13, 152–154). 377. Silvestrov, D.S. (1973). Conditions for convergence of the superposition of random processes in the J topology. Teor. Verojatnost. Primen., 18, 605–607 (English translation in Theory Probab. Appl., 18, 579–582). 378. Silvestrov, D.S. (1974). Limit Theorems for Composite Random Functions. Vysshaya Shkola and Izdatel’stvo Kievskogo Universiteta, Kiev, 318 pp. 379. Silvestrov, D.S. (1979). A remark on limit distributions for times of attainment for asymptotically recurrent Markov chains. Theor. Sluch. Proces., 7, 106–109. 380. Silvestrov, D.S. (1980a). Semi-Markov Processes with a Discrete State Space. Library for the Engineer in Reliability, Sovetskoe Radio, Moscow, 272 pp. 381. Silvestrov, D.S. (1980b). Mean hitting times for semi-Markov processes, and queueing networks. Elektron. Infor. Kybern., 16, 399–415. 382. Silvestrov, D.S. (1981). Theorems of large deviations type for entry times of a sequence with mixing. Teor. Verojatnost. Mat. Stat., 24, 129–135 (English translation in Theory Probab. Math. Statist., 24, 145–151). 383. Silvestrov, D.S. (1995). Exponential asymptotic for perturbed renewal equations. Teor. ˘Imovirn. Mat. Stat., 52, 143–153 (English translation in Theory Probab. Math. Statist., 52, 153–162). 384. Silvestrov, D.S. (1996a). Recurrence relations for generalised hitting times for semi-Markov processes. Ann. Appl. Probab., 6, 617–649. 385. Silvestrov, D.S. (1996b). Hitting times for semi-Markov dynamical systems. Theory Stoch. Process. 2(18), no. 1–2, 260–268. 386. Silvestrov, D.S. (2000a). Nonlinearly perturbed Markov chains and large deviations for lifetime functionals. In: Limnios, N., Nikulin, M. (Eds.) Recent Advances in Reliability Theory: Methodology, Practice and Inference. Birkhäuser, Boston, 135–144. 387. Silvestrov, D.S. (2000b). Limit theorems for generalised exceeding times, renewal and risk type processes. Theory Stoch. Process. 6(22), no. 3–4, 177–238. 388. Silvestrov D.S. (2004). Limit Theorems for Randomly Stopped Stochastic Processes. Probability and Its Applications, Springer, London, xvi+398 pp. 389. Silvestrov, D.S. (2007). Asymptotic expansions for quasi-stationary distributions of nonlinearly perturbed semi-Markov processes. Theory Stoch. Process., 13, no. 1–2, 267–271. 390. Silvestrov D.S. (2010). Nonlinearly perturbed stochastic processes and systems. In: Rykov, V., Balakrishnan, N., Nikulin, M. (Eds). Mathematical and Statistical Models and Methods in Reliability, Chapter 2, Birkhäuser, 19–38. 391. Silvestrov D.S. (2014a). Improved asymptotics for ruin probabilities. In: Silvestrov, D., Martin-Löf, A. (Eds). Modern Problems in Insurance Mathematics, Chapter 5, EAA series, Springer, Cham, 93–110. 392. Silvestrov D.S. (2014b). American-Type Options. Stochastic Approximation Methods, Volume 1. De Gruyter Studies in Mathematics, 56, Walter de Gruyter, Berlin, x+509 pp. 393. Silvestrov D.S. (2015). American-Type Options. Stochastic Approximation Methods, Volume 2. De Gruyter Studies in Mathematics, 57, Walter de Gruyter, Berlin, xi+558 pp.

References

393

394. Silvestrov, D. (2016). Necessary and sufficient conditions for convergence of first-rare-event times for perturbed semi-Markov processes. Theor. ˘Imovirn. Mat. Stat. 95, 119–137 (Also in Theory Probab. Math. Statist. 95, 135–151). 395. Silvestrov, D. (2019). Algorithms of phase space reduction and asymptotics of hitting times for perturbed semi-Markov processes. Research Report 2019:12, Department of Mathematics, Stockholm University, 153 pp. 396. Silvestrov, D.S., Abadov, Z.A. (1984). Asymptotic behaviour for exponential moments of sums of random variables defined on exponentially ergodic Markov chains. Dokl. Acad. Nauk Ukr. SSR, Ser. A, no. 4, 23–25. 397. Silvestrov, D.S., Abadov, Z.A. (1991). Uniform representations of exponential moments of sums of random variables defined on a Markov chain, and of distributions of passage times. 1. Teor. Verojatnost. Mat. Stat., 45, 108–127 (English translation in Theory Probab. Math. Statist., 45, 105–120). 398. Silvestrov, D.S., Abadov, Z.A. (1993). Uniform representations of exponential moments of sums of random variables defined on a Markov chain, and of distributions of passage times. 2. Teor. Verojatnost. Mat. Stat., 48, 175–183 (English translation in Theory Probab. Math. Statist., 48, 125–130). 399. Silvestrov, D.S., Drozdenko, M.O. (2005). Necessary and sufficient conditions for the weak convergence of the first-rare-event times for semi-Markov processes. Dopov. Nac. Akad. Nauk Ukr., Mat. Prirodozn. Tekh. Nauki, no. 11, 25–28. 400. Silvestrov, D.S., Drozdenko, M.O. (2006a). Necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes. I. Theory Stoch. Process., 12(28), no. 3–4, 151–186. 401. Silvestrov, D.S., Drozdenko, M.O. (2006b). Necessary and sufficient conditions for weak convergence of first-rare-event times for semi-Markov processes. II. Theory Stoch. Process., 12(28), no. 3–4, 187–202. 402. Silvestrov, D., Manca, R. (2017a). Recurrent Algorithms for mixed power-exponential moments of hitting times for semi-Markov processes. Proceedings of the 17th Applied Stochastic Models and Data Analysis International Conference, London, 2017. ISAST: International Society for the Advancement of Science and Technology, 905–923. 403. Silvestrov, D., Manca, R. (2017b). Reward algorithms for semi-Markov processes. Methodol. Comput. Appl. Probab., 19, no. 4, 1191–1209. 404. Silvestrov, D., Manca, R., Silvestrova, E. (2014). Computational algorithms for moments of accumulated Markov and semi-Markov rewards. Comm. Statist. Theory, Meth., 43, no. 7, 1453–1469. 405. Silvestrov, D., Petersson, M., Hössjer, O. (2018). Nonlinearly perturbed birth-death-type models. In: Silvestrov, S., Ran˘cić, M., Malyarenko, A. (Eds.) Stochastic Processes and Applications, Chapter 11, Springer Proceedings in Mathematics & Statistics, 271, Springer, Cham, 189–244. 406. Silvestrov, D., Silvestrov, S. (2016). Asymptotic expansions for stationary distributions of perturbed semi-Markov processes. In: Silvestrov, S., Ran˘cić, M. (Eds.) Engineering Mathematics II. Algebraic, Stochastic and Analysis Structures for Networks, Data Classification and Optimization, Chapter 10, Springer Proceedings in Mathematics & Statistics, 179, Springer, Cham, 151–222. 407. Silvestrov, D., Silvestrov, S. (2017a). Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes.1. Methodol. Comput. Appl. Probab., https:// doi.org/10.1007/s11009-017-9605-0, 20 pp., 21, no. 3 (2019), 945–964. 408. Silvestrov, D., Silvestrov, S. (2017b). Asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes.2. Methodol. Comput. Appl. Probab., https:// doi.org//10.1007/s11009-017-9607-yh, 20 pp., 21, no. 3 (2019), 965–984. 409. Silvestrov, D., Silvestrov, S. (2017c). Nonlinearly Perturbed Semi-Markov Processes. Springer Briefs in Probability and Mathematical Statistics, Springer, Cham, xiv+143 pp.

394

References

410. Silvestrov, D., Silvestrov, S. (2017d). Asymptotic expansions for power-exponential moments of hitting times for nonlinearly perturbed semi-Markov processes. Theor. ˘Imovirn. Mat. Stat. 97, 171–187 (Also in Theory Probab. Math. Statist. 97, 183–200). 411. Silvestrov, D., Silvestrov, S., Abola, B., Biganda, P.S., Engstrröm, C., Mango, J.M., Kakuba, G. (2020a). Perturbation analysis for stationary distributions of Markov chains with damping component. In: Silvestrov, S., Ran˘cić, M., Malyarenko, A. (Eds.) Algebraic Structures and Applications, Chapter 38. Springer Proceedings in Mathematics & Statistics, 317, Springer, Cham, 903–934. 412. Silvestrov, D., Silvestrov, S., Abola, B., Biganda, P.S., Engstrröm, C., Mango, J.M., Kakuba, G. (2020b). Perturbed Markov chains with damping component. Methodol. Comput, Appl, Probab., https://doi.org/10.1007/s11009-020-09815-9, 31 pp., 23, no. 1 (2021), 369–397. 413. Silvestrov, D.S., Stenflo, Ö. (1998). Ergodic theorems for iterated function systems controlled by regenerative sequences. J. Theor. Probab., 11, 589–608. 414. Silvestrov, D.S., Teugels, J.L. (2004). Limit theorems for mixed max-sum processes with renewal stopping. Ann. Appl. Probab., 14, 1838–1868. 415. Silvestrov, D.S., Veliki˘ı, Yu.A. (1988). Necessary and sufficient conditions for the convergence of attainment times. In: Zolotarev, V.M., Kalashnikov, V.V. (Eds.) Stability Problems for Stochastic Models. Trudy Seminara, VNIISI, Moscow, 129–137 (English translation in J. Soviet. Math., 57, (1991), 3317–3324). 416. Simon, H.A., Ando, A. (1961). Aggregation of variables in dynamic systems. Econometrica, 29, 111–138. 417. Sirl, D., Zhang, H., Pollett, P. (2007). Computable bounds for the decay parameter of a birth-death process. J. Appl. Probab., 44, no. 2, 476–491. 418. Skorokhod, A.V. (1956). Limit theorems for stochastic processes. Teor. Verojatnost. Primen., 1, 289–319 (English translation in Theory Probab. Appl., 1, 261–290). 419. Skorokhod, A.V. (1957). Limit theorems for processes with independent increments. Teor. Verojatnost. Primen., 2, 145–177 (English translation in Theory Probab. Appl., 2, 138–171) 420. Skorokhod, A.V. (1964). Random Processes with Independent Increments. Probability Theory and Mathematical Statistics, Nauka, Moscow, 278 pp. (English edition: Nat. Lending Library for Sci. and Tech., Boston Spa, 1971). 421. Skorokhod, A.V. (1986). Random Processes with Independent Increments. Second edition, Probability Theory and Mathematical Statistics, Nauka, Moscow, 320 pp. (English edition: Mathematics and its Applications, 47, Kluwer, Dordrecht, 1991, xii+279 pp.). 422. Slepov, N.A. (2021). The convergence rate of random geometric sum distribution to the Laplace law. Teor. Verojatnost. Primen., 66, no. 1, 149–174. 423. Sloothaak, F., Wachtel, V., Zwart, B. (2018). First-passage time asymptotics over moving boundaries for random walk bridges. J. Appl. Probab., 55, no. 2, 627–651. 424. Smith, W.L. (1955). Regenerative Stochastic Processes. Proceedings of the Royal Society, Ser. A: Mathematical, Physical and Engineering Sciences, 232, 6–31. 425. Solov’ev, A.D. (1966). A combinatorial identity and its application to the problem on the first occurrence of a rare event. Teor. Verojatnost. Primen., 11, 313–320 (English translation in Theory Probab. Appl., 11, 276–282). 426. Solov’ev, A.D. (1971). Asymptotical behaviour of the first occurrence time of a rare event in a regenerative process. Izv. Akad. Nauk SSSR, Tekhn. Kibernet., no. 6, 79–89 (English translation in Engrg. Cybern., 9, no. 6, 1038–1048). 427. Solov’ev, A.D. (1972). Asymptotic distribution of the moment of first crossing of a high level by a birth and death process. In: Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, 1970/1971, Vol. III: Probability theory. Univ. California Press, Berkeley, CA, 71–86. 428. Stewart, G.W. (1969). On the continuity of the generalized inverse. SIAM J. Appl. Math., 17, 33–45. 429. Stewart, G.W. (1973). Error and perturbation bounds for subspaces associated with certain eigenvalue problems. SIAM Rev., 15, 727–764.

References

395

430. Stewart, G.W. (1979). Perturbation bounds for the definite generalized eigenvalue problem. Linear Algebra Appl., 23, 69–85. 431. Stewart, G.W. (1983). Computable error bounds for aggregated Markov chains. J. Assoc. Comput. Mach., 30, no. 2, 271–285. 432. Stewart, G.W. (1984a). On the structure of nearly uncoupled Markov chains. In: Iazeolla, G, Courtois, P.J., Hordijk, A. (Eds). Mathematical Computer Performance and Reliability. North-Holland, Amsterdam, 287–302. 433. Stewart, G.W. (1984b). A second order perturbation expansion for small singular values. Linear Algebra Appl., 56, 231–235. 434. Stewart, G.W. (1990). Stochastic perturbation theory. SIAM Rev., 32, no. 4, 579–610. 435. Stewart, G.W. (1991). On the sensitivity of nearly uncoupled Markov chains. In: Stewart, W.J. (Ed.) Numerical Solution of Markov Chains. Probability: Pure and Applied, 8, Marcel Dekker, New York, 105–119. 436. Stewart, G.W. (1993a). Gaussian elimination, perturbation theory, and Markov chains. In: Meyer, C.D., Plemmons, R.J. (Eds.) Linear Algebra, Markov Chains, and Queueing Models. IMA Volumes in Mathematics and its Applications, 48, Springer, New York, 59–69. 437. Stewart, G.W. (1993b). On the perturbation of Markov chains with nearly transient states. Numer. Math., 65, no. 1, 135–141. 438. Stewart, W.J. (1994). Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton, NJ, xx+539 pp. 439. Stewart, G.W. (1998). Matrix Algorithms. Vol. I. Basic Decompositions. SIAM, Philadelphia, PA, xx+458 pp. 440. Stewart, G.W. (2001). Matrix Algorithms. Vol. II. Eigensystems. SIAM, Philadelphia, PA, xx+469 pp. 441. Stewart, G.W. (2003). On the powers of a matrix with perturbations. Numer. Math., 96, no. 2, 363–376. 442. Stewart, G.W., Stewart, W.J., McAllister, D.F. (1994). A two-stage iteration for solving nearly completely decomposable Markov chains. In: Golub, G., Greenbaum, A., Luskin, M. (Eds.) Recent Advances in Iterative Methods. IMA Volumes in Mathematics and its Applications, 60, Springer, New York, 201–216. 443. Stewart, G.W., Sun, J.G. (1990). Matrix Perturbation Theory. Computer Science and Scientific Computing. Academic Press, Boston, xvi+365 pp. 444. Stewart, G.W., Zhang, G. (1991). On a direct method for the solution of nearly uncoupled Markov chains. Numer. Math., 59, no. 1, 1–11. 445. Stewart, W.J. (2009). Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling. Princeton University Press, Princeton, NJ, xviii+758 pp. 446. Stone, C. (1963). Weak convergence of stochastic processes defined on semi-infinite time intervals. Proc. Amer. Math. Soc., 14, 694–696. 447. Sudakova, E.V. (1990). Exponential asymptotics of sums of geometrical number of independent random variables. Theor. ˇImovirn. Mat. Stat., 42, 161–166 (English translation in Theory Probab. Math. Statist., 42, 135–139). 448. Sumita, U., Reiders, M. (1988). A new algorithm for computing the ergodic probability vector for large Markov chains: Replacement process approach. Probab. Eng. Infor. Sci., 4, 89–116. 449. Sun, W., Feuillet, M., Robert, P. (2016). Analysis of large unreliable stochastic networks. Ann. Appl. Probab., 26, no. 5, 2959–3000. 450. Takács, L. (1954). Some investigations concerning recurrent stochastic processes of a certain type. Magyar. Tud. Akad. Mat. Kutató Int. K˘ozl., 3, 114–128. 451. Thorisson, H. (2000). Coupling, Stationarity and Regeneration. Probability and its Applications, Springer, New York, xiv+517 pp. 452. Turbin, A.F. (1971). On asymptotic behavior of time of a semi-Markov process being in a reducible set of states. Linear case. Teor. Verojatnost. Mat. Stat., 4, 179–194 (English translation in Theory Probab. Math. Statist., 4, 167–182). 453. Turbin, A.F. (1972). An application of the theory of perturbations of linear operators to the solution of certain problems that are connected with Markov chains and semi-Markov

396

454. 455. 456.

457. 458. 459. 460.

461. 462.

463. 464. 465. 466.

467.

468. 469.

470. 471.

472. 473. 474.

References processes. Teor. Verojatnost. Mat. Stat., 6, 118–128 (English translation in Theory Probab. Math. Statist., 6, 119–130). Tweedie, R.L. (1980). Perturbations of countable Markov chains and processes. Ann. Inst. Statist. Math., 32, no. 2, 283–290. Tweedie, R.L. (1998). Truncation approximations of invariant measures for Markov chains. J. Appl. Probab., 35, 517–536. Vakhtel’, V.I., Denisov, D.È. (2015). Exact asymptotics for the instant of crossing a curved boundary by an asymptotically stable random walk. Teor. Verojatnost. Primen., 60, no. 3, 459–481 (English translation in Theory Probab. Appl., 60, no. 3, 481–500). van Doorn, E.A. (2017). An orthogonal-polynomial approach to first-hitting times of birthdeath processes. J. Theoret. Probab., 30, no. 2, 594–607. Vantilborgh, H. (1985). Aggregation with an error of O( 2 ). J. Assoc. Comput. Mach., 32, no. 1, 162–190. Vo˘ına, A.A. (1978). Limit Theorems for Shemes of Summation on Processes with Arbitrary Sets of States and their Applications. Candidate of Science dissertation, Kiev State University. Vo˘ına, A.A. (1979). Sojourn time in a fixed subset and the consolidation of states of random processes with an arbitrary phase space. Theor. Verojatnost. Mat. Stat., 20, 30–39 (English translation in Theory Probab. Math. Statist., 20, 35–44). Volker, N. (1980). Semi-Markov Processes. Scientific Paperbacks, Mathematics/Physics Series, 260. Akademie-Verlag, Berlin, 162 pp. Vsekhsvyatskii, S.Yu., Kalashnikov, V.V. (1988). Estimates in the Rényi theorem in terms of renewal theory. Teor. Verojatnost. Primen., 33, no. 2, 369–373 (English translation in Theory Probab. Appl., 33, no. 2, 347–352). Vysotsky, V. (2015). Limit theorems for random walks that avoid bounded sets, with applications to the largest gap problem. Stoch. Process. Appl., 125, no. 5, 1886–1910. Wang Zi Kun (1980). Sojourn times and first passage times for birth and death processes. Sci. Sinica, 23, no. 3, 269–279. Whitt, W. (1980). Some useful functions for functional limit theorems, Math. Oper. Res., 5, 67–85. Whitt, W. (2002). Stochastic-Processes Limits. An Introduction to Stochastic-Process Limits and their Applications to Queues. Springer Series in Operations Research, Springer, New York, 602 pp. Woodroofe, M. (1982). Nonlinear Renewal Theory in Sequential Analysis. CBMS-NSF Regional Conference Series in Applied Mathematics, 39, Soc. Industr. Appl. Math., Philadelphia, v+113 pp. Yin, G.G., Zhang, Q. (1998). Continuous-time Markov Chains and Applications. A Singular Perturbation Approach. Applications of Mathematics, 37, Springer, New York, ivx+349 pp. Yin, G., Zhang, Q. (2003). Discrete-time singularly perturbed Markov chains. In: Yao, D.D., Zhang, H., Zhou, X.Y. (Eds). Stochastic Modeling and Optimization. Springer, New York, 1–42. Yin, G.G., Zhang, Q. (2005). Discrete-Time Markov Chains. Two-Time-Scale Methods and Applications. Stochastic Modelling and Applied Probability, Springer, New York, xix+348 pp. Yin, G.G., Zhang, Q. (2013). Continuous-Time Markov Chains and Applications. A TwoTime-Scale Approach. Stochastic Modelling and Applied Probability, 37. Springer, New York, xxii+427 pp. (2nd revised edition of Continuous-Time Markov Chains and Applications. A Singular Perturbation Approach. Applications of Mathematics, 37, Springer, New York, 1998. xvi+349 pp.). Yin, G., Zhang, Q., Badowski, G. (2000a). Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. Ann. Appl. Probab., 10, 549–572. Yin, G., Zhang, Q., Badowski, G. (2000b). Occupation measures of singularly perturbed Markov chains with absorbing states. Acta Math. Sin., 16, no. 1, 161–180. Yin, G., Zhang, Q., Badowski, G. (2000c). Singularly perturbed Markov chains: convergence and aggregation. J. Multivar. Anal., 72, no. 2, 208–229.

References

397

475. Yin, G., Zhang, Q., Badowski, G. (2003). Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit. Adv. Appl. Probab., 35, no. 2, 449–476. 476. Yin, G., Zhang, Q., Yang, H., Yin, K. (2001). Discrete-time dynamic systems arising from singularly perturbed Markov chains. In: Proceedings of the Third World Congress of Nonlinear Analysts, Part 7, Catania, 2000. Nonlinear Anal., 47, no. 7, 4763–4774. 477. Zakusilo, O.K. (1972a). Thinning semi-Markov processes. Teor. Verojatnost. Mat. Stat., 6, 54–59 (English translation in Theory Probab. Math. Statist., 6, 53–58). 478. Zakusilo, O.K. (1972b). Necessary conditions for convergence of semi-Markov processes that thin. Teor. Verojatnost. Mat. Stat., 7, 65–69 (English translation in Theory Probab. Math. Statist., 7, 63–66). 479. Zakusilo, O.K., Matsak, ¯I.K. (2017). On extreme values of some regenerative processes. Theor. ˘Imovirn. Mat. Stat. 97, 58–71 (English translation in Theory Probab. Math. Statist. 97, 57–71). 480. Zakusilo, O.K., Matsak, ¯I.K. (2021). Estimates for the rate of convergence in the limit theorem for extreme values of regenerative processes. Ukraïn. Mat. Zh., 72, no. 8, 1064–1081 (English translation in Ukrainian Math. J., 72, no. 8, 1230–1250). 481. Zhang, Q., Yin, G. (2004). Exponential bounds for discrete-time singularly perturbed Markov chains. J. Math. Anal. Appl., 293, no. 2, 645–662. 482. Zubkov, A.M., Miha˘ılov, V.G. (1974). Limit distributions of random variables that are connected with long duplications in a sequence of independent trials. Teor. Verojatnost. Primen., 19, 173–181 (English translation in Theory Probab. Appl., 19, 172–179). 483. Zweimüller, R. (2019). Hitting-time limits for some exceptional rare events of ergodic maps. Stochastic Process. Appl., 129, no. 5, 1556–1567.

Index

Symbols J-convergence, 356 J-topology, 31, 356 U-convergence, 355 U-topology, 31, 355 A Algorithm recurrent phase space reduction, 20, 250, 252, 255 Approximation diffusion for ruin probabilities, 144 stable for risk processes, 144 B Buffer bounded queue, 166 unbounded queue, 188 C Coefficient loading, 142 Condition absorbing rate, 254 of asymptotic comparability, 225, 227 local absorbing rate, 250 perturbation, 221–224, 329 regularity, 220 Convergence in distribution, 31 in probability, 31 Convolution, 229 n-fold, 229

Criterion central of convergence, 36 Cumulant, 36 D Device Wold-Cramér, 96, 330 Distribution of hitting time, 262 Markov phase type, 293 semi-Markov phase type, 293 of sojourn time, 220 stationary, 32 steady claim, 142 E Ergodicity asymptotically uniform, 33 Expectation of hitting time, 262 F Family of asymptotically comparable functions, 12, 204 of asymptotically comparable powerexponential-type functions, 211 of asymptotically comparable powerlogorithmic-type functions, 214 of asymptotically comparable power-type functions, 208 complete of asymptotically comparable functions, 204

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 D. Silvestrov, Perturbed Semi-Markov Type Processes I, https://doi.org/10.1007/978-3-030-92403-4

399

400 Flow of rare events, 68 Formula Pollaczek–Khinchine, 142 Function claim distribution, 142 non-ruin distribution, 142 normalisation, 222, 254, 280, 282–284 Functional hitting reward, 330 real-valued hitting reward, 331, 332 vector hitting reward, 330, 331 I Indicator failure, 2 of rare event, 3, 6, 134 Instant of jump, 220 L Limit comparability, 204 M Markov chain asymptotically uniformly ergodic, 7 continuous time, 17, 220 discrete time, 18, 220 embedded, 220, 334 Matrix generator, 11, 146 Model epidemic, 336 population genetics, 336 with regular perturbations, 19 with semi-regular perturbations, 19 with singular perturbations, 19 Modulus of J-compactness, 357 of U-compactness, 356 Moment of jump, 3, 5, 6, 334 N Number of jumps, 220, 334 P Parameter comparability, 208, 211, 215 Perturbation regular, 221 singular, 221

Index Probability hitting, 262, 277 non-ruin, 142 stationary, 32 transition, 220, 333 Process counting, 68 first-rare-event, 6, 31 with independent increments, 36 Lévy, 36, 332 lower rare-event-time, 100 Markov renewal, 2, 6, 17, 30, 133, 220, 330 Markov renewal of birth-death type, 333 Markov renewal with reduced phase space, 240 Markov renewal with removed virtual transitions, 229 Markov renewal with transition period, 102 Poisson, 142 rare-event-time, 134 reduced Markov renewal, 239 reduced semi-Markov, 240 risk, 142 semi-Markov, 3, 6, 17, 220, 334 semi-Markov with removed virtual transitions, 229 stable Lévy, 333 standard renewal, 69 step-sum reward, 35 step-sum with independent increments, 36 upper rare-event-time, 100 vector counting, 86 vector first-rare-event, 94 vector Markov renewal, 94 vector rare-event counting, 88 R Reward accumulated, 328 S Space phase, 220, 333 Sum geometric random, 141 multi-dimensional random, 332 Superposition of stochastic processes, 359 System closed M/M-type queuing, 146, 151 M/M queuing, 166, 188 M/M(m/n) queuing, 336 queuing, 2

Index T Theorem Slutsky, 360 Time directed hitting, 126 failure, 151 first failure, 3, 11 first hitting, 262 first-rare-event, 3, 6, 31, 134 inter-rare-event, 68

401 lower first-rare-event, 98 rare-event, 68 repairing, 151 return, 294 sojourn, 220 upper first-rare-event, 98 vector first-rare-event, 94 Transform Laplace for distribution of hitting time, 262