Lectures on the Theory of Stochastic Processes [Reprint 2018 ed.] 9783110618167, 9789067642064

Citation preview

Lectures on the Theory of Stochastic Processes

LECTURES ON THE THEORY OF STOCHASTIC PROCESSES A. V.

M MSP M

Utrecht, The Netherlands

SKOROKHOD

1996

«TBiMC» Kiev, Ukraine

VSP B V P.O. Box 346 3700 AH Zeist The Netherlands

© VSP BV 1996 First published in 1996 ISBN 90-6764-206-1

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner.

Printed in The Netherlands by Koninklijke Wöhrmann BV, Zutphen.

CONTENTS Preface Chapter I Chapter II Chapter III Chapter Chapter Chapter Chapter Chapter Chapter

IV V VI VII VIII IX

Chapter X Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter

XI XII XIII XIV XV XVI XVII XVIII XIX XX

Chapter XXI Chapter XXII Chapter XXIII Chapter XIV Chapter XXV Chapter XXVI Chapter XXVII

: Stochastic Processes. Definitions. Examples : The Kolmogorov consistency theorem. Classification of processes : Random Walks. Recurrence. Renewal Theorem : Martingales. Inequalities for martingales : Theorems on the limit of a martingale : Stationary sequences. Ergodic theorem : Ergodic theorem. Metric transitivity : Regularization of a process. Continuity : Processes without discontinuities of the second kind : Continuity of processes with independent increments. Martingales with continuous time : Measurable processes : Stopping times. Associated tr-algebras : Completely measurable processes : ¿^-theory : Stochastic integrals : Stationary processes. Spectral representations : Stationary sequences. Regularity and singularity : The prediction of a stationary sequence : Markov processes : Homogeneous Markov processes and associated semigroups : Homogeneous purely discontinuous processes. Conditions for their regularity : Processes with a denumerable set of states : Simple birth and death processes : Branching processes with particles of only one kind : Homogeneous processes and strongly continuous semigroups. Resolvent operator and generator : The Hille-Iosida theorem : Processes with independent increments. Representation of the discontinuous part

1 3 8 14 21 27 34 39 44 50 56 61 65 69 73 80 87 93 98 104 110 115 120 127 132 137 143 149

vi

Chapter XXVIII Chapter XXIX Chapter XXX Chapter XXXI Chapter XXXII References

contents

: General representation of a stochastically continuous process with independent increments : Diffusion processes : Stochastic integrals : Existence, uniqueness, and properties of solutions of stochastic differential equations : It6's formula with some corollaries

155 160 166 172 178 184

PREFACE

This is a text book for a one year course on stochastic processes for graduate students in mathematics and engineering. This book is evolved from several stacks of lecture notes on the theory of stochastic processes given many times by the author for students at the Department of Mathematics of the National University of Ukraine, Kiev, Ukraine. In transforming the overlapping material into a book, I aimed to adhere to the lecture style. The result is that the material is broken up according to the lecture time (one hour and a half), so a topic may occur in several Lectures. On the other hand, occasionally, it was necessary to include different topics into a Lecture because one "homogeneous" topic does not always fill a Lecture. I tried, however, to minimize such cases. In general, I believe t h a t the style used in this book is more convenient for lecturers and students, especially at those Universities where there is no Department of Stochastic Processes but a course on stochastic processes should be given for students. To understand the mathematics in the Lectures, the reader should be familiar with basic notions of Mathematical Analysis, Theory of Complex Functions, Theory of Differential Equations, and Probability Theory. Usually these topics are given at Universities over the first two years of study. Those wishing to read this book will easily understand what mathematical tools they need to learn before hand. The subjects included in the book are different in levels of abstraction. Engineers and students of natural sciences require a somewhat lower level compared with pure mathematic. For this reason, a lecturer should separate Lectures t h a t are needed for his or her students. First of all, a lecturer may exclude Lectures on martingales, general theory of measurability, semi-groups for Markov processes, stochastic differential equations. In so doing, the course will be shorter by a half and will meet in more detail needs of applications. Those wishing to know more about stochastic processes may turn to the book included in the References.

July 1995, Kiev

A. Skorokhod

LECTURE 1 STOCHASTIC

PROCESSES.

DEFINITIONS.

EXAMPLES

The theory of stochastic processes studies infinite families of random variables. We can observe such families in nature, in technology, and in society. Random variables can depend either on an integer parameter or on a real parameter, regarded as a time, or on a vector-valued parameter. In these cases, families of random variables are called either random sequences or stochastic processes or random fields. A stochastic process is regarded as a state of a certain system that is randomly varying with time. If states of this system are observed at discrete time moments then we deal with a random sequence or with a stochastic process with discrete time. Random fields describe changes of random variables with space and time. In what follows, if we are referring to random events, we keep in mind that a probability space is given. Families of random variables considered in Lectures are supposed to be defined on the same probability space. Let us recall some of properties of a probability space. A probability space is a triple ( 0 , 5 , P)i where Í2 is a certain nonempty set called as a set of elementary events, J is a collection of subsets of Ct forming a cr-algebra, and P is a measure on 5 such that P ( 0 ) = 1. Sets from 5 are referred to as random events and P(^4) is referred to as the probability of a set A. A collection 21 of sets from H is an algebra if

ai) ilea, a2) A\B e 21 whenever A € 21 and B G 21. Under these conditions, A U B G 21 and A n B 6 21 whenever A, B E 21. An algebra 5 is a a -algebra if in addition to a l ) and a2) a3) UÍT=i ¿n e 5 whenever An e 5, n > 1. For the sake of completeness we give here the definition of a measure, too.

A

measure is an additive continuous function of sets defined on J, i.e. m l ) P(i4 U B ) = P(A) + P (B) m2

) P (IXLi A O =

l i m n->°o

whenever A,B

with A n B = 0 ,

P ( ^ n ) whenever An C 5 with An C An+i,

n > 1.

If a probability space is given then random events, that is sets from 5, are defined. Each measurable function £ acting from fl to R , that is fi —• R , is said to be a random variable. This means that {U;:£(U;) < I } 6 5 for all i G R . It follows from this property that {w: £ B} € 5 where B G © R and © R is the set of Borel subsets of R. A measurable vector acting from Í1 and having values in R M is said to be a random vector. So, a vector x(u>):Cl —> R M is called a random vector if { w : x(u) £ B } e J for all B £ © r ™ , where B r ™ is the set of Borel subsets of R M . A set X equipped with a c-algebra © is said to be a measurable space. measurable mapping ( X , © ) , i.e. a mapping such that

A €

4

LECTURES ON THE THEORY OF STOCHASTIC PROCESSES

B} € 3 for B £ 05, is said to be a random, element in X. In the case where X is a metric space we take the cr-algebra generated by open balls as the cr-algebra ©. This cr-algebra is Borel cr-algebra if X is separable. In the last case we denote this (7-algebra by © x - A measure on 03 such that fi(B) = P(lo:x(lj) 6 B) is called the distribution of a random element x(u>). Random variables (X = R ) and random vectors (X = R m ) are special cases of the construction described above. Distributions of random variables and random vectors are defined by the distribution function. For X = R the distribution function of a random variable is given by F(x) = P(£(w) 1, ii) n ~ = 1 G whenever An D An+i and An e OT, n > 1. Evidently, an algebra of sets which is a monotone system is a cr-algebra. THEOREM (ON A MONOTONE SYSTEM). Let

21 be an algebra

minima] monotone system containing 21. Then cr(21) =

and M(21)

be

the

M($l).

PROOF. Let X be the set where the algebra 21 is determined. Let also A(X) be the family of all possible algebras on X . For a class of sets 03, we denote by 03CT the set containing all unions |J Ak (even infinite) and by the set containing all intersections f | Ak (even infinite), where Ak € 03. Set [03] = (OS^j n (©i)^. Then [03] D 03. It is clear that [03] is an algebra if 03 is an algebra. We shall say that a collection of algebras A C A(X) is regular if 1) [21] e A whenever 21 e A, ) M k ^ Z A whenever 2t* € A, k > 1, where \J k 2ljt is the minimal algebra containing all 21^. Finally, denote by 7^(21*) the collection of algebras containing 21. Consider now the minimal regular collection Ao from 72.(21). This collection is the intersection of all A from 7?.(21). It has the following properties. 2

1) if © and £ are from Ao then either 03 C £ or C C © (otherwise it is possible to exclude either © or £ from >4o); 2) for all © € Ao, we have © £ >1(21) (because [€} C M(2l) if £ C >i(2i)); 3) |J„ ®n C M(21) if ®„ t and © n C M(21). We derive from these properties that [£ 0 ] C £ for Co = (J £ C -4o- Additionally, Co is a cr-algebra and 21) C Co C vVi(2l). Since a(21) is a monotone class, _M(21) c

ff( 21).

•

1. S T O C H A S T I C P R O C E S S E S . D E F I N I T I O N S . C O R O L L A R Y 1 . Let measures ¡x\ and coincide on the a-algebra CR(51).

5

EXAMPLES

coincide on an algebra

21.

Then

they

This Corollary follows from the fact that ¡j,i and fj,2 coincide on a monotone class. Definition of a r a n d o m function. Let 0 be a certain set and let (X, 25) be a measurable space. We shall call them the parametric set and the phase space respectively. A function x(9,w): 0 x Q —> X is said to be stochastic if x(9,w) is a random element in the phase space (X, 03) for every 9 £ 0 . The finite dimensional distributions are the basic characteristics of stochastic processes. These distributions are sequences of measurable functions Feu...,e„ {BI,...,

BN) = P{X(91,OJ)

£ BU ... ,x{9N,U)

£ BN)

(2)

defined on 0™ x 05". This definition for n = 1 gives one dimensional distributions: F$(B) = P ( x ( 9 , w ) £ B). We say that (2) defines n-dimensional distributions of a stochastic process. Let us consider more carefully real valued stochastic processes, i.e., the case X = R. To describe such processes we also use moment functions. If £(9, ui) is a real valued random function then the k-th moment function is given by mk(9 u...,ek)

=

E((Í

L L W

)-{(Í

T

,U).

(3)

The mean value mi(9) of a stochastic process £(9) is an example of moment functions as well as the correlation function R{9u92)

= E[£(0i,w) -mi(0i)][£(02,w)

-mi(02)]

= m2{9u92)

-

mi(0i)mi(02)(4)

There is a special field of the theory of stochastic processes where only the mean value and correlation functions are used. The characteristic functions 'figi,...,g„{zi,... ,zn) = Eexp {i zk£{9k, w)} are also considered for stochastic processes. These functions determine uniquely the moment functions (2). Now we give some examples of stochastic processes and functions. E X A M P L E 1: A WAITING P R O C E S S . Let r be a random variable and £(t,w) = I{T(u)t) = P(£(i + h) = l / t f t ) = 0)

(5)

of the event that A occurs before the moment t + h provided that A did not occur until the moment t. If expression (5) can be represented as \(t)h + o(h) then the distribution of the occurring moment has the density p(t)

= X(t) e x p I J

A(s)dsJ.

(6)

6

L E C T U R E S ON THE THEORY OF STOCHASTIC PROCESSES

In this case A(t) is called the local intensity of the occurring probability.

v

I A,

If

for t > 0,

then T has the exponential distribution. In this case the event A is occurring on the interval [0, oo) with the constant intensity p(t) and p(t) = Ae~xtI{t>0}.

(7)

EXAMPLE 2: A POISSON PROCESS. Consider a sequence of events occurring randomly one by one on some intervals. The process £(t,u>) denotes the number of events occurred on [0, t]. Here are suitable examples of such a process: - the number of particles recorded by a counter of space particles, - the number of highway accidents, - the number of instances of a disease which is found in nature. The process £(i) is a Poisson process if the intensity of a next event depends on neither the time nor the number of events that have occurred. Let the occurring intensity be a. This means that the occurring probability of only one event on an interval [t, t + h) equals ah + o(h) independently of what has happened on the interval [0,i]. It easy to make sure that the probability that more than one event occurs on an interval [t, t + h] is o(h). Denote by pk(t) the probability that exactly k events occur on an interval [t,t + h]. Then p0(t + h) = p0(t){l — ah + o(h)), pk+1(t + h) = pk+l(t)(l ~ah + o{h)) + pk(t)(ah + o{h)). Therefore, by approaching the limit as h —> 0, we get dPo(t) — —

„„ m = ~apo{t),

= —apk+i(t) +

(8) apk(t).

Additionally, po(0) = 1 and pk{0) = 0, k > 0. Solving these differential equations, step by step, we obtain pk(t) = {ta)k{k\)~1e~at, (9) that is, the number of events occurring has a Poisson distribution. This explains the name of the process. One can determine the finite dimensional distributions of this process, as well. To do this we note that the number of events occurring on an interval [£i, i 2 ] has the same distribution as the number of events occurring on the interval [0,i2 — and does not depend on the number of events occurred on [0,t]. For this reason P(£(ii,a;) = Ai,£(i 2 > w) - £(ti,w) = k2) = (t1a)k^k1\)-l[(t2-t1)a\k^k2)-1e-t^. Analogous formulas hold true for every family of intervals.

(1

1. STOCHASTIC PROCESSES. DEFINITIONS. EXAMPLES

7

EXAMPLE 3: A BROWNIAN MOTION. Consider a probability model of the physical phenomenon of the Brownian motion (or a diffusion). This is a motion of very small particles in an environment with molecules running chaotically. A limpid liquid or a gas may be considered as such kind of an environment. We suppose that 1) this environment is homogeneous, so all directions are interchangeable, 2) the distribution of a shift At of a particle from a certain point x of the space does not depend on this point and on the time when this particle has occupied the point x. Consider such a process in R . This means that we are observing only one of the coordinates of a particle. Let £(t) be a point occupied by this particle at a time t and let A£ = £(i 4- Ai) — £(i) be the shift of the particle over time St. It is clear that A£ does not depend on £(t), and the previous values of the process and its distribution does not depend on t. Let also the following properties be satisfied: a) E A£ = 0, b) E(A£) 2 = bAt, c) E |A£|2+i5 = o(A£), 0. To determine the distribution of £(i) under condition that £(0) = x, we consider the function E[^(£(i))/£(0) = x] = u(t,x). We have u(t + A t,x) = E [ e for some e > 0. According t o this e, we may choose {Fn}

p ( ñ

P(cfc \ P i ) > £ - ¿

^ I-

Cn then Cn i 0 , P ( C n ) > e/2, and C * C ( R m ) n is a compact set. Let 7Ti be the projection of ( R m ) n onto R m such that tti(xi, ... ,xn) = xi, Xi £ R m , i = 1 , . . . , n , and let 71*, k < n, be the projection of ( R m ) " onto ( R m ) * such that TTk{xi,... , x n ) = ( x j , . . . ,ijfe). N o t e that the projection of a compact set If f l L i

Fk

1Tb) > P ( C „ ) - ¿

Assume that t o obtain

=

10

LECTURES ON THE THEORY OF STOCHASTIC PROCESSES

is compact, too. Take now points ( x . . . , xL"') £ C*. By the diagonal method, a sequence

can be chosen such that, for every n > 1, there exists the limit x-n\

x = lim^oo It is clear that (a?i,... ,xjt) € nkC* for all n > k. Therefore the function w(9) belongs to every set Fk C Ck if = x¿. This contradicts the assumption that f ) C k = 0 . So, P can be extended to C ( 0 , R m ) . • REMARK. The proof holds true if X is a complete separable metric space because in this case P ( B ) = sup A - c B P(.?0 for all measures P and Borel sets B, where K is a compact set. Now we are going to give a classification of stochastic processes. At first we separate processes with respect to 0 . We shall consider stochastic processes, that is, the case 0 6 R . If 0 = Z or 0 = Z + then we say that a random sequence is defined. We also distinguish processes by the phase space ( X , 05). With respect to ( X , 05), we consider either real valued or complex valued or vector valued processes. But the basic properties, by which we separate the processes, are the properties of finite dimensional distributions. Processes w i t h independent values. Assume that random variables •• • are independent, that is,

, w),

n

F8l

eAM

,...,A„) = n ^ ( ^ )

(3)

Jfc=l

for all . . . ,0 n G ©• Such processes are considered basically for discrete time interval. One of the important properties of processes with independent values is the Kolmogorov 0-1 law. THEOREM (KOLMOGOROV'S 0 - 1 LAW). Let {£„} be a sequence of independent elements in a phase space (X, 03). Denote by 2ln the a-algebra of events {£n € B}, Then either P ( C ) = 0 B G 05, and 5 n = V * < „ 5m = V*>m 5°° = D m or P(C) = 1 for all C C 5°°PROOF. The a-algebras 5 „ and J N + 1 are independent, since they are generated by independent random variables. The first of them is generated by random variables { £ i , . . . , £ n } , and the second one by {£ n +i,£n+2,• • • }• For this reason a-algebras (J 5 „ and are independent. In particular, this means that P ( i n B ) = P(A)P(B) for all events A € U ^ n and B € It is easy to see that every collection of events satisfying the last property is a monotone class. By the Theorem on a monotone class we conclude that the a-algebras a (|J J n ) and are independent. Since S°° C a ((J3n), every event C 6 3°° does not depend on it itself, i.e. P(C DC) = P(C)P(C), P(C) = P2(C), and P ( C ) [ 1 - P ( C ) ] - 0. • COROLLARY. Let { £ * } be a sequence of independent elements. Assume that a function f(x\ , . . . , ! „ , . . . ) is measurable on X°° and that it does not depend on xi,

2. CLASSIFICATION OF PROCESSES X2,...

,x

n

for

random

n > 1.

every

Then,

with

probability

1, / ( £ i

,

11

.

.

)

is a

This corollary follows from the fact that the probability P ( / ( f i , • • •, equals either 1 or 0 for every t. An example of such a function is given by f{x 1 , . . . , £ „ , . . . ) = lim sup

• • •) < t)

ifin(Xn),

n—>oo

where

constant

variable.

-> R is a measurable function.

*,£(i0)) + » ( f > * , £ ( i i ) - £ ( i o ) ) +••• ^

Set ipti,t2(z) follows

(*)J

(2)

16

L E C T U R E S ON THE THEORY OF STOCHASTIC PROCESSES

R e n e w a l s c h e m e . Let x > 0 and P(£i > 0) = 1. Then a walk { C n , " > 0 } is said to be a renewal scheme. The following interpretation explains the terminology. Let be a working time of the fc-th apparatus. All the apparatus are switching on one by one starting from the moment t = 0. Let u(t) be the number of the apparatus which are working at the time t. According to the scheme described, i/(0) = 1 and u{t) = k if Ot-i < t < (k. The function N(t) = Eu(t) is called the renewal function. It means the average of apparatus which were used over the time t. Since N(t) < t, the renewal function is finite for every t > 0. For renewal schemes, we investigate the behaviour of the function N(t) as t -»• oo. Let us compute first the following series oo J2[N(t) t=l

oo - N(t - 1)]*' = E 5 > ( t ) - u(t - 1 )]zl. t=i

Put Then EzCfc =

(ip(z))k

and v(t) — u(t — 1) = 0 if t — 1 is not represented among numbers C11C2, • • • v(Ct+i) - v{C,i) = 1 for all i > 0. Therefore 00

t=1 OO E k=0

OO = 5>fc(*> = Jt=l

and

a

fc=0 ,/ X rv

'

Hence 1)]*' = ^ L . .

±[N(t)-N(t-

(3)

Assume that the step of £1 is equal to 1. Then tp ( e m ) = f { u ) |it| < 7r. We have for t > 1 N(t) - N(t - 1) = lim I" w v ' Ati 2tt J_n

-!£^L-e-itu 1 - \tp(u)

du = lim [* Ati 2tr J_n

0 for u ^ 0,

1—e1 - \tp(u)

i t u

du

and for t > 0 r J~*

T^rVreitU u> -

1

du = f]Xk k=0

J

f t

t=0

Then

k>t

t=0

k=0

It follows from equality (3) t h a t

Q(z)R(z)

=

1 1 - z

Therefore for all n > 1, ¿ 9 n - t r t = l.

(8)

t=0

According to (5), I f "

qt+1+qt-l-2qt=—

sin(i + l ) u 4- sin(£ — l ) u — 2 sin tu ,

i

1 r = —: I 71'I whence limt_>oo[ o o . So,

N (iqn - e„,Ar) ^ r t < 1, t=o

qn < £n,N + —¡7 • ¿vt=0 T i

Approaching the limit as n —• 00 and then as N —> 00, and taking into account • that rt — +00, we complete t h e proof of Theorem 3.

3. RANDOM WALKS. RECURRENCE. RENEWAL THEOREM

19

Conditional m a t h e m a t i c a l e x p e c t a t i o n s and probabilities. We introduce some notions that will be needed in future considerations. Let (fi, P) be a probability space and © C 5 be some cr-algebra. For a random variable £ > 0, we shall denote by E(£/©) the random variable that is ©-measurable and for all G 6 © E [£IG] = E[E(C/©)/ g ],

(9)

The random variable E(£/©) is said to be the conditional expectation of £ with respect to the a-algebra ©. The conditional expectation is uniquely defined almost certainly. Indeed, let rj and rj be ©-measurable random variables satisfying relation (9) where either r) or rj is taken instead of E(£/©). Then P(rj = rj) = 1. Indeed, in this case E(rj - rj)Ic = 0 for all G £ 0 . Taking G\ = {u>:t) - rj > 0 } and G2 = — rj< 0} we obtain E(r) - r^Ic^ = 0 and E(rj — rj)Ic2 = 0. Therefore E \r) - rj\ = 0 and P(77 / rj) = 0. For general we may define separately E ( £ + / 0 ) and E ( £ _ / 0 ) , where £+ = £ I { i > 0 } and = £ J { € < 0 } . Now we put E(£/©) = E ( £ + / 0 ) + E(£_/)].

REMARK. Let Sj be the trivial cr-algebra containing only sets 0 and Í2. All immeasurable random variables are constants almost surely. Therefore E ( £ / . f t ) = E £ a n d E [ E ( £ / 0 ) ] = E£. DEFINITION. A random variable E ( / ^ / 0 ) is said to be the conditional of the set A with respect to the a-algebra ©.

The following property is said to be the complete probability E[P(¿/0)] = P(¿).

probability

formula:

(11)

20

LECTURES ON THE THEORY OF STOCHASTIC PROCESSES

The next two properties of conditional expectations are also of interest. (5) If £ > 0 then E ( f / 0 ) > 0. So, < E ( & / 0 ) for ft < £2 almost surely provided that both expectations exist. (6) Let £ n —• £ in probability and |£ n | < r¡ for some positive random variable r¡ with ET] < oo. Then E ( f „ / 0 ) -»• E(f/ 0 and E | E ( £ n / 0 ) - E(í/«5)| < E[E(|Í„ - £|/®)] = E |£ n -

0.

It follows from the last two properties that (a) 0 < P{A/&)

< 1,

(b) if Ak n Aj = 0 for k ^ j then p(U^/®) The property (b) is satisfied with probability 1.

(12)

LECTURE 4

MARTINGALES. INEQUALITIES FOR

MARTINGALES

A sequence { & } , k € Z + or k € Z, with values in R is said to be a if for all k from the domain of definition, a) cr-algebras b) E | & | < oo, c) E(&+i/ff*)

are given such that 5*: C S t + i and ^ is

martingale

-measurable,

Here are some examples of martingales. EXAMPLE 1. Let rji be independent random variables with E rji = 0 . Put = rjo + • • • + rjk, k £ Z+, and define the u-algebra Jfc generated by r ) 0 T h e n is a martingale because of E {Vo H

1- Vk+i/Sk)

= m H

1-T)k + Eifc+i =

since rjk+i does not depend on EXAMPLE 2. Let % be independent random variables with E 77, = 1 and aalgebras Jj are defined as in Example 1. Put = 770771 •••r]k- Then is a martingale, too. EXAMPLE 3. This example is more complicated. Consider two sequences of random variables 7 7 ^ , . . . , t ] n \ • • • and t j ^ , . . . , r j n ^ , . . . such that the random vector (77q \ ... ,T)n'), i = 1,2, for every n has the density fn\xo, • • - , x n ) with respect to the Lebesgue measure in R n + 1 . everywhere. Put Cn = /< 2 )

We assume that / ^ ( x o , . . . , x n ) > 0 almost

...,ȣ>),

n = 0,1,2,...

and define cr-algebras 3 „ to be generated by the random variables 7 7 ^ , . . . , Let us show that £ n is a martingale. Indeed, , A2) / (1) 7

n (l)

•

x

fn+

where p n ( - ) is the conditional density of the random variable 77^^ with respect to the er-algebra 3 n . It is easy to verify that

Jn+l IVO >••'>Vn )

LECTURES ON THE THEORY OF STOCHASTIC PROCESSES

22

Therefore E(gn+l/gn)=

, ( 1 ) . M) 1 fn [Vo

ojT

[ J

f P W , - , ^ )

r

In what follows we shall consider supermartingales, erties a), b), and

that is, sequences with prop-

ci) e(6+I/ffk) < & instead of c). replaced by

Also, we shall consider submartingales

for which condition c) is

c 2 ) E(Zk+i/3k) S t o p p i n g t i m e s . A random variable r with values in Z + is said to be a stopping time if {U:T = k} 6 Stopping times are called sometimes optional random variables. A u-algebra is connected with each stopping time. This cr-algebra may be described as follows. A set A belongs to 3V if and only if A n { r = k} € for all k. If Ti and r 2 are stopping times and tx < t 2 then 3V, C 3r2 • Indeed, let A e $ T l . Then A n {t2 = k) = |J [A n { n = ¿} n { r 2 = k}} e i 1 then E ( m a x ( a v O ) )

a

< ( ^ )

a

E (

W

.

(7)

PROOF. Set 77 = maxjt a} for t h e event A defined in t h e proof of T h e o r e m 2. It follows from (5) t h a t AP(R? > a)
a}.

By multiplying b o t h sides of this inequality by a Q ~ 2 and t h e n integrating over a f r o m 0 t o + 0 0 we get t h a t r+oo /

Jo

aa-1P(r)>a)da
a)da = — I aa dP{q < a) = - Et?", a a Jo

we obtain T^M+V0 a — 1 Let us utilize t h e Holder inequality: |E^R/| < ( E £ P ) 1 / P (ETJ9)17" provided t h a t l / p + 1 / g = 1. Taking p = a a n d q = a/a — 1 in this inequality we get

Therefore a — 1 T h e last inequality is equivalent t o (7).

•

4. MARTINGALES. INEQUALITIES FOR MARTINGALES

25

The number of crossings of an interval. Consider a sequence X Q , X \ , . . . , X^ of real numbers. Let a < b. We say that the sequence { n } has exactly k downcrossings of the interval [a, 6] if there exist k + 1 integer numbers no < ni < • • • < rik < N such that x

no —

^

£712 —

—

•••

and there are no k + 2 numbers with this property. The number of down-crossings may be calculated as follows. Find the first number no such that x„0 > b, then find the first number ni > no such that xni < a and so on. Of course, in the same way we may define the number of up-crossings. The dual definition is left to the reader. T h e o r e m 4. Let £o,fi, • • • >&v be asubmartingale and v~{a, b) denote the number of down-crossings of the interval [a, b]. Then

E

y

-(M)
b. We put ro = N if such a number does not exist. In the last case we also put r* = N for all other i. Otherwise, i.e. if ro < N, we denote by t\ the smallest number after ro such that 7i < a. Again, we put T\ = N if such a number does not exist. Otherwise we may define T2 as the smallest number greater than t\ and such that £ r2 > b, and so on. Let

A2m = {Tim < N} n Ur 2 m + 1 < a} = ( M a , 6 ) >m+

1}.

It is clear that A2m € 3v2m+i- Similarly, A2m+1 = {r 2 m + 1 < N} n {£r2m+2 >b}e

3r 2m+2 -

It is easy to see that A2m+i

C A2m. We get

0 < E(fr 2m -i - &2m)IA2m-i

= E(fr 2m + 1 - £T 2 m ) I A 2 m + E(£r2m + 1 ~ &m)lA2m-1\A2m

•

Also, we have for w € A2m &2m + l < a>

£r2m > b,

and £r2m > b for u> € A2m-i-

£r2m + 1 ~ £r2m

Therefore

0 < (b - a) EIa 2 „ + E (£T2m+1 - b) 1. Then = Vi is a martingale (put £0 = 0) and therefore p(jgx|i.| > a ) < ^ M ,

The second inequality above follows from the fact that We give below a more general result in this direction.

is a submartingale.

LEMMA. Let {£„} be a martingale and g(x) be a convex function. E |g(£n)| < 00. Then {