Stochastic Convergence of Weighted Sums of Random Elements in Linear Spaces (Lecture Notes in Mathematics, 672) 3540089292, 9783540089292

Citation preview

Lecture Notes in Mathematics Edited by A. Dold and B. Eckmann

672 Robert L. Taylor

Stochastic Convergence of Weighted Sums of Random Elements in Linear Spaces

Springer-Verlag Berlin Heidelberg New York 1978

Lecture Notes in Mathematics Edited by A. Dold and B. Eckmann

672 Robert L. Taylor

Stochastic Convergence of Weighted Sums of Random Elements in Linear Spaces

Springer-Verlag Berlin Heidelberg New York 1978

Author Robert L. Taylor Department of Mathematics and Computer Science University of South Carolina Columbia, SC 292G8/USA

Library of Congress Cataloging in Publication Data

Taylor, Robert Lee, 1943Stochastic convergence of weighted sums of random elements in linear spaces. (Lecture notes in mathematics ; 672) Bibliography: p , Includes index. l. Linear topological Spaces. 2. Stochastic processes. 3. Limit theorems (Probability theory) 4. Law of large numbers. I. Title. II. Series: Lecture notes in mathematics (Berlin) ; 672. QA3.L28 no. 672 [QA273.43] 5l0'.8s [5l5' . 73] 78-l3024

AMS Subject Classifications (1970): 60B05, 60F15, 60G99

ISBN 3-540-08929-2 Springer-Verlag Berlin Heidelberg New York ISBN 0-387-08929-2 Springer-Verlag New York Heidelberg Berlin This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to the publisher, the amount of the fee to be determined by agreement with the publisher.

© by Springer-Yerlag Berlin Heidelberg 1978 Printed in Germany Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr. 2141/3140-543210

PREFACE Recent interest in representing stochastic processes as random variables in function spaces has inspired the study of the "convergence of weighted sums" for random elements.

The purpose of these

notes is to provide a unified presentation of the results which have been obtained to date concerning the stochastic convergence of weighted sums of random elements in (primarily) linear topological spaces, including some recent work of the author and some colleagues. These notes are somewhat self­contained with only a background knowledge of basic probability theory being absolutely essential, Chapter I presents many of the essential definitions and results from mathematical analysis and topology that are needed, and the definitions and properties of random elements are developed in Chapter II.

The remainder of the notes is concerned with the sto-

chastic convergence results and possible applications. Probabilists, statisticians, and mathematicians who are involved in research concerning convergence theory, stochastic processes, ergodic theory, and related topics will find these notes very

In addition, the notes will be useful for introducing advanced graduate students and researchers to the area of random elements and stochastic convergence of weighted sums in function spaces.

It is

anticipated that these notes will be expanded in a later version to include developing convergence theory results and applications. The author is very gratefUl to research colleagues W. J. Padgett, Duan Wei, and Peter Daffer for work and suggestions on this manuscript and related research.

A special thanks goes to Mrs. Denise

Domeracki for the typing and general appearance of the manuscript. Finally, the author wishes to thank the Department of Mathematics and Computer Science at the University of South Carolina and the

IV

Department of Statistics at the Florida State University for providing the support and resources that made these notes possible. Robert Lee Taylor Columbia, South Carolina December,

TABLE Of CONTENTS GENERAL INTRODUCTION . . . . I.

II.

III.

.

1

MATHEMATICAL PRELIMINARIES

6

l.O

Introduction . . . .

6

l.l

Linear Spaces and Topologies

6

1.2

Some Basic Mathematical Tools.

12

1.3

Basis Theory

14

1.4

Problems . .

20

RANDOM ELEMENTS IN LINEAR SPACES

21

.

2.0

Introduction

2.1

Definitions and Examples of Random Elements

21

2.2

Topological Properties of Random Elements

25

2.3

Probability Properties of Random Elements

30

2.4

Problems

21

.

LAWS OF LARGE NUMBERS, UNCORRELATION, AND CONVERGENCE OF

44

WEIGHTED SUMS OF RANDOM VARIABLES

IV.

42

.

44

3.0

Introduction

3.1

Laws of Large Numbers for Random Variables

45

3.2

Extensions to Separable Hilbert Spaces

49

3.3

Comparisons of Uncorrelation Concepts

54

3.4

Convergence of Weighted Sums of Random Variables

63

3.5

Problems . . .

70

LAWS OF LARGE NUMBERS IN NORMED LINEAR SPACES. 4.0

Introduction . . .

4.l

Laws of Large Numbers for Independent, Identically

71 71

Distributed Random Elements

72

4.2

Distributional Conditions and Laws of Large Numbers.

81

4.3

Beck's Convexity and the Strong Law of Large Numbers

86

VI

4.4

Other Geometric Conditions and the Strong Law of Large Numbers

4.5

4.6 V.

. . .

Other Convergence Results and Extensions to Frechet Spaces

104

Problems . . .

106

CONVERGENCE OF WEIGHTED SUMS IN NORMED LINEAR SPACES 5.0

Introduction .

5.1

Distributional Conditions and Convergence of Weighted Sums

5'.2

5.4 VI.

108

109

120

Geometric Conditions and Convergence of Weighted Sums . .

137

Problems

145

RANDOMLY WEIGHTED SUMS

146

6.0

Introduction

146

6.1

Identically Distributed Results

146

6.2

Convergence in the rth Mean for Randomly Weighted

6.3 VII.

108

Convergence for Weighted Sums of Tight Random Elements . . . . . .

5.3

96

Sums

150

PrLJblems

152

LAWS OF LARGE NUMBERS IN DCO,l]

153

7.0

Introduction

153

7.1

Preliminaries

154

7.2

Weak laws of Large Numbers in D[O,I]

166

7.3

Strong laws of Large Numbers in DCO,I]

171

7.4

A Discussion of Convex Tightness

179

7.5

Problems

184

VII

VIII.

POSSIBLE APPLICATIONS

185

8.0

Introduction

185

8

Applications in Stochastic Processes

186

8.2

Applications in Decision Theory

190

8.3

Applications in Quality Control

195

8.4

Applications in Estimation Problems

196

BIBLIOGRAPHY

202

SUBJECT INDEX

212

GENERAL INTRODUCTION

The consideration of a stochastic process as a random element In a function space (a random variable taking values in a function space) by Doob (l947), Mann (l951), Prohorov (1956), Billingsley (1968), and others has inspired the study of stochastic convergence properties for random elements.

However, a simple example illus-

trates that careful construction of the appropriate framework is needed in these considerations.

For example, if {X t:

t E T}

is a stochastic process with respect to the probability space (n,A,p), then for each WEn Xt(w), t

E T, the sample path can be

regarded as a real-valued function of t E T. {X t:

But in considering

E T} as a mapping from n into RT, the space of real-valued

t

functions on T, certain measurability problems may occur. an identity function X [X Let A

t,

t

= {Xt

:

E T} from n

t

=W

E TJ(w)

= RT

Define

T to R by

= n.

for each WERT

=

TI B(R) where B(R) denotes the Borel subsets of R and let tET P be the probability measure degenerate at the origin. Then

Xt

o

(w)

= wCt O)

for each to E T and is a random variable since

{W: for each a E R.

R) x (-"',ct] E

(TI

t;tt However,

TI

tET able [Doob (1947)J; otherwise

o

B(R)

= B(R T)

TI

tET

B(R)

if and only if T is count-

B(R) c B(R T) where the Borel field t ET ;t T of R is generated by open subsets of the product topology of

T R =

TI

TI R. Thus, the mapping X = {X t t: tET measureable function from (n,A,p) to RT.

E T} may not be a (Borel)

One way of solving the problem of measurability problem is by placing constraints on the parameter space.

For example, a sto-

chstic process with a countable parameter space will be shown to be a random element in the space of sequences, s.

Often the stochastic

2

T. process will take values only in a "small" subspace of R

Recall

that a separable stochastic process may have sample paths which are Borel measurable functions from T into R [Loeve (1963), p. 510] and hence are restricted a.s. to a subspace of RT.

Thus, the sto-

chastic processes may have properties which reduce the ranges of T the mappings from n to interesting subspaces of R where different (possibly stronger) topological structures can be employed.

Char-

acterizations of stochastic properties as random elements in a variety of different spaces will be presented in Chapter II. In an attempt to make the notes reasonably self-contained, Chapter I will consist of basic mathematical notations, definitions, and results for future reference throughout the presentation.

In

Chapter II the concept of a random element will be presented, and some basic properties and definitions will be given.

The emphasis

in Chapter II will be to illustrate the different techniques which are to be used in studying function-valued random variables. numerous

Thus,

examples are presented in Chapter II for the convenience

of the reader and for reference in the remainder of the notes. A weighted sum of a sequence {X } of random variables is an

n

expression of the form (0.1 ) where {a

is a double array of constants. The most classical form n k} of a weighted sum is the mean (or average) where

= 11/n o The convergence of {3

n}

for k = 1,2, ... ,n for k > n.

(0.2)

plays a fundamental role in the relation-

ships between the theoretical and practical aspects of both probability and statistics.

With the consideration of stochastic pro-

cesses as random elements in function spaces, it is very natural to

3

seek function space analogues of the convergence results for weighted sums of random variables.

Thus, Chapter III will be de-

noted to a brief discussion of some results on the stochastic convergence of weighted sums of random variables and some direct ext ensions to separable Hilbert spaces. The law of large numbers, the convergence of {S } wich the n

particular weights listed in (0.2), was examined by Mourier (1953) where a direct extension of Kolmogorov's strong law of large numbers for independent, identically distributed random elements with finite first absolute moments was obtained for separable Banach spaces. Examples by Beck (1963) and Woyczynski Q973) showed that neither the classical second moment condition nor Kolmogorov's condition is sufficient for the strong law of large numbers in certain infinitedimensional spaces.

However, it is sufficient for certain other

infinite-dimensional spaces.

Such spaces have been examined by

Beck (l963) (B-convex spaces) and (Type p spaces, l

,;; P ,;; 2)".

and Pisier (1976)

Woyczynski (973) also showed that

Chung's strong law of large numbers holds in Ga-spaces (0 < a ,;; 1). Basically, the Ga spaces provide the inequality (0.3)

,nk=lEIIXklll+a Ellxl + ... + Xn Il l +a s C t: which is valid for independent random elements {X } where C is n

a constant.

When a = l

in (0.3), then a bound on the variance of a

sum of random elements is available. in the proofs of most results. available.

This bound plays a key role

In general, this inequality is not

These and other laws of large numbers for random elements

in normed linear spaces are presented in Chapter IV. The classical second moment conditions alone do not yield laws of large numbers for sequences of independent random elements.

An

r t h moment bound (r > l) and the assumption of tightness do lead to laws of large number [Taylor and Wei (to appear)].

These results

4

follow from convergence results of weighted sums without requiring any geometric structure for the spaces and are presented in Chapter V.

Tightness of a sequence of random clements has been defined

by LeCam (1957), and tightness characterizations of random elements In

the function spaces C[O,lJ and D[O,lJ have been treated by

Billingsley (1968).

In these notes some properties of tightness

are presented, and several counterexamples are provided for other plausible conjectures.

Other extensions and generalizations of the

results by Chow and Lai (1973), Rohatgi (1971), and Taylor (1972) are also presented in Chapter V.

In particular, convergence in

probability for Toeplitz weighted sums of tight random elements with .. ( r > 1 ) .lS shown to be equlva . 1 ent f or t h e an r th moment condltlon weak linear and norm topologies of a separable Banach space.

Finally,

geometric conditions (type p spaces) are imposed to obtain convergence in probability results for weighted sums of random elements without assuming tightness. Due to the random nature of many problems in the applied sciences, researchers are increasingly forced to switch from deterministic to probabilistic approaches.

Thus, it is important to con-

sider random weighting of random variables, that is, to allow {a to be a double array of random variables. pertain to these considerations.

nk} The results in Chapter VI

Laws of large numbers for the

space D[O,lJ are presented in Chapter VII.

The space D[O,lJ is very

fruitful for applications, but with the Skorohod topology it is not a linear topological space.

Hence, the results and techniques of

previous chapters can not be directly applied but must be appropriately modified and developed in Chapter VII. In the last chapter, some examples are given to demonstrate possible applications of these convergence results.

Since the area

of stochastic processes provided the motivation for the study of random elements, many of the applications will be concerned with

5

stochastic processes.

In addition, weighted sums often occur in

the fields of applied statistics such as robust analysis, regression analysis, control charts, decision theory, and estimation.

Specific

examples are given in Chapter VIII to indicate possible applications in these areas.

CHAPTER I MATHEMATICAL PRELIMINARIES

1.0 INTRODUCTION For the convenience of the reader and for future reference, this chapter will contain some of the basic mathematical definitions, theorems, and notations which will be used throughout the notes. Section 1.1 will present the concept of a linear topological space and will give the notation for particular linear topological spaces which will be used in later chapters.

In Section 1.2 some basic

theorems and results are listed for later reference.

The definition

of a Schauder basis and some of its properties will be given in Section 1.3.

Basis theory provides very useful tools for character-

izing random elements in linear topological spaces and for obtaining convergence results.

A few problems are listed at the end of the

chapter and can be used to check on understanding the basic concepts of this chapter.

1.1 LINEAR SPACES AND TOPOLOGIES The following definitions and notation will be used throughout these notes.

First, R will denote the real numbers.

Definition 1.1.1

A nonempty set X is said to be a (real) linear

space if there is defined an operation of addition which makes X a commutative group and an operation of mUltiplication by scalars (real numbers) which satisfies the distributive, and identity laws; more precisely stated,

(1)

to every pair of elements lx,y)

an element z

E

X such that z = x + y,

E

X x X there corresponds

7

(2) tx

to every x

X and t

R there corresponds an element

X, the operations defined in (1) & (2) satisfy the following

(3)

X and s,t

properties for every x,y,z (i)

=y

x + y

(ii) (iii) (iv) (v) (vi) (vii)

+ x

=x

Cx+y) + z

=x

x + y

=x s(tx) =

R

+ (y+z)

+ z implies y

=z

Ix

Cst)x

= sx + tx s(x+y) = sx + sy.

(s+t)x

Note that Definition 1.1.1 implies the existence of a zero element, 0, and an additive inverse, -x. Definition 1.1.2

A nonempty set M is called a semimetric space

if there is a real-valued function d defined on M x M with the following properties: (i)

d(x,y)

(ii)

= d(y,x) o

d Cxvx )

0 for all (x,y)

for all x

M x M;

M; and

(iii) d Cx c z ) s d Cx yy ) + d Cy c z ) for all x,y.z

M.

If in addition Uv}

d Cxvy )

metric space.

=

0 if and only if x

= y,

then M__ is called a

The real number d(x,y) is called the distance from

x to y, and d is called a semimetric (or a metric when (iv) holds). Definition 1.1.3

A sequence {x

n}

in a metric space M is called

a Cauchy sequence if for every E > 0 there exists an integer N such that dCx , x ) < E whenever n

n

m

Definition 1.1.4

Nand m

N.

A metric space M is said to be complete if

every Cauchy sequence in M converges to an element of M.

8

Definition 1.l.5

If X is a linear space with a topology such

that the two basic operations of addition and scalar multiplication are continuous, then X is said to be a linear topological space or a topological vector space. It is not sufficient for a linear space to have a topology defined on it in order to be a linear topological space.

For

consider the real numbers R and the discrete metric topology. Definition 1.1.6

The collection of all continuous linear

functionals (that is, continuous linear real-valued functions) defined on a linear topological space X is called the dual space of X and will be denoted by X*. If a linear topological space X has a metric d which generates its topology, then X is called a linear metric space.

Also, the

concept of a seminorm or norm is often used to introduce a topology on a linear space.

A detailed discussion of seminorms and norms may

be found in Yosida (1965) and Wilansky (1964). Definition l.1.7

A real-valued function p defined on a linear

space X is said to be a seminorm on X if the following properties are satisfied for all x,y (i)

Ui)

p Cx)

X and t

R;

0;

p(x+y),;; p Cx) + pCy); Csubadditivity)

(iii) p Ct;x )

=

ItlpC-x)

Definition 1.1.8

(homogeneity).

A linear space X is said to be normed if there

is a real-valued function defined on X and denoted by that

Ilxll

II II

II II

such

satisfies (i), Cii), and (iii) of Definition 1.1. 7 and

= 0 if and only if x = O.

The function

II II

is then called

a norm. A norm Cor seminorm) always yields a topology on a linear

9

space

where both addition and scalar multiplication are continuous

operations. space.

A complete normed linear space is called a Banach

A complete linear metric space is called a Frechet space.

Note that a Frechet space (by this definition) need not be locally convex while a normed linear space is locally convex. Definition 1.1.9

Let X be a linear space.

A mapping

from X x X into the real (or complex) numbers is called an inner product if for each x,y,z

X and t

R the following properties

hold: (i)

(ii)

(iii) (iv)

= + = t;

>

;

, the bar denotes the complex conjugation;

a

if and only if x

f

O.

The space X is then saij to be an inner product space. i d by th e norm d e f 1ne

I Ix I I = 1/ 2

If X with

. comp 1 ete, t h en X·1S ca 11 e d 1S

a Hilbert space. Definition 1.1.10

A subset S of a topological space X is said

to be dense in X if its closure (the smallest closed set containing S) equals X.

If X has a countable dense subset, then X is called

a separabie space. Most of the spaces that are discussed in these notes are separable since most results for random elements require separable spaces. Several particular linear topological will now be listed for notational reference [see Taylor (1958), Wilansky (1964), or Yosida (1965) for further discussions of these spacesJ. (1)

The space of all real sequences will be denoted by s.

metric on s can be defined by the Frechet metric d (x ,y)

A

10

for every x = (x1,x 2' ... ) and y = (Y1'Y2"") in s. metric s is a separable Frechet space.

With this

Moreover, for each k,

Pk(x) = Ixkl defines a seminorm.

(2)

The space of all real convergent sequences {x

=

lim x exists} n n+ co

(Xl ,x 2 ' ••• ) :

will be denoted by c.

A norm for c is defined by II x II

With this norm c is a separable Banach space.

=

s up j x

n

Co

Similarly,

c

n

I.

c

will denote the separable Banach space of all null convergent

= sup Ix n

sequences with norm II x II Cco The symbol R )

(3)

I·

denote the subspace of

real sequences which are zero except for a nates.

With norm defined by I Ixl

I

The spaces LP , I

(4) x

=

each of the spaces

=


0

lim P[d(V ,V) 2: EJ = 0, and n .... oo n r V) in the rth mean, r > 0, (V .... if n E[d(Vn,V)r] = 0

where the expected value E[d(V ,V)r] is assumed to exist. n

Other modes of convergence can be defined such as convergence in distribution [Billingsley ClS68)], but they will not be discussed here since these notes will concentrate mainly on developing results

29

using convergence with probability one and convergence in probability.

Convergence results of weighted sums of random elements

in which the convergence is with probability one are called strong laws, and those concerned with convergence in probability are referred to as weak laws. When d(V,Z) is a (non-negative) random variable, the following form of Markov's inequality is valid for random elements: (2.2.1)

PCd(V,Z) for each r > 0 and exists.

£

>

0 whenever the expected value ECd(V,Z)r]

Inequality (2.2.1) is frequently used in obtaining both

strong and weak laws for random elements in Chapters III, IV, V, VI, and VII. Most of the relationships which exist among the various modes of convergence for random variables are also valid for random elements

in semimetric spaces.

The reader may find it instructive

to verify some of the implications (and non-implications) of the following diagram: convergence with

convergence in

probability one

the rth mean

%

# convergence in

i

probability

convergence ln distribution The implications are verified exactly like the random variable case, and the counterexamples can be constructed when the semimetric space is the real numbers.

30

2.3 PROBABILITY PROPERTIES OF RANDOM ELEMENTS A random element in a topological space induces a probability measure on the space and its Borel subsets.

In this section the

definitions of identically distributed and independent random variables will be extended to

elements, and characteriza-

tions of identically distributed random elements and independent random elements will be provided for certain linear topological spaces.

Finally, the Pettis integral will be used to define an

expected value for random elements. Definition 2.3.1

The random elements V and Z are said to be

identically distributed if P[V for each B

B(X).

B] = P[Z

B]

A family of random elements is identically

distributed if every pair is identically distributed. Definition 2.3.2

A finite set of random elements {V ... ,V n} 1, in X is said to be independent if

for every B ... ,B BlX). A family of random elements in X is 1, n said to be independent if every finite subset is independent. It is important to note that independent, identically distributed random elements in a separable Banach space X are strictly stationary random elements in X as defined by Mourier (1953, 1956). Lemma 2.3.1 (a)

Let {V : a

a

A} be a family of identically

distributed random elements in X and let T be a Borel measurable function from X into another topological space Y. a

Then {T(V ): a

A} is a family of identically distributed random elements in Y.

31

(b)

Let {V : a

a

A} be a family of independent random

elements in X and let {T: a

a

A} be a corresponding family of

Borel measurable functions from X into Y.

Then {T (V): a a

a

E

A}

is a family of independent random elements in Y. Proof:

Lemma 2.1.1 provides that T (V ) is a random element a a in Y for each a A. Let Ct , ... ,Ct B(Y). Then A and B ... ,B n l, 1 n

E

Thus, {Ta(V in Y.

a):

a

B ].

n

A} is a family of independent random elements

III

The proof for identical distributions is similar. Definition 2.3.3

Let C be a subfamily of B(X).

probability measures P and Q on B(X) P(D)

= Q(D)

If for all

for each

E

C

implies that P = Q on B(X), then C is called a family of unicity or a determining class for B(X).

[See Grenander (1963), p. 128,

and Billingsley (1968), p. 15.] If C is a field and a(C) = B(X), then C is a family of unicity [Halmos (1950), p. 54].

Also if X is a separable seminormed linear

space, then

c =

{{x:

f Cx) < t l :

f

X* and t

E

R}

is a family of unicity [Grenander 0963), p , 128.J. C[O,l] and C[O,oo), the collection

0.3.1) For the spaces

32

c

= {I x :

forms a family of unicity [Billingsley L1968) and Whitt (1970)J. Lemma 2.3.2

Let X be a separable seminormed linear space.

The random elements V and Z in X are identically distributed if and only if fLV) and fLZ) are identically distributed random variables for each f Proof: in X.

E

X*. Let V and Z be identically distributed random elements

Since each f

X* is a continuous function from X into R,

E

f(V) and feZ) are identically distributed random elements in R (that is, random variables) for each f

X* by Lemma 2.1.1.

E

Suppose that fLV) and feZ) are identically distributed random variables for each f

E

X*

If P

= Pz

v

which is given in (2.3.1), where P

on the family of unicity

v and P z are the probability

measures induced on B(X) by V and Z respectively, then V and Z are identically distributed.

Thus, P[V

E

{x :

P [V

E

f

-1

f Cx) < t}J

(-co, t

)

J

P [ f ( V)

-co

n

(2.3.11)

V dP. n

A necessary and sufficient condition for the Bochner integral to exist for a strongly measurable random element V in a Banach space is that

Jn II V I IdP




ECg (V)g (Z)]. n

n

(3.3.5)

But, g n (V)

= g(U n (V»

(\,n f (V)b ) - g Lk=l k k

and

Hence, for each n

(3.3.6) Condition

(3.3.1) implies that each term of each sum is zero and

hence that ECgn(V)gn(Z)]

=

0 for each n.

Therefore, (3.3.2) is

satisfied since ECg(V)g(Z)]

= lim n-i'-OO

ECg (V)g (Z)] n

n

=

0,

and the random elements V and Z are weakly uncorrelated.

III

Theorem 3.3.1 is also valid for any normed linear space which has a Schauder basis such that

{I Iun I I}

is bounded.

The verification

of Condition (3.3.1) depends on the partiCUlar basis {b

which is n} used, but it must hold for every basis for the random elements to be weakly uncorrelated. hold for k

= n,

Also observe that Condition (3.3.1) must

or for each k

58

Thus, weak uncorrelation implies coordinate uncorrelation for every Schauder basis where the coordinate functionals are continuous. For the random elements V = (V ... ) and Z = (Zl,Z2"") 1,V 2, 2 2 (00) , [with EI Ivi I < 00 and Ej Izi I < ooJ in the sequence spaces c,c O' R and tp(p

1), Theorem 3.3.1 states that V and Z are weakly uncor-

related if and only if Cov (V for each k and n.

n

+ V ,Z +Z ) k n k

=

(3.3.7)

0

Condition (3.3.7) is also necessary and sufficient

for random elements to be weakly uncorrelated in c. verified for cO' R(oo), or tp(p

This is easily

1) by considering the basis

{ (1,0,0,

), (0,1,0,

), ... I .

{(1,1,1,

), (1,0,0,

), (0,1,0, ... ), ... } is a basis and the

For the space c , recall that

coordinate functionals {fn} are given by fO(x) = lim x

m

and fn(x)

= x

- lim x for each n 1. To show that Condition (3.3.7) n m implies Condition (3.3.1), three cases are considered. First, without loss of generality, assume that EV

= 0 = EZ.

Thus,

E(Z ) for each n. n Case 1:

k

o

and n

=

0

First,

But, for each m

and 0, then Elxlll+l/Y < implies that 00

k

with probability one. Let

be an array of real numbers such that A

n

for each n.

= Lk=l \''''

Chow

a2

nk


0 (3.4.10)

or if A

n

=

then

with probability one. Theorem 3.4.8 random variables.

Let {X } be independent, identically distributed n Then EX = 0 and E(Xi) < w if and only if for l

every array {a nk} such that lim n+ w

= 1 it follows that

with probability one. Stout (1968) considered the complete convergence of the sequence {Y in the sense of n} (3.4.11) for each E > O.

Note that Condition (3.4.11) implies that Yn

with probability one.

+

0

Stout's (1968) results involved varying

moment conditions and conditions on the array {a

nk}

in addition

to (3.4.9) and (3.4.10) of Chow (1966). Rohatgi (1971) extended Pruitt's (1966) results to independent, but not necessarily identically distributed random variables {X by k} requiring that there exists a random variable X such that

68

0.4.12) for all t > 0 and all n.

Theorems 3.4.9 and Theorem 3.4.10 are

from Rohatgi (1971). Theorem 3.4.9

Let {X } be a sequence of independent random n

variables such that Condition (3.4.12) holds and let {a nk} be a Toeplitz sequence.

If

max lankl + 0 as n + k r < 00 (ii) for some Elxl

(i)

(iii)

lanklr

s;

C

o


0

max lankl = k

E!X/ 1 + 1/ y
-

00.

3.5 PROBLEMS 3.1

Let X be a normed linear space and let V be a random element

in X. 3.2

Show that E(IIVI1

2)


1 and L > O. Theorem 4.2.3 [Taylor and Padgett (1974)J

Let X be a separable

Let {A } be a sequence of real-valued random

normed linear space.

n

variables and let {V } be a sequence of identically distributed n

random elements in X such that Conditions (4.2.3) and (4.2.4) hold. If {A V } is a sequence of independent random elements and if n n E(A V ) n n

with probability one. A more useful form of Theorem 4.2.3 for applications is obtained by requiring the following conditions to hold:

84

(4.2.5) and

for all n where L >

° [Taylor

and Padgett (1974)].

The following

example gives an application where the Cesaro boundedness of Condition (4.2.6) is satisfied but where the Cesaro convergence to zero of Condition (4.2.2) does not hold.

The example will also

show that identically distributed random elements are often easily obtained from random elements which are not identically distributed. Example 4.2.1

Let {Z } be a sequence of separable Brownian n

motion processes on [0,1] such that {o2 = E[Z2(1)]} satisfies n n

Ok} is a bounded sequence and that

the condition that ,'" 2 2 Ln=l 0n/n < "'.

Each Zn can be regarded as a random element in

C[O,l], and Theorem 4.2.3 can be applied by letting Zn = 0nVn if the {Zn} are independent.

By construction the {V are identically n} distributed since 1 = E[V 2(1)] (Property 2.3.5). Conditions (4.2.5) n

and (4.2.6) are satisfied since An = an and EI lVI'

with probability one since E(Z ) n each n.


1. Also 00

let {A } be a sequence of random variables such that for each n n

1

;

(ECIAklsJ)s s L

= E(AIV 1) for each n) X* the weak law of large numbers holds for the

where L is a positive constant, and let E(AnV n.

For each f

E

sequence {f(A V )} if and only if n n

in probability. If X has a Schauder basis such that

{I Ivn I!}

is a bounded

sequence for the partial sum operators {V}, then the weak law of n

large numbers holding in each coordinate is necessary and sufficien' for the weak law of large numbers in Theorem 4.2.4.

86

4.3 BECK'S CONVEXITY AND THE STRONG LAW OF LARGE NUMBERS In this section the strong law of large numbers for independent random variables with uniformly bounded variances will be extended to separable normed linear spaces which satisfy Beck's convexity condition (see Definition 4.3.1 below).

The convexity condition is

a necessary and sufficient condition for this extension. Example 4.3.1 will show that the strong law of large numbers for random variables which satisfy Kolmogorov's condition on the variances (see Theorem 3.2.4) does not extend to separable normed linear spaces even if the convexity condition is satisfied.

Also included in

this section will be a discussion of Giesy's (1965) results for normed linear spaces with this convexity condition and other implications. A normed linear space X is said to be uniformly convex if for every

E

> 0 there exists a 0 > 0 such that

and Ilx +

vl l

> 2 -

26

implies that Ilx -

II x II ,;; 1, v l l < £ for

II y II

:0;

all x,y

1, E

X.

Beck (1958) extended the strong law of large numbers for random variables with uniformly bounded variances to separable, uniformly' CGDVex

Banach spaces.

This result will follow as a corollary to

Theorem 4.3.1. Definition 4.3.1 convex of

A normed linear space X is said to be

(B) if there is an integer t

that for all xl' ... ,x

t

E

X with Ilx·11 l ± x

t

II

:0;

> 0

1, i

< t

and an E > 0 such 1, ... ,t, then

( l -E)

(4.3.1)

for some choice of + and - signs. Giesy (1965) extensively studied the convexity property of normed linear spaces which is given in Definition 4.2.1.

Finite-

dimensional normed linear spaces, uniformly convex normed linear

87

spaces

(and hence the LP-spaces and iP-spaces, 1 < P < 00), and

inner product spaces are convex of type (B).

Examples of normed

linear spaces which are not convex of type (B) include i1, ioo, and cO'

Giesy (1965) also characterized type (B) convex spaces by

conditions on their first and second conjugate spaces and on factor spaces.

One interesting geometric characterization of spaces which

are not convex of type (B) is that they must have isomorphic copies of finite dimensional i1 for arbitrary finite dimension. that i1 in Example 4.1.1 had this property.

Recall

More detailed results

on spaces which are convex of type (B) may be found in Giesy (1965). The following several pages will be used to reproduce Beck's (1963) strong law of large numbers for random elements in normed linear spaces which are convex of type (B) and whose variances are uniformly bounded. Theorem 4.3.1

If X is a separable normed linear space which is

convex of type (B) and if {V } is a sequence of independent random n 2 elements in X such that EV = a and EI IV 11 $ M for all n where M n

n

is a positive constant, then

with probability one. Since every normed linear space is isomorphic to a dense subspace of a Banach space, it suffices to prove Theorem 4.3.1 for separable Banach spaces which are convex of type (B).

The proof

which will be given is essentially the proof contained in Beck (1963) Definition 4.3.2

A random element V in a normed linear space

X is said to be symmetric if there exists a measure-preserving function ¢ of that PCV

into 0

(that is, PC¢

¢ = -VJ = 1.

-1

(B)J = PCBJ for each B

A)

such

88

Recall that 61 Ivi I denotes the essential supremum of the random variable I Ivl I.

For a sequence of random elements in X, define (4.3.2)

cIV } n A sequence of random elements is said to be of

(A) if they are

bounded in norm by 1, have the zero element as their expected values, and are symmetric and independent.

C(X) = sup {c{V }: n

{v

n

Finally, define (4.3.3)

lis of type (A) in X}

where the supremum is taken over all sequences of random elements where are of type (A) in X.

Note that 0

C(X)

1 since for any

sequence of random elements {V } of type (A) n

6(lim sup 1)

1.

n

In part (a) of the proof of Theorem 4.3.1, it will be shown that C(X) = 0 when X is convex of type (B). Proof of Theorem 4.2.1 - Part (a):

Let X be a separable Banach

space which is convex of type (B) and suppose that C(X) Hence,

=C

#

o.

for n > 0 there exists a sequence of random elements {W } in n

X which are of type (A) and such that c{W } > C n Z

n.

Define (4.3.4)

n

where t is the positive integer which is given by the type B convexity of X.

Without loss of generality, it can be assumed that the

sequence of measure-preserving

n

} which correspond to

the random elements {W } have the property that n

89

pewm

0

¢n = WmJ = 1

(4.3.5)

pewn

0

¢n = -Wn J = 1

(4.3.6)

for m f; n and

for each n, otherwise, consider the infinite product of the original probability space

and let the random element W be identin

fied with the nth coordinate.

Note,

E(Wtn)+ECWtn_l)+" t

E(Z ) n

.+E(W t n_ t+ l)

and the random elements {Z } are independent.

For the symmetry of

n

each Zn' let ¢Z type (A) and

n

= ¢tn

0

¢tn-l

0

•••

0

0,

¢tn-t+l'

Thus, {Zn} is of

c{Zn} = f3

1

£

t'

(4.3.11)

2

and for each n define

11 Y

(4.3.12)

n

it follows that {Y is of type (A) and that c{Zn} = c{Y n}. n} 2 Similar to (4.3.11) it can be shown that EI IZnl 1 < 1, and hence Var

(I

IZnl I) < 1.

But since

{I

IZnl I} is a sequence of independent

random variables, Z

II/II)


O. pleted.


0 be given, let W

o

E

D and define

e4.3.22) Since PeEn)

+

whenever n > N. {feV

00, there exists an N such that peE n ) > 1/2 For each continuous linear functional f E X*,

1 as n

+

is a sequence of independent random variables with expected 2M. 2 values equal to zero and Var (feV» s IIfl1 Var ev) s Ilf11 n n 2M For each f E X* such that I If I I s l and for n > n)}

94

Let D

n,

f denote the set D

For n

max {N, P [E

n

n,f

=

{te :

E: n

D f J = P ( E ) + P ( D f) - P [E u D r J n, n n, n n,

X1, such that Ilfll

from (4.3.23) and (4.3.22).

Thus, for any f

and for any n > max {N,

there exists an element wn

and nence

i;

tevk(w O»

E:

!

E:

f(Vk(w O) - Vk(Wn »

I

(Vk(WO) - Vk(w n )

/1

E:

1

En n Dn,f

+ E:

< 2E:.

Thus, by the Hahn-Banach

whenever n> max {N,

E:

(Corollary 1.2.3)

III

and the proof is completed.

In addition to Theorem 4.3.1, Beck (1963) showed that convexity of type (B) is necessary to obtain the strong law of large numbers for independent random elements with zero expected values and bounded variances.

More precisely, if X is not convex of type (B), then

there exists a sequence of independent random elements {V

n}

with

E(V n) = 0 and Var (Vn) M for all n and such that I Vk' I does not converge to zero with probability one. Example 4.3.1 [Beck (1963), Example 15J will show that Kolmogorov's condition on the variances [rf. Theorem 3.1.4J is not sufficient for the strong law

95

of large numbers even in separable normed linear spaces which satisfies Beck's convexity condition. Let p be a real number such that l < P < 2.

Example 4.3.l

Recall that £P denotes the separable Banach space 1

Ilxll = 0: [x n JP)p

< co}

and that on denotes the element having one for its nth coordinate and 0 in the other coordinates.

I - !P

< q < 2' I

defined by An for each n.

Let q be a real number such that

. . bl es Let { An} b e a sequence of lndependent random varla

= ±n q

each with probability

Thus, {V } is a n

in £P such that E(V ) n

"co

L.n=l

=0

i,

and define V

n

= Anon

of independent random elements

for each nand

Var(V n ) n

since 2 - 2q > 1.

2

But, for n even

(!:.)pq 2

nP

which goes to

slnce 1 -

!

P

< q implies that pq + 1 - P > O.

III

Since t P (p > 1) is also a uniformly convex, reflexive Banach space, Example 4.3.1 also provides a counterexample to the extension of Theorem 3.1.4 to either uniformly convex spaces or reflexive spaces.

It was earlier conjectured that all B-convex spaces were

reflexive.

However, James (l974) presented an example of a non-

reflexive B-convex space.

The material in this section is only

the start of a large area of current research.

The next section will

96

develop one aspect of this area, the G condition, and discuss the a type p spaces, but first the following two theorems of Giesy (to appear) are listed. Theorem 4.3.2 separable. with EV n

Let X be a Banach space whose dual space X* is

Let {V } be a sequence of independent random elements n

=

0 for all n.

If Var(V ) 2 n < n

00

and

I}

n

is a bounded sequence, then {V

satisfies the weak topology strong n} law of large numbers (WTSLLN), that is, there exists c such

=1

that

be such that either

fC! ,n V,_(w)) n Lk=l for each w

c

0

X*.

and f

Let X be a Banach space which is not B-convex,

Theorem 4.3.3 and let {sk}

+

R+ be such that either 2

,00

Ln=l

sn

00

n

,n n Lk=l

or 1

s

k

+

00

Then, there exists a sequence of independent random elements {V } n

with EV

n

= 0 and a(V ) n

s

n

for all n which fail to satisfy the

WTSLLN.

4.4 OTHER GEOMETRIC CONDITIONS AND THE

STRONG LAW OF LARGE NUMBERS In this section geometric conditions are imposed on the Banach spaces to obtain strong laws of large numbers.

As indicated in the

97

introduction of the chapter, not all results in this rapidly developing area will be presented.

The main goal of this section will be

the presentation of Woyczynski's L1973) strong laws of large numbers for independent random elements in "G "spaces. Also, the strong a law of large numbers for independent random elements in "type p" spaces will be listed, and a brief comparison of the different geometric conditions will be given. The G (a a

= 1)

was introduced

by Fortet and Mourier (1955)

and was motivated by the inequality between the moment of a sum of independent random elements and the sum of the moments. Definition 4.4.1

A Banach space X is said to satisfy the

condition G for some a, 0 a G:

X

+


1 - 1 and hence p

co.

4.5 OTHER CONVERGENCE RESULTS AND EXTENSIONS TO FRECHET SPACES Although Chapter IV is concerned with laws of large numbers for separable normed linear spaces, some related results will be listed in this section which are rates of convergence or which are in spaces which are not normed linear spaces.

The laws of large numbers which

will be discussed in this section are concerned with convergence in the rth mean. also be stated.

A result for sums of independent random elements will Finally, the extensions of the laws of large numbers

to certain Frechet spaces will be outlined. Mourier (1956) proved that if X is a separable Banach space and if {V is a sequence of independent, identically distributed n} random elements in X such that EI lVI' ,r < co, I

In addition, if X* is separable and if r

$

r < co, then

2, then there exists a

positive number p such that pn for all n.

-r/2

Also, a reverse inequality may be obtained by restricting

the Banach space to be a G} space [see Mourier (1956)J.

105

For separable Banach spaces which are convex of type US), Giesy (1965) obtained the following result which gives an upper bound on the rate of convergence for a law of large numbers of the Mourier type stated above:

Let 1

P < q

with 2

$

q.

If X is

a separable Banach space which is convex of type (B) and if {Vn} is a sequence of independent random elements in X with EV = 0 and n (EI IVnl Iq)l/q exist

$

M for all n [Sl IVnl

real numbers b

n

I

= b n Lp,q,t,£)

$

M for q = roJ, then there

such that b

n

0 and

for all n. Alf (preprint) extended the results on the convergence rates of Lai (1974) and Heyde and Rohatgi (1967) to independent

elements

ln separable Banach spaces. Keulbs (1976 & 1976) has obtained Banach space versions of the law of the

logarithm in addition to several other related

convergence Ito and Nisio (1968) proved the following result concerning sums of independent random elements: independent 3

n

elements in a separable Banach space and let

VI + .•. + V n' (i) (ii) (iii)

Let {V } be a sequence of n

Then the following conditions are equivalent:

Sn converges with probability one, Sn converges in probability, and the distribution of Sn converges in the

metric

[see Billingsley (1968)J. Recall that the

space F in (10) of Section 1.1 has a

metric which is given by the seminorms {Pk}'

combination of a family of

For each k the space F with seminorm Pk is a semi-

normed linear space which is denoted by F seminormed linear space F

k

and the topology of each k, is weaker than the metric topology of F.

Also, convergence (either in probability or with probability one) in

106

all of the seminormed linear spaces F

k,

k = 1,2, ... , is equivalent

to convergence in the metric topology of F.

I

Ik = Pk(x)

space Fk/N

k

where

with N

k

x denotes =

{y

E

F

k:

For each x

E

Fk , let

the embedded element in the quotient Pk(y)

=

a}.

Thus, if the correspond-

ing moment conditions are defined in terms of the seminorms {Pk}' then several of the laws of large numbers of this chapter extend easily to F since the embeddings preserve the probabilistic proper­ ties.

Additional details on these

extensions are available in

Chapter VI of Padgett and Taylor (1973). The strong law of large numbers for independent, identically distributed random elements in F was obtained by Ahmad (1965) using a different method of proof, but the above embedding technique is simpler and allows further results.

For example, strong laws of

large numbers in F which are not necessarily for identically distri­ buted random elements may be obtained by using the results of Section 4.2.

For the weak law of large numbers to hold for identically dis­ tributed random elements in F, it is necessary and sufficient for the weak law of large numbers to hold in the weak linear topology of F.

This result follows from Theorem 4.1.5 and the indicated

embedding technique since f is a continuous linear functional on F if f is a continuous linear functional on each F

Thus, the weak k. law of large numbers also holds for weakly uncorrelated random elements in F.

4.6 PROBLEMS 4.1

Let Vl"",V

space X.

be random elements in a separable normed linear t Show that for each E > 0

1­

Ilv·11 1

> EJ,;

1­

p[jjv·11 1

> £J. t

107

4.2 Vn

Prove or Disprove +

0 with probability one. ln "n

4.3 V n

Vk

Prove or Disprove +

0 in probability.

"n

n1 [.k=l Vk 4.4

Let {V } be random variables such that n

+

0.

Then with probability one.

Let {V be random variables such that n} Then +

0 in probability.

Show that every uniformly convex space is convex of type (B)

by finding the appropriate t and 4.5 4.6

E.

Verify that c{Z } = c{W } in (4.3.7). n n Show that the inequality in Theorem 4.4.1 does not hold for

I Ixl)

= Llx n I < oo} for any a

>

0 and hence L

I

is

not G . a

4.7

Show that every separable Banach space is of type 1.

4.8

Conjecture:

Define independent random elements {V in c, n}

the space of convergent sequences,

I Ivn I I = 1

and EV

n

= 0 for

all n, but where

(Neither the author nor any of his colleagues have resolved the conjecture, and there is a split opinion on its validity.)

CHAPTER V

CONVERGENCE OF WEIGHTED SUMS IN NORMED LINEAR SPACES

5.0 INTRODUCTION In this chapter the convergence of the weighted sums

are obtained under various conditions on the weights {a random elements {V and the normed linear spaces. n},

nk}

, the

Laws of

large numbers will be special cases of convergence results for weighted sums of random elements.

In particular, in Section 5.2

very general and powerful convergence results are proved for weighted sums of tight random elements with uniformly bounded rth moments (r > 1).

These results yields new laws of large numbers

without requiring geometric conditions on the spaces and supplements the results of Sections 4.1 and 4.2.

Also, in Section 5.2 com-

pari sons of these results with previous work are discussed, and several examples are presented to disprove other plausible conjectures. In Section 5.1 several results for the convergence of weighted sums are obtained using the identical distributions of the random elements, moment conditions on the random elements, and restrictions on the weights.

For identically distributed random elements in a

separable normed linear space, convergence in probability in the weak linear topology is necessary and sufficient for convergence in probability for the weighted sum in the norm topology. lim a nk = 0 for every k, maxla k l = k n

n-

Next, let

for some a > 0,

be uniformly bounded for all n, and {V be independent, identically k}

109

distributed random elements in a separable normed linear space X . h EV l Wlt

= Q•

bilityone.

l .X, Then l• f E II V Il +l / Cl < 00, Sn + 0 lD Wlth pro b al

Conditions similar to Rohatgi's dominance in probability

are introduced in relaxing the identically distributed condition. In Section 5.3 geometric conditions of the normed linear spaces are used to obtain additional results.

For example, it is shown

that funder slightly different conditions on {a

than those nk} stated above) weighted sums of independent, not necessarily identi-

cally distributed, random elements {V with EV = 0 converge with k} k probability one to the zero element if the random elements have uniformly bounded rth absolute moments for some r > 1 and X is a normed linear space satisfying Beck's convexity condition.

Finally,

the type p spaces are used in Section 5.3 to obtain convergence in probability for weighted sums of independent random elements.

5.l DISTRIBUTIONAL CONDITIONS AND CONVERGENCE OF WEIGHTED SUMS Recall that a Toeplitz sequence is -a double array of real number-s {a

n k}

satisfying lim a n+ oo nk

o

for each k,

(5.1.1)

for each n

(5.1.2)

and

where C is some positive constant.

Theorem 5.1.1 [Taylor and

Padgett, (1975)J will show that convergence in probability in each coordinate of a Schauder basis for a Banach space is a necessary and sufficient condition for weighted sums of identically distributed random elements to converge in probability in the norm topology.

110

Theorem 5.1.1

Let X be a Banach space which has a Schauder

basis {b.}, let {a k} be a Toeplitz sequence, and let {V } be a n

1

n

sequence of identically distributed random elements in X such that For each coordinate functional f.

1

in probability if and only if

in probability. The proof of Theorem 5.1.1 follows in the same manner as the proof of Theorem 4.1.3 and hence will not be presented.

In

5.1.2 the dual space will be used ln place of the coordinate functionals, and The proof is obtained by embedding the separable normed linear space isomorphically in C[O,l] (which has a Schauder basis) and by applying Theorem 5.1.1. Theorem 5.1.2

Let X be a separable normed linear space, let

he a Toeplitz sequence, and let {V } be a sequence of identin cally rlistributed random elements in X such that EI IVII I < 00 and

EV l exists.

For each continuous linear functional f

in probability if and only if

in probability. If the identically distributed random elements {V } are weakly n uncorrelated and if the condition that

max lankl 0 as n 00 is lsksn assumed [as in Pruitt (1966), Rohatgi (1971), and Padgett and Taylor

111

(1974)J, then for

£

>

a

(5.1.3) Thus, by (5.1.3) and Theorem 5.1.2 as n

+

00

(5.1.4) in probability.

Moreover, if the identically distributed random

elements are independent, then

max la k lsksn n

l

+

a

as n

+

00

will yield

the convergence in (5.1.4) by [Pruitt's (1966)J Theorem 3.4.5 and Theorem 5.1.2 of this section.

If the Banach space has a Schauder

basis, then the more general (see comparisons in Section 3.3) coordinate uncorrelation will provide the convergence theory.

Also,

Theorem 5.1.1 in conjunction with [Pruitt's (1966)J Theorem 3.4.6 will yield convergence in probability of the weighted sums when the random elements are independent in each coordinate of some Schauder basis for the space. Example 4.1.1 again illustrates some of the futility in trying to obtain convergence results for nonidentically distributed random elements without assuming stringent moment conditions. if

then

For example,

112

In attempting to obtain somewhat less trivial

in probability.

convergence results than the one previously listed, Example 4.1.1 provides the counterexample for many plausible extensions of random variable results.

However, results can be obtained for classes

of random elements which need not be identically distributed by using product sequencffiof random variables and random elements. The product sequences results are in Taylor and Padgett (1976) and will not be reproduced here, Next, let max k

la k n

l

=

(5.1.5)

for some a > O.

Theorem 5.1.3

Let {V be a sequence of independent, identik} cally distributed random elements in a separable normed linear space X with EV = 0 and let {a be a Toeplitz sequence which l nk} i f 1es i Con d 1t1on i ( 5.1.5 ) . If EIIV1111+1/a < "", then sa 't 1S

with probability one, Proof:

be a Schauder basis for C[O,lJ, and let h i} denote the one­to­one bicontinuous linear function from X into C[O,lJ.

Let {b

Define h(V

= V For each coordinate functional of the k) h k. basis, f the sequence of random variables {f (V are indepeni, i h k)} dent and identically distributed by Lemma 2.3.1. For each fixed m, ankV h k) II

= III:=l f i "

ankVhk)bill ankf i (Vh k) Illbill .... 0 (5.1.6)

with probability one as n .... "", since by Theorem 3.4.6 and the fact that

113

it follows that

with probability one. Now, for each m

by Condition (5.1.2) and the hypothesis that {V are identically k} distributed.

But {I 1Qm(Vhk)1

I-EI I QmCVhl) II}

is a sequence of

independent, identically distributed random variables with zero means for each m, and by hypothesis l /a < [I Ihl I CM+l)]l+l/a E ! Ivll I l +

00,

1 l /a IQ (V 1)/ 1 + m h where M is the basis constant.

EI

Hence, by Theorem 3.4.6

(5.1.7) tv..;

th probability one as n Let

-+

00.

be the countable union of the null sets for which (5.1.6)

and (5.1. 7) do not hold for all m

1.

Since

II Qm (Vh l) II

-+

0

pointwise, by Lebesgue's dominated convergence theorem, for s > 0 choose m so that

EI

1Qm(Vhl)1

I


n

Then

with probability one. Proof:

Again, it suffices to prove the result for a Banach

space which has a Schauder basis. constant.

Let m > 0 denote the basis

For each fixed positive integer m, consider

117

I

ankV k) I I = 1, ... ,m, {fiLV

i

ankfi(V k k)}

)!-I Ibil I·

For each

is a sequence of independent, identically

distributed random variables with Ef Elf i LV 1)

I

iLV 1

) = O.

I I f i I IE I IV1 1I


0 be given,

of {5.2.3) has a compact closure, say K. and let Ks be compact such that P[V

n

E

KsJ

>

E

Ke J

1 - s for all n.

Then,

by the choice of K and (5.2.3), P[V

n

- EV n

E

Kc - KJ

P[V

n

1 - s

C5.2.4)

This completes the proof since K - K is compact. c

for all n.

Lemma 5.2.1

III

may not be true without the rth moment condition

even in a separable Hilbert space. Example 5.2.1 sequences.

Let X

= t 2,

the space of all square sumrnable

Let {V be defined by n}

V

n

=

no On with probability lin, { with probability 1 - lin,

where {on} is the standard basis as in Example 4.1.1.

I

Then

= 1 for all n, and {V

is tight by choosing {OJ u {finite n} number of non,s} as the compact set for each s > O. Now EI !Vnl

Cn - l ) On with probability lin, V

n

EV n

{ _on

with probability 1 - lin.

Given 1 > s > 0, any compact set K satisfying c

must contain infinitely many _on,s, which is impossible since then

123

K

E

would contain a subsequence which is not convergent.

Thus,

III

{V - EV } is not tight. n n

Three function spaces in which many stochastic processes take their sample paths are CCO,1],

and DCO,1], the metric space of

functions on CO,1] which are right continuous with left limits and endowed with the Skorohod topology.

In these spaces, tightness can

be characterized by uniform boundedness conditions and uniform equicontinuity conditions (Billingsley 1968 p. 55, p. 125 and Whitt 1970 p. 941) by using the Arzela-Ascoli characterization of compactness.

Other characterizations of compactness may be obtained,

for example, Mangano (1976) characterized the sequential compactness of certain sequences of Gaussian random elements in CCO,l]. For a sequence of random variables {X with uniformly bounded n} rth moments, r > 1, the following lemma concerning their uniform boundedness by a random variable is obtained. Lemma 5.2.2 that EIX

r

n

I

Let {X be a sequence of random variables such n}

r for all n with r

> 0 and r

> 1.

Then there exists

a random variable X such that Ci)

PC/X

n

I

PC Ixi

aJ

a] for all n and for all a

0,

and Cii)

E(lx/ 1 +1 / s )

for 0 < lis < r - 1.


rr

124

1

Also, for a < rr and any n, (5.2.6)

Thus, part (i) is proved.

E(/xI 1 +l

For part lii), observe that 1

/s) =

tl+l/s(rrr/tr+l)dt rr

hl

= rr

L

r/(r_l_l/s)rrs
O. real numbers satisfying

for each f

E

,00

L.k=l

la

X* if and only if

nk

Let {ank}be an array of Then

r

125

Proof:

Since X can be isometrically embedded in a Banach space

with a Schauder basis, it may be assumed without Idss of generctlity that X has a Schauder basis {btl.

Assume further that

r = 1.

The

necessity is immediate since convergence in the norm topology implies convergence in the weak linear topology of X. To prove the sUfficiency, let m be the basis constant that

Let

II Ut II

E >

m andj ] Q t

0 be given.

+

II

s m + 1 for each t.

For each nand t,

For each fixed t,

(5.2.8)

0

since by hypothesis

(Vk)

I

+

0 in probability for each i.

Let K be a compact set in X such that (5.2.9)

for all n.

Then by Lemma 1.3.3, t can be chosen so that (5.2.10)

for all x

E

K.

Thus, by Holder's inequality,

126

0 and r > 1.

a) where 0 < l/a < r - 1, then I f max {a k} = &(nl:o:k:o:n n

with probability one. Convergence in probability for Theorem 5.3.1 is easily obtained from Theorem 5.3.2 (see Problem 5.6).

As in Beck (1963), the proof

138

of this theorem will be given In three parts.

In part

Cal

the

result is obtained by assuming that the random elements are symmetric and uniformly bounded.

Part (b) of the proof will consist

of replacing the boundedness condition by the condition of uni­ formly bounded rth moments (some r > 1). condition is eliminated in part (c). the random variable

I IVI

Finally, the symmetry

The essential supremum of

I will be denoted by 6(/ IVI

I)

throughout

the proof. Proof ­ Part (a):

Let {V be a sequence of independent, n}

symmetric random elements such that let d

n

that r

=

=

min {a k}. lsksn n 1. Then

I Iv n / I

s r for each n, and

It may be assumed without loss of generality

(5.3.3)

since nd

n

s

a

nk

s 1 by Assumption (5.3.1).

But, by Theorem

4.3.1

o.

(5.3.4)

Also, from the condition that I Ivkl l S I for all k and assumption (5.2.3),

139

lim sup Lkn=l (a k - d ) = n n n

o.

(5.3.5)

rherefore, from (5.3.3), (5.3.4), and (5.3.5)

/II

completing part (a) of the proof. Proof - Part (b):

Now, let {V } be a sequence of independent, n r for all n symmetric random elements in X such that and some r > 0 and r > 1.

Without loss of generality, it may be

assumed that r = 1. Let q > 1 be a positive integer and define

and Y = 0 n

and Z - V if n n

II Vn II

> q.

Thus, {Z } and {y } are sequences of independent, symmetric random n n elements with I!Y I I q and EY = 0 = EZ for all n. Hence, .n

n

n

by Part (a) of the proof,

(5.3.6) with probability one.

Also, for each n, by definition of Zn'

Ellznll = 1 r-1

(q)

f

(e II Zn II

+ feliZ 1 r-l

(q)

n

r-ll

I Zn II

II>q]qr-11IZnlldP)

f e "zn II >q] II Zn II r dP

(!)r-1Ellz Il r q

= OJ q

n

dP

140

(5.3.7) since

Elf Vn II r

,:; 1 for all n .

Now, the random variables {liZ I I - EI Iz I I} have mean since

IZ

n

II

n

s (!)r-l < 1, E (I I Zn I I)

- II Zn II E (I I Zn II) - II Zn I I

q

Hence, the event {w for all n.

E

Thus, for a PC

n

< 1 pointwise.

a} =

J

aJ s PC

,:;

II zn II

Ell Zn Il r a

which n

1

r

aJ

,:;

EIIV n Il r

Therefore, by Lemma 5.2.2 and Theorem 3.4.10, as n

such that for every w

if a



° there

Combining this result with (5.3.6)

r

00

with

exists an integer N for

N implies that

by (5.3.7) and the assumptions on Lank}'

a

Thus,

= 1

141

Since q > l was an arbitrary positive integer, this implies that

III

completing part (b) of the proof. Proof - Part (c):

This part of the proof proceeds following

the steps of the proof of Theorem 4.3.1. corresponding step that for n

ankf(V

k)I f on X such that 1 If I I

£J




However, to obtain the

0 and sufficiently large

for all continuous linear functionals

1, simultaneously [see (4.3.23) in Section

4.3J, it is necessary to observe that the Conditions (5.2.1) and (5.2.2) on the weights {a Consider two cases:

n k}

imply that max la k k

1 < r < 2 and r

2.

n

l

+

0 as n

+

00.

If 1 < r < 2, then a > 1

and

since

I If I I

1 and

EI

!Vkl I

r

r for all k.

Thus, from the assump-

la k l = &tn-a) and since a > 1, we have for all f n such that II f II 1

tion that max k

n P[ ILk=l ankf(V k

which is less that

)!

£J

£

-1

a I ) l-a Cn max lank )(1 + r n

for n sufficiently large.

using Chebyshev's inequality

Now, if r

2,

142

(5.3.8) Hence, for su=ficiently large n the right-hand side of (5.3.8)

21

can be made less that

simultaneously for all f such that III

Ilfll:51. It is important to note that the double sequence {a

defined

n k}

by

a

nk

=

j:

k = 1,2, ... ,n k = n+l, ...

satisfies the Conditions (5.3.1) and (5.3.2).

However, this se-

quence does not satisfy the hypothesis of Theorem 5.3.1 that max {a k} = unless r > 2, requiring the uniform boundedness l:5k:5n n of slightly higher order moments than Beck's strong law of large numbers.

This is the case since Part (b) of the above proof re-

quires the uniform boundedness of {liZ II n

EI

IZ

n

I I}

by the random

variable Y in order to use Rohatgi's result, whereas Beck used the strong law of large numbers for random variables to obtain the convergence of

;

IZkl

I.

A number of other possible choice2

for {a are listed in Taylor and Padgett (1975). n k} Theorem 5.3.2

Let {V

n}

be independent random elements in a

Banach space X of type p, 1 < P :5 2.

Let V be a random variable

such that PC II V II

n

for all a > 0 and that assume further that

EV n

aJ :5 PC I V 1

EIVl r


0, there exists N(a,w) such that n

ITn (w) I

< 2 a 13 and

Is n Cw) I

Thus, for W

0

f no

Nla,w} implies that

III

< 013.

To obtain sharper convergence results of weighted sums when {a

are random weights instead of constants, the correlation

n k}

of a

nk

and V must be considered. k

The following simple example

shows that the results of Chapter V cannot be extended directly to random weights. Example 6.l.1

Let {V

k}

be independent, identically distri-

buted random variables such that VI = ±1 each with probability

1/2.

Let a

nk

=

V for 1 k

k

n, a

nk

= 0 otherwise.

Then {a

n k}

is a Toeplitz sequence of random variables such that 1 and lim ank(w)

n.

0 for each w

=1

But,

O. III

It is important to observe that Condition (6.1.5) is not as restrictive as its appearance might indicate.

For example, the

weights (or in the order of) for 1 s k a

nk

n

for k > n

not only satisfy Condition (6.1.5) but

n

\,n 2 1 Lk=l a nk =

as n

0

The choice of weights lin the order of) for 1 s k

n

for k > n also satisfy Condition l6 .l. 5) and illustrates the importance of r < 2.

150

6.2 CONVERGENCE IN THE rth MEAN FOR RANDOMLY WEIGHTED SUMS The following theorem [from Wei and Taylor (to appear, 1978b)] concerns convergence in the rth mean, and hence a weak convergence may then be obtained as a corollary. Theorem 6.2.1

Let {V } be a random elements in a separable n

normed linear space X such that

A for some 1 s r < 2.

Let {a

nk}

, sup Lk=l ,n LL 1 lm n .... co


0:

ay(t)

sup !ax(At) Osts1

sup IAt - t! s E, for some A Osts1

= inf

{E > 0:

sup IX(At) - yet) Osts1

I

s

E

g

and

sup IX(At) - yet) Ostsl

I

s

E

g

and

s

E

and

A}

sup IAt - tl s Osts1

some A inf {E > 0:

I

E,

A}

£

E

sup I At - tl s 0"ts1

some A

g'

A}

la! d(x,y), and similarly for the second inequality.

III

An immediate and unfortunate (for the laws of large numbers since the reverse inequality is more desired) corollary is the following lemma. Lemma 7.1.2

For any xi' Yi

.i n -n d ( l.l= '.

1 s i s n.

,n

l

t., 1 Y') l= l

D, 1 s i s n, 1,n

s d (-l.' n

1

1 ,n

X., -n t:«l= 1 Y·l l

)

•

156

Proof:

Follows from d(:O,ax)

lald(O,x) and

III

d(:O ,x+yl s d(:O ,x) + d(:O ,y). If x,y,u,v e: D, then

Lemma 7

d Cx + u, y + v) s d Cx yy ) + II ull Proof:

Given 8 > 0, find A e:

co

+

l lvl l..

A such that

sup I x0t) - y(t)1 < d(x,y) + 8 Ostsl and sup IAt - tl Ostsl

< d(x,y)

+ 8.

Thus, IX(At) + U(At) -

(y(t) + v Ct r ) + vet) - u Cx t ) I < d Cx yy ) + 8,

and -lu(;A.t) - v Ct ) ] + Ix(;A.t) + u Cx t ) -

Cy Ct ) + v(t»1 < d Cx y ) + 8. j

Hence, SUD I x Cx t ) + u Cx t ) Ost$l s d (x , y) +

Recalling that

(y(t) + v Ct )

II U II

sup IAt - tl Ostsl

co

+

I

s d Cx vy )

I I v I I",

< d(x,y)

+ 8.

+ 8, it follows that

d (x + U, Y + v ) s d Cx , Y ) +

I IU I I

If u = v

cc

+

I Iv I I

co

III

since 8 > 0 is arbitrary. Remark:

SUpIU(At) - v(t)I+8 Ostsl

c = constant, then

d Cx + c , Y + c)

= dCx,y).

157

With the Skorohod metric d, D is a linear space of functions and is a metric space, but it is not a linear topological space. The metric d is not translation invariant, and addition is not a continuous operation [Billingsley (1968), page

However,

scalar mUltiplication is continuous. Example 7

=-x O'

and x x

Then, x

= xn -

n + x

Let x

x

dUc

o

n

I

n

n

...

"c in the Skorohod x o - x = 0 since o

fr

n

=

2,3, ... , x

2 n'

o =

I 1 [2,lJ

topology, but

+ x , 0)

1

for all n.

III

However, the following interesting property holds. Property 7.1.5 x n' Yn' x, y, z Proof:

If x

n

... x, y

n

... y, and x

+ y

n

n

... z with

D, then z = x + y.

for x

of x in 1"= [O,lJ.

D, let Ix denote the set of continuity points

we have by hypothesis y n Iz z(t) [Skorohod con x Ct ) , y (t) ... yet), and x (t) + Yn (t) x (t) n n n vergence implies convergence at continuity pointsJ and thus for t

...

z(t)

x(t) + yet).

I

x

n

I

...

But the complement of Ix n I y n I z is at

most a countable subset of I and since z

D by hypothesis, III

z = x + y.

With the Skorohod topology, D is not locally convex.

It is

locally convex at some points, for example, at the origin and at the constant functions. BS(x

O)

= {x

s} is not necessarily convex.

D:

the convex hull of d(x,x

o)

from X

o

Moreover, the neighborhood ball In fact,

may contain points x whose distance

is arbitrarily large. Let A

1

1 and let 0 < s < 2'

158

Define

o =

X

d-co

Proof:

m 2 (E. Step functions of the form x m = Li=lri_lI E. 1,m 1,m

as above) with r. rational are dense in D with the Skorohod 1

topology (Billingsley, page ll2). there is such an x

m

Let xED.

and a A E A satisfying

To every

E:

> 0

sup IX(At) - Xm(t)

I

£

163

and

sup IAt - tJ O,;t,;l

Find r

, 1 ,; i

i

,; 2

m,

such that

sup IxCL} - r., s c ; thus sup ] T G

sequence which is bounded above by 1, X(t in D[O,lJ.

- X('n+l) + 0 pointwise n) Thus, by the dominated convergence theorem,

EjX(t n) - X('n+l)

I

Theorem 7.3.2

III

for each n is impossible.

>

If {X } are independent, identically distrin

buted random elements in D[O,lJ such that EI [xII I < 00, then

with probability one. In the Ranga Rao's proof of Theorem 7.3.2, the following characterization of compact sets in terms of

[see Billingsley

173

(1968)J is used:

if K is a compact set in D, then for

exists Il > 0 such that [x Ct ) - xW.l1 a

t


I

0 there

+ c whenever

K.

Proof of Theorem 7.3.2: sCyarable and complete, then P compact set K such that

> 0 be given.

Let

x1

Since D is

is tight and there exists a

(7.3.3)

Choose 0 > 0 so that a

uniformly for x {t.: .i,

t < 6 < a + 0 implies that

/x

Ct ) -

£

K.

x (a) /

Ix (6 - 0)

-

x (a)

I

+ e

0.3.4)

By Lemma 7.3.1 there exists a partition

i = 1, ... .m) of [O,lJ such that max

where J

i

for all 1

=

[t _ i 1,ti). i

m.

sup E/X1 (t) - Xl (s) s,t J i

I

It can also be assumed that t

For each nand t

0.3.5)

i

- t _1 < 0 i

[O,lJ

0.3.6) By Theorem 3.1.5 (7.3.7)

with probability one.

For the first term of (7.3.6),

174

+

I

l

(7.3.8) by compactness of K and t

i

- t

i_ 1


l.

Rence by K. L.

Chung's strong law of large numbers,

the first term tends to zero with probability one as n

+

For

the second term, (7.3.ll1 yields (7.3.16) for every n.

r1 / r

Finally, by (7.3.11), (V)

E•

Thus, a null set can be excluded for each m, and the countable union

nO

is obtained.

For E >

0

and w

4 no'

m is chosen large

enough so that (I) and (III) are each less than E.

Then N(E,W)

is chosen large enough so that (II) and (IV) are each less than E./// The following strong law of large numbers by Daffer and Taylor (1977) can be used for the random elements which were excluded from Theorem 7.3.3 by Example 7.1.5.

Let Dt denote the

cone of non-decreasing elements of D. Theorem 7.3.4

Let {X

n}

be a sequence of independent random

elements in D satisfying (i) (ii) (iii)

X n

E

L

Dt almost surely, for each n;

EIIX n


c and [x ,Ct ,) - x ,(t ,-0)1 n n n n n n n n > e.

Since xn,x

have jumps at tn,t n"

n'

respectively, of

magnitudes> (., it follows from (7.4.2) that for i = n,n' IX Ct ) - x Cs) 1 < (./2 n n

sup l

l

and sup l

Now let T

l

= {to,t1, ... ,tm}

{ti-t i_ 1}

8·

(7.4.3)

[x (t) - x (s)1 < (./2. n n

be any partition of [O,lJ with

Since Itn-t n, I < 0, T can contain t n or tn'

or neither one, but not both.

If for example t n

E

T, then tn'

T

and

Ix n (t',) n

+ x ,(t ,) n n

(x (t ,-D) + x

n

Ix ,Ct ,) - x ,(t ,-0)1 n n n n > (. -

n

n

,(t ,-0))1 n

[x Ct; ,) - x Ct; ,-0)1 n

n

n

n

(./2 = (./2

using the relations (7.4.3).

If t

n

,

E

=1

1

Then, Ix - x , - 0 ) 1 > (./4. "2x n + "2x n " 8(t n 8(t n,) 8 T, then t T and the same reasoning as above yields n

Define x

-x(t-O)I > (./4. 8 n - x Ct; -0) I > (./4. 8 n

If t

T and tn' T, then a fortiori n Thus, for any partition T with

181

= l, ... ,m,

and hence w' (Ci};:,; E/4. xCi and so sup w' (Cil ;:,; w' U) ;:,; E/4. xe:coOO x xCi

for some i

But x",

coOO,

u

Since E > 0 is fixed and Ci > 0 is arbitrary, this yields 0, and thus coCK} is not relatively compact

>

in D.

III For the "if" part we first prove the following lemma. If K is a relatively compact subset of D such

Lemma 7.4.2

that SECK} is finite for each E > 0, then to each t following holds:

[O,lJ the

for each E > 0 there exists 0 > 0 such that

sup W ([t,t+o}) < E and sup wx((t-o,t)) < E. x K x x K Proof:

Let

£

> 0 be given and fix to

compactness of K, find 0

0

>

By relative

0 such that supw'(o ) < E/3. X

K

°1 )

sup/x(t) - x(t-O) x K Take x

[O,lJ.

I

x

Now find

0

implies 0.4.4)

< £/3.

K and let T = {til be any finite partition of [O,lJ such

that max sup w ([t. l,t.)) < £/3 and min it. - t. l} i x K x ll i l lpoint of T falls in [to,t o + a point t i

°1 ) ,

If no

then wx([to,t o + 01)) < £/3. (to,t o +

T is such that t i

°0 '

°1 ) ,

If

then

max sup wx([ti_l,t < £/3 yields wx([to,t i)) < £/3 and i)) i x K wx([t i, to + at t

i,

°1 ) )

< £/3.

By Inequality (7.4.4), if x makes a jump

its magnitude is necessarily < E/3.

inequality, wx([to,t

o

Thus, by the triangle

+ all) < £/3 + £/3 + £/3 = £, for any x

Hence, sup wx([to,t + all) o xe:K

and the proof is complete.

£.

In a similar manner, a

°2

>

K.

0 can

III

182

Proof ("if" part): finite.

Let 8E:(K)

Let I k t

... ,tN}'

= Put t

N+l

=

Now, let E: > 0 be given and 8E:(K) be

=

for k kl

=tk

... ,N, where to

and corresponding to t

k

=

0 and

find, by Lemma

7.4.2, oLtkll > 0 such that 0.4.5)

For j I

kl

= [t k l,

Let t

kj manner:

1, inductively define t t

kl

+ oCt

and o(t

kl))

=

and I

'+1 = t + oCt and set k,) k,] k ,] ] Ct t + o(t if j f. 1. k j)) k j, k j

kj be determined alternately in the following

k j) given t ... ,t find by Lemma 7.4.2, oCt > 0, k j) k l, k j,

such that

and

Each t k j

t k+ l since some x

K

a jump at

In this way a sequence {t

... } of points in [tk,t k+ l] k l,tk 2, is obtained, and a sequence of intervals I ... which are all k l,Ik 2, open sets in [tk,t

is obtained. Another application of k+ l] Lemma 7.4.2 yields a oCt > 0 such that k+ l)

The collection of relatively open subintervals {I is an open cover of

which is compact.

there exists an open subcover {Jkl, ... ,J

kN

}.

k

denote the respective centers of these intervals.

k,Ikl,I k 2, ... }

Consequently,

183

Now this can be done for every k, k = 0,1, ... ,N. The coIN N N k lection of points U U {sk'} u U Lt . } forms a partition of J k=O j =1 j=l J [O,lJ, call it T, and denote the points of it in ascending order, by

The claim is now that max i=1, ... ,m-1

sup

w ([s. l's.)) < x ll

E

and

Let' x

L J=

lu, x., x , " K, c . '" 0 and l = 1. J J J J LJ= u,J

Then,

w

=

x

([s.

l-

l's.)) l

sup s,t" [ si_l,si )

:5 Ij=lUj

I

J=

lu.(x.(s) - x.(t))/ J J J

sup [x.(s)-x.(t))! s,t,,[s. l's.) J J l-

l

:5 InJ'=luJ' sup w ([s. l's.)) x eK x .Lrl :5

J

= E. =lC1..E J

Hence, (7.4.6) holds and thus sup xe co Ck) by taking 0
sand k

In these applications,

n is sufficient for the random elements to

be weakly uncorrelated.

While these applications for ceo,oo) will

not be discussed, it is important to note that the less restrictive coordinate uncorrelation suffices in applying the weak laws of large nwnbers.

188

The space of null convergent sequences, cO' is used for the next application.

Let {V } be a sequence of stochastic processes n

where each process has parameter space {1,2, .. . }.

Also, for each n

and k let the stochastic processes V and V have the same finitek n dimensional distributions and let lim V Un) = 0 with probability one for each n.

ill....'"

Finally, let EI Iv11 I

n

= E[sup ill

IV1Un)IJ < '" and for

each m let the weak law of large numbers hold for the random variables {VnCm):

n

I}.

From the results of Chapter II {Vn}

can be regarded as a sequence of random elements in

(EV 11) , EV ... ). 1 1(2),

Co

with EV1

=

Moreover, the random elements are identically

distributed since they have the same finite­dimensional distributions [from Property 2.3.4J.

Theorem 4.1.3 states that for any E > 0

(8.1.2) One consequence of (8.1.2) is the uniform convergence of to EV1Cm) in probability. In essence, the preceding paragraph gives a uniform weak law of large numbers for triangular arrays of random variables.

Let

1, m I} be a family of random variables satisfying n nm the following conditions: {V

lim V (8.1.3) nm = 0 with probability one for each n; m....'" (ii) for each m the weak law of large numbers holds for the (i)

random variables {V : nm (iii) for any n and any j

n

I};

(8.1.4)

1 the stochastic processes

{Vnm: m I} and {V(n+j)m: m dimensional distributions; and Civ)

I} have the same finite­ C8.1.5) C8.1.6)

E[sup IVlmlJ < "'. m 1

n

It follows from the above discussion that n Lk=l V .... EV in km lm

189

probability uniformly for m.

Thus,

= II

Vk - EV.i I I

-s-

0

in probability. Consider now the space I

1

.

Let {V (m):

m

n

1, n

I} be a

family of random variables satisfying the following conditions: (i)

I:=l

Elvl(m)!