U-Statistics in Banach Spaces 9067642002, 9789067642002

Citation preview

U-Statistics in Banach Spaces

U-STATISTICS IN B A N A C H SPACES

Yu.V. Borovskikh

///VSP/// Utrecht, The Netherlands, 1996

VSP BV P.O. Box 346 3700 AH Zeist The Netherlands

© VSP BV 1996 First published in 1996 ISBN 90-6764-200-2

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner.

CIP-DATA KONINKLIJKE BIBLIOTHEEK, DEN HAAG Borovskikh, Yuri V. U-statistics in Banach spaces / Yuri V. Borovskikh. Utrecht: VSP With index, ref. ISBN 90-6764-200-2 bound NUGI815 Subject headings: U-statistics / Banach spaces

Printed

in The Netherlands

by Koninklijke

Wöhrmann

BV,

Zutphen.

Managing Editors:

E.V. Cherkashin International Mathematical Center, Kiev, Ukraine O.A. Cherkashina Institute of Mathematics, Kiev, Ukraine

Language Editor:

J. Thomas University of Salford, Manchester, UK

To My grandson Anton

God, Thou great symmetry, Who put a biting lust in me From whence my sorrows spring, For all the frittered days That 1 have spent in shapeless ways Give me one perfect thing. Anna Wickham {"Envoi", 1921)

Contents Preface

xi

Introduction

1

1 Basic definitions

5

1.1

One-sample [/B-statistics

1.2

Multi-sample l/B-statistics

15

5

1.3

Von Mises' statistics

21

1.4

Banach-valued symmetric statistics

23

1.5

Permanent symmetric statistics

24

1.6

Multiple stochastic integrals

27

1.7

B-valued polynomial chaos

29

2 Inequalities

3

31

2.1

Inequalities based on the Hoeffding formula

31

2.2

Martingale moment inequalities

37

2.3

Maximal inequalities

42

2.4

Contraction and symmetrization inequalities

44

2.5

Decoupling inequalities

48

2.6

Hypercontractive method in moment inequalities

64

2.7

Moment inequalities in Banach spaces of type p

71

Law of large numbers

73

3.1

One-sample L/B-statistics

73

3.2

Multi-sample [7B-statistics

79

3.3

Von Mises' statistics

84

3.4

Estimates of convergence rates

86

4 Weak convergence 4.1

95

Central limit theorem

95

4.2

Convergence to a chaos

102

4.3

Multi-sample [/B-statistics

114 vii

Vili

5

6

7

8

Contents 4.4

Poisson approximation

124

4.5

Stable approximation

129

4.6

Approximation with increasing degrees

132

4.7

Symmetric statistics

146

4.8

[/-statistics with varying kernels

152

4.9

Weighted (/-statistics

160

Functional limit t h e o r e m s

165

5.1

Non-degenerate kernels

166

5.2

Degenerate kernels

170

5.3

Weak convergence to a chaos process

174

5.4

Weak convergence in the Poisson approximation scheme

182

5.5

Invariance principle for symmetric statistics

184

5.6

Functional limit theorems with varying kernels

187

5.7

Weak convergence of (7-processes

191

Approximation estimates

201

6.1

General methods of estimation

201

6.2

Rate of normal approximation of (/K-statistics

205

6.3

Estimates with increasing degree

215

6.4

Nonuniform estimates

224

6.5

Rate of chaos approximation

226

6.6

Normal approximation of [/H-statistics

237

6.7

Multi-sample L/H-statistics

277

6.8

Estimates in central limit theorem in

279

6.9

Rate of Poisson approximation

291

A s y m p t o t i c expansions

295

7.1

Expansions for non-degenerate i/R-statistics

296

7.2

General method of expansions

311

7.3

Expansions with canonical kernels

316

7.4

Expansions with arbitrary kernels

323

Large deviations

327

8.1

Exponential inequalities

327

8.2

Moderate deviations

335

8.3

Power zones of normal convergence

337

8.4

Probabilities of large deviations for l/H-statistics

339

Contents 9

ix

Law of iterated logarithm

349

9.1

i/ft-statistics

350

9.2

l/H-statistics

351

9.3

Bounded LIL

352

9.4

Compact LIL

357

9.5

Functional LIL

360

9.6

Multi-sample [/B-statistics

361

10 D e p e n d e n t variables

365

10.1 Symmetrically dependent random variables

365

10.2 Weakly dependent random variables

366

10.3 Bootstrap variables

369

10.4 Order statistics

375

Bibliographical s u p p l e m e n t s and c o m m e n t s

377

Bibliography

385

Index

419

Preface [/-statistics are universal objects of modern probabilistic summation theory. T h e y appear in various statistical problems and have very important applications.

The

mathematical nature of this class of random variables has a functional character, and therefore, leads to the investigation of probabilistic distributions in infinitedimensional spaces. T h e situation when the kernel of a [/-statistic takes values in a Banach space, turns out to be the most natural and interesting. In this case the construction of the probabilistic theory of such [/-statistics is based on methods of probability theory in B a n a c h spaces. T h e mathematical beauty and universal attractiveness of [/-statistics is clearly explained by the universal idea of s y m m e t r y lying in its definition. T h e idea of s y m m e t r y is dominant in modern probability theory in Banach spaces. T h a t is why [/-statistical objects are natural in all aspects of probability theory in Banach spaces.

In concentrated form this

idea is revealed in the representation of every [/-statistic as an integral functional with respect to a permanent random measure.

A s y m p t o t i c properties of random

permanents tell us that a [/-statistic is a statistical image of an element of chaos, and a f a m i l y of [/-statistics serves as a statistical model of chaos processes. In this book I present in a systematic form the probabilistic theory of [/-statistics with values in B a n a c h spaces (i/B-statistics). which has been developed up to the present time.

T h e exposition of the material in this book is based around the

following topics: - algebraic and martingale properties of [/-statistics; - inequalities; - law of large numbers; - the central limit theorem; - weak convergence to a Gaussian chaos and multiple stochastic integrals; - invariance principle and functional limit theorems; - estimates of the rate of weak convergence; - asymptotic expansion of distributions; - large deviations; - law of iterated logarithm; - dependent variables; xi

xu

Preface

- relation between Banach-valued (/-statistics and functionals from permanent random measures. All asymptotic properties of i/B-statistics depend on Banach space B, rank r, kernel $ and moment limitations for

For r = 1 (non-degenerate kernel) the

asymptotic behaviour of (7B-statistics can essentially be reduced to the asymptotic behaviour of sums of B-valued random elements. If r > 2 (degenerate kernel), then the family of possible limit distributions for (7B-statistics is extended by the distributions of functionals of infinite-dimensional Gaussian vectors defined by kernel

distributions of multiple stochastic integrals

and functionals of them, distributions of Gaussian chaos and Poisson chaos and so on. I hope that this book will serve to introduce many people to the beautiful theory of (/-statistics in Banach spaces. Many people have helped me in the writing of this book, either through discussions or by reading one of the versions of the manuscript. I am extremely grateful to V.S. Korolyuk for his continuous support in my efforts and his diligence in exploring stochastic laws of ¡7-statistics during the last twenty years. My deep gratitude is expressed to O.A. Cherkashina (Adamenko) (Institute of Mathematics, Kiev) and E.V. Cherkashin (International Mathematical Center, Kiev) for their activity in publishing mathematical manuscripts, and especially for their editorial work on several versions of this book. I would also like to thank Joseph Thomas (British) for his English language editing of the text.

Komarovo, near St

Petersburg,

Russia

8th May, 1995

Yuri V. Borovskikh

Introduction In this book we consider (/-statistics £/„ = n-H

£

,Xlm)

with Banach-valued kernels $ : Xm —> B. The exploration of the stochastic laws of random elements of this type is accompanied by all the problems which are inherent in the investigations of stochastic properties of sums of independent Banach-valued random vectors (this corresponds to the case m = 1). In Chapter 1 we present preliminary information which includes the definition of canonical functions gc; rank r of kernel

1 < r < m; canonical Hoeffding

representation; multiple stochastic integrals and Banach-valued polynomial chaos. Naturally, our treatment of these definitions is relevant to (7J3-statistics. Chapter 2 is devoted to inequalities. Here we have moment inequalities related to the geometry of Banach spaces and maximal inequalities following from martingale properties of l/B-statistics. Decoupling inequalities, consisting in the comparison play a very important role in what follows.

between £ $ ( | | i / n | | ) and Here [/ dec =

n

-H

£

is a decoupling version of Un, and random variables {X{,...

,Xl}™ = 1 are indepen-

dent copies of { X i , . . . , Xn}. Laws of large numbers for [/B-statistics and for Banach-valued von Mises' statistics are considered in Chapter 3. The estimates of the rate of convergence in these theorems are obtained by martingale inequalities. Chapter 4 is devoted to the study of the convergence in distribution of UBstatistics and the different functionals of them.

Conditions of the theorems are

formulated in terms of canonical functions gc, c = r , . . . ,m, on rank r, 1 < r < m.

and are dependent

In the non-degenerate case (r = 1) in limit we have

asymptotic normality. If kernel $ is degenerate (r > 2), then Gaussian chaos arises if the considered Banach space is of type 2. Under additional moment conditions for canonical functions gc these statements hold true for multi-sample [/B-statistics and symmetric statistics. We also consider weak convergence under the conditions 1

2

Introduction

of Poisson approximation, stable approximation and with increasing degrees. T h e results for [/-statistics with varying kernels and for weighted [/-statistics are adduced as well. Weak convergence of probability measures generated by [/-statistics in functional spaces is considered in C h a p t e r 5. R a n d o m [/-processes of a stochastic sequence { [ / n ( $ ) } can be defined in different ways. It is possible to consider a partial sum process similar to (i/[ n ( ]($), 0 < t < 1) as a sequence of processes indexed by p a r a m e t e r t when $ is fixed. Another interpretation of ( [ / „ ( $ ) , $ £ J 7 ) is a stochastic process indexed by family T consisting of measurable s y m m e t r i c kernels. In this chapter, for such [/-processes we study t h e weak convergence in non-degenerate and in degenerate cases, and in Poisson approximation schemes. Martingale functional limit theorems are applied to [/-statistics with varying kernels. In C h a p t e r 6, t h e rate of weak convergence is investigated, and various approximation estimates depending on a Banach space

a kernel $ and a distribution P

are obtained. In different situations we apply different m e t h o d s of analysis. For the estimation of t h e rate of Gaussian convergence we apply t h e characteristic function m e t h o d and t h e m e t h o d of compositions. In Poisson approximation cases we apply the C h e n - S t e i n m e t h o d .

Rank r of kernel $ has an effect upon t h e form of t h e

limit distribution. T h a t is why it is natural t h a t t h e rate of convergence depends on this rank r . We consider nonuniform estimates and estimates with increasing degree both in Gaussian approximation and in Poisson approximation schemes. T h e most interesting estimates are obtained for l/H-statistics. C h a p t e r 7 is devoted to t h e problem of t h e refinement of t h e limit theorems of t h e weak convergence of probability distributions generated by [/B-statistics. This problem is connected with a s y m p t o t i c expansion and clarification of t h e conditions for it to be true.

These conditions can depend on the m e t h o d of investigations.

T h e m e t h o d of characteristic functions d e m a n d s t h e well known C r a m e r condition and t h e conditions of t h e m o m e n t s of individual terms. In this problem, rank r of kernel $ has an essential effect. Depending on different values of r this problem has different solutions. But an asymptotic expansion can be written for every r and t h e coefficients of this expansion can be represented in t h e form of multiple stochastic integrals if kernel $ satisfies some additional conditions. In C h a p t e r 8 we consider t h e behaviour of probability P(nr'2\\Un-6\\ in t h e region x = 0(nf3)

>x)

with some 0 < ¡3 < 1. First we obtain some exponential

inequalities for all x > 0, and then we study t h e large deviations of this probability when x = x(n)

—* oo as n —> oo. T h e results depend on rank r of kernel $ and

t h e properties of a Banach space B. For [/H-statistics t h e estimations of t h e rate of convergence in t h e regions of large deviations are obtained.

Introduction

3

Chapter 9 is devoted to the law of iterated logarithm for (7B-statistics.

For

[/-statistics with values in Banach spaces of type 2, the validity of bounded LIL, compact LIL and functional LIL is proved. Corresponding theorems include conditions for canonical functions gc. Chapter 10 serves as an introduction to the theory of l/B-statistics when the random variables Xi,...

,Xn

are dependent.

Theoretically, for this class of ran-

dom 5 - v a l u e d elements, we can consider the whole spectrum of asymptotic and nonasymptotic problems which was considered for independent random variables. T h e author confines himself to the variables for which he has a personal preference: symmetrically dependent random variables, weakly dependent random variables, bootstrap variables and order statistics. T h e bibliographical supplements and comments contain remarks on the scientific works which were used during the writing of this book, and those which influenced the author. Additionally in the bibliography, the principal papers where the exploration of the stochastic laws of [/-statistics were actively conducted, can be found at the beginning with the appearance of the fundamental work of Hoeffding (1948a).

Chapter 1 Basic definitions 1.1

One-sample UB-statistics

Definition of UB-statistics. Let B be a real separable Banach space with a norm || • || and let B* be the dual to space B. Denote by x*(x) the value of functional x* £ B* at x € B. Let Xi,... ,Xn be independent random variables taking values in the measurable space (X, X) and all with identical distribution P. Consider a Bochner integrable symmetric function $ : Xm —• B of m variables given on Xm and taking values in B. We define a U-statistic as follows (1.1.1)

It is clear that Un £ B. Using terminology of Borovskikh (1986) [/-statistic (1.1.1) with a B-valued kernel $ is called a UB-statistic. In particular, if B = R it is called a UR-statistic and if B = H, where H is a real separable Hilbert space, it is called a UH-statistic. Let V = {P} be a class of probability distributions on ( X , X). By 6 : P —> 9(P), we denote a functional given on V and taking values in B where 9{P) = j . . . j $ ( x l 5 . . .

,xm)P{dXl)...P{dxm)

If £|||| < oo, then Un is an unbiased estimate of the 5-valued element 6{P) — E9{XU... ,Xm). In particular, if in (1.1.1), m = 1, then Un = n _1 ($(A"i) + • • • + $(A"„)) is the sum of independent identically distributed (i.i.d.) B-valued random variables. Therefore, [/-statistic (1.1.1) can be treated as a functional-algebraic generalization of usual mean. Without loss of generality, we suppose that $ is a symmetric function of its arguments, since otherwise we can use a symmetric kernel 5 m $ according to the 5

Basic

6

definitions

formula Sm$(xi,...

,xm) = — ,x,J m! where the summation is carried out over all m! permutations ( ¿ 1 , . . . ,im)

of the

numbers ( 1 , . . . , m). Then [/-statistic (1-1.1) is defined as follows £

£/„=n-H

(Xn,... ,Xlm),

(1.1.2)

where rJ" 1 ' = n(n — 1) • • • (n — m + 1). Note, that for all m < n, we have CT

l< ll7 i...^, m n.

Consequently, for

different values of parameter m we have 1)

if m = 1,

then i/n = rT1

2)

i f m = 2,

then Un = (§)

n-1)

n)

if m = n — 1,

i f m = n,

then Un = n

+ • • • + $(Xn))

X3),

Ei : (Ai,...

, Xn), Xn+i,

Xn+2,...

}.

It is clear that 25n D 25 n + i for all n = 1 , 2 , . . . . For the stochastic sequence {Un, ®n). Lemma 1.1.3 holds. E x a m p l e 1 . 1 . 3 . Let X = R and Xni

< Xn2

corresponding to random variables Xi,...

Xn.

®n = a{w : Xni,... the cr-algebra generated by Xni,... It is clear that 5Bn D Un

< ...

< Xnn

be ordered statistics

Denote by

,Xnn,Xn+i,Xn+2,

, Xnn, Xn+i,

• • •}

Xn+2,

•..

.

for every n > 1. Therefore =

E(*(Xil,...,Xim)

=

E(*(Xiit...

|®n) ,Xlm)\Xnl,...

,Xnn)

for all 1 < ¿i < . . . < im < k and m < k < n.

H o e f f d i n g f o r m u l a . Let k = [n/m] be the integral part of the number n / m . We set tpn(xi,...

,xn)

=

k~1(^(x1,...

,xm)

+

+ $ ( z m + l , . . . , 22m) H Let and let

h $(z/fcm-m+l, • • •

,Xkm)).

be the sum over all nl permutations ( ¿ i , . . . ,in) of the numbers ( 1 , . . . , n ) denote the sum over all

combinations ( ¿ i , . . . ,im)

constructed from

the numbers ( 1 , . . . , n). Then £ 1, j = 1 , . . . , c, and taking its values in Banach space B. A multi-sample f /

n = n ( n j ) \mi/ ]=\

UB-statistic

is defined as follows

a = i j u . . . ,ijmj,

J = 1,... ,c),

(1.2.1)

where n = ( n j , . . . , n c ) and the summation is carried out over all 1 < iji < . . . < ijm,


o.

32

Inequalities T h e integrability of Sn has been studied by a number of authors.

T h e most

general result in that direction is the following inequality obtained by T a l a g r a n d (1989): ||5„||,
1.

E\\Un-0\\p 2, where one can choose a„ = ( l 8 \ / 2 ) V -

Note that if B — R, then the module of smoothness pit) is equal to max(0, t — 1). Therefore, p(t) = 0(t2),

t —• oo, and we can assume that the space R is 2-uniformly

smooth. Thus, for the case B = R Corollary 2.2.2 also holds. Corollary 2.2.3. Assume that Banach space B is 2-uniformly

smooth, r = m, and

< oo for some q > 1. Then

for 1 < q < 2, and / \ -g+i E \ \ u

n

\ \ < < a ™ r )

n

2

^ -

2

^

E \ m

q

for q> 2.

Inequalities for multi-sample lIB-statistics. Consider the random element 5 fic j in (1.2.9). smooth and £||jf1j||'7 < oo for

T h e o r e m 2.2.3. Let Banach space B be p-uniformly q > 1. Then the inequality E\\S

h S

\r d, d > r where d = d\ + • • • + dc, r = r^ + • • • + rc and r is the rank vector in (1.2.7); aq is the constant in (2.2.1). Proof. We can write Snd=

in (1.2.9) as follows

'"' 9dix]t» 1 1. Then the inequality £ 1 1 « <
1, and let {a^}, k > 1, be a nonincreasing sequence of nonnegative numbers.

Then

P( max a t ||5 f c c || > t) < r ' { < £ | | S n c | | ' + c n For the reverse martingale (Un, ®„), n = r, r + 1 , . . . , the following assertions are valid, which are similar to Theorem 2.3.2 and Corollary 2.3.2. T h e o r e m 2 . 3 . 3 . Assume nondecreasing

sequence

that

< oo for q > I, and that {&&}, k > 1, is a

of positive numbers.

Then

P( m a x U k - 9 \ \ > t) < n m.

T h e o r e m 2.3.4. The following inequalities are valid: P( sup \\U-k - 0|| > t) < ( " V i k>n

+ Pc-MUn

- 0|| (ln+||£/n - A l l ) ' - 1 ) }

for any t > 0 and £(sup \\u-k - 0||?) < k>n

lcqE\\Un

- 6\\\

q>

1

where In+z = m a x ( l , l n 2 ) , 2 > 0;

A-fe)'.

^ ( ^ r

P r o o f . By Lemma 1.2.2 for n > k

E{Un - 0|®s) = u-k-e. Hence, we obtain

for any q > 1. Thus, the stochastic sequence verse submartingale with multivariate index k.

—

is an i?-valued re-

Therefore, the conclusion of the

present theorem comes from the known inequalities for such submartingales (see, e.g. Fazekas (1983, 1985)).

2.4

Contraction and symmetrization inequalities

Contraction and symmetrization inequalities provide a comparison between moments (and tails) of sums

Xi,

an 0,

P( max ||St|| > t) < 2P(||5„|| > t).

Contraction

and symmetrization

inequalities

45

The proof of Levy's inequalities are in many papers (see, for example, Araujo and Gine (1980), Theorem 2.6). L e m m a 2.4.2. Let ty: B

R be a convex function

Y and EY = 0 in the sense of Bochner's EV(X) In

on B. If X is independent

integration,

< EV{X

of

then

+ Y).

particular, £11X11" < E\\X + Y\\P,

p> 1.

P r o o f . For every x € B we have x = E(x + Y) and \t(x) = ty(E(x + Y))

[by Jensen's inequality]

< EV{x + Y). Then apply Fubini's theorem.

The following inequality is an obvious and simple consequence of Lemma 2.4.1 and Lemma 2.4.2. L e m m a 2.4.3. Let X\,... elements.

,Xn

be independent

zero-mean

Let ^ ( z ) , x > 0, be a convex increasing function.

random

Banach-valued

Then

Proof. Let { X / } be an independent copy of {A";}. Then {Xi — X / } is a sequence of independent symmetric random elements. Further, [by Lemma 2.4.2] [by Lemma 2.4.1] [because of convexity] t=i n

< 2

i=1 t=l

n

46

Inequalities

L e m m a 2.4.4.

Let Xi,...

,Xn

be independent

with values in B. Then, for each sequence a\,... P

2P

(|X>.-*||

and symmetric

random

variables

,an 6 R, and any t > 0,

(,^f) +

X}) + SiiWiXj)

X,-) + + t3(X}:x)

[by formula (2.5.6) on the first term] = E*(sup||4 -

y

£

E^EHiAX^X])-

E ^ ^ x i x , ) -

Y

Y

Exl^3(Xt,X^)-

¿^«m*/,*;)!)
n m / p ) +

Hence, if condition (3.1.1) takes place, then n m P ( | | $ | | >nm/p)^

0

oo J P ( | | $ | | > t) dt.

One-sample

and

75

UB-statistics

oo nm(p-l)/p

J

p(\\Q\\>t)dt->0

n">/p

as n —y oo. If we choose n = [t\plm for t > 1 where [t)dt

=

o o

n

for 1 < p < 2. Then

m ( 2 - p ) / ( 2 p)

J

„m/p

¿P(||$|| >

+

j

tP(||$||>i) 0

valid:

as n - > oo .

P r o o f . T h e case p = 1 is contained in Theorem 3.1.1. We assume in the following that 1 < p < 2. Using formula (3.1.4), we write the following inequality E\\n-m'pSn\\p

< 2p-1£||n-m/p5n($1)||p

+

2p-1£||n-m^5n($1)||p.

B y Corollary 2.7.1 i5||n-'»/P5n($1)r
ml

r

P

nm/p),

i.e. £||n-m/p5n($1)||''0

as


R. It is natural to consider a two-parameter convergence in the strong law of large numbers for Unin2,

i.e. convergence as n\ A n? tends to infinity,

where ni A n2 is the minimum of n\ and n 2 . In (3.1.5) we assume that X i , X 2 , . . . are independent random variables with a common uniform distribution on (0,1). Clearly, EUnin2 where

— 0,

l I 6

" I I 0 0

®(x'y)dxdy-

However, if m i n ( n i , n 2 ) —> oo, then the integrability of $ does not guarantee the validity of the strong law of large numbers. This fact is reflected in the following theorem. Denote

WMTSWFW where l n + z = l n ( m a x ( l , z)).

(3 I6)

-

Multi-sample T h e o r e m 3.1.4. Let

E\$(XI,X

2

)\
nij,

i , j

-

= 1 , . . . , c},

(3.2.2)

0 < A < 1. Denote by a vector q — (qi, ••• ,qc) with components 1 < qi < 2,

i = 1 , . . . , c.

Let f = ( r i , . . . , rc) be a vector of ranks and a vector d = ( d j , . . . , dc) from (1.2.7). Let r = ri H

b rc, d = di H

7 f J = d/(d-r i.e.

E,c=iiri/

n

-

J=i

0

as

n

x

^

oo .

By Corollary 2.7.1 E\\Sni(g})\\2

< ft j^mAY^r'nfEWgA'lilM 3 =1