Markov Processes and Control Theory [Reprint 2022 ed.] 9783112620243, 9783112620236

Citation preview

Mathematica/ Research Markov Processes and Control Theory

H. Langer V. Nollau Volume

54

AKADEMIE-VERLAG

BERLIN

In this series original contributions of mathematical research in all fields are contained, such as — research monographs — collections of papers to a single topic — reports on congresses of exceptional interest for mathematical research. This series is aimed at promoting quick information and communication between mathematicians of the various special branches.

In diese Reihe werden Originalbeiträge zu allen Gebieten der mathematischen Forschung aufgenommen wie — Forschungsmonographien — Sammlungen von Arbeiten zu einem speziellen Thema — Berichte von Tagungen, die für die mathematische Forschung besonders aktuell sind. Die Reihe soll die schnelle Information und gute Kommunikation zwischen den Mathematikern der verschiedenen Fachgebiete fördern.

Manuscripts in English and German comprising at least 100 pages and not more than 500 pages can be admitted to this series. With respect to a quick publication the manuscripts are reproduced photomechanically. Authors w h o are interested in this series please turn directly to the 'Akademie-Verlag'. Here you will get more detailed information about the form of the manuscripts and the modalities of publication.

Manuskripte in englischer und deutscher Sprache, die mindestens 100 Seiten und nicht mehr als 500 Seiten umfassen, können in diese Reihe aufgenommen werden. Im Interesse einer schnellen Publikation werden die Manuskripte auf fotomechanischem Weg reproduziert. Autoren, die an der Veröffentlichung entsprechender Arbeiten in dieser Reihe interessiert sind, wenden sich bitte direkt an den Akademie-Verlag. Sie erhalten dort genauere Informationen über die Gestaltung der Manuskripte und die Modalitäten der Veröffentlichung.

Markov Processes and Control Theory

Mathematical Research

• Mathematische Forschung

Wissenschaftliche Beitrage herausgegeben von der Akademie der Wissenschaften der DDR Karl-Weierstraß-Institut für Mathematik

Band 54 Markov Processes a n d Control Theory

Markov Processes and Control Theory

edited by Heinz Langer and Volker Nollau

Akademie-Verlag Berlin 1989

Herausgeben Prof« Dr. Heinz Langer Doz. Dr. Volker Technische Sektion

Nollait

Universität

Die Titel

dieser

Autoren

reproduziert.

ISBN ISSN

Dresden

Mathematik

Schriftenreihe

werden vom Originalmanuskript

der

3-05-500754-9 0138-3019

Erschienen im Akademie-Verlag Berlin,Leipziger Straße 3-4,Berlin,DDR-io86 (c) Akademie-Verlag Berlin 1989 Lizenz»ummer: 202-100/414/89 Printed in the German Democratic Republic Gesamtherstellung: VEB Kongreß- und Werbedruck, Oberlungwitz, DDR-9273 Lektor: Dr. Reinhard Höppner LSV 1075 Bestellnummer: 764 009 7 (2182/54) 03200

FOREWORD

This volume oontains the major part of the proceedings of the Symposium ISAM 88: "Markov Prooesses and Control Theory" held in Gaufilg near Dresden, GDR, January 10 - 15, 1988. This Symposium was organized by the Department of Mathematics of the Technical University Dresden in collaboration with the "Forschungsrichtung

Markovsche

Prozesse und Prozeßsteuerung".

The lectures presented at the Symposium reviewed recent advances of some aspects of - stochastic differential equations and their applications, - semigroups, Markov processes and diffusions, - statistics of stochastic processes, - stochastic dynamic programming.

He thank the participants of the Symposium for their contributions and lively discussions, Dr. Wilfried Schenk, Mrs. Andrea König and Mrs. Margit Schönherr for their help in preparing this volume and, last but not least, Dr. Reinhard Höppner from the Akademie-Verlag for pleasant collaboration.

Dresden, February 1989

The Editors

TABLE OP CONTENTS

R. Buckdahn: A linear stochastic differential equation with Skorohod integral

9

J.van Casteren: On generalized Schrbdinger semigroups

16

H.-M. Dietz: Representations for solutions of linear functional stochastic differential equations

46

B. Fritzsche, B. Kirstein: On problems with incomplete covariance information

55

W. Grecksch: A prediction problem for a time discrete parabolic Ito equation

65

G. Hognas: Invariant measures and random walks on the semigroup of matrices

77

T. Koski: On Deltamodulators for Gauss-Markov processes: A normal approximation

85

U. Ktichler: A limit theorem for the excursion of quasidiffusions straddling t

100

H. Langer, W. Schenk: Time reversal of transient gap diffusions Yu.N. Lin kovs Asymptotical methods of statistics for semimartingales

115

W. Lipfert: Ober ein stochastisches Entscheidungsmodell mit allgemeinen vektorwertigen Ertragsfunktionalen

140

V. Nollau: Some aspects of vector-valued stochastic dynamic programming

149

S. Fohlenz: Die Verbesserung von Strategien beim mehrarmigen Banditen

158

7

U. RBslen lbs variation diminishing property applied to diffusions

164

M. 3chill: On stoohastlo dynamic programming:. A bridge between Markov deoision processes and gambling

178

B. Schmalfuss: Invariant attracting sets of nonlinear stochastic differential equations

217

H. Sjfrensen: A note on the existence of a consistent maximum likelihood estimator for diffusions with jumps .

229

8

A LINEAR STOCHASTIC DIFFERENTIAL EQUATION WITH SKOROHOD INTEGRAL R. Buokdahn

Abstract: We study a linear atoohaatle differential equation with a random initial oondition and a drift anticipating the future of the driving Wiener process.

1. Introduction Let ^ W(t)}

denote a Brownian motion. Reoently, progress has been made

in developing a useful theory of stochastio Integrals t s Nf(s,o)W(ds) 0 in which the integrand ^ (s,o) anticipates ^ W(s)^. For suoh an Integral constructed by Skorohod in £ 6 ] an extended stoohastio oaloulus on the basis of the Malliavin calculus has been developed by Gaveau-Trauber f and Nualart-Pardoux

. This allows one to formulate a linear stoohastio

differential equation (1)

Z(t) -

t t n + J C >

- E = E

12

| b(s,Ts)l > C } X(t,T t )~ 1 ] = lb(s)|>c} X(s)X(t,Tt)~1

0.

Thus, lb(a,U •»».) | ds ¿ C ,

dtdP-a.e. 18

Analogously, It can be seen that the prooess

essentially

bounded. Henoe, Z is a produot of a process from

and

xeL 2 (C0,l"l x Q ) , i.e., Z is an element of Lg(£o,11 xfl). Thus, it remains to show that Z satisfies (4). The applioation of (5) provides for F 0), then Pv(t), t > 0, maps L'(E,m) oo.

if s u p { p 0 ( < , s , y ) = into L*(E,m), for

(d) In L*(E, m) the family { J V ( ' ) : i > 0} is a self-adjoint positivity preserving strongly continuous semigroup with a self-adjoint generator. Remark 1. Strictly speaking in order to prove that the generator A of the semigroup {JV(i) : t > 0} extends the operator At — V the following condition is used: u m «10

8 U

p ^ i > = 0 . t

U P,{C = oo} = 1, or equivalently, if ||Po(')lloo,oo = 1 for all / > 0, then this condition is automatically verified. Remark 2. If in (c) we assume that, for t > 0, the operator Po(t) maps ^(E^m) in Ca(E), then we may prove that, always for t > 0, the operator Pv(i) maps LP(E, m) in Lq(E, m) (~) C0(E), provided that 1 < p < q < oo, p ^ oo. This will be explained in another paper. Outline of a proof. Let (X(t),Pm) be the strong Markov process associated to the semigroup {Po(t) : ' > 0}; i.e. suppose [Po(*)/l (*) = Ex (f (X(t))) = J f (X(t))dP„

t>o,xeE,fe

C0(E).

Define the semigroup {iV(i) : t > 0} by the Feynman-Kac formula: [JV(0/] 0 0 = Et (exp

£

V (*(,))

dS) f ( X ( 0 ) : C > < ) , / € C0(E),x

eE,t>0

Here £ is the life time of the process: C = inf{« > 0 : X(*) = A } . Define the function pv(i>x>v) by: pv(t, x,y) = Km E* (exp

J^ V (X(e)) daj po(t - s,X(s),y)

:(>jj,(>0,i,i€£

The various assertions in Theorem 1.3. have to be verified. The semigroup {Py(t) : i > 0} is approximated by semigroups {P*,/, m (0 : i > 0} defined by [ J W 0 / K * ) = Ez (exp ( - J * V„AX(»))

.

Here - o o < k < t < oo, V M = max(min(V,£),fc), E = U*"m, Km is compact, Km C int (Xm+i) and Tm is the first exit time from int(ATm): Tm = inf {« > 0 :

€ E* \ int (# T O )} .

The integral kernel py(t,x, y) is approximated by the integral kernels of the semigroups {Pt./.mO : < > 0}. In fact we have

where p*,/ >m (t,®,y) is given by Pk.t,m(i,x,y) = Km£ x (exp

j f V M (JT( s ) .

Here pt,m(t,

v) » defined by Po,m( 0 and y 6 IR". It is useful to observe that the following processes have the same finite dimensional Pz -distributions:

+

o n(< - e - t ) f ] (x)ds

= [i'o.m(e) £

' Po.m(') lV.Sm,n(t - e - s)f ] d*

= [ft,m(£)Sm.»+l ( < - * ) / ] ( * ) • Fix / in b£. Since the function S r a ,„ + i(( — e)f is bounded and measurable, the function * >- [Po,m(e)Sm,n+i(< - e)/] (a) is continuous on iat(Km).

(Notice that functions of the form

[¿VmMs] (*) = Ex (g(X(t)):

Tm > 0)

are continuous on int(Jifm), provided the function g is measurable and bounded on Km). Moreover we have I [S m , n+1 (0 (s) [V.5m,n(< - s)f] (*) t > Tm)

< JS. (exp (2 J\-{X{s))d»^

. (Ex ( | / W « ) ) | 4 ) ) * .P«(C > t >

Tm)l.

Let A' be a compact subset of E. Since the function x >-• PX(C > t > Tm) is continuous on intfifm), Dini's lemma may be applied to conclude that m

lim sup PmU >t> Tm) = 0.

-°°

zEK

Consequently the function x [iV(.||/|||. Hence ||lifP-v.,m(t)||i,oo < Mexp(at) sup{po ( t / 2 , x , y ) : t , y £ By what we proved above we see that the functions (t, x)

24

[JV w , m (.})

( e x p ( 2 j T V_(A-(_jv. (< - a, z, y)dzj X(Tm),y)

Let

K

: Tm < . } ) ) *

(martingale property of the process Po>(< —