Applied Stochastic Control of Jump Diffusions [3rd edition] 3540698256, 9783540698258, 9783540140238, 9783030027797, 3030027791

Citation preview

Second Edition

Universitext

Bernt Øksendal Agnès Sulem

Applied Stochastic Control of Jump Diffusions

 

Bernt Øksendal · Agn`es Sulem

Applied Stochastic Control of Jump Diffusions 2nd Edition With  Figures

Bernt Øksendal University of Oslo Department of Mathematics  Oslo Norway e-mail: [email protected]

Angn`es Sulem INRIA Rocquencourt Domaine de Voluceau  Le Chesnay CX France e-mail: [email protected]

Mathematics Subject Classification (): E, G, G, L, MXX, J, J, B

Library of Congress Control Number: 

ISBN-10: --- Springer Berlin Heidelberg New York ISBN-13: ---- Springer Berlin Heidelberg New York ISBN-13: ---- 1st ed. Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September , , in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com ◦c Springer-Verlag Berlin Heidelberg , 

The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover design: WMXDesign, Heidelberg Typesetting by the authors and SPi using a Springer LATEX macro package Printed on acid-free paper SPIN: 

//SPi      

To my family Eva, Elise, Anders, and Karina B. Ø.

A tous ceux qui m’accompagnent A. S.

Preface to the Second Edition

In this second edition, we have added a chapter on optimal control of random jump fields (solutions of stochastic partial differential equations) and partial information control (Chap. 10). We have also added a section on optimal stopping with delayed information (Sect. 2.3). It has always been our intention to give a contemporary presentation of applied stochastic control, and we hope that the addition of these recent developments will contribute in this direction. We have also made a number of corrections and other improvements, many of them based on helpful comments from our readers. In particular, we would like to thank Andreas Kyprianou for his valuable communications. We are also grateful to (in alphabetical order) Knut Aase, Jean-Philippe Chancelier, Inga Eide, Emil Framnes, Arne-Christian Lund, Jose-Luis Menaldi, Tam´ as K. Papp, Atle Seierstad, and Jens Arne Sukkestad for pointing out errors and suggesting improvements. Our special thanks go to Martine Verneuille for her skillful typing.

Oslo and Paris, November 2006

Bernt Øksendal and Agn`es Sulem

Preface of the First Edition

Jump diffusions are solutions of stochastic differential equations driven by L´evy processes. Since a L´evy process η(t) can be written as a linear combination of t, a Brownian motion B(t) and a pure jump process, jump diffusions represent a natural and useful generalization of Itˆ o diffusions. They have received a lot of attention in the last years because of their many applications, particularly in economics. There exist today several excellent monographs on L´evy processes. However, very few of them – if any – discuss the optimal control, optimal stopping, and impulse control of the corresponding jump diffusions, which is the subject of this book. Moreover, our presentation differs from these books in that it emphasizes the applied aspect of the theory. Therefore, we focus mostly on useful verification theorems and we illustrate the use of the theory by giving examples and exercises throughout the text. Detailed solutions of some of the exercises are given at the end of the book. The exercises to which a solution is provided, are marked with an asterix ∗. It is our hope that this book will fill a gap in the literature and that it will be a useful text for students, researchers, and practitioners in stochastic analysis and its many applications. Although most of our results are motivated by examples in economics and finance, the results are general and can be applied in a wide variety of situations. To emphasize this, we have also included examples in biology and physics/engineering. This book is partially based on courses given at the Norwegian School of Economics and Business Administration (NHH) in Bergen, Norway, during the Spring semesters 2000 and 2002, at INSEA in Rabat, Morocco in September 2000, at Odense University in August 2001 and at ENSAE in Paris in February 2002.

Oslo and Paris, August 2004

Bernt Øksendal and Agn`es Sulem

X

Preface of the First Edition

Acknowledgments We are grateful to many people who in various ways have contributed to these lecture notes. In particular, we thank Knut Aase, Fred Espen Benth, Jean-Philippe Chancelier, Rama Cont, Hans Marius Eikseth, Nils Christian Framstad, Jørgen Haug, Monique Jeanblanc, Kenneth Karlsen, Arne-Christian Lund, Thilo Meyer-Brandis, Cloud Makasu, Sure Mataramvura, Peter Tankov, and Jan Ubøe for their valuable help. We also thank Francesca Biagini for useful comments and suggestions to the text and her detailed solutions of some of the exercises. We are grateful to Dina Haraldsson and Martine Verneuille for proficient typing and Eivind Brodal for his kind assistance. We acknowledge with gratitude the support by the French–Norwegian cooperation project Stochastic Control and Applications, Aur 99-050.

Oslo and Paris, August 2004

Bernt Øksendal and Agn`es Sulem

Contents

1

Stochastic Calculus with Jump Diffusions . . . . . . . . . . . . . . . . . 1.1 Basic Definitions and Results on L´evy Processes . . . . . . . . . . . 1.2 The Itˆ o Formula and Related Results . . . . . . . . . . . . . . . . . . . . . 1.3 L´evy Stochastic Differential Equations . . . . . . . . . . . . . . . . . . . 1.4 The Girsanov Theorem and Applications . . . . . . . . . . . . . . . . . . 1.5 Application to Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 6 10 12 19 21

2

Optimal Stopping of Jump Diffusions . . . . . . . . . . . . . . . . . . . . . 2.1 A General Formulation and a Verification Theorem . . . . . . . . 2.2 Applications and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Optimal Stopping with Delayed Information . . . . . . . . . . . . . . . 2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27 27 31 36 42

3

Stochastic Control of Jump Diffusions . . . . . . . . . . . . . . . . . . . . 3.1 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The Maximum Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Application to Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 45 52 57 61

4

Combined Optimal Stopping and Stochastic Control of Jump Diffusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 A General Mathematical Formulation . . . . . . . . . . . . . . . . . . . . 4.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 66 71 75

5

Singular Control for Jump Diffusions . . . . . . . . . . . . . . . . . . . . . 77 5.1 An Illustrating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.2 A General Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

XII

Contents

5.3 5.4

Application to Portfolio Optimization with Transaction Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6

Impulse Control of Jump Diffusions . . . . . . . . . . . . . . . . . . . . . . . 91 6.1 A General Formulation and a Verification Theorem . . . . . . . . 91 6.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

7

Approximating Impulse Control by Iterated Optimal Stopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 7.1 Iterative Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 7.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 7.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

8

Combined Stochastic Control and Impulse Control of Jump Diffusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 8.1 A Verification Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 8.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 8.3 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 8.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

9

Viscosity Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 9.1 Viscosity Solutions of Variational Inequalities . . . . . . . . . . . . . . 136 9.1.1 Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 9.2 The Value Function is Not Always C 1 . . . . . . . . . . . . . . . . . . . . 139 9.3 Viscosity Solutions of HJBQVI . . . . . . . . . . . . . . . . . . . . . . . . . . 142 9.4 Numerical Analysis of HJBQVI . . . . . . . . . . . . . . . . . . . . . . . . . . 153 9.4.1 Finite Difference Approximation . . . . . . . . . . . . . . . . . . 153 9.4.2 A Policy Iteration Algorithm for HJBQVI . . . . . . . . . . 156 9.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

10

Optimal Control of Random Jump Fields and Partial Information Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 10.1 A Motivating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 10.2 The Maximum Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 10.3 The Arrow Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 10.3.1 Return to Example 10.1 . . . . . . . . . . . . . . . . . . . . . . . . . 169 10.4 Controls Which do not Depend on x . . . . . . . . . . . . . . . . . . . . . 174 10.5 Connection with Partial Observation Control . . . . . . . . . . . . . . 176 10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

11

Solutions of Selected Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 11.1 Exercises of Chapter 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 11.2 Exercises of Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 11.3 Exercises of Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Contents

11.4 11.5 11.6 11.7 11.8 11.9 11.10

Exercises Exercises Exercises Exercises Exercises Exercises Exercises

of of of of of of of

Chapter Chapter Chapter Chapter Chapter Chapter Chapter

XIII

4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Notation and Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

1 Stochastic Calculus with Jump Diffusions

1.1 Basic Definitions and Results on L´ evy Processes In this chapter we present the basic concepts and results needed for the applied calculus of jump diffusions. Since there are several excellent books which give a detailed account of this basic theory, we will just briefly review it here and refer the reader to these books for more information. Definition 1.1. Let (Ω, F, {Ft }t≥0 , P ) be a filtered probability space. An Ft adapted process {η(t)}t≥0 = {ηt }t≥0 ⊂ R with η0 = 0 a.s. is called a L´evy process if ηt is continuous in probability and has stationary and independent increments. ag version (right Theorem 1.2. Let {ηt } be a L´evy process. Then ηt has a c`adl` continuous with left limits) which is also a L´evy process. ⊓ ⊔

Proof. See, e.g., [P, S].

In view of this result we will from now on assume that the L´evy processes we work with are c`adl` ag. The jump of ηt at t ≥ 0 is defined by ∆ηt = ηt − ηt− .

(1.1.1)

¯ does not contain Let B0 be the family of Borel sets U ⊂ R whose closure U 0. For U ∈ B0 we define  N (t, U ) = N (t, U, ω) = XU (∆ηs ). (1.1.2) s:0 0; ηt ∈ U }.

We claim that T1 (ω) > 0 a.s. To prove this note that by right continuity of paths we have lim+ η(t) = η(0) = 0 a.s. t→0

Therefore, for all ε > 0 there exists t(ε) > 0 such that |η(t)| < ε for all t < t(ε). This implies that η(t) ∈ U for all t < t(ε), if ε < dist(0, U ). Next define inductively Tn+1 (ω) = inf{t > Tn (ω); ∆ηt ∈ U }. Then by the above argument Tn+1 > Tn a.s. We claim that Tn → ∞

as

n → ∞, a.s.

Assume not. Then Tn → T < ∞. But then lim η(s)

s→T −

cannot exist,

contradicting the existence of left limits of the paths. It is well known that Brownian motion {B(t)}t≥0 has stationary and independent increments. Thus B(t) is a L´evy process. Another important example is the following. Example 1.4 (The Poisson Process). The Poisson process π(t) of intensity λ > 0 is a L´evy process taking values in N ∪ {0} and such that P [π(t) = n] =

(λt)n −λt e ; n!

n = 0, 1, 2, . . .

Theorem 1.5. [P, Theorem 1.35]. 1. The set function U → N (t, U, ω) defines a σ-finite measure on B0 for each fixed t, ω. The differential form of this measure is written N (t, dz). 2. The set function [a, b)×U → N (b, U, ω)−N (a, U, ω); [a, b) ⊂ [0, ∞), U ∈ B0 defines a σ-finite measure for each fixed ω. The differential form of this measure is written N (dt, dz). 3. The set function ν(U ) = E[N (1, U )], (1.1.3) where E = EP denotes expectation with respect to P , also defines a σ-finite measure on B0 , called the L´evy measure of {ηt }. 4. Fix U ∈ B0 . Then the process πU (t) := πU (t, ω) := N (t, U, ω) is a Poisson process of intensity λ = ν(U ).

1.1 Basic Definitions and Results on L´evy Processes

3

Example 1.6 (The Compound Poisson Process). Let X(n), n ∈ N be a sequence of i.i.d. random variables taking values in R with common distribution μX(1) = μX and let π(t) be a Poisson process of intensity λ, independent of all the X(n)s. The compound Poisson process Y (t) is defined by Y (t) = X(1) + · · · + X(π(t)),

t ≥ 0.

(1.1.4)

An increment of this process is given by π(s)

Y (s) − Y (t) =



X(k),

s > t.

k=π(t)+1

This is independent of X(1), . . . , X(π(t)), and its distribution depends only on the difference (s − t) and on the distribution of X(1). Thus Y (t) is a L´evy process. To find the L´evy measure ν of Y (t) note that if U ∈ B0 then ⎤ ⎡  ν(U ) = E[N (1, U )] = E ⎣ XU (∆Y (s))⎦ s;0 ε) < ε). Let Lucp denote the space of adapted c`agl` ad processes (left continuous with right limits), equipped with the ucp topology. If H(t) is a step function of the form  H(t) = H0 X{0} (t) + Hi X(Ti ,Ti+1 ] (t), i

where Hi ∈ FTi and 0 = T0 ≤ T1 ≤ · · · ≤ Tn+1 < ∞ are Ft -stopping times and X is c` adl` ag, we define  t  JX H(t) := Hs dXs := H0 X0 + Hi (XTi+1 ∧t − XTi ∧t ), t ≥ 0. 0

i

Theorem 1.13. [P, p. 51]. Let X be a semimartingale. Then the mapping JX can be extended to a continuous linear map JX : Lucp → Ducp . This construction allows us to define stochastic integrals of the form  t H(s)dηs 0

for all H ∈ Lucp . (See also Remark 1.18.) In view of the decomposition (1.1.6) (ds, dz), this integral can be split into integrals with respect to ds, dB(s), N and N (ds, dz). This makes it natural to consider the more general stochastic integrals of the form  t  t  t ¯ (ds, dz), γ(s, z, ω)N β(s, ω)dB(s) + α(s, ω)ds + X(t) = X(0) + 0

0

0

R

(1.1.11) where the integrands are satisfying the appropriate conditions for the integrals to exist and we for simplicity have put

¯ (ds, dz) = N (ds, dz) − ν(dz)ds if |z| < R N N (ds, dz) if |z| ≥ R, with R as in Theorem 1.7. As is customary we will use the following shorthand differential notation for processes X(t) satisfying (1.1.11):  ¯ (dt, dz). dX(t) = α(t)dt + β(t)dB(t) + γ(t, z)N (1.1.12) R

We call such processes Itˆ o–L´evy processes.

6

1 Stochastic Calculus with Jump Diffusions

Recall that a semimartingale M (t) is called a local martingale up to time T (with respect to P ) if there exists an increasing sequence of Ft -stopping times τn such that limn→∞ τn = T a.s. and M (t ∧ τn ) is a martingale with respect to P for all n. Note that 1. If



E

T

0



2

R



γ (t, z)ν(dz)dt < ∞,

(1.1.13)

then the process M (t) := is a martingale. 2. If



0

 t T

˜ (dt, dz), γ(t, z)N

0≤t≤T

0

R



γ 2 (t, z)ν(dz)dt < ∞ a.s.,

R

(1.1.14)

then M (t) is a local martingale, 0 ≤ t ≤ T .

1.2 The Itˆ o Formula and Related Results We now come to the important Itˆ o formula for Itˆ o–L´evy processes: If X(t) is given by (1.1.12) and f : R2 → R is a C 2 function, is the process Y (t) := f (t, X(t)) again an Itˆ o–L´evy process and if so, how do we represent it in the form (1.1.12)? If we argue heuristically and use our knowledge of the classical Itˆ o formula it is easy to guess what the answer is: Let X (c) (t) be the continuous part of X(t), i.e., X (c) (t) is obtained by removing the jumps from X(t). Then an increment in Y (t) stems from an increment in X (c) (t) plus the jumps (coming from N (·, ·)). Hence in view of the classical Itˆo formula we would guess that dY (t) =

∂f 1 ∂2f ∂f (t, X(t))dt + (t, X(t))dX (c) (t) + (t, X(t)) · β 2 (t)dt ∂t ∂x 2 ∂x2

+

R

{f (t, X(t− ) + γ(t, z)) − f (t, X(t− ))}N (dt, dz).

It can be proved that our guess is correct. Since

  (c) dX (t) = α(t) − γ(t, z)ν(dz) dt + β(t)dB(t), |z| 0] > 0.

(1.5.8)

Does this market have an arbitrage? To answer this we combine (1.5.5) and (1.5.6) to get φ0 (t) = e−rt (V φ (t) − φ1 (t)S1 (t)) and

φ

φ

−









dV (t) = rV (t)dt + φ1 (t)S1 (t ) (μ − r)dt + γ From this we obtain −rt

d(e

φ

−rt

V (t)) = e

or e−rt V φ (t) = V φ (0) +



0

t

−

φ1 (t)S1 (t ) (μ − r)dt + γ

R

R

 z N (dt, dz) .

 z N (dt, dz)

   (ds, dz) . e−rs φ1 (s)S1 (s− ) (μ − r)ds + γ zN R

Therefore e−rt V φ (t) is a lower bounded local martingale, and hence a supermartingale, with respect to Q. But then 0 = EQ [V φ (0)] ≥ EQ [e−rT V φ (T )], which shows that (1.5.8) cannot hold. We conclude that there is no arbitrage in this market (if (1.5.1) holds). This example illustrates the First Fundamental Theorem of Asset Pricing, which states the connection between (1) the existence of an ELMM and (2) the nonexistence of arbitrage, or No Free Lunch with Vanishing Risk (NFLVR) to be more precise. See, e.g., [DS, LS].

1.6 Exercises Exercise* 1.1. Suppose dX(t) = α dt + σ dB(t) +



R

¯ (dt, dz), γ(z)N

X(0) = x ∈ R,

where α, σ are constants, γ : R → R is a given function.

22

1 Stochastic Calculus with Jump Diffusions

1. Use Itˆo’s formula to find dY (t) when Y (t) = exp(X(t)). 2. How do we choose α, σ, and γ(z) if we want Y (t) to solve the SDE    ¯ (dt, dz) , dY (t) = Y (t− ) β dt + θ dB(t) + λ z N R

for given constants, β, θ, and λ? Exercise* 1.2. Solve the following L´evy SDEs:  ¯ (dt, dz), 1. dX(t) = (m − X(t))dt + σ dB(t) + γ z N R

X(0) = x ∈ R

(m, σ, γ constants) (the mean-reverting L´evy–Ornstein–Uhlenbeck process)  ¯ (dt, dz), X(0) = x ∈ R 2. dX(t) = α dt + γ X(t− ) z N R

(α, γ constants, γ z > −1 a.s. ν). [ Hint: Try to multiply the equation by

   t ¯ θ(z)N (dt, dz) + F (t) := exp − 0

|z| 0, η(t) ∈ K} ≤ ∞ and put τk = τ ∧ k,

k = 1, 2, . . .

2 n Let {fm }∞ m=1 be a sequence of functions in C (R ) such that 2 2 fm (x) = |x| for |x| ≤ R, 0 ≤ fm (x)  ≤ 2R for all x ∈ R, 1 for all m and supp fm ⊂ x ∈ Rn , |x| ≤ R + m fm (x) → |x|2 · χK (x) as m → ∞, for all x ∈ Rn . Use the Dynkin formula I to show that

E a [fm (η(τk ))] = |a|2 + ρ(n) · E a [τk ]

for all m, k.

24

1 Stochastic Calculus with Jump Diffusions

(c) Show that E a [τ ] =

( ( 1 ' 2 1 ' 2 a R P [η(τ ) ∈ K] − |a|2 ≤ R − |a|2 . ρ(n) ρ(n)

In particular, τ < ∞, a.s.

Remark. If we replace η(·) by an n-dimensional Brownian motion B(·), then the corresponding exit time τ (B) satisfies * ) ( 1' 2 E a τ (B) = R − |a|2 n (see, e.g., [Ø1, Example 7.4.2]).

Exercise* 1.6. Show that, under some conditions on γ(s, z) (deterministic),    t    t  (ds, dz) {eγ(s,z) − 1 − γ(s, z)}ν(dz)ds . γ(s, z)N = exp E exp 0

0

R

R

Exercise* 1.7. Let

dXi (t) =



R

(dt, dz), γi (t, z)N

i = 1, 2

be two one-dimensional Itˆo–L´evy processes. Use the two-dimensional Itˆo formula (Theorem 1.16) to prove the following integration by parts formula:  t  t X2 (s− )dX1 (s) X1 (s− )dX2 (s) + X1 (t)X2 (t) = X1 (0)X2 (0) + 0 0  t γ1 (s, z)γ2 (s, z)N (ds, dz). (1.6.1) + 0

R

Remark. The process  t γ1 (s, z)γ2 (s, z)N (ds, dz) [X1 , X2 ](t) := 0 R  t  t (ds, dz) γ1 (s, z)γ2 (s, z)N γ1 (s, z)γ2 (s, z)ν(dz)ds + = 0

R

0

R

(1.6.2)

is called the quadratic covariation of X1 and X2 . See Definition 1.28. Exercise* 1.8. Consider the following market (Bond price) dS0 (t) = 0; S0 (0) = 0 (Stock price 1) dS1 (t) = S1 (t− )[μ1 dt + γ11 dη1 (t) + γ12 dη2 (t)]; S1 (0) = x1 > 0 (Stock price 2) dS2 (t) = S2 (t− )[μ2 dt + γ21 dη1 (t) + γ22 dη2 (t)]; S2 (0) = x2 > 0

1.6 Exercises

25

where μi and γij are constants and η1 (t), η2 (t) are independent L´evy martingales of the form  i (dt, dz), i = 1, 2. zN dηi (t) = R

Assume that the matrix γ := [γij ]1≤i,j≤2 ∈ R2 is invertible, with inverse γ −1 = λ = [λij ]1≤i,j≤2

and assume that 

R

|z|νi (dz) > |λi1 μ1 + λi2 μ2 | for i = 1, 2.

(1.6.3)

Find an ELMM Q for (S1 (t), S2 (t)) and use this to deduce that there is no arbitrage in this market. Exercise* 1.9. Define, with suitable conditions on θ(s, z),  t  (ds, dz) Z(t) = exp ln(1 − θ(s, z))N 0

+

R

 t 0

R

 {ln(1 − θ(s, z)) + θ(s, z)}ν(dz)ds .

Show that dZ(t) = −Z(t− )



R

(ds, dz). θ(s, z)N

In particular, Z(t) is a local martingale.

2 Optimal Stopping of Jump Diffusions

2.1 A General Formulation and a Verification Theorem Fix an open set S ⊂ Rk (the solvency region) and let Y (t) be a jump diffusion in Rk given by  ¯ (dt, dz), Y (0) = y ∈ Rk , dY (t) = b(Y (t))dt+σ(Y (t))dB(t)+ γ(Y (t− ), z)N Rk

k

k

k

k×m

, and γ : Rk × Rk → Rk×ℓ are given where b : R → R , σ : R → R functions such that a unique solution Y (t) exists (see Theorem 1.19). Let τS = τS (y, ω) = inf{t > 0; Y (t) ∈ S}

(2.1.1)

be the bankruptcy time and let T denote the set of all stopping times τ ≤ τS . The results below remain valid, with the natural modifications, if we allow S to be any Borel set such that S ⊂ S 0 where S 0 denotes the interior of S, S 0 its closure. Let f : Rk → R and g : Rk → R be continuous functions satisfying the conditions   τS Ey f − (Y (t))dt < ∞ for all y ∈ Rk . (2.1.2) 0

The family {g − (Y (τ )) · X{τ 0; Y (t) ∈ W } < ∞ a.s. Then U ⊂ {y ∈ S; Φ(y) > g(y)} = D. Hence it is never optimal to stop while Y (t) ∈ U . Proof. Choose y ∈ U and let W ⊂ U be a neighborhood of y with τW < ∞ a.s. Then by the Dynkin formula (Theorem 1.24)  τW  τW   y y y f (Y (t))dt . Ag(Y (t))dt > g(y)−E E [g(Y (τW ))] = g(y)+E 0

0

Hence g(y) < E y



τW

0

 f (Y (t))dt + g(YτW ) ≤ Φ(y),

as claimed. Another useful observation is: Proposition 2.4. Let U be as in Proposition 2.3. Suppose U = ∅. Then Φ(y) = g(y)

and

τ∗ = 0

is optimal.

⊓ ⊔

2.2 Applications and Examples

31

Proof. If U = ∅ then Ag(y) + f (y) ≤ 0 for all y ∈ S. Hence the function φ = g satisfies all the conditions of Theorem 2.2. Therefore D = ∅, g(y) = Φ(y), and ⊓ ⊔ τ ∗ = 0 is optimal.

2.2 Applications and Examples Example 2.5 (The Optimal Time to Sell). Suppose the price X(t) at time t of an asset (a property, a stock. . . ) is a geometric L´evy process given by  * ) (dt, dz) , X(0) = x > 0, dX(t) = X(t− ) α dt + β dB(t) + γ z N R

where α, β, and γ are constants, and we assume that −1 < γ z ≤ 0 a.s. ν.

(2.2.1)

If we sell the asset at time s + τ we get the expected discounted net payoff $ & J τ (s, x) := E s,x e−ρ(s+τ ) (X(τ ) − a)X{τ 0 (the discounting exponent) and a > 0 (the transaction cost) are constants. We seek the value function Φ(s, x) and an optimal stopping time τ ∗ ≤ ∞ such that ∗ (2.2.2) Φ(s, x) = sup J τ (s, x) = J τ (s, x). τ ≤∞

We apply Theorem 2.2 to solve this problem as follows: Put S = R × (0, ∞) and   s+t Y (t) = , t ≥ 0. X(t) Then

⎡

1

⎤

⎡

0

⎦dt +⎣ dY (t) = ⎣ αX(t) βX(t)

⎤

⎢ ⎦dB(t) +⎢ ⎣

and the generator A of Y (t) is Aφ(s, x) =

⎡

⎡ ⎤ s ⎥ ⎥, Y (0) = ⎣ ⎦  ⎦ (dt, dz) γX(t− ) z N x

∂φ ∂φ 1 2 2 ∂ 2 φ + αx + β x ∂s  ∂x 2 ∂x2

+

R

⎤

0

R

∂φ φ(s, x + γxz) − φ(s, x) − γxz ∂x



ν(dz).

If we try a function φ of the form φ(s, x) = e−ρs xλ

for some constant λ ∈ R

(2.2.3)

32

2 Optimal Stopping of Jump Diffusions

we get −ρs

Aφ(s, x) = e



1 − ρxλ + αxλxλ−1 + β 2 x2 λ(λ − 1)xλ−2 2  + {(x + γxz)λ − xλ − γxzλxλ−1 }ν(dz) R

−ρs λ

=e

x h(λ),

where 1 h(λ) = −ρ + αλ + β 2 λ(λ − 1) + 2



R

{(1 + γz)λ − 1 − λγz}ν(dz).

Note that h(1) = α − ρ and

lim h(λ) = ∞.

λ→∞

Therefore, if we assume that α < ρ,

(2.2.4)

then we get that there exists λ1 ∈ (1, ρ/α) such that h(λ1 ) = 0. With this value of λ1 we put

e−ρs Cxλ1 for φ(s, x) = e−ρs (x − a) for

(2.2.5)

(s, x) ∈ D (s, x) ∈ D

(2.2.6)

for some constant C, to be determined. To find a reasonable guess for the continuation region D we use Proposition 2.3. In this case we have f = 0 and g(s, x) = e−ρs (x − a) and hence by (2.2.3) Ag + f = e−ρs (−ρ(x − a) + αx) = e−ρs ((α − ρ)x + ρa). Therefore U = {(s, x); (α − ρ)x + ρa > 0}. Case 1: α ≥ ρ . In this case U = R2 and it is easily seen that Φ = ∞. We can get as high expected payoff as we wish by waiting long enough before stopping. Case 2: α < ρ . In this case  U = (s, x); x
0 (to be determined). We guess that the value function is C 1 at x = x∗ and this gives the following “high contact” conditions: C(x∗ )λ1 = x∗ − a

(continuity at x = x∗ )

Cλ1 (x∗ )λ1 −1 = 1

(differentiability at x = x∗ ).

and

It is easy to see that the solution of these equations is x∗ =

λ1 a , λ1 − 1

C=

1 ∗ 1−λ1 . (x ) λ1

(2.2.11)

It remains to verify that with these values of x∗ and C the function φ given by (2.2.10) satisfies all the conditions (i)–(xi) of Theorem 2.2. To this end, first note that (i) and (ix) hold by construction of φ and by (2.2.1). Moreover, φ = g outside D. Therefore, to verify (ii) we only need to prove that φ ≥ g on D, i.e., that Cxλ1 ≥ x − a for

0 < x < x∗ .

(2.2.12)

Define k(x) = Cxλ1 − x + a. By our chosen values of C and x∗ we have k(x∗ ) = k ′ (x∗ ) = 0. Moreover, k ′′ (x) = Cλ1 (λ1 − 1)xλ1 −2 > 0 for x < x∗ . Therefore k(x) > 0 for 0 < x < x∗ and (2.2.12) holds and hence (ii) is proved. (iii): In this case ∂D = {(s, x); x = x∗ } and hence   ∞  ∞ y P x [X(t) = x∗ ]dt = 0. X∂D (Y (t))dt = E 0

(iv) and (v) are trivial.

0

34

2 Optimal Stopping of Jump Diffusions

(vi): Outside D we have φ(s, x) = e−ρs (x − a) and therefore Aφ + f (s, x) = e−ρs (−ρ(x − a) + αx)    ∂φ (s, x)γxz ν(dz) + φ(s, x + γxz) − φ(s, x) − ∂x R = e−ρs (α − ρ)x + ρa  ! + {C(x + γxz)λ1 − (x − a) − γxz}ν(dz) ≤e

x+γxz −1 a.s. ν. For (viii) to hold it suffices that  ∞  + ,  e−2ρt X 2 (t) + γ 2 z 2 ν(dz)t dt < ∞. Ex 0

R

By the above this holds if 2

2α − 2ρ + β + γ

2



z 2 ν(dz) < 0.

(2.2.14)

R

(x): To check if τD < ∞ a.s. we consider the solution X(t) of (2.2.1), which by (1.2.5) is given by   ! 1 α − β2 − γ zν(dz) t X(t) = x exp 2 R   t ln(1 + γz)N (dt, dz) + βB(t) . + 0

R

2.2 Applications and Examples

35

By the law of iterated logarithm for Brownian motion (see the argument in [Ø1, Chapter 5] we see that if  1 2 (2.2.15) α > β + γ zν(dz) 2 R and z ≥ 0 a.s. ν

then

lim X(t) = ∞

(2.2.16) a.s.

t→∞

and in particular τD < ∞ a.s. (xi): Since φ is bounded on [0, x∗ ] it suffices to check that  −ρτ  e X(τ ) τ ∈T is uniformly integrable.

For this to hold it suffices that there exists a constant K such that E[e−2ρτ X 2 (τ )] ≤ K

for all τ ∈ T .

(2.2.17)

By (2.2.13) and Exercise 1.6 we have  ! ) + −2ρT 2 2 2 E[e zν(dz) τ X (τ )] = x E exp 2α − 2ρ − β − 2γ R  τ ,* ln(1 + γz)N (dt, dz) + 2βB(τ ) +2 0 R  ! ) + 2 = x E exp 2α − 2ρ + β 2 + 2 (ln(1 + γz) − γz)ν(dz) τ R  τ ,* (dt, dz) ln(1 + γz)N +2 0 R  ) + 2 2 = x E exp 2α − 2ρ + β + [2 ln(1+γz) − 2γz+(1 + γz)2 R ! ,* − 1 − 2 ln(1 + γz)]ν(dz) τ  ) + 2 2 = x E exp 2α − 2ρ + β + [(1 + γz)2 ! R,* − 1 − 2γz]ν(dz) τ .

We conclude that if

2α − 2ρ + β 2 + γ 2



R

z 2 ν(dz) ≤ 0

then (2.2.17) holds and hence (xi) holds also. We summarize what we have proved.

36

2 Optimal Stopping of Jump Diffusions

Theorem 2.6. Suppose that (2.2.1), (2.2.4), (2.2.13), (2.2.15), and (2.2.14) hold. Then, with λ1 , C, and x∗ given by (2.2.5) and (2.2.11), the function φ given by (2.2.10) coincides with the value function Φ of problem (2.2.2) and τ ∗ = τD is an optimal stopping time, where D is given by (2.2.8). Remark 2.7. (1) If condition (2.2.1) is relaxed to −1 < γz

a.s. ν

then the situation becomes more complicated. See [Mo] for a solution of a related problem in this case. See also [Ky]. (2) The C 1 -property of the value function assumed in Theorem 2.2 (often call the “high contact” or “smooth pasting” assumption) need not hold in general, although we found that it holds in Example 2.5 (under some conditions). See [AKy] for a discussion of this. See also Sect. 9.1. (3) For other applications of optimal stopping to jump diffusions we refer to [Ma].

2.3 Optimal Stopping with Delayed Information This presentation is based on [Ø4]. Let Y (t) be a jump diffusions in Rk . Let δ ≥ 0 be a fixed constant. In this section we consider optimal stopping problems of the form   α y f (Y (t))dt + g(Y (α)) , (2.3.1) Φδ (y) := sup E α∈Tδ

0

where we interpret g(Y (α)) as 0 if α = ∞. Here Tδ is the set of δ-delayed stopping times, defined as follows. Definition 2.8. A function α : Ω → [δ, ∞] is called a δ-delayed stopping time if (2.3.2) {ω; α(ω) ≤ t} ∈ Ft−δ for all t ≥ δ or, equivalently, {ω; α(ω) ≤ s + δ} ∈ Fs

for all s ≥ 0.

(2.3.3)

The set of all δ-delayed stopping times is denoted by Tδ . In other words, if we interpret α(ω) as the time to stop, then α ∈ Tδ if the decision whether or not to stop at or before time t is based on the information represented by Ft−δ . In particular, if δ = 0 then Tδ = T0 is the family of classical stopping times and (2.3.1) becomes the classical optimal stopping problem, discussed in previous sections.

2.3 Optimal Stopping with Delayed Information

37

In the delayed case problem (2.3.1) models the situation when there is a delay δ > 0 in the flow of information available to the agent searching for the optimal time to stop. An alternative way of stating this is that there is a delay δ > 0 from the decided stopping time τ ∈ T0 (based on the complete current information available from the system) to the time α = τ + δ ∈ Tδ when the system actually stops. This new formulation is based on the following simple observation. τ ∈ T0 ⇐⇒ α := τ + δ ∈ Tδ ,

Lemma 2.9. (i) (ii)

α ∈ Tδ ⇐⇒ τ := α − δ ∈ T0 .

Proof. It suffices to prove (i). First, assume τ ∈ T0 . Then, for t ≥ δ, {ω; τ (ω) + δ ≤ t} = {ω; τ (ω) ≤ t − δ} ∈ Ft−δ , and hence α := τ + δ ∈ Tδ . Conversely, if α := τ + δ ∈ Tδ then {ω; τ (ω) ≤ t} = {ω; τ (ω) + δ ≤ t + δ} = {ω; α(ω) ≤ t + δ} ∈ F(t+δ)−δ = Ft , ⊓ ⊔

and hence τ ∈ T0 .

Remark 2.10. In view of this result we see that it is possible to give another interpretation of problem (2.3.1), namely  τ +δ

Φδ (y) = sup E y τ ∈T0

f (Y (t))dt + g(Y (τ + δ)) .

(2.3.4)

0

In this formulation the problem appears as an optimal stopping problem over classical stopping times τ ∈ T0 , but with delayed effect of the stopping. If the stopping time τ ∈ T0 is chosen, then the system itself is stopped at time τ + δ, i.e., after a delay δ > 0. Note that Tδ ⊂ T0 for δ > 0 and hence Φδ (y) ≤ Φ0 (y) and we can interpret Φ0 (y) − Φδ (y) as the loss of value due to the delay of information. In this section we show that the delayed optimal stopping problem (2.3.1) can be reduced to a classical optimal stopping problem by a simple transformation (Theorem 2.11). We call α∗ ∈ Tδ an optimal stopping time for the problem (2.3.1) if  ∗ α

Φδ (y) = E y

f (Y (t))dt + g(Y (α∗ )) .

0

(2.3.5)

38

2 Optimal Stopping of Jump Diffusions

The result of this section may be regarded as a partial extension of [AK], where the geometric Brownian motion case is studied and solved (see Exercise 2.12), with a more general (Markovian) delay δ(X) ≤ 0. See also [AK]. For a related type of problem involving impulse control with delivery lags, see [BS]. We are now ready to state and prove the main result of this section. Theorem 2.11. (a) Consider the two optimal stopping problems:  α

Φδ (y) := sup E y α∈Tδ

˜ Φ(y) := sup E

y

τ ∈T0

f (Y (t))dt + g(Y (α)) ,



0



τ

f (Y (t))dt + g˜δ (Y (τ )) ,

0

where g˜δ (y) = E

y



f (Y (t))dt + g(Y (δ)) .

0

Then we have

˜ Φδ (y) = Φ(y)

(2.3.7)



δ

(2.3.6)

(2.3.8)

for all y ∈ Rk , δ ≥ 0.

(b) Moreover, α∗ ∈ Tδ is an optimal stopping time for the delayed problem (2.3.6) if and only if α∗ := τ ∗ + δ

(2.3.9)

where τ ∗ ∈ T0 is an optimal stopping time for the nondelayed problem (2.3.7). Proof. (a) Define J

(α)

(y) = E

y



α

J˜(τ ) (y) = E y



α ∈ Tδ ,

(2.3.10)



τ ∈ T0 .

(2.3.11)

f (Y (t))dt + g(Y (α)) ,

0

and



τ

f (Y (t))dt + g˜δ (Y (τ )) ,

0

Choose α ∈ Tδ and put

τ = α − δ ∈ T0 .

Then α = τ + δ and hence J

(α)

(y) = E

y



α

f (Y (t))dt + g(Y (α))

0

=E

y



0



τ +δ



f (Y (t))dt + g(Y (τ + δ))

2.3 Optimal Stopping with Delayed Information

= Ey



τ

f (Y (t))dt +

=E



τ +δ

τ

f (Y (t))dt + E

39

f (Y (t))dt + g(Y (τ + δ))

τ

0

y





y

0



θτ



δ



f (Y (t))dt + g(Y (δ))

0

,

(2.3.12)

where θτ is the shift operator, defined by θτ {h(Y (s))} = h(Y (τ + s)) for s ≥ 0, for all measurable h : Rk −→ R, and we have used that



δ

θτ

f (Y (t))dt

0



=



τ +δ

f (Y (t))dt.

τ

We refer to [BG] for more information about Markov processes. By the strong Markov property we now get from (2.3.12) that      τ τ  y (α) y f (Y (t))dt + g(Y (δ)) Fτ f (Y (t))dt + E θτ J (y) = E 0

0

=E

y



τ

f (Y (t))dt + E

Y (τ )

=E



δ

τ



f (Y (t))dt + g˜δ (Y (τ )) = J˜(τ ) (y).

0



f (Y (t))dt + g(Y (δ))

0

0

y



(2.3.13)

Hence, by Lemma 2.9 (ii), Φδ (y) = sup J (α) (y) = sup J (α) (y) α∈Tδ

α−δ∈T0

˜(α−δ)

= sup J α−δ∈T0

˜ (y) = sup J˜(τ ) (y) = Φ(y), τ ∈T0

as claimed. (b) Suppose τ ∗ ∈ T0 is optimal for (2.3.7). Define α∗ := τ ∗ + δ. Then α∗ ∈ Tδ by Lemma 2.9 and by (2.3.13) combined with (a) we have ∗ ∗ ˜ J (α ) (y) = J˜(τ ) (y) = Φ(y) = Φδ (y).

Hence α∗ is optimal for (2.3.6). Conversely, if α∗ ∈ Tδ is optimal for (2.3.6) a similar argument gives that ∗ τ := α∗ − δ is optimal for (2.3.7). ⊓ ⊔

40

2 Optimal Stopping of Jump Diffusions

We illustrate Theorem 2.11 by solving the following problem. In the no delay case the following example is discussed in [Ø1] (continuous case) and Exercise 2.2 (jump diffusion case). Our example models the situation when there is a time lag δ > 0 between the decided stopping time τ ∈ T0 and the time α = τ + δ ∈ Tδ when the result of the stopping decision comes into effect. Example 2.12 (Optimal Time to Stop Resource Extraction). Suppose the price P (t) at time t per unit of a resource (oil, gas, . . . ) is given by   − ˜ dP (t) = P (t ) μ dt + σ dB(t) + z N (dt, dz) , P (0) = p > 0, (2.3.14) R

where μ and σ are given constants and we assume that z ≥ 0 a.s. with respect to ν. Let Q(t) denote the amount of remaining resources at time t. As long as the extraction field is open, we assume that the extraction rate is proportional to the remaining amount, i.e., dQ(t) = −λQ(t)dt,

Q(0) = q > 0,

(2.3.15)

where λ > 0 is a known constant. If we decide to stop the extraction and close the field at a (delayed) stopping time α ∈ Tδ , then the expected total discounted net profit J α (s, p, q) is assumed to have the form  α J α (s, p, q) = E (s,p,q) e−ρ(s+t) (λP (t)Q(t) − K)dt + θe−ρ(s+α) P (α)Q(α) , 0

(2.3.16) where K > 0 is the (constant) running cost rate and ρ > 0, θ > 0 are other constants. The expectation E (s,p,q) is taken with respect to the probability law P (s,p,q) of the strong Markov process ⎡ ⎤ ⎤ ⎡ s s+t (2.3.17) Y (t) := ⎣ P (t) ⎦, which starts at y = ⎣p⎦ at time t = 0. q Q(t)

The explanation of the quantity J α (s, p, q) in (2.3.16) is the following. As long as the field is open (i.e., as long as t < α) the gross income rate from the production is price times production rate, i.e., P (t)λQ(t). Subtracting the running cost rate K we get the net profit rate λP (t)Q(t) − K

for 0 ≤ t < α.

If the field is closed at time α the net value of the remaining resources is estimated to be θP (α)Q(α). Discounting and integrating/adding these quantities and taking expectation we get (2.3.16).

2.3 Optimal Stopping with Delayed Information

41

We want to find the value function Φδ (s, p, q) and the corresponding optimal delayed stopping time α∗ ∈ Tδ such that ∗

Φδ (y) = Φδ (s, p, q) = sup J α (s, p, q) = J α (s, p, q).

(2.3.18)

α∈Tδ

In the case of no delay (δ = 0) it is shown in Exercise 2.2 that if the following relations between the parameters hold 0 < θ(λ + ρ − μ) < λ

(2.3.19)

then the optimal stopping time τ0∗ ∈ T0 is τ0∗ = inf{t > 0; P (t)Q(t) ≤ w0∗ }, where w0∗ =

(−r2 )K(λ + ρ − μ) , (1 − r2 )ρ(λ − θ(λ + ρ − μ))

(2.3.20)

(2.3.21)

r2 < 0 being the negative solution of the equation  1 h(r) := −ρ+(μ−λ)r + σ 2 r(r −1)+ {(1+z)r −1−rz}ν(dz) = 0. (2.3.22) 2 R In this case we have f (y) = f (s, p, q) = e−ρs (λpq − K) and g(y) = g(s, p, q) = θe−ρs pq. Thus g˜δ (y) = E

y



δ

e

0

=



=

0



(λP (t)Q(t) − K)dt + E y [θe−ρ(s+δ) P (δ)Q(δ)]

δ

e−ρ(s+t) (λE[P (t)Q(t)] − K)dt + θe−ρ(s+δ) E y [P (δ)Q(δ)]

0



−ρ(s+t)

δ

e−ρ(s+t) (λpqe(μ−λ)t − K)dt + θe−ρ(s+δ) pqe(μ−λ)δ 

= e−ρs {(λ + ρ − μ)−1 λ(1 − e−(λ+ρ−μ)δ ) + θe−(λ+ρ−μ)δ }pq K −ρδ − (1 − e ) ρ

= e−ρs [F1 pq − F2 ],

(2.3.23)

42

2 Optimal Stopping of Jump Diffusions

where F1 = (λ + ρ − μ)−1 λ(1 − e−(λ+ρ−μ)δ ) + θe−(λ+ρ−μ)δ

(2.3.24)

and

K (1 − e−ρδ ). ρ Therefore, according to Theorem 2.11 we have  τ −ρ(s+t) y e (λP (t)Q(t) − K)dt Φδ (y) = sup E F2 =

τ ∈T0

y

(2.3.25)

0

−ρ(s+τ )

+ E [e

(F1 P (τ )Q(τ ) + F2 )].

(2.3.26)

The method used in Exercise 2.2 to provide the solutions (2.3.20)–(2.3.22) in the no delay case can easily be modified to find the optimal stopping time τ ∗ for the problem (2.3.26). The result is wδ∗ = We have proved.

(−r2 )K(λ + ρ − μ)e(λ−μ)δ = w0∗ e(λ−μ)δ . (1 − r)ρ[λ − θ(λ + ρ − μ)]

(2.3.27)

Theorem 2.13. The optimal stopping time α∗ ∈ Tδ for the delayed optimal stopping problem (2.3.18) is α∗ = τδ∗ + δ,

(2.3.28)

τδ∗ = inf{t > 0; P (t)Q(t) ≤ wδ∗ },

(2.3.29)

where with

wδ∗

given by (2.3.27).

Remark 2.14. Note that the threshold wδ∗ for the decision to close down in the case of a time lag in the action only differs from the corresponding threshold w0∗ in the no delay case by the factor e(λ−μ)δ . Assume, for example, that λ > μ. Then we should decide to stop sooner in the delay case than in the no delay case, because of the anticipation that P (t)Q(t) will probably decrease during the extra time δ it takes before the closing down actually takes place.

2.4 Exercises Exercise* 2.1. Solve the optimal stopping problem $ & Φ(s, x) = sup E (s,x) e−ρ(s+τ ) (X(τ ) − a) , τ ≥0

where

dX(t) = dB(t) + γ



R

¯ (dt, dz), zN

X(0) = x ∈ R

and ρ > 0, a > 0, and γ are constants, γz ≤ 0 a.s. ν.

2.4 Exercises

43

Exercise* 2.2 (An Optimal Resource Extraction Stopping Problem). Suppose the price P (t) per unit of a resource (oil, gas . . .) at time t is given by  − (dt, dz), P (0) = p > 0 (1) dP (t) = αP (t)dt + βP (t)dB(t) + γP (t ) z N R

and the remaining amount of resources Q(t) at time t is

(2) dQ(t) = −λQ(t)dt,

Q(0) = q > 0,

where λ > 0 is the (constant) relative extraction rate and α, β, γ are constants. We assume that γz ≥ 0 a.s. ν. If we decide to stop extraction and close the field at a stopping time τ ≥ 0, the expected discounted total net profit J τ (s, p, q) is given by  τ

J τ (s, p, q) = E (s,p,q)

0

e−ρ(s+t) (λP (t)Q(t) − K)dt + θe−ρ(s+τ ) P (τ )Q(τ ) ,

where K > 0 is the (constant) running cost rate, ρ > 0 is the (constant) discounting exponent, and θ > 0 another constant. Find Φ and τ ∗ such that ∗

Φ(s, p, q) = sup J τ (s, p, q) = J τ (s, p, q). τ ≥0

[Hint: Try φ(s, p, q) = e−ρs ψ(p · q) for some function ψ : R → R.] Exercise* 2.3. Solve the optimal stopping problem Φ(s, x) = sup E x [e−ρ(s+τ ) |X(τ )|] τ ≥0

where dX(t) = dB(t) +



˜ (dt, dz) θ(X(t), z)N

R

and ρ > 0 is a constant. Assume that there exists ξ > 0 such that θ(x, z) = 0 for a.a. z if |x| < ξ

and

(2.4.1)

θ(x, z)(x − ξ) ≥ 0 for a.a. z if |x| ≥ ξ

(2.4.2)

1 tgh( 2ρ ξ) ≥ √ . 2ρ ξ

(2.4.3)

Exercise* 2.4 (The Optimal Time with Delay to Sell an Asset). This case (without the jump part) was first solved by [AK], with a more general (Markovian) delay δ(X) ≥ 0.

44

2 Optimal Stopping of Jump Diffusions

Suppose the value X(t) of an asset at time t is modeled by a geometric L´evy process of the form   − ˜ dX(t) = X(t ) μ dt + σ dB(t) + z N (dt, dz) , X(0) = x > 0, (2.4.4) R

where μ, σ, and x are constants. We assume that −1 < z < 0

a.s. ν.

(2.4.5)

This guarantees that X(t) never jumps down to a negative value. For convenience, we also assume that E[η 2 (t)] < ∞

for all t ≥ 0.

(2.4.6)

Then by the Itˆ o formula for L´evy processes the solution of (2.4.4) is   1 2 X(t) = x exp μ − σ t + σB(t) 2  t  t ˜ (ds, dz) , ln(1 + z)N {ln(1 + z) − z}ν(dz)ds + + 0

0

R

R

t ≥ 0.

(2.4.7)

Solve the following delayed optimal stopping problem Φδ (s, x) = sup E s,x [e−ρ(s+α) (X(α) − q)],

(2.4.8)

α∈Tδ

where E s,x denotes expectation with respect to the probability law P s,x of the time–space process     dt s dY (t) = , Y (0) = dX(t) x and ρ > 0, q > 0 are constants. We assume that ρ > μ.

(2.4.9)

One possible interpretation of this problem is that Φδ (s, x) represents the maximal expected discounted net payment obtained by selling the asset at a δ-delayed stopping time (ρ is the discounting exponent and q is the transaction cost).

3 Stochastic Control of Jump Diffusions

3.1 Dynamic Programming Fix a domain S ⊂ Rk (our solvency region) and let Y (t) = Y (u) (t) be a stochastic process of the form dY (t) = b(Y (t), u(t))dt + σ(Y (t), u(t))dB(t)  ¯ (dt, dz), Y (0) = y ∈ Rk , + γ(Y (t− ), u(t− ), z)N

(3.1.1)

Rk

where b : Rk × U → Rk ,

σ : Rk × U → Rk×m ,

and γ : Rk × U × Rk → Rk×ℓ

are given functions, U ⊂ Rp is a given set. The process u(t) = u(t, ω) : [0, ∞) × Ω → U is our control process, assumed to be c`adl` ag and adapted. We call Y (t) = Y (u) (t) a controlled jump diffusion. We consider a performance criterion J = J (u) (y) of the form  τS  (u) y f (Y (t), u(t))dt + g(Y (τS )) · X{τS 0; Y (u) (t) ∈ S} (the bankruptcy time)

and f : S → R and g : Rk → R are given continuous functions. We say that the control process u is admissible and write u ∈ A if (3.1.1) has a unique, strong solution Y (t) for all y ∈ S and  τS  − − y f (Y (t), u(t))dt + g (Y (τS )) · X{τS 0, μ > 0, and σ ∈ R are constants. We assume that  ∞ |z|dν(z) < ∞ and μ > r. −1

Assume that at any time t the investor can choose a consumption rate c(t) ≥ 0 (adapted and c` adl` ag) and is also free to transfer money from one investment to the other without transaction cost. Let X1 (t) and X2 (t) be the amounts of money invested in the bonds and the stocks, respectively. Let

48

3 Stochastic Control of Jump Diffusions

θ(t) =

X2 (t) X1 (t) + X2 (t)

be the fraction of the total wealth invested in stocks at time t. Define the performance criterion by   ∞ cγ (t) J (c,θ) (s, x1 , x2 ) = E x1 ,x2 dt , e−δ(s+t) γ 0 where δ > 0, γ ∈ (0, 1) are constants and E x1 ,x2 is the expectation w.r.t. the probability law P x1 ,x2 of (X1 (t), X2 (t)) when X1 (0) = x1 , X2 (0) = x2 . Call the control u(t) = (c(t), θ(t)) ∈ [0, ∞) × [0, 1] admissible and write u ∈ A if the corresponding total wealth (u)

(u)

W (t) = W (u) (t) = X1 (t) + X2 (t) is nonnegative for all t ≥ 0. The problem is to find Φ(s, x1 , x2 ) and u∗ (c∗ , θ∗ ) ∈ A such that ∗

Φ(s, x1 , x2 ) = sup J (u) (s, x1 , x2 ) = J (u ) (s, x1 , x2 ). u∈A

Case 1: ν = 0. In this case the problem was solved by Merton [M]. He proved that if   (μ − r)2 δ>γ r+ 2 , (3.1.8) 2σ (1 − γ) then the value function is

where

Φ0 (s, x1 , x2 ) = K0 e−δs (x1 + x2 )γ ,

(3.1.9)

  γ−1 1 1 γ(μ − r)2 K0 = . δ − γr − 2 γ 1−γ 2σ (1 − γ)

(3.1.10)

c∗0 (t) = (K0 γ)1/(γ−1) (X1 (t) + X2 (t))

(3.1.11)

Moreover, the optimal consumption rate c∗0 (t) is given by

and the optimal portfolio θ0∗ (t) is (the constant) θ0∗ (t) =

μ−r σ 2 (1 − γ)

for all t ∈ [0, ∞).

(3.1.12)

In other words, it is optimal to keep the state (X1 (t), X2 (t)) on the line x2 =

θ0∗ x1 1 − θ0∗

in the (x1 , x2 )-plane at all times (the Merton line). See Fig. 3.1.

(3.1.13)

3.1 Dynamic Programming x2

49

the Merton line (ν = 0) (x1 , x2 ) (X1 (t), X2 (t))

S (x1 , x2 )

x1

S x1 + x2 = 0 Fig. 3.1. The Merton line

Case 2: ν = 0 We now ask: How does the presence of jumps influence the optimal strategy? As in [M] we reduce the dimension by introducing W (t) = X1 (t) + X2 (t). Then we see that dW (t) = ([r(1 − θ(t)) + μθ(t)]W (t) − c(t)) dt + σθ(t)W (t)dB(t)  ∞ − (dt, dz), W (0) = x1 + x2 = w ≥ 0. + θ(t)W (t ) zN −1

(u)

The generator A

of the controlled process     s+t s Y (t) = ; t ≥ 0, Y (0) = y = W (t) w

is A(u) φ(y) =

( ∂φ ∂φ ' + [r(1 − θ) + μθ]w − c + ∂s ∂w ∞+ φ(s, w + θwz) − φ(s, w) − + −1

1 2 2 2 ∂2φ σ θ w 2 ∂w2 , ∂φ (s, w)θwz ν(dz). ∂w

50

3 Stochastic Control of Jump Diffusions

If we try φ(y) = φ(s, w) = e−δs ψ(w) we get (u)

A(u) φ(y) = e−δs A0 ψ(w),

where

1 (u) A0 ψ(w) = −δψ(w) + ([r(1 − θ) + μθ] w − c) ψ ′ (w) + σ 2 θ2 w2 ψ ′′ (w) 2  ∞ ′ + {ψ((1 + θz)w) − ψ(w) − ψ (w)θwz}ν(dz). −1

In particular, if we try ψ(w) = Kwγ we get ' ( (u) A0 ψ(w) + f (w, u) = −δKwγ + [r(1 − θ) + μθ]w − c Kγwγ−1 1 + K · σ 2 θ2 w2 γ(γ − 1)wγ−2 2 ∞

+ Kwγ

−1

{(1 + θz)γ − 1 − γθz}ν(dz) +

cγ . γ

Let h(c, θ) be the expression on the right-hand side. Then h is concave in (c, θ) and the maximum of h is attained at the critical points, i.e., when ∂h = −Kγwγ−1 + cγ−1 = 0 ∂c

(3.1.14)

and ∂h = (μ−r)Kγwγ +Kσ 2 θγ(γ−1)wγ +Kwγ ∂θ From (3.1.14) we get



∞

−1

{γ(1+θz)γ−1 z−γz}ν(dz) = 0.

' (1/(γ−1) w c = cˆ = Kγ

and from (3.1.15) we get that θ = θˆ should solve the equation  ∞   Λ(θ) := μ − r − σ 2 θ(1 − γ) − 1 − (1 + θz)γ−1 zν(dz) = 0.

(3.1.15) (3.1.16)

(3.1.17)

−1

Since Λ(0) = μ − r > 0 we see that if  ∞   σ 2 (1 − γ) + 1 − (1 + z)γ−1 zν(dz) ≥ μ − r −1

then there exists an optimal θ = θˆ ∈ (0, 1].

(3.1.18)

3.1 Dynamic Programming

51

' (1/(γ−1) With this choice of c = cˆ = Kγ w and θ = θˆ (constant) we require that (ˆ u)

or

ˆ) = 0, i.e., A0 ψ(w) + f (w, u ( ' ' ( ˆ + μθ] ˆ − Kγ 1/(γ−1) Kγ − δK + [r(1 − θ)  ∞ 1 2 ˆ2 ˆ γ − 1 − γ θz}ν(dz) ˆ + K σ θ γ(γ − 1) + K {(1 + θz) 2 −1 ' (γ/(γ−1) 1 + Kγ =0 γ

ˆ + μθ] ˆ − (Kγ)1/(γ−1) γ −δ + γ[r(1 − θ)  ∞ 1 1 ˆ γ − 1 − γ θz}ν(dz) ˆ − σ 2 θˆ2 (1 − γ)γ + {(1 + θz) + K 1/(γ−1) · γ γ/(γ−1) · 2 γ −1

or

' or

Kγ

(1/(γ−1)

ˆ + μθ] ˆ + 1 σ 2 θˆ2 (1 − γ)γ [1 − γ] = δ − γ[r(1 − θ) 2  ∞ γ ˆ ˆ − {(1 + θz) − 1 − γ θz}ν(dz) −1



1 1 ˆ + μθ} ˆ + 1 σ 2 θˆ2 (1 − γ)γ K= δ − γ{r(1 − θ) γ 1−γ 2  γ−1  γ ˆ ˆ − {(1 + θz) − 1 − γ θz}ν(dz) .

(3.1.19)

R

We now study condition (iii): Here σ T ∇φ(y) = e−δs σθwKγwγ−1 = e−δs σθKw γ and φ(Y (t) + γ(Y (t), u(t))) − φ(Y (t)) = KW (t)γ e−δs [(1 + θz)γ − 1]. So (iii) holds if   T e−2δt W 2γ (t)dt + [(1 + θz)γ − 1]ν(dz) < ∞. E 0

(3.1.20)

R

We refer to [FØS1] for sufficient conditions on the parameters for (3.1.20) to hold. We conclude that the value function is Φ(s, w) = Φ(s, x1 , x2 ) = e−δs K(x1 + x2 )γ

(3.1.21)

with optimal control u∗ (t) = (c∗ (t), θ∗ (t)) where c∗ = cˆ = (Kγ)1/γ−1 (x1 + x2 ) is given by (3.1.16) and θ∗ = θˆ is given by (3.1.17), with K given by (3.1.19).

52

3 Stochastic Control of Jump Diffusions x2

ν = 0 (classical Merton line)

x2 =

ν > 0 (jump Merton line)

∗ θ0 ∗ x1 1−θ0

x2 =

θ∗ x 1−θ ∗ 1

0

x1

Fig. 3.2. The Merton line for ν = 0 and ν > 0

Finally we compare the solution in the jump case (ν = 0) with Merton’s solution in the no jump case (ν = 0). As before let Φ0 , c∗0 , and θ0∗ be the solution when there are no jumps (ν = 0). Then it can be seen that K < K0

and hence Φ(s, w) = e−δs Kwγ < e−δs K0 wγ = Φ0 (s, w) c∗ (s, w) ≥ c∗0 (s, w) θ∗ ≤ θ0∗ .

So with jumps it is optimal to place a smaller wealth fraction in the risky investment, consume more relative to the current wealth and the resulting value is smaller than in the no jump case. See Fig.3.2. For more details we refer to [FØS1]. Remark 3.3. For more information and other applications of stochastic control of jump diffusions, see [GS, BKR1, BKR2, BKR3, BKR4, BKR5, BKR6, Ma] and the references therein.

3.2 The Maximum Principle Suppose the state X(t) = X (u) (t) of a controlled jump diffusion in Rn is given by

3.2 The Maximum Principle

dX(t) = b(t, X(t), u(t))dt + σ(t, X(t), u(t))dB(t)  (dt, dz). + γ(t, X(t− ), u(t− ), z)N

53

(3.2.1)

Rℓ

(dt, dz) = (N 1 (dt, dz1 ), . . . , N ℓ (dt, dzℓ ))T , where As before N j (dt, dzj ) = Nj (dt, dzj ) − νj (dzj )dt, N

1≤j≤ℓ

(see the notation of Theorem 1.16). The process u(t) = u(t, ω) ∈ U ⊂ Rk is our control. We assume that u is adapted and c` adl` ag, and that the corresponding (3.2.1) has a unique strong solution X (u) (t), t ∈ [0, T ]. Such controls are called admissible. The set of admissible controls is denoted by A. Suppose the performance criterion has the form  T

f (t, X(t), u(t))dt + g(X(T )) ,

J(u) = E

0

u ∈ A,

where f : [0, T ] × Rn × U → R is continuous, g : Rn → R is C 1 , T < ∞ is a fixed deterministic time and  T − − f (t, X(t), u(t))dt + g (X(T )) < ∞ for all u ∈ A. E 0

Consider the problem to find u∗ ∈ A such that J(u∗ ) = sup J(u).

(3.2.2)

u∈A

In Chap. 2 we saw how to solve such a problem using dynamic programming and the associated HJB equation. Here we present an alternative approach, based on what is called the maximum principle. In the deterministic case this principle was first introduced by Pontryagin and his group [PBGM]. A corresponding maximum principle for Itˆ o diffusions was formulated by Kushner [Ku], Bismut [Bi], and subsequently further developed by Bensoussan [Ben1, Ben2, Ben3], Haussmann [H1], and others. For jump diffusions a sufficient maximum principle has recently been formulated in [FØS3] and it is this approach that is presented here, in a somewhat simplified version. Define the Hamiltonian H : [0, T ] × Rn × U × Rn × Rn×m × R → R by H(t, x, u, p, q, r) = f (t, x, u) + bT (t, x, u)p + tr(σ T (t, x, u)q) n  ℓ   γij (t, x, u, zj )rij (t, z)νj (dzj ), + j=1 i=1

R

(3.2.3)

54

3 Stochastic Control of Jump Diffusions

where R is the set of functions r : Rℓ+1 → Rn×ℓ such that the integrals in (3.2.3) converge. From now on we assume that H is differentiable with respect to x. The adjoint equation (corresponding to u and X (u) ) in the unknown processes p(t) ∈ Rn , q(t) ∈ Rn×m , and r(t, z) ∈ Rn×ℓ is the backward stochastic differential equation ⎧ ⎪ dp(t) = −∇x H(t, X(t), u(t), p(t), q(t), r(t, ·))dt ⎪ ⎪  ⎨ (dt, dz), (3.2.4) r(t− , z)N t 0 is the discounting( exponent and θ > 0, a ≥ 0 are constants. Thus e−ρ(s+t) ut (Pt Qt − K1 ) − K0 gives the discounted net profit rate when the field is in operation, while e−ρ(s+τ ) (θPτ Qτ − a) gives the discounted net value of the remaining resources at time τ . (We may interpret a ≥ 0 as a transaction cost.) We assume that the closing time τ is a stopping time with respect to the filtration {Ft }t≥0 , i.e., that {ω; τ (ω) ≤ t} ∈ Ft

for all t.

Thus both the extraction intensity ut and the decision whether to close before or at time t must be based on the information Ft only, not on any future information. The problem is to find the value function Φ(s, p, q) and the optimal control u∗t ∈ [0, m] and the optimal stopping time τ ∗ such that ∗

Φ(s, p, q) = sup J (u,τ ) (s, p, q) = J (u

,τ ∗ )

(s, p, q).

(4.1.4)

ut ,τ

This problem is an example of a combined optimal stopping and stochastic control problem. It is a modification of a problem discussed in [BØ1, DZ]. We will return to this and other examples after presenting a general theory for problems of this type.

4.2 A General Mathematical Formulation Consider a controlled stochastic system of the same type as in Chap. 3, where the state Y (u) (t) = Y (t) ∈ Rk at time t is given by  ¯ (dt, dz), dY (t) = b(Y (t), u(t))dt + σ(Y (t), u(t))dB(t)+ γ(Y (t− ), u(t− ), z)N Rk

k

Y (0) = y ∈ R .

(4.2.1)

Here b : Rk × U → Rk , σ : Rk × U → Rk×m , and γ : Rk × U × Rk → Rk×ℓ are given continuous functions and u(t) = u(t, ω) is our control, assumed to be Ft -adapted and with values in a given closed, convex set U ⊂ Rp .

4.2 A General Mathematical Formulation

67

Associated to a control u = u(t, ω) and an Ft -stopping time τ = τ (ω) belonging to a given set T of admissible stopping times, we assume there is a performance criterion of the form   τ f (Y (t), u(t))dt + g(Y (τ ))χ{τ 0; Y (u) (t) ∈ S}. – The family {g − (Y (u) (τ )); τ ∈ T } is uniformly P y -integrable for all y ∈ S, where g − (y) = max(0, −g(y)).

(4.2.4)

We interpret g(Y (τ (ω))) as 0 if τ (ω) = ∞. Here, and in the following, E y denotes expectation with respect to P when Y (0) = y and S ⊂ Rk is a fixed Borel set such that S ⊂ S 0. We can think of S as the “universe” or “solvency set” of our system, in the sense that we are only interested in the system up to time T , which may be interpreted as the time of bankruptcy. We now consider the following combined optimal stopping and control problem. Let T be the set of Ft -stopping times τ ≤ τS . Find Φ(y) and u∗ ∈ U, ∗ τ ∈ T such that ∗

Φ(y) = sup{J (u,τ ) (y); u ∈ U, τ ∈ T } = J (u

,τ ∗ )

(y).

(4.2.5)

We will prove a verification theorem for this problem. The theorem can be regarded as a combination of the variational inequalities for optimal stopping (Theorem 2.2) and the HJB equation for stochastic control (Theorem 3.1). We say that the control u is Markov or Markovian if it has the form u(t) = u0 (Y (t)) for some function u0 : S¯ → U . If this is the case we usually do not distinguish notationally between u and u0 and write (with abuse of notation) u(t) = u(Y (t)).

68

4 Combined Optimal Stopping and Stochastic Control of Jump Diffusions

If u ∈ U is Markovian then Y (u) (t) is a Markov process whose generator coincides on C02 (Rk ) with the differential operator L = Lu defined for y ∈ Rk by Lu ψ(y) =

k 

bi (y, u(y))

i=1

+

ℓ   j=1

R

k 1  ∂2ψ ∂ψ + (σσ T )ij (y, u(y)) ∂yi 2 i,j=1 ∂yi ∂yj

{ψ(y + γ (j) (y, u(y), zj )) − ψ(y)

− ∇ψ(y).γ (j) (y, u(y), zj )}νj (dzj )

(4.2.6)

for all functions ψ : Rk → R which are twice differentiable at y. Typically the value function Φ will be C 2 outside the boundary ∂D of the continuation region D (see (ii) below) and it will satisfy an HJB equation ¯ Across ∂D the function Φ will not in D and an HJB inequality outside D. 2 1 be C , but it will usually be C , and this feature is often referred to as the “high contact” – or “smooth fit” – principle. This is the background for the verification theorem given below (Theorem 4.2). Note, however, that there are cases when Φ is not even C 1 at ∂D. To handle such cases one can use a verification theorem based on the viscosity solution concept. See Chap. 9 and in particular Sect. 9.2. Theorem 4.2 (HJB-Variational Inequalities for Optimal Stopping and Control). (a) Suppose we can find a function ϕ : S¯ → R such that ¯ (i) ϕ ∈ C 1 (S 0 ) ∩ C(S) 0 (ii) ϕ ≥ g on S Define D = {y ∈ S; ϕ(y) > g(y)}

(the continuation region).

(u) Suppose  τS Y (t) spends 0time on ∂D a.s., i.e., (iii) E y X∂D (Y (u) (t))dt = 0 for all y ∈ S, u ∈ U 0

and suppose that (iv) ∂D is a Lipschitz surface (v) ϕ ∈ C 2 (S 0 \ ∂D) and the second-order derivatives of ϕ are locally bounded near ∂D (vi) Lv ϕ(y) + f (y, v) ≤ 0 on S 0 \ ∂D for all v ∈ U (vii) Y (u) (τS ) ∈ ∂S a.s. on {τS < ∞} and lim ϕ(Y (u) (t)) = g(Y (u) (τS ))χ{τS 0; Y (ˆu) (t) ∈ / D} < ∞ a.s. for all y ∈ S, and (xi) the family {ϕ(Y (ˆu) (τ )); τ ∈ T } is uniformly integrable with respect to P y for all y ∈ D. Suppose u ˆ ∈ U. Then for all y ∈ S.

ϕ(y) = Φ(y)

Moreover, u∗ := u ˆ and τ ∗ := τD are optimal control and stopping times, respectively. Proof. The proof is a synthesis of the proofs of Theorems 2.2 and 3.1. For completeness we give some details: ¯ Choose u ∈ U (a) By Theorem 2.1 we may assume that ϕ ∈ C 2 (S 0 ) ∩ C(S). (u) and put Y (t) = Y (t). Let τ ≤ τS be a stopping time. Then by Dynkin’s formula (Theorem 1.24) we have, for m = 1, 2, . . . ,   τ ∧m $ & u y y L ϕ(Y (t))dt . (4.2.7) E ϕ(Y (τ ∧ m)) = ϕ(y) + E 0

Hence by (vii) and the Fatou lemma   τ ∧m −Lu ϕ(Y (t))dt + ϕ(Y (τ ∧ m)) ϕ(y) = lim E y m→∞ 0   τ y ≥E −Lu ϕ(Y (t))dt + g(Y (τ ))χ{τ θw − a for all w < w0 ,

(4.3.7)

1 − ρF (w) + (α − v)wF ′ (w) + β 2 w2 F ′′ (w) 2  + {F (w + γzw) − F (w) − F ′ (w)γzw}ν(dz) + v(w − K1 ) − K0 ≤ 0 R

for all w < w0 , v ∈ [0, m],

sup v∈[0,m]



(4.3.8)

 1 −ρF (w) + (α − v)wF ′ (w) + β 2 w2 F ′′ (w)+ {F (w + γzw) − F (w) 2 R  − F ′ (w)γzw}ν(dz) + v(w − K1 ) − K0 = 0 for all w > w0 . (4.3.9)

From (4.3.9) and (xi) of Theorem 4.2 we get the following candidate u ˆ for the optimal control: , + v=u ˆ(w) = Argmax v(w(1 − F ′ (w)) − K1 ) v∈[0,m]

m = 0

if F ′ (w) < 1 − (K1 /w) . if F ′ (w) > 1 − (K1 /w)

(4.3.10)

Let Fm (w) be the solution of (4.3.9) with v = m, i.e., the solution of 1 ′′ ′ (w) (w) + β 2 w2 Fm − ρFm (w) + (α − m)wFm 2  + {F (w + γzw) − F (w) − F ′ (w)γzw}ν(dz) = K0 + mK1 − mw. R

(4.3.11)

A solution of (4.3.11) is Fm (w) = C1 wλ1 + C2 wλ2 +

K0 + mK1 mw − , ρ+m−α ρ

(4.3.12)

where C1 , C2 are constants and λ1 > 0, λ2 < 0 are roots of the equation h(λ) = 0 with 1 h(λ) = −ρ+(α−m)λ+ β 2 λ(λ−1)+ 2



R

(4.3.13)

{(1+γz)λ −1−λγz}ν(dz). (4.3.14)

4.3 Applications

73

(Note that h(0) = −ρ < 0 and lim|λ|→∞ h(λ) = ∞.) The solution will depend on the relation between the parameters involved and we will not give a complete discussion, but only consider some special cases. Case 1 Let us assume that α ≤ ρ,

K1 = a = 0,

and

0 1 ⇐⇒ ρ + m > α.

(4.3.16)

C1 = 0

(4.3.17)

Let us try (guess) that and that the continuation region D = {(s, p, q); pq > w0 } is such that (see (4.3.10)) ′ Fm (w) < 1 for all w > w0 . (4.3.18) The intuitive motivation for trying this is the belief that it is optimal to use the maximal extraction intensity m all the time until closure, at least if θ is small enough. These guesses lead to the following candidate for the value function F (w):

θw if 0 ≤ w ≤ w0 F (w) = (4.3.19) K0 mw λ2 Fm (w) = C2 w + ρ+m−α − ρ if w > w0 . We now use continuity and differentiability at w = w0 to determine w0 and C2 : (Continuity) C2 w0λ2 +

(4.3.20)

(Differentiability)

(4.3.21)

mw0 K0 ρ+m−α − ρ = θw0 , m = θ. C2 λ2 w0λ2 −1 + ρ+m−α

Easy calculations show that the unique solution of (4.3.20) and (4.3.21) is w0 =

(−λ2 )K0 (ρ + m − α) (1 − λ2 )ρ[m − θ(ρ + m − α)]

(> 0

by (4.3.15))

(4.3.22)

and C2 =

[m − θ(ρ + m − α)]w01−λ2 (−λ2 )(ρ + m − α)

(> 0 by (4.3.15)).

(4.3.23)

It remains to verify that with these values of w0 and C2 the set D = {(s, p, q); pq > w0 } and the function F (w) given by (4.3.19) satisfies (4.3.6)– (4.3.9), as well as all the other conditions of Theorem 4.2.

74

4 Combined Optimal Stopping and Stochastic Control of Jump Diffusions

To verify (4.3.6) we have to check that (4.3.18) holds, i.e., that ′ (w) = C2 λ2 wλ2 −1 + Fm

m w0 .

Since λ2 < 0 and we have assumed α ≤ ρ (in (4.3.15)) this is clear. So (4.3.9) holds. If we substitute F (w) = θw in (4.3.8) we get −ρθw + (α − m)wθ + mw − K0 = w[m − θ(ρ + m − α)] − K0 . We know that this is 0 for w = w0 by (4.3.20) and (4.3.21). Hence it is less than 0 for w < w0 . So (4.3.8) holds. Condition (4.3.6) holds by definition of D and F . Finally, since F (w0 ) = θw0 , F ′ (w0 ) = θ, and ′′ (w) = C2 λ2 (λ2 − 1)wλ2 −2 > 0 F ′′ (w) = Fm

we must have F (w) > θw for w > w0 . Hence (4.3.7) holds. Similarly one can verify all the other conditions of Theorem 4.2. We have proved. Theorem 4.5. Suppose (4.3.15) holds. Then the optimal strategy (u∗ , τ ∗ ) for problems (4.1.3) and (4.1.4) is u∗ = m,

τ ∗ = inf{t > 0; Pt Qt ≤ w0 },

(4.3.24)

where w0 is given by (4.3.22). The corresponding value function is Φ(s, p, q) = e−ρs F (p·q), where F is given by (4.3.19) with λ2 < 0 as in (4.3.11) and C2 > 0 as in (4.3.23). For other values of the parameters it might be optimal not to produce at all but just wait for the best closing/sellout time. For example, we mention without proof the following cases (see Exercise 4.2): Case 2 Assume that θ = 1 and ρ ≤ α.

(4.3.25)

Then u∗ = 0 and Φ = ∞. Case 3 Assume that θ = 1,

ρ > α,

and K0 < ρa < K0 + ρK1 .

Then u∗ = 0 and τ ∗ = inf{t > 0; Pt Qt ≥ w1 }, for some w1 > 0.

(4.3.26)

4.4 Exercises

75

4.4 Exercises Exercise* 4.1. (a) Solve the following stochastic control problem ∗

Φ(s, x) = sup J (u) (s, x) = J (u ) (s, x), u(t)≥0

where J (u) (s, x) = E x



τS

e−δ(s+t)

0

 uγ (t) dt . γ

Here τS = τS (ω) = inf{t > 0; X(t) ≤ 0}

(the time of bankruptcy)

and dX(t) = (μX(t)−u(t))dt+σX(t)dB(t)+θX(t− )



¯ (dt, dz), zN

X0 = x > 0

R

with γ ∈ (0, 1), δ > 0, μ, σ = 0, θ constants, θz > −1 a.s. ν. The interpretation of this is the following. X(t) represents the total wealth at time t, u(t) = u(t, ω) ≥ 0 represents the chosen consumption rate (the control). We want to find the consumption rate u∗ (t) which maximizes the expected total discounted utility of the consumption up to the time of bankruptcy, τS . [Hint: Try a value function of the form φ(s, x) = Ke−δs xγ for a suitable value of the constant K.] (b) Consider the following combined stochastic control and optimal stopping problem ∗ ∗ Φ(s, x) = sup J (u,τ ) (s, x) = J (u ,τ ) (s, x), u,τ

where J

(u,τ )

(s, x) = E

x



0

τ

e

−δ(s+t) u

 (t) −δ(s+τ ) γ dt + λe X (τ ) γ

γ

with X(t) as in (a), λ > 0 a given constant. Now the supremum is taken over all Ft -adapted controls u(t) ≥ 0 and all Ft -stopping times τ ≤ τS . Let K be the constant found in (a). Show that: 1. If λ ≥ K then it is optimal to stop immediately. 2. If λ < K then it is never optimal to stop.

76

4 Combined Optimal Stopping and Stochastic Control of Jump Diffusions

Exercise 4.2. (a) Verify the statements in Cases 2 and 3 at the end of Sect. 4.3. (b) What happens in the cases Case 4: θ = 1, ρ > α, and ρa ≤ K0 ? Case 5: θ = 1, ρ > α, and K0 + ρK1 ≤ ρa? Exercise 4.3 (A Stochastic Linear Regulator Problem with Optimal Stopping). Consider the stochastic linear regulator problem in Exercise 3.5, with the additional option of stopping, i.e., solve the problem , + Φ(s, x) = inf J (u,τ ) (s, x); u ∈ U, τ ∈ T , where J

(u,τ )

(s, x) = E

s,x



0

τ

e

−ρ(s+t)

'

 ( −ρ(s+τ ) 2 X (t) + θu (t) dt + λe X (τ )χ{τ 0, θ > 0, λ > 0 constants), where the state process is     s+t s ; t ≥ 0, Y (0) = = y ∈ R2 Y (t) = X(t) x with dX(t) = dX (u) (t) = u(t)dt + σ dB(t) +



R

(a) Explain why Φ(s, x) ≤ λe−ρs x2

(dt, dz), zN

X(0) = x.

for all s, x.

(b) Prove that    Φ(s, x) ≤ e−ρs x2 + σ 2 + z 2 ν(dz)

for all s, x.

R

[Hint: What happens if we choose τ = ∞?] (c) Show that if 0 0.

R

Here r > 0, α and β > 0 are given constants, and we assume that z > −1 a.s. ν. Assume that at any time t the investor can choose an adapted and c` adl` ag consumption rate process c(t) = c(t, ω) ≥ 0, taken from the bank account. Moreover, the investor can at any time transfer money from one investment to another with a transaction cost which is proportional to the size of the transaction. Let X1 (t) and X2 (t) denote the amounts of money invested in the bank and in the stocks, respectively. Then the evolution equations for X1 (t) and X2 (t) are dX1 (t) = dX1x1 ,c,ξ (t) = (rX1 (t) − c(t))dt − (1 + λ)dξ1 (t) + (1 − μ)dξ2 (t),

X1 (0− ) = x1 ∈ R,

78

5 Singular Control for Jump Diffusions

dX2 (t) =

dX2x,c,ξ (t)

   = X2 (t ) α dt + β dB(t) + z N (dt, dz) −

R

+ dξ1 (t) − dξ2 (t),

X2 (0− ) = x2 ∈ R.

Here ξ = (ξ1 , ξ2 ), where ξ1 (t) and ξ2 (t) represent cumulative purchase and sale, respectively, of stocks up to time t. The constants λ ≥ 0, μ ∈ [0, 1] represent the constants of proportionality of the transaction costs. Define the solvency region S to be the set of states (x1 , x2 ) such that the net wealth is nonnegative, i.e., (see Fig. 5.1) S = {(x1 , x2 ) ∈ R2 , x1 + (1 + λ)x2 ≥ 0,

and x1 + (1 − μ)x2 ≥ 0}. (5.1.1)

We define the set A of admissible controls as the set of adapted consumption–investment policies (c, ξ) such that c = c(t) ≥ 0 is c`adl` ag and ξ = (ξ1 , ξ2 ) where each ξi (t), i = 1, 2 is right-continuous, nondecreasing, ξ(0− ) = 0. Define / S}. τS = inf{t > 0; (X1 (t), X2 (t)) ∈

x2 ∂2 S : x1 + (1 − μ)x2 = 0

S

x1

∂1 S : x1 + (1 + λ)x2 = 0 Fig. 5.1. The solvency region S

5.2 A General Formulation

The performance criterion is defined by  J c,ξ (s, x1 , x2 ) = E s,x1 ,x2

τS

e−δ(s+t)

0

 cγ (t) dt , γ

79

(5.1.2)

where δ > 0, γ ∈ (0, 1) are constants. We seek (c∗ , ξ ∗ ) ∈ A and Φ(s, x1 , x2 ) such that Φ(s, x1 , x2 ) = sup J c,ξ (s, x1 , x2 ) = J c

∗

,ξ ∗

(s, x1 , x2 ).

(5.1.3)

(c,ξ)∈A

This is an example of a singular stochastic control problem. It is called singular because the investment control measure dξ(t) is allowed to be singular with respect to Lebesgue measure dt. In fact, as we shall see, the optimal control measure dξ ∗ (t) turns out to be singular. We now give a general theory of singular control of jump diffusions and return to the above example afterward.

5.2 A General Formulation Let κ = [κij ] : Rk → Rk×p and θ = [θi ] : Rk →∈ Rp be given continuous functions. Suppose the state Y (t) = Y u,ξ (t) ∈ Rk is described by the equation dY (t) = b(Y (t), u(t))dt + σ(Y (t), u(t))dB(t)  (dt, dz) + κ(Y (t− ))dξ(t), + γ(Y (t− ), u(t− ), z)N Rℓ

Y (0− ) = y ∈ Rk . (5.2.1)

Here ξ(t) ∈ Rp is an adapted c` adl` ag finite variation process with increasing components and ξ(0− ) = 0. Since dξ(t) may be singular with respect to Lebesgue measure dt, we call ξ our singular control or our intervention control. The process u(t) is an adapted c` adl` ag process with values in a given set U (u(t)dt is our absolutely continuous control). Suppose we are given a performance functional J u,ξ (y) of the form  τS u,ξ y f (Y (t), u(t))dt + g(Y (τS )) · X{τS 0; Y u,ξ (t) ∈ S} ≤ ∞ is the time of bankruptcy,

where S ⊂ Rk is a given solvency set , assumed to satisfy S ⊂ S 0 .

80

5 Singular Control for Jump Diffusions

Let A be a given family of admissible controls (u, ξ), contained in the set of (u, ξ) such that a unique strong solution Y (t) of (5.2.1) exists and  τS |f (Y (t), u(t))|dt + |g(Y (τS ))| · X{τS inf 0≤s≤t Z(s). In either case the process ξ1 (t) = sup0≤s≤t Z1− (s) is constant in an interval [t, t + ǫ) for some ǫ > 0, by right continuity of Z(t). Therefore dξ(t) = 0 if Y (t) ∈ D, as claimed. Hence (Y (t), ξ(t)) defined by (5.2.16) solves the Skorohod equation (5.2.8)–(5.2.10) in the case when D and κ are given by (5.2.11) and (5.2.12). The proof in the general case follows by mapping neighborhoods of x ∈ ∂D into neighborhoods of the boundary point 0 of a half-space and applying the argument above. ⊓ ⊔

5.3 Application to Portfolio Optimization with Transaction Costs We now apply this theorem to Example 5.1. In this case our state process is ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ dt 1 0 ⎦ dB(t) dY (t) = ⎣dX1 (t)⎦ = ⎣rX1 (t) − c(t)⎦ dt + ⎣ 0 dX2 (t) αX2 (t) βX2 (t) ⎤ ⎡ ⎤⎡ ⎤ ⎡ 0 dξ1 (t) 0 0 ⎦. ⎦ + ⎣−(1 + λ) (1 − μ)⎦ ⎣ 0 +⎣  − dξ2 (t) 1 −1 X2 (t ) R z N (dt, dz) The generator of Y (t) when there are no interventions is Ac φ(y) =

∂φ 1 ∂φ ∂2φ ∂φ + (rx1 − c) + αx2 + β 2 x22 2 ∂s ∂x1 ∂x2 2 ∂x2  + , ∂φ φ(s, x1 , x2 + x2 z) − φ(s, x1 , x2 ) − x2 z + (s, x1 , x2 ) ν(dz). ∂x2 R

Or, if φ(s, x1 , x2 ) = e−δs ψ(x1 , x2 ) we have Ac φ(s, x1 , x2 ) = e−δs Ac0 ψ(x1 , x2 ), where ∂ψ ∂2ψ ∂ψ 1 Ac0 ψ(x1 , x2 ) = − δψ + (rx1 − c) + αx2 + β 2 x22 2 ∂x1 ∂x2 2 ∂x2  + , ∂ψ ψ(x1 , x2 + x2 z) − ψ(x1 , x2 ) − x2 z + (x1 , x2 ) ν(dz). ∂x2 R

86

5 Singular Control for Jump Diffusions

Here θ = g = 0,

f (y, u) = f (s, x1 , x2 , c) = e−δs

u(t) = c(t),

cγ . γ

Condition (ii) of Theorem 5.2 gets the form −(1 + λ)

∂ψ ∂ψ + ≤0 ∂x1 ∂x2

and (1 − μ)

∂ψ ∂ψ − ≤ 0. ∂x1 ∂x2

The nonintervention region D in (5.2.5) therefore becomes   ∂ψ ∂ψ ∂ψ ∂ψ + < 0 and (1 − μ) − γα − β 2 γ(1 − γ) − γν + 2 where 2

0



∞ −1

{(1 + z)γ − 1}ν(dz),

ν = ν((−1, ∞)) < ∞.

Then Φ ∈ C (S ) ∩ C(S) and

∂Φ !1/(γ−1) ∂x1 $ & ˆ ˆ is optimal and there exist θ1 , θ2 ∈ 0, π/2 with θˆ1 < θˆ2 such that √ D = {reiθ ; θˆ1 < θ < θˆ2 } (i = −1) c∗ (x1 , x2 ) =

is the nonintervention region and the optimal intervention strategy (portfolio) ξ ∗ (t) is the local time of the process (X1 (t), X2 (t)) reflected back into D in the direction parallel to ∂1 S at θ = θˆ1 and in the direction parallel to ∂2 S at θ = θˆ2 . See Fig. 5.2. For proofs and more details we refer to [FØS2]. Remark 5.6. For other applications of singular control theory of jump diffusions see [Ma].

5.4 Exercises x1 + (1 − μ)x2 = 0

87

x2 ∂ψ − (1 − μ) ∂x 1

∂ψ ∂x2

=0

θ = θˆ2

∂2 S (x1 , x2 ) D= θ = θˆ1

(x′1 , x′2 ) (x′1 , x′2 ) −(1 +

∂ψ λ) ∂x 1

+

∂ψ ∂x2

(x1 , x2 ) =0 x1 ∂1 S x1 + (1 + λ)x2 = 0

ℓ = purchase amount, m = sale amount (at transaction) x′1 = x1 − (1 + λ)ℓ + (1 − μ)m (new value of x1 after transaction) (new value of x2 after transaction) x′2 = x2 + ℓ − m Fig. 5.2. The optimal portfolio ξ ∗ (t)

5.4 Exercises Exercise* 5.1 (Optimal Dividend Policy Under Proportional Transaction Costs). Suppose the cash flow X(t) = X (ξ) (t) of a firm at time t is given by (with α, σ, β, λ > 0 constants)  (dt, dz) − (1 + λ)dξ(t), X(0− ) = x > 0, dX(t) = α dt + σ dB(t) + β z N R

where ξ(t) is an increasing, adapted c` adl` ag process representing the total dividend taken out up to time t (our control process). We assume that βz ≤ 0 for a.a. z (ν). Let τS = inf{t > 0; X (ξ) (t) ≤ 0} be the time of bankruptcy of the firm and let   τS −ρ(s+t) ξ s,x J (s, x) = E e dξ(t) ,

ρ > 0 constant

0

be the expected total discounted amount taken out up to bankruptcy time.

88

5 Singular Control for Jump Diffusions

Find Φ(s, x) and a dividend policy ξ ∗ such that ∗

Φ(s, x) = sup J (ξ) (s, x) = J ξ (s, x). ξ

Exercise* 5.2. Let Φ(s, x1 , x2 ) be the value function of the optimal consumption problem (5.1.2) with proportional transaction costs and let Φ0 (s, x1 , x2 ) = Ke−δs (x1 +x2 )γ be the corresponding value function when there are no transaction costs, i.e., μ = λ = 0 (Example 3.2). Use Theorem 5.2a to prove that Φ(s, x1 , x2 ) ≤ Ke−δs (x1 + x2 )γ . Exercise 5.3 (Optimal Harvesting). Suppose the size X(t) at time t of a certain fish population is modeled by a geometric L´evy process, i.e., dX(t) = dX

(ξ)

−



(t) = X(t ) μ dt +

X(0− ) = x > 0,



R

 z N (dt, dz) − dξ(t),

t > 0,

where μ > 0 is a constant, z > −1 a.s. ν(dz) and ξ(t) is an increasing adapted process giving the amount harvested from the population from time 0 up to time t. We assume that ξ(t) is right-continuous. Consider the optimal harvesting problem Φ(s, x) = sup J (ξ) (s, x), ξ

where J

(ξ)

(s, x) = E

s,x



τs

θe

0

−ρ(s+t)

 dξ(t) ,

with θ > 0, ρ > 0 constants and τS = inf{t > 0; X (ξ) (t) ≤ 0}

(extinction time).

If we interpret θ as the price per unit harvested, then J (ξ) (s, x) represents the expected total discounted value of the harvested amount up to extinction time. (a) Write down the integrovariational inequalities (i), (ii), (vi), and (ix) of Theorem 5.2 in this case, with the state process 

 s+t Y (t) = , X(t)

t ≥ 0,

  s Y (0) = y = ∈ R+ × R+ . x

5.4 Exercises

89

(b) Suppose μ ≤ ρ. Show that in this case it is optimal to harvest all the population immediately, i.e., it is optimal to choose the harvesting strategy ξˆ defined by ˆ = x for all t ≥ 0 ξ(t) (sometimes called the “take the money and run” strategy). This gives the value function Φ(s, x) = θe−ρs x. (c) Suppose μ > ρ. Show that in this case Φ(s, x) = ∞.

6 Impulse Control of Jump Diffusions

6.1 A General Formulation and a Verification Theorem Suppose that – if there are no interventions – the state Y (t) ∈ Rk of the system we consider is a jump diffusion of the form  (dt, dz), dY (t) = b(Y (t))dt + σ(Y (t))dB(t) + γ(Y (t− ), z)N (6.1.1) Rℓ

Y (0) = y ∈ Rk ,

where b : Rk → Rk , σ : Rk → Rk×m , and γ : Rk × Rℓ → Rk×ℓ are given functions satisfying the conditions for the existence and uniqueness of a solution Y (t) (see Theorem 1.19). The generator A of Y (t) is Aφ(y) =

k 

bi (y)

i=1

+

ℓ   j=1

R

k 1  ∂2φ ∂φ + (σσ T )ij (y) ∂yi 2 i,j=1 ∂yi ∂yj

{φ(y + γ (j) (y, zj )) − φ(y) − ∇φ(y) · γ (j) (y, zj )}νj (dzj ).

Now suppose that at any time t and any state y we are free to intervene and give the system an impulse ζ ∈ Z ⊂ Rp , where Z is a given set (the set of admissible impulse values). Suppose the result of giving the impulse ζ when the state is y is that the state jumps immediately from y = Y (t− ) to Y (t) = Γ (y, ζ) ∈ Rk , where Γ : Rk × Z → Rk is a given function. An impulse control for this system is a double (possibly finite) sequence v = (τ1 , τ2 , . . . , τj , . . . ; ζ1 , ζ2 , . . . , ζj , . . .)j≤M ,

M ≤ ∞,

where 0 ≤ τ1 ≤ τ2 ≤ · · · are Ft -stopping times (the intervention times) and ζ1 , ζ2 , . . . are the corresponding impulses at these times. We assume that ζj is Fτj -measurable for all j.

92

6 Impulse Control of Jump Diffusions

If v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) is an impulse control, the corresponding state process Y (v) (t) is defined by Y (v) (0− ) = y Y

(v)

and Y (v) (t) = Y (t), ˇ (v)

(τj ) = Γ (Y

(τj− ), ζj ),

0 < t ≤ τ1 ,

j = 1, 2, . . .

(6.1.2) (6.1.3)

(If τ1 = 0 we put Y (v) (τ1 ) = Γ (Y (v) (0− ), ζ1 ) = Γ (y, ζ1 ).) dY (v) (t) = b(Y (v) (t))dt + σ(Y (v) (t))dB(t)  (dt, dz) for τj < t < τj+1 ∧ τ ∗ . + γ(Y (v) (t), z)N

(6.1.4)

Rℓ

If τj+1 = τj , then Y (v) (t) jumps from Yˇ (v) (τj− ) to Γ (Γ (Yˇ (v) (τj− ), ζj ), ζj+1 ), where

Yˇ (v) (τj− ) = Y (v) (τj− ) + ΔN Y (τj ),

(6.1.5)

ΔN Y (v) (t) is as in (5.2.2) the jump of Y (v) stemming from the jump of the random measure N (t, ·) only and τ ∗ = τ ∗ (ω) = lim (inf{t > 0; |Y (v) (t)| ≥ R}) ≤ ∞ R→∞

(6.1.6)

is the explosion time of Y (v) (t). Note that here we must distinguish between the (possible) jump of Y (v) (τj ) stemming from the random measure N , denoted by ΔN Y (v) (τj ) and the jump caused by the intervention v, given by Δv Y (v) (τj ) := Γ (Yˇ (v) (τj− ), ζ) − Yˇ (v) (τj− ).

(6.1.7)

Let S ⊂ Rk be a fixed open set (the solvency region). Define τS = inf{t ∈ (0, τ ∗ ); Y (v) (t) ∈ S}.

(6.1.8)

Suppose we are given a continuous profit function f : S → R and a continuous bequest function g : Rk → R. Moreover, suppose the profit/utility of making an intervention with impulse ζ ∈ Z when the state is y is K(y, ζ), where K : S × Z → R is a given continuous function. We assume we are given a set V of admissible impulse controls which is included in the set of v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) such that a unique solution Y (v) of (6.1.2)–(6.1.4) exists and

and (if M = ∞)

τ ∗ = ∞ a.s.

(6.1.9)

lim τj = τS .

(6.1.10)

j→∞

We also assume that  τS      (v) (Y (t)) Ey f  dt < ∞ 0

for all y ∈ Rk , v ∈ V,

(6.1.11)

6.1 A General Formulation and a Verification Theorem

and

) * E g − (Y (v) (τS ))X{τS 0 from X(t) by applying the transaction cost k(ζ) = c + λζ,

(6.2.2)

where c > 0, λ ≥ 0 are constants. The constant c is called the fixed part and the quantity λζ is called the proportional part, respectively, of the transaction cost. The resulting cash flow X (v) (t) is given by X (v) (t) = X(t) if 0 ≤ t < τ1 ,

(6.2.3)

ˇ (v) (τ − ) − (1 + λ)ζj − c, (6.2.4) X (v) (τj ) = Γ (X (v) (τj− ) + ΔN X(τj ), ζj ) = X j and dX

(v)

(t) = μ dt + σ dB(t) + θ



R

Put

(dt, dz) if τj ≤ t < τj+1 . zN

τS = inf{t > 0; X (v) (t) ≤ 0} and

(time of bankruptcy)

⎡

J (v) (s, x) = E s,x ⎣



τj ≤τS

⎤

e−ρ(s+τj ) ζj ⎦ ,

(6.2.5)

(6.2.6)

(6.2.7)

where ρ > 0 is constant (the discounting exponent). We seek Φ(s, x) and v ∗ = (τ1∗ , τ2∗ , . . . ; ζ1∗ , ζ2∗ , . . .) ∈ V such that ∗

Φ(s, x) = sup J (v) (s, x) = J (v ) (s, x),

(6.2.8)

v∈V

where V is the set of impulse controls s.t. X (v) (t) ≥ 0 for all t ≤ τS . This is a problem of the type (6.1.14), with     s+t s (v) (v) − Y (t) = = y, , t ≥ 0, Y (0 ) = x X (v) (t)   s Γ (y, ζ) = Γ (s, x, ζ) = , x − c − (1 + λ)ζ K(y, ζ) = K(s, x, ζ) = e−ρs ζ,

f = g = 0,

and S = {(s, x); x > 0}. As a candidate for the value function Φ we try φ(s, x) = e−ρs ψ(x).

(6.2.9)

6.2 Examples

97

Then 

x−c Mψ(x) = sup ψ(x − c − (1 + λ)ζ) + ζ, 0 < ζ < 1+λ



.

We now guess that the continuation region has the form D = {(s, x); 0 < x < x∗ } for some x∗ > 0.

(6.2.10)

Then (x) of Theorem 6.2 gives 1 −ρψ(x) + μψ (x) + σ 2 ψ ′′ (x) + 2 ′



R

{ψ(x + θz) − ψ(x) − ψ ′ (x)θz}ν(dz) = 0.

To solve this equation we try a function of the form ψ(x) = erx for some constant r ∈ R. Then r must solve the equation  1 2 2 h(r) := −ρ + μr + σ r + {erθz − 1 − rθz}ν(dz) = 0. 2 R

(6.2.11)

Since h(0) = −ρ < 0 and lim|r|→∞ h(r) = ∞, we see that there exist two solutions r1 , r2 of h(r) = 0 such that r2 < 0 < r1 . Moreover, since erθz − 1 − rθz ≥ 0 for all r, z we have |r2 | > r1 . With such a choice of r1 , r2 we try ψ(x) = A1 er1 x + A2 er2 x ,

Ai constants.

Since ψ(0) = 0

we have A1 + A2 = 0

so we write A1 = A = −A2 > 0 and ' ( ψ(x) = A er1 x − er2 x ,

Define

0 < x < x∗ .

( ' for all x > 0. ψ0 (x) = A er1 x − er2 x

To study Mψ we first consider

g(ζ) := ψ0 (x − c − (1 + λ)ζ) + ζ,

ζ > 0.

(6.2.12)

98

6 Impulse Control of Jump Diffusions

ˆ The first-order condition for a maximum point ζˆ = ζ(x) for g(ζ) is that ˆ = ψ0′ (x − c − (1 + λ)ζ)

1 . 1+λ

Now ψ0′ (x) > 0 for all x and ψ0′′ (x) < 0 iff x < x ˜ :=

2(ln |r2 |−ln r1 ) . r1 −r2

Therefore the equation ψ0′ (x) = 1/(1 + λ) has exactly two solutions x = x and x = x ¯ where ˜ 0 and λ ≥ 0 are constants. Then the expected total discounted cost associated to a given impulse control is  N ∞  −ρ(s+τk ) (v) x −ρ(s+t) (v) 2 e (c + λζk ) . J (s, x) = E e (X (t)) dt + 0

k=1

(6.2.21)

We seek Φ(s, x) and v ∗ = (τ1∗ , τ2∗ , . . . ; ζ1∗ , ζ2∗ , . . .) such that ∗

Φ(s, x) = inf J (v) (s, x) = J (v ) (s, x). v

(6.2.22)

This is an impulse control problem of the type described above, except that it is a minimum problem rather than a maximum problem. Theorem 6.2 still applies, with the corresponding changes. Note that it is not optimal to move X(t) downward if X(t) is already below 0. Hence we may restrict ourselves to consider impulse controls v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) such that τk  j=1

ζj ≤ X(τk ) for all k.

(6.2.23)

We let V denote the set of such impulse controls. We guess that the optimal strategy is to wait until the level of X(t) reaches an (unknown) value x∗ > 0. At this time, τ1 , we intervene and give X(t) an impulse ζ1 , which brings it down to a lower value xˆ > 0. Then we do nothing until the next time, τ2 , that X(t) reaches the level x∗ , etc. This suggests that the continuation region D in Theorem 6.2 has the form D = {(s, x), x < x∗ }.

(6.2.24)

See Fig. 6.3. Let us try a value function ϕ of the form ϕ(s, x) = e−ρs ψ(x), where ψ remains to be determined.

(6.2.25)

6.2 Examples

101

X(t)

x∗ ζ1∗

ζ2∗

x ˆ

t t = τ1∗

t = τ2∗

Fig. 6.3. The optimal impulse control of Example 6.5

Condition (x) of Theorem 6.2 gives that for x < x∗ we should have    1 ′′ −ρs ′ Aϕ + f = e −ρψ(x) + ψ (x) + {ψ(x + z) − ψ(x) − zψ (x)}ν(dz) 2 R + e−ρs x2 = 0.

So for x < x∗ we let ψ be a solution h(x) of the equation  1 {h(x + z) − h(x) − zh′ (x)}ν(dz) + h′′ (x) − ρh(x) + x2 = 0. 2 R We see that any function h(x) of the form h(x) = C1 e

r1 x

+ C2 e

r2 x

1+ 1 + x2 + ρ



z 2 ν(dz) , ρ2

R

(6.2.26)

(6.2.27)

where C1 , C2 are arbitrary constants, is a solution of (6.2.26), provided that r1 > 0, r2 < 0 are roots of the equation  1 K(r) := {erz − 1 − rz}ν(dz) + r2 − ρ = 0. 2 R Note that if we make no interventions at all, then the cost is   ∞ e−ρt (X(t))2 dt J (v) (s, x) = e−ρs E x 0    ∞ 1 2 b = e−ρs x + 2 , e−ρt (x2 + tb)dt = e−ρs ρ ρ 0

(6.2.28)

102

6 Impulse Control of Jump Diffusions

where b = 1 +



R

z 2 ν(dz). Hence we must have b 1 2 x + 2 ρ ρ

0 ≤ ψ(x) ≤

for all x.

(6.2.29)

Comparing this with (6.2.27) we see that we must have C2 = 0. Hence C1 ≤ 0. So we put ψ(x) = ψ0 (x) :=

b 1 2 x + 2 − aer1 x ρ ρ

for x ≤ x∗ ,

(6.2.30)

where a = −C1 remains to be determined. We guess that a > 0. To determine a we first find ψ for x > x∗ and then require ψ to be C 1 at x = x∗ . By (ii) and (6.2.24) we know that for x > x∗ we have ψ(x) = Mψ(x) := inf{ψ(x − ζ) + c + λζ, ζ > 0}.

(6.2.31)

ˆ The first-order condition for a minimum ζˆ = ζ(x) of the function G(ζ) := ψ(x − ζ) + c + λζ, is

ζ>0

ˆ = λ. ψ ′ (x − ζ)

Suppose there is a unique point xˆ ∈ (0, x∗ ) such that ψ ′ (ˆ x) = λ. Then

(6.2.32)

ˆ ˆ x ˆ = x − ζ(x), i.e., ζ(x) =x−x ˆ

and from (6.2.31) we deduce that ψ(x) = ψ0 (ˆ x) + c + λ(x − x ˆ) for x ≥ x∗ . In particular, ψ ′ (x∗ ) = λ

(6.2.33)

ψ(x∗ ) = ψ0 (ˆ x) + c + λ(x∗ − x ˆ).

(6.2.34)

and To summarize we put ⎧ ⎨ 1 x2 + b − aer1 x , for x ≤ x∗ , ρ2 ψ(x) = ρ ⎩ x) + c + λ(x − x ˆ), for x > x∗ , ψ0 (ˆ

(6.2.35)

6.3 Exercises

103

where x ˆ, x∗ , and a are determined by (6.2.32)–(6.2.34), i.e., 2 x ˆ − λ (i.e., ψ ′ (ˆ x) = λ), ρ ∗ 2 ar1 er1 x = x∗ − λ (i.e., ψ ′ (x∗ ) = λ), ρ 1 ∗ 2 r1 x ˆ x)2 ) − c − λ(x∗ − x ˆ). − ae = ((x ) − (ˆ ρ ar1 er1 xˆ =

∗

aer1 x

(6.2.36) (6.2.37) (6.2.38)

One can now prove (see [Ø2, Theorem 2.5]). For each c > 0 there exists a = a∗ (c) > 0, x ˆ = x ˆ(c) > 0, and x∗ = x (c) > x ˆ such that (6.2.36)–(6.2.38) hold. With this choice of a, x ˆ, and x∗ , −ρs the function ϕ(s, x) = e ψ(x) with ψ given by (6.2.35) coincides with the value function Φ defined in (6.2.22). Moreover, the optimal impulse control v ∗ = (τ1∗ , τ2∗ , . . . ; ζ1∗ , ζ2∗ , . . .) is to do nothing while X(t) < x∗ , then move X(t) ˆ) at the first time τ1∗ when X(t) ˆ (i.e., apply ζ1∗ = x∗ − x from x∗ down to x ∗ reaches a value ≥ x , then wait until the next time, τ2∗ , X(t) again reaches the value x∗ , etc. ∗

Remark 6.6. In [Ø2] this result is used to study how the value function Φ(s, x) = Φc (s, x) depends on the fixed part c > 0 of the intervention cost. It is proved that the function c → Φc (s, x) is continuous but not differentiable at c = 0. In fact, we have ∂ Φc (s, x) → ∞ as c → 0+ . ∂c Subsequently this high c-sensitivity of the value function for c close to 0 was proved for other processes as well. See [ØUZ]. Remark 6.7. For applications of impulse control theory in inventory control see, e.g., [S, S2] and the references therein.

6.3 Exercises Exercise* 6.1. Solve the impulse control problem ∗

Φ(s, x) = inf J (v) (s, x) = J (v ) (s, x), v

where J

(v)

(s, x) = E



0

∞

e

−ρ(s+t)

(X

(v)

2

(t)) dt +

N 

k=1

e

−ρ(s+τk )



(c + λ|ζk |) .

104

6 Impulse Control of Jump Diffusions

The inf is taken over all impulse controls v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) with ζi ∈ R and the corresponding process X (v) (t) is given by  ˜ (dt, dz), τk < t < τk+1 , dX (v) (t) = dB(t) + θ(X (v) (t), z)N R

ˇ (v) (τ − ) + ζk+1 , X (v) (τk+1 ) = X k+1

k = 0, 1, 2, . . . ,

where B(0) = 0, x ∈ R, and we assume that there exists ξ > 0 such that θ(x, z) = 0 for a.a. z if |x| < ξ, θ(x, z) ≥ 0 if x ≥ ξ and θ(x, z) ≤ 0 if x ≤ −ξ

for a.a. z,

(6.3.1) (6.3.2)

Exercise* 6.2 (Optimal Stream of Dividends with Transaction Costs from a Geometric L´ evy Process). For v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) with ζi ∈ R+ define X (v) (t) by dX (v) (t)

= μX (v) (t)dt +σX (v) (t)dB(t) ˜ (ds, dz), τi ≤ t ≤ τi+1 , + θX (v) (t− ) z N R

ˇ (v) (τ − ) − (1 + λ)ζi+1 − c, X (v) (τi+1 ) = X i+1 (v) − X (0 ) = x > 0,

i = 0, 1, 2, . . . ,

where μ, σ = 0, θ, λ ≥ 0, and c > 0 are constants (see (6.1.5)), −1 ≤ θz ≤ 0 a.s. ν. Find Φ and v ∗ such that ∗

Φ(s, x) = sup J (v) (s, x) = J (v ) (s, x). v

Here J

(v)

(s, x) = E

x





τk 0 constant)

is the expected discounted total dividend up to time τS , where τS = τS (ω) = inf{t > 0; X (v) (t) ≤ 0} is the time of bankruptcy. (See also Exercise 7.2.) Exercise* 6.3 (Optimal Forest Management (Inspired Y. Willassen [W])). Suppose the biomass of a forest at time t is given by  (t, dz), X(t) = x + μt + σB(t) + θ z N

by

R

where μ > 0, σ > 0, θ > 0 are constants and we assume that z ≤ 0 a.s. ν. At times 0 ≤ τ1 < τ2 < · · · we decide to cut down the forest and replant it, with the cost ˇ − ) = X(τ − ) + ΔN X(τk ), ˇ − ), with X(τ c + λX(τ k k k

6.3 Exercises

105

where c > 0 and λ ∈ [0, 1) are constants and ΔN X(t) is the (possible) jump in X at t coming from the jump in N (t, ·) only, not from the intervention. Find the sequence of stopping times v = (τ1 , τ2 , . . .) which maximizes the expected total discounted net profit J (v) (s, x) given by ∞  − − (v) x −ρ(s+τk ) ˇ ˇ e (X(τ ) − c − λX(τ )) , J (s, x) = E k

k=1

where ρ > 0 is a given discounting exponent.

k

7 Approximating Impulse Control by Iterated Optimal Stopping

7.1 Iterative Scheme In general it is not possible to reduce impulse control to optimal stopping, because the choice of the first intervention time τ1 and the first impulse ζ1 will influence the next and so on. However, if we allow only (up to) a fixed finite number n of interventions, then the corresponding impulse control problem can be solved by solving iteratively n optimal stopping problems. Moreover, if we restrict the number of interventions to (at most) n in a given impulse control problem, then the value function of this restricted problem will converge to the value function of the original problem as n → ∞. Thus it is possible to reduce a given impulse control problem to a sequence of iterated optimal stopping problems. This is useful both for theoretical purposes and numerical applications. We now make this more precise. Using the notation of Chap. 6 consider the impulse control problem ∗

Φ(y) = sup{J (v) (y); v ∈ V} = J (v ) (y), (v)

= inf{t > 0; Y (v) (t) ∈ S},

(v)

(v)

where, with τS = τS J (v) (y) =E

y



0

τS

f (Y

(t))dt + g(Y

(τS ))χ{τS 0; Y (0) (t) ∈ Dn }

(7.1.37)

ζˆ1 = ζ¯1 (Y (0) (ˆ τ1− )),

(7.1.38)

Define and

where ζ¯1 = ζ¯1 (y) is a Borel measurable function such that Mϕn−1 (y) = ϕn−1 (Γ (y, ζ¯1 )) + K(y, ζ¯1 ),

y ∈ S.

(7.1.39)

Then if (ˆ τ1 , . . . , τˆj ; ζˆ1 , . . . , ζˆj ) is defined, put

and

τˆj+1 = inf{t > τˆj ; Y (j) (t) ∈ Dn−j }

(7.1.40)

− τj+1 )), ζˆj+1 = ζ¯j+1 (Y (j) (ˆ

(7.1.41)

where ζ¯j+1 = ζ¯j+1 (y) is a Borel measurable function such that Mϕn−(j+1) (y) = ϕn−(j+1) (Γ (y, ζ¯j+1 )) + K(y, ζ¯j+1 ),

y ∈ S, j + 1 ≤ n. (7.1.42) As before Y (j) (t) denotes the result of applying the impulse control vˆj = (ˆ τ1 , . . . , τˆj ; ζˆ1 , . . . , ζˆj ) to Y . Corollary 7.6. Assume that g ≥ 0 and that f, g, Mϕn ∈ P(Rk ) for n = 0, 1, 2, . . ., where ϕn is as defined in (7.1.11) and (7.1.12). Then ϕn (y) ↑ Φ(y) as n → ∞,

for all y.

Corollary 7.7. Suppose g ≥ 0 and f, g, Mϕn ∈ P(Rk ) for n = 0, 1, 2, . . . Then Φ is a solution of the following nonlinear optimal stopping problem   τ f (Y (t))dt + MΦ(Y (τ ))χ{τ 0

then we are led to the following candidate for ψ1 (see Example 6.5)

1 2 x + ρb2 − a1 exp(rx), x < x∗1 , ψ1 (x) = ρb x ≥ x∗1 , ρ2 + c,

(7.2.8)

where a1 is a constant to determine and r = r1 > 0 is a root of the equation

7.2 Examples

119

ψ1 (x)

x x∗1

x ˆ1

Fig. 7.2. The graph of ψ1 (x)

K(r) :=



1 {erz − 1 − rz}ν(dz) + r2 − ρ = 0. 2 R

See Fig. 7.2. If we require ψ1 to be continuous and differentiable at x = x∗1 , we get the following two equations to determine x∗1 and a1 : 1 ∗ 2 b b (x ) + 2 − a1 exp(rx∗1 ) = 2 + c, ρ 1 ρ ρ

(7.2.9)

2 ∗ x − a1 r exp(rx∗1 ) = 0. ρ 1

(7.2.10)

Combining these two equations we get x∗1 = and

! 1 1 + 1 + cρr2 r

(7.2.11)

2x∗1 exp(−rx∗1 ). ρr

(7.2.12)

a1 =

If we choose these values of x∗1 and a1 , we can now verify that, under some additional assumptions on ν, the candidate ψ1 given by (7.2.8) satisfies all the conditions of the verification theorem and we conclude that ψ1 = Ψ 1 . We now repeat the procedure to find Ψ2 . First note that MΨ1 (x) = inf{Ψ1 (x − ζ) + c; ζ > 0} = Ψ1 (x ∧ x ˆ1 ) + c b 1 ˆ1 )2 + 2 + c − a1 exp(r(x ∧ x ˆ1 )), = (x ∧ x ρ ρ

(7.2.13)

120

7 Approximating Impulse Control by Iterated Optimal Stopping

where x ˆ1 ∈ Argmin Ψ1 (x). Hence Ψ2 (x) = inf E τ ≥0

x



τ

e−ρt X 2 (t)dt

0 −ρτ

+e



 1 b 2 (X(τ ) ∧ x ˆ1 ) + 2 + c − a1 exp(r(X(τ ) ∧ x ˆ1 )) . ρ ρ

The same procedure as above leads us to the candidate ⎧1 b ⎪ x < x∗2 , ⎨ x2 + 2 − a2 exp(rx), ρ ρ ψ2 (x) = b ⎪ ⎩1x ˆ2 + + c − a1 exp(rˆ x1 ), x ≥ x∗2 , ρ 1 ρ2

where x∗2 and a2 are given by   0 1 ∗ 2 2 x1 ))) , x1 + ρ(c − a1 exp(rˆ x2 = 1 + 1 + r (ˆ r a2 =

2x∗2 exp(−rx∗2 ). ρr

(7.2.14)

(7.2.15) (7.2.16)

If we choose these values of x∗2 and a2 > 0 we can verify that the candidate ψ2 given by (7.2.14) coincides with Ψ2 . Note that x∗2 < x∗1 and a2 > a1 . Continuing this inductively we get a sequence of functions ψn of the form ⎧1 b ⎪ x < x∗n , ⎨ x2 + 2 − an exp(rx), ρ ρ (7.2.17) ψn (x) = b ⎪ ⎩1x ˆ2n−1 + 2 + c − an−1 exp(rˆ xn−1 ), x ≥ x∗n , ρ ρ

where x∗n and an are given by   0 1 x∗n = xn−1 ))) x2n−1 + ρ(c − an−1 exp(rˆ 1 + 1 + r2 (ˆ r

(7.2.18)

with x ˆn−1 ∈ Argmin Ψn−1 (x), and an =

2x∗n exp(−rx∗n ). ρr

(7.2.19)

We find that x∗n < x∗n−1 and an > an−1 . Using the verification theorem we conclude that ψn = Ψn . In the limiting case when there is no bound on the number of interventions the corresponding value function will be

7.2 Examples

⎧ 1 2 b ⎪ ⎪ ⎨ ρ x + ρ2 − a exp(rx), Ψ (x) = ⎪1 2 b ⎪ ⎩ x ˆ + 2 + c − a exp(rˆ x), ρ ρ

121

x < x∗ , (7.2.20) x ≥ x∗ ,

where x∗ and a solve the coupled system of equations x∗ =

! 1 1 + 1 + r2 (ˆ x2 + ρ(c − a exp(rˆ x))) , r 2x∗ exp(−rx∗ ), a= ρr

(7.2.21) (7.2.22)

where x ˆ ∈ Argmin Ψ (x). It is easy to see that this system has a unique solution x∗ > 0, a > 0, and that x∗ < x∗n and a > an for all n. The situation in the case n = 3 is shown in Fig. 7.3. Note that with only three interventions allowed the optimal strategy is first to wait until the first time τ1∗ when X(t) ≥ x∗3 , then move X(t) down to x ˆ2 , next wait until the first time τ2∗ > τ1∗ when X(t) ≥ x∗2 , then move X(t) down to x ˆ1 and finally wait until the first time t = τ3∗ > τ2∗ when ∗ X(t) ≥ x1 before making the last intervention, which is moving X(t) down to x ˆ0 = 0. X(t)

x∗1 x∗2 x∗3 D1

x∗ ζ1∗

D3

ζ2∗

D2

D

x ˆ2 x ˆ1

t

x ˆ0 = 0 t = τ1∗

t = τ2∗

t = τ3∗

Fig. 7.3. The optimal impulse control for Ψ3 (Example 7.8)

122

7 Approximating Impulse Control by Iterated Optimal Stopping

7.3 Exercises Exercise* 7.1 (Optimal Forest Management Revisited). Using the notation of Exercise 6.3 let   Φ(x) = sup J (v) (x); v = (τ1 , τ2 , . . .)

be the value function when there are no restrictions on the number of interventions. For n = 1, 2, . . . let   Φn (x) = sup J (v) (x); v = (τ1 , τ2 , . . . , τk ); k ≤ n

be the value function when up to n interventions are allowed. Use Theorem 7.2 to find Φ1 and Φ2 .

Exercise* 7.2 (Optimal Stream of Dividends with Transaction Costs from a Geometric L´ evy Process). This is an addition to Exercise 6.2. Suppose that we at times 0 ≤ τ1 ≤ τ2 ≤ · · · decide to take out dividends of sizes ζ1 , ζ2 , . . . ∈ (0, ∞) from an economic quantity growing like a geometric L´evy process. If we let X (v) (t) denote the size at time t of this quantity when the dividend policy v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) is applied, we assume that X (v) (t) is described by (see (6.2.4)) dX (v) (t) = μX (v) (t)dt + σX (v) (t)dB(t) + θX (v) (t− ) ˇ (v) (τ − ) − (1 + λ)ζi+1 − c, X (v) (τi+1 ) = X i+1



˜ (dt, dz), zN

τi ≤ t < τi+1 ,

i = 0, 1, 2, . . .

(see (6.1.5)),

R

where μ, σ = 0, θ, λ ≥ 0, and c > 0 are constants, −1 ≤ θz ≤ 0, a.s. (ν). Let   Φ(x) = sup J (v) (x); v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .)

and

  Φn (x) = sup J (v) (x); v = (τ1 , τ2 , . . . , τk ; ζ1 , ζ2 , . . . , ζk ); k ≤ n

be the value function with no restrictions on the number of interventions and with at most n interventions, respectively, where   (ρ > 0 constant) e−ρτk ζk J (v) (x) = E x τk 0; X (v) (t) ≤ 0 is the time of bankruptcy. Show that

Φ(x) = Φn (x) = Φ1 (x)

for all n.

Thus in this case we achieve the optimal result with just one intervention.

8 Combined Stochastic Control and Impulse Control of Jump Diffusions

8.1 A Verification Theorem Consider the general situation in Chap. 6, except that now we assume that we, in addition, are free at any state y ∈ Rk to choose a Markov control u(y) ∈ U , where U is a given closed convex set in Rp . Let U be a given family of such Markov controls u. If, as before, v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) ∈ V denotes a given impulse control we call w := (u, v) a combined control . If w = (u, v) is applied, we assume that the corresponding state Y (t) = Y (w) (t) at time t is given by (see (3.1.1) and (6.1.2)–(6.1.5)) Y (0− ) = y, dY (t) = b(Y (t), u(t))dt + σ(Y (t), u(t))dB(t)  ˜ (dt, dz), τj < t < τj+1 , + γ(Y (t− ), u(t− ), z)N

(8.1.1)

Rℓ

− Y (τj+1 ) = Γ (Yˇ (τj+1 ), ζj+1 ),

j = 0, 1, 2, . . .

(8.1.2)

where u(t) = u(Y (t)) and b : Rk × U → Rk , σ : Rk × U → Rk×m , and γ : Rk × U × Rℓ → Rk×ℓ are given continuous functions, τ0 = 0. As before we let our “universe” S be a fixed Borel set in Rk such that S ⊂ S 0 and we define τ ∗ = lim inf{t > 0; |Y (w) (t)| ≥ R} ≤ ∞

(8.1.3)

τS = inf{t ∈ (0, τ ∗ (ω)); Y (w) (t, ω) ∈ S}.

(8.1.4)

R→∞

and (w)

∗

∗

If Y (t, ω) ∈ S for all t < τ we put τS = τ . We assume that we are given a set W = U × V of admissible combined controls w = (u, v) which is included in the set of combined controls w = (u, v) such that a unique strong solution Y (w) (t) of (8.1.1) and (8.1.2) exists and τ∗ = ∞

and

lim τj = τS a.s. Qy for all y ∈ Rk .

j→∞

(8.1.5)

124

8 Combined Stochastic Control and Impulse Control of Jump Diffusions

Define the performance or total expected profit/utility of w = (u, v) ∈ W, v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .), by  τS

J (w) (y) = E y

0

+

f (Y (w) (t), u(t))dt + g(Y (w) (τS ))X{τS Mφ(y)}

(the continuation region).

(8.1.10)

Suppose that Y (w) (t) spends 0 time on ∂D a.s., i.e.,  τS y (w) (iii) E X∂D (Y (t))dt = 0 for all y ∈ S, w ∈ W 0

and suppose that (iv) ∂D is a Lipschitz surface (v) φ ∈ C 2 (S 0 \ ∂D) and the second-order derivatives of φ are locally bounded near ∂D (vi) Lα φ(y) + f (y, α) ≤ 0 for all α ∈ U , y ∈ S 0 \ ∂D (vii) Y (w) (τS ) ∈ ∂S a.s. on {τS < ∞} and φ(Y (w) (t)) → g(Y (w) (τS )) · X{τS τˆk ; Y (k) (t) ∈ / D}, − ¯ (k) (ˆ ζˆk+1 = ζ(Y τk+1 )),

k = 0, 1, . . . ,

where Y (k) (t) is the result of applying the combined control ˆ w /k := (ˆ u, (ˆ τ1 , . . . , τˆk ; ζˆ1 , . . . , ζ)).

Put w / = (ˆ u, vˆ). Suppose w / ∈ W and that (xii) {φ(Y (w/) (τ )); τ ∈ T , τ ≤ τD } is Qy -uniformly integrable for all y ∈ S.

126

8 Combined Stochastic Control and Impulse Control of Jump Diffusions

Then φ(y) = Φ(y)

for all y ∈ S

and w /∈W

is an optimal combined control.

Proof. The proof is similar to the proof of Theorem 6.2 and is omitted.

⊓ ⊔

8.2 Examples Example 8.2 (Optimal Combined Control of the Exchange Rate). This example is a simplification of a model studied in [MØ]. Suppose a government has two means of influencing the foreign exchange rate of its own currency: 1. At all times the government can choose the domestic interest rate r. 2. At times selected by the government it can intervene in the foreign exchange market by buying or selling large amounts of foreign currency. Let r(t) denote the interest rate chosen and let τ1 , τ2 , . . . be the (stopping) times when it is decided to intervene, with corresponding amounts ζ1 , ζ2 , . . . If ζ > 0 the government buys foreign currency, if ζ < 0 it sells. Let v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) be corresponding impulse control. If the combined control w = (r, v) is applied, we assume that the corresponding exchange rate X(t) (measured in the number of domestic monetary units it takes to buy one average foreign monetary unit) is given by X(t) = x −



0

t

F (r(s) − r¯(s))ds + σB(t) +



j:τj ≤t

γ(ζj ),

t ≥ 0,

(8.2.1)

where σ > 0 is a constant, r¯(s) is the (average) foreign interest rate, and F : R → R and γ : R → R are known functions which give the effects on the exchange rate by the interest rate differential r(s) − r¯(s) and the amount ζj , respectively. The total expected cost of applying the combined control w = (r, v) is assumed to be of the form  T  −ρt (w) x −ρτj e {M (X(t) − m)+N (r(t)−¯ r(t))}dt + L(ζj )e , J (s, x) = E s

j;τj ≤T

(8.2.2) where M (X(t)−m) and N (r(t)− r¯(t)) give the costs incurred by the difference X(t)−m between X(t) and an optimal value m, and by the difference r(t)−¯ r(t) between the domestic and the average foreign interest rate r¯(t), respectively. The cost of buying/selling the amount ζj is L(ζj ) and ρ > 0 is a constant discounting exponent.

8.2 Examples

127

The problem is to find Φ(s, x) and w∗ = (r∗ , v ∗ ) such that ∗

Φ(s, x) = inf J (w) (s, x) = J (w ) (s, x). w

(8.2.3)

Since this is a minimum problem, the corresponding HJBQVIs of Theorem 8.1 are changed to minima also and they get the form   ' ( ∂φ ∂φ − F (r − r¯(s)) min inf e−ρs M (x − m) + N (r − r¯(s)) + r∈R ∂s ∂x   1 2 ∂2φ + σ , Mφ(s, x) − φ(s, x) = 0, (8.2.4) 2 ∂x2 where Mφ(s, x) = inf {φ(s, x + γ(ζ)) + e−ρs L(ζ)}. ζ∈R

(8.2.5)

In general this is difficult to solve for φ, even for simple choices of the functions M, N , and F . A detailed discussion on a special case can be found in [MØ]. Example 8.3 (Optimal Consumption and Portfolio with Both Fixed and Proportional Transaction Costs (1)). This application is studied in [ØS]. Suppose there are two investment possibilities, say a bank account and a stock. Let X1 (t) and X2 (t) denote the amount of money invested in these two assets, respectively, at time t. In the absence of consumption and transactions suppose that (8.2.6) dX1 (t) = rX1 (t)dt and −

dX2 (t) = X2 (t ) μ dt + σ dB(t) + θ



R

 ˜ z N (dt, dz) ,

(8.2.7)

where r, μ, θ, and σ = 0 are constants and μ > r > 0,

θz ≥ −1 a.s. ν.

(8.2.8)

Suppose that at any time t the investor is free to choose a consumption rate u(t) ≥ 0. This consumption is automatically drawn from the bank account holding with no extra costs. In addition the investor may at any time transfer money from the bank to the stock and conversely. Suppose that such a transaction of size ζ incurs a transaction cost given by c + λ|ζ|,

(8.2.9)

where c > 0 and λ ∈ [0, 1) are constants. (If ζ > 0 we buy stocks and if ζ < 0 we sell stocks.) Thus the control of the investor consists of a combination of a stochastic control u(t) and an impulse control v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .), where

128

8 Combined Stochastic Control and Impulse Control of Jump Diffusions

τ1 , τ2 , . . . are the chosen transaction times and ζ1 , ζ2 , . . . are corresponding transaction amounts. If such a combined control w = (u, v) is applied, the corresponding system (w) (w) (X1 (t), X2 (t)) = (X1 (t), X2 (t)) gets the form dX1 (t) = (rX1 (t) − u(t))dt,

−

τi < t < τi+1 ,

dX2 (t) = X2 (t ) μ dt + σ dB(t) + θ



(8.2.10) 

˜ (dt, dz) , zN

R

τi < t < τi+1 , (8.2.11)

− ) − ζi+1 − c − λ|ζi+1 |, X1 (τi+1 ) = X1 (τi+1

(8.2.12)

ˇ 2 (τ − ) + ζi+1 . X2 (τi+1 ) = X i+1

(8.2.13)

If we do not allow any negative amounts held in the bank account or in the stock, the solvency region S is given by S = [0, ∞) × [0, ∞).

(8.2.14)

(w)

(w)

We call w = (u, v) admissible if X (w) (t) := (X1 (t), X2 (t)) exists for all t. The set of admissible controls is denoted by W. Let τS = inf{t > 0; X (w) (t) ∈ / S} be the bankruptcy time. The investor’s objective is to maximize  τS γ u (t) J (w) (y) = E y dt , e−ρ(s+t) γ 0

(8.2.15)

where ρ > 0 and γ ∈ (0, 1) are constants and E y with y = (s, x1 , x2 ) denotes the expectation when X1 (0− ) = x1 ≥ 0 and X2 (0− ) = x2 ≥ 0. Thus we seek the value function Φ(y) and an optimal control w∗ = (u∗ , v ∗ ) ∈ W such that ∗

Φ(y) = sup J (w) (y) = J (w ) (y).

(8.2.16)

w∈W

This problem may be regarded as a generalization of optimal consumption and portfolio problems studied by Merton [M] and Davis and Norman [DN] (see also Shreve and Soner [SS]). Merton [M] considers the case with no jumps and no-transaction costs (c = λ = θ = 0). The problem then reduces to an ordinary stochastic control problem and it is optimal to keep the positions (X1 (t), X2 (t)) on the line y = (π ∗ /(1 − π ∗ ))x in the (x, y)-plane at all times (the Merton line), where π ∗ = (μ − r)/(1 − γ)σ 2 (see Example 3.2). Davis and Norman [DN] and Shreve and Soner [SS] consider the case when the cost is proportional (λ > 0), with no fixed component (c = 0) and

8.2 Examples

129

x2

✻






















Γ2

x2 =

π∗ x 1−π ∗ 1


sell
◗ s ◗

✟
✟✟
✟✟

✟ Γ1 ✟✟ ✟✟ ②buy ❳ ✟ ❳ ✟ ✟✟ ✟ ✟ ✲

x1

Fig. 8.1. The no-transaction cone (no fixed cost: c = 0), θ = 0

no jumps (θ = 0). In this case the problem can be formulated as a singular stochastic control problem and under some conditions it is proved that there exists a no-transaction cone N T bounded by two straight lines Γ1 and Γ2 such that it is optimal to make no transactions if (X1 (t), X2 (t)) ∈ N T and make transactions corresponding to local time at ∂(N T ), resulting in reflections back to N T every time (X1 (t), X2 (t)) ∈ ∂(N T ). See Fig. 8.1. These results have subsequently been extended to jump diffusion markets by [FØS2] (see Example 5.1). In the general combined control case numerical results indicate (see [CØS]) that the optimal control w∗ = (u∗ , v ∗ ) has the following form. There exist two pairs of lines, (Γ1 , Γˆ1 ) and (Γ2 , Γˆ2 ) from the origin such that the following is optimal. Make no transactions (only consume at the rate u∗ (t)) while (X1 (t), X2 (t)) belongs to the region D bounded by the outer curves Γ1 , Γ2 , and if (X1 (t), X2 (t)) hits ∂D = Γ1 ∪ Γ2 then sell or buy so as to bring (X1 (t), X2 (t)) to the curve Γˆ1 or Γˆ2 . See Fig. 8.2. Note that if we sell stocks (ζ < 0) then (X1 (t), X2 (t)) = (x1 , x2 ) moves to a point (x′1 , x′2 ) = (x′1 (ζ), x′2 (ζ)) on the line x′1 + (1 − λ)x′2 = x1 + (1 − λ)x2 − c,

(8.2.17)

i.e., (x′1 , x′2 ) lies on the straight line through (x1 , x2 ) with slope −1/(1 − λ). Similarly, if we buy stocks (ζ > 0) then the new position (x′1 , x′2 ) = ′ (x1 (ζ), x′2 (ζ)) is on the line x′1 + (1 + λ)x′2 = x1 + (1 + λ)x2 − c,

(8.2.18)

130

8 Combined Stochastic Control and Impulse Control of Jump Diffusions x2 Γ2 Γˆ2

D

sell

Γˆ1 buy

Γ1

x1 Fig. 8.2. The no-transaction region D (c > 0)

i.e., (x′1 , x′2 ) lies on the straight line through (x1 , x2 ) with slope −1/(1 + λ). If there are no interventions then the process ⎡ ⎤ s+t (8.2.19) Y (t) = ⎣X1 (t)⎦ X2 (t) has the generator

Lu φ(s, x1 , x2 ) =

∂φ ∂2φ ∂φ 1 ∂φ + (rx1 − u) + μx2 + σ 2 x22 2 ∂s ∂x1 ∂x2 2 ∂x2   + φ(s, x1 , x2 + x2 θz) − φ(s, x1 , x2 ) R  ∂φ (s, x1 , x2 ) ν(dz). − x2 θz ∂x2

(8.2.20)

Therefore, if we put φ(s, x1 , x2 ) = e−ρs ψ(x1 , x2 ) the corresponding HJBQVI is   γ u ∂ψ ∂2ψ ∂ψ 1 − ρψ(x1 , x2 ) + (rx1 − u) + μx2 + σ 2 x22 2 max sup γ ∂x1 ∂x2 2 ∂x2 u≥0     ∂ψ (x1 , x2 ) ν(dz) + ψ(x1 , x2 + x2 θz) − ψ(x1 , x2 ) − x2 θz ∂x2 R  ψ(x1 , x2 ) − Mψ(x1 , x2 ) = 0 for all (x1 , x2 ) ∈ S, (8.2.21)

8.3 Iterative Methods

131

where (see (8.2.17) and (8.2.18)) Mψ(x1 , x2 ) = sup{ψ(x′1 (ζ), x′2 (ζ)); ζ ∈ R \ {0}, (x′1 (ζ), x′2 (ζ)) ∈ S}. (8.2.22) See Example 9.12 for a further discussion of this.

8.3 Iterative Methods In Chap. 7 we saw that an impulse control problem can be regarded as a limit of iterated optimal stopping problems. A similar result holds for combined control problems. More precisely, a combined stochastic control and impulse control problem can be regarded as a limit of iterated combined stochastic control and optimal stopping problems. We now describe this in more detail. The presentation is similar to the approach in Chap. 7. For n = 1, 2, . . . let Wn denote the set of all admissible combined controls w = (u, v) ∈ W with v ∈ Vn , where Vn is the set of impulse controls v = (τ1 , . . . , τk ; ζ1 , ζ2 , . . . , ζk ) with at most n interventions (i.e., k ≤ n). Then Wn ⊆ Wn+1 ⊆ W

for all n.

Define, with J (w) (y) as in (8.1.6),   Φn (y) = sup J (w) (y); w ∈ Wn ,

n = 1, 2, . . .

(8.3.1)

(8.3.2)

Then

Φn (y) ≤ Φn+1 (y) ≤ Φ(y)

because Wn ⊆ Wn+1 ⊆ W.

Moreover, we have: Lemma 8.4. Suppose g ≥ 0. Then lim Φn (y) = Φ(y)

n→∞

for all y ∈ S.

Proof. The proof is similar to the proof of Lemma 7.1 and is omitted.

⊓ ⊔

The iterative procedure is the following. Let Y (t) = Y (u,0) (t) be the process in (8.1.1) obtained by using the control u and no interventions. Define  τS y φ0 (y) = sup E f (Y (t), u(t))dt + g(Y (τS ))χ{τS g(y)}.

(9.1.9)

Then if y0 ∈ D we have Φ(y0 ) = g(y0 ) and hence (9.1.6) holds trivially. Next, assume y0 ∈ D. Then by the dynamic programming principle (Lemma 7.3b) we have ) τ * y0 f (Y (t))dt + Φ(Y (τ )) (9.1.10) Φ(y0 ) = E 0

for all bounded stopping times τ ≤ τD = inf{t > 0; Y (t) ∈ D}. Choose hm ∈ C02 (Rk ) such that hm → h and Lhm → Lh pointwise dominatedly on Rk . Then by combining (9.1.10) with the Dynkin formula we get ) τ * y0 f (Y (t))dt + Φ(Y (τ )) Φ(y0 ) = E 0 * ) τ y0 f (Y (t))dt + h(Y (τ )) ≤E 0 ) τ * y f (Y (t))dt + hm (Y (τ )) = lim E 0 m→∞ 0 * ) τ y = h(y0 ) + lim E 0 (Lhm (Y (t)) + f (Y (t)))dt . m→∞

Hence

lim E y0

m→∞

)

τ

0

0

* (Lhm (Y (t)) + f (Y (t)))dt ≥ 0.

In particular, if we choose τ = βj := inf{t > 0; |Y (t) − y0 | ≥ 1/j} ∧ 1/j ∧ τD , we get * )  βj (Lh(Y (t)) + f (Y (t)))dt E y0 0

= lim E y0 m→∞

)

0

βj

* (Lhm (Y (t)) + f (Y (t)))dt ≥ 0.

(9.1.11)

138

9 Viscosity Solutions y

If we divide (9.1.11) by E 0 [βj ] and let j → ∞ we get, by right continuity, Lh(y0 ) + f (y0 ) ≥ 0. Hence (9.1.6) holds and we have proved that Φ is a viscosity subsolution. Finally we show that Φ is a viscosity supersolution. So we assume that h ∈ C 2 (Rk ) and y0 ∈ S are such that h ≤ Φ on S and h(y0 ) = Φ(y0 ). Then by the dynamic programming principle (Lemma 7.3a) we have ) τ * y f (Y (t))dt + Φ(Y (τ )) (9.1.12) Φ(y0 ) ≥ E 0 0

for all stopping times τ ≤ τS 0 . Hence, by the Dynkin formula, with hm and βj as above, Φ(y0 ) ≥ E

y0

)

βj

0

≥ lim E

y0

m→∞

* f (Y (t))dt + φ(Y (τ ))

)

βj

0

= h(y0 ) + lim E

* f (Y (t))dt + h(Y (τ ))

y0

m→∞

= h(y0 ) + E

y0

)

)

βj

(Lh(Y (t))dt + f (Y (t)))dt

0

βj

(Lh(Y (t))dt + f (Y (t)))dt

0

Hence E

y0

)

0

y0

βj

*

*

for all j.

* (Lh(Y (t)) + f (Y (t)))dt ≤ 0

and by dividing by E [βj ] and letting j → ∞ we get, by right continuity, Lh(y0 ) + f (y0 ) ≤ 0. Hence (9.1.7) holds and we have proved that Φ is also a viscosity supersolution. ⊓ ⊔ 9.1.1 Uniqueness One important application of the viscosity solution concept is that it can be used as a verification method. In order to verify that a given function φ is indeed the value function Φ it suffices to verify that the function is a viscosity solution of the corresponding variational inequality. For this method to work, however, it is necessary that we know that Φ is the unique viscosity solution. Therefore the question of uniqueness is crucial. In general we need not have uniqueness. The following simple example illustrates this.

9.2 The Value Function is Not Always C 1

139

Example 9.3. Let Y (t) = B(t) ∈ R and choose f = 0, S = R, and g(y) =

y2 , 1 + y2

y ∈ R.

(9.1.13)

Then the value function Φ of the optimal stopping problem Φ(y) = sup E y [g(B(τ ))]

(9.1.14)

τ ∈T

is easily seen to be Φ(y) ≡ 1. The corresponding VI is   1 ′′ φ (y), g(y) − φ(y) = 0 max 2

(9.1.15)

and this equation is trivially satisfied by all constant functions φ(y) ≡ a for any a ≥ 1. Theorem 9.4 (Uniqueness). Suppose that τS 0 < ∞

a.s. P y for all y ∈ S 0 .

(9.1.16)

¯ be a viscosity solution of (9.1.4) and (9.1.5) with the property Let φ ∈ C(S) that the family {φ(Y (τ )); τ stopping time, τ ≤ τS 0 } is P y -uniformly integrable, for all y ∈ S 0 . Then φ(y) = Φ(y)

(9.1.17)

¯ for all y ∈ S.

Proof. We refer the reader to [ØR] for the proof in the case where there are no jumps. ⊓ ⊔

9.2 The Value Function is Not Always C 1 Example 9.5. We now give an example of an optimal stopping problem where the value function Φ is not C 1 everywhere. In this case Theorem 2.2 cannot be used to find Φ. However, we can use Theorem 9.4. The example is taken from [ØR]: Define ⎧ ⎪ for x ≤ 0 ⎨1 k(x) = 1 − cx for 0 < x < a, (9.2.1) ⎪ ⎩ 1 − ca for x ≥ a

140

9 Viscosity Solutions

where c and a are constants to be specified more closely later. Consider the optimal stopping problem $ & Φ(s, x) = sup E (s,x) e−ρ(s+τ ) k(B(τ )) , (9.2.2) τ ∈T

where B(t) is one-dimensional Brownian motion, B(0) = x ∈ R = S, and ρ > 0 is a constant. The corresponding variational inequality is (see (9.1.4))   ∂φ 1 ∂ 2 φ −ρs + , e k(x) − φ(s, x) = 0. (9.2.3) max ∂s 2 ∂x2

If we try a solution of the form φ(s, x) = e−ρs ψ(x)

(9.2.4)

for some function ψ, then (9.2.3) becomes   1 ′′ max −ρψ(x) + ψ (x), k(x) − ψ(x) = 0. 2

(9.2.5)

Let us guess that the continuation region D has the form D = {(s, x); 0 < x < x1 }

(9.2.6)

for some x1 > a. Then (9.2.5) can be split into the three equations 1 −ρψ(x) + ψ ′′ (x) = 0 ; 2 ψ(x) = 1 ; ψ(x) = 1 − ca ;

0 < x < x1

(9.2.7)

x≤0 x ≥ x1 .

The general solution of (9.2.7) is ψ(x) = C1 e

√

2ρ x

+ C2 e−

√

2ρ x

,

0 < x < x1 ,

where C1 , C2 are arbitrary constants. If we require ψ to be continuous at x = 0 and at x = x1 we get the two equations

C1 e

√

C1 + C2 = 1, 2ρ x1

+ C2 e

√ − 2ρ x1

(9.2.8) = 1 − ca

(9.2.9)

in the three unknowns C1 , C2 , and x1 . If we also guess that ψ will be C 1 at x = x1 we get the third equation √ √ (9.2.10) C1 2ρ e 2ρ x1 − C2 2ρ e− 2ρ x1 = 0.

9.2 The Value Function is Not Always C 1

If we assume that ca < 1 and

-

1 2ρ < ln a

1 − ca 1 − ca(2 − ca)



141

(9.2.11)

then the three equations (9.2.8), (9.2.9), and (9.2.10) have the unique solution ( 1' (9.2.12) C1 = 1 − ca(2 − ca) > 0, C2 = 1 − C1 > 0 2 and   1 − ca 1 x1 = √ ln > a. (9.2.13) 2C1 2ρ With these values of C1 , C2 , and x1 we put ⎧ ⎪ if x ≤ 0 ⎨1 √ √ 2ρ x − 2ρ x ψ(x) = C1 e (9.2.14) + C2 e if 0 < x < x1 . ⎪ ⎩ 1 − ca if x1 ≤ x See Fig. 9.1. We claim that ψ is a viscosity solution of (9.2.5).

(i) First we verify that ψ is a viscosity subsolution: let h ∈ C 2 (R), h ≥ ψ, and h(x0 ) = ψ(x0 ). Then if x0 ≤ 0 or x0 ≥ x1 we have k(x0 ) − ψ(x0 ) = 0. And if 0 < x0 < x1 then h − ψ is C 2 at x = x0 and has a local minimum at x0 so h′′ (x0 ) − ψ ′′ (x0 ) ≥ 0. Therefore

1 1 −ρh(x0 ) + h′′ (x0 ) ≥ −ρψ(x0 ) + ψ ′′ (x0 ) = 0. 2 2 This proves that   1 max −ρh(x0 ) + h′′ (x0 ), k(x0 ) − ψ(x0 ) ≥ 0, 2 so ψ is a viscosity subsolution of (9.2.5).

1 − cx

1

ψ(x) 1 − ca a

0 Fig. 9.1. The function ψ

x1

x

142

9 Viscosity Solutions

(ii) Second, we prove that ψ is a viscosity super solution. So let h ∈ C 2 (R), h ≤ ψ, and h(x0 ) = ψ(x0 ). Note that we always have k(x0 ) − ψ(x0 ) ≤ 0 so in order to prove that   1 ′′ max −ρh(x0 ) + h (x0 ), k(x0 ) − ψ(x0 ) ≤ 0 2 it suffices to prove that 1 −ρh(x0 ) + h′′ (x0 ) ≤ 0. 2 At any point x0 where ψ is C 2 this follows in the same way as in (i) above. So it remains only to consider the two cases x0 = 0 and x0 = x1 . If x0 = 0 then no such h exists, so the conclusion trivially holds. If x0 = x1 then the function h − ψ has a local maximum at x = x0 and it is C 2 to the left of x0 so lim− h′′ (x) − ψ ′′ (x) ≤ 0, x→x0

i.e., h′′ (x0 ) − This gives

ψ ′′ (x− 0)

≤0.

1 1 −ρh(x0 ) + h′′ (x0 ) ≤ −ρψ(x0 ) + ψ ′′ (x− 0 ) = 0, 2 2 and the proof is complete. We have proved. Suppose (9.2.11) holds. Then the value function Φ(s, x) of problem (9.2.2) is given by Φ(s, x) = e−ρs ψ(x) with ψ as in (9.2.14), C1 , C2 , and x1 as in (9.2.12) and (9.2.13). Note in particular that ψ(x) is not C 1 at x = 0.

9.3 Viscosity Solutions of HJBQVI We now turn to the general combined stochastic control and impulse control problem from Chap. 8. Thus the state Y (t) = Y (w) (t) is Y (0− ) = y ∈ Rk ,

dY (t) = b(Y (t), u(t))dt + σ(Y (t), u(t))dB(t)  ˜ (dt, dz), τi < t < τi+1 , + γ(Y (t− ), u(t− ), z)N Rℓ

− Y (τi+1 ) = Γ (Yˇ (τi+1 ), ζi+1 ),

i = 0, 1, 2, . . . ,

where w = (u, v) ∈ W, u ∈ U, v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) ∈ V.

(9.3.1)

9.3 Viscosity Solutions of HJBQVI

143

The performance is given by ⎡ ⎤  τS  − (w) y⎣ J (y) = E K(Yˇ (τj ), ζj )⎦ f (Y (t), u(t))dt + g(Y (τS ))χ{τS 0; Y (w) (t) ∈ S} = 0

for all y ∈ ∂S and all w ∈ W. (9.3.5) These conditions (9.3.4) and (9.3.5) exclude cases where Φ also satisfies certain HJBQVIs on ∂S (see, e.g., [ØS]), but it is often easy to see how to extend the results to such situations. We need to make the following two assumptions on the set of admissible controls W : (1) If w = (u, v), with v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .), belongs to W and ζˆ is any point in Z, then w := (u, vˆ) belongs to W also, when vˆ := (0, τ1 , τ2 , . . . ; ζ, ζ1 , ζ2 , . . .). (2) If α is any constant in U , then the combined control w := (α, 0) (no interventions, just the constant control α) belongs to W . Theorem 8.1 associates Φ to the HJBQVI   α max sup {L Φ(y) + f (y, α)}, MΦ(y) − Φ(y) = 0, α∈U

y∈S

(9.3.6)

with boundary values Φ(y) = g(y),

y ∈ ∂S,

(9.3.7)

where Lα Φ(y) =

k 

bi (y, α)

i=1

+

k 1  ' T( ∂2Φ ∂Φ σσ ij (y, α) + ∂yi 2 i,j=1 ∂yi ∂yj

ℓ  + !  Φ y + γ (j) (y, α, zj ) − Φ(y) j=1

R

, − ∇Φ(y) · γ (j) (y, α, zj ) νj (dzj )

(9.3.8)

144

9 Viscosity Solutions

and   MΦ(y) = sup Φ(Γ (y, ζ)) + K(y, ζ); ζ ∈ Z, Γ (y, ζ) ∈ S .

(9.3.9)

Unfortunately, as we have seen already for optimal stopping problems, the value function Φ need not be C 1 everywhere – in general not even continuous! So (9.3.6) is not well defined, if we interpret the equation in the usual sense. However, it turns out that if we interpret (9.3.6) in the weak sense of viscosity then Φ does indeed solve the equation. In fact, under some assumptions Φ is the unique viscosity solution of (9.3.6) and (9.3.7) (see Theorem 9.11). This result is an important supplement to Theorem 8.1. We now define the concept of viscosity solutions of general HJBQVIs of type (9.3.6) and (9.3.7). ¯ Definition 9.6. Let ϕ ∈ C(S). (i) We say that ϕ is a viscosity subsolution of   α max sup {L ϕ(y) + f (y, α)} , Mϕ(y) − ϕ(y) = 0, α∈U

ϕ(y) = g(y),

y ∈ ∂S

y ∈ S,

(9.3.10) (9.3.11)

if (9.3.11) holds and for every h ∈ C 2 (Rk ) and every y0 ∈ S such that h ≥ ϕ on S and h(y0 ) = ϕ(y0 ) we have   α max sup {L h(y0 ) + f (y0 , α)} , Mϕ(y0 ) − ϕ(y0 ) ≥ 0. (9.3.12) α∈U

(ii) We say that ϕ is a viscosity supersolution of (9.3.10) and (9.3.11) if (9.3.11) holds and for every h ∈ C 2 (Rk ) and every y0 ∈ S such that h ≤ ϕ on S and h(y0 ) = ϕ(y0 ) we have   max sup {Lα h(y0 ) + f (y0 , α)} , Mϕ(y0 ) − ϕ(y0 ) ≤ 0. (9.3.13) α∈U

(iii) We say that ϕ is a viscosity solution of (9.3.10) and (9.3.11) if ϕ is both a viscosity subsolution and a viscosity supersolution of (9.3.10) and (9.3.11). Lemma 9.7. Let Φ be as in (9.3.3). Then Φ(y) ≥ MΦ(y) for all y ∈ S. Proof. Suppose there exists y ∈ S with Φ(y) < MΦ(y),

9.3 Viscosity Solutions of HJBQVI

145

i.e., Φ(y) < sup {Φ(Γ (y, ζ)) + K(y, ζ)} . ζ∈Z

ˆ Then there exist ε > 0 and ζˆ ∈ Z such that, with yˆ = Γ (y, ζ), ˆ − 2ε. Φ(y) < Φ(ˆ y ) + K(y, ζ) Let w = (u, v), with v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) be ε-optimal for Φ at yˆ, in the sense that y ) > Φ(ˆ y ) − ε. J (w) (ˆ ˆ ζ1 , ζ2 , . . .). Then, with τ0 = 0 Define w ˆ := (u, vˆ), where vˆ = (0, τ1 , τ2 , . . . ; ζ, ˆ and ζ0 = ζ,   τS ˆ f (Y (t), u(t))dt + g(Y (τS ))χ{τS 0 and let w = (u, v) ∈ W, with v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) ∈ V, be an ǫ-optimal portfolio, i.e., Φ(y0 ) < J (w) (y0 ) + ǫ. Since τ1 is a stopping time we know that {ω; τ1 (ω) = 0} is F0 -measurable and hence either τ1 (ω) = 0 a.s.

or

τ1 (ω) > 0 a.s.

(9.3.16)

If τ1 = 0 a.s. then Y (w) makes an immediate jump from y0 to the point y = Γ (y0 , ζ1 ) ∈ S and hence ′

′

Φ(y0 ) − ǫ ≤ J (w ) (y ′ ) + K(y0 , ζ1 ) ≤ Φ(y ′ ) + K(y0 , ζ1 ) ≤ MΦ(y0 ), where w′ = (τ2 , τ3 , . . . ; ζ2 , ζ3 , . . .). This is a contradiction if ǫ < Φ(y0 ) − MΦ(y0 ). This proves that (9.3.15) implies that it is impossible to have τ1 = 0 a.s. So by (9.3.16), we can now assume that τ1 > 0 a.s. Choose R < ∞, ρ > 0 and define τ := τ1 ∧ R ∧ inf{t > 0 ; | Y (w) (t) − y0 | ≥ ρ}. By the dynamic programming principle (see Lemma 7.3) we have: for each ε > 0, there exists a control u such that   τ − y0 ˇ f (Y (t), u(t))dt + Φ(Y (τ )) + ε, (9.3.17) Φ(y0 ) ≤ E 0

where, as before, Yˇ (τ − ) = Y (τ − ) + ΔN Y (τ ). Choose hm ∈ C02 (Rk ) such that hm → h and Lu hm → Lu h pointwise dominatedly as m → ∞. Then by (9.3.17) and the Dynkin formula we have, using that Φ ≤ h,   τ f (Y (t), u(t))dt + h(Yˇ (τ − )) + ε Φ(y0 ) ≤ E y0 0   τ f (Y (t), u(t))dt + hm (Yˇ (τ − )) + ε ≤ lim inf E y0 m→∞ 0   τ Lu hm (Y (t)) + f (Y (t), u(t))dt + ε = h(y0 ) + lim inf E y0 m→∞ 0   τ u y0 L h(Y (t)) + f (Y (t), u(t))dt + ε. (9.3.18) = h(y0 ) + E 0

9.3 Viscosity Solutions of HJBQVI

147

Using that h(y0 ) = Φ(y0 ), we obtain   τ {Lu h(Y (t)) + f (Y (t), u(t))}dt ≥ −ε. E y0 0

Dividing by E y0 [τ ] and letting ρ → 0 we get Lα0 h(y0 ) + f (y0 , α0 ) ≥ −ε, where α0 = lim+ u(s). s→0

Since ε is arbitrary, this proves (9.3.14) and hence that Φ is a viscosity subsolution. (b) Next we prove that Φ is a viscosity supersolution. So we choose h ∈ C 2 (Rk ) and y0 ∈ S such that h ≤ Φ on S and h(y0 ) = Φ(y0 ). We must prove that   (9.3.19) max sup {Lα h(y0 ) + f (y0 , α)}, MΦ(y0 ) − Φ(y0 ) ≤ 0. α∈U

Since Φ ≥ MΦ always (Lemma 9.7) it suffices to prove that Lα h(y0 ) + f (y0 , α) ≤ 0 for all α ∈ U. To this end, fix α ∈ U and let wα = (α, 0), i.e., wα is the combined control (uα , vα ) ∈ W where uα = α (constant) and vα = 0 (no interventions). Then by the dynamic programming principle and the Dynkin formula we have, with Y (t) = Y (wα ) (t), τ = τS ∧ ρ, and hm as in (a),   τ f (Y (s), α)ds + Φ(Yˇ (τ − )) Φ(y0 ) ≥ E y0 0

≥E

y0



τ

0

 − ˇ f (Y (s), α)ds + h(Y (τ ))

= h(y0 ) + lim E

y0

m→∞

= h(y0 ) + E y0

0



0

Hence E



0

τ



τ

τ α

{L hm (Y (t)) + f (Y (t), α)}dt

 {Lα h(Y (t)) + f (Y (t), α)}dt .



 {L h(Y (t)) + f (Y (t), α)}dt ≤ 0. α

Dividing by E[τ ] and letting ρ → 0 we get (9.3.19). This completes the proof of Theorem 9.8.

⊓ ⊔

148

9 Viscosity Solutions

Next we turn to the question of uniqueness of viscosity solutions of (9.3.10) and (9.3.11). Many types of uniqueness results can be found in the literature. See the references in the end of this section. Here we give a proof in the case when the process Y (t) has no jumps, i.e. when N (·, ·) = ν(·) = 0. The method we use is a generalization of the method in [ØS, Theorem 3.8]. First we introduce some convenient notation: Define Λ : Rk×k × Rk × RS × Rk → R by

k k  1  Λ(R, r, ϕ, y) := sup (σσ T )ij (y, α)Rij bi (y, α)ri + 2 α∈U i,j=1 i=1 +

ℓ  +  ϕ(y + γ (j) (y, α, zj )) − ϕ(y) j=1

−r·γ

R

(j)

 , (y, α, zj ) νj (dzj ) + f (y, α)

(9.3.20)

for R = [Rij ] ∈ Rk×k , r = (ri , . . . , rk ) ∈ Rk , ϕ : S → R, y ∈ Rk , and define F : Rk×k × Rk × RS × Rk → R by F (R, r, ϕ, y) = max{Λ(R, r, ϕ, y), Mϕ(y) − ϕ(y)}.

(9.3.21)

Note that if ϕ ∈ C 2 (Rk ) then Λ(D2 ϕ, Dϕ, ϕ, y) = sup {Lα ϕ(y) + f (y, α)} , α∈U

where

   ∂2ϕ ∂ϕ D ϕ(y) = (y) and Dϕ(y) = (y). ∂yi ∂yj ∂yi 2



We recall the concepts of “superjets” JS2,+ , JS2,− and J¯S2,+ , J¯S2,− (see [CIL, Sect. 2]):  JS2,+ ϕ(y) := (R, r) ∈ Rk×k × Rk ;    1 lim sup ϕ(η) − ϕ(y) − (η − y)T r − (η − y)T R(η − y) · |η − y|−2 ≤ 0 , η→y 2 η∈S

 J¯S2,+ ϕ(y) := (R, r) ∈ Rk×k × Rk ; for all n there exists

(R(n) , r(n) , y (n) ) ∈ Rk×k × Rk × S such that (R(n) , r(n) ) ∈ JS2,+ ϕ(y (n) ) and  (R(n) , r(n) , ϕ(y (n) ), y (n) ) → (R, r, ϕ(y), y) as n → ∞

9.3 Viscosity Solutions of HJBQVI

and

JS2,− ϕ = −JS2,+ (−ϕ),

149

J¯S2,− ϕ = −J¯S2,+ (−ϕ).

In terms of these superjets one can give an equivalent definition of viscosity solutions as follows. Theorem 9.9. [CIL, Sect. 2] (i) A function ϕ ∈ C(S) is a viscosity subsolution of (9.3.10) and (9.3.11) if and only if (9.3.11) holds and max(Λ(R, r, ϕ, y), Mϕ(y) − ϕ(y)) ≥ 0 for all (R, r) ∈ J¯S2,+ ϕ(y), y ∈ S. (ii) A function ϕ ∈ C(S) is a viscosity supersolution of (9.3.10) and (9.3.11) if and only if (9.3.11) holds and max(Λ(R, r, ϕ, y), Mϕ(y) − ϕ(y)) ≤ 0 for all (R, r) ∈ J¯S2,− ϕ(y), y ∈ S. We have now ready for the second main theorem of this section. Theorem 9.10 (Comparison Theorem). Assume that N (·, ·) = 0.

(9.3.22)

¯ which satisfies the Suppose that there exists a positive function β ∈ C (S) strict quasivariational inequality

 max

sup {Lα β(y)}, sup β(Γ (y, ζ)) − β(y)

α∈U

ζ∈Z

2

≤ −δ(y) < 0,

y ∈ S, (9.3.23)

where δ(y) > 0 is bounded away from 0 on compact subsets of S. Let u be a viscosity subsolution and v be a viscosity supersolution of (9.3.10) and (9.3.11) and suppose that  +  u (y) v − (y) + lim = 0. (9.3.24) β(y) β(y) |y|→∞ Then u(y) ≤ v(y)

for all y ∈ S.

Proof. (Sketch) We argue by contradiction. Suppose that sup{u(y) − v(y)} > 0.

y∈S

Then by (9.3.24) there exists ǫ > 0 such that if we put vǫ (y) := v(y) + ǫβ(y),

y∈S

150

9 Viscosity Solutions

then M := sup{u(y) − vǫ (y)} > 0. y∈S

For n = 1, 2, . . . and (x, y) ∈ S × S define Hn (x, y) := u(x) − v(y) −

ǫ n |x − y|2 − (β(x) + β(y)) 2 2

and Mn :=

sup

Hn (x, y).

(x,y)∈S×S

Then by (9.3.24) we have 0 < Mn < ∞

for all n,

and there exists (x(n) , y (n) ) ∈ S × S such that Mn = Hn (x(n) , y (n) ). Then by Lemma 3.1 in [CIL] the following holds: lim n|x(n) − y (n) |2 = 0

n→∞

and lim Mn = u(ˆ y ) − vǫ (ˆ y ) = sup{u(y) − vǫ (y)} = M,

n→∞

y∈S

for any limit point yˆ of {y (n) }∞ n=1 . Since v is a supersolution of (9.3.10), (9.3.11), and (9.3.23) holds, we see that vǫ is a strict supersolution of (9.3.10), in the sense that ϕ = vǫ satisfies (9.3.13) in the following strict form:   α max sup {L h(y0 ) + f (y0 , α)}, Mvǫ (y0 ) − vǫ (y0 ) ≤ −δ(y0 ), α∈U

with δ(·) as in (9.3.23). By [CIL, Theorem 3.2], there exist k × k matrices P (n) , Q(n) such that, if we put p(n) = q (n) = n(x(n) − y (n) ) then (P (n) , p(n) ) ∈ J¯ 2,+ u(x(n) ) and

and

(Q(n) , q (n) ) ∈ J¯ 2,− vǫ (y (n) )

 (n)    P 0 I −I ≤ 3n , −I I 0 −Q(n)

9.3 Viscosity Solutions of HJBQVI

151

in the sense that ξ T P (n) ξ − η T Q(n) η ≤ 3n|ξ − η|2

for all ξ, η ∈ Rk .

(9.3.25)

Since u is a subsolution we have, by Theorem 9.9, ! max Λ(P (n) , p(n) , u, x(n) ), Mu(x(n) ) − u(x(n) ) ≥ 0

(9.3.26)

! max Λ(Q(n) , q (n) , vǫ , y (n) ), Mvǫ (y (n) ) − vǫ (y (n) ) ≤ 0.

(9.3.27)

and since vǫ is a supersolution we have

By (9.3.25) we get

Λ(P (n) , p(n) , u, x(n) ) − Λ(Q(n) , q (n) , vǫ , y (n) )

k  (n) (n) ≤ sup (bi (x(n) , α) − bi (y (n) , α))(pi − qi ) α∈U

i=1

 k * 1 ) (n) (n) (σσ T )ij (x(n) , α) − (σσ T )ij (y (n) , α) (Pij − Qij ) + 2 i,j=1

≤ 0.

Therefore, by (9.3.27), Λ(P (n) , p(n) , u, x(n) ) ≤ Λ(Q(n) , q (n) , vǫ , y (n) ) ≤ 0 and hence, by (9.3.26), Mu(x(n) ) − u(x(n) ) ≥ 0.

(9.3.28)

On the other hand, since vǫ is a strict supersolution we have Mvǫ (y (n) ) − vǫ (y (n) ) < −δ

for all n,

(9.3.29)

for some constant δ > 0. Combining the above we get Mn < u(x(n) ) − vǫ (y (n) ) < Mu(x(n) ) − Mvǫ (y (n) ) − δ and hence M = lim Mn ≤ lim (Mu(x(n) ) − Mvǫ (y (n) ) − δ) n→∞

n→∞

y) − δ ≤ Mu(ˆ y ) − Mvǫ (ˆ

= sup {u(Γ (ˆ y , ζ)) + K(ˆ y , ζ)} − sup {vǫ (Γ (ˆ y , ζ)) + K(ˆ y , ζ)} − δ ζ∈Z

ζ∈Z

≤ sup {u(Γ (ˆ y , ζ)) − vǫ (Γ (ˆ y , ζ))} − δ ≤ M − δ. ζ∈Z

This contradiction proves Theorem 9.10.

⊓ ⊔

152

9 Viscosity Solutions

Theorem 9.11 (Uniqueness of Viscosity Solutions). Suppose that the process Y (t) has no jumps, i.e., N (·, ·) = 0 ¯ be as in Theorem 9.10. Then there is at most one viscosity and let β ∈ C 2 (S) solution ϕ of (9.3.10) and (9.3.11) with the property that |ϕ(y)| = 0. |y|→∞ β(y)

(9.3.30)

lim

Proof. Let ϕ1 , ϕ2 be two viscosity solutions satisfying (9.3.30). If we apply Theorem 9.10 to u = ϕ1 and v = ϕ2 we get ϕ1 ≤ ϕ2 . If we apply Theorem 9.10 to u = ϕ2 and v = ϕ1 we get ϕ2 ≤ ϕ1 . ⊓ ⊔

Hence ϕ1 = ϕ2 .

Example 9.12 (Optimal Consumption and Portfolio with Both Fixed and Proportional Transaction Costs (2)). Let us return to Example 8.3. In this case (9.3.10) takes the form (8.2.21) and (8.2.22) in S 0 . For simplicity we assume Dirichlet boundary conditions, e.g., ψ = 0, on ∂S. Fix γ ′ ∈ (γ, 1) such that (see (3.1.8))   (μ − r)2 ρ > γ′ r + 2 2σ (1 − γ) and define

′

β(x1 , x2 ) = (x1 + x2 )γ . Then with M as in (8.2.22) we have  γ′

(Mβ − β)(x1 , x2 ) ≤ (x1 + x2 )

c 1− x1 + x2

(9.3.31) γ ′



− 1 < 0.

(9.3.32)

Moreover, with Lu ψ(x1 , x2 ) := − ρψ(x1 , x2 ) + (rx1 − u) ∂2ψ 1 + σ 2 x22 2 (x1 , x2 ), 2 ∂x2

∂ψ ∂ψ (x1 , x2 ) + μx2 (x1 , x2 ) ∂x1 ∂x2

ψ ∈ C 2 (R2 )

(9.3.33)

we get max Lu β(x1 , x2 ) < 0, u≥0

(9.3.34)

9.4 Numerical Analysis of HJBQVI

153

and in both (9.3.32) and (9.3.34) the strict inequality is uniform on compact subsets of S 0 . The proofs of these inequalities are left as an exercise (Exercise 9.3). We conclude that the function β in (9.3.31) satisfies the conditions (9.3.23) of Theorem 9.10. Thus by Theorem 9.11 we have in this example uniqueness of viscosity solutions ϕ satisfying the growth condition ′

lim

|(x1 ,x2 )|→∞

(x1 + x2 )−γ |ϕ(x1 , x2 )| = 0.

(9.3.35)

For other results regarding uniqueness of viscosity solutions of equations associated to impulse control, stochastic control and optimal stopping for jump diffusions, we refer to [Am, AKL, BKR2, CIL, Is1, Is2, Isk, MS, AT, Ph, JK1, FS, BCa, BCe] and the references therein.

9.4 Numerical Analysis of HJBQVI In this section we give some insights in the numerical solution of HJBQVI. We refer, e.g., to [LST] for details on the finite difference approximations and the description of the algorithms to solve dynamic programming equations. Here we focus on the main problem which arises in the case of quasivariational inequalities, i.e., the presence of a nonexpansive operator due to the intervention operator.

9.4.1 Finite Difference Approximation We want to solve the following HJBQVI numerically   α max sup {L Φ(x) + f (x, α)}, MΦ(x) − Φ(x) = 0, α∈U

x∈S

(9.4.1)

with boundary values Φ(x) = g(x),

x ∈ ∂S,

(9.4.2)

where Lα Φ(x) = −ρΦ +

k  i=1

bi (x, α)

k ∂Φ 1  ∂2Φ + aij (x, α) ∂xi 2 i,j=1 ∂xi ∂xj

(9.4.3)

and   MΦ(x) = sup Φ(Γ (x, ζ)) + K(x, ζ); ζ ∈ Z, Γ (x, ζ) ∈ S . (9.4.4) ( ' We have denoted here aij := σσ T ij . We shall also write K ζ (x) for K(x, ζ).

154

9 Viscosity Solutions

We assume that S is bounded, otherwise a change of variable or a localization procedure has to be performed in order to reduce to a bounded domain. Moreover we assume for simplicity that S is a box, i.e., a cartesian product of bounded intervals in Rk . We can also handle Neumann type boundary conditions without additional difficulty. We discretize (9.4.1) by using a finite difference approximation. Let δi denote the finite difference step in each coordinate direction and set δ = (δ1 , . . . , δk ). Denote by ei the unit vector in the ith coordinate direction, and 1 2k 1 2k consider the grid Sδ = S i=1 (δi Z). Set ∂Sδ = ∂S i=1 (δi Z). We use the following approximations: ∂Φ Φ(x + δi ei ) − Φ(x − δi ei ) (x) ∼ ≡ ∂iδi Φ(x) ∂xi 2δi

(9.4.5)

or (see (9.4.16)) ⎧ Φ(x + δi ei ) − Φ(x) ⎪ ⎪ ≡ ∂iδi + Φ(x) if bi (x) ≥ 0, ⎪ ⎨ δi

∂Φ (x) ∼ ⎪ ∂xi ⎪ Φ(x) − Φ(x − δi ei ) ⎪ ⎩ ≡ ∂iδi − Φ(x) if bi (x) ≤ 0. δi ∂2Φ Φ(x + δi ei ) − 2Φ(x) + Φ(x − δi ei ) δi Φ(x). (x) ∼ ≡ ∂ii 2 ∂xi δi2

(9.4.6)

(9.4.7)

If aij (x) ≥ 0, i = j, then ∂2Φ 2Φ(x) + Φ(x + δi ei + δj ej ) + Φ(x − δi ei − δj ej ) (x) ∼ ∂xi ∂xj 2δi δj   Φ(x + δi ei ) + Φ(x − δi ei ) + Φ(x + δj ej ) + Φ(x − δj ej ) − 2δi δj δ δj +

≡ ∂iji

Φ(x).

(9.4.8)

If aij (x) < 0, i = j, then ∂2Φ [2Φ(x) + Φ(x + δi ei − δj ej ) + Φ(x − δi ei + δj ej )] (x) ∼ − ∂xi ∂xj 2δi δj Φ(x + δi ei ) + Φ(x − δi ei ) + Φ(x + δj ej ) + Φ(x − δj ej ) + 2δi δj δ δj −

≡ ∂iji

Φ(x).

(9.4.9)

These approximations can be justified when the function Φ is smooth by Taylor expansions. Using approximations (9.4.5), (9.4.7)–(9.4.9), we obtain the following approximation problem:

9.4 Numerical Analysis of HJBQVI



 α max sup {Lδ Φδ (x) + f (x, α)}, Mδ Φδ (x) − Φδ (x) = 0 α∈U

Φδ (x) = g(x)

155

for all x ∈ Sδ ,

for all x ∈ ∂Sδ ,

(9.4.10)

where ⎫ ⎧ k ⎬ ⎨  |a (x, α)| −a (x, α) ij ii Lα + − ρ Φ(x) = Φ(x) δ ⎭ ⎩ δi2 2δi δj i=1 j =i ⎫ ⎧ ⎨ a (x, α) ⎬  1  (x, α) |a (x, α)| b i ij ii + − + κ Φ(x + κδi ei ) ⎩ δi2 2 i,κ=±1 δi δj δi ⎭ +

and with

1 2

j,j =i



Φ(x + κei δi + λej δj )

i=j,κ=±1,λ=±1

aij (x, α)[κλ] δi δj

  Mδ Φδ (x) = sup Φ(Γ (x, ζ)) + K(x, ζ); ζ ∈ Zδ (x)   Zδ (x) = ζ ∈ Z, Γ (x, ζ) ∈ Sδ .

(9.4.11)

(9.4.12) (9.4.13)

We have used here the notation  + , aij (x, α) ≡ max(0, aij (x, α)) if κλ = 1, [κλ] aij (x, α) = a− ij (x, α) ≡ − min(0, aij (x, α)) if κλ = −1. In (9.4.10), Φδ denotes an approximation of Φ at the grid points. This approximation is consistent and stable if the following condition holds: (see [LST] for a proof) |bi (x, α)| ≤

aii (x, α)  |aij (x, α)| − δi δj j =i

for all α in U , x in Sδ , i = 1, . . . , k.

(9.4.14) In this case φδ converges to the viscosity solution of (9.4.1) when the step δ goes to 0. This can be proved by using techniques introduced by Barles and Souganidis [BS], provided a comparison theorem holds for viscosity sub- and supersolutions of the continuous-time problem. If (9.4.14) does not hold but only the following weaker condition 0≤

aii (x, α)  |aij (x, α)| − δi δj j =i

for all α in U , x in Sδ , i = 1 . . . k. (9.4.15)

is satisfied, then it can be shown that we can also obtain a stable approximation (but of lower order) by using the one-sided approximations (9.4.6) for

156

9 Viscosity Solutions

the approximation of the gradient instead of the centered difference (9.4.5). Instead of (9.4.11), the operator Lα δ is then equal to ⎫ ⎧ k ⎬ ⎨  −aii (x, α) |aij (x, α)| |bi (x, α)| + − −ρ Lα δ Φ(x) = Φ(x) 2 ⎭ ⎩ δi 2δi δj δi i=1 j =i ⎧ ⎫ ⎨ [κ] ⎬   aii (x, α) 1 |aij (x, α)| bi (x, α) + − + Φ(x + κδi ei ) ⎩ δi2 ⎭ 2 i,κ=±1 δi δj δi +

1 2



j,j =i

Φ(x + κei δi + λej δj )

i=j,κ=±1,λ=±1

aij (x, α)[κλ] . δi δj

(9.4.16)

By replacing the values of the function Φδ by their known values on the boundary, we obtain the following equation in Sδ :   ¯ α Φδ (x) + fδ (x, α)}, Mδ Φδ (x) − Φδ (x) = 0, x ∈ Sδ , (9.4.17) max sup {L δ α∈U

¯ α is a square Nδ × Nδ matrix, obtained by retrieving the first and where L δ last column from Lα δ , Nδ = Card(Sδ ), i.e., the number of points of the grid, and fδ (x, α) (which will also be denoted by fδα (x)) takes into account the boundary values.

9.4.2 A Policy Iteration Algorithm for HJBQVI ¯ α is When the stability conditions (9.4.14) or (9.4.15) hold, then the matrix L δ diagonally dominant, i.e., ¯α (L δ )ij ≥ 0 for i = j and

Nδ  j=1

¯α (L δ )ij ≤ −ρ < 0 for all i = 1, . . . , Nδ .

Now let h be a positive number such that h ≤ min i

1 ¯ α )ii + ρ| |(L δ

(9.4.18)

and let Iδ denote the Nδ × Nδ identity matrix. It is easy to check that the matrix ¯ α + ρIδ ) Pδα := Iδ + h(L δ %Nδ is sub-Markovian, i.e., (Pδα )ij ≥ 0 for all i, j and j=1 (Pδα )ij ≤ 1 for all i. Consequently (9.4.17) can be rewritten as     ! 1 α α max sup P Φδ (x) − (1 + ρh)Φδ (x) + fδ (x) , Mδ Φδ (x) − Φδ (x) = 0, h δ α∈U (9.4.19)

9.4 Numerical Analysis of HJBQVI

157

which is equivalent to Φδ (x) = max

sup

α∈U

Lα δ Φδ (x),

ζ



sup B Φδ (x) , ζ∈Zδ (x)

(9.4.20)

where Lα δ Φ(x) :=

Pδα Φ(x) + hfδα (x) , 1 + ρh

B ζ Φ(x) := Φ(Γ (x, ζ)) + K ζ (x).

(9.4.21) (9.4.22)

Let P(Sδ ) denote the set of all subsets of Sδ and for (T, α, ζ) in P(Sδ )×U ×Zδ , denote by OT,α,ζ the operator:

Lα δ v(x) if x ∈ Sδ \T , OT,α,ζ v(x) := (9.4.23) B ζ v(x) if x ∈ T . Problem (9.4.20) is equivalent to the fixed point problem Φδ (x) =

sup T ∈P(Sδ ),α∈U,ζ∈Zδ

OT,α,ζ Φδ (x).

We define Tad as Tad := P(Sδ )\Sδ and restrict ourselves to the following problem Φδ (x) =

sup T ∈Tad ,α∈U,ζ∈Zδ

OT,w,z Φδ (x) =: OΦδ (x).

(9.4.24)

In other words, it is not admissible to make interventions at all points of Sδ (i.e., the continuation region is never the empty set). We can always assume that we order the points of the grid in such a way that it is not admissible to intervene at x1 ∈ Sδ . The operator Lα δ is contractive (because P ∞ ≤ 1 and rh > 0) and satisfies the discrete maximum principle, i.e., α Lα δ v1 − Lδ v2 ≤ v1 − v2 ⇒ v1 − v2 ≥ 0.

(9.4.25)

(If v is a function from Sδ into R, v ≥ 0 means v(x) ≥ 0 for all x ∈ Sδ .) The operator B ζ is nonexpansive and we need some additional hypothesis in order to be able to use a policy iteration algorithm for computing a solution of (9.4.21). We assume There exists an integer function σ : {1, 2, . . . , Nδ } × Zδ → {1, 2, . . . , Nδ } such that for all ζ ∈ Zδ and all i = 1, . . . , Nδ Γ (xi , ζ) = xσ(i,ζ) with σ(i, ζ) < i.

(9.4.26)

158

9 Viscosity Solutions

The operator Bζ defined in (9.4.22) can be rewritten as B ζ v = Bζ v + K ζ , where (Bζ , ζ ∈ Zδ ) is a family of Nδ × Nδ Markovian matrices (except for the first row) defined by: Bzi,j = 1 if j = σ(i, z) and i = 1, and 0 elsewhere. Let ζ(·) be a feedback Markovian control from Sδ into Zδ , and define the function σ ¯ on Sδ by σ ¯ (x) := σ(x, ζ(x)). Condition (9.4.26) implies that the pth composition of σ ¯ starting in T ∈ Tad will end up in Sδ \T after a finite number of iterations. We can now consider the following Howard or policy iteration algorithm to solve problem (9.4.20) in the finite set Sδ . It consists of constructing two sequences of feedback Markovian policies {(Tk , αk , ζk ), k ∈ N} and functions {vk , k ∈ N} as follows. Let v0 be a given initial function in Sδ . For k ≥ 0 we do the following iterations: –

(step 2k) Given vk , compute a feedback Markovian admissible policy (Tk+1 , αk+1 , ζk+1 ) such that (Tk+1 , αk+1 , ζk+1 ) ∈ Argmax{OT,α,ζ vk }.

(9.4.27)

T,α,ζ

In other words αk+1 (x) ∈ Argmax Lα δ vk (x); α∈U

for all x in Sδ ,

ζk+1 (x) ∈ Argmax Bδζ vk (x);

for all x in Sδ ,

β∈Zδ α

Tk+1 = {x ∈ Sδ , Lδ k+1 –

(x)

ζ

vk (x) > Bδ k+1

(x)

vk (x)}.

(step 2k + 1) Compute vk+1 as the solution of vk+1 = OTk+1 ,αk+1 ,ζk+1 vk+1 .

(9.4.28)

Set k ← k + 1 and go to step 2k. It can be proved that if (9.4.15), (9.4.18), and (9.4.26) hold, then the sequence {vk } converges to the solution Φδ of (9.4.20) and the sequence {(Tk , αk , ζk )} converges to the optimal feedback Markovian strategy. See [CMS] for a proof and [BT] for similar problems. For more information on the Howard algorithm, we refer to [Pu, LST]. For complements on numerical methods for HJB equations we refer, e.g., to [KD, LST]. Example 9.13 (Optimal Consumption and Portfolio with Both Fixed and Proportional Transaction Costs (3)). We go back to Example 9.12. We want to solve (8.2.21) numerically. We assume now that S = (0, l) × (0, l) with l > 0, and that the following boundary conditions hold:

9.5 Exercises

ψ(0, x2 ) = ψ(x1 , 0) = 0, ∂ψ ∂ψ (l, x2 ) = (x1 , l) = 0 ∂x1 ∂x2

159

for all (x1 , x2 ) in (0, l) × (0, l).

Moreover we assume that the consumption is bounded by umax > 0 so that U = [0, umax ]. Let δ > 0 be a positive step and let Sδ = {(iδ, jδ), i, j ∈ {1, . . . , N }} be the finite difference grid (we suppose that N = l/δ is an integer). We denote by ψδ the approximation of ψ on the grid. We approximate the operator Lu defined in (9.3.33) by the following finite difference operator on Sδ : 1 δ2 + Luδ ψ := −ρψ + rx1 ∂1δ+ ψ + μx2 ∂2δ+ ψ − u∂1δ− ψ + σ 2 x22 ∂22 ψ 2 and set the following boundary values: ψδ (0, x2 ) = ψδ (x1 , 0) = 0, ψδ (l − δ, x2 ) = ψδ (l, x2 ), ψδ (x1 , l − δ) = ψδ (x1 , l). We then obtain a stable approximation. Take now h≤

σx2 !2 umax μx2 rx1 + + . + δ δ δ δ

We obtain a problem of the form (9.4.20). In order to be able to apply the Howard algorithm described above, it remains to check that (9.4.26) holds. This is indeed the case since a finite number of transactions brings the state to the continuation region. The details are left as an exercise. This problem is solved in [CØS] by using another numerical method based on the iterative methods of Chap. 7.

9.5 Exercises Exercise* 9.1. Let k > 0 be a constant and define

K|x| G(x) = 1

1 for − K ≤x≤ 1 for |x| > K .

1 K,

Solve the optimal stopping problem $ & Φ(s, x) = sup E x e−ρ(s+τ ) G(B(τ )) , τ ≥0

160

9 Viscosity Solutions

where B(t) is a one-dimensional Brownian motion starting at x ∈ R. Distinguish between the two cases √ (a) K ≤ 2ρ/z , where z > 0 is the unique positive solution of the equation tgh(z) = and tgh(z) = (b) K >

√

1 , z

ez − e−z . ez + e−z

2ρ/z .

Exercise* 9.2. Assume that the state X(t) = X (w) (t) at time t obtained by using a combined control w = (u, v), where u = u(t, ω) ∈ R and v = (τ1 , τ2 , . . . ; ζ1 , ζ2 , . . .) with ζi ∈ R given by  ˜ (dt, dz), τi ≤ t < τi+1 , dX(t) = u(t)dt + dB(t) + z N R

− ) + ΔN X(τi+1 ) + ζi+1 , X(τi+1 ) = X(τi+1

X(0− ) = x ∈ R.

Assume that the cost of applying such a control is  ∞  −ρ(s+t) (w) 2 2 (w) x −ρ(s+τi ) e (X (t) + θu(t) )dt + c J (s, x) = E e , 0

i

where ρ, θ, and c are positive constants. Consider the problem to find Φ(s, x) and w∗ = (u∗ , v ∗ ) such that ∗

Φ(s, x) = inf J (w) (s, x) = J (w ) (s, x). w

(9.5.1)

Let Φ1 (s, x) = inf J (u,0) (s, x) u

be the value function if we de not allow any impulse control (i.e., v = 0) and let Φ2 (s, x) = inf J (0,v) (s, x) v

be the value function if u is fixed equal to 0, and only impulse controls are allowed. (See Exercises 3.4 and 6.1, respectively.) Prove that for i = 1, 2, there exists (s, x) ∈ R × R such that Φ(s, x) < Φi (s, x). In other words, no matter how the positive parameter values ρ, θ, and c are chosen it is never optimal for the problem (9.5.1) to choose u = 0 or v = 0 (compare with Exercise 8.2). [Hint: Use Theorem 9.8]. Exercise 9.3. Prove the inequalities (9.3.32) and (9.3.34) and verify that the inequalities hold uniformly on compact subsets of S 0 .

10 Optimal Control of Random Jump Fields and Partial Information Control

10.1 A Motivating Example Example 10.1. Suppose the density Y (t, x) of a fish population at time t ∈ [0, T ] and at the point x ∈ D ⊂ Rn (where D is a given open set) is modeled by a stochastic partial differential equation (SPDE for short) of the form   1 dY (t, x) = ∆Y (t, x) + αY (t, x) − u(t, x) dt 2  − ˜ (dt, dz); (t, x) ∈ (0, T ) × D, + βY (t, x)dB(t) + Y (t , x) z N R

(10.1.1)

where we assume that z ≥ −1 + ε a.s. ν(dz) for some constant ε > 0. The boundary conditions are: Y (0, x) = ξ(x); x ∈ D

(10.1.2)

Y (t, x) = η(t, x); (t, x) ∈ [0, T ) × ∂D.

(10.1.3)

(See Fig. 10.1.) Here dY (t, x) = dt Y (t, x) is the differential with respect to t, n

Δ = Δx =

1  ∂2 2 i=1 ∂x2i

is the Laplacian operator acting on the variable x. We assume that α and β are constants and ξ(x) and η(t, x) are given deterministic functions. The process u(t, x) ≥ 0 is our control, representing the harvesting rate at (t, x). Equation (10.1.1) is an example of a reaction-diffusion equation. With u = 0 and without the Δ-term, the equation reduces to a geometric L´evy equation describing the growth with respect to t. The Δ-term models the diffusion in space of the population.

162

10 Optimal Control of Random Jump Fields and Partial Information Control x Y (t, x) = η(t, x)

Y (0, x) = ξ(x) D (0, T ) × D t Y (t, x) = η(t, x)

T

0 Fig. 10.1. The boundary values of Y (t, x)

Let A be a family of admissible controls, contained in the set of all Ft adapted processes u(t, ω) such that (10.1.1)–(10.1.3) has a unique solution Y (t, x). Suppose the total expected utility from the harvesting rate u(·) and the corresponding terminal density Y (T, x) is given by     T uγ (t, x) J(u) = E dx dt + ρ Y (T, x)dx , (10.1.4) γ 0 D D where γ ∈ (0, 1) and ρ > 0 are constants. We want to find u∗ ∈ A such that sup J(u) = J(u∗ ).

(10.1.5)

u∈A

Such a control u∗ is called an optimal control. This is an example of a stochastic control problem for random jump fields, i.e., random fields which are solutions of stochastic partial differential equations driven by Brownian motions and Poisson random measures. How do we solve problem (10.1.5)? It is possible to use a dynamic programming approach and formulate an infinite-dimensional HJB equation for the value function (see [Mort]) but this HJB equation is difficult to use. Therefore, we will instead formulate a maximum principle for such problems (Theorem 10.2). This principle can be used to solve such stochastic control problems in some cases. We will illustrate this by solving (10.1.5) by this method.

10.2 The Maximum Principle We first give a general formulation of the stochastic control problem we consider. Suppose that the state Y (t, x) = Y (u) (t, x) at (t, x) is described by a stochastic partial differential equation of the form

10.2 The Maximum Principle

163

dY (t, x) = [LY (t, x) + b(t, x, Y (t, x), u(t, x))] dt + σ(t, x, Y (t, x), u(t, x))dB(t)  ˜ (dt, dz); (t, x) ∈ (0, T ) × D + θ(t, x, Y (t, x), u(t, x), z)N R

(10.2.1)

with boundary conditions Y (0, x) = ξ(x); x ∈ D

(10.2.2)

Y (t, x) = η(t, x); (t, x) ∈ [0, T ] × ∂D

(10.2.3)

(see Fig. 10.1.) Here L is a linear integro-differential operator acting on x and b : [0, T ] × D×R×U → R and σ : [0, T ]×D×R×U → R and θ : [0, T ]×D×R×U ×R → R are given functions and U ⊂ Rk is a given closed set of admissible control values. Let f : [0, T ] × D × R × U → R and g : D × R → R be a given profit rate function and bequest rate function, respectively. Let A be a given family of admissible controls , contained in the set of Ft -adapted right-continuous stochastic processes u(t, x) ∈ U such that (10.2.1)–(10.2.3) has a unique solution Y (t, x) and such that     T |g(x, Y (T, x))| dx < ∞. E |f (t, x, Y (t, x), u(t, x))| dx dt + 0

D

D

(10.2.4) If U ∈ A we define its performance functional J(u) by     T f (t, x, Y (t, x), u(t, x))dx dt + g(x, Y (T, x))dx . J(u) = E 0

D

D

(10.2.5)

The problem is to find u∗ ∈ A such that

sup J(u) = J(u∗ ).

(10.2.6)

u∈A

Such a process u∗ is called an it optimal control (if it exists). The number J ∗ = sup J(u)

(10.2.7)

u∈A

is called the value of this problem. We now state the maximum principle for this problem. Let R be the set of functions r : R → R. Define the Hamiltonian H : [0, T ] × D × R × U × R × R × R → R by H(t, x, y, u, p, q, r) = f (t, x, y, u) + b(t, x, y, u)p  + σ(t, x, y, u)q + θ(t, x, y, u, z)r(z)ν(dz). R

(10.2.8)

164

10 Optimal Control of Random Jump Fields and Partial Information Control x p(t, x) = 0

p(T, x) = (0, T ) × D

D

∂g (x, Y ∂y

(T, x))

t p(t, x) = 0

T

0 Fig. 10.2. The boundary values of p(t, x)

For u ∈ A we consider the following backward stochastic partial differential equation (the adjoint equation) in the three unknown adapted, right-continuous processes p(t, x) ∈ R, q(t, x) ∈ R, r(t, x, z) ∈ R; called the adjoint processes:   ∂H ∗ (t, x, Y (t, x), u(t, x), p(t, x), q(t, x), r(t, x, ·)) dt dp(t, x) = − L p(t, x) + ∂y  ˜ (dt, dz); (t, x) ∈ (0, T ) × D, + q(t, x)dB(t) + r(t, x, z)N R

(10.2.9)

∂g (x, Y (T, x)); x ∈ D ∂y

(10.2.10)

p(t, x) = 0; (t, x) ∈ (0, T ) × ∂D.

(10.2.11)

p(T, x) =

(See Fig. 10.2) Here L∗ is the adjoint of the operator L, in the sense that (L∗ ϕ, ψ) = (ϕ, Lψ) for all ϕ, ψ ∈ C0∞ (Rn ),

(10.2.12)

where (ϕ1 , ϕ2 ) =



ϕ1 (x)ϕ2 (x)dx is the inner product in L2 (Rn ).

Rn

The following result is taken from [ØPZ]. For earlier, related results see [Ø5, FØS3]. Theorem 10.2 (A Maximum Principle for Random Jump Fields [ØPZ]). Let u ˆ ∈ A with corresponding solution Yˆ (t, x) of (10.2.1)–(10.2.3) and suppose that pˆ(t, x), qˆ(t, x) and rˆ(t, x, z) is a solution of the adjoint backward SPDE (10.2.9)–(10.2.11). Suppose that the following, (i)–(iv), hold:

10.2 The Maximum Principle

165

(i) The functions y → g(x, y) and (y, u) → H(y, u) := H(t, x, y, u, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·))

(10.2.13)

are concave functions of y and (y, u), respectively, for all (t, x) ∈ ¯ [0, T ] × D. (ii) (The maximum condition) H(t, x, Yˆ (t, x), u ˆ(t, x), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) + , = sup H(t, x, Yˆ (t, x), v, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) v∈U

(10.2.14)

¯ for all (t, x) ∈ [0, T ] × D.

(iii) For all u ∈ A we have      T E (Y (x, t) − Yˆ (t, x))2 qˆ(t, x)2 + rˆ(t, x, z)2 ν(dz) dt dx < ∞ D

0

R

(10.2.15)

and (iv) E

  D

+

T

0



R

 pˆ(t, x)2 σ(t, x, Y (t, x), u(t, x))2

  θ(t, x, Y (t, x), u(t, x), z)2 ν(dz) dtdx < ∞. (10.2.16)

Then u ˆ(t) is an optimal control for the random jump field control problem (10.2.6). Proof. Choose u ∈ A and let Y (t, x) = Y (u) (t, x) be the corresponding solution of (10.2.1)–(10.2.3). Write fˆ = f (t, x, Yˆ (t, x), u ˆ(t, x)), f = f (t, x, Y (t, x), u(t, x)), ˆ gˆ = g(x, Y (T, x)), g = g(x, Y (T, x)), and similarly ˆb = b(t, x, Yˆ (t, x), u ˆ(t, x)), b = b(t, x, Y (t, x), u(t, x)), σ ˆ = σ(t, x, Yˆ (t, x), u ˆ(t, x)), σ = σ(t, x, Y (t, x), u(t, x)), and

θˆ = θ(t, x, Yˆ (t, x), u ˆ(t, x), z),

θ = θ(t, x, Y (t, x), u(t, x), z).

Moreover, put ˆ = H(t, x, Yˆ (t, x), u H ˆ(t, x), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·))

166

10 Optimal Control of Random Jump Fields and Partial Information Control

and H = H(t, x, Y (t, x), u(t, x), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)). Note that since f (t, x, y, u) does not depend on p, q or r we have  ˆ − ˆb · pˆ(t, x) − σ fˆ = H ˆ · qˆ(t, x) − θˆ · rˆ(t, x, z)ν(dz) R

and f = H − b · pˆ(t, x) − σ · qˆ(t, x) − Therefore



R

θ · rˆ(t, x, z)ν(dz).

J(ˆ u) − J(u) = I1 + I2 ,

(10.2.17)

where I1 = E



T

0

=E



0



D

T

 {fˆ − f }dx dt

    ˆ ˆ ˆ H − H − (b − b) · pˆ − (ˆ σ − σ) · qˆ − (θ − θ)rν(dz) dx dt D

D

(10.2.18)

and I2 = E

 {ˆ g − g}dx .



D

(10.2.19)

Since y → g(x, y) is concave, we have g − gˆ ≤ where

∂g (x, Yˆ (T, x))Y˜ (T, x), ∂y

Y˜ (t, x) = Y (t, x) − Yˆ (t, x); 0 ≤ t ≤ T.

(10.2.20)

(10.2.21)

Therefore, by (10.2.10) and integration by parts for jump diffusions (Lemma 3.6) we get   ∂g ˆ ˜ I2 ≥ −E (x, Y (T, x))Y (T, x)dx ∂y D  ˜ = −E pˆ(T, x) · Y (T, x)dx D  T = −E {Y˜ (t, x)dˆ p(t, x) + pˆ(t, x)dY˜ (t, x) pˆ(0, x) · Y˜ (0, x) + D

+ (σ − σ ˆ )ˆ q (t, x)}dt +

0



0

T



D

 ˆ (θ − θ)ˆ r(t, x, z)N (dt, dz) dx

10.2 The Maximum Principle

= −E

167

  ∧ ∂H ∗ ˜ Y (t, x) −L pˆ(t, x) − + pˆ(t, x)[LY˜ (t, x) − (b − ˆb)] ∂y D 0   ˆ r(t, x, z)ν(dz) dtdx , + (σ − σ ˆ )ˆ q (t, x) + (θ − θ)ˆ (10.2.22)

 

T



R

where 

∂H ∂y

∧

=

∂H (t, x, Yˆ (t, x), u ˆ(t, x), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)). ∂y

Combining (10.2.18) and (10.2.22) we obtain    T J(ˆ u) − J(u) ≥ E {Y˜ (t, x)L∗ pˆ(t, x) − pˆ(t, x)LY˜ (t, x)}dx dt 0

D

+

 

T

0

D



ˆ −H + H



∂H ∂y

∧







· Y˜ (t, x) dt dx . (10.2.23)

Since Y˜ (t, x) = pˆ(t, x) = 0 for all (t, x) ∈ (0, T ) × ∂D, we get by an easy extension of (10.2.12) that   ∗ ˜ Y (t, x)L pˆ(t, x)dx = pˆ(t, x)LY˜ (t, x)dx D

for all t ∈ (0, T ). Hence J(ˆ u) − J(u) ≥ E

D

   D

0

T



ˆ −H + H



∂H ∂y

∧







· Y˜ (t, x) dt dx . (10.2.24)

By the concavity assumption (10.2.13) we have ∂H ˆ ˆ ≤ ∂H (Yˆ , u ˆ) · (Y − Yˆ ) + (Y , u ˆ) · (u − u ˆ) H −H ∂y ∂u and the maximum condition (10.2.14) implies that ∂H ˆ (Y , u ˆ) · (u − u ˆ) ≤ 0. ∂u Hence ˆ −H + H which gives



∂H ∂y

∧

· Y˜ ≥ 0,

J(ˆ u) ≥ J(u).

Since u ˆ ∈ A was arbitrary, this shows that u ˆ is optimal.

⊓ ⊔

168

10 Optimal Control of Random Jump Fields and Partial Information Control

10.3 The Arrow Condition In many cases the Hamiltonian h(y, u) := H(t, x, y, u, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) is not concave in both variables y, u. In such cases it might be useful to replace the concavity condition in (y, u) (see (10.2.13)) by a weaker condition, called the Arrow condition: For each fixed t, x the function ˆ h(y) := max H(t, x, y, v, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) v∈V

(10.3.1)

exists and is a concave function of y. We now get the following extension of Theorem 10.2: Corollary 10.3 (Strengthened Maximum Principle). Let u ˆ(t, x), Yˆ (t, x), pˆ(t, x), qˆ(t, x) and rˆ(t, x, ·) be as in Theorem 10.2. Assume that g(x, y) is concave in y for each x and that the Arrow condition (10.3.1) and the maximum condition (10.2.14) hold, in addition to (10.2.15) and (10.2.16). Then u ˆ(t, x) is an optimal control for the stochastic control problem (10.2.6). Proof. We proceed as in the proof of Theorem 10.2 up to and including (10.2.22). Then to obtain  ∧ ∂H ˆ H −H − · (Y − Yˆ ) ≤ 0 (10.3.2) ∂y we note that ˆ− H −H

∂H ˆ (Y , u ˆ) · (Y − Yˆ ) = h(Y (t, x), u(t, x)) − h(Yˆ (t, x), u ˆ(t, x)) ∂y ∂h ˆ − (Y (t, x), u ˆ(t, x))(Y (t, x) − Yˆ (t, x)). ∂y (10.3.3)

This is ≤ 0 by the same argument as in the deterministic case. See [SS], Theorem 5, p. 107–108. For completeness we give the details: Note that by the maximum condition (10.2.14) we have ˆ Yˆ (t, x)). h(Yˆ (t, x), u ˆ(t, x)) = h(

(10.3.4)

ˆ Moreover, by definition of h, ˆ h(y, u) ≤ h(y) for all y, u.

(10.3.5)

10.3 The Arrow Condition

169

Therefore, subtracting (10.3.4) from (10.3.5) gives ˆ ˆ Yˆ (t, x)) h(y, u) − h(Yˆ (t, x), u ˆ(t, x)) ≤ h(y) − h(

for all y, u.

(10.3.6)

Accordingly, to prove (10.3.2) it suffices to prove that (see (10.3.3)) ˆ (t, x)) − h( ˆ Yˆ (t, x)) − ∂h (Yˆ (t, x), u h(Y ˆ(t, x)) · (Y (t, x) − Yˆ (t, x)) ≤ 0. (10.3.7) ∂y ˆ To this end, note that since the function h(y) is concave it follows by a standard separating hyperplane argument (see e.g., [R], Chap. 5, Sect. 23) that ˆ there exists a supergradient a ∈ R for h(y) at y = Yˆ (t, x), i.e., ˆ ˆ Yˆ (t, x)) ≤ a · (y − Yˆ (t, x)) h(y) − h(

for all y.

(10.3.8)

Define ϕ(y) = h(y, u ˆ(t, x)) − h(Yˆ (t, x), u ˆ(t, x)) − a · (y − Yˆ (t, x)) ; y ∈ R. Then by (10.3.6) and (10.3.8) we have ϕ(y) ≤ 0

for all y ∈ R.

Moreover, by definition of ϕ we have ϕ(Yˆ (t, x)) = 0. Therefore

∂h ˆ ϕ′ (Yˆ (t, x)) = (Y (t, x), u ˆ(t, x)) = a. ∂y Combining this with (10.3.8), we obtain (10.3.7) and the proof is complete. ⊓ ⊔ 10.3.1 Return to Example 10.1 As an illustration of the maximum principle let us apply it to solve the problem in Example 10.1. In this case the Hamiltonian is  uγ H(t, x, y, u, p, q, r) = + (αy − u)p + βyq + y r(z)zν(dz) (10.3.9) γ R

which is clearly concave in (y, u). The adjoint equation is    1 dp(t, x) = − ∆p(t, x) + αp(t, x) + βq(t, x) + r(t, x, z)zν(dz) dt 2 R  ˜ + q(t, x)dB(t) + r(t, x, z)N (dt, dz); (t, x) ∈ (0, T ) × D R

(10.3.10)

170

10 Optimal Control of Random Jump Fields and Partial Information Control

p(T, x) = ρ; x ∈ D,

(10.3.11)

p(t, x) = 0 ; (t, x) ∈ (0, T ) × ∂D.

(10.3.12)

Since the coefficients α, β and the boundary value ρ are all deterministic, we see that we can choose q(t, x) = r(t, x, z) = 0 and solve the resulting deterministic equation 1 ∂ p(t, x) = − ∆p(t, x) − αp(t, x) ; (t, x) ∈ (0, T ) × D ∂t 2

(10.3.13)

(together with (10.3.11)–(10.3.12)) for a deterministic solution p(t, x). This is a classical boundary value problem, and it is well known that the solution can be expressed as follows (Fig. 10.3): p(t, x) = ρeα(T −t) P [W x (s) ∈ D

for all s ∈ [t, T ]],

(10.3.14)

where W x (·) denotes an auxiliary n-dimensional Brownian motion starting at x ∈ Rn with probability law P . (See e.g., [KS, Chap. 4], or [Ø1, Chap. 9]). The function  uγ + (αy − u)p + βyq + y r(z)zν(dz) u → H(t, x, y, u, p, q, r) = γ R is maximal when u=u ˆ(t, x) = (p(t, x))1/(γ−1) ,

(10.3.15)

where p(t, x) is given by (10.3.14).

D

x W x (T )

T

t Fig. 10.3. Interpretation of the function p(t, x)

10.3 The Arrow Condition

171

With this choice of u ˆ(t, x) we see that all the conditions of Theorem 10.2 are satisfied and we conclude that u ˆ(t, x) is an optimal harvesting rate for Example 10.1. Example 10.4 ([ØPZ]). The solution u ˆ(t, x) of Example 10.1 is a bit degenerate, in the sense that it is deterministic and hence independent of the history of the population density Y (t, x). The mathematical reason for this is the deterministic parameters of the adjoint equation, including the terminal condition p(T, x) = ρ. Therefore, let us consider a more general situation, where the performance functional J(u) of (10.1.4) is replaced by    T uγ (t, x) dxdt + g(x, Y (T, x))dx , (10.3.16) J(u) = E γ 0 R R where g : R2 → R is a given C 1 -function . The Hamiltonian remains the same as in Example 10.1, and hence the candidate u ˆ(t, x) for the optimal control has the same form as in (10.3.15), i.e., 1

u ˆ(t, x) = (p(t, x)) γ−1 .

(10.3.17)

The difference is that now we have to work harder to find p(t, x). The backward SPDE is now    1 dp(t, x) = − αp(t, x) + βq(t, x) + r(t, x, z)zν(dz) + ∆p(t, x) dt 2 R  ˜ (dt, dz); (t, x) ∈ [0, T ] × R, + q(t, x)dB(t) + r(t, x, z)N R

(10.3.18)

p(T, x) = F (x); x ∈ R.

(10.3.19)

lim p(t, x) = 0; t ∈ [0, T ],

(10.3.20)

|x|→∞

where

∂g (x, Y (T, x)); x ∈ R. ∂y To solve this equation we proceed as follows: Put p˜(t, x) = eαt p(t, x). F (x) = F (x, ω) =

(10.3.21)

(10.3.22)

This transforms (10.3.18)–(10.3.20) into 1 d˜ p(t, x) = −βeαt q(t, x)dt − ∆˜ p(t, x)dt 2  + eαt r(t, x, z)zν(dz)dt + eαt q(t, x)dB(t) R αt ˜ (dt, dz); t < T, +e r(t, x, z)N R

(10.3.23)

172

10 Optimal Control of Random Jump Fields and Partial Information Control

p˜(T, x) = eαT F (x),

(10.3.24)

lim p˜(t, x) = 0.

(10.3.25)

|x|→∞

Define the probability measure P0 on FT by dP0 (ω) = Z(T )dP (ω), where Z(t)

  t 1 ˜ (ds, dz) = exp βB(t) − β 2 t + ln(1 + z)N 2 0 R   t + {ln(1 + z) − z}ν(dz)ds ; 0 ≤ t ≤ T. (10.3.26) 0

R

Then by the Girsanov theorem (use Theorem 1.35 with u = −β and θ(t, z) = −z) the process B0 (t) := B(t) − βt is a Brownian motion with respect to P0 ˜0 (dt, dz) := N ˜ (dt, dz)−zν(dz)dt is a compensated and the random measure N Poisson random measure with respect to P0 . ˜0 (dt, dz) (10.3.23) gets the form In terms of dB0 (t) and N 1 p(t, x)dt + eαt q(t, x)dB0 (t) d˜ p(t, x) = − ∆˜ 2  ˜0 (dt, dz). + eαt r(t, x, z)N

(10.3.27)

R

Suppose E0



R

 F 2 (x)dx < ∞,

(10.3.28)

where E0 denotes the expectation with respect to P0 . Then by the Itˆ o representation theorem (see e.g., [I]), there exists for a.a., x ∈ R a unique pair of adapted processes (ϕ(t, x), ψ(t, x, z)) such that    T

T

ϕ2 (t, x)dt +

E0

0

0

R

ψ 2 (t, x, z)ν(dz)dt < ∞

and eαT F (x) = h(x) +



0

where

T

ϕ(t, x)dB0 (t) +



0

T



˜0 (dt, dz), (10.3.29) ψ(t, x, z)N

R

$ & h(x) = E0 eαT F (x) .

10.3 The Arrow Condition

173

Let Qt be the heat operator, defined by    (x − y)2 (Qt f )(x) = (2πt)−1/2 f (y) exp − dy; f ∈ D 2t R where D is the set of functions f : R → R such that the integral exists. Define   t  t ˜ ψ(s, ·, z)N0 (ds, dz) + h(·) (x) ϕ(s, ·)rmdB0 (s) + pˆ(t, x) := QT −t =



0

0

0

t

QT −t ϕ(s, ·)(x)dB0 (s) +

+ QT −t h(x). Then, since

 t 0

R

R

˜0 (ds, dz) QT −t ψ(s, ·, z)(x)N (10.3.30)

d 1 Qt f = ∆(Qt f ), dt 2

we see that  t 

  1 − Δ(QT −t ϕ(s, ·))(x) dB0 (s) dt 2

dˆ p(t, x)x = QT −t ϕ(t, ·)(x)dB0 (t) + 0  ˜0 (ds, dz) + QT −t ψ(t, ·, z)(x)N R   t  1 ˜ − ∆(QT −t ψ(s, ·, z))(x)N0 (ds, dz) dt + 2 R 0 1 − ∆(QT −t h(x))dt 2  t 1 =− ∆ QT −t ϕ(s, ·)(x)dB0 (s) 2 0   t ˜0 (ds, dz) + QT −t h(x) dt QT −t ψ(s, ·z)(x)N + 0 R  ˜0 (dt, dz) + QT −t ϕ(t, ·)(x)dB0 (t) + QT −t ψ(t, ·, z)(x)N R

1 = − ∆ˆ p(t, x)dt + QT −t ϕ(t, ·)(x)dB0 (t) 2  +

R

˜0 (dt, dz). QT −t ψ(t, ·, z)(x)N

(10.3.31)

Comparing with (10.3.27) we see that the triple (˜ p, q, r) given by p˜(t, x) := pˆ(t, x),

(10.3.32)

q(t, x) := e−αt QT −t ϕ(t, ·)(x),

(10.3.33)

−αt

r(t, x, z) := e

QT −t ψ(t, ·, z)(x)

(10.3.34)

174

10 Optimal Control of Random Jump Fields and Partial Information Control

solves the backward SPDE (10.3.27), and hence it solves (10.3.23), together with the terminal values (10.3.24) and (10.3.25). We have proved: Theorem 10.5. Assume that (10.3.28) holds Then the optimal control of the problem (10.1.1)–(10.1.3), (10.1.5), with performance functional J(u) as in (10.3.16), satisfies u ˆ(t, x) = (p(t, x))1/(γ−1) , where p(t, x) = e−αt pˆ(t, x) with pˆ(t, x) defined by (10.3.30) and (10.3.29).

10.4 Controls Which do not Depend on x In some cases, for example in the application to partial observation control (see Sect. 10.5), it is of interest to consider only controls u(t) = u(t, x) which do not depend on the space variable x. Thus we let the set A0 of admissible controls be defined by A0 = {u ∈ A; u(t, x) = u(t) does not depend on x}.

(10.4.1)

The performance functional J(u) is as before,     T J(u) = E g(x, , Y (T, x))dx . f (t, x, Y (t, x), u(t))dx dt + 0

D

D

(10.4.2)

We want to find J0∗ ∈ R and u∗0 ∈ A0 such that J0∗ = sup J(u) = J(u∗0 ).

(10.4.3)

u∈A0

It turns out that one can formulate an analog of Theorem 10.2 for this case: Theorem 10.6 (Controls Which do not Depend on x). Suppose u ˆ ∈ A0 with corresponding solutions Yˆ (t, x) of (10.2.1)–(10.2.3) and pˆ(t, x), qˆ(t, x), rˆ(t, x, z) of (10.2.9)–(10.2.11), respectively. Assume that (10.2.15)–(10.2.16) hold, together with the following, (10.4.4)–(10.4.5): – The functions y → g(x, y) and (y, u) → H(y, u) := H(t, x, y, u, pˆ(t, x), qˆ(t, x), rˆ(t, x, z)) ; (y, u) ∈ R × U (10.4.4) are concave, for all (t, x) ∈ (0, T ) × D

10.4 Controls Which do not Depend on x

175

– (the average maximum condition)  H(t, x, Yˆ (t, x), u ˆ(t), pˆ(t, x), rˆ(t, x, ·))dx D  = sup H(t, x, Yˆ (t, x), v, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·))dx. (10.4.5) v∈U

D

Then u ˆ(t) is an optimal control for the problem (10.4.3). Proof. We proceed as in the proof of Theorem 10.2. Let u ∈ A0 with corresponding solution Y (t, x) of (10.2.1)–(10.2.3). With uˆ ∈ A0 as in (10.4.5), consider    T {fˆ − f }dxdt + {ˆ g − g}dx , (10.4.6) J(ˆ u) − J(u) = E 0

D

D

where fˆ = f (t, x, Yˆ (t, x), u ˆ(t)), gˆ = g(x, Yˆ (T, x))

f = f (t, x, Y (t, x), u(t)) and g = g(x, Y (T, x)).

ˆ θ and setting Using a similar shorthand notation for ˆb, b, σ ˆ , σ and θ, ˆ = H(t, x, Yˆ (t, x), u H ˆ(t), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) and H = H(t, x, Y (t, x), u(t), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)) we see that (10.4.6) can be written J(ˆ u) − J(u) = I1 + I2 ,

(10.4.7)

where I1 = E



0

T

  D

ˆ − H − (ˆb − b)ˆ H p − (ˆ σ − σ)ˆ q−

and I2 = E



D

 {ˆ g − g}dx .



R

  ˆ (θ − θ)ˆ rν(dz) dx dt (10.4.8)

(10.4.9)

By concavity of the function y → g(x, y) we have   + , ∂g g(x, Y (T, x)) − g(x, Yˆ (T, x)) dx ≤ (x, Yˆ (T, x)) · Y˜ (T, x)dx, D D ∂y (10.4.10)

176

10 Optimal Control of Random Jump Fields and Partial Information Control

where

Y˜ (t, x) = Y (t, x) − Yˆ (t, x).

Therefore, as in the proof of Theorem 10.2,     

∧ T ∂H ∗ ˜ Y (t, x) − − L pˆ(t, x) I2 ≥ −E ∂y 0 D

+ pˆ(t, x)[LY˜ (t, x) + (b − ˆb)] + (σ − σ ˆ )ˆ q (t, x)    ˆ r(t, x, z)ν(dz) dx dt , + (θ − θ)ˆ

(10.4.11)

R

where 

∂H ∂y

∧

=

∂H (t, x, Yˆ (t, x), u ˆ(t), pˆ(t, x), qˆ(t, x), rˆ(t, x, ·)). ∂y

Summing (10.4.8) and (10.4.11) we get, as in (10.2.24),  

 ∧   T ∂H ˆ − H + Y˜ · H J(ˆ u) − J(u) ≥ E dx dt . ∂y 0 D

(10.4.12)

Since H(y, u) is concave, we have ∂H ˆ ˆ ≤ ∂H (Yˆ , u ˆ) · (Y − Yˆ ) + (Y , u ˆ) · (u − u ˆ). H −H ∂y ∂u Combining (10.4.12) and (10.4.13) lead to   T

J(ˆ u) − J(u) ≥ E

0

= −E



0



∂H ˆ (Y , u ˆ) · (u − u ˆ)dx dt − ∂u D

T

∂ ∂v







H(t, x, Yˆ , v, pˆ, qˆ, rˆ)dx

D

(10.4.13)

· ((u(t) − u ˆ(t))dt v=ˆ u(t)



≥ 0, since v = u ˆ(t) maximizes v→



H(t, x, Yˆ (t, x), v, pˆ(t, x), qˆ(t, x), rˆ(t, x, ·))dx.

 ⊓

G

10.5 Connection with Partial Observation Control In this section we present the well-known connection between partial observation control of ordinary stochastic differential equations and full observation control of stochastic partial differential equations, discussed above. See e.g., [Mort, Ben1, Ben2, Ben3], and the references therein.

10.5 Connection with Partial Observation Control

177

To this end we need to recall briefly some key concepts and results from nonlinear filtering theory. We refer to [D2, DM, Kalli, Pa1, Pa2] for more details. See also [MP]. Suppose the state process x(t) = x(u) (t) ∈ Rn and its corresponding observation process ζ(t) ∈ Rn are given by the following stochastic differential equations, (10.5.1)–(10.5.4): – (State process) dx(t) = α(x(t), ζ(t), u(t))dt + β(x(t), ζ(t), u(t))dv(t)  ˜ (dt, dz); t ∈ [0, T ], (10.5.1) + ξ(x(t− ), ζ(t− ), u(t− ), z)N R

where T > 0 is a fixed constant x(0) has density F (x), i.e.,  E[ϕ(x(0))] = ϕ(x)F (x)dx; ϕ ∈ C0 (Rn ).

(10.5.2)

R

– (Observation process) dζ(t) = h(x(t))dt + dw(t); t ∈ [0, T ], ζ(0) = 0.

(10.5.3) (10.5.4)

Here α : Rn ×Rm ×U → Rn , β : Rn ×Rm ×U → Rn×n , ξ : Rn ×Rm ×U ×Rn → Rn×n and h : Rn → Rm are given deterministic functions, v(t) = v(t, ω) ∈ Rn and w(t) = w(t, ω) ∈ Rm are independent Brownian motions on Rn and ˜ (dt, dz) is a compensated Poisson random measure of Rm , respectively, and N dimension n. For simplicity we assume that h is a bounded function. The process u(t) = u(t, ω) is our control process, assumed to have values in a given closed set U ⊂ Rk . We require that u(t) be adapted to the filtration Zt generated by the observations ζ(s); s ≤ t. We call u(t) admissible if, in addition, (10.5.1)– (10.5.4) has a unique strong solution (x(t), ζ(t)) such that  T

E

0

|ℓ(x(t), u(t))|dt + |k(x(T ))| < ∞,

(10.5.5)

where ℓ : Rn × U → R and k : Rn → R are given functions, called the profit rate and the bequest function, respectively. The set of all admissible controls is denoted by AZ . For u ∈ AZ we define the performance functional  T

J(u) = E

ℓ(x(t), u(t))dt + k(x(T )) .

0

(10.5.6)

178

10 Optimal Control of Random Jump Fields and Partial Information Control

The stochastic control problem with partial observation is to find J ∗ ∈ R and u∗ ∈ AZ such that (10.5.7) J ∗ = sup J(u) = J(u∗ ). u∈AZ

We now show that (under some conditions) this problem can be transformed into a full information control problem of an associated SPDE, with controls u(t) not depending on x, as described in Sect. 10.4. Define    t  1 t 2 h (x(s))ds (10.5.8) h(x(s))dw(s) − Mt (ω) = exp − 2 0 0 and define the probability measure Q on FT by dQ(ω) = Mt (ω)dP (ω) on FT .

(10.5.9)

Then Mt is a martingale with respect to P and we have dQ(ω) = Mt (ω)dP (ω) on Ft ; 0 ≤ t ≤ T.

(10.5.10)

Then by the Girsanov theorem the observation process  t ζ(t) = h(x(s))ds + w(t) 0

is a Brownian motion with respect to Q. Put   t  1 t 2 h (x(s))ds h(x(s))dw(s) + Kt (ω) := Mt−1 (ω) = exp 2 0 0  t   t 1 2 = exp h(x(s))dζ(s) − h (x(s))ds . (10.5.11) 2 0 0 Then Kt is a martingale with respect to Q and dP (ω) = Kt (ω)dQ(ω) on Ft ; 0 ≤ t ≤ T. by

(10.5.12)

For fixed ζ ∈ R, u ∈ U define the integro-differential operator A = Aζ,u Aζ,u ϕ(x) =

n 

αi (x, ζ, u)

i=1

+

n ∂ϕ 1  ∂2ϕ (x) + (ββ T )i,j (x, ζ, u) ∂xi 2 i,j=1 ∂xi ∂xj

n  +  ϕ(x + ξ (j) (x, ζ, u, zj )) − ϕ(x) j=1

R

, −∇ϕ(x) · ξ (j) (x, ζ, u, zj ) νj (dzj )

(10.5.13)

10.5 Connection with Partial Observation Control

179

for ϕ ∈ C02 (Rn ), i.e., Aζ,u is the generator of x(t) if we regard ζ and u as known constants. (Here ξ (j) denotes the jth column of the n × n matrix ξ). Let A∗ be the adjoint of A, in the sense that (Aϕ, ψ)L2 (Rn ) = (ϕ, A∗ ψ)L2 (Rn ) ; ϕ, ψ ∈ C02 (Rn ).

(10.5.14)

Suppose there exists a stochastic process y(t, x) = y(t, x, ω) ; (t, x, ω) ∈ [0, T ] × Rn × Ω such that  EQ [ϕ(x(t))Kt | Zt ] = ϕ(x)y(t, x)dx ; ϕ ∈ C0 (Rn ). (10.5.15) Rn

Then y(t, x) is called the unnormalized conditional density of x(t) given Zt . Conditions which imply the existence of y(t, x) can be found in [GK]. Under certain conditions the process y(t, x) satisfies the Duncan– Mortensen–Zakai equation dy(t, x) = A∗ζ(t),u(t) y(t, x)dt + h(x)y(t, x)dζ(t); t > 0,

(10.5.16)

y(0, x) = F (x),

(10.5.17)

where F (x) is the density of x(0) (see (10.5.2)). In terms of y(t, x) we can now rewrite the performance functional (10.5.6) as follows:

J(u) = E





T

ℓ(x(t), u(t))dt + k(x(T ))

0

=



T

0

= EQ

EQ [ℓ(x(t), u(t))Kt ]dt + EQ [k(x(T ))KT ] 

T

EQ [ℓ(x(t), u(t))Kt | Zt ]dt + k(x(T ))KT

0

= EQ



T

EQ [ℓ(x(t), v)Kt | Zt ]v=u(t) dt +

0

= EQ



T

0

= EQ



0



R

T



R

 ℓ(x, v)y(t, x)dx



dt +



k(x)y(T, x)dx

R





k(x)y(T, x)dx

R

v=u(t)

  ℓ(x, u(t))y(t, x)dx dt + k(x)y(T, x)dx . R

(10.5.18)

This is a performance criterion of the type discussed in Sects. 10.1–10.4, with the control u(t) not depending on x.

180

10 Optimal Control of Random Jump Fields and Partial Information Control

We summarize this as follows: Theorem 10.7. The partial observation SDE control problem (10.5.1)– (10.5.7) can be rewritten as the full observation SPDE control problem J ∗ = sup J(u) = J(u∗ ) u∈AZ

where J(u) = EQ



0

T



R

  ℓ(x, u(t))y(t, x)dx dt + k(x)y(T, x)dx , R

with y(t, x) (the unnormalized conditional density of x(t) given Zt ) given by the Duncan–Mortensen–Zakai equation (10.5.16), which is driven by the observation process ζ(t). This process is a Brownian motion with respect to the measure Q defined by (10.5.9).

10.6 Exercises Exercise* 10.1. Transform the following partial observation SDE control problems into complete observation SPDE control problems: (a) (The partially observed linear-quadratic control problem)

dx(t) = (αx(t) + u(t))dt + σdv(t); t > 0, (state): x(0) has density F (x),

dζ(t) = h(x(t))dt + dw(t); t > 0, (observations): ζ(0) = 0,  T   performance: J(u) = E x2 (t) + θu2 (t) dt , 0

J ∗ = inf J(u). u∈AZ

Here α, σ = 0, θ > 0 are constants and F and h are given functions. (b) (The partially observed optimal portfolio problem)

dx(t) = x(t)[αu(t)dt + βu(t)dv(t)]; t > 0 (state): x(0) has density F (x),

dζ(t) = x(t)dt + dw(t); t > 0, (observations): ζ(0) = 0. performance:

J(u) = E[X γ (T )], J ∗ = sup J(u). u∈AZ

Here α > 0, β > 0 and γ ∈ (0, 1) are constants.

10.6 Exercises

181

We may interpret u(t) as the portfolio representing the fraction of the total wealth x(t) invested in the risky asset with price dynamics dS1 (t) = S1 (t)[αdt + βdv(t)]; t > 0, S1 (0) > 0. The remaining fraction 1 − π(t) is then invested in the other investment alternative, being a risk free asset with price S0 (t) = 1 for all t. Exercise 10.2 (Terminal Conditions). Let Y (t, x) = Y (u) (t, x) be as in (10.1.1)–(10.1.3) and define    T ˜ ln u(t, x)dx dt ; u ∈ A, (10.6.1) J(u) = E 0

D

where A˜ is the set of controls in A (see (10.1.3)) such that the terminal constraint   Y (u) (T, x)dx ≥ 0 (10.6.2) E D

holds. We consider the constrained stochastic control problem to find J˜ ∈ R and u ˜ ∈ A˜ such that J˜ = sup J(u) = J(˜ u). (10.6.3) ˜ u∈A

To solve this problem we use the Lagrange multiplier method as follows: (a) Fix λ > 0 and solve the unconstrained control problem to find Jλ∗ ∈ R and u∗λ ∈ A such that (10.6.4) Jλ∗ = sup Jλ (u) = Jλ (u∗λ ), u∈A

where Jλ (u) = E



0

T



D

  ln u(t, x)dx dt + λY (T, x)dx .

(10.6.5)

D

[Hint: Use the method in Example 10.1.] ˆ > 0 such that (b) Suppose there exists λ   ∗ Y (uλˆ ) (T, x)dx = 0, E

(10.6.6)

D

where u∗λˆ is the corresponding solution of the unconstrained problem (10.6.4)– ˆ Show that then in fact u˜ := u∗ ∈ A˜ solves the constrained (10.6.5) with λ = λ. ˆ λ problem (10.6.1)–(10.6.3) and hence J˜ = J ∗ . ˆ λ

(c) Use the above to solve the constrained stochastic control problem (10.6.3). Exercise 10.3 (Controls Which do not Depend on x). Consider Example 10.1 again, but this time we only allow controls u(t, x) = u(t) which do not depend on x. Use Theorem 10.6 to find the optimal control u∗ (t) in this case.

11 Solutions of Selected Exercises

11.1 Exercises of Chapter 1 Exercise 1.1 Choose f ∈ C 2 (R) and put Y (t) = f (X(t)). Then by the Itˆ o formula

dY (t) = f ′ (X(t))[α dt + σ dB(t)] + 12 σ 2 f ′′ (X(t))dt    + f (X(t− ) + γ(z)) − f (X(t− )) − γ(z)f ′ (X(t− )) ν(dz)dt |z| x∗ , by construction. For x < x∗ we must check that Ceλ1 x ≥ x − a .

To this end, put

k(x) = Ceλ1 x − x + a ;

Then

k(x∗ ) = k ′ (x∗ ) = 0

x ≤ x∗ .

and

k ′′ (x∗ ) = λ21 Ceλ1 x > 0 for x ≤ x∗ .

Therefore k ′ (x) < 0 for x < x∗ and hence k(x) > 0 for x < x∗ . Hence (ii) holds. (vi) We know that Aφ + f = Aφ = 0 for x < x∗ , by construction. For x > x∗ we have Aφ(s, x) = e−ρs A0 (x − a) = e−ρs (−ρ(x − a) +



x+γz x∗ } < ∞ a.s.

(11.2.8)

Some conditions are needed on σ, γ, and ν for (11.2.8) to hold. For example, it suffices that ⎫ ⎧ t  ⎬ ⎨ (11.2.9) γ zN (ds, dz) = ∞ a.s. lim X(t) = lim σ B(t) + t→∞ t→∞ ⎩ ⎭ 0

R

(xi) For (xi) to hold it suffices that

sup E x [e−2ρτ X 2 (τ )] < ∞ .

(11.2.10)

τ ∈T

Again it suffices to assume that (11.2.7) holds. Conclusion Assume that (11.2.5), (11.2.6), and (11.2.8) hold. Then the value function is Φ(s, x) = e−ρs ψ(x), where ψ(x) is given by (11.2.1) and (11.2.4). An optimal stopping time is τ ∗ = inf {t > 0; X(t) ≥ x∗ } . Exercise 2.2 Define

⎤ ⎡ ⎡ ⎤ ⎤ ⎤ ⎡ 0 0 1 dt  − ⎥ ⎢ dY (t) = ⎣dP (t)⎦ = ⎣ α P (t) ⎦ dt+⎣β P (t)⎦ dB(t)+⎣γ P (t )z N (dt, dz)⎦ . R 0 −λ Q(t) dQ(t) 0 ⎡

11.2 Exercises of Chapter 2

193

Then the generator A of Y (t) is ∂φ ∂φ 1 2 2 ∂ 2 φ ∂φ + αp − λq + 2β p Aφ(y) = Aφ(s, p, q) = ∂s ∂p ∂q ∂p2    ∂φ (s, p, q)γ zp ν(dz) . + φ(s, p + γ pz, q) − φ(s, p, q) − ∂p R

If we try φ(s, p, q) = e−ρs ψ(w)

with w = p · q ,

then Aφ(s, p, q) = e−ρs A0 ψ(w), where A0 ψ(w) = −ρψ(w) + (α − λ)wψ ′ (w) + 12 β 2 w2 ψ ′′ (w)  + {ψ((1 + γ z)w) − ψ(w) − γ wzψ ′ (w)} ν(dz). R

Consider the set U defined in Proposition 2.3: U = {y; Ag(y) + f (y) > 0} = {(s, p, q); A0 (θ w) + λ w − K > 0} = {(s, p, q); [θ(α − ρ − λ) + λ]w − K > 0} ,

+ K if θ(α − ρ − λ) + λ > 0 (s, p, q) : w > θ(α−ρ−λ)+λ = ∅ if θ(α − ρ − λ) + λ ≤ 0 By Proposition 2.4 we therefore get Case 1: Assume λ ≤ θ(λ + ρ − α). Then τ ∗ = 0 is optimal and Φ(y) = g(y) = e−ρs p · q for all y. Case 2: Assume θ(λ++ ρ − α) < λ. , K Then U = (s, w); w > λ−θ(λ+ρ−α) ⊂ D. In view of this it is natural to guess that the continuation region D has the form D = {(s, w); 0 < w∗ < w}

for some constant w∗ ; 0 < w∗
0 and C remain to be determined. Continuity and differentiability at w = w∗ give θ w∗ = C(w∗ )r +

λ K w∗ − λ+ρ−α ρ

(11.2.12)

θ = C r(w∗ )r−1 +

λ . λ+ρ−α

(11.2.13)

Combining (11.2.12) and (11.2.13) we get w∗ =

(−r)K(λ + ρ − α) (1 − r)ρ(λ − θ(λ + ρ − α))

(11.2.14)

C=

λ − θ(λ + ρ − α) · (w∗ )1−r . −r

(11.2.15)

and

11.2 Exercises of Chapter 2

195

Since we need to have w∗ > 0 we are led to the following condition: Case 2a) θ(λ + ρ − α) < λ and λ + ρ − α > 0. Then we choose r = r2 < 0, and with the corresponding values (11.2.14), (11.2.15) of w∗ and C the function φ(s, p, q) = e−ρs ψ(p · q), with ψ given by (11.2.11), is the value function of the problem. The optimal stopping time τ ∗ is (11.2.16) τ ∗ = inf {t > 0; P (t) · Q(t) ≤ w∗ } , provided that all the other conditions of Theorem 2.2 are satisfied. For condition (vi) to hold it suffices that w∗(λ − θ(λ +  ρ − α)) − K

C(w + γzw)r +

+

γzw>w∗



λ K (w + γzw) − − θ(w + γzw) λ+ρ−α ρ

ν(dz) ≤ 0. (11.2.17)

See also Remark 11.1. Case 2b) θ(λ + ρ − α) < λ and λ + ρ − α ≤ 0, i.e., α≥λ+ρ.

In this case we have Φ∗ (y) = ∞. To see this note that since t t t  (ds, dz), P (t) = p + α P (s)ds + β P (s)dB(s) + γ P (s− )z N 0

0

0

we have

E[P (t)] = p +

t

R

α E[P (s)]ds

0

which gives E[P (t)] = p eαt . Therefore E[e−ρt P (t)Q(t)] = E[pq e−ρt e−λt P (t)] = pq exp {(α − λ − ρ)t} . Hence lim E

T →∞

 T

e

−ρt



P (t)Q(t)dt = lim pq

0

if and only if α ≥ λ + ρ.

T →∞

T 0

exp {(α − λ − ρ)t} dt = ∞

196

11 Solutions of Selected Exercises

Remark 11.1 (On condition (viii) of Theorem 2.2). Consider φ(Y (t)) = e−ρt ψ(P (t)Q(t)), where P (t) = p exp

and

⎧ ⎨ ⎩

α − 12 β 2 − γ





z ν(dz) t +

R

t  0

⎫ ⎬

ln(1 + γ z)N (dt, dz) + β B(t)

⎭

R

Q(t) = q exp(−λ t). We have ⎧   ⎨ 1 2 α − λ − 2 β − γ z ν(dz) t P (t)Q(t) = pq exp ⎩ R

+

⎫ ⎬ ln(1 + γ z)N (dt, dz) + β B(t) ⎭

t  0

R

and e

−ρt

P (t)Q(t) = pq exp



α−λ−ρ−

+

1 2 2β

−γ





z ν(dz) t

R

t  0



ln(1 + γ z)N (ds, dz) + β B(t) .

R

Hence E[(e

−ρt

2

2

⎡

P (t)Q(t)) ] = (pq) E ⎣exp

⎧ ⎨ ⎩

2α − 2λ − 2ρ − β − 2γ

+2

= (pq)2 exp

⎧ ⎨ ⎩

2

 R

t  0

R

· E exp

z ν(dz) t

⎫⎤ ⎬ ln(1 + γ z)N (ds, dz) + 2β B(t) ⎦ ⎭ ⎫   ⎬

2α − 2λ − 2ρ − β 2 − 2γ





t  2

0

R

z ν(dz) t + 2β 2 t

R



ln(1 + γ z)N (dt, dz)

⎭

11.2 Exercises of Chapter 2

197

Using Exercise 1.6 we get ⎧  ⎨ 2 −ρt 2 2 2 2α−2λ−2ρ + β −2γ z ν(dz) E[(e P (t)Q(t)) ] = p q exp ⎩ R

⎫  ⎬   (1 + γ z)2 −1−2 ln(1+γ z) ν(dz) t . + ⎭ R

So condition (viii) of Theorem 2.2 holds if   2 2  γ z − 2 ln(1 + γ z) ν(dz) < 0. 2α − 2λ − 2ρ + β 2 + R

Exercise 2.3 In this case we have g(s, x) = e−ρs |x| and dX(t) = dB(t) +



˜ (dt, dz). zN

R

We look for a solution of the form

φ(s, x) = e−ρs ψ(x). The continuation region is given by D = {(s, x) ∈ R × R : φ(s, x) > g(s, x)} = {(s, x) ∈ R × R : ψ(x) ≥ |x|} Because of the symmetry we assume that D is of the form D = {(s, x) ∈ R × R; −x∗ < x < x∗ } where x∗ > 0. It is trivial that D is a Lipschitz surface and X(t) spends 0 time on ∂D. We must have Aφ ≡ 0

on D

(11.2.18)

where the generator A is given by Aφ =

∂φ 1 ∂ 2 φ + + ∂s 2 ∂x2

  R

φ(s, x + z) − φ(s, x) −

 ∂φ (s, x)z ν(dz). ∂x

198

11 Solutions of Selected Exercises

Hence (11.2.18) becomes  −ρψ(x) + 12 ψ ′′ (x) + {ψ(x + z) − ψ(x) − zψ ′ (x)} ν(dz) = 0.

(11.2.19)

R

For |x| < ξ this equation becomes, by (2.4.1) 1 ′′ −ρψ(x) + ψ (x) = 0, 2 which has the general solution √

ψ(x) =

2ρx

√

+ C2 e− 2ρx , √ where C1 , C2 are arbitrary constants. Let λ = 2ρ and −λ be the two roots of the equation 1 F (λ) := −ρ + λ2 = 0. 2 Because of the symmetry we guess that ψ(x) = C1 e

( C ' λx e + e−λx = C cosh(λx); x ∈ D = {(s, x); |x| < x∗ } 2

for some constants C > 0 and x∗ ∈ (0, ξ). Therefore

' ( C cosh λx for |x| < x∗ ψ(x) = |x| for |x| ≥ x∗ .

In order to find x∗ and C, we impose the continuity and C 1 -conditions on ψ(x) at x = x∗ : ( ' • Continuity: 1 = |x∗ | = C cosh λx∗ • C1 : 1 = Cλ sinh(λx∗ ) It follows that: C=

x∗ cosh(λx∗ )

and x∗ is the solution of

(11.2.20)

' ( 1 . (11.2.21) tgh λx∗ = λx∗ Figure 11.1 illustrates that there exists a unique solution for (11.2.21). Finally we have to verify that the conditions of Theorem 2.2 hold. We check some: (ii) ψ(x) ≥ |x| for (s, x) ∈ D. Define h(x) = C cosh(λx) − x ; x > 0.

11.2 Exercises of Chapter 2

199

y

tgh(λx) 1/λx x λx∗

Fig. 11.1. The value of x∗ y

|x| C cosh(λx)

x −x∗

x∗

Fig. 11.2. The function ψ

Then h(x∗ ) = h′ (x∗ ) = 0 and h′′ (x) = Cλ2 cosh(λx) > 0 for all x. Hence h(x) > 0 for 0 < x < x∗ , so (ii) holds. See Fig. 11.2. ¯ (vi) Aψ ≤ 0 outside D. This holds since, by (2.4.2) and (2.4.3),  Aψ(x) = −ρ|x| + {|x + z| − x − z} ν(dz) ≤ 0 for all x > x∗ . R

Since all the conditions of Theorem 2.2 are satisfied, we conclude that φ(s, y) = e−ρs ψ(y) is the optimal value function and τ ∗ = inf {t > 0; |X(t)| = x∗ }

200

11 Solutions of Selected Exercises

Exercise 2.4 From Example 2.5 we know that in the no delay case (δ = 0) the solution of the problem (2.4.8) is the following (under some additional assumptions on the L´evy measure ν): (11.2.22) Φ0 (s, x) = e−ρs Ψ0 (x), where

x−q Ψ0 (x) = C0 xλ

; ;

x ≥ x∗0 . 0 < x < x∗0

Here λ > 1 is uniquely determined by the equation    (1 + z)λ − 1 − λ z ν(dz) = 0, −ρ + μλ + 12 σ 2 λ(λ − 1) +

(11.2.23)

(11.2.24)

R

and x∗0 and C0 are given by

x∗0 = C0 =

λq , λ−1

1 ∗ 1−λ (x ) . λ 0

(11.2.25) (11.2.26)

The corresponding optimal stopping time τ ∗ ∈ T0 is τ ∗ = inf {t > 0; X(t) ≥ x∗0 } .

(11.2.27)

Thus it is optimal to sell at the first time the price X(t) equals or exceeds the value x∗0 . To find the solution in the delay case (δ > 0) we note that we have f = 0 and g(y) = g(s, x) = e−ρs (x − q) Hence, by (2.4.5), g˜δ (y) = E y [g(Y (δ))] = E s,x [e−ρ(s+δ) (X(δ) − q)] = e−ρ(s+δ) (E x [X(δ)] − q) = e−ρ(s+δ) (x eμδ − q) = e−ρs+δ(μ−ρ) (x − q e−μδ ) = K e−ρs (x − q˜),

(11.2.28)

where K = eδ(μ−ρ)

and q˜ = q e−μδ .

(11.2.29)

Thus g˜δ has the same form as g, so we can apply the results (11.2.22)–(11.2.27) ˜ to find Φ(y) and the corresponding optimal τ ∗ :

11.2 Exercises of Chapter 2

˜ ˜ x) = e−ρs Ψ˜ (x) Φ(y) = Φ(s, where

K(x − q˜) ; Ψ˜ (x) = C˜ xλ ;

x≥x ˜∗ 0 0; X(t) ≥ x ˜∗ } + δ. Remark 11.2. Assume for example that μ > 0. Then comparing (11.2.32) with the non-delayed case (11.2.25) we see that q˜ > q and hence x ˜∗ < x∗0 Thus, in terms of the delayed effect of the stopping time formulation (see (2.3.4)), it is optimal to stop at the first time t = τ˜∗ when X(t) ≥ x ˜∗ . This is sooner than in the non-delayed case, because of the anticipation that during the delay time interval [τ ∗ , τ ∗ + δ] X(t) is likely to increase (since μ > 0). See Fig. 11.3.

202

11 Solutions of Selected Exercises

X(t)

(δ = 0 case)

x∗0

x˜∗

(delay case)

δ 7 89 :

τ˜∗

x∗0 =

λq λ−1

x˜∗ =

λqe−μδ λ−1

τ0∗

α∗ = τ˜∗ + δ

Fig. 11.3. The optimal stopping times for Exercise 2.4 (μ > 0)

11.3 Exercises of Chapter 3 Exercise 3.1 Put Y (t) =

  s+t . X(t)

Note: If we put S = {(s, x); s < T } then τS = inf {t > 0; Y s,x (t, x) ∈ S} = T − s. The generator of Y (t) is ∂φ 1 2 ∂ 2 φ ∂φ Au φ(y) = Au φ(s, x) = + (μ − ρ x − u) + σ ∂s ∂x 2 ∂x2   ∂φ · θ z ν(dz). + φ(s, x + θ z) − φ(s, x) − ∂x R

So the conditions of Theorem 3.1 get the form γ

(i) Au φ(s, x) + e−δs uγ ≤ 0 for all u ≥ 0, s < T (ii) lims→T − φ(s, x) = λ x

t

11.3 Exercises of Chapter 3

203

(iv) {φ− (Y (τ ))}τ ≤τS is uniformly integrable γ (v) Auˆ φ(s, x) + e−δs uˆγ = 0 for s < T in addition to requirements (iii) and (vi). We try a function φ of the form φ(s, x) = h(s) + k(s)x for suitable functions h(s), k(s). Then the conditions above get the form γ

(i)’ h′ (s) + k ′ (s)x + (μ − ρ x − u)k(s) + e−δs uγ  + {h(s) + k(s)(x + γ z) − h(s) − k(s)x − k(s)γ z} ν(dz) ≤ 0 R

i.e. e−δs

uγ γ

+ h′ (s) + k ′ (s)x + (μ − ρ x − u)k(s) ≤ 0

for all s < T, u ≥ 0

(ii)’ h(T ) = 0, k(T ) = λ (iv)’ {h(τ ) + k(τ )X(τ )}τ ≤τS is uniformly integrable. γ (v)’ h′ (s) + k ′ (s)x + (μ − ρ x − u ˆ)k(s) + e−δs uˆγ = 0 From (i)’ and (v)’ we get −k(s) + e−δs u ˆγ−1 = 0 or

' ( 1 u ˆ=u ˆ(s) = eδs k(s) γ−1 .

Combined with (v)’ this gives

(1) k ′ (s) − ρ k(s) = 0 so k(s) = λ eρ(s−T ) γ (2) h′ (s) = (ˆ u(s) − μ)k(s) − e−δs uˆ γ(s) , h(T ) = 0 Note that γ ' δs ( γ−1 1 ' δs ( γ−1 −δs e k(x) k(s) − μ k(s) − e h (s) = e k(s) γ ' γ (

′

δs

γ

−δs 1−

γ−1 = e γ−1 k(s) γ−1 − μ k(s) − e ·   γ δs = e γ−1 k(s) γ−1 1 − γ1 − μ k(s) < 0 .

1 γ

γ

· k(s) γ−1

Hence, since h(T ) = 0, we have h(s) > 0 for s < T . Therefore φ(s, x) = h(s) + k(s)x ≥ 0 . Clearly φ satisfies (i), (ii), (iv) and (v). It remains to check (vi), i.e., that {h(τ ) + k(τ )X(τ )}τ ≤T

204

11 Solutions of Selected Exercises

is uniformly integrable, and to check (iii). For these properties to hold some conditions on ν must be imposed. We omit the details. We conclude that if these conditions hold then   1 (δ + ρ)s − ρ T γ−1 u ˆ(s) = λ exp ; s≤T (11.3.1) γ−1 is the optimal control. Exercise 3.2 Define J(u) = E

 T0 0

e

−δt

uγ (t) dt + λ X(T0 ) γ

where dX(t) = (μ − ρ X(t) − u(t))dt + σ B(t) + γ

 R

The Hamiltonian is H(t, x, u, p, q, r) = e−δt

uγ γ

(dt, dz); zN

+ (μ − ρ x − u)p + σ q +



0 ≤ t ≤ T0 .

γ zr(t, z)ν(dz).

R

The adjoint equation is  ⎧ ⎪ (dt, dz); t < T0 ⎨dˆ p(t) = ρ pˆ(t)dt + σ qˆ(t)dB(t) + rˆ(t, z)N ⎪ ⎩ pˆ(T0 ) = λ

R

Since λ and ρ are deterministic, we guess that qˆ = rˆ = 0 and this gives pˆ(t) = λ eρ(t−T0 ) . Hence ˆ H(t, X(t), u, pˆ(t), qˆ(t), rˆ(t)) = e−δt

uγ γ

ˆ − u)ˆ + (μ − ρ X(t) p(t),

which is maximal when 1 ' ( 1 u=u ˆ(t) = eδt pˆ(t) γ−1 = λ γ−1 exp



(δ + ρ)t − ρ T0 γ−1



.

(11.3.2)

11.3 Exercises of Chapter 3

205

Exercise 3.3 In this case we have ⎡ ⎤ ⎡ ⎤ − − − γ (t, X(t ), u(t ), z) N (dt, dz) u(t , ω)z 1 ⎢ ⎥ ⎥ ⎢ ⎢R ⎥ ⎢R ⎥ ⎢ ⎥ ⎥ ⎢   N (dt, dz) = ⎢ dX(t) = ⎢ ⎥ ⎥ − − 2 ⎣ ⎣ ⎦ γ2 (t, X(t ), u(t ), z)N (dt, dz)⎦ z R

R

so the Hamiltonian is

H(t, x, u, p, q, r) =

 R

  u z r1 (t, z) + z 2 r2 (t, z) ν(dz)

and the adjoint equations are (g(x1 , x2 ) = −(x1 − x2 )2 )  ⎧ ⎪ (dt, dz) ; t < T ⎨dp1 (t) = r1 (t− , z)N R ⎪ ⎩ p1 (T ) = −2(X1 (T ) − X2 (T ))  ⎧ ⎪ (dt, dz) ⎨dp2 (t) = r2 (t− , z)N R ⎪ ⎩ p2 (T ) = 2(X1 (T ) − X2 (T )).

Now X1 (T ) − X2 (T ) =

T  0

R



 (dt, dz). So if u u(t− ) − z N ˆ is a given candi-

date for an optimal control we get

u(t) − z)z, rˆ1 (t, z) = −2(ˆ u ˆ2 (t, z) = 2(ˆ u(t) − z)z. This gives H(t, x, u, pˆ, qˆ, rˆ) =

 R



 u z(−2(ˆ u(t) − z)z) + z 2 2(ˆ u(t) − z)z ν(dz)

= −2u

 R



2

u ˆ(t)z − z

3



ν(dz) + 2

 R



 u ˆ(t)z 3 − z 4 ν(dz).

This is a linear expression in u, so we guess that the coefficient of u is 0, i.e., that  3 z ν(dz) u ˆ(t) = R 2 for all (t, ω) ∈ [0, T ] × Ω. (11.3.3) z ν(dz) R

206

11 Solutions of Selected Exercises

With this choice of u ˆ(t) all the conditions of the stochastic maximum principle are satisfied and we conclude that u ˆ is optimal. Note that this implies that inf E u



F−

T

2

u(t)dS1 (t)

0

=E

 T  0

=

T 0



R

2  2  z −u ˆ(t)z N (dt, dz)

E[(z 2 − u ˆ(t)z)2 ]ν(dz)dt

R

 3 z ν(dz) 2   R 2 z ν(dz). z − 2 =T z ν(dz) R

R

We see that this is 0 if and only if   3 z ν(dz) = z z 2 ν(dz) for a.a. z(ν) R

(11.3.4)

R

i.e. iff ν is supported on one point {z0 }. Only then is the market complete! See [BDLØP] for more information. Exercise 3.4 We try to find a, b such that the function ϕ(s, x) = e−ρs ψ(x) := e−ρs (ax2 + b) satisfies the conditions of (the minimum version of) Theorem 3.1. In this case the generator is Avϕ (s, x) = e−ρs Av0 ψ(x), where 1 Av0 ψ(x) = − ρψ(x) + vψ ′ (x) + σ 2 ψ ′′ (x) 2  + {ψ(x + z) − ψ(x) − zψ ′ (x)} ν(dz). R

Hence condition (i) of Theorem 3.1 becomes Av0 ψ(x) + x2 + θv 2 = −ρ(ax2 + b) + v2ax + 2

2

1 2 σ 2a + a 2



2



z 2 ν(dz) + x2 + θv 2

R

= θv + 2axv + x (1 − ρa) + a σ +



R



z 2 ν(dz) −ρb =: h(v).

11.3 Exercises of Chapter 3

207

The function h is minimal when v = u∗ (x) = −

ax . θ

(11.3.5)

With this value of v condition (v) becomes      a2 x2 1 − ρa − z 2 ν(dz) − ρb = 0. + a σ2 + θ R Hence we choose a > 0 and b such that a2 + ρθa − θ = 0 and b=

a ρ



σ2 +



R

(11.3.6)

 z 2 ν(dz) .

(11.3.7)

With these values of a and b we can easily check that ϕ(s, x) := e−ρs (ax2 + b) satisfies all the conditions of Theorem 3.1. The corresponding optimal control is given by (11.3.5). Exercise 3.5 (b) The Hamiltonian for this problem is H(t, x, u, p, q, r) = x2 + θu2 + up + σq +



˜ (dt, dz). r(t− , z)N

R

The adjoint equation is  ⎧ ⎨dp(t) = −2X(t)dt + q(t)dB(t) + r(t− , z)N ˜ (dt, dz) ; t < T R ⎩ p(T ) = 2λX(T ).

(11.3.8)

By imposing the first and second-order conditions, we see that H(t, x, u, p, q, r) is minimal for 1 (11.3.9) u = u(t) = u ˆ(t) = − p(t). 2θ In order to find a solution of (11.3.8), we consider p(t) = h(t)X(t), where h : R → R is a deterministic function such that h(T ) = 2λ. Note that u(t) = −

h(t)X(t) and 2θ

208

11 Solutions of Selected Exercises

h(t)X(t) dt + σdB(t) + dX(t) = − 2θ



˜ (dt, dz); X(0) = x. zN

R

Moreover, (11.3.8) turns into dp(t) = h(t)dX(t) + X(t)h′ (t)dt   h(t)2 ′ ˜ (dt, dz). + h (t) dt + h(t)σdB(t) + h(t) z N = X(t) − 2θ R Hence h(t) is the solution of

2 h′ (t) = h(t) 2θ − 2; t < T h(T ) = 2λ.

(11.3.10)

The general solution of (11.3.10) is 2t

√

√θ e with β = λ− λ+ θ conclude that

√ 1 + βe √θ h(t) = 2 θ 2T √ 1 − βe θ 2T −√

θ

(11.3.11)

. By using the stochastic maximum principle, we can

h(t) X(t) 2θ is the optimal control, p(t) = h(t)X(t) and q(t) = σh(t), r(t− , z) = h(t)z, where h(t) is given by (11.3.11). u∗ (t) = −

Exercise 3.6 If we try a function of the form ϕ(s, x) = e−δs ψ(x) then equations (i) and (v) for Theorem 3.1 combine to give the equation  1 sup ln c − δψ(x) + (μx − c)ψ ′ (x) + σ 2 x2 ψ ′′ (x) 2 c≥0   + {ψ(x + xθz) − ψ(x) − xθzψ ′ (x)} ν(dz) = 0. R

The function h(c) := ln c − cψ ′ (x); c > 0 is maximal when c = cˆ(x) =

1 ψ ′ (x)

.

11.4 Exercises of Chapter 4

209

If we set ψ(x) = a ln x + b where a, b are constants, a > 0, then this gives cˆ(x) =

x , a

and hence the above equation becomes a a! 1 ln x − ln a − δ(a ln x + b) + μx · − 1 + σ 2 x2 − 2 x 2 x   1 +a ln(x + xθz) − ln x − xθz · ν(dz) = 0 x R or 1 (1 − δa) ln x − ln a − δb + μa − 1 − σ 2 a 2  + a {ln(1 + θz) − θz} ν(dz) = 0, for all x > 0. R

This is possible if and only if a= and b=δ

−2

1 δ

   1 2 {ln(1 + θz) − θz} ν(dz) . δ ln δ − δ + μ − σ + 2 R

One can now verify that if δ > μ then with these values of a and b the function ϕ(s, x) = e−δt (a ln x + b) satisfies all the conditions of Theorem 3.1. We conclude that Φ(s, x) = e−δt (a ln x + b) and that

x a (in feedback form) is an optimal consumption rate. c∗ (x) = cˆ =

11.4 Exercises of Chapter 4 Exercise 4.1 (a) The HJB equation, i.e., (vi) and (ix) of Theorem 4.2, for this problem gets the form

210

11 Solutions of Selected Exercises

 ∂φ ∂φ σ 2 x2 ∂ 2 φ uγ + + (μx − u) + 0 = sup e−δs γ ∂s ∂x 2 ∂x2 u≥0     ∂φ + φ(s, x + θxz) − φ(s, x) − θxz (s, x) dν(z) ∂x R

(11.4.1)

for x > 0. We impose the first-order conditions to find the supremum, which is obtained for 1/(γ−1)

∗ δs ∂φ . (11.4.2) u = u (s, x) = e ∂x We guess that φ(s, x) = Ke−δs xγ with K > 0 to be determined. Then ' (1/(γ−1) u∗ (s, x) = Kγ x

and (11.4.1) turns into

(11.4.3)

! 1 1 (Kγ)γ/(γ−1) − Kδ + μ − (Kγ)1/(γ−1) Kγ + σ 2 Kγ(γ − 1) γ 2  +K {(1 + θz)γ − 1 − γθz} ν(dz) = 0 R

or 1 γ γ/(γ−1) K 1/(γ−1) − δ + μγ − γ γ/(γ−1) K 1/(γ−1) + σ 2 γ(γ − 1) 2  + {(1 + θz)γ − 1 − γθz} ν(dz) = 0. R

Hence

  1 1 σ2 γ(1 − γ) K= δ − μγ + γ 1−γ 2 −



R

 γ−1 {(1 + θz)γ − 1 − γθz} ν(dz)

(11.4.4)

provided that δ − μγ +

σ2 γ(1 − γ) − 2



R

{(1 + θz)γ − 1 − γθz} ν(dz) > 0.

With this choice of K the conditions of Theorem 4.2 are satisfied and we can conclude that φ = Φ is the value function. (b) (i) First assume λ ≥ K. Choose φ(s, x) = λe−δs xγ . By the same computations as in a), condition (vi) of Theorem 4.2 gets the form

11.5 Exercises of Chapter 5

λ≥



211



1 1 1 δ − μγ + σ 2 γ(1 − γ) γ γ−1 2 γ−1  − {(1 + θz)γ − 1 − γθz} ν(dz) .

(11.4.5)

R

Since λ ≥ K, the inequality (11.4.5) holds by (11.4.4). By Theorem 4.2a), it follows that: φ(s, x) = λe−δs xγ ≥ Φ(s, x) where Φ is the value function for our problem. On the other hand, φ(s, x) is obtained by the (admissible) control of stopping immediately (τ = 0). Hence we also have φ(s, x) ≤ Φ(s, x). We conclude that Φ(s, x) = λe−δs xγ in this case and τ ∗ = 0 is optimal. Note that D = ∅. (ii) Assume now λ < K. Choose φ(s, x) = Ke−δs xγ . Then for all (s, x) ∈ R × (0, ∞) we have φ(s, x) > λe−δs xγ . Hence we have D = R × (0, ∞) and by Theorem 4.2a) we conclude that Φ(s, x) ≤ Ke−δs xγ . On the other hand, we have seen in (a) above that if we apply the control ' (1/(γ−1) u∗ (s, x) = Kγ x

∗

and never stop, then we achieve the performance J (u ) (s, x) = Ke−δs xγ . Hence Φ(s, x) = Ke−δs xγ and it is optimal never to stop (τ ∗ = ∞).

11.5 Exercises of Chapter 5 Exercise 5.1 In this case we put         0 0 dt 0 1  + dξ(t). dt+ = dY (t) = dB(t)+ β z N (dt, dz) −(1 + λ) σ α dX(t) 

R

212

11 Solutions of Selected Exercises

The generator if ξ = 0 is Aφ =

∂φ ∂2φ ∂φ +α + 21 σ 2 2 + ∂s ∂x ∂x

   ∂φ φ(s, x + β z) − φ(s, x) − β z (s, x) ν(dz). ∂x R

The non-intervention region D is described by (see (5.2.5)

 k  ∂φ D = (s, x); (y) + θj < 0 for all j = 1, . . . , p κij ∂yi i=1   ∂φ −ρs = (s, x); −(1 + λ) (s, x) + e 0

then by Theorem 5.2 we should have for 0 < x < x∗ .

Aφ(s, x) = 0 We try a solution φ of the form

φ(s, x) = e−ρs ψ(x) and get A0 ψ(x) := −ρ ψ(x) + α ψ ′ (x) + 21 σ 2 ψ ′′ (x) +



{ψ(x + β z) − ψ(x)

R

−β z ψ ′ (x)} ν(dz) = 0. We now choose ψ(x) = er x

for some constant r ∈ R

and get the equation h(r) := −ρ + α r + 21 σ 2 r2 +

 R

Since h(0) < 0 and lim h(r) = r→∞



 er β z − 1 − r β z ν(dz) = 0.

lim h(r) = ∞, we see that the equation

r→−∞

h(r) = 0 has two solutions r1 , r2 such that

r2 < 0 < r1 .

11.5 Exercises of Chapter 5

213

Outside D we require that −(1 + λ)ψ ′ (x) + 1 = 0 or

x + C3 , 1+λ

ψ(x) =

C3 constant.

Hence we put

C1 er1 x + C2 er2 x ; ψ(x) = x 1+λ + C3 ;

0 < x < x∗ x∗ ≤ x

(11.5.1)

where C1 , C2 are constants. To determine C1 , C2 , C3 and x∗ we have the four equations: ψ(0) = 0 ⇒ C1 + C2 = 0.

(11.5.2)

Put C2 = −C1 ∗

∗

ψ continuous at x = x∗ ⇒ C1 (er1 x − er2 x ) = ∗

∗

∗

∗

ψ ∈ C 1 at x = x∗ ⇒ C1 (r1 er1 x − r2 er2 x ) =

x∗ + C3 1+λ

1 1+λ

ψ ∈ C 2 at x = x∗ ⇒ C1 (r12 er1 x − r22 er2 x ) = 0.

(11.5.3) (11.5.4) (11.5.5)

From (11.5.4) and (11.5.5) we deduce that x∗ =

2(ln |r2 | − ln r1 ) . r1 − r2

(11.5.6)

Then by (11.5.4) we get the value for C1 , and hence the value of C3 by (11.5.3). With these values of C1 , C2 , C3 and x∗ we must verify that φ(s, x) = −ρs e ψ(x) satisfies all the requirements of Theorem 5.2: (i) We have constructed φ such that Aφ + f = 0 in D. Outside D, i.e., for x ≥ x∗ , we have eρs (Aφ(s, x) + f (s, x)) = A0 ψ(x) = −ρ

x + C3 1+λ

1 +α· + 1+λ





+

C1 (er1 (x+βz) − er2 (x+βz) )

x+βz 0) F ′ (x) > F ′ (x∗ ) = 0

for x < x∗ .

Hence F (x) < 0 for 0 < x < x∗ . The conditions (iii), (iv), and (v) are left to the reader to verify. (vi) This holds by construction of φ. (vii)–(x) These conditions claim the existence of an increasing process ξˆ such ˆ ˆ is strictly increasing only when ¯ for all times t, ξ(t) that Y ξ (t) stays in D ˆ ¯ Y (t) ∈ D, and if Y (t) ∈ D then ξ(t) brings Y (t) down to a point on ∂D. Such a singular control is called a local time at ∂D of the process Y (t) reflected downwards at ∂D. The existence and uniqueness of such a local time is proved in [CEM]. (xi) This is left to the reader. We conclude that the optimal dividend policy ξ ∗ (t) is to take out exactly the amount of money needed to keep X(t) on or below the value x∗ . If X(t) < x∗ we take out nothing. If X(t) > x∗ we take out X(t) − x∗ . Exercise 5.2 It suffices to prove that the function Φ0 (s, x1 , x2 ) := Ke−δs (x1 + x2 )γ

11.6 Exercises of Chapter 6

215

satisfies conditions (i)–(iv) of Theorem 5.2. In this case we have (see Sect. 5.3) ∂Φ0 ∂Φ0 ∂Φ0 1 ∂ 2 Φ0 + (rx1 − c) + αx2 + β 2 x22 ∂s ∂x1 ∂x2 2 ∂x22

A(v) Φ0 (y) = A(c) Φ0 (y) =



+ Φ0 (s, x1 , x2 + x2 z) − Φ0 (s, x1 , x2 ) R

 ∂Φ0 − x2 z (s, x1 , x2 ) ν(dz) ∂x2

cγ , so condition (i) becomes γ γ c (i)’ Ac Φ0 (s, x1 , x2 ) + e−δs γ ≤ 0 for all c ≥ 0.

and f (s, x1 , x2 , c) = e−δs

This holds because we know by Example 3.2 that (see (3.1.21))   γ (c) −δs c sup A Φ(s, x1 , x2 ) + e = 0. γ c≥0 Since in this case θ = 0 and κ=



−(1 + λ) 1 − μ 1

−1

we see that condition (ii) of Theorem 5.2 becomes 0 (ii)’ −(1 + λ) ∂Φ ∂x1 + 0 (ii)” (1 − μ) ∂Φ ∂x1 −

Since

∂Φ0 ∂x2

∂Φ0 ∂x2

≤0

≤ 0.

∂Φ0 ∂Φ0 = = Ke−δs γ(x1 + x2 )γ−1 ∂x1 ∂x2

we see that (ii)’ and (ii)” hold trivially. We leave the verification of conditions (iii)–(v) to the reader.

11.6 Exercises of Chapter 6 Exercise 6.1 By using the same notation as in Chap. 6, we have here

216

11 Solutions of Selected Exercises

   s+t s (v) − Y (t) = = y ∈ R2 ; t ≥ 0; Y (0 ) = x X (v) (t)   s Γ (y, ζ) = Γ (s, x, ζ) = ; (s, x, ζ) ∈ R3 x+ζ (v)



K(y, ζ) = K(s, x, ζ) = e−ρs (x + λ|ζ|) f (y) = f (s, x) = e−ρs x2 , g(y) = 0. By symmetry we expect the continuation region to be of the form D = {(s, x) : −¯ x x ˆ such that (11.6.3)–(11.6.5) hold. With these values of a, x ˆ and x ¯ it remains to verify that the conditions of Theorem 6.2 hold. We check some of them: (ii) ψ ≤ Mψ = inf {ψ(x − ζ) + c + λζ; ζ > 0}. First suppose x ≥ x ¯. If x − ζ ≥ x ¯ then x) + c + λ(x − ζ − x ˆ) + c + λζ = c + ψ(x) > ψ(x). ψ(x − ζ) + c + λζ = ψ0 (ˆ If 0 < x − ζ < x ¯ then ψ(x − ζ) + c + λζ = ψ0 (x − ζ) + c + λζ,

11.6 Exercises of Chapter 6

219

ψ ′ (x) = λ

ψ ′ (ˆ x) = λ

x x ˆ

x ¯

Fig. 11.5. The function ψ(x) for x > 0

which is minimal when −ψ0′ (x − ζ) + λ = 0 i.e., when

ζ = ζˆ = x − x ˆ.

This is the minimum point because ψ0′′ (ˆ x) > 0. See Fig. 11.5. This shows that ˆ + c + λζˆ = ψ(ˆ Mψ(x) = ψ(x − ζ) x) + c + λ(x − x ˆ) = ψ(x) for x > x ˆ. Next suppose 0 < x < x ¯. Then x) + c + λ(x − x ˆ) > ψ(x) Mψ(x) = ψ0 (ˆ if and only if ψ(x) − λx < ψ(ˆ x) − λˆ x + c. Now the minimum of H(x) := ψ(x) − λx

for 0 < x < x ¯

220

11 Solutions of Selected Exercises

is attained when ψ ′ (x) = λ i.e. when x = x ˆ. Therefore ψ(x) − λx ≤ ψ(ˆ x) − λˆ x < ψ(ˆ x) − λˆ x + c. This shows that Mψ(x) > ψ(x) for all 0 < x < x ¯. Combined with the above we can conclude that Mψ(x) ≥ ψ(x) for all x > 0, which proves (ii). Moreover, Mψ(x) > ψ(x) if and only if 0 < x < x ¯. Hence D ∩ (0, ∞) = (0, x ¯). Finally we verify (vi) Aφ + f ≥ 0 for x > x ¯. For x > x ¯, we have, if x ¯ ≤ ξ (using (6.3.2)), A0 ψ(x) + f (x) = −ρ(ψ0 (ˆ x) + c + λ(x − x ˆ)) + x2 . This is nonnegative for all x > x ¯ iff it is nonnegative for x = x ¯, i.e., iff −ρψ0 (¯ x) + x ¯2 ≥ 0.

(11.6.7)

By construction of ψ0 we know that, for x < x ¯  1 −ρψ0 (x) + ψ0′′ (x) + {ψ0 (x + z) − ψ0 (x) − zψ0′ (x)} ν(dz) + x2 = 0. 2 R Therefore (11.6.7) holds iff  1 ′′ ψ0 (¯ x) + {ψ0 (¯ x + z) − ψ0 (¯ x) − zψ0′ (¯ x)} ν(dz) ≤ 0. 2 R For this it suffices that 

ρ x). z 2 ν(dz) ≤ − ψ0′′ (¯ 2 R

(11.6.8)

Conclusion Suppose x ¯ ≤ ξ and that (11.6.8) holds. Then Φ(s, x) = e−ρs ψ(x), with ψ(x) given by (11.6.3) and a, x ˆ, x ¯ given by (11.6.4)-(11.6.6). The optimal impulse control is to do nothing while |X(t)| < x ¯, then move X(t) down to x ˆ (respectively, up to −ˆ x) as soon as X(t) reaches a value ≥ x ¯ (respectively, a value ≤ −¯ x).

11.6 Exercises of Chapter 6

221

Exercise 6.2 Here we put  s+t Y (t) = X (v) (t)   s Y (v) (0− ) = =y x 

(v)

Γ (y, ζ) = x − c − (1 + λ)ζ

K(y, ζ) = e−ρs ζ f ≡g≡0

S = {(s, x) : x > 0} .

We guess that the value function φ is of the form φ(s, x) = e−ρs ψ(x) and consider the intervention operator   x−c Mψ(x) = sup ψ(x − c − (1 + λ)ζ) + ζ; 0 ≤ ζ ≤ . 1+λ

(11.6.9)

Note that the condition on ζ is due to the fact that the impulse must be positive and x − c − (1 + λ)ζ must belong to S. We distinguish between two cases: (1) μ > ρ. In this case, suppose we wait until time t1 and then take out ζ1 =

X(t1 ) − c . 1+λ

The corresponding value is J (v1 ) (s, x) = E x



e−ρ(t1 +s) (X(t1 ) − c) 1+λ

x



1 ' −ρs (μ−ρ)t1 −ρ(s+t1 ) xe e − ce ) 1+λ

=E

→ ∞ as t1 → ∞.

Therefore we obtain Φ(s, x) = +∞ in this case.

222

11 Solutions of Selected Exercises

(2) μ < ρ. We look for a solution by using the results of Theorem 6.2. In this case condition (x) becomes A0 ψ(x) := − ρψ(x) + μxψ ′ (x) + 21 σ 2 ψ ′′ (x)  + {ψ(x + γxz) − ψ(x) − γzψ ′ (x)} ν(dz) = 0 in D. (11.6.10) R

We try a solution of the form ψ(x) = C1 xγ1 + C2 xγ2 , where γ1 > 1, γ2 < 0 are the solutions of the equation  1 2 F (γ) := −ρ + μγ + σ γ(γ − 1) + {(1 + θz)γ − 1 − θzγ} ν ′ (dz) = 0. 2 R We guess that the continuation region is of the form D = {(s, x) : 0 < x < x ¯} for some x ¯ > 0 (to be determined). We see that C2 = 0, because otherwise lim |ψ(x)| = ∞. x→0

We guess that in this case it is optimal to wait till X(t) reaches or exceeds a value x ¯ > c and then take out as much as possible, i.e., reduce X(t) to 0. Taking the transaction costs into account this means that we should take out x−c ˆ for x ≥ x ¯. ζ(x) = 1+λ

We therefore propose that ψ(x) has the form

C1 xγ1 for 0 < x < x ¯ ψ(x) = x−c for x ≥ x ¯ . 1+λ Continuity and differentiability of ψ(x) at x = x ¯ give the equations C1 x ¯ γ1 =

x ¯−c 1+λ

and C1 γ1 x ¯γ1 −1 =

1 . 1+λ

Combining these we get x ¯=

γ1 c γ1 − 1

and C1 =

x ¯ − c −γ1 x ¯ . 1+λ

With these values of x ¯ and C1 , we have to verify that ψ satisfies all the requirements of Theorem 6.2. We check some of them:

11.6 Exercises of Chapter 6

223

(ii) ψ ≥ Mψ on S. , + x−c . Here Mψ = sup {ψ(x − c − (1 + λ)ζ) + ζ} ; 0 ≤ ζ ≤ 1+λ If x − c − (1 + λ)ζ ≥ x ¯, then

ψ(x − c − (1 + λ)ζ) + ζ =

x − 2c x−c < = ψ(x) 1+λ 1+λ

and if x − c − (1 + λ)ζ < x ¯ then h(ζ) := ψ(x − c − (1 + λ)ζ) + ζ = C1 (x − c − (1 + λ)ζ)γ1 + ζ. Since h

′



x−c 1+λ



= 1 and h′′ (ζ) > 0

x−c , is attained at ζ = we see that the maximum value of h(ζ); 0 ≤ ζ ≤ 1+λ x−c ˆ ζ(x) = 1+λ . Therefore   x − 2c x − c x−c Mψ(x) = max , for all x > c. = 1+λ 1+λ 1+λ

Hence Mψ(x) = ψ(x) for x ≥ x ¯. For 0 < x < x ¯ consider k(x) := C1 xγ1 −

x−c . 1+λ

Since k(¯ x) = k ′ (¯ x) = 0

and k ′′ (x) > 0 for all x,

we conclude that k(x) > 0 for 0 < x < x ¯. Hence ψ(x) > Mψ(x) for 0 < x < x ¯. ¯ i.e., for x > x (vi) A0 ψ(x) ≤ 0 for x ∈ S\D ¯. For x > x ¯, we have 1 x−c + μx · 1+λ 1+λ    x + γxz − c γ1 + C1 (x + γxz) − ν(dz) 1+λ x+γxz x ¯ ⇔ (μ − ρ)x + (ρ + ν)c ≤ 0 for all x > x ¯ ⇔ (μ − ρ)¯ x + (ρ + ν)c ≤ 0 ⇔x ¯≥ ⇔

γ1 c (ρ + ν)c ≥ γ1 − 1 ρ−μ

⇔ γ1 ≤ Since F

(ρ + ν)c ρ−μ

ρ + ν . μ + ν

    ρ 1 ρ ρ ρ −1 >0 ≥ −ρ + μ + σ 2 μ μ 2 μ μ

and F (γ1 ) = 0, γ1 > 1 we conclude that γ1 < small enough.

ρ μ

and hence (vi) holds if ν is

Exercise 6.3 Here f = g = 0, Γ (y, ζ) = (s, 0), K(y, ζ) = −c + (1 − λ)x and S = R2 ; y = (s, x). If there are no interventions, the process Y (t) defined by        0 1 dt 0  dB(t) + dt + = dY (t) = (dt, dz) θzN σ μ dX(t) R

has the generator

∂φ ∂2φ ∂φ +μ + 21 σ 2 2 + ∂s ∂x ∂x

Aφ(y) =

 +

φ(s, x + θ z) − φ(s, x) − θ z

R

,

∂φ (s, z) ν(dz) ; ∂x

y = (s, x).

The intervention operator M is given by M φ(y) = sup {φ(Γ (y, ζ)) + K(y, ζ); ζ ∈ Z and Γ (y, ζ) ∈ S} = φ(s, 0) + (1 − λ)x − c. If we try

11.6 Exercises of Chapter 6

225

φ(s, x) = e−ρs ψ(x), we get that Aφ(s, x) = e−ρs A0 ψ(x), where A0 ψ(x) = −ρψ +μψ

′

(x)+ 12 σ 2 ψ ′′ (x)+



{ψ(x + θ z) − ψ(x) − θ zψ ′ (x)} ν(dz)

R

and M φ(s, x) = e−ρs M0 ψ(x), where M0 ψ(x) = ψ(0) + (1 − λ)x − c. We guess that the continuation region D has the form D = {(s, x); x < x∗ } for some x∗ > 0 to be determined. To find a solution ψ0 of A0 ψ0 +f = A0 ψ0 = 0, we try ψ0 (x) = er x (r constant) and get A0 ψ0 (x) = −ρ er x + μ r er x + 21 σ 2 r2 er x  + , er(x+θ z) − er x − r θ z er x ν(dz) + R

= er x h(r) = 0, where h(r) = −ρ + μ r + 21 σ 2 r2 +

 R

 rθz  e − 1 − r θ z ν(dz).

Choose r1 > 0 such that h(r1 ) = 0

(see the solution of Exercise 2.1).

Then we define

ψ(x) =

⎧ ⎨M er1 x ;

⎩ψ(0) + (1 − λ)x − c = M + (1 − λ)x − c;

x < x∗ x ≥ x∗

(11.6.11)

for some constant M = ψ(0) > 0. If we require continuity and differentiability at x = x∗ we get the equations

226

11 Solutions of Selected Exercises ∗

M er1 x = M + (1 − λ)x∗ − c

(11.6.12)

and ∗

M r1 er1 x = 1 − λ.

(11.6.13)

This gives the following equations for x∗ and M : ∗

k(x∗ ) := e−r1 x + r1 x∗ − 1 −

r1 c = 0, 1−λ

M=

1 − λ −r1 x∗ > 0. (11.6.14) e r1

r1 c Since k(0) = − 1−λ < 0 and lim k(x) = ∞, we see that there exists x∗ > 0 x→∞

s.t. k(x∗ ) = 0. We must verify that with these values of x∗ and M the conditions of Theorem 6.2 are satisfied. We consider some of them: (ii) ψ(x) ≥ M0 ψ(x). For x ≥ x∗ we have ψ(x) = M0 ψ(x) = M + (1 − λ)x − c. For x < x∗ we have ψ(x) = M er1 x and M0 ψ(x) = M + (1 − λ)x − c. Define F (x) = M er1 x − (M + (1 − λ)x − c);

x ≤ x∗ .

See Fig. 11.6. We have F (x∗ ) = F ′ (x∗ ) = 0 and F ′′ (x) = M r12 er1 x > 0. Hence F ′ (x) < 0 and so F (x) > 0 for x < x∗ . Therefore ψ(x) ≥ M0 ψ(x) for all x. (vi) A0 ψ ≤ 0 for x > x∗ :

For x > x∗ we have

A0 ψ(x) = −ρ[M + (1 − λ)x − c] + μ(1 − λ)  + , M er1 (x+θz) − (M + (1 − λ)(x + θz) − c) ν(dz) + x+θz x∗ ⇔ x ≥

cν μ c−M + + ρ 1−λ ρ(1 − λ)

⇔ x∗ ≥

∗ c 1 cν μ + − e−r1 x + ρ 1 − λ r1 ρ(1 − λ)

⇔ x∗ ≥

∗

⇔ e−r1 x + r1 x∗ −

μ cν c ≥ + 1−λ ρ ρ(1 − λ)

cν μ + ρ ρ(1 − λ)

⇔ 1≥

cν ≤ ρ. 1−λ

⇔ μ+ So we need to assume that μ +

for all x > x∗

c ν 1−λ

≤ ρ for (vi) to hold.

Conclusion Let ψ(s, x) = e−ρs ψ(x) where ψ is given by (11.6.11). Assume that μ+

cν ≤ ρ. 1−λ

Then φ(s, x) = sup J (v) (s, x) v

and the optimal strategy is to cut the forest every time the biomass reaches the value x∗ (see Fig. 11.7).

11.7 Exercises of Chapter 7 Exercise 7.1 As in Exercise 6.3, we have f = g = 0, Γ (y, ζ) = (s, 0); y = (s, x) K(y, ζ) = (1 − λ)x − c S = [0, ∞) × R

228

11 Solutions of Selected Exercises

x∗

x∗

cut

cut t

Fig. 11.7. The optimal forest management of Exercise 6.3

If there is no intervention, then φ0 ≡ 0 and Mφ0 = sup {(1 − λ)ζ − c; ζ = x} = (1 − λ)x − c. Hence   −ρ(s+τ ) φ1 (y) = sup E [Mφ0 (Y (τ ))] = sup E e ((1 − λ)X(τ ) − c) . y

y

τ ≤τS

τ ≤τS

(11.7.1) This is an optimal stopping problem that can be solved by exploiting the three basic variational inequalities. We assume that the continuation region D1 = {φ1 > Mφ0 } is of the form D1 = {(s, x); x < x1 } for some x1 > 0

and that the value function has the form φ1 (s, x) = e−ρs ψ1 (x) for some function ψ1 . On D1 , ψ1 is the solution of  ′ 1 2 ′′ −ρψ1 (x)+μψ1 (x)+ 2 σ ψ1 (x)+ {ψ1 (x + θz) − ψ1 (x) − θzψ1′ (x)} ν(dz) = 0. R

(11.7.2)

A solution of (11.7.2) is ψ1 (x) = Aeγ1 x + Beγ2 x where γ2 < 0 and γ1 > 1, A and B arbitrary constants to be determined. We choose B = 0 and put A1 = A > 0. We get

A1 eγ1 x x < x1 ψ1 (x) = (1 − λ)x − c x ≥ x1 . We impose the continuity and differentiability conditions of ψ1 at x = x1 .

11.7 Exercises of Chapter 7

229

(i) Continuity: A1 eγ1 x1 = (1 − λ)x1 − c. (ii) Differentiability: A1 γ1 eγ1 x1 = 1 − λ.

−γ1 x1 We get A1 = (1−λ) and x1 = γ1 e As a second step, we evaluate

1 γ1

+

c 1−λ .

φ2 (y) = sup E y [Mφ1 (Y (τ ))]. τ

We suppose φ2 (s, x) = e−ρs ψ2 (x) and consider Mψ1 (x) = sup {ψ1 (0) + (1 − λ)ζ − c; ζ ≤ x} = ψ1 (0) + (1 − λ)x − c = (1 − λ)x + A1 − c. Hence * ) φ2 (y) = sup E y e−ρ(s+τ ) ((1 − λ)X(τ ) − (c − A1 )) .

(11.7.3)

τ ≤τS

By the same argument as before, we get Φ2 (s, x) = e−ρs ψ2 (x), where

A2 eγ1 x x < x2 ψ2 (x) = (1 − λ)x + A1 − c x ≥ x2 1−λ −γ1 x2 1 . Note that x2 < x1 and A2 > A1 . where x2 = γ11 + c−A 1−λ and A2 = γ1 e Since Mφ0 and Mφ1 have linear growth, the conditions of Theorem 7.2 are satisfied. Hence φ1 and φ2 are the solutions for our impulse control problems when respectively one intervention and two interventions are allowed. The impulses are given by ζ1 = ζ2 = x and τ1 = inf {t : X(t) ≥ x2 } and τ2 = inf {t > τ1 : X(t) ≥ x1 }.

Exercise 7.2 Here we have (see the notation of Chap. 6) f =g≡0

K(x, ζ) = ζ Γ (x, ζ) = x − (1 + λ)ζ − c S = {(s, x); x > 0} .

We put y = (s, x) and suppose φ0 (s, x) = e−ρs ψ0 (x). Since f = g = 0 we have φ0 (y) = 0 + , x−c x−c + and Mψ0 (y) = sup ζ : 0 ≤ ζ ≤ 1+λ = ( 1+λ ) . As a second step, we consider

230

11 Solutions of Selected Exercises x

φ1 (s, x) = sup E [Mφ0 (X(τ ))] = sup E τ ≤τS

τ ≤τS

x



e

−ρ(τ +s)

(X(τ + s) − c)+ . 1+λ (11.7.4)

We distinguish between three cases (a) μ > ρ Then φ1 (s, x) ≥ Hence if t → +∞

xe(μ−ρ)(t+s) − ce−ρ(t+s) . 1+λ

φ1 (s, x) → +∞.

We obtain Mφ1 (s, x) = +∞ and clearly φn = +∞ for all n. In this case, the optimal stopping time does not exist. (b) μ < ρ In this case we try to put φ1 (s, x) = e−ρs ψ1 (x) and solve the optimal stopping problem (11.7.4) by using Theorem 2.2. We guess that the continuation region is of the form D = {0 < x < x∗1 } and solve σ 2 x2 ′′ ψ1 (x) Lψ1 (x) := − ρψ1 (x) + μxψ1′ (x) + 2  + {ψ1 (x + θxz) − ψ1 (x) − θxzψ1′ (x)} ν(dx) = 0.

(11.7.5)

R

A solution of (11.7.5) is ψ1 (x) = c1 xγ1 + c2 xγ2 where γ2 < 0 and γ1 > 1 are solutions of the equation  1 k(γ) := −ρ + μγ + σ 2 γ(γ − 1) + {(1 + θz)γ − 1 − γθz} ν(dz) = 0 2 R and c1 , c2 are arbitrary constants. Since γ2 < 0, we put c2 = 0. We obtain

c1 xγ1 0 < x < x∗1 ψ1 (x) = x−c x ≥ x∗1 . 1+λ By imposing the condition of continuity and differentiability, we can compute c1 and x∗1 . The result is: 1. x∗1 = 2. c1 =

γ1 c γ1 −1 ' γ1 c (1−γ1 1 γ1 (1+λ) γ1 −1

11.7 Exercises of Chapter 7

231

Note that γ1 > 1 and x∗1 > c. We check some of the conditions of Theorem 2.2: (ii) ψ1 (x) ≥ Mψ0 (x) for all x:

We know that φ1 (x) = Mφ0 (x) for x > x∗1 . Consider h1 (x) := ψ1 (x) − Mψ0 (x). We have h1 (x∗1 ) = 0, h′1 (x∗1 ) = 0,

h′′1 (x∗1 ) = c1 γ1 (γ1 − 1)(x∗1 )γ1 −2 > 0. Hence x∗1 is a minimum for h1 and ψ1 (x) 0 < x < x∗1 .

≥

Mψ0 (x) for every

(vi) Lψ1 ≤ 0 for all x > 0:

Clearly Lψ1 = 0 for 0 < x < x∗1 . If x > x∗1 then 1 Lψ1 (x) = ((μ − ρ)x + cρ) 1+λ    x + θxz − c + c1 (x + θxz)γ1 − ν(dz) ≤ 0 1+λ x+θxz 0 for γ ≥ γ1 . Since ρc γ1 c > . F μρ > 0 we have that μρ > γ1 , which implies that x∗1 = γ1 − 1 ρ−μ ∗ Hence Lψ1 (x) ≤ 0 for all x ≥ x1 if ν is small enough. We conclude that under these conditions φ1 (s, x) = e−ρs ψ1 (x) actually solves the optimal stopping problem (11.7.4). Next we consider   x−c (x − c)+ Mψ1 (x) = sup ψ1 (x − (1 + λ)ζ − c) + ζ; 0 ≤ ζ ≤ = 1+λ 1+λ and repeat the same procedure to find ψ2 . By induction, we obtain Mψn = Mψn−1 = Mψ1 = Mψ0 . Consequently, we also have Φ = Φ1

232

11 Solutions of Selected Exercises

and Φ(s, x) = Φn (s, x) for every n. Moreover, we achieve the optimal result with just one intervention. (c) μ = ρ This case is left to the reader.

11.8 Exercises of Chapter 8 Exercise 8.1 In this case we have Γ1 (ζ, x1 , x2 ) = x1 − ζ − c − λ|ζ|

Γ2 (ζ, x1 , x2 ) = x2 + ζ

K(ζ, x1 , x2 ) = c + λ|ζ| g=0 f (s, x1 , x2 , u) =

e−ρs γ u . γ

The generator is given by Lu =

∂ ∂ σ2 2 ∂ 2 ∂ + (rx1 − u) x + αx2 + . ∂t ∂x1 ∂x2 2 2 ∂x22

Let φ(s, x1 , x2 ) be the value function of the optimal consumption problem  ∞ γ y −ρ(s+t) u(t) dt ; y = (s, x1 , x2 ) e sup E γ w∈W 0 with c, λ > 0 and φ0 (s, x1 , x2 ) the corresponding value function in the case when there are no transaction costs, i.e. c = λ = 0. In order to prove that Φ(s, x1 , x2 ) ≤ Ke−ρs (x1 + x2 )γ = Φ0 (s, x1 , x2 ) we check the hypotheses of Theorem 8.1a): (vi)

Lu φ0 + f ≤ 0

Since φ0 is the value function in the absence of transaction costs, we have sup {Lu φ0 + f } = 0 u≥0

in R3 .

11.8 Exercises of Chapter 8

233

Note that Mφ0 (s, x1 , x2 ) =

sup φ0 (s, x1 − ζ − λ|ζ| − c, x2 + ζ)

ζ∈R\{0}

sup Ke−ρs (x1 + x2 − c − λ|ζ|)γ

=

ζ∈R\{0}

= Ke−ρs (x1 + x2 − c)γ . Therefore D = {φ0 > Mφ0 } = R3 . Hence we can conclude that e−ρs (x1 + x2 )γ ≥ Φ(s, x1 , x2 ). Exercise 8.2 The HJBQVI’s for this problem can be formulated in one equation as follows:     ∂ϕ ∂ϕ 1 2 ∂ 2 ϕ 2 2 min inf +u + σ + x + u , ϕ − Mϕ = 0, u∈R ∂s ∂x 2 ∂x2 where Mϕ(s, x) = inf {ϕ(s, x + ζ) + c} . ζ∈R

Since ϕ is a candidate for the value function Φ it is reasonable to guess that, for each s, ϕ(s, z) is minimal for z = 0. Hence Mϕ(s, x) = ϕ(s, 0) + c, attained for ζ = ζ ∗ (x) = −x. The minimum of the function k(u) :=

∂ϕ ∂ϕ 1 2 ∂ 2 ϕ +u + σ + x2 + u2 ; ∂s ∂x 2 ∂x2

u∈R

is attained when u=u ˆ(s, x) := −

1 ∂ϕ (s, x) 2 ∂x

and this gives the minimum value ∂ϕ 1 − kmin = k(ˆ u(s, x)) = ∂s 4



∂ϕ ∂x

2

1 ∂2ϕ + σ 2 2 + x2 . 2 ∂x

Hence the HJBQVI gets the form min

∂ϕ 1 − ∂s 4



∂ϕ ∂x

2

 1 2 ∂2ϕ + σ + x2 , ϕ(s, 0) + c = 0. 2 ∂x2

234

11 Solutions of Selected Exercises

If we guess that the continuation region D := {(s, x); ϕ(s, x) < ϕ(s, 0) + c} has the form D = {(s, x); |x| < x∗ (s), 0 ≤ s ≤ T } for some function x∗ (s), then the HJBQVI can be split into the two equations 1 ∂2ϕ 1 ∂ϕ = − σ2 2 + ∂s 2 ∂x 4



∂ϕ ∂x

2

1 ∂2ϕ + σ 2 2 − x2 for|x| < x∗ (s) 2 ∂x

ϕ(s, x) = ϕ(s, 0) + c for |x| ≥ x∗ (s).

(11.8.1) (11.8.2)

Equation (11.8.1) is known as a Burgers equation, which can be linearized by the transformation 1 w(s, x) := exp(− 2 ϕ(s, x)). (11.8.3) 2σ This transforms (11.8.1) into the equation ∂w 1 ∂2w x2 = − σ2 2 + 2 w ; ∂s 2 ∂x 2σ

|x| < x∗ (s)

(11.8.4)

with boundary values c ); x = ±x∗ (s), s < T 2σ 2 w(T, x) = 1.

w(s, x) = b(s) := w(s, 0) exp(−

(11.8.5) (11.8.6)

By the Feynman–Kac formula, the solution of (11.8.4)–(11.8.6) can be written ⎛ ⎞ τˆ D −s 1 ˆ 2 (t)dt) b(B(ˆ ˆ τD − s))⎠ ; (s, x) ∈ D (11.8.7) w(s, x) = EP˜ ⎝exp(− B 2 0

ˆ = B(t, ˆ ω where B(t) ˆ ) is an auxiliary Brownian motion with law Pˆ starting at x and   1 ∗ ˆ τˆD = τˆD (ˆ (11.8.8) ω ) = inf t > 0; |B(t)| ≥ x (t) ∧ T. σ The high contact condition for determination of the curve x∗ (t) is then 

∂w ∂w , ∂s ∂x



c (s, x (s)) = exp(− 2 ) 2c ∗



 ∂w (s, 0), 0 . ∂s

The suggested shape of x∗ (t) is shown in Fig. 11.8.

(11.8.9)

11.9 Exercises of Chapter 9

235

x

✻

x∗ (t)

? ✲ t τ2

τ1

T Fig. 11.8. The suggested optimal combined control of Exercise 8.2

11.9 Exercises of Chapter 9 Exercise 9.1 Because of the symmetry of h, we assume that the continuation region is of the form D = {(s, x) : −x∗ < x < x∗ } with x∗ > 0. We assume that the value function φ(s, x) = e−ρs ψ(x). On D, φ is the solution of Lφ(s, x) = 0, (11.9.1) where L =

∂ ∂s

+

1 ∂2 2 ∂x2 .

We obtain

L0 ψ(x) := −ρψ(x) + 12 ψ ′′ (x) = 0.

(11.9.2)

The general solution of (11.9.2) is ψ(x) = c1 e

√

2ρ x

+ c2 e−

√

2ρ x

.

We must have ψ(x) = ψ(−x), hence c1 = c2 . We put c1 = 12 c and assume c > 0. We impose continuity and differentiability conditions at x = x∗ (Fig. 11.9): (i) Continuity at x = x∗ √ ∗( 1 ' √2ρ x∗ c e + e− 2ρ x = Kx∗ 2

236

11 Solutions of Selected Exercises y G(x)

1 −K

x

1 K

Fig. 11.9. The function G(x)

(ii) Differentiability at x = x∗ √ ∗( 1 - ' x∗ √2ρ 1 c 2ρ e − e− 2ρ x = K. 2 2

Then x∗ is the solution of

∗√

√

∗

1 ex 2ρ − e− 2ρ x ∗ √ √ √ = = tgh(x 2ρ) ∗ ∗ x∗ 2ρ ex 2ρ + e−x 2ρ and 1 K √ c= √ . 2ρ + sinh(x∗ 2ρ) √ 1 . If we put z ∗ = x∗ 2ρ , then z ∗ is the solution of We must check if x∗ < K 1 = tgh(z ∗ ). z∗ We distinguish between two cases: √ 2ρ 1 Case 1. For K < ∗ = ∗ we have (Fig. 11.10) x z ⎧ 1 ⎪ |x| > K ⎨1 ∗ ψ(x) = K|x| x < |x| < ⎪ √ ⎩ |x| < x∗ c cosh(x 2ρ )

1 K

Since ψ is not C 2 at x = x∗ we prove that ψ is a viscosity solution for our optimal stopping problem. (i)

We first prove that ψ is a viscosity subsolution.

11.9 Exercises of Chapter 9

237

ψ(x)

x∗

1 K

Fig. 11.10. The function ψ(x) in Case 1 for x ≥ 0

Let u belong to C 2 (R) and u(x) ≥ ψ(x) for all x ∈ R and let y0 ∈ R be such that u(y0 ) = ψ(y0 ). Then ψ is a viscosity subsolution if and only if max(L0 u(y0 ), G(y0 ) − u(y0 )) ≥ 0 for all such u, y0 .

(11.9.3)

We need to check (11.9.3) only for y0 = x∗ . We have u(x∗ ) = ψ(x∗ ) = G(x∗ ) i.e., (G − u)(x∗ ) = 0. Hence max(L0 u(x∗ ), G(x∗ ) − u(x∗ )) ≥ (G − u)(x∗ ) = 0. (ii)

We prove that ψ is a viscosity supersolution. Let v belong to C 2 (R) and v(x) ≤ ψ(x) for every x ∈ R and let y0 ∈ R be such that v(y0 ) = ψ(y0 ). Then ψ is a viscosity supersolution if and only if max(L0 v(y0 ), G(y0 ) − v(y0 )) ≤ 0 for all such v, y0 . We check it only for x = x∗ . Then G(x∗ ) = ψ(x∗ ) = v(x∗ ). Since v ≤ ψ, x = x∗ is a maximum point for H := v − ψ. We have H(x∗ ) = 0, H ′ (x∗ ) = 0, H ′′ (x∗ ) = v ′′ (x∗ ) − ψ ′′ (x∗− ) ≤ 0. Hence L0 v(x∗ ) ≤ L0 ψ(x∗− ) ≤ 0. Therefore ψ is a viscosity supersolution. Since ψ is both a viscosity supersolution and subsolution, ψ is a viscosity solution.

238

11 Solutions of Selected Exercises ψ(x)

1 K

1 −K

Fig. 11.11. The function ψ(x) in Case 2 √

Case 2. We consider now the case when K ≥ z2ρ ∗ . In this case, the continuation region is given by   1 1 D= − 0 this is impossible. Next we prove that Φ(x) < Φ2 (x) where Φ2 (x) is the solution of Exercise 6.1. It has the form Φ2 (s, x) = e−ρs ψ2 (x), with

240

11 Solutions of Selected Exercises

¯ ψ0 (x); |x| ≤ x ψ2 (x) = x) + c; |x| > x ¯ ψ0 (ˆ where ψ0 (x) =

b 1 2 x + 2 − a cosh(γx) ρ ρ

is a solution of 1 −ρψ0 (x) + ψ0′′ (x) + 2



{ψ0 (x + z) − ψ0 (x) − zψ0′ (x)} ν(dz) + x2 = 0.

R

Since clearly Φ(s, x) ≤ Φ2 (s, x), it suffices to prove that Φ2 (s, x) does not satisfy (11.9.5) in viscosity sense. In particular, (11.9.5) implies that Lu Φ2 (s, x) + e−ρs (x2 + θu2 ) ≥ 0 for all u ∈ R. For |x| < x ¯ this reads uψ0′ (x) + θu2 ≥ 0 for all u ∈ R, |x| < x ¯.

(11.9.6)

The function h(u) :=



 2x − aγ sinh(γx) u + θu2 ; u ∈ R ρ

is minimal when u=u ˆ=

1 2θ



aγ sinh(γx) −

2x ρ



with corresponding minimum value  2 1 2x . h(ˆ u) = − aγ sinh(γx) − 4θ ρ Hence (11.9.6) cannot possibly hold and we conclude that Φ2 cannot be a viscosity solution of (11.9.5). Hence Φ = Φ2 and hence Φ(x) < Φ2 (x) for some x.

11.10 Exercises of Chapter 10 Exercise 10.1 (a) Here Aφ(x) = (αx + u)φ′ (x) + 12 σ 2 φ′′ (x) and hence 1 d ((αx + u)ψ(x)) + σ 2 ψ ′′ (x) dx 2 1 ′ = −αψ(x) − (αx + u)ψ (x) + σ 2 ψ ′′ (x). 2

A∗ ψ(x) = −

(11.10.1)

11.10 Exercises of Chapter 10

241

Therefore the corresponding complete observation controlled SPDE is ⎧ ) * ∂y 1 2 ∂2 ⎪ ⎪ ⎨dy(t, x) = −αy(t, x) − (αx + u(t)) ∂x (t, x) + 2 σ ∂x2 y(t; x) dt ⎪ ⎪ ⎩y(0, x)

+ h(x)y(t, x)dζ(t) ; t > 0 = F (x)

(11.10.2)

with performance criterion  



T

J(u) = EQ

0

2

2



(x + u (t))y(t, x)dx dt .

R

(11.10.3)

(b) Here Aφ(x) = αxuφ′ (x) + 12 β 2 x2 u2 φ′′ (x) and hence A∗ ψ(x) = −

1 d2 2 2 2 d (αuxψ(x)) + (β u x ψ(x)) dx 2 dx2

1 = (β 2 u2 − αu)ψ(x) + (2β 2 u2 x − αux)ψ ′ (x) + β 2 u2 x2 ψ ′′ (x). 2 (11.10.4) Hence the corresponding controlled complete observation SPDE is ) ⎧ ∂ ⎪ dy(t, x) = (β 2 u2 (t) − αu(t))y(t, x) + (2β 2 u2 (t)x − αu(t)x) ∂x y(t, x) ⎪ ⎨ * 2 ∂ + 12 β 2 u2 (t)x2 ∂x 2 y(t, x) dt + h(x)y(t, x)dζ(t) ⎪ ⎪ ⎩ y(0, x) = F (x), (11.10.5) with performance criterion J(u) = EQ



R

 x y(T, x)dx . γ

(11.10.6)

References

[A] [Aa] [Am]

[AK] [AKL]

[AKy]

[AMS] [AO] [AT] [B] [BG] [B-N] [Be] [BEK] [Ben1]

D. Applebaum: L´evy Processes and Stochastic Calculus. Cambridge University Press 2003. K. Aase: Optimum portfolio diversification in a general continuous-time model. Stoch. Proc. Appl. 18 (1984), 81–98. A.L. Amadori: Nonlinear integro-differential operators arising in option pricing: a viscosity solutions approach. J. Diff. Integral Eqn. 13 (2003), 787–811. L. H. R. Alvarez and J. Keppo: The impact of delivery lags on irreversible investment under uncertainty. Eur. J. Oper. Res. 136 (2002), 173–180. A.L. Amadori, K.H. Karlsen and C. La Chioma: Non-linear degenerate integro-partial differential evolution equations related to geometric L´evy processes and applications to backward stochastic differential equations. Stoch. Stoch. Rep. 76(2) (2004), 147–177(31). L. Alili and A.Kyprianou: Some remarks on first passage of L´evy processes, the American put and pasting principles. Ann. Appl. Probab. (2004). M. Akian, J.L. Menaldi and A. Sulem: On an investment-consumption model with transaction costs. SIAM J. Control Opt. 34 (1996), 329–364. R. Anderson and S. Orey: Small random perturbations of dynamical systems with reflecting boundary. Nagoya Math. J. 60 (1976), 189-216. O. Alvarez and A. Tourin: Viscosity solutions of nonlinear integrodifferential equations. Ann. Inst. Henri Poincar´e 13 (1996), 293–317. ´ G. Barles: Solutions de Viscosit´e des Equations de Hamilton-Jacobi. Math. Appl. 17. Springer, Berlin Heidelberg New York 1994. R.M. Blumenthal and R. K. Getoor: Markov Processes and Potential Theory. Academic, New York 1968. O. Barndorff-Nielsen: Processes of normal inverse Gaussian type. Finance Stoch. 1 (1998), 41–68. J. Bertoin: L´evy Processes. Cambridge University Press 1996. J.S. Baras, R.J. Elliott and M. Kohlmann: The partially observed stochastic minimum principle. SIAM J. Control Opt. 27 (1989), 1279–1292. A. Bensoussan: Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions. Stochastics 9 (1983), 169–222.

244 [Ben2]

References

A. Bensoussan: Stochastic maximum principle for systems with partial information and application to the separation principle. In M. Davis and R. Elliott (eds.): Applied Stochastic Analysis. Gordon Breach, New York 1991, pp. 157–172. [Ben3] A. Bensoussan: Stochastic Control of Partially Observable Systems. Cambridge University Press 1992. [Bi] J.-M. Bismut: Conjugate convex functions in optimal stochastic control. J. Math. Anal. Appl. 44 (1973), 384–404. [BCa] M. Bardi and I. Capuzzo-Dolcetta: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkh¨auser, Basel 1997. [BCe] B. Bassan and C. Ceci: Optimal stopping problems with discontinuous reward: regularity of the value function and viscosity solutions. Stoch. Stoch. Rep. 72 (2002), 55–71. [BDLØP] F.E. Benth, G. Di Nunno, A. Løkka, B. Øksendal and F. Proske: Explicit representation of the minimal variance portfolio in markets driven by L´evy processes. Math. Finance 13 (2003), 55–72. [BK] F.E. Benth and K.H. Karlsen: Portfolio Optimization in L´evy Markets. World Scientific (in preparation). [BKR1] F.E. Benth, K. Karlsen and K. Reikvam: Optimal portfolio management rules in a non-Gaussian market with durability and intertemporal substitution. Finance Stoch. 4 (2001), 447–467. [BKR2] F.E. Benth, K. Karlsen and K. Reikvam: Optimal portfolio selection with consumption and nonlinear integro-differential equations with gradient constraint: a viscosity solution approach. Finance Stoch. 5 (2001), 275–303. [BKR3] F.E. Benth, K.H. Karlsen and K.Reikvam: A note on portfolio management under non-gaussian logreturns. Int. J. Appl. Theor. Finance 4 (2001), 711–732. [BKR4] F.E. Benth, K.H. Karlsen and K. Reikvam: Portfolio optimization in a L´evy market with intertemporal substitution and transaction costs. Stoch. Stoch. Rep. 74 (2002), 517–569. [BKR5] F.E. Benth, K.H. Karlsen and K. Reikvam: A semilinear Black & Scholes partial differential equation for valuing American options. Finance Stoch. 7 (2003), 277–298. [BKR6] F.E. Benth, K.H. Karlsen and K. Reikvam: Mertons portfolio optimization problem in a Black and Scholes market with non-gaussian stochastic volatility of Ornstein-Uhlenbeck type. Math. Finance 13 (2003), 215–244. [BL] A. Bensoussan and J.L. Lions: Impulse Control and Quasi-Variational Inequalities. Gauthiers-Villars, Paris 1984. [BØ1] K.A. Brekke and B. Øksendal: Optimal switching in an economic activity under uncertainty. SIAM J. Control Opt. 32 (1994), 1021–1036. [BØ2] K.A. Brekke and B. Øksendal: A verification theorem for combined stochastic control and impulse control. In L. Decreusefond et al. (eds.): Stochastic Analysis and Related Topics, Vol. 6, Birkh¨auser, Basel 1998, 211– 220. [BR] F.E. Benth and K. Reikvam: On a connection between singular control and optimal stopping. Appl. Math. Opt. 49 (2004), 27–41. [BS] A. Bar-Ilan and A. Sulem: Explicit solution of inventory problems with delivery lags. Math. Oper. Res. 20 (1995), 709–720.

References [BT] [C] [CE]

[CB]

[CEM] [CIL]

[CMS]

[CØS]

[CT] [D1] [D2] [DM]

[DN] [DS] [DZ] [E] [Eb]

[EK] [F]

245

D.P. Bertsekas and J.N.Tsitsiklis: An analysis of stochastic shortest path problems. Math. Oper. Res. 16 (1991), 580-595. T. Chan: Pricing contingent claims on stocks driven by L´evy processes. Ann. Appl. Probab. 9 (1999), 504–528. C.D. Charalambous and R.E. Elliott: Classes of nonlinear partially observable stochastic optimal control problems with explicit optimal control laws. SIAM J. Control Opt. 36 (1998), 542–578. C. Ceci and B. Bassan: Mixed optimal stopping and stochastic control problems with semi-continuous final reward for diffusion processes. Stoch. Stoch. Rep. 76, (2004), 323–347. M. Chaleyat-Maurel, N. El Karoui and B. Marchal: R´eflexion discontinue et syst`emes stochastiques. Ann. Probab. 8 (1980), 1049–1067. M.G. Crandall, H. Ishii and P.-L. Lions: User’s guide to viscosity solutions of second order partial differential equations. Bull. Am. Math. Soc. 27 (1992), 1–67. J. Ph. Chancelier, M. Messaoud and A. Sulem: A policy iteration algorithm for fixed point problems with nonexpansive operators. Mathematical Methods of Operations Research, 65 (2006). J.-Ph. Chancelier, B. Øksendal and A. Sulem: Combined stochastic control and optimal stopping, and application to numerical approximation of combined stochastic and impulse control, Stochastic Financial Mathematics, In A.N. Shiryaev (ed.), Steklov Mathematical Institute, Moscow, vol. 237, 149–173, 2002. R. Cont and P. Tankov: Financial Modelling with Jump Processes. Chapman & Hall/CRC, London/Boca Raton 2003. M.H.A. Davis: Linear Estimation and Stochastic Control. Chapman & Hall, London 1977. M.H.A. Davis: Lectures on Stochastic Control and Nonlinear Filtering. Tata Institute of Fundamental Research, Bombay 1984. M.H.A. Davis and I. Markus: An introduction to nonlinear filtering. In M. Hazewinkel and J.C. Willems (eds.): Stochastic Systems; The Mathematics of Filtering and Identification and Applications. Reidel, Dordrecht 1981, 53–75. M.H.A. Davis and A. Norman: Portfolio selection with transaction costs. Math. Oper. Res. 15 (1990), 676–713. F. Delbaen and W. Schachermayer: A general version of the fundamental theorem of asset pricing. Math. Ann. 300 (1994), 463–520. K. Duckworth and M. Zervos: A model for investment decisions with switching costs. Ann. Appl. Probab. 11 (2001), 239–260. H.M. Eikseth: Optimization of dividends with transaction costs. Manuscript 2001. E. Eberlein: Application of generalized hyperbolic L´evy motion to finance. In O.E. Barndorff-Nielsen (ed.): L´evy Processes. Birkh¨auser, Basel 2001, 319–336. E. Eberlein and U. Keller: Hyperbolic distributions in finance. Bernouilli 1 (1995), 281–299. N.C. Framstad: Combined Stochastic Control for Jump Diffusions with Applications to Economics. Canadian Scientist Thesis, University of Oslo 1997.

246 [FØS1]

[FØS2]

[FØS3]

[Fre] [FS] [GK] [GS] [H1] [H2] [HST] [I] [Is1]

[Is2] [Isk] [J-P] [JK1]

[JK2]

[JS] [J-PS] [K] [Ka] [Kalli]

References N.C. Framstad, B. Øksendal and A. Sulem: Optimal consumption and portfolio in a jump diffusion market. In A. Shiryaev and A. Sulem (eds.): Math. Finance INRIA, Paris 1998, 8–20. N.C. Framstad, B. Øksendal and A. Sulem: Optimal consumption and portfolio in a jump diffusion market with proportional transaction costs. J. Math. Econ. 35 (2001), 233–257. N.C. Framstad, B. Øksendal and A. Sulem: Sufficient stochastic maximum principle for optimal control of jump diffusions and applications to finance. J. Opt. Theor. Appl. 121 (2004), 77–98. M. Freidlin: Functional Integration and Partial Differential Equations. Princeton University Press, 1985. W. Fleming and M. Soner: Controlled Markov Processes and Viscosity Solutions. Springer, Berlin Heidelberg New York 1993. I. Gy¨ ongy and N.V. Krylov: Stochastic partial differential equations with unbounded coefficients I, II. Stochastics 1990. I.I. Gihman and A.V. Skorohod: Controlled Stochastic Processes. Springer, Berlin Heidelberg New York 1979. U.G. Haussmann: A Stochastic Maximum Principle for Optimal Control of Diffusions. Longman, London 1986. U.G. Haussmann: The maximum principle for optimal control of diffusions with partial information. SIAM J. Control Opt. 25 (1987), 341–361. M.J. Harrison, T. Selke and A. Taylor: Impulse control of a Brownian motion. Math. Oper. Res. 8 (1983), 454–466. K. Itˆ o: Spectral type of the shift transformation of differential processes with stationary increments. Trans. Am. Math. Soc. 81 (1956), 253–263. H. Ishii: Viscosity solutions of nonlinear second order elliptic PDEs associated with impulse control problems. Funkciala Ekvacioj. 36 (1993), 132–141. H. Ishii: On the equivalence of two notions of weak solutions, viscosity solutions and distribution solutions. Funkciala Ekvacioj. 38 (1995), 101–120. Y. Iskikawa: Optimal control problem associated with jump processes. Appl. Math. Opt. 50 (2004), 21–65. M. Jeanblanc-Picqu´e: Impulse control method and exchange rate. Math. Finance 3 (1993), 161–177. E.R. Jakobsen and K.H. Karlsen: Continuous dependence estimates for viscosity solutions of integro-PDEs.. J. Differential Equations (2005), 278– 318. E.R. Jakobsen and K.H. Karlsen: A maximum principle for semicontinuous functions applicable to integro-partial differential equations. Nonlinear Diff. Eqn. Appl. 13 (2006), 1–29. J. Jacod and A. Shiryaev: Limit Theorems for Stochastic Processes. Springer, Berlin Heidelberg New York 1987. M. Jeanblanc-Picqu´e and A.N. Shiryaev: Optimization of the flow of dividends. Russian Math. Surv. 50 (1995), 257–277. N.V. Krylov: Controlled Diffusion Processes. Springer, Berlin Heidelberg New York 1980. O. Kallenberg: Foundations of Modern Probability. Second Edition. Springer-Verlag 2002. G. Kallianpur: Stochastic Filtering Theory. Springer, Berlin Heidelberg New York 1980.

References [Ko1] [Ko2]

[Ku] [KD] [KS] [Ky] [La] [Le] [LL]

[LST] [LS]

[L] [LZ]

[M] [Ma] [Me] [MR] [MS]

[Mo] [MØ] [Mort] [MP]

247

R. Korn: Portfolio optimization with strictly positive transaction costs and impulse control. Finance Stoch. 2 (1998), 85–114. R. Korn: Optimal Portfolios: Stochastic Models for Optimal Investment and Risk Management in Continuous Time. World Scientific, Singapore 1997. H.J. Kushner: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control. 10 (1972), 550-565. H.J. Kushner and P. Dupuis: Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag 1992. I. Karatzas and S. Shreve: Brownian Motion and Stochastic Calculus. 2nd Edition. Springer, Berlin Heidelberg New York 1991. A. Kyprianou: Introductory Lectures on Fluctuations of L´evy Processes with Applications. Springer, Berlin Heidelberg New York 2006. B. Larssen: The partially observed stochastic linear quadratic regulator: A direct approach. Preprint, University of Oslo 29/2001. D. Lef`evre: An introduction to utility maximization with partial observation. Finance 23 (2002), 93–126. P.-L. Lions and J.-M. Lasry: Stochastic control under partial information and applications to finance. CEREMADE (UMR 7534), Number 9912, 10/03/1999. B. Lapeyre, A. Sulem and D. Talay: Understanding Numerical Analysis for Financial Models. Cambridge University Press (in press). S. Levental and A.V. Skorohod: A necessary and sufficient condition for absence of arbitrage with tame portfolios. Ann. Appl. Probab. 5 (1995), 906–925. A. Løkka: Martingale representation and functionals of L´evy processes. Stoch. Anal. Appl. 22 (2004), 867–892. R.R. Lumley and M. Zervos: A model for investment in the natural resource industry with switching costs. Math. Oper. Res. 26 (2001), 637– 653. R. Merton: Optimal consumption and portfolio rules in a continuous time model. J. Econ. Theor. 3 (1971), 373–413. C. Makasu: On some optimal stopping and stochastic control problems with jump diffusions. Ph.D. Thesis, University of Zimbabwe 2002. J.-L. Menaldi: Optimal impulse control problems for degenerate diffusions with jumps. Acta Appl. Math. 8 (1987), 165–198. J.-L. Menaldi and M. Robin: On a singular control problem for diffusions with jumps, IEEE Trans. Automatic Control, AC-29 (1984), 991–1004. M. Mnif and A. Sulem: Optimal risk control and divident pay-outs under excess of loss reinsurance. Res. Rep. RR-5010, November 2003, InriaRocquencourt. E. Mordecki: Optimal stopping and perpatual options for L´evy processes. Finance Stoch. 6 (2002), 473–493. G. Mundaca and B. Øksendal: Optimal stochastic intervention control with application to the exchange rate. J. Math. Econ. 29 (1998), 225–243. R.E. Mortensen: Stochastic optimal control with noisy observations. Int. J. Control 4 (1966), 455–464. T. Meyer-Brandis and F. Proske: Explicit solution of a nonlinear filtering problem for L´evy processes with application to finance. Appl. Math. Opt. 50 (2004), 119–134.

248 [Ø1]

References

B. Øksendal: Stochastic Differential Equations. 6th Edition. Springer, Berlin Heidelberg New York 2003. [Ø2] B. Øksendal: Stochastic control problems where small intervention costs have big effects. Appl. Math. Opt. 40 (1999), 355–375. [Ø3] B. Øksendal: An Introduction to Malliavin Calculus with Applications to Economics. NHH Lecture Notes 1996. [Ø4] B. Øksendal: Optimal stopping with delayed information. Stoch. Dynam. 5 (2005), 271–280. [Ø5] B. Øksendal: Optimal control of stochastic partial differential equations. Stoch. Anal. Appl. 23 (2005), 165–179. [ØR] B. Øksendal and K. Reikvam: Viscosity solutions of optimal stopping problems. Stoch. Stoch. Rep. 62 (1998), 285–301. [ØPZ] B. Øksendal, F. Proske and T. Zhang: Backward stochastic partial differential equations with jumps and application to optimal control of random jump fields. Stochastics 77 (2005), 381–399. [ØS] B. Øksendal and A. Sulem: Optimal consumption and portfolio with both fixed and proportional transaction costs. SIAM J. Control Opt. 40 (2002), 1765–1790. [ØUZ] B. Øksendal, J. Ubøe and T. Zhang: Nonrobustness of some impulse control problems with respect to intervention costs. Stoch. Anal. Appl. 20 (2002), 999–1026. [Pa1] E. Pardoux: Stochastic partial differential equations and filtering of diffusion processes. Stochastics 3 (1979), 127–167. [Pa2] E. Pardoux: Filtrage non lin´eaire et ´equations aux d´eriv´ees partielles stochastiques associ´ees. Ecole d’Et´e de Probabilit´es de Saint-Flour 1989. [P] P. Protter: Stochastic Integration and Differential Equations. 2nd Edition. Springer, Berlin Heidelberg New York 2003. [Ph] H. Pham: Optimal stopping of controlled jump diffusion processes: a viscosity solution approach. J. Math. Syst. Estimation Control 8 (1998), 1–27. [Pu] M.L. Puterman: Markov Decision Processes: Discrete Stochastics Dynamic Programming. Probability and Mathematical Statistics: applied probability and statistics section. Wiley 1994. [PBGM] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko: The Mathematical Theory of Optimal Processes. Wiley, New York 1962. [R] R.T. Rockafellar: Convex Analysis. Princeton University Press 1970. [S] K. Sato: L´evy Processes and Infinitely Divisible Distributions. Cambridge University Press 1999. [ST] C. Schwab and R.A. Todor: Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients. IMA J. Numer. Anal. (to appear). [Sh] A. Shiryaev: Optimal Stopping Rules. Springer, Berlin Heidelberg New York 1978. [Sc] W. Shoutens: L´evy Processes in Finance. Wiley, New York 2003. [SeSy] A. Seierstad and K. Sydsæter: Optimal Control Theory with Economic Applications. North-Holland, Amsterdam 1987. [SS] S.E. Shreve and H.M. Soner: Optimal investment and consumption with transaction costs. Ann. Appl. Probab. 4 (1994), 609–692.

References [S1] [S2] [V] [W]

[YZ]

249

A. Sulem: A solvable one-dimensional model of a diffusion inventory system. Math. Oper. Res. 11 (1986), 125–133. A. Sulem: Explicit solution of a two-dimensional deterministic inventory problem. Math. Oper. Res. 11 (1986), 134–146. H. Varner: Some Impulse Control Problems with Applications to Economics. Canadian Scientist Thesis, University of Oslo, 1997. Y. Willassen: The stochastic rotation problem: a generalization of Faustmann’s formula to stochastic forest growth. J. Econ. Dynam. Control 22 (1998), 573–596. J. Yong and X.Y. Zhou: Stochastic Controls. Springer, Berlin Heidelberg New York 1999.

Notation and Symbols

Rn R+ Rn×m Z N B0 Rn ≃ Rn×1 In AT P(Rk ) C(U, V ) C(U ) C0 (U ) C k = C k (U ) C0k = C0k (U ) C k+α C 1,2 (R × Rn ) Cb (U ) |x|2 = x2 x·y x+ x−

n-dimensional Euclidean space the nonnegative real numbers the n × m matrices (real entries) the integers the natural numbers ¯ the family of Borel sets U ⊂ R whose closure U does not contain 0 i.e., vectors in Rn are regarded as n × 1 matrices the n × n identity matrix the transposed of the matrix A set of functions f : Rk → R of at must polynomial growth, i.e., there exists constants C, m such that: |f (y)| ≤ C(1 + |y|m ) for all y ∈ Rk the continuous functions from U into V the same as C(U, R) the functions in C(U ) with compact support the functions in C(U, R) with continuous derivatives up to order k the functions in C k (U ) with compact support in U the functions in C k whose kth derivatives are Lipschitz continuous with exponent α the functions f (t, x); R × Rn → R which are C 1 w.r.t. t ∈ R and C 2 w.r.t. x ∈ Rn the bounded continuous functions on U % n 2 n=1 xi if x = (x %1n, . . . , xn ) the dot product n=1 xi yi if x = (x1 , . . . , xn ), y = (y1 , . . . , yn ) max(x, 0) if x ∈ R max(−x, 0) if x ∈ R

252

Notation and Symbols

sign x sinh(x)



1 if x ≥ 0 −1 if x > 0

hyperbolic sine of x =

ex −e−x 2

cosh(x)

hyperbolic cosine of x =

tgh(x) s∧t s∨t δx Argmaxu∈U f (u) := lim, lim supp f

sinh(x) cosh(x)

∇f

∂G ¯ G G0 χG

!

ex +e−x 2

!

the minimum of s and t (= min(s, t)) the maximum of s and t (= max(s, t)) the unit point mass at x {u∗ ∈ U ; f (u∗ ) ≥ f (u), ∀u ∈ U } equal to by definition the same as lim inf, lim sup the support of the )function f * the same as Df =

∂f ∂xi

n

i=1

the boundary of the set G the closure of the set G the interior of the set G the indicator function of the set G; χG (x) = 1 if x ∈ G, χG (x) = 0 if x ∈ G filtered probability space (Ω, F, (Ft )t≥0 , P ) the jump of ηt defined by ∆ηt = ηt − ηt− ∆ηt P the probability law of ηt N (t, U ) see (1.1.2) ν(U ) E[N (1, U )] see (1.1.3) ν the norm (total mass) of the measure ν, i.e., ν(R) ˜ (dt, dz) N see (1.1.7) B(t) Brownian motion P ≪Q the measure P is absolutely continuous w.r.t. the measure Q P ∼Q P is equivalent to Q, i.e., P ≪ Q and Q ≪ P EQ the expectation w.r.t. the measure Q E the expectation w.r.t. a measure which is clear from the context (usually P )  E[Y ] = E μ [Y ] = Y dμ the expectation of the random variable Y w.r.t. the measure μ [X, Y ] quadratic covariation of X and Y , see Definition 1.28 T set of all stopping times ≤ τS see (2.1.1) the first exit time from the set G of a process Xt : τG τG = inf{t > 0; Xt ∈ G} ΔN Y (t) the jump of Y caused by the jump of N , see (5.2.2) Yˇ (t− ) Y (t− ) + ΔN Y (t) (see (6.1.5)) Δξ Y (t) the jump of Y caused by the singular control ξ Δξ φ see (5.2.3)

Notation and Symbols

ξ c (t) π/K A = AY M VI QVI HJB HJBVI HJBQVI SDE c`adl` ag c`agl` ad i.i.d. iff a.a., a.e., a.s. w.r.t. s.t.

253

continuous part of ξ(t), i.e., the process obtained by removing the jumps of ξ(t) the restriction of the measure π to the set K the generator of jump diffusion Y intervention operator, see Definition 6.1 variational inequality quasivariational inequality Hamilton–Jacobi–Bellman equation Hamilton–Jacobi–Bellman variational inequality Hamilton–Jacobi–Bellman quasivariational inequality Stochastic differential equation right continuous with left limits left continuous with right limits independent identically distributed if and only if almost all, almost everywhere, almost surely with respect to such that

Index

adjoint equation, 54, 164 adjoint operator, 164 adjoint processes, 164 admissible, 21, 45, 48, 53, 58, 78, 80 admissible combined controls, 123 admissible controls, 163 admissible impulse controls, 92 approximation theorem, 28 arbitrage, 20, 21 Arrow condition, 168 average maximum condition, 175 backward stochastic differential equation, 54 backward stochastic partial differential equation, 164 bankruptcy time, 27, 45 c` adl` ag, 1 c` agl` ad, 5 combined control, 123 combined impulse linear regulator problem, 132 combined optimal stopping and stochastic control, 65, 67 combined stochastic control and impulse control, 123, 124 comparison theorem, 149 compensated Poisson random measure, 4 compound Poisson process, 3 continuation region, 29, 68, 93 control process, 45, 53 controlled jump diffusion, 45

controls which do not depend on x, 174 delayed effect, 37 delayed information, 36 delayed optimal stopping, 42, 44 delayed stopping times, 36 diagonally dominant matrix, 156 discrete maximum principle, 157 Duncan–Mortensen–Zakai, 179, 180 dynamic programming, 45, 57 dynamic programming principle, 110, 137, 138 Dynkin formula, 12 equivalent measure, 13 finite difference approximation, 153 first exit time from a ball, 23 first Fundamental Theorem of Asset Pricing, 21 fixed transaction cost, 96 full observation control, 176 geometric L´evy martingales, 22 geometric L´evy process, 7, 22, 23 Girsanov theorem, 12, 14–17 graph of the geometric L´evy process, 23 Hamilton–Jacobi–Bellman, 46 Hamiltonian, 53, 163 high contact principle, 33, 68 HJB-variational inequalities, 68 HJBQVI verification theorem, 124 Howard algorithm, 158

256

Index

impulse control, 91, 93 impulses, 91 infinite-dimentional HJB equation, 162 integration by parts, 24, 55 integration by parts for jump diffusions, 166 integro-variational inequalities for optimal stopping, 29 integro-variational inequalities for singular control, 81 integrodifferential operator, 28 intensity, 2 intervention control, 79 intervention operator, 93 intervention times, 91 Itˆ o formula, 7, 8 Itˆ o representation theorem, 172 Itˆ o–L´evy isometry, 9 Itˆ o–L´evy processes, 5 iterated optimal stopping, 107 iterative methods, 131 iterative procedure, 109 jump (of a L´evy process), 1 jump diffusion, 11 L´evy L´evy L´evy L´evy L´evy L´evy

decomposition, 3 diffusion, 10 martingale, 4 measure, 2 process, 1 stochastic differential equations, 10 L´evy type Black–Scholes market, 47 L´evy–Khintchine formula, 4 Lipschitz surface, 28 local time, 83 Markov controls, 46 maximum principle, 52–54, 162–164 mean-reverting L´evy–Ornstein– Uhlenbeck process, 22 mean-variance portfolio selection problem, 58 non-intervention region, 81 observation process, 177 operator, 164

optimal combined control of the exchange rate, 126 optimal consumption and portfolio, 47, 127, 152, 158 optimal consumption rate under proportional transaction costs, 77 optimal control, 46 optimal dividend policy, 87 optimal forest management, 104 optimal harvesting, 88 optimal resource extraction problem, 43, 65 optimal stopping problem, 27 optimal stopping time, 28 optimal stream of dividends, 95, 104 partial information, 161 partial observation control, 176 partial observation SDE control problem, 180 performance, 28, 45, 48, 53, 67, 79, 124, 163 Poisson process, 2 Poisson random measure, 1 policy iteration algorithm, 156, 158 polynomial growth, 110 portfolio, 20, 58 predictable processes, 10 proportional transaction cost, 96 quadratic covariation, 14, 24 quasi-integrovariational inequalities, 93 quasivariational inequality, 149 random jump field, 161, 162, 165 reaction–diffusion, 161 resource extraction, 40 second Fundamental Theorem of Asset Pricing, 19 self-financing, 21, 58 shift operator, 39 singular control, 77, 79 smooth fit principle, 68 solvency set, 27, 45, 78, 79 state process, 177 stochastic control, 45, 46 stochastic control problem with partial observation, 178

Index stochastic linear regulator problem, 62 stochastic linear regulator problem with optimal stopping, 76 stochastic partial differential equation SPDE, 161 strengthened maximum principle, 168 supergradient, 169 superjets, 148

257

unnormalized conditional density, 179, 180

time homogeneous, 11

value function, 28, 46 verification theorem, 46, 68 viscosity solution, 135, 136, 144 viscosity solution of HJBQVI, 142, 155 viscosity subsolution, 136, 144 viscosity supersolution, 136, 144

uniqueness of viscosity solutions, 138, 152

wealth process, 20, 48, 58

Universitext Aguilar, M.; Gitler, S.; Prieto, C.: Algebraic Topology from a Homotopical Viewpoint Aksoy, A.; Khamsi, M. A.: Methods in Fixed Point Theory Alevras, D.; Padberg M. W.: Linear Optimization and Extensions Andersson, M.: Topics in Complex Analysis Aoki, M.: State Space Modeling of Time Series Arnold, V. I.: Lectures on Partial Differential Equations Arnold, V. I.; Cooke, R.: Ordinary Differential Equations Audin, M.: Geometry Aupetit, B.: A Primer on Spectral Theory Bachem, A.; Kern, W.: Linear Programming Duality Bachmann, G.; Narici, L.; Beckenstein, E.: Fourier and Wavelet Analysis Badescu, L.: Algebraic Surfaces Balakrishnan, R.; Ranganathan, K.: A Textbook of Graph Theory Balser, W.: Formal Power Series and Linear Systems of Meromorphic Ordinary Differential Equations Bapat, R.B.: Linear Algebra and Linear Models Benedetti, R.; Petronio, C.: Lectures on Hyperbolic Geometry Benth, F. E.: Option Theory with Stochastic Analysis Berberian, S. K.: Fundamentals of Real Analysis Berger, M.: Geometry I, and II Bliedtner, J.; Hansen, W.: Potential Theory Blowey, J. F.; Coleman, J. P.; Craig, A. W. (Eds.): Theory and Numerics of Differential Equations Blowey, J. F.; Craig, A.; Shardlow, T. (Eds.): Frontiers in Numerical Analysis, Durham 2002, and Durham 2004

Blyth, T. S.: Lattices and Ordered Algebraic Structures B¨orger, E.; Gr¨adel, E.; Gurevich, Y.: The Classical Decision Problem B¨ottcher, A; Silbermann, B.: Introduction to Large Truncated Toeplitz Matrices Boltyanski, V.; Martini, H.; Soltan, P. S.: Excursions into Combinatorial Geometry Boltyanskii, V. G.; Efremovich, V. A.: Intuitive Combinatorial Topology Bonnans, J. F.; Gilbert, J. C.; Lemarchal, C.; Sagastizbal, C. A.: Numerical Optimization Booss, B.; Bleecker, D. D.: Topology and Analysis Borkar, V. S.: Probability Theory Brunt B. van: The Calculus of Variations B¨uhlmann, H.; Gisler, A.: A Course in Credibility Theory and its Applications Carleson, L.; Gamelin, T. W.: Complex Dynamics Cecil, T. E.: Lie Sphere Geometry: With Applications of Submanifolds Chae, S. B.: Lebesgue Integration Chandrasekharan, K.: Classical Fourier Transform Charlap, L. S.: Bieberbach Groups and Flat Manifolds Chern, S.: Complex Manifolds without Potential Theory Chorin, A. J.; Marsden, J. E.: Mathematical Introduction to Fluid Mechanics Cohn, H.: A Classical Invitation to Algebraic Numbers and Class Fields Curtis, M. L.: Abstract Linear Algebra Curtis, M. L.: Matrix Groups Cyganowski, S.; Kloeden, P.; Ombach, J.: From Elementary Probability to Stochastic Differential Equations with MAPLE Da Prato, G.: An Introduction to Infinite Dimensional Analysis Dalen, D. van: Logic and Structure Das, A.: The Special Theory of Relativity: A Mathematical Exposition

Debarre, O.: Higher-Dimensional Algebraic Geometry Deitmar, A.: A First Course in Harmonic Analysis Demazure, M.: strophes

Bifurcations and Cata-

Devlin, K. J.: Fundamentals of Contemporary Set Theory DiBenedetto, E.: Equations

Degenerate Parabolic

Diener, F.; Diener, M.(Eds.): Nonstandard Analysis in Practice Dimca, A.: Sheaves in Topology Dimca, A.: Singularities and Topology of Hypersurfaces DoCarmo, M. P.: Differential Forms and Applications Duistermaat, J. J.; Kolk, J. A. C.: Lie Groups Dumortier.: Qualitative Theory of Planar Differential Systems Dundas, B. I.; Levine, M.; Østvaer, P. A.; R¨ondip, O.; Voevodsky, V.: Motivic Homotopy Theory Edwards, R. E.: A Formal Background to Higher Mathematics Ia, and Ib Edwards, R. E.: A Formal Background to Higher Mathematics IIa, and IIb Emery, M.: Stochastic Calculus in Manifolds Emmanouil, I.: Idempotent Matrices over Complex Group Algebras Endler, O.: Valuation Theory Engel, K.-J.; Nagel, R.: A Short Course on Operator Semigroups Erez, B.: Galois Modules in Arithmetic Everest, G.; Ward, T.: Heights of Polynomials and Entropy in Algebraic Dynamics Farenick, D. R.: Algebras of Linear Transformations Foulds, L. R.: Graph Theory Applications Franke, J.; Hrdle, W.; Hafner, C. M.: Statistics of Financial Markets: An Introduction Frauenthal, J. C.: Mathematical Modeling in Epidemiology

Freitag, E.; Busam, R.: Complex Analysis Friedman, R.: Algebraic Surfaces and Holomorphic Vector Bundles Fuks, D. B.; Rokhlin, V. A.: Beginner’s Course in Topology Fuhrmann, P. A.: A Polynomial Approach to Linear Algebra Gallot, S.; Hulin, D.; Lafontaine, J.: Riemannian Geometry Gardiner, C. F.: A First Course in Group Theory G˚arding, L.; Tambour, T.: Algebra for Computer Science Godbillon, C.: Dynamical Systems on Surfaces Godement, R.: Analysis I, and II Goldblatt, R.: Orthogonality and Spacetime Geometry Gouvˆea, F. Q.: p-Adic Numbers Gross, M. et al.: Calabi-Yau Manifolds and Related Geometries Gustafson, K. E.; Rao, D. K. M.: Numerical Range. The Field of Values of Linear Operators and Matrices Gustafson, S. J.; Sigal, I. M.: Mathematical Concepts of Quantum Mechanics Hahn, A. J.: Quadratic Algebras, Clifford Algebras, and Arithmetic Witt Groups H´ajek, P.; Havr´anek, T.: Mechanizing Hypothesis Formation Heinonen, J.: Lectures on Analysis on Metric Spaces Hlawka, E.; Schoißengeier, J.; Taschner, R.: Geometric and Analytic Number Theory Holmgren, R. A.: A First Course in Discrete Dynamical Systems Howe, R., Tan, E. Ch.: Non-Abelian Harmonic Analysis Howes, N. R.: Modern Analysis and Topology Hsieh, P.-F.; Sibuya, Y. (Eds.): Basic Theory of Ordinary Differential Equations Humi, M., Miller, W.: Second Course in Ordinary Differential Equations for Scientists and Engineers

Hurwitz, A.; Kritikos, N.: Lectures on Number Theory Huybrechts, D.: Complex Geometry: An Introduction Isaev, A.: Introduction to Mathematical Methods in Bioinformatics Istas, J.: Mathematical Modeling for the Life Sciences Iversen, B.: Cohomology of Sheaves Jacod, J.; Protter, P.: Probability Essentials Jennings, G. A.: Modern Geometry with Applications Jones, A.; Morris, S. A.; Pearson, K. R.: Abstract Algebra and Famous Inpossibilities Jost, J.: Compact Riemann Surfaces Jost, J.: Dynamical Systems. Examples of Complex Behaviour Jost, J.: Postmodern Analysis Jost, J.: Riemannian Geometry and Geometric Analysis Kac, V.; Cheung, P.: Quantum Calculus Kannan, R.; Krueger, C. K.: Analysis on the Real Line

Advanced

Kelly, P.; Matthews, G.: The Non-Euclidean Hyperbolic Plane Kempf, G.: Complex Abelian Varieties and Theta Functions Kitchens, B. P.: Symbolic Dynamics Kloeden, P.; Ombach, J.; Cyganowski, S.: From Elementary Probability to Stochastic Differential Equations with MAPLE Kloeden, P. E.; Platen; E.; Schurz, H.: Numerical Solution of SDE Through Computer Experiments Koralov, L.; Sina, Ya. G.: Theory of Probability and Random Processes Kostrikin, A. I.: Introduction to Algebra Krasnoselskii, M. A.; Pokrovskii, A. V.: Systems with Hysteresis Kuo, H.-H.: Introduction to Stochastic Integration Kurzweil, H.; Stellmacher, B.: The Theory of Finite Groups. An Introduction

Kyprianou, A.E.: Introductory Lectures on Fluctuations of L´evy Processes with Applications Lang, S.: Introduction to Differentiable Manifolds Lefebvre, M.: Applied Stochastic Processes Lorenz, F.: Algebra I: Fields and Galois Theory Luecking, D. H., Rubel, L. A.: Complex Analysis. A Functional Analysis Approach Ma, Zhi-Ming; Roeckner, M.: Introduction to the Theory of (non-symmetric) Dirichlet Forms Mac Lane, S.; Moerdijk, I.: Sheaves in Geometry and Logic Marcus, D. A.: Number Fields Martinez, A.: An Introduction to Semiclassical and Microlocal Analysis Matouˇsek, J.: Using the Borsuk-Ulam Theorem Matsuki, K.: Introduction to the Mori Program Mazzola, G.; Milmeister G.; Weissman J.: Comprehensive Mathematics for Computer Scientists 1 Mazzola, G.; Milmeister G.; Weissman J.: Comprehensive Mathematics for Computer Scientists 2 Mc Carthy, P. J.: Introduction to Arithmetical Functions McCrimmon, K.: A Taste of Jordan Algebras Meyer, R. M.: Essential Mathematics for Applied Field Meyer-Nieberg, P.: Banach Lattices Mikosch, T.: Non-Life Insurance Mathematics Mines, R.; Richman, F.; Ruitenburg, W.: A Course in Constructive Algebra Moise, E. E.: Introductory Problem Courses in Analysis and Topology Montesinos-Amilibia, J. M.: Classical Tessellations and Three Manifolds Morris, P.: Introduction to Game Theory Nicolaescu, L.: An Invitation to Morse Theory

Nikulin, V. V.; Shafarevich, I. R.: Geometries and Groups Oden, J. J.; Reddy, J. N.: Variational Methods in Theoretical Mechanics Øksendal, B.: Stochastic Differential Equations Øksendal, B.; Sulem, A.: Applied Stochastic Control of Jump Diffusions Orlik, P.; Welker, V.: Algebraic Combinatorics Poizat, B.: A Course in Model Theory Polster, B.: A Geometrical Picture Book Porter, J. R.; Woods, R. G.: Extensions and Absolutes of Hausdorff Spaces Procesi, C.: Lie Groups Radjavi, H.; Rosenthal, P.: Simultaneous Triangularization Ramsay, A.; Richtmeyer, R. D.: Introduction to Hyperbolic Geometry Rautenberg, W.: A concise Introduction to Mathematical Logic Rees, E. G.: Notes on Geometry Reisel, R. B.: Elementary Theory of Metric Spaces Rey, W. J. J.: Introduction to Robust and Quasi-Robust Statistical Methods Ribenboim, P.: Classical Theory of Algebraic Numbers Rickart, C. E.: Natural Function Algebras Rotman, J. J.: Galois Theory Rubel, L. A.: Entire and Meromorphic Functions Ruiz-Tolosa, J. R.; Castillo E.: From Vectors to Tensors Runde, V.: A Taste of Topology Rybakowski, K. P.: The Homotopy Index and Partial Differential Equations Sagan, H.: Space-Filling Curves Samelson, H.: Notes on Lie Algebras Sauvigny, F.: Partial Differential Equations I Sauvigny, F.: Partial Differential Equations II Schiff, J. L.: Normal Families

Sengupta, J. K.: Optimal Decisions under Uncertainty S´eroul, R.: Programming for Mathematicians Tools for Computational Seydel, R.: Finance Shafarevich, I. R.: Discourses on Algebra Shapiro, J. H.: Composition Operators and Classical Function Theory Simonnet, M.: Measures and Probabilities Smith, K. E.; Kahanp¨aa¨, L.; Kek¨al¨ainen, P.; Traves, W.: An Invitation to Algebraic Geometry Smith, K. T.: Power Series from a Computational Point of View Smory´nski, C.: Logical Number Theory I. An Introduction Stichtenoth, H.: Algebraic Function Fields and Codes Stillwell, J.: Geometry of Surfaces Stroock, D. W.: An Introduction to the Theory of Large Deviations Sunder, V. S.: An Invitation to von Neumann Algebras ´ Introduction to Etale Tamme, G.: Cohomology Tondeur, P.: Foliations on Riemannian Manifolds Toth, G.: Finite Mbius Groups, Minimal Immersions of Spheres, and Moduli Tu, L. W.: An Introduction to Manifolds Verhulst, F.: Nonlinear Differential Equations and Dynamical Systems Weintraub, S. H.: Galois Theory Wong, M. W.: Weyl Transforms Xamb´o-Descamps, S.: Block Error-Correcting Codes Zaanen, A.C.: Continuity, Integration and Fourier Theory Zhang, F.: Matrix Theory Zong, C.: Sphere Packings Zong, C.: Strange Phenomena in Convex and Discrete Geometry Zorich, V. A.: Mathematical Analysis I Zorich, V. A.: Mathematical Analysis II