Applied computational economics and finance 9780262633093, 0262633094
This book presents a variety of computational methods used to solve dynamic problems in economics and finance. It emphas
364 119 6MB
English Pages 521 Year 2002
Recommend Papers
File loading please wait...
Citation preview
Applied Computational E onomi s and Finan e Mario J. Miranda The Ohio State University
and
Paul L. Fa kler North Carolina State University
Contents Prefa e
xii
1 Introdu tion
1
1.1
Some Apparently Simple Questions
. . . . . . . . . . . . . . . . . . .
1
1.2
An Alternative Analyti Framework . . . . . . . . . . . . . . . . . . .
3
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
2 Computer Basi s and Linear Equations
8
2.1
Computer Arithmeti . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2
Data Storage
2.3
Linear Equations and the L-U Fa torization
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8 12 13
2.4
Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
2.5
Rounding Error
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
2.6
Ill Conditioning
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
2.7
Spe ial Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . .
22
2.8
Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
Bibliographi Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
28
3 Nonlinear Equations
30
3.1
Bise tion Method
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
3.2
Fun tion Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
3.3
Newton's Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
3.4
Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . .
39
3.5
Problems With Newton Methods
43
. . . . . . . . . . . . . . . . . . . .
3.6
Choosing a Solution Method . . . . . . . . . . . . . . . . . . . . . . .
46
3.7
Complementarity Problems
48
3.8
. . . . . . . . . . . . . . . . . . . . . . .
Complementarity Methods . . . . . . . . . . . . . . . . . . . . . . . .
51
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
Bibliographi Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
i
CONTENTS
ii
4 Finite-Dimensional Optimization 4.1 Derivative-Free Methods . 4.2 Newton-Raphson Method 4.3 Quasi-Newton Methods . . 4.4 Line Sear h Methods . . . 4.5 Spe ial Cases . . . . . . . 4.6 Constrained Optimization Exer ises . . . . . . . . . . . . . Bibliographi Notes . . . . . . .
. . . . . . . .
. . . . . . . .
5 Integration and Di erentiation
. . . . . . . .
5.1 Newton-Cotes Methods . . . . . 5.2 Gaussian Quadrature . . . . . . 5.3 Monte Carlo Integration . . . . 5.4 Quasi-Monte Carlo Integration . 5.5 An Integration Toolbox . . . . . 5.6 Numeri al Di erentiation . . . . 5.7 Initial Value Problems . . . . . Exer ises . . . . . . . . . . . . . . . . Bibliographi Notes . . . . . . . . . .
6 Fun tion Approximation 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
Interpolation Prin iples . . . . . . . . Polynomial Interpolation . . . . . . . Pie ewise Polynomial Splines . . . . Pie ewise-Linear Basis Fun tions . . Multidimensional Interpolation . . . Choosing an Approximation Method An Approximation Toolkit . . . . . . Solving Fun tional Equations . . . . 6.8.1 Cournot Oligopoly . . . . . . 6.8.2 Fun tion Inverses . . . . . . . 6.8.3 Boundary Value Problems . . Exer ises . . . . . . . . . . . . . . . . . . . Bibliographi Notes . . . . . . . . . . . . .
7 Dis rete State Models
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63 64 68 70 74 77 79 83 93
94
95 97 100 102 104 107 114 121 126
127
128 131 137 143 145 149 150 158 159 162 164 172 176
177
7.1 Dis rete Dynami Programming . . . . . . . . . . . . . . . . . . . . . 178 7.2 E onomi Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 7.2.1 Mine Management . . . . . . . . . . . . . . . . . . . . . . . . 180
CONTENTS
iii
7.2.2
Asset Repla ement - I
. . . . . . . . . . . . . . . . . . . . . .
181
7.2.3
Asset Repla ement - II . . . . . . . . . . . . . . . . . . . . . .
182
7.2.4
Option Pri ing
. . . . . . . . . . . . . . . . . . . . . . . . . .
183
7.2.5
Job Sear h . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
184
7.2.6
Optimal Irrigation
. . . . . . . . . . . . . . . . . . . . . . . .
185
7.2.7
Optimal Growth
. . . . . . . . . . . . . . . . . . . . . . . . .
186
7.2.8
Renewable Resour e Problem
. . . . . . . . . . . . . . . . . .
187
7.2.9
Bioe onomi Model . . . . . . . . . . . . . . . . . . . . . . . .
188
7.3
Solution Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . .
189
7.4
Dynami Simulation Analysis
. . . . . . . . . . . . . . . . . . . . . .
193
7.5
A Dis rete Dynami Programming Toolbox . . . . . . . . . . . . . . .
195
7.6
Numeri al Examples
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
198
7.6.1
Mine Management
7.6.2
Asset Repla ement - I
7.6.3
Asset Repla ement - II . . . . . . . . . . . . . . . . . . . . . .
202
7.6.4
Option Pri ing
202
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
198 201
7.6.5
Job Sear h . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
204
7.6.6
Optimal Irrigation
. . . . . . . . . . . . . . . . . . . . . . . .
206
7.6.7
Optimal Growth
. . . . . . . . . . . . . . . . . . . . . . . . .
208
7.6.8
Renewable Resour e Problem
. . . . . . . . . . . . . . . . . .
208
7.6.9
Bioe onomi Model . . . . . . . . . . . . . . . . . . . . . . . .
208
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
210
8 Continuous State Models: Theory 8.1 8.2
8.3
218
Continuous State Dynami Programming . . . . . . . . . . . . . . . .
219
Continuous State Dis rete Choi e Models . . . . . . . . . . . . . . . .
221
8.2.1
221
Asset Repla ement
. . . . . . . . . . . . . . . . . . . . . . . .
8.2.2
Timber Cutting . . . . . . . . . . . . . . . . . . . . . . . . . .
222
8.2.3
Ameri an Option Pri ing . . . . . . . . . . . . . . . . . . . . .
222
8.2.4
Industry Entry and Exit . . . . . . . . . . . . . . . . . . . . .
223
8.2.5
Job Sear h . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
224
Continuous State Continuous Choi e Models . . . . . . . . . . . . . .
225
8.3.1
Optimal E onomi Growth . . . . . . . . . . . . . . . . . . . .
227
8.3.2
Publi Renewable Resour e Management . . . . . . . . . . . .
229
8.3.3
Private Nonrenewable Resour e Management . . . . . . . . . .
230
8.3.4
Optimal Water Management . . . . . . . . . . . . . . . . . . .
231
8.3.5
Optimal Monetary Poli y
233
. . . . . . . . . . . . . . . . . . . .
8.3.6
Produ tion-Adjustment Model . . . . . . . . . . . . . . . . . .
234
8.3.7
Produ tion-Inventory Model . . . . . . . . . . . . . . . . . . .
235
8.3.8
Optimal Feeding
236
. . . . . . . . . . . . . . . . . . . . . . . . .
CONTENTS
iv
8.4
Linear-Quadrati Control . . . . . . . . . . . . . . . . . . . . . . . . .
237
8.5
Dynami Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
239
8.5.1
240
8.6
Capital-Produ tion Game
. . . . . . . . . . . . . . . . . . . .
8.5.2
Risk-Sharing Game . . . . . . . . . . . . . . . . . . . . . . . .
241
8.5.3
Marketing Board Game
. . . . . . . . . . . . . . . . . . . . .
241
Rational Expe tations Models . . . . . . . . . . . . . . . . . . . . . .
242
8.6.1
Asset Pri ing Model
. . . . . . . . . . . . . . . . . . . . . . .
244
8.6.2
Competitive Storage
. . . . . . . . . . . . . . . . . . . . . . .
245
8.6.3
Government Pri e Controls
. . . . . . . . . . . . . . . . . . .
247
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
249
9 Continuous State Models: Methods 9.1
Traditional Solution Methods
9.2
The Collo ation Method
. . . . . . . . . . . . . . . . . . . . . . . . .
9.3
Postoptimality Analysis
. . . . . . . . . . . . . . . . . . . . . . . . .
269
9.4
Computational Examples . . . . . . . . . . . . . . . . . . . . . . . . .
271
9.4.1
271
9.5
9.6
9.7
Asset Repla ement
. . . . . . . . . . . . . . . . . . . . . .
257
. . . . . . . . . . . . . . . . . . . . . . . .
258
260
9.4.2
Timber Cutting . . . . . . . . . . . . . . . . . . . . . . . . . .
274
9.4.3
Optimal E onomi Growth . . . . . . . . . . . . . . . . . . . .
276
9.4.4
Publi Renewable Resour e Management . . . . . . . . . . . .
281
9.4.5
Private Nonrenewable Resour e Management . . . . . . . . . .
284
9.4.6
Optimal Monetary Poli y
288
9.4.7
. . . . . . . . . . . . . . . . . . . .
Produ tion-Adjustment Model . . . . . . . . . . . . . . . . . .
292
Dynami Game Methods . . . . . . . . . . . . . . . . . . . . . . . . .
296
9.5.1
Capital-Produ tion Game
301
9.5.2
In ome Redistribution Game . . . . . . . . . . . . . . . . . . .
Rational Expe tations Methods
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
305 309
9.6.1
Asset Pri ing Model
. . . . . . . . . . . . . . . . . . . . . . .
313
9.6.2
Competitive Storage
. . . . . . . . . . . . . . . . . . . . . . .
315
Comparison of Solution Methods
. . . . . . . . . . . . . . . . . . . .
320
Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
322
10 Continuous Time - Theory & Examples 10.1 Arbitrage Based Asset Valuation 10.2 Sto hasti Control
. . . . . . . . . . . . . . . . . . . .
330
330
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
338
10.2.1 Boundary Conditions . . . . . . . . . . . . . . . . . . . . . . .
340
10.2.2 Choi e of the Dis ount Rate . . . . . . . . . . . . . . . . . . .
342
10.2.3 Euler Equation Methods . . . . . . . . . . . . . . . . . . . . .
344
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
345
10.3 Free Boundary Problems . . . . . . . . . . . . . . . . . . . . . . . . .
10.2.4 Examples
355
CONTENTS
v
10.3.1 Impulse Control . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 Barrier Control . . . . . . . . . . . . . . . . . . . . . . . . 10.3.3 Dis rete State/Control Problems . . . . . . . . . . . . . . 10.3.4 Sto hasti Bang-Bang Problems . . . . . . . . . . . . . . . Exer ises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A: Dynami Programming and Optimal Control Theory . . . Appendix B: Deriving the Boundary Conditions for Resetting Problems Appendix C: Deterministi Bang-Bang Problems . . . . . . . . . . . . Bibliographi Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Continuous Time - Methods
11.1 Solving Arbitrage-based Valuation Problems 11.1.1 Extensions and Re nements . . . . . 11.2 Solving Sto hasti Control Problems . . . . 11.3 Free Boundary Problems . . . . . . . . . . . 11.3.1 Multiple Value Fun tions . . . . . . . 11.3.2 Finite Horizon Problems . . . . . . . Exer ises . . . . . . . . . . . . . . . . . . . . . . . Bibliographi Notes . . . . . . . . . . . . . . . . .
A Mathemati al Ba kground
A.1 A.2 A.3 A.4 A.5
Normed Linear Spa es . . . . . . . . . . Matrix Algebra . . . . . . . . . . . . . . Real Analysis . . . . . . . . . . . . . . . Markov Chains . . . . . . . . . . . . . . Continuous Time Mathemati s . . . . . . A.5.1 Ito Pro esses . . . . . . . . . . . A.5.2 Forward and Ba kward Equations A.5.3 The Feynman-Ka Equation . . . Bibliographi Notes . . . . . . . . . . . . . . .
B A MATLAB Primer
B.1 B.2 B.3 B.4 B.5 B.6
The Basi s . . . . . . . Conditional Statements S ripts and Fun tions . Debugging . . . . . . . Other Data Types . . . Programming Style . .
Web Resour es
. . . And . . . . . . . . . . . .
. . . . . Looping . . . . . . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
357 361 363 369 376 388 390 392 395 397
398 401 408 419 430 443 452 461
463
463 466 468 470 471 471 475 478 480
481
481 486 487 492 494 495
497
CONTENTS
vi
Referen es
498
Index
503
List of Tables 5.1. Errors for Sele ted Quadrature Methods . . . . . . . . . . . . . . . . .
98
5.2. Approximation Errors for Alternative Quasi-Monte Carlo Methods . .
103
6.1. Errors for Sele ted Interpolation Methods . . . . . . . . . . . . . . . .
150
7.1
Optimal Labor Parti ipation Rule . . . . . . . . . . . . . . . . . . . .
206
7.2
Survival Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . .
211
7.3
Optimal Foraging Strategy . . . . . . . . . . . . . . . . . . . . . . . .
211
9.1
Exe ution Times and Approximation Error for Sele ted ContinuousSpa e Approximation Methods . . . . . . . . . . . . . . . . . . . . . .
323
10.1. Known Solutions to the Optimal Harvesting Problem . . . . . . . . .
346
10.2. Types of Free Boundary Problems
355
. . . . . . . . . . . . . . . . . . .
11.1. Option Pri ing Approximation Errors
vii
. . . . . . . . . . . . . . . . .
407
List of Figures 3.1. demslv10
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
3.2. demslv11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
3.3. demslv11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
3.4. demslv12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
3.5. demslv11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
3.6. demslv12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
3.7. demslv13
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
3.8. demslv14
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
3.9. demslv15
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
4.1. demopt01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
3.10. demslv16
4.2. demopt02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
4.3. demopt03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
4.4. demopt04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
4.5. demopt04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75
4.6. demopt05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
76
4.7. demopt06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
80
4.8. demopt06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
81
5.1. demqua01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
104
5.2. demqua01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
105
5.3. demdif01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
111
5.4. demdif02
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
112
5.5. demdif03
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
119
6.1. demapp01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
132
6.2. demapp02
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
133
6.3. demapp01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
134
6.4. demapp01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
135
6.5. demapp01
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
139
viii
LIST OF FIGURES 6.6. demapp01 6.7. demapp04 6.8. demapp04 6.9. demapp05 6.10. demapp05 6.11. demapp05 6.12. demapp05 6.13. demapp06 6.14. demapp09 6.15. demapp09 6.16. demapp09 6.17. demapp10 6.18. demapp10 6.19. dembvp02 6.20. dembvp02
ix . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
141 142 143 152 153 154 155 157 162 163 164 165 166 170 171
7.1. demddp01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 7.2. demddp04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 7.3. demddp06 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 9.1. demdp01 . . 9.2. demdp02 . . 9.3. demdp07 . . 9.4. demdp08 . . 9.5. demdp09 . . 9.6. demdp11 . . 9.7. demdp12 . . 9.8. demgame01 . 9.9. demgame02 . 9.10. demrem01 . 9.11. demrem02 .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
273 276 279 283 286 291 294 305 308 315 319
10.1. demfb05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 11.1. 11.2. 11.3. 11.4. 11.5. 11.6. 11.7.
dem n01 dem n01 dem n02 dems 1 . dems 03 . dems 03 . demfb01 .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
401 402 408 415 420 420 425
LIST OF FIGURES 11.8. demfb02 . 11.9. demfb02 . 11.10. demfb03 11.11. demfb03 11.12. demfb03 11.13. demfb03 11.14. demfb04 11.15. demfb04 11.16. demfb04 11.17. dem n04 11.18. dem n04 11.19. dem n04 11.20. demfb05
x . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
429 429 433 435 437 437 442 442 443 446 446 447 451
LIST OF FIGURES
xi
Glossary of Matlab Terms
Matlab terms used in this text. For a omplete list, see Matlab do u-
A list of mentation.
hol diag disp eps eye feval nd inline
inv length norm rand randn realmax reshape sum tril triu
omputes the Cholesky de omposition of a symmetri positive de nite matrix returns the diagonal elements of a matrix as a ve tor displays results on the s reen ma hine pre ision (the largest number that, added to 1, returns 1) returns an order n identity matrix: In=eye(n) evaluates a fun tion referred to by name: feval(f,x,y) produ es an index of values meeting a stated ondition: find([-1 3 2 0℄>=0) returns [2 4℄
reates a fun tion from a string that behaves like a fun tion le: f=inline('x.^2+2*y','x','y')
matrix inverse the number of elements in a ve tor: length([0 5 2℄) returns 3 ve tor or matrix norm (default is the 2-norm) produ es random uniformly distributed values on [0,1℄: x=rand(m,n) produ es random standard normal (Gaussian) variates: x=randn(m,n) the largest real number representable in
hanges the size of a matrix without hanging the total number of elements: reshape([1 1;1 1;2 2;2 2℄,2,4℄) returns [1 2 1 2;1 2 1 2℄ sums the elements of a ve tor or olumns of a matrix: sum([1 2 3℄ returns 6 zeros the above diagonal elements of a matrix: tril([1 2;3 4℄) returns [1 0;3 4℄ zeros the below diagonal elements of a matrix: triu([1 2;3 4℄) returns [1 2;0 4℄
Matlab
Prefa e Many interesting e onomi models annot be solved analyti ally using the standard mathemati al te hniques of Algebra and Cal ulus. This is often true of applied e onomi models that attempt to apture the omplexities inherent in real-world individual and institutional e onomi behavior. For example, to be useful in applied e onomi analysis, the onventional Marshallian partial stati equilibrium model of supply and demand must often be generalized to allow for multiple goods, interegional trade, intertemporal storage, and government interventions su h as tari s, taxes, and trade quotas. In su h models, the stru tural e onomi onstraints are of entral interest to the e onomist, making it undesirable, if not impossible, to \assume an internal solution" to render the model analyti ally tra table. Another lass of interesting models that typi ally annot be solved analyti ally are sto hasti dynami models of rational, forward-looking e onomi behavior. Dynami e onomi models typi ally give rise to fun tional equations in whi h the unknown is not simply a ve tor in Eu lidean spa e, but rather an entire fun tion de ned on a
ontinuum of points. For example, the Bellman and Euler equations that des ribe dynami optima are fun tional equations, as often are the onditions that hara terize rational expe tations and arbitrage pri ing market equilibria. Ex ept in a very limited number of spe ial ases, these fun tional equations la k a known losed-form solution, even though the solution an be shown theoreti ally to exist and to be unique. Models that la k losed-form analyti al solution are not unique to e onomi s. Analyti ally insoluble models are ommon in biologi al, physi al, and engineering s ien es. Sin e the introdu tion of the digital omputer, s ientists in these elds have turned in reasingly to numeri al omputer methods to solve their models. In many
ases where analyti al approa hes fail, numeri al methods are often used to su
essfully ompute highly a
urate approximate solutions. In re ent years, the s ope of numeri al appli ations in the biologi al, physi al, and engineering s ien es has grown dramati ally. In most of these dis iplines, omputational model building and analysis is now re ognized as a legitimate subdis ipline of spe ialization. Numeri al analysis
ourses have also be ome standard in many graduate and undergraduate urri ulums in these elds. xii
PREFACE
xiii
E onomists, however, have not embra ed numeri al methods as eagerly as other s ientists. Many e onomists have shunned numeri al methods out of a belief that numeri al solutions are less elegant or less general than losed form solutions. The former belief is a subje tive, aestheti judgment that is outside of s ienti dis ourse and beyond the s ope of this book. The generality of the results obtained from numeri al e onomi models, however, is another matter. Of ourse, given an e onomi model, it is always preferable to derive a losed form solution|provided su h a solution exists. However, when essential features of an e onomi system being studied annot be aptured neatly in an algebrai ally soluble model, a hoi e must be made. Either essential features of the system must be ignored in order to obtain an algebrai ally tra table model, or numeri al te hniques must be applied. Too often e onomists hose algebrai tra tability over e onomi realism. Numeri al e onomi models are often unfairly riti ized by e onomists on the grounds that they rest on spe i assumptions regarding fun tional forms and parameter values. Su h riti ism, however, is unwarranted when strong empiri al support exists for the spe i fun tional form and parameter values used to spe ify a model. Moreover, even when there is some un ertainty about fun tional forms and parameters, the model may be solved under a variety of assumptions in order to assess the robustness of its impli ations. Although some doubt will persist as to the impli ations of a model outside the range of fun tional forms and parameter values examined, this un ertainty must be weighed against the la k of relevan e of an alternative model that is expli itly soluble, but whi h ignores essential features of the e onomi system of interest. We believe that it is better to derive e onomi insights from a realisti numeri al model of an e onomi system than to derive irrelevant results, however general, from an unrealisti , but expli itly soluble model. Despite resistan e by some, an in reasing number of e onomists are be oming aware of the potential bene ts of numeri al e onomi model building and analysis. This is eviden ed by the re ent introdu tion of journals and an e onomi so iety devoted to the sub-dis ipline of omputational e onomi s. The growing popularity of omputational e onomi s, however, has been impeded by the absen e of adequate textbooks and omputer software. The methods of numeri al analysis and mu h of the available omputer software have been largely developed for non-e onomi dis iplines, most notably the physi al, mathemati al, and omputer s ien es. The s holarly literature an also pose substantial barriers for e onomists, both be ause of its mathemati al prerequisites and be ause its examples are unfamiliar to e onomists. Many available software pa kages, moreover, are designed to solve problems that are spe i to the physi al s ien es. This book addresses the diÆ ulties typi ally en ountered by e onomists attempting to learn and apply numeri al methods in several ways. First, this book emphasizes pra ti al numeri al methods, not mathemati al proofs, and fo uses on te hniques
PREFACE
xiv
that will be dire tly useful to e onomi analysts, not those that would be useful ex lusively to physi al s ientists. Se ond, the examples used in the book are drawn from a wide range of sub-spe ialties of e onomi s and nan e, both in ma ro- and mi roe onomi s, with parti ular emphasis on problems in nan ial, agri ultural, resour e and ma ro- e onomi s. And third, we in lude with the textbook an extensive library of omputer utilities and demonstration programs to provide interested resear hers with a starting point for their own omputer models. We make no attempt to be en y lopedi in our overage of numeri al methods or potential e onomi appli ations. We have instead hosen to develop only a relatively small number of te hniques that an be applied easily to a wide variety of e onomi problems. In some instan es, we have deviated from the standard treatments of numeri al methods in existing textbooks in order to present a simple onsistent framework that may be readily learned and applied by e onomists. In many ases we have ele ted not to over ertain numeri al te hniques when we regard them to be of limited bene t to e onomists, relative to their omplexity. Throughout the book, we try to explain our hoi es learly and to give referen es to more advan ed numeri al textbooks where appropriate. The book is divided into two major se tions. In the rst six hapters, we develop basi numeri al methods, in luding solving linear and nonlinear equation methods,
omplementarity methods, nite-dimensional optimization, numeri al integration and di erentiation, and fun tion approximation. In these hapters, we develop appre iation for basi numeri al te hniques by illustrating their appli ation to equilibrium and optimization models familiar to most e onomists. The last ve hapters of the book are devoted to methods for solving dynami sto hasti models in e onomi and nan e, in luding dynami programming, rational expe tations, and arbitrage pri ing models in dis rete and ontinuous time. The book is aimed at both graduate students, advan ed undergraduate students, and pra ti ing e onomists. We have attempted to write a book that an be used both as a lassroom text and for self-study. We have also attempted to make the various se tions reasonably self- ontained. For example, the se tions on dis rete time
ontinuous state models are largely independent from those on dis rete time dis rete state models. Although this results in some dupli ation of material, we felt that this would in rease the usefulness of the text by allowing readers to skip se tions. Although we have attempted to keep the mathemati al prerequisites for this book to a minimum, some mathemati al training and insight is ne essary to work with omputational e onomi models and numeri al te hniques. We assume that the reader is familiar with ideas and methods of linear algebra and al ulus. Appendix A provides an overview of the basi mathemati s used throughout the text. One barrier to the use of numeri al methods by e onomists is la k of a
ess to fun tioning omputer ode. This presents an apparent dilemma to us as textbook
PREFACE
xv
authors, given the variety of omputer languages available. On the one hand, it is useful to have working examples of ode in the book and to make the ode available to readers for immediate use. On the other hand, using a spe i language in the text ould obs ure the essen e of the numeri al routines for those unfamiliar with the
hosen language. We believe, however, that the latter on ern an be substantially mitigated by onforming to the syntax of a ve tor pro essing language. Ve tor pro essing languages are designed to fa ilitate numeri al analysis and their syntax is often simple enough that the language is transparent and easily learned and implemented. Due to its fa ility of use and its wide availability on university ampus omputing systems, we have hosen to illustrate algorithms in the book using and have provided an toolbox of utilities and demonstration programs to assist interested readers develop their own omputational e onomi appli ations. The CompE on toolbox an be obtained via the internet at the URL: http://?? All of the gures and tables in this book were generated by demonstration les provided with the toolbox (see List of Tables and List of Figures for le names). On e the toolbox is installed, these an be run by typing the appropriate le name at the
ommand line. For those not familiar with the programming language, a primer in provided in Appendix B. The text ontains many ode fragments, whi h, in some ases, have been simpli ed for expositional larity. This generally onsists of eliminating the expli it setting of optional parameters and not displaying ode that a tually generates tabular or graphi al output. The demonstration and fun tion les provided in the toolbox ontain fully fun tioning versions. In many ases the toolbox versions of fun tions des ribed in the text have optional parameters that an be altered by the user user the toolbox fun tion optset. The toolbox is des ribed in detail in ?? on page ??. Our ultimate goal in writing this book is to motivate a broad range of e onomists to use numeri al methods in their work by demonstrating the essential prin iples underlying omputational e onomi models a ross sub-dis iplines. It is our hope that this book will make a
essible a range of omputational tools that will enable e onomists to analyze e onomi and nan ial models that heretofore they were unable to solve within the on nes of traditional mathemati al e onomi analysis.
Matlab
Matlab
Matlab
Matlab
Matlab
Chapter 1 Introdu tion 1.1
Some Apparently Simple Questions
Consider the onstant elasti ity demand fun tion q
=p
0:2
:
This is a fun tion be ause, for ea h pri e p, there is an unique quantity demanded . Given a hand-held al ulator, any e onomist ould easily ompute the quantity demanded at any given pri e. An e onomist would also have little diÆ ulty omputing the pri e that lears the market of a given quantity. Flipping the demand expression about the equality sign and raising ea h side to the power of 5, the e onomist would derive a losed-form expression for the inverse demand fun tion q
p
=q
5
:
Again, using a al ulator any e onomist ould easily ompute the pri e that will exa tly lear the market of any given quantity. Suppose now that the e onomist is presented with a slightly di erent demand fun tion q
= 0:5 p
0:2
+ 0:5 p
0:5
;
one that is the sum a domesti demand term and an export demand term. Using standard al ulus, the e onomist ould easily verify that the demand fun tion is
ontinuous, di erentiable, and stri tly de reasing. The e onomist on e again ould easily ompute the quantity demanded at any pri e using a al ulator and ould easily and a
urately draw a graph of the demand fun tion. 1
CHAPTER 1.
INTRODUCTION
2
However, suppose that the e onomist is asked to nd the pri e that lears the market of, say, a quantity of 2 units. The question is well-posed. A asual inspe tion of the graph of the demand fun tion suggests that its inverse is well-de ned, ontinuous, and stri tly de reasing. A formal argument based on the Intermediate Value and Impli it Fun tion Theorems would prove that this is so. An unique market learing pri e learly exists. But what is the inverse demand fun tion? And what pri e lears the market? After onsiderable e ort, even the best trained e onomist will not nd an expli it answer using Algebra and Cal ulus. No losed-form expression for the inverse demand fun tion exists. The e onomist annot answer the apparently simple question of what the market learing pri e will be. Consider now a simple model of an agri ultural ommodity market. In this market, a reage supply de isions are made before the per-a re yield and harvest pri e are known. Planting de isions are based on the pri e expe ted at harvest: a
= 0:5 + 0:5E [p℄:
After the a reage is planted, a random yield y is realized, giving rise to a supply q
= ay~
that is entirely sold at a market learing pri e p
=3
2q:
Assume the random yield y is exogenous and distributed normally with a mean 1 and varian e 0.1. Most e onomists would have little diÆ ulty deriving the rational expe tations equilibrium of this market model. Substituting the rst expression into the se ond, and then the se ond into the third, the e onomist would write p
=3
2(0:5 + 0:5E [p℄)~y:
Taking expe tations on both sides [ ℄=3
E p
2(0:5 + 0:5E [p℄);
she would solve for the equilibrium expe ted pri e E [p℄ = 1. She would on lude that the equilibrium a reage is a = 1 and the equilibrium pri e distribution has a varian e of 0.4. Suppose now that the e onomist is asked to assess the impli ations of a proposed government pri e support program. Under this program, the government guarantees ea h produ er a minimum pri e, say 1. If the market pri e falls below this level, the
CHAPTER 1.
INTRODUCTION
3
government simply pays the produ er the di eren e per unit produ ed. The produ er thus re eives an e e tive pri e of max(p; 1) where p is the prevailing market pri e. The government program transforms the a reage supply relation to = 0:5 + 0:5E [max(p; 1)℄: Before pro eeding with a formal mathemati al analysis, the e onomist exer ises a little e onomi intuition. The government support, she reasons, will stimulate a reage supply, raising a reage planted. This will shift the equilibrium pri e distribution to the left, redu ing the expe ted market pri e below 1. Pri e would still o
asionally rise above 1, however, implying that the expe ted e e tive produ er pri e will ex eed 1. The di eren e between the expe ted e e tive produ er pri e and the expe ted market pri e represents a positive expe ted government subsidy. The e onomist now attempts to formally solve for the rational expe tations equilibrium of the revised market model. She performs the same substitutions as before and writes p = 3 2(0:5 + 0:5E [max(p; 1)℄)~y: As before, she takes expe tations on both sides E [ p℄ = 3 2(0:5 + 0:5E [max(p; 1)℄): a
In order to solve the expression for the expe ted pri e, the e onomist uses a fairly
ommon and apparently inno uous tri k: she inter hanges the max and E operators, repla ing E [max(p; 1)℄ with max(E [p℄; 1). The resulting expression is easily solved for E [p℄ = 1. This solution, however, asserts the expe ted market pri e and a reage planted remain un hanged by the introdu tion of the government pri e support poli y. This is in onsistent with the e onomist's intuition. The e onomist qui kly realizes her error. The expe tation operator annot be inter hanged with the maximization operator be ause the latter is a nonlinear fun tion. But if this operation is not valid, then what mathemati al operations would allow the e onomist to solve for the equilibrium expe ted pri e and a reage? Again, after onsiderable e ort, our e onomist is unable to nd an answer using Algebra and Cal ulus. No apparent losed-form solution exists for the model. The e onomist annot answer the apparently simple question of how the equilibrium a reage and expe ted market pri e will hange with the introdu tion of the government pri e support program. 1.2
An Alternative Analyti Framework
The two problems dis ussed in the pre eding se tion illustrate how even simple e onomi models annot always be solved using standard mathemati al te hniques.
CHAPTER 1.
4
INTRODUCTION
These problems, however, an easily be solved to a high degree of a
ura y using numeri al methods. Consider the inverse demand problem. An e onomist who knows some elementary numeri al methods and who an write basi
ode would have little diÆ ulty solving the problem. The e onomist would simply write the following elementary program:
Matlab
Matlab
p = 0.25; for i=1:100 deltap = (.5*p^-.2+.5*p^-.5-2)/(.1*p^-1.2 + .25*p^-1.5); p = p + deltap; if abs(deltap) < 1.e-8, break, end end disp(p);
He would then exe ute the program on a omputer and, in an instant, ompute the solution: the market learing pri e is 0.154. The e onomist has used Newton's root nding method, whi h is dis ussed in Se tion 3.3 on page 35. Consider now the rational expe tations ommodity market model with government intervention. The sour e of diÆ ulty in solving this problem is the need to evaluate the trun ated expe tation of a ontinuous distribution. An e onomist who knows some numeri al analysis and who knows how to write basi
ode, however, would have little diÆ ulty omputing the rational expe tation equilibrium of this model. The e onomist would repla e the original normal yield distribution with a dis rete distribution that has identi al lower moments, say one that assumes values y1 ; y2 ; : : : ; yn with probabilities w1 ; w2 ; : : : ; wn . After onstru ting the dis rete distribution approximant, whi h would require only a single all to the CompE on library routine qnwnorm, the e onomist would ode and exe ute the following elementary program:1
Matlab
Matlab
[y,w℄ = qnwnorm(10,1,0.1); a = 1; for it=1:100 aold = a; p = 3 - 2*a*y; f = w'*max(p,1); a = 0.5 + 0.5*f; if abs(a-aold) jz j. One problem that an arise is that y is so big that y 2 over ows. The largest real number representable on a ma hine an be found with the
ommand realmax (it is approximately 21024 10308 for most double pre ision environments). Although this kind of over ow may not happen often, it
ould have unfortunate onsequen es and ause problems that are hard to dete t. Even when y is not that big, if it is big relative to z , several problems an arise. The rst of these is easily dealt with. Suppose we evaluate p y+z y2 + z2 p when jy j is large enough so y + z is evaluated as y . This implies that y 2 + z 2 will be evaluated as jy j. When y < 0, the expression is evaluated as 2y , whi h is orre t to the most signi ant digit. When y > 0, however, we get 0, whi h may be very far from orre t. If the expression is evaluated in the order p y y2 + z2 + z
Matlab
the result will be z , whi h is mu h loser to the orre t answer. An even better approa h is to use p 2 (y; z ) = y 1 sign(y ) 1 + + ; where = z=y . Although this is algebrai ally equivalent, it has very di erent properties. First noti e that the han e of over ow is greatly redu ed be ause 1 1+ 2 2 and so the expression in ( ) is bounded on [; 4℄. If 1 + 2 is evaluated as 1 (i.e., if is less than the square root of ma hine pre ision), this expression yields 2y if y < 0 and y = z if y > 0. This is a lot better, but one further problem arises when y > 0 with jy j jz j. In this ase there is a an ellation due to the expression of the form z
=1
p
1 + 2
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
11
The obvious way of omputing this term will result in loss of pre ision as gets small. Another expression for z is p 2 2 2 1 p1 + 2 + : z = 2 1+ Although this is more ompli ated, it is a
urate regardless of the size of . As gets small, this expression will be approximately 2 =2. Thus, if is about the size of the square root of ma hine pre ision (2 26 on most double pre ision implementations), z would be omputed to ma hine pre ision with the se ond expression, but would be
omputed to be 0 using the rst, i.e., no signi ant digits would be orre t. Putting all of this together, a good approa h to omputing (y; z ) when jy j jz j uses 8 p 2 + ) if y < 0 < y (1 + 1+ p 2 2 2 1 + 1+ (y; z ) = ( p ) if y > 0 : y 2 1+2 where = z=y (reverse z and y if jy j < jz j). has a number of spe ial numeri al representations relevant to this dis ussion. We have already mentioned inf and -inf. These arise not only from over ow but from division by 0. The number realmax is the largest oating point number that an be represented; realmin is the smallest positive (normalized) number representable.1 In addition, eps represents the ma hine pre ision, de ned as the rst number greater than 1 that an be represented as a oating point number. Another way to say this is, for any 0 eps=2, 1 + will be evaluated as 1 (i.e., eps is equal to 21 b ).2 All three of these spe ial values are hardware spe i . In addition, oating point numbers may get set to NaN, whi h stands for not-anumber. This typi ally results from a mathemati ally unde ned operation, su h as inf-inf and 0/0. It does not result, however, from inf/0, 0/inf or inf*inf (these result in inf, 0 and inf). Any arithmeti operation involving a NaN results in a NaN. Roundo error is only one of the pitfalls in evaluating mathemati al expressions. In numeri al omputations, error is also introdu ed by the omputer's inherent inability to evaluate ertain mathemati al expressions exa tly. For all its power, a omputer
an only perform a limited set of operations in evaluating expressions. Essentially this list in ludes the four arithmeti operations of addition, subtra tion, multipli ation and division, as well as logi al operations of omparison. Other ommon fun tions,
Matlab
1 A denormalized number is one that non-zero, but has an exponent equal to its smallest possible value. 2 20 + 2 b = (2b + 1)2 b annot be represented and must be trun ated to (2b 1 )21 b = 1. 0 2 + 21 b = (2b 1 + 1)21 b , on the other hand, an be represented.
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
12
su h as exponential, logarithmi , and trigonometri fun tions annot be evaluated dire tly using omputer arithmeti . They an only be evaluated approximately using algorithms based on the four basi arithmeti operations. For the ommon fun tions very eÆ ient algorithms typi ally exist and these are sometimes \hardwired" into the omputer's pro essor or opro essor. An important area of numeri al analysis involves determining eÆ ient approximations that an be
omputed using basi arithmeti operations. For example, the exponential fun tion has the series representation exp(x) =
X1
n
!
x =n :
i=0
Obviously one annot ompute the in nite sum, but one ould ompute a nite number of these terms, with the hope that one will obtain suÆ ient a
ura y for the purpose at hand. The result, however, will always be inexa t.3 For nonstandard problems, we must often rely on our own abilities as numeri al analysts (or know when to seek help). Being aware of some of the pitfalls should help us avoid them. 2.2
Data Storage
basi data type is the matrix, with a s alar just a 1 1 matrix and an n-ve tor an n 1 or 1 n matrix. Matlab keeps tra k of matrix size by storing row and olumn information about the matrix along with the values of the matrix itself. This is a signi ant advantage over writing in low level languages like Fortran or C be ause it relieves one of the ne essity of keeping tra k of array size and memory allo ation. When one wants to represent an m n matrix of numbers in a omputer there are a number of ways to do this. The most simple way is to store all the elements sequentially in memory, starting with the one indexed (1,1) and working down su
essive olumns or a ross su
essive rows until the (m; n)th element is stored. Di erent languages make di erent hoi es about how to store a matrix. Fortran stores matri es in olumn order, whereas C stores in row order. Matlab, although written in C, stores in olumn order, thereby onforming with the Fortran standard. Many matri es en ountered in pra ti e are sparse, meaning that they onsist mostly of zero entries. Clearly, it is a waste of memory to store all of the zeros, and it is time onsuming to pro ess the zeros in arithmeti matrix operations. Matlab supports a sparse matrix data type, whi h eÆ iently keeps tra k of only the Matlab's
In idently, the Taylor series representation of the exponential fun tion does not result in an eÆ ient omputational algorithm. 3
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
13
non-zero elements of the original matrix and their lo ations. In this storage s heme, the non-zero entries and the row indi es are stored in two ve tors of the same size. A separate ve tor is used to keep tra k of where the rst element in ea h olumn is lo ated. If one wants to a
ess element (i; j ),
he ks the j th element of the
olumn indi ator ve tor to nd where the j th olumn starts and then sear hes the row indi ator ve tor for the ith element (if one is not found then the element must be zero). Although sparse matrix representations are useful, their use in urs a ost. To a
ess element (i; j ) of a full matrix, one simply goes to storage lo ation (i 1)m + j . A
essing an element in a sparse matrix involves a sear h over row indi es and hen e
an take longer. This additional overhead an add up signi antly and a tually slow down a omputational pro edure. A further onsideration in using sparse matri es on erns memory allo ation. If a pro edure repeatedly alters the ontents of a sparse matrix, the memory needed to store the matrix may hange, even if its dimension does not. This means that more memory may be needed ea h time the number of non-zero elements in reases. This memory allo ation is both time onsuming and may eventually exhaust omputer memory. The de ision whether to use a sparse or full matrix representation depends on a balan e between a number of fa tors. Clearly for very sparse matri es (less than 10% non-zero) one is better o using sparse matri es and anything over 67% non-zeros one is better o with full matri es (whi h a tually require less storage spa e at that point). In between, some experimentation may be required to determine whi h is better for a given appli ation. Fortunately, for many appli ations, users don't even need to be aware of whether matri es are stored in sparse or full form. is designed so most fun tions work with any mix of sparse or full representations. Furthermore, sparsity propagates in a reasonably intelligent fashion. For example, a sparse times a full matrix or a spare plus a full matrix results in a full matrix, but if a sparse and a full matrix are multiplied element-by-element (using the \.*" operator) a sparse matrix results.
Matlab
Matlab
2.3
Linear Equations and the L-U Fa torization
The linear equation is the most elementary problem that arises in omputational e onomi analysis. In a linear equation, an n n matrix A and an n-ve tor b are given, and one must ompute the n-ve tor x that satis es Ax
= b:
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
14
Linear equations arise, dire tly or indire tly, in most omputational e onomi appli ations. For example, a linear equation may be solved when omputing the steadystate distribution of a dis rete-state sto hasti e onomi pro ess or when omputing the equilibrium pri es and quantities of a multi ommodity market model with linear demand and supply fun tions. Linear equations also arise as elementary tasks in solution pro edures designed to solve more ompli ated nonlinear e onomi models. For example, a nonlinear partial equilibrium market model may be solved using Newton's method, whi h involves solving a sequen e of linear equations. And the Euler fun tional equation of a rational expe tations model may be solved using a ollo ation method, whi h yields a nonlinear equation that in turn is solved as a sequen e of linear equations. Various pra ti al issues arise when solving a linear equation numeri ally. Digital
omputers are apable of representing arbitrary real numbers with only limited pre ision. Numeri al arithmeti operations, su h as omputer addition and multipli ation, produ e rounding errors that may, or may not, be negligible. Unless the rounding errors are ontrolled in some way, the errors an a
umulate, rendering a omputed solution that may be far from orre t. Speed and storage requirements are also important onsiderations in the design of a linear equation solution algorithm. In some appli ations, su h as the sto hasti simulation of a rational expe tations model, linear equations may have to be solved millions of times. And in other appli ations, su h as
omputing option pri es using nite di eren e methods, linear equations with a very large number of variables and equations may be en ountered. Over the years, numeri al analysts have studied linear equations extensively and have developed algorithms for solving them qui kly, a
urately, and with a minimum of omputer storage. In most applied work, one an typi ally rely on Gaussian elimination, whi h may be implemented in various di erent forms depending on the stru ture of the linear equation. Iterative methods o er an alternative to Gaussian elimination and are espe ially eÆ ient if the A matrix is large and onsists mostly of zero entries. Some linear equations Ax = b are relatively easy to solve. For example, if A is a lower triangular matrix,
2 66 A = 6 64
a21
a22
0 0
a31
a32
a33
:::
0 0 0
an1
an2
an3
:::
ann
a11
0
::: :::
3 7 7 7 ; 7 5
then the elements of x an be omputed re ursively using forward-substitution: x1
=
b1 =a11
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
x3
= (b2 = (b3
xn
= (bn
x2
.. .
15
)
a21 x1 =a22
)
a31 x1
a32 x2 =a33
an1 x1
an2 x2
:::
ann
1 xn 1 )=ann :
This learly works only if all of the diagonal elements are non-zero (i.e., if the matrix is nonsingular). The algorithm an be written more ompa tly using summation notation as
X1
!
i
xi
=
bi
aij xj
=aii
j =1
In the ve tor pro essing language
8
i:
Matlab, this may be implemented as follows:
for i=1:length(b) x(i)=(b(i)-A(i,1:i-1)*x(1:i-1))/A(i,i); end
If A is an upper triangular matrix, then the elements of x an be omputed re ursively using ba kward-substitution. Most linear equations en ountered in pra ti e, however, do not have a triangular A matrix. In su h ases, the linear equation is often best solved using the L-U fa torization algorithm. The L-U algorithm is designed to de ompose the A matrix into the produ t of lower and upper triangular matri es, allowing the linear equation to be solved using a ombination of ba kward and forward substitution. The L-U algorithm involves two phases. In the fa torization phase, Gaussian elimination is used to fa tor the matrix A into the produ t A
= LU
of a row-permuted lower triangular matrix L and an upper triangular matrix U . A row-permuted lower triangular matrix is simply a lower triangular matrix that has had its rows rearranged. Any nonsingular square matrix an be de omposed in this way. In the solution phase of the L-U algorithm, the fa tored linear equation Ax
= (LU )x = L(U x) = b
is solved by rst solving Ly
=b
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
16
for y using forward substitution, a
ounting for row permutations, and then solving Ux
=y
for x using ba kward substitution. Consider, for example, the linear equation Ax = b where
2 3 4 3 A =
3
2 2 0
3 1 0
3 5 and
2 3 10 8 5: b = 4 3
The matrix A an be de omposed into the produ t
2 1 4 1 L = 1
0 0 1
3 0 1 5 and 0
2 3 4 0 U = 0
2 2 0
A
= LU where 3 3 2
3 5:
The matrix L is row-permuted lower triangular; by inter hanging the se ond and third rows, a lower diagonal matrix results. The matrix U is upper triangular. Solving Ly = b for y using forward substitution involves rst solving for y1 , then for y3 , and nally for y2 . Given the solution y = [10 7 2℄>, the linear equation U x = y an the be solved using ba kward substitution, yielding the solution of the original linear equation, x = [ 1 2 1℄>. The L-U fa torization algorithm is faster than other linear equation solution methods that are typi ally presented in elementary linear algebra ourses. For large n, it takes approximately n3 =3+ n2 long operations (multipli ations and divisions) to solve an n n linear equation using L-U fa torization. Expli itly omputing the inverse of A and then omputing A 1 b requires approximately n3 + n2 long operations. Solving the linear equation using Cramer's rule requires approximately (n + 1)! long operations. To solve a 10 10 linear equation, for example, L-U fa torization requires exa tly 430 long operations, whereas matrix inversion and multipli ation requires exa tly 1100 long operations and Cramer's rule requires nearly 40 million long operations. Linear equations arise so frequently in numeri al analysis that most numeri al subroutine pa kages and software programs in lude either a basi subroutine or an intrinsi fun tion for solving a linear equation using L-U fa torization. In , the solution to the linear equation Ax = b is returned by the statement x = A n b. The \n", or \ba kslash", operator is designed to solve the linear equation using L-U fa torization, unless a spe ial stru ture for A is dete ted, in whi h ase may impli itly use another, more eÆ ient method. In parti ular, if dete ts that A is triangular or permuted triangular, it will dispense with L-U fa torization and solve the linear equation dire tly using forward or ba kward substitution. also uses spe ial algorithms when the A matrix is positive de nite (see Se tion 2.7 on page 22).
Matlab
Matlab Matlab Matlab
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
17
Although L-U fa torization is the best general method for solving a linear equation, situations an arise in whi h alternative methods may be preferable. For example, in many omputational e onomi appli ations, one must solve a series of linear equations, all having the same A matrix, but di erent b ve tors, b1 ; b2 ; : : : ; bm . In this situation, it is often omputationally more eÆ ient to dire tly ompute and store the inverse of A rst and then ompute the solutions x = A 1 bj by performing only dire t matrix-ve tor multipli ations. Whether expli itly omputing the inverse is faster than L-U fa torization depends on the size of the linear equation system n and the number of times, m, an equation system is to be solved. Computing x = A n bj a total of m times involves mn3 =3 + mn2 long operations. Computing A 1 on e and then omputing A 1 bj a total of m times requires n3 + mn2 long operations. Thus expli it omputation of the inverse should be faster than L-U fa torization whenever the number of equations to be solved m is greater than three or four. The a tual breakeven point will vary a ross numeri al analysis pa kages, depending on the omputational idiosyn rasies and overhead osts of the L-U fa torization and inverse routines implemented in the pa kage. 2.4
Gaussian Elimination
The L-U fa tors of a matrix A are omputed using Gaussian elimination. Gaussian elimination is based on two elementary row operations: subtra ting a onstant multiple of one row of a linear equation from another row, and inter hanging two rows of a linear equation. Either operation may be performed on a linear equation without altering its solution. The Gaussian elimination algorithm begins with matri es L and U initialized as L = I and U = A, where I is the identity matrix. The algorithm then uses elementary row operations to transform U into an upper triangular matrix, while preserving the permuted lower diagonality of L and the fa torization A = LU : Consider the matrix
2 2 66 4 A = 4 2 2
0 2 2 2
1 1 2 7
3
2 47 7: 35 3
The rst stage of Gaussian elimination is designed to nullify the subdiagonal entries of the rst olumn of the U matrix. The U matrix is updated by subtra ting 2 times the rst row from the se ond, subtra ting 1 times the rst row from the third, and subtra ting 1 times the rst row from the fourth. The L matrix, whi h initially equals
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
18
the identity, is updated by storing the multipliers 2, 1, and 1 as the subdiagonal entries of its rst olumn. These operations yield updated L and U matri es:
2 1 66 2 L = 4 1 1
0 1 0 0
0 0 1 0
0 0 0 1
3 77 5
2 2 6 0 U = 6 4 0 0
0 2 2 2
1 1 1 6
2 0 1 1
3 7 7 5:
After the rst stage of Gaussian elimination, A = LU and L is lower triangular, but U is not yet upper triangular. The se ond stage Gaussian elimination is designed to nullify the subdiagonal entries of the se ond olumn of the U matrix. The U matrix is updated by subtra ting 1 times se ond row from the third and subtra ting 1 times the se ond row from the fourth. The L matrix is updated by storing the multipliers 1 and 1 as the subdiagonal elements of its se ond olumn. These operations yield updated L and U matri es:
2 1 66 2 L = 4 1 1
0 1 1 1
0 0 1 0
0 0 0 1
3 77 5
2 2 6 0 U = 6 4 0 0
0 2 0 0
1 1 0 5
2 0 1 1
3 7 7 5:
After the se ond stage of Gaussian elimination, A = LU and L is lower triangular, but U still is not upper triangular. In the third stage of Gaussian elimination, one en ounters an apparent problem. The third diagonal element of the matrix U is zero, making it impossible to nullify the subdiagonal entry as before. This diÆ ulty is easily remedied, however, by inter hanging the third and fourth rows of U . The L matrix is updated by inter hanging the previously omputed multipliers residing in the third and fourth olumns. These operations yield updated L and U matri es:
2 1 66 2 L = 4 1 1
0 1 1 1
0 0 0 1
0 0 1 0
3 7 7 5
2 2 6 0 U = 6 4 0 0
0 2 0 0
1 1 5 0
2 0 1 1
3 7 7 5:
The Gaussian elimination algorithm terminates with a permuted lower triangular matrix L and an upper triangular matrix U whose produ t is the matrix A. In theory, Gaussian elimination will ompute the L-U fa tors of any matrix A, provided A is invertible. If A is not invertible, Gaussian elimination will dete t this by en ountering a zero diagonal element in the U matrix that annot be repla ed with a nonzero element below it.
CHAPTER 2.
2.5
COMPUTER BASICS AND LINEAR EQUATIONS
19
Rounding Error
In pra ti e, Gaussian elimination performed on a omputer an sometimes render ina
urate solutions due to rounding errors. The e e ts of rounding errors, however,
an often be ontrolled by pivoting. Consider the linear equation
M
1
1
1 1
x1 x2
=
1 2
:
where M is a large positive number. To solve this equation via Gaussian elimination, a single row operation is required: subtra ting M times the rst row from the se ond row. In prin iple, this operation yields the L-U fa torization
M
1
1
1 1
=
1 M
0 1
M
0
1
1 M + 1
:
In theory, applying forward and ba kward substitution yields the solution x1 = M=(M + 1) and x2 = (M + 2)=(M + 1), whi h are both very nearly one. In pra ti e, however, Gaussian elimination may yield a very di erent result. In performing Gaussian elimination, one en ounters an operation that annot be arried out pre isely on a omputer, and whi h should be avoided in omputational work: adding or subtra ting values of vastly di erent magnitudes. On a omputer, it is not meaningful to add or subtra t two values whose magnitude di er by more than the number of signi ant digits that the omputer an represent. If one attempts su h an operation, the smaller value is e e tively treated as zero. For example, the sum of 0:1 and 0:0001 may be 0:1001, but on a hypotheti al ma hine with three digit pre ision the result of the sum is rounded to 0:1 before it is stored. In the linear equation above, adding 1 or 2 to a suÆ iently large M on a omputer simply returns the value M . Thus, in the rst step of the ba kward substitution, x2 is omputed, not as (M + 2)=(M + 1), but rather as M=M , whi h is exa tly one. Then, in the se ond step of ba kward substitution, x1 = M (1 x2 ) is omputed to be zero. Rounding error thus produ es omputed solution for x1 that has a relative error of nearly 100 per ent. Fortunately, there is a partial remedy for the e e ts of rounding error in Gaussian elimination. Rounding error arises in the example above be ause the diagonal element 1 M is very small. Inter hanging the two rows at the outset of Gaussian elimination does not alter the theoreti al solution to the linear equation, but allows one to perform Gaussian elimination with a diagonal element of larger magnitude.
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
20
Consider the equivalent linear equation system after the rows have been inter hanged: 1 1 x1 2 = : 1 M 1 x2 1 After inter hanging the rows, the new A matrix may be fa tored as 1 1 1 0 1 1 = : M 1 1 M 1 1 0 M 1+1 Ba kward and forward substitution yield the theoreti al results x1 = 1 M 1 and x2 = M 1 + 1 + M 1 (1 M 1 ). In evaluating these expressions on the omputer, one again en ounters rounding error. Here, x2 is numeri ally omputed to be exa tly one as before. However, x1 is also omputed to be exa tly one. The omputed solution, though not exa tly orre t, is orre t to the pre ision available on the omputer, and is ertainly more a
urate than the one obtained without inter hanging the rows. Inter hanging rows during Gaussian elimination in order to make the magnitude of diagonal element as large as possible is alled pivoting. Pivoting substantially enhan es the reliability and the a
ura y of a Gaussian elimination routine. For this reason, all good Gaussian elimination routines designed to perform L-U fa torization, in luding the ones implemented in , employ some form of pivoting.
Matlab
2.6
Ill Conditioning
Pivoting annot ure all the problems aused by rounding error. Some linear equations are inherently diÆ ult to solve a
urately on a omputer, despite pivoting. This o
urs when the A matrix is stru tured in su h a way that a small perturbation Æb in the data ve tor b indu es a large hange Æx in the solution ve tor x. In su h ases the linear equation or, more generally, the A matrix are said to be ill- onditioned. One measure of ill- onditioning in a linear equation Ax = b is the \elasti ity" of the solution ve tor x with respe t to the data ve tor b
= sup jjÆbjj>0
jjÆxjj=jjxjj : jjÆbjj=jjbjj
The elasti ity gives the maximum per entage hange in the size of the solution ve tor x indu ed by a one per ent hange the size of the data ve tor b. If the elasti ity is large, then small errors in the omputer representation of the data ve tor b an produ e large errors in the omputed solution ve tor x. Equivalently, the omputed solution x will have far fewer signi ant digits than the data ve tor b.
CHAPTER 2.
21
COMPUTER BASICS AND LINEAR EQUATIONS
The elasti ity of the solution is expensive to ompute and thus is virtually never
omputed in pra ti e. In pra ti e, the elasti ity is estimated using the ondition number of the matrix A, whi h for invertible A is de ned by jjAjj jjA
1
jj:
The ondition number of A is the least upper bound of the elasti ity. The bound is tight in that for some data ve tor b, the ondition number equals the elasti ity. The
ondition number is always greater than or equal to one. Numeri al analysts often use the rough rule of thumb that for ea h power of 10 in the ondition number, one signi ant digit is lost in the omputed solution ve tor x. Thus, if A has a ondition number of 1000, the omputed solution ve tor x will have about three fewer signi ant digits than the data ve tor b. Consider the linear equation Ax = b where Aij = in j and bi = (in 1)=(i 1). In theory, the solution x to this linear equation is a ve tor ontaining all ones for any n. In pra ti e, however, if one solves the linear equation numeri ally using 's \n" operator one an get quite di erent results. Below is a table that gives the supremum norm approximation error in the omputed value of x and the ondition number of the A matrix for di erent n: Approximation Condition n Error Number
Matlab
5 10 15 20 25
2.5e-013 5.2e-007 1.1e+002 9.6e+010 8.2e+019
2.6e+004 2.1e+012 2.6e+021 1.8e+031 4.2e+040
In this example, the omputed answers are a
urate to seven de imals up to n = 10. The a
ura y, however, deteriorates rapidly after that. In this example, the matrix A is a member of the a lass of notoriously ill- onditioned matri es alled the Vandermonde matri es, whi h we will en ounter again in Chapter 6. Ill- onditioning ultimately an be as ribed to the limited pre ision of omputer arithmeti . The e e ts of ill- onditioning an often be mitigated by performing omputer arithmeti using the highest pre ision available on the omputer. The best way to handle ill- onditioning, however, is to avoid it altogether. This is often possible when the linear equation problem is as an elementary task in a more ompli ated solution pro edure, su h as solving a nonlinear equation or approximating a fun tion with a polynomial. In su h ases one an sometimes reformulate the problem or alter the solution strategy to avoid the ill- onditioned linear equation. We will see several examples of this avoidan e strategy later in the book.
CHAPTER 2.
2.7
COMPUTER BASICS AND LINEAR EQUATIONS
22
Spe ial Linear Equations
Gaussian elimination an be a
elerated for matri es possessing ertain spe ial stru tures. Two su h lasses arising frequently in omputational e onomi analysis are symmetri positive de nite matri es and sparse matri es. Linear equations Ax = b in whi h A is a symmetri positive de nite arise frequently in least-squares urve- tting and optimization appli ations. A spe ial form of Gaussian elimination, the Cholesky fa torization algorithm, may be applied to su h linear equations. Cholesky fa torization requires only half as many operations as general Gaussian elimination and has the added advantage that it is less vulnerable to rounding error and does not require pivoting. The essential idea underlying Cholesky fa torization is that any symmetri positive de nite matrix A an be uniquely expressed as the produ t > A = U U of an upper triangular matrix U and its transpose. The matrix U is alled the Cholesky fa tor or square root of A. Given the Cholesky fa tor of A, the linear equation > > Ax = U U x = U (U x) = b may be solved eÆ iently by using forward substitution to solve > U y = b and then using ba kward substitution to solve Ux
= y:
Matlab
The \n" operator will automati ally employ Cholesky fa torization, rather than L-U fa torization, to solve the linear equation if it dete ts that A is symmetri positive de nite. Another situation that often arises in omputational pra ti e are linear equations Ax = b in whi h the A matrix is sparse, that is, it onsists largely of zero entries. For example, in solving di erential equations, one often en ounters tridiagonal matri es, whi h are zero ex ept on or near the diagonal. When the A matrix is sparse, the onventional Gaussian elimination algorithm onsists largely of meaningless, but ostly, operations involving either multipli ation or addition with zero. The Gaussian elimination algorithm in these instan es an often be dramati ally in reased by avoiding these useless operations. has spe ial routines for eÆ iently storing sparse matri es and operating with them. In parti ular, the
ommand S=sparse(A) reates a version S of
Matlab
Matlab
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
23
the matrix A stored in a sparse matrix format, in whi h only the nonzero elements of A and their indi es are expli itly stored. Sparse matrix storage requires only a fra tion of the spa e required to store A in standard form if A is sparse. Also, the operator \n" is designed to re ognize whether a sparse matrix is involved in the operation and adapts the Gaussian elimination algorithm to exploit this property. In parti ular, both x = S n b and x = A n b will ompute the answer to Ax = b. However, the former expression will be exe uted substantially faster by avoiding operations with zeros. 2.8
Iterative Methods
Algorithms based on Gaussian elimination are alled exa t or, more properly, dire t methods be ause they would generate exa t solutions for the linear equation Ax = b after a nite number of operations, if not for rounding error. Su h methods are ideal for moderately-sized linear equations, but may be impra ti al for large ones. Other methods, alled iterative methods an often be used to solve large linear equations more eÆ iently if the A matrix is sparse, that is, if A is omposed mostly of zero entries. Iterative methods are designed to generate a sequen e of in reasingly a
urate approximations to the solution of a linear equation, but generally do not yield an exa t solution after a pres ribed number of steps, even in theory. The most widely-used iterative methods for solving a linear equation Ax = b are developed by hoosing an easily invertible matrix Q and writing the linear equation in the equivalent form Qx
= b + (Q
)
A x
or =Q
x
1
b
+ (I
Q
1
)
A x:
This form of the linear equation suggests the iteration rule (k +1)
x
Q
1
b
+ (I
Q
1
)
(k )
A x
;
whi h, if onvergent, must onverge to a solution of the linear equation. Ideally, the so- alled splitting matrix Q will satisfy two riteria. First, Q 1 b and 1 Q A should be relatively easy to ompute. This is true if Q is either diagonal or triangular. Se ond, the iterates should onverge qui kly to the true solution of the linear equation. If
jj
I
Q
1
jj
A
ii
Xj n
Aij j
8i:
i=1
6
i=j
Diagonally dominant matri es arise naturally in many omputational e onomi appli ations, in luding the solution of di erential equations and the approximation of fun tions using ubi splines, both of whi h will be dis ussed in later se tions. The following s ript solves the linear equation Ax = b using GaussJa obi iteration:
Matlab
d = diag(A); for it=1:maxit dx = (b-A*x)./d; x = x+dx; if norm(dx) and Z and D are real-valued. Another fa torization is the QR de omposition, whi h nds a representation A = QR, where Q is orthonormal and R is triangular. This fa torization is not unique; there are a number of algorithms that produ e di erent values of Q and R, in luding Householder and Givens transformations. A need not be square to apply the QR de omposition. Finally, we mention the singular-value de omposition (SVD), whi h nds U , D and V , with U and V orthonormal and D diagonal, that satis es A = U DV >. The diagonal elements of D are known as the singular values of A and are nonnegative and generally order highest to lowest. In the ase of a square, symmetri A, this is identi al to the eigenvalue/eigenve tor de omposition. The SVD an be used with non-square matri es. The SVD is the method of hoi e for determining matrix ondition and rank. The
ondition number is the ratio of the highest to the lowest singular value; the rank is the number of non-zero singular values. In pra ti e, one would treat a singular value Djj as zero if Djj < maxi (Dii ), for some spe i ed value of ( sets equal to the value of the ma hine pre ision eps times the maximum of the number of rows
Matlab
CHAPTER 2.
COMPUTER BASICS AND LINEAR EQUATIONS
and olumns of
A).
29
We have only tou hed on iterative methods. These are mainly useful when solving large sparse systems that annot be stored dire tly. See Golub and Ortega, Se tion 9.3, for further details and referen es. Numerous software libraries that perform basi linear algebra omputations are available, in luding LINPACK, LAPACK, IMSL and NAG. See Notes on Web Resour es (page 497).
Chapter 3 Nonlinear Equations and Complementarity Problems One of the most basi numeri al operations en ountered in omputational e onomi s is to nd the solution of a system of nonlinear equations. Nonlinear equations generally arise in one of two forms. In the nonlinear root nding problem, a fun tion f mapping f (x) = 0:
x
This imposes only a one-sided bound on x at 0. Doubly bounded problems are often alled mixed omplementarity problems (MCPs) and are typi ally formulated as solving max(min(f (x); x
)
b
)
x
a ;x
)=0
rather than min(max(f (x); a
x ;b
) = 0;
as we have done. If standard software for MCPs is used, the sign of f should be reversed. A number of approa hes exist for solving CPs other than reformulation as a root nding problem. A well-studied and robust algorithm based on su
essive linearization is in orporated in the PATH algorithm des ribed by Ferris et al., and Ferris and Munson. The linear omplementarity problem (LCP) has re eived onsiderable attention and forms the underpinning for methods based on su
essive linearization. Lemke's method is perhaps the most widely used and robust LCP solver. It is des ribed in the standard works ited above. Re ent work on LCPs in ludes Kremers and Talman. We have not dis ussed homotropy methods for solving nonlinear equations, but these may be desirable to explore, espe ially if good initial values are hard to guess. Judd, hapter 5, ontains a good introdu tion, with e onomi appli ations and referen es for further study.
Chapter 4 Finite-Dimensional Optimization In this hapter we examine methods for optimizing a fun tion with respe t to a nite number of variables. In the nite-dimensional optimization problem, one is given a real-valued fun tion f de ned on X f (x) for all x 6= x in N \ X . If x is a lo al maximum of f that resides in the interior of X and f is twi e di erentiable there, then f 0 (x ) = 0 and f 00 (x ) is negative semide nite. Conversely, if f 0 (x ) = 0 and f 00 (x) is negative semide nite in an -neighborhood of x ontained in X , then x is a lo al maximum; if, additionally, f 00 (x ) is negative de nite, then x is a stri t lo al maximum. By the Lo al-Global Theorem, if f is
on ave, X is onvex, and x is a lo al maximum of f , then x is a global maximum of f on X .2 4.1
Derivative-Free Methods
As was the ase with univariate root nding, optimization algorithms exist that will pla e progressively smaller bra kets around a lo al maximum of a univariate fun tion. Su h methods are relatively slow, but do not require the evaluation of fun tion derivatives and are guaranteed to nd a lo al optimum to a pres ribed toleran e in a known number of steps. The most widely-used derivative-free method is the golden sear h method. Suppose we wish to nd a lo al maximum of a ontinuous univariate fun tion f (x) on the interval [a; b℄. Pi k any two numbers in the interior of the interval, say x1 and x2 with x1 < x2 . Evaluate the fun tion and repla e the original interval with [a; x2 ℄ if f (x1 ) > f (x2 ) or with [x1 ; b℄ if f (x2 ) f (x1 ). A lo al maximum must be ontained in the new interval be ause the endpoints of the new interval are lower than a point on the interval's interior (or the lo al maximum is at one of the original endpoints). 2 These
results also hold for minimization, provided one hanges on avity of f to onvexity and negative (semi) de niteness of f 00 to positive (semi) de niteness.
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
65
We an repeat this pro edure, produ ing a sequen e of progressively smaller intervals that are guaranteed to ontain a lo al maximum, until the length of the interval is shorter than some desired toleran e level. A key issue is how to pi k the interior evaluation points. Two simple riteria lead to the most widely-used strategy. First, the length of the new interval should be independent of whether the upper or lower bound is repla ed. Se ond, on su
essive iterations, one should be able to reuse an interior point from the previous iteration so that only one new fun tion evaluation is performed per iteration. These onditions are uniquely satis ed by sele ting xi = a + i (b a), where
p
5
p
5 1 : 2 2 The value 2 is known as the golden ratio, a number dear to the hearts of Greek philosophers and Renaissan e artists. The following s ript omputes a lo al maximum of a univariate fun tion f on an interval [a; b℄ using the golden sear h method. The s ript assumes that the user has written a routine f that evaluates the fun tion at an arbitrary point. The s ript also assumes that the user has spe i ed interval endpoints a and b and a onvergen e toleran e tol:
1 =
3
and 2 =
Matlab Matlab
alpha1 = (3-sqrt(5))/2; alpha2 = (sqrt(5)-1)/2; x1 = a+alpha1*(b-a); f1 = f(x1); x2 = a+alpha2*(b-a); f2 = f(x2); d = alpha1*alpha2*(b-a); while d>tol d = d*alpha2; if f2f1 x = x2; else x = x1; end
CHAPTER 4.
66
FINITE-DIMENSIONAL OPTIMIZATION
Matlab
The toolbox a
ompanying the textbook in ludes a fun tion golden that
omputes a lo al maximum of a univariate fun tion using the golden sear h method. To apply this fun tion, one rst de nes a fun tion that returns the value of the obje tive fun tion at an arbitrary point. One then passes the name of this fun tion, along with the lower and upper bounds for the sear h interval, to golden. For example, to ompute a lo al maximum of f (x) = x os(x2 ) 1 on the interval [0; 3℄, one exe utes the following s ript:
Matlab
Matlab
f = inline('x* os(x^2)-1'); x = golden(f,0,3)
Exe ution of this s ript yields the result x = 0:8083. As an be seen in Figure 4.1, this point is a lo al maximum, but not a global maximum in [0; 3℄. The golden sear h method is guaranteed to nd the global maximum when the fun tion is on ave. However, as the present example makes lear, this need not be true when the optimand is not on ave. Maximization of x cos(x2) via golden search 3
2
1
0
−1
−2
−3
0
0.5
1
1.5
2
2.5
3
Figure 4.1 Another widely-used derivative-free optimization method for multivariate fun tions is the Nelder-Mead algorithm. The algorithm begins by evaluating the obje tive fun tion at n + 1 points. These n + 1 points form a so- alled simplex in the
CHAPTER 4.
67
FINITE-DIMENSIONAL OPTIMIZATION
n-dimensional de ision spa e. This is most easily visualized when x is 2-dimensional,
in whi h ase a simplex is a triangle. At ea h iteration, the algorithm determines the point on the simplex with the lowest fun tion value and alters that point by re e ting it through the opposite fa e of the simplex. This is illustrated in Figure 4.2 (Re e tion), where the original simplex is lightly shaded and the heavily shaded simplex is the simplex arising from re e ting point A. If the re e tion su
eeds in nding a new point that is higher than all the others on the simplex, the algorithm he ks to see if it is better to expand the simplex further in this dire tion, as shown in Figure 4.2 (Expansion). On the other hand, if the re e tion strategy fails to produ e a point that is at least as good as the se ond worst point, the algorithm ontra ts the simplex by halving the distan e between the original point and its opposite fa e, as in Figure 4.2 (Contra tion). Finally, if this new point is not better than the se ond worst point, the algorithm shrinks the entire simplex toward the best point, point B in Figure 4.2 (Shrinkage). Simplex Transformations in the Nelder−Mead Algorithm Reflection
Expansion
B
B
A
A
C
Contraction
Shrinkage
B
A
C
B
A
C
C
Figure 4.2 One thing that may not be lear from the des ription of the algorithm is how to
ompute a re e tion. For a point xi , the re e tion is equal to xi + 2di where xi + di is the point in the enter of the opposite fa e of the simplex from xi . That entral point
an be found by averaging the n other point of the simplex. Denoting the re e tion
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
by ri , this means that ri
= xi + 2
1X n
! xj
j =i 6
xi
2X
n
=
n
xj
j =1
1 1+ 2
68
xi :
An expansion an then be omputed as 1:5ri
0:5xi
and a ontra tion as 0:25ri + 0:75xi : The Nelder-Mead algorithm is simple, but slow and unreliable. However, if a problem involves only a single optimization or ostly fun tion and derivative evaluations, the Nelder-Mead algorithm is worth trying. In many problems an optimization problem that is embedded in a larger problem must be solved repeatedly, with the fun tion parameters perturbed slightly with ea h iteration. For su h problems, whi h are ommon is dynami models, one generally will want to use a method that moves more qui kly and reliably to the optimum, given a good starting point. The toolbox a
ompanying the textbook in ludes a fun tion neldermead that maximizes a multivariate fun tion using the Nelder-Meade method. To apply this fun tion, one must rst de ne a fun tion f that returns the value of the obje tive fun tions at an arbitrary point and then pass the name of this fun tion along with a starting value x to neldermeade. Consider, for example, maximizing the \banana" fun tion f (x) = 100(x2 x21 )2 (1 x1 )2 , so- alled be ause its ontours resemble bananas. Assuming a starting value of (1; 0), the Nelder-Meade pro edure may be exe uted in as follows:
Matlab
Matlab
Matlab
f = inline('-100*(x(2)-x(1)^2)^2-(1-x(1))^2'); x = neldmead(f,[1; 0℄);
Exe ution of this s ript yields the result x = (1; 1), whi h indeed is the global maximum of the fun tion. The ontours of the banana fun tion and the path followed by the Nelder-Meade iterates are illustrated in Figure 4.3. 4.2
Newton-Raphson Method
The Newton-Raphson method for maximizing an obje tive fun tion uses su
essive quadrati approximations to the obje tive in the hope that the maxima of the approximants will onverge to the maximum of the obje tive. The Newton-Raphson method is intimately related to the Newton method for solving root nding problems.
CHAPTER 4.
69
FINITE-DIMENSIONAL OPTIMIZATION
Nelder−Mead Maximization of Banana Function 1.2
1
0.8
x
2
0.6
0.4
0.2
0
−0.2 −0.2
0
0.2
0.4
0.6
x
0.8
1
1.2
1
Figure 4.3 Indeed, the Newton-Raphson method is identi al to applying Newton's method to
ompute the root of the gradient of the obje tive fun tion. More generally, the Newton-Raphson method begins with the analyst supplying a guess x(0) for the maximum of f . Given x(k) , the subsequent iterate x(k+1) is omputed by maximizing the se ond order Taylor approximation to f about x(k) : (k ) f (x) f x + f 0 x(k) x x(k) + 1 x x(k) >f 00 x(k) x x(k) : 2
Solving the rst order ondition 0 x(k) + f 00 x(k) x x(k) = 0; f yields the iteration rule (k +1)
x
(k )
x
00 f
(k )
x
1
f
0
(k ) x
:
In theory, the Newton-Raphson method onverges if f is twi e ontinuously di erentiable and if the initial value of x supplied by the analyst is \suÆ iently" lose to a lo al maximum of f at whi h the Hessian f 00 is negative de nite. There is, however, no generally pra ti al formula for determining what suÆ iently lose is. Typi ally,
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
70
an analyst makes a reasonable guess for the maximum of f and ounts his blessings if the iterates onverge. The Newton-Raphson method an be robust to the starting value if f is well behaved, for example, if f is globally on ave. The Newton-Raphson method, however, an be very sensitive to starting value if the fun tion is not globally
on ave. Also, in pra ti e, the Hessian f must be well- onditioned at the optimum, otherwise rounding errors in the vi inity of the optimum an make it diÆ ult to
ompute a pre ise approximate solution. The Newton-Raphson algorithm has numerous drawba ks. First, the algorithm requires omputation of both the rst and se ond derivatives of the obje tive fun tion. Se ond, the Newton-Raphson algorithm o ers no guarantee that the obje tive fun tion value may be in reased in the dire tion of the Newton step. Su h a guarantee is (k ) available only if the Hessian f x is negative de nite; otherwise, one may a tually move towards a saddle point of f (if the Hessian is inde nite) or even a minimum (if Hessian is positive de nite). For this reason, the Newton-Raphson method is rarely used in pra ti e, and then only if the obje tive fun tion is globally on ave. 00
00
4.3
Quasi-Newton Methods
Quasi-Newton methods employ a similar strategy to the Newton-Raphson method, but repla e the Hessian of the obje tive fun tion (or its inverse) with a negative de nite approximation, guaranteeing that fun tion value an be in reased in the dire tion of the Newton step. The most eÆ ient quasi-Newton algorithms employ an approximation to the inverse Hessian, rather than the Hessian itself, in order to avoid performing a linear solve, and employ updating rules that do not require se ond derivative information to ease the burden of implementation and the ost of omputation. In analogy with the Newton-Raphson method, quasi-Newton methods use a sear h dire tion of the form (k ) d
=
B
(k )
f
0
(k )
x
where B (k) is an approximation to the inverse Hessian of f at the kth iterate x(k) . The ve tor d(k) is alled the Newton or quasi-Newton step. The more robust quasi-Newton methods do not ne essarily take the full Newton step, but rather shorten it or lengthen it in order to obtain improvement in the obje tive fun tion. This is a
omplished by performing a line-sear h in whi h one (k ) (k ) seeks a step length s > 0 that maximizes or nearly maximizes f x + sd . Given the omputed step length s(k) , one updates the iterate as follows: (k +1)
x
= x(k) + s(k) d(k) :
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
71
Line sear h methods are dis ussed in the following se tion. Quasi-Newton method di er in how the inverse Hessian approximation B k is onstru ted and updated. The simplest quasi-Newton method sets B k = I , where I is the identity matrix. This leads to a Newton step that is identi al to the gradient of the obje tive fun tion at the urrent iterate: (k ) d = f 0 x(k) : The hoi e of gradient as a step dire tion is intuitively appealing be ause the gradient always points in the dire tion whi h, to a rst order, promises the greatest in rease in f . For this reason, this quasi-Newton method is alled the method of steepest as ent. The steepest as ent method is simple to implement, but is numeri ally less eÆ ient in pra ti e than ompeting quasi-Newton methods that in orporate information regarding the urvature of the obje tive fun tion. The most widely-used quasi-Newton methods that employ urvature information produ e a sequen e of inverse Hessian estimates that satisfy two onditions. First, given that (k ) d f 00 1(x(k) ) f 0(x(k) + d(k) ) f 0(x(k) ) ; the inverse Hessian estimate Ak is required to satisfy the so- alled
ondition: (k ) d = B (k) f 0 (x(k) + d(k) ) f 0 (x(k) ) :
quasi-Newton
Se ond, the inverse Hessian estimate A(k) is required to be both symmetri and negative-de nite, as must be true of the inverse Hessian at a lo al maximum. The negative de niteness of the Hessian estimate assures that the obje tive fun tion value
an be inreased in the dire tion of the Newton step. Two methods that satisfy the quasi-Newton and negative de niteness onditions are the Davidson-Flet her-Powell (DFP) and Broyden-Flet her-Goldfarb-Shano (BFGS) updating methods. The DFP method uses the updating s heme > Buu>B dd ; B B + > > d u u Bu where d
= x(k+1)
u
= f 0 (x(k+1) )
(k )
x
and f
0 (x(k) ):
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
72
The BFGS method uses the update s heme > w u 1 > > > B B + > wd + dw > dd ; d u d u where w = d Bu: The BFGS algorithm is generally onsidered superior to DFP, although there are problems for whi h DFP outperforms BFGS. However, ex ept for the updating formulae, the two methods are identi al, so it is easy to implement both and give users the hoi e.3 The following s ript omputes the maximum of a user-supplied multivariate fun tion f using the quasi-Newton method. The s ript assumes that the user has written a routine f that evaluates the fun tion at an arbitrary point and that the user has spe i ed a starting point x, an initial guess for the inverse Hessian A, a onvergen e toleran e tol, and a limit on the number of iterations maxit. The s ript uses an auxiliary algorithm optstep to determine the step length (dis ussed in the next se tion). The algorithm also o ers the user a hoi e on how to sele t the sear h dire tion, sear hmeth (1-steepest as ent, 2-DFP, 3-BFGS).
Matlab Matlab
k = size(x,1); [fx0,g0℄ = f(x); if all(abs(g0)
f (x + sd) s
f (x)
1f (x) 0
>
d;
CHAPTER 4.
76
FINITE-DIMENSIONAL OPTIMIZATION
for some values of 0 < 0 0:5 1 < 1. Unlike the Armijo sear h, whi h is both a method for sele ting andidate values of the stepsize s and a stopping rule, the Goldstein riteria is simply a stopping rule that an be used with a variety of sear h approa hes. Figure 4.6 illustrates the typi al situation at a given iteration. The gure plots the obje tive fun tion, expressed as deviations from f (x), i.e., f (x + sd) f (x), against the step size s in the Newton dire tion d. The obje tive fun tion is highlighted and the line tangent to it at the origin has slope equal to the dire tional derivative f (x) d. The values 0 and 1 de ne a one within whi h the fun tion value must lie to be
onsidered an a
eptable step. In Figure 4.6 the one is bounded by dashed lines with 0 = 0:25 and 1 = 0:75. These values are for illustrative purposes and de ne a far narrower one than is desirable; typi al values are on the order of 0.0001 and 0.9999. 0
Step Length Determination
−5
12
>
x 10
10
8
f(x+sd)
6
BHHHSTEP: s = 0.00097656 STEPBT: s = 0.0010499 GOLDSTEP: s = 0.001054
4
2
0
−2
−4 −5
0
5
10
s
15
20 −4
x 10
Figure 4.6 A simple strategy for lo ating an a
eptable point is to rst nd a point in or above the one using step doubling (doubling the value of s at ea h iteration). If a point above the one is found rst, we have a bra ket within whi h points in the one must lie. We an then narrow the bra ket using the golden sear h method. We all
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
77
this the bhhhstep approa h. Another approa h, stepbt, he ks to see if s = 1 is in the one and, if so, maximizes a quadrati approximation to the obje tive fun tion in the Newton dire tion
onstru ted from knowledge of f (x), f 0 (x)d and f (x + d). If the omputed step s is a
eptable, it is taken. Otherwise, the algorithm iterates until an a
eptable step is found using a ubi approximation to the obje tive fun tion in the Newton dire tion
onstru ted from knowledge of f (x), f 0 (x)d, f (x + s(j 1) d) and f (x + s(j ) d). stepbt is fast and generally gives good results. It is re ommended as the default lines sear h pro edure for general maximization algorithms. In Figure 4.6 we have in luded three stars representing the step lengths determined by stepbhhh, stepbt and our implementation of the golden sear h step length maximizer, stepgold (also listed below). stepgold rst bra kets a maximum in the dire tion d and then uses the golden sear h approa h to narrow the bra ket. This method di ers from the other two in that it terminates when the size of the bra ket is less than a spe i ed toleran e (here set at 0.0004). In this example, the three methods took 11, 4 and 20 iterations to nd an a
eptable step length, respe tively. Noti e that stepbt found the maximum in far fewer steps than did stepgold. This will generally be true when the fun tion is reasonably smooth and hen e well approximated by a ubi fun tion. It is diÆ ult to make generalizations about the performan e of the step line sear h algorithm, however. In this example, the step size was very small, so both stepbhhh and stepgold take many iterations to get the order of magnitude orre t. In many ases, if the initial distan e is well hosen, the step size will typi ally be lose to unity in magnitude, espe ially as the maximizer approa hes the optimal point. When this is true, the advantage of stepbt is less important. Having said all of that, we re ommend stepbt as a default. We have also implemented our algorithm to use stepgold if the other methods fail. 4.5
Spe ial Cases
Two spe ial ases arise often enough in e onomi pra ti e (espe ially in e onometri s) to warrant additional dis ussion. Nonlinear least squares and the maximum likelihood problems have obje tive fun tions with spe ial stru tures that give rise to their own spe ial quasi-Newton methods. The spe ial methods di er from other Newton and quasi-Newton methods only in the hoi e of the matrix used to approximate the Hessian. Be ause these problems generally arise in the ontext of statisti al appli ations, we alter our notation to onform with the onventions for those appli ations. The optimization takes pla e with respe t to a k-dimensional parameter ve tor and n will refer to the number of observations.
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
78
The nonlinear least squares problem takes the form min 12 f ()>f ()
where f : f ():
i=1
The Hessian of the obje tive fun tion is n
X 2 f ( ) : f 0 ( )> f 0 ( ) + fi ( ) > i=1
If we ignore the se ond term in the Hessian, we are assured of having a positive de nite matrix with whi h to determine the sear h dire tion: 0 1 0 d= f ( )> f 0 ( ) f ( )> f ( ): All other aspe ts of the problem are identi al to the quasi-Newton methods already dis ussed, ex ept for the adjustment to minimization. It is also worth pointing out that, in typi al appli ations, f () omposed of error terms ea h having expe tation 0. Assuming that the usual entral limit assumptions apply to the error term, the inverse of the approximate Hessian
1
f 0 ( )> f 0 ( )
;
an be used as a ovarian e estimator for . Maximum likelihood problems are spe i ed by a hoi e of a distribution fun tion for the data, y , that depends on a parameter ve tor, . The log-likelihood fun tion is the sum of the logs of the likelihoods of ea h of the data points: l
(; y ) =
n X
ln f (; yi):
i=1
The s ore fun tion is de ned as the matrix of derivatives of the log-likelihood fun tion evaluated at ea h observation: si ( ; y ) =
l ( ; yi )
:
(viewed as a matrix, the s ore fun tion is n k).
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
79
A well-known result in statisti al theory is that the expe tation of the inner produ t of the s ore fun tion is equal to the negative of the expe tation of the se ond derivative of the likelihood fun tion, whi h is known as the information matrix. Either the information matrix or the sample average of the inner produ t of the s ore fun tion provides a positive de nite matrix that an be used to determine a sear h dire tion. In the later ase the sear h dire tion is de ned by d=
s( ; y )>s(; y )
1
s(; y )>1n ;
where 1n is an n-ve tor of ones. This approa h is known as the modi ed method of s oring.4 As in the ase of the nonlinear least squares, a ovarian e estimator for is immediately available using
s( ; y )>s(; y )
4.6
1
:
Constrained Optimization
The simplest onstrained optimization problem involves the maximization of an obje tive fun tion subje t to simple bounds on the hoi e variable: max f (x):
axb
A
ording to the Karush-Kuhn-Tu ker theorem, if f is di erentiable on [a; b℄, then is a onstrained maximum for f only if it solves the omplementarity problem CP(f 0 ; a; b):5
x
bi ) fi0(x) 0 ) fi0(x) 0:
ai xi xi > ai xi < bi
Conversely, if f is on ave and di erentiable on [a; b℄ and x solves the omplementarity problem CP(f 0 (x); a; b), then x is a onstrained maximum of f ; if additionally f is stri tly on ave on [a; b℄, then the maximum is unique. Two bounded maximization problems are displayed in Figure 4.7. In this gure, the bounds are displayed with dashed lines and the obje tive fun tion with a solid line. In Figure 4.7A the obje tive fun tion is on ave and a hieves its unique global maximum on the interior of the feasible region. At the maximum, the derivative of f must be zero, for otherwise one ould improve the obje tive by moving either up or 4 If the information matrix is known in losed form, it ould be used rather than s> s and the method would be known as the method of s oring. 5 Complementarity problems are dis ussed in Se tion 3.7 on page 48.)
CHAPTER 4.
80
FINITE-DIMENSIONAL OPTIMIZATION
down, depending on whether the derivative is positive or negative. In Figure 4.7B we display a more ompli ated ase. Here, the obje tive fun tion is onvex. It a hieves a global maximum at the lower bound and a lo al, non-global maximum at the upper bound. It also a hieves a global minimum in the interior of the interval. One−Dimensional Maximization Problems 2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5 −0.5
0
0.5
1
1.5
−0.5 −0.5
a) f(x) = 1.5−(x−3/4)2, x* = 3/4
0
0.5
1
1.5
b) f(x) = −2+(x−3/4)2, x* = 0 & 1
Figure 4.7 In Figure 4.8 we illustrate the omplementarity problem presented by the KarushKuhn-Tu ker onditions asso iated with the bounded optimization problems in Figure 4.7. The omplementarity problems are represented in their equivalent root nding formulation min(max(f 0 (x); a x); b x) = 0. In Figure 4.8A we see that the KarushKuhn-Tu ker onditions possess an unique solution at the unique global maximum of f . In Figure 4.8B there are three solutions to the Karush-Kuhn-Tu ker onditions,
orresponding to the two lo al maxima and the one lo al minimum of f on [a; b℄. These gures illustrate that one may reliably solve a bounded maximization problem using standard omplementarity methods only if the obje tive fun tion is on ave. Otherwise, the omplementary algorithm ould lead to lo al, non-global maxima or even minima. The sensitivity of the optimal value of the obje tive fun tion f to hanges in the bounds of the bounded optimization problem are relatively easy to hara terize. A
ording to the Envelope theorem, df
da df
db
= min (0; f 0 (x )) = max (0; f 0 (x )) :
More generally, if f , a, and b all depend on some parameter p, then df
dp
=
f p
+ min 0;
f x
da dp
+ max 0;
f x
db dp
;
CHAPTER 4.
81
FINITE-DIMENSIONAL OPTIMIZATION
Complementarity Conditions for Maximization Problems 2
2
1.5
1.5
1
1
0.5
0.5
0
0
−0.5
−0.5
−1
−1
−1.5
−1.5
−2 −0.5
0
0.5
1
1.5
−2 −0.5
a) f’(x) = −2(x−3/4)
0
0.5
1
1.5
b) f’(x) = 2(x−3/4)
Figure 4.8 where the derivatives of f , a, and b are evaluated at (x ; p). The most general onstrained nite-dimensional optimization problem that we
onsider is max f (x); s.t. R(x) S r;
axb
where R : [a; b℄ ! > ; ; q p R(x) r
where the values of p and q depend on the type of onstraint:
pi qi
0 1 1 1 1 0 =
A point x is regular if the gradients of all onstraint fun tions Ri that satisfy Ri (x) = ri are linearly independent.6 Conversely, if f is on ave, R is onvex and (x; ) satis es the Karush-Kuhn-Tu ker onditions, then x solves the general onstrained optimization problem. 6 The
on ave,
regularity onditions may be omitted if either the onstraint fun tion
R
is onvex, and the feasible set has nonempty interior.
R
is linear, or if
f
is
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
82
In the Karush-Kuhn-Tu ker onditions, the i are alled Lagrangian multipliers or shadow pri es. The signi an e of the shadow pri es is given by the Envelope Theorem, whi h asserts that under mild regularity onditions,
f = ; r that is, i is the rate at whi h the optimal value of the obje tive will hange with
hanges in the onstraint onstant ri . The sensitivity of the optimal value of the obje tive fun tion f to hanges in the bounds on the hoi e variable are given by: df = min 0; f 0 (x) R0 (x)> da df = max 0; f 0 (x) R0 (x)> : db
The Karush-Kuhn-Tu ker omplementarity onditions typi ally have a natural arbitrage interpretation. Consider the problem of maximizing pro ts from ertain e onomi a tivities when the a tivities employ xed fa tors or resour es that are available in limited supply. Spe i ally, suppose x1 ; x2 ; : : : ; xn are the levels of n e onomi a tivities, whi h must be nonnegative, and the obje tive is to maximize pro t f (x) generated by those a tivities. Also suppose that these a tivities employ m resour es and that the usage of the ith resour e Ri (x) annot ex eed a given availability ri . Then i represents the opportunity ost or shadow pri e of the ith resour e and
MPj =
f xj
X i
i
Ri xj
represents the e onomi marginal pro t of the j th a tivity, a
ounting for the opportunity ost of the resour es employed in the a tivity. The Karush-Kuhn-Tu ker
onditions may thus be interpreted as follows: xj 0 a tivity levels are nonnegative MPj 0 otherwise, raise pro t by raising xj xj > 0 ) MPj 0 otherwise, raise pro t by lowering xj i 0 Shadow pri e of resour e is nonnegative Ri (x) ri resour e use annot ex eed availability i > 0 ) Ri (x) = ri valuable resour es should not be wasted There are many approa hes to solving general optimization problems that would take us beyond what we an hope to a
omplish in this book. Solving general optimization problems is diÆ ult and the best advi e we an give here is that you should obtain a good pa kage and use it. However, if your problem is reasonably well behaved
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
83
in the sense that the Karush-Kuhn-Tu ker are both ne essary and suÆ ient, then the problem is simply to solve the Karush-Kuhn-Tu ker onditions. This means writing the Karush-Kuhn-Tu ker onditions as a omplementarity problem and solving the problem using the methods of the previous hapter.
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
84
Exer ises 4.1. Suppose that the probability density fun tion of a non-negative random variable, y , is exp( yi =i )=i where i = Xi for some observable data Xi (Xi is 1 k and is k 1). (a) Show that the rst order onditions for the maximum likelihood estimator of an be written as
X
Xi >Xi = (Xi )2
X
Xi >yi : (Xi )2
(b) Use this result to de ne a re ursive algorithm to estimate . ( ) Write a fun tion of the form [beta,sigma℄=example(y,X) that
omputes the maximum likelihood estimator of and its asymptoti ovarian e matrix . The fun tion should be a stand-alone pro edure (i.e., do not all any optimization or root- nding solvers) that implements the re ursive algorithm. (d) Show that the re ursive algorithm an be interpreted as a quasi-Newton method. Explain fully.
Matlab
4.2. The two-parameter gamma probability distribution fun tion has density: f (x; ) =
21 x1 1 e (1 )
2 x
:
(a) Derive the rst order onditions asso iated with maximizing the log-likelihood asso iated with this distribution. Note that the rst and se ond derivatives of the log of the fun tion are the psi and trigamma fun tions. The toolbox ontains pro edures to evaluate these spe ial fun tions. (b) Solve the rst order ondition for 2 in terms of 1 . Use this to derive an optimality ondition for 1 alone. ( ) Write a fun tion that is passed a ve tor of observations (of positive numbers) and returns the maximum likelihood estimates of and their ovarian e matrix. Implement the fun tion to use Newton's method without alling any general optimization or root- nding solvers.
Matlab
Matlab
CHAPTER 4.
85
FINITE-DIMENSIONAL OPTIMIZATION
Noti e that the maximum Pn likelihood estimator of depends on the Pndata only through Y = n i xi , the arithmeti mean, and Y = exp( n i ln(xi)), the geometri mean (Y and Y are known as suÆ ient statisti s for ). Your ode should exploit this by only omputing these suÆ ient statisti s on e. (d) Plot as a fun tion of Y =Y over the range [1:1; 3℄. 4.3. CIR Bond Pri ing The so- all Cox-Ingersoll-Ross (CIR) bond pri ing model uses the fun tion Z (r; ; ; ; ) = A( ) exp( B ( )r) with 1
1
2
=1
1
1
2
= 2
e = A( ) = ( + )(e 1) + 2
and B ( ) =
( + )
=1
2
1
1
2
2
2
2(e 1) ( + )(e 1) + 2 ;
p where = + 2 . Here r is the urrent instantaneous rate of interest, is the time to maturity of the bond, and , and are model parameters. The per ent rate of return on a bond is given by r( ) = 100 ln(Z (r; ))= : In the following table, a tual rates of return on Treasury bonds for 9 values of are given for 5 onse utive Wednesdays in early 1999. Date .25 .5 1 2 3 5 7 10 30 1999/01/07 4.44 4.49 4.51 4.63 4.63 4.62 4.82 4.77 5.23 1999/01/13 4.45 4.48 4.49 4.61 4.61 4.60 4.84 4.74 5.16 1999/01/20 4.37 4.49 4.53 4.66 4.66 4.65 4.86 4.76 5.18 1999/01/27 4.47 4.47 4.51 4.57 4.57 4.57 4.74 4.68 5.14 1999/02/03 4.48 4.55 4.59 4.72 4.73 4.74 4.91 4.83 5.25 2
2
7
7 A tually,
the data is onstru ted by a smoothing and tting pro ess and thus these returns do not ne essarily represent the market pri es of a tual bonds; for the purposes of the exer ise, however, this fa t an be ignored.
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
86
a) For ea h date, nd the values of r, , and that minimize the squared di eren es between the model and the a tual rates of return. This is one way that model parameters an be \ alibrated" to the data and ensures that model parameters yield a term stru ture that is lose to the observed term stru ture. b) In this model the values of the parameters are xed, but the value of r varies over time. In fa t, part (a) showed that the three parameters values vary from week to week. As an alternative, nd the values of the parameters and the 5 values of r that minimize the squared deviations between the model and a tual values. Compare these to the parameter values obtained by alibrating to ea h date separately. 4.4. Option-based Risk-neutral Probabilities An important theorem in nan e theory demonstrates that the value of a European put option is equal to the expe ted return on the option, with the expe tation taken with respe t to the so- alled risk-neutral probability measure8
V (k ) =
Z1k 0
p)+ f (p)dp =
(
Z
k
0
(k
p)f (p)dp
where f (p) is the probability distribution of the pri e of underlying asset at the option's maturity, k is the option's strike pri e and (x)+ = max(0; x). This relationship has been used to ompute estimates of f (p) based on observed asset pri es. There are two approa hes that have been taken. The rst is to
hoose a parametri form for f and nd the parameters that best t the observed option pri e. To illustrate, de ne the dis repan y between observed and model values as
e(k) = Vk
Z
k
0
(k
p)f (p; )dp
Xe k
and then t by, e.g., minimizing the sum of squared errors: min
( j )2 :
j
The other approa h is to dis retize the pri e, pi , and its probability distribution, fi . Values of the fi an be omputed that orre tly reprodu e observed option 8 This
is stri tly true only if the interest rate is 0 or, equivalently, if the option values are interest
rate adjusted appropriately. the interest rate.
Also, the pri e of the underlying asset should not be orrelated with
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
87
value and that satisfy some auxiliary ondition. That ondition ould be a smoothness ondition, su h as minimizing the sum of the fi+1 2fi + fi 1 ; if the pi are evenly spa ed this is proportional to an approximation to the se ond derivative of f (p). An alternative is to ompute the maximum entropy values of the fi : max ffi g
X f ln(f ); i
i
i
subje t to the onstraints that the fi are non-negative and sum to 1. It is easy to show that the fi satisfy
P P f =P i
exp( j j (kj pi )+ ) ; pi )+ ) i exp( j j (kj
where j is the Lagrange multiplier on the onstraint that the j th option is
orre tly pri ed. The problem is thus onverted to the root nding problem of solving for the Lagrange multipliers:
Vj
X f (k i
j
pi )+ = 0;
i
where the fi are given above. Write a MATLAB program that takes as input a ve tor of pri e nodes, p, and asso iated ve tors of strike pri es, k, and observed put option values, v , and returns a ve tor of maximum entropy probabilities, f , asso iated with p: f=RiskNeutral(p,k,v)
The fun tion an pass an auxiliary fun tion to a root nding algorithm su h as Newton or Broyden. The pro edure just des ribed has the pe uliar property that (if put options alone are used), the upper tail probabilities are all equal above the highest value of the kj . To orre t for this, one an add in the onstraint that the expe ted pri e at the option's expiration date is the urrent value of the asset, as would be true in a 0 interest rate situation. Thus modify the original program to a
ept the urrent value of the pri e of the underlying asset: f=RiskNeutral(p,k,v,p0)
To test your program, use the s ript le RiskNeutD.m.
CHAPTER 4.
88
FINITE-DIMENSIONAL OPTIMIZATION
4.5. Consider the Quadrati Programming problem >Dx + >x
max
1 x 2
x
s:t:
Ax b x0
where D is a symmetri n n matrix, A is an m n matrix, b is an m-ve tor. (a) Write the Karush-Kuhn-Tu ker ne essary onditions as a linear omplementarity problem. (b) What ondition on D will guarantee that the Karush-Kuhn-Tu ker onditions are suÆ ient for optimality? 4.6. A onsumer's preferen es over the ommodities x1, x2 , and x3 are hara terized by the Stone-Geary utility fun tion
X ln(x U ( x) = 3
i
i )
i
i=1
where > 0 and x > 0. The onsumer wants to maximize his utility subje t to the budget onstraint i
i
i
Xp x I 3
i
i
i=1
where p > 0 denotes the pri e of x , I denotes in ome, and I P3=1 p > 0: (a) Write the Karush-Kuhn-Tu ker ne essary onditions for the problem. (b) Verify that the Karush-Kuhn-Tu ker onditions are suÆ ient for optimality. ( ) Derive analyti ally the asso iated demand fun tions. (d) Derive analyti ally the shadow pri e and interpret its meaning. (e) Prove that the onsumer will utilize his entire in ome. i
i
i
i
i
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
89
4.7. Derive and interpret the Karush-Kuhn-Tu ker onditions for the lassi al transportation problem: n X m X min x{| x{| { | n X s:t: x{| d| { Xm x{| s{ =1
=1
=1
|=1 x{| 0
| = 1; : : : ; m { = 1; : : : ; n { = 1; : : : ; n; | = 1; : : : ; m
State suÆ ient onditions for the transportation problem to have an optimal feasible solution. 4.8. Demand for a ommodity in regions A and B is given by: Region A : p = 200 Region B : p = 100
2q 4q
Supply is given by: Region A : p = 20 + 8q Region B : p = 10 + 6q:
The transportation ost between regions is $10 per unit. Formulate an optimization problem that hara terizes the ompetitive spatial pri e equilibrium. Derive, but do not solve, the Karush-Kuhn-Tu ker onditions. Interpret the shadow pri es. 4.9. Portfolio Choi e Suppose that the returns on a set of n assets has mean (n 1) and varian e (n n). A portfolio of assets an be hara terized by a set of share weights, ! , an n 1 ve tor of non-negative values summing to 1. The mean return on portfolio is >! and its varian e is !>!. A portfolio is said to be on the mean-varian e eÆ ient frontier if its varian e is as small as possible for a given mean return.
CHAPTER 4.
90
FINITE-DIMENSIONAL OPTIMIZATION
Write a MATLAB program that al ulates and plots a mean-varian e eÆ ient frontier. Write it so it returns two ve tors that provide points on the frontier: [mustar,Sigmastar℄=MV(mu,Sigma,n) Here represents the desired number of points. Run the program MVDemo.m to test your program. Hint: Determine the mean return from the minimium varian e portfolio and determine the maximum mean return portfolio. These provide lower and upper bounds for mustar. Then solve the optimization problem for the remaining 2 values of mustar. 4.10. Consider the nonlinear programming problem max 22 2 1 21 2 2 1+ 2 1 1 0 2 0 (a) Write the Karush-Kuhn-Tu ker ne essary onditions for the problem. (b) What points satisfy the Karush-Kuhn-Tu ker ne essary onditions. ( ) Are the Karush-Kuhn-Tu ker onditions suÆ ient for optimality? (d) How do you know that problem possesses an optimum? (e) Determine the optimum, if any. 4.11. A tomato pro essor operates two plants whose hourly variable osts (in dollars) are, respe tively, n
n
x
x1 ;x2
s:t:
x
x
1
2
x
x
x
;x
:
= 80 + 2 0 1 + 0 001 = 90 + 1 5 2 + 0 002
2
: x
:
x1
: x
:
x2 ;
2
where is the number of ases produ ed per hour at plant . In order to meet
ontra tual obligations, he must produ e at a rate of at least 2000 ases per hour ( 1 + 2 2000.) He wishes to do so at minimal ost. (a) Write the Karush-Kuhn-Tu ker ne essary onditions for the problem. (b) Verify that the Karush-Kuhn-Tu ker onditions are suÆ ient for optimality. ( ) Determine the optimal levels of produ tion. (d) Determine the optimal value of the shadow pri e and interpret its meaning. xi
x
i
x
CHAPTER 4.
91
FINITE-DIMENSIONAL OPTIMIZATION
4.12. Consider the problem of allo ating a s ar e resour e, the total supply of whi h is b > 0, among n tasks with separable rewards: max
x1 ;x2 ;:::;x
f1 (x1 ) + f2 (x2 ) + : : : + f (x ) n
n
n
x1 + x2 + : : : + x b x1 0; x2 0; : : : ; x 0:
s:t:
n
n
Assume ea h f is stri tly in reasing and di erentiable but not ne essarily on ave. i
(a) (b) ( ) (d) (e)
How do you know that problem possesses an optimum? Write the Karush-Kuhn-Tu ker ne essary onditions. Prove that the s ar e resour e will be ompletely utilized. Interpret the shadow pri e asso iated with the resour e onstraint. Given a marginal in rease in the supply of the resour e, to whi h task(s) would you allo ate the additional amount.
4.13. Consider a one-output two-input produ tion fun tion y = f(x1 ; x2 ) = x21 + x22 : Given the pri es of inputs 1 and 2, w1 and w2 , the minimum ost of produ ing a given level of output, y, is obtained by solving the onstrained optimization problem min C = w1 x1 + w2 x2
x1 ;x2
s:t:
f(x1 ; x2 ) y:
Letting denote the shadow pri e asso iated with the produ tion onstraint, answer the following questions: (a) (b) ( ) (d)
Write the Karush-Kuhn-Tu ker ne essary onditions. Find expli it expressions for the optimal x1 , x2 , and C . Find an expli it expression for the optimal and interpret its meaning. Di erentiate the expression for C to on rm that = . C
y
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
92
4.14. A salmon annery produ es Q 1-lb. ans of salmon a
ording to a te hnology 1 1 given by Q = 18K 4 L 3 , where apital K is xed at 16 units in the shortrun and labor L may be hired in any quantity at a wage rate of w dollars per unit. Ea h unit of output provides a pro t ontribution of 1 dollar. (a) Derive the rm's shortrun demand for labor. (b) If w = 3, how mu h would the rm be willing to pay to rent a unit of
apital. 4.15. Consider the nonlinear programming problem min
x1 ;:::;x4
s:t:
x01 25 x03 50 x04 25 :
:
:
x1 + x2 + x3 + x4 4 x1 ; x2 ; x3 ; x4 0:
(a) What an you say about the optimality of the point (1; 0; 2; 1)? (b) Does this program possess all the orre t urvature properties for the Karush-Kuhn-Tu ker onditions to be suÆ ient for optimality throughout the feasible region? Why or why not? ( ) How do you know that problem possesses an optimal feasible solution? 4.16. Consider the non-linear programming problem min
x1 ;x2
s:t:
2x21
12x1 + 3x22
3x1 + x2 12 x1 + x2 6 x1 ; x2 0:
18x2 + 45
The optimal solution to this problem is: x1 = 3 and x2 = 3. (a) Verify that the Karush-Kuhn-Tu ker onditions are satis ed by this solution. (b) Determine the optimal values for the shadow pri es 1 and 2 asso iated with the stru tural onstraints, and interpret 1 and 2 . ( ) If the se ond onstraint were hanged to x1 + x2 5, what would be the e e t on the optimal values of x1 , x2 , 1 , and 2 ?
CHAPTER 4.
FINITE-DIMENSIONAL OPTIMIZATION
93
Bibliographi Notes A number of very useful referen es exist on omputational aspe ts of optimization. Perhaps the most generally useful for pra titioners are Gill et al. and Flet her. Ferris and Sinapiromsaran dis usses solving non-linear optimization problems by formulating them as CPs.
Chapter 5 Numeri al Integration and Di erentiation In many omputational e onomi appli ations, one must ompute the de nite integral of a real-valued fun tion f with respe t to a \weighting" fun tion w over an interval n I of < :
Z
( ) ( ) dx:
f x w x
I
The weighting fun tion may be the identity, w 1, in whi h ase the integral represents the area under the fun tion f . In other appli ations, w may be the probability density of a random variable X~ , in whi h ase the integral represents the expe tation of f (X~ ) when I repesents the whole support of X~ . In this hapter, we dis uss three lasses of numeri al integration or numeri al quadrature methods. All methods approximate the integral with a weighted sum of fun tion values:
Z
( ) ( ) dx
f x w x
I
X n
( )
wi f xi
i=0
The methods di er only in how the quadrature weights wi and the quadrature nodes are hosen. Newton-Cotes methods approximate the integrand f between nodes using low order polynomials, and sum the integrals of the polynomials to estimate the integral of f . Newton-Cotes methods are easy to implement, but are not parti ularly eÆ ient for omputing the integral of a smooth fun tion. Gaussian quadrature methods hoose the nodes and weights to satisfy moment mat hing onditions, and are more powerful than Newton-Cotes methods if the integrand is smooth. Monte Carlo and quasi-Monte Carlo integration methods use \random" or \equidistributed" xi
94
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
95
nodes, and are simple to implement and are espe ially useful if the integration domain is of high dimension or irregularly shaped. In this hapter, we also present an overview of how to ompute nite di eren e approximations for the derivatives of a real-valued fun tion. As we have seen in previous hapters, it is often desirable to ompute derivatives numeri ally be ause analyti derivative expressions are diÆ ult or impossible to derive, or expensive to evaluate. Finite di eren e methods an also be used to solve di erential equations, whi h arise frequently in dynami e onomi models, espe ially models formulated in
ontinuous time. In this hapter, we introdu e numeri al methods for di erential equations and illustrate their appli ation to initial value problems. 5.1
Newton-Cotes Methods
Newton-Cotes quadrature methods are designed to approximate the integral of a realvalued fun tion f de ned on a bounded interval [a; b℄ of the real line. Newton-Cotes methods approximate the integrand f between nodes using low order polynomials, and sum the integrals of the polynomials to form an estimate the integral of f . Two Newton-Cotes rules are widely used in pra ti e: the trapezoid rule and Simpson's rule. Both rules are very easy to implement and are typi ally adequate for omputing the area under a ontinuous fun tion. The simplest way to ompute an approximate integral of a real-valued fun tion f over a bounded interval [a; b℄ < is to partition the interval into subintervals of equal length, approximate f over ea h subinterval using a straight line segment that linearly interpolates the fun tion values at the subinterval endpoints, and then sum the areas under the line segments. This is the so- alled trapezoid rule, whi h draws its name from the fa t that the area under f is approximated by a series of trapezoids. More formally, let xi = a + (i 1)h for i = 1; 2; : : : ; n, where h = (b a)=n. The nodes xi divide the interval [a; b℄ into n 1 subintervals of equal length h. Over the th subinterval, [x ; x i i i+1 ℄, the fun tion f may be approximated by the line segment passing through the two graph points (xi ; f (xi )) and (xi+1 ; f (xi+1 )). The area under this line segment de nes a trapezoid that provides an estimate of the area under f over this subinterval:
Z
xi+1
( ) dx
f x
xi
Z
Z
xi+1
^( ) dx = h [f (xi ) + f (xi+1 )℄: 2
f x
xi
Summing up the areas of the trapezoids a ross subintervals yields the trapezoid rule: b
( ) dx
f x
a
X n
i=1
( )
wi f xi
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
96
where w1 = wn = h=2 and wi = h, otherwise. The trapezoid rule is simple and robust. Other Newton-Cotes methods will be more a
urate if the integrand f is smooth. However, the trapezoid rule will often be more a
urate if the integrand exhibits dis ontinuities in its rst derivative, whi h
an o
ur in e onomi appli ations exhibiting orner solutions. The trapezoid rule is said to be rst order exa t be ause in theory it exa tly omputes the integral of any rst order polynomial, that is, a line. In general, if the integrand is smooth, the trapezoid rule yields an approximation error that is O(h2), that is, the error shrinks quadrati ally with the size of the sampling interval. Simpson's rule is based on pie e-wise quadrati , rather than pie e-wise linear, approximations to the integrand f . More formally, let xi = a +(i 1)h for i = 1; 2; : : : ; n, where h = (b a)=(n 1) and n is odd. The nodes xi divide the interval [a; b℄ into an even number n 1 of subintervals of equal length h. Over the j th pair of subintervals, [x2j 1 ; x2j ℄ and [x2j ; x2j +1 ℄, the fun tion f may be approximated by the unique quadrati fun tion f^j that passes through the three graph points (x2j 1 ; f (x2j 1 )) (x2j ; f (x2j )), and (x2j +1 ; f (x2j +1 )). The area under this quadrati fun tion provides an estimate of the area under f over the subinterval:
Z
x2j +1
x2j
1
( ) dx
f x
Z
x2j +1 x2j
1
^ ( ) dx =
fj x
h
3
(f (x2j 1 ) + 4f (x2j ) + f (x2j +1 )) :
Summing up the areas under the quadrati approximants a ross subintervals yields Simpson's rule:
Z
b
( ) dx
f x
a
X n
i=1
( )
wi f xi
where w1 = wn = h=3 and, otherwise, wi = 4h=3 if i is odd and wi = 2h=3 if i is even. Simpson's rule is almost as simple as the trapezoid rule, and thus not mu h harder to program. Simpson's rule, moreover, will yield more a
urate approximations if the integrand is smooth. Even though Simpson's rule is based on lo ally quadrati approximation of the integrand, it is third order exa t. That is, it exa tly omputes the integral of any third order (e.g., ubi ) polynomial. In general, if the integrand is smooth, Simpson's rule yields an approximation error that is O(h4 ), and thus falls at twi e the geometri rate as the error asso iated with the trapezoid rule. Simpson's rule is the Newton-Cotes rule most often used in pra ti e be ause it retains algorithmi simpli ity while o ering an adequate degree of approximation. Newton-Cotes rules of higher order may be de ned, but are more diÆ ult to work with and thus are rarely used.
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
97
Through the use of tensor produ t prin iples, univariate Newton-Cotes quadrature s hemes an be generalized for higher dimensional integration. Suppose one wishes to integrate a real-valued fun tion de ned on a re tangle f(x1 ; x2 )ja1 x1 b1 ; a2 2 x2 b2 g in < . One way to pro eed, is to ompute the Newton-Cotes nodes and weights f(x1i ; w1i )ji = 1; 2; : : : ; n1 g for the real interval [a1 ; b1 ℄ and the Newton-Cotes nodes and weights f(x2j ; w2j )jj = 1; 2; : : : ; n2 g for the real interval [a2 ; b2 ℄. The tensor produ t Newton-Cotes rule for the re tangle would be omprised of the n = n1 n2 grid points of the form f(x1i ; x2j )ji = 1; 2; : : : ; n1 ; j = 1; 2; : : : ; n2 g with asso iated weights fwij = w1iw2j ji = 1; 2; : : : ; n1 ; j = 1; 2; : : : ; n2 g. This onstru tion prin iple an be applied to an arbitrary dimension using repeated tensor produ t operations. In most omputational e onomi appli ations, it is not possible to determine a priori how many partition points are needed to ompute an integral to a desired level of a
ura y using a Newton-Cotes quadrature rule. One solution to this problem is to use an adaptive quadrature strategy whereby one in reases the number of points at whi h the integrand is evaluated until the sequen e of estimates of the integral onverge. EÆ ient adaptive Newton-Cotes quadrature s hemes are espe ially easy to implement. One simple, but powerful, s heme alls for the number of intervals to be doubled with ea h iteration. Be ause the new partition points in lude the partition points used in the previous iteration, the omputational e ort required to form the new integral estimate is ut in half. More sophisti ated adaptive Newton-Cotes quadrature te hniques relax the requirement that the intervals be equally spa ed and on entrate new evaluation points in those areas where the integrand appears to be most irregular. 5.2
Gaussian Quadrature
Gaussian quadrature rules are onstru ted with respe t to spe i weighting fun tions. Spe i ally, for a weighting fun tion w de ned on an interval I < of the real line, and for a given order of approximation n, the quadrature nodes x1 ; x2 ; : : : ; xn and quadrature weights w1 ; w2 ; : : : ; wn are hosen so as to satisfy the 2n \momentmat hing" onditions:
Z
k
( ) dx =
x w x
I
X n
k
wi xi ;
for k = 0; : : : ; 2n
1:
i=1
Integral approximations are then formed using weighted sums of values of f at sele ted nodes:
Z
( ) ( ) dx
f x w x
I
X n
i=1
( )
wi f xi :
CHAPTER 5.
98
INTEGRATION AND DIFFERENTIATION
Gaussian quadrature over a bounded interval with respe t to the identity weighting fun tion, w(x) 1, is alled Gauss-Legendre quadrature. Gauss-Legendre quadrature may be used to ompute the area under a urve, and an easily be generalized to integration on higher dimensional spa es using tensor produ t prin iples. By onstru tion, an n-point Gauss-Legendre quadrature rule will exa tly ompute the integral of any polynomial of order 2n 1 or less. Thus, if f an be losely approximated by a polynomial, a Gauss-Legendre quadrature should provide an a
urate approximation to the integral. Furthermore, Gauss-Legendre quadrature is onsistent for Riemann integrable fun tions. That is, if f is Riemann integrable, then the approximation a orded by Gauss-Legendre quadrature an be made arbitrarily pre ise by in reasing the number of nodes n. Sele ted Newton-Cotes and Gaussian quadrature methods are ompared in Table 5.1. The table illustrates that Gauss-Legendre quadrature is the numeri al integration method of hoi e when f possesses ontinuous derivatives, but should be applied with great aution otherwise. If the fun tion f possesses known kink points, it is often possible to break the integral into the sum of two or more integrals of smooth fun tions. If these or similar steps do not produ e smooth integrands, then NewtonCotes quadrature methods may be more eÆ ient than Gaussian quadrature methods be ause they limit the error aused by the kinks and singularities to the interval in whi h they o
ur. Table 5.1: Errors for Sele ted Quadrature Methods
Fun tion exp(
x)
(1 + 25x2 )
jxj0 5 :
1
Degree (n)
Trapezoid Rule
Simpson Rule
GaussLegendre
10 20 30
1.36e+001 3.98e+000 1.86e+000
3.57e-001 2.31e-002 5.11e-003
8.10e-002 2.04e-008 1.24e-008
10 20 30
8.85e-001 6.34e-001 4.26e-001
9.15e-001 6.32e-001 3.80e-001
8.65e-001 2.75e+001 1.16e+004
10 20 30
7.45e-001 5.13e-001 4.15e-001
7.40e-001 4.75e-001 3.77e-001
6.49e-001 1.74e+001 4.34e+003
When the weighting fun tion w(x) is the ontinuous probability density for some random variable X~ , Gaussian quadrature has a very straightforward interpretation.
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
99
In this ontext, Gaussian quadrature essentially \dis retizes" the ontinuous random variable X~ by onstru ting a dis rete random variable with mass points xi and probabilities wi that approximates X~ in the sense that both random variables have the same moments of order less than 2n:
X w x = E[X~ ℄ for k = 0; : : : ; 2n n
i=1
k i i
k
1:
Given the mass points and probabilities of the dis rete approximant, the expe tation of any fun tion of the ontinuous random variable X~ may be approximated using the expe tation of the fun tion of the dis rete approximant, whi h requires only the
omputation of a weighted sum: E [f (X~ )℄ =
Z f (x) w(x) dx X f (x )w : n
i
i
i=1
For example, the three-point approximation to the standard univariate normal distribution Z~ is hara terized by the ondition that moments 0 through 5 mat h those of the standard normal: E Z~0 = 1, E Z~ 1 = 0, E Z~2 = 1, E Z~3 = 0, E Z~4 = 3, and E Z~ 5 = 0. One an easily verify that these p by a dis rete p onditions are satis ed random variable with mass points x1 = 3, x2 = 0, and x3 = 3 and asso iated probabilities w1=1/6, w2 = 2=3, and w3 = 1=6. Computing the n-degree Gaussian nodes and weights is a non-trivial task whi h involves solving the 2n nonlinear equations for fxig and fwig. EÆ ient, spe ialized numeri al routines for omputing Gaussian quadrature nodes and weights are available for di erent weighting fun tions, in luding virtually all the better known probability distributions, su h as the uniform, normal, gamma, exponential, Chisquare, and beta distributions. Gaussian quadrature with respe t to the identity weight is alled Gauss-Legendre quadrature; Gaussian quadrature with respe t to normal probability densities is related to Gauss-Hermite quadrature.1 As was the ase with Newton-Cotes quadrature, tensor produ t prin iples may be applied to univariate Gaussian quadrature rules to develop quadrature rules for multivariate integration. Suppose, for example, that X~ is a d-dimensional normal random variable with mean ve tor and varian e- ovarian e matrix . Then X~ is ~ where R is the Cholesky square root of (e.g., = R>R) distributed as + ZR ~ and Z is a row d-ve tor of independent standard normal variates. If fzi ; wig are the degree n Gaussian nodes and weights for a standard normal variate, then an nd degree 1 Gauss-Hermite quadrature applies to the weighting fun tion weighting fun tion for the standard normal density
w(x) = exp(
w(x) =pexp( x2 ), x2 =2)= 2 .
as opposed the
CHAPTER 5.
100
INTEGRATION AND DIFFERENTIATION
approximation for X~ may be onstru ted using tensor produ ts. For example, in two dimensions the nodes and weights would take the form
xij = (1 + R11 zi + R21 zj ; 2 + R12 zi + R22 zj ) and
pij = pi pj : The Gaussian quadrature s heme for normal variates may also be used to develop a reasonable s heme for dis retizing lognormal random variates. By de nition, Y~ is lognormally distributed with parameters and 2 if, and only if, it is distributed as exp(X~ ) were X~ is normally distributed with mean and varian e 2 . It follows that if fxi ; wi g are nodes and weights for a Normal(; 2) distribution, then fyi; wi g, where yi = exp(xi ), provides a reasonable dis rete approximant for a Lognormal(; 2) distribution. Given this dis rete approximant for the lognormal distribution, one an estimate the expe tation of a fun tion of Y~ as follows: Ef (Y~ ) = f (y ) w(y ) dy n f (yi )wi This integration rule for lognormal distributions will be exa t if f is a i=1 polynomial of degree 2n 1 and less in log(y ) (not in y ).
R
P
5.3
Monte Carlo Integration
Monte Carlo integration methods are motivated by the Strong Law of Large Numbers. One version of the Law states that if x1 ; x2 ; : : : are independent realizations of a random variable X~ and f is a ontinuous fun tion, then 1 n!1 n lim
X f (x ) = Ef (X~ ) n
i
i=1
with probability one. The Monte Carlo integration s heme is thus a simple one. To ompute an approximation to the expe tation of f (X~ ), one draws a random sample x1 ; x2 ; : : : ; xn from the distribution of X~ and sets
h
E f (X~ )
i
n1
X f (x ): n
i
i=1
Matlab o ers two intrinsi random number generators. The routine rand gen-
erates a random sample from the Uniform(0,1) distribution stored in either ve tor or matrix format. Similarly, the routine randn generates a random sample from the standard normal distribution stored in either ve tor or matrix format. In parti ular,
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
101
a all of the form x=rand(m,n) or x=randn(m,n) generates a random sample of mn realizations and stores it in an m n matrix. The uniform random number generator is useful for generating random samples from other distributions. Suppose X~ has a umulative distribution fun tion ( ) = Pr(X~
F x
)
x
whose inverse has a well-de ned losed form. If U~ is uniformly distributed on (0; 1), then ~ = F 1 (U~ )
X
has the desired distribution F . Thus, to generate a random sample x1 ; x2 ; : : : ; xn from the X~ distribution, one generates a random sample u1 ; u2 ; : : : ; un from the uniform distribution and sets xi = F 1 (ui ). The standard normal random number generator is useful for generating random samples from related distributions. For example, to generate a random sample of n lognormal variates, one may use the s ript x = exp(mu+sigma*randn(n));
where mu and sigma are the mean and standard deviation of the distribution. To generate a random sample of n d-dimensional normal variates one may use the s ript x = randn(n,d)* hol(Sigma)+mu(ones(n,1),:);
where Sigma is the d by d varian e- ovarian e matrix and mu is the mean ve tor in row form. A fundamental problem that arises with Monte Carlo integration is that it is almost impossible to generate a truly random sample of variates for any distribution. Most ompilers and ve tor pro essing pa kages provide intrinsi routines for omputing so- alled random numbers. These routines, however, employ iteration rules that generate a purely deterministi , not random, sequen e of numbers. In parti ular, if the generator is repeatedly initiated at the same point, it will return the same sequen e of \random" variates ea h time. About all that an be said of numeri al random number generators is that good ones will generate sequen es that appear to be random, in that they pass ertain statisti al tests for randomness. For this reason, numeri al random number generators are more a
urately said to generate sequen es of \pseudo-random" rather than random numbers. Monte Carlo integration is easy to implement and may be preferred over Gaussian quadrature if the a routine for omputing the Gaussian mass points and probabilities is not readily available or if the integration is over many dimensions. Monte Carlo
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
102
integration, however, is subje t to a sampling error that annot be bounded with
ertainty. The approximation an be made more a
urate, in a statisti al sense, by in reasing the size of the random sample, but this an be expensive if evaluating f or generating the pseudo-random variate is ostly. Approximations generated by Monte Carlo integration will vary from one integration to the next, unless initiated at the same point, making the use of Monte Carlo integration in onjun tion within other iterative s hemes, su h as dynami programming or maximum likelihood estimation, problemati . So- alled quasi Monte-Carlo methods an ir umvent some of the problems asso iated with Monte-Carlo integration. 5.4
Quasi-Monte Carlo Integration
Although Monte-Carlo integration methods originated using insights from probability theory, re ent extensions have severed that onne tion and, in the pro ess, demonstrated ways in whi h the methods an be improved. Monte-Carlo methods rely on sequen es fxi g with the property that lim
n
!1
b
a n
X1 i=1
( )=
f xi
Z
b
( ) dx:
f x
a
Any sequen e that satis es this ondition for arbitrary (Riemann) integrable fun tions
an be used to approximate an integral on [a; b℄. Although the Law of Large Numbers assures us that this is true when the xi are independent and identi ally distributed random variables, other sequen es also satisfy this property. Indeed, it an be shown that sequen es that are expli itly non-random, but instead attempt to ll in spa e in a regular manner exhibit improved onvergen e properties. There are numerous s hemes for generating equidistributed sequen es. The best known are the Neiderreiter, Weyl, and Haber. The following s ript generates equidistributed sequen es of length n for the unit hyper ube:
Matlab
eds_pp=sqrt(primes(7920)); i=(1:n)'; swit h upper(type(1))
ase 'N' % Neiderreiter j=2.^((1:d)/(d+1)); x=i*j; x=x-fix(x);
ase 'W' % Weyl j=eds_pp(1:d); x=i*j;
CHAPTER 5.
103
INTEGRATION AND DIFFERENTIATION
x=x-fix(x);
ase 'H' j=eds_pp(1:d); x=(i.*(i+1)./2)*j; x=x-fix(x); end
% Haber
Matlab
The toolbox a
ompanying the textbook in ludes a fun tion qnwequi that generates the equidistributed nodes for integration over an arbitrary bounded interval in a spa e of arbitrary dimension. The alling sequen e takes the form [x,w℄ = qnwequi(n,a,b,type);
where x are the nodes, w are the weights, n is the number of nodes and weights, a is the ve tor of left endpoints, b is the ve tor of right endpoints, and type refers to the type of equidistributed sequen e (`N'-Neiderrieter, `W'-Weyl, and `H'-Haber). For example, suppose one wished to ompute the integral of exp(x1 + x2 ) over the re tangle [1; 2℄ [0; 5℄ in 0 for a lower bound and h < 0 for an upper bound). To obtain formulii for se ond derivatives we an use the same approa h but in order to obtain se ond order a
ura y, we will (in general) require a weighted sum
omposed of 4 points 0
0
f (x) af(x) + bf(x + h) + f(x + h) + df(x + h): 0
We also expand the Taylor series to the fourth order, obtaining af(x) + bf(x + h) + f(x + h) + df(x + h) = 2
(a + b + + d)f(x) + h(b + + d )f (x) + h2 (b + 2 + d 2 )f (x) 3 4 h h 3 3 (4) 4 (4) 4 (4) + 6 (b + + d )f (x) + 24 bf (z1 ) + f (z2 ) + d f (z3 ) : 0
00
000
The onstraints a + b + + d = 0, b + + d = 0, b + 2 + d b + 3 + d 3 = 0 uniquely determine a, b, and d: 2 3 1++ 6 7 6 7 6 7 6 7 2 3 2 2 6 7 a 6 7 6 b 7 2 6 ( 1)( 1)( ) 7 6 7= 6 7 7 4 5 h2 6 2 6 7 1 6 7 d 6 ( 1)( 1)( ) 7 6 7 6 7 4 5 1 2 ( 1)( 1)( )
2
= 2=h2 and
CHAPTER 5.
INTEGRATION AND DIFFERENTIATION
110
with af(x) + bf(x + h) + f(x + h) + df(x + h) = f 00 (x) + O(h2 ): Thus, by using 4 points, we an ensure that the approximation onverges at a quadrati rate in h. Some spe ial ases of importan e arise when the evaluations points are evenly spa ed. When x lies halfway between x+h and one of the other two points (i.e., when either = 1 or = 1), we obtain the entered nite di eren e approximation 2f(x) + f(x h) + O(h2); h2 whi h is se ond order a
urate even though only three approximation points are used. If = 2 and = 3 we obtain a formula that is useful when a derivative is needed at a boundary of the domain. In this ase f 00 (x) =
f(x + h)
1 [2f(x) 5f(x + h) + 4f(x + 2h) f(x + 3h)℄ + O(h2 ): 2 h An important use of se ond derivatives is in omputing Hessian matri es. Given some fun tion f : 0, there exists a polynomial p su h that
jjf
p
jj1 =
sup jf (x) 2[ ℄
x
j
p(x) < :
a;b
The Weierstrass theorem provides strong motivation for using polynomials to approximate ontinuous fun tions. The theorem, however, is not very pra ti al. It gives no guidan e on how to nd a good polynomial approximant. It does not even state what order polynomial is required to a hieve the required level of a
ura y. One apparently reasonable way to onstru t a n -degree polynomial approximant for a fun tion f is to form the unique (n 1) -order polynomial th
th
p(x)
= 1 + 2 x + 3 x2 + : : : +
n
n
x
1
that interpolates f at the n evenly spa ed interpolation nodes i 1 b a 8i = 1; 2; : : : ; n: x =a+ n 1 In pra ti e, however, polynomial interpolation at evenly spa ed nodes often does not produ e an a
urate approximant. In fa t, there are well-behaved fun tions for whi h polynomial approximants with evenly spa ed nodes rapidly deteriorate, rather than improve, as the degree of approximation n rises. Numeri al analysis theory and empiri al experien e both suggest that polynomial approximants over a bounded interval [a; b℄ should be onstru ted by interpolating the underlying fun tion at the so- alled Cheby hev nodes: i
xi
=
a+b
2
+
b
a
2
os
n
i + 0:5 n
;
8i = 1; 2; : : : ; n:
As illustrated in Figure 6.1 for n = 9, the Cheby hev nodes are not evenly spa ed. They are more losely spa ed near the endpoints of the interpolation interval and less so near the enter.
CHAPTER 6.
132
FUNCTION APPROXIMATION
Chebychev Nodes 1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 6.1 Cheby hev-node polynomial interpolants possess some strong theoreti al properties. A
ording to Rivlin's Theorem, Cheby hev-node polynomial interpolants are very nearly optimal polynomial approximants. Spe i ally, the approximation error asso iated with the nth -degree Cheby hev-node polynomial interpolant annot be larger than 2 log(n) + 2 times the lowest error attainable with any other polynomial approximant of the same order. For n = 100, this fa tor is approximately 30, whi h is very small when one onsiders that other polynomial interpolation s hemes typi ally produ e approximants with errors that are orders of magnitude, that is, powers of 10, larger then the optimum. In pra ti e, the a
ura y a orded by the Cheby hevnode polynomial interpolant is often mu h better than indi ated by Rivlin's bound, espe ially if the fun tion being approximated is smooth. Another theorem, Ja kson's theorem, provides a more useful result. Spe i ally, if f is ontinuously di erentiable, then the approximation error a orded by the nth degree Cheby hev-node polynomial interpolant pn an be bounded above: jjf pnjj n6 jjf jj(b a)(log(n)= + 1): This error bound an often be a
urately estimated in pra ti e, giving the analyst a good indi ation of the a
ura y a orded by the Cheby hev-node polynomial in0
CHAPTER 6.
133
FUNCTION APPROXIMATION
terpolant. More importantly, however, the error bound goes to zero as rises. In
ontrast to polynomial interpolation with evenly spa ed nodes, one an a hieve any desired degree of a
ura y with Cheby hev-node polynomial interpolation by in reasing the degree of approximation. To illustrate the di eren e between Cheby hev and evenly spa ed node polynomial interpolation, onsider approximating the fun tion ( ) = exp( ) on the interval [ 1 1℄. The approximation error asso iated with ten node polynomial interpolants are illustrated in Figure 6.2. The Cheby hev node polynomial interpolant exhibits errors that os illate fairly evenly throughout the interval of approximation, a ommon feature of Cheby hev node interpolants. The evenly spa ed node polynomial interpolant, on the other hand, exhibits signi ant instability near the endpoints of the interval. The Cheby hev node polynomial interpolant avoids su h endpoint instabilities be ause the nodes are more heavily on entrated near the endpoints. n
f x
x
;
−9
4
x 10
Approximation Error for exp(−x) Chebychev Nodes Uniform Nodes
3.5
3
2.5
y
2
1.5
1
0.5
0
−0.5
−1
0
x
Figure 6.2 The most intuitive basis for expressing polynomials, regardless of the interpolation nodes hosen, is the monomial basis onsisting of the simple power fun tions 1 2 3 , illustrated in Figure 6.3 for the interval 2 [0 1℄. However, the monomial basis produ es an interpolation matrix that is a so- alled Vandermonde ; x; x ; x ; : : :
x
;
CHAPTER 6.
134
FUNCTION APPROXIMATION
matrix:
2 66 = 64
3
n 2 n 1 1 1 1 1 n 2 n 1 7 1 2 2 2 ... ... . . . ... ... 775 n 2 n 1 1 n n n Vandermonde matri es are notoriously ill- onditioned, and in reasingly so as the degree of approximation is in reased. Thus, e orts to ompute the basis oeÆ ients of the monomial basis polynomials often fail due to rounding error, and attempts to
ompute in reasingly more a
urate approximations by raising the number of interpolation nodes are often futile. x
:::
x
x
x
:::
x
x
:
x
:::
x
x
n
Monomial Basis Functions 1
1
0
1
0 0
1
1
0 0
1
1
0 1
1
1
1
1
0
1
1
0 0
0
0 0
1
0
1
1
0 0
0
0 0
1
Figure 6.3 Fortunately, alternatives to the standard monomial basis exist. In fa t, any sequen e of polynomials having exa t orders 0 1 2 1 an serve as a basis for all polynomials of order less than . One su h basis for the interval [ ℄ on the real line is the Cheby hev polynomial basis. De ning = 2( )( ) 1, to normalize the domain to the interval [-1,1℄, the Cheby hev polynomials are de ned n
;
;
;:::;n
n
a; b
z
x
a = b
a
CHAPTER 6.
135
FUNCTION APPROXIMATION
re ursively as:2
j (x) = Tj 1 (z ) where
T0 (z ) T1 (z ) T2 (z ) T3 (z )
= = = = .. .
1
z
2z 2 4z 3
1 3z
Tj (z ) = 2zTj 1 (z )
Tj 2 (z ):
The rst twelve Cheby hev basis polynomials for the interval x 2 [0; 1℄ are displayed in Figure 6.4. Chebychev Polynomial Basis Functions 1
1
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
1
0
1
0
1
0
1
0
1
Figure 6.4 2 The
Cheby hev polynomials also possess the alternate trigonometri de nition
os ar
os(z )j on the domain [a; b℄.
( ) =
Tj z
CHAPTER 6.
FUNCTION APPROXIMATION
136
Cheby hev polynomials are an ex ellent basis for onstru ting polynomials that interpolate fun tion values at the Cheby hev nodes. Cheby hev basis polynomials in
ombination with Cheby hev interpolation nodes yields an extremely well- onditioned interpolation equation that an be a
urately and eÆ iently solved, even with high degree approximants. The interpolation matrix asso iated with the Cheby hev interpolation has typi al element ij = os((n i + 0:5)(j 1)=n): This Cheby hev interpolation matrix is orthogonal > = diagfn; n=2; n=2; : : : ; n=2g p and has a ondition number 2 regardless of the degree of interpolation, whi h is very near the ideal minimum of 1. This implies that the Cheby hev basis oeÆ ients
an be omputed qui kly and a
urately, regardless of the degree of interpolation. Derivatives and integrals of polynomials are also polynomials. Di erentiation de reases the polynomial order by 1 and integration in reases it by 1. A di erential operator that maps the oeÆ ients of a polynomial in the Chebyshev basis is given by the n 1 n matrix operator with ij th element given by
8 > > < Dij = > > :
2(j
if i = 1 and i + j is odd if i > 1, i + j is odd and i > > > > > < => > > > > > :
b a 2
b a 8
(
1)j (b
2j (j
a) 2)
b a 4j b a 4(j 2)
0
if i 2 and j = 1 if i = 1 and j = 2 if i = 1 and j > 2 if i > 1 and j = i 1 if i > 1 and j = i + 1 otherwise.
maps Chebyshev oeÆ ients into the oeÆ ients of the integral (normalized so the integral is 0 at a).
CHAPTER 6.
6.3
137
FUNCTION APPROXIMATION
Pie ewise Polynomial Splines
Pie ewise polynomial splines, or simply splines for short, are a ri h, exible lass of fun tions that may be used instead of high degree polynomials to approximate a real-valued fun tion over a bounded interval. Generally, an order k spline onsists of series of kth order polynomial segments spli ed together so as to preserve ontinuity of derivatives of order k 1 or less. The points at whi h the polynomial pie es are spli ed together, 1 < 2 < : : : < p , are alled the breakpoints of the spline. By onvention, the rst and last breakpoints are the endpoints of the interval of approximation [a; b℄. A general order k spline with p breakpoints may be hara terized by (p 1)(k +1) parameters, given that ea h of the p 1 polynomial segments is de ned by its k + 1
oeÆ ients. By de nition, however, a spline is required to be ontinuous and have
ontinuous derivatives up to order k 1 at ea h of the p 2 interior breakpoints, whi h imposes k(p 2) onditions. Thus, an order k spline with p breakpoints is a tually hara terized by n = (k + 1)(p 1) k(p 2) = p + k 1 free parameters. It should not be surprising that a general order k spline with p breakpoints an be written as a linear ombination of n = p + k 1 basis fun tions. There are many ways to express bases for splines, but for applied numeri al work the most useful are the so- alled B-splines. The B-splines for an order k spline with breakpoint ve tor an be omputed using the re ursive de nition j +1 x x j k k 1; Bj 1 (x) + B k 1; (x); Bjk; (x) = j j k j +1 j +1 k j for i = 1; : : : ; n, with the re ursion starting with B 0; (x) =
j
1 0
if j x < j +1 otherwise.
This de nition requires that we extend the breakpoint ve tor, , for j < 1 and j > p: j =
a if j b if j
1 p:
Additionally, at the endpoints we set the terms B0k 1; Bnk 1; = = 0: 1 1 k n+1 n k+1 Given a B-spline representation of a spline, the spline an easily be di erentiated by omputing simple di eren es, and an be integrated by omputing simple sums. Spe i ally: dBjk; (x) dx
=
j
k j
k
Bjk 11; (x)
k j +1 j +1
k
Bjk 1; (x)
CHAPTER 6.
and
Zx a
Bjk;
FUNCTION APPROXIMATION
(z )dz =
n X i
i k
i=j
k
138
+1; (x): Bik+1
Although these formulae appear a bit ompli ated, their appli ation in omputer programs is relatively straightforward. First noti e that the derivative of a B-spline of order k is a weighted sum of two order k 1 B-splines. Thus, the derivative of an order k spline is an order k 1 spline with the same breakpoints. Similarly, the integral of a B-spline an be represented as the sum of B-splines of order k + 1. Thus, the antiderivative of an order k spline is an order k + 1 spline with the same breakpoints. This implies that the family of splines are losed under di erentiation and integration, with the oder k de reasing or in reasing by 1 and with the breakpoints remaining un hanged. Two lasses of splines are often employed in pra ti e. A rst-order or linear spline is a series of line segments spli ed together to form a ontinuous fun tion. A thirdorder or ubi spline is a series of ubi polynomials segments spli ed together to form a twi e ontinuously di erentiable fun tion. Linear spline approximants are parti ularly easy to onstru t and evaluate in pra ti e, whi h explains their widespread popularity. Linear splines use line segments to onne t points on the graph of the fun tion to be approximated. A linear spline with n evenly spa ed breakpoints on the interval [a; b℄ may be written as a linear
ombination f^(x) =
n X i=1
i (x) i
of the basis fun tions:
(
j (x) =
1 0
jx
j h
j
if jx j j h otherwise.
Here, h = (b a)=(n 1) is the distan e between breakpoints and j = a + (j 1)h, j = 1; 2; : : : ; n, are the breakpoints. The linear spline basis fun tions are popularly
alled the \hat" fun tions, for reasons that are lear from Figure 6.5. This gure illustrates the basis fun tions for a degree 12, evenly spa ed breakpoint linear spline on the interval [0; 1℄. Ea h hat fun tion is zero everywhere, ex ept over a narrow support of width 2h. The basis fun tion a hieves a maximum of 1 at the midpoint of its support. One an x the oeÆ ients of an n-degree linear spline approximant for a fun tion f by interpolating its values at any n points of its domain, provided that the resulting
CHAPTER 6.
139
FUNCTION APPROXIMATION
Linear Spline Basis Functions 1
1
0
1
0 0
1
1
0 0
1
1
0 1
1
1
1
1
0
1
1
0 0
0
0 0
1
0
1
1
0 0
0
0 0
1
Figure 6.5 interpolation matrix is nonsingular. However, if the interpolation nodes x1 ; x2; : : : ; x are hosen to oin ide with the spline breakpoints 1; 2 ; : : : ; , then omputing the basis oeÆ ients of the linear spline approximant be omes a trivial matter. In this
ase (x ) equals one if i = j , but equals zero otherwise; that is, the interpolation matrix is simply the identity matrix and the interpolation equation redu es to the identity = y, where y is the ve tor of fun tion values at the interpolation nodes. The linear spline approximant of f when nodes and breakpoints oin ide thus takes the form n
n
i
j
f^(x) =
X n
i=1
i (x)f (xi ):
When interpolation nodes and breakpoints oin ide, no omputations other than fun tion evaluations are required to form the linear spline approximant. For this reason linear spline interpolation nodes in pra ti e are always hosen to be the spline's breakpoints. Evaluating a linear spline approximant and its derivative at an arbitrary point x is also straightforward. Sin e at most two basis fun tions are nonzero at any point, only
CHAPTER 6.
FUNCTION APPROXIMATION
140
two basis fun tion evaluations are required. Spe i ally, if i is the greatest integer less than 1 + (x a)=h, then x lies in the interval [i ; i+1 ℄. Thus, f^(x) = ((x i ) i+1 + (i+1 x) i )=h and f^0 (x) = ( i+1 i )=h: Higher order derivatives are zero, ex ept at the breakpoints, where they are unde ned. Linear splines are attra tive for their simpli ity, but have ertain limitations that often make them a poor hoi e for omputational e onomi appli ations. By onstru tion, linear splines produ e rst derivatives that are dis ontinuous step fun tions and se ond derivative that are zero almost everywhere. Linear spline approximants thus typi ally do a very poor job of approximating the rst derivative of a nonlinear fun tion and are in apable of approximating its se ond derivative. In some e onomi appli ations, the derivative represents a measure of marginality that is of as mu h interest to the analyst as the fun tion itself. The rst and maybe se ond derivatives of the fun tion also may be needed to solve for the root of the fun tion using Newtonlike method and in the ontinuous time dynami models en ountered in Chapters 10 and 11 are expressed in terms of se ond order di erential equations. Cubi spline approximants o er a higher degree of smoothness while retaining mu h of the exibility and simpli ity of linear spline approximants. Be ause ubi splines possess ontinuous rst and se ond derivatives, they typi ally produ e adequate approximations for both the fun tion and its rst and se ond derivatives. The basis fun tions for n-degree, evenly spa ed breakpoint ubi splines on the interval [a; b℄ are generated using the n 2 breakpoints j = a + h(j 1), j = 1; 2; : : : ; n 2, where h = nb a3 . Cubi spline basis fun tion generated with evenly spa ed breakpoints are nonzero over a support of width 4h. As su h, at any point of [a; b℄, at most four basis fun tions are nonzero. The basis fun tions for a degree 12, evenly spa ed breakpoint ubi spline on the interval [0; 1℄ are illustrated in Figure 6.6. Although spline breakpoints are often hosen to be evenly spa ed, this need not be the ase. Indeed, the ability to distribute breakpoints unevenly and to sta k them on top of one another adds onsiderably to the exibility of splines, allowing them to a
urately approximate a wide range of fun tions. In general, fun tions that exhibit wide variations in urvature are diÆ ult to approximate numeri ally with polynomials of high degree. With splines, however, one an often nesse urvature diÆ ulties by
on entrating breakpoints in regions displaying the highest degree of urvature. To illustrate the importan e of breakpoint lo ation, onsider the problem of forming a ubi spline approximant for Runge's fun tion 1 f (x) = 1 + 25x2
CHAPTER 6.
141
FUNCTION APPROXIMATION
Cubic Spline Basis Functions 1
1
0
1
0 0
1
1
0 0
1
1
0 1
1
1
1
1
0
1
1
0 0
0
0 0
1
0
1
1
0 0
0
0 0
1
Figure 6.6 on the interval x 2 [ 5; 5℄. Figure 6.7 displays two ubi spline approximations, one using thirteen evenly spa ed breakpoints, the other using thirteen breakpoints that luster around zero (the breakpoints are indi ated by `x' symbols). Figure 6.8 shows the asso iated approximation errors (note that the errors for the unevenly spa ed approximation have been multiplied by 100). In Figure 6.7 the unevenly spa ed breakpoints approximation lies almost on top of the a tual fun tion, whereas the even spa ing leads to signi ant errors, espe ially near zero. The gures learly demonstrate the power of spline approximations with good breakpoint pla ement. The pla ement of the breakpoints an also be used to a e t the ontinuity of the spline approximant and its derivatives. By sta king breakpoints on top of one another, we an redu e the smoothness at the breakpoints. Normally, an order k spline has ontinuous derivatives to order k 1 at the breakpoints. By sta king q breakpoints, we an redu e this to k q ontinuous derivatives at this breakpoint. For example, with two equal breakpoints, a ubi spline possesses a dis ontinuous se ond derivative at the point. With three equal breakpoints, a ubi spline possesses a dis ontinuous rst derivative at that point, that is, it exhibits a kink there. Sta king breakpoints is a useful pra ti e if the fun tion is known a priori to exhibit a kink at a
CHAPTER 6.
142
FUNCTION APPROXIMATION
Runge’s Function with Spline Approximations 1.2 Runge Even Spacing Uneven Spacing 1
0.8
y
0.6
0.4
0.2
0
−0.2 −5
−4
−3
−2
−1
0
1
2
3
4
5
x
Figure 6.7 given point. Kinks arise in the pri ing of options, whi h display a kink in their payo fun tion and in dynami optimization problems with dis rete hoi e variables, whi h display kinks in their marginal value fun tion (or its derivative). Regardless of the pla ement of breakpoints, splines have several important and useful properties. We have already ommented on the limited domain of the basis fun tion. This limited support implies that spline interpolation matri es are sparse and for this reason an be stored and manipulated using sparse matrix methods. This property is extremely useful in high-dimensional problems for whi h a fully expanded interpolation matrix would strain any omputer's memory. Another useful feature of splines is that their values are bounded, thereby redu ing the likelihood that s aling e e ts will ause numeri al diÆ ulties. In general, the limited support and bounded values make spline basis matri es well- onditioned. If the spline interpolation matrix must be reused, one must resist the temptation to form and store its inverse, parti ularly if the size of the matrix is large. Inversion destroys the sparsity stru ture. More spe i ally, the inverse of the interpolation matrix will be dense, even though the interpolation matrix is not. When n is large, solving the sparse n by n linear equation using sparse L-U fa torization will generally be less ostly than performing the matrix-ve tor multipli ation required with the
CHAPTER 6.
143
FUNCTION APPROXIMATION
Approximation Errors for Runge’s Function 0.6 Even Spacing Uneven Spacing 0.4
0.2
y
0
−0.2
−0.4
−0.6 Uneven spacing errors 100x −0.8 −5
−4
−3
−2
−1
0
1
2
3
4
5
x
Figure 6.8 dense inverse interpolation matrix (a
omplished with the \n" operator in 6.4
Matlab).
Pie ewise-Linear Basis Fun tions
Despite their simpli ity, linear splines have many virtues. For problems in whi h the fun tion being approximated is not-smooth and may even exhibit dis ontinuities, linear splines an still provide reasonable approximations. Unfortunately, derivatives of linear splines are dis ontinuous, pie ewise onstant fun tions. There is no reason, however, to limit ourselves to using the a tual derivative of the approximating fun tion, if a more suitable alternative exists. If a fun tion is approximated by a linear spline, a reasonable andidate for an approximation of its derivative is a linear spline onstru ted using nite di eren e approximations to the derivative (see Se tion 5.6, page 107). Given a breakpoint sequen e for the fun tion's approximant, this an be a
omplished by de ning a new breakpoint sequen e with n 1 values pla ed at the midpoints of the original sequen e: zi = (i + i+1 )=2, i = 1; : : : ; n 1. The new fun tion is set to equal the entered nite di eren e
CHAPTER 6.
FUNCTION APPROXIMATION
144
approximation to the derivative at the new breakpoints: f 0 (zi )
f (i+1) i+1
f (i ) : i
Values between and beyond the zi sequen e an be obtained by linear interpolation and extrapolation. We leave it as an exer ise to show that this pie ewise linear fun tion, evaluated at the original breakpoints (the i ), is equal to the entered nite di eren e approximations derived in the last hapter. Approximations to higher order derivatives an be obtained be repeated appli ation of this idea. For ompleteness, we de ne an approximate integral that is also a linear spline, with a breakpoint sequen e zi+1 = (i + i+1 )=2 for i = 2; : : : ; n and with additional breakpoints de ned by extrapolating beyond the original sequen e: z1 = (31 2 )=2 and zn+1 = (3n n 1 )=2. The approximation to the integral, Z x F (x) = f (x)dx 1
at the new breakpoints is F (zi ) = F (zi 1 ) + (zi
zi 1 )f (i 1 );
where F (z1 ) = 12 (1
2 )f (1 )
(this ensures the normalization that F (1 ) = 0).3 This de nition produ es an approximation to the integral at the original breakpoints that is equal to the approximation obtained by applying the trapezoid rule (see Se tion 5.1, page 95): Z +1 f (x)dx 12 (i+1 i )(f (i+1 ) + f (i )): i
i
(we leave the veri ation of this assertion as exer ise for the reader). As with the other families of fun tions dis ussed, the family of pie ewise linear fun tions obtained using these approximations is losed under di erentiation and integration. Unlike splines, however, for whi h di erentiation and integration de reases or in reases the order of the pie ewise segments, leaving the breakpoint sequen e 3 It
should be pointed out that the breakpoint sequen e obtain by integrating and then di erentiating will not produ e the original breakpoint sequen e unless the original breakpoints are evenly spa ed. This leads to the unfortunate property that di erentiating the integral will only produ e the original fun tion if the breakpoints are evenly spa ed. It an also be shown that, although the rst derivatives are O(h2 ), the se ond derivatives are only O(h) when the breakpoints are not evenly spa ed.
CHAPTER 6.
145
FUNCTION APPROXIMATION
un hanged, with the pie ewise linear family, di erentiation and integration do not
hange the polynomial order of the pie es (they remain linear) but de rease or in rease the number of breakpoints. The pie ewise linear family makes omputation using nite di eren e operators quite easy, without a need for spe ial treatment to distinguish them from other families of basis fun tions (in luding nite element families su h as splines). We will return to this point in Chapter 11 when we dis uss solving partial di erential equations (PDEs). 6.5
Multidimensional Interpolation
The univariate interpolation methods dis ussed in the pre eding se tions may be extended in a natural way to multivariate fun tions through the use of tensor produ ts. To illustrate, onsider the problem of approximating a bivariate real-valued fun tion f (x; y ) de ned on a bounded interval I = f(x; y ) j ax x bx ; ay y by g in 1 & ~isempty(tvals) x=funeval( , def,tvals); dx=funeval( , def,tvals,1); r=feval(model.fun ,'r',tvals,x,dx,model.params{:}); end
In addition to omputing the oeÆ ient matrix, , the pro edure is implemented to, optionally, take a ve tor of time values tvals and to return the solution and residual fun tions at those values (x and r). The solver instru ts the root nding algorithm broyden to nd the roots of the fun tion BVPRes, whi h in turn alls the model.fun le to ompute the residual and boundary fun tions. fun tion r=bvpres( ,model,fspa e,tnode,Phi,Phi1,tb,phi,phi1); n=size(Phi,2);
CHAPTER 6.
FUNCTION APPROXIMATION
169
m=length(tb);
=reshape( ,n,m); % Compute residuals at nodal values x=Phi* ; dx=Phi1* ; r=feval(model.fun ,'r',tnode,x,dx,model.params{:}); % Compute boundary onditions and on atenate to residuals x=phi* ; dx=phi1* ; b=feval(model.fun ,'b',tb,x,dx,model.params{:}); r=[r(:);b(:)℄;
The demonstration le dembvp01 ontains the ode to solve the example problem using Chebyshev polynomial approximants and plots both the approximation error fun tions and the residual fun tions. The pro edure solves in a single iteration of the root nding algorithm be ause it is a linear problem. An e onomi appli ation of these pro edures is illustrated next with a simple market equilibrium example. Example: Commodity Market Equilibrium
At time t = 0 there are available for onsumption S0 units of a periodi ally produ ed
ommodity. No more of the good will be produ ed until time t = 1, at whi h time all of the urrently available good must be onsumed. The hange in the level of the sto ks is the negative of the rate of onsumption, whi h is given by the demand fun tion, here assumed to be of the onstant elasti ity type: 0 s (t) = q = D ( p) = p : To prevent arbitrage and to indu e storage, the pri e must rise at a rate that overs the ost of apital, r and the physi al storage harges, C : 0 p (t) = rp + C: It is assumed that no sto ks are arried into the next produ tion y le, whi h begins at time t = 1; hen e the boundary ondition that s(1) = 0. This is a two variable system of rst order di erential equations with two boundary
onditions, one at t = 0 and the other at t = 1. De ning x = [p; s℄, the residual fun tion is 0 0 [rx + C r (t; x; x ) = x x ℄ 1 1
and the boundary onditions are x2 (0) S0 = 0 and x2 (1) = 0. The model stru ture an be reated using
CHAPTER 6.
170
FUNCTION APPROXIMATION
model.fun ='pbvp02'; model.tb=[0;1℄; model.params={A};
The problem de nition le pbvp02 for this problem is
fun tion out1=pbvp02(flag,t,x,dx,r,C,eta,S0); swit h flag
ase 'r' out1=dx-[r*x(:,1)+C -x(:,1).^(-eta)℄;
ase 'b' out1=x(:,2)-[S0;0℄; end
A demonstration le, dembvp02 is available that uses the parameters r = 0:10, C = 0:5, = 2 and S0 = 1. It approximates the solution using a degree n = 6 Chebyshev polynomial approximation. The resulting solution and residual fun tions are shown in Figures 6.19 and 6.20. It is evident in the latter that the approximation Equilibrium Price and Stock Level 1.4
1.2 price 1
0.8
0.6
stocks
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
t
Figure 6.19 a hieves a high degree of a
ura y even with a low order approximation; the maximum sizes of the pri e and sto ks residual fun tions are approximately 10 10 and 10 3 , respe tively.
CHAPTER 6.
FUNCTION APPROXIMATION
Residual Functions
−4
10
171
x 10
P’ x 106 S’ 8
6
4
2
0
−2
−4
−6
−8
0
0.1
0.2
0.3
0.4
0.5
0.6
t
Figure 6.20
0.7
0.8
0.9
1
CHAPTER 6.
172
FUNCTION APPROXIMATION
Exer ises 6.1. Constru t the 5- and 50-degree approximants for the fun tion f (x) = exp( x2 ) on the interval [ 1; 1℄ using ea h of the interpolation s hemes below. For ea h s heme and degree of approximation, estimate the sup norm approximation error by omputing the maximum absolute deviation between the fun tion and approximant at 201 evenly spa ed points. Also, graph the approximation error for the degree 5 approximant. (a) (b) ( ) (d)
Uniform node, monomial basis polynomial approximant Cheby hev node, Cheby hev basis polynomial approximant Uniform node, linear spline approximant Uniform node, ubi spline approximant
6.2. In the Cournot model ea h rm takes the output of the other rms as given when determining its output level. An alternative assumption is that ea h rm takes its ompetitiors' output de ision fun tions as given when making its own output hoi e. This an be expressed as the assumption that dp dqi
X dSj (p) dp 1 1+ D (p) dp dqi j =i
1 X dqj = = D (p) dqi j =1 n
0
0
!
:
6
Solving this for dp=dqi yields dp dqi
=
D
0
(p)
P1
j 6=i
0
( )
Sj p
:
In an industry with m identi al rms, ea h rm assumes the other rms will rea t in the same way it does, so this expression simpli es to dp dq
=
D
0
( p)
1
(m
1)S (p) 0
:
This expression di ers from the Cournot ase in the extra term in the denominator (whi h only equals 0 in the monopoly situation of m = 1). Noti e also that, unlike the Cournot ase, the rm's \supply" fun tion depends on the number of rms in the industry. Write a fun tion to solve this problem analogous to the one des ribed in Se tion 6.8.1 (page 159) and a demo le to produ e the analogous plots. The
CHAPTER 6.
FUNCTION APPROXIMATION
173
fun tion must take the parameters (in luding m, the industry size) and it must also ompute the derivative of the q = S (p) fun tion to ompute the residual fun tion. 6.3. Consider the potato market model dis ussed in the Chapter 3 (page 60). Constru t a 5th degree Cheby hev polynomial approximant for the fun tion relating the period 1 pri e to initial supply s over the interval s 2 [1; 3℄. Interpolate the polynomial at s = 1, s = 2, and s = 3 and ompare to the interpolated values to those obtained earlier. 6.4. Consider again the potato market model. Assume now that supply s is the produ t of a reage a and yield y where yield an a hieve one of two equiprobable out omes, a low yield 0:75 and a high yield 1:25, and that a reage is a fun tion of the pri e expe ted in the harvest period: a
= 0:5 + 0:5E [p1 ℄:
The rational expe tations equilibrium a reage level and expe ted pri e satisfy the a reage supply fun tion and [ ℄ = 0:5f (0:75a) + 0:5f (1:25a)
E p1
where f is the fun tion approximated in the pre eding problem. Compute the rational expe tations equilibrium of the model using the 10th degree Cheby hev polynomial approximation for f omputed in the pre eding problem. 6.5. Using ollo ation with the basis fun tions of your hoi e and without using BVPSOLVE numeri ally solve the following di erential equation for x 2 [0; 1℄: (1 + x2 )v (x)
v
00 (x) = x2 ;
with v (0) = v (1) = 0. Plot the residual fun tion to ensure that the maximum value of the residual is less than 1e-8. What degree of approximation is needed to a hieve this level of a
ura y. 6.6. Lifetime Consumption A simple model of lifetime savings/ onsumption hoi e onsiders an agent with a proje ted in ome ow by w(t), who must hoose a onsumption rate (t) to maximize dis ounted lifetime utility: Z T t max e U (C (t))dt C (t)
0
CHAPTER 6.
FUNCTION APPROXIMATION
174
subje t to an intertemporal wealth onstraint dW=dt = rW + w(t) C , where r is the rate of return on investments (or the interest rate on borrowed funds, if W < 0). The solution to this optimal ontrol problem an be expressed as the system of di erential equations C0 =
U 0 (C ) (r U 00 (C )
)
and W 0 = rW + w(t) C:
It is assumed that the agent begins with no wealth (W (0) = 0) and leaves no bequests (W (T ) = 0). a) Use BVPSOLVE to solve this BVP using the CARA utility fun tion U (C ) = (C 1 1)=(1 ) and the parameters values T = 45, r = 0:1, = 0:6, = 0:5 and w(t) = w0=(1+ e t), with w0 = 1 and = 0:15. Plot the solution fun tion and the residual fun tions. b) In part (a) the agent works until time T and then dies. Suppose, instead, that the agent retires at time T and lives an additional R = 20 retirement years with no additional in ome (w(t) = 0 for T < t T + R). Resolve the problem with this assumption. What additional problem is en ountered? How an the problem be addressed? 6.7. The omplementary Normal CDF is de ned as (x)
Z1 1 e =p 2 x
z 2 =2 dz:
De ne u(x) = ex =2 (x): 2
(a) Express u as a di erential equation with boundary ondition u(1) = 0. (b) Use the hange of variable t = x=(K + x) (for some onstant K) to de ne a di erential equation for the fun tion v(t) = u(x), for v 2 [0; 1℄. ( ) Write a Matlab fun tion to solve this di erential equation using ollo ation with Chebyshev polynomials. Do this by writing the ollo ation equations in the form B = f , where is an n-ve tor of oeÆ ients. Then solve this linear system dire tly.
CHAPTER 6.
FUNCTION APPROXIMATION
175
(d) Plot the residual fun tion for a range of values of K between 0.1 and 20. Make a re ommendation about the best hoi e of K .
Matlab
6.8. Write a fun tion that automates the approximation of fun tion inverses. The fun tion should have the following syntax: fun tion =finverse(f,fspa e,varargin)
You will also need to write an auxiliary fun tion to ompute the appropriate residuals used by the root nding algorithm.
CHAPTER 6.
FUNCTION APPROXIMATION
176
Bibliographi Notes Most introdu tory texts on numeri al analysis ontain some dis ussion of interpolation via Chebyshev polynomials and splines; see, for example, Press et al., or, for a dis ussion fo used on solving di erential equations, see Golub and Ortega (Chapter 6). Collo ation is one of a more general lass of approximation methods known as weighted residual methods. The general idea of weighted residual methods is to nd an approximate that minimizes the residual fun tion for some fun tional norm. In addition to ollo ation, two ommon approa hes of this general lass are least squares methods, whi h (for the simple fun tional equation problem, solve: Z b min r2 (x; (x) )dx
a
and Galerkin methods (also alled Bubnov-Galerkin methods), whi h solve Z b r(x; (x) ) i (x)dx = 0; for i = 1; : : : ; n: a
When the integrals in these expressions an be solved expli itly, they seem to be somewhat more eÆ ient than ollo ation, espe ially the Galerkin approa h. Unless r has a
onvenient stru ture, however, these methods will ne essitate the use of some kind of dis retization to ompute the ne essary integrals, redu ing any potential advantages these methods may have relative to ollo ation.
Chapter 7 Dis rete Time Dis rete State Dynami Models With this hapter, we begin our study of dynami e onomi models. Dynami e onomi models often present three ompli ations rarely en ountered together in dynami physi al s ien e models. First, humans are ogent, future-regarding beings
apable of assessing how their a tions will a e t them in the future as well as in the present. Thus, most useful dynami e onomi models are future-looking. Se ond, many aspe ts of human behavior are unpredi table. Thus, most useful dynami e onomi models are inherently sto hasti . Third, the predi table omponent of human behavior is often omplex. Thus, most useful dynami e onomi models are inherently nonlinear. The ompli ations inherent in forward-looking, sto hasti , nonlinear models make it impossible to obtain expli it analyti solutions to all but a small number of dynami e onomi models. However, the proliferation of a ordable personal omputers, the phenomenal in rease of omputational speed, and developments of theoreti al insights into the eÆ ient use of omputers over the last two de ades now make it possible for e onomists to analyze dynami models mu h more thoroughly using numeri al methods. The next three hapters are devoted to the numeri al analysis of dynami e onomi models in dis rete time and are followed by three hapters on dynami e onomi models in ontinuous time. In this hapter we study the simplest of these models: the dis rete time, dis rete state Markov de ision model. Though the model is simple, the methods used to analyze the model lay the foundations for the methods developed in subsequent hapters to analyze more ompli ated models with ontinuous states and time.
177
CHAPTER 7.
7.1
178
DISCRETE STATE MODELS
Dis rete Dynami Programming
The dis rete time, dis rete state Markov de ision model has the following stru ture: in every period t, an agent observes the state of an e onomi pro ess st , takes an a tion xt , and earns a reward f (xt ; st ) that depends on both the state of the pro ess and the a tion taken. The state spa e S , whi h enumerates all the states attainable by the pro ess, and the a tion spa e X , whi h enumerates all a tions that may be taken by the agent, are both nite. The state of the e onomi pro ess follows a
ontrolled Markov probability law. That is, the distribution of next period's state,
onditional on all urrently available information, depends only on the urrent state of the pro ess and the agent's a tion:
j
j
Pr(st+1 = s0 xt = x; st = s; other information at t ) = P (s0 x; s):
f g
The agent seeks a poli y xt Tt=1 that pres ribes the a tion xt = xt (st ) that should be taken in ea h state at ea h point in time so as to maximize the present value of urrent and expe ted future rewards over time, dis ounted at a per-period fa tor Æ (0; 1℄: " T # X t max E Æ f (xt ; st ) : fxt gT t=0 t=0
2
1
A dis rete Markov de ision model may have an in nite horizon (T = ) or a nite horizon (T < ). The model may also be either deterministi or sto hasti . It is deterministi if next period's state is known with ertainty on e the urrent period's state and a tion are known. In this ase, it is bene ial to dispense with the probability transition law as a des ription of how the state evolves and use instead a deterministi state transition fun tion g , whi h expli itly gives the state transitions:
1
st+1
= g (xt ; st ):
Dis rete Markov de ision models may be analyzed and understood using the dynami programming prin iples developed by Ri hard Bellman (1956). Dynami programming is an analyti approa h in whi h a multiperiod model is e e tively de omposed into a sequen e two period models. Dynami programming is based on the Prin iple of Optimality, whi h was arti ulated by Bellman as follows: \An optimal poli y has the property that, whatever the initial state and de ision are, the remaining de isions must onstitute an optimal poli y with regard to the state resulting from the rst de ision."
CHAPTER 7.
179
DISCRETE STATE MODELS
The Prin iple of Optimality an be formally expressed in terms of the value fun tions Vt . For ea h period t and state s, Vt (s) spe i es the maximum attainable sum of urrent and expe ted future rewards, given that the pro ess is in state s and the
urrent period is t. Bellman's Prin iple implies that the value fun tions must satisfy Bellman's re ursion equation X Vt (s) = max f (x; s) + Æ P (s0 x; s)Vt+1 (s0 ) s S: x2X (s) s 2S Bellman's equation aptures the essential problem fa ed by a dynami , futureregarding optimizing agent: the need to balan e the immediate reward f (xt ; st ) with expe ted present value of future rewards ÆEt Vt+1 (st+1 ). Given the value fun tions, the optimal poli ies xt (s) are simply the solutions to the optimization problems embedded in Bellman's equation. In a nite horizon model, we adopt the onvention that the optimizing agent fa es de isions up to and in luding a nal de ision period T < . The agent fa es no de isions after the terminal period T , but may earn a nal reward VT +1 (sT +1 ) in the subsequent period that depends on the realization of the state in that period. The terminal value is typi ally xed by some e onomi ally relevant terminal ondition. In many appli ations, VT +1 is identi ally zero, indi ating that no rewards are earned by the agent beyond the terminal de ision period. In other appli ations, VT +1 may spe ify a salvage value earned by the agent after making his nal de ision in period T. For the nite horizon dis rete Markov de ision model to be well posed, the terminal value VT +1 must be spe i ed by the analyst. Given the terminal value fun tion, the nite horizon de ision model in prin iple may be solved re ursively by repeated appli ation of Bellman's equation: having VT +1 , solve for VT (s) for all states s; having VT , solve for VT 1 (s) for all states s; having VT 1 , solve for VT 2 (s) for all states s; and so on. The pro ess ontinues until V0 (s) is derived for all states s. Be ause only nitely many a tions are possible, the optimization problem embedded in Bellman's equation an always be solved by performing nitely many arithmeti operations. Thus, the value fun tions of a nite horizon dis rete Markov de ision model are always well-de ned, although in some ases more than one poli y of state- ontingent a tions may yield the maximum expe ted stream of rewards, that is, the optimal a tion may not be unique. If the de ision problem has an in nite horizon, the value fun tions will not depend on time t. We may, therefore, disregard the time subs ripts and denote the ommon value fun tion by V . Bellman's equation therefore be omes the ve tor xed-point equation
f
0
j
g 2
1
"
V (s)
= max x2X (s)
f (x; s) + Æ
X
s 2S 0
j
#
P (s0 x; s)V (s0 ) ;
s
2 S:
CHAPTER 7.
DISCRETE STATE MODELS
180
If the dis ount fa tor Æ is less than one, the mapping underlying Bellman's equation is a strong ontra tion. The Contra tion Mapping Theorem thus guarantees the existen e and uniqueness of the in nite horizon value fun tion.1 7.2
E onomi Examples
Spe i ation of a dis rete Markov de ision model requires several pie es of information: the state spa e, the a tion spa e, the reward fun tion, the state transition fun tion or state transition probabilities, the dis ount fa tor Æ , the time horizon T , and, if the model has nite horizon, the terminal value VT +1 . This se tion provides seven e onomi examples that illustrate how the ne essary information is spe i ed and how the Bellman equation is formulated. 7.2.1
Mine Management
A mine operator must determine the optimal ore extra tion s hedule for a mine that will be shut down and abandoned after T years of operation. The pri e of extra ted ore is p dollars per ton and the total ost of extra ting x tons of ore in any year is
= x2 =(1+ s) dollars, where s is the tons of ore remaining in the mine at the beginning of the year. The mine urrently ontains s tons of ore. Assuming the amount of ore extra ted in any year must be an integer number of tons, what extra tion s hedule maximizes pro ts? This is a nite horizon, deterministi model with time t = f1; 2; : : : ; T g measured in years. The state variable s
2 S = f0; 1; 2; : : : ; sg
denotes tons of ore remaining in the mine at the beginning of the year. The a tion variable x
2 X (s) = f0; 1; 2; : : : ; sg
denotes tons of ore extra ted over the year. The state transition fun tion is s0
= g (s; x) = s
x:
The reward fun tion is f (s; x) = px x2 =(1 + s): 1 Value
fun tions in in nite horizon problems ould be time dependent if f , P , or Æ displayed time dependen e. However, this reates diÆ ulties in developing solution methods, and we have hosen not to expli itly onsider this possibility. Fortunately, most in nite horizon e onomi model do not display su h time dependen e.
CHAPTER 7.
181
DISCRETE STATE MODELS
The value of the mine, given it ontains s tons of ore at the beginning of year t, satis es Bellman's equation V (s) = max fpx x2 =(1 + s) + ÆV +1 (s x)g; s2S 012 t
t
x2f ; ; ;:::;sg
subje t to the terminal ondition VT +1 (s)
7.2.2
= 0;
s
2 S:
Asset Repla ement - I
At the beginning of ea h year, a manufa turer must de ide whether to ontinue operating with an aging physi al asset or repla e it with a new one. An asset that is a years old yields a pro t ontribution p(a) up to n years, after whi h the asset be omes unsafe and must be repla ed by law. The ost of a new asset is . What repla ement poli y maximizes pro ts? This is an in nite horizon, deterministi model with time t = f1; 2; : : : ; T g measured in years. The state variable
2 A = f1; 2; : : : ; ng
a
denotes the age of the asset in years. The a tion variable
x
keep; repla eg 2 X (a) = ffrepla e g
a 1 p t r 1 2 1 q = 2 + 2 2 Æ = exp( r t): Here, r is the annualized interest rate, ontinuously ompounded, is the annualized volatility of the ommodity pri e, and t is the length of a period in years. Assuming the urrent pri e of the ommodity is p0, what is the value of an Ameri an put option if it has a strike pri e p and if it expires T years from today? This is a nite horizon, sto hasti model where time t 2 f0; 1; 2; : : : ; N g is measured in periods of length t = T =N years ea h. The state is2 p = ommodity pri e p 2 S = fp1 u ji = N 1; N; : : : ; N; N + 1g: The a tion is x = de ision to keep or exer ise x 2 X = fkeep; exer iseg; the state transition probability rule is 8 p = pu < q 1 q p = p=u P (p jx; p) = : 0 otherwise the reward fun tion is 0 x = keep f (p; x) = p p x = exer ise i
0
0
0
2 In this example, we alter our notation to onform with standard treatments of option valuation. Thus, the state is the pri e, denoted by
p, the number of time periods until expiration is N , and T
reserved for the time to expiration (in years).
CHAPTER 7.
184
DISCRETE STATE MODELS
The value fun tion Vt (p)
= option value at t, if ommodity pri e is p,
must satisfy Bellman's equation Vt (p)
= maxf p
p; qÆVt+1 (pu) + (1
q )ÆVt+1 (p=u)
g
subje t to the post-terminal ondition VN +1 (p)
=0
Note that if the option is exer ised, the owner re eives p p. If he does not exer ise the option, however, he earns no immediate reward but will have an option in hand the following period worth Vt+1 (pu) with probability q and Vt+1 (p=u) with probability 1 q . In option expires in the terminal period, making it valueless the following period; as su h, the post-terminal salvage value is zero. 7.2.5
Job Sear h
At the beginning of ea h week, an in nitely-lived worker nds himself either employed or unemployed and must de ide whether to be a tive in the labor market over the
oming week by working, if he is employed, or by sear hing for a job, if he is unemployed. An a tive employed worker earns a wage w. An a tive unemployed worker earns an unemployment bene t u. An ina tive worker earns a psy hi bene t v from additional leisure, but no in ome. An unemployed worker that looks for a job will nd one with probability p by the end of the week. An employed worker that remains at his job will be red with probability q at the end of the week. What is the worker's optimal labor poli y? This is a in nite horizon, sto hasti model with time t = f1; 2; : : : ; 1g measured in weeks. The state is s s
= employment state 2 S = funemployed(0); employed(1)g
and the a tion is x x
= labor for e parti ipation de ision 2 X = fina tive(0); a tive(1)g:
CHAPTER 7.
185
DISCRETE STATE MODELS
The state transition probability rule is
j
P (s0 s; x)
=
8 1 > > > 1 > < p q
> > > : 10
= 0; s = 0 = 1; s = 0; s = 1; s = 0; s = 1; s = 1; s = 1; s = 1; s otherwise;
x p x x x q x
0
0
0
0
0
=0 =1 =0 =1
(ina tive worker) (sear hes, nds no job) (sear hes, nds job) (works, loses job) (works, keeps job)
and the reward fun tion is
8 0:
Renewable Resour e Problem
A so ial planner wishes to maximize the dis ounted sum of net so ial surplus from harvesting a renewable resour e over an in nite horizon. For year t, let st denote the resour e sto k at the beginning of the year, let xt denote the amount of the resour e harvested, let t = (xt ) denote the total ost of harvesting, and let pt = p(xt ) denote the market learing pri e. Growth in the sto k level is given by st+1 = g (st xt ). What is the so ially optimal harvest poli y? This is an in nite horizon, deterministi model with time t 2 f0; 1; 2; : : :g measured in years. There is one state variable, st = sto k of resour e at beginning of year t st 2 [0; 1);
and one a tion variable, xt = amount of resour e harvested in year t,
subje t to the onstraint 0 xt
st :
The net so ial surplus is Z xt p( ) d (xt ): 0
CHAPTER 7.
188
DISCRETE STATE MODELS
State transitions are governed by st+1
= g (st
xt ) :
The value fun tion, whi h gives the net so ial value of resour e sto k, satis es Bellman's equation V (s) 7.2.9
Zx
= max f 0xs 0
p( ) d
(x) + ÆV (g (s
g
x)) :
Bioe onomi Model
In order to survive, an animal must forage for food in one of m distin t areas. In area x, the animal survives predation with probability px , nds food with probability qx , and, if it nds food, gains ex energy units. The animal expends one energy unit every period and has a maximum energy arrying apa ity s. If the animal's energy sto k drops to zero, it dies. What foraging pattern maximizes the animal's probability of surviving T years to reprodu e at the beginning of period T + 1? This is a nite horizon, sto hasti model with time t = f1; 2; : : : ; T g measured in foraging periods. The state is s s
= sto k of energy 2 S = f0; 1; 2; : : : ; sg;
the a tion is x x
= foraging area 2 X = f1; 2; : : : ; mg:
The state transition probability rule is, for s = 0, 0 1 s =0 (death is permanent) 0 P (s js; x) = 0 otherwise; and, for s > 0,
8 px q x > > < px (1 qx ) P (s0 js; x) = (1 px ) > > : 0
The reward fun tion is f (s; x)
= 0:
s0 s0 s0
= min(s; s =s 1 =0 otherwise.
1 + ex ) (survive, nds food) (survive, no food) (does not survive)
CHAPTER 7.
189
DISCRETE STATE MODELS
Here, s = 0 is an absorbing state that, on e entered, is never exited. More to the point, an animal whose energy sto ks fall to zero dies, and remains dead. The reward fun tion for periods 1 through T is zero, be ause there is only one payo , surviving to pro reate, and this payo is earned in period T + 1. The value fun tion ( ) = probability of pro reating, given energy sto ks s in period t
Vt s
must satisfy Bellman's equation ( ) = maxfpx qx Vt+1 (min(s; s x2X
Vt s
1 + e)) + px (1
) +1 (s
qx Vt
1)g;
for t 2 1; : : : ; T , with Vt (0) = 0, subje t to the terminal ondition VT
7.3
+1 (s) =
0 1
=0 s > 0 s
Solution Algorithms
Below, we develop numeri al solution algorithms for sto hasti dis rete time, dis rete spa e Markov de ision models. The algorithms apply to deterministi models as well, provided one views a deterministi model as a degenerate spe ial ase of the sto hasti model for whi h the transition probabilities are all zeros or ones. To develop solution algorithms, we must introdu e some ve tor notation and operations. Assume that the states S = f1; 2; : : : ; ng and a tions X = f1; 2; : : : ; mg are indexed by the rst n and m integers, respe tively. Let v 2
0 =) 0;
x 0 (0), the model admits only one steady state: s = x = 0, = p 0 (0), and = (1 Æ ) . That is, the mine will be worked until its
ontents are depleted. Until su h time that the ontent of the mine is depleted, pt
= 0t + Æt
t
= Æt+1 :
where pt is the market pri e and 0t is the marginal ost of extra tion. That is, the market pri e of extra ted ore equals the shadow pri e of unextra ted ore plus the marginal ost of extra tion. Also, the urrent-valued shadow pri e of unextra ted ore will grow at the rate of interest, or equivalently, the present-value shadow pri e will remain onstant. 8.3.4
Optimal Water Management
A water planner wishes to maximize so ial bene ts derived from the water olle ted in a reservoir. The water may be used either for irrigation or re reation. Irrigation during the spring bene ts agri ultural produ ers, but redu es the reservoir level during the summer, damaging re reational users. Spe i ally, if s is the water level at the beginning of spring and an amount x is released for irrigation, produ er bene ts will be a(x) and re reational user bene ts will be u(s x). Water levels are replenished during the winter months by i.i.d. random rainfalls , implying that the reservoir level at the beginning of ea h year is a ontrolled Markov pro ess st+1
= st
xt
+ t+1 :
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
232
What irrigation poli y maximizes the sum of urrent and expe ted future bene ts to produ ers and users ombined? This is an in nite horizon, sto hasti model with one state variable, the reservoir water level at beginning of the year s, and one hoi e variable, the amount of water released for irrigation x, whi h is subje t to the onstraint 0 x s: The urrent reward is the sum of produ er and user bene ts f (s; x) = a(x) + u(s x): The value of the water in the reservoir satis es Bellman's equation V (s) = max fa(x) + u(s x) + ÆEV (s x + )g: 0xs Assuming a0 (0) and u0 (0) are suÆ iently large, the onstraints will not be binding at an optimal solution and the shadow pri e of water (s) will satisfy the Euler equilibrium onditions a0 (x) u0 (s x) ÆE(s x + ) = 0 (s) = u0 (s x) + ÆE(s x + ): It follows that along the optimal path a0t = t = u0t + ÆEt+1 where a0t and u0t are the marginal produ er and user bene ts, respe tively. Thus, on the margin, the bene t re eived by produ ers this year from releasing one unit of water must equal the marginal bene t re eived by users this year from retaining the unit of water plus the bene ts of having that unit available for either irrigation or re reation the following year. The ertainty-equivalent steady-state water level s , irrigation level x , and shadow pri e solve the equation system x = a0 (x ) = u0 (s x ) = (1 Æ )a0 (x ) where is mean annual rainfall. These onditions imply that the ertainty-equivalent steady-state irrigation level and shadow pri e of water are not a e ted by the interest rate. The ertainty-equivalent steady-state reservoir level, however, is a e ted by the interest rate. Totally di erentiating the above equation system and making reasonable assumptions about the urvature of the bene t fun tions: Æ 2 a0 s = < 0: r
00
u
That is, as the interest rate rises, the ertainty-equivalent steady-state reservoir level falls.
CHAPTER 8.
8.3.5
CONTINUOUS STATE MODELS: THEORY
233
Optimal Monetary Poli y
A monetary authority wishes to ontrol the nominal interest rate x in order to minimize the volatility of the in ation rate s1 and the gross domesti produ t (GDP) gap s2. Spe i ally, the authority wishes to minimize expe ted dis ounted stream of weighted squared deviations from zero targets 1 L(s) = s> s 2 where s is a 2x1 ve tor ontaining the in ation rate and the GDP gap and is a 2x2
onstant positive de nite matrix of weights. The in ation rate and the GDP gap are a joint ontrolled linear Markov pro ess s +1 = + s + x + where and are 2x1 onstant ve tors, is a 2x2 onstant matrix, and is a 2x1 random ve tor with mean zero. For politi al reasons, the nominal interest rate x annot be negative. What monetary poli y minimizes the sum of urrent and expe ted future losses? This is an in nite horizon, sto hasti model with two state variables, the in ation rate s1 and the GDP gap s2, and one hoi e variable, the nominal in ation rate x, whi h is subje t to the onstraint x 0: In order to formulate this problem as a maximization problem, one may posit a reward fun tion that equals the negative of the loss fun tion f (s) = L(s): Given this assumption, the sum of urrent and expe ted future rewards V (s) satis es Bellman's equation V (s) = max f L(s) + ÆEV (g(s; x; ))g: 0 t
t
x
Given the stru ture of the model, one annot pre lude the possibility that the nonnegativity onstraint on the optimal nominal interest rate will be binding in ertain states. As su h, the shadow pri e fun tion (s) is hara terized by the Euler
onditions Æ >E(g (s; x; ) = (s) = s + Æ >E(g (s; x; )) where the nominal interest rate x and the long-run marginal reward from in reasing the nominal interest rate must satisfy the omplementarity ondition x 0; 0; x > 0 =) = 0:
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
234
It follows that along the optimal path Æ >Et t+1 = t t = st + Æ >Et+1 xt 0; t 0; xt > 0 =) t = 0: Thus, in any period, the nominal interest rate is redu ed until either the long-run marginal reward or the nominal interest rate is driven to zero. 8.3.6
Produ tion-Adjustment Model
A ompetitive pri e-taking rm wishes to manage produ tion so as to maximize long run pro ts, given that produ tion is subje t to adjustment osts. In parti ular, if the rm produ es a quantity q, it in urs produ tion osts (q) and adjustment osts 0:5a(q l)2, where l is the pre eding period's (lagged) produ tion. The rm an sell any quantity it produ es at the prevailing market pri e, whi h is an exogenous Markov pro ess pt+1 = g (pt ; t+1 ): What produ tion poli y maximizes the value of the rm? This is an in nite horizon, sto hasti model with two state variables, the urrent market pri e p and lagged produ tion l, and one hoi e variable, produ tion q, whi h is subje t to the nonnegativity onstraint q 0. The urrent reward, short-run pro ts, equals revenue less produ tion and adjustment osts f (p; l; q ) = p q (q ) 0:5a(q l)2 : The value V (p; l) of the rm, given the market pri e p and the previous period's produ tion l, satis es Bellman's equation V (p; l) = maxfp q (q ) a(q l) + ÆEV (g (p; ); q )g: 0q
Assuming a positive optimal produ tion level in all states, the shadow pri e of lagged produ tion (p; l) will satisfy the Euler equilibrium onditions p 0 (q ) a0 (q l) + ÆE(g (p; ); q ) = 0 (p; l) = a0 (q l): It follows that along the optimal path pt = 0t + (a0t ÆEa0t+1 )
CHAPTER 8.
235
CONTINUOUS STATE MODELS: THEORY
where 0 and a0 are the marginal produ tion and adjustment osts in period t. Thus, pri e equals the marginal ost of produ tion plus the net ( urrent less future) marginal adjustment ost. The ertainty-equivalent steady-state produ tion q is obtained by assuming p is xed at its long-run mean p: t
t
p = 0 (q ) + (1
8.3.7
Æ )a0 (0):
Produ tion-Inventory Model
A ompetitive pri e-taking rm wishes to manage produ tion and inventories so as to maximize long-run pro ts. The rm begins ea h period with a predetermined sto k of inventory s and de ides how mu h to produ e q and how mu h to store x, buying or selling the resulting di eren e s + q x on the open market at the prevailing pri e p. The rm's produ tion and storage osts are given by (q ) and k (x), respe tively, and the market pri e follows a purely exogenous Markov pro ess pt+1
= g (s ; +1 ): t
t
What produ tion poli y and inventory poli y maximizes the value of the rm? This is an in nite horizon, sto hasti model with two state variables, the urrent market pri e p and beginning inventories s, and two hoi e variables, produ tion q and ending inventories x, both of whi h are subje t to nonnegativity onstraints q 0 and x 0. The urrent reward, short-run pro ts, equals net revenue from marketing sales or pur hases, less produ tion and storage osts: f (p; s; q; x)
= p( s + q
x)
(q )
k (x):
The value V (p; s) of the rm, given market pri e satis es Bellman's equation V (p; s)
= max 0 0 q;
fp(s + q
x
x)
(q )
p
and beginning inventories s,
g
k (x) + ÆEV (g (p; ); x) :
If produ tion is subje t to in reasing marginal osts and 0 (0) is suÆ iently small, then produ tion will be positive in all states and the shadow pri e of beginning inventories (p; s) will satisfy the Euler equilibrium onditions: p
= 0 (q )
ÆE(g (p; ); x) (p; s)
=p
p
k 0 (x)
=
CHAPTER 8.
x
0;
236
CONTINUOUS STATE MODELS: THEORY
0;
x>
0 =) = 0:
It follows that along the optimal path, p = 0 t
t
xt
0;
Et pt+1
pt
kt0
0;
x>
0 =) Et pt+1
pt
kt0
= 0:
where pt denotes the market pri e, 0t denotes the marginal produ tion ost, and kt0 denotes the marginal storage ost. Thus, the rm's produ tion and storage de isions are independent. Produ tion is governed by the onventional short-run pro t maximizing ondition that pri e equal the marginal ost of produ tion. Storage, on the other hand, is entirely driven by intertemporal arbitrage pro t opportunities. If the expe ted marginal pro t from storing is negative, then no storage is undertaken. Otherwise, sto ks are a
umulated up to the point at whi h the marginal ost of storage equals the present value expe ted appre iation in the market pri e. The ertainty-equivalent steady-state obtains when p is xed at its long-run mean p, in whi h ase no appre iation an take pla e and optimal inventories will be zero. The ertainty-equivalent steady-state produ tion is impli itly de ned by the short-run pro t maximization ondition. 8.3.8
Optimal Feeding
An livesto k produ er feeds his sto k up to period T and then sells it at the beginning of period T + 1 at a xed pri e p per unit weight. Ea h period, the produ er must determine how mu h grain x to feed his livesto k, given that grain sells at a onstant unit ost . The weight of the livesto k at the beginning of ea h period is a ontrolled rst-order deterministi pro ess st+1
= g (st ; xt ):
What feeding poli y maximizes pro t, given that the weight of the livesto k in the initial period, t = 0, is s? This is an nite horizon, deterministi model with one state variable, the livesto k weight at beginning of the period s 2 [s; 1), and one hoi e variable, the amount of feed pur hased x 2 [0; 1), whi h is subje t to the onstraint x 0: The value of livesto k weighing s in period t satis es Bellman's equation Vt (s)
= maxf x0
subje t to the terminal ondition VT +1 (s)
= ps:
g
x + ÆVt+1 (g (s; x)) ;
CHAPTER 8.
237
CONTINUOUS STATE MODELS: THEORY
If the marginal weight gain g at zero feed is suÆ iently large, the nonnegativity
onstraint of feed will never be binding. Under these onditions, the shadow pri e of livesto k weight in period t, (s), will satisfy the Euler equilibrium onditions: x
t
Æt+1 (g (s; x))gx(s; x) t (s)
=
= Æ +1 (g (s; x))g (s; x) t
s
subje t to the terminal ondition T +1 (s)
= p:
It follows that along the optimal path Æt+1 gx;t t
=
= Æ +1 g t
s;t
where g and g represent, respe tively, the marginal weight gain from feed and the marginal de line in the livesto k's ability to gain weight as it grows in size. Thus, the ost of feed must equal the value of the marginal weight gain. Also, the present valued shadow pri e grows at a rate that exa tly ounters the marginal de line in the livesto k's ability to gain weight. x;t
8.4
s;t
Linear-Quadrati Control
The linear-quadrati ontrol problem is an un onstrained Markov de ision model with a quadrati reward fun tion f (s; x) = F0 + F s + F x + 0:5s> F s + s> F x + 0:5x> F x s
x
ss
sx
xx
and a linear state transition fun tion g (s; x; )
= G0 + G s + G s
x
x + :
Here, s is an n-by-1 state ve tor, x is an m-by-1 a tion ve tor, F0 is a known onstant, is a known 1-by-n ve tor, F is a known 1-by-m ve tor, F is a known n-by-n matrix, F is a known n-by-m matrix, F is a known m-by-m matrix, G0 is a known n-by-1 ve tor, G is a known n-by-n matrix, and G is a known n-by-m ve tor. Without loss of generality, the sho k is assumed to have a mean of zero. The linear-quadrati ontrol model is of spe ial importan e be ause it is one of the few dis rete time ontinuous state Markov de ision models with a nite-dimensional solution. By a on eptually simple but algebrai ally burdensome indu tion proof Fs
x
sx
ss
xx
s
x
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
238
omitted here, one an show that the optimal poli y and shadow pri e fun tions of the in nite horizon linear-quadrati ontrol model are both linear in the state variable: x(s)
=
(s)
= 0 + s s:
0
+
ss
Here, 0 is an m-by-1 ve tor, s is an m-by-n matrix, 0 is an n-by-1 ve tor, and s is an n-by-n matrix. The parameters 0 and s of the shadow pri e fun tion are hara terized by the nonlinear ve tor xed point Ri
ati equations [ÆGs >s Gx + Fsx℄[ÆGx >s Gx + Fxx >℄ 1 [ÆGx >[s G0 + 0 ℄ + Fx >℄ +ÆGs> [sG0 + 0 ℄ + Fs > s = [ÆGs >s Gx + Fsx℄[ÆGx >s Gx + Fxx >℄ 1 [ÆGx >sGs + Fsx >℄ +ÆGs> sGs + Fss:
0 =
These nite-dimensional xed-point equations an typi ally be solved in pra ti e using fun tion iteration. The re ursive stru ture of these equations allow one to rst solve for s by applying fun tion iteration to the se ond equation, and then solve for 0 by applying fun tion iteration to the rst equation. On e the parameters of the shadow pri e fun tion have been omputed, one an ompute the parameters of the optimal poli y via algebrai operations: 0
= [ÆGx >sGx + Fxx >℄ 1 [ÆGx>[s G0 + 0 ℄ + Fx >℄
s
= [ÆGx >s Gx + Fxx>℄ 1 [ÆGx >sGs + Fsx >℄
The relative simpli ity of the linear-quadrati ontrol problem derives from the fa t that the optimal poli y and shadow pri e fun tions are known to be linear, and thus belong to a nite dimensional family. The parameters of the linear fun tions, moreover, are hara terized as the solution to a well-de ned nonlinear ve tor xedpoint equation. Thus, the apparently in nite-dimensional Euler fun tional xedpoint equation may be onverted into nite-dimensional ve tor xed-point equation and solved using standard nonlinear equation solution methods. This simpli ation, unfortunately, is not generally possible for other types of dis rete time ontinuous state Markov de ision models. A se ond simplifying feature of the linear-quadrati ontrol problem is that the shadow pri e and optimal poli y fun tions depend only on the mean of the state sho k, but not its varian e or higher moments. This is known as the ertainty-equivalen e property of the linear-quadrati ontrol problem. It asserts that the solution of the
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
239
sto hasti problem is the same as the solution of the deterministi problem obtained by xing the state sho k at its mean of zero. Certainty equivalen e also is not a property of more general dis rete time ontinuous state Markov de ision models. Be ause linear-quadrati ontrol models are relatively easy to solve, many analysts
ompute approximate solutions to more general Markov de ision models using the method of linear-quadrati approximation. Linear quadrati approximation alls for all onstraints of the general problem to be dis arded and for its reward and transition fun tions to be repla ed with by their se ond- and rst-order approximations about the steady-state. This approximation method, whi h is illustrated in the following
hapter, works well in some instan es, for example, if the state transition rule is linear,
onstraints are non-binding or non-existent, and if the sho ks have relatively small variation. However, in most e onomi appli ations, linear-quadrati approximation will often render highly ina
urate solutions that di er not only quantitatively but also qualitatively from the true solution. For this reason, we strongly dis ourage the use of linear-quadrati approximation, ex ept in those ases where the assumptions of the linear quadrati model are known to hold globally, or very nearly so. 8.5
Dynami Games
Dynami game models attempt to apture strategi intera tions among a small number of dynami ally optimizing agents when the a tions of one agent a e ts the welfare of the others. To simplify notation, we onsider only in nite horizon games. The theory and methods developed, however, an be easily adapted to a
ommodate nite horizons. The dis rete time ontinuous state Markov m-agent game has the following stru ture: In every period, ea h agent i observes the state of an e onomi pro ess s 2 S , takes an a tion xi 2 X , and earns a reward fi (s; xi ; x i ) that depends on the state of the pro ess and both the a tion taken by the agent and the a tions taken by the m 1 other agents x i . The state of the e onomi pro ess is a jointly ontrolled Markov pro ess. Spe i ally, the state of the e onomi pro ess in period t + 1 will depend on the state in period t, the a tions taken by all m agents in period t, and an exogenous random sho k t+1 that is unknown in period t: st+1
= g (st ; xt ; t+1 ):
As with stati games, the equilibrium solution to a Markov game depends on the information available to the agents and the strategies they are assumed to pursue. We will limit dis ussion to non ooperative Markov perfe t equilibria, that is, equilibria that yield a Nash equilibrium in every proper subgame. Under the assumption that ea h agent an perfe tly observe the state of the pro ess and knows the poli ies
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
240
followed by the other agents, a Markov perfe t equilibrium is a set of m poli ies of state- ontingent a tions xi : S 7! X , i = 1; 2; : : : ; m, su h that poli y xi maximizes the present value of agent i's urrent and expe ted future rewards, dis ounted at a per-period fa tor Æ , given that other agents pursue their poli ies x i (). That is, for ea h agent i, xi () solves T X max E Æ t f (s ; x (s ); x i (st )) fx()g 0 t=0 i t i t The Markov perfe t equilibrium for the m-agent game is hara terized by a set of m simultaneous Bellman equations Vi (s) = max fi (s; x; x i (s)) + ÆE Vi (g (s; x; x i (s); )) : x2X (s) whose unknowns are the value fun tions Vi () and optimal poli ies xi (), i = 1; 2; : : : ; m of the di erent agents. Here, Vi (s) denotes the maximum urrent and expe ted future rewards that an be earned by agent i, given that other agents pursue their optimal strategies. i
8.5.1
Capital-Produ tion Game
Consider two in nitely-lived rms that produ e perishable goods that are lose substitutes (say, donuts and bagels). Ea h rm i begins period t with a predetermined
apital sto k ki and must de ide how mu h to produ e qi . Its produ tion ost i (qi ; ki ) depends on both the quantity produ ed and the apital sto k. Pri es are determined by short-run market learing onditions (Cournot ompetition). More spe i ally, rm i re eives pri e pi = Pi (q1 ; q2 ) that depends both on its output and the output of its ompetitor. The rm must also de ide how mu h to invest in apital. Spe i ally, if the rm invests in new apital xi , it in urs a ost hi (xi ) and its apital sto k at the beginning of the following period will be (1 )ki + xi where is the apital depre iation rate. What are the two rm's optimal produ tion and investment poli ies? This is an in nite horizon, deterministi 2-agent dynami game with two state variables, the apital sto ks of the two produ ers, k1 and k2 . Ea h agent i has two de ision variables, produ tion qi and investment xi , whi h are subje t to the nonnegativity onstraints qi 0 and xi 0. His urrent reward, net revenue, equals Pi (q1 ; q2 )qi
i (qi ; ki ) hi (xi ). The Markov perfe t equilibrium for the produ tion apital game is represented by a pair of Bellman equations, one for ea h rm, whi h take the form ^1 ; k^i)g Vi (k1 ; k2 ) = max fPi (q1 ; q2 )qi i (qi ; ki ) hi (xi ) + ÆE Vi (k q 0;x 0 where k^i = (1 )ki + xi . i
i
CHAPTER 8.
8.5.2
CONTINUOUS STATE MODELS: THEORY
241
Risk-Sharing Game
Consider two in nitely-lived agents who must make onsumption-investment de isions. Ea h period, ea h agent i begins with a predetermined level of wealth si , of whi h an amount xi is invested, and the remainder is onsumed, yielding an utility ui (si xi ). Agent i's wealth at the beginning of period t + 1 is determined entirely by his investment in period t and an in ome sho k t+1 , whi h is unknown at the time the investment de ision is made. More spe i ally, wealth follows a ontrolled Markov pro ess sit+1
= gi (xit ; it+1 ):
Suppose now that the two agents o-insure against exogenous in ome risks by agreeing to share their wealth in perpetuity. Spe i ally, the agents agree that, at the beginning of any given period t, the wealthier of the two agents will transfer a
ertain proportion of the wealth di erential to the poorer agent. Under this s heme, agent i's wealth in period t + 1, after the transfer, will equal sit+1
= (1
)gi (xit ; it+1 ) + gj (xjt ; jt+1 ):
where j 6= i. If the wealth transfer is enfor eable, but agents are free to onsume and invest freely, moral hazard will arise. In parti ular, both agents will have in entives to shirk investment in favor of urrent onsumption when o-insured. How will insuran e a e t the agents' behavior, and for what initial wealth states s1 and s2 and share parameter will both agents be willing to enter into the insuran e ontra t? How does the orrelation in the wealth sho ks a e t the value of the insuran e ontra t? This is an in nite horizon, sto hasti 2-agent dynami game with two state variables, the wealth levels of the two agents s1 and s2 . Ea h agent i has a single de ision variable, his investment xi , whi h is subje t to the onstraint 0 xi s^i . His urrent period reward, urrent utility, equals ui (^si xi ). The Markov perfe t equilibrium for the redistribution game is represented by a pair of Bellman equations, one for ea h agent, whi h take the form Vi (s1 ; s2 )
= max fui(^si 0x ^s i
i
g
xi ) + ÆE Vi (^ s1 ; s^2 ) ;
where s^i = (1 )gi (xi ; i ) + gj (xj ; j ). Here, Vi (s1 ; s2 ) denotes the maximum expe ted lifetime utility that an be obtained by agent i. 8.5.3
Marketing Board Game
Suppose that two ountries are the sole produ ers of a ommodity and that, in ea h
ountry, a government marketing board has the ex lusive power to sell the ommodity
CHAPTER 8.
242
CONTINUOUS STATE MODELS: THEORY
on the world market. The marketing boards ompete with ea h other, attempting to maximize the present value of their own urrent and expe ted future in ome from
ommodity sales. More spe i ally, the marketing board in ountry i begins ea h period with a pre-determined supply si of the ommodity, of whi h it exports a quantity qi and stores the remainder si qi at a total ost i (si qi ). The world market pri e will depend on the total amount exported by both ountries, p = p(q1 + q2 ). The supplies available in the two ountries at the beginning period t + 1 are given by sit+1
= xit + yit
where new produ tion in both ountries, y1t and y2t , are assumed to be exogenous and independently and identi ally distributed over time. What are the optimal export strategies for the two marketing boards? This is an in nite horizon, sto hasti 2-agent dynami game with two state variables, the beginning supplies in the two ounties s1 and s2 . The marketing board for ountry i has a single de ision variable, the export level qi , whi h is subje t to the onstraint 0 xi si . Country i's urrent reward, net in ome, equals p(q1 + q2 )qi
i (si qi ). The Markov perfe t equilibrium for the marketing board game is aptured by a pair of Bellman equations, one for ea h marketing board, whi h take the form Vi (s1 ; s2 )
= max fp(q1 + q2 )qi 0qi si
i (si
qi ) + ÆEy Vi (s1
q1
+ y 1 ; s2
q2
+ y2 )g:
Here, Vi (s1 ; s2 ) denotes the maximum urrent and expe ted future in ome that an be earned by marketing board i, given that marketing board j remains ommitted to its export poli y. 8.6
Rational Expe tations Models
We now examine dynami sto hasti models of e onomi systems in whi h arbitragefree equilibria are enfor ed through the olle tive, de entralized a tions of atomisti dynami ally optimizing agents. We assume that agents are rational in the sense that their expe tations are onsistent with the impli ations of the model as whole. Examples of phenomenon that may be studied in a rational expe tations framework in lude asset returns in a pure ex hange e onomy, pri ing of primary ommodities, and agri ultural produ tion subje t to pri e ontrols. We limit attention to dynami models of the following form: At the beginning of period t, an e onomi system emerges in a state st . Agents observe the state of the system and, by pursuing their individual obje tives, produ e a systemati response xt
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
243
governed by an equilibrium ondition that depends on expe tations of the following period's state and a tion f (st ; xt ; Et h(st+1 ; xt+1 ))
= 0:
The e onomi system then evolves to a new state st+1 that depends on the urrent state st and response xt , and an exogenous random sho k t+1 that is realized only after the system responds at time t: st+1
= g (st ; xt ; t+1 ):
In many appli ations, the equilibrium ondition f = 0 admits a natural arbitrage interpretation. In these instan es, fi > 0 indi ates a tivity i generates pro ts on the margin, so that agents have a olle tive in entive to in rease xi ; fi < 0 indi ates that a tivity i generates loses on the margin, so that agents have a olle tive in entive to de rease xi . An arbitrage-free equilibrium exists if and only if f = 0. The state spa e S 2 0. The unit ost of storage is
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
253
k dollars per period; due to spoilage, a proportion of inventories held at the
beginning of one period do not survive to the following period. The rm's initial inventories are x0 where 0 < x0 < Q. The rm wishes to minimize the present value of the ost of meeting its ontra tual obligation; assume a dis ount fa tor Æ < 1. (a) Identify the state and a tion variables, the reward fun tion, and the transition fun tion of this problem. (b) Write Bellman's fun tional equation. What does the value fun tion represent? ( ) Derive the Euler onditions and interpret them. What does the shadow pri e fun tion represent? (d) Assuming in reasing marginal ost, > 0, qualitatively des ribe the optimal produ tion plan. (e) Assuming de reasing marginal ost, < 0, qualitatively des ribe the optimal produ tion plan. 00
00
8.12. A subsisten e farmer grows and eats a single rop. Produ tion, yt , depends on how mu h seed is on hand at the beginning of the year, kt , a
ording to yt = kt where 0 < < 1. The amount kept for next year's seed is the di eren e between the amount produ ed and the amount onsumed, t :
kt+1 = yt
t :
The farmer has a time-additive logarithmi utility fun tion and seeks to maximize
XÆ T
t=0
t ln(
t ):
subje t to having an initial sto k of seed, k0 . (a) Identify the state and a tion variables, the reward fun tion, and the transition fun tion of this problem. (b) Write Bellman's fun tional equation. What does the value fun tion represent? ( ) Derive the Euler onditions and interpret them. What does the shadow pri e fun tion represent?
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
254
(d) Show that the value fun tion has the form (kt ) = A + B ln(kt )
V
and that the optimal de ision rule for this problem is
+1 = C yt ;
kt
nd the values for A,
B
, and C .
8.13. A rm ompetes in a mature industry whose total pro t is a xed amount X every year. If the rm aptures a fra tion pt of total industry sales in year t, it makes a pro t pt X . The fra tion of sales aptured by the rm in year t is a fun tion pt = f (pt 1 ; at 1 ) of the fra tion it aptured the pre eding year and its advertising expenditures the pre eding year, at 1 . Find the advertising poli y that maximizes the rm's dis ounted pro ts over a xed time horizon of T years. Assume p0 and a0 are known. (a) Identify the state and a tion variables, the reward fun tion, and the transition fun tion of this problem. (b) Write Bellman's fun tional equation. What does the value fun tion represent? ( ) Derive the Euler onditions and interpret them. What does the derivative of value fun tion represent? (d) What onditions hara terize the steady-state optimal solution? 8.14. A orn produ er's net per-a re revenue in year t is given by = pyt
t
xt
wlt
where p is the unit pri e of orn ($/bu.), yt is the orn yield (bu./a re), is the unit ost of fertilizer ($/lb.), xt is the amount of fertilizer applied (lbs./a re), w is the wage rate ($/man-hour), and lt is the amount of labor employed (manhours/a re). The per-a re rop yield in year t is a fun tion yt
= f (lt ; xt ; st )
of the amount of labor employed and fertilizer applied in year t and the level of fertilizer arryin st from the pre eding year. Fertilizer arryout in year t is a fun tion
+1 = f (xt ; st )
st
CHAPTER 8.
CONTINUOUS STATE MODELS: THEORY
255
of the amount of fertilizer applied and the level of fertilizer arryin in year t. Assume that future orn pri es, fertilizer osts, and wage rates are known with ertainty. The orn produ er wishes to maximize the expe ted present value of net revenues over a nite horizon of T years. Formulate the produ er's optimization problem. Spe i ally, (a) Identify the state and a tion variables, the reward fun tion, and the transition fun tion of this problem. (b) Write Bellman's fun tional equation. What does the value fun tion represent? ( ) Derive the Euler onditions and interpret them. What does the derivative of value fun tion represent? (d) What onditions hara terize the steady-state optimal solution? 8.15. The role of ommodity storage in intertemporal allo ation has often been ontroversial. In parti ular, the following laims have often been made: a) Competitive storers, in sear h of spe ulative pro ts, tend to hoard a ommodity|that is, they olle tively store more than is so ially optimal, and b) A monopolisti storer tends to dump a ommodity at rst in order to extra t monopoly rents in the future|that is, he/she stores less than is so ially optimal. Explore these two propositions in the ontext of a simple intraseasonal storage model in whi h a given amount Q of a ommodity is to be allo ated between two periods. Consumer demand is given by pi = a qi for periods i = 1; 2, and the unit ost of storage between periods is k. There is no new produ tion in period 2, so q1 + q2 = Q. Spe i ally, answer ea h of the following: (a) Determine the amount stored under the assumption that there are a large number of ompetitive storers. (b) Determine the amount stored under the assumption that there is a single pro t-maximizing storer who owns the entire supply Q at the beginning of period 1. ( ) Taking expe ted total onsumer surplus less storage osts as a measure of so ietal welfare, determine the so ially optimal level of storage. Address the two omments above. (d) Consider an E onomist who reje ts net total surplus as a measure of so ial welfare. Why might he/she still wish to nd the level of storage that maximizes total surplus? To simplify the analysis, assume that the dis ount fa tor is 1 and that the storer(s) are risk neutral and possess perfe t pri e foresight.
CHAPTER 8.
256
CONTINUOUS STATE MODELS: THEORY
8.16. Consider an industry of identi al pri e taking rms. For the representative rm, let s denote beginning apital sto k, let x denote newly pur hased apital sto k, let q = f (s + x ) denote produ tion, let k denote the unit ost of new
apital, and let > 0 denote the survival rate of apital. Furthermore, let p = p(q ) be the market learing pri e. Find the perfe t foresight ompetitive equilibrium for this industry. t
t
t
t
t
t
t
8.17. Show that the ompetitive storage model in se tion 8.6.2 an be formulated with the equilibrium storage fun tion as the sole unknown. Hint: Write the arbitrage storage ondition in the form f (s ; x ; E h(x +1 )) = 0 for some appropriately de ned fun tion h. t
t
t
t
8.18. Show that the ompetitive storage model of se tion 8.6.2 an be re ast as a dynami optimization problem. In parti ular, formulate a dynami optimization problem in whi h a hypotheti al so ial planner maximizes the dis ounted expe ted sum of onsumer surplus less storage osts. Derive the Euler onditions to show that, under a suitable interpretation, they are identi al to the rational expe tations equilibrium onditions of the storage model. 8.19. Consider the produ tion-inventory model of se tion 8.3.7. Show that the value fun tion is of the form V (p; s) = ps + W (p) where W is the solution to a Bellman fun tional equation. Can you derive general onditions under whi h one an redu e the dimensionality of a Bellman equation? 8.20. Consider the monetary poli y model of se tion 8.3.5. Derive the ertaintyequivalent steady-state in ation rate, GDP gap, nominal interest rate, and shadow pri es under the simplifying assumption that the nominal interest rate is un onstrained. 8.21. Demonstrate that the problem
" V (s ; d ; p ) = max E0 t
t
t
t 2
[0
;dt st
℄
X
#
1
=0
Æ U ( ) s.t. s +1 = s + (d s t
t
t
leads to the Euler ondition ÆE [U ( +1 )(p +1 + d +1 )℄ = U ( )p : 0
t
0
t
t
t
t
t
t
t
t
t
)=p : t
t
Chapter 9 Dis rete Time Continuous State Dynami Models: Methods This hapter dis usses numeri al methods for solving dis rete time ontinuous state dynami e onomi models. Su h models give rise to fun tional equations whose unknowns are entire fun tions de ned on a subset of Eu lidean spa e. For example, the unknown of Bellman's equation V (s)
= max ff (s; x) + ÆEV (g (s; x; ))g 2
x X (s)
is the value fun tion
ondition
V ().
And the unknown of a rational expe tations equilibrium
f (s; x(s); E h(g (s; x(s); ); x(g (s; x(s); ))))
=0
is the response fun tion x(). In most appli ations, these fun tional equations la k known losed form solution and an only be solved approximately using omputational methods. Among the omputational methods available, linear-quadrati approximation and spa e dis retization histori ally have been popular among e onomists due to the relative ease with whi h they an be implemented. However, in most appli ations, these methods either provide una
eptably poor approximations or are omputationally ineÆ ient. In re ent years, e onomists have begun to experiment with proje tion methods pioneered by physi al s ientists. Among the various proje tion methods available, the
ollo ation method is the most useful for solving dynami models in E onomi s and Finan e. In most appli ations, the ollo ation method is exible, a
urate, and numeri ally eÆ ient. It an also be developed dire tly from basi numeri al integration, approximation, and root nding te hniques.
257
CHAPTER 9.
CONTINUOUS STATE MODELS: METHODS
258
The ollo ation method employs a on eptually straightforward strategy to solve fun tional equations. Spe i ally, the unknown fun tion is approximated using a linear ombination of n known basis fun tions whose n oeÆ ients are xed by requiring the approximant to satisfy the fun tional equation, not at all possible points of the domain, but rather at n pres ribed points alled the ollo ation nodes. The ollo ation method e e tively repla es an in nite-dimensional fun tional equation with a nite-dimensional nonlinear equation that an be solved using standard numeri al root nding, xed-point, and omplementarity te hniques. Unfortunately, the widespread appli ability of the ollo ation method to e onomi and nan ial models has been hampered by the absen e of publi ly available general purpose omputer ode. We address this problem by developing omputer routines that perform the essential omputations for a broad lass of dynami e onomi and nan ial models. Below, the ollo ation method is developed in greater detail for single- and multiple-agent de ision Bellman equations and rational expe tations models. Appli ation of the method is illustrated with a variety of examples. 9.1
Traditional Solution Methods
Before dis ussing ollo ation methods for ontinuous state Markov de ision models in greater detail, let us brie y examine the two numeri al te hniques that histori ally have been popular among e onomists for omputing approximate solutions to su h models: spa e dis retization and linear-quadrati approximation. Spa e dis retization alls for the ontinuous state Markov de ision model to be repla ed with a dis rete state and a tion de ision model that losely resembles it. The resulting dis rete state and a tion model is then solved using the dynami programming methods dis ussed in Chapter 7. To \dis retize" the state spa e of a ontinuous state Markov de ision problem, one partitions the state spa e S into nitely many regions, S1 ; S2 ; : : : ; Sn . If the a tion spa e X is also ontinuous, it too is partitioned into nitely many regions X1 ; X2 ; : : : ; Xm . On e the spa e and a tion spa es have been partitioned, the analyst sele ts representative elements, si 2 Si and xj 2 Xj , from ea h region. These elements serve as the state and a tion spa es of the approximating dis rete state dis rete a tion Markov de ision problem. The transition probabilities of the dis rete state dis rete a tion spa e problem are omputed by integrating with respe t to the density of the random sho k:
j
P (si0 si ; xj )
= Pr[g (si; xj ; ) 2 Si ℄: 0
When the state and a tion spa es are bounded intervals on the real line, say, S = [smin ; smax ℄ and X = [xmin ; xmax ℄, it is often easiest to partition the spa es so that the nodes are equally-spa ed and the rst and nal nodes orrespond to the endpoints
CHAPTER 9.
259
CONTINUOUS STATE MODELS: METHODS
of the intervals. Spe i ally, set si = smin + (i 1)ws and xj = xmin + (j 1)wx, for i = 0; 1; : : : ; n and j = 0; 1; : : : ; m, where ws = (smax smin )=(n 1) and wx = (xmax xmin )=(m 1). If the model is sto hasti , the transition probabilities of the approximating dis rete state de ision model are given by
j
P (si0 si ; xj )
= Pr[si
0
ws =2
g(si; xj ; ) si
0
+ ws =2℄:
Another popular method for solving dynami optimization models is linear-quadrati approximation. Linear-quadrati approximation alls for the state transition fun tion g and obje tive fun tion f to be repla ed with linear and quadrati approximants, respe tively. Linear-quadrati approximation is motivated by the fa t that an un onstrained Markov de ision problem with linear transition and quadrati obje tive has a losed-form solution that is relatively easy to derive numeri ally. Typi ally, the linear and quadrati approximants of g and f are onstru ted by forming the rst- and se ond-order Taylor expansions around the ertainty-equivalent steady-state. When passing to the linear-quadrati approximation, any onstraints on the a tion, in luding nonnegativity onstraints, must be dis arded. The rst step in deriving an approximate solution to a ontinuous state Markov de ision problem via linear-quadrati approximation is to ompute the ertaintyequivalent steady-state. If denotes the mean sho k, the ertainty-equivalent steadystate state s , optimal a tion x , and shadow pri e are hara terized by the nonlinear equation system: fx (s ; x ) + Æ gx (s ; x ; )
=0
= fs (s ; x ) + Æ gs(s ; x ; )
s
= g (s ; x ; ):
Typi ally, the nonlinear equation may be solved for the steady-state values of s , x , and using standard nonlinear equation methods. In one-dimensional state and a tion models, the onditions an often be solved analyti ally. Here, fx, fs , gx , and gs denote partial derivatives whose dimensions are 1 m, 1 n, n m, and n n, respe tively, where n and m are the dimensions of the state and a tion spa es, respe tively. Here, the ertainty-equivalent steady-state shadow pri e is expressed as a 1 n row ve tor. The se ond step is to repla e the state transition fun tion g and the reward fun tion f , respe tively, with their rst- and se ond-order Taylor series approximants expanded around the ertainty-equivalent steady-state: f (s; x)
f + fs(s +(s
s ) + fx (x
s )> fsx (x
x ) + 0:5(s
x ) + 0:5(x
s )> fss (s
x )> fxx (x
x )
s )
CHAPTER 9.
g (s; x; )
CONTINUOUS STATE MODELS: METHODS
g + gs(s
s ) + gx (x
260
x ):
, and f are the values and partial derivatives Here, f , g , fs , fx , gs, gx , fss , fsx xx of f and g evaluated at the ertainty-equivalent steady-state. If n and m are the dimensions of the state and a tion spa es, respe tively, then the orders of these ve tors and matri es are as follows: f is a onstant, fs is 1 n, fx is 1 m, fss is n n, is n m, f is m m, g is n 1, g is n n, and g is n m. fsx xx s x The shadow pri e and optimal poli y fun tions of the resulting linear-quadrati
ontrol problem will be linear. Spe i ally: x(s)
= x + (s
s )
(s)
= + (s
s ):
The slope matri es of the poli y and shadow pri e fun tions, and , are hara terized by the nonlinear ve tor xed point equations ℄[Æg >g + f >℄ 1 [Æg >g + f >℄ + Æg >g + f = [Ægs >gx + fsx x x xx x s sx s s ss >℄ 1 [Æg >g + f >℄: = [Ægx >gx + fxx sx x s
These xed point equations an usually be solved using numeri ally by fun tion iteration, typi ally with initial guess = 0, or, if the problem is one dimensional, analyti ally by applying the quadrati formula. In parti ular, if the problem has = f 2 , a ondition often enone dimensional state and a tion spa es, and if fss fxx sx
ountered in e onomi problems, then the slope of the shadow pri e fun tion may be
omputed analyti ally as follows: = [fss gx2 9.2
g g + f g 2 2fss fxx s x xx s
=Æ ℄=g 2 fxx x
The Collo ation Method
In order to des ribe the ollo ation method for solving ontinuous state Markov de ision models, we will limit our dis ussion to in nite-horizon models with onedimensional state and a tion spa es and univariate sho ks. The presentation generalizes to models with higher dimensional states, a tions, and sho ks, but at the expense of umbersome additional notation required to tra k the di erent dimensions.1 1 The
routines in luded in the Compe on library a
ompanying the book admit higher dimensional
states, a tions, and sho ks.
CHAPTER 9.
261
CONTINUOUS STATE MODELS: METHODS
Consider, then, Bellman's equation for an in nite horizon dis rete time ontinuous state dynami de ision problem V (s) = max ff (s; x) + ÆE V (g (s; x; ))g: x2X s ( )
Assume that the state spa e is a bounded interval of the real line, S = [smin; smax ℄, and the a tions either are dis rete or are ontinuous and subje t to simple bounds a(s) x b(s) that are ontinuous fun tions of the state. Further assume that the reward fun tion f (s; x) and state transition fun tion g(s; x; ) are twi e ontinuously di erentiable fun tions of their arguments. To ompute an approximate solution to Bellman's equation via ollo ation, one employs the following strategy: First, write the value fun tion approximant as a linear
ombination of known basis fun tions ; ; : : : ; n whose oeÆ ients ; ; : : : ; n are to be determined:
X ( )
1
n
V s
2
1
2
()
j j s :
j =1
Se ond, x the basis fun tion oeÆ ients ; ; : : : ; n by requiring the approximant to satisfy Bellman's equation, not at all possible states, but rather at n states s ; s ; : : : ; sn , alled the ollo ation nodes. Many ollo ation basis-node s hemes are available to the analyst, in luding Cheby hev polynomial and spline approximation s hemes. The best hoi e of basis-node s heme is appli ation spe i , and often depends on the urvature of the value and poli y fun tions. The ollo ation strategy repla es the Bellman fun tional equation with a system of n nonlinear equations in n unknowns. Spe i ally, to ompute the value fun tion approximant, or more pre isely, to ompute the n oeÆ ients ; ; : : : ; n in its basis representation, one must solve the nonlinear equation system 1
1
2
2
X j
( ) = x2max ff (si; x) + ÆE X s
j j si
( i)
X n
1
((
2
))g:
j j g si ; x;
j =1
The nonlinear equation system may be ompa tly expressed in ve tor form as the
ollo ation equation
= v( ): Here, , the ollo ation matrix, is the n by n matrix whose typi al ij th element is the j th basis fun tion evaluated at the ith ollo ation node ij = j (si)
CHAPTER 9.
CONTINUOUS STATE MODELS: METHODS
262
and v, the ollo ation fun tion, is the fun tion from 0 and x is onstrained su h that x = 0 if S = 0. Zero is a natural boundary for this pro ess, meaning that S(t) 0 with probability 1. In this ase, it may not be ne essary to impose onditions on the boundary at S = 0. An intuitive way to think of this situation is that a se ond order di erential equation be omes
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
342
e e tively rst order as the varian e goes to zero. We may, therefore, not need to impose further onditions to a hieve a well de ned solution. Several examples we will dis uss have singular boundary onditions. 10.2.2
Choi e of the Dis ount Rate
The hoi e of the appropriate dis ount rate to use in dynami hoi e problems has been a topi of onsiderable dis ussion in the orporate nan e literature. The arbitrage theory dis ussed in se tion 10.1 an be applied fruitfully to this issue. In parti ular, there is an equivalen e between the hoi e of a dis ount rate and the pri e of risk assigned to the various sour es of risk a e ting the problem. In general, if there is a market for assets that depend on a spe i risk, S , then arbitrage onstrains the hoi e of the dis ount rate that should be used to value an investment proje t. If an inappropriate dis ount rate is used, a potential arbitrage opportunity is reated by either overvaluing or undervaluing the risk of the proje t. To see this note that the on entrated Bellman's equation for a dynami proje t an be written V
= ÆV + Vt + S VS +
1 2 S VSS ;
where ÆV = f (S; x ; t) and x = g (S; x ; t). To avoid arbitrage, however, (10.1) must hold. Together these relationships imply that
= r + S VS =V = r + V =V
(10.6)
In pra ti e we an eliminate the need to determine the appropriate dis ount rate by using the risk-free rate as the dis ount rate and a ting as if the pro ess S has instantaneous mean of either ^S
= S
S S
or, if S is the value of a traded asset, ^S
= rS
ÆS :
Whi h form is more useful depends on whether it is easier to obtain estimates of the market pri e of risk for S , S , or in ome stream generated by S , ÆS . Even if the proje t involves a non-traded risk, it may be easier to guess the market pri e of that risk than to de ne the appropriate dis ount rate. For example, if the risk is idiosyn rati and hen e an be diversi ed away, then a well-diversi ed agent would set the market pri e of risk to zero. An appropriate dis ount rate is parti ularly diÆ ult to sele t when there are multiple sour e of risk (state variables) be ause the dis ount rate be omes a ompli ated fun tion of the various market pri es of risk.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
343
Having said that, there may be ases in whi h the appropriate dis ount rate is easier to set. For rm level apital budgeting, the dis ount rate is the required rate of return on the proje t and, in a well fun tioning apital market, should equal the rm's ost of apital. Thus the total return on the proje t must over the ost of funds: V
= ÆV + V = rV + S V :
The ost of funds, , therefore impli itly determines the market pri e of risk (using 10.6). Summarizing, there are three alternative ases to onsider: 1.
S
is a traded asset for whi h S
2.
= rS
ÆS
is not a traded asset but there is a traded asset the value of whi h, W , depends on S and the market pri e of risk an be determined a
ording to S
3.
S
= (W + ÆW
rW )=W
S represents a non-pri ed risk and either means.
or
must be determined by other
When S is in uen ed by the ontrol x, the payment stream, Æ (S; t), be omes f (S; x; t) and the drift term, (S; t), be omes g (S; x; t). There are three forms of Bellman's equation: A)
rV
= maxx f (S; x; t) + Vt + (rS
ÆS ) VS
+
B)
rV
= maxx f (S; x; t) + Vt + (g (S; x; t)
C)
V
= maxx f (S; x; t) + Vt + g (S; x; t)VS +
1 2 (S; t)V SS 2
(S; t)) VS
+
1 2 (S; t)V SS 2
1 2 (S; t)V SS 2
Any of the three forms an be used when S is a traded asset, although (A) and (B) are preferred in that they rely on market information rather than on guesses on erning the appropriate dis ount rate. When S is not a traded asset but represents a risk pri ed in the market, (B) is the preferred form. If S represents a non-pri ed asset then either form (B) or (C) may be used, depending on whether it is easier to determine appropriate values for or for .
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES 10.2.3
344
Euler Equation Methods
As in the dis rete time ase, it may be possible to eliminate the value fun tion (or
ostate variable) and express the optimality onditions in terms of the state and
ontrol alone (see dis ussion for dis rete time in Se tion ??). Su h an expression is known as an Euler equation. In the dis rete time ase, this was most useful in problems for whi h gS (x; S; ) = 0. In the ontinuous time ase, however, Euler equation methods are most useful in deterministi problems. As before, we dis uss the in nite horizon ase and leave the reader to work out the details for the nite horizon ase. Suppose dS
= g (x; S )dt:
The Bellman Equation is V (S )
= max f (x; S ) + g (x; S )V (S ); 0
x
with FOC fx (x; S ) + gx (x; S )V
0
(S ) = 0:
Let h(x; S ) = fx (x; S )=gx (x; S ), so the FOC an be written V (S ) = h(x; S ):
(10.7)
0
Using the Envelope Theorem applied to the Bellman Equation, (
gS (x; S ))V
0
(S )
fS (x; S )
= g (x; S )V (S ): 00
Using h and its total derivative with respe t to S : V
00
(S ) = hS (x; S ) + hx (x; S )
dx dS
;
the terms involving V an be eliminated: (
gS (x; S ))h(x; S )
fS (x; S )
= g (x; S )
hS (x; S ) + hx (x; S )
dx dS
:
(10.8)
This is a rst-order di erential equation that an be solved for the optimal feedba k rule, x(S ). The \boundary" ondition is that the solution pass through the steady state at whi h dS=dt = 0 and dx=dt = 0. The rst of these onditions is that g (x; S ) = 0, whi h in turn implies that the left hand side of (10.8) equals 0: (
gS (x; S ))h(x; S )
fS (x; S )
= 0:
These two equations are solved simultaneously (either expli itly or numeri ally) to yield boundary onditions for the Euler Equation. This approa h is applied in the example beginning on page 353.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES 10.2.4
345
Examples
Example: Optimal Renewable Resour e Extra tion
The sto k of a resour e, S , is governed by the ontrolled sto hasti pro ess
dS = (B (S ) q )dt + Sdz; where B (S ) is a biologi al growth fun tion and q is the harvest rate of the resour e. The marginal ost of harvesting the resour e depend only on the sto k of the resour e with the spe i fun tional form
C (q ) = (S )q: The total surplus ( onsumer plus produ er) is Z q f (S; q ) = D 1 (z )dz (S )q; 0
where D is the demand fun tion for the resour e. With a dis ount rate of , the Bellman Equation for this optimization problem is Z q V = max D 1 (z )dz (S )q + (B (S ) q ) VS + 21 2 S 2 VSS : q
0
The FOC for the optimal hoi e of q is
D 1 (q ) (S ) VS (S ) = 0; or
q = D( (S ) + VS ): Noti e that the FOC implies that the marginal surplus of an additional unit of the harvested resour e is equal to the marginal value of an additional unit of the in situ sto k: f (S; q ) D 1 (q ) (S ) = V (S ): q
S
To make the problem operational, it must be parameterized. Spe i ally the biologi al growth fun tion is
B (S ) = S (1 (S=K ) );
the demand fun tion is
D(p) = bp
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
346
and the marginal ost fun tion is
(S ) = S : In general this model must be solved numeri ally, but spe ial ases do exist whi h admit an expli it solution. Spe i ally, if = 1 + and = 1=(1 + ) the value fun tion has the form 1 V (S ) = +
S
K
where solves
and
1+
= 0;
+ = b 1+
2 2b
1+
:
It is straightforward to solve for using a standard root nding solver (see Chapter 3) and for some values of a omplete solution is possible. Table 10.1 provides three spe ial ases dis ussed by Pindy k that have losed form solutions, in luding the limiting ase as ! 0. Example: Sto hasti Growth
An e onomy is hara terized by a fun tion des ribing the produ tivity of apital, K , that depends, both in mean and varian e, on an exogenous te hnology sho k, denoted Y . Y is governed by
p
dY = (1 Y )dt + Y dz: With denoting urrent onsumption (the ontrol), the apital sto k dynami s are
p
dK = ( KY
)dt + K Y dz; where the same Brownian motion, dz , that drives the te hnology sho ks also auses
volatility in the produ tivity of apital. The so ial planner's optimization problem is to maximize the present value of the utility of onsumption, taken here to be the log utility fun tion, using dis ount rate . Before dis ussing the solution it is useful to onsider the form of the te hnology assumed here. The expe ted growth rate in apital, ignoring onsumption, is aÆne in the apital sto k and depends on the size of the te hnology sho k. The te hnology sho k, in turn, has an expe ted growth pattern given by
dEY = (aEY
b)dt:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
Table 10.1: Known Solutions to the Optimal Harvesting Problem
1
1 2
2
0
1
1
2
1 2
1 2
B (S ) S (1 S=K ) S ln(K=S ) q K 2 S 1 S
where
1
=
2
=
3
=
2
0
1
V (S ) 1
S + K
b +ln(S ) + 2 p 3 S + K
p
s
q (S ) p +b S 1
V S (S ) 1 S2 b ( +)S p3 2 S
2
1
bS b
3 =2)2 S
(
b 1 + 1 + + A 2 + b b 1 + 2 ln(b )
+ ln(K ) b 21 ; = + b + ( + ) s 2b
2 +
+ + 2 =8
2
2
347
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
348
This di erential equation an be solved for the expe ted value of Y :
b=a)ea(T
Et YT = (Yt
t)
+ b=a:
Roughly speaking, this implies that, for a given apital sto k, the produ tivity of
apital is expe ted to grow at a onstant rate (a) if Y is greater than b=a and to shrink at the same rate when Y is less than b=a. The Bellman Equation for this problem is
V = max ln( ) + VK ( KY
+
1 2
) + VY (aY
b)
VKK 2 K 2 Y + 12 VY Y 2 Y + VKY KY
Let us guess that the solution is one with onsumption proportional to the apital sto k
= K: The FOC ondition asso iated with the Bellman equation tells us that the optimal satis es 1= = VK : If our guess is right, it implies that V (K; Y ) = ln(K )= + f (Y ), where f (Y ) is yet to be determined. To verify that this guess is orre t, substitute it into the Bellman equation:
K)
ln(
+ f (Y ) = ln( K ) +
1 + f 0 (Y )(aY
Y
b)
2 Y 2
+ 21 f 00 (Y ) 2 Y:
Colle ting terms and simplifying, we see that = and that f (Y ) solves a ertain se ond order di erential equation. Rather than try to solve f (Y ) dire tly, however, a more instru tive approa h is to solve for the value fun tion dire tly from its present value form. If our guess is
orre t then Z 1 Z 1 ln() t V (K; Y ) = E e ln(K )dt = + e t E [ln(K )℄ dt (10.9)
0
0
The only diÆ ulty presented here is to determine the time path of E [ln(K )℄. Using Ito's Lemma and = K
d ln(K ) =
dK K
1 2
2 Y dt =
h
1 2
2 Y
i
p
dt + Y dz:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES Taking expe tations and using the previously obtained result for
h
dE [ln(K )℄
=
1 2
2 E [Y ℄
dt
1 2
2
b a
EY
yields
i
h
=
349
Y0
eat + ab
i
dt
[ 0 aeat + 1 ℄ dt;
= where
0 =
1 =
1 2
2
a b a
1 2
b a
Y0
2
:
Integrating both sides and hoosing the onstant of integration to ensure that, at
t = 0, the expe ted when = K :
E [ln(Kt )℄ = ln(K0 ) produ es
value of
an expression for
E [ln(K )℄
0 + 0 eat + 1 t:
E [ln(K )℄ = ln(K0 )
One step remains; we must use the formula for
E [ln(K )℄ to omplete the derivation
of the present value form of the value fun tion. Re alling (10.9)8
V (K; Y )
= =
Z 1 Z0 1
t
e
E [ln(K )℄ dt +
(ln(K0 )
0
=
ln(K0 )
0
Substituting in the values of
+
0
t
0 ) e
and
0
a
1
+
V (K; Y ) = 8
ln(K )
+
(
1 2
2 a)
+ 0 e
(a
1 2
ln()
+
0
Y
+
)t
+ 1 te
dt +
ln()
:
1
ln()
b(
1 2
(
If the third line is problemati for you it might help to note that Z e t 1 t te dt = t+ :
t
and rearranging we obtain an expression for
the value fun tion
ln()
2 )
a)
1 1A :
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
350
(the subs ripts on K and Y are no longer ne essary). Noti e that this does indeed have the form ln(K )= + f (Y ), with f (Y ) a linear fun tion of Y . We have therefore satis ed the essential part of Bellman's equation, namely verifying that = K is an optimal ontrol. We leave as an exer ise the task of ompleting the veri ation that Bellman's equation is satis ed by our expression for V (K; Y ). Let's review the steps we took to solve this problem. First, we guessed a solution for the ontrol and then used the rst order onditions from Bellman's equation to determine a fun tional form for V (K; Y ) that must hold for this to be an optimal
ontrol. We then evaluated the present value form of the value fun tion for this ontrol, thereby obviating the need to worry about the appropriate boundary onditions on Bellman's equation (whi h we have seen is a deli ate subje t). We were able to obtain an expression for the value fun tion that mat hed the fun tional form obtained using the rst order onditions, verifying that we do indeed have the optimal ontrol. This strategy is not always possible, of ourse, but when it is, we might as well take advantage of it. Example: Portfolio Choi e
The previous examples had a small number of state and ontrol variables. In the example we are about to present, we start out with a large number of both state variables and ontrols, but with a spe i assumption about the state dynami s, the dimension of the state is redu ed to one and the ontrol to two. Su h a redu tion transforms a problem that is essentially impossible to solve in general into one that is relatively straightforward to solve. If a spe i lass of reward fun tions is used, the problem an be solved expli itly (we leave this as an exer ise). Suppose investors have a set of n assets from whi h to invest, with the per unit pri e of these assets generated by an n dimensional Ito pro ess dP = (P )dt + (P )dz; where (P ) is an n k matrix valued fun tion (i.e., : < ! < ), and dz is a k-dimensional ve tor of independent Wiener pro esses. We assume that = >, the instantaneous ovarian e matrix for pri es, is non-singular, implying that there are no redundant assets or, equivalently, that there is no riskless asset.9 A portfolio
an be de ned by the number of shares, N , invested in ea h asset or as the fra tion of wealth held in ea h asset: w = N P =W: Expressed in terms of N the wealth pro ess an be des ribed by n
i
i
dW = 9 The
i
i
X N dP
i
n
i
i
i=1
ase in whi h a riskless asset is available is treated in an exer ise.
n
k
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
351
whereas, in terms of w , it is given by i
dW=W
=
n X
w dP =P : i
i
i
i
=1
The latter expression is parti ularly useful if pri es are multivariate geometri Brownian motion pro esses, so (P ) = P and (P ) = P (where and are onstants), implying that: dW=W = w dt + w dz; i.e., W is itself a geometri Brownian motion pro ess. This means that portfolio de isions an be expressed in terms of wealth alone, without referen e to the pri es of the underlying assets in the portfolio. Geometri Brownian motion, therefore, allows for a very signi ant redu tion in the dimension of the state (from n to 1). Consider an investor who draws o a ow of onsumption expenditures C . The wealth dynami s are then dW = W w C dt + W w dz: Suppose the investor seeks to maximize the dis ounted stream of satisfa tion derived from onsumption, where utility is given by U (C ) and the dis ount rate is . The Bellman's Equation for this problem is10 >
>
>
P
= max U (C ) + W w C V + 21 W 2w wV
V
s.t.
>
>
>
W
C;w
WW
= 1. The FOC asso iated with this maximization problem are U (C ) = V ; i
w
i
0
W
W V + W 2V W
and
;
X
w
i
= 1;
WW
w 1 = 0;
(10a) (10b) (10 )
i
where is a Lagrange multiplier introdu ed to handle the adding-up onstraint on the w . A bit of linear algebra applied to (10b) and (10 ) will demonstrate that the 10 If pri es were not geometri Brownian motion the oeÆ ients and would be fun tions of
urrent pri es and the Bellman's Equation would have additional terms representing derivatives of the value fun tion with respe t to pri es, whi h would make the problem onsiderably harder to solve. i
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
352
optimal portfolio weight ve tor, w, an be written as a linear ombination of ve tors, and , that are independent of the investor's preferen es: w = + (W ); (11) where 11 = > 1 ; 1 1 > 1 1 1 = > 1 1 1 1 and (W ) =
VW ; W VW W
This has a ni e e onomi interpretation. When asset pri es are generated by geometri Brownian motion, a portfolio separation result o
urs, mu h like in the stati CAPM model. Only two portfolios are needed to satisfy all investors, regardless of their preferen es. One of the portfolios has weights proportional to 11, the other to 1 ( ( >)1). The relative amounts held in ea h portfolio depend on the investor's preferen es, with more of the rst portfolio being held as the degree of risk aversion rises (as (W ) de reases). This is understandable when it is noti ed that the rst portfolio is the minimum risk portfolio, i.e., solves the problem min > ; s.t. > 1 = 1: Furthermore, the expe ted return on the minimum risk portfolio is >; hen e the term ( >)1 an therefore be thought of as an \ex ess" return ve tor, i.e., the expe ted returns over the return on the minimum risk portfolio. The problem is therefore redu ed to determining the two de ision rule fun tions for onsumption and investment de isions, C (W ) and (W ), that satisfy: U 0 (C (W )) = VW (W ) and VW (W ) : (W ) = W VW W (W ) Noti e that the two fund separation result is a result of the assumption that asset pri es follow geometri Brownian motions and not the result of any assumption about
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
353
preferen es. Given the enormous simpli ation that it allows, it is small wonder that nan ial e onomists like this assumption. Example: Neo lassi al Growth Model
Ramsey introdu ed what has be ome a standard starting pla e for studying optimal e onomi growth. The basi model has been re ned and extended in numerous ways. We present here simple version.11 A single (aggregate) good e onomy is governed by a produ tion te hnology, f . The net output from the produ tion pro ess depends on the level of the apital sto k K . That output an either be onsumed at rate C or invested, thereby in reasing the apital sto k. A so ial utility fun tion depends on the rate of onsumption, U (C ); a so ial planner attempts to maximize the dis ounted stream of so ial utility over an in nite time horizon, using a onstant dis ount fa tor . The optimization problem an be expressed in terms of K , the state variable, and C , the ontrol variable, as follows Z 1 max e t U (C )dt; C (t)
0
subje t to the state transition fun tion K 0 = q (K ) C . This is a deterministi in nite time problem so VKK and Vt do not enter the Bellman's equation. The Bellman's equation for this problem is V (K ) = max U (C ) + V 0 (K ) (q (K ) C ) C
The maximization problem requires that U 0 (C ) = V 0 (K ):
(12)
We an derive an Euler equation form for this problem by eliminating the value fun tion, thereby obtaining a di erential equation for onsumption in terms of urrent
apital sto k. Applying the Envelope Theorem, V 0 (K ) = q 0 (K )V 0 (K ) + [q (K )
C ℄V 00 (K ):
(13)
Combining (12), (13) and the fa t that V 00 (K ) = U (C )C 0 (K ) yields U 0 (C ) 0 (q (K ) ) = (q (K ) C )C 0 (K ): U 00 (C ) 00
Thus the optimal de ision rule C (K ) solves a rst order di erential equation. 11 We
alter Ramsey's original formulation by in luding a dis ount fa tor in the optimization prob-
lem, as is standard in modern treatments.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
354
The boundary ondition for this di erential equation is that the solution passes through the point (K ; C ) whi h simultaneously solves dK=dt = 0 and the Euler
ondition: q 0 (K ) = C = q (K ): Example: Non-Renewable Resour e Management
A rm that manages a non-renewable resour e obtains a net return ow of Ax1 per unit of time, where x is the rate at whi h the resour e is extra ted. The sto k of the resour e is governed by dS = xdt: The extra tion rate is bounded below by 0 (x 0) and onstrained to equal 0 if the sto k is 0 (S = 0 ) x = 0). The manager seeks to solve Z 1 1 V (S ) = max E e Ax d : x
0
The Bellman's equation for the problem is V (S ) = max Ax1 xVs : x
The boundary ondition is that V (0) = 0. The rst order optimality ondition is VS (S ) = (1 )Ax : Using an Euler equation approa h, apply the envelope theorem to obtain VS = xVSS : We have an expression for VS from the optimality ondition and this an be di erentiated with respe t to S to obtain
(1 )Ax
= (1
Simplifying, this yields
dx = ; dS
)Ax
dx :
dS
whi h, with the boundary ondition that x = 0 when S = 0, produ es the optimal
ontrol x(S ) = S
and the value fun tion
V (S ) =
AS 1 :
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES 10.3
355
Free Boundary Problems
We have already seen how boundary onditions are needed to determine the solution to dynami models in ontinuous time. Many important problems in e onomi s, however, involve boundaries in the state spa e whi h must be determined as part of the solution. Su h problems are known as free boundary problems. The boundary marks the lo ation where some dis rete a tion is taken, generally taking the form of e e ting an instantaneous hange in the value of a ontinuous or a dis rete state.12 Table 10.2 ontains a lassi ation of di erent free boundary problems that have appeared in the e onomi s literature. An important distin tion, both in understanding the e onomi s and in solving the problem numeri ally, is whether the boundary
an be rossed. If the ontrol is su h that it maintains a state variable within some region de ned by the free boundary, the problem is a barrier problem and we will solve a di erential equation in this region only. For example, the sto k of a sto hasti renewable resour e an be harvested in su h a way as to keep the sto k level below some spe i ed point. If the sto k rises to this point, it is harvested in su h a way as to maintain it at the boundary (barrier ontrol) or to move it to some point below the boundary (impulse ontrol). Table 10.2: Types of Free Boundary Problems
BARRIERS: Problem
A tion at Boundary
Impulse ontrol Jump from trigger to target Barrier ontrol
Move along boundary
TRANSITIONAL BOUNDARIES: Problem
A tion at Boundary
Dis rete states
Change state
Bang-bang
Swit h between ontrol extrema
In barrier ontrols problems, the barrier de nes a trigger point at whi h, if rea hed, one maintains the state at the barrier by exa tly o setting any movements a ross the 12 In
the physi al s ien es free boundary problems are also known as Stefan problems. A ommonly
used example is the lo ation of the phase hange between liquid and i e, where the state spa e is measured in physi al spa e oordinates.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
356
barrier. Typi ally, su h a ontrol is optimal when there are variable osts asso iated with exerting the ontrol. In su h a situation it is only optimal to exert the ontrol if the marginal hange in the state o sets the marginal ost of exerting the ontrol. In impulse ontrol problems, if the barrier is rea hed one takes an a tion that instantaneously moves the state to a point inside the barrier. An (s; S ) inventory
ontrol system is an example of an impulse ontrol in whi h the state is the level of inventory, whi h is subje t to random demand. When the inventory drops to the level s, an order to replenish it to level S is issued. Typi ally su h ontrols are optimal when there is a xed ost asso iated with exerting the ontrol; the ontrol is exerted only when the bene t from exerting the ontrol overs the xed ost. The other major type of free boundary problem arises when, in addition to one or more ontinuous state variables, there is also a state that an take on dis rete set of values. In this ase, boundaries represent values of the ontinuous states at whi h a hange in the dis rete state o
urs. For example, onsider a rm that an either be a tively produ ing or an be ina tive (a binary state variable). The hoi e of whi h state is optimal depends on a randomly u tuating net output pri e. Two boundaries exist that represent the pri es at whi h the rm hanges from a tive to ina tive or from ina tive to a tive (it should be lear that the latter must be above the former to prevent the rm from having to be ontinuously hanging!). An important spe ial ase of the dis rete state problem is the so- alled optimal stopping problem; the exer ise of an Ameri an option is perhaps the most familiar example. Stopping problems arise when the hoi e of one of the dis rete state values is irreversible. Typi ally the dis rete state takes on two values, a tive and ina tive. Choosing the ina tive state results in an immediate one time payout. An Ameri an put option, for example, an be exer ised immediately for a reward equal to the option's exer ise pri e less the pri e of the underlying asset. It is optimal to exer ise when the underlying asset's pri e is so low that it is better to have the ash immediately and reinvest it than to wait in hopes that the pri e drops even lower. Another important spe ial ase is the so- alled sto hasti bang-bang problem. Su h problems arise when it is optimal to exert a bounded ontinuous ontrol at either its maximum or minimum level. E e tively, therefore, there is a binary state variable that represents whi h ontrol level is urrently being exerted. The free boundary determines the values of the ontinuous variables at whi h it is optimal to hange the binary state. A ouple points should be mentioned now and borne in mind whenever onsidering free boundary problems. First, it is useful to distinguish between the value fun tion evaluated using an arbitrary boundary and the value fun tion using the optimal hoi e of the boundary. The value fun tion (the present value of the return stream) using an arbitrary barrier ontrol is des ribed by a se ond order partial differential equation subje t to the appropriate boundary onditions; this is the message
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
357
of the Feynman-Ka equation (see Appendix A, Se tion A.5.3). The optimal hoi e of the boundary must then add additional restri tions that ensure its optimality. We therefore distinguish in Table 10.2 between a point, S a, on an arbitrary boundary and a point, S , on the optimal boundary. As we shall see in the next hapter, this distin tion is parti ularly important when using a strategy to nd the free boundary that involves guessing its lo ation, omputing the value fun tion for that guess, and then he king whether the optimality ondition holds. Related to this is an understanding the number of boundary onditions that must be applied. Here are some rules that should help you avoid problems. First, any non-sto hasti ontinuous state will have one partial derivative and will require one boundary ondition. On the other hand, any sto hasti state variable will have se ond order derivatives and will generally need two boundary onditions.13 These statements apply to arbitrary ontrols. For optimality we will require an additional boundary ondition for ea h possible binary hoi e. The additional onstraints an be derived formally by maximizing the value fun tion for an arbitrary barrier with respe t to the lo ation of the barrier, whi h for single points means solving an ordinary maximization problem and for fun tional barriers means solving an optimal ontrol problem. In all of these ases one an pro eed as before by de ning a Bellman's Equation for the problem and solving the resulting maximization problem. The main new problem that arises lies in determining the region of the state spa e over whi h the Bellman's Equation applies and what onditions apply at the boundary of this region. We will
ome ba k to these points so, if they are not lear now, bear with us. Now let us
onsider ea h of the main types of problem and illustrate them with some examples. 10.3.1
Impulse Control
Impulse and barrier ontrol problems arise when the reward fun tion in ludes the size of the hange in a state variable aused by exerting some ontrol. Su h problems typi ally arise when there are transa tions osts asso iated with exerting a ontrol, in whi h ase it may be optimal to exert the ontrol at an in nite rate at dis rete sele ted times. In addition, the reward fun tion need not be ontinuous in S . The idea of an in nite value for the ontrol may seem puzzling at rst and one may feel that it is unrealisti . Consider that in many appli ations en ountered in e onomi s the ontrol represents the rate of hange in a state variable. The state is typi ally a sto k of some asset measured in quantity units. The ontrol is thus a
ow rate, measured in quantity units per unit time. If the ontrol is nite, the state 13 The
ex eption to this rule of thumb involves pro esses that exhibit singularities at natural
boundaries, whi h an eliminate the need to spe ify a ondition at this boundary
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
358
annot hange qui kly; essentially the size of the hange in the state must be small if the time interval over whi h the hange is measured is small. In many situations, however, we would like to have the ability to hange the state very qui kly in relation to the usual time s ale of the problem. For example, the time it takes to ut down a timber stand may be very small in relation to the time it takes for the stand to grow to harvestable size. In su h situations, allowing the rate of hange in the state to be ome in nite allows us to hange the state very qui kly (instantaneously). Although this makes the mathemati s somewhat more deli ate, it also results in simpler optimality onditions with intuitive e onomi interpretations. Consider the single state ase in whi h the state variable governed by dS = [(S ) + x℄dt + (S )dz and the reward fun tion that is subje t to xed and variable osts asso iated with exerting the ontrol: 8 > > < r (S ) ( S ) F if x < 0 f (S; S; x) = r0 (S ) if x = 0 > > : r+(S ) +(S ) F + if x > 0 with (0) = +(0) = 0. In this formulation there are xed osts, F and F +, and variable osts, and +, asso iated with exerting the ontrol, both of whi h depend on the sign of the ontrol. Typi ally, we would assume that the xed osts are nonnegative. The variable osts, however, ould be negative; onsider the salvage value from selling o assets. To rule out the possibility of arbitrage pro ts, when the reward is in reasing in the state (rS 0), we require that F + + + (z ) + F + ( z ) > 0 for any positive z; thereby preventing in nite pro ts to be made by ontinuous hanges in the state. With ontinuous time di usion pro esses, whi h are very wiggly, any strategy that involved ontinuous readjustment of a state variable would be ome in nitely expensive and ould not be optimal. Instead the optimal strategy is to hange the state instantly in dis rete amounts, thereby in urring the osts of those states only at isolated points in time. An impulse ontrol strategy would be optimal when there are positive xed osts (F +; F > 0). Barrier ontrol strategies (whi h we dis uss in the next se tion) arise when the xed ost omponents of altering the state are zero. With impulse ontrol, the state of the system is reset to a new position (a target) when a boundary is rea hed (a trigger). It may be the ase that either or both the trigger and target points are endogenous. For example, in a ash management
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
359
situation, a bank manager must determine when there is enough ash-on-hand (the trigger) to warrant investing some of it in an interest bearing a
ount and must also de ide how mu h ash to retain (the target). Alternatively, in an inventory repla ement problem, an inventory is resto ked when it drops to zero (the trigger), but the resto king level (the target) must be determined (resto king o
urs instantaneously so there is no reason not to let inventory fall to zero). A third possibility arises in an asset repla ement problem, where the age at whi h an old ma hine is repla ed by a new one must be determined (the trigger), but the target is known (the age of a new asset). In any impulse ontrol problem, a Feynman-Ka Equation governs the behavior of the value fun tion on a region where ontrol is not being exerted. The boundaries of the region are determined by value mat hing onditions that equate the value at the trigger point with the value at the target point less the ost of making the jump. Furthermore, if the trigger is subje t to hoi e, a smooth pasting ondition is imposed that the marginal value of hanging the state is equal to the marginal ost of making the hange. A similar ondition holds at the target point if it is subje t to hoi e. Example: Asset Repla ement
Consider the problem of when to repla e an asset that produ es a physi al output, y (A), where A is the state variable representing the age of the asset. The asset's value also depends on the net pri e of the output, P , and the net ost of repla ing the asset, . This is a deterministi problem in whi h the state dynami s are simply dA = dt. The reward fun tion is y (A)P . Thus the Bellman equation is V (A) = y (A)P + V 0 (A):
This di erential equation is solved on the range A 2 [0; A ℄, where A is the optimal repla ement age. The boundary onditions are given by the value mat hing ondition: V (0) = V (A ) +
and the optimality (smooth pasting) ondition: V 0 (A ) = 0
The smooth pasting ondition may not be obvious, but it is intuitively reasonable if one onsiders that an asset whi h is older than A should always be immediately repla ed. On e past the age of A , therefore, the value fun tion is onstant: V (A) = V (A ) = V (0) , for A A . No optimality ondition is imposed at the lower boundary (A = 0) be ause this boundary is not a de ision variable.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
360
Before leaving the example, a potentially misleading interpretation should be dis-
V (A) as the value of an age A asset, this V (A) represents the value of the urrent asset, together
ussed. Although it is not unusual to refer to is not quite orre t. In fa t,
with the right to earn returns from future repla ement assets. The urrent asset will be repla ed at age
A
and has value equal to the dis ounted stream of returns it
generates:
Z
A A
0
e t P y (A + t)dt;
but the value fun tion is
V (A) =
Z
A A
0
e t P y (A + t)dt + e
(A A)
V (A )
A has value
Thus the urrent asset at age
V (A) e
(A A)
V (A ):
Example: Timber Harvesting
The previous example examined an asset repla ement problem in whi h the asset generated a ontinuous stream of net returns. In some ases, however, the returns are generated only at the repla ement time. Consider a forest stand that will be lear- ut on a date set by the manager. The stand is allowed to grow naturally at a biologi ally determined rate a
ording to
p
dS = (m S )dt + Sdz: The state variable here represents the biomass of the stand and the parameter
m rep-
resents a biologi al equilibrium point. When the stand is ut, it is sold for a net return of
P S.
In addition, the manager in urs a ost of
has size
S = 0.
stand size, using a dis ount rate of
V for
=
C
to replant the stand, whi h now
The de ision problem is to determine the optimal utting/replanting
.
The Bellman equation is
(m S )V 0 (S ) + 21 2 SV 00 (S );
S 2 [0; S ℄, where S is determined by boundary onditions V (S ) = V (0) + P S
C
value mat hing
and
V 0 (S ) = P
smooth pasting.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
361
If the stand starts at a size above S it is optimal to ut/replant immediately. Clearly the marginal value of additional timber when S > S is the net return from the immediate sale of an additional unit of timber. Hen e, for S > S , V (S ) = V (S ) + p(S S ) and V 0 (S ) = P: As in the previous example, the value fun tion refers not to the value of the timber on the stand but rather to the right to ut the timber on the land in perpetuity. 10.3.2
Barrier Control
In barrier ontrol problems it is optimal to maintain the state within a region by keeping it on the region's boundary whenever it would otherwise tend to move outside of it and to do nothing when the state is in the interior of the region. This, of
ourse, assumes that the state is suÆ iently ontrollable so that su h a poli y is feasible. Barrier ontrol problems an be thought of as limiting ases of impulse ontrol problems as the size of any xed osts go to zero. When this happens, the size of the jump goes to zero, so the trigger and target points be ome equal. This represents something of a dilemma be ause the value mat hing ondition between the target and jump points be omes meaningless when these points are equal. The resolution of this dilemma is to shift the value mat hing ondition to the rst derivative and the smooth pasting to the se ond derivative (the latter is sometimes referred to as a super- onta t ondition). Example: Capa ity Choi e
A rm an install apital, K , to produ e an output with a net return of P . Capital produ es Q(K ) units of output per unit of time, but the apital depre iates at rate Æ . The rm wants to determine Z 1 h i V (Kt ) = max e P q (K ) CI d ; I
t
s.t. dK
= (I
ÆK )dt;
together with the onstraint that I 0. This is an in nite horizon, deterministi
ontrol problem. The Bellman's Equation for this problem is V (K ) = max P q (K ) CI (I ÆK )V 0 (K ): I
The KKT ondition asso iated with optimal I is C V 0 (K ) 0; I 0 and (V 0 (K ) C )I = 0:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
362
This suggests that the rate of investment should be 0 when the marginal value of
apital is less than C and that the rate should be suÆ iently high (in nite) to ensure that the marginal value of apital never falls below C . We assume that apital exhibits positive but de lining marginal produ tivity. The optimal ontrol is spe i ed by a value K su h that investment is 0 when K > K (implying low marginal value of apital) and is suÆ ient high to ensure that K does not fall below K . If K starts below K , the investment poli y will be to invest at an in nite rate so as to move instantly to K , in urring a ost of (K K )C in the pro ess. If K starts at K , the investment rate should be just suÆ ient to ountera t the e e t of depre iation. Summarizing, the optimal investment poli y is 8 > > > < 1 for K < K I = ÆK for K = K : > > > :0 for K > K The value fun tion for any arbitrary value of K an now be determined. To do so, rst de ne the fun tion T (K; K ) to represent the time it takes for the apital sto k to rea h K is it is urrent equal to K . Clearly T = 0 if K K . In the absen e of investment, the apital sto k evolves a
ording to Kt+h = e Æh Kt ; hen e T = ln(K=K )=Æ for K > K . The value fun tion is 8 < 1 P q (K ) ÆK C (K K )C for K < K V (K ; K ) = : (14) : R T (K;K ) e P q (e Æ K )d + e T V (K ) for K > K 0 It is lear that value mat hing holds; i.e. the value fun tion is ontinuous. Not so obvious is the fa t that the derivative of the value fun tion is also ontinuous. The left hand derivative is learly equal to C and a few simple omputations will show that the right hand derivative is also.14 This illustrates the point made above that the usual smooth-pasting ondition (i.e., the ontinuity of the rst derivative) is not an optimality ondition in barrier ontrol problems. To determine the optimal hoi e of K , noti e that we an use the Bellman equation to express the value fun tion for K > K : 1 V (K ; K ) = [P q (K ) ÆKVK (K ; K )℄ :
14 For K > K
V 0 (K ) =
Z 0
T
e P q 0 (e Æ K )d + e T P q (e ÆT K )
dT : e T V (K ) dK
Using dT =dK = 1=(ÆK ) and T (K ; K ) = 0 and simplifying yields the desired result.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES Setting the derivative of this expression with respe t to VK (K ; K )
=
ÆK
K
363
to 0
VKK (K; K );
and noting that VK (K ; K ) = C implies that VKK (K ; K ) = VKK (K ; K ), demonstrates that the appropriate optimality ondition is to nd K su h that VKK (K )
= 0;
implying ontinuity of the se ond derivative. To omplete the problem we apply the Envelope Theorem to the Bellman equation to note that VK
= P q 0 (K )
ÆVK
ÆKVKK :
Using the fa t that VK (K ) = C and VKK (K ) = 0 yields the ondition satis ed by the optimal K : C
=
P +Æ
q 0 (K ):
This is the same expression one obtains by setting the derivatives in (14) equal to 0 and solving for K = K .15 10.3.3
Dis rete State/Control Problems
We turn now to problems involving transitional boundaries. In su h problems, ontrols are not exerted on a ontinuous state variable to for e it to remain within some region. Instead, the boundary typi ally represents the values of the state at whi h a de ision is made to hange from one dis rete state to another. In the simplest form of these problems, a termination de ision is taken. So- alled optional stopping problems in lude the exer ise of Ameri an style options and asset abandonment. More ompli ated problems arise when proje ts an be a tivated and dea tivated. The problem 15 We should express a note of aution here. When the depre iation rate is 0, the se ond derivative
ondition does not apply and, in fa t, the se ond derivative is dis ontinuous at the optimal K . The intuition is lear, however. When Æ = 0 the apital sto k stays the same unless a tively moved. Sin e disinvestment is not allowed (I
0), if the marginal value of apital is less than the ost of
apital, no hange in the apital sto k is warranted. However, if the marginal value of apital is less than C , then the apital sto k should be immediately in reased to the point where the present value of the marginal unit of apital equals the ost of apital:
C = P q 0 (K )=:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
364
then be omes one of determining the value of the proje t if a tive, given that one
an dea tivate it, together with the value of the proje t if ina tive, given that it an be a tivated. The solution involves two boundaries, one whi h determines when the proje t should be a tivated (given that it is urrently ina tive), the other when it should be dea tivated (given that it is urrently a tive). The hallmark of transitional boundary problems is that there is a distin t value fun tion on either side of the boundary and there are onditions that must apply to both of these fun tions at the boundary. Thus the boundary and the value fun tions on both sides must all be simultaneously determined. For arbitrary spe i ations of the boundary, we require that the two value fun tions, net of swit hing osts, are equal at the boundary (value mat hing) and for the optimal boundary, we require that their derivatives are equal at the boundary (smooth-pasting or high onta t).
Optimal Stopping Problems The optimal stopping problem is in many ways the simplest of the free boundary problems and arises in situations involving a on e and for all de ision. For example, suppose a rm is attempting to de ide whether a ertain proje t should be undertaken. The value of the proje t depends on a sto hasti return that the proje t, on e developed, will generate. The state variable an therefore be taken to be the present value of the developed proje t. Furthermore, the rm must invest a spe i ed amount to develop the proje t. In this simple framework, the state spa e is partitioned into a region in whi h no investment takes pla e (when the present value of the developed proje t is low) and a region in whi h the proje t would be undertaken immediately. The boundary between these two areas represents the value of the state, that, if rea hed from below, would trigger the investment. It is important to emphasize that optimal stopping problems, although they have a binary ontrol, di er from other binary ontrol problems in that one value of the
ontrol pays out an immediate reward, after whi h no further de isions are made. The one time nature of the ontrol makes the problem quite di erent from and, a tually, easier to solve than problems with binary ontrols that an be turned on and o . Stopping problems in ontinuous time are hara terized by a random state governed by
dS = (S )dt + (S )dz; a reward stream f (S ) that is paid so long as the pro ess is allowed to ontinue and a payout fun tion R(S ) that is re eived when the pro ess is stopped (for now we
onsider only in nite time dis ounted time autonomous problems; this will be relaxed presently).
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
365
Another way to view the stopping problem is as a problem of hoosing an optimal time to stop a pro ess. This leads to the following formal statement of the problem "Z # t (S ) V (S ) = max E e f (S )d + e t (S ) R(S ) : t (S )
0
This value fun tion is des ribed by the di erential equation
V (S ) = f (S ) + (S )VS (S ) + 21 2 (S )VSS (S )
(15)
The optimal ontrol problem onsists of nding the boundary between the regions on whi h the pro ess should be stopped and those on whi h it should be allowed to
ontinue. For the present, assume that there is a single su h swit hing point, S , with S < S indi ating that the pro ess should be allowed to ontinue. Thus the di erential equation is satis ed on [S; S ℄, where S is a (known) lower bound on the state. Any spe i hoi e of a ontrol onsists of a hoi e of the stopping point, say S . At this point the value fun tion, to be ontinuous, must equal the reward
V (S a ) = R (S a ); (the value-mat hing ondition). The optimal hoi e of S a is determined by the smooth pasting ondition VS (S ) = R0 (S ); the optimal hoi e makes the derivative of the value fun tion equal the derivative of the reward fun tion at the boundary between the ontinuation and stopping regions. Intuitively, the value mat hing and smooth pasting onditions are indi eren e relations; at S the de ision maker is indi erent between ontinuing and stopping. The value fun tion must, therefore, equal the reward and the marginal value of an additional unit of the state variable must be equal regardless of whether the pro ess is stopped or allowed to ontinue. This is the simplest of the optimal stopping problems. We an make them more
omplex by allowing time to enter the problem either through non-autonomous rewards, state dynami s or stopping payment or by imposing a nite time horizon. In the following example we examine a nite horizon problem. Example: Exer ising an Ameri an Put Option
An Ameri an put option, if exer ised, pays K P , where K is the exer ise or strike pri e. P is the random pri e of the underlying asset, whi h evolves a
ording to
dP = (P )dt + (P )dz:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
366
The option pays nothing when it is being held, so f (P ) = 0. Let T denote the option's expiration date, meaning that it must be exer ised on or before t = T (if at all). In general, the option is written on a traded asset so we may use the form of the Bellman's Equation that is dis ounted at the risk-free rate and with mean fun tion repla ed by rP ÆP (see Se tion 10.2.2): rV
= Vt + (rP
ÆP ) V P
+
1 2 (P )V PP 2
on the ontinuation region, where Æ represents the in ome ow (dividend, onvenien e yield, et .) from the underlying asset. Noti e that the onstraint that t T means that the value fun tion is a fun tion of time and so Vt must be in luded in the Bellman's Equation. The solution involves determining the optimal exer ise boundary, P (t). Unlike the previous problem, in whi h the optimal stopping boundary was a single point, the boundary here is a fun tion of time. For puts, P (t) is a lower bound so the ontinuation region on whi h the Bellman's Equation is de ned is [P ; 1). The boundary onditions for the put option are V (P; T )
= max(K
V ( P ; t)
=K
VP (P ; t)
P
= 1
P; 0)
(terminal ondition) (value mat hing) (smooth-pasting)
and, V(
1; t) = 0:
Example: Ma hine Abandonment
Consider a situation in whi h a ma hine produ es an output worth where dP
P
per unit time,
= P dt + P dz;
i.e., that P is a geometri Brownian motion pro ess. The ma hine has an operating
ost of per unit time. If the ma hine is shut down, it must be totally abandoned and thus is lost. Furthermore, at time T , the ma hine must be abandoned. At issue is the optimal abandonment poli y for an agent who maximizes the ow of net returns from the ma hine dis ounted at rate . For the nite time ase de ne as equal to the time remaining until the ma hine must be abandoned, so = T t and d = dt. The optimal poli y an be de ned in terms of a fun tion, P ( ); for P > P ( ) it is optimal to keep the ma hine running, whereas for P < P ( ) it is optimal to abandon it.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
367
The urrent value of the operating ma hine satis es the Bellman's equation V = P
V + P VP + 21 2 P 2 VP P :
and boundary onditions V (P; 0) = 0
terminal ondition VP (1; ) = (1 e )=( ) natural boundary ondition V (P ; ) = 0 value mat hing ondition VP (P ; ) = 0 smooth pasting ondition The rst boundary ondition states that the ma hine is worthless when it must be abandoned. The se ond ondition is derived by onsidering the expe ted value of an in nitely-lived ma hine that is never abandoned: V (P; ) =
P
1 e
(the derivation of this result is left as an exer ise). An alternative upper boundary ondition is that VP P (1; ) = 0. The remaining two onditions are the value mat hing and smooth pasting onditions at P ( ).
General Transitional Boundaries
An optimal stopping problem exhibits omplete irreversability; an ina tivated (exer ised) option annot be rea tivated. Optimal stopping problems an put into a more general framework by viewing them as having a binary state variable representing the a tive/ina tive states. In an optimal stopping problem, the ost of moving from the ina tive to the a tive state is e e tively in nite, thus pre luding this possibility. More generally, however, it may be possible to move ba k and forth between states. In general, suppose that there is a state variable, D, that an take on integer values from 1 to n, as well as a ontinuous state variable (possibly ve tor-valued) governed by dS = (S )dt + (S )dz: The ontrol variable is one that allows any of the n dis rete state to be hosen; hen e x(S; D) takes on values in the set f1; : : : ; ng. In addition, there is a ost to swit hing from D = i to D = j of F ij . A de ision rule de nes the n sets i as the values of S su h that S 2 i ) x(S; i) = i. i is the set of values of S for whi h a de ision rule spe i es that the
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
368
dis rete state remain un hanged. The boundaries of the i represent states at whi h it is optimal to hange the value of D. It is important to note, however, that the presen e of xed transition osts makes it possible that the i are overlapping sets ( i \ j 6= ;). The solution an be expressed as a set of n fun tions, V i(S ), representing the value fun tion for ea h of the n values of the dis rete state. From a omputational point of view, it is only ne essary to de ne V i(S ) over the points in i , and to determine the set of swit hing points S ij , for ea h (i; j ). The Bellman equation is ( ) = f (S; i) + (S )VSi (S ) + 21 2(S )VSSi (S );
rV i S
for S 2 i. In addition, at any points, S ij on the boundary of i at whi h a swit h from i to j is e e ted, it must be true that V i (S ij ) = V j (S ij ; j ) F ij to ensure that no arbitrage opportunities exist. Furthermore, if the swit h point is optimal, the smooth pasting ondition also holds j VSi (S ij ) = VS (S ij ): Example: Entry/Exit
Consider a rm that an either be not produ ing at all or be a tively produ ing q units of a good per period at a ost of per unit. In addition to the binary state Æ (Æ = 0 for ina tive, Æ = 1 for a tive), there is also an exogenous sto hasti state representing the return per unit of output, P , whi h is a geometri Brownian motion pro ess: Pt = (P )dt + (P )dz: We assume there are xed osts of a tivating and dea tivating of I and E , with I + E 0 (to avoid arbitrage opportunities). The value fun tion is (
V P; Æ
)=E
Z 1 0
e
t
(
Æ P
)
dt
the dis ounted osts of swit hing states;
where Æ = 1 if a tive, 0 if ina tive. For positive transition osts, it is reasonable that su h swit hes should be made infrequently. Furthermore it is intuitively reasonable that the optimal ontrol is to a tivate when P is suÆ iently high, P = Ph, and to dea tivate when the pri e is suf iently low, P = Pl . It should be lear that Pl < Ph, otherwise in nite transa tions
osts would be in urred. The value fun tion an therefore be thought of as a pair of
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
369
fun tions, one for when the rm is a tive, V a , and one for when it is ina tive, V i . The former is de ned on the interval [Pl ; 1), the latter on the interval [0; Ph℄. On the interior of these regions the value fun tions satisfy the Feynman-Ka equations
V a = P
+ (P )VPa + 2 (P )VPaP
V i = (P )VPi + 2 (P )VPi P
:
(16)
At the upper boundary point, Ph, the rm will hange from being ina tive to a tive at a ost of I . Value mat hing requires that the value fun tions di er by the swit hing
ost: V i (Ph) = V a (Ph) I . Similarly at the point Pl the rm hanges from an a tive state to an ina tive one; hen e V i (Pl ) E = V a (Pl ). Value mat hing holds for arbitrary hoi es of Pl and Ph . For the optimal hoi es the smooth pasting onditions must also be satis ed:
VPi (Pl ) = VPa (Pl ) and
VPi (Ph ) = VPa (Ph ): In this problem, the exit is irreversible in the sense that reentry is as expensive as initial investment. A re nement of this approa h is to allow for temporary suspension of produ tion, with a per unit time maintenan e hange. Temporary suspension is generally preferable to omplete exit (so long as the maintenan e harge is not prohibitive). The dis rete state for this problem takes on 3 values; its solution is left as an exer ise. 10.3.4
Sto hasti Bang-Bang Problems
Bang-bang ontrol problems arise when both the reward fun tion and the state transition dynami s are linear in the ontrol and the ontrol is bounded. In su h ases it is optimal to set the ontrol at either its upper or lower bound. The ontrol problem thus be omes one of dividing the state spa e into a set of points at whi h the ontrol is at its upper bound and a set at whi h it is at its lower bound. Equivalently, the problem is to nd the boundary between the two sets. If there is no ost to swit hing the ontrol from the lower to upper bound, we are in pre isely the same situation that we dis ussed in the last se tion when the swit hing osts go to zero. The optimal value fun tion and ontrol is found in a similar fashion: de ne a Feynman-Ka Equation on ea h side of the boundary and require that the value fun tions on either side of the boundary are equal up to their se ond derivative.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
370
The general bang-bang problem has reward fun tion of the form f0 (S ) + f1 (S )x
and state dynami s of the form dS = [g0 (S ) + g1 (S )x℄dt + (S )dz:
Furthermore the ontrol is onstrained to lie on a given interval: xa
x xb :
The Bellman's equation for this problem is V = max f0 (S ) + f1 (S )x + [g0 (S ) + g1 (S )x℄VS + 21 2 (S )VSS x
subje t to the ontrol onstraint. The Karush-Kuhn-Tu ker onditions for this problem indi ate that
8 < xa x= : xb
if f1 (S ) + g1 (S )VS (S ) > 0 if f1 (S ) + g1 (S )VS (S ) < 0
This suggests that there is a point, S , at whi h f1 (S ) + g1 (S )VS (S ) = 0:
(17)
Assuming that VS is de reasing in S , this suggests that we must solve for two fun tions, one for S < S that solves a V a = f0 (S ) + f1 (S )xa + [g0 (S ) + g1 (S )xa ℄VSa + 21 2 (S )VSS
(18)
and the other for S > S that solves b : V b = f0 (S ) + f1 (S )xb + [g0 (S ) + g1 (S )xb ℄VSb + 12 2 (S )VSS
(19)
We will need three side onditions at S to ompletely spe ify the problem and to nd the optimal lo ation of S , namely that V a (S ) = V b (S ) VSa (S ) = VSb (S ) f1 (S ) + g1 (S )VS (S ) = 0:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
371
Combining these onditions with (18) and (19) we see that a b VSS (S ) = VSS (S );
i.e., with the optimal hoi e of S the value fun tion is ontinuous up to its se ond derivative. Example: Harvesting a Renewable Resour e
Consider a manager of a renewable biologi al resour e who must determine the optimal harvesting strategy. The state variable, the sto k of the resour e, is sto hasti ,
u tuating a
ording to
dS = [ S (1 S ) hS ℄dt + Sdz; where h, the ontrol, is the proportional rate at whi h the resour e is harvested. Assume that the per unit return is p and that 0 h C . The manager seeks to solve Z 1 t V (S ) = max E e phSdt : h
0
In the notation of general problem, xa = 0, xb = C , f0 (S ) = 0, f1 (S ) = pS , g0 (S ) = S (1 S ) and g1 (S ) = S . The Bellman equation for this problem is
V = max phS + ( S (1 S ) hS ) VS + 21 2 S 2 VSS : h
The assumptions the sto k dynami s imply that V (0) = 0 (on e the sto k rea hes zero it never re overs and hen e the resour e is worthless). At high levels of the sto k, the marginal value of an additional unit to the sto k be omes onstant and hen e VSS (1) = 0. The rst order onditions for this problem suggest that it is optimal to set h = C if VS < p and set h = 0 if VS > p. The interpretation of these onditions is straightforward: only harvest when the value of a harvested unit of the resour e is greater than an unharvested one and then harvest at maximum rate. Thus the problem be omes one of nding the sets
S 0 = fS : VS > pg and
S C = fS : VS < pg
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
372
where
V
S (1 S )VS
V
( S (1
1 2
2 S 2 VSS = 0
on S 0
and
S ) CS ) VS
1 2
2 S 2 VSS
pCS = 0 on S C
The solution must also satisfy the boundary onditions at 0 and 1 and the ontinuity
onditions at any points S su h that VS (S ) = p. The fa t that S (1 S ) hS is
on ave in S implies that S will be a single point, with S 0 = [0; S ) and S C = (S ; 1). Example: Produ tion with a Learning Curve
More ompli ated bang-bang problems arise when there are two state variables. The free boundary is then a urve, whi h typi ally must be approximated. Consider the
ase of a rm that has developed a new produ tion te hnique. Initially produ tion
osts are relatively high but de rease as the rm gains more experien e with the pro ess. It therefore has an in entive to produ e more than it otherwise might due to the future ost redu tions it thereby a hieves. To make this on rete, suppose that marginal and average osts are onstant at any point in time but de line at an exponential rate in umulative produ tion until a minimum marginal ost level is a hieved. The problem fa ing the rm is to determine the produ tion rule that maximizes the present value of returns (pri e less ost times output) over an in nite horizon: 1 max e rt (P C (Q)) x dt;
Z
x
0
where r is the risk-free interest rate and the two state variables are P , the output pri e, and Q, the umulative produ tion to date. The state transition equations are
dP = P dt + P dz and
dQ = x dt; where x is the produ tion rate, whi h is onstrained to lie of the interval [0; x ℄. The pri e equation should be interpreted as a risk-neutral pro ess. The ost fun tion is given by
8 < e C (Q) = : e
Q
Qm
if Q < Qm
= if Q Qm
;
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
373
on e Q Qm , the per unit produ tion ost is a onstant but for Q < Qm it de lines exponentially. The Bellman equation for this problem is = max (P
rV
x
C (Q)) x + xVQ
+ P VP +
1 2P 2V PP 2
s.t. 0 x x . The problem thus is of the sto hasti bang-bang variety with the optimality onditions given by: C (Q) + VQ
0
P P
) )
=0 x = x :
x
Substituting the optimal produ tion rate into the Bellman Equation and rearranging yields the partial di erential equation rV (P; Q)
= P VP (P; Q) +
1 2P 2V PP 2
+ max(0; P
C (Q) + VQ (P; Q))x :
The boundary onditions for this problem require that V (0; Q)
=0
1; Q) = x =Æ
VP (
and that V , VP , and VQ be ontinuous. The rst boundary ondition re e ts the fa t that 0 is an absorbing state for P ; hen e is P rea hes 0, no revenue will ever be generated and hen e the rm has no value. The se ond ondition is derived from
omputing the expe ted revenue if the rm always produ es at maximum apa ity, as it would if the pri e were to get arbitrarily large (i.e., if the probability that the pri e falls below marginal ost be omes arbitrarily small). The derivative of the expe ted revenue is x =Æ . As illustrated in Figure 10.1, the (Q; P ) state spa e for this problem is divided by a urve P (Q) that de nes a low pri e region in whi h the rm is ina tive and a high pri e region in whi h it is a tive. Furthermore, for Q > Qm the lo ation of P (Q) is equal to be ause, on e the marginal ost is at its minimum level, there is nothing to be gained from produ tion when the pri e is less than the marginal produ tion ost. For Q > Qm the problem is simpli ed by the fa t that VQ = 0. Thus V is a fun tion of P alone and the value of the rm satis es rV (P )
= P VP (P ) +
1 2P 2V PP 2
+ max(0; P
C (Q))x :
For Q < Qm , on the other hand, VQ 6= 0 and the lo ation of the boundary must be determined simultaneously with the value fun tion.
P (Q)
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
374
Sequential Learning: Optimal Activation Boundary 50 Qm 45
Production Region:
40
Production
V(P,Q) must be 35
Region:
computed numerically
V known
P
30 *
25
P
20
15 Non−Production Region:
10
A(Q) computed from
Non−Production
value matching condition
5
Region: V known
0
0
5
10
15
20
25
Q
Figure 10.1 A third boundary ondition
V (P; Qm ) = V (P ) (de ned below) is a \terminal" ondition in Q. On e Qm units have been produ ed the rm has rea hed its minimum marginal ost. Further produ tion de isions do not depend on Q nor does the value of the rm, V . An expli it solution an be derived for Q > Qm :
8 < A P V (P ) = : A P 1
2
1
2
+ PÆ
r
if P ; if P
where the solve the quadrati equation 1 2 (1 2
) + (r
Æ )
r=0
and the A1 and A2 are omputed using the ontinuity of V and VP . The ontinuity requirements on the value fun tion, even though the ontrol is dis ontinuous, allow us to determine a free boundary between the regions of the state spa e in whi h produ tion will and will not o
ur. Intuitively, there is a fun tion P (Q) above whi h the pri e is high enough to justify urrent produ tion and below whi h no produ tion is justi ed.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
375
Noti e that below the free boundary the Bellman's equation takes a parti ularly simple form rV (P; Q) = (r
Æ )P VP (P; Q) + 21 2 P 2 VP P ;
whi h together with the rst boundary ondition (V (0; Q) = 0), is solved by V (P; Q) = A1 (Q)P 1 ; where A1 (Q) is yet to be determined (we know, of ourse, that A1 (Qm ) = )). Above the boundary, however, there is no losed form solution. A1 (Q); P (Q) and V (P; Q) for P P must be omputed numeri ally. The solution methods for this problem depend on being able to determine the position of the free boundary. It is therefore worth exploring some of the onsequen es of the ontinuity onditions on V . First, onsider the known form of the value fun tion below the free boundary and its derivative: V (P; Q) = A1 (Q)P 1 VP (P; Q) = 1 A1 (Q)P 1 1 : Eliminating A1 (Q) yields P VP (P; Q) = 1 V (P; Q): This ondition holds everywhere below the boundary and at it as well. By the ontinuity of the V and VS , it must also hold as the boundary is approa hed from above. Another relationship that is useful to note on erns the ontinuity in the Q dire tion. Below the boundary, VQ(P; Q) = A01 (Q)P 1 : The derivative of A1 is onstant in P and may therefore be related to VQ as it approa hes the boundary from above, whi h is known from the Bellman equation: VQ(P; Q) = A01 (Q)P 1 1 P = (LV (P ; Q) (P C (Q))) P where the di erential operator L is de ned as
LV (P; Q) = rV (P; Q)
(r Æ )P VP (P; Q) 12 2 P 2 VP P (P; Q) But we have already seen that P C (Q)+ VQ(P ; Q) = 0 and therefore LV (P ; Q) = 0. Summarizing these results, we see that 8 < (P C (Q)) P 1 for P P P VQ(P; Q) = : LV (P; Q) (P C (Q)) for P P
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
376
Exer ises 10.1. Pri ing Bonds De ne P (t; T ) to be the urrent (time t) pri e of a pure dis ount bond maturing at time T , i.e., a bond that pays $1 at time T . The pri e of a bond of any maturity depends on the instantaneous interest rate, r. It an be shown that
P (r; t; T ) = E^ exp
Z
T t
r( )d
;
where the expe tation is taken with respe t to the risk adjusted pro ess governing the instantaneous interest rate. Assuming that this pro ess is
dr = (r; t)dt + (r; t)dz an extended version of the Feynman-Ka Formula implies that P is the solution to
rP = Pt + (r; t)Pr + 21 2 (r; t)Prr ; subje t to the boundary ondition that P (r; T; T ) = 1. Suppose that the instantaneous interest rate pro ess is
dr = ( r)dt + dz: Show that P has the form
P (r; t; T ) = A(t; T ) exp( B (t; T )r) and, in doing so, determine the fun tions A and B . 10.2. Pri ing Bonds Continued Given the setting of the previous problem, suppose we take the instantaneous interest rate pro ess to be
p
dr = ( r)dt + rdz: Verify numeri ally that P has the form
P (r; t; T ) = A(t; T ) exp( B (t; T )r)
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES with
2 e( +) =2 A( ) = ( + )(e 1) + 2
377
=2 2
and B ( ) =
p
2(e ( + )(e
1) ; 1) + 2
where = 2 + 2 2 . This an be a
omplished by writing a fun tion that returns the proposed value of the bond. This fun tion an be di erentiated with respe t to t and r to obtain the partial derivatives. The arbitrage ondition should be lose to 0 for all values of r, t and all parameter values. 10.3. Futures Pri es A futures ontra t maturing in periods on a ommodity whose pri e is governed by dS = (S; t)dt + (S; t)dz
an be shown to satisfy V (S; ) = (rS
Æ (S; t))VS (S; ) + 21 2 (S; t)VSS (S; )
subje t to the boundary ondition V (S; 0) = S . Here Æ is interpreted as the
onvenien e yield, i.e., the ow of bene ts that a
rue to the holders of the
ommodity but not to the holders of a futures ontra t. Suppose that the volatility term is (S; t) = S:
In a single fa tor model one assumes that Æ is a fun tion of S and t. Two
ommon assumptions are Æ (S; t) = Æ
and Æ (S; t) = ÆS:
In both ases the resulting V is linear in S . Derive expli it expressions for V given these two assumptions.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
378
10.4. Futures Pri es Continued Continuing with the previous question, suppose that the onvenien e yield is
Æ (S; t) = ÆS where Æ is a sto hasti mean-reverting pro ess governed by
dÆ = (m Æ )dt + Æ dw; with Edzdw = Æ . Furthermore, suppose that the market pri e of the onvenien e yield risk is a onstant . Then the futures pri e solves
V = (r
Æ )SVS + ( (m Æ ) ) VÆ + 12 2 S 2 VSS + Æ SVSÆ + 21 Æ2 VÆÆ ;
with V (S; 0)=S. Verify that the solution has the form V = exp(A( ) derive expression for A( ) and B ( ).
B ( )Æ )S and in doing so
10.5. Lookba k Options with Geometri Brownian Motion Suppose the risk neutral pro ess asso iated with a sto k pri e follows
dS = (r
Æ )Sdt + 12 SdW:
Show that a lookba k strike put option an be written in the form
V (S; M; t) = Sv (y; t); where y = M=S . Derive the PDE and boundary onditions satis ed by v . 10.6. Portfolio Choi e with CRRA Utility For the portfolio hoi e problem on page 350 show that a utility fun tion of the form U (C ) = (C 1 )=(1 ) implies an optimal onsumption rule of the form C (W ) = W . Determine the onstant a and, in the pro ess, determine the value fun tion and the optimal investment rule (W ).
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
379
10.7. Portfolio Choi e with a Risk Free Asset Suppose that, in addition to n risky assets, there is also a risk-free asset that earns rate of return r. The ontrols for the investor's problem are again C , the
onsumption rate, and the n-ve tor w, the fra tions of wealthPheld in the risky > assets. The fra tion of wealth held in the riskless asset is 1 i wi = 1 w 1. a) Show that the wealth pro ess an be follows dW W
= [W (r + w>(
r1))
C ℄dt + w>dz:
b) Write the Bellman's Equation for this problem and the asso iated rst order
ondition.
) Show that it is optimal to hold a portfolio onsisting of the risk-free asset and a mutual fund with weights proportional to 1 ( r1). d) Derive expressions for w>( r1) and w>w and use them to on entrate the Bellman equation with respe t to w. 1 C e) Suppose that U (C ) = 1 . Verify that the optimal onsumption rate is proportional to the wealth level and nd the onstant of proportionality. 10.8. Portfolio Choi e Continued Continuing the previous problem, de ne (W ) = V 0(W )=V 00 (W ). Show that C (W ) and (W ) satisfy a system of rst order di erential equations. Use this result to verify that C is aÆne in W and is a onstant when U (C ) = e C . 10.9. Sto hasti Nonrenewable Resour e Management Suppose that the resour e sto k dis ussed on page 354 evolved a
ording to dS
=
xdt + Sdz:
Verify that the optimal ontrol has the form x(S ) = S ;
and, in so doing, determine the values of and . Also obtain an expression for the value fun tion. You should he k that your answer in the limiting ase that = 0 is the same as that given on page 354.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
380
10.10. Nonrenewable Resour es with Sto hasti Pri es As in the example on page 354, a resour e is extra ted at rate x, yielding a ow of returns Ax1 . The sto k of the resour e is governed by dS = xdt. Here, however, we treat A as a random sho k pro ess due to randomness in the pri e of the resour e governed by
dA = (A)dt + (A)dz: The rm would like to maximize the expe ted present value of returns to extra tion, using a dis ount rate of . a) State the rm's optimization problem. b) State the asso iated Bellman's equation.
) State the rst order optimality ondition and solve for the optimal extra tion rate (as a fun tion of the value fun tion and its derivatives). 10.11. Timber Lease A government timber lease allows a timber ompany to ut timber for T years on a stand with B units of biomass. The pri e of ut timber is governed by
p
dp = (p p)dt + pdW: With a utting rate of x and a utting ost of Cx2 =2, dis uss how the ompany
an de ide what to pay for the lease, given a urrent pri e of p and a dis ount rate of (assume that the ompany sells timber as it is ut). Hint: introdu e a remaining stand size, S , with dS = xdt (S is bounded below by 0) and set up the dynami programming problem. 10.12. Timber Harvesting with Deterministi Growth Suppose that the timber harvesting problem dis ussed on page 360 is nonsto hasti . The Bellman equation an then be rewritten in the form
V0 =
V : m S
Verify that the solution is of the form
V = k (m S )
= ;
where k is a onstant of integration to be determined by the boundary onditions. There are two unknowns to be determined, k and S . Solve for k in terms of S and derive an optimality ondition for S as a fun tion of parameters.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
381
10.13. Fishery Management A monopolist manager of a shery fa es a state transition fun tion
dS = [ S (M
S ) x℄dt + 2 (S )dW:
The pri e is onstant and the ost fun tion has a onstant marginal ost that is inversely proportional to the sto k level. In addition, a xed ost of F is in urred if any shing a tivity takes pla e. The reward fun tion an thus be written
p
C x F Æx>0 : S
This is an impulse ontrol problem with two endogenous values of the state,
Q and R, with Q < R. When S R, the sto k of sh is harvested down to Q. Express the Bellman equation for S R and the boundary onditions that determine the lo ation of Q and R (assume a dis ount rate of ). V (S ) = S (M
S )VS (S ) +
2 (S ) 2
VSS (S ); for S 2 [0; R℄:
V (R) V (Q) = p(R Q) C ln(R=Q) F: Vs(R) = p C=R Vs(Q) = p C=Q: 10.14. Capital Investment Consider an investment situation in whi h a rm an add to its apital sto k, K , at a ost of C per unit. The apital produ es output at rate q (K ) and the net return on that output is P . Hen e the reward fun tion fa ing the rm is
f (K; P; I ) = P q (K ) CI: K is learly a ontrollable state, with dK = Idt:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
382
P , on the other hand, is sto hasti and is assumed to be governed by dP = P dt + P dz; (geometri Brownian motion). Using a dis ount rate of , the Bellman equation for this problem is
V (K; P ) = max P q (K ) I
CI + IVK (K; P ) + P VP (K; P ) + 21 2 P 2 VP P (K; P ):
There are, however, no onstraints on how fast the rm an add apital and hen e it is reasonable to suppose that, when it invests, it does so at an in nite rate, thereby keeping its investment osts to a minimum. The optimal poli y, therefore, is to add apital whenever the pri e is high enough and to do so in su h a way that the apital sto k pri e remains on or above a
urve K (P ). If K > K (P ), no investment takes pla e and the value fun tion therefore satis es
V (K; P ) = P q (K ) + P VP (K; P ) + 12 2 P 2 VP P (K; P ): This is a simpler expression be ause, for a given K , it an be solved more or less dire tly. It is easily veri ed that the solution has the form
V (K; P ) = A1 (K )P 1 + A2 (K )P 2 +
P q (K )
where the i solves 12 2 ( 1) + = 0. It an be shown, for > > 0, that 2 < 0 < 1 < 1 . For the assumed pro ess for P , 0 is an absorbing barrier so the term asso iated with the negative root must be for ed to equal zero by setting A2 (K ) = 0 (we an drop the subs ripts on A1 (K ) and 1 ). At the barrier, the marginal value of apital must just equal the investment
ost:
VK (K (P ); P ) = C:
(20)
Consider now the situation in whi h the rm nds itself with K < K (P ) (for whatever reason). The optimal poli y is immediately to invest enough to bring the apital sto k to the barrier. The value of the rm for states below the
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
383
barrier, therefore, is equal to the value at the barrier (for the same P ) less the
ost of the new apital:
V (K; P ) = V (K (P ); P )
(K (P )
K )C:
This suggests that the marginal value of apital equals C when K < K (P ) and hen e does not depend on the urrent pri e. Thus, in addition to (20), it must be the ase that
VKP (K (P ); P ) = 0:
(21)
Use the barrier onditions (20) and (21) to obtain expli it expressions for the optimal trigger pri e P (K ) and the marginal value of apital, A0 (K ). Noti e that to determine A(K ) and therefore to ompletely determine the value fun tion, we must solve a di erential equation. The optimal poli y, however, does not depend on knowing V , and, furthermore, we have enough information now to determine the marginal value of apital for any value of the state (K; P ). Write a program to ompute and plot the optimal trigger pri e urve are displayed in using the parameters
= 0 = 0:2 = 0:05
= 1 and the following two alternative spe i ations for q (K ):
q (K ) = ln(K + 1) q (K ) =
p
K:
10.15. Cash Management Consider the manager of a ash a
ount subje t to random deposits and withdrawals. In the absen e of a tive management the a
ount is des ribed by absolute Brownian motion
dS = dt + dz:
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
384
The manager must maintain a positive ash balan e. When the a
ount hits 0, the manager must draw funds from an interest bearing a
ount. To in rease the ash a
ount by z units, the manager bears a ost of f + z , i.e., there are both xed and proportional variable osts of ontrol. Similarly, the manager
an pla e funds in the interest bearing a
ount by withdrawing an amount z from the ash a
ount, in urring osts of F + Cz . Suppose the manager uses a dis ount rate of and the interest bearing a
ount generates interest at rate r. It is lear that the manager will want to adjust the a
ount only at dis rete times so as to minimize the adjustment osts. A ontrol poli y an therefore be des ribed as a hoi e of three ash levels, S1 S2 S3 , where S1 is the amount of the addition to the fund when it hits 0, S3 is the trigger level for withdrawing funds (adding them to the interest bearing a
ount) and S2 is the target level (i.e., S3 S2 units are withdrawn when the fund hits S3 ). The value fun tion asso iated with this problem solves the Bellman equation16
V (S ) = V (S ) + 12 2 V (S ); for S 2 [0; S3 ℄ 00
0
with the side onditions that
V (0) = V (S1 )
f
(r= + )S1
and
V (S3 ) = V (S2 )
F + (r=
C )(S3
S2 ):
Furthermore, an optimal poli y satis es
V (S1 ) = (r= + ) 0
and
V (S3 ) = V (S2 ) = (r= 0
16 Although
0
C ):
it is not ne essary to solve the problem, it is useful to understand why these onditions are appropriate. The value fun tion here is interpreted as the present value of the urrent ash position, whi h does not depend on how mu h money is in the interest bearing a
ount at the present moment. Cash pays no urrent ows and hen e the Bellman equation is homogeneous (no reward term). The ost of withdrawing funds from the interest bearing a
ount equals the ontrol
ost plus the opportunity ost of the lost interest, whi h is equal to r= times the amount withdrawn. The ost of adding funds to the interest bearing a
ount equals the ontrol ost less the present value of the interest earned on the funds put into the a
ount (r= times the amount of these funds).
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
385
The Bellman equation an be solved expli itly:
V (S ) = A exp( S ) + B exp( S ); where and are hosen to solve the di erential equation and A and B are
hosen to satisfy the side onditions. Write a MATLAB pro edure that a
epts the parameters , , , r, f , F , , and C and returns the parameters A, B , , , S1 , S2 , and S3 . Also determine how the program needs to be modi ed if the proportional osts ( and C ) are zero. Che k your ode using the following parameter values: = 0, = 0:5, = 0:4, r = 0:5, f = 1, F = 0:5, = 0:1, and C = 0:1. You should obtain the result that S1 = 0:7408, S2 = 0:8442, and S3 = 2:2216. 10.16. Entry/Suspension/Exit The Entry/Exit problem dis ussed beginning on page 368 an be extended to allow for temporary suspension of produ tion. Suppose that a maintenan e fee of m is needed to keep equipment potentially operative. In the simple entry/exit problem there were two swit hing osts, I and E . Now there are 6 possible swit hing osts, whi h will generi ally be alled F ij . With D = 1 representing the a tive produ tion state, D = 2 the temporarily suspended state and D = 3 the exited state, de ne the Bellman equations and boundary onditions satis ed by the solution. 10.17. Non-Renewable Resour e Management The demand for a nonrenewable resour e is given by
p = D (q ) = q ; where q is the extra tion rate. For simpli ity, assume the resour e an be extra ted at zero ost. The total sto k of the resour e is denoted by S (with S (0) = S0 ), and is governed by the transition fun tion
dS = qdt: a) For the so ial planner's problem, with the reward fun tion being the so ial surplus, state the Bellman's equation and the optimality ondition, using dis ount rate . Use the optimality ondition to nd the on entrated Bellman's equation. b) Guess that V (S ) = S . Verify that this is orre t and, in doing so, determine and .
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
386
) Determine the time value, T , at whi h the resour e is exhausted. d) Solve the problem using an optimal ontrol (Hamiltonian) approa h and verify that the solutions are the same. 10.18. Renewable Resour e Management with Adjustment Costs Consider an extension to the renewable resour e problem dis ussed on page 371. Suppose that the harvest rate is still onstrained to lie on [0; C ℄ but that it
annot be adjusted instantaneously. Instead assume that the rate of adjustment in the harvest rate, x, must lie on [a; b℄, with a < 0 < b, with the proviso that x 0 is h = 0 and x 0 is h = C . This problem an be addressed by de ning h to be a se ond state variable with a deterministi state transition equation:
dh = xdt: The optimal ontrol for this problem is de ned by two regions, one in whi h x = a and one in whi h x = b. The boundary between these regions is a urve in the spa e [0; 1) [0; C ℄. Write the PDEs that must be satis ed by the value fun tions in ea h region and the value-mat hing and smooth pasting onditions that must hold at the boundaries. 10.19. Optimal Sales from an Inventory Consider a situation in whi h an agent has an inventory of S0 units of a good in inventory, all of whi h must be sold within T periods. It osts k dollars per unit of inventory per period to store the good. In this problem there is a single
ontrol, the sales rate q , and two state variables, the pri e P and the inventory level S . The pri e is an exogenously given Ito pro ess:
dP = (P; t)dt + (P; t)dz: The amount in storage evolves a
ording to
dS = qdt: Furthermore it is assumed that both the state and the ontrol must be nonnegative. The latter assumes that the agent annot pur hase additional amounts to replenish the inventory, so that sales are irreversible.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
387
The problem an be written as V (S; P; t)
= max
Z
q (S;P;t)
Et
T
e
rt
t
(qP
kS ) dt
subje t to the above onstraints. What is Bellman's equation for this problem? Treat the problem as an optimal stopping problem so q = 0 when the pri e is low and q = 1 when the pri e is high. At or above the stopping boundary all inventory is sold instantaneously. State the Bellman's equation for the regions above and below the stopping boundary. State the value-mat hing and smooth-pasting onditions that hold at the boundary. 10.20. Learning-By-Doing with Deterministi Pri e Suppose in the sequential learning that the pri e is deterministi ( = 0) and the r Æ . In this ase, on e produ tion is initiated, it is never stopped. Use this to derive an expli it expression for V (P; Q), where P P (Q). In this
ase, be ause produ tion o
urs at all times, Z 1 V (P; Q) = e r (P C (Q ))d ; 0
where Pt solves the homogeneous rst order di erential equation dPt dt
= (r
Æ )P
and Z1 0
e
r
C (Q )d
=
Z Qm 0
Q
e
r
C (Q
Q)d
+
Z 1 m
Q
e
r
d :
Q
Also show that, for P < P , the value fun tion an be written in the form f (P; P (Q))V (P (Q); Q). Combining these two results, determine the optimal a tivation boundary in the deterministi ase. Verify that your answer satis es the Bellman equation.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
388
Appendix A Dynami Programming and Optimal Control Theory Many e onomists are more familiar with optimal ontrol theory than with dynami programming. This appendix provides a brief dis ussion of the relationship between the two approa hes. As stated previously, optimal ontrol theory is not naturally applied to sto hasti problems but it is used extensively in deterministi ones. The Bellman equation in the deterministi ase is V = max f(S; x) + Vt + g(S; x)VS ; x
where x is evaluated at its optimal level. Suppose we totally di erentiate the marginal value fun tion with respe t to time: dS dVS = VSt + VSS = VSt + VSS g(S; x): dt dt Now apply the Envelope Theorem to the Bellman equation to determine that VS = fS (S; x) + VtS + g(S; x)VSS + VS gS (S; x): Combining these expressions and rearranging yields dVS = VS fS VS gS : (22) dt This an be put in a more familiar form by de ning = VS . Then (22), ombined with the FOC for the maximization problem and the state transition equation an be written as the following system 0 = fx (S; x) + gx (S; x) d = dt
fS (S; x)
gS (S; x)
and dS = g(S; x): dt These relationships are re ognizable as the Hamiltonian onditions from optimal ontrol theory, with the ostate variable representing the shadow pri e of the state variable (expressed in urrent value terms).17 17 See
Kamien and S hwartz, pp. 151-152 for further dis ussion.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
389
The message here is that dynami programming and optimal ontrol theory are just two approa hes to arrive at the same solution. It is important to re ognize the distin tion between the two approa hes, however. Optimal ontrol theory leads to three equations, two of whi h are ordinary di erential equations in time. Optimal
ontrol theory therefore leads to expressions for the time paths of the state, ontrol and ostate variables as fun tions of time: S(t), x(t) and (t). Dynami programming leads to expressions for the ontrol and the value fun tion (or its derivative, the
ostate variable) as fun tions of time and the state. Thus dynami programming leads to de ision rules rather than time paths. In the sto hasti ase, it is pre isely the de ision rules that are of interest, be ause the future time path, even when the optimal
ontrol is used, will always be un ertain. For deterministi problems, however, DP involves solving partial di erential equations, whi h tend to present more hallenges than ordinary di erential equations.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
390
Appendix B Deriving the Boundary Conditions for Resetting Problems It is instru tive to view the resetting problem from another perspe tive. In a simple resetting problem an asset is repla ed at a dis rete set of times when S = S , at whi h point a reward, f (S ) is obtained. Let us de ne (S; S ) to be the (random) time until the state rst hits S , given that it is now equal to S . The rst time the state hits S a reward worth f (S )e (S;S ) (in urrent units of a
ount) will be generated and the state is reset to 0. The time elapsing after a resetting until the state next hits S depends on a random variable that has the same distributional properties as (0; S ) and is independent of previous hitting times (by the Markov property). The expe ted dis ounted rewards (i.e., the value fun tion) an be therefore be written as 1 (S;S ) X i V (S ; S ) = f (S )E e E e (0;S ) i=0
f (S )E e (S;S ) : = 1 E [e (0;S ) ℄
To simplify the notation, let (S; S ) = E e (S;S ) ; so the value fun tion is f (S ) (S; S ) V (S ; S ) = : 1 (0; S ) From the de nition of it is lear that (S ; S ) = 0 so (S ; S ) = 1. Hen e the boundary ondition that f (S ) V (S ; S ) = : 1 (0; S ) Combining this with the lower boundary ondition f (S ) (0; S ) V (0; S ) = 1 (0; S ) leads to the value mat hing ondition that V (S ; S ) = V (0; S ) + f (S ): Noti e that value mat hing does not indi ate anything about the optimality of the hoi e of S . One way to obtain an optimality ondition is to set the derivative
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES of
V (S; S )
is, for every 0
with respe t to
S,
S
f (S ) (S; S
) + f (S )
391
equal to zero. After suitable rearrangement the FOC
(S; S ) (S; S ) (0; S ) + S 1 (0; S ) S
= 0:
(23)
In order to show that this is equivalent to the smooth pasting ondition we will use two properties of
.
First,
(S ; S ) is identi ally equal to 1, so (S ; S )=S = 0.
Combined with the fa t that
dS dS S =S
= 1;
this implies
d (S ; S ) dS
=
(S ; S ) (S ; S ) + S S
=0
and hen e that
(S ; S ) S
=
(S ; S ) : S
The se ond fa t, a result of the Markov assumption, is that
(S; S + dS ) = (S; S ) (S ; S + dS ): taking limits as
dS ! 0 we
(S ; S ) = (S; S ) : S we evaluate (23) at S = S and rearrange, it is straightforward to see (S; S ) S
If
see that
0
f (S
)
= = = = =
(S ; S ) (S ; S ) (0; S ) f (S ) + S 1 (0; S ) S (0; S ) (S ; S ) f (S ) 1 + 1 (0; S ) S f (S ) (S ; S ) 1 (0; S ) S f (S ) (S ; S ) 1 (0; S ) S V (S ; S ) S
whi h is the desired result.
that
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
392
Appendix C Deterministi Bang-Bang Problems
The general form for a deterministi bang-bang type problem has a reward fun tion f0 (S ) + f1 (S )x
state dynami s dS = [g0 (S ) + g1 (S )x℄dt
and ontrol onstraint xa
x xb :
Suppose we use a ontrol, not ne essarily optimal, with S as a swit hing point, e.g., set x = xa for S < S and x = xb for S > S .18 At S = S we hoose x in su h a way that dS=dt = 0. Summarizing, de ne 8 > xa if S < S a > > < g0 (S ) x(S; S a ) = if S = S a ; g1 (S ) > > > : x if S > S a b
with xa < g0 (S a )=g1 (S a ) < xb . The value fun tion satis es the di erential equation h i 1 f0 (S ) + f1 (S )x(S; S a ) + g0 (S ) + g1 (S )x(S; S a ) VS (S; S a ) ; (24) V (S; S a ) =
whi h, evaluated at S = S , yields a 1 a a a a g0 (S ) V (S ; S ) = f0 (S ) f1 (S ) : g1 (S a )
(25)
In spite of the dis ontinuity of the ontrol at S , the value fun tion is ontinuous, as is readily apparent by writing it as Z1 a V (S; S ) = e t (f0 (S ) + f1 (S )x(S; S a )) dt; 0
and noting that as S approa hes S from below (above), the amount of time during whi h the ontrol is set at xa (xb ) goes to 0. 18 This
assumes that the state is growing when
a simple matter to reverse these inequalities.
xa
is used and is shrinking when
xb
is used. It is
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
393
The ontinuity of V an be used to demonstrate the ontinuity of VS (S; S ) at S = S , and to thereby determine its value:19 f1 (S a ) a a : (26) VS (S ; S ) = g1 (S a ) So far, however, we have only onsidered the value fun tion for the ontrol S . To
hoose the ontrol optimally, we must pi k S to satisfy VS a (S; S a ) = 0:
For S 6= S a we an di erentiate (24) to see that h i VS a (S; S a ) = 1 f1 (S ) + g1 (S )VS (S; S a ) xS a (S; S a ) +
(27)
g1 (S )x(S; S a )VSS a (S; S a ):
However, ex ept at S = S , xS (S; S ) and VSS a (S; S a ) are zero and hen e we only need to set this derivative to zero at S = S . (27) is not well de ned at S = S be ause the derivative xS (S; S ) is unde ned at this point. Instead we use the relationship dV (S a ; S a ) = VS (S a ; S a ) + VS a (S a ; S a ): a dS
Rearranging this and using (25) and (26) we get dV (S a ; S a ) a a a VS (S ; S ) = VS (S a ; S a ) a dS a a f1 ( S a ) d f ( S ) g ( S ) 0 0 g1 (S a ) f1 (S a ) 1 + : = dS a g1 (S a ) 19 To determine the limit from below, note that ontinuity of
%
lim
S Sa
V (S; S a )
= = =
%
lim
1
S Sa 1
1
a
[f0 (S ) + f1 (S )xa + (g0 (S ) + g1 (S )xa ) VS (S; S )℄
f0 (S a )
f1 (S a )g0 (S a ) g1 (S a )
Rearranging, we see this expression implies that
a a (g0 (S ) + g1 (S )xa )
%
implies that
f0 (S a ) + f1 (S a )xa + (g0 (S a ) + g1 (S a )xa )
V
lim
S Sa
V (S; S a )
= =
VS (S; S a ) S %S a lim
V (S a ; S a ):
f1 (S a )g0 (S a ) f1 (S a )xa + g1 (S a ) a f1 (S ) a a g0 (S ) + g1 (S )xa : g1 (S a )
The same exer ise an be applied to solving for the limit from above.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
394
Thus the optimal swit hing points are found by solving for the roots of this expression. It is possible that there are multiple roots, leading to a situation in whi h VS may be dis ontinuous at a root; this root represents an unstable equilibrium at whi h x is unde ned.
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES
395
Bibliographi Notes
Arbitrage methods for solving nan ial asset pri ing problems originated with Bla k and S holes and Merton. The literature is now vast. A good introdu tory level dis ussion in found in Hull. For a more hallenging dis ussion, see DuÆe. The mathemati al foundations for modern asset pri ing theory are dis ussed at an introdu tory level in Hull and Neft i. See also Shimko. Dis ussions of exoti option pri ing models are found in Hull and Wilmott. Goldman et al. ontains the original derivation of the boundary ondition for lookba k options. AÆne di usion models are dis ussed in DuÆe and Kan, Dai and Singleton and Fa kler. Introdu tory level treatments of sto hasti ontrol problems are available in Dixit and Dixit and Pindy k. These books ontain numerous examples as well as links to other referen es. A more rigorous treatment in found in Fleming and Rishel. The renewable resour e harvesting problem is from Pindy k, the optimal investment from Cox et al., the portfolio hoi e example from Merton (1969) and Merton (1971). Boundary onditions asso iated with sto hasti pro esses are dis ussed by Feller, who devised a lassi ation s heme for di usion pro esses with singular boundaries (see dis ussion by Bharu ha-Reid, se . 3.3, and Karlin and Taylor (1981), Chap. 15.). Kamien and S hwartz is a lassi text on solving dynami optimization problems in e onomi s; its primary fo us is on deterministi problems solved via al ulus of variations and use of Hamiltonian methods but ontains a brief treatment of dynami programming and ontrol of Ito pro esses (Chapters 20 and 21). Other useful treatments of deterministi problems are found in Dorfman and Chiang. Malliaris and Bro k ontains an overview of Ito pro esses and sto hasti ontrol, with numerous examples in e onomi s and nan e. DuÆe ontains a brief introdu tory treatment of sto hasti ontrol, with a detailed dis ussion of the portfolio hoi e problem rst posed in Merton (1969) and Merton (1971). A standard advan ed treatment of sto hasti
ontrol is Fleming and Rishel. Free boundary problems are in reasingly ommon in e onomi s. Dixit (1991), Dixit (1993a) and Dixit and Pindy k ontain useful dis ussions of these problems. Several of the examples are dis ussed in these sour es. Dixit (1993b) provides a good introdu tion to sto hasti ontrol, with an emphasis on free boundary problems. Dumas dis usses optimality onditions; see also the arti les in Lund and Oksendal. Ho man and Sprekels and Antonsev et al. are pro eedings of onferen es on free boundary problems with an emphasis on problems arising in physi al s ien es. The original solution to the timber harvesting problem with replanting is attributed to Martin Faustmann, who dis ussed it in an arti le published in 1849. Irving Fisher dis ussed the related problem with abandonment in The Theory of Interest. For further dis ussion see Ga ney, Hershleifer. Re ently, ?? dis ussed the
CHAPTER 10. CONTINUOUS TIME - THEORY & EXAMPLES sto hasti version of the problem.
396
The entry/exit example originates with Brennan
and S hwartz and M Donald and Siegel.
Numerous authors have dis ussed renew-
able resour e management problems; see espe ially Mangel. The sto hasti bang-bang problem is dis ussed most fully in Ludwig and Ludwig and Varrah, where detailed proofs and a dis ussion of the multiple equilibria situation an be found. The proof in the appendix to this hapter is modeled after a similar proof in Ludwig.
The
learning-by-doing example is from Majd and Pindy k and is also dis ussed in Dixit and Pindy k.
Chapter 11 Continuous Time Models: Solution Methods In the previous hapter we saw how ontinuous time e onomi models, whether deterministi or sto hasti , result in solution fun tions that satisfy di erential equations. Ordinary di erential equations (ODEs) arise in in nite horizon single state models or in deterministi problems solved in terms of time paths. Partial di erential equations (PDEs) arise in models with multiple state variables or in nite horizon ontrol problems. From a numeri al point of view the distin tion between ODEs and PDEs is less important than the distin tion between problems whi h an be solved in a re ursive or evolutionary fashion or those that require the entire solution be omputed simultaneously be ause the solution at one point (in time and/or spa e) depends on the solution everywhere else. This is the distin tion between initial value problems (IVPs) and boundary value problems (BVPs) that we dis ussed in Se tions 5.7 and 6.8.3. With an IVP, the solution is known at some point or points and the solution near these points an then be (approximately) determined. This, in turn, allows the solution at still other point to be approximated and so forth. When possible, it is usually faster to use su h re ursive solution te hniques. Numerous methods have been developed for solving PDEs. We on entrate on a parti ular approa h that en ompasses a number of the more ommon methods and whi h builds ni ely on the material already overed in this book. Spe i ally, the true but unknown solution will be repla ed with a onvenient approximating fun tion, the parameters of whi h will be determined using ollo ation. For initial value type problems (IVPs), this approa h will be ombined with a re ursive algorithm. We will also dis uss free boundary problems. The basi approa h for su h problems is to solve the model taking the free boundary as given and then use the optimality ondition to identify the lo ation of the boundary. 397
CHAPTER 11.
398
CONTINUOUS TIME - METHODS
There are a number of methods for solving PDEs and sto hasti ontrol problems that we do not dis uss here. These in lude binary and trinomial tree methods, simulation methods and methods that dis retize ontrol problems and solve the related dis rete problem. Although all of these methods have their pla e, we feel that providing a general framework that works to solve a wide variety of problems and builds on general methods developed in previous hapters is of more value than an en y lopedi a
ount of existing approa hes. Mu h of what is dis ussed here should look and feel familiar to readers that have persevered up to this point.1 We do, however, in lude some referen es to other approa hes in the bibliographi al notes at the end of the
hapter. 11.1
Solving Arbitrage-based Valuation Problems
In the previous hapter it was shown that nan ial assets often satisfy an arbitrage
ondition in the form of the PDE
r(S )V = Æ (S ) + Vt + VSS (S ) + 21 tra e( (S ) (S ) VSS ): >
The spe i asset depends on the boundary onditions. For an asset that has a single payout at time T , the only boundary ondition is of the form V (S; T ) = R(S ) and, as there are no dividends, Æ = 0. For zero- oupon default-free bonds the boundary
ondition is R(S ) = 1. For futures, European all options and European put options written on an underlying asset with pri e p = P (S ), the boundary onditions are, respe tively, R(S ) = P (S ) and R(S ) = max(0; P (S ) K ) and R(S ) = max(0; K P (S )), where K is the option's strike pri e. Asset pri ing problems of this kind are more easily expressed in terms of time-tomaturity rather than alendar time; let = T t. We will work with V (S; ) rather than V (S; t), ne essitating a hange of sign of the time derivative: V = Vt . The problem, of ourse, is that the fun tional form of V is unknown. Suppose, however, it is approximated with a fun tion of the form V (S; ) (S ) ( ), where is a suitable n-dimensional family of approximating fun tions and ( ) is an n-ve tor of time varying oeÆ ients. When the state variable is one-dimensional, the arbitrage
ondition an be used to form a residual equation of the form2 h
(S ) ( ) (S ) (S ) + 12 2 (S ) (S ) 0
0
00
i
r(S ) (S ) ( ) = (S ) ( ):
1 It would be useful to at least be familiar with the material in Chapter 6.
(1)
2 For multi-dimensional states the prin iple is the same but the implementation is a bit messier. We dis uss this in Appendix 11.A.
CHAPTER 11.
CONTINUOUS TIME - METHODS
399
A ollo ation approa h to determining the ( ) is to sele t a set of n values for S , si , and to solve the residual equation with equality at these values. The residual fun tion an then be written in the form 0 ( ) = B ( ); where and B are both n n matri es. This is a rst order system of ordinary di erential equations in , with the known solution ( ) = exp( 1 B ) 0; where 0 satis es the boundary ondition R(S ) evaluated at the n values of the si (note: the exponential is a matrix exponential whi h an be omputed using the Matlab fun tion expm). Before illustrating this approa h, a few additional omments are in order. It may be desirable to impose additional boundary onditions. This would require rede ning and B using fewer than n nodes and to on atenate to these any additional equations needed to impose the boundary onditions. Generally, this is not needed if the behavior at the boundaries is regular enough. The issue be omes more riti al when free boundary problems are en ountered. Another issue is espe ially important in option pri ing problems, for whi h the terminal boundary is not smooth but rather has a kink at the strike pri e K . This suggests that the use of a polynomial approximation may not be appropriate. Instead a ubi spline, possibly with extra nodes at S = K , or a pie e-wise linear approximation with nite di eren e derivatives may be in order. Example: Pri ing Bonds
The CIR (Cox-Ingersoll-Ross) bond pri ing model assumes that the risk-neutral pro ess for the short interest rate is given by p dr = ( r)dt + rdz: Expressing the value of a bond in terms of time-to-maturity ( ), a bond paying 1 unit of a
ount at maturity, has value V (r; ) that solves V = ( r)Vr + 12 2 rVrr
rV;
with initial ondition V (r; 0) = 1. To solve this model, rst hoose a family of approximating fun tions with basis (r) and n ollo ation nodes, ri . Letting the basis fun tions and their rst two derivatives at these points be de ned as the n n matri es 0 , 1 and 2 , a system of ollo ation equations is given by 0 0( ) = [( r)1 + 21 2r2 r0 ℄ ( ) = B ( ):
CHAPTER 11.
CONTINUOUS TIME - METHODS
400
The term r0 is an abuse of notation; it indi ates multiplying the n 1 ve tor r by an n n matrix 0 . Su h a term is more properly written as diag(r)0 and an be obtained in Matlab using element-by-element multipli ation as r(:,ones(n,1)).*Phi0
(see ode below). The same omments also apply to the rst and se ond order terms. The following fun tion solves the CIR bond pri ing problem. fun tion = irbond(fspa e,tau,kappa,alpha,sigma) % Define nodes and basis r=funnode(fspa e); Phi0=funbas(fspa e,r,0); Phi1=funbas(fspa e,r,1); Phi2=funbas(fspa e,r,2); % Evaluate parameters m=kappa*(alpha-r); s=0.5*sigma.^2*r; % Define and solve the linear differential equation in the oeffi ients u=ones(size(r,1),1); B=m(:,u).*Phi1+s(:,u).*Phi2-r(:,u).*Phi0; B=Phi0\B;
0=Phi0\u;
=expm(full(tau*B))* 0;
The fun tion's input arguments in lude a fun tion de nition stru ture fspa e indi ating the family of approximating fun tions desired, the time to maturity, , and the model parameters, , and . The fun tion returns the oeÆ ient ve tor
. A s ript le, demfin01.m, demonstrates the use of the pro edure. It uses a Chebyshev polynomial approximation of degree n = 20 on the interval [0; 2℄. The solution fun tion for a 30-year bond with parameter values = 0:1, = 0:05 and = 0:1 is plotted in Figure 11.1. Two points are in order on erning the pro edure. First, Matlab's matrix exponential fun tion expm requires that its argument be a full rather than a sparse matrix. Some basis fun tions (e.g., spline and pie ewise linear) are stored as sparse matri es, so this ensures that the ode will work regardless of the family of fun tions used. Se ond, the pro edure uses the standard nodes for a given approximating family of fun tions. This typi ally requires upper and lower bounds to be spe i ed. For the pro ess used in the bond pri ing example, a natural lower bound of 0 an be used. The upper bound is tri kier, be ause the natural upper bound is 1. Knowledge of the underlying nature of the problem, however, should suggest an upper bound for the rate of interest. We have used 2, whi h should more than suÆ e for ountries that are not experien ing hyper-in ation.
CHAPTER 11.
401
CONTINUOUS TIME - METHODS
Zero−Coupon Bond Price 0.45
0.4
0.35
V(r)
0.3
0.25
0.2
0.15
0.1
0.05
0
0.05
0.1
0.15
0.2
0.25
r
Figure 11.1 More generally, one should use an upper bound that ensures the result is not sensitive to the hoi e in regions of the state spa e that are important. In pra ti e, this may ne essitate some experimentation. A useful rule of thumb is that the omputed value of V (S ) is not sensitive to the hoi e of S if the probability that ST = S, given St = S , is negligible. For in nite horizon problems with steady state probability distributions, one would like the steady state probability of S to be negligible. For this example, a known solution to the bond pri ing problem exists (see exer ise 10.2 on p. 376). The losed form solution an be used to ompute the approximation error fun tion, whi h is shown in Figure 11.2. The example uses a Chebyshev polynomial basis of degree n = 20; it is evident that this is more than suÆ ient to obtain a high degree of a
ura y. 11.1.1 Extensions and Re nements
The approa h to solving the asset pri ing problems just des ribed repla es the original arbitrage ondition with one of the form 0 ( ) = B ( ); with (0) = V0 . The known solution ( ) = exp( 1 B ) 1 V0 an be put in re ursive form
( + ) = exp( 1 B ) ( ) = A ( ):
CHAPTER 11.
402
CONTINUOUS TIME - METHODS
Zero−Coupon Bond Approximation Errors
−10
3
x 10
2
1
0
−1
−2
−3
−4
−5
0
0.05
0.1
0.15
0.2
0.25
r
Figure 11.2 The n n matrix A need only be omputed on e and the re ursive relationship may then be used to ompute solution values for a whole grid of evenly spa ed values of . In the approa h taken above, the existen e of a known solution to the ollo ation di erential equation is due to the linearity of the arbitrage ondition in V and its partial derivatives. If linearity does not hold, we will still be able to express the system in the form (t) = B ( (t)), whi h an be solved using any onvenient initial value solver su h as the Runge-Kutta algorithm des ribed in Se tion 5.7 or any of the suite of Matlab ODE solvers. This approa h has been alled the extended method of lines. The name omes from a te hnique alled the method of lines, whi h treats = I and uses nite di eren e approximations for the rst and se ond derivatives in S . The values ontained in the (t) ve tor are then simply the n values of V (s ; t). The extended method of lines simply extends this approa h by allowing for arbitrary basis fun tions. We should point out that the system of ODEs in the extended method of lines is often \sti ". This is a term that is diÆ ult to de ne pre isely and a full dis ussion is beyond the s ope of this book. SuÆ e it to say, a sti ODE is one that operates on very di erent time s ales. The pra ti al import of this is that ordinary evolutionary solvers su h as Runge-Kutta and its re nements must take very small time steps to solve sti problems. Fortunately, so- alled impli it methods for solving sti problems 0
n
i
CHAPTER 11.
CONTINUOUS TIME - METHODS
403
do exist. The Matlab ODE suite provides two sti solvers, ode15s and ode23s. It is also possible to use nite di eren e approximations for ; indeed, this is perhaps the most ommon approa h to solving PDEs for nan ial assets. Expressed in terms of time-to-maturity ( ), a rst order approximation with a forward di eren e (in ) transforms (1) to
( + ) ( ) = (S ) ( ); (S ) or, equivalently, (S ) ( + ) = [ (S ) + (S )℄ ( ): Expressing this in terms of basis matri es evaluated at n values of S leads to
( + ) = [In + 1 B ℄ ( ): This provides an evolutionary rule for updating ( ), given the initial values (0). [In + 1 B ℄ is a rst order Taylor approximation (in ) of exp( 1 B ). Hen e the rst order di eren ing approa h leads to errors of O(2). A ba kwards (in ) di eren ing s heme an also be used
( ) ( ) (S ) = (S ) ( ); leading to
( ) = [In 1 B ℄ ( ) or
( + ) = [In 1 B ℄ 1 ( ): [In 1 B ℄ 1 is also a rst order Taylor approximation (in ) of exp( 1 B ) so this method also has errors of O(2). Although it may seem like the forward and ba kwards approa hes are essentially the same, there are two signi ant di eren es. First, the ba kwards approa h de nes
( ) impli itly and the update requires a linear solve using the matrix [In 1 B ℄. The forward approa h is expli it and requires no linear solve. This point is relatively unimportant when the oeÆ ients of the di erential equation are onstant in be ause the inversion would need to be arried out only on e. The point be omes more signi ant when the oeÆ ients are time varying. As we shall see, this an happen even when the state pro ess has onstant (in time) oeÆ ients, espe ially in free boundary problems.
CHAPTER 11.
404
CONTINUOUS TIME - METHODS
The need for a linear solve would seem to make the ba kward (impli it) approa h less desirable. It is possible, however, that the expli it forward approa h is unstable. Both approa hes repla e the di erential system of equations with a system of di eren e equations of the form + = It is well known that su h a system is explosive if any of the eigenvalues of are greater than 1 in absolute value. In appli ations of the kind found in nan ial appli ations, the matrix = [ + 1 ℄ an be assured of having small eigenvalues only by making small enough. On the other hand, the impli it method leads to a di eren e equation for whi h = [ 1 ℄ 1, whi h an be shown to be stable for any . Pra ti ally speaking, this means that the expli it may not be faster than the impli it method and may produ e garbage if is not hosen properly. If the matrix is explosive, small errors in the approximation will be magni ed as the re ursion progresses, ausing the
omputed solution to bear no resemblan e to the the true solution. x
Ax :
A
A
In
B
A
In
B
A
A Matlab Asset Pri ing Fun tion: FINSOLVE
Due to the ommon stru ture of the arbitrage pri ing equation a ross a large lass of nan ial assets, it is possible to write general pro edures for asset pri ing. Su h a pro edure, finsolve, for solving an arbitrage ondition for an asset that pays no dividends is shown in below. The fun tion requires ve inputs, model, fspa e, alg, snodes and N. The rst input, model, is a stru ture variable with the following elds: fun the name of the problem de nition le T the time to maturity of the asset params a ell array of additional parameters to be passed to model.fun A template for the fun tion de nition le is: out1=fun (flag,S,t,additional parameters); swit h flag
ase 'rho' out1 = instantaneous risk-free interest rate
ase 'mu' out1 = drift on the state pro ess
ase 'sigma' out1 = volatility on the state pro ess
ase 'V0' out1 = exer ise value of the asset end
CHAPTER 11.
CONTINUOUS TIME - METHODS
405
The fun tion uses the modi ed method of lines by default if alg is unspe i ed but expli it (forward) or impli it (ba kward) nite di eren es an also be used to represent the derivative in by spe ifying the alg argument to be either 'impli it' or 'expli it' (the default is 'lines'). In addition, a method known as the CrankNi holson method, whi h averages the impli it and expli it methods, an be obtained by spe ifying 'CN'. fun tion [ ,V,A℄=finsolve(model,fspa e,alg,s,N) if ~exist('alg','var') | isempty(alg), alg='lines'; end if ~exist('N','var') | isempty(N), N=1; end probfile=model.fun ; T=model.T; n=prod(fspa e.n); % Compute ollo ation matrix mu=feval(probfile,'mu',s,[℄,model.params{:}); sigma=feval(probfile,'sigma',S,[℄,model.params{:}); rho=feval(probfile,'rho',S,[℄,model.params{:}); V0=feval(probfile,'V0',S,[℄,model.params{:}); n=fspa e.n; Phi0=funbas(fspa e,s,0); % ompute basis matri es Phi1=funbas(fspa e,s,1); Phi2=funbas(fspa e,s,2); v=0.5*sigma.*sigma; u=ones(n,1); B=mu(:,u).*Phi1+v(:,u).*Phi2-rho(:,u).*Phi0; B=funfitxy(fspa e,Phi0,B);
0=funfitxy(fspa e,Phi0,V0); Delta=T/N; swit h method
ase 'lines' A=expm(full(Delta*B));
ase 'expli it' A=eye(n)+Delta*B;
ase 'impli it' A=inv(eye(n)-Delta*B);
ase 'CN'
CHAPTER 11.
CONTINUOUS TIME - METHODS
406
A=(inv(eye(n)-Delta*B) + eye(n)+Delta*B)/2; otherwise error('Method option is invalid') end
=zeros(n,N+1);
(:,1)= 0; for i=2:N+1
(:,i)=A* (:,i-1); end
The next se tion uses this fun tion to solve the Bla k-S holes option pri ing model (the le demfin01 optionally uses finsolve to solve the CIR bond pri ing example).
Matlab
Example: Bla k-S holes Option Pri ing Formula
In Se tion 10.1, the Bla k-S holes option pri ing formula was introdu ed. The assumption underlying this formula is that the pri e of a dividend prote ted sto k has risk-neutral dynami s given by dS = rSdt + Sdz:
The arbitrage ondition is V = rSVS + 21 2 S 2 VSS
rV
with the initial ondition V (S; 0) = max(S K; 0). A s ript le using the finsolve fun tion is given below (see demfin02) . The family of pie e-wise linear fun tions with nite di eren e approximations for the derivatives (with pre x lin) is used. 50 evenly spa ed nodal values on [0; 2K ℄ are used, along with 75 time steps using the impli it (ba kward in ) method. Noti e that the fun tion finsolve returns an n N + 1 matrix of oeÆ ients; the rst olumn
ontains oeÆ ients that approximate the terminal value. If only the values for timeto-maturity T are desired, all but the last olumn of the output an be dis arded (hen e the = (:,end); line). The approximation an be evaluated at arbitrary values of S using funeval( ,fspa e,S). The delta and gamma of the option an also be evaluated using funeval. % Define parameters r=.05; delta=0; sigma=0.2; K=1; T=1; put=0;
CHAPTER 11.
CONTINUOUS TIME - METHODS
407
lear model model.fun ='pfin02'; model.T=T; model.params={r,delta,sigma,K,put}; n=50; fspa e=fundefn('lin',n,log(K/3),log(3*K)); s=funnode(fspa e); % Call solution algorithm
=finsolve(model,fspa e,'impli it',s,75);
= (:,end);
The problem de nition le for this example follows: fun tion out1=pfin02(flag,S,t,r,delta,sigma,K,put); n=size(S,1); swit h flag
ase 'rho' out1=r+zeros(n,1);
ase 'mu' out1= (r-delta-0.5*sigma.^2);
ase 'sigma' out1=sigma;
ase 'V0' if put out1=max(0,K-exp(S)); else out1=max(0,exp(S)-K); end end
As a losed form solution exists for this problem, we an plot the approximation errors produ ed the method (the pro edure Bla kS h is available in the CompE on toolbox). These are shown in Figure 11.3. The maximum absolute error is 5:44 10 4. It is simple to experiment with the alternate methods by hanging the alg argument to the finsolve fun tion. The family of approximating fun tions an also be hanged easily by hanging the input to the fundef fun tion. The approximation errors in Table 11.1 were obtained in this manner. The approximation errors for all of these methods are roughly equivalent, with a slight preferen e for the ubi spline. Note, however, that the expli it approa h
CHAPTER 11.
408
CONTINUOUS TIME - METHODS
Table 11.1: Option Pri ing Approximation Errors
Method Fun tion Family Lines Impli it Expli it Pie ewise-linear 4.85 5.44 5.31 2.89 2.31 3.20 Cubi Splines Maximum absolute errors on [0; 2K ℄ times 10 4 . Expli it ubi spline uses 250 time steps; other methods use 50 steps.
Call Option Approximation Errors
−3
2
x 10
1.5
1
0.5
0
−0.5
−1 −1.5
−1
−0.5
0
0.5
1
1.5
S
Figure 11.3 using the ubi spline basis is explosive with under 200 time steps and the table gives approximation errors using 250 time steps. It should also be noted that a polynomial approximation does a very poor job in this problem due to its inability to adequately represent the initial ondition, whi h has a dis ontinuity at K in its rst derivative, and, more generally, be ause of the high degree of urvature near S = K . 11.2
Solving Sto hasti Control Problems
In the previous hapter we saw that for problems of the form Z 1 V (S ) = max e f (S; x)d ; s.t. dS = g (S; x)dt + (S )dz; x(S )
t
CHAPTER 11.
CONTINUOUS TIME - METHODS
Bellman's equation takes the form V (S ) = max f (S; x) + g (S; x)V 0 (S ) + xS ( )
1 2
409
2 (S )V 00 (S );
possibly subje t to boundary onditions. When the fun tional form of the solution is unknown, the basi strategy for solving su h problems will be essentially the same as in the dis rete time ase. The value fun tion will be approximated using V (S ) (S ) , where is an n-ve tor of oef ients. For in nite horizon problems, an be found using either a value fun tion and a poli y fun tion iteration. Given a guess of the value of , the optimal value of the ontrol an be solved (in prin iple) for a set of nodal values of S . (si ) = max f (si ; xi ) + g (si ; xi ) 0 (si ) + (si ) 00 (si ) : x 1 2
i
2
This leads to the rst order onditions fx (si ; xi ) + gx (si ; xi ) 0 (si ) = 0; whi h may admit a losed form solution, xi = x (si; ). If there are no relevant boundary onditions, n values of S are used to form the n n basis matri es , and . The three ve tors de ned by fi = f (si ; xi ), mi = g (si; xi ) and vi = 0:5 (si ) are also omputed. A fun tion iteration algorithm
an then be expressed as 1 f + [m + v ℄
0
0
1
1 2
2
1
2
(as noted earlier, terms like m are an abuse of notation and signify diag(m) ). Poli y fun tion iteration uses
= [ m v ℄ f: If there are relevant boundary onditions, the number of nodal values of S an be less than n by the number of additional onditions. Generally boundary onditions are linear in the value fun tion and its derivatives and hen e are linear in the approximation oeÆ ients. These onditions an, therefore, be appended to the ollo ation
onditions from the Bellman's Equation and an be updated in essentially the same manner. This approa h will be used extensively for solving free boundary in the next se tion. An alternative when one an solve expli itly for the optimal ontrol (in terms of the value fun tion) is to substitute the ontrol out of the Bellman Equation. This results in (generally) a nonlinear di erential equation in S , whi h an be solved dire tly 1
0
1
2
1
1
CHAPTER 11.
CONTINUOUS TIME - METHODS
410
using ollo ation. If the di erential equation is nonlinear, however, the ollo ation equations are also nonlinear and hen e must be solved using a root nding algorithm. A general solver routine for sto hasti ontrol problems with ontinuous ontrol variables an be developed along the same lines as the solver for the dis rete time
ase (dis ussed beginning on p. ??). The method starts with an initial guess of the
oeÆ ients of an approximation of the value fun tion. It then omputes the optimal
ontrol given this guess. The value fun tion oeÆ ients are then updated using either poli y or fun tion iteration. This iterative pro ess o
urs until a onvergen e riterion is met. There are, in fa t, some distin t omputational advantages to using ontinuous time models be ause of three related fa ts. First, the Bellman equation is not sto hasti ; there is no need to perform numeri al integration to ompute an expe tation. Se ond, to implement a Newton solution to the optimization problem, the rst order
ondition, fx (s; x) + Vs (s)gx(s; x) = 0, and its derivative need only be evaluated at nodal values of the state. Third, the rst order ondition is relatively simple and an often be solved expli itly in the form x = x(s; Vs ), thereby eliminating entirely the need to perform use numeri al optimization methods to determine the onditional optimal ontrol. Even if no expli it solution is available, the omputation of se ond derivative of the onditional value fun tion is easier.3 In the dis rete time ase vxx
= fxx + Vsgxx + gx>Vss gx:
In the ontinuous time ase, the last of these terms does not appear be ause the value of the state at whi h V is evaluated is independent of x. A general solver for ontinuous time sto hasti ontrol problems is provided by s solve. This solver requires that the user spe ify a model stru ture and problem de nition le as well as a family of approximating a fun tions, a set of ollo ation nodes and initial values for the value fun tion at the ollo ation nodes. Optionally, an initial value for the optimal ontrol may be passed. The model stru ture should 3 If
the varian e term is a fun tion of x, the omputation of the rst and se ond derivatives must a
ount for this.
vx = fx + Vs gx + ve (Vss )> x and
vxx = fxx + Vs gxx + x > (Id Vss )x + ve (Vss )> xx : Here x is d2 p and x is d2 p p. The matrix produ t ve (Vss )> xx multiplies a 1 d2 matrix by a d2 p p array and returns a p p array. In this type of matrix multipli ation is a
omplished by reshape(A*reshape(B,d*d,p*p),p,p).
Matlab
CHAPTER 11.
CONTINUOUS TIME - METHODS
411
ontain the following elds: fun the name of the problem de nition le params a ell array of additional parameters to be passed to model.fun For many ontinuous time problems, a solution to the rst order onditions (as a fun tion of S and VS ) an be obtained in losed form. If this is the ase, the problem de nition le should return these values when a flag value of x is passed. A template for the problem de nition le is: fun tion out1=probdef(flag,s,x,Vs,additional parameters) swit h flag
ase 'x' out1 = optimal value of the ontrol
ase 'f' out1 = reward fun tion
ase 'g' out1 = drift term in state transition equation
ase 'sigma' out1 = diffusion term in state transition equation
ase 'rho' out1 = dis ount rate end
A simpli ed (one-dimensional state and a tion) version of the solver is provided below. This version assumes that the optimal ontrol an be expli itly al ulated. % SCSOLVE1 Solves sto hasti ontrol problems % Simplified to handle the single state, single ontrol ase fun tion v=s solve1(model,fspa e,snodes,v0); maxiters=100; tol=sqrt(eps); % Compute part of ollo ation matrix that does not depend on x u=ones(1,fspa e.n); rho=feval(s file,'rho',s,[℄,[℄,varargin{:}); Phi0=funbas(fspa e,s,0); B=rho(:,u).*Phi0; sigma=feval(s file,'sigma',s,[℄,[℄,varargin{:}); sigma=0.5*sigma.*sigma; B=B-sigma(:,u).*funbas(fspa e,s,2); % The part of the ollo ation matrix that does depend on x Phi1=funbas(fspa e,s,1);
CHAPTER 11.
CONTINUOUS TIME - METHODS
412
% Initialize oeffi ient and ontrol ve tors if isempty(v0)
=zeros(fspa e.n,1); v0=zeros(fspa e.n,1); else
=Phi0\v0; end v=v0; % Poli y fun tion iteration loop for i=1:maxiters Vs=Phi1* ; x=feval(s file,'x',s,[℄,Vs,varargin{:}); f=feval(s file,'f',s,x,[℄,varargin{:}); g=feval(s file,'g',s,x,[℄,varargin{:});
=(B-g(:,u).*Phi1)\f; v0=v; v=Phi0* ; e=max(abs(v-v0)); if e=maxiters, % print warning message disp(['Algorithm did not onverge. Maximum error: ' num2str(e)℄); end
If no expli it solution for the optimal ontrol exists, the problem de nition le must return derivative information that an be used to numeri ally solve for the optimal ontrol. In this ase the a template for the problem de nition le is: fun tion [out1,out2,out3℄=probdef(flag,s,x,Vs,additional parameters) swit h flag
ase 'f' out1 = reward fun tion out2 = derivative of the reward fun tion with respe t to x out3 = se ond derivative of the reward fun tion with respe t to x
ase 'g' out1 = drift term in state transition equation out2 = derivative of the drift fun tion with respe t to x out3 = se ond derivative of the drift fun tion with respe t to x
ase 'sigma' out1 = diffusion term in state transition equation
ase 'rho'
CHAPTER 11.
CONTINUOUS TIME - METHODS
413
out1 = dis ount rate end
Example: Renewable Resour e Management
The problem of optimally managing a renewable resour e was dis ussed beginning on page 345. There are a total of 9 parameters in the model, , , K , b, , , , and . The problem de nition le for this example is provided below. It is ompli ated by the need to handle separately the ases in whi h = 1 or = 0 in order to avoid division by 0. out1=ps 01(flag,s,x,Vs,alpha,beta,K,b,eta,C,gamma,sigma,rho) swit h flag
ase 'x' Cost=C*s.^(-gamma); out1=b*(Cost+Vs).^(-eta);
ase 'f' Cost=C*s.^(-gamma); if eta~=1 % handle demand elasti ity 1 fa tor1=1-1/eta; % ase separately to avoid fa tor0=b.^(1/eta)/fa tor1; % division by 0; see iteration loop out1=fa tor0*x.^fa tor1-Cost.*x; else % demand elasti ity = 1 out1=b*log(x)-Cost.*x; end
ase 'g' if beta~=0 % need to handle beta=0 Growth=alpha/beta*s.*(1-(s/K).^beta); % ase separately else % to avoid division by 0 Growth=alpha*s.*log(K./s); end out1=Growth-x;
ase 'sigma' out1=sigma*s;
ase 'rho' out1=rho+zeros(size(s,1),1); end
Noti e that an expli it expression for the optimal ontrol is used so there is no need to provide derivative information.
CHAPTER 11.
CONTINUOUS TIME - METHODS
414
A s ript le using s solve to obtain a solution is provided below. The parameter values used are = 0:5, = 1, K = 1, b = 1, = 0:5, = 0:1, = 2, = 0:05, and = 0:1. The natural state spa e is S 2 [0; 1): As dis ussed below, an approximation is feasible only over a bounded interval. The demonstration uses a pie ewise linear approximation with 75 breakpoints, evenly spa ed on the interval [0:2; 1:2℄. % Set parameter values beta=1; eta=0.5; gam=2; alpha=0.5; K=1; b=1; C=5; rho=0.05; sigma=0.1; beta=1; eta=0.5; gam=2; % Define model variable model.fun ='ps 01'; model.params={alpha,beta,K,b,eta,C,gam,sigma,rho}; % Define the approximating family and nodes lo=0.2; hi=1.2; n=75; fspa e=fundefn('plin',n,lo,hi); s=funnode(fspa e); % Call the sto hasti ontrol solver
v=s solve(model,fspa e,s);
An expli it solution exists for the parameter values used in the demonstration, as displayed in Table 10.1 on page 347. The demonstration le plots the relative error fun tions over the range of approximation for the marginal value fun tion and optimal ontrol (i.e., 1 V^ =V and 1 x^=x). The resulting plot is displayed in Figure 11.4. It an be seen that the errors in both fun tions are quite small, ex ept at the upper end of the range of approximation. As with all problems involving an unbounded state spa e, a range of approximation must be sele ted. For in nite horizon problems, the general rule of thumb is that the range should in lude events that the stationary (long-run) distribution pla es a non-negligible probability of o
urren e. In this ase, there is little probability that the resour e sto k, optimally harvested, will ever be lose to the biologi al arrying
apa ity of K . It is also never optimal to let the sto k get too small.4 The renewable resour e problem has a natural lower bound on S of 0. It turns out, however, that 0 is a poor hoi e for the lower bound of the approximation be ause the value fun tion goes to 1 and the marginal value fun tion to 1 as S ! 0. Su h behavior is extremely hard to approximate with splines or polynomials. In lusion 4 For
the parameters values used in the example, the long run distribution has a known losed form. It an be shown that the probability of values of S less that 0.2 or greater than 0.8 are e e tively 0.
CHAPTER 11.
415
CONTINUOUS TIME - METHODS
Renewable Resource Approximation Errors
−3
6
x 10
Marginal Value Function Optimal Control (x100) 4
2
0
−2
−4
−6
−8
−10
−12
−14 0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
S
Figure 11.4 of basis fun tions that exhibit su h behavior is possible but requires more work. It is also unne essary, be ause the approximation works well with a smaller range of approximation. In pra ti e, of ourse, we may not know the stationary distribution or how large the errors are. There are several methods to address whether the approximation
hoi es are good ones. First, he k whether the approximation provides reasonable results. Se ond, he k the residual fun tion at inter-nodal values of S . Third, he k whether in reasing the range of approximation hanges the value of the approximating fun tion in a signi ant manner. Before leaving this example, a te hnique for improving the a
ura y of a solution will be demonstrated. For a number of reasons, a pie ewise linear approximation with nite di eren e derivatives is a fairly robust hoi e of approximating fun tions. It is a shape preserving hoi e, in the sense that an approximation to a monotoni and/or
onvex fun tion will also be monotoni and/or onvex. This makes it far more likely that the iterations will onverge from arbitrary starting values. The pie ewise linear approximation, however, requires a large number of ollo ation nodes to obtain a
urate approximations. It may therefore be useful to obtain a rough solution with a pie ewise linear approximation and use that approximation as a starting point for a se ond, more a
urate approximation. The following ode fragment ould be appended to the s ript le on page 414 to obtain a polynomial approximation on the same interval.
CHAPTER 11.
CONTINUOUS TIME - METHODS
416
s2=linspa e(lo,hi,101)'; fspa e2=fundefn(' heb',35,lo,hi);
v2=s solve(model,fspa e,s2,funeval( v,fspa e,s2));
This ode fragment de nes a new family of approximating fun tions and uses the previously omputed solution to provide starting values. The polynomial approximation with 35 ollo ation nodes produ es an approximation with a relative error of under 10 10 ex ept at the upper end of the range of approximation (above 0.9). Example: Optimal Growth
The neo lassi al optimal growth model, dis ussed beginning on page 353, solves Z1 max e t U (C )dt; C (t)
0
subje t to the state transition fun tion K 0 = q (K ) C . This problem ould be solved using the s solve fun tion; we leave this as an exer ise (page 452). The optimal
ontrol (C ) satis es the Euler ondition U 0 (C ) 0 (q (K ) ) = (q (K ) C )C 0 (K ); U 00 (C ) a rst order di erential equation. The solution C (K ) passes through the steady state point (K ; C ) whi h simultaneously solves dK=dt = 0 and the Euler ondition: q 0 (K ) = C = q (K ):
Our solution approa h requires that three fun tions be de ned. The rst is q (K ), the se ond is s(K ) = q 0 (K ) , whi h measures the ex ess marginal produ tivity over the dis ount rate and the third is the absolute risk aversion fun tion, r(C ) = U 00 (C )=U 0 (C ). The steady state apital sto k solves s(K ) = 0. The optimal ontrol an be approximated on the interval [a; b℄ with a fun tion (K ) . The Euler ondition is used to form the residual fun tion s(K ) : 0 e(K ) = q (K ) (K ) 0 (K ) r( (K ) ) This equation an be solved at a set of nodal values, ki , simultaneously with the boundary ondition that (K ) C = 0.
CHAPTER 11.
CONTINUOUS TIME - METHODS
417
To make the problem on rete, de ne q (K ) = ln(K + 1) ÆK; implying that s(K ) = q 0 (K ) =
K +1
Æ
:
The steady state apital sto k is therefore K = Æ + 1. As the units in whi h K is denominated are arbitrary, it is onvenient to set K = 1 by de ning = 2(Æ + ). Furthermore, let the utility fun tion be of the onstant relative risk aversion (CRRA) form, U (C ) = (C 1 1)=(1 ), whi h implies that r(C ) = =C: The relative risk aversion parameter, , takes on values between 0 and 1, with 0 implying risk neutrality and 1 resulting in logarithmi utility. Spe i parameter values are = 0:05, Æ = 0:02, = 2( + Æ ) = 0:14 and = 0:5. A s ript le to solve the problem is displayed in Code Box 1. The ode uses \inline" fun tions to de ne q(K ), s(K ) and r(C ) at the beginning of the le. These three fun tions an be thought of as the parameters de ning the model and an be
hanged without altering the rest of the ode to explore the impli ations of alternative parameters. The next step is to determine the steady state values. We pass s(K ) to root nding algorithm broyden to obtain K . This is a bit gratuitous in this example given that we already know that K = 1, but it illustrates how K an be found if s(K ) = 0 annot be solved dire tly. Given the model parameters, the steady state
onsumption rate is C = 0:077 or 7:7% of the apital sto k. The next step is to set up the ollo ation problem. We use a Chebyshev polynomial approximation of degree n = 20. To impose the boundary ondition, it is ne essary, however, to solve the Euler equation at n 1 nodes and impose the additional restri tion that (K ) = C . In pra ti e, it turns out that it is useful to impose the additional restri tion that no onsumption an o
ur when the apital sto k is 0: C (0) = 0. Thus, we use n 2 ollo ation nodes on the interval [0; 2K ℄ for the Euler equation, plus the two boundary onditions. The Euler equation nodes used are simply the Chebyshev nodes for degree n 2. These n 2 nodes are then appended to 0 and K and the basis matri es for the fun tion and its derivative are
omputed (0 and 1). An initial oeÆ ient ve tor is t to the straight line through (0; 0) and (K ; C ) using the funfitxy pro edure. To solve the ollo ation problem, we will need to pass a residual fun tion to a root nding algorithm. The residual fun tion is de ned in a separate le shown in Code Box 2. The residual le takes values of , the approximation oeÆ ient ve tor, and returns the ollo ation residual ve tor. A number of pre omputed values are
CHAPTER 11.
CONTINUOUS TIME - METHODS
Code Box 11.1: Neo lassi al Growth Model
% DEMSC03 Neo- lassi al Optimal Growth Problem ( ont. time) % Define Problem Parameters % q(K)=alpha*log(K+1) - delta*K q=inline('0.14*log(K+1) - 0.02*K','K'); % s(K)=q'(K)-rho s=inline('0.14./(K+1) - 0.02 - 0.05','K'); % r(C)=-U''(C)/U'(C) r=inline('0.5./C','C'); % Find steady state solution Kstar=broyden(s,1); Cstar=q(Kstar); disp('Steady State Capital and Consumption') disp([Kstar Cstar℄) % Define nodes and basis matri es n=20; a=0; b=2*Kstar;
def=fundef({' heb',n-2,a,b}); K=[0;Kstar;funnode( def)℄;
def=fundef({' heb',n,a,b}); Phi0=funbas( def,K,0); Phi1=funbas( def,K,1); svals=s(K); qvals=q(K); % Get initial value of C, linear in K k=linspa e( def.a, def.b,301)';
=funfitxy( def,k,Cstar/Kstar*k);
=broyden('fs 03', ,[℄,r,svals,qvals,Phi0,Phi1,Cstar); % Plot optimal ontrol fun tion figure(1) C=funeval( , def,k); C(1)=0; plot(k,C,Kstar,Cstar,'*') title('Optimal Consumption Rule') xlabel('K') ylabel('C') % Plot residual fun tion figure(2) dC=funeval( , def,k,1); warning off % avoid divide by zero warning e=(q(k)-C).*dC-s(k)./r(C); warning on plot(k,e) title('Growth Model Residual Fun tion') xlabel('K') prtfigs(mfilename)
418
CHAPTER 11.
CONTINUOUS TIME - METHODS
419
passed to this fun tion as auxiliary parameters. These in lude the basis matri es, 0 and 1 , the steady state ontrol, C , and the risk aversion fun tion r(C ). The values of q(K ) and s(K ) are also needed to ompute the residual fun tion. Unlike the values of r(C ), however, these do not hange with , so they an be pre omputed and passed as ve tors rather than \inline" fun tions. The residual fun tion rst omputes the residuals for the Euler equation and then alters the rst two values to impose the boundary onditions at K = 0 and K = K . Code Box 11.2: Residual Fun tion for Neo lassi al Growth Model
% FSC03 Residual fun tion for neo lassi al optimal growth problem % See DEMSC03 for usage fun tion e=fs 03( ,r,svals,qvals,Phi0,Phi1,Cstar) C=Phi0* ; dC=Phi1* ; warning off e=(qvals-C).*dC-svals./feval(r,C); warning on e(1)=C(1); e(2)=C(2)-Cstar;
After using broyden to nd the oeÆ ient ve tor for the optimal onsumption fun tion, the demonstration le then produ es a plot of this fun tion, whi h is shown in Figure 11.5. The steady state equilibrium point (K ; C ) is marked with a \*". The demonstration ode also plots the residual fun tion de ned by the Euler equation. This provides an estimate of the error of the approximation. As seen in Figure 11.6 the residual fun tion is relatively small, even with a degree 20 approximation. 11.3
Free Boundary Problems
Many of the problems dis ussed in the previous hapter involved free boundaries whi h represent endogenously determined state values at whi h some a tion is taken. In their most simple form, with a single state variable in an in nite horizon situation, these problems involve solving a se ond order linear di erential equation of the form (S )V (S ) = f (S ) + (S )V 0 (S ) + 21 2 (S )V 00 (S ); (2) where this equation holds on some interval [a; b℄. The usual boundary value problem takes both a and b as known and requires boundary onditions su h as V (a) = ga and V (b) = gb to be met, where ga and gb are known values. As dis ussed in Se tion 6.8.3
CONTINUOUS TIME - METHODS
420
Optimal Consumption Rule 0.2
0.18
0.16
0.14
0.12
C
CHAPTER 11.
0.1
0.08
0.06
0.04
0.02
0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
1.6
1.8
2
K
Figure 11.5
Growth Model Residual Function
−4
1
x 10
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
0
0.2
0.4
0.6
0.8
1
1.2
K
Figure 11.6
1.4
CHAPTER 11.
421
CONTINUOUS TIME - METHODS
(page 164), one an approximate the solution using a fun tion parameterized by an n-ve tor , V (S ) (S ) , with is hosen so that (S ) satis es (2) at n 2 points and satis es the boundary onditions. This yields n equations in the n unknown parameters. In the free boundary problem one or both of the boundary lo ations a and b are unknown and must be determined by satisfying some additional onditions. Suppose, for example that the lo ation of the upper boundary, b, is unknown but is known to satisfy V (b) = hb , where hb is a known onstant. Thus there are three boundary
onditions and one additional parameter, b, implying that one must solve n + 1 equation in n + 1 unknowns. If both boundaries are free, with V (a) = ha , the problem be omes one with n + 2 equations and n + 2 parameters. A general strategy treats the solution interval [a; b℄ as known, nds an approximate solution on this interval using the di erential equations along with V (a) = ga and V (b) = gb . This allows the residual fun tions V (a) ha and V (b) hb to be de ned. These are passed to a root nding algorithm to determine the lo ation of the free boundaries. A problem with this method, however, is that the approximating fun tions we use typi ally de ne the fun tion in terms of its boundaries. Here, however, the interval on whi h the approximating fun tion is to be de ned is unknown. Fortunately, this problem is easily addressed using a hange in variable. To illustrate, onsider rst the ase in whi h b is unknown and, for simpli ity, a = 0. De ne 0
0
0
0
y = S=b; so the di erential equation is de ned on y 2 [0; 1℄. De ne the fun tion v (y ) su h that
V (S ) = v (y ): Using the hain rule it an be seen that
V (S ) = v (y ) 0
0
and
dy v (y ) = dS b 0
2
dy V (S ) = v (y ) dS 00
00
d2 y v (y ) v (y ) + v (y ) 2 = : dS b2 00
0
0
Inserting these de nitions into (2) demonstrates that the original problem is equivalent to 2 (by ) (by ) v (y ) + v (y ); (3) (by )v (y ) = f (by ) + b 2b 0
00
CHAPTER 11.
CONTINUOUS TIME - METHODS
422
for y 2 [0; 1℄, with v (0) = ga ; v (1) = gb ;
(4)
v 0 (1) = bhb :
(5)
and An approximation, v (y ; b) = (y ) (b) an be found for arbitrary values of b using (3) and (4). The optimal hoi e of b an then be determined using a root nding algorithm to solve (5) using 0 (1) (b) bhb = 0. More ompli ated problems will use a similar strategy, but may need to handle multiple value fun tions, multiple boundaries and/or boundaries that are urves or surfa es in the state spa e rather than isolated points. After illustrating the basi approa h in the simplest of ases, we dis uss ways to handle the ompli ations. Before pro eeding, we should point out that there is a vast and growing literature on free boundary problems. Our goal here is rather modest. We want to present a framework that an solve su h problems, while building on the methods already developed in this book. As su h, we are not attempting to provide the most eÆ ient approa hes to spe i problems, but rather ones that will provide useful answers without a lot of spe ial oding. We also make no attempt to provide general solvers for these models but rather demonstrate by example how they an be solved. Example: Asset Repla ement
The in nite time horizon resetting problem with a single state variable is perhaps the simplest example of a free boundary problem to solve. Re all the asset repla ement problem (page 359) in whi h an asset's produ tivity depends on its age, A, yielding a net return of Q(A)P . The Bellman equation is V (A) = Q(A)P + V 0 (A);
and applies on the range A 2 [0; A ℄, where A is the optimal repla ement age. The asset an be repla ed by paying a xed ost C. The boundary onditions are given by the value mat hing ondition: V (0) = V (A ) + C
and the optimality (smooth pasting) ondition: V 0 (A ) = 0:
CHAPTER 11.
423
CONTINUOUS TIME - METHODS
We rst transform the problem in terms of y = A=A , de ning v (y ) = V (A) and, therefore, v 0 (y ) = V 0 (A)A . Thus, the Bellman's equation an be expressed in terms of y : v (y ) = Q(A y )P + v 0 (y )=A
with boundary onditions v (0) v (1) = C and v 0 (1) = 0. We approximate the value fun tion using v (y ) (y ) , for y 2 [0; 1℄, where is an n-ve tor of oeÆ ients. Then the Bellman's equation an be expressed as the residual fun tion [ (y )
0 (y )=A℄ = Q(A y )P:
The Bellman equation is solved with equality at n 1 nodal points on [0; 1℄ simultaneously with the value-mat hing ondition [ (0) (1)℄ = C . This is a linear system in and hen e an be solved dire tly. The system of n 1 ollo ation equations plus the value-mat hing ondition provides a value of that is onditional on the hoi e of the free boundary A . This
hoi e is optimal when 0 (1) (A ) = 0; this is a single equation in a single unknown and is easily solved using a nonlinear root nding algorithm. The method is demonstrated below for the ase in whi h Q(A) = 1 + 0:05A 0:003A2 , P = 2, C = 3 and = 0:1. % DEMFB01 Asset Repla ement Demonstration % Define parameters Q=inline('1+.05*A-.003*A.^2','A'); P=2; C=3; rho=0.1; % Define nodes and basis matri es n=15;
def=fundefn(' heb',n-1,0,1); y=funnode( def);
def=fundefn(' heb',n,0,1); rPhi0=funbas( def,y); Phi1=funbas( def,y,1); phiVM=funbas( def,0)-funbas( def,1); phiSP=funbas( def,1,1); % Call rootfinder Astar=100; % initial guess
% % % %
rho*phi(y) phi'(y) phi(0)-phi(1) for value mat hing phi'(1) for smooth pasting
CHAPTER 11.
CONTINUOUS TIME - METHODS
424
Astar=broyden('ffb01',Astar,[℄,Q,P,C,y,rPhi0,Phi1,phiVM,phiSP); [e, ℄=ffb01(Astar,Q,P,C,y,rPhi0,Phi1,phiVM,phiSP);
The nodal values of y are de ned as the standard Chebyshev nodes for degree 1 rather then for degree n, to a
ommodate the addition of the value-mat hing
ondition. Furthermore, in addition to the basis matri es for the Bellman's equation (0 and 1 ), we pre ompute basis ve tors (1 n) for the value-mat hing and smoothpasting onditions. The latter ve tors are de ned as V M = (0) (1) and SP = 0 (1), respe tively. After setting A to an initial guess of 100, broyden is alled to solve for the optimal A . This requires that a separate residual fun tion le be de ned: n
fun tion [e, ℄=Repla eRes(Astar,Q,P,C,y,rPhi0,Phi1,phiVM,phiSP) B=[rPhi0-Phi1./Astar;phiVM℄; b=[feval(Q,Astar*y)*P;C℄;
=B\b; e=phiSP* ;
The residual fun tion rst determines (A) by solving 3 3 2 2 Q(A y1 )P (y1) 0 (y1 )=A 7 7 6 6 7 6 6
= 5: 4 (yn 1) 0 (yn 1 )=A 5 4 Q(A yn 1 )P 7 C (0) (1) It then returns the value e = 0(1) (A ). It will also return the omputed value of , whi h an then be used to evaluate v(y) and hen e V (A). A plot of V (A) is shown in Figure 11.7. The value-mat hing ondition appears in the gure as the di eren e of C = 3 between V (0) and V (A) (A 17:64). The smooth-pasting ondition appears as the 0 slope of V at A . Example: Investment Timing
In the previous example, a state variable was ontrolled in su h a way as to keep it within a region of the state spa e bounded by a free boundary. In other problems, the state an wander outside of the region de ned by the free boundary but the problem is either known or has no meaning outside of the region. In su h ase the value fun tion need only be approximated on the region of interest, using appropriate boundary onditions to de ne both the value fun tion and the boundary itself. From a omputational perspe tive su h problems require no new onsiderations. To illustrate, onsider a simple irreversible investment problem in whi h an investment of I will generate a return stream with present value of S , where S is des ribed by the Ito pro ess dS = (m S )Sdt + Sdz:
CHAPTER 11.
425
CONTINUOUS TIME - METHODS
Asset Replacement Value Function 2.5
2
1.5
V
1
0.5
0
−0.5
−1
0
5
10
15
20
25
30
35
A
Figure 11.7 This pro ess an be shown to have a mean reverting rate of return, with long-run mean m (see Appendix A, Se tion A.5.2). When the investment is made, it has net value S I . Prior to making the investment, however, the value of the right to make su h an investment is V (S ), whi h is the solution to the following di erential equation 1 2
2 S 2 V 00 (S ) + (m S )SV 0 (S ) rV (S ) = 0;
where r is the risk-free interest rate. The lower boundary, S = 0, is asso iated with an investment value of 0, be ause on e the pro ess S goes to 0, it stays equal to 0 forever; hen e V (0) = 0. The upper boundary is de ned as the value, S , at whi h investment a tually o
urs. At this value two onditions must be met. The value mat hing ondition states that at S the value of investing and not investing are equal: V (S ) = S I . The smooth-pasting optimality ondition requires that V 0 (S ) = 1. Applying the hange of variables (z = S=S ) yields the equivalent problem 1 2
2 z 2 v 00 (z ) + (m zS )zv 0 (z ) rv (z ) = 0;
(6)
on the interval [0; 1℄, with v (0) = 0, v (1) = S I , and v 0 (1) = S . To solve the problem we use an approximation of the form v (z ) (z ) . Chebyshev polynomials are a natural hoi e for this problem be ause v (z ) should be relatively smooth. The parameter ve tor and the optimal investment trigger S are sele ted to satisfy (6)
CHAPTER 11.
CONTINUOUS TIME - METHODS
426
at n 2 appropriately hosen nodes on the interior of [0; 1℄ (e.g., the roots of the order n 2 Chebyshev polynomial) and to satisfy the three boundary onditions. To make this a bit more expli it, given a guess of S , de ne the n 2 n matrix
B
Bij = 21 2 zi2 00j (zi ) + (m zi S )zi 0j (zi ) r j (zi ) for i = 1; : : : ; n 2. Then on atenate the basis fun tions for the boundary onditions to the bottom of this matrix: Bn 1;j = j (0) and Bn;j = j (1). This produ es an n n matrix. The oeÆ ients, onditional on the guess of S , are given by 0n 1 1 :
(S ) = B
S
I
Given we an de ne a residual fun tion in one dimension to solve for S using the smooth-pasting ondition:
e(S ) = S
0 (1) (S ):
This approa h works well in some ases but this example has one additional problem that must be addressed. For some parameter values, the approximate solution obtained be omes unstable, exhibiting wide os illations at low values of z . The solution value for S , however, remains reasonable. The problem, therefore, seems due to the approximation having trouble satisfying the lower boundary. It an be shown that, for some parameter values, the derivative of v be omes unbounded as S approa hes 0: lim V 0 (S ) = 1:
S &0
This type of behavior annot be well approximated by polynomials, the derivatives of whi h (at every order) are bounded on a bounded domain. Fortunately this problem an be easily addressed by simply eliminating the lower boundary onstraint and evaluating (6) at n 1 rather than n 2 nodes. This
auses some error at very small values of z (or S ) but does not ause signi ant problems at higher values of z . The e onomi ontext of the problem pla es far more importan e on the values of z near 1, whi h de nes the lo ation of S and hen e determines the optimal investment rule.
ode solving the problem using fun tion approximation is displayed in Code Boxes 3 and 4. This parti ular problem has a partially known solution. It an be shown that the solution an be written as
Matlab
V (S ) = AS H ( S ; ; );
CHAPTER 11.
CONTINUOUS TIME - METHODS
Code Box 11.3: Solution Fun tion for Optimal Investment Problem
% % % % % % % % % % % % % %
OPTINVEST Solves the optimal investment problem with mean-reverting return Finds the optimal investment rule for a proje t that has return pro ess dS=alpha(m-S)dt + sigma*SdW USAGE [e,vstar,V,dV℄=OptInvest(v,r,alpha,m,sigma,I,n); INPUTS r : interest rate alpha, m, sigma : return pro ess paramters I : fixed investment ost n : number of nodes used for Chebyshev ollo ation OUTPUTS Sstar : the trigger return level (invest if S>=Sstar)
: oeffi ients for value fun tion approximation fspa e : fun tion family definition stru ture
fun tion [Sstar, ,fspa e℄=optinvest(r,alpha,m,sigma,I,n) if nargin 0. Denoting these regions Sl and Su , the value fun tion must satisfy V
(g0 + g1 xl ) VS
1 2 2 VSS
(f0 + f1 xl ) = 0 on Sl
V
(g0 + g1 xu ) VS
1 2 2 VSS
(f0 + f1 xu ) = 0 on Su
and value-mat hing and smooth pasting at points where f1 = g1 VS (plus any additional boundary onditions at S = a and S = b). For on reteness suppose that there is a single point S su h that f1 (S ) = g1 (S )VS (S ) and that Sl onsists of points less than S and Su of points greater than S (usually the ontext of the problem will suÆ e to determine the general nature of these sets). The numeri al problem is to nd this S and the value fun tion V (S ). To solve this we will use two fun tions, one on Sl , the other on Su that approximately satisfy the Bellman equations and the boundary onditions and also that, for any guess of S , satisfy value mat hing and smooth pasting at this guess.
CHAPTER 11.
CONTINUOUS TIME - METHODS
431
Let the approximations be de ned by i(S ) i , for i = l; u and de ne the fun tion Bi (S ) as
Bi (S ) = i (S ) [g0 (S ) + g1 (S )xi ℄ 0i (S )
1 2
2 (S ) 00i (S )
The i an be determined by making
Bi (S ) i
[f0 (S ) + f1 (S )xi ℄ = 0
at a sele ted set of ollo ation nodes, together with the boundary onditions and
l (S ) l 0l (S ) l
u (S ) u = 0 (value mat hing) 0u (S ) u = 0 (smooth pasting).
Determining the i for some guess of S , therefore, amounts to solving a system of linear equations (assuming any additional boundary onditions are linear in V ). On e the i are determined, the residual
r(S ) = f1 (S ) + g1 (S ) 0l (S ) l
an be omputed. The optimal value of S is then hosen to make r(S ) = 0. Example: Optimal Fish Harvest
In the optimal sh harvesting problem (page 345) the value fun tion solves the oupled PDE
V =
S (1 S=K )VS + 12 2 S 2 VSS for S < S P HS + ( S (1 S=K ) HS ) VS + 12 2 S 2 VSS for S > S
with S determined by P = VS (S ) and ontinuity of V and VS at S . For simpli ity, we impose the s ale normalization P = K = 1 (by hoosing units for money and sh quantity). To solve this problem we rst transform the state variable by setting
y = ln(S ) ln(S ): This transformation has two e e ts: rst, it simpli es the di erential equation by making the oeÆ ients onstant or linear in S , and, se ond, it pla es the boundary between the two solution fun tions at y = 0. The transformation ne essitates rewriting the value fun tion in terms of y , say as v (y ). The transformation implies that
S = S ey ; SVS (S ) = vy (y )
CHAPTER 11.
432
CONTINUOUS TIME - METHODS
and
S 2 VSS (S ) = vyy (y ) vy (y ): The transformed Bellman equation with the s ale normalizations is
v =
1 2 1 2
2 vyy + (1 S ey ) 2 vyy + (1 S ey )
1 2 1 2
2 vy for y < 0 2 y H vy + S He for y > 0 :
It will be useful to rewrite this to isolate the S terms v
v
1 2 1 2
2 vy 12 2 vyy + S ey vy = 0 for y < 0 1 2 2 y y H vy 2 vyy + S e vy = S He for y > 0 :
The two fun tions are oupled by imposing ontinuity of v and vy at y = 0. Te hni ally there are also boundary onditions as y goes to 1 and 1 , but we will ignore these for the time being. Now let's approximate the two fun tions using l (y ) l and u(y ) u, where the i are ni -element basis ve tors and the i are the oeÆ ients asso iated with these bases. For a spe i guess of S , the Bellman equation an be written
Dl (y ) l + S [ ey 0l (y )℄ l = 0 for y < 0 : 0 y 0 y [Du (y ) + H u(y )℄ u + S [ e u (y )℄ u = S He for y > 0
where Di (y ) = i (y ) 21 2 0i(y ) 12 2 00i (y ). Evaluating this expression at a set of nodes, yl 2 [a; 0℄, and yu 2 [0; b℄, where a and b are arbitrary upper and lower bounds, with a < 0 and b > 0. The boundary onditions at y = 0 for a given S are
l (0) l
u (0) u = 0
0l (0) l
0u (0) u = 0:
and If we hoose yl and yu to have nl 1 and nu nl + nu system of linear equations: 02 B6 B6 4
Bl
0 l (0) 0l (0)
0
Bu u (0) 0u (0)
whi h has the form (B + S D) = S f:
3
2
Dl
6 7 7 + S 6 0 4 0 5
0
0
Du
1 elements, respe tively, this yields the 31
7C 7C 0 5A
0
l
u
2 6
=6 4
0
S Heyu 0 0
3 7 7: 5
CHAPTER 11.
433
CONTINUOUS TIME - METHODS
The unknowns here are S (a s alar) and (an n0 + n1 ve tor). The matri es B , D and f do not depend on either S or and therefore an be prede ned. Furthermore, this system of equations is linear in and hen e an be easily solved for a given S , thereby obtaining an approximation to the value fun tion, v . We an therefore view
as a fun tion of S :
(S ) = (B + S D) 1 S f: (7) The optimal S is then determined by solving the (non-linear) equation S 0 (0) (S ) = 0: l
Matlab
l
(8)
A implementation is displayed in Code Box 5. The pro edure fishh de nes the approximating fun tions and pre omputes the matri es needed to evaluate (7) and (8). The a tual evaluation of these equations is performed by the auxiliary pro edure fishhr displayed in Code Box 6, whi h is passed to the root nding algorithm broyden by fishh. A s ript le whi h omputes and plots results is given in Code Box 7; this was used to produ e Figures 11.10-11.13. Fish Harvesting: Value Function 1.4
1.2
1
V
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
S
Figure 11.10 The pro edure fishh allows additional onstraints to be imposed at the lower and upper bounds of the approximating interval (a and b). By setting nbl to 2, the additional onstraint that v 00 (a) = 0 is imposed. Similarly, by setting nbu to 2, the additional onstraint that v 00 (b) = 0 is imposed. Although neither is ne essary to l
u
CHAPTER 11.
CONTINUOUS TIME - METHODS
Code Box 11.5: Collo ation File for Fish Harvesting Problem
% FISHH Solution fun tion for fish harvesting problem % See DEMFB03 for demonstration fun tion [sstar, l, defl, u, defu℄=fishh(rho,alpha,sigma,H) % Set up basis matri es a=log(0.005); % lower bound b=log(10); % upper bound nl=25; nu=15; % number of nodes for fun tions nbl=2; nbu=1; % number of boundary onstraints on fun tions
defl=fundef({' heb',nl-nbl,a,0});
defu=fundef({' heb',nu-nbu,0,b}); yl=funnode( defl); yu=funnode( defu);
defl=fundef({' heb',nl,a,0});
defu=fundef({' heb',nu,0,b}); eyl=exp(yl); eyu=exp(yu); Dl=funbas( defl,yl,1); Du=funbas( defu,yu,1); B=rho*funbas( defl,yl)... -(alpha-0.5*sigma.^2)*Dl... -(0.5*sigma.^2)*funbas( defl,yl,2); temp=rho*funbas( defu,yu)... -(alpha-0.5*sigma.^2-H)*Du... -(0.5*sigma.^2)*funbas( defu,yu,2); B=[B zeros(nl-nbl,nu);zeros(nu-nbu,nl) temp℄; % Add boundary onstraints B=[B; ... funbas( defl,0) -funbas( defu,0); ... % V ontinuous at y=0 funbas( defl,0,1) -funbas( defu,0,1)℄; % Vx ontinuous at y=0 if nbl==2; B=[B;funbas( defl,a,2) zeros(1,nu)℄; end % lower boundary if nbu==2; B=[B;zeros(1,nl) funbas( defu,b,2)℄; end % upper boundary % Basis for Vy D=[alpha*eyl*ones(1,nl).*Dl zeros(nl-nbl,nu); ... zeros(nu-nbu,nl) alpha*eyu*ones(1,nu).*Du; ... zeros(nbl+nbu,nl+nu)℄; % RHS of DE residual fun tion f=[zeros(nl-nbl,1);H*eyu;zeros(nbl+nbu,1)℄; % Basis for residual fun tion (Vy(0)=S*) phil10=funbas( defl,0,1); % find the utoff sto k level sstar=broyden('fishhr',0.5*(1-rho/alpha),[℄,B,D,f,phil10,nl); % Break apart the oeffi ient ve tor and reate stru tures to return
=(B+sstar*D)\(sstar*f);
l= (1:nl);
u= (nl+1:end);
434
CHAPTER 11.
435
CONTINUOUS TIME - METHODS
Code Box 11.6: Residual File for Fish Harvesting Problem
% FISHHR residual fun tion for fish harvesting problem % Used by FISHH fun tion [e, ℄=fishhr(sstar,B,D,f,phil10,nl)
=(B+sstar*D)\(sstar*f); e=sstar-phil10* (1:nl);
Fish Harvesting: Marginal Value Function 5
4.5
4
3.5
V’
3
2.5
2
1.5
1
0.5
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
S
Figure 11.11 obtain useful results, the former is e e tive in enfor ing sensible behavior in the value fun tion and its derivatives for low sto k levels. Figure 11.10 illustrates a numeri al approximation to the value fun tion for the problem with = 0:05, = 0:1, = 0:2 and H = 1. Figures 11.11 and 11.12 display the rst and se ond derivatives of the value fun tion. S is indi ated in these plots with an \*". Noti e that the value fun tion is ontinuous up to its se ond derivative, but that V 00 exhibits a kink at S = S . This indi ates why it is a good idea to break the value fun tion apart and approximate it on ea h region separately, and pasting the two approximations together at the ut-o sto k level. It also allows us to use the high degree of a
ura y that polynomial approximations provide. Eviden e of the quality of the approximation is provided by the plot of the residual fun tion, shown
CHAPTER 11.
CONTINUOUS TIME - METHODS
Code Box 11.7: S ript File for Fish Harvesting Problem
% DEMFB03 Demo for fish harvesting problem rho=0.05; alpha=0.1; sigma=0.2; H=1; [sstar, l, defl, u, defu℄=fishh(rho,alpha,sigma,H); % CODE TO GENERATE PLOTS a= defl.a; b= defu.b; N=101; yl=linspa e(a,0,N)'; yu=linspa e(0,log(1/sstar),N)'; s=sstar*exp([yl;yu℄); figure(1) v=[funeval( l, defl,yl);funeval( u, defu,yu)℄; plot(s,v,sstar,v(N),'*'); title('Fish Harvesting: Value Fun tion'); xlabel('S'); ylabel('V'); figure(2) v1=[funeval( l, defl,yl,1);funeval( u, defu,yu,1)℄; plot(s,v1./s,sstar,v1(N)./sstar,'*'); title('Fish Harvesting: Marginal Value Fun tion'); xlabel('S'); ylabel('V'''); axis([0 1 0 5℄); figure(3) v2=[funeval( l, defl,yl,2);funeval( u, defu,yu,2)℄-v1; plot(s,v2./(s.^2),sstar,v2(N)./sstar.^2,'*'); title('Fish Harvesting: Curvature of Value Fun tion'); xlabel('S'); ylabel('V"'); axis([0 1 -1 0℄); figure(4) e=rho*v-(alpha-alpha*s).*v1-(0.5*sigma.^2)*v2; e=e+(H*(v1-s)).*(s>=sstar); plot([yl;yu℄,e) title('Fish Harvesting: Residual Fun tion'); xlabel('y') ylabel('e') prtfigs(mfilename)
436
CHAPTER 11.
437
CONTINUOUS TIME - METHODS
Fish Harvesting: Curvature of Value Function 0
−0.1
−0.2
−0.3
V"
−0.4
−0.5
−0.6
−0.7
−0.8
−0.9
−1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
S
Figure 11.12
Fish Harvesting: Residual Function
−9
1
x 10
0
−1
−2
e
−3
−4
−5
−6
−7
−8
−9 −6
−5
−4
−3
−2
−1
0
1
2
y
Figure 11.13
in Figure 11.13. One ould, of ourse, approximate the entire value fun tion with, say, a ubi spline, so long as you ensured that y=0 was a node (see problem 7 on
CHAPTER 11.
438
CONTINUOUS TIME - METHODS
page 454). This would avoid the need to de ne two fun tions and thus has something to re ommend it. However, it would require more nodes to a hieve the same level of a
ura y. Example: Entry-Exit
In sto hasti bang-bang problems, the state spa e is divided into a region in whi h the ontrol is set to it highest possible value and another region where it is set to its lowest possible value. In other problems with transitional boundaries, the state spa e may not be partitioned in this way. Instead, multiple free boundaries may need to be determined. In the entry-exit problem (page 368), a rm is either a tive or ina tive. The two value fun tions satisfy
V a = P C + (P )VPa + 2 (P )VPaP V i = (P )VPi + 2 (P )VPi P
for P for P
2 [Pl ; 1) : 2 [0; Ph℄
(9)
with
and
V i (Ph ) = V a (Ph ) I V i (Pl ) E = V a (Pl ) VPi (Pl ) = VPa (Pl ) VPi (Ph ) = VPa (Ph ):
When P is a geometri Brownian motion pro ess, i.e.,
dP = P dt + P dz; the solution is known for arbitrary levels of Pl and Ph. The general form of the solution is
V a = Aa1 P 1 + Aa2 P 2 + P=(
)
C=
V i = Ai1 P 1 + Ai2 P 2 where the four A terms will be pinned down by the boundary onditions and the i solve 1 2 2 (
1) +
= 0:
It an be shown that, for > 0, one of the is negative and the other is greater than one; de ne 1 > 1 and 2 < 0. (It is easy to verify that these solutions solve (9)). Two of the unknown onstants an be eliminated by onsidering the boundary
onditions at P = 0 and P = 1. At P = 0 only V i is de ned and the geometri
CHAPTER 11.
439
CONTINUOUS TIME - METHODS
Brownian motion pro ess is absorbed; hen e V i (0) = 0, whi h requires that Ai2 = 0. For large P , only V a is de ned and the probability of dea tivation be omes vanishingly small; hen e the value fun tion would approa h P=( ) C=, requiring that Aa1 = 0. We still have two unknown onstants to determine, Ai1 and Aa2 (we shall hen eforth refer to these as A1 and A2 , as there is no possible onfusion on erning whi h fun tion they belong to). The value mat hing onditions require that, V a (Ph )
I = A2 Ph 2 + Ph =(
)
C=
I = A1 Ph 1 = V i (Ph )
and V a (Pl ) = A2 Pl 2 + Pl =(
C= = A1 Pl 1
)
E = V i (Pl )
E:
The optimality onditions on Pl and Ph are that the derivatives of V a and V i are equal at the two boundary lo ations: VPa (P ) = 2 A2 P 2
1
+ 1=(
) = 1 A1 P 1
1
= VPi (P )
at P = Pl and P = Ph . Taken together, the value mat hing and smooth pasting
onditions yield a system of four equations in four unknowns, A1 , A2 , Pl and Ph . This is a simple root- nding problem (the toolbox fun tion entex2.m solves this problem). For general pro esses, we will have to approximate the value fun tions. We rst de ne two transformed variables that will make the boundaries of the approximating fun tions simple. Let y a = P=Pl and y i = P=Ph. The value fun tions an then be approximated using V j (P ) = v j (y j ) j (y j ) j for j = a; i, with v a de ned on [1; ya℄ and v i on [0; 1℄ (the hoi e of ya, the upper bound on y a is dis ussed below). Given the linearity of the Bellman's Equation and the boundary onditions, it will again be the ase that the oeÆ ient ve tors i and a , for given values of the boundary points, satisfy a system of linear equations. Our strategy will be write a pro edure that is passed trial values of P = [Pl ; Ph℄, omputes i and a for these free boundary points and then returns the di eren e in marginal value fun tions at the two boundary points:
2 i vy (1) vya (Ph=Pl ) 66 Ph + Pl r(P ) = 6 i 4 vy (Pl =Ph) vya (1) Ph
3 7 7 : 7 5
Pl At the optimal hoi e of P , r(P ) = 0. The pro edure that returns the residuals an be passed to a root nding algorithm to determine P .
CHAPTER 11.
CONTINUOUS TIME - METHODS
440
To see how the oeÆ ient values an be determined, rst write the Bellman's equation in terms of the yj and vj : i (y i ) (Phy i )vyi (y i) 2 (Phy i)vyy i i = 0: v (y ) Ph 2Ph2 and a (y a ) (Pl y a )vya (y a) 2 (Pl y a )vyy a a v (y ) = Pl y a C: 2 Pl 2Pl Suppose the approximating fun tions have degree ni and na , respe tively. We will evaluate the Bellman's Equation at ni 2 values of yi and na 2 values of ya. The addition of the two end point onditions (vi(0) = 0 and vyy (ya) = 0) and the two value-mat hing onditions yields a system of ni + na linear equations in the same number of unknowns. Spe i ally, the linear system is 3 2 3 2 3 2 0 0 i (0) 0 6 0 7 7 6 7 6 Di 0 7 6 7 i 6 0 7 6 i a 6 0 7 7 6 7 6 I
(1) ( P =P ) ( P =P ) h l h l 7 a =6 7 6 7 6 6 E 7 + Pl 6 0 7 ; 7 6 (Pl =Ph) i (Pl =Ph) a (1) 7 6 7 6 7 6 4 Ya 5 4 C1 5 5 4 0 Da 0 0 0 a (y a) where (PhY i ) i 2 (PhY i ) i Di = i0 1 Ph 2Ph2 2 and (Pl Y a ) a 2 (Pl Y a ) i a a D = 0 1 Pl 2Pl2 2 : The system is of the form B = b0 + Pl b1 , where b0 and b1 do not depend on Pl and Ph and hen e an be pre omputed. Furthermore, all of the basis matri es used in B ex ept for i(Pl =Ph ) and a (Ph=Pl ) an be pre omputed. The drift and di usion terms and are evaluated at points that may depend on Pl and Ph and hen e must be omputed ea h time the residual fun tion is evaluated. The user must supply a spe i fun tion to evaluate these terms. On e the drift, di usion and dis ount rates are known, the Di and Da matri es an be omputed and the matrix B an be formed. 00
CHAPTER 11.
CONTINUOUS TIME - METHODS
441
The residuals themselves an be written in the form R : # " Ph a 0 (P =P ) i i 0 (1)
h l Pl : r(P ) = Pl i 0 0 a
a (1) Ph (Pl =Ph ) The pro edure for omputing the residuals is found in the toolbox fun tion entexres. Most of the inputs to this fun tion are pre omputed basis and other parameter matri es, in luding b0 , b1 and elements of B and R. A pro edure that solves the entry/exit problem is provided in the toolbox fun tion entex. This pro edure takes as inputs the problem parameters and approximation information. It then pre omputes basis matri es and the other oeÆ ients used by the residual fun tion entexres, and passes entexres to the root nding algorithm broyden to nd Pl and Ph . entex returns the value of P , the value fun tion oeÆ ient ve tors ( i and a) and their asso iated fun tion de nition stru tures (fspa ei and fspa ea). In addition, a pro edure must be de ned that a
epts values of P (along with additional parameters as needed) and returns values of , and for a spe i ed model.6 An example of su h a le for geometri Brownian motion with a onstant dis ount rate is given by: fun tion [r,m,s℄=gbm(P,rho,mu,sigma) n=size(P,1); r=rho+zeros(n,1); m=mu*P; s=sigma*P;
The le demfb04.m demonstrates the use of this approa h for the geometri Brownian motion ase and ompares the resulting solution to the (essentially) losed form solution dis ussed above. Figure 11.14 shows the value fun tions for the ina tive and a tive states. Figures 11.15 and 11.16 show, respe tively, the approximation errors and the residual fun tions for the ollo ation approximation relative to the \ losed form" solution. An important point to note about this problem on erns the hoi e of the upper bound for v a . Even though the optimal boundary values are Pl = 0:41815 and Ph = 2:1996, it is ne essary to set the upper bound for y a to a rather large number. Intuitively the reason for this stems from the non-stationarity of the geometri Brownian motion pro ess oupled with the in nite time horizon. Together, these imply that the probability of getting a large value of P , even when starting at a relatively low urrent value, is non-negligible over time horizons that ontribute to the present value. We should expe t, therefore, that relatively larger values of the upper bound will be needed as shrinks. 6 Although
appli ations of this model generally treat as a xed onstant, the approa h taken here provides the exibility to make it depend on P if desired.
CHAPTER 11.
CONTINUOUS TIME - METHODS
442
Value Functions for Entry/Exit Problem 50 Inactive Active
40
30
20
10
0
−10
0
0.5
1
1.5
2
2.5
3
2.5
3
P
Figure 11.14
Errors in Approximate Value Functions
−5
x 10
8
6
4
2
0
0
0.5
1
1.5
P
Figure 11.15
2
CHAPTER 11.
443
CONTINUOUS TIME - METHODS
Residual Functions
−5
4
x 10
3
2
1
0
−1
−2
−3
0
0.5
1
1.5
2
2.5
3
P
Figure 11.16 11.3.2
Finite Horizon Problems
Thus far we have dis ussed single state, in nite horizon free boundary problems. Somewhat greater diÆ ulties arise in nite horizon problems. The lo ation of the free boundaries in su h problems are not isolated points but are fun tions of time. For example, when an Ameri an put option is near to expiration, it is optimal to exer ise it at higher pri es of the underlying asset than when it is far from expiration. In nite time problems, we know the value fun tion at the terminal date. This means that we an employ evolutionary methods that work their way ba kwards in time from the end point. We present two ways of handling su h problems. The rst impli itly assumes that the ontrol an be exer ised only at dis rete points in time. Although simple, this method approximates the free boundary with a step fun tion. To obtain a smoother approximation of the boundary, without requiring a dense set of state variable nodes, we use an expli it nite di eren e approximation for the time derivative, while simultaneously solving for the free boundary. The two approa hes are des ribed in the following two examples. Example: Pri ing Ameri an Options
In Se tion 11.1 we solved problems of valuing European style options using the extended method of lines, whi h approximates the value of an option using V (S; )
CHAPTER 11.
CONTINUOUS TIME - METHODS
444
( ) ( ). By evaluating (S ) at a set of n nodal values, we derived a di erential equation of the form
0 ( ) = 1 B ( ): whi h is solved by
( + ) = A ( ); where A = exp( 1 B ) and (0) equals the terminal payout, R(S ), evaluated at the nodal state values. The most ommonly used strategy for pri ing Ameri an style options solves the
losely related problem of determining the value of an option that an be exer ised only at a dis rete set of dates. Clearly, as the time between dates shrinks, the value of this option onverges to the value of one that an be exer ised at any time before expiration. Between exer ise dates, the option is e e tively European and hen e an be approximated using7
^( + ) = A ( ): The value of (S )^ ( + ) an then be ompared to the value of immediate exer ise, R(S ), and the value fun tion set to the maximum of the two: V (S; + ) max(R(S ); (S )^
( + )): The oeÆ ient ve tor is updated to approximate this fun tion, i.e.,
( + ) = 1 max(R; ^
( + )): The fun tion finsolve des ribed in Se tion 11.1 requires only minor modi ation to implement this approa h to pri ing Ameri an style assets. First, add an additional eld to the model variable, model.ameri an, whi h takes values of 0 or 1. Then,
hange the main iteration loop to the following: S
for i=2:N+1
(:,i)=A* (:,i-1); if model.ameri an
(:,i)=iPhi*max(V0,Phi* (:,i)); end end
Commonly used nite di eren e and binomial tree methods dis retize the time derivatives, repla ing ( + ) with ( + ) ( ) =. 7
0
CHAPTER 11.
CONTINUOUS TIME - METHODS
445
The le demfin04.m demonstrates the use of this feature and produ es Figures 11.17-11.19. It losely follows the ode des ribed on page 406, but two di eren es are noteworthy. First, a losed form solution does not exist for the Ameri an put option, even when the underlying pri e is geometri Brownian motion. To assess the quality of the approximation, we have omputed a di erent approximation due to Baron-Adesi and Whaley (see bibliographi notes), whi h is implemented in the toolbox fun tion baw. The di eren es between the approximations are plotted in Figure 11.18. The other distin tion lies in the determination of the optimal exer ise boundary, whi h is plotted in Figure 11.19. This is obtained by determining whi h nodal points that are less than or equal to K are asso iated with an option that is equal to its intrinsi value of K S . The exer ise boundary is taken to be the highest su h nodal value of S . This provides a step fun tion approximation to the early exer ise boundary. Unfortunately, this approximation an only be re ned by in reasing the number of nodal values so they are fairly dense in the region where early exer ise may o
ur (just below the strike pri e). Su h a dense set of nodal values is rarely needed to improve the a
ura y of the value fun tion, however. On the positive side, the method of nding an approximation to the value of an option with a dis rete set of exer ise dates has two overriding advantages. It is very simple and it extends in an obvious way to multiple state situations. On its negative side, it does not produ e a smooth representation of the optimal exer ise boundary. If a smooth approximation is needed or desired, the approa h des ribed in the next example an be used. Example: Sequential Learning
In the sequential learning problem on page 372, the umulative produ tion, Q, a ts like a time variable. There is a known terminal ondition at Q = Qm and the solution
an be obtained in an evolutionary fashion by working ba kwards in Q from Qm . Identifying the lo ation of the free boundary, however, is somewhat more involved than with the Ameri an option pri ing problem. Re all that the problem involved solving rV = P
(Q) + VQ + (r
Æ )P VP + 21 2 P 2 VP P
on [P (Q); 1) [0; Qm ℄, where P (Q) is a free boundary to be determined. The boundary onditions are8 P (Q)VP (P (Q); Q) = V (P (Q); Q)
and P (Q)VP P (P (Q); Q) = ( 8 We
ignore the limiting ondition as
1)VP (P (Q); Q); P
!1
.
CHAPTER 11.
446
CONTINUOUS TIME - METHODS
Put Option Premium 1 American European Optimal Exercise Boundary
0.9
0.8
0.7
V(S)
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
1.6
1.8
2
S
Figure 11.17
Put Option Approximation Errors
−4
6
x 10
4
2
0
−2
−4
−6
−8
0
0.2
0.4
0.6
0.8
1
1.2
S
Figure 11.18
where
is the positive solution to
1 2 2 (
1) + (r
Æ )
r = 0:
1.4
CHAPTER 11.
447
CONTINUOUS TIME - METHODS
Early Exercise Boundary 1
0.98
0.96
0.94
S*
0.92
0.9
0.88
0.86
0.84
0.82
0.8
0
0.1
0.2
0.3
0.4
0.5
τ
0.6
0.7
0.8
0.9
1
Figure 11.19 Also a terminal ondition at Q = Qm is known and, for states below the free boundary, the value fun tion is known up to a onstant: V
(P; Q) = A(Q)P :
The diÆ ulty with free boundaries is the unknown shape of the spa e over whi h the di erential equation must hold. To get around this problem, we use a transformation method that regularizes the boundary; spe i ally, y
= ln(P )
ln(P (Q))
with v (y; Q) = V (P; Q). The PDE must be solved for values of P on [P (Q); 1), whi h translates into values on y on [0; 1) (in pra ti e we will trun ate y ). Given this transformation it is straightforward to verify the following relationships between the original and the transformed problem: (
vy y; Q vyy
) = P VP (P; Q)
vy
= P 2 VP P (P; Q)
and VQ
= vQ
P P
0
(Q) v : (Q) y
CHAPTER 11.
CONTINUOUS TIME - METHODS
448
Substituting these expressions into the Bellman equation and the boundary onditions yields: rv = P ey C (Q) + vQ + (r Æ 12 2 P 0 =P )vy + 21 2 vyy ; vy (0; Q) v (0; Q) = 0 and vyy (0; Q) vy (0; Q) = 0: We an approximate v(y; Q) with the fun tion (y) (Q), where (Q) : [0; Qm ℄ ! Qm beta1=0.5-(r-delta)/sigma.^2 + sqrt((0.5-(r-delta)/sigma.^2).^2+2*r/sigma.^2); beta2=0.5-(r-delta)/sigma.^2 - sqrt((0.5-(r-delta)/sigma.^2).^2+2*r/sigma.^2); % Impose value mat hing and smooth pasting at P= bar temp=[
bar.^beta1 -( bar.^beta2) ; ... beta1* bar.^(beta1-1) -beta2* bar.^(beta2-1)℄; temp=temp\[ bar/delta- bar/r ; 1/delta℄; A1=temp(1); A2=temp(2); % Define the approximating fun tions and nodal values Delta=Qm/N; Q=linspa e(0,Qm,N+1);
def=fundef({' heb',n-1,0,hi}); y=funnode( def);
def=fundef({' heb',n,0,hi}); % Set up ollo ation matri es D=funbasx( def,y,[0;1;2℄); Phi0=D.vals{1}; Phi1=D.vals{2}; D=r*Phi0-(r-delta-0.5*sigma^2)*Phi1-0.5*sigma.^2*D.vals{3}; phi=funbasx( def,0,[0;1;2℄); B=[ Phi0 zeros(n-1,1); phi.vals{2}-beta1*phi.vals{1} 0 ; phi.vals{3}-(beta1)*phi.vals{2} 0 ℄; A=[Phi0-Phi1-Delta*D Delta*exp(y); zeros(2,n+1) ℄; % Compute ost fun tion values gamma=log(C/ bar)/Qm; Cost=Delta* bar*exp(gamma*(Qm-Q)); Cfa tor=[ones(n-1,1);0;0℄; % Initialize at terminal boundary p=[ bar; bar*exp(y)℄;
=zeros(n+1,N+1);
(1:n,N+1)=[phi.vals{1};Phi0℄\(A2*p.^beta2+p/delta- bar/r);
(end,N+1)= bar; % Iterate ba kwards in Q for i=N+1:-1:2 B(1:n-1,end)=Phi1* (1:n,i)/(- (end,i));
(:,i-1)=B\(A* (:,i)-Cost(i)*Cfa tor); end % Extra t Pstar pstar= (end,:)';
(end,:)=[℄;
CONTINUOUS TIME - METHODS
451
Sequential Learning: Value Function 500
450
400
350 Q=Qm=20
300
V(P,Q)
CHAPTER 11.
250
Q=0
200
150
100
50
0
0
5
10
15
20
25
P
Figure 11.20
30
35
40
CHAPTER 11.
452
CONTINUOUS TIME - METHODS
Exer ises 11.1. A Generalized Term Stru ture Model A one fa tor model that en ompasses many term-stru ture models appearing in the literature is based on the pro ess for the short interest rate (under the risk neutral measure) given by:
dr = [ 1 + 2 r + 3 r log(r)℄dt + [ 1 + 2 r℄ dW: State the PDE satis ed by a zero- oupon bond maturing in periods, along with the asso iated boundary ondition. Write a fun tion analogous to irbond on p. 400 for this ase. The fun tion should have the following input/output format:
=Bond(fspa e,alpha1,alpha2,alpha3,beta1,beta2,nu,tau)
Noti e that this fun tion returns the oeÆ ients of a fun tion of r de ned by the fun tion de nition stru ture variable fspa e. Verify that your fun tion reprodu es orre tly the results obtained using CIRBOND, whi h is a spe ial ase of the generalized model. To do this, use the parameters = 30, = :1, = :05 and = 0:2. 11.2. Bond Option Pri ing Consider an option that gives its holder the right, in o periods, to buy a bond that pays 1 unit of a
ount b periods after the option expires, at a strike pri e of K . Using the model for the short rate des ribed in the previous exer ise, write a fun tion that omputes the value of su h an option.The fun tion should have the following input/output format:
Matlab
=BondOption(fspa e,alpha1,alpha2,alpha3,beta1,beta2,nu,K,tauo,taub)
Determine the value of an option for the CIR model with K = 1, o = 1, b = 0:25, = :1, = :05 and = 0:2.
11.3. Neo lassi al Optimal Growth Use s solve to solve the neo lassi al optimal growth model, dis ussed beginning on page 416: Z 1 max e t U (C )dt; C (t)
0
subje t to the state transition fun tion K 0 = q (K )
q (K ) = ln(K + 1) ÆK
C , where
CHAPTER 11.
CONTINUOUS TIME - METHODS
453
and
U (C ) = (C 1
1)=(1
)
using the parameter values = 0:05, Æ = 0:02, = 2( + Æ ) = 0:14 and = 0:5. Compare your results to those obtained using the Euler equation approa h. 11.4. Cow Repla ement Convert the dis rete time and state ow repla ement problem on page ?? to a
ontinuous time and ontinuous state problem and solve the problem. 11.5. Asset Repla ement with Sto hasti Quality In the asset repla ement problem dis ussed on pages 359 and 422, the produ tivity of the asset depended only on its age. Suppose instead that the output of the ma hine is governed by
q
dQ = Qdt + Q(Q Q )dz; where Q is the produ tivity of a new asset. Noti e that the pro ess is singular . At Q = Q the drift rate is negative, so produ tivity is at Q = 0 and Q = Q de reasing, whereas Q = 0 is an absorbing barrier. The in ome ow rate from the asset is P Q, for some onstant P , the repla ement
ost of the asset is C and the dis ount rate is . Intuitively, there is some value Q = at whi h it is optimal to repla e the asset. a) State the Bellman's equation and boundary onditions for this problem (be sure to onsider what happens if = 0). What is the value fun tion for Q < ? b) Write a le that has the following input/output format:
Matlab
[beta, ,fspa e℄=Repla e(mu,sigma,rho,P,C,Qbar,n)
where and fspa e are the oeÆ ients and the fun tion de nition stru ture ℄, and n is the degree of the de ning the value fun tion on the interval [ ; Q approximation. You may assume an interior solution ( > 0).
) Call the fun tion you wrote with the line [beta, ,fspa e℄=Repla e(0.02,0.05,0.1,1,2,1,50);
℄ (not on [ ; Q ℄) and mark the point Plot the value fun tion on the interval [0; Q ( ; V ( )) with a \*". 11.6. Timber Harvesting
CHAPTER 11.
CONTINUOUS TIME - METHODS
454
a) Solve the timber harvesting example from page 360 with the parameters = 0:1, m = 1, 0, P = 1, C = 0:15 and = 0:08. Plot the value fun tion, indi ating the lo ation of the free boundary with an \*". b) Resolve the problem under the assumption that the land will be abandoned and the replanting ost will not be in urred. Add this result to the plot generated for part (a). 11.7. Fish Harvesting Using Cubi Splines Modify the ode in the sh harvesting example (page 431) to ompute the value fun tion using a single ubi spline approximation with a double breakpoint at y = 0. Plot the value fun tion and its 1st and 2nd derivatives as fun tions of S (not y ) and the residual fun tion for the di erential equation as a fun tion of y . 11.8. Fish Harvesting - Unbounded E ort a) Consider the sh harvesting problem (page 431) under the assumption that the ontrol is not bounded (H ! 1), making the problem of the barrier ontrol type. Write a program to solve for the value fun tion and the optimal sto k level that triggers harvesting. Use the same parameter values as in the bounded e ort model ( = 0:1, = 0:05, = 0:2). b) Compute and plot the optimal trigger sto k level as a fun tion of the maximal harvest rate (H ), using the above values for other parameters. Demonstrate that the limiting value as H ! 1 omputed in part (a) is orre t. 11.9. Cost Un ertainty Consider the problem of determining an investment strategy when a proje t takes time to omplete and ompletion osts are un ertain. The ost un ertainty takes two forms. The rst, te hni al un ertainty, arises be ause of unforeseen te hni al problems that develop as the proje t progresses. Te hni al un ertainty is assumed to be diversi able and hen e the market pri e of risk is zero. The se ond type of un ertainty is fa tor ost un ertainty, whi h is assumed to have market pri e of risk . De ne K to be the expe ted remaining ost to omplete a proje t that is worth V upon ompletion. The dynami s of K are given by
p
dK = Idt + IKdz + Kdw; where I , the ontrol, is the urrent investment rate and dz and dw are independent Weiner pro esses. The proje t annot be ompleted immediately be ause I is onstrained by 0 I k. Given the assumptions about the market pri e
CHAPTER 11.
CONTINUOUS TIME - METHODS
455
of risk, we onvert the K pro ess to its risk neutral form and use the risk free interest rate, r, to dis ount the future. Thus we a t \as if" dK =
p
(I + K )dt + IKdz + Kdw
and solve F (K ) = max E e
rT
I (t)
Z
T
V
e
rt
I (t)dt ;
0
where T is the (un ertain) ompletion time given by K (T ) = 0. The Bellman equation for this problem is rF = max I I
(I + K )F 0 (K ) + 21 ( 2 IK + 2 K 2 )F 00 (K );
with boundary onditions F (0) = V F (1) = 0:
The optimal ontrol is of the bang-bang type: 0 if K > K I= k if K < K where K solves 1 2
2 KF 00 (K )
F 0 (K )
1 = 0:
Noti e that te hni al un ertainty in reases with the level of investment. This is a ase in whi h the varian e of the pro ess is in uen ed by the ontrol. Although we have not dealt with this expli itly, it raises no new problems. a) Solve F up to an unknown onstant for K > K . b) Use the result in (a) to obtain a boundary ondition at K = K by utilizing the ontinuity of F and F 0 .
) Solve the deterministi problem ( = = 0) and show that K = k ln(1 + rV =k)=r. d) Write the Bellman equation for K < K and transform it from the domain [0; K ℄ to [0; 1℄ using z = K=K : Also transform the boundary onditions.
CHAPTER 11.
CONTINUOUS TIME - METHODS
456
e) Write a omputer program using Chebyshev ollo ation to solve for F and K using the following parameters: V r k
= = = = = =
10 0:05 0 2 0:5 0:25:
g) What alterations are needed to handle the ase when = 0 and why are they needed. 11.10. Investment with Time-to-Build Constraints Consider an investment proje t whi h, upon ompletion, will have a random value V and will generate a return ow of ÆV . The value of the ompleted proje t evolves, under the risk neutral measure, a
ording to dV = (r
Æ )V dt + V dz;
where r is the risk free rate of return. The amount of investment needed to
omplete the proje t is K , whi h is a ompletely ontrolled pro ess: dK =
Idt;
where the investment rate is onstrained by 0 I k. In this situation it is optimal to either be investing at the maximum rate or not at all. Let the value of the investment opportunity in these two ases by denoted F (V; K ) and f (V; K ), respe tively. These fun tions are governed by the following laws of motion: 1 2 2 2 V F
V V + (r
Æ )V FV
rF
V V + (r
Æ )V fV
rf = 0;
kFK
and 1 2 2 2 V f
k=0
CHAPTER 11.
CONTINUOUS TIME - METHODS
457
subje t to the boundary onditions F (V; 0) = V
lim FV (V; K ) = e
V !1
ÆK=k
f (0; K ) = 0 f (V ; K ) = F (V ; K )
fV (V ; K ) = FV (V ; K ):
V is the value of the ompleted proje t needed to make a positive investment. It an be shown that f (V ) = A(K )V , where
1 = 2
r
2
Æ
+
s
1 2
r
2
Æ
2
+
2r : 2
(10)
and A(K ) is a fun tion that must be determined by the boundary onditions. This may be eliminated by ombining the free boundary onditions to yield F (V ; K ) = V FV (V ; K ):
Summarizing, the problem is to solve the following partial di erential equation for given values of , r, Æ and k: 1 2V 2F VV 2
+ (r
Æ )V FV
rF
kFK
k = 0;
subje t to F (V; 0) = V
lim FV (V; K ) = e ÆK=k F (V ; K ) = V FV (V ; K ); V !1
where is given by (10). This is a PDE in V and K , with an initial ondition for K = 0, a limiting boundary ondition for large V and a lower free boundary for V that is a fun tion of K . The problem as stated is solved by F = Ve
Æ=kK
K
CHAPTER 11.
CONTINUOUS TIME - METHODS
458
with optimal uto boundary V (K ) =
K
1
eÆ=kK :
However, the optimal solution must, in addition, satisfy FK (V ; K ) = 1:
Matlab
Write
ode to solve the time-to-build problem for the following parameter values: Æ=0 r = 0:02 = 0:2 k=1
11.11. Sequential Learning Continued Review the sequential learning model dis ussed on pages 372 and 445. Note that the Bellman equation provides an expression for VQ when P > P . For P < P , the value fun tion has the form A(Q)P 1 and so VQ (P; Q) = A0 (Q)P 1 . a) Derive an expression for VQ for P > P . b) Show that VQ is ontinuous at P = P for > 0.
) Use this fa t to determine A0 (Q). d) Plot VQ as a fun tion of P for Q = 0 using the parameters r = Æ = 0:05 and for the values =0.1, 0.2, 0.3, 0.4 and 0.5. e) When = 0, VQ is dis ontinuous at P = P . This ase was dis ussed in Problem 20 in Chapter 10. Add this ase to the plot from part (d).
CHAPTER 11.
CONTINUOUS TIME - METHODS
Appendix A Basis Matri es for Multivariate Models
459
The basi PDE used in ontrol problems arising in e onomi s and nan e takes the form r (S )V
= Æ (S ) + Vt + VSS (S ) + 12 tra e( (S ) (S )
>
VSS ):
Generi ally, S is a d-dimensional ve tor, VS is (1 d), (S ) is d 1, VSS is d d, and (S ) is d d. The rst derivative term an be omputed (for a single S ve tor) using VS (S )
(S )
>
0 (S )
where
2 6 (S ) = 6 4 0
3
01 (S1 ) 1 (S1 )
2(S2) : : : d (Sd)
2(S2) : : : d (Sd) 77 :
1 (S1 )
2(S2) : : : d (Sd)
0
:::
5
0
The se ond derivative term an be omputed using > 1 2 tra e( (S ) (S ) VSS )
where
00
2 66 66 66 66 66 (S ) = 6 66 66 66 66 64
> > 00 1 2 ve ( (S ) (S ) ) (S )
001 (S1 ) 01 (S1 )
2(S2 ) : : : d(Sd) 3
2(S2) : : : d(Sd ) 77
01 (S1 ) 01 (S1 ) 1 (S1 )
2(S2) : : : d(Sd )
2(S2) : : : d(Sd )
2 (S2 ) : : : d(Sd)
1 (S1 )
2(S2) : : : d(Sd )
01 (S1 ) 1 (S1 )
2(S2) : : : d(Sd )
2(S2) : : : d(Sd )
1 (S1 )
2(S2) : : : d (Sd )
0
:::
0
0
00
:::
0
0
:::
0
0
0
:::
7 7 7 7 7 7 7 7 7 : 7 7 7 7 7 7 7 7 7 5
00
It would be even more eÆ ient to avoid the double omputations arising from symmetry by using the ve h operator: > 1 2 tra e( (S ) (S ) VSS )
ve h((S )(S )
>
) ^ (S ) >
00
CHAPTER 11.
CONTINUOUS TIME - METHODS
where
2 66 66 66 ^ (S ) = 6 66 66 64 00
1 (S1 ) 2 (S2 ) : : : d (Sd ) 1 (S1 ) 2 (S2 ) : : : d (Sd ) ::: 1 (S1 ) 2 (S2 ) : : : d (Sd ) 1 (S1 ) 12 2 (S2 ) : : : d (Sd ) ::: 1 (S1 ) 2 (S2 ) : : : d (Sd ) ::: 1 (S1 ) 2 (S2 ) : : : 12 d (Sd ) 1 2
00
0
0
0
0
00
0
0
00
3 7 7 7 7 7 7 7 : 7 7 7 7 7 5
460
CHAPTER 11.
CONTINUOUS TIME - METHODS
461
Bibliographi Notes A standard referen e on solving PDEs is Ames. It ontains a good dis ussion of stability and onvergen e analysis; the se tion on paraboli PDEs is espe ially relevant for e onomi appli ations. Golub and Ortega ontains a useful introdu tory treatment the extended method-of-lines for solving PDEs (Se tion 8.4), whi h they all a semidis rete method. Most treatments of PDEs begin with a dis ussion of nite di eren e methods and may then pro eed to nite element and weighted residual methods. The approa h we have taken reverses this order by starting with a weighted residual approa h ( ollo ation) and demonstrating that nite di eren e methods an be viewed as a spe ial ase with a spe i hoi e of basis fun tions. We have not dis ussed nite element methods expli itly, but the same remarks apply to them. Pie ewise linear
ubi splines bases are ommon examples of nite element methods. The investment under un ertainty with mean reversion in the risk neutral return pro ess is due to Dixit and Pindy k (pp. 161-163). We have simpli ed the notation by taking as given the risk-neutral pro ess for the value of the ompleted investment. Numerous referen es ontaining dis ussions of numeri al te hniques for solving nan ial asset models now exist. Hull ontains a good overview of ommonly used te hniques. See also DuÆe and Wilmott. In addition to nite di eren e methods, binomial and trinomial trees and Monte Carlo methods are the most ommonly used approa hes. Tree approa hes represent state dynami s using a bran hing pro ess. Although the on eptual framework seems di erent from the PDE approa h, tree methods are
omputationally losely related to expli it nite di eren e methods for solving PDEs. If the solution to an asset pri ing model for a given initial value of the state is the only output required from a solution, trees have an advantage over nite di eren e methods be ause they require evaluation of far fewer nodal points. If the entire solution fun tion and/or derivatives with respe t to the state variable and to time are desired, this advantage disappears. Furthermore, the extended method of lines is quite ompetitive with tree methods and far more simple to implement. Monte Carlo te hniques are in reasingly being used, espe ially in situations with a high dimensional state spa e. The essential approa h simulates paths for the state variable using the risk-neutral state pro ess. Many assets an then be pri ed as the average value of the returns to the asset evaluated along ea h sample path. This approa h is both simple to implement and avoids the need for spe ial treatment of boundary onditions with exoti assets. Numerous re nements exist to in rease the eÆ ien y of the approa h, in luding the use of varian e redu tion te hniques su h as antitheti and ontrol variates, as well as the use of quasi-random numbers (low dis repan y sequen es). Monte Carlo approa hes have been applied to the al ulation of Ameri an style assets with early exer ise features but this requires more work.
CHAPTER 11.
CONTINUOUS TIME - METHODS
462
Other approa hes to solving sto hasti ontrol problems in lude dis retization methods; see, e.g., Kushner and Dupuis. Several of the exer ises are based on problems in the literature. The generalized model of the short interest rate appears in DuÆe, pp. 131-133. The sh harvesting problem with adjustment osts was developed by Ludwig and Ludwig and Varrah. The ost un ertainty model is dis ussed in Dixit and Pindy k, pp. 345-351. The time-to-build exer ise is from Majd and Pindy k and is also dis ussed in Dixit and Pindy k (pp. 328-339).
Appendix A Mathemati al Ba kground A.1
Normed Linear Spa es
A linear spa e or ve tor spa e is a nonempty set X endowed with two operations, ve tor addition + and s alar multipli ation , that satisfy
x + y = y + x for all x; y 2 X
(x + y ) + z = x + (y + z ) for all x; y; z 2 X
there is a 2 X su h that x + = x for all x 2 X
for ea h x 2 X there is a y 2 X su h that x + y =
( ) x = ( x) for all ;
(x + y ) = x + y for all 2 < and x; y 2 X
( + ) x = x + y for all ; 2 < and x 2 X
1 x = x for all x 2 X .
2