Introduction to the Calculus of Variations and Control with Modern Applications 146657139X, 9781466571396

Citation preview

Introduction to

he Calculus of Variations and Control with Modern Applications

CHAPMAN & HALL/CRC APPLIED MATHEMATICS AND NONLINEAR SCIENCE SERIES Series Editor H. T. Banks

Published Titles Advanced Differential Quadrature Methods, Zhi Zong and Yingyan Zhang Computing with hp-ADAPTIVE FINITE ELEMENTS, Volume 1, One and Two Dimensional Elliptic and Maxwell Problems, Leszek Demkowicz Computing with hp-ADAPTIVE FINITE ELEMENTS, Volume 2, Frontiers: Three Dimensional Elliptic and Maxwell Problems with Applications, Leszek Demkowicz, Jason Kurtz, David Pardo, Maciej Paszy´nski, Waldemar Rachowicz, and Adam Zdunek CRC Standard Curves and Surfaces with Mathematica®: Second Edition, David H. von Seggern Discovering Evolution Equations with Applications: Volume 1-Deterministic Equations, Mark A. McKibben Discovering Evolution Equations with Applications: Volume 2-Stochastic Equations, Mark A. McKibben Exact Solutions and Invariant Subspaces of Nonlinear Partial Differential Equations in Mechanics and Physics, Victor A. Galaktionov and Sergey R. Svirshchevskii Fourier Series in Several Variables with Applications to Partial Differential Equations, Victor L. Shapiro Geometric Sturmian Theory of Nonlinear Parabolic Equations and Applications, Victor A. Galaktionov Green’s Functions and Linear Differential Equations: Theory, Applications, and Computation, Prem K. Kythe Group Inverses of M-Matrices and Their Applications, Stephen J. Kirkland and Michael Neumann Introduction to Fuzzy Systems, Guanrong Chen and Trung Tat Pham Introduction to non-Kerr Law Optical Solitons, Anjan Biswas and Swapan Konar Introduction to Partial Differential Equations with MATLAB®, Matthew P. Coleman Introduction to Quantum Control and Dynamics, Domenico D’Alessandro Introduction to The Calculus of Variations and Control with Modern Applications, John A. Burns Mathematical Methods in Physics and Engineering with Mathematica, Ferdinand F. Cap Mathematical Theory of Quantum Computation, Goong Chen and Zijian Diao Mathematics of Quantum Computation and Quantum Technology, Goong Chen, Louis Kauffman, and Samuel J. Lomonaco Mixed Boundary Value Problems, Dean G. Duffy Modeling and Control in Vibrational and Structural Dynamics, Peng-Fei Yao Multi-Resolution Methods for Modeling and Control of Dynamical Systems, Puneet Singla and John L. Junkins Nonlinear Optimal Control Theory, Leonard D. Berkovitz and Negash G. Medhin Optimal Estimation of Dynamic Systems, Second Edition, John L. Crassidis and John L. Junkins Quantum Computing Devices: Principles, Designs, and Analysis, Goong Chen, David A. Church, Berthold-Georg Englert, Carsten Henkel, Bernd Rohwedder, Marlan O. Scully, and M. Suhail Zubairy A Shock-Fitting Primer, Manuel D. Salas Stochastic Partial Differential Equations, Pao-Liu Chow

CHAPMAN & HALL/CRC APPLIED MATHEMATICS AND NONLINEAR SCIENCE SERIES

Introduction to

he Calculus of Variations and Control with Modern Applications

John A. Burns Virginia Tech Blacksburg, Virginia, USA

CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2014 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper Version Date: 20130715 International Standard Book Number-13: 978-1-4665-7139-6 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com

Contents Preface

xi

Acknowledgments

I

xvii

Calculus of Variations

1 Historical Notes on the Calculus of Variations 1.1 Some Typical Problems . . . . . . . . . . . . . . 1.1.1 Queen Dido’s Problem . . . . . . . . . . 1.1.2 The Brachistochrone Problem . . . . . . 1.1.3 Shape Optimization . . . . . . . . . . . . 1.2 Some Important Dates and People . . . . . . . .

1 . . . . .

. . . . .

3 6 6 7 8 11

2 Introduction and Preliminaries 17 2.1 Motivating Problems . . . . . . . . . . . . . . . . . 17 2.1.1 Problem 1: The Brachistochrone Problem . . 17 2.1.2 Problem 2: The River Crossing Problem . . 18 2.1.3 Problem 3: The Double Pendulum . . . . . 20 2.1.4 Problem 4: The Rocket Sled Problem . . . 21 2.1.5 Problem 5: Optimal Control in the Life Sciences . . . . . . . . . . . . . . . . . . . . . . 22 2.1.6 Problem 6: Numerical Solutions of Boundary Value Problems . . . . . . . . . . . . . . 24 2.2 Mathematical Background . . . . . . . . . . . . . . 26 2.2.1 A Short Review and Some Notation . . . . . 26 2.2.2 A Review of One Dimensional Optimization 35 2.2.3 Lagrange Multiplier Theorems . . . . . . . . 42 v

vi 2.3

2.4

2.5

Function Spaces . . . . . . . . . . . . . . . . . . . 2.3.1 Distances between Functions . . . . . . . . 2.3.2 An Introduction to the First Variation . . . Mathematical Formulation of Problems . . . . . . . 2.4.1 The Brachistochrone Problem . . . . . . . . 2.4.2 The Minimal Surface of Revolution Problem 2.4.3 The River Crossing Problem . . . . . . . . . 2.4.4 The Rocket Sled Problem . . . . . . . . . . 2.4.5 The Finite Element Method . . . . . . . . . Problem Set for Chapter 2 . . . . . . . . . . . . . .

57 64 68 69 69 72 73 74 76 86

3 The Simplest Problem in the Calculus of Variations 3.1 3.2 3.3

3.4

3.5

The Mathematical Formulation of the SPCV . . . . The Fundamental Lemma of the Calculus of Variations . . . . . . . . . . . . . . . . . . . . . . . . . . The First Necessary Condition for a Global Minimizer . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Examples . . . . . . . . . . . . . . . . . . . Implications and Applications of the FLCV . . . . 3.4.1 Weak and Generalized Derivatives . . . . . . 3.4.2 Weak Solutions to Differential Equations . Problem Set for Chapter 3 . . . . . . . . . . . . .

91 91 95 102 112 117 118 124 125

4 Necessary Conditions for Local Minima 131 4.1 Weak and Strong Local Minimizers . . . . . . . . . 132 4.2 The Euler Necessary Condition - (I) . . . . . . . . . 135 4.3 The Legendre Necessary Condition - (III) . . . . . . 139 4.4 Jacobi Necessary Condition - (IV) . . . . . . . . . . 146 4.4.1 Proof of the Jacobi Necessary Condition . . 152 4.5 Weierstrass Necessary Condition - (II) . . . . . . . 155 4.5.1 Proof of the Weierstrass Necessary Condition 159 4.5.2 Weierstrass Necessary Condition for a Weak Local Minimum . . . . . . . . . . . . . . . . 171 4.5.3 A Proof of Legendre’s Necessary Condition . 176 4.6 Applying the Four Necessary Conditions . . . . . . 178 4.7 Problem Set for Chapter 4 . . . . . . . . . . . . . 180

vii 5 Sufficient Conditions for the Simplest 5.1 A Field of Extremals . . . . . . . . . 5.2 The Hilbert Integral . . . . . . . . . 5.3 Fundamental Sufficient Results . . . 5.4 Problem Set for Chapter 5 . . . . . .

Problem . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

6 Summary for the Simplest Problem

185 186 190 192 197 203

7 Extensions and Generalizations 213 7.1 Properties of the First Variation . . . . . . . . . . . 213 7.2 The Free Endpoint Problem . . . . . . . . . . . . . 215 7.2.1 The Euler Necessary Condition . . . . . . . 218 7.2.2 Examples of Free Endpoint Problems . . . . 221 7.3 The Simplest Point to Curve Problem . . . . . . . . 224 7.4 Vector Formulations and Higher Order Problems . . 238 7.4.1 Extensions of Some Basic Lemmas . . . . . 242 7.4.2 The Simplest Problem in Vector Form . . . 249 7.4.3 The Simplest Problem in Higher Order Form 252 7.5 Problems with Constraints: Isoperimetric Problem . 255 7.5.1 Proof of the Lagrange Multiplier Theorem . 259 7.6 Problems with Constraints: Finite Constraints . . . 263 7.7 An Introduction to Abstract Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 265 7.7.1 The General Optimization Problem . . . . 265 7.7.2 General Necessary Conditions . . . . . . . . 267 7.7.3 Abstract Variations . . . . . . . . . . . . . . 271 7.7.4 Application to the SPCV . . . . . . . . . . . 273 7.7.5 Variational Approach to Linear Quadratic Optimal Control . . . . . . . . . . . . . . . 274 7.7.6 An Abstract Sufficient Condition . . . . . . 275 7.8 Problem Set for Chapter 7 . . . . . . . . . . . . . . 278 8 Applications 8.1 Solution of the Brachistochrone Problem . . . . 8.2 Classical Mechanics and Hamilton’s Principle . 8.2.1 Conservation of Energy . . . . . . . . . . 8.3 A Finite Element Method for the Heat Equation

. . . .

. . . .

283 283 287 292 295

viii 8.4

II

Problem Set for Chapter 8 . . . . . . . . . . . . . . 303

Optimal Control

307

9 Optimal Control Problems 9.1 An Introduction to Optimal Control Problems . . . 9.2 The Rocket Sled Problem . . . . . . . . . . . . . . 9.3 Problems in the Calculus of Variations . . . . . . . 9.3.1 The Simplest Problem in the Calculus of Variations . . . . . . . . . . . . . . . . . . . 9.3.2 Free End-Point Problem . . . . . . . . . . . 9.4 Time Optimal Control . . . . . . . . . . . . . . . . 9.4.1 Time Optimal Control for the Rocket Sled Problem . . . . . . . . . . . . . . . . . . . . 9.4.2 The Bushaw Problem . . . . . . . . . . . . . 9.5 Problem Set for Chapter 9 . . . . . . . . . . . . . .

309 309 313 315

10 Simplest Problem in Optimal Control 10.1 SPOC: Problem Formulation . . . . . . . . . . . . . 10.2 The Fundamental Maximum Principle . . . . . . . 10.3 Application of the Maximum Principle to Some Simple Problems . . . . . . . . . . . . . . . . . . . 10.3.1 The Bushaw Problem . . . . . . . . . . . . 10.3.2 The Bushaw Problem: Special Case γ = 0 and κ = 1 . . . . . . . . . . . . . . . . . . . 10.3.3 A Simple Scalar Optimal Control Problem . 10.4 Problem Set for Chapter 10 . . . . . . . . . . . . .

341 341 343

11 Extensions of the Maximum Principle 11.1 A Fixed-Time Optimal Control Problem . . . . . . 11.1.1 The Maximum Principle for Fixed t1 . . . . 11.2 Application to Problems in the Calculus of Variations . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 The Simplest Problem in the Calculus of Variations . . . . . . . . . . . . . . . . . . . 11.2.2 Free End-Point Problems . . . . . . . . . . 11.2.3 Point-to-Curve Problems . . . . . . . . . .

373 373 375

315 318 319 319 333 338

351 351 358 362 367

377 377 384 385

ix 11.3 Application to the Farmer’s Allocation Problem . . 11.4 Application to a Forced Oscillator Control Problem 11.5 Application to the Linear Quadratic Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 11.5.1 Examples of LQ Optimal Control Problems 11.5.2 The Time Independent Riccati Differential Equation . . . . . . . . . . . . . . . . . . . 11.6 The Maximum Principle for a Problem of Bolza . . 11.7 The Maximum Principle for Nonautonomous Systems . . . . . . . . . . . . . . . . . . . . . . . . 11.8 Application to the Nonautonomous LQ Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 11.9 Problem Set for Chapter 11 . . . . . . . . . . . . .

393 400 404 410 419 429 436 446 453

12 Linear Control Systems 12.1 Introduction to Linear Control Systems . . . . . . . 12.2 Linear Control Systems Arising from Nonlinear Problems . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Linearized Systems . . . . . . . . . . . . . . 12.2.2 Sensitivity Systems . . . . . . . . . . . . . 12.3 Linear Quadratic Optimal Control . . . . . . . . . . 12.4 The Riccati Differential Equation for a Problem of Bolza . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Estimation and Observers . . . . . . . . . . . . . . 12.5.1 The Luenberger Observer . . . . . . . . . . 12.5.2 An Optimal Observer: The Kalman Filter . 12.6 The Time Invariant Infinite Interval Problem . . . 12.7 The Time Invariant Min-Max Controller . . . . . . 12.8 Problem Set for Chapter 12 . . . . . . . . . . . . .

459 459

Bibliography

519

Index

539

473 474 475 478 480 490 494 498 506 509 511

Preface It is fair to say that variational calculus had its beginnings in the 17th century when many of the mathematical giants of that time were focused on solving “calculus of variations” problems. In modern terminology, these early problems in the calculus of variations may be formulated as optimization problems over infinite dimensional spaces of functions. Although this might seem to be a very specialized area, many of the mathematical ideas that were developed to analyze such optimization problems provided the foundations of many areas of modern mathematics. The roots of functional analysis, optimal control, mechanics and the modern theory of partial differential equations can all be traced back to the classical calculus of variations. In addition to its historical connections to many branches of modern mathematics, variational calculus has applications to a wide range of current problems in engineering and science. In particular, variational calculus provides the mathematical framework for developing and analyzing finite element methods. Thus, variational calculus plays a central role in modern scientific computing. Note that the word “modern” appears five times in the previous paragraph. This is no accident. Too often the calculus of variations is thought of as an old area of classical mathematics with little or no relevance to modern mathematics and applications. This is far from true. However, during the first half of the 20th century, most mathematicians in the United States focused on the intricacies of the mathematics and ignored many of the exciting new (modern) applications of variational calculus. This point was not lost on E. J. McShane who made many fundamental contributions to this area. In a 1978 lecture on the history of calculus of variations and xi

xii control theory (see [133] and [134]), McShane explained why his fundamental papers on the classical Bolza problem in the calculus of variations “... burst on the mathematical world with the e´clat of a butterfly’s hiccough.” McShane observed: The problem of Bolza was the most general of the single-integral problems of the calculus of variations. Its mastery gave us the power to answer many deep and complicated questions that no one was asking. The whole subject was introverted. We who were working in it were striving to advance the theory of the calculus of variations as an end in itself, without attention to its relation with other fields of activity. In the same lecture, McShane provided the one reason why Pontryagin and his followers lead the development of optimal control theory: In my mind, the greatest difference between the Russian approach and ours was in mental attitude. Pontryagin and his students encountered some problems in engineering and in economics that urgently asked for answers. They answered the questions, and in the process they incidentally introduced new and important ideas into the calculus of variations. I think it is excusable that none of us in this room found answers in the 1930’s for questions that were not asked until the 1950’s. But I for one am regretful that when the questions arose, I did not notice them. Like most mathematicians in the United States, I was not paying attention to the problems of engineers. The importance of applications as noted by McShane is still valid today. Optimal control is a vibrant and important offshoot of the classical calculus of variations. The development of a modern framework for the analysis and control of partial differential equations was developed by J. L. Lions and is based on variational

xiii theory (see [123], [128], [127] and [125]). Moreover, variational approaches are essential in dealing with problems in stochastic control [21] and differential games [20]. Perhaps one of the most important current applications of variational theory is to modern computational science. The 1972 book by Aubin [12] was in some sense ahead of its time. This book uses variational theory to develop a very general framework for constructing numerical approximation of elliptic boundary problems. Finite element methods produce numerical algorithms that are based on variational (weak) solutions to partial differential equations and provide a powerful approach to simulating a wide variety of physical systems. It is interesting to note that recent advances in computational science have come about because many people in this research community have “paid attention to applications”. Thus, this is a case where focusing on a good application can lead to exciting new mathematics and pave the way for major breakthroughs in computational algorithms. The main goal of this book is to provide an introduction to the calculus of variations and (finite dimensional) optimal control theory with modern applications. The book is based on lecture notes that provided the raw reading material for a course I have taught at Virginia Tech for the past thirty five years. However, the examples and motivating applications have changed and evolved over the years. The object of the course is to introduce the main ideas in a completely rigorous fashion, but to keep the content at a level that is accessible by first year graduate students in engineering and science. For example, we focus mostly on function spaces of piecewise continuous and piecewise smooth functions and thereby avoid measure theory. This is sufficient for the variational calculus and the simplest problem in optimal control. In Part I we develop the calculus of variations and provide complete proofs of the main results. Detailed proofs are given, not for the sake of proving theorems, but because the ideas behind these proofs are fundamental to the development of modern optimization and control theory. Indeed, many of these ideas provide the foundations to all of modern applied and computational mathematics including: functional analysis, distribution theory, the theory of partial differential equations, optimization and finite element methods.

xiv In Part II we focus on optimal control problems and show how optimal control is a natural extension of the classical calculus of variations to more complex problems. Although the proof of the Maximum Principle was a tour de force in applied mathematics, the basic Maximum Principle for the simplest problem is given without proof. A complete (rigorous) proof is not given for two reasons. First, from the author’s teaching experience I have found that the time spent to develop and present a proof of the Maximum Principle adds very little to the understanding of key ideas. Basically there are two approaches to proving the Maximum Principle. One approach is based on functional analysis techniques and would require that the student have a working knowledge of Lebesgue integration and measure theory. This approach is typical in the more mathematical treatments such as found in [100] and [119]. Although there is a second approach that only uses advanced calculus and geometric ideas, the complete proof is rather lengthy. This approach can be found in several excellent references such as [18], [101] and [120] and will not be repeated here. However, the basic Maximum Principle is used to rigorously develop necessary conditions for more complex problems and for optimal control of linear systems. The author feels that going through a rigorous development for extensions of the simplest optimal control problem provides the student with the basic mathematical tools to attack new problems of this type in their particular application area. During the past fifty years a huge number of texts have been published on calculus of variations, optimization, design and control. Clearly this book can not capture this entire body of work. A major objective of this book is to provide the fundamental background required to develop necessary conditions that are the starting points for theoretical and numerical approaches to variational and control problems. Although we focus on necessary conditions, we present some classical sufficient conditions and discuss the importance of distinguishing between the two. In all cases the emphasis is on understanding the basic ideas and their mathematical development so that students leave the course with mathematical tools that allow them to attack new problems of this type. After

xv a thorough treatment of the simplest problems in the Calculus of Variations and Optimal Control, we walk through the process of moving from the simplest problem to more complex problems to help the student see how one might begin to modify the basic optimality conditions to address more difficult problems. This is important since we stress the point: It is impossible to solve all problems in a course and when the student moves on to the ”real working world” there will be new applications and problems to be solved. This book provides the ideas and methodologies that might be used as a starting point to address yet unsolved problems. It is assumed that the reader has a basic background in differential equations and advanced calculus. The notes focus on the fundamental ideas that are needed to rigorously develop necessary and sufficient conditions and to present cases where these ideas have impact on other mathematical areas and applications. In Part I we provide complete proofs of the main theorems on necessary and sufficient conditions. One goal is to make sure the student has a clear understanding of the difference between necessary conditions and sufficient conditions and when to use them. Although this may seem like a trivial issue to mathematicians, the author has found that some students in other disciplines have trouble distinguishing between the two which can lead to mistakes. Moreover, since very little “advanced mathematics” is assumed, the initial proofs are very detailed and in a few cases these details are repeated to emphasize the important ideas which have applications to a variety of problems. In order to keep the book at a reasonable length and to keep the mathematical requirements at the advanced calculus level, we have clearly omitted many important topics. For example, multiintegral problems are not discussed and “direct methods” that require more advanced mathematics such as Sobolev Space theory are also missing. The interested reader is referred to [86] for an elementary introduction to these topics and more advanced treatments may be found in [64], [65], [89] and [186].

xvi Suggested References and Texts The best textbooks to supplement this book are George Ewing’s book, Calculus of Variations with Applications [77] and Leitmann’s book, The Calculus of Variations and Optimal Control [120]. Also, the books by Joseph Z. Ben-Asher [18] and Lee and Markus [119] provide nice introductions to optimal control. Other books that the reader might find useful are [65], [122] and [135]. Finally, the following texts are excellent and will be cited often so that the interested reader can dive more deeply into individual topics. B. Anderson and J. Moore [6] A. Bensoussan, G. Da Prato, M. Delfour and S. Mitter [22] and [23] L. D. Berkovitz [24] G. Bliss [29] O. Bolza [31] A. Bryson and Y. Ho [46] C. Caratheodory [54] F. Clarke [57] R. Courant [59] R. Curtain and H. Zwart [63] A. Forsyth [80] I. Gelfand and S. Fomin [87] L. Graves [95] H. Hermes and J. LaSalle [100] M. Hestenes [101] and [102]

A. Isidori [105] T. Kailath [110] H. Knobloch, A. Isidori and D. Flockerzi [114] H. Kwakernaak and R. Sivan [115] B. Lee and L. Markus [119] J. L. Lions [123] and [125] D. Luenberger [131] L. Neustadt [144] D. Russell [157] H. Sagan [158] D. Smith [166] and [167] J. Troutman [180] F. Wan [183] L. Young [186]

Disclaimer:This book is based on “raw” lecture notes developed over the years and the notes were often updated to include new applications or eliminate old ones. Although this process helped the author find typos and errors, it also meant the introduction of new typos and errors. Please feel free to send the author a list of typos, corrections and any suggestions that might improve the book for future classes.

Acknowledgments I would like to acknowledge all my students who for the past thirty five years have provided feedback on the raw notes that were the basis of this book. Also, I thank my long time colleagues Gene Cliff and Terry Herdman who have provided valuable input, guidance and inspiration over the past three decades. This constant feedback helped me to update the subject matter as the years progressed and to find both typos and technical errors in the material. However, as noted above the constant updating of the material means that even this version is sure to have some typos. This is not the fault of my students and colleagues, so I wish to apologize in advance for not catching all these errors. I especially wish to thank my student Mr. Boris Kramer for a careful reading of the current version. I also thank the reviewers for their comments and valuable suggestions for improving the manuscript, the acquisitions editor Bob Stern at Taylor & Francis for his help and support and the series editor H. T. Banks for encouraging this project. Most importantly, I wish to thank my family for their support and understanding, particularly my wife Gail. They graciously gave up many nights of family time while this manuscript was being written.

xvii

Part I Calculus of Variations

Chapter 1 Historical Notes on the Calculus of Variations It is widely quoted that the calculus of variations (as a mathematical subject) had its beginnings in 1686 when Newton proposed and solved a problem of finding a body of revolution that produces minimum drag when placed in a flow. The problem and its solution were given in his 1687 Philosophiae Naturalis Principia Mathematica (Mathematical Principles of Natural Philosophy). A second milestone occurred in 1696 when Johann (John) Bernoulli proposed the brachistochrone problem as a mathematical challenge problem. In 1697 his brother Jacob (James) Bernoulli published his solution and proposed a more general isoperimetric problem. In addition to the Bernoulli brothers, Newton, Leibniz and L’Hˆopital also gave correct solutions to the brachistochrone problem. This is clearly a precise time in history where a “new” field of mathematics was born. Between 1696 and 1900 a large core of mathematical giants worked in this area and the book by Herman H. Goldstine [91] provides a detailed treatment of this body of work. In particular, John and James Bernoulli, Leonhard Euler, Isaac Newton, Joseph-Louis Lagrange, Gottfried Wilhelm von Leibniz, Adrien Marie Legendre, Carl G. J. Jacobi and Karl Wilhelm Theodor Weierstrass were among the main contributors in this field. Other important contributions during this period were made by Paul du Bois-Reymond, 3

4

Chapter 1. Historical Notes

Johann Peter Gustav Lejeune Dirichlet, William Rowan Hamilton and Pierre-Louis Moreau de Maupertuis. At the end of the 19th century and into the first half of the 20th century, the single integral problem in the calculus of variations was expanded and refined by David Hilbert (Germany), Leonida Tonelli (Italy), Oskar Bolza (Germany and U.S.A.) and the “Chicago school” including G. A. Bliss, L. M. Graves, M. R. Hestenes, E.J. McShane, and W. T. Reid. Around 1950 the basic problem of minimizing an integral subject to differential equation constraints became a major problem of interest because of various military applications in the USA and the USSR. These problems required the treatment of “hard constraints” which were basically ignored in the classical calculus of variations and led to the theory of optimal control. The history of the development of optimal control is less precise and the subject of varying opinions. The paper “300 Years of Optimal Control: From the Brachystochrone to the Maximum Principle” by Sussmann and Willems [172] clearly states that optimal control was born in 1667. Although everyone agrees that optimal control is an extension of the classical calculus of variations, others ([149], [47]) suggest that optimal control theory had its beginning around 1950 with the “discovery” of the Maximum Principle by various groups. The road from the “classical” calculus of variations to “modern” optimal control theory is certainly not linear and it can honestly be argued that optimal control theory had its beginnings with the solution to the brachistochrone problem in 1697 as suggested in [172]. However, two important steps in moving from classical variational approaches to modern control theory occurred between 1924 and 1933. In L. M. Graves’ 1924 dissertation [93] he treated the derivative as an independent function and hence distinguished between state and control variables. In 1926 C. Carath´eodory gave the first formulation of the classical Weierstrass necessary condition in terms of a Hamiltonian [53] which, as noted in [172], is the “first fork in the road” towards modern control theory. Finally, in 1933 L. M. Graves [94] gave a control formulation of the classical Weierstrass condition for a Bolza type problem. These ideas are key to understanding the power of modern optimal control

Chapter 1. Historical Notes

5

methods. The papers [149], [154] and [172] provide a nice historical summary of these results and their impact on modern optimal control. Clearly, everyone agrees to some level that the classical calculus of variations is a starting point for modern optimal control theory. However, what is often lost in this historical discussion is that the calculus of variations also laid the foundations for the creation of other “new” fields in both applied and pure mathematics. Modern functional analysis, the theory of distributions, Hamiltonian mechanics, infinite (and finite dimensional) optimization and the modern theory of partial differential equations all trace their roots to the classical calculus of variations. Perhaps even more relevant today is the role that variational theory plays in modern scientific computing. A key theme in this book is that all these problems fall within the purview of optimization. Although the theoretical issues first appear to be no more difficult than those that occur in finite dimensional optimization problems, there are major differences between infinite and finite dimensional optimization. Moreover, the computational challenges are different and a direct reduction to a finite dimensional problem through approximation is not always the best approach. For example, information about the form and properties of a solution to an infinite dimensional optimization problem can be lost if one introduces approximation too early in the problem solution. The paper [113] by C. T. Kelley and E. W. Sachs provides an excellent example of how the theory of infinite dimensional optimization can yield improved numerical algorithms that could not be obtained from finite dimensional theory alone. We specifically mention that Lucien W. Neustadt’s book on optimization [144] contains the most complete presentation of general necessary conditions to date. This book provides necessary conditions for optimization problems in topological vector spaces. Although a rather deep and broad mathematical background is required to follow the development in this book, anyone seriously considering research in infinite dimensional optimization, control theory and their applications should be aware of this body of work.

6

Chapter 1. Historical Notes

Most early problems in the calculus of variation were motivated by applications in the physical sciences. During the past century, calculus of variations was key to the development of game theory, existence theory for partial differential equations and convergence theory for finite element methods for numerical approximation of partial differential equations. Modern applications in optimal control, especially of systems governed by partial differential equations (PDEs), and computational science has sparked a renewed interest in variational theory and infinite dimensional optimization. The finite element method is based on minimizing certain functionals over spaces of functions and PDE optimization and control problems have applications ranging from fluid flow control to large space structures to the design and control of energy efficient buildings. All these applications lead to problems that require infinite dimensional optimization. In this book we focus on necessary conditions for the classical calculus of variations and then provide a short introduction to modern optimal control and the Maximal Principle.

1.1

Some Typical Problems

The term “calculus of variations” originally referred to problems involving the minimization of a functional defined by an integral J(x(·)) =

Zt1

f (s, x (s) , x˙ (s)) ds

(1.1)

t0

over a suitable function space. Most of the problems we consider will have an integral cost functions of the form (1.1). However, because of significant developments during the past century, we will expand this classical definition of the calculus of variations to include problems that now fall under the topic of optimal control theory. To set the stage, we begin with some standard problems.

1.1.1

Queen Dido’s Problem

As noted above, the calculus of variations as a mathematical subject has its beginnings in 1696, when John Bernoulli suggested the

Chapter 1. Historical Notes

7

brachistochrone problem. Although folklore presents Queen Dido’s problem as one of the first problems in the calculus of variations, real scientific and mathematical investigations into such problems probably started with Galileo and Newton. In any case Queen Dido’s problem provides a nice illustrative example to start the discussion. In Roman mythology, Dido was the Queen of Carthage (modern-day Tunisia). She was the daughter of a king of Tyre. After her brother Pygmalion murdered her husband, she fled to Libya, where she founded and ruled Carthage. The legend has it that she was told that she could rule all the land around the coast that she could “cover with the hide of a cow”. Being very clever, she cut the hide into one continuous thin string and used the string to outline the area she would rule. Thus, Queen Dido’s problem is to find the maximum area that can be encompassed with a string of fixed length.

1.1.2

The Brachistochrone Problem

Galileo discovered that the cycloid is a brachistochrone curve, that is to say it is the curve between two points that is covered in the least time by a body rolling down the curve under the action of constant gravity. In 1602 Galileo provided a geometrical demonstration that the arc of circumference is the brachistochrone path. The apparatus is a wooden frame, mounted on two feet with adjusting screws and carrying a cycloidal groove. Parallel to this is a straight groove, the inclination of which may be varied by inserting wooden pins in the brass-reinforced openings made just underneath the cycloid. A lever with two small clamps allows the release of two balls along the two channels simultaneously. The ball that slides down the cycloidal groove arrives at the point of intersection of the two channels in less time than the ball rolling along an inclined plane. This device can still be seen in Museo Galileo at the Institute and Museum of the History of Science in Florence, Italy. The mathematical formulation of the problem of finding the curve of least descent time, the brachistochrone, was proposed by

8

Chapter 1. Historical Notes

John Bernoulli in 1696. In 1697 the problem was solved by John, his brother James, Newton and others. The solution curve was shown to be a cycloid and Jakob’s solution contained the basic ideas leading to the theory of the calculus of variations.

1.1.3

Shape Optimization

As noted above, in 1686 Newton proposed the problem of finding a body of revolution (nose cone) that produces minimum drag when placed in a flow. In modern terms, the problem is to find a shape of the nose cone to minimize the drag with the constraint that this shape is defined by a surface of revolution. From a historical point of view this is one of the first mathematical formulations of a “shape optimization” problem. It is also interesting to note that because of Newton’s choice of model for the aerodynamic forces, his solution is not accurate for subsonic flows (see [136], [137] and page 52 in [46]). On the other hand, Newton’s assumption is fairly accurate for a hypersonic flow which is important today. In any case, this problem was an important milestone in the calculus of variations. Today Newton might have considered the problem of finding a shape of a body attached to the front of a jet engine in order to produce a flow that matches as well as possible a given flow into the jet (a forebody simulator). The idea is to use a smaller forebody so that it can be placed in a wind tunnel and the engine tested. This problem had its beginnings in 1995 and is based on a joint research effort between the Air Force’s Arnold Engineering Design Center (AEDC) and The Interdisciplinary Center for Applied Mathematics (ICAM) at Virginia Tech. The goal of the initial project was to help develop a practical computational algorithm for designing test facilities needed in the free-jet test program. At the start of the project, the main bottleneck was the time required to compute cost function gradients used in an optimization loop. Researchers at ICAM attacked this problem by using the appropriate variational equations to guide the development of efficient computational algorithms. This initial idea has since been

9

Chapter 1. Historical Notes

FBS

ENGINE INLET

Figure 1.1: The AEDC Test Section refined and has now evolved into a practical methodology known as the Continuous Sensitivity Equation Method (CSEM) for optimal design. The wind tunnel is large enough to hold the engine and a smaller “forebody simulator” (FBS), but not large enough to hold the entire front of the airplane. The objective is to determine the shape of the forebody simulator and the inflow conditions (MACH number, angle, etc.) so that the flow going into the engine inlet matches (as well as possible) a given flow. This given flow is the flow that would be present if the jet were in free flight. This data can be generated by flight test or full 3D simulation. Consider a 2D version of the problem. The green sheet represents a cut through the engine reference plane and leads to the following 2D problem. The goal is to find a shape Γ (that is constrained to be 1/2 the length of the long forebody) and an inflow mach number M0 to match the flow entering the engine inlet generated by the long forebody. This shorter curve is called the “forebody simulator”. This problem is much more complex than Newton’s minimum drag problem because it is not assumed that the forebody is a surface of revolution. Although both problems are concerned with finding optimal shapes (or curves), the forebody simulator problem is still a challenge for mathematics and numer-

10

Chapter 1. Historical Notes

OUT FLOW IN FLOW

ENGINE INLET

Long Forebody

OUT FLOW IN FLOW



ENGINE INLET

FBS

Figure 1.2: The 2D Forebody Simulator Problem ical computations. Given data generated over the long forebody, the goal is to find a shorter (maybe “fatter”) forebody simulator to optimally match the “real data” at the engine inlet. Modern “optimal design” is one area that has its roots in the calculus of variations. However, the computer has changed both the range of problems that are possible to solve and the type of methods used to solve these problems. For more on the analysis, design and optimization of modern engineering systems see [8], [9], [13], [19], [38], [34], [32], [33], [35], [40], [41], [36], [37], [43], [42], [67], [68], [82], [83], [90], [98], [99], [108], [109], [111], [142], [150], [163], [168], [174], [177] and [178].

Chapter 1. Historical Notes

1.2

11

Some Important Dates and People

The following dates provide some insight into the history and the people who contributed to the development of calculus of variations and its modern realization in Optimal Control.

1600-1900 • 1630 - Galileo Galilei (1564-1642) formulated the brachistochrone problem. • 1686 - Isaac Newton (1642-1727) proposed and gave solution to the surface of revolution of minimum drag when the body is moved through a fluid. • 1696 - John Bernoulli (1667-1748) proposed the brachistochrone problem as a challenge to all mathematicians, but to his brother James Bernoulli (1654-1705) in particular. The problem was also solved by Newton, Leibniz (1646-1716), L’Hˆopital (1661-1704), as well as both Bernoulli brothers. • 1697 - James Bernoulli (1654-1705) published his solution and proposed a more general isoperimetric problem. • 1724 - Jacopo Francesco Riccati (1676-1754) wrote his famous paper on the Riccati Equation. James Bernoulli had also worked on this equation. • 1744 - Leonard Euler (1707-1783) extended James Bernoulli’s methods (geometric, analytical) to general problems, and discovered the “Euler Equation”. He also derived Newton’s second law from “the principle of least action”. • 1760 - Joseph Louis Lagrange (1736-1813) first used the term “calculus of variations” to describe the methods he used in his work. • 1762 and 1770 - Lagrange devised an analytic method for general constrained problems. He also indicated that mechanics and variational theory are connected, and he introduced the notation y (x) + δy (x). This symbol, δy (·), was called the variation of y (·).

12

Chapter 1. Historical Notes • 1786 - Lagrange published his necessary condition for a minimum of Newton’s problem. • 1786 - Adrien-Marie Legendre (1752-1833) studied the second variation δ 2 J. • 1788 - Lagrange showed that a large part of Newtonian dynamics could be derived from the principle of least action for “conservative” systems. • 1788 - Lagrange showed that a curve could satisfy the EulerLagrange equation for Newton’s problem and not minimize the functional. He used a proof essentially equivalent to Weierstrass’s condition. • 1835 - Sir William Rowan Hamilton (1805-1865) expanded the principle of least action to “Hamilton’s Principle” of stationary action. • 1837 - Karl Gustav Jacob Jacobi (1804-1851) used some of Legendre’s ideas to construct Jacobi’s (second order) necessary condition. • 1842 - Jacobi gave an example to show that the principle of least action does not hold in general. • 1879 - Karl Theodor Wilhelm Weierstrass (1815-1897) gave his necessary condition for a strong local minimum. • 1879 - Paul David Gustav Du Bois-Reymond (1831-1889) gave proof of the fundamental lemma. In addition to providing a correct proof of the FLCV, in 1873 he gave an example of a continuous function with divergent Fourier series at every point. The term “integral equation” is also due to Du Bois-Reymond. • 1898 - Adolf Kneser (1862-1930) defined focal point.

Chapter 1. Historical Notes

13

1900-1965 • 1900 - David Hilbert (1862-1943) gave a derivation of the “Hilbert Invariant Integral”. • 1904 - Hilbert gave his famous existence proof for the simplest problem. • 1913 - Oskar Bolza (1857-1942) stated the problem of Bolza. The problem of Bolza is the forerunner of modern control problem. • 1933 - Lawrence M. Graves (1896-1973) transformed the problem of Lagrange into a control theory formulation, and proved a “maximum principle” for normal problems. Between 1958 and 1962, V. G. Boltyanskii, R. Gamkrelidze, E. F. Mischenko and L. S. Pontryagin established the Pontryagin Maximum Principle for more general optimal control problems. • 1937 - Lawrence C. Young (1905-2000) introduced generalized curves and relaxed controls. • 1940 - Edward James McShane (1904-1989) established the existence of a relaxed control and proved that a generalized curve was real curve under certain convexity conditions. • 1950 - Magnus R. Hestenes (1906-1991) formulated the first optimal control problems, and gave the maximum principle first published in a Rand Report. • 1952 - Donald Wayne Bushaw (1926-2012) gave a mathematical solution to a simple time optimal control problem by assuming the bang-bang principle. • 1959 - Joesph P. LaSalle (1916-1983) gave the first proof of (LaSalle’s) Bang-Bang Principle. He also extended the classical Lyapunov theory to the LaSalle Invariance Principle.

14

Chapter 1. Historical Notes • 1959 - Richard F. Bellman (1920-1984) developed the dynamic programming principle of optimality for control problems. • 1962 - Lev Semenovich Pontryagin (1908-1988) derived the Pontryagin Maximum Principle along with V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko.

Other Important Players • Gilbert Ames Bliss (1876-1951) - Bliss’s main work was on the calculus of variations and he produced a major book, Lectures on the Calculus of Variations, on the topic in 1946. As a consequence of Bliss’s results a substantial simplification of the transformation theories of Clebsch and Weierstrass was achieved. His interest in the calculus of variations came through two sources, firstly from lecture notes of Weierstrass’s 1879 course of which he had a copy, and secondly, from the inspiring lectures by Bolza which Bliss attended. Bliss received his doctorate in 1900 for a dissertation The Geodesic Lines on the Anchor Ring which was supervised by Bolza. Then he was appointed as an instructor at the University of Minnesota in 1900. He left Minnesota in 1902 to spend a year in G¨ottingen where he interacted with Klein, Hilbert, Minkowski, Zermelo, Schmidt, Max Abraham, and Carath´eodory. • Constantin Carath´eodory (1873-1950) - Carath´eodory made significant contributions to the calculus of variations, the theory of point set measure, and the theory of functions of a real variable. He added important results to the relationship between first order partial differential equations and the calculus of variations. • Jean Gaston Darboux (1842-1917) - Darboux studied the problem of finding the shortest path between two points on a surface and defined a Darboux point.

Chapter 1. Historical Notes

15

• Ernst Friedrich Ferdinand Zermelo (1871-1953) - His doctorate was completed in 1894 when the University of Berlin awarded him the degree for a dissertation Untersuchungen zur Variationsrechnung which followed the Weierstrass approach to the calculus of variations. In this thesis he extended Weierstrass’ method for the extrema of integrals over a class of curves to the case of integrands depending on derivatives of arbitrarily high order. He also introduced the notion of a “neighborhood” in the space of curves. • George M. Ewing (1907-1990) - Ewing’s 1959 book, Calculus of Variations with Applications remains one of the best introductions to the classical theory. • Christian Gustav Adolph Mayer (1839-1907) - Mayer focused on the principle of least action and is credited with formulating variational problems in “Mayer form” where the cost functional is given in terms of end conditions. • Harold Calvin Marston Morse (1892-1977) - Morse developed “variational theory in the large” and applied this theory to problems in mathematical physics. He built his “Morse Theory” on the classical results in the calculus developed by Hilbert. • William T. Reid (1907-1977) - Reid’s work in the calculus of variations combined Sturm-Liouville theory with variational theory to study second order necessary and sufficient conditions. Reid established a generalization of Gronwall’s Inequality which is known as the Gronwall-Reid-Bellman Inequality. The Reid Prize awarded by SIAM for contributions to Differential Equations and Control Theory is named after W. T. Reid. • Frederick A. Valentine (1911-2002) - Valentine attended the University of Chicago where he received his Ph.D. in mathematics in 1937. His dissertation was entitled “The Problem of Lagrange with Differential Inequalities as Added Side Conditions” and was written under the direction of Bliss. Most

16

Chapter 1. Historical Notes of his work was in the area of convexity and his book Convex Sets [182] is a classic. • Vladimir Grigorevich Boltyanskii (1925-) - Boltyanskii was one of the four authors of the book The Mathematical Theory of Optimal Processes and was awarded the Lenin Prize for the work presented in that book on optimal control. • Revaz Valer’yanovich Gamkrelidze (1927-) - Gamkrelidze was one of the four authors of the book The Mathematical Theory of Optimal Processes and was awarded the Lenin Prize for the work presented in that book on optimal control. • Evgenii Frolovich Mishchenko (1922-2010) - Mishchenko was one of the four authors of the book The Mathematical Theory of Optimal Processes and was awarded the Lenin Prize for the work presented in that book on optimal control. • Lev Semenovich Pontryagin (1908-1988) - Pontryagin was 14 when an accident left him blind. The article [4] describes how Pontryagin’s mother, Tat’yana Andreevna Pontryagin took complete responsibility for seeing that her son was educated and successful. As noted in [4]: For many years she worked, in effect, as Pontryagin’s secretary, reading scientific works aloud to him, writing in the formulas in his manuscripts, correcting his work and so on. In order to do this she had, in particular, to learn to read foreign languages. Pontryagin’s early work was on problems in topology and algebra. In the early 1950’s Pontryagin began to study applied mathematics, differential equations and control theory. In 1961 he published The Mathematical Theory of Optimal Processes with his students V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko.

Chapter 2 Introduction and Preliminaries In this chapter we discuss several problems that will serve as models and examples for the theory to follow. We first provide rough overviews of some classical problems to give a preliminary indication of the type of mathematical concepts needed to formulate these problems as mathematical optimization problems. We introduce some notation, discuss various classes of functions and briefly review some topics from calculus, advanced calculus, and differential equations. Finally, we close this chapter with precise mathematical statements of some of these model problems.

2.1

Motivating Problems

In order to motivate the topic and to set the stage for future applications, we provide a brief description of some problems that are important from both historical and scientific points of view.

2.1.1

Problem 1: The Brachistochrone Problem

This problem is perhaps the first problem presented in every endeavor of this type. It is called the brachistochrone problem and it

17

18

Chapter 2. Introduction and Preliminaries

, ●

x

a

y(x)

b

, ● Figure 2.1: The Brachistochrone Problem

was first considered by Galileo Galilei (1564-1642). The mathematical formulation of the problem was first given by John Bernoulli in 1696, and is often quoted as the beginning of the classical calculus of variations. Suppose that P1 = [0 0]T and P2 = [a b]T are two points in the plane with P1 “higher” than P2 (see Figure 2.1). Suppose that we slide a (frictionless) bead down a wire that connects the two points. We are interested in finding the shape of the wire down which the bead slides from P1 to P2 in minimum time. At first glance one might think that the wire should be the straight line between P1 and P2 , but when we solve this problem it will be seen that this is not the case. The mathematical formulation of this problem will be derived below. But first, we describe other “typical problems” in the calculus of variations.

2.1.2

Problem 2: The River Crossing Problem

Another “minimum time” problem is the so called river crossing problem. We assume that the river of a fixed width of one mile has

19

Chapter 2. Introduction and Preliminaries

y v(x)



x

x=1

Figure 2.2: The River Crossing Problem parallel banks and we let the y-axis be the left bank (see Figure 2.2). The current is directed downstream along the y-axis, and the velocity v(·) of the current depends only on x, i.e. v = v (x) is the velocity of the current at a distance of x feet from the left bank. Given v (x) and the assumption that the boat travels at a constant velocity (relative to the surrounding water), we wish to find the “steering angle” of the boat that will move the boat from the point [0 0]T to the right bank in minimum time. Note that we are not concerned with where along the right bank the boat lands. The goal is to find the path of the boat that starts at a prescribed point and reaches the opposite bank in minimum time. This problem is similar to the brachistochrone problem, except that landing site is not prescribed so that the downstream location is free. Thus, this is an example of a “free endpoint problem”. Intuition would seem to imply that the shape of the minimum time crossing path is not dependent on the starting point. In particular, if the boat were to start at [0 y0 ]T with y0 > 0 rather than [0 0]T , then the shape (graph) of the minimum time crossing path would look the same, only shifted downstream by an additional distance of y0 .

20

2.1.3

Chapter 2. Introduction and Preliminaries

Problem 3: The Double Pendulum

Consider an idealized double pendulum with masses m1 and m2 attached to weightless rods of lengths ℓ1 and ℓ2 as shown below. Suppose that we observe the pendulum at two times t0 and t1 and note the angles α1 (t0 ), α2 (t0 ), α1 (t1 ), and α2 (t1 ) and/or angular velocities α˙ i (tj ), i, j = 0, 1. The problem is to write down the governing equations for the pendulum. In particular, we wish to find a system of differential equations in α1 (t) and α2 (t) that describe the motion of the double pendulum. If you recall Newton’s laws of motion and a little mechanics, then the problem is not very difficult. However, we shall see that the problem may be solved by using what is known as Hamilton’s Principle in dynamics. Although Problem 3 does not appear to be related to the first two “optimization problems”, it happens that in order to apply Hamilton’s Principle one must first be able to solve problems similar to each of these.

1(t )

l1 m1

l2

 2 (t )

m2

Figure 2.3: The Double Pendulum Problem

Chapter 2. Introduction and Preliminaries

2.1.4

21

Problem 4: The Rocket Sled Problem

The orbit transfer problem cannot be solved by the classical techniques of the calculus of variations. The more modern theory of optimal control is required to address problems of this type. Another very simple example that falls under the category of optimal control is the so-called rocket car problem. This example will illustrate some of the elementary mathematics that are needed in order to solve the problems we have presented earlier. Consider a rocket sled illustrated in Figure 2.4 below. We assume the car is on a frictionless track, that the sled is of mass m, and that it is controlled by two rocket engines thrusting in opposite directions. It is assumed that the mass m is so large that the weight of the fuel is negligible compared to m. Also, the thrusting force of the rockets is bounded by a maximum of 1. Let x (t) denote the position of the sled with respect to the reference R, at time t. Given that at time t = 0, the sled is at an initial position x0 with initial velocity v0 , we wish to find a thrust force action that will move the sled to position R and stop it there, and we want to accomplish this transfer in minimum time. We derive a simple mathematical formulation of this problem. Let x (t) be as in the figure and let u (t) denote the thrusting force due to the rockets at time t. Newton’s second law may be written as m¨ x (t) = u(t),

STATION x=0

x(t ) Figure 2.4: The Rocket Sled Control Problem

22

Chapter 2. Introduction and Preliminaries

or equivalently, x¨ (t) =



1 m

The initial conditions are given by



u (t) .

x (0) = x0 , x˙ (0) = v0 .

(2.1)

(2.2)

The fact that the thrust force is bounded is written as the constraint |u (t)| ≤ 1. (2.3) The problem we wish to solve may be restated as the following optimal control problem. Find a control function u∗ (t) satisfying the constraint (2.3) such that if x∗ (t) satisfies (2.1)-(2.2), then x∗ (t∗ ) = 0 = x˙ ∗ (t∗ ) for some time t∗ . Moreover, if u (t) is any other control function satisfying (2.3)

with x (t) satisfying (2.1)-(2.2) and x t = 0 = x t for some t, then t∗ ≤ t. Although this problem seems to be the easiest to understand and to formulate, it turns out to be more difficult to solve than the first three problems. In fact, this problem falls outside of the classical calculus of variations and to solve it, one must use the modern theory of optimal control. The fundamental new ingredient is that the control function u(t) satisfies the “hard constraint” |u(t)| ≤ 1. In particular, u(t) can take values on the boundary of the interval [−1, +1].

2.1.5

Problem 5: Optimal Control in the Life Sciences

Although many motivating problems in the classical calculus of variations and modern optimal control have their roots in the physical sciences and engineering, new applications to the life sciences is a very active area of current research. Applications to cancer treatment and infectious diseases present new challenges

23

Chapter 2. Introduction and Preliminaries









 







 



                   

PROLIFERATING CELLS





x1 (t )



 



eu (t )



 

QUIESCENT CELLS

x2 (t )



Figure 2.5: Cancer Control Problem and opportunities for control (see [1], [14], [26], [62], [74], [106], [117], [118], [141], [140], [145], [146], [148], [165], [173]). The following optimal chemotherapy treatment problem may be found in the paper by Fister and Panetta [148]. The cancer cells are divided into two types. The proliferating cells are in a drug-sensitive phase and quiescent cells are in a drug-resistant phase (see Figure 2.5). The growth dynamics are given by the system x˙ 1 (t) = (γ − δ − α − eu(t)) x1 (t) + βx2 (t) x˙ 2 (t) = αx1 (t) − (λ + β) x2 (t) with initial data x1 (0) = x1,0 and x2 (0) = x2,0 . Here, x1 (·) is the cell mass of proliferating cancer cells and x1 (·) is the cell mass of quiescent cells in the bone marrow. The parameters are all constant. Here γ is the cycling cells’ growth rate, α is the transition rate from proliferating to resting, δ is the natural cell death rate, β is the transition rate from resting to proliferating, λ is cell differentiation (where mature bone marrow cells leave the

24

Chapter 2. Introduction and Preliminaries

bone marrow and enter the blood stream as various types of blood cells) and e is the effectiveness of the treatment. The function u(·) is the control describing the effects of the chemotherapeutic treatment which has impact only on the proliferating cells. The control function is assumed to be P W C(0, T ) and satisfies 0 ≤ u(t) ≤ 1. The cost function is defined by  Z T b 2 ˜ J(u(·)) =− (1 − u(s)) − q(x1 (t) + x2 (t)) ds 2 0 where b and q are weighting parameters and the goal is to maximize ˜ on the set J(·) Θ = {u(·) ∈ P W C(0, T ) : u(t) ∈ [0, 1]} . This cost function is selected so that one can give as much drug as possible while not excessively destroying the bone marrow. The weighting parameters b and q are selected depending on the importance of the terms. Notice the negative sign in front of the integral ˜ is equivalent to minimizing so that maximizing J(·)  Z T b 2 (1 − u(s)) − q(x1 (t) + x2 (t)) ds J(u(·)) = 2 0 on Θ. We note that similar problems occur in the control of HIV (see [1] and [14]) and the models vary from application to application. In addition, the models of cell growth have become more complex and more realistic leading to more complex control problems. As indicated by the current references [14], [26], [62], [74], [117], [118], [146] and [165], this is a very active area of research.

2.1.6

Problem 6: Numerical Solutions of Boundary Value Problems

The finite element method is a powerful computational method for solving various differential equations. Variational theory and the calculus of variations provide the mathematical framework required to develop the method and to provide a rigorous numerical

25

Chapter 2. Introduction and Preliminaries

analysis of the convergence of the method. In order to describe method and to illustrate the key ideas, we start with a simple two-point boundary problem. Given an integrable function f (·), find a twice differentiable function x(·) such that x(·) satisfies the differential equation − x¨(t) + x(t) = f (t),

0 < t < 1,

(2.4)

subject to the Dirichlet boundary conditions x(0) = 0,

x(1) = 0.

(2.5)

Observe that it is assumed that x(·) has two derivatives since x¨(·) appears in the equation. The goal here is to find a numerical solution of this two-point boundary value problem by approximating x(·) with a continuous piecewise linear function xN (·) as shown in Figure 2.6 below. The idea is to divide the interval [0, 1] into N + 1 subintervals (called elements) with nodes 0 = tˆ0 < tˆ1 < tˆ2 < . . . < tˆN −1 < tˆN < tˆN +1 = 1 and construct the

x

xN () 

x()

0

1 Figure 2.6: A Piecewise Linear Approximation

t

26

Chapter 2. Introduction and Preliminaries

approximation xN (·) so that it is continuous on all of [0, 1] and linear between the nodes. It is clear that such a continuous piecewise linear approximating function xN (·) may not be differentiable at the nodes and hence can not satisfy the differential equation (2.4). The finite element method is based on developing a weak (or variational) form of the two-point boundary value problem and using this formulation to devise a computational method for constructing the approximating solution xN (·). We will work through this process in Section 2.4.5 below.

2.2

Mathematical Background

In order to provide a mathematical formulation of the above problems, we must first introduce some notation and discuss various classes of functions. At this point it is assumed that the reader is familiar with the basics of advanced calculus and elementary differential equations. However, we briefly review some topics from calculus, advanced calculus, and differential equations. Since this is a review of background material, we will not go into much detail on any topic.

2.2.1

A Short Review and Some Notation

In mathematical writings employing consistent and careful notation helps improve the understanding of the theory and reduces common errors in the application of the theory to problems. However, as is typical in mathematical texts it is sometimes useful to abuse notation as long as the precise meaning is made clear in the presentation. Every attempt is made to use precise notation and when notation is abused, we point out the abuse to keep the exposition as clear as possible. These observations are especially important for the notation used to describe functions since understanding functions is key to the development of the material in this book. In particular, it is essential to distinguish between a function, its name and the value of the function at a point in its domain.

Chapter 2. Introduction and Preliminaries

27

Remark 2.1 Let X and Y be two sets and assume that F (·) is a function with domain in X and range in Y . Observe that we use the symbol F (·) to denote the function rather than using the “name” of the function F . For example, if x is a real number and one writes F (x) = x2 , then the corresponding function F (·) has domain and range contained in the set of real numbers and for each x in the domain of F (·), the value of F (·) at x is given by F (x) = x2 . Thus, we distinguish between the function F (·), its name F and its value at a specific point F (x). At first glance these distinctions may seem like “overkill”. However, using the same symbol for a function and its name is often a major source of confusion for students and leads to a lack of understanding of the theory and its application. Let F (·) be a function with domain D ⊆ X and range R ⊆ X. To emphasize the relationship between the function F (·), its domain and its range we write D(F ) = D and R(F ) = R. This notation merely states the obvious that the names of the domain and range of a function F (·) should be attached to the name F of the function F (·). We denote this by writing F : D(F ) ⊆ X → Y. If the domain of F (·) is all of X, i.e. if D(F ) = X, then we write F : X → Y. For completeness, recall that the range of a function F (·) is given by R(F ) = {y ∈ Y | y = F (x), x ∈ D(F )} .

In elementary calculus the sets X and Y are often intervals of the form [a, b], (a, b], [a, b) or (a, b). In advanced calculus X and Y may be n-dimensional Euclidean spaces. In the calculus of variations and optimal control X and Y are most often spaces of functions. The following notation is rather standard and will be used throughout the book. • The space of real numbers is denoted by R and the space of complex numbers by C, respectively. However, in some cases

28

Chapter 2. Introduction and Preliminaries we may use the notation R1 to emphasize the fact that the real line is also the state space for a 1-dimensional system. • Real n-dimensional Euclidean space, denoted by Rn , is the space of n-dimensional real column vectors       x1       x2     n R = x =  ..  : xi ∈ R .    .        xn Likewise, complex n-dimensional Euclidean space, denoted by Cn , is the space of n-dimensional complex column vectors       z 1       z2     n C = z =  ..  : zi ∈ C .    .        zn

Remark 2.2 Note that we use boldface letters for vectors so that if one writes x ∈ Rn , then one knows that x is a vector with n real components. Also, in special cases we will abuse notation and use the symbols R1 for R and C1 for C, respectively. • If x ∈ R, then the absolute value of x is denoted by |x| and if z = x p + iy ∈ C, then the complex modulus is denoted by |z| = x2 + y 2 . The complex conjugate of z = x + iy is z¯ = x − iy.   z1  z2    • The transpose of a column vector  ..  is a row vector  .  zn given by  T z1  z2       ..  , z1 z2 · · · zn  .  zn

29

Chapter 2. Introduction and Preliminaries and conversely, 

z1 z2 · · ·

zn

T



  = 

z1 z2 .. . zn



  . 

T  ∈ Rn , then the Euclidean (or • If x = x1 x2 · · · xn 2) norm of x is given by v u n uX kxk = kxk2 = t |xi |2 i=1

 T and likewise, if z = z1 z2 · · · zn ∈ Cn , then the Euclidean (or 2) norm of z is given by v u n uX |zi |2 . kzk = kzk2 = t i=1

 T • If x = x1 x2 · · · xn ∈ Rn and 1 ≤ p < +∞, then the p-norm of x is given by v " n #1/p u n X uX p |xi |p = |xi |p kxkp = t i=1

i=1

T  ∈ Cn , then the and likewise, if z = z1 z2 · · · zn p-nor m of z is given by v " n #1/p u n X uX p p kzkp = t |zi |p = |zi | . i=1

i=1

The norms above are special cases of the general concept of a norm which measures the magnitude of a vector and provides a mechanism to define distance between vectors. In general, we have the following definition.

30

Chapter 2. Introduction and Preliminaries

Definition 2.1 A norm on Rn (or Cn ) is a function k·k : Rn (or Cn ) → R satisfying: 1. kxk ≥ 0 and kxk = 0 if and only if x = 0. 2. kα · xk = |α| · kxk , α ∈ R (or C) and x ∈ Rn (or Cn ). 3. kx + yk ≤ kxk + kyk , for x, y ∈ Rn (or Cn ) (the triangle inequality). Let k·k : Rn → R be a norm on Rn . If x ˆ ∈ Rn and δ > 0, then the (open) δ-neighborhood of x ˆ is the open ball of radius δ centered at x ˆ given by U (ˆ x, δ) = {x ∈ Rn : kx − x ˆk < δ} . On the real line, U (ˆ x, δ) is the open interval centered at xˆ with radius δ. We sometimes abbreviate the δ -neighborhood by δ-nbd. If f : D(f ) ⊆ Rn −→ R is a real valued function of n real variables and x ∈ D(f ), then we often write f (x) = f (x1 , x2 , ..., xn ) rather than   x1  x 2    T    x1 x2 · · · xn . f (x) = f  ..  = f  .   xn

Remark 2.3 The use of f (x) = f (x1 , x2 , ..., xn ) rather than f (x) = f ([x1 , x2 , ..., xn ]T ) is standard “abuse of notation” and should cause little misunderstanding in the material. We shall use various notations for partial derivatives. For example, we use subscripts for partial derivatives such as fxi (x), = fxi (x1 , x2 , ..., xn ) = and

∂f (x) ∂f (x1 , x2 , ..., xn ) = ∂xi ∂xi

∂ 2 f (x, y) . fxy (x, y) = ∂x∂y

Chapter 2. Introduction and Preliminaries

31

If x : I → Rn is a vector-valued function with domain defined by an interval I ⊂ R, then it follows that there exist n real valued functions xi (·), i = 1, 2, . . . , n, such that   x1 (t)  x2 (t)   T   . x(t) =  ..  = x1 (t) x2 (t) · · · xn (t)  .  xn (t)

We use the standard definitions of continuity and differentiability for real valued functions. Assume x : [t0 , t1 ] −→ R is a real valued function. If t = tˆ ∈ [t0 , t1 ), then the right-hand limit of x(·) at t = tˆ is defined by
x tˆ+ = lim [x(t)] , t→tˆ+

provided that this limit exist (it could be infinite). Likewise, if t = tˆ ∈ (t0 , t1 ], then the left-hand limit of x(·) at t = tˆ is defined by
x tˆ− = lim [x(t)] , t→tˆ−

provided that this limit exist (it could be infinite). If x : [t0 , t1 ] −→ R is a real valued function which is differentˆ) all denote the derivative of tiable at t = tˆ, then x( ˙ tˆ), x′ (tˆ), and dx( dt ˆ x(·) at t = t. If t = t0 , then x(t ˙ 0 ) denotes the right-hand derivative x˙ + (t0 ) defined by   x(t) − x(t0 ) d+ x(t0 ) + = x˙ (t0 ) = lim+ dt t − t0 t→t0 and if t = t1 , then x(t ˙ 1 ) denotes the left-hand derivative x˙ − (t0 ) defined by   d− x(t1 ) x(t) − x(t1 ) − , = x˙ (t1 ) = lim− dt t − t1 t→t1 provided these limits exist and are finite.

32

Chapter 2. Introduction and Preliminaries


+ ˆ Remark 2.4 It is important to note that even if d dtx(t) and x˙ tˆ+ both exist at a point tˆ, they may not be the same so that in general
d+ x(tˆ) 6= x˙ tˆ+ . x˙ + (tˆ) = dt

Clearly, the same is true for the left-hand derivatives and limits. In words, the one-sided derivative of a function at a point is not the one-sided limit of the derivative at that point. Example 2.1 Let x : [−1, 1] −→ R, be  2  t sin(1/t), 0 x(t) =  t2

defined by t > 0, t = 0, t < 0.

Computing the right-hand derivative at tˆ = 0, it follows that    2  d+ t sin(1/t) − 0 x(t) − x(0) + = lim+ x(0) = x˙ (0) = lim+ t→0 t→0 dt t t = lim+ [t sin(1/t)] = 0 t→0

exists and is finite. On the other hand, if 0 < t ≤ 1, then x(t) ˙ exists and x(t) ˙ = 2t sin(1/t) − cos(1/t) and

lim [2t sin(1/t) − cos(1/t)]

t→0+

does not exist. Hence, d+ x(0) = x˙ + (0) 6= x(0 ˙ + ). dt However, it is true that d− x(0) = x˙ − (0) 6= x(0 ˙ − ) = 0. dt Although, as the previous example illustrates, one can not interchange the limit process for one-sided derivatives, there are important classes of functions for which this is true. Consider the following example.

Chapter 2. Introduction and Preliminaries

33

Example 2.2 If x(t) = |t|, then x(t) − x (0) |t| d+ x(0) = lim+ = lim+ =1 t→0 t→0 dt t−0 t and d− x(0) x(t) − x (0) |t| −t = lim− = lim− = lim− = −1. t→0 t→0 t→0 dt t−0 t t Note also that

d+ x(0) = lim+ x(t) ˙ =1 t→0 dt

and

d− x(0) = lim− x(t) ˙ = −1 t→0 dt are the left and right-hand limits of x(·) ˙ at 0. Note that x(t) = |t| is differentiable at all points except t = 0. Definition 2.2 Let I denote an interval and assume that x : I ⊆ R1 −→ R1 is a real valued function. For a given integer k ≥ 1, we say that x(·) is C k on I if x(·) and all its derivatives of order k exist and are continuous at all points t ∈ I. We say that x(·) is C 0 on I if x(·) is continuous and x(·) is C ∞ on I if x(·) is C k for all k ≥ 0. If x(·) is C ∞ on I, then we call x(·) a smooth function. Thus, we define the function spaces C k (I) = C k (I; R1 ) by  C k (I) = C k (I; R1 ) = x : I ⊆ R1 −→ R1 : x(·) is C k on I . Definition 2.3 Let I denote an interval and assume that x : I → Rn is a vector-valued function. We say that the function x(·) =  T x1 (·) x2 (·) · · · xn (·) is continuous at ˆt if, for each ǫ > 0, there is a δ > 0 such that if t ∈ I and 0 0 such that f (x∗ ) ≤ f (x) for all x ∈ U (x∗ , δ) ∩ I. If in addition, f (x∗ ) < f (x) for all x ∈ U (x∗ , δ) ∩ I with x 6= x∗ , then we say that x∗ provides a proper local minimum for f (·) on I. If x∗ ∈ I is such that f (x∗ ) ≤ f (x) for all x ∈ I, then x∗ is said to provide a global minimum for f (·) on I. In theory, there is always one way to “find” global minimizers. Simply pick x1 ∈ I and “test” x1 to see if f (x1 ) ≤ f (x) for all x ∈ I. If so, then x1 is a global minimizer, and if not, there is a x2 with f (x2 ) < f (x1 ). Now test x2 to see if f (x2 ) ≤ f (x) for all x ∈ I, x 6= x1 . If so, x2 is the minimizer, and if not, we can find x3 so that f (x3 ) < f (x2 ) < f (x1 ). Continuing this process generates sequence of points {xk : k = 1, 2, 3, . . .} satisfying . . . f (xk+1 ) < f (xk ) < . . . < f (x3 ) < f (x2 ) < f (x1 ) .

36

Chapter 2. Introduction and Preliminaries

Such a sequence is called a minimizing sequence and under certain circumstances the sequence (or a subsequence) will converge to a point x∗ and if x∗ ∈ I it will be a minimizer. The above “direct procedure” may be impossible to accomplish in finite time. Therefore, we need some way to reduce the number of possible candidates for x∗ . We can do this by applying elementary necessary conditions from calculus. Clearly, if x∗ is a global minimizer, then x∗ is a local minimizer. We shall try to find all the local minima of f (·) by using standard necessary conditions. The following theorem can be found in almost any calculus book. Theorem 2.1 Suppose that f : I → R is continuous, where I is an interval with endpoints a < ∗b and assume that x∗ ∈ I is a local (x ) exists, then minimizer for f (·) on I. If dfdx df (x∗ ) = 0, if a < x∗ < b, dx

(2.6)

df (x∗ ) ≥ 0, if x∗ = a, dx

(2.7)

df (x∗ ) ≤ 0, if x∗ = b. dx

(2.8)

or or

In addition, if

d2 f (x∗ ) dx2

= f ′′ (x∗ ) exists, then f ′′ (x∗ ) ≥ 0.

(2.9)

It is important to emphasize that Theorem 2.1 is only a necessary condition. The theorem says that if x∗ is a local minimizer, then x∗ must satisfy (2.6), (2.7) or (2.8). It does not imply that points that satisfy (2.6)-(2.8) are local minimizers. Necessary conditions like Theorem 2.1 can be used to reduce the number of possible candidates that must be tested to see if they provide a minimum to f (·). Consider the following example that illustrates the above theorem.

37

Chapter 2. Introduction and Preliminaries

Example 2.3 Let I = [−2, 3) and define f (x) = x4 − 6x2 . Let us try to find the local (maximum and) minimum. In order to apply Theorem 2.1 we compute the derivatives f ′ (x) = 4x3 − 12x and f ′′ (x) = 12x2 − 12.

If x∗ is a local minimum and −2 < x∗ < 3, then


f ′ (x∗ ) = 4[x∗ ]3 − 12[x∗ ] = 4x∗ [x∗ ]2 − 3 = 0.

Solving this equation, we find that x∗ could be either 0, Checking (2.9) at x∗ = 0, yields

√

√ 3 , − 3.

f ′′ (0) = 12 · 0 − 12 = −12 < 0 so that condition (2.9) fails and x∗ = 0 is not a local minimizer. Also, checking condition (2.7) at x∗ = −2, yields f ′ (−2) = −32 + 24 = −8 < 0 so, x∗ = −2 is not a candidate. Thus, an application of Theorem √ 2.1 has √ reduced the process to that of “testing” the two points 3 and− 3. The second derivative test (2.9) yields √ √ f ′′ (± 3) = 12[± 3]2 − 12 = 12(3 − 1) = 24 > 0 √ √ so that x∗ = − 3, and x∗ = + 3 are possible local minimum. This is clear from the graph, but what one really needs are sufficient conditions. Theorem 2.2 Suppose that f (·) is as in Theorem 2.1. If x∗ ∈ (a, b), f ′ (x∗ ) = 0 and f ′′ (x∗ ) > 0, then x∗ provides a local minimum for f (·) on [a, b]. We close this section with a proof of Theorem 2.1. The proof is very simple, but the idea behind the proof happens to be one of the key ideas in much of optimization.

38

Chapter 2. Introduction and Preliminaries

30

25

20

15

10

5

0

−5

−10 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

3

Figure 2.7: Plot of the Cost Function Proof of Theorem 2.1: First assume that a < x∗ < b, is a local minimizer. This means that there is a δ > 0 such that f (x∗ ) ≤ f (x) for all x ∈ U (x∗ , δ) ∩ I. Let δ1 = 21 min{δ, b − x∗ , x∗ − a} > 0 and note that the open interval U (x∗ , δ1 ) is contained in I. In particular, if −δ1 < ε < δ1 , then the “variation” x∗ + ε ∈ U (x∗ , δ1 ) ⊂ U (x∗ , δ) ∩ I. Hence x∗ satisfies f (x∗ ) ≤ f (x∗ + ε), or equivalently, 0 ≤ f (x∗ + ε) − f (x∗ ).

(2.10)

Chapter 2. Introduction and Preliminaries

39

Observe that this inequality holds for positive and negative ε. Dividing both sides of (2.10) by ε > 0, yields the inequality 0≤

f (x∗ + ε) − f (x∗ ) , ε

and passing to the limit as ε → 0+ it follows that 0 ≤ f ′ (x∗ ).

(2.11)

Likewise, dividing both sides of (2.10) by ε < 0, yields the reverse inequality f (x∗ + ε) − f (x∗ ) 0≥ , ε and passing to the limit as ε → 0− it follows that 0 ≥ f ′ (x∗ ).

(2.12)

Combining (2.11) with (2.12) it follows that f ′ (x∗ ) = 0 and we have established (2.6). Consider now the case where x∗ = a. Let δ1 = 12 min{δ, b − x∗ } > 0 and note that if 0 < ε < δ1 , then the “variation” x∗ + ε = a + ε ∈ U (x∗ , δ1 ) ∩ I ⊂ U (a, δ) ∩ I. Hence x∗ = a satisfies f (a) ≤ f (a + ε), or equivalently, 0 ≤ f (a + ε) − f (a).

Observe that this inequality holds for 0 < ε < δ1 . Dividing both sides of this inequality by ε > 0, yields 0≤

f (a + ε) − f (a) , ε

and passing to the limit as ε → 0+ it follows that 0 ≤ f ′ (a). This completes the proof of (2.7). The case x∗ = b is completely analogous. 

40

Chapter 2. Introduction and Preliminaries

Remark 2.7 The important idea in the above proof is that when a < x∗ < b, the variations x∗ + ε belong to U (x∗ , δ) ∩ I for both positive and negative values of ε. In particular, one can “approach” x∗ from both directions and still be inside U (x∗ , δ) ∩ I. However, when x∗ = a, the variations x∗ + ε belong to U (x∗ , δ) ∩ I only when ε is positive. Thus, one can “approach” x∗ only from the right and still remain in U (x∗ , δ) ∩ I. This simple observation is central to much of what we do in deriving necessary conditions for the simplest problem in the calculus of variations. Remark 2.8 It is extremely important to understand that care must be exercised when applying necessary conditions. A necessary condition usually assumes the existence of of an optimizer. If an optimizer does not exist, then the necessary condition is vacuous. Even worse, one can draw the incorrect conclusion by applying the necessary condition. Perron’s Paradox (see [186]) provides a very simple example to illustrate the danger of applying necessary conditions to a problem with no solution. Let Φ = {N : N is a positive integer}

(2.13)

J(N ) = N.

(2.14)

and define ˆ ∈ Φ maximizes J(·) on the set Φ, i.e. that N ˆ is the Assume that N ˆ ≥ 1 which implies that N ˆ2 ≥ N ˆ. largest positive integer. Thus, N 2 ˆ is a positive integer and N ˆ is the largest posiHowever, since N 2 ˆ ˆ ˆ2 ≤ N ˆ ≤N ˆ2 tive integer, it follows that N ≥ N . Consequently, N ˆ ≤1≤N ˆ so that N ˆ = 1. Therefore, if one which implies that N assumes that the optimization problem (2.13) - (2.14) has a solution, then one can (correctly) prove that the largest positive integer ˆ = 1. Of course the issue is a point of logic, where a false asis N sumption can be used to prove a false conclusion. If one assumes an optimizer exists and it does not, then necessary conditions can be used to produce incorrect answers. Unlike the simple Perron Paradox, it is often difficult to establish the existence of solutions to calculus of variation and optimal control problems. One should take this remark as a warning when applying necessary conditions to such problems.

41

Chapter 2. Introduction and Preliminaries

3 2.5 2 1.5 1 0.5 0 -0.5 -1 0

0.2

0.4

0.6

0.8

1

Figure 2.8: Plot of the Cost Function f (·) on (0, 1] Example 2.4 Let I = (0, 1] and define f (x) = 5x2 − x4 − 1. The plot of f (·) on the interval (0, 1] is shown in Figure 2.8. It is also clear that f (·) does not have a global nor local minimizer on the (0, 1] since 0 ∈ / (0, 1]. However, if one “relaxes” (expands) the problem and considers the problem of minimizing f (·) on the closure of I given by I¯ = [0, 1], then x¯∗ = 0 ∈ (0, 1] solves the “relaxed problem”. In addition, there exist a sequence of points xk ∈ I = (0, 1] satisfying xk −→ x¯∗ = 0 and f (xk ) −→ f (¯ x∗ ) = −1. This process of relaxing the optimization problem by expanding the set of admissible points is an important idea and leads to the concepts of generalized curves and relaxed controllers. The key point

42

Chapter 2. Introduction and Preliminaries

is that by relaxing the problem one obtains existence of a minimizer in a larger set. But equally important, it can be shown that the solution to the relaxed problem can be approximated by points in the original constraint set. This general idea also has profound applications to the theory of nonlinear partial differential equations. In particular, a “viscosity solution” is an extension of the classical concept of what is meant by a “solution” to certain partial differential equations. The basic idea is to define a viscosity solution as a limit of (classical) solutions that are parameterized by parameter that tends to zero. This method produces existence of (weak) solution and these solutions are close related to the concept of Young’s “generalized curves”. The references [24], [61], [60], [75] and [186] provide an introduction to these ideas.

2.2.3

Lagrange Multiplier Theorems

The constraint set for the general optimization problem is often defined in terms of equality (or inequality) constraints. The isoperimetric problem in the calculus of variations is such a problem. In this section assume there are two vector spaces Z and Y and two functions J : D(J ) ⊆ Z −→ R1 (2.15)

and

G : D(G) ⊆ Z −→ Y.

(2.16)

The function J (·) is called the cost function and G(·) is called the constraint function. Define the constraint set ΘG ⊆ Z by ΘG = {z ∈ D(G) : G(z) = 0 ∈ Y } ⊂ D(G).

(2.17)

The Equality Constrained Optimization Problem is defined to be: Find an element z ∗ ∈ ΘG ∩ D(J ) such that J (z ∗ ) ≤ J (z)

for all z ∈ ΘG ∩ D(J ).

43

Chapter 2. Introduction and Preliminaries

Observe that since ΘG ⊂ D(G), it follows that ΘG ∩ D(J ) ⊂ D(G) ∩ D(J ). Therefore, the equality constrained optimization problem is equivalent to finding z ∗ ∈ D(G) ∩ D(J ) such that z ∗ minimizes J (z) subject to G(z) = 0 ∈ Y . We first discuss special cases and then move to the more abstract versions. Lagrange Multiplier Theorem in Rn We consider the finite dimensional constrained optimization problem in n variables and m equality constraints. For the sake of simplicity of presentation, we assume that the cost function J : Rn −→ R1 has domain equal to all Rn , i.e. D(J ) = Rn . Also, we assume that the constraint function G : Rn −→ Rm has domain equal to all Rn , i.e. D(G) = Rn and m < n. In particular, there are m real-valued functions g i : Rn −→ R1 ,i = 1, 2, . . . , m, such that



  G(z) =  

g 1 (z) g 2 (z) .. . g m (z)



  , 

T  ∈ Rn . We assume that all the where z = x1 x2 · · · xn functions gi (·), i = 1, 2, ..., n, are C 1 real valued functions of the T  ∈ Rn so that the gradin real variables z = x1 x2 · · · xn ents       ∇J (z) =   

∂J(z) ∂x1 ∂J(z) ∂x2

.. .

∂J(z) ∂xn

     and ∇g i (z) =     

∂ g i (z) ∂x1 ∂ g i (z) ∂x2

.. .

∂ g i (z) ∂xn

    

44

Chapter 2. Introduction and Preliminaries

exist and are continuous on Rn . The (equality) constrained minimization problem is to minimize J (z) subject to G(z) = 0 ∈ Rm . Define the Lagrangian L : R1 × Rm × Rn → R1 by L(λ0 , λ, z) , λ0 J (z) +

m X

λi g i (z),

(2.18)

i=1

 where λ = λ1 λ2 · · · can be written as

λm

T

∈ Rm . Observe that L(λ0 , λ, z)

L(λ0 , λ, z) = λ0 J (z) + hλ, G(z)im where h·, ·im is the standard inner product on Rm . Also, the gradient ∇z L(λ0 , λ, z) is given by ∇z L(λ0 , λ, z) = λ0 ∇J (z) +

m X i=1

λi ∇g i (z).

(2.19)

Finally, we shall need the Jacobian of G(·) which is the m × n matrix defined by   ∂ g 1 (z) ∂ g 1 (z) ∂ g 1 (z) · · · ∂x2 ∂xn 1  ∂ ∂x g 2 (z) ∂ g 2 (z) ∂ g 2 (z)    ∂x · · · ∂x2 ∂xn 1 , (2.20) JG(z) ,  .. .. .. ..   . . . .   ∂ g m (z) ∂x1

∂ g m (z) ∂x2

···

which can be written as  JG(z) = ∇g 1 (z) ∇g 2 (z) · · ·

∂ g m (z) ∂xn

∇g m (z)

As above we set

T

.

ΘG = {z ∈ Rn : G(z) = 0 ∈ Rm } and state the Lagrange Multiplier Theorem for this problem. Theorem 2.3 (Multiplier Theorem for the n-D Problem) If z ∗ ∈ ΘG minimizes J (·) on ΘG then there exists a constant λ∗0  T and vector λ∗ = λ∗1 λ∗2 · · · λ∗m ∈ Rm such that

45

Chapter 2. Introduction and Preliminaries (i) |λ∗0 | + kλ∗ k 6= 0 and (ii) the minimizer z ∗ satisfies ∇z L(λ∗0 , λ∗ , z ∗ )

=

λ∗0 ∇J

∗

(z ) +

m X i=1

λ∗i ∇g i (z ∗ ) = 0. (2.21)

(iii) If in addition the gradients ∇g 1 (z ∗ ), ∇g 2 (z ∗ ), . . . , ∇g m (z ∗ ) are linearly independent, i.e. the Jacobian JG(z) =  T ∇g 1 (z ∗ ) ∇g 2 (z ∗ ) · · · ∇g m (z ∗ ) has maximal rank m, then λ∗0 is not zero. Remark 2.9 Note that condition (iii) implies that the linear operator T : Rn → Rm defined by T h = [JG(z ∗ )]h is onto all of Rm . This form of condition (iii) is the key to more abstract forms of the Lagrange Multiplier Theorem. We say that the minimizer z ∗ is a normal minimizer if λ∗0 6= 0. Example 2.5 (A Normal Problem) Consider the problem in R2 of minimizing J (x, y) = x2 + y 2 subject to the single (m = 1) equality constraint G (x, y) = y − x2 − 1 = 0. The gradients of J (·) and G(·) are given by  T ∇J (x, y) = 2x 2y

and

∇G (x, y) =



−2x 1

T

,

respectively.  T Assume that z ∗ = x∗ y ∗ minimizes J (·, ·) subject to G(x, y) = 0. The Lagrange Multiplier Theorem above implies that there exist λ∗0 and λ∗1 ∈ R1 such that λ∗0 ∇J (x∗ , y ∗ ) + λ∗1 ∇G (x∗ , y ∗ ) = 0

(2.22)

46

Chapter 2. Introduction and Preliminaries

and |λ∗0 | + |λ∗1 | = 6 0. Observe that if λ∗0 = 0, then it follows from (2.22) that  T  T λ∗1 ∇G (x∗ , y ∗ ) = λ∗1 −2x∗ 1 = −λ∗1 2x∗ λ∗1 = 0,

which implies that λ∗1 = 0. Since λ∗0 = 0 this would imply that |λ∗0 | + |λ∗1 | = 0 which contradicts the theorem. Therefore, λ∗0 6= 0 and if we define λ = λ∗1 /λ∗0 , then (2.22) is equivalent to ∇J (x∗ , y ∗ ) + λ∇G (x∗ , y ∗ ) = 0.

(2.23)

Consequently, the Lagrange Multiplier Theorem yields the following system 2x∗ − λ2x∗ = 2x∗ (1 − λ) = 0, 2y ∗ + λ = 0, g (x∗ , y ∗ ) = y ∗ − [x∗ ]2 − 1 = 0.

Therefore, either λ = 1 or x∗ = 0. However, if λ = 1, then it would follow that y ∗ = −λ/2 = −1/2 and −1/2 = y ∗ = [x∗ ]2 + 1 > 0 which is impossible. The only solution to the system is x∗ = 0, y ∗ = 1 and λ = −2. The previous example was normal in the sense that one could show that λ∗0 6= 0 and hence the Lagrange Multiplier Theorem resulted in three (nonlinear) equations to be solved for for three unknowns x∗ , y ∗ and λ. When the problem is not normal this is not always the case. This is easily illustrated by the following example. Example 2.6 (A Non-Normal Problem) Consider the problem of minimizing J (x, y) = x2 + y 2 subject to the equality constraint G (x, y) = x2 − (y − 1)3 = 0. The gradients of J (·) and G(·) are given by  T ∇J (x, y) = 2x 2y

47

Chapter 2. Introduction and Preliminaries

2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 -1

-0.5

0

0.5

1

Figure 2.9: A Non-Normal Problem and ∇G (x, y) =



2x −3 (y − 1)2

T

,

respectively. It is clear from Figure 2.9 that the minimizer is given by  T  T ∗ z = x∗ y ∗ = 0 1 . The Lagrange Multiplier Theorem implies that there exist λ∗0 and λ∗1 such that λ∗0 ∇J (x∗ , y ∗ ) + λ∗1 ∇G (x∗ , y ∗ ) = 0

(2.24)

and |λ∗0 | + |λ∗1 | 6= 0. If λ∗0 6= 0 and if we define λ = λ∗1 /λ∗0 , then (2.24) is equivalent to ∇J (x∗ , y ∗ ) + λ∇G (x∗ , y ∗ ) = 0. (2.25)  T  T Using the fact that z ∗ = x∗ y ∗ = 0 1 is the minimizer,

48

Chapter 2. Introduction and Preliminaries

(2.24) and (2.25) imply that ∇J (0, 1) + λ∇G (0, 1) = 0, or equivalently, that  T  T  T 0 1 +λ 0 0 = 0 0 .

 T  T Clearly, 0 1 6= 0 0 so that λ∗0 = 0 is the only possible choice for λ∗0 and λ∗1 can be any non-zero value. The reason that the only choice for λ∗0 is λ∗0 = 0 is because the gradient of G (·) at T   T  T z ∗ = x∗ y ∗ = 0 1 is ∇G (0, 1) = 0 0 = 0. Proofs of the 2D Lagrange Multiplier Theorem

We shall present two proofs of the Lagrange Multiplier Theorem 2.3 in the special case where n = 2. The first proof is based on a variational method and can be easily modified to deal with isoperimetric problems in the calculus of variations discussed in Chapter 7 below. The second proof is geometric and relies on a separation result. Both proofs can be extended to a very general setting and yield a Lagrange Multiplier Theorem in abstract spaces. The first proof makes use of the Inverse Mapping Theorem from advanced calculus. The Inverse Mapping Theorem provides conditions that ensure a function will have an inverse (at least locally). Let T : O ⊂ R2 → R2 be a function from an open set O into the plane defined by T (α, β) = [p(α, β) q(α, β)]T where p(α, β) and q(α, β) are smooth functions. Assume that ˆ T ∈ O and [ˆ α β] ˆ = [p(ˆ ˆ q(ˆ ˆ = [ˆ T (ˆ α, β) α, β) α, β)] p qˆ]T . Roughly speaking, the Inverse Mapping Theorem implies that if ˆ T is non-singular (i.e. invertible), then the Jacobian matrix at [ˆ α β] ˆ T and an open neighborhood V there is a neighborhood U of [ˆ α β]

49

Chapter 2. Introduction and Preliminaries

ˆ = [ˆ of T (ˆ α, β) p qˆ]T so that T (α, β) restricted to U is a one-to-one and onto mapping from U to V with a continuous inverse function. ˆ T is given by Recall that the Jacobian matrix at [ˆ α β] " " # ˆ ˆ # ∂p(α, ˆ β) ∂p(α,β) ∂p(α,β) ∂p(α, ˆ β) ∂α ∂β ∂α ∂β ˆ = = JT (ˆ α, β) , ˆ ˆ ∂q(α,β) ∂α

∂q(α,β) ∂β

ˆT [α β]T =[α ˆ β]

∂q(α, ˆ β) ∂α

∂q(α, ˆ β) ∂β

(2.26) ˆ in one-to-one and onto (i.e. non-singular) if and only and JT (ˆ α, β) if the determinant " ˆ ˆ # ∂p(α, ˆ β) ∂p(α, ˆ β) ∂α ∂β ˆ = det det JT (ˆ α, β) 6= 0. ˆ ˆ ∂q(α, ˆ β) ∂α

∂q(α, ˆ β) ∂β

The following version of the Inverse Mapping Theorem follows from Theorem 41.8 on page 381 in Bartle’s book [15]. Theorem 2.4 (Inverse Function Theorem) Let T : O ⊂ R2 → R2 be a C 1 function from the open set O into the plane defined by T (α, β) = [p(α, β) q(α, β)]T where p(α, β) and q(α, β) are smooth functions. Assume that ˆ T ∈ O with [ˆ α β] ˆ = [p(ˆ ˆ q(ˆ ˆ = [ˆ T (ˆ α, β) α, β) α, β)] p qˆ]T

ˆ T , JT (ˆ ˆ is non-singular. Then and that the Jacobian at [ˆ α β] α, β) ˆ T and V of T (ˆ ˆ = there are open neighborhoods U of [ˆ α β] α, β) T [ˆ p qˆ] such that T (α, β) restricted to U is a one-to-one and onto mapping from U onto V. Moreover, if T (α, β) , T (α, β)|U

denotes the restriction of T (α, β) to U , then T (α, β) : U → V has a continuous inverse T −1 (p, q) : V → U belonging to C 1 , [α β]T = [α(p, q) β(p, q)]T = T −1 (p, q) ,

and J[T −1 (p, q)] = [JT (α(p, q), β(p, q))]−1 = [JT (T (p, q))]−1 for all [p q]T ∈ V.

50

Chapter 2. Introduction and Preliminaries

Proof of Theorem 2.3 for the 2D Case: Assume z ∗ = [ x∗ y ∗ ]T ∈ R2 minimizes J (x, y) subject to G (x, y) = g (x, y) = 0, where J : R2 −→ R1 and G : R2 −→ R1 . Since there is only one constraint (i.e. m = 1), the Jacobian T ∇g 1 (z ∗ ) ∇g 2 (z ∗ ) · · · ∇g m (z ∗ )  T = [∇g(z ∗ )]T = g x (x∗ , y ∗ ) g y (x∗ , y ∗ )

JG(z ∗ ) =



has maximal rank m = 1, if and only if ∇g(z ∗ ) 6= [0 0]T . First consider the case where z ∗ = [ x∗ y ∗ ]T satisfies     g x (x∗ , y ∗ ) 0 ∗ ∗ ∗ ∗ ∇G (x , y ) = ∇g (x , y ) = . = g y (x∗ , y ∗ ) 0 In this case set λ∗0 = 0 and λ∗1 = 1. It follows that |λ∗0 |+|λ∗1 | = 1 6= 0 and ∇z L(λ∗0 , λ∗ , z ∗ ) = λ∗0 ∇J (z ∗ ) + λ∗ ∇g(z ∗ ) = 0∇J (z ∗ ) + 1∇g(z ∗ ) = 0. Hence, λ∗0 ∇J (z ∗ ) + λ∗1 ∇g(z ∗ ) = 0 and the theorem is clearly true. Now consider the case where z ∗ = [ x∗ y ∗ ]T satisfies     g x (x∗ , y ∗ ) 0 ∗ ∗ ∗ ∗ ∇G (x , y ) = ∇g (x , y ) = . 6= ∗ ∗ g y (x , y ) 0

51

Chapter 2. Introduction and Preliminaries

In particular, at least one of the partial derivatives is not zero. Without loss of generality we assume g x (x∗ , y ∗ ) 6= 0 and define λ∗0 = g x (x∗ , y ∗ ) 6= 0 and λ∗1 = −J x (x∗ , y ∗ ) .

We now show that z ∗ = [x∗ y ∗ ]T satisfies

λ∗0 ∇J (z ∗ ) + λ∗1 ∇g(z ∗ ) = 0. Observe that λ∗0 ∇J (z ∗ ) + λ∗1 ∇g(z ∗ ) = g x (z ∗ ) ∇J (z ∗ ) − J x (z ∗ ) ∇g(z ∗ ), so that λ∗0 J x (z ∗ ) + λ∗1 g x (z ∗ ) = g x (z ∗ ) J x (z ∗ ) − J x (z ∗ ) g x (z ∗ ) = 0 (2.27) and λ∗0 J y (z ∗ ) + λ∗1 g y (z ∗ ) = g x (z ∗ ) J y (z ∗ ) − J x (z ∗ ) g y (z ∗ )   g x (z ∗ ) J x (z ∗ ) . = det g y (z ∗ ) J y (z ∗ ) Therefore, to establish that z ∗ = [ x∗ y ∗ ]T satisfies λ∗0 ∇J (z ∗ ) + λ∗1 ∇g(z ∗ ) = g x (z ∗ ) ∇J (z ∗ ) − J x (z ∗ ) ∇g(z ∗ ) = 0, we must show that     g x (z ∗ ) J x (z ∗ ) J x (z ∗ ) J y (z ∗ ) det = det = 0. (2.28) g y (z ∗ ) J y (z ∗ ) g x (z ∗ ) g y (z ∗ ) This is accomplished by applying the Inverse Mapping Theorem 2.4 above. Define T : R2 → R2 by T (α, β) = [(p(α, β) q(α, β)]T ,

(2.29)

52

Chapter 2. Introduction and Preliminaries

where p(α, β) = J (x∗ + α, y ∗ + β)

(2.30)

q(α, β) = G (x∗ + α, y ∗ + β) ,

(2.31)

and respectively. Observe that T (α, β) maps the open set R2 to R2 , is defined ˆ T = [0 0]T and by (2.29) - (2.31) with [ˆ α β] ˆ = T (0, 0) = [J (x∗ , y ∗ ) G(x∗ , y ∗ )]T = [J (z ∗ ) 0]T = [ˆ T (ˆ α, β) p 0]T . ˆ T = [0 0]T is given by The Jacobian of T (α, β) at [ˆ α β] #  "  ∂p(0,0) ∂p(0,0) J x (x∗ , y ∗ ) J y (x∗ , y ∗ ) ∂α ∂β = . ∂q(0,0) ∂q(0,0) g x (x∗ , y ∗ ) g y (x∗ , y ∗ ) ∂α ∂β Assume that (2.28) is not true. This assumption implies that # "   ∂p(0,0) ∂p(0,0) J x (x∗ , y ∗ ) J y (x∗ , y ∗ ) ∂α ∂β 6= 0 det ∂q(0,0) ∂q(0,0) = det g x (x∗ , y ∗ ) g y (x∗ , y ∗ ) ∂α ∂β ˆ T = [0 0]T so and the Jacobian of T (α, β) is non-singular at [ˆ α β] we may apply the Theorem n2.4. In particular, (see Figure 2.10) o p there is a neighborhood U = [α β]T : α2 + β 2 < γ of [0 0]T

and a neighborhood V of [J (x∗ , y ∗ ) 0]T = [ˆ p 0]T such that the restriction of T (α, β) to U , T (α, β) : U → V, has a continuous inverse T −1 (p, q) : V → U belonging to C 1 . Let [˜ p 0]T ∈ V be any point with p˜ < J (x∗ , y ∗ ) and let ˜ T = T −1 (˜ [˜ α β] p, 0) ∈ U. Observe that ˜ = p(˜ ˜ = p˜ < J (x∗ , y ∗ ) = J (z ∗ ) J (x∗ + α ˜ , y ∗ +β) α, β) and

˜ = q(˜ ˜ = 0. G(x∗ + α ˜ , y ∗ +β) α, β)

(2.32) (2.33)

53

Chapter 2. Introduction and Preliminaries





T

[  ]T

U  [  ]T :  2   2  

.



[ˆ ˆ ]  [0 0]

p  J ( x* , y * )

V

p

[  ]T  T 1 ( p ,0)

T

[ p( ,  ) q( ,  )]T [ p 0]T



T

q

T 1

T (0,0)  [J ( x* , y * ) 0]T  [ pˆ 0]T

Figure 2.10: 2D Lagrange Multiplier Proof ˜ T satisfies Thus, the vector z˜ = [x∗ + α ˜ y ∗ +β] ˜ = q(˜ ˜ =0 G(˜ z ) = G(x∗ + α ˜ , y ∗ +β) α, β) and ˜ = p(˜ ˜ = p˜ < J (x∗ , y ∗ ) = J (z ∗ ) J (˜ z ) = J (x∗ + α ˜ , y ∗ +β) α, β) which contradicts the assumption that z ∗ = [x∗ y ∗ ]T minimizes J (x, y) subject to G (x, y) = 0. Therefore, it follows that # "   ∂p(0,0) ∂p(0,0) J x (x∗ , y ∗ ) J y (x∗ , y ∗ ) ∂α ∂β 6= 0 det ∂q(0,0) ∂q(0,0) = det g x (x∗ , y ∗ ) g y (x∗ , y ∗ ) ∂α ∂β must be false and hence g x (x∗ , y ∗ ) J y (x∗ , y ∗ ) − J x (x∗ , y ∗ ) g y (x∗ , y ∗ )   J x (x∗ , y ∗ ) J y (x∗ , y ∗ ) = − det g x (x∗ , y ∗ ) g y (x∗ , y ∗ ) = 0.

(2.34)

54

Chapter 2. Introduction and Preliminaries

Consequently, (2.27) and (2.34) together imply that λ∗0 ∇J (z ∗ ) + λ∗1 ∇g(z ∗ ) = g x (z ∗ ) ∇J (z ∗ ) − J x (z ∗ ) ∇g(z ∗ ) = 0 and λ∗0 = g x (x∗ , y ∗ ) 6= 0 which completes the proof.  We also outline another proof that is geometric in nature. The details can be found in Hestenes’ book [102]. Second Proof of Theorem 2.3 for the 2D Case: Assume z ∗ = [x∗ y ∗ ]T ∈ R2 minimizes J (x, y) subject to G (x, y) = g (x, y) = 0. Observe that if ∇g(z ∗ ) =0 = [0 0]T , then λ∗0 = 0 and λ∗ = 1 produces ∇z L(λ∗0 , λ∗ , z ∗ ) = λ∗0 ∇J (z ∗ ) + λ∗ ∇g(z ∗ ) = 0 and hence we need only consider the case where ∇g(z ∗ ) 6= 0. Also, in the trivial case when ∇J (z ∗ ) = 0 one can set λ∗0 = 1 and λ∗ = 0 so that without loss of generality we can consider the case where ∇g(z ∗ ) 6= 0 and ∇J (z ∗ ) 6= 0. Under this assumption, the condition that λ∗0 ∇J (z ∗ ) + λ∗ ∇g(z ∗ ) = 0 ¯ such that with λ∗ 6= 0 is equivalent to the existence of a λ 0

∗ ¯ ∇J (z ∗ ) + λ∇g(z ) = 0,

¯ = (λ∗ /λ∗ ). In particular, the nonzero gradients ∇g(z ∗ ) where λ 0 and ∇J (z ∗ ) must be collinear. To establish this we define the level set L∗ by  L∗ , [x y]T : J (x, y) = m∗ = J (x∗ , y ∗ ) and the constraint set C by  C = [x y]T : G (x, y) = 0 ,

Chapter 2. Introduction and Preliminaries

55

J ( x* , y* )

B( z,  )  {z  [ x y ]T : z  z*   }

S   {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

[ x* y * ]T

S  {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

L *  {[ x y ]T : J ( x, y )  m*}

Figure 2.11: The Level Set respectively. Thus, in a sufficiently small ball B(z ∗ , δ) about z ∗ = [x∗ y ∗ ]T the level set separates the ball B(z ∗ , δ) into two sets  S− = [x y]T ∈ B(z ∗ , δ) : J (x, y) < m∗ = J (x∗ , y ∗ ) and

 S + = [x y]T ∈ B(z ∗ , δ) : J (x, y) > m∗ = J (x∗ , y ∗ ) ,

respectively (see Figure 2.11). If we assume that ∇g(z ∗ ) and ∇J (z ∗ ) are not collinear, then as shown in Figure 2.12 the support line to S− at z ∗ = [ x∗ y ∗ ]T (i.e. the line orthogonal to ∇J (z ∗ )) must cross the line orthogonal to ∇g(z ∗ ) at z ∗ = [x∗ y ∗ ]T . In particular, the constraint set C T ∗ ∗ must intersect S− = [x y] ∈ B(z , δ) : J (x, y) < m and there is a point [˜ x y˜]T ∈ C ∩ S− (see Figure 2.13). However,  C ∩ S− = [x y]T : G (x, y) = 0 and J (x, y) < m∗ = J (x∗ , y ∗ )

56

Chapter 2. Introduction and Preliminaries J ( x* , y* )

B( z,  )  {z  [ x y ]T : z  z*   }

S   {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

G ( x* , y* )

C  {[ x y ]T : G( x, y )  0} [ x* y * ]T

S  {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

L *  {[ x y ]T : J ( x, y )  m*}

Figure 2.12: Non-collinear Gradients so that [˜ x y˜]T ∈ C ∩ S− satisfies G (˜ x, y˜) = 0 and J (˜ x, y˜) < m∗ = J (x∗ , y ∗ ) . Therefore, [x∗ y ∗ ]T is not a local minimizer of J (x, y) on the set  C = [x y]T : G (x, y) = 0

and hence can not minimize J (x, y) subject to G (x, y) = 0. Consequently, ∇g(z ∗ ) and ∇J (z ∗ ) must be collinear and this completes the proof.  We note that this geometric proof depends on knowing that ∗ the line to the gradient ∇J (z ) is a support plane for  orthogonal T ∗ ∗ S− = [x y] ∈ B(z , δ) : J (x, y) < m at z ∗ = [ x∗ y ∗ ]T . This

Chapter 2. Introduction and Preliminaries

57

J ( x* , y* )

B( z,  )  {z  [ x y ]T : z  z*   }

S   {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

G ( x* , y* )

C  {[ x y ]T : G( x, y )  0} [ x* y * ]T

J ( x, y )  m*

[ x y ]T S  {[ x y ]T  B( z* ,  ) : J ( x, y )  m*}

L *  {[ x y ]T : J ( x, y )  m*}

Figure 2.13: Contradiction to Assumption “theme” will be important when we derive the simplest Maximum Principle for the time optimal control problem.

2.3

Function Spaces

Problems in the calculus of variations and optimal control involve finding functions that minimize some functional over a set of prescribed (admissible) functions. Therefore, we need to specify the precise space of functions that will be admissible. We start with the basic real valued piecewise continuous functions defined on an interval I = [t0 , t1 ]. Definition 2.7 Let I = [t0 , t1 ] be a closed interval. Then x : I → R is said to be piecewise continuous (PWC) on [t0 , t1 ] if: • The function x (·) is bounded on I.

58

Chapter 2. Introduction and Preliminaries
• The right-hand limit x tˆ+ = limt→tˆ+ [x(t)] exists (is finite) for all tˆ ∈ [t0 , t1 ).
• The left-hand limit x tˆ− = limt→tˆ− [x(t)] exists (is finite) for all tˆ ∈ (t0 , t1 ]. • There is a finite partition of [t0 , t1 ], t0 = tˆ0 < tˆ1 < tˆ2 < . . . < tˆp−1 < tˆp = t1 , such
that x (·) is continuous on each open subinterval tˆi−1 , tˆi .

Note that if x (·) is defined and bounded on [t0 , t1 ], tˆ ∈ [t0 , t1 ] and limt→tˆ± [x(t)] exists (even one-sided), then this limit must be finite. Thus, for piecewise continuous functions the one-sided limits exist and are finite at all points. Example 2.7 If x (·) is defined as in Figure 2.14, then x (·) is piecewise continuous. Note however that x (·) is not continuous. Example 2.8 The function shown in Figure 2.15 is not piecewise continuous since it is not bounded on [t0 , t1 ).

tˆ0  t0

tˆ1

tˆ2

tˆ3

tˆ4

t1  tˆ5

Figure 2.14: A Piecewise Continuous Function

59

Chapter 2. Introduction and Preliminaries

t 0  tˆ0

tˆ1

tˆ2

…….. tˆp 2

tˆp 1

tˆp  t1

Figure 2.15: A Non-Piecewise Continuous Function Example 2.9 Let tN = [0, 1] −→ R by x˜(t) =

1 , 2N −1

1 2N −1

,

for N = 1, 2, 3, · · · and define x˜ : 1 1 < t ≤ N −1 N 2 2

and set x˜(0) = 0 (see Figure 2.16). Note that x˜(·) is defined and bounded on [0, 1] and given any tˆ ∈ [0, 1] it follows that the limits limt→tˆ± [˜ x(t)] exist and are finite. Clearly, x˜(·) is bounded, but there is no finite partition of [0, 1] such that x˜(·) is continuous on this partition. Therefore, x˜(·) is not piecewise continuous since it has an infinite number of discontinuous jumps. Definition 2.8 A function x : [t0 , t1 ] → R is called piecewise smooth (PWS) on [t0 , t1 ] if (i) x(·) is continuous on [t0 , t1 ] and (ii) there exists a piecewise continuous function g(·) and a con-

60

Chapter 2. Introduction and Preliminaries

1

1 2 1 4

…

1 8

… 1

16

1 8

1 4

1 2

1

Figure 2.16: A Bounded Non-Piecewise Continuous Function stant c such that for all t ∈ [t0 , t1 ] x (t) = c +

Zt

g (s) ds.

t0

Note that if x : [t0 , t1 ] → R is PWS on [t0 , t1 ] and there are two PWC functions gi (·) and constants ci , i = 1, 2 such that for all t ∈ [t0 , t1 ] Zt x (t) = c1 + g1 (s) ds t0

and x (t) = c2 +

Zt

g2 (s) ds,

t0

then c1 = x (t0 ) = c2 . Also, Zt

t0

(g1 (s) − g2 (s))ds = 0

61

Chapter 2. Introduction and Preliminaries

for all t ∈ [t0 , t1 ]. It is straightforward to show that g1 (t) = g2 (t) except at a finite number of points, g1 (t+ ) = g2 (t+ ) and g1 (t− ) = g2 (t− ) for all t ∈ (t0 , t1 ). Hence, g1 (t) and g2 (t) can only have discontinuities at the same discrete points tˆi and between these points g1 (t) = g2 (t) if t 6= tˆi . If x : [t0 , t1 ] → R is PWS on [t0 , t1 ], then one could redefine x : [t0 , t1 ] → R at a finite number of points, but the resulting function would no longer be continuous. In particular, if xˆ(·) and x(·) are equal except at a finite number of points and xˆ(·) is continuous, then xˆ(·) is uniquely defined. Thus, piecewise smooth functions are continuous and x˙ (t) = g (t) exists except at a finite number of points. In addition, at the points tˆi , i = 1, 2, ..., p − 1 the right-hand and left-hand derivatives exist and are given by
d+ x(tˆi ) = x˙ + (tˆi ) = g tˆ+ = x( ˙ tˆ+ i i ) dt

t 0  tˆ0

tˆ1

tˆ2

…….. tˆp 2

tˆp 1

Figure 2.17: A Piecewise Smooth Function

(2.35)

tˆp  t1

62

Chapter 2. Introduction and Preliminaries

and




d− x(tˆi ) (2.36) = x˙ tˆ− = x˙ − tˆi = g tˆ− i , i dt respectively. Moreover, at the endpoints t0 and t1 it follows that

and

respectively.


d+ x(t0 ) = x˙ + (t0 ) = g t+ ˙ + 0 = x(t 0) dt

d− x(t1 ) ˙ t− = x˙ − (t1 ) = g t− 1 , 1 = x dt

(2.37)

(2.38)

Definition 2.9 If x (·) is
piecewise
smooth on [t0 , t1 ] and t0 < tˆ < t1 is such that x˙ tˆ+ 6= x˙ tˆ− , then we say that x (·) has
a corner at tˆ. From the remarks above, it follows that x˙ tˆ+ =


˙ = g tˆ− always limt→tˆ+ [x(t)] ˙ = g tˆ+ and x˙ tˆ− = limt→tˆ− [x(t)] exist and are finite since x˙ (t) = g (t) except at a finite number of points and g (·) is piecewise continuous. Before discussing specific spaces of functions, it is worthwhile to discuss what one means when we say that two functions are “equal” on a fixed interval [t0 , t1 ]. For example, the functions x(·) and z(·) plotted in Figure 2.18 below are not equal at each point. However, for all “practical purposes” (like integration) they are essentially the same functions. Their values are the same except at a finite number of points. If x : [t0 , t1 ] −→ R and z : [t0 , t1 ] −→ R are two functions such that x(t) = z(t) except at an finite number of points we shall write x(·) = z(·) e.f.

(2.39)

and, unless otherwise noted, we will rarely distinguish between x(·) and z(·). The functions x(t) and z(t) in Figure 2.18 are equal e.f. We denote the space of all real-valued piecewise continuous functions defined on [t0 , t1 ] by P W C(t0 , t1 ) and the space of all real-valued piecewise smooth functions defined on [t0 , t1 ] by P W S(t0 , t1 ).

Chapter 2. Introduction and Preliminaries

x(t )

63

z (t )

Figure 2.18: The Functions x(·) and z(·) are Equal e.f. Remark 2.10 Important Remark on Notation: Recall that each z(·) ∈ P W S(t0 , t1 ) is continuous. Therefore, if x(·) ∈ P W S(t0 , t1 ) and z(·) ∈ P W S(t0 , t1 ), then x(·) = z(·) e.f. if and only if x(t) = z(t) for all t ∈ [t0 , t1 ]. Also note that if x(·) ∈ P W C(t0 , t1 ), z(·) ∈ P W S(t0 , t1 ) and x(·) = z(·) e.f., then the left and right limits of x(·) are equal at all points of (t0 , t1 ). In particular, x(t+ ) = z(t+ ) = z(t− ) = x(t− ). Clearly, z(·) is the only continuous function satisfying x(·) = z(·) e.f. and since x(t+ ) = z(t+ ) = z(t− ) = x(t− ) we shall make no distinction between x(·) and its continuous representation z(·). Thus, for future reference we shall always use the equivalent continuous representation z(·) of x(·) when it exists and not distinguish between x(·) and z(·). If [t0 , +∞) is a semi-infinite interval, then we say that x(·) ∈ P W S(t0 , +∞) if x(·) ∈ P W S(t0 , T ) for all t0 < T < +∞. Thus, we denote the space of all real-valued piecewise continuous functions defined on [t0 , +∞) by P W C(t0 , +∞) and the space of all real-valued piecewise smooth functions defined on [t0 , +∞) by P W S(t0 , +∞).

64

2.3.1

Chapter 2. Introduction and Preliminaries

Distances between Functions

We shall need to consider what is meant by two functions “being close” in some sense. Although there are many possible definitions of “the distance” between x(·) and z(·) ∈ P W S(t0 , t1 ), we shall consider only two specific metrics. Definition 2.10 If x(·) and z(·) ∈ P W S(t0 , t1 ), then the d0 distance between two piecewise smooth functions x(·) and z(·) with domain [t0 , t1 ] is defined by d0 (x(·), z(·)) , sup {|x(t) − z(t)|}.

(2.40)

t0 ≤t≤t1

In this case we can define a norm on P W S(t0 , t1 ) by kx(·)k0 = sup {|x(t)|}, t0 ≤t≤t1

and note that d0 (x(·), z(·)) = kx(·) − z(·)k0 .



xˆ()



t0

t1

Figure 2.19: A U0 (ˆ x(·), δ)-neighborhood of xˆ(·)

Chapter 2. Introduction and Preliminaries

65

Definition 2.11 If xˆ(·) ∈ P W S(t0 , t1 ) and δ > 0, the U0 (ˆ x(·), δ)neighborhood (or Strong Neighborhood) of xˆ(·) is defined to be the open ball U0 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d0 (ˆ x(·), x(·)) < δ}. It is easy to visualize what such neighborhoods look like. Given xˆ(·) and δ > 0, the U0 (ˆ x(·), δ)-neighborhood of xˆ(·) is the set of all x(·) ∈ P W S(t0 , t1 ) with graphs in a tube of radius δ about the graph of xˆ(·) (see the figure below). As we see from above, x(·) and z(·) ∈ P W S(t0 , t1 ) are “close” in the d0 metric if their graphs are close. However, the derivatives can be greatly different. For example the two functions shown in the Figure 2.20 below have very different derivatives. In order to “fix this problem” we need a different metric. Recall that if x(·) and z(·) ∈ P W S(t0 , t1 ), then their derivatives x˙ (·) and z(·) ˙ are PWC on I. In particular, it follows that: • x˙ (·) and z(·) ˙ are bounded on I. • x˙ (t+ ) and z˙ (t+ ) exist (are finite) on [t0 , t1 ). • x˙ (t− ) and z˙ (t− ) exist (are finite) on (t0 , t1 ]. • There is a (finite) partition of [t0 , t1 ], say t0 = tˆ0 < tˆ1 < tˆ2 < . . . < tˆp−1 < tˆp = t1 , such that both x˙ (t) and z˙ (t) exist and are
continuous (and bounded) on each open subinterval tˆi−1 , tˆi . We can now define a weaker notion of distance.

Definition 2.12 The d1 distance between two piecewise smooth functions x(·) and z(·) with domain [t0 , t1 ] is defined by d1 (x(·), z(·)) = sup {|x(t) − z(t)| : t0 ≤ t ≤ t1 } (2.41)  + sup |x(t) ˙ − z(t)| ˙ : t0 ≤ t ≤ t1 , t 6= tˆi .

Definition 2.13 If xˆ(·) ∈ P W S(t0 , t1 ) and δ > 0, the U1 (ˆ x(·), δ)neighborhood (or weak neighborhood) of xˆ(·) is defined to be the open ball U1 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d1 (ˆ x(·), x(·)) < δ} .

66

Chapter 2. Introduction and Preliminaries



x()

xˆ ()



t0

t1

Figure 2.20: A Non-smooth Function in a U0 (ˆ x(·), δ)-neighborhood of xˆ(·) Remark 2.11 Note that the d1 distance between x(·) and z(·) is given by d1 (x(·), z(·)) = d0 (x(·), z(·)) + sup{|x(t) ˙ − z(t)| ˙ : t0 ≤ t ≤ t1 , t 6= tˆi }. (2.42) If x(·) and z(·) ∈ P W S(t0 , t1 ) and d1 (x(·), z(·)) = 0, then x(t) = z(t) for all t ∈ [t0 , t1 ] and x(t) ˙ = z(t) ˙ e.f. Also, since d0 (x(·), z(·)) ≤ d1 (x(·), z(·)), it follows that if d1 (x(·), z(·)) < δ, then d0 (x(·), z(·)) < δ. It is important to note that this inequality implies that U1 (ˆ x(·), δ) ⊂ U0 (ˆ x(·), δ) ⊂ P W S(t0 , t1 ),

(2.43)

so that the U1 (ˆ x(·), δ)-neighborhood U1 (ˆ x(·), δ) is smaller than the U0 (ˆ x(·), δ)-neighborhood U0 (ˆ x(·), δ) (see Figure 2.21).

Chapter 2. Introduction and Preliminaries

67

U 0 ( xˆ (),  )

xˆ ()

PWS (t0 , t1 )

U1 ( xˆ (),  )

Figure 2.21: A Comparison of U0 (ˆ x(·), δ) and U1 (ˆ x(·), δ) neighborhoods of xˆ(·) Two functions x(·) and z(·) are “close” in the d0 sense if their graphs are within δ of each other at all points. It is helpful (although not quite accurate) to think of two functions x(·) and z(·) as “close” in the d1 sense if their graphs and the graphs of x(·) ˙ and z(·) ˙ are within δ of each other except at a finite number of points. Remark 2.12 Another Remark on Notation: If x(·) is a piecewise smooth function, then x(t) ˙ is defined except perhaps at a ˆ finite number of points say, t0 = t0 < tˆ1 < tˆ2 < . . . < tˆp−1 < tˆp = t1 . However, at these points the left and right derivatives exist and are given by x( ˙ tˆ− ˙ tˆ+ i ) and x( i ), respectively. In order to keep notation at a minimum, when we use x(t) ˙ in any expression, we mean that this expression holds for all points t where x(t) ˙ exists. Also, recall ˆ ˆ that at corners t where x( ˙ t) do not exist, both x( ˙ tˆ− ) and x( ˙ tˆ+ ) exist and are finite. We shall treat conditions at corners as they arise in the development of the material.

68

2.3.2

Chapter 2. Introduction and Preliminaries

An Introduction to the First Variation

In order to extend Theorem 2.1 to infinite dimensional optimization problems characteristic of those in the calculus of variations and optimal control, we need to introduce the concept of a “Variation” of a functional. Although this is a very general concept, we focus on functionals defined on the space P W S(t0 , t1 ) and extend the definition in a later section. Assume that J : D(J) ⊆ P W S(t0 , t1 ) −→ R is a real-valued functional defined on a subset of P W S(t0 , t1 ) and let xˆ(·) ∈ D(J) be given. We say η(·) ∈ P W S(t0 , t1 ) is an admissible direction for xˆ(·) if there is an interval (−ˆ ε, +ˆ ε) with εˆ > 0 such that xˆ(·) + εη(·) ∈ D(J) ⊆ P W S(t0 , t1 ) for all ε ∈ (−ˆ ε, +ˆ ε). If η(·) ∈ P W S(t0 , t1 ) is an admissible direction, then F (ε) = J(ˆ x(·) + εη(·))

(2.44)

defines a real-valued function of the real variable ε on the interval (−ˆ ε, +ˆ ε). In this case we have the following definition. Definition 2.14 If xˆ(·) ∈ P W S(t0 , t1 ), η(·) ∈ P W S(t0 , t1 ) is an admissible direction for xˆ(·) and F (·) : (−ˆ ε, +ˆ ε) −→ R has a derivative at ε = 0, then the first variation of J(·) at xˆ(·) in the direction of η(·) is denoted by δJ(ˆ x(·); η(·)) and is defined by d d δJ(ˆ x(·); η(·)) = F (ε) [J(ˆ x(·) + εη(·))] . (2.45) = dε dε ε=0 ε=0

Likewise, we say η(·) ∈ P W S(t0 , t1 ) is a right admissible direction for xˆ(·) if there is an εˆ > 0 such that that xˆ(·) + εη(·) ∈ D(J) ⊆ P W S(t0 , t1 ) for all ε ∈ [0, +ˆ ε). If η(·) ∈ P W S(t0 , t1 ) is a right admissible direction, then F (ε) = J(ˆ x(·) + εη(·)) defines a real-valued function of the real variable ε on the interval [0, +ˆ ε). If the right-hand derivative d+ d+ F (ε) [J(ˆ x(·) + εη(·))] = (2.46) dε dε ε=0 ε=0

Chapter 2. Introduction and Preliminaries

69

exists, then d+ d+ δ J(ˆ x(·); η(·)) = = F (ε) [J(ˆ x(·) + εη(·))] dε dε ε=0 ε=0 +

is the right-hand first variation of J(·) at xˆ(·) in the direction of η(·). We will abuse notation and write δJ(ˆ x(·); η(·)) = δ + J(ˆ x(·); η(·)) even when η(·) ∈ P W S(t0 , t1 ) is only a right admissible direction for xˆ(·). If xˆ(·) ∈ P W S(t0 , t1 ), η(·) ∈ P W S(t0 , t1 ) is an admissible direction for xˆ(·) and F (·) : (−ˆ ε, +ˆ ε) −→ R has a second derivative at ε = 0, then the second variation of J(·) at xˆ(·) in the direction of η(·) is denoted by δ 2 J(ˆ x(·); η(·)) and is defined by d2 d2 2 = 2 [J(ˆ x(·) + εη(·))] . (2.47) δ J(ˆ x(·); η(·)) = 2 F (ε) dε dε ε=0 ε=0

2.4

Mathematical Formulation of Problems

In this section we return to some of the motivating problems outlined at the beginning of this chapter. Now that we have carefully defined specific classes of functions, it is possible to develop precise mathematical formulations of these problems.

2.4.1

The Brachistochrone Problem

We consider the problem where the particle starts at (0, 0) and slides down a wire to the point [a b]T and there is no friction as again illustrated in Figure 2.22. Let g be the acceleration due to gravity and m be the mass of the bead. The velocity of the bead when it is located at [x y]T is denoted by v (x). The kinetic energy of the bead is given by 1 T = mv 2 (x) 2

70

Chapter 2. Introduction and Preliminaries

, ●

a

x y(x)

, ●

b

Figure 2.22: Mathematical Formulation of The Brachistochrone Problem and the potential energy is given by V = mg (−y (x)) . The total energy of the bead is T +V and conservation of energy requires that T + V remains constant at all times. Since at x = 0, the kinetic energy is 0 (i.e. v (0) = 0) and the potential energy is 0 (i.e. y (0) = 0), we have that T = −V, or equivalently,

1 2 mv = mgy. 2 Now, we use some elementary calculus. The length of the path described by the graph of y (x) is given by Zx q 1 + [y ′ (τ )]2 dτ. s (x) = 0

71

Chapter 2. Introduction and Preliminaries The velocity is given by q 1 + [y ′ (x)]2

d dx d s (x) = s (x) = dt dx dt dt p However, v (x) = 2gy (x) and hence we obtain q 1 + [y ′ (x)]2 dt = p dx. 2gy (x) v (x) =

dx.

Integration gives the time of travel from [0 0]T to [x y(x)]T as the integral q Zx 1 + [y ′ (x)]2 p dx. t (x) = 2gy (x) 0

Hence, the time required for the bead to slide from [0 0]T to [a b]T is given by q Za 1 + [y ′ (x)]2 p dx. (2.48) J(y(·)) = t (a) = 2gy (x) 0

Thus, we have derived a formula (2.48) for the time it takes the bead to slide down a given curve y(·). Observe that the time (2.48) depends only on the function y(·) and is independent of the mass of the bead. We can now state the mathematical formulation of the brachistochrone problem. Among the set of all smooth functions y : [0, a] → R satisfying y (0) = 0, y (a) = b, find the function that minimizes q Za 1 + [y ′ (x)]2 p J(y(·)) = dx. 2gy (x)

(2.49)

0

Note that in the derivation of (2.49) we assumed that y (x) is smooth (i.e., that y (x) and y ′ (x) are continuous). Although

72

Chapter 2. Introduction and Preliminaries

we never explicitly assumed that y (x) ≥ 0, this is needed to make the terms under the square root sign non-negative. Also, observe that at x = 0, y(0) = 0, so that the integrand is singular. All of these “little details” are not so important for the brachistochrone problem, but can become important in other cases.

2.4.2

The Minimal Surface of Revolution Problem

Consider the problem of generating a surface by revolving a continuous curve about the x-axis. Suppose we are given two points P0 = [a0 b0 ]T and P1 = [a1 b1 ]T in the plane. These two points are joined by a curve which is the graph of the function y(·). We assume that y(·) is smooth and we generate a surface by rotating the curve about the x-axis (see Figure 2.23 below). What shape should y(·) be so that this surface of revolution has minimum surface area?

b1 b0

[a1 b1 ]T 

[a0 b0 ]T 

y ( x) a0 

x

a1 

Figure 2.23: Minimal Surface of Revolution Problem

Chapter 2. Introduction and Preliminaries

73

Again, we return to elementary calculus and recall the formula for surface area. If y(·) ≥ 0, then the surface area is given by S = 2π

Za1

a0

q y (x) 1 + [y ′ (x)]2 dx.

We are seeking, among the set of all smooth functions y : [a1 , a2 ] → R satisfying y (a0 ) = b0 , y (a1 ) = b1 , a function y ∗ (·) that minimizes J(y(·)) = S = 2π

Za1

a0

y (x)

q 1 + [y ′ (x)]2 dx.

(2.50)

At this point you should have noticed that the brachistochrone problem and the problem of minimal surface of revolution are “almost” identical in form. Both problems involve finding a function, with fixed endpoints, that minimizes an integral among all sufficiently smooth functions that pass through these same endpoints.

2.4.3

The River Crossing Problem

Consider the river crossing problem discussed in section 2.1.2 and as illustrated in Figure 2.2. If the path of the boat is represented by the graph of the function x(t), then it can be shown that the time it takes for the boat to cross the river is given by q Z1 c2 (1 + [x˙ (s)]2 ) − [v(s)]2 − v(s)x˙ (s) ds, (2.51) time = c2 − [v(s)]2 0

where c is the constant speed of the boat in still water and we assume that c2 − [v(t)]2 > 0. The point of departure is given by t1 = 0, x1 = 0 so that x (0) = 0. (2.52)

74

Chapter 2. Introduction and Preliminaries

Thus, the river crossing problem is to find a smooth function x∗ : [0, 1] −→ R that minimizes q Z1 c2 (1 + [x˙ (s)]2 ) − [v(s)]2 − v(s)x˙ (s) J(x(·)) = ds (2.53) c2 − [v(s)]2 0

among all smooth functions satisfying (2.52). Observe that unlike the brachistochrone problem and the problem of minimal surface of revolution, there is no specified condition at x = 1. Thus, this is a simple example of the classical free-endpoint problem.

2.4.4

The Rocket Sled Problem

Consider the rocket sled described in Section 2.1.4 above. We assume the sled is on a frictionless track, the sled is of mass m, it is controlled by two rocket engines thrusting in opposite directions and that the thrusting force of the rockets is bounded by a maximum of 1 so that |u(t)| ≤ 1. Let x (t) denote the displacement of the sled from the reference point R, at time t. Given that at time t = 0, the sled is at an initial position x0 with initial velocity v0 , we wish to find a thrust force action that will move the sled to position R and stop it there, and we want to accomplish this transfer in minimal time. Newton’s second law may be written as m¨ x(t) = u(t), where x(t) is the displacement from the base station at time t, m is the mass of the sled and u(t) is the thrust. We assume that at time t = 0 the initial data is given by the initial displacement x(0) = x0 , and initial velocity x(0) ˙ = v0 . The time optimal control problem is to find a control u∗ (t) that transfers the system from [x0 v]T to [0 0]T in minimal time, given that |u(t)| ≤ 1.

Chapter 2. Introduction and Preliminaries

75

To formulate the problem we write this as a first order system by defining x1 (t) = x(t) = position at time t, x2 (t) = x(t) ˙ = velocity at time t, with initial data x1 (0) = x1,0 , x0 , x2 (0) = x2,0 , v0 . The system becomes        d x1 (t) 0 x1 (t) 0 1 u(t), + = 1/m x2 (t) 0 0 dt x2 (t)

(2.54)

or x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0

(2.55)

where the matrices A and B are defined by     0 0 1 , and B = A= 1/m 0 0

 T  T respectively. Here, x0 = x1,0 x2,0 and x(t) = x1 (t) x2 (t) is the trajectory in the plane R2 . Given a control u(·) we let x(t; u(·)) denote the solution of the initial value problem (2.55) with control input u(·). T  to a state We say that a control u(t) steers x0 = x1,0 x2,0  T x1 = x1,1 x2,1 in time t1 if there is a solution x(t; u(·)) to (2.55) satisfying x(0; u(·)) = x0 (2.56) and x(t1 ; u(·)) = x1

(2.57)

for some finite time t1 > 0. The time optimal problem is to find the control u∗ (·) such that |u∗ (t)| ≤ 1,

76

Chapter 2. Introduction and Preliminaries

 T T  in to x1 = x1,1 x2,1 and u∗ (·) steers x0 = x1,0 x2,0 minimal time. Observe that here the final time t1 is not fixed and the cost function is very simple. In particular, the cost is the final time and this can be represented by the integral J(x(·), u(·)) = t1 =

Zt1

1ds.

(2.58)

0

This time optimal control problem can be “solved” by several means, including a Lagrange Multiplier method. Also, since this particular problem is rather simple, one can use a geometric analysis of the problem and the optimal controller can be synthesized by means of a switching locus. We will present this solution in Chapter 9 because the basic ideas are useful in understanding the maximum principle and how one might derive this necessary condition.

2.4.5

The Finite Element Method

In its purest form the finite element method is intimately connected to variational calculus, optimization theory and Galerkin approximation methods. The books by Strang and Fix (see [170, 171]) provide an excellent introduction to the finite element method. The references [44], [96], [98], and [179] provide examples where the finite element method has been applied to a variety of areas in science and engineering. The finite element method and its extensions are among the most powerful computational tools for solving complex ordinary and partial differential equations. We shall illustrate how one formulates the finite element approximation for the simple two-point boundary value problem described by (2.4)-(2.5) above. Later we shall see how the calculus of variations provides the mathematical framework and theory necessary to analyze the convergence of the method. Here we focus on the application of the finite element method to

77

Chapter 2. Introduction and Preliminaries

a simple example defined by the two-point boundary value problem − x¨(t) + x(t) = f (t),

0 < t < 1,

(2.59)

subject to the Dirichlet boundary conditions x(0) = 0,

x(1) = 0.

(2.60)

A strong (or classical) solution is a twice differentiable function x(·) that satisfies (2.59)-(2.60) at every value 0 ≤ t ≤ 1. We are interested in developing a numerical algorithm for approximating solutions. As noted above, in order to construct the finite element approximation one divides the interval (0, 1) into N subintervals (called elements) with nodes 0 = tˆ0 < tˆ1 < tˆ2 < . . . < tˆN −1 < tˆN < tˆN +1 = 1 and constructs the approximation xN (·) so that it is continuous on all of [0, 1] and linear between the nodes. Since continuous piecewise linear approximating functions xN (·) are not typically differentiable at the nodes, it is not possible to insert this approximation directly into equation (2.59). In particular, the piecewise smooth function xN (·) has only a piecewise continuous derivative x˙ N (·) and hence x¨N (·) does not exist. In order to deal with this lack of smoothness, we must define the concept of weak solutions. Before introducing this approximation, one derives a weak (or variational) form of this equation. If x(·) is a solution to (2.59) (2.60), then multiplying both sides of (2.59) by any function η(·) yields [−¨ x(t) + x(t)]η(t) = f (t)η(t), 0 < t < 1. (2.61) If one assumes that η(·) is piecewise continuous, then integrating both sides of (2.61) implies that Z 1 Z 1 f (t)η(t)dt, [−¨ x(t) + x(t)]η(t)dt = 0

0

or equivalently, Z Z 1 −¨ x(t)η(t)dt + 0

1

x(t)η(t)dt = 0

Z

1

f (t)η(t)dt, 0

(2.62)

78

Chapter 2. Introduction and Preliminaries

for any function η(·) ∈ P W C(0, 1). If η(·) is piecewise smooth, then one can use integration by parts on the first term in (2.62) which implies that Z 1 Z 1 Z 1 t=1 f (t)η(t)dt, x(t)η(t)dt = x(t) ˙ η(t)dt ˙ + − x(t)η(t)| ˙ t=0 + 0

0

0

(2.63) for any function η(·) ∈ P W S(0, 1). If in addition η(·) ∈ P W S(0, 1) satisfies the boundary conditions (2.60), then the boundary terms in (2.63) are zero so that Z 1 Z 1 Z 1 f (t)η(t)dt, (2.64) x(t)η(t)dt = x(t) ˙ η(t)dt ˙ + 0

0

0

for all η(·) ∈ P W S(0, 1) satisfying η(0) = 0,

η(1) = 0.

(2.65)

What we have shown is that if x(·) is a solution of the two-point boundary value problem (2.59)-(2.60), then x(·) satisfies (2.64) for all η(·) ∈ P W S(0, 1) that satisfy the Dirichlet boundary conditions (2.65). Observe that the equation (2.64) does not involve the second derivative of the function x(·) and equation (2.64) makes sense as long as x(·) is just piecewise smooth on (0, 1). Let P W S0 (0, 1) denote the space of all functions z(·) ∈ P W S(0, 1) satisfying z(0) = 0 and z(1) = 0. In particular, P W S0 (0, 1) = {z(·) ∈ P W S(0, 1) : z(0) = 0,

z(1) = 0} . (2.66) We now define what we mean by a weak solution of the boundary value problem (2.59)-(2.60). Definition 2.15 (Weak Solution) We say that the function x(·) is a weak solution of the two-point boundary value problem (2.59)-(2.60), if: (1) x(·) ∈ P W S0 (0, 1), (2) x(·) satisfies (2.64) for all η(·) ∈ P W S0 (0, 1). Observe that we have shown that a solution (in the strong sense) to the two-point boundary value problem (2.59)-(2.60) is

Chapter 2. Introduction and Preliminaries

79

always a weak solution. We shall show later that the Fundamental Lemma of the Calculus of Variations can be used to prove that a weak solution is also a strong solution. Therefore, for this one dimensional problem, weak and strong solutions are the same. Now we return to the issue of approximating this solution. We begin by dividing the interval [0, 1] into N + 1 subintervals (called elements) of length ∆ = 1/(N +1) with nodes 0 = tˆ0 < tˆ1 < tˆ2 < . . . < tˆN −1 < tˆN < tˆN +1 = 1, where for i = 0, 1, 2, . . . , N, N + 1, tˆi = i · ∆. For i = 0, 1, 2, . . . , N, N + 1, define the hat functions hi (·) on [0, 1] by  (tˆ1 − t)/∆, 0 ≤ t ≤ tˆ1 h0 (t) = , 0, tˆ1 ≤ t ≤ 1  (t − tˆN )/∆, tˆN ≤ t ≤ 1 hN +1 (t) = , (2.67) 0, 0 ≤ t ≤ tˆN   (t − tˆi−1 )/∆, tˆi−1 ≤ t ≤ tˆi hi (t) = (tˆ − t)/∆, tˆi ≤ t ≤ tˆi+1 , for i = 1, 2, . . . , N.  i+1 0, t∈ / (tˆi−1 , tˆi+1 )

Plots of these functions are shown in Figures 2.24, 2.25 and 2.26 below. These hat functions provide a basis for all continuous piecewise linear functions with (possible) corners at the nodes tˆ1 < tˆ2 < . . . < tˆN −1 < tˆN . Therefore, any continuous piecewise linear function yN (·) with corners only at these nodes can be written as yN (t) =

N +1 X

yi hi (t),

(2.68)

i=0

where the numbers yi determine the value of yN (t) at tˆi . In particular, yN (tˆi ) = yi and in order to form the function yN (·) one must provide the coefficients yi for i = 0, 1, 2, . . . , N, N + 1. Moreover, if yN (t) is assumed to satisfy the Dirichlet boundary conditions (2.60), then yN (tˆ0 ) = yN (0) = y0 = 0 and yN (tˆN +1 ) = yN (1) = yN +1 = 0 and yN (·) can be written as yN (t) =

N X i=1

yi hi (t).

(2.69)

80

Chapter 2. Introduction and Preliminaries 1 0.9 0.8 0.7

h0(t)

0.6 0.5 0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

0.8

1

Figure 2.24: The Hat Function h0 (·) If we seek an approximate continuous piecewise linear solution xN (·) to the weak form of the two-point value problem (2.59)1 0.9 0.8 0.7 0.6

h (t) i

0.5 0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

0.8

Figure 2.25: The Hat Function hi (·)

1

81

Chapter 2. Introduction and Preliminaries 1 0.9 0.8 0.7 0.6

h

0.5

(t)

N+1

0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

0.8

1

Figure 2.26: The Hat Function hN +1 (·) (2.60), then xN (·) must have the form xN (t) =

N X

xi hi (t)

(2.70)

i=1

and we need only to “compute” the values xi for i = 1, 2, . . . , N . N P We do this by substituting xN (t) = xi hi (t) into the weak form i=1

of the equation given by (2.64). In particular, xN (t) is assumed to satisfy Z 1 Z 1 Z 1 f (t)η(t)dt, (2.71) xN (t)η(t)dt = x˙ N (t)η(t)dt ˙ + 0

0

0

for all η(·) ∈ P W S0 (0, 1). N P Substituting xN (t) = xi hi (t) into the weak equation (2.71) i=1

82

Chapter 2. Introduction and Preliminaries

yields Z

N X

1 0

!

xi h˙ i (t) η(t)dt ˙ +

i=1

=

Z

1

Z

N X

1 0

!

xi hi (t) η(t)dt

i=1

f (t)η(t)dt, 0

for all η(·) ∈ P W S0 (0, 1). This equation can be written as N X

xi

i=1

Z =

1



h˙ i (t)η(t)dt ˙ +

0

Z

1

N X

xi

i=1

Z

1

hi (t)η(t)dt 0



f (t)η(t)dt,

(2.72)

0

for all η(·) ∈ P W S0 (0, 1). In order to use the variational equation to compute the coefficients xi for i = 1, 2, . . . , N , we note that each basis function hi (·) belongs to P W S0 (0, 1) for i = 1, 2, . . . , N . Therefore, setting η(·) = hj (t) ∈ P W S0 (0, 1) for each index j = 1, 2, . . . , N , yields N equations N X i=1

xi

Z

=

Z

1



h˙ i (t)h˙ j (t)dt +

0 1

N X i=1

xi

Z

1

hi (t)hj (t)dt 0

f (t)hj (t)dt.

(2.73)

0

Define the N × N mass matrix M = M N by M = M N = [mi,j ]i,j=1,2,...,N , where the entries mi,j of M N are given by the integrals mi,j =

Z



1

hj (t)hi (t)dt. 0

Likewise, define the N × N stiffness matrix K = K N by K = K N = [ki,j ]i,j=1,2,...,N ,

83

Chapter 2. Introduction and Preliminaries where the entries ki,j of K N are given by the integrals Z 1 ki,j = h˙ j (t)h˙ i (t)dt. 0

Finally, let f N be the N × 1 (column) vector defined by fN =



f1 f2 · · ·

fN

T

,

where entries fj of f N are given by the integrals Z 1 hj (t)f (t)dt. fj = 0

If x is the solution vector 

  x= 

x1 x2 .. . xN



  , 

of (2.73), then x satisfies the matrix system KN x + M N x = f N .

(2.74)

Defining LN = K N + M N yields the algebraic equation LN x = f N

(2.75)

which must be solved to find xi for i = 1, 2, . . . , N . This approach provides a mechanism for finding the (finite element) apN P proximation xN (t) = xi hi (t). In particular, one solves the mai=1  T trix equation (2.75) for x = x1 x2 · · · xN and then constructs the piecewise smooth finite element approximate function N P xN (t) = xi hi (t). i=1

84

Chapter 2. Introduction and Preliminaries

There are many issues that need to be addressed in order to prove that this method actually produces accurate approximations of the solution to the two-point boundary value problem (2.59) (2.60). What one might expect (hope) is that as N → +∞ the functions xN (·) converge to x(·) in some sense. We shall return to this issue later, but we note here that this process breaks down if the matrix LN is not invertible. In particular, one needs to prove that LN is non-singular. For this problem one can show that the mass and stiffness matrices are given by   4 1 0 0 ··· 0 0  1 4 1 0 ··· 0 0     0 1 4  1 · · · 0 0  ∆  0 0 1 4 ··· 0 0  MN =   6  .. .. .. .. .. ..  . .  . . . . . .  .    0 0 ··· ··· 1 4 1  0 0 ··· ··· 0 1 4

and



KN

   1   =  ∆   

2 −1 0 0 ··· 0 0 −1 2 −1 0 ··· 0 0 0 −1 2 −1 · · · 0 0 0 0 −1 2 ··· 0 0 .. .. .. .. . . .. .. . . . . . . . 0 0 · · · · · · −1 2 −1 0 0 ··· ··· 0 −1 2

respectively. Thus, LN = K N + M N is given by  4∆

LN

     =    

6 ∆ 6

+ −

2 ∆ 1 ∆

0 0 .. . 0 0

∆ 6 4∆ 6 ∆ 6

− + −

1 ∆ 2 ∆ 1 ∆

0 .. . 0 0

0 ∆ 6 4∆ 6 ∆ 6

− + −

1 ∆ 2 ∆ 1 ∆

.. . ··· ···

··· ··· ··· ··· .. .

0 0 ∆ 6 4∆ 6

− +

1 ∆ 2 ∆

.. . ··· ···

∆ 6

−

1 ∆

0



     ,     0 0 0 0 .. .

4∆ 6 ∆ 6

+ −

2 ∆ 1 ∆

∆ 6 4∆ 6

 0 0  0  0  .. .  1 −∆ 2 +∆

and one can show that LN is a symmetric positive definite matrix. Thus, LN is non-singular (see [92]).

85

Chapter 2. Introduction and Preliminaries 0 -0.05

x3(t)

-0.1 -0.15 -0.2 -0.25 -0.3 -0.35 0

x(t) = t(t-1)

0.2

0.4

t

0.6

0.8

1

Figure 2.27: N = 3 Finite Element Approximation Consider the case where the function f (·) is given by f (t) = t2 − t − 2. It is easy to show (by direct substitution) that x(t) = t(t − 1) is the solution to the boundary value problem (2.59)-(2.60). In Figures 2.27 and 2.28 below, we compare the finite element solutions for N = 3 and N = 7. Clearly, the approximating solutions converge very rapidly. Although it might seem like the problem discussed above has nothing to do with classical problems such as the brachistochrone and minimal surface area problems, this is not true. In fact, we shall show later that finding solutions to the two-point boundary problem (2.59) - (2.60) is equivalent to solving the following simplest problem in the calculus of variations. Find x∗ (·) ∈ P W S(0, 1) to minimize the integral Z o 2 1 1 n 2 J(x(·)) = x(s)] ˙ + [x(s) − 2x(s)f (s) ds, (2.76) 2 0

86

Chapter 2. Introduction and Preliminaries 0 -0.05 -0.1

x7(t)

-0.15 -0.2 -0.25 -0.3 -0.35 0

x(t) = t(t-1)

0.2

0.4

t

0.6

0.8

1

Figure 2.28: N = 7 Finite Element Approximation subject to the end conditions x(0) = 0 and x(1) = 0.

(2.77)

Again, this is a problem of finding a function passing through two given points that minimize an integral where f (t, x, u) is given by f (t, x, u) =

1 2 u + x2 − 2xf (t) . 2

In fact, the three problems above are examples of the so called Simplest Problem in the Calculus of Variations (SPCV) to be discussed in Chapter 3.

2.5

Problem Set for Chapter 2

Problem 2.1 Let f : [0, 2] → R be defined by f (x) = xe−x . Find all local minimizers for f (·) on [0, 2]. Problem 2.2 Let f : (0, 2] → R be defined by f (x) = xe−x . Find all local minimizers for f (·) on (0, 2].

87

Chapter 2. Introduction and Preliminaries

Problem 2.3 Let f : [−1, +1] → R be defined by f (x) = |x|. Find all local minimizers for f (·) on [−1, +1]. 2

Problem 2.4 Let f : [−1, +1] → R be defined by f (x) = x3 e−x . Find all local minimizers for f (·) on [−1, +1]. 2

Problem 2.5 Let f : [−2, +2] → R be defined by f (x) = x3 e−x . Find all local minimizers for f (·) on [−2, +2]. Problem 2.6 Let x : [−1, 1] → R be defined by  |t| , t 6= 0, x(t) = 1, t = 0. Prove that z(t) = |t| is the only continuous function satisfying z(t) = x(t) except at a finite number of points. Problem 2.7 Let x : [−1, 1] → R and z : [−1, 1] → R be defined by x(t) = |t| and z(t) = t2 , respectively. Compute d0 (x(·); z(·)) and d1 (x(·); z(·)). Problem 2.8 Let xN : [−π, π] → R and z : [−π, π] → R be defined by xN (t) = N1 sin(N t) and z(t) = 0, respectively. Compute d0 (xN (·); z(·)) and d1 (xN (·); z(·)). Problem 2.9 Consider the functional J(x(·))

=

Rπ 0

{[x˙ (s)]2

−[x (s)]2 }ds and let xˆ(·) = sin(t). Compute the first and second variation of J(·) at xˆ(·) in the direction of η(·) ∈ P W S(0, π). Problem 2.10 Consider the functional J(x(·)) =

R2π [x (s)]2 ds and 0

let xˆ(·) = |t − 1|. Compute the first and second variation of J(·) at xˆ(·) in the direction of η(·) ∈ P W C(0, π). Problem 2.11 Let J : P W S(0, 1) −→ R be defined by Z 1 p 2 ds x(s) 1 + [x(s)] ˙ J(x(·)) = 0

88

Chapter 2. Introduction and Preliminaries

and let xˆ(s) = sin(s). Show that any η(·) ∈ P W S(0, 1) is an admissible direction for xˆ(s) = sin(s). Compute the first and second variation of J(·) at xˆ(·) in the direction of η(·). You may assume it is allowable to pull the partial derivative with respect to ε inside the integral. Problem 2.12 Let J : P W S(0, 1) −→ R be defined by Z t1 f (s, x(s), x(s))ds ˙ J(x(·)) = t0

2

where f (t, x, u) is a C function. Given any xˆ(·) ∈ P W S(t0 , t1 ), show that η(·) is an admissible direction for all η(·) ∈ P W S(0, 1). Compute the first and second variation of J(·) at xˆ(·) in the direction of η(·). You may assume it is allowable to pull the partial derivative with respect to ε inside the integral. Problem 2.13 Show that for the river crossing problem that the time to cross is given by the integral q Z1 c2 (1 + [x˙ (s)]2 ) − [v(s)]2 − v(s)x˙ (s) ds. J(x(·)) = c2 − [v(s)]2 0

Problem 2.14 Use the Lagrange Multiplier Theorem to find all possible minimizers of J(x, y, x) = x2 + y 2 + z 2 subject to the constraints g1 (x, y, z) = x2 + y 2 − 4 = 0, g2 (x, y, z) = 6x + 3y + 2z − 6 = 0. Problem 2.15 Use the Lagrange Multiplier Theorem to find all possible minimizers of J(x, y, x) = x2 + y 2 + z 2 subject to the constraints g1 (x, y, z) = x2 + y 2 + z 2 − 1 = 0, g2 (x, y, z) = x + y + z − 1 = 0.

Chapter 2. Introduction and Preliminaries

89

Advanced Problems Problem 2.16 Given any x(·) ∈ P W S(t0 , t1 ), let kx(·)k0 = sup {|x(t)| : t0 ≤ t ≤ t1 }. Prove that k·k0 is a norm on P W S(t0 , t1 ). Problem 2.17 Use the finite element method to solve the two point boundary problem −¨ x(t) + 4x(t) = t, x(0) = x(1) = 0.

Chapter 3 The Simplest Problem in the Calculus of Variations We turn now to the Simplest Problem in the Calculus of Variations (SPCV) with both endpoints fixed and focus on obtaining (first order) necessary conditions. We begin with global minimizers and then return to the problem for local minimizers later.

3.1

The Mathematical Formulation of the SPCV

We shall make the standard assumption that f = f (t, x, u) is a C 2 function of three real variables t, x and u. In particular, f : D(f ) ⊆ R3 → R is real-valued, continuous and all the partial derivatives ft (t, x, u), fx (t, x, u), fu (t, x, u), ftx (t, x, u), ftu (t, x, u), fxu (t, x, u), and ftt (t, x, u), fxx (t, x, u), fuu (t, x, u), exist and are continuous on the domain D(f ). Recall that by Schwarz’s Theorem (see page 369 in [15]) this implies all the mixed

91

92

Chapter 3. The Simplest Problem

derivatives of 2 or less are equal. Thus, we have ∂2 ∂2 ∂2 ∂2 f (t, x, u) = f (t, x, u), f (t, x, u) = f (t, x, u) ∂t∂x ∂x∂t ∂t∂u ∂u∂t and ∂2 ∂2 f (t, x, u) = f (t, x, u). ∂x∂u ∂u∂x In most cases the domain D(f ) will have the form D(f ) = [t0 , t1 ] × R2 . Although, there are problems where f (t, x, u) will not be defined for all [x u]T ∈ R2 and one should be aware of these cases, the main ideas discussed in this book are not greatly impacted by this detail. Remark 3.1 It is important to comment on the “notation” f (t, x, u) used in this book. Using the symbols t, x and u for the independent variables is not standard in many classical books on the calculus of variations. However, the choice of the symbol u as the third independent variable is rather standard in treatments of modern control theory. Consequently, using the same notation for both classical and modern problems avoids having to “switch” notation as the material moves from variational theory to control. Many older texts on the calculus of variations use notation such as f (t, x, x) ˙ or f (x, y, y) ˙ so that one sees terms of the form ∂ ∂2 ∂ f (t, x, x), ˙ f (x, y, y) ˙ and f (x, y, y). ˙ ∂ x˙ ∂ y˙ ∂ y∂x ˙ This notation often leads to a misunderstanding (especially by students new to these subjects) of the difference between the independent variable x˙ and the derivative of the function x(·). Ewing’s book [77] was one of the first texts on the calculus of variations to use the notation f (x, y, r) and Leitmann [120] uses a similar notation f (t, x, r) to avoid this issue. Using t, x and u for the independent variables is mathematically more pleasing, leads to fewer mistakes and is consistent with modern notation. Note that if x : [t0 , t1 ] → R is PWS on [t0 , t1 ], then the function g : [t0 , t1 ] → R defined by g(s) = f (s, x(s), x(s)) ˙

Chapter 3. The Simplest Problem

93

is PWC on [t0 , t1 ]. Therefore, the integral Zt1

f (s, x(s), x˙ (s)) ds

t0

exists and is finite. Let X = P W S(t0 , t1 ) denote the space of all real-valued piecewise smooth functions defined on [t0 , t1 ]. For each PWS function x : [t0 , t1 ] → R, define the functional (a “function of a function”) by J(x(·)) =

Zt1

f (s, x(s), x˙ (s)) ds.

(3.1)

t0

Assume that the points [t0 x0 ]T ∈ R1 × R1 and [t1 x1 ]T ∈ R1 × R1 are given and define the set of PWS functions Θ by Θ = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 , x (t1 ) = x1 } .

(3.2)

Observe that J : X → R is a real valued function on X = P W S(t0 , t1 ). The Simplest Problem in the Calculus of Variations (SPCV) is the problem of minimizing J(·) on Θ. In particular, the goal is to find x∗ (·) ∈ Θ such that J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s)) ds,

t0

=

t0

for all x (·) ∈ Θ. In the brachistochrone problem, q 1 + [u]2 , f (t, x, u) = √ 2gx

(3.3)

94

Chapter 3. The Simplest Problem

x1 (t ) x1

x0

x * (t )

x2 (t )

t1

t0

Figure 3.1: The Simplest Problem in the Calculus of Variations

so that J(x(·)) =

Za

q

0

Z 1 + [x˙ (s)]2 p ds = f (s, x(s), x˙ (s)) ds. 2gx (s) a

0

For the problem of finding the surface of revolution of minimum area, q f (t, x, u) = 2πx 1 + [u]2 ,

so that

J(x(·)) = S = 2π

Za1

a0

x (s)

q

2

1 + [x˙ (s)] ds =

Za1

f (s, x(s), x˙ (s)) ds.

a0

The Simplest Problem in the Calculus of Variations will be the focus of the next few chapters. We will move on to other (more general) problems in later chapters. The basic goal is

Chapter 3. The Simplest Problem

95

the development of the classical necessary conditions for (local) minimizers. In the following chapters we shall derive necessary conditions for local minimizers. However, we start with global minimizers. Before we derive the first necessary condition, we develop a few fundamental lemmas that provide the backbone to much of variational theory. In addition, we review some basic results on differentiation.

3.2

The Fundamental Lemma of the Calculus of Variations

The Fundamental Lemma of the Calculus of Variations (FLCV) is also known as the Du Bois-Reymond Lemma. To set the stage for the lemma, we need some additional notation. Let V0 denote the set V0 = P W S0 (t0 , t1 ) = {η(·) ∈ P W S(t0 , t1 ) : η (t0 ) = η (t1 ) = 0} . (3.4) Observe that V0 ⊆ P W S(t0 , t1 ) is the set of all PWS functions on (t0 , t1 ) that vanish at both ends (see Figure 3.2 below). The space V0 defined by (3.4) is called the space of admissible variations for the fixed endpoint problem. Lemma 3.1 (Fundamental Lemma of the Calculus of Variations) Part (A): If α(·) is piecewise continuous on [t0 , t1 ] and Zt1

α (s) η˙ (s) ds = 0

(3.5)

t0

for all η(·) ∈ V0 , then α (t) is constant on [t0 , t1 ] except at a finite number of points. In particular, there is a constant c and a finite

96

Chapter 3. The Simplest Problem

1 (t )

2 ( t )

3 ( t )

t0

t1

Figure 3.2: Admissible Variations set of points t0 < tˆ1 < tˆ2 < ... < tˆp < t1 , in (t0 , t1 ) such that for each t ∈ (t0 , t1 ) with t 6= tˆi , i = 1, 2, ...., p α (t) = c. The converse is also true. Part (B): If α(·) and β(·) are piecewise continuous functions on [t0 , t1 ] and Zt1

[α (s) η (s) + β (s) η˙ (s)] ds = 0

(3.6)

t0

for all η(·) ∈ V0 , then there is a constant c such that β (t) = c +

Zt

α (s) ds

t0

except at a finite number of points. In particular, there is a finite set of points t0 < tˆ1 < tˆ2 < ... < tˆq < t1 , in (t0 , t1 ) such that for

97

Chapter 3. The Simplest Problem each t ∈ (t0 , t1 ) with t 6= tˆi , i = 1, 2, ...., q, β˙ (t) exists and β˙ (t) = α (t) . The converse is also true.

Proof of Part (A): Assume (3.5) holds for all η(·) ∈ V0 . Let 1 c= t1 − t0

Zt1

α (s) ds,

t0

and define the function ηˆ(·) by ηˆ (t) =

Zt

[α (s) − c] ds.

t0

Observe that ηˆ(·) is PWS since it is the integral of a piecewise continuous function. Also, ηˆ (t0 ) =

Zt0

[α (s) − c] ds = 0,

(3.7)

t0

and ηˆ (t1 ) =

[α (s) − c] ds =

Zt1

α (s) ds − c[t1 − t0 ] = 0.

α (s) ds −

t0

t0

=

Zt1

Zt1

Zt1

cds

t0

(3.8)

t0

Therefore, it follows from (3.7) and (3.8) that ηˆ(·) ∈ V0 . Hence, the assumption implies that Zt1

t0

α (s) [

d ηˆ (s)]ds = 0. ds

98

Chapter 3. The Simplest Problem

However, since c is constant and ηˆ (t0 ) = ηˆ (t1 ) = 0, it follows that Zt1

c[

d ηˆ (s)]ds = c [ˆ η (t1 ) − ηˆ (t0 )] = 0. ds

t0

Therefore, we have that Zt1

d [α (s) − c] [ ηˆ (s)]ds = ds

Zt1

d α (s) [ ηˆ (s)]ds− ds

t0

t0

and since [

Zt1

c[

d ηˆ (s)]ds = 0 ds

t0

d ηˆ (s)] = [α (t) − c] , ds

it follows that Zt1

2

[α (s) − c] ds =

Zt1

[α (s) − c] [

d ηˆ (s)]ds = 0. ds

(3.9)

t0

t0

The assumption that α (·) is piecewise continuous implies that there is a partition of [t0 , t1 ], say t0 = tˆ0 < tˆ1 < tˆ2 < ... < tˆp < tˆp+1 = t1 , such that α (·) is continuous on each subinterval (tˆi , tˆi+1 ). On the other hand (3.9) implies that Ztˆi+1 [α (s) − c]2 ds = 0 tˆi

on these subintervals. Consequently, for s ∈ (tˆi , tˆi+1 ) it follows that [α (s) − c]2 = 0 and hence on each subinterval (tˆi , tˆi+1 ) α (s) = c, which establishes the result.

Chapter 3. The Simplest Problem

99

For the converse, simply note that if η(·) ∈ V0 and α (t) = c. for each t ∈ (t0 , t1 ) with t 6= tˆi , i = 1, 2, ...., p, then Zt1

t0

p

X d α (s) [ η (s)]ds = ds i=0

Ztˆi+1 d α (s) [ η (s)]ds ds tˆi

tˆ p Zi+1 X d c[ η (s)]ds = ds i=0 tˆi

tˆ p Zi+1 X d η (s) ds] = c[ ds i=0 tˆi

=c

Zt1

[

d η (s)]ds = c [η (t1 ) − η (t0 )] = 0, ds

t0

and this proves the converse of Part (A). Proof of Part (B): Assume (3.6) holds for all for all η(·) ∈ V0 . Let g (·) be defined by g (t) =

Zt

α (s) ds,

t0

and observe that g (·) is piecewise smooth with d g (t) = g(t) ˙ = α (t) dt except at a finite number of points in (t0 , t1 ). Integrate (3.6) by

100

Chapter 3. The Simplest Problem

parts, to obtain Zt1 

t0

 d α (s) η (s) + β (s) [ η (s)] ds ds

=

=

Zt1 

t0

 d d [ g (s)]η (s) + β (s) [ η (s)] ds ds ds

Zt1 

  Zt1  d d [ g (s)]η (s) ds + β (s) [ η (s)] ds ds ds

t0

t0

= g (t) η

t=t1 (t)|t=t 0

−

Zt1 

t0

=

Zt1 

t0

=

Zt1

  Zt1  d d g (s) [ η (s)] ds + β (s) [ η (s)] ds ds ds t0

 d −g (s) [ η (s)] ds + ds

[−g (s) + β (s)] [

Zt1 

t0

 d β (s) [ η (s)] ds ds

d η (s)]ds. ds

t0

Therefore, it follows that Zt1

t0

[−g (s) + β (s)] η˙ (s) ds =

Zt1

[α (s) η (s) + β (s) η˙ (s)] ds = 0,

t0

for all η(·) ∈ V0 . Applying Part (A) yields the existence of a constant c such that −g (t) + β (t) = c,

except at a finite number of points. Therefore, β (t) = c + g(t) = c +

Zt

α (s) ds

(3.10)

t0

except at a finite number of points. Since α(·) is piecewise continuous, there is a partition of [t0 , t1 ], say t0 = tˆ0 < tˆ1 < tˆ2
0 for all (t, x, u) ∈ D(f ), then the integrand f (t, x, u) is said to be regular and the SPCV is said to be a regular problem.
If tˆ and xˆ are fixed, then the function ρ(u) , f tˆ, xˆ, u is called the figurative (or indicatrix) at (tˆ, xˆ). Note that if f (t, x, u) is a regular integrand, then the figurative is a strictly convex
d2 ˆ function since du2 ρ(u) = fuu t, xˆ, u > 0. This implies that
d ˆ, xˆ, u is a strictly increasing function. This obsert ρ(u) = f u du vation leads to the following result. Theorem 3.5 If the integrand f (t, x, u) is regular, then all extremals are of class C 2 . In particular, extremals for a regular problem cannot have corners. Proof: Assume that f (t, x, u) is regular, and suppose that x(·) is an extremal with a corner at tˆ, i.e.

x˙ tˆ+ 6= x˙ tˆ− . Without loss of generality we may assume that

u1 , x˙ tˆ+ < x˙ tˆ− , u2 .

The derivative of the figurative ρ′ (u) , fu (tˆ, x(tˆ), u) at (tˆ, x(tˆ)) is strictly increasing so that d d ρ(u1 ) < ρ(u2 ) = fu (tˆ, x(tˆ), u2 ). fu (tˆ, x(tˆ), u1 ) = du du

(3.26)

However, the corner condition (3.25) implies that

fu (tˆ, x(tˆ), u1 ) = fu (tˆ, x(tˆ), x˙ tˆ+ ) = fu (tˆ, x(tˆ), x˙ tˆ− ) = fu (tˆ, x(tˆ), u2 ), which contradicts (3.26). Therefore, x(·) cannot have a corner at tˆ. Since x(·) ˙ has no corners, it follows that x(·) ˙ ∈ C 1 (t0 , t1 ) and 2 hence x(·) ∈ C (t0 , t1 ) and this completes the proof.  Actually, Hilbert proved a much stronger result. We state his theorem below.

111

Chapter 3. The Simplest Problem

Theorem 3.6 (Hilbert’s Differentiability Theorem) If x(·) ∈ P W S(t0 , t1 ) is an extremal, tˆ is not a corner of x(·), and fuu (tˆ, x(tˆ), x( ˙ tˆ)) 6= 0, then there exists a δ > 0 such that x(·) has a continuous second derivative for all t ∈ (tˆ − δ, tˆ + δ) and [fuu (t, x(t), x˙ (t))] · x¨ (t) = − [fut (t, x(t), x(t))] ˙ − [fux (t, x(t), x˙ (t))] · x˙ (t) + [fx (t, x(t), x˙ (t))] .

(3.27)

If in addition, f (t, x, u) is of class C p , p ≥ 2, then any extremal x(·) is also of class C p on (tˆ − δ, tˆ + δ). Observe that Theorem 3.6 implies that, for regular integrands, all extremals x(·) have continuous second derivatives x¨(·) since f is assumed to be of class C 2 . Therefore, we may differentiate (3.23) by applying the chain rule to obtain d [fu (t, x(t), x˙ (t))] = [fut (t, x(t), x˙ (t))] dt + [fux (t, x(t), x˙ (t))] · x˙ (t) + [fuu (t, x(t), x˙ (t))] · x¨ (t) . Hence, the Euler Differential Equation (3.23) becomes the second order differential equation [fuu (t, x(t), x˙ (t))] · x¨ (t) = [fx (t, x(t), x˙ (t))] − [fut (t, x(t), x(t))] ˙ (3.28) − [fux (t, x(t), x˙ (t))] · x˙ (t) . Observe that since fuu (t, x (t) , x˙ (t)) > 0, this differential equation may be written as x ¨ (t) =

[fx (t, x(t), x˙ (t))] − [fut (t, x(t), x˙ (t))] − [fux (t, x(t), x˙ (t))] · x˙ (t) . [fuu (t, x(t), x˙ (t))] (3.29)

Note that Hilbert’s Theorem is valid even if the problem is not regular. The key is that along an extremal x(·), the function ρ(t) = fuu (t, x(t), x(t)) ˙ 6= 0 for all points t where x(t) ˙ exists. Such extremals have a special name leading to the following definition.

112

Chapter 3. The Simplest Problem

Definition 3.3 If x(·) ∈ P W S(t0 , t1 ) is an extremal, then x(·) is called a non-singular extremal if fuu (t, x(t), x(t)) ˙ 6= 0 for all t ∈ (t0 , t1 ) where x(t) ˙ exists. If x(·) ∈ P W S(t0 , t1 ) is an extremal, then x(·) is called a regular extremal if fuu (t, x(t), x(t)) ˙ > 0 for all t ∈ (t0 , t1 ) where x(t) ˙ exists.

3.3.1

Examples

We shall go through a few examples to illustrate the application of the necessary condition. It is important to note that at this point we can say very little about the existence of a minimum except in some special cases. The following three examples illustrate that the interval [t0 , t1 ] plays an important role in SPCV. Example 3.1 Find a PWS function x∗ (·) satisfying x (0) = 0, x (π/2) = 1 and such that x∗ (·) minimizes J(x(·)) =

Zπ/2 0


1 [x˙ (s)]2 − [x (s)]2 ds. 2

We note that t0 = 0, t1 = π/2, x0 = 0, and x1 = 1. The integrand f (t, x, u) is given by 1 f (t, x, u) = ([u]2 − [x]2 ) 2 and hence, fx (t, x, u) = −x fu (t, x, u) = +u fuu (t, x, u) = +1 > 0. We see that f (t, x, u) is regular and hence the minimizer cannot have corners. Euler’s Equation d [fu (t, x∗ (t), x˙ ∗ (t))] = [fx (t, x∗ (t), x˙ ∗ (t))] dt becomes

d ∗ [x˙ (t)] = [−x∗ (t)] , dt

Chapter 3. The Simplest Problem

113

or equivalently, x¨∗ (t) + x∗ (t) = 0. The general solution is x∗ (t) = α cos(t) + β sin(t), and applying the boundary conditions 0 = x∗ (0) = α cos(0) + β sin(0) = α, 1 = x∗ (π/2) = α cos(π/2) + β sin(π/2) = β, it follows that x∗ (t) = sin(t) is the only solution to the Euler Necessary Condition as given in Theorem 3.3. Observe that we do not know if x∗ (t) = sin(t) minimizes J(·). However, if there is a minimizer, then x∗ (t) = sin(t) must be the minimizer since it is the only function satisfying the necessary condition. Example 3.2 Find a PWS function x∗ (·) satisfying x (0) = 0, x (3π/2) = 0 and such that x∗ (·) minimizes J(x(·)) =

3π/2 Z 0


1 [x˙ (s)]2 − [x (s)]2 ds. 2

Observe that the integrand f (t, x, u) is the same as in Example 3.1 so that the Euler Equation is the same x¨∗ (t) + x∗ (t) = 0 and has the general solution x∗ (t) = α cos(t) + β sin(t). The boundary conditions 0 = x∗ (0) = α cos(0) + β sin(0) = α, 0 = x∗ (π/2) = α cos(3π/2) + β sin(3π/2) = −β,

114

Chapter 3. The Simplest Problem

imply that x∗ (t) ≡ 0

is the only solution to the Euler Necessary Condition as given in Theorem 3.3. Again, at this point we do not know if x∗ (t) ≡ 0 minimizes J(·). However, if there is a minimizer, then x∗ (t) ≡ 0 must be the minimizer since it is the only function satisfying the necessary condition. Example 3.3 Find a PWS function x∗ (·) satisfying x (0) = 0, x (2π) = 0 and such that x∗ (·) minimizes J(x(·)) =

Z2π 0


1 [x˙ (s)]2 − [x (s)]2 ds. 2

Again, the integrand f (t, x, u) is the same as in Example 3.1 so that the Euler Equation is x¨∗ (t) + x∗ (t) = 0, and has the general solution x∗ (t) = α cos(t) + β sin(t). However, the boundary conditions 0 = x∗ (0) = α cos(0) + β sin(0) = α, 0 = x∗ (2π) = α cos(2π) + β sin(2π) = α, only imply that x∗ (t) = β sin(t). Therefore, there are infinitely many solutions to the Euler Necessary Condition as given in Theorem 3.3, and we do not know if any of these functions x∗ (t) = β sin(t) actually minimizes J(·). Example 3.4 Find a PWS function x∗ (·) satisfying x (−1) = 0, x (1) = 1 and such that x∗ (·) minimizes J(x(·)) =

Z1

−1

[x (s)]2 [x˙ (s) − 1]2 ds.

115

Chapter 3. The Simplest Problem The integrand f (t, x, u) is given by f (t, x, u) = [x]2 [u − 1]2 and hence, fx (t, x, u) = 2x[u − 1]2 fu (t, x, u) = 2[x]2 [u − 1] fuu (t, x, u) = 2[x]2 ≥ 0.

Note that the integrand is not regular since fuu (t, 0, u) = 0. The Euler equation is [fu (t, x∗ (t), x˙ ∗ (t))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds,

−1

or equivalently, ∗

2

∗

2[x (t)] [x˙ (t) − 1] = c +

Zt

−1



 2x∗ (s)[x˙ ∗ (s) − 1]2 ds.

This equation is not as simple as in the previous examples. However, it is possible to find the solution to this problem by “inspection” of the cost function. Observe that J(x(·)) =

Z1

[x (s)]2 [x˙ (s) − 1]2 ds ≥ 0

−1

for all functions x (·), and J(x(·)) =

Z1

[x (s)]2 [x˙ (s) − 1]2 ds = 0

−1

if x (s) = 0 or x˙ (s) − 1 = 0. Consider the function defined by  0, −1 ≤ t ≤ 0, ∗ x (t) = t, 0 ≤ t ≤ 1.

116

Chapter 3. The Simplest Problem

Note that ∗

J(x (·)) =

Z1

[x∗ (s)]2 [x˙ ∗ (s) − 1]2 ds

−1

=

Z0

2

2

[x∗ (s)] [x˙ ∗ (s) − 1] ds +

=

[x∗ (s)]2 [x˙ ∗ (s) − 1]2 ds

0

−1

Z0

Z1

[0]2 [0 − 1]2 ds +

Z1

[s]2 [1 − 1]2 ds = 0,

0

−1

and hence ∗

J(x (·)) = 0 ≤

Z1

[x (s)]2 [x˙ (s) − 1]2 ds = J(x(·)),

−1

for all x(·). Hence, x∗ (·) is a global minimizer for J(·) on Θ = {x(·) ∈ P W S(−1, 1) : x (−1) = 0, x (1) = 1} . Remark 3.5 This example illustrates one important point about “solving” problems. Always think about the problem before you start to “turn the crank” and compute. Example 3.5 Minimize the functional J(x(·)) =

R1

[x˙ (s)]3 ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Here, f (t, x, u) = u3 , fu (t, x, u) = 3u2 , and fx (t, x, u) = 0. Euler’s equation becomes 3[x˙ ∗ (t)]2 = fu (t, x∗ (t), x˙ ∗ (t)) = c +

Zt 0

=c+

Zt 0

0ds = c,

fx (s, x∗ (s), x˙ ∗ (s))ds

Chapter 3. The Simplest Problem

117

or equivalently, 3[x˙ ∗ (t)]2 = c. Therefore, x˙ ∗ (t) = ±

p

c/3 = ±k

and all we know is that x∗ (t) is piecewise linear. Since x∗ (0) = 0 and x∗ (1) = 1, a possible candidate is x∗ (t) = t. Although we have derived a first order necessary condition for the simplest problem, the basic idea can be extended to very general problems. In particular, we shall see that the simplest problem is a special case of a class of “infinite dimensional” optimization problems. We shall discuss this framework in later sections and apply this to the problem of finding local minimizers for the simplest problem. However, we first discuss some other applications of the FLCV.

3.4

Implications and Applications of the FLCV

In the previous section we applied the FLCV to develop the Euler Necessary Condition Theorem 3.3 for a global minimum. However, the FLCV also plays a key role in the development of many ideas that provide the basis for the modern theories of distributions and partial differential equations. Although these ideas are important and interesting, a full development of the material lies outside the scope of these notes. However, we present two simple examples to illustrate other applications of the FLCV and to provide some historical perspective on the role the calculus of variations has played in the development of modern mathematics.

118

3.4.1

Chapter 3. The Simplest Problem

Weak and Generalized Derivatives

Recall that Part (B) of the FLCL Lemma 3.1 states that if α(·) and β(·) are piecewise continuous on [t0 , t1 ] and Zt1

[α (s) η (s) + β (s) η˙ (s)] ds = 0

(3.30)

t0

for all η(·) ∈ V0 , then there is a constant c such that β (t) = c +

Zt

α (s) ds

t0

except at a finite number of points. The converse is also true. In particular, β (·) = βc (·) e.f. where βc (·) is the PWS function defined by Zt βc (t) ≡ c + α (s) ds t0

and at points t where α(·) is continuous β˙ c (t) = α (t) . If we rewrite (3.30) as Zt1

t0

β (s) η˙ (s) ds = −1

Zt1

α (s) η (s) ds,

(3.31)

t0

then the expression (3.31) can be used to define the “weak derivative” of a piecewise continuous function β (·). Definition 3.4 Let β (·) ∈ P W C(t0 , t1 ). We say that β (·) has a weak derivative on [t0 , t1 ], if there is a PWC function α (·) ∈ P W C(t0 , t1 ) such that Zt1

t0

β (s) η˙ (s) ds = (−1)

1

Zt1

t0

α (s) η (s) ds

119

Chapter 3. The Simplest Problem

for all η(·) ∈ V0 . The function α (·) ∈ P W C(t0 , t1 ) is called the weak derivative of β (·) on [t0 , t1 ]. Remark 3.6 It is important to note that the concept of a weak derivative as defined here is dependent on the specific interval [t0 , t1 ]. In particular, it is possible that a function can have a weak derivative on the interval [−1, 0], and a weak derivative on [0, +1], but not have a weak derivative on the interval [−1, +1]. Observe that Part (B) of the FLCV implies that if β (·) ∈ P W C(t0 , t1 ) has a weak derivative on [t0 , t1 ], say α (·) ∈ P W C(t0 , t1 ), then β (·) has a ordinary derivative except at a finite number of points in [t0 , t1 ] and at points where α(·) is continuous β˙ (t) = α (t) . Therefore, if the weak derivative of β (·) exist, then the ordinary (strong) derivative of β (·) exists except at a finite number of points and is given by β˙ (t) = α (t). The FLCV also implies the converse is true. Moreover, as noted in Remark 3.2 above there is a unique PWS function βc (·) such that β (·) = βc (·) e.f. and we can identify β (·) with its “equivalent” PWS representation βc (·). Thus, with this convention one can say that if β (·) ∈ P W C(t0 , t1 ) has a weak derivative on [t0 , t1 ], then β (·) ∈ P W S(t0 , t1 ). It may appear that the notion of a weak derivative does not bring anything very new to the table and in one dimension this is partially true because of the FLCV. However, consider how one might extend the notion to higher order derivatives. A natural extension to higher order derivatives would be to define a function α (·) ∈ P W C(t0 , t1 ) to be a weak second derivative of β (·) ∈ P W C(t0 , t1 ), if Zt1

t0

β (s) η¨ (s) ds = (−1)

2

Zt1

α (s) η (s) ds

(3.32)

t0

for all η(·) ∈ V0 with η(·) ˙ ∈ V0 . Later we shall see that extensions of the FLCV can be used to show that (3.31) implies that β˙ (·) ∈ P W S(t0 , t1 ) and β¨ (·) exists except at a finite number of points. It

120

Chapter 3. The Simplest Problem

is easy to see, the key idea is to use integration by parts (like in the proof of Part (B) of the FLCV) to “move” the derivatives from β (·) ∈ P W C(t0 , t1 ) to the functions η(·). Again, it may appear that the notion of a weak 2nd derivative does not bring anything new to the table. However, the real power of of this idea comes when it is applied to functions of several variables. Example 3.6 Consider the PWC function β(·) defined on [−1, +1] by β(t) = |t| and α(·) defined by 

α(t) =

−1, −1 ≤ t ≤ 0, +1, 0 < t ≤ +1.

If η (·) ∈ V0 , then Z+1 Z0 Z+1 β (s) η˙ (s) ds = |s| η˙ (s) ds + |s| η˙ (s) ds 0

−1

−1

=−

Z+1 sη˙ (s) ds + sη˙ (s) ds

Z0

0

−1

= − [sη

(s)] |s=0 s=−1

+

Z0

η (s) ds + [sη (s)] |s=+1 s=0

−1

−

Z+1

η (s) ds

0

= −0 +

Z0

−1

=

Z0

−1

Z+1 η (s) ds + 0 − η (s) ds

η (s) ds −

0

Z+1

η (s) ds.

0

121

Chapter 3. The Simplest Problem On the other hand Z+1 Z0 Z+1 α (s) η (s) ds = α (s) η (s) ds + α (s) η (s) ds 0

−1

−1

=−

Z0

η (s) ds +

Z+1

η (s) ds,

0

−1

so that Z+1 Z0 Z+1 β (s) η˙ (s) ds = η (s) ds − η (s) ds −1

−1

0



= − −

Z0

−1

=−

Z+1

 Z+1 η (s) ds + η (s) ds 0

α (s) η (s) ds.

−1

Hence, β(t) = |t| has a PWC weak derivative on [−1, +1] and the weak derivative is α(·).

Example 3.7 Consider the PWC function β(·) defined on [−1, +1] by  −1/2, −1 ≤ t ≤ 0, β(t) = +1/2, 0 ≤ t ≤ +1. We show that β(·) does not have a weak derivative on [−1, +1]. Assume the contrary, that there is a function α(·) ∈ P W S(−1, +1) such that Z+1 Z+1 β (s) η˙ (s) ds = − α (s) η (s) ds

−1

−1

122

Chapter 3. The Simplest Problem

for all η (·) ∈ V0 (−1, +1). Note that Z+1 Z0 Z+1 β (s) η˙ (s) ds = (−1/2)η˙ (s) ds + (+1/2)η˙ (s) ds −1

−1

=



0

1 − 2

Z0

η˙ (s) ds +

Z+1 0

−1



η˙ (s) ds

  1 1 = −η(0− ) − η(0+ ) = − η(0− ) + η(0+ ) 2 2 = −η(0), since η (·) ∈ V0 (−1, +1) is continuous. Thus, if Z+1 Z+1 β (s) η˙ (s) ds = − α (s) η (s) ds −1

−1

for all η (·) ∈ V0 (−1, +1), then Z+1 α (s) η (s) ds = η(0)

(3.33)

−1

for all η (·) ∈ V0 (−1, +1). To see that there is no function α(·) ∈ P W S(−1, +1) satisfying (3.33) for all η(·) ∈ V0 (−1, +1), assume such a function α(·) ∈ P W S(−1, +1) exists and let α ˆ be such that |α(s)| ≤ α ˆ,

− 1 ≤ s ≤ +1.

For m = 1, 2, 3, . . . let η m (·) ∈ V0 (−1, +1) be given by   +m(t − 1/m), −1/m ≤ t ≤ 0, −m(t + 1/m), 0 ≤ t ≤ 1/m, η m (t) =  0, elsewhere, and note that η m (0) = 1 while

Z+1 Z+1 1 m η (s) ds = |η m (s)| ds = . m

−1

−1

123

Chapter 3. The Simplest Problem Select M > 1 so that

α ˆ < 1/2 M

and observe that +1 Z+1 Z Z+1 M η (s) ds α (s) η M (s) ds ≤ α (s) η M (s) ds ≤ |ˆ α | −1

−1

=

−1

α ˆ < 1/2. M

However, η M (·) ∈ V0 (−1, +1) and η M (0) = 1, but Z+1 α (s) η M (s) ds ≤ 1/2 < 1 = η M (0)

−1

and hence (3.33) does not hold for η M (·) ∈ V0 (−1, +1). Consequently,  −1/2, −1 ≤ t ≤ 0, β(t) = +1/2, 0 ≤ t ≤ +1,

does not have a weak derivative on [−1, +1]. Observe that β(·) does have a weak derivative on [−1, 0] and a weak derivative on [0, +1] and on each of these intervals the weak derivative is zero. Remark 3.7 The extension of weak derivatives to a more general setting requires the development of “distribution theory” and will not be discussed in this book. However, this extension leads to the modern definition of a generalized derivative (or distribution) that covers the example above. In particular, for β(·) defined on (−∞, +∞) by  0, t ≤ 0, β(t) = +1, 0 < t.

the generalized derivative of β(·) (on R) is the Dirac “delta function”, denoted by δ(·) and is not a PWC function. In fact, δ(·) is not a function in the usual sense and hence the generalized derivative of β(·) is a distribution (see the references [2], [88] and [161]). Modern theories of partial differential equations make extensive

124

Chapter 3. The Simplest Problem

use of weak and generalized derivatives. These derivatives are used to define weak solutions that are key to understanding both theoretical and computational issues in this field (see [55], [56], [76], [96], [126] and [179]).

3.4.2

Weak Solutions to Differential Equations

In order to set up the finite element method in Section 2.4.5 to solve the two-point boundary value problem − x¨(t) + x(t) = f (t),

0 < t < 1,

(3.34)

subject to the Dirichlet boundary conditions x(0) = 0,

x(1) = 0,

(3.35)

we discussed strong and weak solutions to (3.34) - (3.35). If x(t) is a solution of (3.34) - (3.35) in the classical sense, then multiplying both sides of (3.34) by η(·) ∈ V0 = P W S0 (0, 1) and integration by parts produced the variational equation Z 1 Z 1 Z 1 f (t)η(t)dt, (3.36) x(t)η(t)dt = x(t) ˙ η(t)dt ˙ + 0

0

0

which must hold for all η(·) ∈ P W S(0, 1) satisfying η(0) = 0,

η(1) = 0.

(3.37)

Thus, Z

1 0

x(t) ˙ η(t)dt ˙ =−

Z

Z

1

1

f (t)η(t)dt x(t)η(t)dt + 0 0 Z 1 1 [x(t) − f (t)]η(t)dt = (−1) 0

for all η(·) ∈ V0 and hence β(t) = x(t) ˙ satisfies Z 1 Z 1 1 β(t)η(t)dt ˙ = (−1) α(t)η(t)dt 0

0

Chapter 3. The Simplest Problem

125

for all η(·) ∈ V0 , where α(t) = [x(t) − f (t)]. Consequently, β(t) = x(t) ˙ has a weak derivative on [0, 1] given by [x(t) − f (t)] and it follows that if x(·) ∈ P W S0 (0, 1), then is a weak solution of the two-point boundary value problem (3.34) - (3.35). Weak and generalized (distributional) derivatives and the notion of weak solutions to differential equations are key concepts in modern analysis. In multi-dimensional settings where one is interested in partial differential equations, the mathematical background required to properly address the theory of partial differential equations is more complex. However, the basic ideas have their roots in the classical problems discussed above. For more advanced readers we suggest the references [64], [65], [89] and [186].

3.5

Problem Set for Chapter 3

Consider the Simplest Problem in the Calculus of Variations (SPCV): Find x∗ (·) to minimize the cost function Z t1 J(x(·)) = f (s, x(s), x(s))ds, ˙ t0

subject to x(t0 ) = x0 , x(t1 ) = x1 . For each of the following problems: (A) Write out the integrand f (t, x, u). (B) Determine the endpoints t0 and t1 . (C) Determine the endpoints x0 and x1 . (D) Compute all the partial derivatives ft (t, x, u), fx (t, x, u), fu (t, x, u), ftt (t, x, u), fxx (t, x, u), fuu (t, x, u), ftx (t, x, u), ftu (t, x, u) and fxu (t, x, u).

126

Chapter 3. The Simplest Problem

(E) What can you say about possible minimizing functions x∗ (·) for these problems? Write a short summary of what you know and don’t know about each problem. Here x(t) ˙ =

dx(t) dt

is the derivative.

Problem 3.1 Minimize the functional J(x(·)) =

R1

x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Problem 3.2 Minimize the functional J(x(·)) =

R1

x (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Problem 3.3 Minimize the functional J(x(·)) =

R1

sx (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Rb Problem 3.4 Minimize the functional J(x(·)) = [x˙ (s)]3 ds, 0

subject to the endpoint conditions x (0) = 0 and x (b) = x1 . Problem 3.5 Minimize the functional [J(x(·)) =

Z1

{[x˙ (s)]2 + [x (s)]2 + 2es x (s)}ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = e/2. Problem 3.6 Minimize the functional J(x(·)) =

R2

s−3 [x˙ (s)]2 ds,

1

subject to the endpoint conditions x (1) = 1 and x (2) = 16. Problem 3.7 Minimize the functional J(x(·)) =

Z4

[x˙ (s) − 1]2 [x˙ (s) + 1]2 ds,

0

subject to the endpoint conditions x (0) = 0 and x (4) = 2.

Chapter 3. The Simplest Problem Problem 3.8 Minimize the functional Zπ/2 J(x(·)) = {[x˙ (s)]2 − [x (s)]2 }ds, 0

subject to the endpoint conditions x (0) = 0 and x (π/2) = 0. Problem 3.9 Minimize the functional J(x(·)) =

Zπ

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0. Problem 3.10 Minimize the functional

J(x(·)) =

3π/2 Z

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (3π/2) = 0. Problem 3.11 Minimize the functional J(x(·)) =

Zb

x(s)

0

p 2 ds, 1 + [x(s)] ˙

subject to the endpoint conditions x (0) = 1 and x (b) = 2. Problem 3.12 Minimize the functional J(x(·)) =

Zb s

2 1 + [x(s)] ˙ ds, 2gx(s)

0

subject to the endpoint conditions x (0) = 1 and x (b) = 0.

127

128

Chapter 3. The Simplest Problem

Problem 3.13 Minimize the functional Z2

J(x(·)) =

2 {[x(s)] ˙ − 2sx(s)}ds,

1

subject to the endpoint conditions x (1) = 0 and x (2) = −1. Problem 3.14 Minimize the functional J(x(·)) =

Zπ

2 {[x(s)]2 [1 − [x(s)] ˙ }ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0. Problem 3.15 Minimize the functional J(x(·)) =

Z3

{[3s − x(s)]x(s)}ds,

1

subject to the endpoint conditions x (1) = 1 and x (3) = 9/2. Problem 3.16 Minimize the functional J(x(·)) = 4πρv

2

ZL

3 {[x(s)] ˙ x(s)}ds,

0

subject to the endpoint conditions x (0) = 1 and x (L) = R. Here, ρ, v 2 , L > 0 and R > 0 are all constants. Problem 3.17 Minimize the functional J(x(·)) =

Z2

{x(s)[1 ˙ + s2 x(s)]}ds, ˙

1

subject to the endpoint conditions x (1) = 3 and x (2) = 5.

Chapter 3. The Simplest Problem

129

Advanced Problems Problem 3.18 Show that if x∗ (·) minimize the functional J(x(·)) =

Z1 0

1 [x˙ (s)]2 + [x (s)]2 − 2es x (s) ds, 2

then x∗ (·) satisfies the two point boundary value problem −¨ x(t) + x(t) = et , x(0) = x(1) = 0. Problem 3.19 Use the finite element method to solve the two point boundary problem −¨ x(t) + x(t) = et , x(0) = x(1) = 0.

Chapter 4 Necessary Conditions for Local Minima We turn now to the problem of obtaining necessary conditions for local minimizers. As in the previous chapters, let X = P W S(t0 , t1 ) denote the space of all real-valued piecewise smooth functions defined on [t0 , t1 ]. For each PWS function x : [t0 , t1 ] → R, define the functional J : X → R (a “function of a function”) by J(x(·)) =

Zt1

f (s, x(s), x˙ (s)) ds.

(4.1)

t0

Assume that the points [t0 x0 ]T and [t1 x1 ]T are given and define the subset Θ of P W S(t0 , t1 ) by Θ = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 , x (t1 ) = x1 } .

(4.2)

Observe that J : X → R is a real valued function on X. The Simplest Problem in the Calculus of Variations (the fixed endpoint problem) is the problem of minimizing J(·) on

131

132

Chapter 4. Necessary Conditions for Local Minima

Θ. In particular, the goal is to find x∗ (·) ∈ Θ such that ∗

J(x (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s))ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s))ds,

t0

=

t0

for all x (·) ∈ Θ. The basic goal in this chapter is the development of the classical necessary conditions for local minimizers. We begin with a review of the basic definitions.

4.1

Weak and Strong Local Minimizers

In order to define local minimizers for the SPCV, we must have a measure of distance between functions in P W S(t0 , t1 ). Given two functions x(·) and z(·) ∈ P W S(t0 , t1 ) there are many choices for a distance function, but we will focus on the weak and strong metrics defined in section 2.3.1. Recall that the d0 distance between x(·) and z(·) is defined by d0 (x(·), z(·)) , sup {|x(t) − z(t)|}.

(4.3)

t0 ≤t≤t1

In this case we can define a norm on P W S(t0 , t1 ) by kx(·)k0 = sup {|x(t)|}

(4.4)

t0 ≤t≤t1

and note that d0 (x(·), z(·)) = kx(·) − z(·)k0 . Given xˆ(·) ∈ P W S(t0 , t1 ) and δ > 0, the U0 (ˆ x(·), δ)neighborhood (or Strong Neighborhood) of xˆ(·) is defined

Chapter 4. Necessary Conditions for Local Minima

133

to be the open ball U0 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d0 (ˆ x(·), x(·)) < δ} . Likewise, the d1 distance between x(·) and z(·) is defined by d1 (x(·), z(·)) = sup {|x(t) − z(t)| : t0 ≤ t ≤ t1 }  (4.5) + sup |x(t) ˙ − z(t)| ˙ : t0 ≤ t ≤ t1 , t 6= tˆi = d0 (x(·), z(·)) + sup {|x(t) ˙ − z(t)| ˙ : t0 ≤ t ≤ t1 , t 6= tˆi .

In this case the 1-norm is defined on P W S(t0 , t1 ) by kx(·)k1 = d1 (x(·), 0(·)),

(4.6)

where 0(·) is the zero function and as before, d1 (x(·), z(·)) = kx(·) − z(·)k1 .

(4.7)

If xˆ(·) ∈ P W S(t0 , t1 ) and δ > 0, the U1 (ˆ x(·), δ)neighborhood (or Weak Neighborhood) of xˆ(·) is defined to be the open ball U1 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d1 (ˆ x(·), x(·)) < δ}. Remark 4.1 Recall that if d1 (x(·), z(·)) = 0, then x(t) = z(t) for all t ∈ [t0 , t1 ] and x(t) ˙ = z(t) ˙ e.f. Also, d0 (x(·), z(·)) ≤ d1 (x(·), z(·)) and it follows that if d1 (x(·), z(·)) < δ, then d0 (x(·), z(·)) ≤ δ. This is an important inequality since it implies that U1 (ˆ x(·), δ) ⊂ U0 (ˆ x(·), δ) ⊂ P W S(t0 , t1 ),

(4.8)

so that the U1 (ˆ x(·), δ)-neighborhood U1 (ˆ x(·), δ) is smaller than the U0 (ˆ x(·), δ)-neighborhood U0 (ˆ x(·), δ). In addition to defining global minimizers, the metrics d0 and d1 defined on P W S(t0 , t1 ) allows us to define two types of local minimizers for the SPCV.

134

Chapter 4. Necessary Conditions for Local Minima

Definition 4.1 The function x∗ (·) ∈ Θ, provides a global minimizer for J(·) on Θ if J(x∗ (·)) ≤ J(x(·)) for all x(·) ∈ Θ. Definition 4.2 The function x∗ (·) ∈ Θ, provides a strong local minimizer for J(·) on Θ if there is a δ > 0 such that J(x∗ (·)) ≤ J(x(·)) for all x(·) ∈ U0 (x∗ (·), δ) ∩ Θ. Definition 4.3 The function x∗ (·) ∈ Θ, provides a weak local minimizer for J(·) on Θ if there is a δ > 0 such that J(x∗ (·)) ≤ J(x(·)) for all x(·) ∈ U1 (x∗ (·), δ) ∩ Θ. Remark 4.2 Recall that U1 (x∗ (·), δ) ⊂ U0 (x∗ (·), δ) ⊂ P W S(t0 , t1 ). Therefore, it follows that a global minimizer is a strong local minimizer, and a strong local minimizer is a weak local minimizer. It is important to note that a necessary condition for weak local minimum is also a necessary condition for a strong local minimum, and a necessary condition for strong local minimum is also a necessary condition for a global minimum. In particular, any necessary condition for a weak local minimum applies to strong and global minima. However, a necessary condition obtained by assuming that x∗ (·) is a global minimum may not apply to a local minimum. The important point is that if one can derive a necessary condition assuming only that x∗ (·) is a weak local minimizer, then it is more powerful (i.e. applies to more problems) than a necessary condition obtained by assuming that x∗ (·) is a strong or global minimizer. Remark 4.3 In the following sections we derive four necessary conditions for weak and strong local minimizers for the SPCV.

135

Chapter 4. Necessary Conditions for Local Minima

These necessary conditions are numbered I, II, III and IV . This numbering system is used to match what appears in the classical work of Gilbert Bliss [29] and follows the convention used by Ewing [77]. The numbers do not reflect the historical development of the conditions and should not be thought of as an order for solution of practical problems.

4.2

The Euler Necessary Condition (I)

In this section we extend the Euler Necessary Condition from the previous chapter to weak local minimizers. Assume that x∗ (·) ∈ Θ is a weak local minimizer for J(·) on Θ. In particular, there is a δ > 0 such that ∗

J(x (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s))ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s))ds,

t0

=

(4.9)

t0

for all x (·) ∈ U1 (x∗ (·), δ) ∩ Θ. Let η(·) ∈ V0 and consider the variation ϕ(t, ε) , x∗ (t) + εη(t).

(4.10)

Recall that for each ε ∈ R the variation ϕ(t, ε) satisfies the following conditions: (i) ϕ(t0 , ε) = x∗ (t0 ) + εη(t0 ) = x0 + εη(t0 ) = x0 + ε0 = x0 , (ii) ϕ(t1 , ε) = x∗ (t1 ) + εη(t1 ) = x1 + εη(t1 ) = x1 + ε0 = x1 , (iii) ϕ(t, ε) = x∗ (t) + εη(t) ∈ P W S(t0 , t1 ).

136

Chapter 4. Necessary Conditions for Local Minima

It follows that if η(·) ∈ V0 , then for all ε ∈ R, ϕ(·, ε) = x∗ (·) + εη(·) ∈ Θ. However, it is not always true that ϕ(·, ε) = x∗ (·) + εη(·) ∈ U1 (x∗ (·), δ) unless ε is small. Let kη(·)k1 = sup {|η(t)| : t0 ≤ t ≤ t1 }  + sup |η(t)| ˙ : t0 ≤ t ≤ t1 , t 6= tˆi

and note that kη(·)k1 = 0 if and only if η(t) = 0 for all t ∈ [t0 , t1 ]. The case kη(·)k1 = 0 is trivial, so assume that kη(·)k1 6= 0 and select ε such that δ −δ 0 such that J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s))ds ≤ J(x(·)) =

t0

for all x (·) ∈ U1 (x∗ (·), δ) ∩ Θ.

Zt1

t0

f (s, x(s), x˙ (s))ds,

140

Chapter 4. Necessary Conditions for Local Minima Let η(·) ∈ V0 and again we consider the classical variation ϕ(x, ε) , x∗ (t) + εη(t).

−δ , δ ), then d1 (x∗ (·), ϕ(·, ε)) < δ and As before, if ε ∈ ( kη(·)k 1 kη(·)k1 ϕ(·, ε) = x∗ (·) + εη(·) ∈ U1 (x∗ (·), δ) ∩ Θ and is admissible. Since x∗ (·) ∈ Θ minimizes J(·) on U1 (x∗ (·), δ) ∩ Θ, it follows that

J(x∗ (·)) ≤ J(x∗ (·) + εη(·)) −δ δ −δ δ for all ε ∈ ( kη(·)k , kη(·)k ). Define F : ( kη(·)k , kη(·)k ) −→ R by 1

1

∗

F (ε) = J(x (·) + εη(·)) =

1

Zt1

1

f (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))ds, ˙

t0

and note that F (0) = J(x∗ (·)) ≤ J(x∗ (·) + εη(·)) = F (ε) −δ , δ ). Therefore, F (·) has a minimum on for all ε ∈ ( kη(·)k 1 kη(·)k1 −δ ( kη(·)k , δ ) at ε∗ = 0. This time we apply the second order 1 kη(·)k1 condition as stated in Theorem 2.2 from Chapter 2. In particu d2 lar, if dε2 F (ε) exists then ε=0

d2 F (ε) dε2

d2 ∗ , δ J(x (·); η(·)) = 2 [J(x (·) + εη(·))] ≥ 0. dε ε=0 (4.21) 2

ε=0

∗

Observe that (4.21) holds for all η(·) ∈ V0 . To use (4.21) we must compute the second variation δ 2 J(x∗ (·); η(·)). The first variation of F (ε) =

Zt1

t0

f (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))ds ˙

Chapter 4. Necessary Conditions for Local Minima

141

is given by d d F (ε) = dε dε

Zt1

f (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))ds ˙

t0

=

Zt1

[fx (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η(s)ds

t0

+

Zt1

[fu (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η˙ (s) ds.

t0

Differentiating once again yields d2 d F (ε) = 2 dε dε

Zt1

[fx (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η(s)ds

t0

d + dε

Zt1

[fu (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η˙ (s) ds

t0

=

Zt1

[fxx (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · [η(s)]2 ds

t0

+

Zt1

[fxu (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η(s) · η(s)ds ˙

Zt1

[fux (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · η˙ (s) · η(s)ds

Zt1

[fuu (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))] ˙ · [η˙ (s)]2 ds,

t0

+

t0

+

t0

142

Chapter 4. Necessary Conditions for Local Minima

and setting ε = 0 produces Zt1 d2 F (ε) = [fxx (s, x∗ (s), x˙ ∗ (s))] · [η(s)]2 ds dε2 ε=0 t0

+

Zt1

[fxu (s, x∗ (s), x˙ ∗ (s))] · η(s) · η˙ (s) ds

Zt1

[fux (s, x∗ (s), x˙ ∗ (s))] · η˙ (s) · η(s)ds

Zt1

[fuu (s, x∗ (s), x˙ ∗ (s))] · [η˙ (s)]2 ds.

t0

+

t0

+

t0

Since, fux (s, x∗ (s), x˙ ∗ (s)) = fxu (s, x∗ (s), x˙ ∗ (s)), it follows that Zt1 d2 = [fxx (s, x∗ (s), x˙ ∗ (s))] · [η(s)]2 ds F (ε) dε2 ε=0

(4.22)

t0

+

Zt1

[2fxu (s, x∗ (s), x˙ ∗ (s))] · η(s) · η(s)ds ˙

Zt1

[fuu (s, x∗ (s), x˙ ∗ (s))] · [η˙ (s)]2 ds.

t0

+

t0

In order to simplify notation, we set ∗ fxx (t) = fxx (t, x∗ (t), x˙ ∗ (t)),

(4.23)

∗ fxu (t) = fxu (t, x∗ (t), x˙ ∗ (t)),

(4.24)

∗ fuu (t) = fuu (t, x∗ (t), x˙ ∗ (t)).

(4.25)

and

143

Chapter 4. Necessary Conditions for Local Minima

Therefore, we have established that the second variation is given by Zt1 ∗ ∗ (s)[η(s)]2 + 2fxu (s)[η(s)η(s)] ˙ δ 2 J(x∗ (·); η(·)) = {fxx t0

∗ + fuu (s)[η˙ (s)]2 }ds.

(4.26)

Consequently, (4.21) is equivalent to the condition that Zt1

∗ ∗ ∗ {fxx (s)[η(s)]2 + 2fxu (s)[η(s)η(s)] ˙ + fuu (s)[η˙ (s)]2 }ds ≥ 0

t0

holds for all η(·) ∈ V0 . Therefore, we have established the following necessary condition. Theorem 4.3 If x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then Zt1

∗ ∗ ∗ 2 {fxx (s)[η(s)]2 +2fxu (s)[η(s)η(s)]+f ˙ uu (s)[η˙ (s)] }ds ≥ 0, (4.27)

t0

for all η(·) ∈ V0 . Theorem 4.3 above is not very useful as stated. We need to extract useful information about x∗ (·) from this inequality. The first result along this line is the Legendre Necessary Condition. Theorem 4.4 (Legendre Necessary Condition - (III)) If x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then,

∗ (L-1) fuu (t) = fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0, for all t0 ≤ t ≤ t1 ,

(L-2) x∗ (t0 ) = x0 , (L-3) x∗ (t1 ) = x1 .

Remark 4.4 It is important to note that condition (L-1) holds at corners. In particular, since fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0

(4.28)

144

Chapter 4. Necessary Conditions for Local Minima

it follows that fuu t, x∗ (t), x˙ ∗ t+ for all t ∈ [t0 , t1 ), and fuu t, x∗ (t), x˙ ∗ t− for all t ∈ (t0 , t1 ].









≥0 ≥0

Remark 4.5 The condition that fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0

(4.29)

for all t ∈ [t0 , t1 ] is called the Legendre Condition. Note that the Legendre Necessary Condition is often easy to check. In the case where fuu (t, x, u) ≥ 0 for all (t, x, u), it is not very helpful. However, the Strengthened Legendre Condition fuu (t, x∗ (t), x˙ ∗ (t)) > 0

(4.30)

for all t ∈ [t0 , t1 ] will be very useful. Recall that an extremal x(·) ∈ P W S(t0 , t1 ) is called a regular extremal if fuu (t, x(t), x(t)) ˙ >0 for all t ∈ [t0 , t1 ] such that x(t) ˙ exists. Also, if f (t, x, u) is a regular integrand, then all extremals are regular. Example 4.1 Consider the functional J(x(·)) =

R1

[x˙ (s)]3 ds.

0

Here, f (t, x, u) = u3 , fu (t, x, u) = 3u2 , fuu (t, x, u) = 6t, and fx (t, x, u) = 0. As noted in Example 3.5 all extremals are piecewise linear functions. In particular, Euler’s Integral Equation is given by ∗

2

∗

∗

3[x˙ (t)] = fu (t, x (t), x˙ (t)) = c +

Zt 0

=c+

Zx 0

0ds = c,

fx (s, x∗ (s), x˙ ∗ (s))ds

Chapter 4. Necessary Conditions for Local Minima

145

or equivalently, 3[x˙ ∗ (t)]2 = c. Therefore, x˙ ∗ (t) = ±

p

c/3 = ±k

and hence it follows that x∗ (·) is piecewise linear with slope restricted to either ±k. Hence, fuu (t, x∗ (t), x˙ ∗ (t)) = 6x˙ ∗ (t) = ±6k

and as long as k 6= 0 the extremal is non-singular. On the other hand, the only regular extremals are those satisfying fuu (t, x(t), x˙ ∗ (t)) = 6x˙ ∗ (t) = ±6k > 0, which means that the derivative must always be positive. In particular, x∗ (t) = mt + r with m > 0. Example 4.2 Consider the problem of minimizing the functional J(x(·)) =

Z1

[x˙ (s)]3 ds

0

subject to the endpoint conditions x (0) = 0 and x (1) = b. We know that if x˙ ∗ (·) is a weak local minimizer it is an extremal so that x˙ ∗ (t) = ±6k

for some k. Applying the Legendre Necessary Condition, it must be the case that fuu (t, x∗ (t), x˙ ∗ (t)) = 6x˙ ∗ (t) = ±6k , m ≥ 0,

(4.31)

and the derivative cannot change sign. Thus, x∗ (t) = mt + r for all t ∈ [0, 1] where m ≥ 0. The endpoint conditions x(0) = 0 and x(1) = b imply that m = b and x∗ (t) = bt is the only possible minimizer. If b < 0, then x∗ (t) = bt fails to satisfy the Legendre Condition (4.31) and there is no local minimizer. If b ≥ 0, then x∗ (t) = bt will satisfy the Legendre Condition (4.31) and perhaps can be a minimizer.

146

Chapter 4. Necessary Conditions for Local Minima

4.4

Jacobi Necessary Condition - (IV)

The general second order necessary condition Theorem 4.3 can also be used to obtain another necessary condition due to Karl Gustav Jacob Jacobi. In 1837 Jacobi used some of Legendre’s basic ideas on the second variation and constructed what is known as Jacobi’s (second order) Necessary Condition. In order to state the result we need to introduce some additional terms and definitions. Recall that if x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then the general result Theorem 4.3 implies that 2

∗

δ J(x (·); η(·)) =

Zt1

∗ ∗ {fxx (s)[η(s)]2 + 2fxu (s)[η(s)η(s)] ˙

t0

∗ + fuu (s)[η˙ (s)]2 }ds ≥ 0,

(4.32)

for all η(·) ∈ V0 . Jacobi noted that if ηo (·) ∈ V0 is defined to be the zero function, ηo (t) ≡ 0 for all t ∈ [t0 , t1 ], then 2

∗

δ J(x (·); ηo (·)) =

Zt1

∗ ∗ ∗ ˙ + fuu {fxx (s)[0]2 + 2fxu (s)[00] (s)[0]2 }ds = 0.

t0

(4.33) ∗ ∗ ∗ Again, remember that the functions fxx (·), fxu (·), and fuu (·) are fixed functions of t given by ∗ fxx (t) = fxx (t, x∗ (t), x˙ ∗ (t)), ∗ fxu (t) = fxu (t, x∗ (t), x˙ ∗ (t)),

and ∗ fuu (t) = fuu (t, x∗ (t), x˙ ∗ (t)),

respectively. Using this notation we define the function F(t, η, ξ) by 1 ∗ ∗ ∗ (t)η 2 + 2fxu (t)ηξ + fuu (t)ξ 2 ] F(t, η, ξ) = [fxx 2

(4.34)

147

Chapter 4. Necessary Conditions for Local Minima and consider the functional J : P W S(t0 , t1 ) −→ R given by J (η(·)) =

Zt1

F(s, η(s), η(s))ds. ˙

(4.35)

t0

Let ΘS ⊂ P W S(t0 , t1 ) be defined by ΘS = {η(·) ∈ P W S(t0 , t1 ) : η (t0 ) = 0, η (t1 ) = 0} = V0 ,

(4.36)

and consider the so called Accessory (Secondary) Minimum Problem. The Accessory (Secondary) Minimum Problem: Find η ∗ (·) ∈ ΘS , such that ∗

J (η (·)) =

Zt1

F(s, η ∗ (s), η˙ ∗ (s))ds ≤ J (η(·))

Zt1

F(s, η(s), η(s))ds, ˙

t0

=

t0

for all η (·) ∈ ΘS . There are two key observations that make the Accessory Minimum Problem important and useful. (1) The answer to the Accessory Minimum Problem is known. In view of (4.32) and (4.33), we know that if x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then the zero function ηo (t) ≡ 0, satisfies J (ηo (·)) =

Zt1

F(s, ηo (s), η˙ o (s))ds = 0 ≤ J (η(·))

Zt1

F(s, η(s), η(s))ds, ˙

t0

=

t0

for all η (·) ∈ V0 = ΘS . In particular, η ∗ (x) = ηo (x) ≡ 0 is a global minimizer for J on ΘS .

148

Chapter 4. Necessary Conditions for Local Minima

(2) The Accessory Minimum Problem is a special case of the Simplest Problem in the Calculus of Variations with the change of variables (t, x, u) ←→ (t, η, ξ), f (t, x, u) ←→ F(t, η, ξ), J(x(·)) ←→ J (η(·)), (t0 , x0 ) ←→ (t0 , 0), (t1 , x1 ) ←→ (t1 , 0), and Θ ←→ ΘS . Therefore, we can apply the Euler Necessary Condition to the Accessory Problem. In particular, if η ∗ (·) ∈ ΘS , is any minimizer of Zt1 ˙ J (η(·)) = F(s, η(s), η(s))ds t0

on ΘS , then there is a constant c such that η ∗ (·) satisfies the Euler’s Integral Equation ∗

∗

[Fξ (t, η (s), η˙ (s))] = c +

Zt

[Fη (s, η ∗ (s), η˙ ∗ (s))] ds.

(4.37)

t0

In addition, between corners the function Fξ (t, η ∗ (t), η˙ ∗ (t)) is differentiable and d [Fξ (t, η ∗ (t), η˙ ∗ (t))] = [Fη (t, η ∗ (t), η˙ ∗ (t))] . dt

(4.38)

The equation [Fξ (t, η(t), η˙ (t))] = c +

Zt

t0

[Fη (s, η(s), η˙ (s))] ds

(4.39)

149

Chapter 4. Necessary Conditions for Local Minima is called Jacobi’s Integral Equation and d [Fξ (t, η(t), η˙ (t))] = [Fη (t, η(t), η˙ (t))] dt

(4.40)

is called Jacobi’s Differential Equation. Observe that Jacobi’s Equation is Euler’s Equation for the case where f (t, x, u) is replaced by F(t, η, ξ). Definition 4.4 A PWS function η(·) satisfying Jacobi’s Integral Equation (4.39) (or (4.40)) is called a secondary extremal. We are interested in secondary extremals and what they can tell us about the minimizer x∗ (·) to the original SPCV. Thus, it is important to look at the specific form of the Jacobi equations. Since 1 ∗ ∗ ∗ (t)η 2 + 2fxu (t)ηξ + fuu (t)ξ 2 ] F(t, η, ξ) = [fxx 2

(4.41)

it is obvious that ∂ ∗ ∗ F(t, η, ξ) = [fxx (t)η + fxu (t)ξ], ∂η ∂ ∗ ∗ Fξ (x, η, ξ) = F(t, η, ξ) = [fxu (t)η + fuu (t)ξ] ∂ξ Fη (t, η, ξ) =

(4.42) (4.43)

and ∂2 ∗ (t)] = fuu (t, x∗ (t), x˙ ∗ (t)). Fξξ (t, η, ξ) = 2 F(t, η, ξ) = [fuu ∂ξ Legendre’s Necessary Condition applied to the Accessory Minimum Problem implies that Fξξ (t, η, ξ) = fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0. However, in order to go further, we must assume that x∗ (·) is a non-singular extremal. In this case Fξξ (t, η, ξ) = fuu (t, x∗ (t), x˙ ∗ (t)) > 0,

(4.44)

150

Chapter 4. Necessary Conditions for Local Minima

which in turn implies that the corresponding Accessory Minimum Problem is regular. In particular, we know that all secondary extremals η (·) are smooth and, in this case, we need only consider Jacobi’s Differential Equation (4.40). In view of (4.41) Jacobi’s Differential Equation has the form d ∗ ∗ ∗ ∗ [fxu (t)η(t) + fuu (t)η(t)] ˙ = [fxx (t)η(t) + fxu (t)η(t)]. ˙ dt

(4.45)

Remark 4.6 We will focus on Jacobi’s Differential Equation (4.45). Recall that solutions of Jacobi’s Differential Equation are secondary extremals. It is important to note that Jacobi’s Differential Equation is a second order linear differential equation in η(·). Consequently, Jacobi’s Differential Equation with initial conditions of the form η(tˆ) = pˆ and η( ˙ tˆ) = vˆ has a unique solution. This point is important in the proof of Jacobi’s Necessary Condition. Example 4.3 Minimize the functional J(x(·)) =

Zπ/2  0

[x˙ (s)]2 − [x (s)]2 ds,

subject to the endpoint conditions x (0) = 0 and x(π/2) = 0. Here f (t, x, u) = u2 − x2 , fx (t, x, u) = −2x, fxx (t, x, u) = −2, fu (t, x, u) = 2u, fuu (t, x, u) = 2, and fxu (t, x, u) = fxu (t, x, u) = 0. Thus, if x∗ (·) is any minimizer of J(·) on Θ, ∗ fux (t) = fux (t, x∗ (t), x˙ ∗ (t)) = 0, ∗ fxu (t) = fxu (t, x∗ (t), x˙ ∗ (t)) = 0, ∗ fxx (t) = fxx (t, x∗ (t), x˙ ∗ (t)) = −2,

and ∗ fuu (t) = fuu (t, x∗ (t), x˙ ∗ (t)) = 2 > 0.

Therefore, Jacobi’s Equation (4.45) d ∗ ∗ ∗ ∗ [f (t)η(t) + fuu (t)η(t)] ˙ = [fxx (t)η(x) + fxu (t)η(t)], ˙ dt xu

151

Chapter 4. Necessary Conditions for Local Minima reduces to d [0 · η(t) + 2η(t)] ˙ = [−2η(t) + 0 · η(t)], ˙ dt or equivalently, η¨(t) = −η(t). This implies that all secondary extremals have the form η(t) = c1 cos(t) + c2 sin(t).

Definition 4.5 A value tˆc is said to be a conjugate value to t0 , if t0 < tˆc , and there is a solution ηc (·) to Jacobi’s Equation (4.45) satisfying (i) ηc (t0 ) = ηc (tˆc ) = 0 and ηc (t) 6= 0, for some t ∈ (t0 , tˆc ). In particular, ηc (·) does not vanish on (t0 , tˆc ). The point [tˆc x∗ (tˆc ]T ∈ R2 on the graph of x∗ (·) is said to be a conjugate point to the initial point [t0 x∗ (t0 )]T ∈ R2 . Figure 4.1 illustrates the definition. We now state the Jacobi Necessary Condition. Theorem 4.5 (Jacobi’s Necessary Condition - (IV)) Assume that x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ. If x∗ (·) is smooth and regular, then

c (t )

t0

conjugate values

tˆc

tc

t1

Figure 4.1: Definition of a Conjugate Value

152

Chapter 4. Necessary Conditions for Local Minima

(J-1) there cannot be a value tˆc conjugate to t0 with tˆc < t1 ,

(4.46)

(J-2) x∗ (t0 ) = x0 , (J-3) x∗ (t1 ) = x1 .

4.4.1

Proof of the Jacobi Necessary Condition

Assume that x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, x∗ (·) is smooth and fuu (t, x∗ (t), x˙ ∗ (t)) > 0. The proof is by contradiction. Assume that there is a value tˆc conjugate to t0 with tˆc < t1 . Without loss in generality we may assume that tˆc is the “first” conjugate value so that (see Figure 4.1), there is a secondary extremal ηc (·) ∈ P W S(t0 , t1 ) such that ηc (·) satisfies the Jacobi Equation d ∗ ∗ ∗ ∗ [f (t)ηc (t) + fuu (t)η˙ c (t)] = [fxx (t)ηc (t) + fxu (t)η˙ c (t)], dt xu with ηc (t0 ) = 0, ηc (tˆc ) = 0 and ηc (t) 6= 0, t0 < t < tˆc .

(4.47)

Since Fξξ (t, η, ξ) = fuu (t, x∗ (t), x˙ ∗ (t)) > 0, the accessory problem is regular and Hilbert’s Differentiability Theorem implies that all secondary extremals are smooth. Thus, ηc (·) cannot have a corner. Let ηˆ(t) be the piecewise smooth function defined by  ηc (t), t0 ≤ t ≤ tˆc ηˆ(t) = 0, tˆc ≤ t ≤ t1 and note that ηˆ(t) 6= 0 for t0 < t < tˆc (see Figure 4.2). We shall show that ηˆ(·) ∈ V0 = ΘS minimizes J (η(·)) =

Zt1

t0

F(s, η(s), η(s))ds, ˙

Chapter 4. Necessary Conditions for Local Minima

c (t )

t0

tˆc

153

ˆ (t )

t1

tˆc

t0

t1

Figure 4.2: Definition of ηˆ(·) and hence is also a secondary extremal. Observe that for each t0 < t < tˆc ,   d d ηˆ(t)Fξ (t, ηˆ(t), ηˆ(t)) dt dt     d d d d = ηˆ(t) Fξ (t, ηˆ(t), ηˆ(t)) + ηˆ(t) Fξ (t, ηˆ(t), ηˆ(t)) dt dt dt dt   d Fξ (t, ηc (t), η˙ c (t)) + η˙ c (t) [Fξ (t, ηc (t), η˙ c (t))] = ηc (t) dt  h i d ∗ ˆ = ηc (t) f (t)ηc (t) + fuu (t)η˙ c (t) dt xu ∗ ∗ + η˙ c (t) [fxu (t)ηc (t) + fuu (t)η˙ c (t)] ∗ ∗ = ηc (t) [fxx (t)ηc (t) + fxu (t)η˙ c (t)] ∗ ∗ + η˙ c (t) [fxu (t)ηc (t) + fuu (t)η˙ c (t)]  ∗  ∗ = fxx (t)[ηc (t)]2 + fxu (t)ηc (t)η˙ c (t)   ∗ ∗ + fux (t)ηc (t)η˙ c (t) + fuu (t)[η˙ c (t)]2 = 2F(t, ηc (t), η˙ c (t)). Hence, it follows that 2J (ˆ η (·)) =

Zt1

t0

2F(s, ηˆ(s),

d ηˆ(s))ds ds

154

Chapter 4. Necessary Conditions for Local Minima

=

Ztˆc

2F(s, ηc (s), η˙ c (s))ds +

Ztˆc

2F(s, ηc (s), η˙ c (s))ds

Ztˆc

  d d ηˆ(s) · Fξ (s, ηˆ(s), ηˆ(s)) ds ds ds

t0

=

Zt1

2F(s, 0, 0)ds

tˆc

t0

=

t0

 t=tˆc d = ηˆ(t) · Fξ (t, ηˆ(t), ηˆ(t)) = 0. ds t=t0 

Therefore,

2J (ˆ η (·)) =

Zt1

2F(s, ηˆ(s),

d (s)ˆ η )ds = 0 ≤ 2J (η(·)) ds

t0

for all η(·) ∈ V0 = ΘS , and hence ηˆ(·) minimizes J (η(·)) on ΘS . However, this means that ηˆ(·) satisfies Jacobi’s Equation (4.40) and is a secondary extremal. Since secondary extremals cannot have corners, it follows that d d ˆ+ d ˆ ηˆ(tc ) = ηˆ(tˆ− ηˆ(t ) = 0, c ) = dt dt dt c and, by Hilbert’s Differentiability Theorem, ηˆ(·) ∈ C 2 . Thus, ηˆ(·) satisfies the linear second order initial value problem d ∗ ∗ ∗ ∗ [f (t)η(t) + fuu (t)η(t)] ˙ = [fxx (t)η(t) + fxu (t)η(t)], ˙ dt xu with initial condition η(tˆc ) = 0, η( ˙ tˆc ) = 0. However, the only solution to a linear second order initial value problem, with zero initial data, is ηˆ(t) ≡ 0. It follows that for t0 < t < tˆc , ηc (t) = ηˆ(t) = 0,

Chapter 4. Necessary Conditions for Local Minima

155

which contradicts (4.47) above. Therefore, there cannot be a value tˆc with t0 < tˆc < t1 conjugate to t0 and this completes the proof. 

4.5

Weierstrass Necessary Condition (II)

The three previous necessary conditions are valid for weak local minimizers. In this section we assume we have a strong local minimizer and derive the Weierstrass’ Necessary Condition for a strong local minimum (Karl Theodor Wilhelm Weierstrass 1879). Although the techniques are similar to the idea used for Euler’s Necessary condition, the “variations” are different. Weierstrass’ Necessary Condition is much closer to the Maximum Principle we will study in Optimal Control. In particular, for the SPCV the Weierstrass Necessary Condition can be stated as a Maximum Principle which is equivalent to the Pontryagin Maximum principle in optimal control. In order to formulate the Weierstrass Necessary Condition, we need to define the Excess Function. Definition 4.6 The Weierstrass Excess Function E [t0 , t1 ] × R3 −→ R is defined by E(t, x, u, v) = [f (t, x, v) − f (t, x, u)] − [v − u]fu (t, x, u)

:

(4.48)

for all [t0 , t1 ] × R3 . Theorem 4.6 (Weierstrass Necessary Condition - (II)) If x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ, then, (W-1) E(t, x∗ (t), x˙ ∗ (t), v) ≥ 0 for all t ∈ [t0 , t1 ] and v ∈ R, (W-2) x∗ (t0 ) = x0 , (W-3) x∗ (t1 ) = x1 . Condition (W-1 ) E(t, x∗ (t), x˙ ∗ (t), v) ≥ 0,

(4.49)

156

Chapter 4. Necessary Conditions for Local Minima

is the essential new information in Weierstrass’ Necessary Condition. Moreover, (4.49) holds at all t ∈ [t0 , t1 ], including corners. In particular, for all v ∈ R E(t, x∗ (t), x˙ ∗ (t+ ), v) ≥ 0,

(4.50)

E(t, x∗ (t), x˙ ∗ (t− ), v) ≥ 0.

(4.51)

and Before proving the Weierstrass Necessary Condition Theorem 4.6 we note some results that follow from this theorem. First we restate the Weierstrass Necessary Condition as a Maximum Principle by defining a new function. Given x∗ (·), let H(t, v) , −E(t, x∗ (t), x˙ ∗ (t), v)

(4.52)

and observe that Weierstrass’ Necessary Condition may be written as H(t, v) = −E(t, x∗ (t), x˙ ∗ (t), v) ≤ 0

for all v ∈ R. However, if v = u∗ (t) = x˙ ∗ (t), then using the definition of the excess function, one has that H(t, x˙ ∗ (t)) = H(t, u∗ (t)) = −E(t, x∗ (t), x˙ ∗ (t), u∗ (t)) = −{[f (t, x∗ (t), u∗ (t)) − f (t, x∗ (t), x˙ ∗ (t))] − [u∗ (t) − x˙ ∗ (t)]fu (t, x∗ (t), x˙ ∗ (t)} = −{[f (t, x∗ (t), x˙ ∗ (t)) − f (t, x∗ (t), x˙ ∗ (t))] − [x˙ ∗ (t) − x˙ ∗ (t)]fu (t, x∗ (t), x˙ ∗ (t))} = 0. Consequently, H(t, v) ≤ 0 = H(t, x˙ ∗ (t)) = H(t, u∗ (t)), for all v ∈ R, and we have the following equivalent version of Weierstrass’ Necessary Condition.

157

Chapter 4. Necessary Conditions for Local Minima

Theorem 4.7 (Weierstrass Maximum Principle) If x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ and H(t, v) = −E(t, x∗ (t), x˙ ∗ (t), v), then, (WMP-1) v = u∗ (t) = x˙ ∗ (t) maximizes H(t, v), i.e. for all t ∈ [t0 , t1 ], H(t, u∗ (t)) = H(t, x˙ ∗ (t)) = max H(t, v) = 0, v∈R

(4.53)

(WMP-2) x∗ (t0 ) = x0 , (WMP-3) x∗ (t1 ) = x1 . In addition to the above necessary conditions, one can add stronger corner conditions when x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ. In particular, observe that (4.50) and (4.51) imply that if tˆ is a corner of x∗ (·), then for any v ∈ R E(tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ), v) ≥ 0 and E(tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ), v) ≥ 0. Therefore, {[f (tˆ, x∗ (tˆ), v) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))] − [v − x˙ ∗ (tˆ+ )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))} ≥ 0

(4.54)

and {[f (tˆ, x∗ (tˆ), v) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))] − [v − x˙ ∗ (tˆ− )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))} ≥ 0

(4.55)

both hold for any v ∈ R. Setting v = x˙ ∗ (tˆ− ) in equation (4.54) and set v = x˙ ∗ (tˆ+ ) in equation (4.55) yields E + , [f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))] (4.56) ∗ ˆ− ∗ ˆ+ ∗ ∗ + −[x˙ (t ) − x˙ (t )]fu (tˆ, x (tˆ), x˙ (tˆ )) ≥ 0 and E − , [f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))] −[x˙ ∗ (tˆ+ ) − x˙ ∗ (tˆ− )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) ≥ 0,

158

Chapter 4. Necessary Conditions for Local Minima

respectively. Note that E + = −E − since − {[f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))]

− [x˙ ∗ (tˆ+ ) − x˙ ∗ (tˆ− )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))} = {[f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))]

− [x˙ ∗ (tˆ− ) − x˙ ∗ (tˆ+ )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))} = {[f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))] − [x˙ ∗ (tˆ− ) − x˙ ∗ (tˆ+ )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))},

where the last equality follows by replacing the term fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) by fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )). This step is valid because of the WeierstrassErdmann Corner Condition Theorem 3.4. In particular, we have shown that E + = −E − ≤ 0 and from inequality (4.56) E + ≥ 0 so that E + = 0 = −E − = 0 = E − . In particular, [f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ ))] − [x˙ ∗ (tˆ− ) − x˙ ∗ (tˆ+ )]fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) = [f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))]

− [f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− ))].

Rearranging terms and again using the fact that fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) = fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) yields 2f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − 2x˙ ∗ (tˆ− )fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) = 2f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − 2x˙ ∗ (tˆ+ )fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )).

Dividing both sides of this expression by 2 provides a proof of the following result. Theorem 4.8 (Second Corner Condition of Erdmann) If x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ and tˆ is a corner, then f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) − x˙ ∗ (tˆ+ ) · fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ+ )) = f (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) − x˙ ∗ (tˆ− ) · fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )).

Chapter 4. Necessary Conditions for Local Minima

159

The remainder of this section is devoted to proving the Weierstrass Necessary Condition Theorem 4.6.

4.5.1

Proof of the Weierstrass Necessary Condition

Assume that x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ. Therefore, there is a δ > 0 such that J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s))ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s))ds,

t0

=

(4.57)

t0

for all x (·) ∈ U0 (x∗ (·), δ) ∩ Θ. Recall that strong local minimizers are also weak local minimizers so that Euler’s Necessary Condition must hold. In particular, there is a constant c such that [fu (t, x∗ (t), x˙ ∗ (t))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds,

t0

and, between corners, d [fu (t, x∗ (t), x˙ ∗ (t))] = [fx (t, x∗ (t), x˙ ∗ (t))] . dt Let t0 < tˆ1 < tˆ2 < ··· < tˆp < t1 be the corners of x∗ (·). On each subinterval (tˆi , tˆi+1 ) the minimizer x∗ (·) and x˙ ∗ (·) are continuous (see Figure 4.3). Select a subinterval (tˆj , tˆj+1 ) and let z be any point in (tˆj , tˆj+1 ). In particular, z is not a corner and there is a ρ > 0 such that tˆj < z < z + ρ < tˆj+1 .

160

Chapter 4. Necessary Conditions for Local Minima

t0

tˆj 1 t1

tˆj

x* () Figure 4.3: The Strong Local Minimizer

Note that if α is any number satisfying z < α ≤ z + ρ, then [z, α] ⊂ (tˆj , tˆj+1 ), so that x∗ (·) is smooth on [z, α]. In particular, the derivative x˙ ∗ (t) is continuous on [z, z + ρ] so there exists a bound M = M (ρ) ≥ 0 such that if z ≤ t ≤ z + ρ, then |x˙ ∗ (t)| ≤ |M (ρ)| . ˆ =M ˆ (v, ρ) = |v| + M (ρ) and let α = α(δ, v) be a number Let M satisfying z < α ≤ z + ρ and ˆ · |α − z| < δ. M

(4.58)

Observe that if z ≤ ε ≤ α, then ˆ · |α − z| < δ. |v| · |ε − z| < M

(4.59)

The Mean Value Theorem implies that if z ≤ t ≤ z + ρ, then there is a tˆ ∈ (z, t) such that x∗ (t) − x∗ (z) = x˙ ∗ (tˆ)(t − z) and hence ˆ · |α − z| < δ. |x∗ (t) − x∗ (z)| ≤ x˙ ∗ (tˆ) · |t − z| < M (4.60)

We shall use these inequalities to prove the results.

Chapter 4. Necessary Conditions for Local Minima

161

For v ∈ R and α as above with z ≤ ε ≤ α, define the functions xv (t) = x∗ (z) + v(t − z), and θ(t, ε) = x∗ (t) + [xv (ε) − x∗ (ε)]

(α − t) , (α − ε)

respectively. Using these functions we construct a “variation” ϕ (·, ε) of x∗ (t) by  ∗ x (t), t0 ≤ t < z,    xv (t), z ≤ t ≤ ε, . ϕ (t, ε) = θ (t, ε) , ε ≤ t ≤ α,    ∗ x (t), α ≤ t ≤ t1 .

Figure 4.4 provides a plot of ϕ (·, ε) in terms of its pieces. It is clear that for each z ≤ ε ≤ α, the variation ϕ (·, ε) is piecewise smooth, ϕ (t0 , ε) = x0 , and ϕ (t1 , ε) = x1 . Thus, ϕ (·, ε) ∈ Θ. Also,

 (t ,  )

xv (t )

t0

tˆj

z 

 tˆj 1 t1 x* ()

Figure 4.4: Defining the Variation: Step One

162

Chapter 4. Necessary Conditions for Local Minima

observe that

 0, t0 ≤ t < z,    ∗ |x (t) − xv (t)|, z ≤ t ≤ ε |x∗ (t) − ϕ(t, ε)| = ∗ |x (t) − θ (t, ε) |, ε < t ≤ α    0, α ≤ t ≤ t1 ,  0, t0 ≤ t < z,    |x∗ (t) − [x∗ (z) + v(t − z)]|, z ≤ t ≤ ε   (α−t) |x∗ (t) − {x∗ (t) + [xv (ε) − x∗ (ε)] (α−ε) }|, =    ε z such that z < α ≤ α0 and d0 (x∗ (·), ϕ(·, ε)) ≤ |v| · |α − z| < δ/2. Consider the error in the derivatives given by  0, t0 ≤ t < z,    d ∗ ∂ϕ(t, ε) | x (t) − dtd xv (t)|, z ≤ t ≤ ε dt |= |x˙ ∗ (t) − | d x∗ (t) − dtd θ (t, ε) |, ε < t ≤ α  ∂t   dt 0, α ≤ t ≤ t1 ,  0, t0 ≤ t < z,    | d x∗ (t) − d [x∗ (z) + v(t − z)]|, z ≤ t ≤ ε   dt dt (α−t) }|, | dtd x∗ (t) − dtd {x∗ (t) + [xv (ε) − x∗ (ε)] (α−ε) =    ε 0 such that (RW-1) E(t, x∗ (t), x˙ ∗ (t), v) ≥ 0 for all t ∈ [t0 , t1 ] and v satisfying |x˙ ∗ (t) − v| < ρ,

(4.77)

and

(RW-2) x∗ (t0 ) = x0 , (RW-3) x∗ (t1 ) = x1 . Using this restricted form of the Weierstrass Necessary Condition, one can prove the Legendre Necessary Condition without using second order variations. For completeness we provide this proof below.

4.5.3

A Proof of Legendre’s Necessary Condition

Assume x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ. If (4.28) does not hold, then there is a fixed zˆ ∈ [t0 , t1 ] such that fuu (ˆ z , x∗ (ˆ z ), x˙ ∗ (ˆ z )) < 0.

We consider the case where zˆ ∈ (t0 , t1 ) is not a corner. The case at a corner is treated the same except one works to the left or to the right. Since the function Ψ(u) , fuu (ˆ z , x∗ (ˆ z ), x˙ ∗ (ˆ z ) + u)

177

Chapter 4. Necessary Conditions for Local Minima is continuous, there is a ρ1 > 0 such that Ψ(u) , fuu (ˆ z , x∗ (ˆ z ), x˙ ∗ (ˆ z ) + u) < 0

for all u ∈ [−ρ1 , ρ1 ]. However, applying Theorem 4.9, there is a ρ > 0 such that if |v − x˙ ∗ (ˆ z )| < ρ, then E(ˆ z , x∗ (ˆ z ), x˙ ∗ (ˆ z ), v) ≥ 0. Let γ(v) be the function γ(v) = f (ˆ z , x∗ (ˆ z ), v), and select vˆ such that 0 < |ˆ v − x˙ ∗ (ˆ z )| < min{ρ, ρ1 }. Note that γ(·) is twice continuously differentiable and hence by Taylor’s Theorem ˆ 0 0, so we can apply the Jacobi Necessary Condition - (IV). Since f (t, x, u) = u3 , it follows that fxu (t, x∗ (t), x˙ ∗ (t)) = 0, fxx (t, x∗ (t), x˙ ∗ (t)) = 0 and fuu (t, x∗ (t), x˙ ∗ (t)) = 6x˙ ∗ (t) = 6. Hence, Jacobi’s Equation is given by d [0 · η(t) + 6η(t)] ˙ = [0 · η(t) + 0 · η(t)] ˙ = 0, dt or equivalently, η¨(t) = 0. Thus, all secondary extremals have the form η(t) = pt + q,

180

Chapter 4. Necessary Conditions for Local Minima

and if there is a value 0 < tˆc < 1 such that η(0) = q = 0 and η(tˆc ) = ptˆc + q = 0, then η(t) ≡ 0

for all t ∈ R. Hence there are no values conjugate to t0 = 0 and Jacobi’s Necessary Condition - (IV) is satisfied. Summarizing, we know that x∗ (t) = t satisfies all the necessary conditions for a weak local minimum, but fails to satisfy Weierstrass’ Necessary Condition. All we can say at this point is that x∗ (t) = t is not a strong local minimizer.

4.7

Problem Set for Chapter 4

Consider the Simplest Problem in the Calculus of Variations (SPCV): Find x∗ (·) to minimize the cost function Z t1 f (s, x(s), x(s))ds, ˙ J(x(·)) = t0

subject to x(t0 ) = x0 , x(t1 ) = x1 . Use the four necessary conditions to completely analyze the problems below. State exactly what you can say about each problem at this point. Be sure to distinguish between weak, strong and global minimizers when possible. Problem 4.1 Minimize the functional J(x(·)) =

R1

x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Problem 4.2 Minimize the functional J(x(·)) =

R1

x (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1.

181

Chapter 4. Necessary Conditions for Local Minima Problem 4.3 Minimize the functional J(x(·)) =

R1

sx (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Rb Problem 4.4 Minimize the functional J(x(·)) = [x˙ (s)]3 ds, 0

subject to the endpoint conditions x (0) = 0 and x (b) = x1 . Problem 4.5 Minimize the functional J(x(·)) =

Z1

{[x˙ (s)]2 + [x (s)]2 + 2es x (s)}ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = e/2. Problem 4.6 Minimize the functional J(x(·)) =

R2

s−3 [x˙ (s)]2 ds,

1

subject to the endpoint conditions x (1) = 1 and x (2) = 16. Problem 4.7 Minimize the functional J(x(·)) =

Z4

[x˙ (s) − 1]2 × [x˙ (s) + 1]2 ds,

0

subject to the endpoint conditions x (0) = 0 and x (4) = 2. Problem 4.8 Minimize the functional Zπ/2 J(x(·)) = {[x˙ (s)]2 − [x (s)]2 }ds, 0

subject to the endpoint conditions x (0) = 0 and x (π/2) = 0. Problem 4.9 Minimize the functional J(x(·)) =

Zπ

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0.

182

Chapter 4. Necessary Conditions for Local Minima

Problem 4.10 Minimize the functional 3π/2 Z

{[x˙ (s)]2 − [x (s)]2 }ds,

J(x(·)) =

0

subject to the endpoint conditions x (0) = 0 and x (3π/2) = 0. Problem 4.11 Minimize the functional J(x(·)) =

Zb 0

x(s)

p

2 ds, 1 + [x(s)] ˙

subject to the endpoint conditions x (0) = 1 and x (b) = 2. Problem 4.12 Minimize the functional J(x(·)) =

Rb q 1+[x(s)] 2 ˙ 0

2gx(s)

subject to the endpoint conditions x (0) = 1 and x (b) = 0. Problem 4.13 Minimize the functional J(x(·)) =

Z2

2 {[x(s)] ˙ − 2sx(s)}ds,

1

subject to the endpoint conditions x (1) = 0 and x (2) = −1. Problem 4.14 Minimize the functional J(x(·)) =

Zπ

2 {[x(s)]2 (1 − [x(s)] ˙ )}ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0. Problem 4.15 Minimize the functional J(x(·)) =

Z3

{[3s − x(s)]x(s)}ds,

1

subject to the endpoint conditions x (1) = 1 and x (3) = 9/2.

ds,

Chapter 4. Necessary Conditions for Local Minima

183

Problem 4.16 Minimize the functional J(x(·)) = 4πρv 2

ZL

3 {[x(s)] ˙ x(s)}ds,

0

subject to the endpoint conditions x (0) = 1 and x (L) = R. Here, ρ, v 2 , L > 0 and R > 0 are all constants. Problem 4.17 Minimize the functional J(x(·)) =

Z2

{x(s)[1 ˙ + s2 x(s)]}ds, ˙

1

subject to the endpoint conditions x (1) = 3 and x (2) = 5.

Advanced Problems Problem 4.18 Consider the problem of minimizing the functional Z1  [x˙ (s) + 1]2 [x˙ (s) − 1]2 ds ≥ 0, J(x(·)) = −1

with x(−1) = x(1) = 0. Show that there is a sequence of functions xN (·) such that J(xN (·)) −→ 0

and d0 (xN (·), 0) → 0, but x0 (t) = 0 is not a minimizer of J(·).

Problem 4.19 Consider the problem of minimizing the functional J(x(·)) =

Z1 0



2 2 [x(s)]2 + [x(s)] ˙ (1 − [x(s)] ˙ ) ds,

with x(0) = x(1) = 0. Show that x0 (t) = 0 is a weak local minimizer, but not a strong local minimizer.

Chapter 5 Sufficient Conditions for the Simplest Problem Although it is difficult to derive “useful” sufficient conditions, there are some results that combine the necessary conditions. We shall not spend much time on these conditions. However, there is merit in looking at some of the techniques used to derive such conditions. First recall Hilbert’s form of Euler’s Equations. Observe that Hilbert’s Differentiability Theorem 3.6 implies that smooth extremals x(·) have continuous second derivatives. Therefore, we may again differentiate Euler’s Differential Equation d [fu (t, x(t), x˙ (t))] = [fx (t, x(t), x˙ (t))] , dt

(5.1)

and apply the chain rule to obtain Hilbert’s (second order) differential equation x¨ (t) · [fuu (t, x(t), x˙ (t))] = [fx (t, x(t), x˙ (t))] − [fut (t, x(t), x˙ (t))] (5.2) − [fux (t, x(t), x˙ (t))] · x˙ (t) .

185

186

5.1

Chapter 5. Sufficient Conditions

A Field of Extremals

Assume that x0 (·) is a fixed smooth extremal satisfying Hilbert’s equation (5.2). Also assume that there exists a one parameter family of solutions ϕ(·, α) to (5.2) with the following properties: (FE-i) For each parameter α in an open interval (α0 − γ, α0 + γ), the function xα (·) defined by xα (t) , ϕ(t, α) is a solution to Hilbert’s equation on the interval t0 ≤ t ≤ t1 and x0 (t) = ϕ(t, α0 ). (FE-ii) The function ϕ(t, α) and all the partial derivatives ∂ ∂ ∂2 ∂2 ϕ(t, α), ∂α ϕ(t, α), ∂t∂α ϕ(t, α), and ∂t 2 ϕ(t, α) exist and are ∂t continuous on the set [t0 , t1 ] × (α0 − γ, α0 + γ). (FE-iii) The equation x − ϕ(t, α) = 0

(5.3)

implicitly defines a function α : S −→ R on a region S in the (t, x) plane defined by S = [t0 , t1 ] × {x : x0 (t) − δ < x < x0 (t) + δ} for some δ > 0. In particular, α = α(t, x) satisfies x − ϕ(t, α(t, x)) = 0,

(5.4)

for all [t x]T ∈ S. (FE-iv) The partial derivatives are continuous on S.

∂ α(t, x) ∂t

and

∂ α(t, x) ∂x

exist and

It is helpful to think of the family of solutions ϕ(·, α) as providing a “strip” of graphs of smooth extremals about the graph of x0 (·).

187

Chapter 5. Sufficient Conditions x 2 ()   (,  2 )



S x0 ()   (,  0 )

t0

x1 ()   (, 1 )

t1

Figure 5.1: A Field of Extremals Note that (FE-i) through (FE-iv) imply that through each fixed point [tˆ xˆ]T ∈ S, there exists exactly one extremal xˆ(t) = xαˆ (t) = ϕ(t, α ˆ ) = ϕ(t, α(tˆ, xˆ)).

Let p(tˆ, xˆ) denote the slope of the unique extremal that goes through the point [tˆ xˆ]T . Note that at a specific point [tˆ xˆ]T ∈ S there is a value α ˆ = α(tˆ, xˆ) such that the value of the slope at T [tˆ xˆ] is the slope of the extremal xˆ(t) = xαˆ (t) = ϕ(t, α ˆ) at t = tˆ. Thus, it follows that

∂ ∂ ∂ ϕ(t, α ˆ ) = ϕ(tˆ, α ˆ ) = ϕ(tˆ, α(tˆ, xˆ)) p(tˆ, xˆ) = x˙ αˆ (tˆ) = ∂t ∂t ∂t t=tˆ (5.5) T ˆ and since (5.5) holds at each [t xˆ] ∈ S one has that

∂ ∂ ϕ(t, α(t, x)) = ϕ(t, α) (5.6) ∂t ∂t holds for all [t x]T ∈ S. The function p(t, x) is called the slope function on S. x˙ α (t) = p(t, x) =

188

Chapter 5. Sufficient Conditions x 2 ()   (,  2 ) p(tˆ, xˆ )

(tˆ, xˆ )



S x0 ()   (,  0 )

t0

x1 ()   (, 1 )

t1

Figure 5.2: An Extremal Through a Specified Point and the Slope Function Definition 5.1 The pair F = (S, p(·, ·)) is called a field of extremals about x0 (·). Here S is the domain of the function α(t, x), and we say that x0 (·) is embedded in F. The existence of a field of extremals will provide the basic tool for developing sufficient conditions. However, we need a few preliminary results to set the stage for the development of these results. We begin by establishing the following theorem about the slope function. Assume x0 (·) is embedded in the field F = (S, p(·, ·)) with slope function p(·, ·). Theorem 5.1 (Basic PDE for Slope) The slope function p (t, x) satisfies the partial differential equation {pt (t, x) + p(t, x)px (t, x)} · fuu (t, x, p(t, x)) = fx (t, x, p(t, x)) − p(t, x) · fux (t, x, p(t, x)) − fut (t, x, p(t, x)) for all [t x]T ∈ S.

(5.7)

189

Chapter 5. Sufficient Conditions

Proof: For each α in the open interval (α0 − γ, α0 + γ), the extremal xα (t) = ϕ(t, α) satisfies Hilbert’s Equation x¨α (t) · fuu (t, xα (t), x˙ α (t)) = fx (t, xα (t), x˙ α (t)) (5.8) − x˙ α (t) · fux (t, xα (t), x˙ α (t)) − fut (t, xα (t), x˙ α (t)).

Given any “fixed” pair [t x]T , let α = α(t, x) so that xα (t) = ϕ(t, α) = ϕ(t, α(t, x)) and 5.4 in (FE-iii) implies that x = xα (t) = ϕ(t, α) = ϕ(t, α(t, x)).

Moreover, since ϕ(t, α) = xα (t) and ϕt (t, α) = x˙ α (t), then (t, xα (t), x˙ α (t)) = (t, ϕ(t, α), p(t, x)) = (t, ϕ(t, α(t, x)), p(t, x)) = (t, x, p(t, x)). (5.9) Differentiating both sides of (5.6) p(t, x) = p(t, ϕ(t, α)) = ϕt (t, α) = x˙ α (t),

(5.10)

yields pt (t, ϕ(t, α)) + px (t, ϕ(t, α)) · ϕt (t, α) = ϕtt (t, α) = x¨α (t).

In addition, since x = ϕ(t, α), p (t, x) = ϕt (t, α) and ϕtt (t, α) = x¨α (t) it follows that {pt (t, x) + px (t, x) p (t, x)} = x¨α (t).

Multiplying both sides of this equation by fuu (t, xα (t), x˙ α (t)) = fuu (t, x, p(t, x)) and substituting (5.10) into (5.8) yields {pt (t, x) + px (t, x)p(t, x)} · fuu (t, xα (t), p(t, x)) = x¨α (t) · fuu (t, xα (t), x˙ α (t)) = fx (t, xα (t), x˙ α (t)) − x˙ α (t) · fux (t, xα (t), x˙ α (t)) − fut (t, xα (t), x˙ α (t)) = fx (t, x, p(t, x)) − p(t, x) · fux (t, x, p(t, x)) − fut (t, x, p(t, x))

190

Chapter 5. Sufficient Conditions

and this completes the proof. 

5.2

The Hilbert Integral

Assume that F = (S, p(·, ·)) is a field of extremals about x0 (·) and suppose q(t, x) is any function defined on S. Given any x(·) ∈ P W S(t0 , t1 ), the Hilbert Integral is defined by Jq∗ (x(·))

=

Zt1

{f (s, x(s), q(s, x(s)))

t0

+ [x(s) ˙ − q(s, x(s))] · fu (s, x(s), q(s, x(s)))}ds t Z1 = {f (s, x(s), q(s, x(s))) − q(s, x(s)) · fu (s, x(s), t0

q(s, x(s)))}ds +

Zt1

{

d x(s) · fu (s, x(s), q(s, x(s)))}ds. ds

t0

If one defines P(t, x) and Q(t, x) by P(t, x) = {f (t, x, q(t, x)) − q(t, x) · fu (t, x, q(t, x))},

(5.11)

and Q(t, x) = fu (t, x, q(t, x)),

(5.12)

respectively, then Jq∗ (x(·)) is a line integral given by Jq∗ (x(·)) =

Zt1

P(s, x)ds + Q(s, x)dx.

t0

We are interested in the case where q(s, x) renders the line integral Jq∗ (x(·)) independent of path. Recall from vector calculus (see Section 9.5 in [48]) that Jq∗ (x(·)) is independent of path if and

191

Chapter 5. Sufficient Conditions only if Px (t, x) = Qt (t, x). A direct calculation yields

Px (t, x) = fx (t, x, q(t, x)) + fu (t, x, q(t, x))qx (t, x) − qx (t, x)fu (t, x, q(t, x)) − q(t, x)[fux (t, x, q(t, x)) + fuu (t, x, q(t, x))qx (t, x)] = fx (t, x, q(t, x)) − q(t, x)fux (t, x, q(t, x)) − fuu (t, x, q(t, x))q(t, x)qx (t, x). Likewise, a direct calculation yields Qt (t, x) = fut (t, x, q(t, x)) + fuu (t, x, q(t, x))qt (t, x). Hence, Qt (t, x) = Px (t, x),

if and only if

fut (t, x, q(t, x)) + fuu (t, x, q(t, x))qt (t, x) = fx (t, x, q(t, x)) − q(t, x)fux (t, x, q(t, x)) − fuu (t, x, q(t, x))q(t, x)qx (t, x). Therefore, Jq∗ (x(·)) is independent of path if and only if for all [t x]T ∈ S, q (t, x) satisfies the partial differential equation {qt (t, x) + q(t, x)qx (t, x)} · fuu (t, x, q(t, x)) = fx (t, x, q(t, x)) − q(t, x) · fux (t, x, q(t, x)) − fut (t, x, q(t, x)).

(5.13)

But we know from (5.7) that the slope function satisfies this partial differential equation. Hence, if q(t, x) = p(t, x), then the Hilbert Integral Jp∗ (x(·)) =

Zt1

{f (s, x(s), p(s, x(s)))

t0

+ [x(s) ˙ − p(s, x(s))] · fu (s, x(s), p(s, x(s)))}ds, is independent of path. Thus we have established the following result.

192

Chapter 5. Sufficient Conditions

Theorem 5.2 (Hilbert’s Integral Theorem) Assume that F = (S, p(·, ·)) is a field of extremals about x0 (·) and let x1 (·) and x2 (·) ∈ P W S(t0 , t1 ) be any two functions with graphs contained in S. If x1 (t0 ) = x2 (t0 ), and x1 (t1 ) = x2 (t1 ), then Jp∗ (x1 (·)) = Jp∗ (x2 (·)).

5.3

Fundamental Sufficient Results

If there exists a field of extremals F = (S, p(·, ·)) about a smooth function x0 (·), then Hilbert’s Integral Theorem can be exploited to yield sufficient conditions. The key result relates the cost function to the Weierstrass Excess Function. Theorem 5.3 (Weierstrass-Hilbert) Suppose x0 (·) is a smooth function embedded in a field F = (S, p(·, ·)) and the graph of x(·) ∈ P W S(t0 , t1 ) is in S. If x(t0 ) = x0 (t0 ), and x(t1 ) = x0 (t1 ), then J(x(·)) − J(x0 (·)) =

Zt1

E(s, x(s), p(s, x(s)), x(s))ds. ˙

(5.14)

t0

Proof: Consider Hilbert’s integral with q(t, x) = p(t, x) . Evaluating Jp∗ (·) at x0 (·), it follows that Jp∗ (x0 (·))

=

Zt1

f (s, x0 (s), p(s, x0 (s))ds

+

Zt1

t0

t0

[x˙ 0 (s) − p(s, x0 (s))] · [fu (s, x0 (s), p(s, x0 (s))]ds

193

Chapter 5. Sufficient Conditions and since x˙ 0 (s) = p(s, x0 (s)) the Hilbert integral reduces to Jp∗ (x0 (·))

=

Zt1

f (s, x0 (s), p(s, x0 (s))ds = J(x0 (·)).

(5.15)

t0

However, Hilbert’s Integral Theorem above implies that Jp∗ (x0 (·)) = Jp∗ (x(·)).

(5.16)

Therefore it follows from (5.15) and (5.16) above that J(x(·)) − J(x0 (·)) = J(x(·)) − Jp∗ (x0 (·)) = J(x(·)) − Jp∗ (x(·)) Zt1 Zt1 ˙ − f (s, x(s), p(s, x(s))ds = f (s, x(s), x(s))ds t0

t0

−

Zt1

[x(s) ˙ − p(s, x(s))] · [fu (s, x(s), p(s, x(s))]ds

t0

=

Zt1

E(s, x(s), p(s, x(s)), x(s))ds, ˙

t0

and this completes the proof.  We now have the following fundamental sufficiency condition based on the Weierstrass excess function. Note that we need the existence of a field of extremals. Theorem 5.4 (Fundamental Sufficiency Theorem) Assume that x0 (·) is a smooth extremal embedded in a field F = (S, p(·, ·)) satisfying x0 (t0 ) = x0 and x0 (t1 ) = x1 . If x(·) ∈ Θ is any other piecewise smooth function with graph in S and if for all t ∈ [t0 , t1 ] E(t, x(t), p(t, x(t)), x(t)) ˙ ≥ 0, then J(x0 (·)) ≤ J(x(·)).

194

Chapter 5. Sufficient Conditions

In order to make use of the Fundamental Sufficiency Theorem, one needs to have the answer to the following two questions: (A) When can a smooth function x0 (·) be embedded in a field F = (S, p(·, ·))? and (B) When is E(t, x(t), p(t, x(t)), x(t)) ˙ ≥ 0? One answer to the first question is given by the following result. The proof of this result is outlined in Ewing’s book [77] and details can be found in Bliss [27] and Bolza [31]. Theorem 5.5 (Fundamental Field Theorem) If x0 (·) is smooth and satisfies (F 1) Euler’s Equation d [fu (t, x0 (t), x˙ 0 (t))] = [fx (t, x0 (t), x˙ 0 (t))] , dt

(5.17)

(F 2) the Strengthen Legendre Condition fuu (t, x0 (t), x˙ 0 (t)) > 0, t0 ≤ t ≤ t1 ,

(5.18)

(F 3) the Strengthen Jacobi Condition that there is no value tˆc conjugate to t0 satisfying tˆc ≤ t1 ,

(5.19)

then there exists a field of extremals F = (S, p(·, ·)) about x0 (·). The following result connects the Weierstrass and Legendre conditions and is the key to obtaining the basic sufficient conditions. Theorem 5.6 (Excess Function Expansion) If x0 (·) is embedded in a field with slope function p(t, x), then for each v ∈ R and [t x]T ∈ S there is a function θ = θ(t, x, v) such that 0 < θ(t, x, v) < 1 and E(t, x, p(t, x), v) = 1/2[v − p(t, x)]2 fuu (t, x, p(t, x) + θ(t, x, v)[v − p(t, x)]). (5.20)

Chapter 5. Sufficient Conditions

195

Proof : Assume x0 (·) is embedded in a field of extremals F = (S, p(·, ·)) about x0 (·) with slope function p(t, x). For each [tˆ xˆ]T ∈ S define the function r(v) = f (tˆ, xˆ, v) so that

d r(v) = r′ (v) = fu (tˆ, xˆ, v), dv

and

d2 r(v) = r′′ (v) = fuu (tˆ, xˆ, v). dv 2 Let pˆ = p(tˆ, xˆ). Taylor’s theorem with remainder implies for any v that there is a θ = θ(tˆ, xˆ, v), with 0 < θ < 1 such that r(v) = r(ˆ p) + [v − pˆ]r′ (ˆ p) + 1/2[v − pˆ]2 [r′′ (ˆ p + θ[v − pˆ])] = f (tˆ, xˆ, pˆ) + [v − pˆ][fu (tˆ, xˆ, pˆ)] + 1/2[v − pˆ]2 [fuu (tˆ, xˆ, pˆ + θ[v − pˆ])],

which implies f (tˆ, xˆ, v) = f (tˆ, xˆ, pˆ) + [v − pˆ][fu (tˆ, xˆ, pˆ)] + 1/2[v − pˆ]2 [fuu (tˆ, xˆ, pˆ + θ[v − pˆ])], or equivalently, f (tˆ, xˆ, v) − f (tˆ, xˆ, pˆ) − [v − pˆ][fu (tˆ, xˆ, pˆ)] = 1/2[v − pˆ]2 [fuu (tˆ, xˆ, pˆ + θ[v − pˆ])]. The previous equality implies that E(tˆ, xˆ, pˆ, v) = 1/2[v − pˆ]2 fuu (tˆ, xˆ, pˆ + θ[v − pˆ]).

In particular, since [tˆ xˆ]T ∈ S is arbitrary, it follows that the function θ = θ(t, x, v) exists for all [tˆ xˆ]T ∈ S, 0 < θ(t, x, v) < 1 and E(t, x, p(t, x), v) = 1/2[v − p(t, x)]2 fuu (t, x, p(t, x) + θ(t, x, v)[v − p(t, x)]). (5.21) This completes the proof and leads to the first sufficient condition for a strong local minimum. 

196

Chapter 5. Sufficient Conditions

Theorem 5.7 (Sufficient Condition (1)) If the problem is regular (i.e. fuu (t, x, u) > 0 for all (t, x, u)), x0 (·) ∈ Θ is smooth, and satisfies (S1–1) Euler’s Differential Equation d [fu (t, x0 (t), x˙ 0 (t))] = [fx (t, x0 (t), x˙ 0 (t))] , dt (S1–2) the Strengthen Jacobi Condition that there is no value tˆc conjugate to t0 satisfying tˆc ≤ t1 , then x0 (·) provides a strong local minimum for J(·) on Θ. Proof: By Theorem 5.5 it follows that the smooth function x0 (·) can be embedded in a field F = (S, p(·, ·)), x0 (t0 ) = x0 and x0 (t1 ) = x1 . Also, there is a δ > 0 such that if x(·) is any piecewise smooth function satisfying x (t0 ) = x0 , x (t1 ) = x1 and d0 (x0 (·), x(·)) < δ, then x(·) ∈ Θ has its graph in S. Thus, (5.20) implies that there exists a θ(t) = θ(t, x(t), x(t)) ˙ with 0 < θ(t) < 1 such that E(t, x(t), p(t, x(t)), x(t)) ˙ = 1/2[x(t) ˙ − p(t, x(t))]2 · [fuu (t, x(t), p(t, x(t)) + θ(t)[x(t) ˙ − p(t, x(t))])] ≥0 for all t ∈ [t0 , t1 ] and the result follows from the Fundamental Sufficiency Condition above.  We also have the following sufficient condition for a weak local minimum. The proof is similar to the proof to Theorem 5.7 above and is left as an exercise. Theorem 5.8 (Sufficient Condition (2)) If x0 (·) smooth, and satisfies (S2 − 1) Euler’s Equation d [fu (t, x0 (t), x˙ 0 (t))] = [fx (t, x0 (t), x˙ 0 (t))] , dt

∈

Θ is

197

Chapter 5. Sufficient Conditions (S2 − 2) the Strengthen Legendre Condition fuu (t, x0 (t), x˙ 0 (t)) > 0, t0 ≤ t ≤ t1 ,

(S2 − 3) the Strengthen Jacobi Condition that there is no value tˆc conjugate to t0 satisfying tˆc ≤ t1 , then x0 (·) provides a weak local minimum for J(·) on Θ.

5.4

Problem Set for Chapter 5

Consider the Simplest Problem in the Calculus of Variations (SPCV): Find x∗ (·) to minimize the cost function Z t1 f (s, x(s), x(s))ds, ˙ J(x(·)) = t0

subject to x(t0 ) = x0 , x(t1 ) = x1 . Determine if the sufficient conditions above are helpful in analyzing the following problems. Be sure to distinguish between weak, strong and global minimizers. Problem 5.1 Minimize the functional J(x(·)) =

R1

x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Problem 5.2 Minimize the functional J(x(·)) =

R1

x (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Problem 5.3 Minimize the functional J(x(·)) =

R1

sx (s) x˙ (s) ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = 1.

198

Chapter 5. Sufficient Conditions

Rb Problem 5.4 Minimize the functional J(x(·)) = [x˙ (s)]3 ds, 0

subject to the endpoint conditions x (0) = 0 and x (b) = x1 . Problem 5.5 Minimize the functional J(x(·)) =

Z1

{[x˙ (s)]2 + [x (s)]2 + 2es x (s)}ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) = e/2. Problem 5.6 Minimize the functional J(x(·)) =

Z2

s−3 [x˙ (s)]2 ds,

1

subject to the endpoint conditions x (1) = 1 and x (2) = 16. Problem 5.7 Minimize the functional J(x(·)) =

Z4

[x˙ (s) − 1]2 [x˙ (s) + 1]2 ds,

0

subject to the endpoint conditions x (0) = 0 and x (4) = 2. Problem 5.8 Minimize the functional Zπ/2 J(x(·)) = {[x˙ (s)]2 − [x (s)]2 }ds, 0

subject to the endpoint conditions x (0) = 0 and x (π/2) = 0. Problem 5.9 Minimize the functional J(x(·)) =

Zπ

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0.

199

Chapter 5. Sufficient Conditions Problem 5.10 Minimize the functional 3π/2 Z

{[x˙ (s)]2 − [x (s)]2 }ds,

J(x(·)) =

0

subject to the endpoint conditions x (0) = 0 and x (3π/2) = 0. Problem 5.11 Minimize the functional J(x(·)) =

Zb 0

x(s)

p 2 ds, 1 + [x(s)] ˙

subject to the endpoint conditions x (0) = 1 and x (b) = 2. Problem 5.12 Minimize the functional J(x(·)) =

Rb q 1+[x(s)] 2 ˙ 0

2gx(s)

subject to the endpoint conditions x (0) = 1 and x (b) = 0. Problem 5.13 Minimize the functional J(x(·)) =

Z2

2 {[x(s)] ˙ − 2sx(s)}ds,

1

subject to the endpoint conditions x (1) = 0 and x (2) = −1. Problem 5.14 Minimize the functional J(x(·)) =

Zπ

2 {[x(s)]2 (1 − [x(s)] ˙ )}ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) = 0. Problem 5.15 Minimize the functional J(x(·)) =

Z3

{[3s − x(s)]x(s)}ds,

1

subject to the endpoint conditions x (1) = 1 and x (3) = 9/2.

ds,

200

Chapter 5. Sufficient Conditions

Problem 5.16 Minimize the functional J(x(·)) = 4πρv

2

ZL 0



3 [x(s)] ˙ x(s) ds,

subject to the endpoint conditions x (0) = 1 and x (L) = R. Here, ρ, v 2 , L > 0 and R > 0 are all constants. Problem 5.17 Minimize the functional J(x(·)) =

Z2

{x(s)[1 ˙ + s2 x(s)]}ds, ˙

1

subject to the endpoint conditions x (1) = 3 and x (2) = 5. Advanced Problems Problem 5.18 Prove the Fundamental Field Theorem 5.5. Problem 5.19 Consider the functional J(x(·)) =

Z2

3 {[x(s)] ˙ + [sin(s)]2 }ds,

0

subject to the endpoint conditions x (0) = 1 and x (2) = 1. Find a field of extremals about the extremal x0 (t) = 1. Problem 5.20 Consider the functional J(x(·)) =

Z2

3 {[x(s)] ˙ + [sin(s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (2) = 4. Find a field of extremals about the extremal x0 (t) = 2t.

Chapter 5. Sufficient Conditions

201

Problem 5.21 Consider the functional J(x(·)) =

Z1

1 ˙ {x(s)[2t ˙ − x(s)]}ds, 2

−1

subject to the endpoint conditions x (−1) = 0 and x (1) = 12 . Find a field of extremals about the extremal x0 (t) = t2 + 14 t − 43 . Problem 5.22 Prove the Sufficient Condition (2) as stated in Theorem 5.8. Problem 5.23 Consider the problem of minimizing Z T 2 {[x(s)] ˙ − [x(s)]2 }ds, 0

subject to x(0) = 0, x(T ) = 0. [(a)] For which T is it the case that x(t) ≡ 0 is a strong local minimizer? [(b)] How does your analysis extend to the more general problem of minimizing Z T 2 {p(s)[x(s)] ˙ + q(s)[x(s)]2 }ds, 0

subject to x(0) = 0 and x(T ) = 0. Here, p(·) and q(·) are real valued smooth functions defined on [0, ∞) with p(t) > 0.

Chapter 6 Summary for the Simplest Problem This is a summary of the definitions and fundamental results for the Simplest Problem in the Calculus of Variations. Recall that f (t, x, u), t0 , t1 , x0 and x1 are given and we assume that f (t, x, u) belongs to C 2 . Let X = P W S(t0 , t1 ) denote the space of all real-valued piecewise smooth functions defined on [t0 , t1 ]. For each PWS function x : [t0 , t1 ] → R, define the functional J : X → R by J(x(·)) =

Zt1

f (s, x(s), x˙ (s)) ds.

(6.1)

t0

Assume that the points [t0 x0 ]T and [t1 x1 ]T are given and define the set of PWS functions Θ by Θ = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 , x (t1 ) = x1 } . Observe that J : X → R is a real valued function on X. ———————————————————————–

203

(6.2)

204

Chapter 6. Summary for the Simplest Problem

The Simplest Problem in the Calculus of Variations: Find x∗ (·) ∈ Θ such that J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s))ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s))ds,

t0

=

t0

for all x (·) ∈ Θ. ———————————————————————– If x(·) and z(·) ∈ P W S(t0 , t1 ), then d0 (x(·), z(·)) , sup {|x(s) − z(s)|}

(6.3)

t0 ≤s≤t1

defines the d0 distance between x(·) and z(·). Given xˆ(·) ∈ P W S(t0 , t1 ) and δ > 0, the U0 (ˆ x(·), δ)-neighborhood of xˆ(·) is defined to be the open ball U0 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d0 (ˆ x(·), x(·)) < δ} . ———————————————————————– For x(·) and z(·) ∈ P W S(t0 , t1 ), there is a (finite) partition of [t0 , t1 ], say t0 = tˆ0 < tˆ1 < tˆ2 < · · · < tˆp−1 < tˆp = t1 , such that x˙ (t) and z˙ (t) exist and
are continuous (and bounded) on each open subinterval tˆi−1 , tˆi . The d1 distance between x(·) and z(·) is defined by d1 (x(·), z(·)) , sup {|x(t) − z(t)|} + t0 ≤t≤t1

sup t0 ≤s≤t1 , s6=tˆi

{|x(t) ˙ − z(t)|} ˙ (6.4)

= d0 (x(·), z(·)) +

sup t0 ≤t≤t1 , s6=tˆi

{|x(t) ˙ − z(t)|}. ˙

The U1 (ˆ x(·), δ)-neighborhood of xˆ(·) is defined to be the open ball U1 (ˆ x(·), δ) = {x(·) ∈ P W S(t0 , t1 ) : d1 (ˆ x(·), x(·)) < δ} .

205

Chapter 6. Summary for the Simplest Problem ———————————————————————–

If x∗ (·) ∈ Θ satisfies J(x∗ (·)) ≤ J(x(·)) for all x (·) ∈ Θ, then x∗ (·) is called a global minimizer for J(·) on Θ. If there is a δ > 0 and a x∗ (·) ∈ Θ, such that J(x∗ (·)) ≤ J(x(·)), for all x (·) ∈ U0 (x∗ (·), δ) ∩ Θ, then x∗ (·) is called a strong local minimizer for J(·) on Θ. Similarly, if there is a δ > 0 and a x∗ (·) ∈ Θ, such that J(x∗ (·)) ≤ J(x(·)), for all x (·) ∈ U1 (x∗ (·), δ) ∩ Θ, then x∗ (·) is called a weak local minimizer for J(·) on Θ. ———————————————————————– Theorem 6.1 (Euler Necessary Condition - (I)) If x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then, (E-1) there is a constant c such that for all t ∈ [t0 , t1 ], ∗

∗

[fu (t, x (t), x˙ (t))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds,

(6.5)

t0

(E-2) x∗ (t0 ) = x0 , (E-3) x∗ (t1 ) = x1 . (E-4) Between corners of x∗ (·) the function fu (t, x∗ (t), x˙ ∗ (t)) is differentiable and if t is not a corner of x∗ (·), then d [fu (t, x∗ (t), x˙ ∗ (t))] = [fx (t, x∗ (t), x˙ ∗ (t))] . dt

(6.6)

———————————————————————– Any piecewise smooth function x(·) satisfying (6.5) is called an extremal. The Euler Necessary Condition 6.1 implies that any local minimizer of J(·) on Θ is an extremal. Observe that extremals do not have to satisfy the boundary conditions.

206

Chapter 6. Summary for the Simplest Problem

If fuu (t, x, u) 6= 0 for all (t, x, u), t0 ≤ t ≤ t1 , then the integrand f (t, x, u) is called non-singular. If fuu (t, x, u) > 0 for all (t, x, u), then the integrand f (t, x, u) is said to be regular and the SPCV is called a regular problem. If x(·) ∈ P W S(t0 , t1 ) is an extremal, then x(·) is called a non-singular extremal if at all points t ∈ [t0 , t1 ] where x(t) ˙ is defined, fuu (t, x(t), x(t)) ˙ 6= 0. If x(·) ∈ P W S(t0 , t1 ) is an extremal, then x(·) is called a regular extremal if at all points t ∈ [t0 , t1 ] where x(t) ˙ is defined, fuu (t, x(t), x(t)) ˙ > 0. ———————————————————————– Weierstrass-Erdmann Corner Condition. If x(·) P W S(t0 , t1 ) is an extremal, then fu (tˆ, x(tˆ), x( ˙ tˆ+ )) = fu (tˆ, x(tˆ), x( ˙ tˆ− ))

∈

(6.7)

for all tˆ ∈ (t0 , t1 ). ———————————————————————– Theorem 6.2 (Hilbert’s Differentiability Theorem) If x(·) ∈ P W S(t0 , t1 ) is an extremal, tˆ is not a corner of x(·), and fuu (tˆ, x(tˆ), x( ˙ tˆ)) 6= 0, then there exists a δ > 0 such that x(·) has a continuous second derivative for all t ∈ (tˆ − δ, tˆ + δ) and [fuu (t, x(t), x˙ (t))] · x¨ (t) + [fux (t, x(t), x˙ (t))] · x˙ (t) + [fut (t, x(t), x(t))] ˙ = [fx (t, x(t), x˙ (t))] .

(6.8)

If in addition, f (t, x, u) is of class C p , p ≥ 2, then any nonsingular extremal x(·) is also of class C p . ———————————————————————– The Weierstrass Excess Function is defined by E(t, x, u, v) = [f (t, x, v) − f (t, x, u)] − [v − u]fu (t, x, u) for all (t, x, u, v) ∈ [t0 , t1 ] × R3 .

(6.9)

207

Chapter 6. Summary for the Simplest Problem ———————————————————————–

Theorem 6.3 (Weierstrass Necessary Condition - (II)) If x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ, then, (W-1) E(t, x∗ (t), x˙ ∗ (t), v) ≥ 0 for all t ∈ [t0 , t1 ] and v ∈ R, (W-2) x∗ (t0 ) = x0 , (W-3) x∗ (t1 ) = x1 . Condition (W-1 ) E(t, x∗ (t), x˙ ∗ (t), v) ≥ 0,

(6.10)

is the essential new information in Weierstrass’ Necessary Condition. Moreover, (6.10) holds at all t ∈ [t0 , t1 ], including corners. In particular, for all v ∈ R E(t, x∗ (t), x˙ ∗ (t+ ), v) ≥ 0,

(6.11)

E(t, x∗ (t), x˙ ∗ (t− ), v) ≥ 0.

(6.12)

and ———————————————————————– Assume that x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ and define H(t, v) , −E(t, x∗ (t), x˙ ∗ (t), v),

(6.13)

and note that Weierstrass’ Necessary Condition may be written as H(t, v) = −E(t, x∗ (t), x˙ ∗ (t), v) ≤ 0 for all v ∈ R. However, if v = u∗ (t) = x˙ ∗ (t), then using the

208

Chapter 6. Summary for the Simplest Problem

definition of the excess function, one has that H(t, x˙ ∗ (t)) = H(t, u∗ (t)) = −E(t, x∗ (t), x˙ ∗ (t), u∗ (t)) = −{[f (t, x∗ (t), u∗ (t)) − f (t, x∗ (t), x˙ ∗ (t))] − [u∗ (t) − x˙ ∗ (t)]fu (t, x∗ (t), x˙ ∗ (t)} = −{[f (t, x∗ (t), x˙ ∗ (t)) − f (t, x∗ (t), x˙ ∗ (t))] − [x˙ ∗ (t) − x˙ ∗ (t)]fu (t, x∗ (t), x˙ ∗ (t))} = 0. Consequently, H(t, v) ≤ 0 = H(t, x˙ ∗ (t)) = H(t, u∗ (t)), for all v ∈ R, and we have the following equivalent version of Weierstrass’ Necessary Condition. Theorem 6.4 (Weierstrass Maximum Principle) If x∗ (·) ∈ Θ provides a strong local minimum for J(·) on Θ, then, (WMP-1) v = u∗ (t) = x˙ ∗ (t) maximizes H(t, v) (WMP-2) x∗ (t0 ) = x0 , (WMP-3) x∗ (t1 ) = x1 . In particular, H(t, u∗ (t)) = H(t, x˙ ∗ (t)) = max H(t, v) = 0 v∈R

(6.14)

for all t ∈ [t0 , t1 ]. ———————————————————————– Theorem 6.5 (Legendre Necessary Condition - (III)) If x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, then, (L-1) fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0, for all t ∈ [t0 , t1 ],

209

Chapter 6. Summary for the Simplest Problem (L-2) x∗ (t0 ) = x0 , (L-3) x∗ (t1 ) = x1 .

It is important to again note that condition (L − 1) holds at corners. In particular, fuu (t, x∗ (t), x˙ ∗ (t)) ≥ 0 implies that fuu t, x∗ (t), x˙ ∗ t+ for all t ∈ [t0 , t1 ), and fuu t, x∗ (t), x˙ ∗ t− for all t ∈ (t0 , t1 ].





≥0





≥0

(6.15)

———————————————————————– If x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ, ∗ ∗ ∗ then define the functions fxx (t), fxu (t), and fuu (t) by ∗ fxx (t) = fxx (t, x∗ (t), x˙ ∗ (t)),

∗ fxu (t) = fxu (t, x∗ (t), x˙ ∗ (t)),

and ∗ fuu (t) = fuu (t, x∗ (t), x˙ ∗ (t)),

respectively. Also, define the function F(t, η, ξ) by 1 ∗ ∗ ∗ F(t, η, ξ) = [fxx (t)η 2 + 2fxu (t)ηξ + fuu (t)ξ 2 ], 2

(6.16)

and consider the functional J : P W S(t0 , t1 ) −→ R given by J (η(·)) =

Zt1

F(s, η(s), η(s))ds. ˙

(6.17)

t0

Let ΘS ⊂ P W S(t0 , t1 ) be defined by ΘS = {η(·) ∈ P W S(t0 , t1 ) : η (t0 ) = 0, η (t1 ) = 0} = V0 ,

(6.18)

210

Chapter 6. Summary for the Simplest Problem

and consider the so called Accessory (Secondary) Minimum Problem: Find η ∗ (·) ∈ ΘS , such that Zt1 Zt1 ∗ ∗ ˙ J (η (·)) = F(s, η (s), η˙ (s))ds ≤ J (η(·)) = F(s, η(s), η(s))ds, ∗

t0

t0

for all η (·) ∈ ΘS . If η ∗ (·) ∈ ΘS is any minimizer of J (η(·)) on ΘS , then there is a constant c such that η ∗ (·) satisfies Jacobi’s Integral Equation ∗

∗

[Fξ (t, η (t), η˙ (t))] = c +

Zt

[Fη (s, η ∗ (s), η˙ ∗ (s))] ds.

(6.19)

t0

In addition, between corners the function Fξ (t, η ∗ (t), η˙ ∗ (t)) is differentiable and d [Fξ (t, η ∗ (t), η˙ ∗ (t))] = [Fη (t, η ∗ (t), η˙ ∗ (t))] . dt

(6.20)

Observe that Jacobi’s Differential Equation is Euler’s Differential Equation for the case where f (t, x, u) is replaced by F(t, η, ξ). In particular, Jacobi’s Differential Equation (6.20) has the form d ∗ ∗ ∗ ∗ [f (t)η(t) + fuu (t)η(t)] ˙ = [fxx (t)η(t) + fxu (t)η(t)]. ˙ dt xu

(6.21)

A PWS function η(·) satisfying Jacobi’s equation (6.19) (or (6.20)) is called a secondary extremal. ———————————————————————– A value tˆc is said to be a conjugate value to t0 , if t0 < tˆc , and there is a solution ηc (·) to Jacobi’s Equation (6.19) (or (6.20)) satisfying (i) ηc (t0 ) = ηc (tˆc ) = 0 and ηc (t) 6= 0, for some t ∈ (t0 , tˆc ). In particular, ηc (·) does not vanish on (t0 , tˆc ). The point [tˆc x∗ (tˆc )]T ∈ R2 on the graph of x∗ (·) is said to be a conjugate point to the initial point [t0 x∗ (t0 )]T inR2 . ———————————————————————–

211

Chapter 6. Summary for the Simplest Problem

c (t )

t0

conjugate values

tˆc

tc

t1

Figure 6.1: Definition of a Conjugate Value

Theorem 6.6 (Jacobi’s Necessary Condition - (IV)) Assume that x∗ (·) ∈ Θ provides a weak local minimum for J(·) on Θ. If x∗ (·) is smooth and regular (i.e. fuu (t, x∗ (t), x˙ ∗ (t)) > 0), then (J-1) there can not be a value tc conjugate to t0 with t c < t1 ,

(6.22)

(J-2) x∗ (t0 ) = x0 , (J-3) x∗ (t1 ) = x1 . ———————————————————————– Theorem 6.7 (Sufficient Condition (1)) If the problem is regular (i.e. fuu (t, x, u) > 0 for all (t, x, u)), x0 (·) ∈ Θ is smooth, and satisfies (S1–1) Euler’s Equation d [fu (t, x0 (t), x˙ 0 (t))] = [fx (t, x0 (t), x˙ 0 (t))] , dt

212

Chapter 6. Summary for the Simplest Problem

(S1–2) the Strengthen Jacobi Condition that there is no value tˆc conjugate to t0 satisfying tˆc ≤ t1 , then x0 (·) provides a strong local minimum for J(·) on Θ. ———————————————————————– Theorem 6.8 (Sufficient Condition (2)) If x0 (·) smooth, and satisfies

∈

Θ is

(S2−1) Euler’s Equation d [fu (t, x0 (t), x˙ 0 (t))] = [fx (t, x0 (t), x˙ 0 (t))] , dt (S2−2) the Strengthen Legendre Condition fuu (t, x0 (t), x˙ 0 (t)) > 0, t0 ≤ t ≤ t1 , (S2−3) the Strengthen Jacobi Condition that there is no value tˆc conjugate to t0 satisfying tˆc ≤ t1 , then x0 (·) provides a weak local minimum for J(·) on Θ.

Chapter 7 Extensions and Generalizations In this chapter we discuss several extensions of the Simplest Problem of the Calculus of Variations. Although we focus on the first necessary condition (Euler’s Necessary Condition), there are extensions of all the Necessary conditions (I) - (IV) presented above. We shall derive the necessary conditions for global minimizers and simply note that the proofs may be modified as in Chapter 4 for weak local minimizers.

7.1

Properties of the First Variation

We recall the definition of the first variation of a functional defined on P W S(t0 , t1 ) at a given xo (·) ∈ P W S(t0 , t1 ). We restate Definition 2.14 in Chapter 2. If xo (·) and η(·) ∈ P W S(t0 , t1 ), then the first variation of J(·) at xo (·) in the direction of η(·) is denoted by δJ(xo (·); η(·)) and is defined by d (7.1) [J(xo (·) + εη(·))] . δJ(xo (·); η(·)) = dε ε=0 As in Chapter 3 one uses Leibniz’s formula (Lemma 3.2) to show that the first variation exists for all η(·) ∈ P W S(t0 , t1 ). Moreover, the first variation δJ(xo (·); η(·)) of J(·) at xo (·) in the direction of 213

214

Chapter 7. Extensions and Generalizations

η(·) has the form δJ(xo (·); η(·)) =

Zt1

{fx (s, xo (s), x˙ o (s))η(s)

t0

+ fu (s, xo (s), x˙ o (s))η(s)}ds. ˙

(7.2)

Observe that the above definition is valid for any function xo (·) ∈ P W S(t0 , t1 ). When xo (·) is an extremal, the first variation has a special form. Assume now that xo (·) is an extremal so that there is a constant c such that for all t ∈ [t0 , t1 ] [fu (t, xo (t), x˙ o (t))] = c +

Zt

[fx (s, xo (s), x˙ o (s))] ds.

t0

Between corners, xo (·) is differentiable and d [fu (t, xo (t), x˙ o (t))] = fx (t, xo (t), x˙ o (t)) . dt

(7.3)

Using the fact that xo (·) satisfies the Euler equation (7.3), then the first variation (7.2) becomes δJ(xo (·); η(·)) =

Zt1

{[fx (s, xo (s), x˙ o (s))] · η (s)

t0

+ [fu (s, xo (s), x˙ o (s))] · η˙ (s)}ds Zt1 d = { [fu (s, xo (s), x˙ o (s))] · η (s) ds t0

+ [fu (s, xo (s), x˙ o (s))] · η˙ (s)}ds Zt1 d [fu (s, xo (s), x˙ o (s)) · η (s)] ds = ds t0

1 = [fu (t, xo (t), x˙ o (t)) · η (t)]|t=t t=t0 .

Chapter 7. Extensions and Generalizations

215

Therefore, we have shown that if xo (·) is any extremal and η (·) ∈ P W S(t0 , t1 ), then 1 δJ(xo (·); η(·)) = [fu (t, xo (t), x˙ o (t)) · η (t)]|t=t t=t0

= fu (t1 , xo (t1 ), x˙ o (t1 )) · η (t1 ) − fu (t0 , xo (t0 ), x˙ o (t0 )) · η (t0 ) .

We summarize this as the following Lemma. Lemma 7.1 If xo (·) ∈ P W S(t0 , t1 ) is an extremal and η(·) ∈ P W S(t0 , t1 ), then the first variation of J(·) at xo (·) in the direction of η(·) exists and is given by t=t1 δJ(xo (·); η(·)) = [fu (t, xo (t), x˙ o (t)) · η (t)]|t=t . 0

7.2

(7.4)

The Free Endpoint Problem

Recall the river crossing problem discussed in Section 2.1.2 and Section 2.4.3. The river crossing problem is to find a smooth function x∗ : [0, 1] −→ R that minimizes q Z1 c2 (1 + [x˙ (s)]2 ) − [v(s)]2 − v(s)x˙ (s) ds (7.5) J(x(·)) = c2 − [v(s)]2 0

among all smooth functions satisfying x (0) = 0.

(7.6)

Observe that unlike the SPCV, there is no specified value at t = 1. In particular, the value of x(1) is “free” and must be determined as part of finding the minimizing function x∗ (·). This a typical example of the so-called free-endpoint problem. In this section we consider the free-endpoint problem and focus on obtaining (first order) necessary conditions. As in the previous chapters, let X = P W S(t0 , t1 ) denote the space of all real-valued

216

Chapter 7. Extensions and Generalizations

piecewise smooth functions defined on [t0 , t1 ]. For each PWS function x : [t0 , t1 ] → R, define the functional J : X → R by J(x(·)) =

Zt1

f (s, x(s), x˙ (s)) ds.

(7.7)

t0

Assume that the interval [t0 , t1 ] and initial value x0 are given (no value is assigned at t1 ) and define the set of PWS functions ΘL by ΘL = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 } .

(7.8)

The Free Endpoint Problem is the problem of minimizing J(·) on ΘL . In particular, the goal is to find x∗ (·) ∈ ΘL such that ∗

J(x (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s)) ds,

t0

=

t0

for all x (·) ∈ ΘL .

x2 () x * ()

x0

o t0

x1 ()

t

t1

Figure 7.1: The Free Endpoint Problem

217

Chapter 7. Extensions and Generalizations

In order to derive the first order necessary condition for minimizing J(·) on the set ΘL we must decide what class of variations are admissible. Assume that x∗ (·) ∈ ΘL minimizes J(·) on ΘL . We wish to make sure that for all ε (or for all ε sufficiently small) the variations ϕ(·, ε) = x∗ (·) + εη(·) belongs to ΘL . In particular, we need to show that ϕ(t, ε) = x∗ (t) + εη(t) ∈ ΘL .

(7.9)

It is clear that if VL (t0 , t1 ) = {η(·) ∈ P W S(t0 , t1 ) : η (t0 ) = 0} ,

(7.10)

then (7.9) is satisfied for all ε. Observe that V0 (t0 , t1 ) ⊂ VL (t0 , t1 ) ⊂ P W S(t0 , t1 ) so that (7.9) also holds for any η(·) ∈ V0 (t0 , t1 ) ⊂ VL (t0 , t1 ) ⊂ P W S(t0 , t1 ). The space VL (t0 , t1 ) defined by (7.10) is called the space of “admissible variations” for the free endpoint problem. We turn now to the derivation of Euler’s Necessary Condition for the free endpoint problem. Although the presentation below is almost identical to the derivation for the SPCV in Chapter 3, we present the details again to re-enforce the basic idea. The main

1 ()

t0

2 ()

t1

Figure 7.2: Admissible Variations for the Free Endpoint Problem

218

Chapter 7. Extensions and Generalizations

difference here is that the free endpoint condition leads to a new “natural boundary condition” that is an essential component of the necessary condition.

7.2.1

The Euler Necessary Condition

Assume that x∗ (·) ∈ ΘL is a (global) minimizer for J(·) on ΘL . In particular, assume that x∗ (·) ∈ ΘL satisfies ∗

J(x (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s)) ds,

t0

=

(7.11)

t0

for all x (·) ∈ ΘL . Let η(·) ∈ VL (t0 , t1 ) and consider the “variation” ϕ(t, ε) = x∗ (t) + εη(t).

(7.12)

Observe that for each ε ∈ R, ϕ(t, ε) ∈ ΘL since ϕ(t, ε) ∈ P W S(t0 , t1 ) and satisfies ϕ(t0 , ε) = x∗ (t0 )+εη(t0 ) = x0 +εη(t0 ) = x0 . Hence, if η(·) ∈ VL (t0 , t1 ), then for all ε ∈ R the variation ϕ(t, ε) , x∗ (t) + εη(t) ∈ ΘL , i.e. it is admissible. Since x∗ (·) ∈ ΘL minimizes J(·) on ΘL , it follows that J(x∗ (·)) ≤ J(x∗ (·) + εη(·))

(7.13)

for all ε ∈ (−∞, +∞). Define F : (−∞, +∞) −→ R by ∗

F (ε) = J(x (·) + εη(·)) =

Zt1

f (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))ds, ˙

t0

(7.14) and note that (7.13) implies that F (0) = J(x∗ (·)) ≤ J(x∗ (·) + εη(·)) = F (ε)

Chapter 7. Extensions and Generalizations

219

for all ε ∈ (−∞, +∞). Therefore, F (·) has a minimum on (−∞, +∞) at ε∗ = 0 and applying Theorem 2.1 it follows that the first variation must be zero. That is, d d ∗ ∗ = = 0, δJ(x (·); η(·)) , F (ε) [J(x (·) + εη(·))] dε dε ε=0 ε=0 (7.15) for all η(·) ∈ VL (t0 , t1 ). However, (7.2) implies that ∗

δJ(x (·); η(·)) =

Zt1

{[fx (s, x∗ (s), x˙ ∗ (s))] η (s)

t0

+ [fu (s, x∗ (s), x˙ ∗ (s))] η˙ (s)}ds

(7.16)

and (7.15) yields Zt1

{[fx (s, x∗ (s), x˙ ∗ (s))] η (s) + [fu (s, x∗ (s), x˙ ∗ (s))] η˙ (s)} ds = 0

t0

(7.17) for all η(·) ∈ VL (t0 , t1 ). Since (7.17) holds for all η(·) ∈ VL (t0 , t1 ) and V0 (t0 , t1 ) ⊂ VL (t0 , t1 ) it follows that Zt1

{[fx (s, x∗ (s), x˙ ∗ (s))] η (s) + [fu (s, x∗ (s), x˙ ∗ (s))] η˙ (s)} ds = 0

t0

holds for all η(·) ∈ V0 (t0 , t1 ). Therefore, the Fundamental Lemma of the Calculus of Variations 3.1, Part (B) implies that there is a c such that [fu (t, x∗ (t), x˙ ∗ (t))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds

t0

e.f. on t ∈ [t0 , t1 ] and hence x∗ (·) is an extremal. Moreover, between corners d [fu (t, x∗ (t), x˙ ∗ (t))] = fx (t, x∗ (t), x˙ ∗ (t)) . dt

(7.18)

220

Chapter 7. Extensions and Generalizations

Consequently, Lemma 7.1 implies that the first variation of J(·) at x∗ (·) in the direction of η(·) exists and is given by 1 δJ(x∗ (·); η(·)) = [fu (t, x∗ (t), x˙ ∗ (t)) η (t)]|t=t t=t0 .

(7.19)

Returning to (7.15) we have that 1 0 = δJ(x∗ (·); η(·)) = [fu (t, x∗ (t), x˙ ∗ (t)) η (t)]|t=t t=t0

must hold for all η(·) ∈ VL (t0 , t1 ). Hence, if η(·) ∈ VL (t0 , t1 ), then η(t0 ) = 0 and it follows that 1 0 = [fu (t, x∗ (t), x˙ ∗ (t)) η (t)]|t=t t=t0

= fu (t1 , x∗ (t1 ), x˙ ∗ (t1 )) η (t1 ) − fu (t0 , x∗ (t0 ), x˙ ∗ (t0 )) η (t0 ) = fu (t1 , x∗ (t1 ), x˙ ∗ (t1 )) η (t1 ) . However, for η(·) ∈ VL (t0 , t1 ) there is no restriction on η (t1 ) so that fu (t1 , x∗ (t1 ), x˙ ∗ (t1 )) η (t1 ) = 0 (7.20) must hold for arbitrary values of η (t1 ). Hence (7.20) holds for any value of η (t1 ) which implies that fu (t1 , x∗ (t1 ), x˙ ∗ (t1 )) = 0.

(7.21)

Condition (7.21) is called the natural boundary condition for the free endpoint problem. Thus, we have derived the (global) Euler necessary condition for the Free Endpoint Problem. It is straightforward to extend this proof to the case where we only assume that x∗ (·) ∈ ΘL is a weak local minimizer in ΘL . In particular, we have the following Euler Necessary Condition for the free endpoint problem. Theorem 7.1 (Euler Necessary Condition for the Free Endpoint Problem) If x∗ (·) ∈ ΘL is a weak local minimizer for J(·) on ΘL , then (EF-1) there is a constant c such that for all t ∈ [t0 , t1 ], ∗

∗

fu (t, x (t), x˙ (t)) = c +

Zt

t0

fx (s, x∗ (s), x˙ ∗ (s)) ds,

(7.22)

221

Chapter 7. Extensions and Generalizations (EF-2) x∗ (t0 ) = x0 , (EF-3) fu (t1 , x∗ (t1 ), x˙ ∗ (t1 )) = 0.

(EF-4) Between corners the function fu (t, x∗ (t), x˙ ∗ (t)) is differentiable and if t is not a corner of x∗ (·) d fu (t, x∗ (t), x˙ ∗ (t)) = fx (t, x∗ (t), x˙ ∗ (t)) . dt

(7.23)

As in the previous chapters, equation (7.22) is called Euler’s Equation in integral form, while equation (7.23) is called Euler’s Differential Equation. Therefore, we have shown that a minimizer x∗ (·) of J(·) on ΘL must be an extremal. This implies that the Erdmann corner condition

fu tˆ, x∗ (tˆ), x˙ ∗ tˆ+ = fu (tˆ, x∗ (tˆ), x˙ ∗ (tˆ− )) also holds, and if the problem is regular, then extremals can not have corners. The Hilbert Differentiability Theorem also holds and applying the chain rule to the left side of (7.23) yields d [fu (t, x∗ (t), x˙ ∗ (t))] = [fut (t, x∗ (t), x˙ ∗ (t))] dt + [fux (t, x∗ (t), x˙ ∗ (t))] · x˙ ∗ (t) + [fuu (t, x∗ (t), x˙ ∗ (t))] · x¨∗ (t) . Hence, the Euler Equation (7.23) becomes the second order differential equation [fuu (t, x∗ (t), x˙ ∗ (t))] · x¨∗ (t) = [fx (t, x∗ (t), x˙ ∗ (t))] − [fut (t, x∗ (t), x˙ ∗ (t))] (7.24) − [fux (t, x∗ (t), x˙ ∗ (t))] · x˙ ∗ (t) .

7.2.2

Examples of Free Endpoint Problems

We shall go through a couple of simple examples to illustrate the application of the necessary condition.

222

Chapter 7. Extensions and Generalizations

Example 7.1 Find a PWS function x∗ (·) satisfying x∗ (0) = 0 and such that x∗ (·) minimizes

J(x(·)) =

Zπ/2 0


1 [x˙ (s)]2 − [x (s)]2 ds. 2

We note that t0 = 0, t1 = π/2, x0 = 0, and x(π/2) is free. The integrand f (t, x, u) is given by 1 f (t, x, u) = ([u]2 − [x]2 ), 2 and hence fx (t, x, u) = −x fu (t, x, u) = +u fuu (t, x, u) = +1 > 0. We see that f (t, x, u) is regular and hence the minimizer can not have corners. Euler’s Equation d [fu (t, x∗ (t), x˙ ∗ (t))] = [fx (t, x∗ (t), x˙ ∗ (t))] dt becomes

d ∗ [x˙ (t)] = [−x∗ (t)] , dt

or equivalently, x¨∗ (t) + x∗ (t) = 0. The general solution is x∗ (t) = α cos(t) + β sin(t), and applying the boundary condition at t = t0 = 0 yields 0 = x∗ (0) = α cos(0) + β sin(0) = α, so that x∗ (t) = β sin(t).

Chapter 7. Extensions and Generalizations

223

The natural boundary at t = t1 = π/2 becomes fu (π/2, x∗ (π/2) , x˙ ∗ (π/2)) = x˙ ∗ (π/2) = 0. However, x˙ ∗ (t) = β cos(t) so that x˙ ∗ (π/2) = β cos(π/2) = 0. Since cos(π/2) = 0, it follows that β can be any number and hence x∗ (t) = β sin(t) are possible minimizers. Observe that we do not know that x∗ (·) minimizes J(·) on ΘL for any number β since we only checked the necessary condition. Example 7.2 Consider the functional J(x(·)) =

Z1

[x˙ (s)]2 ds

0

with x(0) = 0 and the endpoint x(1) free. The integrand is given by f (t, x, u) = u2 , fu (t, x, u) = 2u, fuu (t, x, u) = 2, and fx (t, x, u) = 0. Since the problem is regular, all extremals are regular and Euler’s Integral Equation is given by ∗

∗

∗

2x˙ (t) = fu (t, x (t), x˙ (t)) = c +

Zt 0

=c+

Zt

0ds = c,

0

or equivalently, x˙ ∗ (t) = k for some constant k. Therefore, x∗ (t) = kt + b

fx (s, x∗ (s), x˙ ∗ (s))ds

224

Chapter 7. Extensions and Generalizations

and the condition x(0) = 0 implies x∗ (t) = kt. The natural boundary condition at t1 = 1 takes the form fu (1, x∗ (1), x˙ ∗ (1)) = 2x˙ ∗ (1) = 2k = 0 which means that x∗ (t) = 0. Hence, the only extremal satisfying the necessary condition Theorem 7.1 is x∗ (t) = 0. Clearly, x∗ (t) = 0 is a global minimizer.

7.3

The Simplest Point to Curve Problem

We assume that the initial time t0 and initial value x0 are given and that there is a given smooth function φ(·) defined on the interval (t0 , +∞). The problem is to minimize Z t1 f (s, x(s), x˙ (s)) ds, J (x(·)) = t0

subject to x (t0 ) = x0 and x (t1 ) = φ (t1 ), where t0 < t1 and t1 is a point where the graph of the function x (·) intersects the graph of φ (·). This problem is illustrated in Figure 7.3. Note that t1 is not fixed and different functions may intersect φ(·) at different “final times”. To formulate the optimization problem we first define the space of piecewise smooth functions on [t0 , +∞) by P W S∞ = P W S(t0 , +∞), where P W S∞ = {x(·) : [t0 , +∞) → R : x (·) ∈ P W S(t0 , T ) for all T > t0 }. (7.25)

225

Chapter 7. Extensions and Generalizations

x2 ()

x1 () x0

 t0

 () x *()

t

t1*

t11

t12

Figure 7.3: Point to Curve Problem Note that P W S∞ = P W S(t0 , +∞) is the set of all “locally” piecewise smooth functions defined on the half line [t0 , +∞). Assume that t0 , x0 and φ(·) are given as above and define the set of PWS functions (intersecting the graph of φ) Θφ by Θφ = {x(·) ∈ P W S∞ : x (t0 ) = x0 , x (t1 ) = φ(t1 ) for some t1 > t0 }.

(7.26)

The simplest Point to Curve Problem is the problem of minimizing J(·) on Θφ . In particular, the goal is to find x∗ (·) ∈ Θφ and a t∗1 > t0 such that x∗ (t∗1 ) = φ(t∗1 ) and ∗

J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s)) ds,

t0

=

t0

for all x (·) ∈ Θφ .

226

Chapter 7. Extensions and Generalizations

We will derive a necessary condition for a global minimizer, but the result holds for a weak local minimizer and the following derivation is easily extended to this case. We start with a few basic lemmas. Lemma 7.2 If x∗ (·) ∈ Θφ is piecewise smooth and minimizes J(·) on Θφ , with x∗ (t∗1 ) = φ(t∗1 ), then (EPC-1) there is a constant c such that for all t ∈ [t0 , t∗1 ], ∗

∗

[fu (t, x (t), x˙ (t))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds,

(7.27)

t0 ∗

(EPC-2) x (t0 ) = x0 , (EPC-3) x∗ (t∗1 ) = φ(t∗1 ). (EPC-4) Between corners of x∗ (·) the function is differentiable and

∂ f ∂u

(t, x∗ (t), x˙ ∗ (t))

d [fu (t, x∗ (t), x˙ ∗ (t))] = [fx (t, x∗ (t), x˙ ∗ (t))] . (7.28) dt Proof. This lemma follows from the standard derivation for a fixed endpoint problem. In particular, for η(·) ∈ V0∗ = P W S0 (t0 , t∗1 ), the variation x∗ (·) + εη(·) belongs to Θφ and hence ∗

δJ(x∗ (·); η(·)) =

Zt1

[fx (s, x∗ (s), x˙ ∗ (s))η(s)

t0

+ fu (s, x∗ (s), x˙ ∗ (s))η(s)]ds ˙ = 0.

(7.29)

Since this holds for all η(·) ∈ V0∗ = P W S0 (t0 , t∗1 ), the Fundamental Lemma of the Calculus of Variations implies that x∗ (·) is an extremal and this completes the proof.  Note that the Euler equations (7.27) or (7.28) must be solved on the interval [t0 , t∗1 ]. However, we need an additional piece of information to determine the extra unknown parameter t∗1 . To obtain this condition, we must enlarge the space of variations.

227

Chapter 7. Extensions and Generalizations



x0

 ()

t0

x *()

t

t1*

xˆ *()

Figure 7.4: Extending the Optimal Curve In order to derive the transversality condition we define another class of variations that are admissible for this problem. If x∗ (·) ∈ Θφ minimizes J(·) on Θφ , with x∗ (t∗1 ) = φ(t∗1 ), then we need to make sure that for all ε (or for all ε sufficiently small) the variations x∗ (t) + εη(t) belong to Θφ . In particular, we need to find a set of admissible variations Vφ so that x∗ (t) + εη(t) ∈ Θφ

(7.30)

for all η(·) ∈ Vφ and all ε sufficiently small. Thus, we must define Vφ in such a way that x∗ (t) + εη(t) intersects φ(·) at some time tε1 > t0 . Given the minimizer x∗ (·), define xˆ∗ (·) by  ∗ x (t), t0 ≤ t ≤ t∗1 , ∗ xˆ (t) = x˙ ∗ (t∗1 )(t − t∗1 ) + x∗ (t∗1 ), t∗1 ≤ t.

Also, given any η(·) ∈ VL∗ = {η(·) ∈ P W S(t0 , t∗1 ) : η (t0 ) = 0} define ηˆ(·) by ηˆ(t) =



η(t), t0 ≤ t ≤ t∗1 , η(t ˙ ∗1 )(t − t∗1 ) + η(t∗1 ), t∗1 ≤ t.

(7.31)

228

Chapter 7. Extensions and Generalizations

ˆ2 ()

2 ()

1 ()  t0

ˆ1 ()

t1*

Figure 7.5: Extending η(·) to ηˆ(·)

Remark 7.1 The functions xˆ∗ (·) and ηˆ(·) are continuously differentiable in a neighborhood of t∗1 . To see this, pick tˆL < t∗1 so that neither x∗ (·) nor η(·) has a corner tˆc with tˆL < tˆc . By construction xˆ∗ (·) and ηˆ(·) will be continuously differentiable for all t > tˆL . Now we define Vφ by Vφ = {ˆ η (·) ∈ P W S(t0 , +∞) : η(·) ∈ VL∗ } ,

(7.32)

where ηˆ(·) ∈ Vφ is defined by (7.31) above. What we want to show is that for sufficiently small ε, x∗ (·) + εη(·) intersects φ(·) at some time tε1 > t0 . Observe that all functions ηˆ(·) ∈ Vφ are linear on t∗1 ≤ t. Also, ηˆ(·) and xˆ∗ (·) are smooth on an interval about t∗1 . Note that for any ηˆ(·) ∈ Vφ , the function xˆ∗ (t) + εˆ η (t), is defined for all t ≥ t0 . Since at ε = 0 xˆ∗ (t) + εˆ η (t) = xˆ∗ (t),

229

Chapter 7. Extensions and Generalizations

xˆ * ()  1ˆ ()

xˆ * ()  ˆ ()



x0

t0

 () x * ()

t

t1* t1

Figure 7.6: The Variation xˆ∗ (·) + εˆ η (·) one might expect that for sufficiently small ε the curve xˆ∗ (t) + εˆ η (t) would intersect φ(·) at some time tε1 > t0 (see Figure 7.6). For one important case we shall show that this is true. In order to show this we need the Implicit Function Theorem. The following theorem follows directly from Theorem 41.9 and Lemma 41.10 in Bartle’s book [15] (see pages 382 - 391). Theorem 7.2 (Implicit Function Theorem) Suppose H : R× R → R satisfies (i.e. H (ε, γ)): i) H (0, 0) = 0. ii) H is of class C 1 on a neighborhood of [0 0]T . iii)

∂ H ∂γ

(0, 0) = Hγ (0, 0) 6= 0.

Then, there is a neighborhood of 0 (i.e. (−δ, +δ)) such that one can solve for γ in terms of ε. In particular, there exists a function γ(·) : (−δ, +δ) → R such that for all −δ < ε < δ H (ε, γ (ε)) = 0,

230

Chapter 7. Extensions and Generalizations

and γ (0) = 0. Moreover, the function γ(ε) is continuously differentiable with respect to ε and Hε (ε, γ (ε)) d γ(ε) = γ˙ (ε) = − . dε Hγ (ε, γ (ε)) ˆ γ) and apply the ImWe shall define a specific function H(ε, plicit Function Theorem. Again, we are trying to show that for sufficiently small ε the variation xˆ∗ (t) + εˆ η (t) will intersect φ(·) ε ε ∗ at some time t1 > t0 . If t1 = t1 + γ(ε), then we are asking if for each ε on an interval (−δ, +δ), is there a γ = γ(ε) such that φ(t∗1 + γ) − xˆ∗ (t∗1 + γ) − εˆ η (t∗1 + γ) = 0? ˆ γ) by Thus, we are motivated to define H(ε, ˆ γ) = φ(t∗ + γ) − xˆ∗ (t∗1 + γ) − εˆ η (t∗1 + γ). H(ε, 1

(7.33)

ˆ γ) defined above has the following properObserve that H(ε, ties: ˆ (0, 0) = φ (t∗ ) − xˆ∗ (t∗ ) = 0. i) H 1 1 ˆ γ (ε, γ) = d φ (t∗ + γ) − ˆ ε (ε, γ) = −ˆ ii) H η (t∗1 + γ) and H 1 dt d ∗ ∗ d ∗ ˆ γ) x ˆ (t + γ)−ε η ˆ (t + γ) are continuous and hence H(ε, 1 1 dt dt
1 is of class C on (−∞, ∞) × tˆL , +∞ . ˆ γ (0, 0) = iii) H

d φ (t∗1 ) dt

−

d ∗ xˆ dt

(t∗1 ) = φ˙ (t∗1 ) − x˙ ∗ (t∗1 ).

ˆ γ) satisfies the asTherefore, if φ˙ (t∗1 ) − x˙ ∗ (t∗1 ) 6= 0, then H(ε, sumption of the Implicit Function Theorem. In particular, if φ˙ (t∗1 )− x˙ ∗ (t∗1 ) 6= 0, then there exists a function γ(·) : (−δ, +δ) → R such that for all −δ < ε < δ ˆ γ(ε)) = φ(t∗1 +γ(ε))− xˆ∗ (t∗1 +γ(ε))−εˆ 0 = H(ε, η (t∗1 +γ(ε)). (7.34)

231

Chapter 7. Extensions and Generalizations

Equation (7.34) implies that for each ε ∈ (−δ, +δ), the variations xˆ∗ (·) + εˆ η (·) intersects φ(·) at the time β(ε) = t∗1 + γ(ε). Moreover, β˙ (ε) exists and equals β˙ (ε) = γ˙ (ε) "

# −ˆ η (t∗1 + γ (ε)) =− d . φ (t∗1 + γ (ε)) − dtd xˆ∗ (t∗1 + γ (ε)) − ε dtd ηˆ (t∗1 + γ (ε)) dt

In particular, β˙ (0) =

η (t∗1 ) η (t∗1 ) = = γ˙ (0) . d φ (t∗1 ) − dtd x∗ (t∗1 ) φ˙ (t∗1 ) − x˙ ∗ (t∗1 ) dt

(7.35)

We now have established the following result. Theorem 7.3 If x∗ (·) satisfies ∗

J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f (s, x(s), x˙ (s)) ds,

t0

=

t0

on Θφ and φ˙ (t∗1 ) − x˙ ∗ (t∗1 ) 6= 0, then there exists a function γ(·) : (−δ, +δ) → R such that for all −δ < ε < δ the variations xˆ∗ (·) + εˆ η (·) ∈ Θφ and intersect φ(·) at the time β(ε) = t∗1 + γ(ε). Define the function F : (−δ, +δ) → R by F (ε) = J (ˆ x∗ (·) + εˆ η (·))  Zβ(ε)  d d ∗ ∗ f s, xˆ (s) + εˆ η (s), xˆ (s) + ε ηˆ (s) ds = ds ds t0

Zβ(ε) G (s, ε) ds, = t0

232

Chapter 7. Extensions and Generalizations

where G (t, ε) = f



 d d ∗ t, xˆ (t) + εˆ η (t), xˆ (t) + ε ηˆ(t) . dt dt ∗

Therefore, for all −δ < ε < δ we have β(0) Z f (s, x∗ (s), x˙ ∗ (s)) ds F (0) = t0

 Zβ(ε)  d d ∗ ∗ ≤ f s, xˆ (s) + εˆ η (s), xˆ (s) + ε ηˆ (s) ds = F (ε), ds ds t0

so that ε∗ = 0 minimizes F (ε) on the open interval (−δ, +δ). d Consequently, it follows that dε F (0) = F ′ (0) = 0. In order to ′ compute F (ε), we apply Leibniz’s Formula which yields Zβ(ε) Gε (s, ε) ds. F ′ (ε) = G (β (ε) , ε) β˙ (ε) + t0

In particular, since β (ε) = t∗1 + γ (ε) and γ (0) = 0, it follows that β(0) Z F ′ (0) = G (β (0) , 0) β˙ (0) + Gε (s, 0) ds t0

=f

(t∗1 , x∗

(t∗1 ), x˙ ∗

(t∗1 )) β˙

(0)

t∗1

+

Z

[fx (s, x∗ (s), x˙ ∗ (s)) η (s) + fu (s, x∗ (s), x˙ ∗ (s)) η˙ (s)]ds

t0

= f (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) ∗

+

Zt1

t0

η (t∗1 ) [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

(fx∗ (s)η(s) + fu∗ (s)η(s)) ˙ ds.

Chapter 7. Extensions and Generalizations

233

Hence, for all η(·) ∈ Vφ we have ∗

f (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) η (t∗1 ) 0= + [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

Zt1

(fx∗ (s)η(s) + fu∗ (s)η(s)) ˙ ds.

t0

In particular, (7.3) is valid for all η(·) satisfying η (t∗1 ) = 0, so we obtain Euler’s equation d ∗ f (t) = fx∗ (t), dt u which must hold on [t0 , t∗1 ]. Substituting dtd fu∗ (t) = fx∗ (t) into the above equation yields ∗

0=

f

(t∗1 , x∗

(t∗1 ), x˙ ∗ (t∗1 )) η [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

f ∗ (t∗1 )η (t∗1 ) = + [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

∗ Zt1 

t0

∗

∗

=

f (t∗1 )η (t∗1 ) [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

+

(t∗1 )

Zt1 

t0

+

Zt1

(fx∗ (s)η(s) + fu∗ (s)η(s)) ˙ ds

t0

 d ∗ ∗ ˙ ds [ fu (s)]η(s) + fu (s)η(s) ds  d ∗ [f (s)η(s)] ds ds u

f ∗ (t∗1 )η (t∗1 ) = + fu∗ (t∗1 )η (t∗1 ) − fu∗ (t0 )η (t0 ) . ∗ ∗ ∗ ˙ [φ (t1 ) − x˙ (t1 )] Using the fact that η (t0 ) = 0, it follows that 0=

f ∗ (t∗1 )η (t∗1 ) + fu∗ (t∗1 )η (t∗1 ), [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]

or equivalently, 0 = {f ∗ (t∗1 ) + [φ˙ (t∗1 ) − x˙ ∗ (t∗1 )]fu∗ (t∗1 )}η (t∗1 ) . Since η (t∗1 ) is arbitrary, we obtain the transversality condition f ∗ (t∗1 ) + fu∗ (t∗1 )[φ˙ (t∗1 ) − x˙ ∗ (t∗1 )] = 0.

(7.36)

234

Chapter 7. Extensions and Generalizations

Combining Lemma 7.2 with the transversality condition (7.36) we have proven the following theorem. Theorem 7.4 (Euler Necessary Condition for the Point to Curve Problem) If x∗ (·) ∈ Θφ is piecewise smooth and minimizes J(·) on Θφ , with x∗ (t∗1 ) = φ(t∗1 ), then (EPC-1) there is a constant c such that for all t ∈ [t0 , t∗1 ], [fu (t, x∗ (t), x˙ ∗ (x))] = c +

Zt

[fx (s, x∗ (s), x˙ ∗ (s))] ds,

(7.37)

t0

(EPC-2) x∗ (t0 ) = x0 , (EPC-3) x∗ (t∗1 ) = φ(t∗1 ). (EPC-3’) If φ˙ (t∗1 ) 6= x˙ ∗ (t∗1 ), then f ∗ (t∗1 ) + fu∗ (t∗1 )[φ˙ (t∗1 ) − x˙ ∗ (t∗1 )] = 0.

(7.38)

∂ f ∂u

(t, x∗ (t), x˙ ∗ (x))

(EPC-4) Between corners of x∗ (·) the function is differentiable and

d [fu (t, x∗ (t), x˙ ∗ (x))] = [fx (t, x∗ (t), x˙ ∗ (x))] . dt

(7.39)

These results can clearly be extended to two curves, curve to free endpoint, etc. To motivate the term “transversality condition”, consider the class of equations where f (t, x, u) has the form √ f (t, x, u) = g (t, x) 1 + u2 , (e.g. the brachistochrone problem, minimal surface area of revolution problem, etc.). Here, √ u u fu (t, x, u) = g (t, x) √ = [g (t, x) 1 + u2 ][ ] 1 + u2 1 + u2 u f (t, x, u). = 1 + u2

235

Chapter 7. Extensions and Generalizations Hence, ˙ − x˙ ∗ (t)] f ∗ (t) + fu∗ (t)[φ(t)   x˙ ∗ (t) ∗ ∗ ˙ − x˙ ∗ (t)] [φ(t) = f (t) + f (t) ∗ 2 1 + [x˙ (t)] ∗ ∗ ˙ − x˙ ∗ (t)]) f (t)(1 + [x˙ (t)]2 + [x˙ ∗ (t)][φ(t) = [1 + [x˙ ∗ (t)]2 ] ˙ − [x˙ ∗ (t)]2 ) f ∗ (t)(1 + [x˙ ∗ (t)]2 + x˙ ∗ (t)φ(t) = [1 + [x˙ ∗ (t)]2 ]   f ∗ (t) ˙ = [1 + x˙ ∗ (t)φ(t)]. 1 + [x˙ ∗ (t)]2

The transversality condition at t∗1 implies that   f ∗ (t∗1 ) ∗ ∗ ∗ ∗ ˙ ∗ ∗ ∗ ˙ ∗ )] = 0 [1+x˙ ∗ (t∗1 )φ(t f (t1 )+fu (t1 )[φ (t1 )−x˙ (t1 )] = 1 1 + [x˙ ∗ (t∗1 )]2 and hence if f ∗ (t∗1 ) 6= 0, then it follows that ˙ ∗ )] = 0. [1 + x˙ ∗ (t∗1 )φ(t 1 In this case x˙ ∗ (t∗1 ) =

−1 ˙ ∗) φ(t 1

˙ are perwhich means that the slopes of the curves x˙ ∗ (·) and φ(·) ∗ pendicular at the intersecting value t = t1 . That is, the optimal trajectory x˙ ∗ (·) must be orthogonal (transversal) to the curve φ(·) at the intersection point. Example 7.3 Consider the problem of minimizing Z t1 p J (x(·)) = 1 + [x˙ (s)]2 ds, t0

subject to

x (t0 ) = ϕ (t0 ), x (t1 ) = ψ (t1 ), where ϕ (t) = − (t + 1) , ψ (t) =



5 t− 2

2

.

236

Chapter 7. Extensions and Generalizations

Since the problem is regular, we need only consider the Euler Differential Equation d fu (t, x∗ (t), x˙ ∗ (t)) = fx (t, x∗ (t), x˙ ∗ (t)) dt which has the form d dt or equivalently,

x˙ ∗ (t)

p 1 + [x˙ ∗ (t)]2 x˙ ∗ (t)

p

1 + [x˙ ∗ (t)]2

!

= 0,

= c.

Solving this equation we get

which implies that


[x˙ ∗ (t)]2 = c2 1 + [x˙ ∗ (t)]2 ,

c2 , 1 − c2 or equivalently, there is a constant a such that [x˙ ∗ (t)]2 =

x˙ ∗ (t) = a and the extremals are straight lines x(t) = at + b. The transversality conditions become a = x˙ ∗ (t∗0 ) =

−1 −1 = = 1, ∗ ϕ˙ (t0 ) −1

and hence it follows that a = 1. At the other end t1 = t∗1 , the transversality condition becomes a = 1 = x˙ ∗ (t1 ) =

−1 −1
= 2 t1 − 25 ψ˙ (t1 )

and solving for t1 it follows that   5 2 t1 − = −1, 2

237

Chapter 7. Extensions and Generalizations so that t∗1 = t1 =

5 1 − = 2. 2 2

Since a = 1, x∗ (t) = t + b, and from above t∗1 = 2 so that the relationship x∗ (t∗1 ) = ψ (t∗1 ) implies that x

∗

(t∗1 )

=2+b= =

t∗1

+b=

1 = ψ (t∗1 ) . 4



t∗1

5 − 2

Hence b=−

2

=



5 2− 2

2

=



1 − 2

2

7 4

so that

7 x∗ (t) = t − , 4 ∗ ∗ and t1 = 2. To find t0 we know that x∗ (t∗0 ) = ϕ (t∗0 ) which implies t∗0 − and hence

7 = − (t∗0 + 1) , 4

3 t∗0 = . 8 We have found that the optimal curve is given by x∗ (t) = t − 47
2 and it intersects ϕ (t) = −(t + 1) at t∗0 = 38 and ψ (t) = t − 52 at t∗1 = 2.

238

Chapter 7. Extensions and Generalizations

7.4

Vector Formulations and Higher Order Problems

In this section we discuss extensions of the SPCV to vector systems and higher order models. Consider the problem where we look for two (or more) functions x1 (·), x2 (·) to minimize a functional of the form J (x1 (·), x2 (·)) =

Zt1

f (s, x1 (s), x2 (s), x˙ 1 (s), x˙ 2 (s)) ds,

t0

subject to x1 (t0 ) = x1,0 x1 (t1 ) = x1,1 , x2 (t0 ) = x2,0 x2 (t1 ) = x2,1 .

3

()

2

1

0

() x*()

-1

-2

-3 -3

-2

-1

0 t* =3/8 0

1

t*1= 2

Figure 7.7: The Optimal Curve to Curve Solution

3

Chapter 7. Extensions and Generalizations

239

Here we assume that f (t, x1 , x2 , u1 , u2 ) is a C 2 function and we will apply the same ideas used in the SPCV. This will yield an Euler Necessary Condition where the Euler Differential Equation is a system of equations of the form d fu (t, x1 (t), x2 (t), x˙ 1 (t), x˙ 2 (t)) = fx1 (t, x1 (t), x2 (t), x˙ 1 (t), x˙ 2 (t)), dt 1 d fu (t, x1 (t), x2 (t), x˙ 1 (t), x˙ 2 (t)) = fx2 (t, x1 (t), x2 (t), x˙ 1 (t), x˙ 2 (t)). dt 2 Higher order problems are a special case of the vector formulation. For example, consider the problem of minimizing J(x(·)) =

Zt1

f (s, x(s), x(s), ˙ x¨(s))ds,

t0

subject to x (t0 ) = x0,

x (t1 ) = x1 ,

x˙ (t0 ) = u0,

x˙ (t1 ) = u1 ,

where f (t, x, u, v) is a C 2 function. Here we assume that x(·) is smooth and x(·) ˙ is piecewise smooth on [t0 , t1 ]. In this case setting the first variation to zero and using the higher order Fundamental Lemma of the Calculus of Variations will lead to the higher order Euler Differential Equation −

d2 d f (t, x(t), x(t), ˙ x ¨ (t)) + fu (t, x(t), x(t), ˙ x¨(t)) v dt2 dt = fx (t, x(t), x(t), ˙ x¨(t)).

In order to formulate the vector problem we assume that t0 and t1 are given and fixed and f (t, x1 , x2 , u1 , u2 ) is a C 2 function. Let P W S(t0 , t1 ; R2 ) denote the space of all R2 -valued piecewise smooth functions defined on [t0 , t1 ]. In particular, n  T P W S(t0 , t1 ; R2 ) = x(·) = x1 (·) x2 (·) o : xi (·) ∈ P W S(t0 , t1 ), i = 1, 2 . (7.40)

240

Chapter 7. Extensions and Generalizations

 T For each x(·) = x1 (·) x2 (·) ∈ P W S(t0 , t1 ; R2 ), define the functional J : P W S(t0 , t1 ; R2 ) → R by J(x(·)) = J (x1 (·), x2 (·)) =

Zt1

f (s, x1 (s), x2 (s), x˙ 1 (s), x˙ 2 (s)) ds.

t0

(7.41) Assume that the points x1,0 , x2,0 , x1,1 and x2,1 are given. Define the set of PWS functions Θ2 by  Θ2 = x(·) ∈ P W S(t0 , t1 ; R2 ) : xi (tj ) = xi, j , i = 1, 2, j = 0, 1 (7.42) 2 Observe that J : P W S(t0 , t1 ; R ) → R is a real valued function defined on P W S(t0 , t1 ; R2 ). The Simplest Problem in Vector Form (the fixed endpoint problem) is the problem of minimizing J(·) on Θ2 . In particular, the goal is to find x∗ (·) = [x∗1 (·) x∗2 (·)]T ∈ Θ2 such that J(x∗ (·)) =

Zt1

f (s, x∗1 (s), x∗2 (s), x˙ ∗1 (s), x˙ ∗2 (s)) ds

t0

≤ J(x(·)) =

Zt1

f (s, x1 (s), x2 (s), x˙ 1 (s), x˙ 2 (s)) ds,

t0

for all x (·) ∈ Θ2 . In order to formulate the higher order problem we assume that t0 and t1 are given and fixed and f (t, x, u, v) is a C 2 function. Let P W S 2 (t0 , t1 ) denote the space of all real-valued piecewise smooth functions x(·) defined on [t0 , t1 ] such that x(·) is smooth and x(·) ˙ is piecewise smooth on [t0 , t1 ]. In particular,  P W S 2 (t0 , t1 ) = x : [t0 , t1 ] −→ R :x(·) ∈ C 1 ([t0 , t1 ]) and x(·) ˙ ∈ P W S(t0 , t1 ) (7.43)

Chapter 7. Extensions and Generalizations

241

Note that if x(·) ∈ P W S 2 (t0 , t1 ), then x¨(·) ∈ P W C(t0 , t1 ). For each x(·) ∈ P W S 2 (t0 , t1 ), define the functional J : P W S 2 (t0 , t1 ) → R by J(x(·)) =

Zt1

f (s, x(s), x(s), ˙ x¨ (s)) ds.

(7.44)

t0

Since x¨(·) ∈ P W C(t0 , t1 ), the cost function defined by (7.44) is well defined. Assume that the points x0 , x1 , u0 and u1 are given and define the set of functions Θ2 by  Θ2 = x(·) ∈ P W S 2 (t0 , t1 ) : x (ti ) = xi , x˙ (ti ) = ui , i = 0, 1 . (7.45) The Simplest Problem in Higher Order Form (the fixed endpoint problem) is the problem of minimizing J(·) on Θ2 . In particular, the goal is to find x∗ (·) ∈ Θ2 such that J(x∗ (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s), x¨∗ (s)) ds

t0

≤ J(x(·)) =

Zt1

f (s, x(s), x(s), ˙ x¨ (s)) ds,

t0

for all x (·) ∈ Θ2 . Remark 7.2 Observe that the Simplest Problem in Higher Order Form is a special case of the Simplest Problem in Vector Form. To make this precise, we set x1 (·) = x(·) and x2 (·) = x(·) ˙ so that x˙ 2 (·) = x¨(·) and the cost function (7.44) becomes ˆ 1 (·), x2 (·)) = J(x

Zt1

fˆ (s, x1 (s), x2 (s), x˙ 1 (s), x˙ 2 (s)) ds

t0

= J(x(·)) =

Zt1

t0

f (s, x(s), x(s), ˙ x¨ (s)) ds,

242

Chapter 7. Extensions and Generalizations

where fˆ(t, x1 , x2 , u1 , u2 ) does not explicitly depend u1 and is defined by fˆ(t, x1 , x2 , u1 , u2 ) = f (t, x1 , x2 , u2 ). Before deriving necessary conditions, we present additional lemmas that extend the Fundamental Lemma of the Calculus of Variations.

7.4.1

Extensions of Some Basic Lemmas

In order to derive first order necessary conditions for vector and higher order problems, we present a few fundamental lemmas that provide the backbone to much of variational theory. These results are natural extensions of the FLCV. In addition, we review some basic results on differentiation. The Fundamental Lemma of the Calculus of Variations 3.1 involves only scalar functions and first order derivatives. We shall extend this lemma to higher order derivatives and to vector valued functions. To set the stage for the lemmas, we need some additional notation. Let p ≥ 1 be a given integer and define the set  P W S p (t0 , t1 ) = x(·) : x[k] (·) ∈ P W S(t0 , t1 ), k = 0, 1, . . . , p − 1 , (7.46) where dk x(·) x[k] (·) , (7.47) dtk is the k th derivative of x(·). For notational purposes we define the zero derivative as the function x[0] (·) , x(·). Observe that if x(·) ∈ P W S p (t0 , t1 ), then the pth derivative of x(·) exists except at a finite number of points and x[p] (·) ∈ P W C(t0 , t1 ). Likewise, we define V0p =V0p (t0 , t1 ) = P W S0p (t0 , t1 ) to be the set  V0p (t0 , t1 ) = η(·) ∈ P W S p (t0 , t1 ) :η [k] (t0 ) = 0 = η [k] (t1 ) , k = 0, 1, . . . , p − 1 . (7.48) Note that V0p (t0 , t1 ) ⊆ P W S p (t0 , t1 ) is the set of all functions η(·) ∈ P W S p (t0 , t1 ) with the property that η(·) and all its derivatives up to order p − 1 are zero at both ends of the interval [t0 , t1 ]. Also, by definition P W S 1 (t0 , t1 ) = P W S(t0 , t1 ) and

243

Chapter 7. Extensions and Generalizations

V01 (t0 , t1 ) = V0 (t0 , t1 ). The following is the basic result needed for all extensions of the FLCV. Lemma 7.3 Let p ≥ 1 be any integer. If v (·) is piecewise continuous on [t0 , t1 ] and Zt1

v (s) η (s) ds = 0

(7.49)

t0

for all η(·) ∈ V0p (t0 , t1 ), then v (·) ∈ P W S(t0 , t1 ) and v (t) = 0,

(7.50)

except at a finite number of points. The converse is also true. Proof: Since v (·) is piecewise continuous on [t0 , t1 ], there exist a finite set of points t0 < tˆ1 < tˆ2 < ... < tˆk < t1 in (t0 , t1 ) such that v (·) is continuous on each subinterval (tˆj , tˆj−1 ). Assume that v (·) is not zero on one of these intervals. In particular, assume that there is a point zˆ ∈ (tˆj , tˆj−1 ) so that v (ˆ z ) > 0. Since v (·) is continuous on (tˆj , tˆj−1 ), there is a δ > 0 and a < b such that (ˆ z −δ, zˆ+δ) ⊂ [a, b] ⊂ (tˆj , tˆj−1 ) and v (t) > 0 for all t ∈ (ˆ z −δ, zˆ+δ). Define the function  [(t − a)(b − t)]p , t ∈ [a, b] η˜(t) = 0, t∈ / [a, b] and note that η˜(t) has the following properties: (i) η˜(·) ∈ V0p (t0 , t1 ), (ii) η˜(t) > 0 for all t ∈ (ˆ z − δ, zˆ + δ), (iii) v (t) η˜ (t) > 0 for all t ∈ (ˆ z − δ, zˆ + δ). Consequently, it follows that Zt1

t0

Zzˆ+δ v (s) η˜ (s) ds > 0 v (s) η˜ (s) ds ≥ zˆ−δ

244

Chapter 7. Extensions and Generalizations

which contradicts the assumption (7.49). Hence, v (t) = 0 on each of the subintervals (tˆj , tˆj−1 ) and this completes the proof.  We now state a higher order form on the Fundamental Lemma of the Calculus of Variations. Lemma 7.4 (High Order Form of the FLCV) Let p ≥ 1 be any integer. If v (·) is piecewise continuous on [t0 , t1 ] and Zt1

v (s) η [p] (s) ds = 0

(7.51)

t0

V0p (t0 , t1 ),

for all η(·) ∈ . . . ,ap−1 such that

then there exist p constants a0 , a1 , a2 ,

v (t) = ap−1 tp−1 + ap−2 tp−2 + · · · + a2 t2 + a1 t + a0 except at a finite number of points. In particular, v (·) ∈ P W S p (t0 , t1 ) and at all points t where v [p−1] (·) is continuous v [p−1] (t) = (p − 1)!ap−1 . The converse is also true. Observe that this is a powerful result that implies the function v (·) ∈ P W S p (t0 , t1 ) is equal e.f. to a polynomial of degree p − 1. The proof of this result is nontrivial and will not be given here (see pages 112 to 117 in Reid’s book [153] and Lemma 13.1 (page 105) in Hestenes’ book [101]). However, if one assumes that v (·) ∈ P W S p (t0 , t1 ), the proof follows from an easy integration by parts and applying Lemma 7.3 above. Example 7.4 Consider the case where p = 2. Lemma 7.4 implies that if v(·) ∈ P W C(t0 , t1 ) and Z t1 v(s)¨ η (s)ds = 0 t0

for all η(·) ∈ V02 (t0 , t1 ) where V02 (t0 , t1 ) = {η(·) ∈ P W C(t0 , t1 ) :η(·), η(·) ˙ ∈ P W S(t0 , t1 ), η(t0 ) = η(t ˙ 0 ) = η(t1 ) = η(t ˙ 1 ) = 0},

Chapter 7. Extensions and Generalizations

245

then there are constants a1 and a2 such that v(t) = a1 t + a0 except at a finite number of points on [t0 , t1 ]. Thus, v(·) is equal to a linear function e.f. If one assumes that there are two piecewise continuous functions α(·) and β(·) such that Z t1 {α(s)¨ η (s) + β(s)η(s)} ds = 0 t0

RtRs for all η(·) ∈ V02 (t0 , t1 ), then setting γ(t) = 0 0 β(µ)dµ and integrating by parts twice one has Z t1 Z t1 {α(s)¨ η (s) + γ(s)¨ η (s)} ds {α(s)¨ η (s) + β(s)η(s)} ds = 0= t0 t0 Z t1 {α(s) + γ(s)} η¨(s)ds = t0

for all η(·) ∈ V02 (t0 , t1 ). Consequently, it follows that Z tZ s α(t) + γ(t) = α(t) + β(µ)dµ = a1 t + a0 , 0

0

or equivalently, α(t) = a1 t + a0 −

Z tZ 0

s

β(µ)dµ. 0

Thus, α(·) is twice differentiable (except at a finite number of points) and α ¨ (t) = −β(t), e.f. Let P W C(t0 , t1 ; Rn ) denote the space of all Rn -valued piecewise smooth functions defined on [t0 , t1 ]. In particular,    T x(·) = x1 (·) x2 (·) · · · xn (·) : n . P W C(t0 , t1 ; R ) = and xi (·) ∈ P W C(t0 , t1 ), i = 1, 2, . . . , n (7.52)

246

Chapter 7. Extensions and Generalizations

Likewise, let P W S(t0 , t1 ; Rn ) denote the space of all Rn -valued piecewise smooth functions defined on [t0 , t1 ]. In particular,    T x(·) = x1 (·) x2 (·) · · · xn (·) : n . P W S(t0 , t1 ; R ) = and xi (·) ∈ P W S(t0 , t1 ), i = 1, 2, . . . , n (7.53) n n Also, P W S0 (t0 , t1 ; R ) will denote the space of all R -valued piecewise smooth functions defined on [t0 , t1 ] satisfying zero boundary conditions x(t0 ) = 0 = x(t1 ) and we define V0 (t0 , t1 ; Rn ) to be V0 (t0 , t1 ; Rn )    T η(·) = η1 (·) η2 (·) · · · ηn (·) ∈ P W S(t0 , t1 ) : = . and ηi (tj ) = 0, i = 1, 2, . . . , n, j = 0, 1 (7.54) Note that V0 (t0 , t1 ; Rn ) = P W S0 (t0 , t1 ; Rn ) which leads to following extension of the FLCV Part A to the vector case. Lemma 7.5 Let p ≥ 1 be any integer. If v (·) = [v1 (·) v2 (·) · · · vn (·)]T ∈ P W C(t0 , t1 ; Rn ) is piecewise continuous on [t0 , t1 ] and Zt1

< v (s) , η (s) > ds = 0

(7.55)

t0

for all η(·) ∈ V0 (t0 , t1 ; Rn ), then v (·) ∈ P W S(t0 , t1 ; Rn ) and v (t) = 0,

(7.56)

except at a finite number of points. The converse is also true. Proof: Since v (·) is piecewise continuous on [t0 , t1 ], there exists a finite set of points t0 < tˆ1 < tˆ2 < ... < tˆk < t1 , in (t0 , t1 ) such that v (·) is continuous on each subinterval (tˆj , tˆj−1 ). Assume that v (·) is not zero on one of these intervals. In particular, assume that there is a point zˆ ∈ (tˆj , tˆj−1 ) and m, with 1 ≤ m ≤ n so

247

Chapter 7. Extensions and Generalizations

that vm (ˆ z ) > 0. Since vm (·) is continuous on (tˆj , tˆj−1 ), there is an δ > 0 and a < b such that (ˆ z − δ, zˆ + δ) ⊂ [a, b] ⊂ (tˆj , tˆj−1 ) and vm (t) > 0 for all t ∈ (ˆ z − δ, zˆ + δ). Define the function  [(t − a)(b − t)]p , t ∈ [a, b] ηm (t) = 0, t∈ / [a, b] and let η(·) ˜ note the function η˜ (·) = V0 (t0 , t1 ; Rn ) defined by  ηm (t), ηj (t) = 0,



η1 (·) η2 (·) · · ·

ηn (·)

T

∈

j=m . j 6= m

Observe that η(·) ˜ has the following properties: (i) η(·) ˜ ∈ V0 (t0 , t1 ; Rn ), (ii) ηm (t) > 0 for all t ∈ (ˆ z − δ, zˆ + δ), (iii) vm (t) ηm (t) > 0 for all t ∈ (ˆ z − δ, zˆ + δ). Consequently, it follows that Zt1

< v(s), η (s) > ds =

Zt1

vm (s) ηm (s) ds

t0

t0

Zzˆ+δ vm (s) ηm (s) ds > 0, ≥ zˆ−δ

which contradicts the assumption (7.55). Hence, v (t) = 0 on each of the subintervals (tˆj , tˆj−1 ) and this completes the proof.  Lemma 7.6 (FLCV: Part A in Vector Form)  T Let p ≥ 1 be any integer. If v (·) = v1 (·) v2 (·) · · · vn (·) ∈ P W C(t0 , t1 ; Rn ) is piecewise continuous on [t0 , t1 ] and Zt1

t0

< v(s), η˙ (s) > ds = 0

(7.57)

248

Chapter 7. Extensions and Generalizations

for all η(·) ∈ V0 (t0 , t1 ; Rn ), then v (·) ∈ P W S(t0 , t1 ; Rn ) and there is a constant vector c such that v (t) = c,

(7.58)

except at a finite number of points. The converse is also true. Proof: Assume that v (·) is piecewise continuous on [t0 , t1 ]. Let η(·) be any piecewise smooth function in P W S0 (t0 , t1 ) and define the function η(·) ∈ V0 (t0 , t1 ; Rn ) by  η(t), j=m ηj (t) = . 0, j 6= m Observe that that η(·) ∈ V0 (t0 , t1 ; Rn ) and hence Zt1

vm (s) η(t)ds ˙ =

Zt1

< v(s), η˙ (s) > ds = 0.

t0

t0

Since η(·) ∈ P W S0 (t0 , t1 ) is arbitrary, it follows from the Fundamental Lemma of the Calculus of Variations that there is a constant cm such that vm (t) = cm except at a finite number of points. This can be repeated for each m = 1, 2, · · · , n and if T  c = c1 c2 · · · cn , then v (t) = c except at a finite number of points and this completes the proof.  The following lemma follows immediately from the previous result. Lemma 7.7 (FLCV: Part B in Vector Form)  T Let p ≥ 1 be any integer. If α (·) = α1 (·) α2 (·) · · · αn (·) and  T β (·) = β1 (·) β2 (·) · · · βn (·) ∈ P W C(t0 , t1 ; Rn ) are piecewise continuous on [t0 , t1 ] and Zt1

t0

{hα(s), η (s)i + hβ(s), η˙ (s)i} ds = 0

(7.59)

249

Chapter 7. Extensions and Generalizations

for all η(·) ∈ V0 (t0 , t1 ; Rn ), then β (·) ∈ P W S(t0 , t1 ; Rn ) and there is a constant vector c such that Z t β (t) = c+ α (s) ds, (7.60) t0

except at a finite number of points. The converse is also true. We can now formulate the general vector and higher order forms of the Simplest Problem in the Calculus of Variations.

7.4.2

The Simplest Problem in Vector Form

Let t0 and t1 be given and suppose the function f : [t0 , t1 ] × Rn × Rn −→ R is C 2 . In particular, f (t, x, u) has the form f (t, x, u) = f (t, x1 , x2 , . . . , xn , u1 , u2 , . . . , un ), where all the partial derivatives of order 2 exist and are continuous. Here, ∂f (t, x, u) = f t (t, x, u) = f t (t, x1 , x2 , . . . , xn , u1 , u2 , . . . , un ), ∂t ∂f (t, x, u) = f xi (t, x, u) = f xi (t, x1 , x2 , . . . , xn , u1 , u2 , . . . , un ) ∂xi and ∂f (t, x, u) = f ui (t, x, u) = f ui (t, x1 , x2 , . . . , xn , u1 , u2 , . . . , un ). ∂ui We use standard notation for the gradients   f x1 (t, x, u)  f (t, x, u)    x2 ∇x f (t, x, u) ,   ..   . f xn (t, x, u)

and



  ∇u f (t, x, u) ,  

f u1 (t, x, u) f u2 (t, x, u) .. . f un (t, x, u)



  . 

250

Chapter 7. Extensions and Generalizations

 T For each x(·) = x1 (·) x2 (·) · · · xn (·) ∈ P W S(t0 , t1 ; Rn ), define the functional J : P W S(t0 , t1 ; Rn ) → R by J (x(·)) =

Zt1

f (s, x (s) , x˙ (s))ds.

(7.61)

t0

Assume that the vectors x0 and x1 are given. Define the set of piecewise smooth vector functions Θn by Θn = {x(·) ∈ P W S(t0 , t1 ; Rn ) : x (t0 ) = x0 , x (t1 ) = x1 }. (7.62) The Simplest Problem in Vector Form (the fixed endpoint problem) is the problem of minimizing J (·) on Θn . In particular,  T the goal is to find x∗ (·) = x∗1 (·) x∗2 (·) · · · x∗n (·) ∈ Θn such that ∗

J (x (·)) =

Zt1

f (s, x∗ (s), x˙ ∗ (s)) ds

t0

≤ J (x(·)) =

Zt1

f (s, x(s), x˙ (s)) ds

t0

for all x (·) ∈ Θn . Theorem 7.5 (Vector Form of the Euler Necessary Condition) If x∗ (·) ∈ Θn minimizes J (·) on Θn , then (1) there is a vector c such that ∗

∗

[∇u f (t, x (t) , x˙ (t))] = c+

Zt

[∇x f (s, x∗ (s), x˙ ∗ (s))] ds, (7.63)

t0

except at a finite number of points, (2) x∗ (t0 ) = x0 ,

251

Chapter 7. Extensions and Generalizations (3) x∗ (t1 ) = x1 .

(4) Between corners of x∗ (·) the function ∇u f (t, x∗ (t), x˙ ∗ (t)) is differentiable and d [∇u f (t, x∗ (t), x˙ ∗ (t))] = [∇x f (t, x∗ (t), x˙ ∗ (t))] . dt

(7.64)

Proof: Suppose that x∗ (·) ∈ Θn minimizes J (·) on Θn and η(·) ∈ P W S(t0, t1 ; Rn ). Define the function g (t, ε) = f (t, x∗ (t) + εη(t), x˙ ∗ (t) + ε η˙ (t)) . Since x˙ ∗ (t) and η˙ (t) are continuous on a finite partition t0 = tˆ1 < tˆ2 < . . . < tˆk = t1 of [t0 , t1 ], it follows that g (t, ε) and g ε (t, ε) are both continuous on each subinterval (tˆi , tˆi+1 ). Without loss of generality we may assume that x∗ (·) and η(·) are smooth. It follows that F (ε) = J (x∗ (·) + εη(·)) Zt1 Zt1 ˙ = g(s, ε)ds = f (s, x∗ (s) + εη(s), x˙ ∗ (s) + εη(s))ds t0

t0

and the goal is to differentiate F (ε) at ε = 0. Applying the chain rule we have d [f (t, x∗ (t) + εη(t), x˙ ∗ (s) + εη(t))] ˙ dε n X = f xi (t, x∗ (t) + εη(t), x˙ ∗ (t) + εη(t)) ˙ ηi (t)

g ε (t, ε) =

i=1

˙ η˙ i (t) . + f ui (t, x∗ (t) + εη(t), x˙ ∗ (t) + εη(t))

Applying Leibniz’s Lemma 3.2 yields Zt1 d d F (ε) [J (x∗ (·) + εη(·))] = = g ε (s, 0)ds. (7.65) dε dε ε=0 ε=0 t0

252

Chapter 7. Extensions and Generalizations

Again, note that we really only need to compute g ε (t, ε) on each subinterval (tˆi , tˆi+1 ) where both x˙ ∗ (·) and η(·) ˙ are continuous and the chain rule produces an explicit formula for the first variation δJ(x∗ (·); η(·)) of J(·) at x∗ (·) in the direction of η(·). In particular, ∗

δJ (x (·); η(·)) =

Zt1 X n

f xi (s, x∗ (s) + εη(s), x˙ ∗ (s)

i=1

t0

+ εη(s))η ˙ i (s) ds Zt1 X n + f ui (s, x∗ (s) + εη(s), x˙ ∗ (s) t0

i=1

+ εη(s)) ˙ η˙ i (s) ds. Consequently, it follows that ∗

δJ (x (·); η(·)) =

Zt1

{h∇x f (s, x∗ (s), x˙ ∗ (s)) , η(s)i

t0

+ h∇u f (s, x∗ (s), x˙ ∗ (s)) , η(s)i}ds. ˙ In addition, if η(·) ∈ P W S0 (t0, t1 ; Rn ), then Zt1

{h∇x f (s, x∗ (s), x˙ ∗ (s)) , η(s)i

t0

+ h∇u f (s, x∗ (s), x˙ ∗ (s)) , η(s)i}ds ˙ =0 and the theorem follows from the Fundamental Lemma Part B in Vector Form given in Lemma 7.7 above. 

7.4.3

The Simplest Problem in Higher Order Form

Assume p ≥ 1 is a given integer and the endpoints t0 and t1 are fixed. Let f : [t0 , t1 ] × R1+p −→ R be a C 2 real valued function of the form f (t, x, u1 , u2 , . . . , up ),

Chapter 7. Extensions and Generalizations

253

where all the partial derivatives of order 2 exist and are continuous. As above, we let P W S p (t0 , t1 ) denote the space of all real-valued piecewise smooth functions x(·) defined on [t0 , t1 ] such that x[k] (·) is piecewise smooth on [t0 , t1 ] for all k = 0, 1, . . . , p − 1. In particular,  P W S p (t0 , t1 ) = x(·) : x[k] (·) ∈ P W S(t0 , t1 ), k = 0, 1, . . . , p − 1 . (7.66) Observe that if x(·) ∈ P W S p (t0 , t1 ), then x[p] (·) ∈ P W C(t0 , t1 ). For each x(·) ∈ P W S p (t0 , t1 ), define the functional J : P W S p (t0 , t1 ) → R by J(x(·)) =

Zt1

t0


f s, x(s), x(s), ˙ x¨(s), . . . , x[p] (s) ds.

(7.67)

Since x[p] (·) ∈ P W C(t0 , t1 ), the cost function defined by (7.67) is well defined. Assume that the points xk,0 and xk,1 for k = 0, 1, 2, . . . , p − 1 are given and define the set of functions Θp by Θp = {x(·) ∈ P W S p (t0 , t1 ) : x[k] (ti ) = xk,i , k = 0, 1, 2, . . . , p − 1, i = 0, 1}. (7.68)

The Simplest Problem in Higher Order Form (the fixed endpoint problem) is the problem of minimizing J(·) on Θp . In particular, the goal is to find x∗ (·) ∈ Θp such that ∗

J(x (·)) =

Zt1

t0


f s, x∗ (s), x˙ ∗ (s), x¨∗ (s), . . . , (x∗ )[p] (s) ds

≤ J(x(·)) =

Zt1

t0


f s, x(s), x(s), ˙ x¨(s), . . . , x[p] (s) ds,

for all x (·) ∈ Θp . Remark 7.3 Note that the Simplest Problem in Higher Order Form is a special case of the Simplest Problem in Vector Form. To make this precise, we set x1 (·) = x(·), x2 (·) = x(·), ˙ x3 (·) = x¨(·), . . . , xp (·) = x[p−1] (·)

254

Chapter 7. Extensions and Generalizations

so that x˙ p (·) = x[p] (·). Define fˆ : [t0 , t1 ] × Rp × Rp −→ R by fˆ(t, x, u)) = fˆ(t, x1 , x2 , . . . , xp , u1 , u2 , . . . , up ) , f (t, x, u1 , u2 , . . . , up ) and observe that for x(·) =



x(·) x(·) ˙ ···

x[p−1] (·)

fˆ(t, x(t), x(t)) ˙ has the form

T

=



x1 (·) x2 (·) · · ·

xp (·)

T

fˆ(t, x(t), x(t)) ˙ = fˆ(t, x1 (t), x2 (t), . . . , xp (t), x˙ 1 (t), x˙ 2 (t), . . . , x˙ p (t))
= f t, x(t), x(t), ˙ x¨(t), . . . , x[p] (t) .

Consequently, the cost function (7.67) becomes ˆ J(x(t)) =

Zt1

fˆ(s, x(s), x(s))ds ˙

t0

=

Zt1

t0


f s, x(s), x(s), ˙ x¨(t), . . . , x[p] (s) ds = J(x(·)),

where f (t, x1 , x2 , . . . , xp , u1 , u2 , . . . , up ) does not explicitly depend uk for k = 1, 2, . . . , p − 1. In order to simplify expressions, recall that we use f ∗ (t) to denote the evaluation of f (·) along the optimal curve so that
f ∗ (t) = f t, x∗ (t), x˙ ∗ (t), x¨∗ (t), . . . , (x∗ )[p] (t) ,
fx∗ (t) = fx t, x∗ (t), x˙ ∗ (t), x¨∗ (t), . . . , (x∗ )[p] (t) ,
fu∗i (t) = fui t, x∗ (t), x˙ ∗ (t), x¨∗ (t), . . . , (x∗ )[p] (t) , i = 1, 2, ...p. Also, we set

fu∗0 (t) = fx∗ (t).

,

255

Chapter 7. Extensions and Generalizations

Theorem 7.6 (High Order Form of the Euler Necessary Condition) If x∗ (·) ∈ Θp minimizes J(·) on Θp , then

(1) there are constants ci , i = 1, 2, . . . , p − 1 such that for all t ∈ [t0 , t1 ], fu∗p

p−1 X (i − t)p−1−i ci (t) = (p − 1 − i)! i=0

(7.69)

 Zt X p−1  (s − t)p−1−i ∗ + fui (s) ds (p − 1 − i)! i=0 t0

except at a finite number of points, (2) x[k] (t0 ) = xk,0 , k = 0, 1, 2, . . . , p − 1, (3) x[k] (t1 ) = xk,1 , k = 0, 1, 2, . . . , p − 1.

(4) Between corners of (x∗ )[p−1] (·), x∗ (·) satisfies p−1 i i X p−i h dp h ∗ p−i d ∗ (−1) (−1) p fup (t) + fup−i (t) + fx∗ (t) = 0, p−i dt dt i=1 (7.70) where p

 dk ∗
dk  ∗ ∗ ∗ ∗ ∗ [p] f (t) = [f t, x (t), x ˙ (t), x ¨ (t), . . . , (x ) (t) ]. u u k dk dk k

7.5

Problems with Constraints: Isoperimetric Problem

Here we impose a “functional” constraint on the curve x(·). For example, consider the problem of finding x∗ (·) ∈ P W S(t0 , t1 ) such that x∗ (·) minimizes Z t1 f (s, x(s), x˙ (s)) ds, J (x(·)) = t0

subject constraint x (t0 ) = x0 ,

x (t1 ) = x1 ,

256

Chapter 7. Extensions and Generalizations

and G (x(·)) =

Z

t1

g (s, x(s), x˙ (s)) ds = 0.

(7.71)

t0

Example 7.5 Find the curve of length 5 that passes through the points [−1 0]T and [1 0]T and minimizes Z 1 −x (s) ds J (x(·)) = −1

subject to to the length constraint Z 1p 1 + [x˙ (s)]2 ds = 5. −1

Observe that

G (x(·)) =

Z

1

g (s, x(s), x˙ (s)) ds, −1

where g(t, x, u) =

p

1 + [u]2 − 5/2.

In order to obtain a first order necessary condition for this constrained problem, we recall the definition of an extremal. An extremal for the functional Z t1 f (s, x(s), x˙ (s)) ds J (x(·)) = t0

is any function x(·) ∈ P W S(t0 , t1 ) that satisfies the integral form of the Euler equation fu (t, x(t), x˙ (t)) = c +

Zt

fx (s, x(s), x˙ (s)) ds

t0

for some c. Note that because of the Fundamental Lemma of the Calculus of Variations, x(·) ∈ P W S(t0 , t1 ) is an extremal for J(·) if and only if δJ(x(·); η(·)) = 0 (7.72)

Chapter 7. Extensions and Generalizations

257

for all η(·) ∈ P W S0 (t0 , t1 ). Again, δJ(x(·); η(·)) is the first variation of J(·) at x(·) in the direction of η(·) and is given by δJ(x(·); η(·)) =

Zt1

{fx (s, x(s), x˙ (s)) η (s)

t0

+ fu (s, x(s), x˙ (s)) η˙ (s)}ds.

(7.73)

Likewise, we say that x(·) ∈ P W S(t0 , t1 ) is an extremal for the functional Z t 1

G (x(·)) =

g (s, x(s), x˙ (s)) ds

t0

if x(·) ∈ P W S(t0 , t1 ) satisfies the Euler Integral Equation gu (t, x(t), x˙ (t)) = c +

Zt

gx (s, x(s), x˙ (s)) ds

t0

for some c. Thus, the Fundamental Lemma of the Calculus of Variations implies that x(·) ∈ P W S(t0 , t1 ) is an extremal for G(·) if and only if δG(x(·); η(·)) = 0 (7.74) for all η(·) ∈ P W S0 (t0 , t1 ), where δG(x(·); η(·)) is the first variation of G(·) at x(·) in the direction of η(·) and is given by δG(x(·); η(·)) =

Zt1

{gx (s, x(s), x˙ (s)) η (s)

t0

+ gu (s, x(s), x˙ (s)) η˙ (s)}ds.

(7.75)

We now can state a necessary condition which is an infinite dimensional form of the Lagrange Multiplier Theorem. We outline a proof in Section 7.5.1 below. Theorem 7.7 (Multiplier Theorem for the Isoperimetric Problem) If x∗ (·) ∈ P W S(t0 , t1 ) is smooth and minimizes Z t1 f (s, x(s), x(s)) ˙ ds, J (x(·)) = t0

258

Chapter 7. Extensions and Generalizations

subject to x (t0 ) = x0 , and G (x(·)) =

Z

x (t1 ) = x1 ,

t1

g (s, x(s), x(s)) ˙ = 0, t0

then there exist constants λ0 and λ1 such that (i) |λ0 | + |λ1 | = 6 0 and

(ii) x∗ (·) is an extremal of λ0 J (x(·)) + λ1 G (x(·)) .

(7.76)

(iii) If in addition x∗ (·) is not an extremal of G(·), then the con-

stant λ0 is not zero.

To make the notation more compact we define the Lagrangian by L(λ0 , λ1 , x(·)) = λ0 J (x(·)) + λ1 G (x(·)) .

(7.77)

If we define l(λ0 , λ1 , t, x, u) by l(λ0 , λ1 , t, x, u) = λ0 f (t, x, u) + λ1 g(t, x, u), then L(λ0 , λ1 , x(·)) =

Z

t1

l (λ0 , λ1 , s, x(s), x(s)) ˙ ds.

(7.78)

t0

Remark 7.4 The statement that x∗ (·) is an extremal of L(λ0 , λ1 , x(·)) = λ0 J (x(·)) + λ1 G (x(·)) implies that x∗ (·) is a solution of the corresponding Euler equation Zt ˙ [λ0 fu (t, x(t), x(t)) ˙ + λ1 gu (t, x(t), x(t))]= ˙ c + [λ0 fx (s, x(s), x(s))]ds t0

Zt ˙ + [λ1 gx (s, x(s), x(s))]ds t0

Chapter 7. Extensions and Generalizations

259

for some c. Also, if x∗ (·) is smooth then x∗ (·) satisfies the differential equation d [λ0 fu (t, x(t), x(t)) ˙ + λ1 gu (t, x(t), x(t))] ˙ dt = [λ0 fx (t, x(t), x(t)) ˙ + λ1 gx (t, x(t), x(t))] ˙ . In the case (iii) above, when, x∗ (·) is not an extremal of G(·), then the minimizer x∗ (·) is called a normal minimizer. Thus, if x∗ (·) is a normal minimizer, then λ0 6= 0. This definition of normality of a minimizing arc was first given by Bliss in [28] where he noted that (global) normality implied that the multiplier λ0 was not zero. As we shall see later, this idea is true for general Lagrange multiplier theorems.

7.5.1

Proof of the Lagrange Multiplier Theorem

In this section we prove Theorem 7.7. The proof is essentially the same as the proof for the necessary condition for the 2D Lagrange Multiplier Theorem in Section 2.2.3. Proof of Theorem 7.7: Assume x∗ (·) ∈ C 1 (t0 , t1 ) minimizes Z t1 J (x(·)) = f (s, x(s), x(s)) ˙ ds, t0

subject to x (t0 ) = x0 , and G (x(·)) =

Z

x (t1 ) = x1 ,

t1

g (s, x(s), x(s)) ˙ = 0. t0

If x∗ (·) is an extremal of G(·), then δG(x∗ (·); η(·)) = 0 for all η(·) ∈ P W S0 (t0 , t1 ). In this case set λ0 = 0 and λ1 = 1. It

260

Chapter 7. Extensions and Generalizations

follows that |λ0 | + |λ1 | = 1 6= 0 and δ [λ0 J(x∗ (·); η(·))] + δ [λ1 G(x∗ (·); η(·))] = λ0 δJ(x∗ (·); η(·)) + λ1 δG(x∗ (·); η(·)) = 0δJ(x∗ (·); η(·)) + δG(x∗ (·); η(·)) = δG(x∗ (·); η(·)) = 0, for all η(·) ∈ P W S0 (t0 , t1 ). Hence x∗ (·) is an extremal of λ0 J (x(·)) + λ1 G (x(·)) and the theorem is clearly true. Now consider the case where x∗ (·) is not an extremal of G(·). Since x∗ (·) is not extremal of G(·), there exists a η(·) ¯ ∈ ∗ P W S0 (t0 , t1 ) such that δG(x (·); η(·)) ¯ 6= 0. Clearly, η¯(·) is not the zero function since δG(x(·); 0(·)) = 0. Let λ0 = δG(x∗ (·); η¯(·)) 6= 0 and λ1 = −δJ(x∗ (·); η¯(·)).

We now show that x∗ (·) is an extremal of λ0 J(x(·))+λ1 G(x(·)) = [δG(x∗ (·); η¯(·))]J(x(·))+[δJ(x∗ (·); η¯(·))]G(x(·)). Observe that δ[λ0 J(x∗ (·); η(·)) + λ1 G(x∗ (·); η(·))] = λ0 [δJ(x∗ (·); η(·))] + λ1 [δG(x∗ (·); η(·))] = [δG(x∗ (·); η¯(·))][δJ(x∗ (·); η(·))] − [δJ(x∗ (·); η¯(·))][δG(x∗ (·); η(·))], or equivalently, δ[λ0 J(x∗ (·); η(·)) + λ1 G(x∗ (·); η(·))]   [δJ(x∗ (·); η(·))] [δJ(x∗ (·); η¯(·))] . = det [δG(x∗ (·); η(·))] [δG(x∗ (·); η¯(·))]

Chapter 7. Extensions and Generalizations

261

Therefore, to establish that x∗ (·) is an extremal of λ0 J(x(·)) + λ1 G(x(·)) = [δG(x∗ (·); η¯(·))]J(x(·)) + [δJ(x∗ (·); η¯(·))]G(x(·)) we must show that   [δJ(x∗ (·); η(·))] [δJ(x∗ (·); η¯(·))] =0 det [δG(x∗ (·); η(·))] [δG(x∗ (·); η¯(·))]

(7.79)

for all η(·) ∈ P W S0 (t0 , t1 ). This is accomplished by applying the Inverse Mapping Theorem 2.4. Define T : R2 → R2 by T (α, β) = [p(α, β) q(α, β)]T ,

(7.80)

p(α, β) = J(x∗ (·) + αη(·) + β η¯(·))

(7.81)

q(α, β) = G(x∗ (·) + αη(·) + β η¯(·)),

(7.82)

where and respectively. Note that T (α, β) maps the open set R2 to R2 and T (0, 0) = [p(0, 0) q(0, 0)]T = [J(x∗ (·)) 0]T = [ˆ p qˆ]T . Also, the Jacobian of ˆ T = [0 0]T is given by T (α, β) at [ˆ α β] #  "  ∂p(0,0) ∂p(0,0) [δJ(x∗ (·); η(·))] [δJ(x∗ (·); η¯(·))] ∂α ∂β . = ∗ ∗ ∂q(0,0) ∂q(0,0) [δG(x (·); η(·))] [δG(x (·); η ¯ (·))], ∂α ∂β Assume that (7.79) is not true. This assumption implies that the ˆ T = [0 0]T so we may Jacobian of T (α, β) is non-singular at [ˆ α β] apply the Inverse Mapping Theorem 2.4. Here, T (α, β) as defined ˆ T = [0 0]T and by (7.80), (7.81) and (7.82) with [ˆ α β] ˆ = T (0, 0) = [J(x∗ (·)) G(x∗ (·))]T = [ˆ T (ˆ α, β) p 0]T . o n p In particular, there is a neighborhood U = [α β]T : α2 + β 2 < γ

of [0 0]T and a neighborhood V of [J(x∗ (·)) 0]T such that the restriction of T (α, β) to U , T (α, β) : U → V, has a continuous inverse T −1 (p, q) : V → U belonging to C 1 .

262

Chapter 7. Extensions and Generalizations

˜T = Let [˜ p 0]T ∈ V be any point with p˜ < J(x∗ (·)) and let [˜ α β] ˜ T, T −1 (˜ p, 0) ∈ U. Observe that for [˜ α β] ˜ = p˜ < J(x∗ (·)) J(x∗ (·) + α ˜ η(·) + β˜η¯(·)) = p(˜ α, β) and

G(x∗ (·) + α ˜ η(·) + β˜η¯(·)) = 0.

(7.83) (7.84)

Therefore, by construction the function x˜(·) = x∗ (·) + α ˜ η(·) + β˜η¯(·) satisfies all the constraints of the Isoperimetric Problem and J(˜ x(·)) < J(x∗ (·)) which contradicts the assumption that x∗ (·) ∈ C 1 (t0 , t1 ) minimizes Z t1 f (s, x(s), x(s)) ˙ ds, J (x(·)) = t0

subject to x (t0 ) = x0 , and G (x(·)) =

Z

x (t1 ) = x1 ,

t1

g (s, x(s), x(s)) ˙ = 0. t0

Therefore the assumption that # "   ∂p(0,0) ∂p(0,0) [δJ(x∗ (·); η(·))] [δJ(x∗ (·); η¯(·))] ∂α ∂β det ∂q(0,0) ∂q(0,0) = det 6= 0 [δG(x∗ (·); η(·))] [δG(x∗ (·); η¯(·))] ∂α ∂β must be false. Hence, x∗ (·) is an extremal of λ0 J(x(·))+λ1 G(x(·)) = [δG(x∗ (·); η¯(·))]J(x(·))+[δJ(x∗ (·); η¯(·))]G(x(·)) which completes the proof. 

263

Chapter 7. Extensions and Generalizations

7.6

Problems with Constraints: Finite Constraints

Here we consider a non-integral constraint. Assume that the function g(t, x) is C 2 and consider the problem of minimizing Z t1 J (x(·)) = f (s, x(s), x(s)) ˙ ds, t0

subject to x (t0 ) = x0 ,

x (t1 ) = x1 ,

and the “finite constraint” g (t, x (t)) = 0.

(7.85)

Observe that equation (7.85) implies that the trajectory x(·) must lie on the curve  Mg = [t x]T : g(t, x) = 0 . (7.86)

Theorem 7.8 (Multiplier Theorem for the Finite Constraint Problem) If x∗ (·) ∈ P W S(t0 , t1 ) is smooth and minimizes Z t1 J (x(·)) = f (s, x(s), x(s)) ˙ ds, t0

subject to x (t0 ) = x0 ,

x (t1 ) = x1 ,

and the finite constraint g (t, x (t)) = 0, then there exists a constant λ0 and a function λ1 (·) such that (i) |λ0 | + |λ1 (t)| 6= 0 for all t0 ≤ t ≤ t1 and

(ii) x∗ (·) is an extremal of Z t1 (λ0 f (s, x(s), x(s)) ˙ + λ1 (s)g (s, x(s)))ds. L(λ0 , λ1 (·), x(·)) = t0

(7.87)

264

Chapter 7. Extensions and Generalizations

T (iii) If in addition, gx (t, x) 6= 0 for all [t x] on the surface T Mg = [t x] : g(t, x) = 0 , then λ0 6= 0. Thus, there exists a function λ(t) = λ1 (t)/λ0 such that x∗ (·) is an extremal of the functional Z t1 {f (s, x(s), x˙ (s)) + λ (s) g (s, x (s))}ds. t0

Let F (t, x, u) = λ0 f (t, x, u) + λ1 (t)g (t, x) so that F x (t, x, u) = λ0 fx (t, x, u) + λ1 (t)gx (t, x) and F u (t, x, u) = λ0 fu (t, x, u) . The statement that x∗ (·) is an extremal of Z t1 L(λ0 , λ1 (·), x(·)) = (λ0 f (s, x(s), x(s)) ˙ + λ1 (s)g (s, x(s)))ds t0

implies that x∗ (·) is a solution of the corresponding Euler Integral Equation λ0 fu (t, x(t), x(t)) ˙ = c+

Zt

(λ0 fx (s, x(s), x(s)) ˙ + λ1 (s)gx (s, x(s))) ds

t0

for some c. Also, between corners x∗ (·) satisfies d λ0 fu (t, x(t), x(t)) ˙ = [λ0 fx (t, x(t), x(t)) ˙ + λ1 (t)gx (t, x(t))] . dt Remark 7.5 It is worth noting that Theorem 7.8 does not provide information about the function λ1 (·). In particular, there is no claim that λ1 (·) is piecewise smooth or even piecewise continuous. In order to obtain additional information of this kind requires mathematical techniques beyond advanced calculus and will not be considered here.

Chapter 7. Extensions and Generalizations

265

We close this chapter with a very brief introduction to abstract optimization problems. Although this material is not required reading, it does help the reader to see how the idea of the first variation can be extended to rather general settings. It is extensions of this type that provide the foundations for the modern development of necessary and sufficient conditions for optimization with applications to optimal control.

7.7

An Introduction to Abstract Optimization Problems

In this section we provide an basic introduction to the theory of necessary conditions for general optimization problems. Generally speaking, the subjects of the calculus of variations and optimal control theory belong to the larger discipline of infinite dimensional optimization theory. Problems in the calculus of variations came from classical physics and date back 300 years. The theory of optimal control is relatively new and some mathematicians place its formal beginning in 1952, with Bushaw’s Ph.D. thesis (see Hermes and La Salle [100]). However, many of the basic ideas date back into the last century. Here we review some basic finite and infinite dimensional unconstrained and constrained optimization theory. The reader is referred to [102], [131] and [166] for details. The book by Neustadt [144] contains very general results on necessary conditions with applications to a wide variety of problems.

7.7.1

The General Optimization Problem

We shall consider problems that are special cases of the following general optimization problem. Let Z be a vector space and assume that Θ ⊆ Z is given a given (constraint) set. Also, let J : D(J ) ⊆ Z −→ R1 be a real valued function defined on the domain D(J ). The General Optimization Problem (GOP) is defined by: Find an element z ∗ ∈

266

Chapter 7. Extensions and Generalizations

Θ ∩ D(J ) such that J (z ∗ ) ≤ J (z) for all z ∈ Θ ∩ D(J ). Of course, specific problems have much more structure and the challenge is to formulate the problem in such a way that there is a solution and one can “compute” z ∗ . In addition to the Simplest Problem in the Calculus of Variations, this framework includes much of finite dimensional optimization as well as problems in optimal control. The following examples will be used to motivate the discussion in this section. Example 7.6 Let J (·) be a real-valued function defined on some interval I = [a, b]. In ordinary calculus one considers the problem of minimizing J (·) on [a, b]. In this example, Z = R1 and Θ = [a, b]. Example 7.7 To give another elementary example, let us consider the problem of finding the point on the plane with equation 2x + 3y − z = 5 which is nearest to the origin in R3 . In this problem, we set  Θ = [x y z]T : 2x + 3y − z − 5 = 0

and let J (·) be the square of the distance to the origin; namely J (z) = J (x, y, z) = x2 + y 2 + z 2 .

Here, Z = R3 is the vector space. Example 7.8 The problem is to find a piecewise continuous (control) function u(t), 0 ≤ t ≤ 3 such that the cost functional J (x(·), u(·)) =

Z3 0

[1 − u (s) · x (s)] ds

Chapter 7. Extensions and Generalizations

267

is minimized where x(·) satisfies the differential equation x˙ (t) = u (t) x (t) with initial condition x (0) = 1 and u(·) is constrained by 0 ≤ u (t) ≤ 1. This is the so called Farmer’s Allocation Problem. For this example we set X = P W S(0, 3) × P W C(0, 3) and define the constraint set Θ by   z(·) = [x(·) u(·)]T : u(·) ∈ P W C(0, 3), 0 ≤ u (t) ≤ 1, . Θ= x˙ (t) = u (t) x(t), x (0) = 1 The cost function is defined on J : Z = P W S(0, 3) × P W C(0, 3) −→ R1 by J (z(·)) = J (x(·), u(·)). We shall see later that there is a more efficient way to describe the constraint set Θ in terms of an equality and inequality constraint. Example 7.9 Returning to Example 2.6 we let J : R2 −→ R1 be defined by J (x, y) = x2 + y 2 and G : R2 −→ R1 be given by G (x, y) = x2 − (y − 1)3 . In this problem the constraint set is the “level curve” defined by  Θ = ΘG = z = [x y]T ∈ R2 : G (x, y) = 0 .

Recall that the minimizer is z ∗ = [x∗ y ∗ ]T = [0 1]T , but the Lagrange Multiplier Rule is not very helpful.

7.7.2

General Necessary Conditions

In this section we present the basic first order necessary conditions for the general optimization problem described above. The problem of finding the minimum of J (·) over a constraint set Θ can range from the trivial problem in Example (7.6) to the difficult optimal control problem in Example (7.8). The approach most often

268

Chapter 7. Extensions and Generalizations

used to “solve” such problems has been the application of necessary conditions for a minimum. The basic idea behind the use of a necessary condition is really very simple. A necessary condition is used to reduce the problem of minimizing J (·) over all of Θ to a problem of minimizing J (·) over a smaller set ΘS ⊆ Θ. We illustrate this idea with the following trivial example. Example 7.10 Consider the problem of minimizing the continuous function 2z 2 − z 4 + 3 J(z) = 4 on the compact interval Θ = [−2, 1]. From elementary calculus we know that there is a z ∗ ∈ Θ that minimizes J(·) on the compact interval [−2, 1]. Moreover, if J (z ∗ ) ≤ J (z) for all z ∈ Θ = [−2, 1] then J ′ (z ∗ ) = 0 if − 2 < z ∗ < 1 J ′ (z ∗ ) ≥ 0 if z ∗ = −2 J ′ (z ∗ ) ≤ 0 if z ∗ = 1. We use the first derivative test and solve for all z ∗ ∈ (−2, 1) satisfying dJ (z) = J ′ (z) = z − z 3 . 0= dz Clearly, z = −1 and z = 0 are the only solutions in the open interval (−2, 1). At the left endpoint, we find that J ′ (−2) = 6 > 0 and at the right endpoint J ′ (1) = 0 ≤ 0. Thus, at this point we have reduced the possible set of minimizers to ΘS = {−2, −1, 0, 1} .

It is now trivial to find the global minimum, since

J (−2) = −5/4, J (0) = 3/4, and J (−1) = J (1) = 1.

Chapter 7. Extensions and Generalizations

269

Thus, z ∗ = −2 is the global minimum of J(·) on Θ = [−2, 1]. Note that we could have reduced the set ΘS even more by applying the second order condition that d2 J (z) = J ′′ (z ∗ ) ≥ 0 if − 2 < z ∗ < 0. dz 2 For in this problem J ′′ (z) = 1 − 3z 2 ,

so J ′′ (−1) = −2 ≤ 0, while J ′′ (0) = 1 ≥ 0. Thus, z = −1 could have been removed from ΘS leading to the problem of minimizing J(·) on ΘSS = {−2, 0, 1} . Recall from Section 2.2.2 that the proof of the basic necessary condition used in the previous example is rather simple. However, the idea can be extended to very general settings. Consider the problem of minimizing J (·) on a set Θ ⊆ Z. Assume that z ∗ ∈ Θ∩D(J ) provides a (global) minimizer for J : D(J ) ⊆ Z −→ R1 on Θ ∩ D(J ). Thus, J (z ∗ ) ≤ J (z) , for all z ∈ Θ∩D(J ). Furthermore, assume that there is a function ϕ(·) : (−δ, +δ) → Θ ∩ D(J )

(7.88)

ϕ (0) = z ∗ .

(7.89)

satisfying Define F : (−δ, +δ) → R1 by F (ε) = J (ϕ(ε)), and observe that, since ϕ(ε) ∈ Θ ∩ D(J ) for all ε ∈ (−δ, +δ), F (0) = J (ϕ(0)) = J (z ∗ ) ≤ J (ϕ(ε)) = F (ε), for all ε ∈ (−δ, +δ). If the derivative d d ′ F (0) = F (ε) J (ϕ(ε)) = dε dε ε=0 ε=0

270 exists, then

Chapter 7. Extensions and Generalizations d = 0. J (ϕ(ε)) F (0) = dε ε=0 ′

(7.90)

Note that this is the basic necessary condition and to be “useful” one must construct functions ϕ(·) and have enough freedom in the choice of these functions so that the condition (7.90) implies something about the minimizer z ∗ ∈ Θ ∩ D(J ). In the following sections we show that this general framework can be applied to a wide variety of problems. Note that in all of the above examples Θ is a subset of a vector space Z. Assume that z ∗ ∈ Θ ∩ D(J ) provides a (global) minimizer for J : D(J ) ⊆ Z −→ R1 on Θ ∩ D(J ). We focus on variations ϕ(·) : (−δ, +δ) → Θ∩D(J ) of the form ϕ(ε) = z ∗ +εη where η ∈ Z is “limited” to a specific subset of Z. The key issue is to make sure that, for a suitable class of η, the variations ϕ(ε) = z ∗ + εη are admissible in the following sense. Definition 7.1 The vector ϕ(ε) = z ∗ + εη is called an admissible variation of z ∗ if z ∗ + εη ∈ Θ ∩ D(J ) on some nontrivial interval containing ε = 0. It is clear that some “directions” η may lead to admissible variations and others may not. Also, sometimes it is sufficient to assume that ϕ(·) : [0, +δ) → Θ ∩ D(J ) and consider only one sided derivatives at ε = 0. Thus, we are led to the following concepts. Definition 7.2 Suppose A is a subset of the vector space Z with z ∈ A and η ∈ Z. We say that z ∈ A is an internal point of A in the direction of η ∈ Z, if there is a δ = δ(η) > 0 such that (z + εη) ∈ A for all ε in the open interval −δ(η) < ε < δ(η). The point z ∈ A is called a radial point of A in the direction of η, if there is an δ = δ (η) > 0 such that (z + εη) ∈ A for all ε in the interval 0 ≤ ε < δ (η). The point z ∈ A is called a core point of A if z is an internal point of A for all directions η ∈ Z. Observe that the concept of internal, core and radial points are valid in a general vector space and require no notion of distance. In particular, there is no requirement that Z be a topological vector space.

Chapter 7. Extensions and Generalizations

7.7.3

271

Abstract Variations

Let J : D(J ) ⊆ Z −→ R1 . If z is an internal point of D(J ) in the direction of η then the function ϕ (ε) = z + εη maps the interval (−δ (η) , δ (η)) into D(J ). On the other hand, if z ∈ D(J ) is a radial point of D(J ) in the direction of η then ϕ (ε) = z + εη maps [0, δ(η)) into D(J ). If z ∈ D(J ), η ∈ Z and z is either an internal point or radial point of D(J ) in the direction of η , then we define the first variation of J (·) at z in the direction η by d , (7.91) J (z + εη) δJ (z; η) , dε ∈=0

provided the derivative exists. Here the derivative is two-sided if z is an internal point and one-sided if z is a radial point of D(J ). If the two-sided or one-sided second derivatives d2 J (z + εη) dε2 ∈=0

exist, then we say that J (·) has a second variation at z in the direction η and denote this by d2 2 . (7.92) δ J (z; η) , 2 J (z + εη) dε ∈=0

If z is a core point of D(J ) and the first variation of J (·) at z in the direction η exists for all η ∈ Z, then we say that J (·) has a Gˆ ateaux variation at z. In other words, if δJ (z; η) exists for all η ∈ Z, then δJ (z; η) is called the Gˆateaux variation (or weak differential) of J (·) at z in the direction η. Likewise, if δ 2 J (z; η) exists for all η ∈ Z, then δ 2 J (z; η) is called the second Gˆ ateaux variation (or weak second differential) of J (·) at z in the direction η (see [143] for details). Observe that J (·) has Gˆateaux variation at z ∈ D(J ) if and only if

272

Chapter 7. Extensions and Generalizations

1. z is a core point of D(J ) and 2. δJ (z; η) exists for all η ∈ Z.

Moreover, if J (·) has Gˆateaux variation at z ∈ D(J ), then the derivative (7.91) is two-sided since η ∈ Z implies that −η ∈ Z. It is now rather easy to establish a very general necessary condition for the abstract problem. In particular, assume that z ∗ ∈ Θ∩D(J ) provides a (global) minimizer for J : D(J ) ⊆ Z −→ R1 on Θ ∩ D(J ). In order for the necessary condition to be useful, the set of all η for which z ∗ ∈ Θ is an internal point or radical point of A , Θ ∩ D(J ) in the direction η needs to be sufficiently “large”. Thus, we define two spaces of admissible variations given by VAI = VAI (z ∗ ) , {η ∈ Z : z ∗ is an internal point of A in the direction of η} and VAR = VAR (z ∗ ) , {η ∈ Z : z ∗ is an radial point of A in the direction of η}. The following theorem provides the starting point for much of the theory of general necessary conditions in optimization. Theorem 7.9 (Fundamental Abstract Necessary Condition) Let J : D(J ) ⊆ Z −→ R1 be a real valued function defined on a vector space Z and assume that Θ ⊆ Z is a given subset of Z. Let A , Θ ∩ D(J ) and assume that z ∗ ∈ A satisfies J (z ∗ ) ≤ J (z) for all z ∈ A. (A) If η ∈ VAI and δJ (z ∗ ; η) exists, then

δJ (z ∗ ; η) = 0.

(7.93)

Moreover, if δ 2 J (z ∗ ; η) exists, then δ 2 J (z ∗ ; η) ≥ 0.

(7.94)

(B) If η ∈ VAR and δJ (z ∗ ; η) exists, then

δJ (z ∗ ; η) ≥ 0.

(7.95)

Chapter 7. Extensions and Generalizations

7.7.4

273

Application to the SPCV

To apply the abstract necessary condition given by Theorem 7.9 to the Simplest Problem in the Calculus of Variations, first note that Z = P W S(t0 , t1 ) is a vector space. Also, J : Z → R1 is defined on the domain D(J) = Z by J(x(·)) =

Zt1

f (s, x(s), x(s))ds. ˙

t0

Since the constraint set is Θ = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 , x (t1 ) = x1 } , it follows that xˆ(·) ∈ A , Θ ∩ D(J) is an internal point of A , Θ ∩ D(J) = Θ in the direction of η(·) if η(·) ∈ V0 , where V0 is the set of “admissible variations” given by V0 = {η(·) ∈ P W S(t0 , t1 ) : η(t0 ) = 0, η (t1 ) = 0} . Thus, if xˆ(·) ∈ A , Θ ∩ D(J), then V0 ⊂ VAI (ˆ x(·)) , {η(·) ∈ Z : xˆ(·) is an internal point of A in the direction of η(·)} and the first variation δJ(ˆ x(·); η(·)) exists. However, in general it is sometimes neither obvious nor easy to pick a “good” set of admissible variations. Note that for the Simplest Problem in the Calculus of Variations, the constraint set Θ = {x(·) ∈ P W S(t0 , t1 ) : x (t0 ) = x0 , x (t1 ) = x1 } , has no core points! For example, given any xˆ(·) ∈ Θ, let η¯(·) be the constant function η¯(t) ≡ 1. Since xˆ(t0 ) + ε¯ η (t0 ) = x0 + ε 6= x0 for any ε 6= 0, it follows that [ˆ x(·) + ε¯ η (·)] ∈ / Θ, and hence xˆ(·) can not be a core point of A , Θ ∩ D(J). Therefore, requiring that J(·) have Gˆateaux variation at a minimizer x∗ (·) ∈ Θ is too strong for even the Simplest Problem of the Calculus of Variations.

274

7.7.5

Chapter 7. Extensions and Generalizations

Variational Approach to Linear Quadratic Optimal Control

Although the second part of this book is devoted to optimal control problems, some of these problems may be formulated as abstract optimization problems and the variational approach as set out in Theorem 7.9 may be applied. To illustrate this we consider a simple linear quadratic control problem governed by the linear system (LS)

x(t) ˙ = Ax(t), +Bu(t),

0 < t ≤ 1,

(7.96)

where A is an n × n constant matrix and B is an n × m constant matrix. The initial conditions are given by x(0) = x0 ∈ Rm .

(7.97)

The function u(·) ∈ P W C(0, 1; Rm ) is called the control and the solution x(·) = x(·; u(·)) to the initial value problem (7.96) - (7.97) is called the state. Given a control u(·) ∈ P W C(0, 1; Rm ), the (quadratic) cost functional is defined by 1 J(x(s), u(·)) = 2

Z1 0



kx(s)k2 + ku(s)k2 ds,

(7.98)

where x(·) = x(·; u(·)) ∈ P W S(0, 1; Rn ) is the piecewise smooth solution to the initial value problem (7.96) - (7.97). In particular, At

x(t; u(·)) = e x0 +

Zt

eA(t−τ ) Bu(τ )dτ.

(7.99)

0

Therefore, 1 J(u(·)) = 2

Z1 0



kx(s)k2 + ku(s)k2





2 

Zs Z1  

As 1

e x0 + eA(s−τ ) Bu(τ )dτ + ku(s)k2 ds =

 2 

0

0

Chapter 7. Extensions and Generalizations

275

depends only on u(·) ∈ P W C(0, 1; Rm ). If one defines Z = P W C(0, 1; Rm ) and J : Z → R by J (u(·)) = J(u(·)) and the set of all admissible controllers by Θ = {u(·) ∈ P W C(0, 1; Rm )} ,

(7.100)

then the optimal control problem is equivalent to the general problem of minimizing J (·) on the set of all admissible controllers Θ. One can apply the abstract necessary condition given by Theorem 7.9 to this problem and obtain the standard necessary conditions. Observe that the domain of J(·), is the entire space D(J) = P W C(0, 1; Rm ). Remark 7.6 This variational approach is valid because there are no constraints on the control. In particular, Θ = P W C(0, 1; Rm ) is a vector space and any vector u∗ (·) ∈ P W C(0, 1; Rm ) is a core point of A = D(J ) ∩ P W C(0, 1; Rm ) = P W C(0, 1; Rm ). We note that the classical variational approach for optimal control was the first method used in optimal control of both ODE systems and systems governed by partial differential equations. Also, as long as the control is not constrained, as is the case above, this variational method works for nonlinear systems as well (see Lee and Markus [119], page 18 to 22).

7.7.6

An Abstract Sufficient Condition

The derivation of sufficient conditions is not as easy, nor is it a well developed subject. However, for certain convex problems (which almost never occur in the calculus of variations) there is a simple result. Again we assume that J : D(J ) ⊆ Z −→ R1 is a real valued function defined on a vector space Z and that Θ ⊆ Z is a given subset of Z. Definition 7.3 The set Θ is called convex if z 1 and z 2 ∈ Θ implies that for all λ ∈ [0, 1] [λz 1 + (1 − λ)z 2 ] ∈ Θ.

276

Chapter 7. Extensions and Generalizations

In particular, the line segment between z 1 and z 2 lies inside Θ. Figure 7.8 illustrates a convex set. On the other hand, the set shown in Figure 7.9 is not convex.

z1

O z1  (1  O ) z2 0 z2

Figure 7.8: A Convex Set Definition 7.4 If Θ is convex, then a function J : D(J ) ⊆ Z −→ R1 is said to be a convex function on Θ if Θ ∩ D(J ) is convex and for all z 1 , z 2 ∈ Θ ∩ D(J ) and 0 ≤ λ ≤ 1, J (λz 1 + (1 − λ) z 2 ) ≤ λJ (z 1 ) + (1 − λ) J (z 2 ). The function J (·) is said to be a strictly convex function on Θ if for all z 1 , z 2 ∈ Θ ∩ D(J ) and 0 < λ < 1, J (λz 1 + (1 − λ) z 2 ) < λJ (z 1 ) + (1 − λ) J (z 2 ). The following theorem applies to general abstract convex problems. Also, the proof is a rather simple extension of the ideas found in Section 2.2.2. Theorem 7.10 (Abstract Sufficient Condition for Convex Problems) Let

Chapter 7. Extensions and Generalizations

277

z1

O z1  (1  O ) z2 0 z2

Figure 7.9: A Non-convex Set Θ be convex and J (·) be a convex function on Θ. If z 0 ∈ Θ∩D(J ), δJ (z 0 ; η) exists and δJ (z 0 ; η) ≥ 0 for all η such that (z 0 + η) ∈ Θ ∩ D(J ), then J (·) has a minimum on Θ at z 0 . Moreover, if J (·) is strictly convex, z 0 is unique. Proof : If z ∈ Θ and 0 < λ ≤ 1, then λz + (1 − λ) z 0 ∈ Θ and J ([λz + (1 − λ) z 0 ]) ≤ λJ (z) + (1 − λ) J (z 0 ), or equivalently, J ([λz + (1 − λ) z 0 ]) − J (z 0 ) ≤ λ[J (z) − J (z 0 )]. Hence, J ([λz + (1 − λ) z 0 ]) − J (z 0 ) ≤ [J (z) − J (z 0 )], λ and it follows that J ([λz + (1 − λ) z 0 ]) − J (z 0 ) λ→0 λ ≤ [J (z) − J (z 0 )].

δJ (z 0 ; [z − z 0 ]) = lim [

278

Chapter 7. Extensions and Generalizations

However, z 0 + [z − z0 ] = z ∈ Θ, so that the assumption that 0 ≤ δJ (z 0 ; [z − z ∗ ])

implies that 0 ≤ δJ (z 0 ; [z − z 0 ]) ≤ [J (z) − J (z 0 )], Consequently, it follows that J (z 0 ) ≤ J (z)

(7.101)

and since (7.101) holds for any z ∈ Θ, z 0 minimizes J (·) on Θ. The second part of the theorem is trivial.  Remark 7.7 Although control of PDE systems lies outside of the scope of this book, we note that the abstract variational approach represented by Theorem 7.9 above was the basic method used in this field. As noted in the 1971 review paper by A. C. Robinson [156], most of the early work on necessary conditions for PDE control systems was in fact done by variational methods (see page 374 and the cited references). The idea of using abstract variational calculus to derive the necessary conditions for PDE systems was the main theme of the 1960’s and 1970’s literature. J. L. Lions certainly used this approach in his fundamental book [124]. This abstract approach was popular in the early Russian school as illustrated in [144] and [151].

7.8

Problem Set for Chapter 7

Apply the Euler Necessary Condition to the following free endpoint problems. Problem 7.1 Minimize the functional q Z1 c2 (1 + [x˙ (s)]2 ) − [v(s)]2 − v(s)x˙ (s) J(x(·)) = ds c2 − [v(s)]2 0

among all piecewise smooth functions satisfying x (0) = 0, and x (1) is free.

Chapter 7. Extensions and Generalizations

279

Problem 7.2 Minimize the functional J(x(·)) =

Z1

{[x˙ (s)]2 + [x (s)]2 + 2es x (s)}ds,

0

subject to the endpoint conditions x (0) = 0 and x (1) is free. Problem 7.3 Minimize the functional J(x(·)) =

Zπ

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) is free. Problem 7.4 Minimize the functional

J(x(·)) =

3π/2 Z

{[x˙ (s)]2 − [x (s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0 and x (3π/2) is free. Problem 7.5 Minimize the functional J(x(·)) =

Zb 0

x(s)

p 2 ds, 1 + [x(s)] ˙

subject to the endpoint conditions x (0) = 1 and x (b) is free. Problem 7.6 Minimize the functional J(x(·)) =

Z2

2 {[x(s)] ˙ − 2sx(s)}ds,

1

subject to the endpoint conditions x (1) = 0 and x (2) is free.

280

Chapter 7. Extensions and Generalizations

Problem 7.7 Minimize the functional J(x(·)) =

Zπ

2 {[x(s)]2 (1 − [x(s)] ˙ )}ds,

0

subject to the endpoint conditions x (0) = 0 and x (π) is free. Problem 7.8 Minimize the functional J(x(·)) =

Z3

{[3s − x(s)]x(s)}ds,

1

subject to the endpoint conditions x (1) = 1 and x (3) is free. Problem 7.9 Minimize the functional J(x(·)) =

Z2

{x(s)[1 ˙ + s2 x(s)]}ds, ˙

1

subject to the endpoint conditions x (1) is free and x (2) = 5. Problem 7.10 Minimize the functional J(x(·)) =

Z1

1 2 { [x(s)] ˙ + x(s)x(s) ˙ + x(s) ˙ + x(s)}ds, 2

0

subject to the endpoint conditions x (0) is free and x (1) is free. Problem 7.11 Consider the point to curve problem of minimizing the functional ) Zt1 ( p 2 1 + [x(s)] ˙ J(x(·)) = ds, x(s) 0

subject to the endpoint condition x (0) = 0 and x (t1 ) = ϕ(t1 ), where .ϕ(t) = t − 5. Use the necessary condition to find all possible minimizers.

281

Chapter 7. Extensions and Generalizations Problem 7.12 Minimize the functional J(x(·)) =

Z1

[¨ x(s)]2 ds,

0

subject to the endpoint conditions x (0) = 1, x(0) ˙ = 0, x(1) = 2 and x(1) ˙ = 0. Problem 7.13 Minimize the functional J(x(·)) =

Z1

{1 + [¨ x(s)]2 }ds,

0

subject to the endpoint conditions x (0) = 0, x(0) ˙ = 1, x(1) = 1 and x(1) ˙ = 1. Problem 7.14 Minimize the functional J(x(·)) =

R1

[¨ x(s)]2 ds,

−1

subject to the endpoint conditions x (−1) = 0, x(0) = 1, and x(1) = 0. Hint: Note that this requires an extension of the necessary condition for higher order problems. Problem 7.15 Minimize the functional J(x1 (·), x2 (·)) =

Zπ/2  0

[x1 (s)]2 + 2x1 (s)x2 (s) + [x2 (s)]2 ds,

subject to the endpoint conditions x1 (0) = 0, x1 (π/2) = 1, x2 (0) = 0, x2 (π/2) = 1.

Advanced Problems Problem 7.16 Prove the Euler Necessary Condition for the Free Endpoint Problem 7.1 assuming that x∗ (·) provides only a weak local minimum.

282

Chapter 7. Extensions and Generalizations

Problem 7.17 Consider the Euler Necessary Condition for the point to curve problem as stated in Theorem 7.4. Modify the proof to allow for piecewise smooth functions x(·) and ϕ(·). Problem 7.18 Prove the High Order Form of the FLCV as stated in Lemma 7.4. Hint: See Problem Set III.2 in Reid’s book [153]. Problem 7.19 Prove Theorem 7.7 assuming that x∗ (·) is piecewise smooth and provides only a weak local minimum. Problem 7.20 Assume one has a partition 0 = tˆ0 < tˆ1 < · · · < tˆp−1 < tˆp = 1 of [0, 1], and fixed points xˆ0 , xˆ1 , xˆ2 , xˆp−1 . Let J(·) be the functional Z1 x(s)]2 ds J(x(·)) = [¨ 0

and consider the problem of finding x∗ (·) to minimize J(·) on the set  ΘI = x(·) ∈ P W S 2 (0, 1) : x(tˆi ) = xˆi , for all i = 0, 1, . . . , p .

Derive the Euler Necessary Condition for this problem. Hint: Look into the theory of splines and check the references [3], [159] and [160].

Chapter 8 Applications In this chapter we apply the theory from the previous chapters to some specific problems. We begin with the brachistochrone problem.

8.1

Solution of the Brachistochrone Problem

Recall that the brachistochrone problem leads to the minimization of s Z a 1 + [x˙ (s)]2 ds, J (x(·)) = 2gx (s) 0 subject to x (0) = 0, x (a) = b. Here we assume that the minimizer has no corners so that x∗ (·) is C 2 . In order to solve this problem we note that the integrand does not depend explicitly on t. In particular, the functional has the form Z a f (x (s) , x˙ (s)) ds, J (x(·)) = 0

where

f (x, u) =

s

1 + u2 1 √ 1 + u2 , =√ 2gx 2gx 283

284

Chapter 8. Applications

and

u . 1 + u2 The Euler Differential Equation becomes fu (x, u) = f (x, u)

d fu (x∗ (t), x˙ ∗ (t)) = fx (x∗ (t), x˙ ∗ (t)) dt or in second order form (Hilbert’s Equation) fux (x∗ (t), x˙ ∗ (t))x˙ ∗ (t) + fuu (x∗ (t), x˙ ∗ (t))¨ x∗ (t) = fx (x∗ (t), x˙ ∗ (t)). (8.1) ∗ ∗ ∗ Again, recall that we shall use f (t), fu (t) and fx (t), etc. to denote the functions evaluated at (x∗ (t), x˙ ∗ (t)). Multiply both sides of (8.1) by x˙ ∗ (t) to obtain ∗ ∗ fux (t)[x˙ ∗ (t)]2 + fuu (t)¨ x∗ (t)x˙ ∗ (t) − fx∗ (t)x˙ ∗ (t) = 0.

(8.2)

However, d ∗ [fu (t)x˙ ∗ (t) − f ∗ (t)] = fu∗ (t)¨ x∗ (t) dt ∗ ∗ + [fux (t)x˙ ∗ (t) + fuu (t)¨ x∗ (t)]x˙ ∗ (t) − [fx∗ (t)x˙ ∗ (t) + fu∗ (t)¨ x∗ (t)] ∗ = fu∗ (t)¨ x∗ (t) + fux (t)[x˙ ∗ (t)]2 ∗ + fuu (t)¨ x∗ (t)x˙ ∗ (t) − fx∗ (t)x˙ ∗ (t) − fu∗ (t)¨ x∗ (t) ∗ = fux (t)[x˙ ∗ (t)]2 ∗ + fuu (t)¨ x∗ (t)x˙ ∗ (t) − fx∗ (t)x˙ ∗ (t) = 0. Hence, we have that d [fu (x∗ (t), x˙ ∗ (t))x˙ ∗ (t) − f (x∗ (t), x˙ ∗ (t))] = 0, dt or equivalently, [fu (x∗ (t), x˙ ∗ (t))x˙ ∗ (t) − f (x∗ (t), x˙ ∗ (t))] = c.

(8.3)

285

Chapter 8. Applications

Thus, equation (8.3) is equivalent to the Euler Integral Equation for the brachistochrone problem and hence the Euler Integral Equation has the equivalent form s ∗ 2 1 + [x˙ ∗ (t)]2 [x˙ (t)] p = c. − 2gx∗ (t) 2gx∗ (t) (1 + [x˙ ∗ (t)]2 ) Simplifying, we have that p

[x˙ ∗ (t)]2 x∗ (t) (1 + [x˙ ∗ (t)]2 )

or equivalently,

1 + [x˙ ∗ (t)]2

−p

x∗ (t) (1 + [x˙ ∗ (t)]2 )

=

p

2gc2 ,

p −1 p = 2gc2 . x∗ (t) (1 + [x˙ ∗ (t)]2 )

Solving for [x˙ ∗ (t)]2 we obtain the differential equation [x˙ ∗ (t)]2 =

1 a + x∗ (t) . , a = x∗ (t) 2gc2

Therefore, since the slope of x∗ (t) is non-positive we obtain the differential equation s a + x∗ (t) . x˙ ∗ (t) = − x∗ (t) Solving this differential equation is not an easy task. In fact we shall make the (not so obvious) substitution x∗ (t) = q [sin(θ (t) /2)]2 and observe that by the chain rule x˙ ∗ (t) = −2q sin(θ (t) /2) cos(θ (t) /2) Thus, s

q + x∗ (t) = x∗ (t) =

s

q [1 − [sin(θ (t) /2)]2 ] = q[sin(θ (t) /2)]2

[cos(θ (t) /2)] [sin(θ (t) /2)]

s

θ˙ (t) . 2

[cos(θ (t) /2)]2 [sin(θ (t) /2)]2

286

Chapter 8. Applications

and the differential equation for θ (t) becomes 2q sin(θ (t) /2) cos(θ (t) /2)

θ˙ (t) [cos(θ (t) /2)] = . 2 [sin(θ (t) /2)]

Multiplying by sin(θ (t) /2) and dividing out cos(θ (t) /2) yields q[sin(θ (t) /2)]2 θ˙ (t) = 1. However, the identity 2[sin(θ (t) /2)]2 = 1 − cos θ (t) implies that

q (1 − cos(θ (t)) θ˙ (t) = 1, 2

or equivalently, q˙ q θ (t) − cos θ (t) θ˙ (t) = 1. 2 2 We write this as

q dθ q dθ − cos θ = 1 2 dt 2 dt

so that

q q dθ − cos θdθ = dt 2 2 and integrating, we obtain q q θ − sin θ = t + t0 2 2

and since t0 = 0 we have q q θ − sin θ = t. 2 2 Thus, we have a parametric description of the solution given by q t = θ − sin θ 2 and

x = q [sin(θ/2)]2 =

q (1 − cos θ) . 2

287

Chapter 8. Applications The curve

q (θ − sin θ) , 2 q x = (1 − cos θ) , 2 is a cycloid and the graph of −x∗ (·) lies on this curve (recall that x∗ (·) is the distance from the curve to y = 0 axis) and hence the optimal curve is part of a cycloid. t=

8.2

Classical Mechanics and Hamilton’s Principle

What we call classical mechanics today was initiated in the late 1600’s by Newton and others. Newton changed our understanding of the universe by enumerating his three “Laws of Motion”: [First Law] Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it. [Second Law] The rate of change of momentum ( p = mass·velocity) is proportional to the impressed force and is in the direction in which the force acts: d[m(t)~v (t)] d~p(t) = . F~ (t) = dt dt [Third Law] For every action there is an equal and opposite reaction. Although the second law is often stated as F~ (t) = m~a(t),

(8.4)

which is valid under the assumption that the mass is constant, it is important to note the correct version. Also, around 300 BC Aristotle had stated a “Second Law of Motion” which, for constant mass, becomes F~ (t) = m~v (t). (8.5)

288

Chapter 8. Applications

Aristotle’s Second Law seems to be more in accord with common sense and even today we hear reports about the “force of impact” when a 3000 pound car traveling at 50 miles per hour hits a brick wall. The point is that “Laws of Motion” are not “laws”, but are mathematical models of observed behavior and often evolve and change over time as our understanding improves. In short, these are assumptions that lead to models that must be validated through experiments. In 1744 Euler showed Newton’s Second Law (8.4) could be obtained by another assumption called the Principle of Least Action and in 1788 Lagrange showed that, in the case of conservative forces, a major part of Newtonian mechanics could be derived from this principle. As we shall see below, the Principle of Least Action is not true for a general time interval (this was pointed out by Jacobi in 1842), but in 1834 and 1835 William Hamilton provided an alternative principle that extended the application to more general forces and set the stage for what we now call Hamiltonian mechanics. Suppose a particle of fixed constant mass m is  T located at the position x(t) = x1 (t) x2 (t) x3 (t) at time t  T and moves under a force F (t) = f1 (t) f2 (t) f3 (t) . By Newton’s Second Law, the force F (t) will cause the particle to move along a path in R3 such that m¨ x(t) = F (t), i.e. m¨ xi (t) = fi (t), i = 1, 2, 3.

(8.6)

The particle is said to be in a conservative field if there is a function U = U (t, x1 , x2 , x3 ) such that F (t) = −∇x U (t, x1 (t), x2 (t), x3 (t)),

(8.7)

or equivalently, fi (t) = −

∂ U (t, x1 (t), x2 (t), x3 (t)), i = 1, 2, 3. ∂xi

The function U : R × R3 → R is called the potential and P (t) = U (t, x1 (t), x2 (t), x3 (t)) = U (t, x(t))

(8.8)

289

Chapter 8. Applications

is called the potential energy of the particle. The kinetic energy of the particle is defined to be 3

1 1 X 2 ˙ . [x˙ i (t)]2 = m kx(t)k K(t) , m 2 i=1 2

(8.9)

The total energy of the particle is given by H(t) = K(t) + P (t).

(8.10)

The Lagrangian L(t, x, u) : R × R3 × R3 → R is defined by 1 L(t, x, u) = m kuk2 − U (t, x) 2

(8.11)

L(t) = L(t, x(t), x(t)) ˙ , K(t) − P (t)

(8.12)

so that is the difference between the kinetic and potential energy of the particle. Finally, given two times t0 < t1 , the action of the particle is defined to be the integral Z t1 Z t1 A(x(·)) , [K(s) − P (s)] ds = [L(s, x(s), x(s))] ˙ ds t0

t0

(8.13) and given the special form of the force and kinetic energy, it follows that # Z t1 " 3 1 X m [x˙ i (s)]2 − U (s, x(s)) ds. A(x(·)) = 2 t0 i=1

The Principle of Least Action states that the particle will move between x(t0 ) and x(t1 ) along a path x∗ (·) that minimizes the action integral. In particular, A(x∗ (·)) ≤ A(x(·)) for all curves joining x∗ (t0 ) and x∗ (t1 ). Note that this is a 3 dimensional version of the Simplest Problem of the Calculus of

290

Chapter 8. Applications

Variations and hence if x∗ (·) minimizes the action integral (8.13) then x∗ (·) must satisfy δA(x∗ (·); η(·)) = 0

(8.14)

for all η(·) ∈ P W S(t0 , t1 ; R3 ) satisfying η(t0 ) = η(t1 ) = 0. Consequently, by Euler’s Necessary Condition x∗ (·) satisfies Euler’s Differential Equation d ∇u L(t, x∗ (t), x˙ ∗ (t)) = ∇x L(t, x∗ (t), x˙ ∗ (t)) dt for t0 < t < t1 . Using the definition of 1 L(t, x, u) = m kuk2 − U (s, x) 2 3 1 X 2 = m [ui ] − U (s, x), 2 i=1 it follows that

and

 mu1 ∇u L(t, x, u) =  mu2  mu3 

∇x L(t, x, u) = −∇x U (t, x).

Hence, along the motion of the particle we have     x˙ 1 (t) mx˙ 1 (t) ∇u L(t, x∗ (t), x˙ ∗ (t)) =  mx˙ 2 (t)  = m  x˙ 2 (t)  x˙ 3 (t) mx˙ 3 (t)

and

 f1 (t) ∇x L(t, x∗ (t), x˙ ∗ (t)) = −∇x U (t, x∗ (t)) =  f2 (t)  f3 (t)

which becomes



   f1 (t) x˙ 1 (t) d m  x˙ 2 (t)  =  f2 (t)  , dt f3 (t) x˙ 3 (t) 

(8.15)

291

Chapter 8. Applications or equivalently, m¨ x(t) = F (t) which is Newton’s Second Law.

Example 8.1 Consider the one dimensional version where a particle of mass m = 1 moves under the force defined by the potential U (x) = κ2 x2 so that f (t) = −κx(t). The kinetic energy is given 2 ˙ and the Lagrangian is defined by by K(t) = 12 (x(t)) 1 L(x, u) = [u2 − κx2 ]. 2 Here the action integral becomes Z 1 t1 2 [(x(s)) ˙ − κ(x(s))2 ]ds A(x(·)) = 2 t0

(8.16)

and Euler’s equation is d [x(t)] ˙ = −κx(t). dt Therefore, the particle will move along a path x∗ (·) defined by the differential equation x¨∗ (t) + κx∗ (t) = 0. This is the same equation that comes directly from Newton’s Second Law. However, consider the case where κ = 1 and the particle starts at x0 = 0 at time t0 = 0 and stops at x1 = 0 at time t1 = 2π. We know from the Jacobi Necessary Condition - (IV) that the action integral (8.16) does not have a minimum on this interval. Thus, the Principle of Least Action does not always hold. Hamilton stated a new principle which is known as Hamilton’s Principle of Stationary Action. In particular, Hamilton’s principle states that the particle will move between x(t0 ) and x(t1 ) along a path x∗ (·) that makes the action integral stationary. In particular, A(x∗ (·)) has first variation zero for all η(·) ∈ P W S(t0 , t1 ; R3 )

292

Chapter 8. Applications

satisfying η(t0 ) = η(t1 ) = 0. Thus, Hamilton’s Principle of Stationary Action is equivalent to the condition that δA(x∗ (·); η(·)) = 0

(8.17)

all η(·) ∈ P W S(t0 , t1 ; R3 ) satisfying η(t0 ) = η(t1 ) = 0. Observe that Newton’s Second Law follows from (8.17) since the Fundamental Lemma of the Calculus of Variations implies that the Euler Differential Equation holds. Clearly, the Euler Differential Equation in this case reduces to m¨ x(t) = F (t), which is Newton’s Second Law.

8.2.1

Conservation of Energy

Here we again assume that a particle of mass m is moving in a conservative field with potential energy given by P (t) = U (t, x1 (t), x2 (t), x3 (t)) = U (t, x(t))

(8.18)

and kinetic energy given by 3

1 X [x˙ i (t)]2 . K(t) , m 2 i=1 Also, with L(t, x, u) = 12 m kuk2 − U (t, x) = U (t, x), let pi (t) be defined by pi (t) , mx˙ i (t) =

(8.19) 1 m 2

P3

i=1 [ui ]

∂ L(t, x(t), x(t)) ˙ ∂ui

2

−

(8.20)

and observe that the total energy H(t) = K(t) + P (t) can be written as 3

1 X H(t) = m [x˙ i (t)]2 + U (t, x(t)) 2 i=1 3

3

X 1 X [x˙ i (t)]2 + m [x˙ i (t)]2 + U (t, x(t)) =− m 2 i=1 i=1

293

Chapter 8. Applications 3

3

X 1 X [x˙ i (t)]2 + U (t, x(t)) + x˙ i (t)[mx˙ i (t)] =− m 2 i=1 i=1 = −[K(t) − U (t, x(t))] + = −L(t, x(t), x(t)) ˙ +

3 X

3 X

x˙ i (t)[

i=1

∂ L(t, x(t), x(t))] ˙ ∂ui

x˙ i (t)pi (t).

i=1

If we define the Hamiltonian function by H(t, x, u, p) , −L(t, x, u) +

3 X

ui pi ,

(8.21)

i=1

then H(t, x(t), x(t), ˙ p(t)) = −L(t, x(t), x(t)) ˙ +

3 X

x˙ i (t)pi (t) (8.22)

i=1

is the total energy of the system. Now consider the case where the potential is independent of time so that P (t) = U (x1 (t), x2 (t), x3 (t)) = U (x(t)) (8.23) P and since L(x, u) = 12 m kuk2 − U (x) = 12 m 3i=1 [ui ]2 − U (x), the Euler Differential Equation d ∇u L(x∗ (t), x˙ ∗ (t)) = ∇x L(x∗ (t), x˙ ∗ (t)) dt has the form d [mx˙ i (t)] = Lxi (x∗ (t), x˙ ∗ (t)), i = 1, 2, 3. dt Note that for i = 1, 2, 3, we have ∂ H(x, u, p) = −Lxi (x, u) ∂xi

294

Chapter 8. Applications

and

∂ H(x, u, p) = ui . ∂pi

Therefore, x˙ i (t) =

∂ H(x(t), x(t), ˙ p(t)) ∂pi

(8.24)

and ∂ d [mx˙ i (t)] = Lxi (x∗ (t), x˙ ∗ (t)) = − H(x(t), x(t), ˙ p(t)), dt ∂xi so that p˙i (t) = −

∂ H(x(t), x(t), ˙ p(t)). ∂xi

(8.25)

Combining (8.24) and (8.25) it follows that Euler’s Equation can be written as the system ∂ H(x(t), x(t), ˙ p(t)) ∂pi ∂ H(x(t), x(t), ˙ p(t)). p˙i (t) = − ∂xi

x˙ i (t) =

(8.26) (8.27)

Now we differentiate H(x(t), x(t), ˙ p(t)) with respect to time. In particular, since H(x(t), x(t), ˙ p(t)) = −L(x(t), x(t)) ˙ +

3 X

x˙ i (t)pi (t),

i=1

it follows that  3  X d ∂ H(x(t), x(t), ˙ p(t)) = H(x(t), x(t), ˙ p(t))x˙ i (t) dt ∂x i i=1  3  X ∂ H(x(t), x(t), ˙ p(t))¨ xi (t) + ∂u i i=1  3  X ∂ + H(x(t), x(t), ˙ p(t))p˙i (t) . ∂pi i=1

295

Chapter 8. Applications However, H(x, u, p) , −L(x, u) + so that

3 X

ui p i

i=1

∂ H(x, u, p) = −Lui (x, u) + pi = −mui + pi ∂ui and hence ∂ H(x(t), x(t), ˙ p(t)) = −mx˙ i (t) + pi (t) ≡ 0. ∂ui We now have proven the following result concerning the conservation of energy. Theorem 8.1 (Conservation of Energy) Assume the potential is independent of time so that the potential energy is given by P (t) = U (x1 (t), x2 (t), x3 (t)) = U (x(t)). If x(t) satisfies the Principle of Stationary Action (i.e. δA(x∗ (·); η(·)) = 0 for all η(·) ∈ P W S(t0 , t1 ; R3 ) satisfying η(t0 ) = η(t1 ) = 0), then the Hamiltonian is constant along the trajectory x(t). In particular, there is a constant H0 such that the total energy H(t) satisfies H(t) = H(x(t), x(t), ˙ p(t)) ≡ H0

8.3

A Finite Element Method for the Heat Equation

In this section we discuss how the Fundamental Lemma of the Calculus of Variations can be applied to the theory and numerical solutions of systems governed by partial differential equations (PDEs). The FLCV plays two important roles here. First, as noted in Section 3.4, the FLCV provides the motivation for defining “weak derivatives” which will be used to define “weak solutions” of the PDE. Moreover, this weak formulation of the PDE is the first step in developing numerical methods based on finite element

296

Chapter 8. Applications

schemes. We illustrate this approach for a simple PDE describing the heat flow in a uniform rod given by ∂2 ∂ θ (t, x) = k 2 θ (t, x) + g(x)u(t), t > 0, 0 < x < 1, (8.28) ∂t ∂x where u(t) is a heat source (control) and g(·) : (0, 1) → R1 is a given (piecewise continuous) function. We also specify boundary conditions θ (t, 0) = θ (t, 1) = 0, (8.29) and initial data θ (0, x) = ϕ (x) , 0 < x < 1.

(8.30)

A strong (or classical) solution is a function θ (·, ·) of t and x such ∂ ∂2 ∂ θ (t, x), ∂x θ (t, x) and ∂x that ∂t 2 θ (t, x) are continuous and θ (·, ·) satisfies (8.28) at every value 0 < x < 1 and all t > 0 and the boundary condition (8.29) at x = 0, x = 1 and all t > 0. We are interested in developing a numerical algorithm for approximating solutions. The finite element method can be used to approximate the PDE problem (8.28) - (8.30) in much as the same way it was employed in Section 2.4.5 above to approximate the two-point boundary value problem. Indeed, the two problems are linked and the finite element method in Section 2.4.5 can be extended to the heat equation above. Recall that the basic steps begin with multiplying both sides of (8.28) by an arbitrary function η(·) so that if θ (t, x) is a solution to (8.28), then for all t > 0 we have ∂2 ∂ θ (t, x) η(x) = k 2 θ (t, x) η(x) + b(x)u(t)η(x). ∂t ∂x If η(·) ∈ P W C(0, 1), then η(·) is integrable so one can integrate both sides to obtain  Z 1 ∂ θ (t, x) η(x)dx ∂t 0  Z 1 Z 1 2 ∂ [g(x)u(t)] η(x)dx k 2 θ (t, x) η(x)dx + = ∂x 0 0  Z 1  Z 1 2 ∂ k 2 θ (t, x) η(x)dx + = g(x)η(x)dx u(t). ∂x 0 0

297

Chapter 8. Applications

d If in addition, η(·) ∈ P W S(0, 1), then dx η(·) is integrable andh we can iuse integration by parts on the first term R1 ∂2 k ∂x 2 θ (t, x) η(x)dx which yields 0

Z

1

0



  x=1 ∂2 ∂ k 2 θ (t, x) η(x)dx = k θ (t, x) η(x) ∂x ∂x x=0   Z 1 d ∂ k θ (t, x) − η(x) dx. ∂x dx 0

Finally, if η(·) ∈ P W S0 (0, 1) so that η(0) = η(1) = 0, then 

∂ k θ (t, x) η(x) ∂x

x=1

=0

x=0

and it follows that    Z 1 Z 1 ∂ ∂ d k θ (t, x) θ (t, x) η(x)dx = − η(x) dx ∂t ∂x dx 0 0 (8.31) Z 1  + g(x)η(x)dx u(t), 0

for all η(·) ∈ P W S0 (0, 1). The equation (8.31) is called the weak (or variational) form of the heat equation defined by the heat equation (8.28) with boundary condition (8.29). Definition 8.1 (Weak Solution of the Heat Equation) We say that the function θ(t, ·) is a weak solution of the heat equation ∂ θ(t, x) are continuous (8.28)-(8.29), if for each t > 0, θ(t, x) and ∂t functions of t and x and ∂ θ(t, ·) ∈ P W S0 (0, 1), (1) θ(t, ·) ∈ P W S0 (0, 1), ∂t (2) θ(t, ·) satisfies (8.31) for all η(·) ∈ P W S0 (0, 1). As in Section 2.4.5 above, the finite element element method actually produces an approximation of the weak form of the heat equation. Observe that we have shown that a strong solution to the heat equation (8.28) - (8.29) is always a weak solution. Following the approach in Section 2.4.5, we focus on the simplest piecewise

298

Chapter 8. Applications

linear approximations and hence we divide the interval [0, 1] into N +1 subintervals (called elements) of length ∆x = 1/(N +1) with nodes 0 = xˆ0 < xˆ1 < xˆ2 < . . . < xˆN −1 < xˆN < xˆN +1 = 1, where for i = 0, 1, 2, . . . , N, N + 1, xˆi = i∆x . The approximation θN (t, x) will be continuous in x on all of [0, 1] and linear between the nodes. Since continuous piecewise linear approximating functions θN (t, x) are not typically differentiable in x at the nodes, it is not possible to insert this approximation directly into the equation (8.28). In particular, the piecewise smooth function θN (t, x) has only a ∂2 ∂ θN (t, x) and hence ∂x piecewise continuous derivative ∂x 2 θN (t, x) does not exist. In order to deal with this lack of smoothness, we use the weak form (8.31). Define the spatial hat functions hi (·) on [0, 1] as in 2.67 by  (ˆ x1 − x)/∆x , 0 ≤ x ≤ xˆ1 , h0 (x) = 0, xˆ1 ≤ x ≤ 1  (x − xˆN )/∆x , xˆN ≤ x ≤ 1 hN +1 (x) = , (8.32) 0, 0 ≤ x ≤ xˆN   (x − xˆi−1 )/∆x , xˆi−1 ≤ x ≤ xˆi (ˆ xi+1 − x)/∆x , xˆi ≤ x ≤ xˆi+1 , hi (x) =  0, x∈ / (ˆ xi−1 , xˆi+1 )

for i = 1, 2, . . . , N.

Plots of these hat functions are identical to the plots in Figures 2.24, 2.25 and 2.26 in Section 2.4.5 with the variable t replaced by the variable x. These hat functions provide a basis for all continuous piecewise linear functions with (possible) corners at the internal nodes 0 < xˆ1 < xˆ2 < . . . < xˆN −1 < xˆN < 1. Therefore, any continuous piecewise linear function θN (t, x) with corners only at these nodes can be written as θN (t, x) =

N +1 X

θi (t)hi (x),

(8.33)

i=0

where the functions θi (t) determine the value of θN (t, x) at x = xˆi . In particular, θN (t, xˆi ) = θi (t) and in order to form

299

Chapter 8. Applications

the function θN (t, ·) one must provide the coefficients θi (t) for i = 0, 1, 2, . . . , N, N + 1. Moreover, since θN (t, ·) is assumed to satisfy the Dirichlet boundary conditions (8.29), then θN (t, xˆ0 ) = θN (t, 0) = θ0 (t) = 0 and θN (t, xˆN +1 ) = θN (t, 1) = θN +1 (t) = 0 and θN (t, ·) can be written as θN (t, x) =

N X

θi (t)hi (x).

(8.34)

i=1

We seek an approximate continuous piecewise linear solution θN (t, ·) of the form (8.34) to the weak form of the two-point value problem (8.31). In order to compute the functions θi (t) for i = N P 1, 2, . . . , N , we substitute θN (t, x) = θi (t)hi (x) into the weak i=1

form of the equation given by (8.31). In particular, θN (t, x) is assumed to satisfy    Z 1 Z 1 ∂ d ∂ k θN (t, x) θN (t, x) η(x)dx = − η(x) dx ∂t ∂x dx 0 0 (8.35)  Z 1 g(x)η(x)dx u(t), + 0

for all η(·) ∈ P W S0 (0, 1). Observe that N P

i=1

∂ θ (t, x) ∂t N

=

N P θ˙i (t)hi (x) and

i=1

∂ θ (t, x) ∂x N

=

d θi (t) dx hi (x) is piecewise continuous so that substituting

θN (t, x) =

N P

i=1

Z

1

0

θi (t)hi (x) into the weak equation (8.35) yields "

N X

#

θ˙i (t)hi (x) η(x)dx

i=1

#  Z 1" X N d d η(x) dx k =− θi (t) hi (x) dx dx 0 i=1 Z 1  + g(x)η(x)dx u(t) 0

300

Chapter 8. Applications

for all η(·) ∈ P W S0 (0, 1). This equation can be written as  Z 1 N X hi (x)η(x)dx θ˙i (t) 0

i=1

Z

   d d = −k θi (t) hi (x) η(x) dx dx dx 0 i=1 Z 1  + g(x)η(x)dx u(t), N X

1



(8.36)

0

for all η(·) ∈ P W S0 (0, 1). In order to use the variational equation to compute the coefficients θi (t) for (8.34), we note that for i = 1, 2, . . . , N the basis function hi (·) belongs to P W S0 (0, 1). Therefore, setting η(·) = hj (·) ∈ P W S0 (0, 1) for each index j = 1, 2, . . . , N , yields N equations  Z 1 N X ˙θi (t) [hi (x)] [hj (x)] dx i=1

0

   d d hi (x) hj (x) dx = −k θi (t) dx dx 0 i=1  Z 1 g(x)hj (x)dx u(t). + Z

N X

1



(8.37)

0

Again, as in Section 2.4.5 we define the N × N mass matrix M = M N by M = M N = [mi,j ]i,j=i,2,...,N , where the entries mi,j of M N are given by the integrals  Z 1 [hi (x)] [hj (x)] dx . mi,j = 0

Likewise, define the N × N stiffness matrix K = K N by K = K N = [ki,j ]i,j=i,2,...,N , where the entries ki,j of K N are given by the integrals Z 1     d d hi (x) hj (x) dx . ki,j = dx dx 0

301

Chapter 8. Applications Finally, let g N be the N × 1 (column) vector defined by  T g N = g1 g2 · · · gN ,

where entries gj of g N are given by the integrals  Z 1 hj (x)g(x)dx . gj =

(8.38)

(8.39)

0

If θ N (t) is the solution vector



  θ N (t) =  

θ1 (t) θ2 (t) .. . θN (t)



  , 

of (8.37), then θ N (t) satisfies the matrix differential equation M N θ˙ N (t) = − kK N θ N + g N u(t). Defining and

(8.40)

AN = −k [M N ]−1 K N B N = [M N ]−1 g N

yields the linear control system θ˙ N (t) = AN θ N + B N u(t).

(8.41)

which must be solved to find the coefficients θi (t) for i = 1, 2, . . . , N . In order to find the correct initial data for the finite element model (8.41), one approximates θ (0, x) = ϕ (x), 0 < x < 1 by ϕN (x) =

N X

ϕi hi (x)

i=1

and selects the coefficients ϕi for i = 1, 2, . . . , N so that ϕN (x) is the “best” approximation of ϕ (x). To make this precise, define the subspace V0h (0, 1) ⊆ P W S0 (0, 1) by ( ) N X V0h (0, 1) = span {hi (x) : i = 1, 2, . . . , N } = ψ(x) = αi hi (x) . i=1

302

Chapter 8. Applications

Thus, we seek the function ϕN (x) =

N P

ϕi hi (x) such that ϕN (·)

i=1

minimizes

Z

1 0

|ϕN (x) − ϕ(x)|2 dx.

Elementary geometry implies that ϕN (·) − ϕ(·) must be “orthogonal to V0h (0, 1)” so that Z 1 [ϕN (x) − ϕ(x)]η(x))dx = 0 0

for all η(·) ∈ V0h (0, 1). In particular, Z 1 [ϕN (x) − ϕ(x)]hj (x)dx = 0 0

for all i = 1, 2, . . . , N so that # Z 1 "X N ϕi hi (x) − ϕ(x) hj (x)dx = 0 0

i=1

which implies N X

ϕi

i=1

Z



1 0

[hi (x)] [hj (x)] dx −

N Z X i=1

1

[ϕ(x)] [hj (x)] dx 0



= 0.

In matrix form we have [M N ]ϕN = ϕ ˜N , where ϕ˜j = and ϕ ˜N =

Z 

1

[ϕ(x)] [hj (x)] dx 0

ϕ˜1 ϕ˜2 · · ·

ϕ˜N



T

.

Therefore, the initial condition for the finite element model (8.41) is θ N (0) = ϕN

303

Chapter 8. Applications where ϕN = [M N ]−1 ϕ ˜N

(8.42)

and the finite dimensional “finite element” system is defined by θ˙ N (t) = AN θ N + B N u(t)

(8.43)

with initial condition θ N (0) = ϕN .

(8.44)

The finite element ODE system (8.41) can be solved using standard numerical methods. Again, the key to developing the ODE model (8.41) is to approximate the weak form of the PDE model (8.28) - (8.29). This is a fundamental idea that is the basis of the finite element method.

8.4

Problem Set for Chapter 8

Problem 8.1 Consider the problem of minimizing the functional J(x(·)) =

Zb 0



[x˙ (s)]2 − [x (s)]2 + f (s)x(s) ds

among all piecewise smooth functions satisfying x (0) = 0, x (b) = 0, where f (·) is a given continuous function. Assume x∗ (·) minimizes J(·). (A) Compute the first variation δJ(x∗ (·), η(·)) for this problem. (B) Write out the equation δJ(x∗ (·), η(·)) = 0. (C) Show that if x∗ (·) minimizes J(·) subject to x (0) = 0, x (1) = 0, then x∗ (·) is a weak solution to the two point boundary value problem x¨(t) + x(t) = f (t), x (0) = 0, x (b) = 0.

(8.45)

(D) Show that there are solutions of (8.45) that do not minimize J(x(·)).

304

Chapter 8. Applications

Problem 8.2 Consider the problem of minimizing the functional J(x(·)) =

Z1 0



[x˙ (s)]2 + 2x(s)x(s) ˙ + [x(s)]2 + 2es x (s) ds,

subject to the endpoint conditions x (0) = 0 and x (1) = 1. Assume x∗ (·) is piecewise smooth and minimizes J(·). (A) Compute the first variation δJ(x∗ (·), η(·)) for this problem. (B) Write out the equation δJ(x∗ (·), η(·)) = 0. (C) What two point boundary value problem will x∗ (·) satisfy? Problem 8.3 Consider the problem of minimizing the functional J(x(·)) =

Z3

{[3s − x(s)]x(s)} ds,

1

subject to the endpoint conditions x (1) = 1 and x (3) is free. Assume x∗ (·) is piecewise smooth and minimizes J(·). (A) Compute the first variation δJ(x∗ (·), η(·)) for this problem. (B) Write out the equation δJ(x∗ (·), η(·)) = 0. (C) What two point boundary value problem will x∗ (·) satisfy? Problem 8.4 Consider the problem of minimizing the functional J(x(·)) =

Z2 1



x(s)[1 ˙ + s2 x(s)] ˙ ds,

subject to the endpoint conditions x (1) is free and x (2) = 5. Assume x∗ (·) is piecewise smooth and minimizes J(·). (A) Compute the first variation δJ(x∗ (·), η(·)) for this problem. (B) Write out the equation δJ(x∗ (·), η(·)) = 0. (C) What two point boundary value problem will x∗ (·) satisfy? Problem 8.5 Apply Hamilton’s Principle to the idealized double pendulum described in Section 2.1.3 to derive the equations of motion for the system. Hint: Use polar coordinates and Figure 2.3 to determine the kinetic and potential energy.

305

Chapter 8. Applications Advanced Problems

Problem 8.6 Let k = .25, g(x) = x and assume the initial function is given by ϕ(x) = 1. Construct the finite element model (8.43) - (8.44) for N = 4, 8, 16, 32 and 64. Let u(t) = e−t and use a numerical method to solve the finite element equations (8.43) - (8.44) on the interval 0 < t ≤ 2. Hint: Since NP +1 i )hi (x), it follows that g(x) = x = ( N +1 i=1

gj =

Z

1

hj (x)g(x)dx 0



can be computed exactly. In particular, note  4 0 0, 0, η2∗ (t) = 0, u∗ (t) =  −1, η2∗ (t) < 0

and u∗ (t) = ±1 can change sign at most one time (when η2∗ (t) crosses the axis).

327

Chapter 9. Optimal Control Problems

To finish the problem we need to find a way to compute η2∗ (·). Note that the definitions (9.40)-(9.41) imply η˙ 1∗ (t) = 0, and

d {ˆ η1 (t∗1 − t) + ηˆ2 } = −ˆ η1 = −η1∗ (t). dt We can write this as a system of the form   ∗    ∗ η1 (t) 0 0 η˙ 1 (t) , = η2∗ (t) −1 0 η˙ 2∗ (t) η˙ 2∗ (t) =

with terminal condition 

η1∗ (t∗1 ) η2∗ (t∗1 )



=



ηˆ1 ηˆ2



.

(9.42)

(9.43)

It is important to observe that the system (9.42) can be written as η˙ ∗ (t) = −AT η ∗ (t), where A=



0 1 0 0



(9.44)

is the matrix defining the state equation. The system defined by (9.44) is called the adjoint system, or the co-state system. We shall return to this point later. Since the optimal control is determined by the sign of the linear function η2∗ (t) = ηˆ1 (t∗1 − t) + ηˆ2 , we know that u∗ (t) = 1 when η2∗ (t) > 0 and u∗ (t) = −1 when η2∗ (t) < 0. This type of control is known as a bang-bang control, meaning that the optimal control takes on the extreme values of the constraint set Ω = [−1, 1]. Thus, we need only consider controls that take the value of +1 or −1 and this allows us to “synthesize” the optimal controller by the following method. Rather than fixing the initial condition and integrating for T ward in time, we fix the terminal condition at x0 = 0 0 and integrate the equation backwards in time to see what possible

328

Chapter 9. Optimal Control Problems

states can be reached by controls of this form. Thus, consider the problem x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t) with initial values x1 (0) = 0,

x2 (0) = 0

and integrate backward in time −∞ < t < 0. When u(t) = −1 the solutions are given by −t2 < 0, 2 x2 (t) = −t > 0. x1 (t) =

Observe that x2 (t) = +

p −2x1 (t)

so that the backward trajectory lies on the parabola √  Γ− = [x y]T : y = + −2x, −∞ < x ≤ 0 . Likewise, if u(t) = +1, then

t2 > 0, 2 x2 (t) = t < 0.

x1 (t) =

In this case we have x2 (t) = −

p 2x1 (t)

and the backward trajectory lies on the parabola n o √ + T Γ = [x y] : y = − 2x, 0 ≤ x < +∞ .

Let Γ = Γ− ∪ Γ+ denote the union of the curves Γ− and Γ+ denote the switching curve as shown in Figure 9.2.

329

Chapter 9. Optimal Control Problems

4 -

3 2 1 0 -1 +

-2 -3 -4 -5

0

5

Figure 9.2: The Switching Curves When time is positive and if u(t) = −1, then any initial state on the curve Γ− will follow the parabola until it reaches x1 = 0 at a time t∗1 . If the control is then turned off (i.e. u(t) = 0 for t ≥ t∗1 ), then  −1, 0 ≤ t ≤ t∗1 , ∗ u (t) = 0, t∗1 < t, is optimal. See Figure 9.3. Likewise, if the initial state is on the curve Γ+ , then  +1, 0 ≤ t ≤ t∗1 , ∗ u (t) = 0, t∗1 < t

will be optimal. Here, t∗1 is the time it takes for the trajectory to reach x1 = 0. See Figure 9.4. When the initial data lies above Γ one applies a control of u∗ (t) = −1 until the trajectory intersects the switching curve on Γ+ at some time t∗s . At this time the optimal control is switched to u∗ (t) = +1 until the (optimal) time t∗1 when the trajectory reaches x1 = 0. At t = t∗1 the control is set to u∗ (t) = 0 for all time t > t∗1 .

330

Chapter 9. Optimal Control Problems

4

3

Γ−

−∞ < t ≤ 0 u(t)=−1

2

1

0

−1

−2

u(t)=+1

−3

−4 −5

−4

−3

−2

−1

0

1

2

3

Γ+

4

5

Figure 9.3: Switching Curves for Negative Times Thus, the optimal control is   −1, ∗ +1, u (t) =  0,

0 ≤ t ≤ t∗s , t∗s < t ≤ t∗1 , t∗1 < t.

When the initial data lies below Γ one applies a control of u∗ (t) = +1 until the trajectory intersects the switching curve Γ − at some time t∗s . At this time the optimal control is switched to u∗ (t) = −1 until the (optimal) time t∗1 when the trajectory reaches x1 = 0. At t = t∗1 the control is set to u∗ (t) = 0 for all time t > t∗1 . Thus, the optimal control is  0 ≤ t ≤ t∗s ,  +1, −1, t∗s < t ≤ t∗1 , u∗ (t) =  0, t∗1 < t. Let W : R −→ R be defined by p  − p 2x1, x1 ≥ 0, x2 = W (x1 ) = + −2x1, x1 < 0.

(9.45)

331

Chapter 9. Optimal Control Problems

4

3

Γ−

0 ≤ t < +∞ u(t)=−1

2

1

0

−1

−2

u(t)=+1

−3

−4 −5

−4

−3

−2

−1

0

1

2

3

Γ+

4

5

Figure 9.4: Switching Curves for Positive Times Define the function Ψ(x1 , x2 ) : R2 −→ R by   −1, if x2 > W (x1 ) or (x1 , x2 ) ∈ Γ − , 0, if x1 = x2 = 0, Ψ(x1 , x2 ) =  +1, if x2 < W (x1 ) or (x1 , x2 ) ∈ Γ + .

(9.46)

It follows that the optimal control is given by the feedback control law u∗ (t) = Ψ(x∗1 (t), x∗2 (t)) = Ψ(x∗ (t)), (9.47)  ∗  x1 (t) where x∗ (t) = is the optimal trajectory. x∗2 (t) The previous synthesis can be summarized as follows. The state equation is given by x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0

where the matrices A and B are defined by     0 0 1 , and B = A= 1 0 0

(9.48)

332

Chapter 9. Optimal Control Problems

4

3

Γ−

0 ≤ t < +∞ u(t)=−1

2

u*(t)=−1 1

0

u*(t)=+1 −1

−2

u(t)=+1

−3

−4 −5

−4

−3

−2

−1

0

1

2

3

Γ+

4

5

Figure 9.5: Optimal Trajectories for Initial State above Γ T  T  and x(t) = x1 (t) x2 (t) . respectively. Here, x0 = x1,0 x2,0 The adjoint system (9.42) is defined by η˙ ∗ (t) = −AT η ∗ (t), η ∗ (t∗1 ) = η, ˆ (9.49) T  where η ∗ (t) = η1∗ (t) η2∗ (t) satisfies the terminal boundary   T condition η ∗ (t∗1 ) = ηˆ = ηˆ1 ηˆ2 . The optimal control is given by
u∗ (t) = sgn(η2∗ (t)) = sgn B T η ∗ (t) . (9.50)

If we substitute (9.50) into (9.48), then we obtain the two point boundary value problem defined by the coupled system
x˙ ∗ (t) = Ax∗ (t) + B[sgn B T η ∗ (t) ] (9.51) η˙ ∗ (t) = −AT η ∗ (t)

and boundary conditions x∗ (0) = x0

ˆ η ∗ (t∗1 ) = η,

(9.52)

333

Chapter 9. Optimal Control Problems

4

3

Γ−

0 ≤ t < +∞ u(t)=−1

2

u*(t)=−1

1

0

−1

−2

u*(t)=+1 u(t)=+1

−3

−4 −5

−4

−3

−2

−1

0

1

2

3

Γ+

4

5

Figure 9.6: Optimal Trajectories for Initial State below Γ where x∗ (t) is the optimal trajectory. Consequently, if one can compute the outer normal η, ˆ then the optimal control can be computed by solving the two point boundary value problem (9.51) - (9.52) for x∗ (t) and η ∗
(t) and using (9.50) to give u∗ (t) = sgn(η2∗ (t)) = sgn B T η ∗ (t) .

9.4.2

The Bushaw Problem

Here we consider the so-called Bushaw Problem [52]. The general Bushaw type problem is concerned with second order control systems of the form x¨(t) + 2γ x(t) ˙ + κx(t) = u(t),

(9.53)

where γ ≥ 0 and κ > 0. This model can be thought of as a damped mechanical oscillator. We write this equation as a second

334

Chapter 9. Optimal Control Problems

order system        d x1 (t) 0 x1 (t) 0 1 u(t). + = 1 x2 (t) −κ −2γ dt x2 (t)

(9.54)

In particular, we write the system as x(t) ˙ = Ax(t) + Bu(t)

(9.55)

x(0) = x0 ,

(9.56)

with initial condition where the matrices A and B are defined by     0 0 1 , and B = A= , 1 −κ −2γ  T respectively. Here, x0 = x1,0 x2,0 where x1,0 is the initial displacement and x2,0 is the initial velocity of the oscillator defined by (9.53). Given a control u(t), the solution to (9.55) - (9.56) is  T denoted by x(t; u(·)) = x1 (t; u(·)) x1 (t; u(·)) and defines the state trajectory at time t with the initial state x(0; u(·)) = x0 ∈ R2 . Again, we are interested in time optimal transfer of the initial  T state x0 to a fixed terminal state x1 = 0 0 ∈ R2 . We shall introduce some notation to make the statement of this problem more precise and formal. Recall that a control u(·) is said to steer the state x0 to the state x1 in time t1 if there is a solution to (9.55) - (9.56) and a finite time t1 > 0 such that x(t1 ; u(·)) = x1 ∈ R2 . Let Ω = [−1, 1] ⊂ R1 be the control constraint set so that the condition |u(t)| ≤ 1 can be written as u(t) ∈ Ω = [−1, 1]. Recall that the optimal controller for the rocket sled problem above was “bang-bang” and hence discontinuous. It turns out that many time optimal controllers are bang-bang (see [97] and [100]) and hence to ensure the existence of an optimal control we need to allow for such controls to be “admissible”. Consequently, the set of admissible controls is a subset Θ of all piecewise continuous functions defined on [0, +∞) defined by Θ = Θ(x0 , x1 ) ,{u(·) ∈ P W C[0, +∞) : u(·) steers x0 to x1 , u(t) ∈ Ω e.f.} (9.57)

335

Chapter 9. Optimal Control Problems where

P W C[0, +∞) = {u(·) ∈ P W C[0, T ] : for all finite T > 0} . (9.58) Let f0 : R2 × R1 −→ R1 be defined by f0 (x, u) = f0 (x1 , x2 , u) ≡ 1 and define the cost functional J : Θ −→ R1 by Z t1 f0 (x(s;u(s)), u(s))ds. J(u(·)) ,

(9.59)

0

Note that if u(·) ∈ Θ, then u(·) steers x0 to x1 in time t1 and x(s;u(s)) is the corresponding solution of (9.55) - (9.56). Hence, J(·) is well defined and only depends on the control u(·) ∈ Θ. For the time optimal control problem considered here Z t1 Z t1 J(u(·)) , f0 (x(s;u(s)), u(s))ds = 1ds = t1 (9.60) 0

0

∗

is the time. If u (·) ∈ Θ steers x0 to x1 in time t∗1 , then u∗ (·) is said to be a time optimal control if J(u∗ (·)) ≤ J(u(·)) for any control u(·) ∈ Θ. The Time Optimal Control Problem is the problem of minimizing the functional J(·) on the set of admissible controls Θ. In particular, the goal is to find u∗ (·) ∈ Θ such that Z t∗1 ∗ ∗ t1 = J(u (·)) = f0 (x(s;u∗ (s)), u∗ (s))ds ≤ J(u(·)) 0 Z t1 f0 (x(s;u(s)), u(s))ds = t1 . = 0

for all u(·) ∈ Θ. To obtain the optimal control we proceed as in the rocket sled problem and arrive at the same conclusion that the optimal control u∗ (·) maximizes the inner product hη, ˆ x(t∗1 ;u(·))i

336

Chapter 9. Optimal Control Problems

so that hη, ˆ x(t∗1 ;u(·))i = max hη, ˆ x(t∗1 ;u(·))i = 0. u(·)∈Θ

Applying the Variation of Parameters formula Z t∗1 ∗ ∗ At∗1 x(t1 ;u(·)) = e x0 + eA(t1 −s) Bu(s)ds 0

we find that hη, ˆ

x(t∗1 ;u(·))i



At∗1

Z

t∗1

A(t∗1 −s)



e Bu(s)ds  Z t∗  1 

A(t∗1 −s) At∗1 ˆ e Bu(s)ds = η, ˆ e x0 + η, 0 Z t∗1

 

∗ At∗1 η, ˆ [eA(t1 −s) ]Bu(s) ds. = η, ˆ e x0 + =

η, ˆ e

x0 +

0

0

However, Z t∗1 Z t∗1 

A(t∗ −s) T 

A(t∗1 −s) ˆ Bu(s) ds [e 1 ] η, ]Bu(s) ds = η, ˆ [e 0 0 Z t∗1 D E AT (t∗1 −s) = [e η], ˆ Bu(s) ds 0

and if we define η ∗ (s) by η ∗ (s) = [eA

T (t∗ −s) 1

η], ˆ

then hη, ˆ

x(t∗1 ;u(·))i



= η, ˆ e

= η, ˆ e In order to maximize hη, ˆ

x(t∗1 ;u(·))i



At∗1

At∗1

= η, ˆ e



x0 + 

x0 +

At∗1



x0 +

Z

Z

t∗1 0 t∗1



t∗1



0

Z

hη ∗ (s), Bu(s)i ds

0

 B T η ∗ (s), u(s) ds.  B T η ∗ (s), u(s) ds

337

Chapter 9. Optimal Control Problems

over all u(·) ∈ Θ it is clear that one needs only maximize the integral containing u(·). Hence, the optimal control satisfies Z

t∗1 0



T

∗



∗

B η (s), u (s) ds = max

u(·)∈Θ

Z

t∗1 0



 B T η ∗ (s), u(s) ds

and, like in the rocket sled problem, u∗ (·) must be u∗ (s) = sgn[B T η ∗ (s)].

(9.61)

The only issue is how do we compute B T η ∗ (s)? Differentiating η ∗ (s) = [eA

T (t∗ −s) 1

η] ˆ

it follows that η˙ ∗ (s) = −AT [eA

T (t∗ −s) 1

η] ˆ = −AT η ∗ (s).

(9.62)

Note that η ∗ (s) = [eA

T (t∗ −s) 1

T

η] ˆ = [e−A s ][eA

T t∗ 1

T

η] ˆ = [e−A s ]η, ˜

so that η ∗ (s) is the solution to the adjoint equation (9.62) with initial data T ∗ η ∗ (0) =η˜ = [eA t1 η]. ˆ Therefore, to compute η ∗ (·) from the outward normal ηˆ we set T ∗ η˜ = [eA t1 η] ˆ and solve the initial value problem η˙ ∗ (s) = −AT η ∗ (s),

η ∗ (0) = η. ˜

The optimal control is then given by T

u∗ (s) = sgn[B T η ∗ (s)] = sgn[B T e−A s η] ˜

(9.63)

as indicated by (9.61). We will return to this problem and provide a complete solution for the special case where γ = 0 and κ = 1.

338

9.5

Chapter 9. Optimal Control Problems

Problem Set for Chapter 9

Problem 9.1 Consider the system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), and assume |u(t)| ≤ 1. Compute and sketch the attainable set A(t) for t = 1 and t = 2. Problem 9.2 Consider the system x˙ 1 (t) = u1 (t), x˙ 2 (t) = u2 (t), x˙ 3 (t) = −1. Show that there is a control [u1 (t) u2 (t)]T that steers [0 0 1]T to [0 0 0]T and satisfies [u1 (t)]2 + [u2 (t)]2 ≤ 1. Problem 9.3 Consider the system x˙ 1 (t) = −x2 (t) + u1 (t), x˙ 2 (t) = u2 (t), and assume |u1 (t)| ≤ 1 and |u2 (t)| ≤ 1. Compute and sketch the attainable set A(t) for t = 2. Problem 9.4 Consider the system x(t) ˙ = x(t) + u(t), and assume u(t) ∈ Ω. Compute and sketch the attainable set A(t) for t = 1 when Ω = [−1, 0], Ω = [−1, 1] and Ω = [0, 1]. Problem 9.5 Consider the system x˙ 1 (t) = u1 (t), x˙ 2 (t) = u2 (t), and assume |u1 (t)| ≤ 1 and |u2 (t)| ≤ 1. Show that for t > 0 the attainable set A(t) is a square.

Chapter 9. Optimal Control Problems

339

Problem 9.6 Consider the system x˙ 1 (t) = u1 (t), x˙ 2 (t) = u2 (t), and assume |u1 (t)|2 + |u2 (t)|2 ≤ 1. Compute and sketch the attainable set A(t) for t = 1. Problem 9.7 Consider the system x˙ 1 (t) = x2 (t), x˙ 2 (t) = −x1 (t) + u(t), and assume |u(t)| ≤ 1. Show that the attainable set A(t) has “corners” for 0 < t < π and is a circle of radius 2 for t = π. Problem 9.8 Compute the distances d(A, B) and dH (A, B) between the following sets in R1 : (1) A = {1, 2, 3, 4} and B = {2, 3}; (2) A = {2, 3} and B = {1, 2, 3, 4}; (3) A = [0, 1] and B = [0, 2]; (4) A = [0, 2] and B = [0, 1]. Problem 9.9 Let A be the constant matrix   −1 R , A= 0 −2 where R ≥ 0. Compute the matrix exponential eAt . Problem 9.10 Let A be the constant matrix   0 1 . A= −1 0 Compute the matrix exponential eAt . Problem 9.11 Consider the system x(t) ˙ = Ax(t) + Bu(t),

340

Chapter 9. Optimal Control Problems

where A is a n × n constant matrix and B is a n × m constant matrix. Derive the variation of parameters formula Z t At eA(t−s) Bu(s)ds. x(t) = e x0 + 0

Advanced Problems Problem 9.12 Prove Lemma 9.1. Problem 9.13 Prove Lemma 9.2. Problem 9.14 Prove Lemma 9.3. Problem 9.15 Consider the control system x(t) ˙ = Ax(t) + Bu(t), where A is a n × n constant matrix and B is a n × m constant matrix. Assume Ω ⊆ Rm is a compact and convex set. Show that the attainable set A(t) is convex and bounded. Also, prove that if Ω is a convex polyhedron, then A(t) is closed. (Hint: See [97] and [121]. )

Chapter 10 Simplest Problem in Optimal Control As in the previous chapters, let P W C(a, b; Rm ) denote the space of all Rm -valued piecewise continuous functions defined on [a, b]. Also, P W C(a, +∞; Rm ) denotes the set of all functions u(·) : [a, +∞) → Rm such that u(·) ∈ P W C(a, T ; Rm ) for all a < T < +∞. A function u(·) ∈ P W C(t0 , +∞; Rm ) is called a control or control function. We start with a statement of the Simplest Problem in Optimal Control (SPOC) and state the basic Maximum Principle for this simplest problem.

10.1

SPOC: Problem Formulation

Assume we are given functions of class C 2 f : Rn × Rm → Rn , f0 : Rn × Rm → R, initial set and time X0 ⊆ Rn , t0 ∈ R,

terminal set (but no final time t1 )

X1 ⊆ R n , and control constraint set Ω ⊆ Rm . 341

342

Chapter 10. Simplest Problem in Optimal Control

The state equation (or control system or control process) is defined by the system of differential equations (S)

x(t) ˙ = f (x(t), u(t)),

t0 < t.

(10.1)

The initial conditions are given by x(t0 ) = x0 ∈ X0 .

(10.2)

Definition 10.1 We say that a control u(·) ∈ P W C(t0 , +∞; Rm ) steers the state x0 to the state x1 if there is a solution x(·) = x(·; x0 ; u(·)) to the initial value problem (10.1) - (10.2) satisfying x(t1 ) = x1 for some finite time t0 < t1 . A control u(·) ∈ P W C(t0 , +∞; Rm ) steers the set X0 to the set X1 if for some x0 ∈ X0 , there is a solution x(·) = x(·; x0 ; u(·)) to the initial value problem (10.1) (10.2) satisfying x(t1 ) = x1 ∈ X1 for some finite time t0 < t1 . We make the following standing assumption about the initial value problem. Standing Assumption for Optimal Control: For each x0 ∈ X0 and u(·) ∈ P W C(t0 , t1 ; Rm ), the initial value problem (10.1) - (10.2) has a unique solution defined on [t0 , t1 ]. We denote this solution by x(·; x0 ; u(·)). Definition 10.2 The set of admissible controllers is the subset of P W C(t0 , +∞; Rm ) defined by   u(·) ∈ P W C(t0 , +∞; Rm ) : u(t) ∈ Ω e.f. Θ= and u(·) steers X0 to X1 . = Θ(t0 , X0 , X1 , Ω).

(10.3)

Chapter 10. Simplest Problem in Optimal Control

343

Again, the abbreviation e.f. stands for “except at a finite number of points”. Given a control u(·) ∈ P W C(t0 , +∞; Rm ) and a time t1 > t0 , the cost functional at time t1 > t0 is defined by J(u(·)) =

Zt1

f0 (x(s), u(s))ds =

t0

Zt1

f0 (x(s; u(·)), u(s))ds, (10.4)

t0

where x(·) = x(·; u(·)) is the solution to the initial value problem (10.1) - (10.2). The Simplest Problem in Optimal Control is the problem of minimizing J(·) on the set of all admissible controllers Θ. In particular, an optimal control is a function u∗ (·) ∈ Θ such that u∗ (·) steers X0 to X1 in time t∗1 > t0 and ∗

∗

J(u (·)) =

Zt1

∗

f0 (x(s; u (·)), u(s))ds ≤

t0

= J(u(·))

Zt1

f0 (x(s; u(·)), u(s))ds

t0

(10.5)

for all u(·) ∈ Θ.

10.2

The Fundamental Maximum Principle

In this section we state the Basic Maximum Principle that applies to the SPOC stated above. We will not prove the basic result here. The proof may be found in several standard references. The books The Calculus of Variations and Optimal Control by Leitmann [120] and Foundations of Optimal Control Theory [119] by Lee and Markus both contain a complete proof of the basic result. These references provide the necessary background and alternative proofs of the theorem. In order to state the Maximum Principle we first have to introduce some notation. Given a vector z = [z1 z2 ... zn ]T ∈ Rn and a real number z0 , define an augmented vector zˆ = [z0 z1 z2 ... zn ]T ∈

344

Chapter 10. Simplest Problem in Optimal Control

Rn+1 by T

zˆ = [z0 z1 z2 ... zn ] = Also, given the smooth functions



z0 z



∈ Rn+1 .

f : Rn × Rm → Rn , f0 : Rn × Rm → R, where f : Rn × Rm → Rn is given by    f1 (x1 , x2 , ..., xn , u1 , u2, ..., um ) f1 (x, u)  f2 (x, u)   f2 (x1 , x2 , ..., xn , u1 , u2, ..., um )       f (x, u) =  f3 (x, u)  =  f3 (x1 , x2 , ..., xn , u1 , u2, ..., um )    .. ..    . . fn (x1 , x2 , ..., xn , u1 , u2, ..., um ) fn (x, u) it follows that all the partial derivatives



   ,  

∂fi (x1 , x2 , ..., xn , u1 , u2, ..., um ) ∂xj and

∂f0 (x1 , x2 , ..., xn , u1 , u2, ..., um ) ∂xj exist and are continuous. Define the augmented vector field fˆ : Rn+1 × Rm → Rn+1 by 

   ˆ f (ˆ x, u) =    

   =  

f0 (ˆ x, u) f1 (ˆ x, u) f2 (ˆ x, u) .. . fn (ˆ x, u)





      =    

f0 (x, u) f1 (x, u) f2 (x, u) .. . fn (x, u)

      

f0 (x1 , x2 , ..., xn , u1 , u2, ..., um ) f1 (x1 , x2 , ..., xn , u1 , u2, ..., um ) f2 (x1 , x2 , ..., xn , u1 , u2, ..., um ) .. . fn (x1 , x2 , ..., xn , u1 , u2, ..., um )



   .  

(10.6)

345

Chapter 10. Simplest Problem in Optimal Control The augmented control system is defined by d x ˆ(t) = fˆ(ˆ x(t), u(t)), dt

(10.7)

or equivalently, by the system d x0 (t) = f0 (x(t), u(t)) dt d xi (t) = fi (x(t), u(t)), for i = 1, 2, ..., n. dt

(10.8) (10.9)

Observe that if x0 (t) =

Zt

f0 (x(s; u(·)), u(s))ds,

t0

then x0 (·) satisfies (10.8) with initial data x0 (t0 ) = 0. In particular, x0 (t) represents the cost of transferring the state from an initial x(t0 ) = x0 ∈ X0 to x(t; u(·)) by the control u(·). Assume that (x∗ (·), u∗ (·)) is an optimal pair for the SPOC. In particular, u∗ (·) ∈ Θ steers x∗0 ∈ X0 to x∗1 ∈ X1 in time t∗1 and x∗ (·) = x(·; u∗ (·)) is the corresponding optimal trajectory that satisfies the initial value problem (10.1) - (10.2) with x0 = x∗0 ∈ ˆ by X0 . Define the (n + 1) × (n + 1) matrix A(t) " # ˆi (x0 , x1 , x2 , ..., xn , u1 , u2, ..., um ) ∂ f ˆ = A(t) ∂xj |(ˆ x∗ (t),u∗ (t)) " # ∂ fˆi (ˆ x∗ (t), u∗ (t)) = , (10.10) ∂xj so that  ∂f0 (ˆx∗ (t),u∗ (t))

∂x0 ∂f1 (ˆ x (t),u∗ (t)) ∂x0

∂f0 (ˆ x∗ (t),u∗ (t)) ∂x1 ∂f1 (ˆ x∗ (t),u∗ (t)) ∂x1

∂f0 (ˆ x∗ (t),u∗ (t)) ∂x2 ∂f1 (ˆ x∗ (t),u∗ (t)) ∂x2

∂fn (ˆ x∗ (t),u∗ (t)) ∂x0

∂fn (ˆ x∗ (t),u∗ (t)) ∂x1

∂fn (ˆ x∗ (t),u∗ (t)) ∂x2

 ˆ = A(t) 

∗

.. .

.. .

.. .

··· ··· .. . ···

∂f0 (ˆ x∗ (t),u∗ (t)) ∂xn ∂f1 (ˆ x∗ (t),u∗ (t)) ∂xn

.. .

∂fn (ˆ x∗ (t),u∗ (t)) ∂xn



 . 

346

Chapter 10. Simplest Problem in Optimal Control

Moreover, since fˆi (ˆ x, u) = fˆi (x, u) = fi (x, u) = fi (x1 , x2 , ..., xn , u1 , u2, ..., um ) does not depend on x0 , then ∂fi (ˆ x∗ , u∗ ) =0 ∂x0 for all i = 0, 1, 2, ..., n and it follows that # " ∗ ∗ ˆi (ˆ ∂ f x (t), u (t)) ˆ = A(t) ∂xj  ∗ ∂f0 (ˆ x∗ (t),u∗ (t)) (t),u∗ (t)) 0 ∂f0 (ˆx ∂x 1 2  ∂f1 (ˆx∗ (t),u∗ (t)) ∂f1 (ˆx∗∂x (t),u∗ (t)) 0 ∂x1 ∂x2 = .. ..  .. . . . 0

∂fn (ˆ x∗ (t),u∗ (t)) ∂x1

∂fn (ˆ x∗ (t),u∗ (t)) ∂x2

··· ··· .. . ···



∂f0 (ˆ x∗ (t),u∗ (t)) ∂xn ∂f1 (ˆ x∗ (t),u∗ (t))   ∂xn 

.. .

∂fn (ˆ x∗ (t),u∗ (t)) ∂xn

. 

ˆ It is important to note that the first column of A(t) is all T n zeros. Also, note that for x = [x1 x2 ... xn ] ∈ R , the gradient of f0 (x, u) with respect to x at (x, u) is defined by     ∇x f0 (x, u) ,   

∂f0 (x,u) ∂x1 ∂f0 (x,u) ∂x2

.. .

∂f0 (x,u) ∂xn

    

and the Jacobian matrix of f (x, u) with respect to x = [x1 x2 ... xn ]T ∈ Rn at (x, u) is given by   ∂f1 (x,u) ∂f1 (x,u) ∂f1 (x,u) · · · ∂x2 ∂xn 1  ∂f2∂x ∂f2 (x,u)  ∂f2 (x,u) (x,u)  ∂x  · · · ∂x2 ∂xn 1 . Jx f (x, u) =  . . . ..   .. .. .. .   ∂fn (x,u) ∂fn (x,u) ∂fn (x,u) ··· ∂x1 ∂x2 ∂xn Consequently, #  "  ∗ ∗ ∗ ∗ T ˆi (ˆ ∂ f x (t), u (t)) 0 [∇ f (ˆ x (t), u (t))] x 0 ˆ = = A(t) 0 [Jx f (ˆ x∗ (t), u∗ (t))] ∂xj

347

Chapter 10. Simplest Problem in Optimal Control

and the transpose matrix can be written as   0 0 T ˆ [A(t)] = . (10.11) ∇x f0 (ˆ x∗ (t), u∗ (t)) [Jx f (ˆ x∗ (t), u∗ (t))]T We now define the adjoint equation by d ˆ T η(t) η(t) ˆ = −[A(t)] ˆ dt

(10.12)

where ηˆ(t) given by T

η(t) ˆ = [η0 (t) η1 (t) η2 (t) ... ηn (t)] =



η0 (t) η(t)



is called the adjoint state (or co-state) variable. We need to define two more functions. The augmented ˆ : Rn+1 × Rn+1 × Rm → R1 defined Hamiltonian is the function H by ˆ η, ˆ 0 , η1 , η2 , ..., ηn , x0 , x1 , x2 , ..., xn , u1 , u2, ..., um ) H( ˆ x ˆ, u) = H(η (10.13) = η0 f0 (x, u) + hη, f (x, u)i n X = η0 f0 (x, u) + ηi fi (x, u). i=1

Hence, ˆ η, H( ˆ x ˆ, u) = η0 f0 (x0 , x1 , x2 , ..., xn , u1 , u2, ..., um ) n X + ηi fi (x0 , x1 , x2 , ..., xn , u1 , u2, ..., um ). i=1

ˆ : Rn+1 × is a function of 2(n + 1) + m variables. Finally, let M n+1 R → R be defined by ˆ (η, ˆ η, M ˆ x ˆ) = max H( ˆ x ˆ, u) u∈Ω

(10.14)

ˆ (η, when M ˆ x ˆ) exists. We can now state the (Pontryagin) Maximum Principle as the following theorem.

348

Chapter 10. Simplest Problem in Optimal Control

Theorem 10.1 (Maximum Principle) Assume that f : Rn × Rm → Rn , f0 : Rn × Rm → R1 , X0 ⊆ Rn , t0 ∈ R, X1 ⊆ Rn and Ω ⊆ Rm are given as above and consider the control system (S)

x(t) ˙ = f (x(t), u(t)),

t0 < t,

(10.15)

with piecewise continuous controllers u(·) ∈ P W C(t0 , +∞; Rm ) satisfying u(·) ∈ Ω ⊆ Rm e.f. If   u(·) ∈ P W C(t0 , +∞; Rm ) : u(t) ∈ Ω e.f. ∗ u (·) ∈ Θ = and u(·) steers X0 to X1 . (10.16) minimizes Zt1 (10.17) J(u(·)) = f0 (x(s), u(s))ds t0

on the set of admissible controls Θ with optimal response x∗ (·) satisfying x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t∗1 ) = x∗1 ∈ X1 at time t∗1 > t0 , then there exists a non-trivial solution   ∗ η0 (t) T ∗ ∗ ∗ ∗ ∗ (10.18) ηˆ (t) = [η0 (t) η1 (t) η2 (t) ... ηn (t)] = η ∗ (t) to the augmented adjoint equation d ˆ T η(t), η(t) ˆ = −[A(t)] ˆ dt

(10.19)

such that ˆ ηˆ∗ (t), x ˆ (ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = M ˆ∗ (t)) ˆ ηˆ∗ (t), x = maxH( ˆ∗ (t), u). (10.20) u∈Ω

Moreover, there is a constant η0∗ ≤ 0 such that η0∗ (t) ≡ η0∗ ≤ 0

(10.21)

and for all t ∈ [t0 , t∗1 ] ˆ (ηˆ∗ (t), x ˆ ηˆ∗ (t), x M ˆ∗ (t)) = max H( ˆ∗ (t), u) ≡ 0. u∈Ω

(10.22)

349

Chapter 10. Simplest Problem in Optimal Control

Also, if X0 ⊆ Rn and X1 ⊆ Rn are manifolds with tangent spaces T0 and T1 at x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t∗1 ) = x∗1 ∈ X1 , respectively, then  T ηˆ∗ (t) = [η0∗ (t) η1∗ (t) η2∗ (t) ... ηn∗ (t)]T = η0∗ (t) η ∗ (t)

can be selected to satisfy the transversality conditions η ∗ (t0 ) ⊥ T0

(10.23)

η ∗ (t∗1 ) ⊥ T1 .

(10.24)

and

Remark 10.1 We have not discussed the precise definitions of “manifolds” and “tangent spaces”. A rigorous presentation of the definitions of manifolds and tangent spaces would require a background in differential geometry and topology which is outside the

 * (t1* )

 * (t0 )

x (t )  x (t ; u ()) *

T0

x0*

X0

*

T1 x1*

x (t ; u2 ())

x (t ; u1 ())

Figure 10.1: The Maximum Principle

X1

350

Chapter 10. Simplest Problem in Optimal Control

scope of this book. However, roughly speaking a manifold of dimension k is a topological space that near each point “resembles” k-dimensional Euclidean space. Two dimensional manifolds are called surfaces. Examples of manifolds are the entire space Rn , kdimensional subspaces of Rn , hyperplanes and 0-dimensional manifolds which are points and may be viewed as a translation of the zero subspace. Also, given p ≤ n and p smooth functions g1 (·), g2 (·), ... ,gp (·) satisfying gi : Rn −→ R, define the set Xg = {x ∈ Rn : gi (x) = 0, for i = 1, 2, ..., p} .

(10.25)

If the vectors ∇gi (x), i = 1, 2, ..., p

are linearly independent, then Xg is a (smooth) manifold of dimension k = n − p. Note that this condition is equivalent to the statement that the matrix  ∂g1 (x) ∂g1 (x)  · · · · · · ∂g∂x1 (x) ∂x1 ∂x2 n  ∂g2 (x) ∂g2 (x) · · · · · · ∂g2 (x)   ∂x1  ∂x2 ∂xn  . .. ..  ..  .. . . .     .. ..  .. . .  . . . .  ∂gp (x) ∂gp (x) ∂gp (x) ··· ··· ∂x1 ∂x2 ∂xn

has maximal rank p. In this case the tangent plane Tg (x) at x ∈ Xg is given by Tg (x) = Tg1 (x) ∩ Tg2 (x) ∩ . . . ∩ Tgp (x)

where Tgi (x) is the tangent plane of the smooth surface Xgi = {x ∈ Rn : gi (x) = 0} with normal vector given by ∇gi (x). Therefore, a vector ηˆ ∈ Rn is orthogonal to Tg (x) at x ∈ Xg if and only if n X T ηˆ z = ηˆj zz = 0 j=1

for z ∈ Rn satisfying

n X ∂gi (x) i=1

∂xj

zj = 0, for all i = 1, 2, ..., p.

Chapter 10. Simplest Problem in Optimal Control

351

We will use this result when the initial and target sets are defined by (10.25). The special cases where X = Rn and X = x ˆ are simple since the tangent space to any subspace is that space and the tangent space to a single point is the zero subspace. Thus, every vector ηˆ is transversal to a single point and only the zero vector is transversal to the whole space Rn . The references [107], [116], [147] and [184] provide introductions to differential geometry with rigorous definitions.

10.3

Application of the Maximum Principle to Some Simple Problems

Here we apply the Maximum Principle to some of the problems described above. We focus on the Simplest Problem (i.e. t1 not fixed) and then move to other variations of this problem in later sections.

10.3.1

The Bushaw Problem

The Bushaw problem is governed by the control system        d x1 (t) 0 x1 (t) 0 1 u(t), + = 1 x2 (t) −κ −2γ dt x2 (t)

(10.26)

which has the form x(t) ˙ = Ax(t) + Bu(t)

(10.27)

x(0) = x0 ∈ R2 ,

(10.28)

with initial condition

where the matrices A and B are defined by     0 0 1 , and B = A= 1 −κ −2γ

352

Chapter 10. Simplest Problem in Optimal Control

 T is the initial condition. respectively. Here, x0 = x1,0 x2,0 Given a control u(·), the solution to (10.27) - (10.28) is denoted by x(·; u(·)). We are interested in time optimal transfer of the  T initial state x0 to a fixed terminal state x1 = 0 0 ∈ R2 by a control satisfying the condition |u(t)| ≤ 1. We formulate this time optimal control problem as a SPOC described above. Formulation as a Simplest Problem in Optimal Control: In order to set up the optimal control problem as a SPOC and apply the Maximum Principle, we need to: 1. Identify the initial time t0 ; 2. Identify the initial set X0 ⊆ R2 , the terminal set X1 ⊆ R2 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R2 × R1 −→ R1 , f0 : R2 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). Remark 10.2 We will work through these steps for every example we discuss and each time the above list will be repeated. Although this may seem like overkill, it helps the reader focus on the essential steps required to formulate and apply the Maximum Principle and its extensions. Moreover, many problems require additional steps before one can formulate the problem as an optimal control problem and explicitly listing these six steps can be helpful in determining the proper formulation (see the point-to-curve problem in Section 11.2.3). Recall that the time optimal cost functional can be written as Z t1 J(u(·)) = 1ds = t1 0

and the initial time is t0 = 0. Since there are two state equations and one control, the state space is R2 and the control space is R1

Chapter 10. Simplest Problem in Optimal Control

353

so that n = 2 and m = 1. Let Ω = [−1, 1] ⊂ R1 be the control constraint set; so that the condition |u(t)| ≤ 1 can be written as u(t) ∈ Ω = [−1, 1]. The initial and terminal sets are single points so that X0 = {x0 } ⊆ R2 and X1 = {0} ⊆ R2 define these sets. The functions f : R2 × R1 −→ R2 and f0 : R2 × R1 −→ R1 are defined by   x2 f (x, u) = f (x1 , x2 , u) = Ax + Bu = −κx1 − 2γx2 + u

and f0 (x, u) = f0 (x1 , x2 , u) ≡ 1,

respectively. Hence,

f (x, u) = f (x1 , x2 , u) =



f1 (x1 , x2 , u) f2 (x1 , x2 , u)



=



x2 −κx1 − 2γx2 + u



defines the functions f1 (x1 , x2 , u) and f2 (x1 , x2 , u). Now to set up the Maximum Principle we start by letting T  denote the augmented state so that the augx ˆ = x0 x1 x2 mented function fˆ : R3 × R1 −→ R3 is given by   f0 (x1 , x2 , u) fˆ(ˆ x, u) = fˆ(x0 , x1 , x2 , u) = fˆ(x1 , x2 , u) =  f1 (x1 , x2 , u)  f2 (x1 , x2 , u)   1 . x2 = −κx1 − 2γx2 + u

ˆ : R3 × We also need to compute the augmented Hamiltonian H R3 × R1 −→ R1 defined by ˆ η, H( ˆ x ˆ, u) = η0 [f0 (x1 , x2 , u)] + η1 [f1 (x1 , x2 , u)] + η2 [f2 (x1 , x2 , u)], T  T  where ηˆ = η0 η = η0 η1 η2 . By direct substitution we have that ˆ η, H( ˆ x ˆ, u) = η0 [1] + η1 [x2 ] + η2 [−κx1 − 2γx2 + u] = η0 + η1 x2 + η2 [−κx1 − 2γx2 ] + η2 u.

354

Chapter 10. Simplest Problem in Optimal Control

To set up the adjoint equation we need to compute the matrix  ∂f (x ,x ,u) ∂f (x ,x ,u) ∂f (x ,x ,u)  0

 Jxˆ (ˆ x, u) = Jxˆ (x, u) = 

1

2

∂x0 ∂f1 (x1 ,x2 ,u) ∂x0 ∂f2 (x1 ,x2 ,u) ∂x0

0

1

2

∂x1 ∂f1 (x1 ,x2 ,u) ∂x1 ∂f2 (x1 ,x2 ,u) ∂x1

0

1

2

∂x2 ∂f1 (x1 ,x2 ,u) ∂x2 ∂f2 (x1 ,x2 ,u) ∂x2

which is easily seen to be 

 ,

   0 0 0 0 0   1 . = Jxˆ (ˆ x, u) = 0 0 0 A 0 −κ −2γ

Now assume that (x∗ (·), u∗ (·)) is an optimal pair so that the maˆ can be computed by trix A(t)   0 0 0 ˆ = Jxˆ (ˆ 1  A(t) x, u)|(x∗ (t),u∗ ()) = Jxˆ (x∗ (t), u∗ (t)) =  0 0 0 −κ −2γ   0 0 . = 0 A Observe that in this special case     0 0 0 0 0 ˆ = Aˆ =  0 0 1 = A(t) 0 A 0 −κ −2γ

is a constant matrix and does not depend on (x∗ (·), u∗ (·)). The augmented adjoint equation d ˆ T η(t) η(t) ˆ = −[A(t)] ˆ dt

(10.29)

has the form         η0 (t) 0 0 0 η0 (t) 0 0 0 η0 (t) d  η1 (t) = − 0 0 −κ  η1 (t) = 0 0 κ  η1 (t) . dt η2 (t) 0 −1 2γ η2 (t) 0 1 −2γ η2 (t)

Observe that this system is given by     0 η0 (t) d   κη2 (t) η1 (t)  =  dt −η1 (t) + 2γη2 (t) η2 (t)

355

Chapter 10. Simplest Problem in Optimal Control and the first equation implies d η0 (t) = 0 dt

which means that all solutions to the augmented adjoint equations have η0 (t) ≡ η0 .

Also for this special case, the non-augmented adjoint state   η1 (t) η(t) = η2 (t) satisfies the system d η(t) = −AT η(t) dt

(10.30)

where the matrix A is the state matrix in (10.27). Application of The Maximum Principle: We are now ready to apply the Maximum Principle. Assume (x∗ (·), u∗ (·)) is an optimal pair. Then there exists a non-trivial solution   ∗   ∗ η0 (t) η0 (t) =  η1∗ (t)  ηˆ∗ (t) = η ∗ (t) η2∗ (t) to the augmented adjoint equation (10.29) such that η0∗ (t) ≡ η0∗ ≤ 0 and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max H( ˆ∗ (t), u) ≡ 0. u∈Ω

Since ˆ ηˆ∗ (t), x H( ˆ∗ (t), u) = η0∗ + η1∗ (t)x∗2 (t) + η2∗ (t)[−κx∗1 (t) − 2γx∗2 (t)] + η2∗ (t)u, ˆ ηˆ∗ (t), x it follows that in order to maximize H( ˆ∗ (t), u) on [−1, 1] one needs only to maximize the last term in this expression. In particular, the optimal control must maximize the expression η2∗ (t)u

356

Chapter 10. Simplest Problem in Optimal Control

for all t where −1 ≤ u ≤ +1. Clearly, this occurs when u∗ (t) = sgn[η2∗ (t)] and the problem now reduces to the case of computing η2∗ (t). Let’s see what the transversality conditions tell us. Since X0 = {x0 } ⊆ R2 and X1 = {0} ⊆ R2 and the tangent plane to a point is the “zero plane”, i.e. T0 = {0} ⊆ R2 and T1 = {0} ⊆ R2 , the conditions η ∗ (0) ⊥ T0 and η ∗ (t∗1 ) ⊥ T1 are satisfied by any vectors η ∗ (0) and η ∗ (t∗1 ). Thus, the transversality conditions give us no additional information. However, we do know that x∗ (0) = x0 and x∗ (t∗1 ) = 0 and

d ∗ η (t) = −AT η ∗ (t) dt

(10.31)

so that η ∗ (t) = e−A

T (t−t∗ ) 1

T

η ∗ (t∗1 ) = e−A t eA

where η˜∗ , eA Hence, η2∗ (t) and

=



0 1





T t∗ 1

T t∗ 1

T

η ∗ (t∗1 ) = e−A t η˜∗

η ∗ (t∗1 ).

η1∗ (t) η2∗ (t)



T

= B T e−A t η˜∗ T

u∗ (t) = sgn[η2∗ (t)] = sgn[B T e−A t η˜∗ ] T ∗

where η˜∗ , eA t1 η ∗ (t∗1 ) is not known since (at this point) neither t∗1 nor η ∗ (t∗1 ) are known. Observe that the Maximum Principle says that η ∗ (t∗1 ) can be selected so that η ∗ (t∗1 ) ⊥ T1 . However, any vector satisfies this condition. We might try the simple case where η ∗ (t∗1 ) = 0 and see what happens. In this case, η ∗ (t) satisfies the linear differential equation dtd η ∗ (t) = −AT η ∗ (t) with zero value at η ∗ (t∗1 ). Therefore, η ∗ (t) must be identically zero for all t i.e. η ∗ (t) ≡ 0. However,

Chapter 10. Simplest Problem in Optimal Control

357

this would imply that u∗ (t) = sgn(η2∗ (t)) ≡ 0 and the augmented Hamiltonian has the form ˆ ηˆ∗ (t), x H( ˆ∗ (t), u) = η0∗ + η1∗ (t)x∗2 (t) + η2∗ (t)[−κx∗1 (t) − 2γx∗2 (t)] + η2∗ (t)u = η0∗ + 0x∗2 (t) + 0[−κx∗1 (t) − 2γx∗2 (t)] + 0u ≡ η0∗ . On the other hand, the Maximum Principle yields ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x η0∗ = H( ˆ∗ (t), u∗ (t)) = max H( ˆ∗ (t), u) ≡ 0 u∈Ω

and the constant η0∗ would be zero also. Thus, ηˆ∗ (t) ≡ 0 and this contradicts the statement (in the Maximum Principle) that ηˆ∗ (t) is a non-trivial solution to the augmented adjoint equation. ConseT ∗ quently, we know that η ∗ (t∗1 ) 6= 0, and hence η˜∗ , eA t1 η ∗ (t∗1 ) 6= 0. Therefore, we know that if u∗ (t) is a time optimal controller, then the following conditions hold. There is a non-trivial solution to the adjoint equation d ∗ η (t) = −AT η ∗ (t) dt with initial data η ∗ (t∗1 ) 6= 0 and the optimal control has the form T

u∗ (t) = sgn(B T e−A t η˜∗ ). The optimal trajectories x∗ (t) satisfy d ∗ T x (t) = Ax∗ (t) + Bsgn(B T e−A t η˜∗ ), dt along with the boundary conditions x∗ (0) = x0 , x∗ (t∗1 ) = 0.

358

Chapter 10. Simplest Problem in Optimal Control

Moreover, ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x 0 = max H( ˆ∗ (t), u) ≡ H( ˆ∗ (t), u∗ (t)) = =

u∈Ω η0∗ + η0∗ +

η1∗ (t)x∗2 (t) + η2∗ (t)[−κx∗1 (t) − 2γx∗2 (t)] + η2∗ (t)(sgn[η2∗ (t)]) η1∗ (t)x∗2 (t) + η2∗ (t)[−κx∗1 (t) − 2γx∗2 (t)] + |η2∗ (t)|

holds for all 0 ≤ t ≤ t∗1 . In addition, since η0∗ ≤ 0, it follows that η1∗ (t)[x∗2 (t)] + η2∗ (t)[−κx∗1 (t) − 2γx∗2 (t)] + |η2∗ (t)| ≥ 0 and this is all we can say at this point about this problem.

10.3.2

The Bushaw Problem: Special Case γ = 0 and κ = 1

Here we synthesize the optimal controller for a special case of Bushaw’s problem. Since   0 1 A= −1 0 it follows that T

−A = and hence e

−AT s

=e

As

Therefore T −AT s

B e



=

0 1 −1 0







= A,

cos(s) sin(s) − sin(s) cos(s) 





.

cos(s) sin(s) η˜ = 0 1 − sin(s) cos(s)     η˜1 = − sin(s) cos(s) η˜2



η˜

= −˜ η1 sin(s) + η˜2 cos(s).

Using the standard identity −˜ η1 sin(s) + η˜2 cos(s) =

p ˜ [˜ η1 ]2 + [˜ η2 ]2 sin(s + φ)

Chapter 10. Simplest Problem in Optimal Control where

 

359

 p 2 2 η1 ] + [˜ η2 ] if η˜1 ≤ 0, arcsin η˜2 / [˜  p  φ˜ =  π − arcsin η˜2 / [˜ η1 ]2 + [˜ η2 ]2 if η˜1 > 0, 

it follows from (9.63) that   u∗ (s) = sgn B T η ∗ (s) = sgn [−˜ η1 sin(s) + η˜2 cos(s)] i hp ˜ . [˜ η1 ]2 + [˜ η2 ]2 sin(s + φ) = sgn

p p ˜ and η1 ]2 + [˜ η2 ]2 > 0, we have that [˜ η1 ]2 + [˜ η2 ]2 sin(s+ φ) Since [˜ ˜ have the same sign. Hence the optimal control is given sin(s + φ) by hp i h i ˜ = sgn sin(t + φ) ˜ . u∗ (t) = sgn [˜ η1 ]2 + [˜ η2 ]2 sin(t + φ) (10.32) The expression (10.32) provides considerable information about the optimal control. In particular: • The optimal control u∗ (·) is bang-bang just like the rocket sled problem. • Switches (if they occur) occur exactly π seconds apart. To complete the synthesis we solve the equations for controls u(·) that are bang-bang with switching time intervals of length π. Thus, we focus on the cases where u(t) = ±1. If u(t) = +1, then the state equation has the form x˙ 1 (t) = x2 (t) x˙ 2 (t) = −x1 (t) + 1. Observe that

so that

x2 dx1 = dx2 −x1 + 1 (−x1 + 1)dx1 = x2 dx2

360

Chapter 10. Simplest Problem in Optimal Control

and hence integration yields Z Z c + (−x1 + 1)dx1 = x2 dx2 . Thus, c+

1 −1 [−x1 + 1]2 = [x2 ]2 2 2

which implies that 2c = [−x1 + 1]2 + [x2 ]2 > 0, or equivalently, [−x1 + 1]2 + [x2 ]2 = α2 > 0. Therefore, the trajectories are circles centered at [1 0]T with radius α. Likewise, if u(t) = −1, then [−x1 − 1]2 + [x2 ]2 = α2 > 0 and trajectories are circles centered at [−1 0]T with radius α. Since an optimal control is bang-bang, optimal trajectories consist of portions of circles centered about the two points [1 0]T and [−1 0]T with trajectories moving clockwise. If time is reversed the backward trajectories move counter clockwise. Moreover, since the switching times are multiples of π only semi-circles can be used to construct optimal trajectories. First assume that 0 < φ˜ ≤ π, set u(t) = 1 and integrate backwards from [0 0]T . The solution is given by x1 (t) = 1 − cos(t) > 0 x2 (t) = sin(t) < 0 ˜ = for −π ≤ t ≤ 0. To be an optimal control, u(t) = sgn[sin(t + φ)] s ˜ 1 must switch from +1 to −1 at t1 = −φ. Thus, u(t) = +1 until ts1 = −φ˜ and then one starts applying u(t) = −1 with initial data ˜ = 1 − cos(−φ) ˜ >0 x1 (−φ) ˜ = sin(−φ) ˜ < 0. x2 (−φ)

361

Chapter 10. Simplest Problem in Optimal Control

4

−∞ < t ≤ 0 3 2

u(t)=−1 Γ−

1 0 −1

Γ+ u(t)=+1

−2 −3 −4 −6

−4

−2

0

2

4

6

Figure 10.2: Switching Curves for Negative Time However, now u(t) = −1 and the trajectory moves counter clockwise along the circle centered about [−1 0]T with radius q ˜ + 1]2 + [x2 (−φ)] ˜ 2 α1 = [x1 (−φ)

until ts2 = −(φ˜ + π) when the control takes the value u(t) = 1 again. The trajectory will now move counter clockwise along the circle centered about [1 0]T with radius q α2 = [x1 (−(φ˜ + π)) − 1]2 + [x2 (−φ˜ + π)]2 .

Continuing this procedure and reversing time to positive values we arrive at the switching curves as shown in the Figures 10.2 and 10.3 below. Let W = Γ− ∪ Γ+ ∪ H − ∪ H + where H − is the curve defined by the positive semicircles to the left of Γ− and H + is the curve defined by the negative semicircles to the right of Γ+ . The curve

362

Chapter 10. Simplest Problem in Optimal Control

4

0 ≤ t < +∞ 3 2

u*(t)=−1 Γ−

1 0 −1

Γ+ u*(t)=+1

−2 −3 −4 −6

−4

−2

0

2

4

6

Figure 10.3: Switching Curves for Positive Time W is the switching curve for this problem. Therefore, the optimal control has the feedback law given by u∗ (t) = Ψ(x∗1 (t), x∗2 (t)) = Ψ(x∗ (t)),

(10.33)

where

 −1, if (x1 , x2 ) is above W,     or (x1 , x2 ) ∈ H − or (x1 , x2 ) ∈ Γ − ,  0, if x1 = x2 = 0, Ψ(x1 , x2 ) =   +1, if (x1 , x2 ) is below W,    or (x1 , x2 ) ∈ H + or (x1 , x2 ) ∈ Γ + . (10.34)

10.3.3

A Simple Scalar Optimal Control Problem

Consider the linear control system x(t) ˙ = −x(t) + 1 − u(t)

(10.35)

Chapter 10. Simplest Problem in Optimal Control

363

with initial data and target

x(0) = 1 ∈ R1

(10.36)

x(t1 ) = 0 ∈ R1 .

(10.37)

Here t0 = 0, x0 = 1 ∈ R1 and x1 = 0 ∈ R1 are given and the controls belong to the space P W C(0, +∞; R1 ). The quadratic cost functional is defined by J(x(·), u(·)) =

Zt1 0

1 1 + [u(s)]2 ds, 2

(10.38)

where x(·) is the solution to the system (10.35) - (10.37). The optimal control problem is to find u∗ (·) ∈ P W C(0, +∞; R1 ) so that ∗

∗

∗

J(x (·), u (·)) =

Zt1 0

1 1 + [u∗ (s)]2 ds ≤ 2

= J(x(·), u(·))

Zt1 0

1 1 + [u(s)]2 ds 2

for all u(·) ∈ P W C(0, +∞; R1 ) that steer x0 = 1 to x1 = 0, where x∗ (s) , x∗ (s; u∗ (·)) is the optimal trajectory. Observe that this is a Simplest Problem in Optimal Control since the final time t1 is not fixed. Formulation as a Simplest Problem in Optimal Control: In order to set up the optimal control problem as a SPOC and apply the Maximum Principle, we need to: 1. Identify the initial time t0 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u);

364

Chapter 10. Simplest Problem in Optimal Control

ˆ 6. Form the augmented adjoint system matrix A(t). It is clear that X0 = {1} and X1 = {0}. Also, since there is no bound on the control, it follows that Ω = R1 . The functions f : R1 × R1 −→ R1 and f0 : R1 × R1 −→ R1 are given by f (x, u) = −x + 1 − u and

1 f0 (x, u) = {1 + [u]2 }, 2 respectively. The augmented Hamiltonian is thus defined by 1 ˆ η, H( ˆ x ˆ, u) = η0 f0 (x, u) + ηf (x, u) = η0 {1 + [u]2 } + η(−x + 1 − u). 2 Moreover, since the augmented right-hand side is given by   1   f0 (x, u) {1 + [u]2 } 2 ˆ = f (ˆ x, u) = , f (x, u) −x + 1 − u it follows that ˆ = A(t) =

"



∂f0 (x,u) ∂x0 ∂f (x,u) ∂x0

0 0 0 −1



∂f0 (x,u) ∂x ∂f (x,u) ∂x

#

|(x,u)=(x∗ (t),u∗ (t))

= |(x,u)=(x∗ (t),u∗ (t))



0 0 0 −1



.

The augmented adjoint equation is defined by      d d η0 (t) η (t) 0 0 0 T ˆ , ˆ = −[A(t)] η(t) ˆ = = η(t) η(t) 0 1 dt η(t) dt (10.39) so that η˙ 0 (t) = 0 and η(t) ˙ = η(t). Thus, as stated in the Maximum Principle, η0 (t) ≡ η0 ≤ 0 is a non-positive constant and hence η(t) = cet

Chapter 10. Simplest Problem in Optimal Control

365

for some constant c. If (x∗ (·), u∗ (·)) is an optimal then it  pair, ∗ η0 (t) to the follows that there is a non-trival solution ηˆ∗ (t) = η ∗ (t) augmented adjoint equation (10.39) such that u = u∗ (t) maximizes the augmented Hamiltonian 1 ˆ ηˆ∗ (t), x H( ˆ∗ (t), u) = η0∗ {1 + [u]2 } + η ∗ (t)(−x∗ (t) + 1 − u). 2 Since Ω = R1 it follows that ∂ ˆ ∗ ∂ ˆ ∗ H(ηˆ (t), x ˆ∗ (t), u)|u=u∗ (t) = H(ηˆ (t), x ˆ∗ (t), u∗ (t)) = 0. ∂u ∂u However, a direct calculation yields ∂ ˆ ∗ H(ˆ η (t), xˆ∗ (t), u) = η0∗ u − η ∗ (t) ∂u so that 0 = [η0∗ u − η ∗ (t)]|u=u∗ (t) = [η0∗ u∗ (t) − η ∗ (t)].

(10.40)

Note that η0∗ can not be zero since if η0∗ = 0, then (10.40) would imply that 0 = [η0∗ u∗ (t) − η ∗ (t)] = η ∗ (t) which contradicts the statement that ηˆ∗ (t) is non-trival. Hence we can solve (10.40) for the optimal control  ∗  η (t) η ∗ (t) ∗ . u (t) = ∗ = − η0 −η0∗ If we define

 η ∗ (t) , λ (t) , −η0∗ 

∗

then

λ˙ ∗ (t) = λ∗ (t)

and λ∗ (t) = Aet , for some constant A. Thus, u∗ (t) = −λ∗ (t) = −Aet ,

366

Chapter 10. Simplest Problem in Optimal Control

and since t1 is not given it follows that ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x max H( ˆ∗ (t), u) = H( ˆ∗ (t), u∗ (t)) ≡ 0. u∈R1

At t = 0 we have 1 ˆ ηˆ∗ (o), x H( ˆ∗ (0), u∗ (0)) = η0∗ {1 + [u∗ (0)]2 } 2 + η ∗ (0)(−x∗ (0) + 1 − u∗ (0)) = 0.

Consequently,  ∗  1 η (0) ∗ 2 − {1 + [u (0)] } + (−x∗ (0) + 1 − u∗ (0)) = 0, ∗ 2 −η0 and 1 − {1 + [u∗ (0)]2 } + λ∗ (0)(−x∗ (0) + 1 − u∗ (0)) = 0 2 so that 1 − {1 + [−A]2 } + A(−1 + 1 + A) = 0. 2 This equation becomes 1 1 1 − {1 + A2 } + A2 = − + A2 = 0 2 2 2 or A2 = 1.

There are two possible roots A = 1 and A = −1. In any case, the optimal state is given by the Variation of Parameters Formula Z t ∗ −t e−(t−s) [1 − u∗ (s)]ds x (t) = e + Z0 t e−(t−s) [1 + Aes ]ds = e−t + Z t Z0 t −(t−s) −t e−(t−s) [Aes ]ds e [1]ds + =e + 0 Z0 t   s=t e−t+2s ds +A = e−t + e−(t−s) |s=0 



0

A  −t+2s  s=t =e + 1−e + |s=0 e 2   A t  e − e−t . = e−t + 1 − e−t + 2 −t

−t

Chapter 10. Simplest Problem in Optimal Control

367

Therefore,

1 1 x∗ (t) = 1 + Aet − Ae−t 2 2 and using the end condition x∗ (t1 ) = 0

it follows that 1 1 1 = −A[ et1 − e−t1 ] = −A sinh(t1 ). 2 2 Thus, at t1 = t∗1

1 , −A where from above A = ±1. Clearly, A = −1 and the optimal final time is t∗1 = sinh−1 (1). sinh(t∗1 ) =

Completing the problem, we have that the optimal control is given by u∗ (t) = −Aet = et , 0 ≤ t ≤ t∗1 = sinh−1 (1) and the optimal trajectory is 1 1 x∗ (t) = 1 − et + e−t . 2 2

10.4

Problem Set for Chapter 10

Problem 10.1 Consider the control system x(t) ˙ = u(t), with initial condition x(0) = 1, and terminal condition x(t1 ) = 0.

368

Chapter 10. Simplest Problem in Optimal Control

Let the control constraint set be Ω = [−1, 0]. Investigate the optimal control problem for the cost functional J(u(·)) = t1 =

Zt1

1ds.

0

Problem 10.2 Consider the control system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), with initial condition x1 (0) = 1, x2 (0) = 1 and terminal condition x1 (t1 ) = 0, x2 (t1 ) = 0. Let the control constraint set be Ω = [−1, +1]. Investigate the optimal control problem for the cost functional J(u(·)) = t1 =

Zt1

1ds.

0

Problem 10.3 Consider the control system x˙ 1 (t) = −x1 (t) + 10x2 (t) + u(t), x˙ 2 (t) = −x2 (t) + u(t), with initial condition x1 (0) = 1, x2 (0) = 0, and terminal condition x1 (t1 ) = 0, x2 (t1 ) = 1. Let the control constraint set be Ω = [−1, +1]. Investigate the optimal control problem for the cost functional J(u(·)) = t1 =

Zt1 0

1ds.

Chapter 10. Simplest Problem in Optimal Control

369

Problem 10.4 Consider the control system x˙ 1 (t) = u1 (t) + u2 (t), x˙ 2 (t) = u1 (t) − u2 (t), with initial condition x1 (0) = 1, x2 (0) = 1, and terminal condition x1 (t1 ) = 0, x2 (t1 ) = 0. Let the controls be constrained by |u1 (t)| ≤ 1 and |u2 (t)| ≤ 1. Investigate the optimal control problem for the cost functional J(u(·)) = t1 =

Zt1

1ds.

0

Problem 10.5 Consider the control system x(t) ˙ = u(t), with initial condition x(0) = 0, and terminal condition x(t1 ) = ϕ(t1 ), where ϕ(t) = t − 5. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional ) Zt1 ( p 1 + [u(s)]2 ds. J(u(·)) = x(s) 0

Problem 10.6 Consider the control system x˙ 1 (t) = −x1 (t) + x2 (t), x˙ 2 (t) = u(t),

370

Chapter 10. Simplest Problem in Optimal Control

with initial condition x1 (0) = 0, x2 (0) = 0, and terminal condition [x(t1 )]2 + [x(t1 )]2 = ϕ(t1 ), where ϕ(t) = t2 + 1. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional J(u(·)) =

Zt1

[u(s)]2 ds.

0

Problem 10.7 Consider the control system x(t) ˙ = −x(t) + u(t), with initial condition x(0) = 1, and terminal condition x(t1 ) = 0. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional J(u(·)) =

Zt1 0



1 + [x(t)]2 + [u(t)]2 ds.

Problem 10.8 Consider the control system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), with initial condition x1 (0) = 1, x2 (0) = 1

Chapter 10. Simplest Problem in Optimal Control

371

and terminal condition x1 (t1 ) = ϕ(t1 ), x2 (t1 ) = 0, where ϕ(t) = −t2 . Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional J(u(·)) =

Zt1 0

[u(s)]2 ds.

Chapter 11 Extensions of the Maximum Principle We now modify the SPOC by changing the problem data. In this chapter we treat the cases where the final time t1 is fixed and consider the case where the system is nonautonomous. In addition, we discuss two other formulations of the optimal control problem and show that all the formulations are equivalent.

11.1

A Fixed-Time Optimal Control Problem

We start with a statement of an optimal control problem defined on a given fixed time interval [t0 , t1 ] with t0 < t1 and t1 is fixed and given. Let P W C(t0 , t1 ; Rm ) denote the space of all Rm -valued piecewise continuous functions defined on [t0 , t1 ]. A function u(·) ∈ P W C(t0 , t1 ; Rm ) is called a control or control function. Therefore, we are given [t0 , t1 ], smooth functions f : R1 × Rn × Rm → Rn , f0 : R1 × Rn × Rm → R, an initial set a terminal set

X0 ⊆ R n , X1 ⊆ R n , 373

374

Chapter 11. Extensions of the Maximum Principle

and control constraint set Ω ⊆ Rm . As for the SPOC, the state equation is defined by the system of differential equations (S)

x(t) ˙ = f (t, x(t), u(t)),

t0 < t < t 1 .

(11.1)

The initial conditions are given by x(t0 ) = x0 ∈ X0 .

(11.2)

Again, we make the following standing assumption about the initial value problem. Standing Assumption for Optimal Control: For each x0 ∈ X0 and u(·) ∈ P W C(t0 , t1 ; Rm ), the initial value problem (11.1) - (11.2) has a unique solution defined on [t0 , t1 ]. We denote this solution by x(·; x0 ; u(·)). The set of admissible controllers is the subset of P W C(t0 , t1 ; Rm ) defined by   u(·) ∈ P W C(t0 , t1 ; Rm ) : u(t) ∈ Ω e.f. Θ= and u(·) steers X0 to X1 at time t1 . = Θ(t0 , t1 , X0 , X1 , Ω).

(11.3)

Given a control u(·) ∈ P W C(t0 , t1 ; Rm ), the cost functional is defined by J(u(·)) =

Zt1

t0

f0 (s, x(s), u(s))ds =

Zt1

f0 (s, x(s; x0 ; u(·)), u(s))ds,

t0

(11.4) where x(·) = x(·; x0 ; u(·)) is the solution to the initial value problem (11.1) - (11.2). Consider the optimal control problem of minimizing J(·) on the set of all admissible controllers Θ. In particular, an optimal

375

Chapter 11. Extensions of the Maximum Principle

control is a function u∗ (·) ∈ Θ such that u∗ (·) steers X0 to X1 at time t1 > t0 and J(u∗ (·)) =

Zt1

f0 (s, x(s; x∗0 ; u∗ (·)), u(s))ds

t0

(11.5) ≤

Zt1

f0 (s, x(s; x0 ; u(·)), u(s))ds = J(u(·))

t0

for all u(·) ∈ Θ. Remark 11.1 As before, if u∗ (·) ∈ Θ is an optimal control that steers x∗0 ∈ X0 to x∗1 ∈ X1 at time t1 > t0 , then the corresponding optimal trajectory x(·; x∗0 ; u∗ (·)) will be denoted by x∗ (·). In particular, x∗ (·) satisfies the two point boundary value problem (S ∗ ) (BCs∗ )

x˙ ∗ (t) = f (t, x∗ (t), u∗ (t)), t0 < t < t1 , x∗ (t0 ) = x∗0 ∈ X0 , x∗ (t1 ) = x∗1 ∈ X1 .

(11.6)

This notation is consistent since we have assumed that the initial value problem (11.1) - (11.2) has a unique solution. Remark 11.2 For the Simplest Problem in Optimal Control the final time t1 > t0 is free and finding the optimal time t∗1 > t0 is part of the problem. The time optimal control problem falls into this category. In the fixed-time optimal control problem the final time t1 > t0 is specified. Later we shall consider optimal control problems defined on the interval 0 ≤ t < +∞ which is a “special” fixed time problem.

11.1.1

The Maximum Principle for Fixed t1

When the final time t1 is given at a fixed value t0 < t1 , the Maximum Principle is the same except that the condition (10.22) is replaced by ˆ ηˆ∗ (t), x ˆ (ηˆ∗ (t), x ˆ∗ (t), u) ≡ c, M ˆ∗ (t)) = max H( u∈Ω

(11.7)

376

Chapter 11. Extensions of the Maximum Principle

where the constant c is not necessarily 0. In particular, the following theorem is valid in this case. Theorem 11.1 (Maximum Principle for t1 Fixed) Assume that f : Rn × Rm → Rn , f0 : Rn × Rm → R, X0 ⊆ Rn , t0 ∈ R, t1 ∈ R, X1 ⊆ Rn and Ω ⊆ Rm are given as above and consider the control system (S)

x(t) ˙ = f (x(t), u(t)),

t0 < t ≤ t 1 ,

(11.8)

with piecewise continuous controllers u(·) ∈ P W C(t0 , t1 ; Rm ) satisfying u(t) ∈ Ω ⊆ Rm e.f. If   u(·) ∈ P W C(t0 , t1 ; Rm ) : u(t) ∈ Ω e.f. ∗ u (·) ∈ Θ = and u(·) steers X0 to X1 at time t1 . (11.9) minimizes Zt1 (11.10) J(u(·)) = f0 (x(s), u(s))ds t0

on the set of admissible controls Θ with optimal response x∗ (·) satisfying x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t1 ) = x∗1 ∈ X1 , then there exists a non-trivial solution   ∗ η0 (t) T ∗ ∗ ∗ ∗ ∗ (11.11) ηˆ (t) = [η0 (t) η1 (t) η2 (t) ... ηn (t)] = η ∗ (t) to the augmented adjoint equation d ˆ T η(t), η(t) ˆ = −[A(t)] ˆ dt

(11.12)

such that ˆ ηˆ∗ (t), x ˆ (ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = M ˆ∗ (t)) = max H( ˆ∗ (t), u). u∈Ω

(11.13) Moreover, there are constants η0∗ ≤ 0 and c such that η0∗ (t) ≡ η0∗ ≤ 0

(11.14)

Chapter 11. Extensions of the Maximum Principle and for all t ∈ [t0 , t1 ], ˆ (ηˆ∗ (t), x ˆ ηˆ∗ (t), x M ˆ∗ (t)) = max H( ˆ∗ (t), u) ≡ c. u∈Ω

377

(11.15)

Also, if X0 ⊆ Rn and X1 ⊆ Rn are manifolds with tangent spaces T0 and T1 at x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t1 ) = x∗1 ∈ X1 , respectively, then  T ηˆ∗ (t) = [η0∗ (t) η1∗ (t) η2∗ (t) ... ηn∗ (t)]T = η0∗ (t) η ∗ (t)

can be selected to satisfy the transversality conditions η ∗ (t0 ) ⊥ T0

(11.16)

η ∗ (t1 ) ⊥ T1 .

(11.17)

and

11.2

Application to Problems in the Calculus of Variations

We start with some classical problems in the calculus of variations. First we assume the cost function does not explicitly depend on time. In particular, the cost functional is given by J(x(·)) =

Zt1

f0 (x(s), x˙ (s)) ds

t0

where f0 (x, u) is independent of time. We will address the time dependent case later.

11.2.1

The Simplest Problem in the Calculus of Variations

Assume t0 , t1 , x0 , and x1 are given and the cost functional is of the form Zt1 J(x(·)) = f0 (x(s), x˙ (s)) ds. t0

378

Chapter 11. Extensions of the Maximum Principle

Let Ψ1 = Ψ(t0 , t1 , x0 , x1 ) be the set of PWS functions defined by Ψ = {x(·) ∈ P W S(t0 , +∞) : x (t0 ) = x0 , x (t1 ) = x1 } and consider the SPCV where the objective is to find x∗ (·) ∈ Ψ such that ∗

J(x (·)) =

Zt1

f0 (x∗ (s), x˙ ∗ (s)) ds ≤ J(x(·))

Zt1

f0 (x(s), x˙ (s)) ds = J(x(·))

t0

=

t0

for all x(·) ∈ Ψ. In order to formulate an equivalent Simplest Problem in Optimal Control (with fixed t1 ), we define the state equation by x(t) ˙ = u(t)

(11.18)

x(t0 ) = x0 .

(11.19)

with initial condition Define the sets X0 = {x0 } ⊆ R1

and X1 = {x1 } ⊆ R1 ,

and let the control constraint set be Ω = R1 . The set of admissible controllers is the subset of P W C(t0 , t1 ) defined by   u(·) ∈ P W C(t0 , t1 ) : u(t) ∈ Ω e.f. and Θ= u(·) steers X0 to X1 at time t1 . = Θ(t0 , t1 , x0 , x1 , Ω).

(11.20)

Given a control u(·) ∈ P W C(t0 , t1 ), the cost functional is defined by Zt1 (11.21) J(u(·)) = f0 (x(s), u(s))ds, t0

Chapter 11. Extensions of the Maximum Principle

379

where x(·) = x(·; u(·)) is the solution to the initial value problem (11.18) - (11.19). This variational problem is equivalent to the optimal control problem of minimizing J(·) on the set of admissible controllers Θ. In particular, the goal is to find an optimal control u∗ (·) ∈ Θ such that u∗ (·) steers X0 to X1 at time t1 > t0 and J(u∗ (·)) =

Zt1

t0

f0 (x∗ (s), u∗ (s)) ds ≤

Zt1

f0 (x(s), u (s)) ds

t0

for all u(·) ∈ Θ. Formulation as an Optimal Control Problem: In order to set up the Simplest Problem in the Calculus of Variations as a fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). Clearly, X0 = {x0 } ⊆ R1 , X1 = {x1 } ⊆ R1 and Ω = R1 . Moreover, f (x, u) = u and f0 (x, u) = f0 (x, u). Therefore, the augmented function fˆ : R2 × R1 −→ R2 is defined by   f (x, u) 0 , fˆ(ˆ x, u) = fˆ(x0 , x, u) = u

380

Chapter 11. Extensions of the Maximum Principle

ˆ η, where x ˆ = [x0 x]T and the augmented Hamiltonian H( ˆ x ˆ, u) : 2 2 1 1 R × R × R −→ R is given by ˆ η, H( ˆ x ˆ, u) = η0 f0 (x, u) + ηf (x, u) = η0 f0 (x, u) + ηu where ηˆ = [η0 η]T . The augmented Jacobian is given by   (x,u) 0 ∂f0∂x ˆ ˆ Jxˆ f (ˆ x, u) = Jxˆ f (x, u) = 0 0 so that if (x∗ (·), u∗ (·)) = (x∗ (·), x˙ ∗ (·)) is an optimal pair, then the ˆ is given by augmented matrix A(t)  ∂f0 (x∗ (t),u∗ (t))  0 ∗ ∗ ˆ = Jxˆ fˆ(ˆ ∂x . A(t) x, u)|(x∗ (t),u∗ (t)) = Jxˆ fˆ(x (t), u (t)) = 0 0 Therefore, ˆ −[A(t)] =− T



0 0

∂f0 (x∗ (t),u∗ (t)) ∂x

0

T

=



0 − ∂f0 (x

∗ (t),u∗ (t))

∂x

0 0



and the augmented adjoint equation becomes        0 0 η0 (t) d η0 (t) T η0 (t) ˆ ∗ (t),u∗ (t)) = = −[A(t)] . η(t) − ∂f0 (x ∂x 0 η(t) dt η(t) (11.22) Consequently, d η0 (t) = 0 dt   d ∂f0 (x∗ (t), u∗ (t)) η(t) = − η0 dt ∂x which implies (as always) that the zero adjoint state is a constant η0 (t) ≡ η0 . If (x∗ (·), u∗ (·)) = (x∗ (·), x˙ ∗ (·)) is an optimal pair for the SPCV, then the Maximum Principle implies that there is a nontrivial solution  ∗  η0 (t) ∗ ηˆ (t) = ∈ R2 η ∗ (t)

Chapter 11. Extensions of the Maximum Principle

381

to the augmented adjoint equation (11.22) such that η0∗ (t) ≡ η0∗ ≤ 0. Observe that   d ∗ ∂f0 (x∗ (t), u∗ (t)) ∗ η0 (11.23) η (t) = − dt ∂x so that ∗

∗

η (t) = η (t0 ) −

η0∗

Z t t0

 ∂f0 (x∗ (s), u∗ (s)) ds. ∂x

(11.24)

The Maximum Principle implies that ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max1 H( ˆ∗ (t), u) u∈R

= max1 [η0∗ f0 (x∗ (t), u) + η ∗ (t)u] ≡ c (11.25) u∈R

ˆ ηˆ∗ (t), x and since Ω = R1 , the derivative of H( ˆ∗ (t), u) with respect to u at u = u∗ (t) must be zero. In particular, ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u)|u=u∗ (t) = Du H( ˆ∗ (t), u∗ (t)) = 0, where ∂ ∗ [η f0 (x∗ (t), u) + η ∗ (t)u] ∂u 0  ∗ ∗ ∂f0 (x (s), u) = η0 + η ∗ (t). ∂u

ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u) =

Thus, when u = u∗ (t)   ∗ ∗ ∗ ∗ ∗ ∂f0 (x (t), u (t)) ˆ + η ∗ (t) = 0 Du H(ηˆ (t), x ˆ (t), u)|u=u∗ (t) = η0 ∂u (11.26) and we have   ∗ ∗ ∗ ∗ ∂f0 (x (t), u (t)) η (t) = −η0 . (11.27) ∂u Observe that (11.27) implies that η0∗ 6= 0 because if η0∗ = 0, then we would also have η ∗ (t) = 0 which contradicts the statement that ηˆ∗ (t) is nontrivial. Combining (11.24) and (11.27) we find that    Z t ∗ ∗ ∂f0 (x∗ (s), u∗ (s)) ∗ ∂f0 (x (s), u (t)) ∗ ∗ −η0 = η (t0 ) − η0 ds ∂u ∂x t0

382

Chapter 11. Extensions of the Maximum Principle

and since η0∗ < 0, we divide both sides of this equation by −η0∗ to obtain    Z t ∂f0 (x∗ (s), u∗ (s)) ∂f0 (x∗ (s), u∗ (t)) =c+ ds (11.28) ∂u ∂x t0 where the constant c is given by c=

η ∗ (t0 ) . −η0∗

Note that (11.28) is the Euler Integral Equation and hence we have shown that the optimal pair (x∗ (·), u∗ (·)) with x˙ ∗ (t) = u∗ (t) satisfies the Euler Integral Equation    Z t ∂f0 (x∗ (s), x˙ ∗ (t)) ∂f0 (x∗ (s), x˙ ∗ (s)) =c+ ds, t0 < t < t1 . ∂u ∂x t0 (11.29) ∗ Also, between corners of x (·),     ∂f0 (x∗ (s), x˙ ∗ (s)) d ∂f0 (x∗ (s), x˙ ∗ (t)) = , t0 < t < t1 dt ∂u ∂x (11.30) and x∗ (t0 ) = x0 and x∗ (t1 ) = x1 which is the Euler Differential Equation for the SPCV. Consequently, we have used the Maximum Principle to derive the classical Euler Necessary Condition. The Weierstrass Necessary Condition One can also use the Maximum Principle to derive the Weierstrass Necessary Condition. Observe that for the SPCV ˆ η, H( ˆ x ˆ, u) = [η0 f0 (x, u) + ηu]

(11.31)

and the Maximum Principle (11.25) implies that ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max1 H( ˆ∗ (t), u) ≥ H( ˆ∗ (t), u) u∈R

(11.32)

Chapter 11. Extensions of the Maximum Principle

383

for all u ∈ R1 . Using the form of the augmented Hamiltonian ˆ η, H( ˆ x ˆ, u) : R2 × R2 × R1 −→ R1 given by ˆ η, H( ˆ x ˆ, u) = η0 f0 (x, u) + ηf (x, u) = η0 f0 (x, u) + ηu,

it follows that [η0∗ f0 (x∗ (t), u∗ (t))+η ∗ (t)u∗ (t)] ≥ [η0∗ f0 (x∗ (t), u)+η ∗ (t)u]. (11.33) Consequently, dividing by η0∗ < 0 yields f0 (x∗ (t), u∗ (t)) +

η ∗ (t) η ∗ (t) ∗ ∗ u (t) ≤ f (x (t), u) + u. 0 η0∗ η0∗

However, from equation (11.27) above, it follows that η ∗ (t) ∂f0 (x∗ (t), u∗ (t)) . = − η0∗ ∂u Therefore, substituting this into the previous inequality yields f0 (x∗ (t), u∗ (t)) −

∂f0 (x∗ (t), u∗ (t)) ∗ u (t) ≤f0 (x∗ (t), u) ∂u ∂f0 (x∗ (t), u∗ (t)) − u ∂u

which implies 0 ≤ f0 (x∗ (t), u) − f0 (x∗ (t), u∗ (t)) − [u − u∗ (t)]

∂f0 (x∗ (t), u∗ (t)) ∂u

for all u ∈ R1 . Recalling the definition of the Weierstrass Excess Function we have E(t, x∗ (t), u∗ (t), u) = f0 (x∗ (t), u) − f0 (x∗ (t), u∗ (t)) ∂f0 (x∗ (t), u∗ (t)) − [u − u∗ (t)] ∂u and hence it follows that 0 ≤ E(t, x∗ (t), u∗ (t), u) for all u ∈ R1 . This is the Weierstrass Necessary Condition - (II) for the SPCV.

384

Chapter 11. Extensions of the Maximum Principle

11.2.2

Free End-Point Problems

Formulation as an Optimal Control Problem: In order to set up the End-Point Problem as an optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). This problem is the same as the previous problem except that x1 is not given. Therefore, X0 = {x0 } ⊆ R1 and X1 = R1 and the tangent spaces are given by T0 = {0} and T1 = R1 , respectively. Therefore, the transversality condition at t1 implies that η ∗ (t1 ) = 0. (11.34) All the analysis above for the SPCV holds. In particular,   ∂f0 (x∗ (t), u∗ (t)) ∗ d ∗ η0 η (t) = − dt ∂x and ˆ ηˆ (t), x Du H( ˆ (t), u)|u=u∗ (t) = ∗

∗

η0∗



 ∂f0 (x∗ (t), u∗ (t)) + η ∗ (t) = 0, ∂u

which implies ∗

η (t) =

−η0∗



 ∂f0 (x∗ (t), u∗ (t)) . ∂u

(11.35)

Chapter 11. Extensions of the Maximum Principle

385

Also, (11.35) implies that η0∗ 6= 0. Therefore, we have shown the optimal pair (x∗ (·), u∗ (·)) with ∗ x˙ (t) = u∗ (t) satisfies Euler’s equation in integral form    Z t ∂f0 (x∗ (s), x˙ ∗ (s)) ∂f0 (x∗ (s), x˙ ∗ (t)) =c+ ds, t0 < t < t1 , ∂u ∂x t0 with initial condition x∗ (t0 ) = x0 . In order to obtain the “natural boundary condition”, we apply the transversality condition (11.34) to (11.35) and obtain   ∗ ∗ ∗ ∗ ∂f0 (x (t1 ), u (t1 )) . 0 = η (t1 ) = −η0 ∂u Again, since η0∗ 6= 0, it follows that the natural boundary condition is given by ∂f0 (x∗ (t1 ), u∗ (t1 )) ∂f0 (x∗ (t1 ), x˙ ∗ (t1 )) = =0 ∂u ∂u which matches the classical results.

11.2.3

Point-to-Curve Problems

Recall that in this problem we are given t0 , x0 , and a “curve” defined by the graph of a smooth function ϕ : R1 −→ R1 . The point-to-curve problem is to find x∗ (·) ∈ P W S(t0 , +∞) that minimizes Zt1 ˙ J(x(·)) = f0 (s, x(s), x(s))ds t0

on the set Ψϕ = {x(·) ∈ P W S(t0 , +∞) : x (t0 ) = x0 , x (t1 ) = ϕ(t1 ) for a finite time t1 > t0 }.

386

Chapter 11. Extensions of the Maximum Principle

In particular, one must find t∗1 and x∗ (·) ∈ P W S(t0 , t∗1 ) so that ∗

J(x∗ (·)) =

Zt1

f0 (s, x∗ (s), x˙ ∗ (s)) ds

≤

Zt1

t0

f0 (s, x(s), x˙ (s)) ds = J(x(·))

t0

for all x∗ (·) ∈ Ψϕ . Remark 11.3 In order to properly formulate this problem we note that the cost functional integrand f0 (s, x, u) is time dependent and does not allow a direct application of the previous Maximum Principles. In order to overcome this issue, again use the idea of an “augmented state” to reformulate the problem as a time independent optimal control problem. This basic idea is also used to extend the Maximum Principle for the SPOC to the case where the integrands and differential equations defining the state equations are time dependent (see Section 11.7). The basic idea is to treat time t as a “state” and “augment” the system with this new state. In particular, we define x1 (t) = t and x2 (t) = x(t). Observe that x˙ 1 (t) = 1 and x1 (t0 ) = t0 ∈ R1 , and x˙ 2 (t) = x(t) ˙ = u(t) and x2 (t0 ) = x0 ∈ R1 ,

x2 (t1 ) = x1 ∈ R1 .

Therefore, the control problem is governed by the two dimensional control system     d x1 (t) 1 , = u(t) dt x2 (t)

387

Chapter 11. Extensions of the Maximum Principle with initial condition  and terminal condition 

x1 (t0 ) x2 (t0 )

x1 (t1 ) x2 (t1 )





=



=



t1 ϕ(t1 )

t0 x0

 

,

where t1 is not specified. Note that because the integrand f0 (·) is a time dependent function f0 (t, x, u), the t variable becomes an additional “state” so the control problem is a 2-dimensional problem. Formulation as a Simplest Problem in Optimal Control: In order to set up the Point-to-Curve Problem as an SPOC and apply the Maximum Principle, we need to: 1. Identify the initial time t0 ; 2. Identify the initial set X0 ⊆ R2 , the terminal set X1 ⊆ R2 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R2 × R1 −→ R1 , f0 : R2 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). Define the sets X0 ⊆ R2 and X0 ⊆ R2 by   t0 X0 = ⊆ R2 x0 and X1 =



respectively.

x1 x2



=



t1 x2



2

∈ R : x2 = ϕ(x1 ) = ϕ(t1 )



⊆ R2 ,

388

Chapter 11. Extensions of the Maximum Principle

Observe that t1 , and hence x1 (t1 ) = t1 is not specified (other than t0 ≤ t1 ). Moreover, if we let G : R2 → R1 be defined by G(x1 , x2 ) = ϕ(x1 ) − x2 , then    x1 2 X1 = ∈ R : G(x1 , x2 ) = ϕ(x1 ) − x2 = 0 . x2  ∗   ∗  x1 x1 If ∈ X1 , then the tangent plane T1 at is defined by ∗ x2 x∗2  ∗  x1 . In particular, the gradient of G(x1 , x2 ) at x∗2     x1 ∗ ∗ T1 = x = : x ⊥ ∇G(x1 , x2 ) x2   η1 and hence a non-zero vector is orthogonal to T1 if and only η2 if     η1 ϕ(x ˙ ∗1 ) ∗ ∗ (11.36) = k∇G(x1 , x2 ) = k η2 −1

for some constant k 6= 0. We use this transversality condition several places below. Finally, note that Ω = R1 since there are no bounds on the control u(·). Observe that f : R2 × R1 −→ R2 is defined by     1 f1 (x1 , x1 , u) , = f (x1 , x2 , u) = u f2 (x1 , x1 , u) and f0 : R2 × R1 −→ R1 is given as the integrand f0 (x1 , x2 , u). Hence, the augmented function fˆ : R3 × R1 −→ R3 is defined by     f0 (x1 , x2 , u) f0 (x1 , x2 , u)  1 fˆ(ˆ x, u) = fˆ(x0 , x, u) =  f1 (x1 , x2 , u)  =  u f2 (x1 , x2 , u)

ˆ η, and the augmented Hamiltonian H( ˆ x ˆ, u) : R3 × R3 × R1 −→ R1 is given by ˆ η, H( ˆ x ˆ, u) = η0 f0 (x1 , x2 , u) + η1 f1 (x1 , x2 , u) + η2 f2 (x1 , x2 , u).

389

Chapter 11. Extensions of the Maximum Principle In particular, the augmented Hamiltonian has the form ˆ η, H( ˆ x ˆ, u) = η0 f0 (x1 , x2 , u) + η1 + η2 u,

where ηˆ = [η0 η]T . The augmented Jacobian is given by  ∂f0 (x1 ,x2 ,u) ∂f0 (x1 ,x2 ,u) ∂f0 (x1 ,x2 ,u)  Jxˆ fˆ(ˆ x, u) = Jxˆ fˆ(x, u) = 

∂x0

∂x1

0 0

∂x2

0 0

0 0



so that if (x∗ (·), u∗ (·)) = (x∗ (·), x˙ ∗ (·)) is an optimal pair, then the ˆ is given by augmented matrix A(t)  ∂f0 (x∗ (t),x∗ (t),u∗ (t)) ∂f0 (x∗ (t),x∗ (t),u∗ (t))  1 2 1 2 0 ∂x1 ∂x2 ˆ = Jxˆ fˆ(ˆ . A(t) x, u)|(x∗ (t),u∗ (t)) = 0 0 0 0 0 0 Therefore,



0

∂f0 (x∗1 (t),x∗2 (t),u∗ (t)) ˆ T = −[A(t)]  − ∂x1 ∂f (x∗ (t),x∗ (t),u∗ (t)) − 0 1 ∂x22

 0 0 0 0   0 0

and the augmented adjoint equation becomes     η0 (t) η0 (t) d  ˆ T  η1 (t)  . η1 (t)  = −[A(t)] dt η2 (t) η2 (t)

(11.37)

Consequently,

d η0 (t) = 0 dt   ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) d η0 η1 (t) = − dt ∂x1   ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) d η0 η2 (t) = − dt ∂x2 which implies (as always) that the zero adjoint state is a constant η0 (t) ≡ η0 .

390

Chapter 11. Extensions of the Maximum Principle

If (x∗ (·), u∗ (·)) = (x∗ (·), x˙ ∗ (·)) is an optimal pair for the SPCV, then the Maximum Principle implies that there is a nontrivial solution   ∗ η0 (t) ηˆ∗ (t) =  η1∗ (t)  ∈ R3 η2∗ (t)

to the augmented adjoint equation (11.37) such that η0∗ (t) ≡ η0∗ ≤ 0. Observe that   d ∗ ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) η0 (11.38) η (t) = − dt 2 ∂x2 so that η2∗ (t)

=

η2∗ (t0 )

−

η0∗

Z t t0

 ∂f0 (x∗1 (s), x∗2 (s), u∗ (s)) ds. ∂x2

(11.39)

Since t1 is free, the Maximum Principle implies that ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max1 H( ˆ∗ (t), u) = 0 u∈R

and for this problem we have ˆ ηˆ∗ (t), x max1 H( ˆ∗ (t), u) = max1 [η0∗ f0 (x∗1 (t), x∗2 (t), u) + η1∗ (t) + η2∗ (t)u] u∈R

u∈R

≡ 0.

(11.40)

ˆ ηˆ∗ (t), x Since Ω = R1 , the derivative of H( ˆ∗ (t), u) with respect to u at u = u∗ (t) must be zero. In particular, since t1 is free we have ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u)|u=u∗ (t) = Du H( ˆ∗ (t), u∗ (t)) = 0, where ∂ ∗ [η f0 (x∗1 (t), x∗2 (t), u) + η1∗ (t) + η2∗ (t)u] ∂u 0  ∗ ∗ ∗ ∂f0 (x1 (t), x2 (t), u) = η0 + η2∗ (t). ∂u

ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u) =

391

Chapter 11. Extensions of the Maximum Principle Thus, when u = u∗ (t) ˆ ηˆ (t), x Du H( ˆ (t), u)|u=u∗ (t) = ∗

∗

η0∗

=0



 ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) + η2∗ (t) ∂u (11.41)

and we have η2∗ (t)

=

−η0∗



 ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) . ∂u

(11.42)

Observe that (11.42) and (11.40) together imply that η0∗ 6= 0. To see this assume that η0∗ = 0. Equation (11.42) then implies that η2∗ (t) ≡ 0. The Maximum Principle (11.40) yields ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = η0∗ f0 (x∗1 (t), x∗2 (t), u∗ (t))+η1∗ (t)+η2∗ (t)u∗ (t) ≡ 0 so that if η0∗ = 0, then η2∗ (t) ≡ 0 and η0∗ f0 (x∗1 (t), x∗2 (t), u∗ (t)) + η1∗ (t) + η2∗ (t)u∗ (t) = η1∗ (t) ≡ 0. Hence, it would follow that ηˆ∗ (t) ≡ 0 which contradicts the statement that ηˆ∗ (t) is nontrivial. Consequently, it follows that η0∗ < 0. Combining (11.39) and (11.42) we find that   ∗ ∗ ∗ ∗ ∂f0 (x1 (t), x2 (t), u (t)) =η2∗ (t0 ) −η0 ∂u  Z t ∂f0 (x∗1 (s), x∗2 (s), u∗ (s)) ∗ − η0 ds ∂x2 t0 and since η0∗ < 0, we divide both sides of this equation by −η0∗ to obtain    Z t ∂f0 (x∗1 (t), x∗2 (t), u∗ (t)) ∂f0 (x∗1 (s), x∗2 (s), u∗ (s)) ds =c+ ∂u ∂x2 t0 (11.43) where the constant c is given by c=

η2∗ (t0 ) . −η0∗

392

Chapter 11. Extensions of the Maximum Principle

Observe that (11.43) is Euler’s equation in integral form. Therefore, we have shown that the optimal solution (x∗1 (·), x∗2 (·), u∗ (·)) with x˙ ∗2 (t) = x˙ ∗ (t) = u∗ (t) and x∗1 (·) = t satisfies Euler’s equation in integral form    Z t ∂f0 (t, x∗ (t), x˙ ∗ (t)) ∂f0 (t, x∗ (t), x˙ ∗ (t)) = c+ ds, t0 < t < t∗1 . ∂u ∂x t0 (11.44) Also, we have that between corners of x∗ (·),     d ∂f0 (t, x∗ (t), x˙ ∗ (t)) ∂f0 (t, x∗ (t), x˙ ∗ (t)) = , t0 < t < t∗1 dt ∂u ∂x (11.45) and x∗ (t0 ) = x0 and x∗ (t∗1 ) = ϕ(t∗1 ). To complete the problem we need to use the transversality condition at t∗1 . The transversality condition (11.36) implies that      ∗ ∗  ϕ(x ˙ ∗1 (t∗1 )) η1 (t1 ) ϕ(t ˙ ∗1 ) ∗ ∗ ∗ ∗ =k = k∇G(x1 (t1 ), x2 (t1 )) = k −1 η2∗ (t∗1 ) −1 and hence it follows that η1∗ (t∗1 ) = k ϕ(t ˙ ∗1 )

(11.46)

and η2∗ (t∗1 )

=

−η0∗



 ∂f0 (x∗1 (t∗1 ), x∗2 (t∗1 ), u∗ (t∗1 )) = −k. ∂u

(11.47)

Again, the Maximum Principle (11.40) implies that at t = t∗1 ∗ ∗ ∗ ∗ ˆ ηˆ∗ (t∗ ), x 0 = H( 1 ˆ (t1 ), u (t1 )) = η0∗ f0 (x∗1 (t∗1 ), x∗2 (t∗1 ), u∗ (t∗1 )) + η1∗ (t∗1 ) + η2∗ (t∗1 )u∗ (t∗1 ) = η0∗ f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) + η1∗ (t∗1 ) + η2∗ (t∗1 )x˙ ∗ (t∗1 ),

and applying (11.46) - (11.47) we have

Chapter 11. Extensions of the Maximum Principle

393

0 = η0∗ f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) + k ϕ(t ˙ ∗)   1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∂f0 (x1 (t1 ), x2 (t1 ), u (t1 )) x˙ ∗ (t∗1 ) − η0 ∂u   ∗ ∗ ∗ ˙ ∗ (t∗1 )) ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∂f0 (t1 , x (t1 ), x ϕ(t ˙ ∗1 ) = η0 f0 (t1 , x (t1 ), x˙ (t1 )) + η0 ∂u   ∗ ∗ ∗ ˙ ∗ (t∗1 )) ∗ ∗ ∗ ∂f0 (t1 , x (t1 ), x − η0 x˙ (t1 ). ∂u Dividing by η0∗ < 0 yields   ∂f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) ∗ ∗ ∗ ∗ ∗ 0 = f0 (t1 , x (t1 ), x˙ (t1 )) + ϕ(t ˙ ∗1 ) ∂u   ∂f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) ∗ ∗ − x˙ (t1 ) ∂u   ∂f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) ∗ ∗ ∗ ∗ ∗ [ϕ(t ˙ ∗1 ) − x˙ ∗ (t∗1 )] , = f0 (t1 , x (t1 ), x˙ (t1 )) + ∂u which implies the classical natural transversality condition   ∂f0 (t∗1 , x∗ (t∗1 ), x˙ ∗ (t∗1 )) ∗ ∗ ∗ ∗ ∗ f0 (t1 , x (t1 ), x˙ (t1 )) + [ϕ(t ˙ ∗1 ) − x˙ ∗ (t∗1 )] = 0. ∂u (11.48)

11.3

Application to the Farmer’s Allocation Problem

Here we apply the Maximum Principle to the Farmer’s Allocation Problem first presented as Example 7.8 in Section 7.7.1 above. We refer to Luenburger’s book ([131]) for details. The basic problem is that a farmer is assumed to produce a single crop (wheat, rice, etc.) and when he sells his crop he stores the crop or else sells the crop and reinvests the money into his business to increase his production rate. The farmer’s goal is to maximize the total amount

394

Chapter 11. Extensions of the Maximum Principle

of crop stored up to two years. Luenberger formulates this as the following optimal control problem. Given the initial time t0 = 0, the final time t1 = 2 and the initial state 1 > 0, find a control u∗ (·) to minimize J(u(·)) =

Z2

(u(s) − 1)x(s)ds,

0

subject to x(t) ˙ = u(t)x(t)

(11.49)

x(0) = 1.

(11.50)

with initial condition The control constraint is given by u(t) ∈ [0, 1] . Here, u(t) is the fraction of the production rate that is reinvested at time t. It is important to note that since u(t) ≥ 0 and x(t) ˙ = u(t)x(t), all solutions to the system (11.49) - (11.50) satisfy x(t) ≥ 1.

(11.51)

Formulation as an Optimal Control Problem: In order to set up the Farmer’s Allocation Problem as a fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t).

Chapter 11. Extensions of the Maximum Principle

395

It is obvious that t0 = 0, t1 = 2, X0 = {x0 } = {1} ⊆ R1 and X1 = R1 . Note that the corresponding tangent spaces are given by T0 = {0} and T1 = R1 and hence the transversality condition on the adjoint equation at t = 2 is η ∗ (2) = 0. (11.52) The control constraint set is Ω = [0, 1] ⊆ R1 , the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 are given by f (x, u) = ux and f0 (x, u) = (u − 1)x,

respectively. The set of admissible controllers is the subset of P W C(0, 2) defined by   u(·) ∈ P W C(0, 2) : u(t) ∈ Ω e.f. and Θ= u(·) steers X0 to X1 at time 2. = Θ(0, 2, x0 = 1, Ω).

(11.53)

Given a control u(·) ∈ P W C(0, 2), the cost functional is defined by J(u(·)) =

Z2

(u(s) − 1)x(s)ds =

Z2

f0 (x(s), u(s))ds,

(11.54)

0

0

where x(·) = x(·; u(·)) is the solution to the initial value problem (11.18) - (11.19). This optimal control problem is equivalent to the problem of minimizing J(·) on the set of admissible controllers Θ. In particular, the goal is to find an optimal control u∗ (·) ∈ Θ such that u∗ (·) steers X0 to X1 at time 2 and ∗

J(x (·)) =

Z2 0

∗

∗

f0 (x (s), u (s)) ds ≤

Z2 0

f0 (x(s), u (s)) ds = J(x(·))

396

Chapter 11. Extensions of the Maximum Principle

for all u(·) ∈ Θ. The augmented function fˆ : R2 × R1 −→ R2 is defined by     (u − 1)x f (x, u) 0 , = fˆ(ˆ x, u) = fˆ(x0 , x, u) = ux f (x, u) ˆ η, where x ˆ = [x0 x]T . The augmented Hamiltonian H( ˆ x ˆ, u): R2 × R2 × R1 −→ R1 is given by ˆ η, H( ˆ x ˆ, u) = η0 f0 (x, u) + ηf (x, u) = η0 (u − 1)x + ηux, where ηˆ = [η0 η]T . The augmented Jacobian is given by " #   ∂f0 (x,u) 0 0 (u − 1) ∂x ˆ ˆ Jxˆ f (ˆ x, u) = Jxˆ f (x, u) = = (x,u) 0 u 0 ∂f ∂x so that if (x∗ (·), u∗ (·)) is an optimal pair, then the augmented ˆ is given by matrix A(t)   ∗ 0 (u (t) − 1) ∗ ∗ ˆ = Jxˆ fˆ(ˆ . A(t) x, u)|(x∗ (t),u∗ (t)) = Jxˆ fˆ(x (t), u (t)) = 0 u∗ (t) Therefore, ˆ −[A(t)] =− T



0 (u∗ (t) − 1) 0 u∗ (t)

T

=



0 0 −(u∗ (t) − 1) −u∗ (t)



and the augmented adjoint equation becomes        d η0 (t) η (t) 0 0 η (t) 0 0 T ˆ . = = −[A(t)] η(t) −(u∗ (t) − 1) −u∗ (t) η(t) dt η(t) (11.55) Consequently, d η0 (t) = 0 dt d η(t) = −η0 [u∗ (t) − 1] − u∗ (t)η(t) dt which implies (as always) that the zero adjoint state is a constant η0 (t) ≡ η0 .

Chapter 11. Extensions of the Maximum Principle

397

If (x∗ (·), u∗ (·)) is an optimal pair for the Farmer’s Allocation Problem, then the Maximum Principle implies that is a nontrivial solution   ∗ η0 (t) ∗ ∈ R2 ηˆ (t) = η ∗ (t)

to the augmented adjoint equation (11.55) such that η0∗ (t) ≡ η0∗ ≤ 0. Observe that the transversality condition (11.52) implies that η ∗ (t) satisfies d ∗ η (t) = −u∗ (t)η ∗ (t) + η0∗ [1 − u∗ (t)] , η ∗ (2) = 0. dt

(11.56)

It again follows that η0∗ 6= 0 because if η0∗ = 0, then η ∗ (t) would be a solution to the homogenous linear initial value problem with zero initial data d ∗ η (t) = −u∗ (t)η ∗ (t), η ∗ (2) = 0, dt

(11.57)

which would imply that η ∗ (t) = 0. Dividing (11.56) by −η0∗ > 0 yields   ∗   ∗   η (t) η (2) d η ∗ (t) ∗ ∗ = −u (t) − [1 − u (t)] , = 0. ∗ ∗ dt −η0 −η0 −η0∗ (11.58) ∗ Defining the normalized adjoint variable λ (t) by  ∗  η (t) ∗ λ (t) = −η0∗ yields d ∗ λ (t) = −u∗ (t)λ∗ (t) − [1 − u∗ (t)] , λ∗ (2) = 0. dt

(11.59)

Also, since −η0∗ > 0 it follows that maximizing the augmented Hamiltonian ˆ ηˆ∗ (t), x H( ˆ∗ (t), u) = [η0∗ (u − 1)x∗ (t) + η ∗ (t)x∗ (t)u]

398

Chapter 11. Extensions of the Maximum Principle

is equivalent to maximizing 1 ∗ [η (u − 1)x∗ (t) + η ∗ (t)x∗ (t)u] −η0∗ 0  ∗  η (t) ∗ ∗ x (t)u] = [−(u − 1)x (t) + −η0∗ = [−(u − 1)x∗ (t) + λ∗ (t)x∗ (t)u] where λ∗ (t) satisfies (11.59). Note that max [−(u−1)x∗ (t)+λ∗ (t)ux∗ (t)] = max [(1−u)x∗ (t)+λ∗ (t)ux∗ (t)]

0≤u≤1

0≤u≤1

and since x∗ (t) ≥ 1 it follows that ∗

∗

∗

∗



max [(1 − u) + λ (t)u] x∗ (t)   ∗ = max [(1 + (λ (t) − 1)u] x∗ (t).

max [(1 − u)x (t) + λ (t)ux (t)] =

0≤u≤1



0≤u≤1

0≤u≤1

Thus, the optimal control must maximize the term (λ∗ (t) − 1)u on the interval [0, 1] which implies that  1, if λ∗ (t) > 1, ∗ u (t) = 0, if λ∗ (t) < 1.

(11.60)

We need only compute λ∗ (t) using the adjoint equation d ∗ λ (t) = −u∗ (t)λ∗ (t) − [1 − u∗ (t)] , λ∗ (2) = 0. dt

(11.61)

However, at t = 2 we know that λ∗ (2) = 0 and since λ∗ (t) is continuous, there is an interval [T, 2] with T < 2 such that λ∗ (t) < 1 for all t ∈ (T, 2].

Chapter 11. Extensions of the Maximum Principle

399

On this interval the optimal control must be u∗ (t) = 0 and the corresponding adjoint equation (11.61) has the form d ∗ λ (t) = −1, λ∗ (2) = 0. dt Thus, λ∗ (t) = −t + 2

until the time where

1 = λ∗ (T ) = −T + 2 and then the optimal control switches to u∗ (t) = 1. Clearly, T = 1 and hence the optimal control on the interval [0, T ] = [0, 1] is given by u∗ (t) = 1. Again, returning to the adjoint equation (11.61) on the interval [0, 1] yields d ∗ λ (t) = −u∗ (t)λ∗ (t)−[1 − u∗ (t)] = −λ∗ (t), λ∗ (1) = 1 (11.62) dt which has the solution λ∗ (t) = e1−t . Therefore, we know that  1−t e , ∗ λ (t) = −t + 2, and ∗

u (t) =



1, 0,

if 0 ≤ t ≤ 1, if 1 ≤ t ≤ 2

if 0 ≤ t ≤ 1, if 1 ≤ t ≤ 2

(11.63)

is the corresponding optimal control. Finally, the corresponding optimal trajectory is given by  t e , if 0 ≤ t ≤ 1, ∗ x (t) = (11.64) e, if 1 ≤ t ≤ 2 and if the optimal control problem has a solution, then the optimal controller is given by (11.63) with corresponding optimal trajectory (11.64).

400

11.4

Chapter 11. Extensions of the Maximum Principle

Application to a Forced Oscillator Control Problem

Suppose we have a forced oscillator with its equilibrium position at the origin and driven by the force u(t). The equation of motion is given by the second order equation x¨(t) + x(t) = u(t) and we consider the cost function given by Z t1 1 2 u (s) ds, J(u(·)) = 2 0

(11.65)

where t1 < 12 π is fixed. Note that if the terminal time t1 was free or if t1 ≥ 21 π, then one can reach the target without applying any control in a quarter-period of the oscillation and the cost is then zero. Thus, the problem is only interesting if t1 < 21 π is fixed. If x1 (t) = x(t) denotes the position, then x2 (t) = x(t) ˙ is the velocity and the state equations have the form        d x1 (t) 0 x1 (t) 0 1 u(t). (11.66) + = 1 x2 (t) −1 0 dt x2 (t) Suppose that initially at time t0 = 0, the system is at rest at a distance 2 (miles) from the equilibrium position 0 = [0 0]T and that we wish to move to the zero position at a prescribed fixed time t1 , without requiring that we arrive there with zero velocity. Formulation as an Optimal Control Problem: In order to set up the forced oscillator control problem as a fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and the final time t1 ; 2. Identify the initial set X0 ⊆ R2 , the terminal set X1 ⊆ R2 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R2 × R1 −→ R1 , f0 : R2 × R1 −→ R1 ;

Chapter 11. Extensions of the Maximum Principle

401

ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). The initial and final states are given by     2 x1 (0) = 0 x2 (0) and



x1 (t1 ) x2 (t1 )



∈



0 x2



: x2 ∈ R

1



= {0} × R1 ,

respectively. Therefore, the initial and final sets are given by   2 and X1 = {0} × R1 , (11.67) X0 = 0 respectively. Since since there is no hard constraint on the control, the control constraint set is Ω = R1 . The functions f0 : R2 ×R1 −→ R1 and f : R2 × R1 −→ R1 are defined by 1 f (x1 , x2 , u) = u2 2 and     x2 f1 (x1 , x2 , u) , f (x1 , x2 , u) = = f2 (x1 , x2 , u) (−x1 + u) so that the augmented function fˆ : R3 × R1 → R3 is given by   1 2 u 2 . x2 fˆ (x0 , x1 , x2 , u) =  (−x1 + u)

The augmented Hamiltonian is given by

1 ˆ η, H( ˆ x ˆ, u) = η0 u2 + η1 x2 + η2 (−x1 + u). 2 If (x∗ (·), u∗ (·)) is an optimal pair, then     0 0 0 0 0 0 ˆ = 0 0 1  = 0 0 1  A(t) 0 −1 0 0 −1 0 |x=x∗ (t),u=u∗ (t)

402

Chapter 11. Extensions of the Maximum Principle

and the adjoint equations become   0 0 0 d ˆ η(t) ˆ =  0 0 1  η(t). dt 0 −1 0

(11.68)

The Maximum Principle implies that there exists a nonzero function ηˆ∗ (·) = [η0∗ (·) η1∗ (·) η2∗ (·)]T which is a solution to the adjoint equation (11.68) such that η0∗ (t) ≡ η0∗ ≤ 0 and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max1 H( ˆ∗ (t), u) ≡ c. u∈R

Since Ω = R1 it follows that ∂ ˆ ∗ H(ηˆ (t), x ˆ∗ (t), u)|u=u∗ (t) ≡ 0 ∂u and hence we have η0∗ u∗ (t) + η2∗ (t) ≡ 0.

(11.69)

Moreover, the transversality condition states that η ∗ (t1 ) can be taken to be normal to the tangent plane for the target set X1 and since X1 = {0} × R1 is a subspace it follows that T1 = {0} × R1 . Consequently, if   ∗ η1 (t1 ) ∗ ⊥{0} × R1 , η (t1 ) = η2∗ (t1 ) then η2∗ (t1 ) = 0, so that there is a constant A so that η1∗ (t) = A cos(t1 − t) and η2∗ (t) = A sin(t1 − t).

Observe that if η0∗ = 0, then (11.69) implies that η2∗ (t) ≡ 0 so that A = 0. Hence, it would follow that η1∗ (t) = A cos(t1 − t) ≡ 0

Chapter 11. Extensions of the Maximum Principle

403

which can not happen since ηˆ∗ (·) is nonzero. Therefore, η0∗ < 0 and we can solve (11.69) for u∗ (t) to yield u∗ (t) = −

η2∗ (t) = A˜ sin(t1 − t) η0∗

(11.70)

where A˜ = −A/η0∗ . When u∗ (t) = A˜ sin(t1 − t), the solution to the state equation (11.66) is easily found to be given by 1˜ ˜ sin t, x∗1 (t) = At cos(t1 − t) + 2 cos t + B (11.71) 2 1˜ 1 ˜ cos t, (11.72) sin(t1 − t) − 2 sin t + B x∗2 (t) = A˜ cos(t1 − t) + At 2 2 ˜ However, x∗2 (0) = 0 and x∗1 (t1 ) = 0 so that for some constant B. 1 ˜ 0 = x∗2 (0) = A˜ cos(t1 ) + B 2 and

1˜ ˜ 0 = x∗1 (t1 ) = At 1 + 2 cos t1 + B sin t1 . 2 ˜ we find Solving these two equations for A˜ and B A˜ = and

−4 cos t1 t1 − sin t1 cos t1

˜ = − 1 A˜ cos t1 . B 2 Finally, the optimal cost can be found by substituting u∗ (t) = A˜ sin(t1 − t)

into the cost function (11.65) and integrating we find J(u∗ (·)) =

4 cos2 t1 . t1 − sin t1 cos t1

(11.73)

Note that if t1 = 12 π, then A˜ = 0 and the optimal cost is zero. For each t1 < 12 π, the optimal cost is positive and as t1 → 0, J(u∗ (·)) → +∞ so that the cost grows to infinity as the time interval shrinks.

404

11.5

Chapter 11. Extensions of the Maximum Principle

Application to the Linear Quadratic Control Problem

We apply the Maximum Principle to a Linear Quadratic (LQ) optimal control problem. Let A be a n × n constant real matrix and B be a n×m real constant matrix. Consider the linear control system x(t) ˙ = Ax(t) + Bu(t) (11.74) with initial data x(0) = x0 ∈ Rn .

(11.75)

We assume that t0 = 0, x0 ∈ Rn and 0 < t1 are given. Also, let Q be a n × n constant symmetric real matrix and R be a m × m constant symmetric real matrix such that QT = Q ≥ 0 and RT = R > 0. Here, the inequality Q ≥ 0 is equivalent to the condition that Q is non-negative hQx, xi ≥ 0 for all x ∈ Rn , and R > 0 is equivalent to the condition that R is positive definite hRu, ui > 0 for all u ∈ Rm with u 6= 0. We assume that the controls belong to the space P W C(0, t1 ; Rm ). The quadratic cost functional is defined by Z 1 t1 {hQx(s), x(s)i + hRu(s), u(s)i} ds, (11.76) J(u(·)) = 2 0 where x(t) = x(t; u(·)) is the solution to the system (11.74) (11.75).The Linear Quadratic (LQ) Optimal Control problem is to find u∗ (·) ∈ P W C(0, t1 ; Rm ) so that Z 1 t1 ∗ {hQx(s; u∗ (·)), x(s; u∗ (·))i + hRu∗ (s), u∗ (s)i} ds J(u (·)) = 2 t0 Z 1 t1 {hQx(s; u(·)), x(s; u(·))i ≤ J(u(·)) = 2 t0 + hRu(s), u(s)i}ds,

Chapter 11. Extensions of the Maximum Principle

405

for all u(·) ∈ P W C(0, t1 ; Rm ). Formulation as an Optimal Control Problem: In order to set up the Quadratic Optimal Control Problem as a fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ Rn , the terminal set X1 ⊆ Rn ; 3. Identify the control constraint set Ω ⊆ Rm ; 4. Identify the functions f : Rn × Rm −→ Rn , f0 : Rn × Rm −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). The initial time is t0 = 0 and the final time t1 is given and fixed. The initial set is the single initial vector X0 = {x0 } and since there is no terminal constraint, the terminal set is the whole state space X1 = Rn . Also, there is no constraint on the control u(·) ∈ P W C(0, t1 ; Rm ), hence the control constraint set is all of Rm and Ω = Rm . The functions f : Rn × Rm −→ Rn and f0 : Rn × Rm −→ R1 are given by f (x, u) = Ax + Bu, and f0 (x, u) =

1 1 hQx, xi + hRu, ui ≥ 0, 2 2

respectively. ˆ : Rn+1 × Rn+1 × Rm −→ R1 is The augmented Hamiltonian H defined by η0 ˆ η, {hQx, xi + hRu, ui} + hη, Ax + Bui (11.77) H( ˆ x ˆ, u) = 2 η0 η0 = xT Q x+ uT Ru + [Ax]T η + [Bu]T η 2 2 η0 T η0 T = x Q x+ u Ru + xT AT η + uT B T η, 2 2

406

Chapter 11. Extensions of the Maximum Principle

 T  T where x ˆ = x0 x ∈ Rn+1 , ηˆ = η0 η ∈ Rn+1 and u ∈ Rm . The augmented function f : Rn+1 × Rm −→ Rn+1 is defined by    1  1 T hQx, xi + 21 hRu, ui xT Qx + 21 u Ru 2 ˆ 2 , = f (ˆ x, u) = Ax + Bu, Ax + Bu, (11.78) so that the Jacobian is given by   0 [Qx]T ˆ ˆ . (11.79) Jxˆ f (ˆ x, u) = Jxˆ f (x, u) = 0 A Now assume that (x∗ (·), u∗ (·)) is an optimal pair so that the ˆ is given by matrix A(t) ˆ = Jxˆ fˆ(ˆ A(t) x, u)|(x∗ (t),u∗ (t)) ∗



0 0 ∗ [Qx (t)] AT



= Jxˆ fˆ(x (t), u (t)) = ∗

0 [Qx∗ (t)]T 0 A



(11.80)

and ˆ T =− −[A(t)]



=



0 0 ∗ − [Qx (t)] −AT

The augmented adjoint equation has the form     d η0 (t) η0 (t) T ˆ = −[A(t)] η(t) dt η(t)    0 0 η0 (t) = − [Qx∗ (t)] −AT η(t)



.

(11.81)

which is equivalent to the system d η0 (t) = 0 dt d η(t) = −η0 (t)Qx∗ (t) − AT η(t), dt where again x∗ (·) is the optimal trajectory. Again, the first equation above implies that the zero adjoint state is a constant η0 (t) ≡

407

Chapter 11. Extensions of the Maximum Principle

η0 . The second equation is coupled to the state equation by the term −η0 (t)Qx∗ (t). If (x∗ (·), u∗ (·)) is an optimal pair for the LQ optimal control problem, then the Maximum Principle implies that is a nontrivial solution   ∗ η0 (t) ∗ ∈ Rn+1 ηˆ (t) = η ∗ (t)

to the augmented adjoint equation (11.81) such that η0∗ (t) ≡ η0∗ ≤ 0, d ∗ (11.82) η (t) = −η0∗ Qx∗ (t) − AT η ∗ (t) dt and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = maxm H( ˆ∗ (t), u) ≡ c. u∈R

Since Ω = Rm is open, then ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u)|u=u∗ (t) = Du H( ˆ∗ (t), u∗ (t)) = 0, where  η∗ ∂ η0∗ ∗ T ∗ ∗ ˆ Du H(ηˆ (t), x ˆ (t), u) = [x (t)] Q x+ 0 uT Ru ∂u 2 2 ∗

T

T

∗

T

T

∗

+[x (t)] A η (t) + u B η (t)  ∗  η0 T ∂ T T ∗ u Ru + u B η (t) = ∂u 2 = η0∗ Ru + B T η ∗ (t),



so that when u = u∗ (t) ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u)|u=u∗ (t) = η0∗ Ru + B T η ∗ (t) |u=u∗ (t) (11.83) = η0∗ Ru∗ (t) + B T η ∗ (t) = 0.

Applying the transversality condition at x∗ (0) = x0 ∈ X0 = {x0 } yields that η ∗ (0) can be any vector since T0 = {x0 }. However,

408

Chapter 11. Extensions of the Maximum Principle

at t1 we have that x∗ (t1 ) ∈ X1 = Rn and since T1 = X1 = Rn , the transversality condition η ∗ (t∗1 ) ⊥ T1 = Rn implies that η ∗ (t1 ) = 0.

(11.84)

This boundary condition in turn implies that η0∗ < 0. To see this assume that η0∗ = 0 and observe that the adjoint equation (11.82) reduces to the linear system d ∗ η (t) = −η0∗ Qx∗ (t) − AT η ∗ (t) dt = −AT η ∗ (t). Therefore, η ∗ (·) would be a solution of the homogenous linear initial value problem d ∗ η (t) = −AT η ∗ (t), dt

η ∗ (t1 ) = 0

and hence it follows that η ∗ (t) ≡ 0. Consequently, we have shown that if η0∗ = 0, then η ∗ (t) ≡ 0 and hence   ∗ η0 (t) ∗ ≡0 ηˆ (t) = η ∗ (t) which contradicts the statement that ηˆ∗ (t) is a nontrivial solution of the augmented adjoint equation (11.81). Since η0∗ < 0 we can solve (11.83) for the optimal control. We have η0∗ Ru∗ (t) + B T η ∗ (t) = 0 which yields η0∗ Ru∗ (t) = −B T η ∗ (t) and Ru∗ (t) =

−1 T ∗ B η (t). η0∗

Chapter 11. Extensions of the Maximum Principle

409

By assumption, the matrix R = RT > 0 is nonsingular and hence we have the following expression for the optimal control  ∗  ∗ −1 T η (t) . (11.85) u (t) = R B −η0∗ Summarizing, it follows that the optimal trajectory can be obtained by solving the two point boundary value problem defined by the coupled state and adjoint equations d ∗ x (t) dt d ∗ η (t) dt

= Ax∗ (t) − η1∗ BR−1 B T η ∗ (t), x∗ (0) = x0 , 0 = −η0∗ Qx∗ (t) − AT η ∗ (t), η ∗ (t1 ) = 0,

and setting ∗

−1

u (t) = R B

T



(11.86)

 η ∗ (t) . −η0∗

To eliminate the η0∗ term, we divide the adjoint equation above by −η0∗ which yields  ∗    ∗   d η ∗ (t) η (t1 ) ∗ T η (t) = Qx (t) − A , = 0. dt −η0∗ −η0∗ −η0∗ Defining the normalized adjoint state λ∗ (t) by λ∗ (t) ,

η ∗ (t) −η0∗

produces the optimality conditions d ∗ x (t) dt d ∗ λ (t) dt

= Ax∗ (t) + BR−1 B T λ∗ (t), = Qx∗ (t) − AT λ∗ (t),

x∗ (0) = x0 , λ∗ (t1 ) = 0,

(11.87)

where the optimal control is defined by u∗ (t) = R−1 B T λ∗ (t). We can write the optimality system as      ∗ d x∗ (t) A BR−1 B T x (t) = , ∗ Q −AT λ∗ (t) dt λ (t)

(11.88)

(11.89)

410

Chapter 11. Extensions of the Maximum Principle

with boundary conditions ∗

x (0) = and ∗

λ (t1 ) =

 



In×n 0n×n

0n×n In×n





x∗ (0) λ∗ (0)



x∗ (t1 ) λ∗ (t1 )



= x0 ,

(11.90)



(11.91)

= 0.

Thus, if one solves the two point boundary value problem (11.89) - (11.91), the optimal control is defined by (11.88).

11.5.1

Examples of LQ Optimal Control Problems

Here we apply the Maximum Principle to specific LQ optimal control problem. We consider a problem with fixed final x1 and a problem where x1 is not specified. LQ Optimal Control Problem: Example 1 Consider the linear control system x(t) ˙ = x(t) + u(t)

(11.92)

x(0) = 1/2 ∈ R1

(11.93)

x(1) = 0 ∈ R1 .

(11.94)

with initial data and terminal condition

We assume that t0 = 0, 0 < t1 = 1, x0 = 1/2 ∈ R1 and x1 = 0 ∈ R1 are given. The quadratic cost functional is defined by J(u(·)) =

Z1 0

1 [u(s)]2 ds. 2

(11.95)

Chapter 11. Extensions of the Maximum Principle

411

The Linear Quadratic (LQ) optimal control problem is to find u∗ (·) ∈ P W C(0, 1) so that ∗

J(u (·)) =

Z1

1 ∗ [u (s)]2 ds ≤ J(u(·)) = 2

Z1

1 [u(s)]2 ds 2

0

0

for all u(·) ∈ P W C(0, 1) that steer x0 = 1/2 to x1 = 0. Formulation as an Optimal Control Problem: In order to set up the fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ; ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). The initial time is t0 = 0 and the final time t1 = 1 is given and fixed. The initial set is X0 = {1/2} and the terminal constraint set is X1 = {0}. Also, there is no constraint on the control u(·) ∈ P W C(0, 1), hence the control constraint set is all of R1 and Ω = R1 . The functions f : R1 × R1 −→ R1 and f0 : R1 × R1 −→ R1 are given by f (x, u) = x + u, and

1 f0 (x, u) = u2 ≥ 0, 2

respectively. ˆ : R2 × R2 × R1 −→ R1 defined The augmented Hamiltonian H by 1 ˆ η, (11.96) H( ˆ x ˆ, u) = η0 u2 + η(x + u) 2

412

Chapter 11. Extensions of the Maximum Principle

 T  T where x ˆ = x0 x ∈ R2 , ηˆ = η0 η ∈ R2 and u ∈ R1 . The augmented function fˆ : R2 × R1 −→ R2 is defined by  1 2  u 2 , (11.97) fˆ(ˆ x, u) = x+u so that the Jacobian is given by Jxˆ fˆ(ˆ x, u) = Jxˆ fˆ(x, u) =



0 0 0 1



.

(11.98)

Now assume that (x∗ (·), u∗ (·)) is an optimal pair so that the ˆ is given by matrix A(t)   0 0 ∗ ∗ ˆ = Jxˆ fˆ(ˆ A(t) x, u)|(x∗ (t),u∗ (t)) = Jxˆ fˆ(x (t), u (t)) = 0 1 (11.99) and     0 0 0 0 T ˆ . = −[A(t)] = − 0 −1 0 1

The augmented adjoint equation has the form        d η0 (t) η0 (t) 0 0 η0 (t) T ˆ = = −[A(t)] η(t) 0 −1 η(t) dt η(t) (11.100) which is equivalent to the system d η0 (t) = 0 dt d η(t) = −η(t), dt where again x∗ (t) is the optimal trajectory. The first equation above implies that the zero adjoint state is a constant η0 (t) ≡ η0 . Observe that η(t) = ke−t for some k. We can now apply the Maximum Principle.

413

Chapter 11. Extensions of the Maximum Principle

If (x∗ (·), u∗ (·)) is an optimal pair for the LQ optimal control problem, then the Maximum Principle implies that there is a nontrivial solution   ∗ η0 (t) ∗ ∈ R2 ηˆ (t) = η ∗ (t)

to the augmented adjoint equation (11.100) such that η0∗ (t) ≡ η0∗ ≤ 0, d ∗ η (t) = −η ∗ (t) (11.101) dt and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max1 H( ˆ∗ (t), u) ≡ c. u∈R

ˆ ηˆ∗ (t), x Since Ω = R1 is open and H( ˆ∗ (t), u) is maximized on the 1 ˆ ηˆ∗ (t), x open set Ω = R , then the derivative of H( ˆ∗ (t), u) with respect to u at u = u∗ (t) must be zero. In particular, ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u∗ (t)) = 0, ˆ∗ (t), u)|u=u∗ (t) = Du H( where ∂ ∗1 2 ˆ ηˆ∗ (t), x [η0 u + η ∗ (t)(x∗ (t) + u)] Du H( ˆ∗ (t), u) = ∂u 2 = [η0∗ u + η ∗ (t)], so that when u = u∗ (t) ˆ ηˆ∗ (t), x Du H( ˆ∗ (t), u)|u=u∗ (t) = [η0∗ u∗ (t) + η ∗ (t)] = 0. Thus, η0∗

η0∗ u∗ (t) = −η ∗ (t). η0∗ u∗ (t)

(11.102) ∗

Note that if = 0, then 0 = = −η (t) which would imply ∗ that η (t) ≡ 0. However, this in turn implies that     ∗ 0 η0 (t) ∗ ∈ R2 ≡ ηˆ (t) = ∗ 0 η (t)

which would contradict the fact that ηˆ∗ (t) is a nontrivial solution. Hence, η0∗ < 0 and we can solve (11.102) for u∗ (t) yielding u∗ (t) =

η ∗ (t) , λ∗ (t), −η0∗

414

Chapter 11. Extensions of the Maximum Principle

where we normalize the adjoint variable by defining λ∗ (t) =

η ∗ (t) . −η0∗

Note that by (11.101) η˙ ∗ (t) η ∗ (t) λ˙ ∗ (t) = = = −λ∗ (t), −η0∗ η0∗ so that λ∗ (t) = Ae−t for some constant A and x˙ ∗ (t) = x∗ (t) + u∗ (t) = x∗ (t) + Ae−t .

(11.103)

Solving (11.103) and applying the boundary conditions at x (0) = 1/2 and x∗ (1) = 0 yields that ∗

x∗ (t) =

1 2(e−1

− e)

[et−1 − e1−t ]

and

1 , 1 − e−2 so the optimal control is given by A=

u∗ (t) = [

1 ]e−t . 1 − e−2

The second example is Linear Quadratic (LQ) optimal control problem with hard constraint on the control. In this case, the Maximum Principle does not reduce to taking the derivative of the augmented Hamiltonian and setting the resulting equation to zero. One must compute a solution to a constrained optimization problem.

Chapter 11. Extensions of the Maximum Principle

415

LQ Optimal Control Problem: Example 2 In this problem we allow the final state to be free and place a bound on the control. Thus, we consider the linear control system x(t) ˙ = x(t) + u(t)

(11.104)

x(0) = 1 ∈ R1 .

(11.105)

|u(t)| ≤ 1

(11.106)

with initial data We assume that t0 = 0, x0 = 1 ∈ R1 and 0 < t1 = 1 are given. The control is required to be bounded

and the quadratic cost functional is defined by J(u(·)) =

Z1 0

1 [x(s; u(·))]2 ds = 2

Z1

1 [x(s)]2 ds, 2

(11.107)

0

where x(·) = x(·; u(·)) is the solution to initial value problem (11.104) - (11.105). The Linear Quadratic (LQ) optimal control problem is to find u∗ (·) ∈ P W C(0, 1) satisfying (11.106) so that ∗

J(u (·)) =

Z1 0

1 ∗ [x (s; u(·))]2 ds ≤ J(u(·)) = 2

Z1

1 [x(s; u(·))]2 ds 2

0

for all u(·) ∈ P W C(0, 1) that steer x0 = 1/2 to x1 ∈ R1 .

Formulation as an Optimal Control Problem: In order to set up the fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ R1 , the terminal set X1 ⊆ R1 ; 3. Identify the control constraint set Ω ⊆ R1 ; 4. Identify the functions f : R1 × R1 −→ R1 , f0 : R1 × R1 −→ R1 ;

416

Chapter 11. Extensions of the Maximum Principle

ˆ η, 5. Define the augmented Hamiltonian H( ˆ x ˆ, u); ˆ 6. Form the augmented adjoint system matrix A(t). The initial time is t0 = 0 and the final time t1 = 1 is given and fixed. The initial set is the single initial vector X0 = {1} and since there is no terminal constraint, the terminal set is the whole state space X1 = R1 . The constraint on the control u(·) ∈ P W C(0, 1) is |u(t)| ≤ 1 which implies that Ω = [−1, 1]. The functions f : R1 × R1 −→ R1 and f0 : R1 × R1 −→ R1 are given by f (x, u) = x + u, and

1 f0 (x, u) = x2 ≥ 0, 2

respectively. ˆ : R2 × R2 × R1 −→ R1 defined The augmented Hamiltonian H by 1 ˆ η, H( ˆ x ˆ, u) = η0 x2 + η(x + u) (11.108) 2  T  T where x ˆ = x0 x ∈ R2 , ηˆ = η0 η ∈ R2 and u ∈ R1 . The augmented function fˆ : R2 × R1 −→ R2 is defined by  1 2  x 2 ˆ , (11.109) f (ˆ x, u) = x+u so that the Jacobian is given by Jxˆ fˆ(ˆ x, u) = Jxˆ fˆ(x, u) =



0 x 0 1



.

(11.110)

Now assume that (x∗ (·), u∗ (·)) is an optimal pair so that the ˆ is given by matrix A(t) ˆ = Jxˆ fˆ(ˆ A(t) x, u)|(x∗ (t),u∗ (t)) = Jxˆ fˆ(x∗ (t), u∗ (t)) =



0 x∗ (t) 0 1



(11.111)

417

Chapter 11. Extensions of the Maximum Principle and ˆ T =− −[A(t)]



0 0 0 1



=



0 0 ∗ −x (t) −1



.

The augmented adjoint equation has the form        d η0 (t) η0 (t) 0 0 η0 (t) T ˆ = = −[A(t)] η(t) −x∗ (t) −1 η(t) dt η(t) (11.112) which is equivalent to the system d η0 (t) = 0 dt d η(t) = −η0 (t)x∗ (t) − η(t), dt where again x∗ (t) is the optimal trajectory. The first equation above implies (as always) that the zero adjoint state is a constant η0 (t) ≡ η0. Observe that d η(t) = −η(t) − η0 x∗ (t) dt

(11.113)

is a linear nonhomogeneous equation. We can now apply the Maximum Principle. If (x∗ (·), u∗ (·)) is an optimal pair for the LQ optimal control problem, then the Maximum Principle implies that there is a nontrivial solution   ∗ η0 (t) ∗ ∈ R2 ηˆ (t) = η ∗ (t)

to the augmented adjoint equation (11.112) such that η0∗ (t) ≡ η0∗ ≤ 0, d ∗ η (t) = −η ∗ (t) − η0∗ x∗ (t) (11.114) dt and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = max H( ˆ∗ (t), u) ≡ c. u∈[−1,1]

In particular, 1 ˆ ηˆ∗ (t), x H( ˆ∗ (t), u) = η0∗ x2 + η ∗ (t)(x∗ (t) + u), 2

418

Chapter 11. Extensions of the Maximum Principle

so that 1 ˆ ηˆ∗ (t), x ˆ∗ (t), u) = max {η0∗ x2 + η ∗ (t)(x∗ (t) + u)} max H( u∈[−1,1] u∈[−1,1] 2 ∗ = max {η (t)u)} u∈[−1,1]

1 + η0∗ x2 + η ∗ (t)x∗ (t) ≡ c. 2 Thus, u = u∗ (t) must be selected to maximize {η ∗ (t)u)} on the interval −1 ≤ u ≤ 1. Clearly, this implies that u∗ (t) = sgn[η ∗ (t)].

(11.115)

Applying the transversality condition at t1 = 1, we find that T1 = R1 so η ∗ (t1 )⊥T1 implies that η ∗ (t1 ) = η ∗ (1) = 0.

(11.116)

Returning to the adjoint equation (11.113) we see that η ∗ (t) satisfies the terminal value problem d ∗ η (t) = −η ∗ (t) − η0∗ x∗ (t), dt

η ∗ (1) = 0.

(11.117)

One can see that η0∗ 6= 0 since η0∗ = 0 would imply that η ∗ (t) solves the linear equation problem d ∗ η (t) = −η ∗ (t), dt

η ∗ (1) = 0,

and hence η ∗ (t) ≡ 0. Thus, if η0∗ = 0, then η ∗ (t) ≡ 0 which contra T dicts the statement that ηˆ∗ (t) = η0∗ η ∗ (t) is nonzero. ∗ Hence, η0 < 0 and we can simplify the adjoint equation (11.117). Divide both sides of (11.117) by the positive number −η0∗ to obtain   ∗   ∗   d η ∗ (t) η (t) η (1) ∗ =− + x (t), = 0. ∗ ∗ dt −η0 −η0 −η0∗

Chapter 11. Extensions of the Maximum Principle

419

Let λ∗ (t) be defined by 

η ∗ (t) λ (t) , −η0∗ ∗



and note that since −η0∗ > 0, λ∗ (t) and η ∗ (t) have the same sign. Also, λ˙ ∗ (t) = −λ∗ (t) + x∗ (t), λ∗ (1) = 0 (11.118) and u∗ (t) = sgn[η ∗ (t)] = sgn[λ∗ (t)].

(11.119)

Combining (11.118) - (11.119) with the state equation, we find that the system x˙ ∗ (t) = x∗ (t) + sgn[λ∗ (t)], λ˙ ∗ (t) = −λ∗ (t) + x∗ (t),

x∗ (0) = 1,

(11.120)

λ∗ (1) = 0,

needs to be solved in order to compute u∗ (t) = sgn(η ∗ (t)). This is a nonlinear two-point boundary value problem that must be solved numerically. At this point it is helpful to review numerical methods for solving two point boundary value problems. The following references [11], [25], [103], [104], [112], [162], [169] and [185] contain some useful results on this topic.

11.5.2

The Time Independent Riccati Differential Equation

We return to the general LQ optimal control problem and focus on the optimality system defined by the two point boundary value problem (11.89) - (11.91). We show that one can transform the state variable to the adjoint variable by a matrix that satisfies a Riccati differential equation. This transformation is a key step in connecting the theory of Riccati equations with the optimality conditions for linear quadratic optimal control. Also, the resulting Riccati equation provides one method for developing numerical methods for solving LQ optimal control problems.

420

Chapter 11. Extensions of the Maximum Principle

First write (11.89) as the linear system        d x(t) A BR−1 B T x(t) x(t) (11.121) = ,F λ(t) Q −AT λ(t) dt λ(t) where λ(t1 ) = 0 and F =



A BR−1 B T Q −AT



.

The solution to (11.121) has the form       x(t1 ) x(t1 ) x(t) F (t−t1 ) F (t−t1 ) . =e =e 0 λ(t1 ) λ(t) Let Ψ(t) = e

Ft

=



ψ11 (t) ψ12 (t) ψ21 (t) ψ22 (t)



,

where ψij (t), i, j = 1, 2 are n×n square matrix functions. It follows that        ψ11 (t − t1 ) ψ12 (t − t1 ) x(t1 ) x(t1 ) x(t) F (t−t1 ) = =e ψ21 (t − t1 ) ψ22 (t − t1 ) 0 λ(t) 0 so that x(t) = ψ11 (t − t1 )x(t1 ),

(11.122)

λ(t) = ψ21 (t − t1 )x(t1 ).

(11.123)

and If ψ11 (t − t1 ) is non-singular for 0 ≤ t ≤ t1 , then we can solve (11.122) for x(t1 ). In particular, x(t1 ) = [ψ11 (t − t1 )]−1 x(t) which, when substituted into (11.123), yields λ(t) = [ψ21 (t − t1 )(ψ11 (t − t1 ))−1 ]x(t). If P (t) is the n × n matrix defined by P (t) , −[ψ21 (t − t1 )(ψ11 (t − t1 ))−1 ],

(11.124)

Chapter 11. Extensions of the Maximum Principle

421

then we have that λ(t) and x(t) are linearly related by the matrix P (t) and the relationship is given by λ(t) = −P (t)x(t). The choice of the negative sign in defining P (·) is made to be consistent with much of the existing literature. In order to make this step rigorous, one needs to prove that ψ11 (t−t1 ) is non-singular for 0 ≤ t ≤ t1 . On the other hand, we could simply ask the question: Is there a matrix P (t) so that λ(t) = −P (t)x(t) and how can P (t) be computed? We will address the issue of the existence of P (t) later. However, assume for the moment that x(·) and λ(t) satisfying (11.89) - (11.91) and λ(t) = −P (t)x(t), (11.125)

with P (t) differentiable. Differentiating the equation (11.125) one obtains     d d d λ(t) = − P (t) x(t) − P (t) x(t) dt dt dt     d =− P (t) x(t) − P (t) Ax(t) + BR−1 B T λ(t) dt     d =− P (t) x(t) − P (t) Ax(t) − BR−1 B T P (t)x(t) dt   d P (t) x(t) − P (t)Ax(t) + P (t)BR−1 B T P (t)x(t). =− dt

However, from (11.121) it follows that d λ(t) = Qx(t) − AT λ(t) dt = Qx(t) + AT P (t)x(t) so that

 d P (t) x(t) − P (t)Ax(t) Qx(t) + A P (t)x(t) = − dt + P (t)BR−1 B T P (t)x(t). T



422

Chapter 11. Extensions of the Maximum Principle

Rearranging the terms we have   d P (t) x(t) = AT P (t)x(t) + P (t)Ax(t) − dt − P (t)BR−1 B T P (t)x(t) + Qx(t), or equivalently   d − P (t) x(t) dt   = AT P (t) + P (t)A − P (t)BR−1 B T P (t) + Q x(t). (11.126) Consequently, P (t) satisfies (11.126) along the trajectory x(t). Observe that (11.126) is satisfied for any solution of the system (11.121) with λ(t1 ) = 0 and all values of x(t1 ). Therefore, if λ(t) = −P (t)x(t), then P (t) satisfies the matrix Riccati differential equation − P˙ (t) = AT P (t) + P (t)A − P (t)BR−1 B T P (t) + Q, 0 ≤ t < t1, (11.127) with terminal condition P (t1 ) = 0n×n ,

(11.128)

since −P (t1 )x(t1 ) = λ(t1 ) = 0

and x(t1 ) can be any vector in Rn . We shall show below that under the assumption that there is a solution P (t) to the Riccati differential equation (11.127) satisfying (11.128), then the LQ optimal control problem has a solution and the optimal control is given by u∗ (t) = −R−1 B T P (t)x∗ (t).

(11.129)

In order to provide a rigorous treatment of this problem, we present two lemmas. These results relate the existence of a solution to the Riccati equation (11.127) to the existence of an optimal

Chapter 11. Extensions of the Maximum Principle

423

control for the LQ optimal control problem. First we note that any solution to the Riccati differential equation must be symmetric. In particular, P (t) = [P (t)]T for all t. Lemma 11.1 Suppose that P (t) = [P (t)]T is any n × n matrix function with P (t) differentiable on the interval [t0 , t1 ]. If u(·) ∈ P W C(t0 , t1 ; Rm ) and x(t) ˙ = Ax(t) + Bu(t), t0 ≤ t ≤ t1 , then hP (s)x(s), x(s)i |tt10 =

Z t1Dh

i E T ˙ P (s)+P (s)A+A P (s) x(s),x(s) ds t0 Z t1 + hP (s)Bu(s), x(s)i ds (11.130) t0 Z t1

T  B P (s)x(s), u(s) ds. + t0

Proof: Observe that

hP (s)x(s), x(s)i |tt10

Z

t1

d hP (s)x(s), x(s)i ds t0 ds Z t1 D E P˙ (s)x(s), x(s) ds = t0 Z t1 + hP (s)x(s), ˙ x(s)i ds t0 Z t1 + hP (s)x(s), x(s)i ˙ ds

=

t0

and by substituting Ax(s) + Bu(s) for x(s) ˙ we obtain Z t1 D E t1 ˙ P (s)x(s), x(s) ds hP (s)x(s), x(s)i |t0 = t0 Z t1 + hP (s) [Ax(s) + Bu(s)] , x(s)i ds t0 Z t1 + hP (s)x(s), [Ax(s) + Bu(s)]i ds. t0

424

Chapter 11. Extensions of the Maximum Principle

Simplifying this expression we obtain Z t1 D E t1 ˙ P (s)x(s), x(s) ds hP (s)x(s), x(s)i |t0 = t0 Z t1 hP (s) [Ax(s)] , x(s)i ds + t0 Z t1 + hP (s) [Bu(s)] , x(s)i ds t0 Z t1 + hP (s)x(s), [Ax(s)]i ds t0 Z t1 + hP (s)x(s), [Bu(s)]i ds t0

and collecting terms yields Z t1 D E t1 [P˙ (s) + P (s)A + AT P (s)]x(s), x(s) ds hP (s)x(s), x(s)i |t0 = t0 Z t1 + hP (s)Bu(s), x(s)i ds t0 Z t1

T  + B P (s)x(s), u(s) ds. t0

which establishes (11.130). 

Lemma 11.2 Assume that the Riccati differential equation (11.127) has a solution P (t) = [P (t)]T for t0 ≤ t < t1 and P (t1 ) = 0n×n . If u(·) ∈ P W C(t0 , t1 ; Rm ) and x(t) ˙ = Ax(t) + Bu(t), t0 ≤ t ≤ t1 , then the cost function J(·) has the representation Z t1

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds J(u(·)) = t0

+ hP (t0 )x(t0 ), x(t0 )i .

Chapter 11. Extensions of the Maximum Principle

425

Proof: Let N (x(·), u(·)) =

Z

t1 t0

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds

and expanding N (x(·), u(·)) we obtain Z t1

1/2 N (x(·), u(·)) = R u(s) + R−1/2 B T P (s)x(s), R1/2 u(s) t0  + R−1/2 B T P (s)x(s) ds Z t1

1/2  R u(s), R1/2 u(s) ds = t0 Z t1

1/2  + R u(s), R−1/2 B T P (s)x(s) ds t Z 0t1

−1/2 T  + R B P (s)x(s), R1/2 u(s) ds t Z 0t1

−1/2 T  + R B P (s)x(s), R−1/2 B T P (s)x(s) ds. t0

Simplifying each term we have Z t1

1/2 1/2  R R u(s), u(s) ds N (x(·), u(·)) = t0 Z t1

 u(s), R1/2 R−1/2 B T P (s)x(s) ds + t Z 0t1

1/2 −1/2 T  R R B P (s)x(s), u(s) ds + t Z 0t1

−1/2 −1/2 T  + R R B P (s)x(s), B T P (s)x(s) ds, t0

426

Chapter 11. Extensions of the Maximum Principle

which implies N (x(·), u(·)) =

Z

t1

hRu(s), u(s)i ds Z t1

 u(s), B T P (s)x(s) ds + t Z 0t1

T  + B P (s)x(s), u(s) ds t Z 0t1

−1 T  + R B P (s)x(s), B T P (s)x(s) ds, t0

t0

or equivalently N (x(·), u(·)) =

Z

t1

hRu(s), u(s)i Z t1

 u(s), B T P (s)x(s) ds (11.131) + t0 Z t1

T  + B P (s)x(s), u(s) ds t Z 0t1

 + P (s)BR−1 B T P (s)x(s), x(s) ds. t0

t0

Since the matrix P (s) satisfies the Riccati equation (11.127), it follows that i h T −1 T ˙ P (s)BR B P (s)x(s) = P (s) + A P (s) + P (s)A + Q x(s) and the last term above becomes Z t1 Dh i E T ˙ P (s) + A P (s) + P (s)A + Q x(s), x(s) ds. t0

427

Chapter 11. Extensions of the Maximum Principle

Substituting this expression into (11.131) and rearranging yields Z t1 Z t1 hQx(s), x(s)i ds hRu(s), u(s)i ds + N (x(·), u(·)) = t0 t0 Z t1 Dh i E + P˙ (s) + AT P (s) + P (s)A x(s), x(s) ds t Z 0t1

 u(s), B T P (s)x(s) ds + t Z 0t1

T  B P (s)x(s), u(s) ds, + t0

which implies

N (x(·), u(·)) = J(u(·)) Z t1 Dh i E P˙ (s) + AT P (s) + P (s)A x(s), x(s) ds + t Z 0t1

 + u(s), B T P (s)x(s) ds t Z 0t1

T  + B P (s)x(s), u(s) ds. t0

Applying (11.130) from the previous Lemma yields

N (x(·), u(·)) = J(u(·)) + hP (s)x(s), x(s)i |tt10 , or equivalently J(u(·)) = N (x(·), u(·)) − hP (s)x(s), x(s)i |tt10 . However, since P (t1 ) = 0n×n we conclude that Z t1

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds J(u(·)) = t0

+ hP (t0 )x(t0 ), x(t0 )i ,

(11.132)

which completes the proof.  We now have the fundamental result on the relationship between solutions to the Riccati equation and the existence of an optimal control for the LQ optimal problem.

428

Chapter 11. Extensions of the Maximum Principle

Theorem 11.2 (Existence of LQ Optimal Control) If the Riccati differential equation (11.127) has a solution P (t) = [P (t)]T for 0 ≤ t < t1 and P (t1 ) = 0n×n , then there is a control u∗ (·) ∈ P W C(0, t1 ; Rm ) such that u∗ (·) minimizes Z t1 J(u(·)) = {hQx(s), x(s)i + hRu(s), u(s)i} ds 0

on the set P W C(0, t1 ; Rm ), where the state equation is given by x(t) ˙ = Ax(t) + Bu(t)

(11.133)

x(0) = x0 ∈ Rn .

(11.134)

with initial data In addition, the optimal control is a linear feedback law u∗ (t) = −R−1 B T P (t)x∗ (t)

(11.135)

and the minimum value of J(u(·)) is J(u∗ (·)) = hP (0)x0 , x0 i .

(11.136)

Proof : Let t0 = 0 and apply the identity (11.132) above. In particular, it follows that J(·) is minimized when the quadratic term Z t1

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds ≥ 0 0

is minimized. If u∗ (t) = −R−1 B T P (t)x∗ (t), then

R1/2 u∗ (t) + R−1/2 B T P (t)x∗ (t) = 0 and ∗

J(u (·)) =

Z

t1 0

1/2 ∗

R u (s) + R−1/2 B T P (s)x∗ (s) 2 ds

+ hP (0)x∗ (0), x∗ (0)i = hP (0)x0 , x0 i .

429

Chapter 11. Extensions of the Maximum Principle Consequently, for any u(·) ∈ P W C(0, t1 ; Rm ) it follows that J(u∗ (·)) = hP (0)x0 , x0 i ≤ hP (0)x0 , x0 i + = J(u(·)),

Z

t1 0

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds

which completes the proof.  Later we will return to linear quadratic control problems and consider more general time dependent problems. Also, we will generalize the cost function to include a terminal penalty such as Z 1 t1 J(u(·)) = hGx(t1 ), x(t1 )i + {hQx(s), x(s)i 2 t0 + hRu(s), u(s)i} ds, (11.137) where G = GT ≥ 0 is a symmetric non-negative matrix.

11.6

The Maximum Principle for a Problem of Bolza

Consider the case where there is an additional explicit cost on the terminal state J(u(·)) = G(x(t1 )) +

Zt1

f0 (x(s), u(s))ds

(11.138)

t0

where G : Rn → R is a C 2 function. In order to keep the discussion simple, we begin with the simple case where X0 = {x0 } ⊆ Rn is a single vector and t1 is fixed. We show that by augmenting the problem we can construct an equivalent problem without the terminal cost defined by G. Also note that the final target set is taken to be X1 = R n .

430

Chapter 11. Extensions of the Maximum Principle

The state equation is augmented by adding a new variable at the end of the vector. In particular, let x ˜ = [x1 x2 ...xn xn+1 ]T = [x xn+1 ]T ∈ Rn+1 and define fn+1 : Rn+1 × Rm −→ R1 by fn+1 (˜ x, u) = 0. By adding the equation x˙ n+1 (t) = fn+1 (˜ x(t), u(t) ) = 0 to the system x(t) ˙ = f (x(t), u(t)),

t0 < t ≤ t 1 ,

we obtain the new state equation   d f (x(t), u(t)) , x ˜(t) = f˜(˜ x(t), u(t)) = 0 dt

t0 < t ≤ t 1 . (11.139)

Consider the new cost functional ˜ J(u(·)) =

Zt1

{xn+1 (s) + f0 (x(s), u(s))}ds =

Zt1

f˜0 (˜ x(s), u(s))ds

t0

t0

where

f˜0 (˜ x, u) , xn+1 + f0 (x, u).

Observe that since xn+1 (s) is a constant (recall that x˙ n+1 (s) = 0), then ˜ J(u(·)) =

Zt1

{xn+1 (s) + f0 (x(s), u(s))}ds

t0

= xn+1 (t1 )(t1 − t0 ) +

Zt1

t0

f0 (x(s), u(s))ds.

431

Chapter 11. Extensions of the Maximum Principle Hence, if we require that xn+1 (t1 ) =

G(x(t1 )) , t1 −t0

˜ J(u(·)) = xn+1 (t1 )(t1 − t0 ) +

Zt1

it follows that

f0 (x(s), u(s))ds

t0

= G(x(t1 )) +

Zt1

f0 (x(s), u(s))ds.

t0

Thus, we can reformulate the problem as the equivalent optimal control problem in Rn+1 . Let  ˜ 0 = {x0 } × R1 = x X ˜ = [x0 y]T : y ∈ R1 ⊆ Rn+1 (11.140)

and

n o ˜1 = x ˜ x) , xn+1 − G(x)/(t1 − t0 ) = 0 X ˜ = [x xn+1 ]T : G(˜ ⊆ Rn+1 .

(11.141)

Therefore, minimizing the cost functional (11.138) among all controls that steer X0 = {x0 } to X1 = Rn is equivalent to minimizing the cost functional Zt1 Zt1 ˜ J(u(·)) = f˜0 (x(s), u(s))ds = {xn+1 (s) + f0 (x(s), u(s))}ds t0

t0

(11.142) 1 ˜ among all controls that steer X0 = {x0 } × R to n o ˜1 = x ˜ x) , xn+1 − G(x)/(t1 − t0 ) = 0 X ˜ = [x xn+1 ]T : G(˜

with the state equation defined by (11.139) above. We now apply the Maximum Principle to this equivalent problem. ˜ 0 = {x0 }×R1 is a subspace of Rn+1 with tangent Observe that X ˜ 0 = {0} × R1 so that a vector η˜ = [η, ηn+1 ]T ∈ Rn+1 is space T ˜ 0 = {0} × R1 if and only if ηn+1 = 0. Since the orthogonal to T target set n o T ˜ ˜ X1 = x ˜ = [x xn+1 ] : G(˜ x) , xn+1 −G(x)/(t1 −t0 ) = 0 ⊆ Rn+1

432

Chapter 11. Extensions of the Maximum Principle

˜ : Rn+1 → R1 , then a is defined by the level set of the function G ˜ 1 at x vector η˜ = [η, ηn+1 ]T ∈ Rn+1 is orthogonal to T ˜(t1 ) if     η −∇G(x(t ))/(t − t ) 1 1 0 ˜ x(t1 )) = α η˜ = = α∇G(˜ ηn+1 1 (11.143) for some nonzero α. The augmented state and co-state are given by b x ˜ = [x0 x xn+1 ]T = [x0 x ˜]T ∈ Rn+2

and

b˜ = [η0 η ηn+1 ]T = [η0 η] η ˜ T ∈ Rn+2 ,

respectively. The augmented state equation is given by   ˜0 (˜ f x (t), u(t)) db b x ˜(t) = fb˜(x ˜(t), u(t)) =  f (˜ x(t), u(t))  dt fn+1 (˜ x(t), u(t))   f˜0 (˜ x(t), u(t))  = f (x(t), u(t))  , t0 < t ≤ t1 , 0 so that the augmented Hamiltonian has the form D E b˜ η b˜∗ (t), x b H( ˜∗ (t), u) , η0 f˜0 (˜ x, u) + η, ˜ f˜(˜ x, u)

(11.144)

= η0 [xn+1 + f0 (x, u)] + hη,f (x, u)i + ηn+1 · 0 = η0 [xn+1 + f0 (x, u)] + hη,f (x, u)i .

To construct the corresponding adjoint system we construct the Jacobian 

      ˜Jx˜ (˜ x, u) =      

∂ f˜0 (˜ x,u) ∂x0 ∂ f˜1 (˜ x,u) ∂x0 ∂ f˜2 (˜ x,u) ∂x0

.. .

∂ f˜0 (˜ x,u) ∂x1 ∂ f˜1 (˜ x,u) ∂x1 ∂ f˜2 (˜ x,u) ∂x1

.. .

∂ f˜0 (˜ x,u) ∂x2 ∂ f˜1 (˜ x,u) ∂x2 ∂ f˜2 (˜ x,u) ∂x2

.. .

∂ f˜n (˜ x,u) ∂ f˜n (˜ x,u) ∂ f˜n (˜ x,u) ∂x0 ∂x1 ∂x2 x,u) ∂ f˜n+1 (˜ x,u) ∂ f˜n+1 (˜ x,u) ∂ f˜n+1 (˜ ∂x0 ∂x1 ∂x2

··· ···

··· .. . ··· ···

∂ f˜0 (˜ x,u) ∂xn ∂ f˜1 (˜ x,u) ∂xn ∂ f˜2 (˜ x,u) ∂xn

.. .

∂ f˜0 (˜ x,u) ∂xn+1 ∂ f˜1 (˜ x,u) ∂xn+1 ∂ f˜2 (˜ x,u) ∂xn+1

.. .

∂ f˜n (˜ x,u) ∂ f˜n (˜ x,u) ∂xn ∂xn+1 ∂ f˜n+1 (˜ x,u) ∂ f˜n+1 (˜ x,u) ∂xn ∂xn+1

            

433

Chapter 11. Extensions of the Maximum Principle which becomes       J˜x˜ (˜ x, u) =     

0 0 0 .. . 0 0

∂f0 (˜ x,u) ∂x1 ∂ f˜1 (˜ x,u) ∂x1 ∂ f˜2 (˜ x,u) ∂x1

∂f0 (˜ x,u) ∂x2 ∂ f˜1 (˜ x,u) ∂x2 ∂ f˜2 (˜ x,u) ∂x2

∂ f˜n (˜ x,u) ∂x1

∂ f˜n (˜ x,u) ∂x2

0

0

.. .

.. .

··· ···

··· ... ··· ···

∂f0 (˜ x,u) ∂xn ∂ f˜1 (˜ x,u) ∂xn ∂ f˜2 (˜ x,u) ∂xn

∂ f˜1 (˜ x,u) ∂xn+1 ∂ f˜2 (˜ x,u) ∂xn+1

∂ f˜n (˜ x,u) ∂xn

∂ f˜n (˜ x,u) ∂xn+1

0

0

.. .

or equivalently,

1

.. .



     ,    



 0 [∇f0 (x, u)]T 1 J˜x˜ (˜ x, u) =  0 Jx f (x, u) 0  , 0 0 0

where Jx f (x, u) is the Jacobian of f (x, u). Consequently, if (˜ x∗ (t), u∗ (t)) is an optimal pair, then     0 [∇f0 (x∗ (t), u∗ (t))]T 1 ˆ A(t) 1 b ∗ ∗ ˜ =  0 Jx f (x (t), u (t)) 0  = A(t) 0 0 0 0 0

and the augmented co-state equation becomes db b˜ T d η(t), b˜ η(t) ˜ = −[A(t)] dt dt

which has the form   η0 (t) d  η(t)  dt ηn+1 (t)    η0 (t) 0 0 0 = −∇f0 (x∗ (t), u∗ (t)) −[Jx f (x∗ (t), u∗ (t))]T 0  η(t)  . ηn+1 (t) −1 0 0 (11.145) Observe that the solutions of (11.145) satisfy η0 (t) ≡ η0 and ηn+1 (t) = −η0 t + k

(11.146)

434

Chapter 11. Extensions of the Maximum Principle

for some constant k. In addition, η(t) satisfies the equation η(t) ˙ = −η0 ∇f0 (x∗ (t), u∗ (t)) − [Jx f (x∗ (t), u∗ (t))]T η(t) (11.147) which is the same as the adjoint equation for the Simplest Problem in Optimal Control. With the observations above it is now straightforward to establish the following Maximum Principle for a Problem of Bolza. Theorem 11.3 (Maximum Principle for the Problem of Bolza) Assume that f : Rn × Rm → Rn , f0 : Rn × Rm → R, G : R → R, X0 = {x0 } ⊆ Rn , t0 ∈ R, t1 is fixed, X1 = Rn and Ω ⊆ Rm are given as above. Consider the control system (S)

x(t) ˙ = f (x(t), u(t)),

t0 < t ≤ t 1 ,

(11.148)

with piecewise continuous controllers u(·) ∈ P W C(t0 , t1 ; Rm ) satisfying u(t) ∈ Ω ⊆ Rm e.f. If   u(·) ∈ P W C(t0 , t1 ; Rm ) : u(t) ∈ Ω e.f. ∗ u (·) ∈ Θ = and u(·) steers X0 to X1 at time t1 . (11.149) minimizes J(u(·)) = G(x(t1 )) +

Zt1

f0 (x(s), u(s))ds,

(11.150)

t0

on the set of admissible controls Θ with optimal response x∗ (·) satisfying x∗ (t0 ) = x∗0 , then there exists a non-trivial solution   ∗ η0 (t) T ∗ ∗ ∗ ∗ ∗ (11.151) ηˆ (t) = [η0 (t) η1 (t) η2 (t) ... ηn (t)] = η ∗ (t) to the augmented adjoint equation d ˆ T η(t), η(t) ˆ = −[A(t)] ˆ dt

(11.152)

such that ˆ ηˆ∗ (t), x ˆ (ηˆ∗ (t), x ˆ ηˆ∗ (t), x H( ˆ∗ (t), u∗ (t)) = M ˆ∗ (t)) = max H( ˆ∗ (t), u). u∈Ω

(11.153)

435

Chapter 11. Extensions of the Maximum Principle Moreover, there are constants η0∗ ≤ 0 and c such that η0∗ (t) ≡ η0∗ ≤ 0

(11.154)

and for all t ∈ [t0 , t1 ], ˆ ηˆ∗ (t), x ˆ (ηˆ∗ (t), x M ˆ∗ (t)) = max H( ˆ∗ (t), u) ≡ c. u∈Ω

(11.155)

Also, the transversality condition at t = t1 is given by η ∗ (t1 ) = η0∗ ∇G(x∗ (t1 )).

(11.156)

Proof. To prove this result, we need only to establish the final transversality condition (11.156) above. Since the tangent space ˜ 0 = {0}×R1 , it follows from the Maximum Principle that η˜∗ (t) = T ∗ ˜ 0 = {0} × R1 and this [η ∗ (t0 ) ηn+1 (t0 )]T ∈ Rn+1 is orthogonal to T ∗ ∗ (t) = −η0∗ t+k is true if and only if ηn+1 (t0 ) = 0. From (11.146) ηn+1 ∗ and ηn+1 (t0 ) = 0 implies that ∗ ηn+1 (t) = −η0∗ (t − t0 ).

Also, the condition (11.143) implies that     ∗ η ∗ (t1 ) η (t1 ) ∗ = η˜ (t1 ) = ∗ −η0∗ (t1 − t0 ) ηn+1 (t1 )   ∗ −α∇G(x (t ))/(t − t ) 1 1 0 ˜ x(t1 )) = = α∇G(˜ α and hence −η0∗ (t1 − t0 ) = α. Therefore, η ∗ (t1 ) = −α∇G(x∗ (t1 ))/(t1 − t0 ) = η0∗ (t1 − t0 )∇G(x∗ (t1 ))/(t1 − t0 ) = η0∗ ∇G(x∗ (t1 )) and this completes the proof. 

436

Chapter 11. Extensions of the Maximum Principle

11.7

The Maximum Principle for Nonautonomous Systems

When the system is nonautonomous the Maximum Principle must be modified. However, we show that the general nonautonomous problem can be reduced to the SPOC and the Maximum Principle for the SPOC can be used to obtain the corresponding Maximum Principle. Thus, we assume that f : R × Rn × Rm → Rn , f0 : R × Rn × Rm → R, X0 ⊆ Rn , t0 ∈ R, X1 ⊆ Rn and Ω ⊆ Rm are given with X0 ⊆ Rn , X1 ⊆ Rn and Ω ⊆ Rm nonempty. Here we assume that t1 ∈ R is free as in the SPOC. The control system is given by (S)

x(t) ˙ = f (t, x(t), u(t)),

t0 < t ≤ t 1

and the cost functional is defined by Z t1 f0 (s, x(s), u(s))ds. J(u(·)) =

(11.157)

(11.158)

t0

The admissible controllers are given by   u(·) ∈ P W C(t0 , +∞; Rm ) : u(t) ∈ Ω e.f. and Θ= u(·) steers X0 to X1 at a finite time t1 . (11.159) and the problem is to find u∗ (·) ∈ Θ such that Z t1 ∗ J(u (·)) = f0 (s, x∗ (s), u∗ (s))ds t Z 0t1 ≤ J(u(·)) = f0 (s, x(s), u(s))ds t0

for all u(·) ∈ Θ. In order to state the Maximum Principle for this nonautonomous case we again have to introduce some notation. Given a vector z = [z1 , z2 , ..., zn ]T ∈ Rn , define the n + 2 dimensional augmented vector z˜ ∈ Rn+2 by     z0 z ˆ T ∈ Rn+2 , z˜ = [z0 z1 z2 ... zn zn+1 ] =  z  = zn+1 zn+1

Chapter 11. Extensions of the Maximum Principle

437

where as before zˆ = [z0 z1 z2 ... zn ]T ∈ Rn+1 . Also, given the functions f : R1 ×Rn × Rm → Rn , f0 : R1 ×Rn × Rm → R,

where f : R1 ×Rn × Rm → Rn is    f1 (t, x, u)  f2 (t, x, u)         f (t, x, u) =  f3 (t, x, u)  =     ..    . fn (t, x, u)

given by f1 (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) f2 (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) f3 (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) .. . fn (t, x1 , x2 , ..., xn , u1 , u2, ..., um )

it follows that all the partial derivatives



   ,  

∂fi (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) , ∂xj ∂fi (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) ∂uj exist. Remark 11.4 For the moment we make a rather strong assumption that the partial derivatives ∂fi (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) ∂t also exist. This is made to simplify the derivation of the Maximum Principle for the nonautonomous problem and is not essential. Define the (time) augmented vector field f˜ : Rn+2 × Rm → Rn+2 by

 ˜    f0 (˜ x, u) f0 (xn+1 , x1 , x2 , ..., xn , u1 , u2, ..., um )  f˜1 (˜   x, u)     f1 (xn+1 , x1 , x2 , ..., xn , u1 , u2, ..., um )     f˜ (˜   2 x, u)   f2 (xn+1 , x1 , x2 , ..., xn , u1 , u2, ..., um )  ˜ f (˜ x, u) =  , . .. ..     . .      f˜n (˜ x, u)  fn (xn+1 , x1 , x2 , ..., xn , u1 , u2, ..., um ) 1 f˜n+1 (ˆ x, u) (11.160)

438

Chapter 11. Extensions of the Maximum Principle

The time augmented control system is defined by d x ˜(t) = f˜(˜ x(t), u(t)), dt

(11.161)

or equivalently by the system d x0 (t) = f0 (xn+1 (t), x(t), u(t)), dt d x(t) = f (xn+1 (t), x(t), u(t)), dt d xn+1 (t) = fn+1 (xn+1 (t), x(t), u(t)) = 1. dt

(11.162) (11.163) (11.164)

Observe that x0 (t) =

Z

t

f0 (xn+1 (s), x(s), u(s))ds, t0

and xn+1 (t0 ) = t0 so that the initial state for (11.161) is x ˜(t0 ) = [ 0 x0 t0 ]T .

(11.165)

In particular, x0 (t) represents the cost of transferring the state from an initial x(t0 ) = x0 ∈ X0 by the control u(·) and xn+1 (t) = t is the time variable. Assume that (x∗ (·), u∗ (·)) is an optimal pair. In particular, u∗ (·) ∈ Θ steers x∗0 ∈ X0 to x∗1 ∈ X1 in time t∗1 and x∗ (·) is the corresponding optimal trajectory that satisfies the initial value problem (10.1) - (10.2) with x0 = x∗0 ∈ X0 . Define the (n + 2) × ˜ by (n + 2) matrix A(t) # " ˜i (x0 , x, xn+1 , u) ∂ f ˜ = (11.166) A(t) ∂xj | ∗ ∗ " # (ˆx (t),u (t)) ∂ f˜i (˜ x∗ (t), u∗ (t)) = , i, j = 0, 1, 2, ..., n, n + 1. ∂xj

439

Chapter 11. Extensions of the Maximum Principle Thus, ˜ = A(t)

#

"

∂ f˜i (ˆ x∗ (t), u∗ (t)) ∂xj



∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂x0 ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂x0 ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂x0

∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂x1 ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂x1 ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂x1

. . .

. . .

       =      

∂ f˜n (ˆ x (t),u (t)) ∂x0 ˜ ∂ fn+1 (ˆ x∗ (t),u∗ (t)) ∂x0 ∗

∗

··· ··· ..

∂ f˜n (ˆ x (t),u (t)) ∂x1 ˜ ∂ fn+1 (ˆ x∗ (t),u∗ (t)) ∂x1 ∗

···

∗

∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂xn ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂xn ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂xn

. . .

.

··· ···

∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂xn+1 ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂xn+1 ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂xn+1

. . .

∂ f˜n (ˆ x (t),u (t)) ∂xn ˜ ∂ fn+1 (ˆ x∗ (t),u∗ (t)) ∂xn ∗

∗

∂ f˜n (ˆ x (t),u∗ (t)) ∂xn+1 ∂ f˜n+1 (ˆ x∗ (t),u∗ (t)) ∂xn+1 ∗



       .      

Since f˜i (˜ x, u) = f˜i (ˆ x, xn+1 , u) = fi (xn+1 , x, u) = fi (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) does not depend on x0 , it follows that ∂fi (˜ x∗ (t), u∗ (t)) =0 ∂x0 for all i = 0, 1, 2, ..., n, n + 1 and since f˜n+1 (˜ x, u) = 1 we have that ∂ f˜n+1 (˜ x∗ (t), u∗ (t)) =0 ∂xj for all i = 0, 1, 2, ..., n, n + 1.  ˜ ∗ (t),u∗ (t)) 0 ∂ f0 (ˆx ∂x 1  ∗ ∗  0 ∂ f˜1 (ˆx (t),u (t)) ∂x1   ∂ f˜2 (ˆ x∗ (t),u∗ (t)) 0  ˜ = ∂x1 A(t) ..  .. .  .  ∗ ∗  0 ∂ f˜n (ˆx (t),u (t)) ∂x1 0 0

Consequently, ··· ···

··· .. . ··· ···

∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂xn ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂xn ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂xn

∂ f˜0 (ˆ x∗ (t),u∗ (t)) ∂xn+1 ∂ f˜1 (ˆ x∗ (t),u∗ (t)) ∂xn+1 ∂ f˜2 (ˆ x∗ (t),u∗ (t)) ∂xn+1

∂ f˜n (ˆ x∗ (t),u∗ (t)) ∂xn

∂ f˜n (ˆ x∗ (t),u∗ (t)) ∂xn+1

0

0

.. .

and using the definition of f˜i (˜ x, u) ∗ ∗ fi (t, x (t), u (t)), it follows that

=

.. .

          

(11.167) fi (xn+1 , x, u) =

∂fi (t, x∗ (t), u∗ (t)) ∂ f˜i (˜ x∗ (t), u∗ (t)) = . ∂xn+1 ∂t

440

Chapter 11. Extensions of the Maximum Principle

˜ Therefore, the matrix A(t)  ∗ ∗ (t)) 0 ∂f0 (t,x∂x(t),u 1  ∂f1 (t,x∗ (t),u∗ (t))  0 ∂x1  ∗ ∗  0 ∂f2 (t,x (t),u (t)) ∂x1 ˜  A(t) =  . ..  .. .  ∗ ∗  0 ∂fn (t,x (t),u (t)) ∂x1 0 0

is given by ··· ··· ··· ...

∂f0 (t,x∗ (t),u∗ (t)) ∂xn ∂f1 (ˆ x∗ (t),u∗ (t)) ∂xn ∂f2 (ˆ x∗ (t),u∗ (t)) ∂xn

∂f0 (t,x∗ (t),u∗ (t)) ∂t ∂f1 (t,x∗ (t),u∗ (t)) ∂t ∂f2 (t,x∗ (t),u∗ (t)) ∂t

··· ···

∂fn (t,x∗ (t),u∗ (t)) ∂xn

∂fn (t,x∗ (t),u∗ (t)) ∂t

0

0

.. .

.. .

which has the block form  ∗ ∗ 0 [∇hx f0 (t, x∗ (t), u∗i(t))]T h ∂f0 (t,x ∂t(t),u (t)) i ∂fi (t,x∗ (t),u∗ (t)) ∂fi (t,x∗ (t),u∗ (t)) ˜ = A(t) 0  ∂xj

0

0

∂t

n×n

0

˜ can be written in block form Observe that A(t) " # ∂ fˆ(t,x∗ (t),u∗ (t)) ˆ ](n+1)×1 ˜ = A(t) [ ∂t A(t) 0 0 where ˆ = A(t)

"

0 [∇hx f0 (t, x∗ (t), u∗i(t))]T ∂fi (t,x∗ (t),u∗ (t)) 0 ∂xj n×n

#

n×1



    ,    



 .

(11.168)

is time dependent because of the explicit dependence of time in the problem as well as the inclusion of the optimal state and control. ˜ is given by The negative transpose of A(t)   0 [0]1×n 0 h iT   ∂fi (t,x∗ (t),u∗ (t)) ∗ ∗  T −[∇ f (t, x (t), u (t))] − 0 x 0 n×1 ˜ ∂xj −[A(t)] =  iTn×n  h  ∗ ∗ ∗ ∗ (t),u (t)) 0 − ∂f (t,x ∂t − ∂f0 (t,x ∂t(t),u (t)) n×1

so that the augmented adjoint system     η0 (t) η0 (t) d  ˜ T  η(t)  η(t)  = −[A(t)] dt ηn+1 (t) ηn+1 (t)

Chapter 11. Extensions of the Maximum Principle

441

is equivalent to d η0 (t) = 0, (11.169) dt d η(t) = −η0 (t) [∇x f0 (t, x∗ (t), u∗ (t))] dt −[Jx f (t, x∗ (t), u∗ (t)]T η(t), (11.170) ∗ ∗ d ∂f0 (t, x (t), u (t)) ηn+1 (t) = −η0 (t) dt ∂t n X ∂fi (t, x∗ (t), u∗ (t)) − . (11.171) ηi (t) ∂t i=1 Note that the first two equations have the form     d η0 (t) η0 (t) T ˆ , = −[A(t)] η(t) dt η(t) ˆ is given by (11.168). where the matrix A(t) The augmented Hamiltonian function is given by ˜ η, ˜ 0 , η1 , η2 , ..., ηn , ηn+1 , x0 , x1 , x2 , ..., H( ˜ x ˜, u) = H(η xn , xn+1 , u1 , u2, ..., um ) (11.172) = η0 f0 (xn+1 , x, u) + hη,f (xn+1 , x, u)i + ηn+1 n X = η0 f0 (xn+1 , x, u) + ηi fi (xn+1 , x, u) + ηn+1 . i=1

Since x ˜ = [ˆ x xn+1 ]T and η˜ = [ηˆ ηn+1 ]T , we can write ˜ η, ˆ η, H( ˜ x ˜, u) = H(t, ˆ x ˆ, u) + ηn+1 , where

ˆ η, H(t, ˆ x ˆ, u)

is called the time dependent augmented Hamiltonian. Note also ˜ : Rn+2 × Rn+2 → R defined by that M ˜ (η, ˜ η, M ˜ x ˜) = max H( ˜ x ˜, u) u∈Ω

(11.173)

442

Chapter 11. Extensions of the Maximum Principle

has the form ˜ (η, ˜ η, ˆ η, M ˜ x ˜) = max H( ˜ x ˜, u) = max H(t, ˆ x ˆ, u) + ηn+1 u∈Ω

u∈Ω

ˆ (t, η, =M ˆ x ˆ) + ηn+1 .

(11.174)

˜ η, so that maximizing H( ˜ x ˜, u) over Ω is equivalent to maximizing ˆ H(t, η, ˆ x ˆ, u) over Ω. We now state the (Pontryagin) Maximum Principle for the nonautonomous optimal control problem and use the basic Maximum Principle to prove this result. Theorem 11.4 (Maximum Principle for Nonautonomous Systems) Assume that f : R1 ×Rn × Rm → Rn , f0 : R1 ×Rn × Rm → R, X0 ⊆ Rn , t0 ∈ R, X1 ⊆ Rn and Ω ⊆ Rm are given as above and consider the control system (S)

x(t) ˙ = f (t, x(t), u(t)),

t0 < t ≤ t 1 ,

(11.175)

with piecewise continuous controllers u(·) ∈ P W C(t0 , +∞; Rm ) satisfying u(t) ∈ Ω ⊆ Rm e.f. If   u(·) ∈ P W C(t0 , t1 ; Rm ) : u(t) ∈ Ω e.f. and ∗ u (·) ∈ Θ = u(·) steers X0 to X1 at a finite time t1 . (11.176) minimizes Z t1 J(u(·)) = f0 (s, x(s), u(s))ds, (11.177) t0

on the set of admissible controls Θ with optimal response x∗ (·) satisfying x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t∗1 ) = x∗1 ∈ X1 , then there exists a non-trivial solution   ∗ η0 (t) (11.178) η˜∗ (t) =  η ∗ (t)  ∗ ηn+1 (t) to the augmented adjoint equation     η0 (t) η0 (t) d  ˜ T  η(t)  , η(t)  = −[A(t)] dt ηn+1 (t) ηn+1 (t)

(11.179)

443

Chapter 11. Extensions of the Maximum Principle such that

˜ η˜∗ (t), x ˜ (η˜∗ (t), x ˜ η˜∗ (t), x H( ˜∗ (t), u∗ (t)) = M ˜∗ (t)) = max H( ˜∗ (t), u). u∈Ω

(11.180) Moreover, there is a constant η0∗ ≤ 0 such that η0∗ (t) ≡ η0∗ ≤ 0

(11.181)

and for all t ∈ [t0 , t∗1 ],

˜ (η˜∗ (t), x ˜ η˜∗ (t), x M ˜∗ (t)) = max H( ˜∗ (t), u) ≡ 0. u∈Ω

(11.182)

The above maximal principle is equivalent to the maximization of the time augmented Hamiltonian ˆ ηˆ∗ (t), x H(t, ˆ∗ (t), u) and ˆ ηˆ∗ (t), x max H(t, ˆ∗ (t), u) = u∈Ω

Z tX n t∗1 i=0

ηi∗ (s)

∂fi (s, x∗ (s), u∗ (s)) ds. ∂t (11.183)

The transversality conditions imply that ∗ ηn+1 (t∗1 ) = 0

(11.184)

and hence ˆ ∗1 , ηˆ∗ (t∗1 ), x ˆ (t∗1 , ηˆ∗ (t∗1 ), x ˆ∗ (t∗1 ), u) = M ˆ∗ (t∗1 )) = 0. max H(t u∈Ω

(11.185) Also, if X0 ⊆ R and X1 ⊆ R are manifolds with tangent spaces T0 and T1 at x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t1 ) = x∗1 ∈ X1 , respectively, then n

n

ηˆ∗ (t) = [η0∗ (t) η1∗ (t) η2∗ (t) ... ηn∗ (t)]T = [η0∗ (t) η ∗ (t)]T can be selected to satisfy the transversality conditions η ∗ (t0 ) ⊥ T0

(11.186)

η ∗ (t∗1 ) ⊥ T1 .

(11.187)

and

444

Chapter 11. Extensions of the Maximum Principle

The proof of this result comes from a direct application of the the Maximum Principle given in Theorem 11.4 to this case where x is replaced by [x xn+1 ]T (see pages 318 - 322 in Lee and Markus [119]). Remark 11.5 One can still derive a Maximum Principle where the functions f : R1 ×Rn × Rm → Rn and f0 : R1 ×Rn × Rm → R are not differentiable in time. However, some information is lost since (11.171) and (11.183) do not make sense when the partial ∂ fi (t, x1 , x2 , ..., xn , u1 , u2, ..., um ) do not exist. Also, it derivatives ∂t ˜ given by equation (11.167) is not well is important to note that A(·) defined unless fi (t, x, u) are differentiable in time. Finally, note ∗ that if the adjoint states η0∗ and η ∗ (·) are both zero, then ηn+1 (·) would be a constant and the transversality condition (11.184) im∗ plies that ηn+1 (·) = 0. Thus, η0∗ and η ∗ (·) can not be both zero. We provide a statement of the Maximum Principle for the nonautonomous control problem that does not require smoothness in time. The result can be found in the books by Young [186] and Fleming [79]. Theorem 11.5 (Second Maximum Principle for Nonautonomous Systems) Assume that f : R1 ×Rn × Rm → Rn , f0 : R1 ×Rn × Rm → R, X0 ⊆ Rn , t0 ∈ R, X1 ⊆ Rn and Ω ⊆ Rm are given as above and consider the control system (S)

x(t) ˙ = f (t, x(t), u(t)),

t0 < t ≤ t 1 ,

(11.188)

with piecewise continuous controllers u(·) ∈ P W C(t0 , +∞; Rm ) satisfying u(t) ∈ Ω ⊆ Rm e.f. If   u(·) ∈ P W C(t0 , t1 ; Rm ) : u(t) ∈ Ω e.f. and ∗ u (·) ∈ Θ = u(·) steers X0 to X1 at a finite time t1 . (11.189) minimizes Z t1 f0 (s, x(s), u(s))ds, (11.190) J(u(·)) = t0

445

Chapter 11. Extensions of the Maximum Principle

on the set of admissible controls Θ with optimal response x∗ (·) satisfying x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t∗1 ) = x∗1 ∈ X1 , then there exists a non-trivial solution   ∗ η0 (t) ∗ (11.191) ηˆ (t) = η ∗ (t) to the time dependent augmented adjoint equation     d η0 (t) η (t) 0 T ˆ , = −[A(t)] η(t) dt η(t)

(11.192)

such that the time dependent augmented Hamiltonian is maximized ˆ ηˆ∗ (t), x ˆ (t, ηˆ∗ (t), x H(t, ˆ∗ (t), u∗ (t)) = M ˆ∗ (t)) ˆ ηˆ∗ (t), x ˆ∗ (t), u). = max H(t, u∈Ω

(11.193)

Moreover, there is a constant η0∗ ≤ 0 such that η0∗ (t) ≡ η0∗ ≤ 0.

(11.194)

The transversality conditions imply that ∗ ∗ ˆ ∗ , ηˆ∗ (t∗ ), x ˆ ∗ ˆ∗ (t∗1 ), x max H(t ˆ∗ (t∗1 )) = 0. 1 1 ˆ (t1 ), u) = M (t1 , η u∈Ω

(11.195) Also, if X0 ⊆ Rn and X1 ⊆ Rn are manifolds with tangent spaces T0 and T1 at x∗ (t0 ) = x∗0 ∈ X0 and x∗ (t1 ) = x∗1 ∈ X1 , respectively, then ηˆ∗ (t) = [η0∗ (t) η1∗ (t) η2∗ (t) ... ηn∗ (t)]T = [η0∗ (t) η ∗ (t)]T can be selected to satisfy the transversality conditions η ∗ (t0 ) ⊥ T0

(11.196)

η ∗ (t∗1 ) ⊥ T1 .

(11.197)

and If in addition, the functions fi (t, x, u), i = 0, 1, 2, . . . , n are C 1 in t, then Z tX n ∂fi (s, x∗ (s), u∗ (s)) ∗ ∗ ˆ ds. (11.198) ηi∗ (s) M (t, ηˆ (t), x ˆ (t)) = ∂t t∗1 i=0

446

Chapter 11. Extensions of the Maximum Principle

11.8

Application to the Nonautonomous LQ Control Problem

The Maximum Principle Theorem 11.5 will be applied to the Nonautonomous Linear Quadratic (LQ) control problem. We use the notation M : I → Rp×q

to denote a p × q (real valued) matrix function defined on an interval I. In particular, if M : I → Rp×q , then   m1,1 (t) m1,2 (t) · · · m1,q (t)  m2,1 (t) m2,2 (t) · · · m2,q (t)    M (t) =   .. .. .. . .   . . . . mp,1 (t) mp,2 (t) · · · mp,q (t)

where each entry is a real valued function defined on the interval I. Likewise we say that M (·) is piecewise continuous, continuous, etc. if each component is piecewise continuous, continuous, etc. We assume that t0 , x0 ∈ Rn and t0 < t1 are given and that A : [t0 , t1 ] → Rn×n and B : [t0 , t1 ] → Rn×m are piecewise continuous matrix valued functions. Consider the nonautonomous linear control system x(t) ˙ = A(t)x(t) + B(t)u(t) (11.199) with initial data x(0) = x0 ∈ Rn .

n×n

(11.200) m×m

Also, let Q : [t0 , t1 ] → R and R : [t0 , t1 ] → R be piecewise continuous and symmetric matrix valued functions such that QT (t) = Q(t) ≥ 0 and RT (t) = R(t) > 0, for all t ∈ [t0 , t1 ]. In addition we require that there is an α > 0 such that for all t ∈ [t0 , t1 ] we have 0 < αIm×m ≤ R(t). Condition (11.201) implies that for any u ∈ Rm , α kuk2 = α hu, ui ≤ hR(t)u, ui

(11.201)

Chapter 11. Extensions of the Maximum Principle

447

and [R(t)]−1 exists. Let w ∈ Rm and define u = [R(t)]−1 w ∈ Rm so that

2 α [R(t)]−1 w = α kuk2 ≤ hR(t)u, ui ≤ kR(t)uk kuk

= kwk [R(t)]−1 w .

If w 6= 0, then [R(t)]−1 w 6= 0 and dividing by k[R(t)]−1 wk it follows that

α [R(t)]−1 w ≤ kwk so that



[R(t)]−1 w ≤ kwk α m for all w ∈ R . Therefore,

[R(t)]−1 ≤ 1 α

and [R(t)]−1 is bounded on [t0 , t1 ]. Since R : [t0 , t1 ] → Rm×m is piecewise continuous, it then follows that R−1 : [t0 , t1 ] → Rm×m is also piecewise continuous. This is important since we need the mapping Λ : [t0 , T ] → Rm×n defined by Λ(t) = [B(t)][R(t)]−1 [B(t)]T to also be piecewise continuous. We assume that the controls belong to the space P W C(t0 , T ; Rm ) and the quadratic cost functional is defined by J(u(·)) = hSx(t1 ), x(t1 )i Z 1 t1 {hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i} ds, + 2 t0 (11.202) where S = S T ≥ 0 is a constant symmetric matrix and x(t) = x(t; u(·)) is the solution to the system (11.199) - (11.200). The

448

Chapter 11. Extensions of the Maximum Principle

Linear Quadratic (LQ) Optimal Control problem is to find u∗ (·) ∈ P W C(t0 , t1 ; Rm ) so that J(u∗ (·)) = hSx∗ (T ), x∗ (T )i Z 1 t1 {hQ(s)x∗ (s), x∗ (s)i + hR(s)u∗ (s), u∗ (s)i} ds + 2 t0 ≤ hSx(T ), x(T )i Z 1 t1 + {hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i} ds, 2 t0 for all u(·) ∈ P W C(0, t1 ; Rm ).

Formulation as an Optimal Control Problem: In order to set up the nonautonomous linear quadratic optimal control problem as a fixed time optimal control problem and apply the Maximum Principle, we need to: 1. Identify the initial time t0 and final time t1 ; 2. Identify the initial set X0 ⊆ Rn , the terminal set X1 ⊆ Rn ; 3. Identify the control constraint set Ω ⊆ Rm ; 4. Identify the functions f : R1 × Rn × Rm −→ Rn , f0 : R1 × Rn × Rm −→ R1 ; ˆ η, 5. Define the time dependent augmented Hamiltonian H(t, ˆ x ˆ, u); 6. Form the time dependent augmented adjoint system matrix ˆ A(t).

The initial time is t0 and the final time t1 is given and fixed. The initial set is the single initial vector X0 = {x0 } and since there is no terminal constraint, the terminal set is the whole state space X1 = Rn . Also, there is no constraint on the control u(·) ∈ P W C(t0 , t1 ; Rm ), hence the control constraint set is all of Rm and Ω = Rm . The functions f : R1 × Rn × Rm −→ Rn and f0 : R1 × Rn × Rm −→ R1 are given by f (t, x, u) = A(t)x + B(t)u,

Chapter 11. Extensions of the Maximum Principle and f0 (t, x, u) =

449

1 1 hQ(t)x, xi + hR(t)u, ui ≥ 0, 2 2

respectively. ˆ : R1 × Rn+1 × The time dependent augmented Hamiltonian H Rn+1 × Rm −→ R1 is defined by

η0 ˆ η, {hQ(t)x, xi + hR(t)u, ui} H(t, ˆ x ˆ, u) = 2 + hη, A(t)x + B(t)ui (11.203) η0 η0 = xT Q(t) x+ uT R(t)u + [A(t)x]T η 2 2 T + [B(t)u] η η0 η0 T = x Q(t) x+ uT R(t)u + xT [A(t)]T η 2 2 T T +u [B(t)] η,

 T  T where x ˆ = x0 x ∈ Rn+1 , ηˆ = η0 η ∈ Rn+1 and u ∈ Rm . The time dependent augmented function f : R1 × Rn+1 × Rm −→ Rn+1 is defined by  1  1 hQ(t)x, xi + hR(t)u, ui 2 fˆ(t, x ˆ, u) = 2 A(t)x + B(t)u,  1  T xT Q(t)x + 12 u R(t)u 2 = , (11.204) A(t)x + B(t)u, so that the time dependent Jacobian is given by   0 [Q(t)x]T ˆ ˆ . Jxˆ f (t, x ˆ, u) = Jxˆ f (t, x, u) = 0 A(t)

(11.205)

Now assume that (x∗ (·), u∗ (·)) is an optimal pair so that the ˆ is given by matrix A(t) ˆ = Jxˆ fˆ(t, x A(t) ˆ, u)|(x∗ (t),u∗ (t)) = Jxˆ fˆ(t, x∗ (t), u∗ (t))   0 [Q(t)x∗ (t)]T (11.206) = 0 A(t)

450

Chapter 11. Extensions of the Maximum Principle

and ˆ T =− −[A(t)]  =



0 0 ∗ [Q(t)x (t)] [A(t)]T



0 0 ∗ − [Q(t)x (t)] −[A(t)]T



.

The time dependent augmented adjoint equation has the form     d η0 (t) η0 (t) T ˆ = −[A(t)] η(t) dt η(t)    0 0 η0 (t) = (11.207) − [Q(t)x∗ (t)] −[A(t)]T η(t) which is equivalent to the system d η0 (t) = 0 dt d η(t) = −η0 (t)Q(t)x∗ (t) − [A(t)]T η(t), dt where again x∗ (·) is the optimal trajectory. The first equation above implies (as always) that the zero adjoint state is a constant η0 (t) ≡ η0 . The second equation is coupled to the state equation by the term −η0 (t)Q(t)x∗ (t). If (x∗ (·), u∗ (·)) is an optimal pair for the LQ optimal control problem, then the Maximum Principle implies that there is a nontrivial solution   ∗ η0 (t) ∗ ∈ Rn+1 ηˆ (t) = η ∗ (t)

to the augmented adjoint equation (11.81) such that η0∗ (t) ≡ η0∗ ≤ 0, d ∗ η (t) = −η0∗ Q(t)x∗ (t) − [A(t)]T η ∗ (t) (11.208) dt and ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x H(t, ˆ∗ (t), u∗ (t)) = maxm H(t, ˆ∗ (t), u) ≡ c(t). u∈R

Since Ω = Rm is open, then ˆ ηˆ∗ (t), x ˆ ηˆ∗ (t), x Du H(t, ˆ∗ (t), u)|u=u∗ (t) = Du H(t, ˆ∗ (t), u∗ (t)) = 0,

Chapter 11. Extensions of the Maximum Principle

451

where ˆ ηˆ∗ (t), x Du H(t, ˆ∗ (t), u)  ∗  η0 ∗ T η0∗ T ∂ ∗ [x (t)] Q(t)x (t) + u R(t)u = ∂u 2 2 ∂  ∗ T [x (t)] [A(t)]T η ∗ (t) + uT [B(t)]T η ∗ (t) + ∂u  ∗  ∂ η0 T T T ∗ = u R(t)u + u [B(t)] η (t) ∂u 2 = η0∗ R(t)u + [B(t)]T η ∗ (t). Hence, when u = u∗ (t) it follows that ˆ ηˆ∗ (t), x Du H(t, ˆ∗ (t), u)|u=u∗ (t) = η0∗ R(t)u + [B(t)]T η ∗ (t) |u=u∗ (t) (11.209) = η0∗ R(t)u∗ (t) + [B(t)]T η ∗ (t) = 0. Applying the transversality condition at x∗ (0) = x0 ∈ X0 = {x0 } yields that η ∗ (0) can be any vector since T0 = {x0 }. However, at t1 we have the transversality condition η ∗ (t1 ) = η0 Sx∗ (t1 ).

(11.210)

This boundary condition in turn implies that η0∗ < 0. To see this assume that η0∗ = 0 and observe that the adjoint equation (11.82) reduces to the linear system d ∗ η (t) = −η0∗ Qx∗ (t) − AT η ∗ (t) dt = −AT η ∗ (t). Therefore, η ∗ (·) would be a solution of the homogenous linear initial value problem d ∗ η (t) = −AT η ∗ (t), η ∗ (t∗1 ) = η0 Sx∗ (t1 ) = 0 dt and hence it follows that η ∗ (t) ≡ 0. Consequently, we have shown that if η0∗ = 0, then η ∗ (t) ≡ 0 and hence   ∗ η0 (t) ∗ ≡0 ηˆ (t) = η ∗ (t)

452

Chapter 11. Extensions of the Maximum Principle

which contradicts the statement that ηˆ∗ (t) is a nontrivial solution of the time augmented adjoint equation (11.207). Since η0∗ < 0 we can solve (11.209) for the optimal control. We have η0∗ R(t)u∗ (t) + [B(t)]T η ∗ (t) = 0 which yields η0∗ R(t)u∗ (t) = −[B(t)]T η ∗ (t) and ∗

−1

u (t) = [R(t)] [B(t)]

T



 η ∗ (t) . −η0∗

Since the matrix R(t) = [R(t)]T > 0 is nonsingular and [R(t)]−1 is piecewise smooth we have the following expression for the (possibly piecewise continuous) optimal control  ∗  ∗ −1 T η (t) . (11.211) u (t) = [R(t)] [B(t)] −η0∗ Summarizing, it follows that the optimal trajectory can be obtained by solving the two point boundary value problem defined by the coupled state and adjoint equations d ∗ x (t) dt d ∗ η (t) dt

= A(t)x∗ (t) − η1∗ [B(t)][R(t)]−1 [B(t)]T η ∗ (t), x∗ (0) = x0 , 0 = −η0∗ Q(t)x∗ (t) − [A(t)]T η ∗ (t), η ∗ (t1 ) = η0 Sx∗ (t1 ), (11.212) and setting  ∗  ∗ −1 T η (t) . u (t) = [R(t)] [B(t)] −η0∗

To eliminate the η0∗ term, we divide the adjoint equation above by −η0∗ which yields  ∗   ∗    d η ∗ (t) η (t1 ) ∗ T η (t) = Q(t)x (t)−[A(t)] , = −Sx∗ (t1 ). dt −η0∗ −η0∗ −η0∗ Defining the normalized adjoint state λ∗ (t) by λ∗ (t) ,

η ∗ (t) −η0∗

Chapter 11. Extensions of the Maximum Principle

453

produces the optimality conditions = A(t)x∗ (t) + [B(t)][R(t)]−1 [B(t)]T λ∗ (t), x∗ (0) = x0 , = Q(t)x∗ (t) − [A(t)]T λ∗ (t), λ∗ (t1 ) = −Sx∗ (t1 ), (11.213) where the optimal control is defined by d ∗ x (t) dt d ∗ λ (t) dt

u∗ (t) = R−1 (t)B T (t)λ∗ (t).

(11.214)

We can write the optimality system as     ∗  d x∗ (t) A(t) [B(t)][R(t)]−1 [B(t)]T x (t) , = ∗ Q(t) −[A(t)]T λ∗ (t) dt λ (t) (11.215) with boundary conditions     x∗ (0) ∗ x (0) = In×n 0n×n = x0 , (11.216) λ∗ (0)

and

∗

λ (t1 ) =



−S 0n×n





x∗ (t1 ) λ∗ (t1 )



.

(11.217)

Thus, if one solves the two point boundary value problem (11.215) - (11.217), the optimal control is defined by (11.214).

11.9

Problem Set for Chapter 11

Problem 11.1 Consider the control system x˙ 1 (t) = x2 (t) + u1 (t), x˙ 2 (t) = u1 (t), with initial condition x1 (0) = 1, x2 (0) = 1, and terminal condition x2 (2) = 0.

454

Chapter 11. Extensions of the Maximum Principle

Let the control constraint set be Ω = R2 . Investigate the optimal control problem for the cost functional 1 J(u(·)) = 2

Z2 p [u1 (s)]2 + [u2 (s)]2 ds. 0

Problem 11.2 Consider the control system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), with initial condition x1 (0) = 1, x2 (0) = 1 and terminal condition x1 (1) = 0, x2 (1) is free. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional 1 J(u(·)) = 2

Z1

[u(s)]2 ds.

0

Problem 11.3 Consider the control system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), with initial condition x1 (0) = 1, x2 (0) = 1 and terminal condition x1 (1) = 0, x2 (1) = 0. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional 1 J(u(·)) = 2

Z1 0

[u(s)]2 ds.

Chapter 11. Extensions of the Maximum Principle

455

Problem 11.4 Consider the control system x˙ 1 (t) = x2 (t), x˙ 2 (t) = u(t), with initial condition x1 (0) = 1, x2 (0) = 1, and terminal condition x1 (t1 ) = 0, x2 (1) = 0. Let the control constraint set be Ω = [−1, +1]. Investigate the optimal control problem for the cost functional Z1 p J(u(·)) = 1 + [u(s)]2 ds. 0

Problem 11.5 Consider the control system x(t) ˙ = x(t) + u(t), with initial condition x(0) = 1, and terminal condition x(2) is free. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional 1 1 J(u(·)) = [x(2)]2 + 2 2

Z2 0



[x(s)]2 + [u(s)]2 ds.

Problem 11.6 Consider the control system x(t) ˙ = x(t) + u(t),

456

Chapter 11. Extensions of the Maximum Principle

with initial condition x(0) = 1, and terminal condition x(1) is free. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional 1 J(u(·)) = 2

Z1 0



[x(s) − e−s ]2 + [u(s)]2 ds.

Problem 11.7 Consider the control system x(t) ˙ = x(t) + u(t), with initial condition x(0) = 1, and terminal condition x(1) is free. Let the control constraint set be Ω = [−1, +1]. Investigate the optimal control problem for the cost functional 1 J(u(·)) = 2

Z2

[x(s) − e−s ]2 ds.

0

Problem 11.8 Consider the control system x˙ 1 (t) = x2 (t) + cos(u(t)), x˙ 2 (t) = sin(u(t)), with initial condition x1 (0) = 0, x2 (0) = 0,

Chapter 11. Extensions of the Maximum Principle

457

and terminal condition x1 (1) and x2 (1) are free. Let the control constraint set be Ω = [−π, +π]. Investigate the optimal control problem for the cost functional J(u(·)) = −

Z1

{x2 (s) + cos(u(s))} ds.

0

Problem 11.9 Consider the control system x(t) ˙ = −x(t) + u(t), with initial condition x(0) = 1, and terminal condition x(2) is free. Let the control constraint set be Ω = R1 . Investigate the optimal control problem for the cost functional J(u(·)) =

Z2 0



[x(s)]2 + [u(s)]2 ds.

Derive and solve the Riccati differential equation that defines the optimal feedback controller. Problem 11.10 Consider the control system x(t) ˙ = x(t) + bu(t), with initial condition x(0) = 1, and terminal condition x(1) is free.

458

Chapter 11. Extensions of the Maximum Principle

Here, b 6= 0 and the control constraint set is Ω = R1 . Investigate the optimal control problem for the cost functional J(u(·)) =

Z1 0



q[x(s)]2 + [u(s)]2 ds,

where q ≥ 0. Derive and solve the Riccati differential equation that defines the optimal feedback controller. What happens when q −→ 0?

Chapter 12 Linear Control Systems Although we have focused on control and optimal control of nonlinear control systems, linear control systems play a key role in practical applications of modern control. Linear systems arise naturally when the dynamic model is linear and when linearization is used to produce an approximate system. We discuss linearization later, but first we present a short introduction to the basic ideas and control system properties. Good references for this section include [7], [45], [110] and [115].

12.1

Introduction to Linear Control Systems

We focus on linear control systems defined by a linear differential equation x(t) ˙ = A(t)x(t) + B(t)u(t) + G(t)w(t) (12.1) with initial data x(t0 ) = x0 ∈ Rn .

(12.2)

The inputs to the system are defined by the control u(·) and a “disturbance” w(·). As before, we assume that u(·) is piecewise continuous and in this section we assume that the disturbance w(·) is also piecewise continuous. The sensed output is defined by y(t) = C(t)x(t) + H(t)v(t) ∈ Rp 459

(12.3)

460

Chapter 12. Linear Control Systems

where y(·) is called the sensed (or measured) output and v(·) represents sensor noise. Again, in this section it is useful to think of v(·) as a piecewise continuous function. However, later it will be important to view both w(·) and v(·) as white noise. In the case with no disturbances, the control system is defined by  x(t) ˙ = A(t)x(t) + B(t)u(t), (12.4) y(t) = C(t)x(t). Recall that many of the previous optimal control problems are formulated in terms of finding a control u(·) that steers an initial state x0 to a terminal state x1 in a finite time t1 > t0 . Thus, we must consider the following question. Given an initial state x0 ∈ Rn and a final state x1 ∈ Rn is there a time t1 > t0 and a corresponding control u(·) ∈ P W C(t0 , t1 ; Rm ) such that u(·) steers x0 ∈ Rn to x1 ∈ Rn at time t1 ? This question motivates the following definition. Definition 12.1 (Controllability) The system x(t) ˙ = A(t)x(t) + B(t)u(t)

(12.5)

is said to be (completely) controllable if for any initial time t0 and states x0 ∈ Rn and x1 ∈ Rn , there exists a finite time t1 > t0 and a piecewise continuous control u(·) ∈ P W C(t0 , t1 ; Rm ) such that the solution to (12.5) satisfying the initial condition x(t0 ) = x0 also satisfies x(t1 ) = x1 . In particular, (12.5) is controllable if and only if x0 can be steered to x1 in finite time t1 − t0 . We shall also be interested in the concept of observability. Loosely speaking, observability implies that one can reconstruct the initial state of the system from given input - output pairs.

461

Chapter 12. Linear Control Systems Definition 12.2 (Observability) The system  x(t) ˙ = A(t)x(t) + B(t)u(t) y(t) = C(t)x(t)

is (completely) observable if for all t0 , there exists a t1 > t0 such that if y(t; t0 , x0 , u(·)) = C(t)x(t; t0 , x0 , u(·)) = y(t; t0 , x ˜0 , u(·)) = C(t)x(t; t0 , x ˜0 , u(·)) for all t ∈ [t0 , t1 ] and all controls u(·) ∈ P W C(t0 , t1 ; Rm ), then x0 = x ˜0 . Here, x(t; t0 , x0 , u(·)) denotes the solution to (12.5) with initial condition x(t0 ) = x0 . In particular, the initial state x0 at time t0 is uniquely determined by the output y(t; t0 , x0 , u(·)) on the interval [t0 , t1 ]. Remark 12.1 In the special case where the matrix C(t) is an n × n nonsingular matrix, then y(t0 ) = C(t0 )x(t0 ) implies x0 = x(t0 ) = [C(t0 )]−1 y(t0 ) so that the system is trivially observable. Note that controllability is determined by the matrices A(·) and B(·) and observability is determined by the matrices A(·) and C(·). Thus, we shall simply say that the pair (A(·), B(·)) is controllable or that (A(·), C(·)) is observable. In the previous section we saw where linear control systems occur either naturally or from linearization. In this section we consider the LQ control problem with time varying matrix coefficients. Most of the following material can be found in the standard books [6], [5], [7], [16], [45], [71] and [73], [81], [115] and [187]. Consider the linear system on the time interval [t0 , t1 ] given by x(t) ˙ = A(t)x(t) + B(t)u(t), x(t0 ) = x0 ∈ Rn .

(12.6)

462

Chapter 12. Linear Control Systems

Recall that the state transition matrix is defined by d Φ(t, s) = A(t)Φ(t, s), dt

Φ(τ, τ ) = In×n

(12.7)

and the solution to (12.6) is given by the Variation of Parameters Formula Z t x(t) = Φ(t, t0 )x0 + Φ(t, s)B(s)u(s)ds. (12.8) t0

Let Wc (t0 , t1 ) , and Wr (t0 , t1 ) ,

Z Z

t1

Φ(t0 , s)B(s)B T (s)ΦT (t0 , s)ds

t0 t1

Φ(t1 , s)B(s)B T (s)ΦT (t1 , s)ds.

t0

The matrices Wc (t0 , t1 ) and Wr (t0 , t1 ) are called the controllability Gramian and reachability Gramian, respectively (see Chapter 3 in [7]). Observe that Φ(t1 , t0 )Wc (t0 , t1 )ΦT (t1 , t0 )  Z t1  T T = Φ(t1 , t0 ) Φ(t0 , s)B(s)B (s)Φ (t0 , s)ds ΦT (t1 , t0 ) t0  Z t1  T T T = Φ(t1 , t0 )Φ(t0 , s)B(s)B (s)Φ (t0 , s)Φ (t1 , t0 )ds t0  Z t1  T T = [Φ(t1 , t0 )Φ(t0 , s)]B(s)B (s)[Φ(t1 , t0 )Φ(t0 , s)] ds t0  Z t1  T T = [Φ(t1 , s)]B(s)B (s)[Φ(t1 , s)] ds t0

= Wr (t0 , t1 ),

so that Wr (t0 , t1 ) = Φ(t1 , t0 )Wc (t0 , t1 )ΦT (t1 , t0 ). Moreover, since Φ(t1 , t0 ) is non-singular it follows that Wr (t0 , t1 ) and Wc (t0 , t1 ) have the same range.

Chapter 12. Linear Control Systems

463

Theorem 12.1 (Minimum Energy Theorem) The state x0 at time t0 can be transferred to the state x1 at time t1 if and only if w = x1 − Φ(t1 , t0 )x0 ∈ Range(Wr (t0 , t1 )). If w = x1 − Φ(t1 , t0 )x0 ∈ Range(Wr (t0 , t1 )), then there is a vector v ∈ Rn such that w = x1 − Φ(t1 , t0 )x0 = Wr (t0 , t1 )v and a control that transfers x0 at time t0 to x1 at time t1 is given by u ˆ(t) = B T (t)ΦT (t1 , t)v.

(12.9)

If uo (·) is any other controller that steers x0 at time t0 to the state x1 at time t1 , then Z Z 1 t1 T 1 t1 T u ˆ (s)ˆ u(s)ds ≤ u (s)uo (s)ds (12.10) 2 t0 2 t0 o and u ˆ(t) = B T (t)ΦT (t1 , t)v minimizes the “energy” functional defined by Z Z 1 t 1 t E = J(u(·)) = hu(s), u(s)i ds = u(s)T u(s)ds. 2 t0 2 t0 Moreover, if [Wr (t0 , t1 )]−1 exists, then the energy required to make this transfer is given by Z 1 1 t1 T E= u ˆ (s)ˆ u(s)ds = wT [Wr (t0 , t1 )]−1 w 2 t0 2  1

[Wr (t0 , t1 )]−1 w, w . (12.11) = 2

Proof : Note that if x1 − Φ(t1 , t0 )x0 = Wr (t0 , t1 )v and u ˆ(t) = B T (s)ΦT (t1 , t)v, the Variation of Parameters formula implies Z t x(t) = Φ(t, t0 )x0 + Φ(t, s)B(s)ˆ u(s)ds. t0 Z t = Φ(t, t0 )x0 + Φ(t, s)B(s)B T (s)ΦT (t1 , s)vds t0

464

Chapter 12. Linear Control Systems

so that x(t1 ) = Φ(t1 , t0 )x0 +

Z

t1 t0

 Φ(t1 , s)B(s)B (s)Φ (t1 , s)ds v T

T

= Φ(t1 , t0 )x0 + Wr (t0 , t1 )v. Hence, we have

x(t1 ) = Φ(t1 , t0 )x0 + Wr (t0 , t1 )v = x1 so that u ˆ(t) = B T (s)ΦT (t1 , t)v steers x0 at time t0 to x1 at time t1 . Assume now that uo (·) is any other controller that steers x0 at time t0 to the state x1 at time t1 . The Variation of Parameters formula implies Z t1 Φ(t1 , s)B(s)uo (s)ds. x1 = Φ(t1 , t0 )x0 + t0

Likewise, x1 = Φ(t1 , t0 )x0 +

Z

t1

Φ(t1 , s)B(s)ˆ u(s)ds t0

and subtracting the two we obtain Z t1 Φ(t1 , s)B(s)[uo (s) − u ˆ(s)]ds = 0. t0

Pre-multiplying by v T yields Z t1 T 0=v Φ(t0 , s)B(s)[uo (s) − u ˆ(s)]ds t0 Z t1 = v T Φ(t0 , s)B(s)[uo (s) − u ˆ(s)]ds t0 Z t1  T T ˆ(s)]ds B (s)ΦT (t1 , t)v [uo (s) − u = t0

and since

u ˆ(t) = B T (t)ΦT (t1 , t)v,

465

Chapter 12. Linear Control Systems it follows that 0=

Z

t1 t0

[ˆ u(t)]T [uo (s) − u ˆ(s)]ds.

Therefore, we have Z 1 t1 [uo (s) − u ˆ(s)]T [uo (s) − u ˆ(s)]ds 0≤ 2 t0 Z  1 t1  T = uo (s)uo (s) − 2ˆ uT (s)uo (s) + u ˆT (s)ˆ u(s) ds 2 t0 Z  1 t1  T uo (s)uo (s) − 2ˆ uT (s)uo (s) + 2ˆ uT (s)ˆ u(s) − u ˆT (s)ˆ u(s) ds = 2 t0 Z t1 Z  T  1 t1 T u ˆ (s)uo (s) − u ˆT (s)ˆ u(s) ds uo (s)uo (s)ds − = 2 t0 t0 Z t1 1 u ˆT (s)ˆ u(s)ds − 2 t0 Z t1 Z 1 t1 T u ˆT (s) [uo (s) − u ˆ(s)] ds uo (s)uo (s)ds − = 2 t0 t0 Z 1 t1 T − u ˆ (s)ˆ u(s)ds 2 t0 Z Z 1 t1 T 1 t1 T u (s)uo (s)ds − u ˆ (s)ˆ u(s)ds = 2 t0 o 2 t0 and hence 1 2

Z

t1 t0

1 u ˆ (s)ˆ u(s)ds ≤ 2 T

Z

t1 t0

uTo (s)uo (s)ds,

so that u ˆ(t) = B T (t)ΦT (t1 , t)v minimizes the energy needed to transfer x0 at time t0 to the state x1 at time t1 . Finally, if [Wr (t0 , t1 )]−1 exists then v = [Wr (t0 , t1 )]−1 [x1 − Φ(t1 , t0 )x0 ] = [Wr (t0 , t1 )]−1 w] and the control is given by u ˆ(t) = B T (t)ΦT (t1 , t) v = B T (t)ΦT (t1 , t)[Wr (t0 , t1 )]−1 w.

466

Chapter 12. Linear Control Systems

Thus, the minimum energy required to transfer x0 at time t0 to the state x1 at time t1 is given by Z Z 1 t 1 t hu(s), u(s)i ds = u(s)T u(s)ds E= 2 t0 2 t0 Z 1 t T = [B (s)ΦT (t1 , s)[Wr (t0 , t1 )]−1 w]T 2 t0 1 = 2

Z

× [B T (s)ΦT (t1 , s)[Wr (t0 , t1 )]−1 w]ds t

wT [[Wr (t0 , t1 )]−1 ]T Φ(t1 , s)B T (s) t0

× B T (s)ΦT (t1 , s)[Wr (t0 , t1 )]−1 w]ds Z t  1 T −1 T T T T = w [[Wr (t0 , t1 )] ] Φ(t1 , s)B (s)B (s)Φ (t1 , s)ds 2 t0 × [Wr (t0 , t1 )]−1 w] 1 = wT [[Wr (t0 , t1 )]−1 ]T [Wr (t0 , t1 )] [Wr (t0 , t1 )]−1 w] 2 1 = wT [[Wr (t0 , t1 )]−1 ]T w]. 2 Since [Wr (t0 , t1 )]T = Wr (t0 , t1 ), it follows that [[Wr (t0 , t1 )]−1 ]T = [Wr (t0 , t1 )]−1 and hence 1 1 E = wT [[Wr (t0 , t1 )]−1 ]T w = wT [Wr (t0 , t1 )]−1 w 2 2

(12.12)

and this completes the proof.  Observe that the previous theorem provides a characterization of the (complete) controllability of the system (12.6). In particular, we have the following result. Theorem 12.2 The system (12.6) is completely controllable at time t0 if and only if there is a time t1 > t0 such that Wr (t0 , t1 ) has maximal rank n. Since rank[Wr (t0 , t1 )] = rank[Wc (t0 , t1 )], it also follows that (12.6) is completely controllable at time t0 if and only if there is a time t1 > t0 such that Wc (t0 , t1 ) has maximal rank n.

467

Chapter 12. Linear Control Systems For time invariant systems x(t) ˙ = Ax(t) + Bu(t)

(12.13)

one has the following result (see [7], [45], [110] and [115]). Theorem 12.3 For the time invariant system (12.13), Φ(t, s) = eA(t−s) and Range(CM) = Range(Wr (0, t)) for all time t > 0, where CM = [B, AB, A2 B, . . . , An−1 B]. Remark 12.2 It is important to note that for time invariant systems where Φ(t, s) = eA(t−s) the transpose of Φ(t, s) is given by [Φ(t, s)]T = eA

T (t−s)

so that

d T [Φ(t, s)]T = AT eA (t−s) = AT [Φ(t, s)]T . dt However, in general one has  T d d T [Φ(t, s)] = [Φ(t, s)] = [A(t)Φ(t, s)]T = [Φ(t, s)]T A(t)T . dt dt Moreover, (in general) Φ(t, s) and A(t) do not commute so that [Φ(t, s)]T A(t)T 6= A(t)T [Φ(t, s)]T

and one cannot imply that dtd [Φ(t, s)]T = A(t)T [Φ(t, s)]T . For example, consider the problem with   0 1 A(t) = −2 2 t2

t

for t > 1. The state transition matrix is given by  2t t2  2 ( s − s2 ) ( ts − t) Φ(t, s) = 2 ( 2s − 2ts2 ) ( 2ts − 1)

and it is easy to show that

[Φ(t, s)]A(t) 6= A(t)[Φ(t, s)].

468

Chapter 12. Linear Control Systems

Example 12.1 Consider the model of the rocket sled        d x1 (t) 0 x1 (t) 0 1 u(t). + = 1/m x2 (t) 0 0 dt x2 (t) A direct calculation yields Φ(t, s) = e

A(t−s)

=



1 (t − s) 0 1



and

  1 (t − s)2 (t − s) . Φ(t, s)BB [Φ(t, s)] = 2 (t − s) 1 m Therefore,  t3 t2  Z t 1 T T Φ(t1 , s)B(s)B (s)Φ (t1 , s)ds = 2 t23 2 Wr (0, t) = t m 0 2 T

T

is non-singular. Again, in the special case where the system is autonomous (i.e. time-invariant) and has the form  x(t) ˙ = Ax(t) + Bu(t) ∈ Rn , (12.14) y(t) = Cx(t) ∈ Rp one has the following results which can be found in [7], [45], [110], [115] and [119]. Theorem 12.4 Consider the linear autonomous system (12.14). (i) The autonomous linear system (12.14) is controllable if and only if the n × nm controllability matrix CM = [B, AB, A2 B, . . . , An−1 B]

(12.15)

has rank n. (ii) The linear autonomous system (12.14) is observable if and only if the n × np observability matrix OM = [C T , AT C T , [AT ]2 C T , . . . , [AT ]n−1 C T ] has rank n.

(12.16)

Chapter 12. Linear Control Systems

469

The previous results point to an important dual relationship between controllability and observability. Given the system (12.14) we define the dual system by  z(t) ˙ = AT z(t) + C T v(t) ∈ Rn (12.17) ζ(t) = B T z(t) ∈ Rm where C T is now an n × p matrix and B T is an m × n matrix. Note that the dual system has inputs from the output space of the system (12.14) and outputs from the input space of (12.14). Thus, we have the following duality result. Theorem 12.5 The system (12.14) is controllable if and only if the dual system (12.17) is observable and (12.14) is observable if and only if the dual system (12.17) is controllable. Example 12.2 Consider the rocket control problem defined by the system        d x1 (t) 0 x1 (t) 0 1 u(t), (12.18) + = 1/m x2 (t) 0 0 dt x2 (t)

or

x(t) ˙ = Ax(t) + Bu(t),

(12.19)

where the matrices A and B are defined by     0 0 1 , and B = A= 1/m 0 0  T respectively. Here, x(t) = x1 (t) x2 (t) , where x1 (t) is the position of the sled and x1 (t) is the velocity. Since n = 2, the controllability matrix is given by   0 1/m CM = [B, AB] = 1/m 0 and clearly rank(CM) = 2 so that the system is controllable. Consider the output given by     x1 (t) = x2 (t). y(t) = Cx(t) = 0 1 x2 (t)

470

Chapter 12. Linear Control Systems

Here C =



0 1



and hence T

C =

Moreover, T

A =





0 1



0 0 1 0

. 

so that the observability matrix is given by   0 0 T T T OM = [C , A C ] = . 1 1 Since rank(OM) = 1 the system is not observable. On the other hand, consider the case where the the output is given by     x1 (t) = x1 (t). y(t) = Cx(t) = 1 0 x2 (t)   Here C = 1 0 and hence   1 T C = 0 so that the observability matrix is given by   1 0 T T T OM = [C , A C ] = 0 1 and rank(OM) = 2. In this case the system is observable. We see that sensing (measuring) the velocity leads to an unobservable system while sensing the position produces an observable system. Example 12.3 Consider the system        d x1 (t) 0 x1 (t) −1 0 u(t), + = 1 x (t) 0 1 x (t) dt 2 2

(12.20)

with output y(t) = y(t) = Cx(t) =



1 1





x1 (t) x2 (t)



= x1 (t) + x2 (t).

471

Chapter 12. Linear Control Systems Here, A=



−1 0 0 1



, B=



0 1



so that CM = [B, AB] = and T

T

T

and C = 

OM = [C , A C ] =



0 0 1 1



1 −1 1 1





1 1



.

Since rank(CM) = 1 the system is not controllable, but since rank(OM) = 2 the system is observable. Stability is of primary importance in many control systems and ensuring stability often is a major goal in feedback design. We recall the basic results for stability of linear autonomous systems. Theorem 12.6 Let A be a constant n × n matrix and suppose that all the eigenvalues λi , i = 1, 2, . . . , n of A have negative real part. Then every solution of x(t) ˙ = Ax(t) is exponentially asymptotically stable. In particular, there exists constants M ≥ 1 and γ > 0 such that kx(t)k ≤ M e−γt kx(0)k (12.21) for all t ≥ 0. On the other hand, if A has one eigenvalue with positive real part, then every solution of x(t) ˙ = Ax(t) is unstable.

Since the stability of the system x(t) ˙ = Ax(t) is determined by the eigenvalues of A, we say that A is a stable matrix if and only if ℜ(λi ) < 0 (12.22) for all eigenvalues λi , i = 1, 2, . . . , n of A. As observed in the previous chapter (see Section 11.5), linear quadratic optimal control problems yield optimal controllers that are of the form of linear state feedback u∗ (t) = −K(t)x∗ (t).

472

Chapter 12. Linear Control Systems

We shall see later that in some cases the gain matrix K(t) is constant so that the feedback controller has the form u∗ (t) = −Kx∗ (t).

(12.23)

If we apply a feedback control law of the form (12.23) to the control system x(t) ˙ = Ax(t) + Bu(t), then the closed-loop system has the form x(t) ˙ = Ax(t) − BKx(t) = [A − BK]x(t).

(12.24)

Consequently, the closed-loop system is stable if and only if the matrix [A − BK] is a stable matrix. Thus, we are interested in the issue of finding a gain matrix such that [A − BK] is a stable matrix and this motivates the following definitions. Definition 12.3 The control system  x(t) ˙ = Ax(t) + Bu(t) ∈ Rn y(t) = Cx(t) ∈ Rp

(12.25)

is stabilizable if there exists a m × n matrix K such that [A − BK] is a stable matrix. Definition 12.4 The control system  x(t) ˙ = Ax(t) + Bu(t) ∈ Rn y(t) = Cx(t) ∈ Rp

(12.26)

is detectable if there exists a n × p matrix F such that [A − F C] is a stable matrix. Observe that if one considers the dual system (12.17)  z(t) ˙ = AT z(t) + C T v(t) ∈ Rn , (12.27) ζ(t) = B T z(t) ∈ Rm ˜ such then the dual system is stabilizable if there is a matrix K T T ˜ that [A − C K] is a stable matrix. However, since a matrix and its transpose have the same eigenvalues it follows that

473

Chapter 12. Linear Control Systems

˜ T = [A − K ˜ T C] is a stable matrix. Thus, if we set [AT − C T K] ˜ T , then [A − F C] is a stable matrix and the system (12.26) F =K is detectable. Likewise, we see that if the dual system (12.27) is detectable, then the system (12.26) is stabilizable. Consequently, like controllability and observability, stabilizability and detectability are dual concepts for autonomous systems. Consider the example above defined by the system        d x1 (t) 0 x1 (t) −1 0 u(t). (12.28) + = 1 x (t) 0 1 x (t) dt 2 2   Recall that (12.28) is not controllable. However, let K = 0 4 and compute      0  −1 0 0 4 − [A − BK] = 1 0 1       −1 0 0 0 −1 0 − = . = 0 1 0 −3 0 4 Since the eigenvalues of

[A − BK] =



−1 0 0 −3



are λ1 = −1 and λ2 = −3, [A − BK] is a stable matrix and hence (12.28) is stabilizable. Thus, uncontrollable systems can be stabilizable. Likewise, there are detectable systems that are not observable so that stabilizability and detectability are weaker conditions than controllability and observability.

12.2

Linear Control Systems Arising from Nonlinear Problems

Although linear control systems occur naturally in many applications, linear systems also play a central role in control problems governed by nonlinear systems. In some cases, linearization offers the only practical approach to the analysis and design of control systems. Also, of immense importance is the issue of robustness and sensitivity of optimal controllers with respect to parameters and disturbances. We briefly discuss these processes below.

474

Chapter 12. Linear Control Systems

12.2.1

Linearized Systems

Consider the case where u ¯(·) ∈ P W C(t0 , t1 ) and x ¯(·) ∈ P W S(t0 , t1 ) is the response to the nonlinear control system (N S)

d x ¯(t) = f (¯ x(t), u ¯(t)), dt

(12.29)

with initial condition x ¯(t0 ) = x ¯0 . We call u ¯(·) the nominal control and x ¯(·) the nominal trajectory. In many cases the nominal control could be an optimal controller for a given problem, but for the moment we simply assume it is a given control. Let u(t) = u ¯(t) + v(t) where we assume that v(t) is “small” and let x(t) = x ¯(t) + z(t) be the response to the control u(t) = u ¯(t) + v(t) with initial condition x(t0 ) = x ¯(t0 ) + z(t0 ) = x ¯0 + z 0 . Assuming that both v(t) and z 0 are small, one would expect that z(t) is small. Applying Taylor’s theorem we have d d d x(t) = x ¯(t) + z(t) = f (¯ x(t) + z(t), u ¯(t) + v(t)) dt dt dt = f (¯ x(t), u ¯(t)) + [Jx f (¯ x(t), u ¯(t))]z(t) + [Ju f (¯ x(t), u ¯(t))]v(t) + HOT (t). Here, [Jx f (¯ x(t), u ¯(t))] and [Ju f (¯ x(t), u ¯(t))] are the Jacobian matrices and the higher order terms HOT (t) are “small” with respect to the control v(t) and response z(t). Hence, d d x ¯(t) + z(t) = f (¯ x(t), u ¯(t)) + [Jx f (¯ x(t), u ¯(t))]z(t) dt dt + [Ju f (¯ x(t), u ¯(t))]v(t) + HOT (t),

475

Chapter 12. Linear Control Systems and since

d x ¯(t) = f (¯ x(t), u ¯(t)), dt

it follows that d z(t) = [Jx f (¯ x(t), u ¯(t))]z(t) + [Ju f (¯ x(t), u ¯(t))]v(t) dt + HOT (t). Therefore, the “variation” z(t) is an approximate solution to the linear system d z(t) = [Jx f (¯ x(t), u ¯(t))]z(t) + [Ju f (¯ x(t), u ¯(t))]v(t) dt with input v(t). Defining A(t) , [Jx f (¯ x(t), u ¯(t))] and B(t) , [Ju f (¯ x(t), u ¯(t))] leads to the linearized system z(t) ˙ = A(t)z(t) + B(t)v(t).

(12.30)

The theory of ordinary differential equations can be used to provide a rigorous basis of the linearization (see [58], [138] and [153]). In the special case where u ¯(t) ≡ u ¯ and x ¯(t) ≡ x ¯ are constants, then A , [Jx f (¯ x, u ¯)] and B , [Ju f (¯ x, u ¯)] are time independent and the linearized system (12.30) is an autonomous system z(t) ˙ = Az(t) + Bv(t).

12.2.2

(12.31)

Sensitivity Systems

Assume the control is parameterized, say u(t) = u(t, q) where q ∈ Rp . For a fixed parameter q¯ let the nominal control be defined by u ¯(t) = u(t, q) ¯ with nominal trajectory x ¯(t) = x(t, q). ¯ The raw sensitivity of x(t, q) at q = q¯ is defined by the partial derivative ∂x(t, q) ¯ = ∂q x(t, q) ¯ = xq (t, q)|q=q¯ = xq (t, q). ¯ ∂q

476

Chapter 12. Linear Control Systems

Observe that for a fixed tˆ, the raw sensitivity is nothing more than the derivative of the nonlinear mapping G(tˆ) : Rp −→ Rn defined by G(tˆ)q , x(tˆ, q). Hence, for each t and q ¯ the derivative ∂q x(t, q) ¯ is a linear operator p n ∂q x(t, q) ¯ : R −→ R which has the matrix representation  ∂x1 (t,q)  ¯ ∂x1 (t,q) ¯ ∂x1 (t,q) ¯ · · · ∂q ∂q2 ∂qp  ∂x2 (t,1 q) ∂x2 (t,q) ¯  ¯ 2 (t,q)  ∂q1 ¯ ∂x∂q  · · · ∂qp 2  . Jq x(t, q) ¯ = (12.32) .. .. ..  ... . . .   ∂xn (t,q) ¯ ∂q1

∂xn (t,q) ¯ ∂q2

···

∂xn (t,q) ¯ ∂qp

We are interested in computing the raw sensitivity as a function of time. Given a parameter q ∈ Rp and the control u(t) = u(t, q), then the corresponding trajectory x(t) = x(t, q) satisfies the nonlinear system (12.29) which explicitly has the form x(t, ˙ q) =

d x(t, q) = f (x(t, q), u(t, q)). dt

It is important to note that x(t, q) is a function of two variables t ˙ q) we mean and q so that when we write dtd x(t, q) or x(t, d ∂ x(t, q) = x(t, q) dt ∂t so that

∂ x(t, q) = f (x(t, q), u(t, q)). (12.33) ∂t Consequently, taking the partial derivatives of both sides of (12.33) with respect to q yields   ∂ ∂ ∂ x(t, q) = [f (x(t, q), u(t, q))] ∂q ∂t ∂q

477

Chapter 12. Linear Control Systems and applying the chain rule we obtain   ∂ ∂ ∂ x(t, q) = [f (x(t, q), u(t, q))] ∂q ∂t ∂q ∂ ∂ f (x(t, q), u(t, q)) ◦ x(t, q) = ∂x ∂q ∂ ∂ u(t, q), + f (x(t, q), u(t, q)) ◦ ∂u ∂q

where “◦” denotes function composition. Assuming that x(t, q) is smooth in both parameters, we can interchange the order of integration to produce   ∂ ∂ ∂ x(t, q) = [f (x(t, q), u(t, q))] ∂t ∂q ∂q ∂ ∂ f (x(t, q), u(t, q)) ◦ x(t, q) = ∂x ∂q ∂ ∂ + f (x(t, q), u(t, q)) ◦ u(t, q). ∂u ∂q In terms of the matrix representations of the operators, we have that ∂ [Jq x(t, q)] = [Jx f (x(t, q), u(t, q))] [Jq x(t, q)] ∂t + [Ju f (x(t, q), u(t, q))] [Jq u(t, q)] . Again, we define ¯ , Jx f (x(t, q), ¯ , Ju f (x(t, q), A(t) ¯ u(t, q)) ¯ and B(t) ¯ u(t, q)) ¯ and let s(t, q) ¯ , Jq x(t, q). ¯ It now follows that the sensitivity (matrix) s(t, q) ¯ satisfies the linear matrix equation ∂ ¯ ¯ [Jq u(t, q)] [s(t, q)] ¯ = A(t)s(t, q) ¯ + B(t) ¯ , ∂t where Jq u(t, q) is the Jacobian matrix for u(t, q).

(12.34)

478

Chapter 12. Linear Control Systems

Therefore, the linear matrix differential equation (12.34) defines the dynamics of the time evolution of the sensitivities s(t, q) , Jq x(t, q) at q = q. ¯ Equation (12.34) is called the Continuous Sensitivity Equation. If we introduce the additional notation S(t) , s(t, q) ¯ = Jq x(t, q) ¯ and V (t) , Jq u(t, q), ¯ then the continuous sensitivity equation has the form ¯ ¯ ˙ S(t) = A(t)S(t) + B(t)V (t).

(12.35)

Observe that if the initial state x ¯(t0 ) = x ¯0 does not depend on q, then the initial condition for the continuous sensitivity equation (12.35) is given by S(t0 ) = 0. However, if x ¯(t0 ) = x ¯0 (q) also depends on the parameter q, then the appropriate initial condition for the matrix sensitivity equation (12.35) is given by S(t0 ) = Jq x0 (q). ¯ (12.36) Even in the special case where x(t) = x(t, q) is a scalar, the sensitivity s(t, q) ¯ is a vector, s(t, q) ¯ , Jq x(t, q) ¯ = ∇q x(t, q) ¯ and the sensitivity system has the form ∂ ¯ ¯ [∇q u(t, q)] [∇q x(t, q)] ¯ = A(t)[∇ ¯ + B(t) ¯ . q x(t, q)] ∂t

12.3

Linear Quadratic Optimal Control

In the previous sections we saw where linear control systems occur naturally as the basic model, from linearization of a nonlinear system and as the sensitivity equations for a nonlinear problem. In

Chapter 12. Linear Control Systems

479

many cases these systems can be time dependent (nonautonomous) even if the nonlinear system is autonomous. In this section we focus on the LQ optimal control problem with time varying matrix coefficients and a terminal cost. Thus, we consider nonautonomous problems of Bolza type. Most of the following material can be found in the standard books [6], [5], [16], [45], [71] and [73], [81], [115] and [187]. We begin with the LQ optimal control problem defined on a fixed time interval [t0 , t1 ] which is governed by the system x(t) ˙ = A(t)x(t) + B(t)u(t), x(t0 ) = x0

(12.37)

with cost function of Bolza type J(u(·), x0 ) = hSx(t1 ), x(t1 )i Z t1 [hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i]ds. + t0

(12.38)

Here, we assume that S = S T ≥ 0, Q(t) = Q(t)T ≥ 0 and R(t) = R(t)T ≥ αIm×m > 0 for some α > 0. The results in Section 11.8 can be extended to this Bolza problem. In particular, a direct combination of the Maximum Principle for the Problem of Bolza (Theorem 11.3) and the Maximum Principle for nonautonomous systems (Theorem 11.4) yields the following result. Theorem 12.7 If an optimal control exists, then the optimality conditions are given by = A(t)x∗ (t) + [B(t)][R(t)]−1 [B(t)]T λ∗ (t), x∗ (t0 ) = x0 , = Q(t)x∗ (t) − [A(t)]T λ∗ (t), λ∗ (t1 ) = −Sx∗ (t1 ), (12.39) where the optimal control is defined by d ∗ x (t) dt d ∗ λ (t) dt

u∗ (t) = R−1 (t)B T (t)λ∗ (t).

(12.40)

We can write the optimality system as     ∗  d x∗ (t) A(t) [B(t)][R(t)]−1 [B(t)]T x (t) , = ∗ λ∗ (t) Q(t) −[A(t)]T dt λ (t) (12.41)

480

Chapter 12. Linear Control Systems

with boundary conditions ∗

x (t0 ) = and ∗



In×n 0n×n

λ (t1 ) =







x∗ (t0 ) λ∗ (t0 )

−S 0n×n







= x0 ,

(12.42)



(12.43)

x∗ (t1 ) λ∗ (t1 )

.

Thus, if one solves the two point boundary value problem (12.41) - (12.43), the optimal control is defined by (12.40). We will show that the optimal control has the form u∗ (t) = −K(t)x∗ (t),

(12.44)

where the gain matrix K(t) is given by K(t) = R(t)−1 B(t)T P (t)

(12.45)

and P (t) is a non-negative definite solution to the Riccati differential equation (RDE) − P˙ (t) = P (t)A(t) + A(t)T P (t) − P (t)B(t)R(t)−1 B(t)T P (t) + Q(t) (12.46) with final data P (t1 ) = S. (12.47) In addition, the optimal cost is given by J(u(·), x0 ) = hP (t0 )x(t0 ), x(t0 )i .

(12.48)

In order to establish this result, we follow the basic approach first used in Sections 11.5 and 11.8.

12.4

The Riccati Differential Equation for a Problem of Bolza

Here we consider the Bolza type LQ optimal control problem for the general nonautonomous system discussed in the previous section. We extend the results on Riccati equations presented in Section 11.5.2 to this case. Again, the starting point is the optimality

481

Chapter 12. Linear Control Systems

system defined by the two point boundary value problem (12.41) (12.43) and the goal is to show that one can transform the “state variable” to the “adjoint variable” by a matrix that satisfies a Riccati differential equation. First write (12.41) as the linear system      d x(t) A(t) B(t)[R(t)]−1 [B(t)]T x(t) = Q(t) −[A(t)]T λ(t) dt λ(t)   x(t) (12.49) , H(t) λ(t) where H(t) is the 2n × 2n matrix   A(t) B(t)[R(t)]−1 [B(t)]T H(t) = . Q(t) −[A(t)]T

(12.50)

Let Ψ(t, τ ) be the fundamental matrix for H(t). In particular, Ψ(t, τ ) satisfies the matrix differential equation d Ψ(t, τ ) = H(t)Ψ(t, τ ), dt

(12.51)

with initial conditions Ψ(τ, τ ) = I2n×2n .

(12.52)

The solution to (12.41) - (12.43) has the form       x(t1 ) x(t1 ) x(t) = Ψ(t, t1 ) , = Ψ(t, t1 ) −Sx(t1 ) λ(t1 ) λ(t) since λ(t1 ) = −Sx(t1 ). Setting   ψ11 (t, τ ) ψ12 (t, τ ) , Ψ(t, τ ) = ψ21 (t, τ ) ψ22 (t, τ ) it follows that     x(t1 ) x(t) = Ψ(t, t1 ) −Sx(t1 ) λ(t)    x(t1 ) ψ11 (t, t1 ) ψ12 (t, t1 ) = −Sx(t1 ) ψ21 (t, t1 ) ψ22 (t, t1 )

482

Chapter 12. Linear Control Systems

so that x(t) = ψ11 (t, t1 )x(t1 ) − ψ12 (t, t1 )Sx(t1 ) = [ψ11 (t, t1 ) − ψ12 (t, t1 )S]x(t1 ),

(12.53)

λ(t) = ψ21 (t − t1 )x(t1 ) − ψ22 (t, t1 )Sx(t1 ) = [ψ21 (t − t1 ) − ψ22 (t, t1 )S]x(t1 ).

(12.54)

and

If [ψ11 (t, t1 ) − ψ12 (t, t1 )S] is non-singular for 0 ≤ t ≤ t1 , then we can solve (12.53) for x(t1 ). In particular, x(t1 ) = [ψ11 (t, t1 ) − ψ12 (t, t1 )S]−1 x(t) which, when substituted into (12.54), yields λ(t) = [ψ21 (t − t1 ) − ψ22 (t, t1 )S][ψ11 (t, t1 ) − ψ12 (t, t1 )S]−1 x(t). If P (t) is the n × n matrix defined by P (t) , −[ψ21 (t − t1 ) − ψ22 (t, t1 )S][ψ11 (t, t1 ) − ψ12 (t, t1 )S]−1 , (12.55) then we have that λ(t) and x(t) are linearly related by the matrix P (t) and the relationship is given by λ(t) = −P (t)x(t). The choice of the negative sign in defining P (·) is made to be consistent with much of the existing literature. In order to make this step rigorous, one needs to prove that [ψ11 (t, t1 ) − ψ12 (t, t1 )S] is non-singular for t0 ≤ t ≤ t1 . On the other hand, we could simply ask the question: Is there a matrix P (t) so that λ(t) = −P (t)x(t) and how can P (t) be computed? We will address the issue of the existence of P (t) later. However, assume for the moment that x(·) and λ(t) satisfying (12.41) - (12.43) and λ(t) = −P (t)x(t), (12.56)

Chapter 12. Linear Control Systems

483

with P (t) differentiable. Differentiating the equation (12.56) one obtains     d d d λ(t) = − P (t) x(t) − P (t) x(t) dt dt dt   d P (t) x(t) =− dt   − P (t) A(t)x(t) + B(t)[R(t)]−1 [B(t)]T λ(t)   d =− P (t) x(t) dt   − P (t) A(t)x(t) − B(t)[R(t)]−1 [B(t)]T P (t)x(t)   d =− P (t) x(t) dt − P (t)A(t)x(t) + P (t)B(t)[R(t)]−1 [B(t)]T P (t)x(t). However, from (12.49) we have d λ(t) = Q(t)x(t) − [A(t)]T λ(t) dt = Q(t)x(t) + [A(t)]T P (t)x(t) so that  d P (t) x(t) − P (t)A(t)x(t) Q(t)x(t) + [A(t)] P (t)x(t) = − dt + P (t)B(t)[R(t)]−1 [B(t)]T P (t)x(t). T



Rearranging the terms we have   d P (t) x(t) = [A(t)]T P (t)x(t) + P (t)A(t)x(t) (12.57) − dt − P (t)B(t)[R(t)]−1 [B(t)]T P (t)x(t) + Q(t)x(t). Consequently, P (t) satisfies (12.57) along the trajectory x(t). Observe that (12.57) is satisfied for any solution of the system (12.49) with λ(t1 ) = −Sx(t1 ) and all values of x(t1 ). Therefore, if λ(t) = −P (t)x(t),

484

Chapter 12. Linear Control Systems

then P (t) satisfies the matrix Riccati differential equation − P˙ (t) = AT P (t) + P (t)A − P (t)BR−1 B T P (t) + Q, t0 ≤ t < t1, (12.58) with terminal condition P (t1 ) = S,

(12.59)

since −P (t1 )x(t1 ) = λ(t1 ) = −Sx(t1 )

and x(t1 ) can be any vector in Rn . We shall show below that under the assumption that there is a solution P (t) to the Riccati differential equation (12.58) satisfying (12.59), then the LQ optimal control problem has a solution and the optimal control is given by u∗ (t) = −[R(t)]−1 [B(t)]T P (t)x∗ (t).

(12.60)

In order to provide a rigorous treatment of this problem, we present two lemmas. These results relate the existence of a solution to the Riccati equation (12.58) to the existence of an optimal control for the LQ optimal control problem. First we note that any solution to the Riccati differential equation must be symmetric. In particular, P (t) = [P (t)]T for all t. Lemma 12.1 Suppose that P (t) = [P (t)]T is any n × n matrix function with P (t) differentiable on the interval [t0 , t1 ]. If u(·) ∈ P W C(t0 , t1 ; Rm ) and x(t) ˙ = A(t)x(t) + B(t)u(t), t0 ≤ t ≤ t1 ,

then

hP (s)x(s), x(s)i |tt10 Z t1 Dh i E P˙ (s) + P (s)A(s) + [A(s)]T P (s) x(s), x(s) ds = t0

+ +

Z

(12.61)

t1

t Z 0t1 t0

hP (s)B(s)u(s), x(s)i ds

 [B(s)]T P (s)x(s), u(s) ds

(12.62)

Chapter 12. Linear Control Systems

485

Proof: Observe that hP (s)x(s), x(s)i |tt10

Z

t1

d hP (s)x(s), x(s)i ds t0 ds Z t1 D E P˙ (s)x(s), x(s) ds = t0 Z t1 hP (s)x(s), ˙ x(s)i ds + t0 Z t1 hP (s)x(s), x(s)i ˙ ds +

=

t0

and by substituting Ax(s) + Bu(s) for x(s) ˙ we obtain Z t1 D E t1 hP (s)x(s), x(s)i |t0 = P˙ (s)x(s), x(s) ds t0 Z t1 + hP (s) [Ax(s) + Bu(s)] , x(s)i ds t0 Z t1 + hP (s)x(s), [Ax(s) + Bu(s)]i ds. t0

Simplifying this expression we obtain Z t1 D E t1 P˙ (s)x(s), x(s) ds hP (s)x(s), x(s)i |t0 = t0 Z t1 + hP (s) [A(s)x(s)] , x(s)i ds t0 Z t1 + hP (s) [B(s)u(s)] , x(s)i ds t0 Z t1 + hP (s)x(s), [A(s)x(s)]i ds t0 Z t1 + hP (s)x(s), [B(s)u(s)]i ds t0

486

Chapter 12. Linear Control Systems

and collecting terms yields hP (s)x(s), x(s)i |tt10 Z t1 D E T ˙ = [P (s) + P (s)A(s) + [A(s)] P (s)]x(s), x(s) ds t0 Z t1 hP (s)B(s)u(s), x(s)i ds + t0 Z t1

 [B(s)]T P (s)x(s), u(s) ds + t0

which establishes (12.62). 

Lemma 12.2 Assume that the Riccati differential equation (12.58) has a solution P (t) = [P (t)]T for t0 ≤ t < t1 and P (t1 ) = 0n×n . If u(·) ∈ P W C(t0 , t1 ; Rm ) and x(t) ˙ = A(t)x(t) + B(t)u(t), t0 ≤ t ≤ t1 , Rt then the cost function J(u(·)) = hSx(t1 ), x(t1 )i + t01 [hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i]ds has the representation Z t1

[R(s)]1/2 u(s) + [R(s)]−1/2 [B(s)]T P (s)x(s) 2 ds J(u(·)) = t0

+ hP (t0 )x(t0 ), x(t0 )i .

Proof: Let

N (x(·), u(·)) =

Z

t1

t0

[R(s)]1/2 u(s)+[R(s)]−1/2 [B(s)]T P (s)x(s) 2 ds

and expanding N (x(·), u(·)) we obtain Z t1

 N (x(·), u(·)) = [R(s)]1/2 u(s), [R(s)]1/2 u(s) ds t Z0 t1

 [R(s)]1/2 u(s), [R(s)]−1/2 [B(s)]T P (s)x(s) ds + t Z 0t1

 [R(s)]−1/2 [B(s)]T P (s)x(s), [R(s)]1/2 u(s) ds + t Z0 t1

[R(s)]−1/2 [B(s)]T P (s)x(s), [R(s)]−1/2 + t0  × [B(s)]T P (s)x(s) ds.

Chapter 12. Linear Control Systems

487

Simplifying each term we have N (x(·), u(·)) = Z t1

 [R(s)]1/2 [R(s)]1/2 u(s), u(s) ds t0 Z t1

 + u(s), [R(s)]1/2 [R(s)]−1/2 [B(s)]T P (s)x(s) ds t Z 0t1

 + [R(s)]1/2 [R(s)]−1/2 [B(s)]T P (s)x(s), u(s) ds t Z 0t1

 + [R(s)]−1/2 [R(s)]−1/2 [B(s)]T P (s)x(s), [B(s)]T P (s)x(s) ds, t0

which implies

N (x(·), u(·)) =

Z

t1

hR(s)u(s), u(s)i ds

t0

+ +

Z

t

t1

Z 0t1 t

+

Z 0t1 t0

or equivalently N (x(·), u(·)) =

Z





 u(s), [B(s)]T P (s)x(s) ds  [B(s)]T P (s)x(s), u(s) ds

 [R(s)]−1 [B(s)]T P (s)x(s), [B(s)]T P (s)x(s) ds,

t1

hR(s)u(s), u(s)i

t0

+ +

Z

t1

t Z 0t1 t

+

Z 0t1 t0



 u(s), [B(s)]T P (s)x(s) ds



 P (s)B(s)[R(s)]−1 [B(s)]T P (s)x(s), x(s) ds.



(12.63)

 [B(s)]T P (s)x(s), u(s) ds

Since the matrix P (s) satisfies the Riccati equation (12.58), it

488

Chapter 12. Linear Control Systems

follows that P (s)B(s)[R(s)]−1 [B(s)]T P (s)x(s) i h = P˙ (s) + [A(s)]T P (s) + P (s)A(s) + Q(s) x(s)

and the last term above becomes Z t1 Dh i E P˙ (s) + [A(s)]T P (s) + P (s)A(s) + Q(s) x(s), x(s) ds. t0

Substituting this expression into (11.131) and rearranging yields N (x(·), u(·)) = Z t1 Z t1 hR(s)u(s), u(s)i ds + hQ(s)x(s), x(s)i ds t0 t0 Z t1 D h i E + P˙ (s) + [A(s)]T P (s) + P (s)A(s) x(s), x(s) ds t Z 0t1

 + u(s), [B(s)]T P (s)x(s) ds t Z 0t1

 [B(s)]T P (s)x(s), u(s) ds, + t0

which implies

N (x(·), u(·)) = J(u(·)) − hSx(t1 ), x(t1 )i Z t1 D h i E + P˙ (s) + [A(s)]T P (s) + P (s)A(s) x(s), x(s) ds t Z 0t1

 u(s), [B(s)]T P (s)x(s) ds + t Z 0t1

 [B(s)]T P (s)x(s), u(s) ds. + t0

Applying (12.62) from the previous Lemma yields

N (x(·), u(·)) = J(u(·)) − hSx(t1 ), x(t1 )i + hP (s)x(s), x(s)i |tt10 = J(u(·)) − hSx(t1 ), x(t1 )i + hP (t1 )x(t1 ), x(t1 )i − hP (t0 )x(t0 ), x(t0 )i ,

489

Chapter 12. Linear Control Systems or equivalently

J(u(·)) = N (x(·), u(·)) + hSx(t1 ), x(t1 )i − hP (t1 )x(t1 ), x(t1 )i + hP (t0 )x(t0 ), x(t0 )i . Since (12.59) holds, we have that P (t1 ) = S and hence J(u(·)) = N (x(·), u(·)) + hSx(t1 ), x(t1 )i − hSx(t1 ), x(t1 )i + hP (t0 )x(t0 ), x(t0 )i = N (x(·), u(·)) + hP (t0 )x(t0 ), x(t0 )i . We conclude that J(u(·)) =

Z

t1 t0

1/2

R u(s) + R−1/2 B T P (s)x(s) 2 ds

+ hP (t0 )x(t0 ), x(t0 )i

(12.64)

which completes the proof.  We now have the fundamental result on the relationship between solutions to the Riccati equation and the existence of an optimal control for the LQ optimal problem. Theorem 12.8 (Existence of LQ Optimal Control) If the Riccati differential equation (12.58) has a solution P (t) = [P (t)]T for t0 ≤ t < t1 and P (t1 ) = S, then there is a control u∗ (·) ∈ P W C(t0 , t1 ; Rm ) such that u∗ (·) minimizes J(u(·)) = hSx(t1 ), x(t1 )i Z t1 {hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i}ds + t0

on the set P W C(t0 , t1 ; Rm ). In addition, the optimal control is a linear feedback law u∗ (t) = −[R(t)]−1 [B(t)]T P (t)x∗ (t)

(12.65)

and the minimum value of J(·) is J(u∗ (·)) = hP (t0 )x(t0 ), x(t0 )i .

(12.66)

490

Chapter 12. Linear Control Systems

Proof : The identity (12.64) above implies that J(·) is minimized when the quadratic term Z t1

[R(s)]1/2 u(s) + [R(s)]−1/2 [B(s)]T P (s)x(s) 2 ds ≥ 0 t0

is minimized. If u∗ (s) = −[R(s)]−1 [B(s)]T P (s)x∗ (s), then [R(s)]1/2 u∗ (s) + [R(s)]−1/2 [B(s)]T P (s)x∗ (s) = 0 and ∗

J(u (·)) =

Z

t1 t0



[R(s)]1/2 u∗ (s) + [R(s)]−1/2 [B(s)]T P (s)x∗ (s) 2 ds

+ hP (t0 )x∗ (t0 ), x∗ (t0 )i = hP (t0 )x∗ (t0 ), x∗ (t0 )i .

Consequently, if u(·) ∈ P W C(t0 , t1 ; Rm ), then J(u∗ (·)) = hP (t0 )x0 , x0 i ≤ hP (t0 )x0 , x0 i Z t1

[R(s)]1/2 u(s) + [R(s)]−1/2 [B(s)]T P (s)x(s) 2 ds + t0

= J(u(·)),

which completes the proof.  The important observation here is that the optimal control is linear state feedback with gain operator K(t) defined by (12.45). Thus, to implement the control law one needs the full state x∗ (t). However, when the full state is not available one must develop a method to estimate those states that can not be sensed. This leads to the construction of state estimators or so-called observers.

12.5

Estimation and Observers

Assume that the system is described by the differential equation x(t) ˙ = A(t)x(t) + B(t)u(t),

x(t0 ) = x0 ∈ Rn ,

t ∈ [t0 , t1 ] (12.67)

Chapter 12. Linear Control Systems

491

with sensed or measured output y(t) = C(t)x(t) ∈ Rp .

(12.68)

If one has a feedback control law of the form u(t) = −K(t)x(t)

(12.69)

for some control gain matrix K(t), then in order to compute the control u(t) at a time t one must have the full state x(t) at time t. Since we can only sense y(t) and the control law requires x(t), we are led to the problem of finding a method to estimate x(t) from the data y(t). In particular, we look for a system of the form x˙ e (t) = A(t)xe (t) + F (t) [y(t) − C(t)xe (t)] + B(t)u(t), xe (t0 ) = xe,0 ∈ Rn , (12.70) such that the solution xe (t) provides an estimate of the state x(t) on the interval of interest. Thus, we seek an observer gain matrix F (t) so that the system (12.70) which is driven by the measured (sensed) output y(t) produces values xe (t) that are as close to x(t) as possible. In this case, one replaces the full state feedback law (12.69) with u(t) = ue (t) = −K(t)xe (t).

(12.71)

In order for this to work and to be practical, one needs to know that xe (t) ≃ x(t) and that the equation (12.70) can be solved in real time. Under suitable conditions we shall say that the system (12.70) is an observer (or state estimator) for the system (12.67)-(12.68). The selection of the system (12.70) as the form of an observer is not arbitrary, but follows naturally. A “good” observer should have the same structure as the system and be driven by the measured output y(t) = C(t)x(t). Moreover, we require that if the initial data for the estimator is exactly the initial data for the system, then the estimated state should equal the actual state. Therefore, if xe (t0 ) = xe,0 = x0 = x(t0 ), then xe (t) = x(t) for all t ≥ t0 . Since the system (12.67)-(12.68) is linear, it is natural to think that the

492

Chapter 12. Linear Control Systems

observer equation should also be linear so that an observer would have the form xe (t0 ) = xe,0 ∈ Rn , (12.72) for some matrices Ae (t), F (t) and G(t). The corresponding error e(t) , xe (t) − x(t) would satisfy the non-homogenous linear differential equation x˙ e (t) = Ae (t)xe (t) + F (t)y(t) + G(t)u(t),

e(t) ˙ = x˙ e (t) − x(t) ˙ = Ae (t)xe (t) + F (t)y(t) + G(t)u(t) − A(t)x(t) − B(t)u(t) = Ae (t)xe (t) − Ae (t)x(t) + Ae (t)x(t) +F (t)y(t) − A(t)x(t) + [G(t) − B(t)]u(t) = Ae (t)[xe (t) − x(t)] +Ae (t)x(t) + F (t)y(t) − A(t)x(t) + [G(t) − B(t)]u(t) = Ae (t)e(t) + Ae (t)x(t) + F (t)C(t)x(t) − A(t)x(t) +[G(t) − B(t)]u(t) = Ae (t)e(t) + [Ae (t) + F (t)C(t) − A(t)]x(t) +[G(t) − B(t)]u(t) with initial data given by the error vector e(t0 ) = xe (t0 ) − x(t0 ). The requirement that if e(t0 ) = xe (t0 ) − x(t0 ) = 0, then e(t) = xe (t) − x(t) ≡ 0 for all states x(t) and controls u(t) implies that the matrices [Ae (t) + F (t)C(t) − A(t)] and [G(t) − B(t)] should be zero. Thus, [Ae (t) + F (t)C(t) − A(t)] = 0

and

which implies that

[G(t) − B(t)] = 0, Ae (t) = A(t) − F (t)C(t)

and G(t) = B(t),

493

Chapter 12. Linear Control Systems

respectively. Therefore, the observer (12.72) must have the form x˙ e (t) = Ae (t)xe (t) + F (t)y(t) + G(t)u(t) = [A(t) − F (t)C(t)]xe (t) + F (t)y(t) + B(t)u(t) = A(t)xe (t) + F (t)[y(t) − C(t)xe (t)] + B(t)u(t) which is the form of the observer (12.70). Let e(t) , xe (t) − x(t) denote the error vector and note that e(t) ˙ = x˙ e (t) − x(t) ˙ = [A(t)xe (t) + F (t) [y(t) − C(t)xe (t)] + B(t)u(t)] −[A(t)x(t) + B(t)u(t)] = A(t)[xe (t) − x(t)] + F (t)C(t)x(t) − F (t)C(t)xe (t) = [A(t) − F (t)C(t)][xe (t) − x(t)] = [A(t) − F (t)C(t)]e(t). Hence, the error satisfies the linear differential equation e(t) ˙ = [A(t) − F (t)C(t)]e(t)

(12.73)

e(t0 ) = xe (t0 ) − x(t0 ) = xe,0 − x0 .

(12.74)

with initial data

Since the error equation (12.73) is linear and homogenous, if the error in the initial estimate xe,0 of x0 is zero (i.e. xe,0 = x0 ), then e(t) = xe (t) − x(t) ≡ 0 and the estimate of x(t) is exact as desired. Moreover, it follows that e(t) = ΨF (t, t0 )e(t0 ), where Ψ(t, τ ) is the fundamental matrix operator for (12.73). In particular, ΨF (t, τ ) satisfies the matrix differential equation d ΨF (t, τ ) = [A(t) − F (t)C(t)]ΨF (t, τ ) dt

494

Chapter 12. Linear Control Systems

with initial condition ΨF (τ, τ ) = In×n . This leads to a bound on the error of the form ke(t)k = kΨF (t, t0 )e(t0 )k ≤ kΨF (t, t0 )k ke(t0 )k

(12.75)

for t0 ≤ t ≤ t1 . The selection of the observer gain F (t) determines this error and hence one approach might be to select F (t) so that sup kΨF (t, t0 )k

t0 ≤t≤t1

is minimized. We shall return to this issue later. There are many approaches to “designing” a good observer, but we focus on the Luenberger Observer and the Kalman Filter.

12.5.1

The Luenberger Observer

As noted above, many optimal controllers require full state feedback which could be implemented if all of x(t) were available. However, in most complex systems the entire state is not available through sensing and hence such control laws cannot be implemented in practice. Thus, one must either consider an alternate form of the control or else devise a method for approximating the state x(t) using only knowledge about the system dynamics and sensed measurements. In 1964 Luenberger first considered this problem from a deterministic point of view (see [130], [132] and [129]). Although Luenberger considered only time invariant systems, the main results can be modified and extended to nonautonomous and even non-linear systems. One of the basic results concerns the question of observing an output to the uncontrolled system given by x(t) ˙ = A(t)x(t),

x(t0 ) = x0

(12.76)

using only sensed measurements y(t) = C(t)x(t),

(12.77)

495

Chapter 12. Linear Control Systems

where A(t) and C(t) are given n × n and p × n matrix functions, respectively. The idea is to use the output y(t) to drive another linear system z(t) ˙ = Ae (t)z(t) + F (t)y(t),

z(t0 ) = z 0 ,

(12.78)

so that one matches a desired output r(t) = T (t)x(t),

(12.79)

where T (t) is a given q × n matrix function. Observe that if Ae (t) is a l × l matrix, then F (t) must be a l × p matrix since there are p measured outputs. Moreover, since F (t)y(t) = F (t)C(t)x(t) then H(t) , F (t)C(t) must be a l × n matrix. The goal is to find Ae (t) and F (t) so that if z(t0 ) = r(t0 ) = T (t0 )x(t0 ), then z(t) = r(t) for all t > t0 . Clearly this implies that l = q so that the size of system (12.78) is q. The following result is due to Luenberger and may be found in [132]. Theorem 12.9 Let x(t) be a solution to the system (12.76) which drives the system (12.78). Suppose the matrix function T (t) is differentiable and there exist Ae (t) and F (t) such that T˙ (t) + [T (t)A(t) − Ae (t)T (t)] = H(t) = F (t)C(t)

(12.80)

on the interval t0 < t < t1 . If z(t0 ) = r(t0 ) = T (t0 )x(t0 ), then z(t) = r(t) = T (t)x(t) for all t0 < t < t1 . Proof : Let the error vector be defined by e(t) = z(t) − r(t) = z(t) − T (t)x(t) and note that d [T (t)x(t)] = z(t) ˙ − T˙ (t)x(t) − T (t)x(t) ˙ dt = Ae (t)z(t) + F (t)y(t) − T˙ (t)x(t) − T (t)[A(t)x(t)] = Ae (t)z(t) + F (t)C(t)x(t) − T˙ (t)x(t) − T (t)A(t)x(t)

e(t) ˙ = z(t) ˙ −

= Ae (t)z(t) + H(t)x(t) − T˙ (t)x(t) − T (t)A(t)x(t).

496

Chapter 12. Linear Control Systems

However, since H(t) = T˙ (t) + T (t)A(t) − Ae (t)T (t) it follows that e(t) ˙ = Ae (t)z(t) + [T˙ (t) + T (t)A(t) − Ae (t)T (t)]x(t) − T˙ (t)x(t) − T (t)A(t)x(t) = Ae (t)z(t) − Ae (t)T (t)x(t) = Ae (t)[z(t) − T (t)x(t)] = Ae (t)e(t).

Therefore, e(t) is given by e(t) = Ψe (t, t0 )e(t0 ) where Ψe (t, s) is the state transition matrix satisfying ∂ Ψe (t, s) = Ae (t)Ψe (t, s), ∂t Hence we have

Ψe (s, s) = Iq×q .

z(t) − T (t)x(t) = Ψe (t, t0 )[z(t0 ) − T (t0 )x(t0 )], so that z(t) = T (t)x(t) + Ψe (t, t0 )[z 0 − r 0 ].

(12.81)

Consequently, if z(t0 ) = r(t0 ) then z(t) = r(t) and this completes the proof.  Observe that equation (12.81) implies that the error e(t) = z(t) − r(t) = z(t) − T (t)x(t) is equal to Ψe (t, t0 )[z 0 − r 0 ] so that ke(t)k = kz(t) − T (t)x(t)k = kΨe (t, t0 )[z 0 − r 0 ]k ≤ kΨe (t, t0 )k kz 0 − r 0 k

is bounded by the norm of the transition matrix. Consequently,   kz(t) − T (t)x(t)k ≤ sup kΨe (t, t0 )k kz 0 − r 0 k . t0 ≤t≤t1

Since Ψe (t, t0 ) depends on Ae (t), and hence implicitly on F (t) through the equation (12.80), the error can be made “small” if one can choose Ae (t) and F (t) satisfying (12.80) such that sup kΨe (t, t0 )k

t0 ≤t≤t1

is small. Note that in the special autonomous case one has the following result which was first given in [130].

497

Chapter 12. Linear Control Systems

Corollary 12.1 Let x(t) be a solution to the system x(t) ˙ = Ax(t) which drives the system z(t) ˙ = Ae z(t)+F y(t) where y(t) = Cx(t). Suppose there exist Ae and F such that [T A − Ae T ] = H = F C.

(12.82)

If z(t0 ) = r(t0 ) = T x(t0 ), then z(t) = r(t) = T x(t) for all t0 < t < t1 . Remark 12.3 It is important to note that for a given T (t) (or T in the time invariant case) it may not be possible to find Ae (t) and F (t) satisfying (12.80) (or (12.82)). See [130], [132] and [129] for more details. Luenberger actually posed the following question: Given H = F C and Ae , then what conditions on T will insure that z(t0 ) = r(t0 ) = T x(t0 ) implies z(t) = r(t) = T x(t) for all t0 < t < t1 ? Clearly, the answer is that T must satisfy [T A − Ae T ] = H. Remark 12.4 Note that if T is an n × n nonsingular matrix so that T −1 exists, then given any F one can solve (12.82) for Ae . In particular, Ae T = T A − H = T A − F C and hence Ae = [T A − H]T −1 = T AT −1 − F CT −1 satisfies (12.82). If T = In×n , then Ae = A − F C and z(t) ˙ = Ae z(t) + F y(t) is called the identity observer. We will return to these issues later and consider related problems of asymptotic tracking. However, we turn now to a stochastic version of this problem which produces an “optimal observer” called the Kalman Filter.

498

12.5.2

Chapter 12. Linear Control Systems

An Optimal Observer: The Kalman Filter

In order to fully develop this topic requires a background in stochastic systems and control and is outside the scope of these notes. However, Section 1.10 in the book [115] by Kwakernaak and Sivan contains a very nice and brief discussion of the basic material. However, it is essential that we discuss the definition of a stochastic process and present a few fundamental definitions and results. A Short Review of Random Variables and Stochastic Processes We review the concepts needed to precisely define a stochastic process. This requires we discuss measures and some measure theory concepts. We begin with the concept of a σ-algebra. Definition 12.5 Given a set Ω, F is called a σ-algebra of subsets of Ω if F is a collection of subsets of Ω and: (1) F is non-empty: i.e. there is at least one set E ⊆ Ω in F, (2) F is closed under complementation: i.e. if E is in Ω, then so is its complement Ω\E, where Ω\E is defined by Ω\E , {x ∈ Ω : x ∈ / E} , (3) F is closed under countable unions: i.e. if E1 , E2 , E3 , . . . are +∞ S Ei is in F. in F, then i=1

Definition 12.6 A pair (Ω, F), where Ω is a set and F is a σalgebra of subsets of Ω, is called a measurable space. A function µ : F → [0, +∞] is a (countably additive) measure on (Ω, F) provided that (1) µ(∅) = 0, where ∅ is the empty subset of Ω; (2) if Ei ∈ F for each i = 1, 2, 3, . . . are disjoint sets, then µ

+∞ S i=1

Ei



=

+∞ X i=1

µ(Ei ).

Chapter 12. Linear Control Systems

499

If µ : F → [0, +∞), then µ is called a finite measure. If Ω can be decomposed into a countable union of measurable sets of finite measure, then µ is called a σ-finite measure. The triple (Ω, F, µ) is called a measure space and sets in F are called measurable sets. Although these definitions may seem rather abstract, consider the simple example where Ω = I is a subinterval of the real line (I could be all of R1 ) and let FB be the “smallest” collection of sets containing all open and closed subintervals and their countable unions and intersections. The σ-algebra FB is called the Borel σ-algebra. If m : FB → [0, +∞) is the usual Lebesgue measure (see [176]), then (I, FB, m) is a measure space. If I is a finite subinterval, then µ is a finite measure. If I is an infinite interval, then µ is a σ-finite measure. This measure space plays the central role in the development of modern integration theory. We will make use of the standard measure space (Rn , FB, m) where again FB is the Borel σ-algebra on Rn and m is the usual Lebesgue measure. Definition 12.7 A probability space is a measure space (Ω, F, p) where the probability measure p satisfies (1) 0 ≤ p(E) ≤ 1; (2) p(Ω) = 1. The set Ω is called the sample space, F is called the event space and p is called the probability measure. Definition 12.8 Let (Ω1 , F1 , µ1 ) and (Ω2 , F2 , µ2 ) be measure spaces. A function X(·) : Ω1 → Ω2 is said to be a measurable function if for every measurable set S ∈ F2 , the inverse image X −1 (S) = {x ∈ Ω1 : X(x) ∈ S} is measurable, i.e. X −1 (S) ∈ F1 . If (Ω1 , F1 , µ1 ) = (Ω, F, p) is a probability space, then a measurable function X(·) : Ω → Ω2 is called a random variable. In particular, random variables are measurable functions from a probability space into a measure space.

500

Chapter 12. Linear Control Systems

Remark 12.5 Note that we have used capital letters to denote random variables. This is done to distinguish between the random variable X(·) and the value of this function at a particular x ∈ Ω. Thus, X(x) is not the same as X(·). This is the same notation we have consistently used before. For example, recall that we write u(·) ∈ P W S(t0 , t1 ) to mean that u(·) is a function and for a given fixed t ∈ [t0 , t1 ], u(t) is the value of u(·) at t. However, this precise notation is often abused so that one sees things like “consider the function u = t”. Of course we know that this means that u(·) is a function such that u(·) evaluated at t is just t or, u(t) = t. This same abuse of notation occurs in probability and statistics where you will see statements like “consider the random variable x”. What this really means is that one is talking about the random variable X(·) where X(x) = x. Definition 12.9 Let (Ω, F, p) be a probability space and (Rn , FB, m) be the standard Lebesgue measure space. Let X denote the set of all random variables X(·) : Ω → Rn . In particular, X = {X(·) : Ω → Rn : X(·) is measurable} . Let I be a subinterval of the real line. A (vector-valued) stochastic process on I is a family of Rn -valued random variables {[Xt ](·) ∈ X : t ∈ I} , indexed by t ∈ I. In particular, for each t ∈ I, Xt (·) is a random variable (i.e. a measurable function). Thus, a stochastic process is a function from I to X. We use the notation X(t, ·) = Xt (·) to indicate that for each fixed t ∈ I, X(t, ·) is the random variable with range in Rn . In this setting, we may consider a stochastic process as a function of two variables X(·, ·) : I × Ω → Rn and if ω ∈ Ω, we write X(t, ω) = Xt (ω).

501

Chapter 12. Linear Control Systems

If X(t, ·) = Xt (·) is a vector-valued stochastic process on I, then  T Xt (·) = [Xt ]1 (·) [Xt ]2 (·) [Xt ]3 (·) · · · [Xt ]n (·) ,

where each random variable [Xt ]i (·) is a real-valued random variable. Thus, we write  T Xt (·) = X(t, ·) = X1 (t, ·) X2 (t, ·) X3 (t, ·) · · · Xn (t, ·)

so that

Xt (ω) = X(t, ω)  = [X1 (t)](ω) [X2 (t)](ω) [X3 (t)](ω) · · ·

or equivalently,  X(t, ω) = X1 (t, ω) X2 (t, ω) X3 (t, ω) · · ·

[Xn (t)](ω)

Xn (t, ω)

T

T

.

At this point, one really starts to abuse notation. In particular, the statement that  T Vt (·) = [Vt ]1 (·) [Vt ]2 (·) [Vt ]3 (·) · · · [Vt ]n (·)

is a Rn -valued stochastic process on I, is often written by stating that  T v(t) = v1 (t) v2 (t) v3 (t) · · · vn (t)

is a Rn -valued stochastic process on I. It is important that one fully appreciates what is meant by this simplification of notation. In particular, always keep in mind that vi (t) is “representing” a random variable (i.e. a function) [Vt ]i (·). If this is well understood, then one has the background required to read Section 1.10 in the book [115]. The Kalman Filter and an Optimal Observer We return now to the problem of constructing an observer that is optimal in some sense. As noted above, one use of an observer is to provide state estimates from measured (sensed) outputs that can be used in a practical feedback law. In particular, u(t) = −K(t)x(t)

,

502

Chapter 12. Linear Control Systems

is replaced by ue (t) = −K(t)xe (t) and if xe (t) ≃ x(t), then one would expect that under suitable conditions the controllers satisfy ue (t) ≃ u(t). In this application, the selection of an observer for a given system depends on the choice of the gain matrix K(t). However, we can formulate an “optimal observer” problem independent of the control problem and focus on the problem of finding an optimal observer. In addition, this approach allows us to deal with some practical issues where the control system is subject to disturbances and the sensors produce noisy signals. This is done by making specific assumptions concerning these disturbances and measurement errors that occur in the system that is to be observed. We follow the approach in [115]. It is assumed that the system equations are described by the stochastic differential equation on t0 ≤ t ≤ T given by x(t) ˙ = A(t)x(t) + B(t)u(t) + G(t)w(t),

x(t0 ) = x0 ∈ Rn , (12.83)

with sensed output y(t) = C(t)x(t) + E(t)v(t) ∈ Rp ,

(12.84)

where w(t) and v(t) represent system disturbances and sensor noise, respectively. A Comment on Notation: If z is a random variable, then we use E for the expectation so that E [z] denotes the  Toperator  expected value of z and E zz is the covariance matrix. We assume that the disturbance w(t) and sensor noise v(t) are white, Gaussian, have zero mean and are independent. Mathematically, this implies that for all t, τ ∈ [t0 , T ],   E [v(t)] ≡ 0, (12.85) E w(t)w(τ )T = W (t)δ(t − τ ),   E [w(t)] ≡ 0, (12.86) E v(t)v(τ )T = V (t)δ(t − τ ),

and

  E v(t)w(τ )T = 0,

(12.87)

503

Chapter 12. Linear Control Systems

where W (t) = W (t)T ≥ 0 and V (t) = V (t)T ≥ 0 are the so-called intensity matrices. We define the n × n and matrices M (t) and N (t) by M (t) = G(t)W (t)[G(t)]T and N (t) = E(t)V (t)[E(t)]T , (12.88) respectively. In addition, we make the standing assumption that there is an α > 0 such that N (t) = E(t)V (t)[E(t)]T = N (t)T ≥ αI > 0.

(12.89)

The initial condition x0 is assumed to be a Gaussian random variable with mean m0 and covariance Π0 so that  (12.90) E [x0 − m0 ][x0 − m0 ]T = Π0 and E[x0 ] = m0 .

Finally, we assume that x0 is independent of the noise terms so that E[x0 v(t)T ] = E[x0 w(t)T ] = 0

for all t ∈ [t0 , T ].

(12.91)

Suppose we are seeking an observer of the form x˙ e (t) = A(t)xe (t) + B(t)u(t) + F (t) [y(t) − C(t)xe (t)] , xe (t0 ) = xe,0 ∈ Rn , (12.92) to minimize the expected value of the error between the state and the estimate of the state. In particular, let e(t) = xe (t) − x(t)

(12.93)

denote the error. The optimal estimation problem is to find the observer gain F (t) and the initial estimate xe,0 to minimize the weighted error E[hΣ(t)e(t), e(t)i] = E[e(t)T Σ(t)e(t)]  = E [xe (t) − x(t)]T Σ(t)[xe (t) − x(t)] , (12.94) where Σ(t) = Σ(t)T ≥ 0. The following theorem provides the answer to the optimal estimation problem and is known as the Kalman Filter.

504

Chapter 12. Linear Control Systems

Theorem 12.10 (Kalman Filter) The optimal observer is obtained by choosing the observer gain matrix to be F opt (t) = Π(t)C T (t)N (t)−1

(12.95)

where Π(t) = [Π(t)]T is the solution to the matrix Riccati differential equation ˙ Π(t) = A(t)Π(t) + Π(t)A(t)T − Π(t)C(t)T N (t)−1 C(t)Π(t) + M (t), (12.96) with initial condition Π(t0 ) = Π0 . (12.97) The optimal initial condition should be xe (t0 ) = m0 = E[x(t0 )] = E[x0 ].

(12.98)

If conditions (12.95) - (12.98) hold, then  E [xe (t) − x(t)][xe (t) − x(t)]T = Π(t),

(12.99)

while the mean square reconstruction error is  E[hΣ(t)e(t), e(t)i] = E [xe (t) − x(t)]T Σ(t)[xe (t) − x(t)] = trace [Π(t)Σ(t)] . (12.100) In addition to the properties given in Theorem 12.10, one can show that Π(t) is the “minimal” nonnegative solution to (12.96) in the following sense. If F (t) is any observer gain and Φ(t) is a solution to the Lyapunov equation ˙ Φ(t) =[A(t) − F (t)C(t)]Φ(t) + Φ(t)[A(t) − F (t)C(t)]T (12.101) + F (t)V (t)F (t)T + M (t)

with initial data (12.97), then 0 ≤ trace[Π(t)Σ(t)] ≤ trace[Φ(t)Σ(t)].

(12.102)

Let Φ(t) denote the variance matrix of the error e(t) and let em (t) denote the mean e(t) so that  E [e(t) − em (t)][e(t) − em (t)]T = Ψ(t) and E[e(t)] = em (t). (12.103)

505

Chapter 12. Linear Control Systems

It can be shown (see [115], Section 4.3) that em (t) satisfies the ordinary differential equation e˙ m (t) = [A(t) − F (t)C(t)]em (t) and Φ(t) is the solution to the Lyapunov equation (12.101) with initial condition Φ(t0 ) = Π0 . To see how this is related to the estimation problem, assume that F (t) = Φ(t)C T (t)N (t)−1

(12.104)

and Φ(t) satisfies the Lyapunov equation (12.101) above. Then, by substitution ˙ Φ(t) = [A(t) − F (t)C(t)]Φ(t) + Φ(t)[A(t) − F (t)C(t)]T + F (t)N (t)F (t)T + M (t)

= A(t)Φ(t) − [Φ(t)C T (t)N (t)−1 ]C(t)Φ(t) + Φ(t)A(t)T − Φ(t)[Φ(t)C T (t)N (t)−1 C(t)]T

+ [Φ(t)C T (t)N (t)−1 ]W (t)[Φ(t)C T (t)N (t)−1 ]T + M (t) = A(t)Φ(t) + Φ(t)A(t)T − Φ(t)C T (t)N (t)−1 C(t)Φ(t) − Φ(t)C(t)T N (t)−1 C(t)Φ(t)

+ [Φ(t)C T (t)N (t)−1 N (t)[N (t)−1 C(t)Φ(t)] + M (t)

= A(t)Φ(t) + Φ(t)A(t)T − 2Φ(t)C T (t)N (t)−1 C(t)Φ(t) + Φ(t)C T (t)N (t)−1 C(t)Φ(t) + M (t)

= A(t)Φ(t) + Φ(t)A(t)T − Φ(t)C T (t)V (t)−1 C(t)Φ(t) + M (t) and hence Φ(t) satisfies the observer Riccati differential equation (12.96). If Φ(t) satisfies the initial condition Φ(t0 ) = Π0 , then Φ(t) = Π(t) and we have the optimal filter. Note that the time averaged error is given by Z T Z T 1 1 E[hΣ(s)e(s), e(s)i]ds = trace [Π(s)Σ(s)] ds. T − t 0 t0 T − t 0 t0 (12.105) This cost has been used for optimal sensor location problems (see [10], [39], [50], [51], [49], [69], [70], [78], [152], [181]). A similar approach has been used for optimal actuator location problems (see [66], [139]).

506

12.6

Chapter 12. Linear Control Systems

The Time Invariant Infinite Interval Problem

In the case where t1 = +∞ and the linear system is time invariant, the linear quadratic optimal control problem takes the form x(t) ˙ = Ax(t) + Bu(t), x(0) = x0 , with cost functional Z +∞ {hQx(s), x(s)i + hRu(s), u(s)i} ds. J(u(·)) =

(12.106)

(12.107)

0

Here, Q = DT D ≥ 0, and R = RT > 0. The linear quadratic optimal control problem defined by (12.106)-(12.107) is called the Linear Quadratic Regulator Problem, or LQR Problem for short. If an optimal control exists, then one can show (see [115]) that it has the form of state feedback u∗ (t) = −Kopt x∗ (t),

(12.108)

where the constant gain matrix Kopt is given by Kopt = R−1 B T Popt

(12.109)

and Popt is a non-negative definite solution to the Algebraic Riccati Equation (ARE) P A + AT P − P BR−1 B T P + Q = 0.

(12.110)

The proof of this statement follows by considering LQ optimal control problems on 0 ≤ t1 < +∞ and then taking the limit as t1 → +∞ (see [115] and [119]). In particular, Popt is the minimal nonnegative solution to (12.110) in the sense that if P is any nonnegative definite solution of (12.110), then P ≥ Popt ≥ 0.

(12.111)

Moreover, the optimal cost is given by Z +∞ ∗ {hQx∗ (s), x∗ (s)i + hRu∗ (s), u∗ (s)i} dt J(u (·)) = 0

= hPopt x0 , x0 i ,

(12.112)

507

Chapter 12. Linear Control Systems where x(0) = x0 is the initial state. Some important questions are:

1. When does the LQR problem defined by (12.106)-(12.107) have a solution? 2. When is this solution unique? 3. How is this related to solutions of the ARE (12.110)? As noted above we consider the case where the weighting matrix Q can be factored as Q = DT D

(12.113)

for a q × n matrix D. Hence, we define the controlled output by z(t) = Dx(t),

(12.114)

where the matrix D represents the mapping D : Rn → Rq . Thus, the cost function (12.107) can be written as Z +∞ {hQx(s), x(s)iRn + hRu(s), u(s)i} ds J(u(·)) = 0 Z +∞  T  = D Dx(s), x(s) + hRu(s), u(s)i ds Z0 +∞ {hDx(s), Dx(s)i + hRu(s), u(s)i} ds = 0 Z +∞ n o hz(s), z(s)iRq + hRu(s), u(s)i ds. = 0

In order to guarantee the existence of an optimal controller we must place additional conditions on the system. The following result can be found many control theory books. In particular, see Section 3.4.3 in [115]. Theorem 12.11 (Existence for LQR Problem) If (A, B) is stabilizable and (A, D) is detectable, then the optimal LQR control problem has a unique solution and is given by (12.108)-(12.110). Moreover, there is only one non-negative definite solution Popt to (12.110) and (12.112) holds.

508

Chapter 12. Linear Control Systems

Now assume that the full state is not available for feedback. Thus, we have a sensed output y(t) = Cx(t) + Ev(t) ∈ Rp ,

(12.115)

and we must build an state estimator (observer). We will employ a Kalman filter of the form xe (t0 ) = xe,0 ∈ Rn , (12.116) where the disturbances v(t) and w(t) are white, gaussian, have zero mean and are independent. In particular, we assume that for all t > 0 and τ > 0,   E [v(t)] ≡ 0, (12.117) E v(t)v(τ )T = In×n δ(t − τ ),   E [w(t)] ≡ 0, (12.118) E w(t)w(τ )T = Ip×p δ(t − τ ), x˙ e (t) = Axe (t)+Bu(t)+F [y(t) − Cxe (t)] ,

and

  E v(t)w(τ )T = 0.

(12.119)

Hence, M = GGT and condition (12.89) implies N = N N T ≥ αIp×p > 0 so that the optimal filter is defined by the observer gain F and given by F = ΠC T N −1 , (12.120) where AΠ + ΠAT − ΠC T N −1 CΠ + M = 0.

(12.121)

Now consider the closed-loop system obtained by using the filter defined by (12.120) - (12.121). It follows that if one sets u(t) = −Kxe (t), then the controlled system becomes x(t) ˙ = Ax(t) + Bu(t) + Gw(t) = Ax(t) − BKxe (t) + Gw(t)

(12.122)

Chapter 12. Linear Control Systems

509

and the observer equation (12.116) becomes x˙ e (t) = Axe (t) + Bu(t) + F [y(t) − Cxe (t)] = Axe (t) − BKxe (t) + F y(t) − F Cxe (t) = Axe (t) − BKxe (t) + F [Cx(t) + Ev(t)] − F Cxe (t) = [Axe (t) − BKxe (t) − F Cxe (t)] + F Cx(t) + F Ev(t) = [A − BK − F C]xe (t) + F Cx(t) + F Ev(t). Consequently, the closed-loop system has the form        d x(t) A −BK Gw(t) x(t) = , + FC Ae F Ev(t) xe (t) dt xe (t) (12.123) where the observer matrix is given by Ae = A − BK − F C.

(12.124)

The controller defined by the observer (12.116) with feedback law (12.122) is called the Linear Quadratic Gaussian Regulator or LQG Controller for short. The rigorous development of these concepts can be found in any modern control book. The closed-loop system defined by (12.123) is stable and for many years it was thought to be robust. However, it is well known that this feedback control is not robust (see the famous paper by John Doyle [72]) so there needs to be a way to deal with both performance and robustness. One approach is to apply the so-called Min-Max theory (see [16]) to obtain a more robust controller. This theory is based on a game theory approach to controller design and will not be discussed here. However, in the next section we summarize the basic approach for the linear regulator problem above.

12.7

The Time Invariant Min-Max Controller

Observe that the solution to the LQG problem involves solving two uncoupled Riccati equations (12.110) and (12.121). In particular, there is no relationship between the optimal controller determined

510

Chapter 12. Linear Control Systems

by the solution of the Riccati equation (12.110) and the optimal observer determined by the Riccati equation (12.121). Thus, the controller design does not consider disturbance and sensor noise. This is one source of the lack of robustness. In the papers [155] and [175] an optimal control problem based on a game theoretic formulation is used to develop an optimal controller and estimator where the control, disturbances and noise are included in the formulation. The cost function is formulated as a quadratic function of the state history, the initial state, the system disturbances, sensor noise and the control. We shall not discuss the derivation of this control here, but briefly summarize the approach. The books by Basar and Bernhard [16] and [17] contain a very nice presentation of the theory of H ∞ and Min-Max control. Let θ2 ≥ 0 and Q = DT D, M = GGT , N = EE T and R = RT > 0 be as above. Consider the Riccati equations P A + AT P − P [BR−1 B T − θ2 M ]P + Q = 0

(12.125)

AΠ + ΠAT − Π[C T N −1 C − θ2 Q]Π + M = 0

(12.126)

and

and let Pθ and Πθ be the minimal nonnegative definite solutions of (12.125) and (12.126), respectively. If Πθ > 0 is positive definite, then for θ2 sufficiently small [Πθ ]−1 − θ2 Pθ > 0 and

exists. Define



I − θ 2 Πθ P θ

−1

Kθ = R−1 B ∗ Pθ , −1  Πθ C T N −1 , F θ = I − θ 2 Πθ P θ Aθ = A − BKθ − Fθ C + θ2 M Pθ

(12.127) (12.128) (12.129)

and consider the closed-loop system        d x(t) A −BKθ Gv(t) x(t) = . + Fθ C Aθ Fθ Ew(t) xθ (t) dt xθ (t) (12.130)

Chapter 12. Linear Control Systems

511

This system is called the Min-Max closed-loop system and provides some robustness when it exists (see[16]). One can show that there is a θˆ2 > 0 so that if 0 ≤ θ2 < θˆ2 , then there exist minimal positive definite solutions Pθ and Πθ to the Ricatti equations (12.125) and (12.126), [I − θ2 Πθ Pθ ] is non-singular and the closed-loop system (12.130) is stable (robustly). Roughly speaking, larger θˆ2 implies more robustness. Note that when θ2 = 0, the Min-Max controller reduces to the LQG controller. Remark 12.6 The short treatment of LQ optimal control presented here is clearly incomplete. The primary purpose of discussing this topic was to illustrate how optimal control and the Maximum Principle can be used to address problems of this type. There are numerous excellent books devoted to these topics and to more advanced applications involving systems governed by delay and partial differential equations. Hopefully, this brief introduction will encourage and motivate the reader to dive more deeply into these exciting areas.

12.8

Problem Set for Chapter 12

Consider the linear control system x(t) ˙ = A(t)x(t) + B(t)u(t),

(12.131)

with measured output y(t) = C(t)x(t)

(12.132)

z(t) = D(t)x(t).

(12.133)

and controlled output

Here, we assume that A : [t0 , t1 ] → Rn×n , B : [t0 , t1 ] → Rn×m , C : [t0 , t1 ] → Rp×n and D : [t0 , t1 ] → Rq×n are piecewise continuous matrix valued functions. Let Q(t) = [D(t)]T D(t) and assume R : [t0 , t1 ] → Rm×m is a piecewise continuous symmetric matrix valued function such that 0 < αIm×m ≤ R(t),

512

Chapter 12. Linear Control Systems

for some α > 0. Finally, we assume that the controls belong to the space P W C(t0 , T ; Rm ) and the quadratic cost functional is defined by J(u(·)) = hSx(t1 ), x(t1 )i (12.134) Z t1 1 {hQ(s)x(s), x(s)i + hR(s)u(s), u(s)i} ds, + 2 t0 (12.135) where S = S T ≥ 0 is a constant symmetric matrix. All the following problems concern these LQ control problems. Problem 12.1 Consider the control system        d x1 (t) 0 x1 (t) −1 Re = u(t), + 1 x (t) 0 −2 x (t) dt 2 2 with output y(t) = x1 (t) + x2 (t) =



1 1





x1 (t) x2 (t)



,

where Re ≥ 0 is a constant. (a) For what values of Re is this system controllable? (b) For what values of Re is the system observable? Problem 12.2 Consider the control system        d x1 (t) −1 Re α x1 (t) u(t), = + β x (t) x (t) 0 −2 dt 2 2 with output y(t) = x1 (t) + x2 (t) =



1 1





x1 (t) x2 (t)



.

(a) For what values of Re , α and β is this system controllable? (b) For what values of Re , α and β is the system stabilizable?

513

Chapter 12. Linear Control Systems Problem 12.3 Consider the control system        d x1 (t) 0 x1 (t) −1 10 u(t) + = 1 x2 (t) 0 2 dt x2 (t) with output y(t) = αx1 (t) + βx2 (t) =



α β





x1 (t) x2 (t)



.

(a) Is this system controllable? (b) For what values of α and β is the system observable? (c) Is this system stabilizable? (d) For what values of α and β is the system detectable? Problem 12.4 Consider the control system        d x1 (t) δ x1 (t) 1 0 u(t) + = ǫ x2 (t) 0 −1 dt x2 (t) with output y(t) = x1 (t) + x2 (t) =



1 1





x1 (t) x2 (t)



.

For what values of δ and ǫ is this system (a) controllable, (b) observable, (c) stabilizable, (d) detectable? Problem 12.5 Consider the LQR control problem for the system        d x1 (t) δ x1 (t) 1 0 u(t) + = ǫ x2 (t) 0 −1 dt x2 (t) with quadratic cost Z J(u(·)) =

+∞ 0



[x1 (s)]2 + [x2 (s)]2 + [u(s)]2 ds.

For each δ and ǫ, find the solution P = P (δ, ǫ) of the algebraic Riccati equation AT P + P A − P BB T P + Q = 0

514

Chapter 12. Linear Control Systems

and compute the optimal feedback gain K(δ, ǫ) = B T P (δ, ǫ). Discuss these solutions as δ −→ 0 and ǫ −→ 0. Problem 12.6 Let Re ≥ 0 be a constant and consider the LQR control problem for the system        d x1 (t) 1 x1 (t) −1 Re u(t) + = 1 x2 (t) 0 −2 dt x2 (t) with quadratic cost Z J(u(·)) =

+∞ 0



[x1 (s)]2 + [x2 (s)]2 + [u(s)]2 ds.

For each Re > 0, find the solution P = P (Re ) of the algrbraic Riccati equation AT P + P A − P BB T P + Q = 0 and compute the optimal feedback gain K(Re ) = B T P (Re ). Discuss these solutions as Re −→ 0 and as Re −→ +∞. Problem 12.7 Consider the control system        d x1 (t) 0 x1 (t) 0 1 u(t). + = 1 x2 (t) 0 0 dt x2 (t) Show that this system is controllable. Find the minimum energy control that transfers a state x1 to zero in time t1 = 2. Hint: the matrix exponential is given by   1 t At . e = 0 1

515

Chapter 12. Linear Control Systems Problem 12.8 Consider the control system        d x1 (t) 0 x1 (t) −1 e2t + u(t). = e−t x2 (t) 0 −1 dt x2 (t)

Show that this system is completely controllable for all t1 > 0. Find the minimum energy control that transfers the state x1 = [1 1]T to zero in time t1 = 2. Hint: the state transition matrix is given by  −(t−s) 1 (t+s)  −(t−−3s) e [e − e ] 2 Φ(t, s) = . 0 e−(t−s) Problem 12.9 Consider the control system      d x1 (t) x1 (t) 0 1 = x2 (t) 0 0 dt x2 (t) with measured output      x1 (t) 2 0 2x1 (t) . = y(t) = x2 (t) 0 1 x2 (t) Construct a Luenberger observer for this system. Problem 12.10 Consider the control system        d x1 (t) 0 x1 (t) 0 1 u(t). + = 1 x2 (t) 0 0 dt x2 (t) with measured output      x1 (t) 2 0 2x1 (t) . = y(t) = x2 (t) 0 1 x2 (t) Let Q=



2 0 0 1



, R = 1, M =



1 0 0 1



, and N = 1.

Construct the Kalman (LQG) observer for this system.

516

Chapter 12. Linear Control Systems Advanced Problems

These problems require MatlabT M or some other numerical toolbox. Problem 12.11 Consider the control system        d x1 (t) 0 x1 (t) −0.4 Re + u(t), = 1 x2 (t) 0 −0.2 dt x2 (t) with output y(t) = x1 (t) + x2 (t) =



1 1





x1 (t) x2 (t)



,

where Re ≥ 0 is a constant. Let     1 0 1 0 , and N = 1. , R = 1, M = Q= 0 1 0 1 For each Re = 1 and 2, compute the LQR gain K(Re ) = R−1 B T P (Re ) and the Kalman filter gain F (Re ) = ΠC T N −1 , where P (Re ) and Π(Re ) satisfy AT P + P A − P BR1 B T P + Q = 0 and AΠ + ΠAT − ΠC T N −1 CΠ + M = 0,

respectively. Construct the optimal LQG closed-loop system and simulate the response to the initial conditions x0 = [1 0]T , x0 = [0 0]T and x0 = [0 1]T , with initial state estimates xe,0 = (0.95) · x0 . Problem 12.12 Consider the nonlinear control system      d x1 (t) −0.4 Re x1 (t) = 0 −0.2 x2 (t) dt x2 (t)    p x (t) 0 −1 1 + [x1 (t)]2 + [x1 (t)]2 x2 (t) +1 0   0 u(t). + 1

517

Chapter 12. Linear Control Systems

where Re = 2. Use the LQR optimal controller from Problem 12.11 above and simulate the nonlinear response to the initial conditions x0 = [1 0]T , x0 = [0 0]T and x0 = [0 1]T . Problem 12.13 Consider the nonlinear control system      d x1 (t) x1 (t) −0.4 Re = x2 (t) ε −0.2 dt x2 (t)    p x (t) 0 −1 1 + [x1 (t)]2 + [x1 (t)]2 x2 (t) +1 0   0 u(t). + 1 where Re = 2 and ε = 0.05. Use the LQR optimal controller from Problem 12.11 above and simulate the nonlinear response to the initial conditions x0 = [1 0]T , x0 = [0 0]T and x0 = [0 1]T . Problem 12.14 Consider the system x˙ N (t) = AN xN (t) + BN u(t), where for each N = 2, 4, 8, 16, ..., given by  −2 1  1 −2   1  0 AN = (N + 1)2  . ..  .. .   0 0 0 0

the N × N system matrix An is 0 1

··· 0

0 0 ··· 0 . . . .. −2 1 . .. .. .. . . . 0 · · · 1 −2 1 ··· 0 1 −2

        

N ×N

518 and

Chapter 12. Linear Control Systems 

1/(N + 1) 2/(N + 1) 3/(N + 1) .. .

    BN =     (N − 1)/(N + 1) N/(N + 1)

        

For N = 2, 4, 8, use MatlabT M to solve the LQR problem where Q = IN ×N is the N × N identity matrix and R = 1. (This system comes from approximating an optimal control problem for heat transfer in a rod.)

Bibliography [1] B. M. Adams, H. T. Banks, H. D. Kwon, and H. T. Tran. Dynamic multidrug therapies for hiv: Optimal and sti control approaches. Mathematical Biosciences and Engineering, 1(2):223–241, 2004. [2] R. A. Adams. Sobolev Spaces. Academic Press, New York, 1975. [3] J. H. Ahlberg, E. N. Nilson, and J. L. Walsh. The Theory of Splines and Their Applications. Academic Press, 1967. [4] P. S. Aleksandrov, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko. Lev Semenovich Pontryagin (on his sixtieth birthday). Uspekhi Mat. Nauk, 23:187–196, 1968. [5] B. D. Anderson and J. B. Moore. Linear Optimal Control. Prentice-Hall, Englewood Cliffs, 1971. [6] B. D. O. Anderson and J. B. Moore. Optimal Control: Linear Quadratic Methods. Prentice-Hall, Inc., Upper Saddle River, 1990. [7] P. J. Antsaklis and A. N. Michel. Birkhauser, Boston, 2005.

Linear Systems.

[8] J. Appel, E. Cliff, M. Gunzburger, and A. Godfrey. Optimization-based design in high-speed flows. In CFD for Design and Optimization, ASME, New York, pages 61–68, 1995.

519

520

Bibliography

[9] E. Arian and S. Ta’asan. Multigrid one shot methods for optimal control problems: Infinite dimensional control. Technical Report ICASE Report No. 94–52, NASA, Langley Research Center, Hampton VA 23681–0001, 1994. [10] A. Armaou and M. A. Demetriou. Optimal actuator/sensor placement for linear parabolic PDEs using spatial H2 norm. Chemical Engineering Science, 61(22):7351–7367, 2006. [11] B. S. Attili and M. I. Syam. Efficient shooting method for solving two point boundary value problems. Chaos, Solitons and Fractals, 35(5):895–903, 2008. [12] J. P. Aubin. Approximation of Elliptic Boundary-Value Problems. RE Krieger Pub. Co., 1972. [13] S. Badrinarayanan and N. Zabaras. A sensitivity analysis for the optimal design of metal-forming processes. Computer Methods in Applied Mechanics and Engineering, 129(4):319– 348, 1996. [14] H. T. Banks, S. Hu, T. Jang, and H. D. Kwon. Modelling and optimal control of immune response of renal transplant recipients. Journal of Biological Dynamics, 6(2):539–567, 2012. [15] R. G. Bartle. The Elements of Real Analysis: Second Edition. John Wiley & Sons, New York, 1976. [16] T. Basar and P. Bernhard. H ∞ -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach. Birkhauser, 1995. [17] T. Ba¸sar and P. Bernhard. H-Infinity Optimal Control and Related Minimax Design Problems: a Dynamic Game Approach. Birkh¨auser, Boston, 2008. [18] J. Z. Ben-Asher. Optimal Control Theory with Aerospace Applications. AIAA, 2010.

Bibliography

521

[19] J. A. Bennett and M. E. Botkin. Structural shape optimization with geometric description and adaptive mesh refinement. AIAA Journal, 23(3):458–464, 1985. [20] A. Bensoussan and A. Friedman. Nonlinear variational inequalities and differential games with stopping times. Journal of Functional Analysis, 16(3):305–352, 1974. [21] A. Bensoussan and J. L. Lions. Applications of Variational Inequalities in Stochastic Control, volume 12. NorthHolland, 1982. [22] A. Bensoussan, G. Da Prato, M. C. Delfour, and S. K. Mitter. Representation and Control of Infinite Dimensional Systems - I. Systems & Control: Foundations & Applications. Birkh¨auser, Boston, 1992. [23] A. Bensoussan, G. Da Prato, M. C. Delfour, and S. K. Mitter. Representation and Control of Infinite Dimensional Systems - II. Systems & Control: Foundations & Applications. Birkh¨auser, Boston, 1992. [24] L. D. Berkovitz. Optimal Control Theory, volume 12 of Applied Mathematical Sciences. Springer-Verlag, New York, 1974. [25] J. T. Betts. Practical Methods for Optimal Control Using Nonlinear Programming. Advances in Design and Control. Society for Industrial and Applied Mathematics, Philadelphia, 2001. [26] F. Billy, J. Clairambault, and O. Fercoq. Optimisation of cancer drug treatments using cell population dynamics. Mathematical Methods and Models in Biomedicine, pages 263–299, 2012. [27] G. A. Bliss. Calculus of Variations. Carus Mathematical Monographs, Chicago, 1925. [28] G. A. Bliss. Normality and abnormality in the calculus of variations. Trans. Amer. Math. Soc, 43:365–376, 1938.

522

Bibliography

[29] G. A. Bliss. Lectures on the Calculus of Variations. Phoenix Science Series, University of Chicago Press, Chicago, 1963. [30] V. G. Boltyanski. The maximum principle - how it came to be? report 526. Technical report, Mathematisches Institut, 1994. [31] O. Bolza. Lectures on the Calculus of Variations. University of Chicago Press, Chicago, 1904. [32] J. Borggaard. The Sensitivity Equation Method for Optimal Design. PhD thesis, Virginia Tech, Blacksburg, VA, December 1994. [33] J. Borggaard. On the presence of shocks in domain optimization of Euler flows. In M. Gunzburger, editor, Flow Control, volume 68 of Proceedings of the IMA. Springer-Verlag, 1995. [34] J. Borggaard and J. Burns. A sensitivity equation approach to optimal design of nozzles. In Proc. 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, pages 232–241, 1994. [35] J. Borggaard and J. Burns. A sensitivity equation approach to shape optimization in fluid flows. In M. Gunzburger, editor, Flow Control, volume 68 of Proceedings of the IMA. Springer-Verlag, 1995. [36] J. Borggaard and J. Burns. Asymptotically consistent gradients in optimal design. In N. Alexandrov and M. Hussaini, editors, Multidisciplinary Design Optimization: State of the Art, pages 303–314, Philadelphia, 1997. SIAM Publications. [37] J. Borggaard and J. Burns. A PDE sensitivity equation method for optimal aerodynamic design. Journal of Computational Physics, 136(2):366–384, 1997. [38] J. Borggaard, J. Burns, E. Cliff, and M. Gunzburger. Sensitivity calculations for a 2D, inviscid, supersonic forebody

Bibliography

523

problem. In H.T. Banks, R. Fabiano, and K. Ito, editors, Identification and Control of Systems Governed by Partial Differential Equations, pages 14–24, Philadelphia, 1993. SIAM Publications. [39] J. Borggaard, J. Burns, and L. Zietsman. On using lqg performance metrics for sensor placement. In American Control Conference (ACC), 2011, pages 2381–2386. IEEE, 2011. [40] J. Borggaard, E. Cliff, J. Burkardt, M. Gunzburger, H. Kim, H. Lee, J. Peterson, A. Shenoy, and X. Wu. Algorithms for flow control and optimization. In J. Borggaard, J. Burkardt, M. Gunzburger, and J. Peterson, editors, Optimal Design and Control, pages 97–116. Birkhauser, 1995. [41] J. Borggaard and D. Pelletier. Computing design sensitivities using an adaptive finite element method. In Proceedings of the 27th AIAA Fluid Dynamics Conference, 1996. [42] J. Borggaard and D. Pelletier. Observations in adaptive refinement strategies for optimal design. In J. Borggaard, J. Burns, E. Cliff, and S. Schreck, editors, Computational Methods for Optimal Design and Control, pages 59–76. Birkhauser, 1998. [43] J. Borggaard and D. Pelletier. Optimal shape design in forced convection using adaptive finite elements. In Proc. 36th AIAA Aerospace Sciences Meeting and Exhibit, 1998. AIAA Paper 98-0908. [44] F. Brezzi and M. Fortin. Mixed and Hybrid Finite Elements. Springer-Verlag, New York, 1991. [45] R. W. Brockett. Finite Dimensional Linear Systems. Wiley, New York, 1970. [46] A. E. Bryson and Y. C. Ho. Applied Optimal Control. John Wiley & Sons, New York, 1975. [47] A.E. Bryson Jr. Optimal control-1950 to 1985. Control Systems Magazine, IEEE, 16(3):26–33, 1996.

524

Bibliography

[48] C. Buck. Advanced Calculus. McGraw Hill, New York, 1978. [49] J. A. Burns, E. M. Cliff, C. Rautenberg, and L. Zietsman. Optimal sensor design for estimation and optimization of pde systems. In American Control Conference (ACC), 2010, pages 4127–4132. IEEE, 2010. [50] J. A. Burns and B. B. King. Optimal sensor location for robust control of distributed parameter systems. In Proceedings of the 33rd IEEE Conference on Decision and Control, volume 4, pages 3967–3972. IEEE, 1994. [51] J. A. Burns, B. B. King, and Y. R. Ou. A computational approach to sensor/actuator location for feedback control of fluid flow systems. In J.D. Paduano, editor, Sensing, Actuation, and Control in Aeropropulsion, volume 2494 of Proc. International Society for Optical Engineering, pages 60–69, 1995. [52] D. Bushaw. Optimal discontinuous forcing terms. In Contributions to the Theory of Non-linear Oscillations, pages 29–52, Princeton, 1958. Princeton University Press. [53] C. Carath´eodory. Die methode der geod¨atischen a¨quidistanten und das problem von lagrange. Acta Mathematica, 47(3):199–236, 1926. [54] C. Carath´eodory. Calculus of Variations and Partial Differential Equations of First Order. American Mathematical Society, Providence, 1999. [55] P. G. Ciarlet. The Finite Element Method for Elliptic Problems. North-Holland, Amsterdam, 1978. [56] P. G. Ciarlet. Introduction to Numerical Linear Algebra and Optimisation. Cambridge Press, Cambridge, 1989. [57] F. H. Clarke. Necessary Conditions for Nonsmooth Problems in Optimal Control and the Calculus of Variations. PhD thesis, University of Washington, 1973.

Bibliography

525

[58] E. A. Coddington and N. Levinson. Theory of Ordinary Differential Equations. McGraw-Hill, New York, 1972. [59] R. Courant and K. L. Friedrichs. Supersonic Flow and Shock Waves. Interscience Publishers, New York, 1948. [60] M. G. Crandall, H. Ishii, and P. L. Lions. Users guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc, 27(1):1–67, 1992. [61] M. G. Crandall and P. L. Lions. Viscosity solutions of hamilton-jacobi equations. Trans. Amer. Math. Soc, 277(1):1–42, 1983. [62] J. J. Crivelli, J. F¨oldes, P. S. Kim, and J. R. Wares. A mathematical model for cell cycle-specific cancer virotherapy. Journal of Biological Dynamics, 6(sup1):104–120, 2012. [63] R. F. Curtain and H. Zwart. An Introduction to InfiniteDimensional Linear Systems Theory. Springer, New YorkBerlin - Heidelberg, 1995. [64] B. Dacorogna. Direct Methods in the Calculus of Variations, volume 78. Springer, 2007. [65] B. Dacorogna. Introduction to the Calculus of Variations. Imperial College Press, London, 2nd edition, 2009. [66] N. Darivandi, K. Morris, and A. Khajepour. Lq optimal actuator location in structures. In American Control Conference (ACC), 2012, pages 646–651. IEEE, 2012. [67] M. C. Delfour and J. P. Zolesio. Shape sensitivity analysis via MinMax differentiability. SIAM Journal of Control and Optimization, 26(4), July 1988. [68] M. C. Delfour and J. P. Zolesio. Velocity method and lagrangian formulation for the computation of the shape hessian. SIAM Journal of Control and Optimization, 29(6):1414–1442, November 1991.

526

Bibliography

[69] M. A. Demetriou. Integrated actuator-sensor placement and hybrid controller design of flexible structures under worst case spatiotemporal disturbance variations. Journal of Intelligent Material Systems and Structures, 15(12):901, 2004. [70] M.A. Demetriou and J. Borggaard. Optimization of a joint sensor placement and robust estimation scheme for distributed parameter processes subject to worst case spatial disturbance distributions. In American Control Conference, 2004. Proceedings of the 2004, volume 3, pages 2239–2244. IEEE, 2004. [71] P. Dorato, V. Cerone, and C. T. Abdallah. Linear-Quadratic Control. Prentice Hall, Englewood Cliffs, 1995. [72] J. C. Doyle. Guaranteed margins for LQG regulators. Transactions on Automatic Control, 23:756–757, 1981. [73] J. C. Doyle, B. A. Francis, and A. R. Tannenbaum. Feedback Control Theory. Macmillan Publishing Company, New York, 1992. [74] M. Engelhart, D. Lebiedz, and S. Sager. Optimal control for selected cancer chemotherapy ode models: A view on the potential of optimal schedules and choice of objective function. Mathematical Biosciences, 229(1):123–134, 2011. [75] L. C. Evans. On solving certain nonlinear partial differential equations by accretive operator methods. Israel Journal of Mathematics, 36(3):225–247, 1980. [76] L. C. Evans. Partial Differential Equations. Graduate Studies in Mathematics: Vol. 19. American Mathematical Society, Providence, 1998. [77] George M. Ewing. Calculus of Variations with Applications. Dover Publications, New York, 1985.

Bibliography

527

[78] A. L. Faulds and B. B. King. Sensor location in feedback control of partial differential equation systems. In Proceedings of the 2000 IEEE International Conference on Control Applications, pages 536–541. IEEE, 2000. [79] W. H. Fleming and R. W. Rishel. Deterministic and Stochastic Optimal Control. Springer, New York, 1975. [80] A. R. Forsyth. Calculus of Variations. Cambridge University Press, New York, 1927. [81] B. A. Francis. A Course in H ∞ -Control Theory, Lecture Notes in Control, Vol. 88. Springer-Verlag, New York, 1987. [82] P. D. Frank and G. R. Shubin. A comparison of optimization-based approaches for a model computational aerodynamics design problem. Journal of Computational Physics, 98:74–89, 1992. [83] R. H. Gallagher and O. C. Zienkiewicz, editors. Optimum Structural Design: Theory and Application, John Wiley & Sons, New York, 1973. [84] R. V. Gamkrelidze. Discovery of the maximum principle. Journal of Dynamical and Control Systems, 5(4):437–451, 1999. [85] R. V. Gamkrelidze. Discovery of the maximum principle. Mathematical Events of the Twentieth Century, pages 85– 99, 2006. [86] I. M. Gelfand and S. V. Fomin. Calculus of Variations. Prentice-Hall, Inc., Englewood Cliffs, 1963. [87] I. M. Gelfand and S. V. Fomin. Calculus of Variations. Prentice-Hall, Inc., Englewood Cliffs, 1963. [88] I. M. Gelfand, G. E. Shilov, E. Saletan, N. I. A. Vilenkin, and M. I. Graev. Generalized Functions, volume 1. Academic Press, New York, 1968.

528

Bibliography

[89] M. Giaquinta. Multiple Integrals in the Calculus of Variations and Nonlinear Elliptic Systems.(AM-105), volume 105. Princeton University Press, 1983. [90] A. Godfrey. Using sensitivities for flow analysis. In J. Borggaard, J. Burns, E. Cliff, and S. Schreck, editors, Computational Methods for Optimal Design and Control, pages 181–196. Birkhauser, 1998. [91] H. H. Goldstine. A History of the Calculus of Variations from the 17th through the 19th Century. Springer-Verlag, 1980. [92] G. H. Golub and C. F. Van Loan. Matrix Computations: Third Edition. Johns Hopkins University Press, Baltimore, 1996. [93] L. M. Graves. The Derivative as Independent Function in the Calculus of Variations. PhD thesis, University of Chicago, Department of Mathematics, 1924. [94] L. M. Graves. A transformation of the problem of Lagrange in the calculus of variations. Transactions of the American Mathematical Society, 35(3):675–682, 1933. [95] L. M. Graves. Calculus of Variations and Its Applications. McGraw-Hill, New York, 1958. [96] M. Gunzburger. Finite Element Methods for Viscous Incompressible Flows. Academic Press, 1989. [97] H. Halkin. A generalization of LaSalle’s bang-bang principle. Journal of the Society for Industrial & Applied Mathematics, Series A: Control, 2(2):199–202, 1964. [98] J. Haslinger and P. Neittaanmaki. Finite Element Approximation for Optimal Shape Design: Theory and Application. John Wiley & Sons, 1988.

Bibliography

529

[99] E. J. Haug, K. K. Choi, and V. Komkov. Design Sensitivity Analysis of Structural Systems, volume 177 of Mathematics in Science and Engineering. Academic Press, Orlando, FL, 1986. [100] H. Hermes and J. P. LaSalle. Functional Analysis and Time Optimal Control. Number 56 in Mathematics in Science and Engineering. Academic Press, New York, 1969. [101] M. R. Hestenes. Calculus of Variations and Optimal Control Theory. John Wiley & Sons, New York, 1966. [102] M. R. Hestenes. Optimization Theory: The Finite Dimensional Case. Wiley, 1975. [103] R. Holsapple, R. Venkataraman, and D. Doman. A modified simple shooting method for solving two point boundary value problems. In Proc. of the IEEE Aerospace Conference, Big Sky, MT, 2003. [104] R. Holsapple, R. Venkataraman, and D. Doman. New, fast numerical method for solving two-point boundary-value problems. Journal of Guidance Control and Dynamics, 27(2):301–303, 2004. [105] A. Isidori. Nonlinear Control Systems. Springer, 1995. [106] M. Itik, M. U. Salamci, and S. P. Banks. Optimal control of drug therapy in cancer treatment. Nonlinear Analysis: Theory, Methods & Applications, 71(12):e1473–e1486, 2009. [107] V. G. Ivancevic and T. T. Ivancevic. Applied Differential Geometry: A Modern Introduction. World Scientific Publishing Company Incorporated, 2007. [108] A. Jameson. Aerodynamic design via control theory. Journal of Scientific Computing, 3:233–260, 1988. [109] A. Jameson and J. Reuther. Control theory based airfoil design using the Euler equations. In Proc.

530

Bibliography 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, pages 206–222, 1994.

[110] T. Kailath. Linear Systems. Prentice-Hall, 1980. [111] J. H. Kane and S. Saigal. Design sensitivity analysis of solids using BEM. Journal of Engineering Mechanics, 114:1703– 1722, 1988. [112] H. B. Keller. Numerical Methods for Two-Point BoundaryValue Problems. Blaisdell, 1968. [113] C. T. Kelley and E. W. Sachs. Quasi-Newton methods and unconstrained optimal control problems. SIAM Journal on Control and Optimization, 25:1503, 1987. [114] H. W. Knobloch, A. Isidori, and D. Flockerzi. Topics in Control Theory. Birkh¨auser Verlag, 1993. [115] H. Kwakernaak and R. Sivan. Linear Optimal Control Systems. Willey-Interscience, New York, 1972. [116] S. Lang. Fundamentals of Differential Geometry, volume 160. Springer Verlag, 1999. [117] U. Ledzewicz, M. Naghnaeian, and H. Sch¨attler. Optimal response to chemotherapy for a mathematical model of tumor–immune dynamics. Journal of Mathematical Biology, 64(3):557–577, 2012. [118] U. Ledzewicz and H. Sch¨attler. Multi-input optimal control problems for combined tumor anti-angiogenic and radiotherapy treatments. Journal of Optimization Theory and Applications, pages 1–30, 2012. [119] E. B. Lee and L. Markus. Foundations of Optimal Control Theory, The SIAM Series in Applied Mathematics. John Wiley & Sons, 1967. [120] G. Leitmann. The Calculus of Variations and Optimal Control. Springer, Berlin, 1981.

Bibliography

531

[121] N. Levinson. Minimax, Liapunov and “bang-bang,” Journal of Differential Equations, 2:218–241, 1966. [122] D. Liberzon. Calculus of Variations and Optimal Control Theory: A Concise Introduction. Princeton University Press, Princeton, 2012. [123] J. L. Lions. Optimal Control of Systems Governed by Partial Differential Equations, vol. 170 of Grundlehren Math. Wiss. Springer-Verlag, Berlin, 1971. [124] J. L. Lions. Optimal Control of Systems Governed by Partial Differential Equations, vol. 170 of Grundlehren Math. Wiss. Springer-Verlag, 1971. [125] J.-L. Lions. Control of Distributed Singular Systems. Gauthier-Villars, Kent, 1985. [126] J. L. Lions and E. Magenes. Non-homogeneous boundary value problems and applications, Vol. 1, 2. Springer, 1972. [127] J. L. Lions and E. Magenes. Non-homogeneous Boundary Value Problems and Applications. Volume 3 (Translation of Problemes aux limites non homogenes et applications). 1973. [128] J. L. Lions, E. Magenes, and P. Kenneth. Non-homogeneous Boundary Value Problems and Applications, volume 1. Springer-Verlag Berlin, 1973. [129] D. G. Luenberger. Observing the state of a linear system. Military Electronics, IEEE Transactions on, 8(2):74– 80, 1964. [130] D. G. Luenberger. Observers for multivariable systems. Automatic Control, IEEE Transactions on, 11(2):190–197, 1966. [131] D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons, New York, 1969.

532

Bibliography

[132] D. G. Luenberger. An introduction to observers. Automatic Control, IEEE Transactions on, 16(6):596–602, 1971. [133] E. J. McShane. The calculus of variations from the beginning through optimal control theory. In W. G. Kelley, A. B. Schwarzkopf and S. B. Eliason, editors, Optimal Control and Differential Equations, pages 3 –49. Academic Press, New York, 1978. [134] E. J. McShane. The calculus of variations from the beginning through optimal control theory. SIAM Journal on Control and Optimization, 27:916, 1989. [135] M. Mesterton-Gibbons. A Primer on the Calculus of Variations and Optimal Control Theory, volume 50. American Mathematical Society, Providence, 2009. [136] A. Miele. Theory of Optimum Aerodynamic Shapes. Academic Press, New York, 1965. [137] A. Miele and H.Y. Huang. Missile shapes of minimum ballistic factor. Journal of Optimization Theory and Applications, 1(2):151–164, 1967. [138] R. K. Miller and A. N. Michel. Ordinary Differential Equations. Dover Publications, New York, 2007. [139] K. Morris. Linear-quadratic optimal actuator location. Automatic Control, IEEE Transactions on, 56(1):113–124, 2011. [140] J. M. Murray. Optimal control for a cancer chemotheraphy problem with general growth and loss functions. Mathematical Biosciences, 98(2):273–287, 1990. [141] J. M. Murray. Some optimal control problems in cancer chemotherapy with a toxicity limit. Mathematical Biosciences, 100(1):49–67, 1990.

Bibliography

533

[142] R. Narducci, B. Grossman, and R. Haftka. Design sensitivity algorithms for an inverse design problem involving a shock wave. Inverse Problems in Engineering, 2:49–83, 1995. [143] M. Z. Nashed. Some remarks on variations and differentials. American Mathematical Monthly, pages 63–76, 1966. [144] L. W. Neustadt. Optimization: A Theory of Necessary Conditions. Princeton University Press, 1976. [145] G. Newbury. A Numerical Study of a Delay Differential Equation Model for Breast Cancer. PhD thesis, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, 2007. [146] K. O. Okosun, O. D. Makinde, and I. Takaidza. Impact of optimal control on the treatment of hiv/aids and screening of unaware infectives. Applied Mathematical Modelling, 2012. [147] B. O’neill. Elementary Differential Geometry. Academic Press, second edition, 1997. [148] J. C. Panetta and K. R. Fister. Optimal control applied to cell-cycle-specific cancer chemotherapy. SIAM Journal on Applied Mathematics, 60(3):1059–1072, 2000. [149] H. J. Pesch and M. Plail. The maximum principle of optimal control: A history of ingenious ideas and missed opportunities. Control and Cybernetics, 38(4A):973–995, 2009. [150] O. Pironneau. Optimal Shape Design for Elliptic Systems. Springer-Verlag, 1983. [151] BN Pshenichny. Convex analysis and optimization. Moscow: Nauka, 25:358, 1980. [152] C. W. Ray and B. A. Batten. Sensor placement for flexible wing shape control. In American Control Conference (ACC), 2012, pages 1430–1435. IEEE, 2012.

534

Bibliography

[153] W. T. Reid. Ordinary Differential Equations. John Wiley and Sons, New York, 1971. [154] W. T. Reid. A historical note on the maximum principle. SIAM Review, 20:580, 1978. [155] I. Rhee and J. L. Speyer. A game theoretic controller and its relationship to H ∞ and linear-exponential-gaussian synthesis. In Decision and Control, 1989, Proceedings of the 28th IEEE Conference on, pages 909–915. IEEE, 1989. [156] A. C. Robinson. A survey of optimal control of distributedparameter systems. Automatica, 7(3):371–388, 1971. [157] D. L. Russell. Mathematics of Finite Dimensional Control Systems: Theory and Design. M. Dekker, 1979. [158] H. Sagan. Introduction to the Calculus of Variations. Courier Dover Publications, 1992. [159] M. H. Schultz. Spline Analysis. Prentice-Hall, 1973. [160] L. Schumaker. Spline Functions: Basic Theory. Cambridge University Press, 2007. [161] L. Schwartz. Th´eorie des distributions. ii, hermann. Paris, France, 1959. [162] L. F. Shampine, J. Kierzenka, and M. W. Reichelt. Solving boundary value problems for ordinary differential equations in Matlab with bvp4c. Manuscript, available at ftp://ftp. mathworks. com/pub/doc/papers/bvp, 2000. [163] L. L. Sherman, A. C. Taylor III, L. L. Green, P.A. Newman, G.J.-W. Hou, and V.M. Korivi. First- and secondorder aerodynamic sensitivity derivatives via automatic differentiation with incremental iterative methods. In Proc. 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, pages 87–120, 1994.

Bibliography

535

[164] D. B. Silin. On the variation and riemann integrability of an optimal control in linear systems. Dokl. Akad. Nauk SSSR, 257(3):548–550, 1981. English Translation in Soviet Math. Doklady, 23(2): 309–311, 1981. [165] M. H. N. Skandari, H. R. Erfanian, A. V. Kamyad, and S. Mohammadi. Optimal control of bone marrow in cancer chemotherapy. European Journal of Experimental Biology, 2(3):562–569, 2012. [166] D. R. Smith. Variational Methods in Optimization. PrenticeHall, Englewood Cliffs, 1974. [167] D. R. Smith. Variational Methods in Optimization. Courier Dover Publications, 1998. [168] D. Stewart. Numerical Methods for Accurate Computation of Design Sensitivities. PhD thesis, Virginia Tech, 1998. [169] J. Stoer, R. Bulirsch, W. Gautschi, and C. Witzgall. Introduction to Numerical Analysis. Springer Verlag, 2002. [170] G. Strang and G. Fix. An Analysis of the Finite Element Method. Prentice-Hall, Englewood Cliffs, 1973. [171] G. Strang and G. Fix. An Analysis of the Finite Element Method. Wellesley-Cambridge Press, Wellesley, 1988. [172] H. J. Sussmann and J. C. Willems. 300 years of optimal control: from the brachystochrone to the maximum principle. IEEE Control Systems Magazine, 17(3):32–44, 1997. [173] G. W. Swan. Role of optimal control theory in cancer chemotherapy. Mathematical Biosciences, 101(2):237–284, 1990. [174] S. Ta’asan. One shot methods for optimal control of distributed parameter systems I: Finite dimensional control. Technical Report ICASE Report No. 91-2, NASA Langley Research Center, Hampton, VA, 1991.

536

Bibliography

[175] M. Tahk and J. Speyer. Modeling of parameter variations and asymptotic LQG synthesis. Automatic Control, IEEE Transactions on, 32(9):793–801, 1987. [176] A. E. Taylor. General Theory of Functions and Integration. Dover Publications, 2010. [177] A. C. Taylor III, G. W. Hou, and V. M. Korivi. A methodology for determining aerodynamic sensitivity derivatives with respect to variation of geometric shape. In Proc. AIAA/ASME/ASCE/AHS/ASC 32nd Structures, Structural Dynamics, and Materials Conference, Baltimore, MD, April 1991. AIAA paper 91-1101. [178] A. C. Taylor III, G. W. Hou, and V. M. Korivi. Sensitivity analysis, approximate analysis and design optimization for internal and external viscous flows. In Proc. AIAA Aircraft Design Systems and Operations Meeting, 1991. AIAA paper 91-3083. [179] R. Temam. Navier-Stokes Equations: Theory and Numerical Analysis, volume 2 of Studies in Mathematics and its Applications. North-Holland Publishing Company, New York, 1979. [180] J. L. Troutman. Variational Calculus with Elementary Convexity. Springer-Verlag, 1983. [181] D. Uci´ nski. Optimal sensor location for parameter estimation of distributed processes. Int. J. Control, 73:1235–1248, 2000. [182] F.A. Valentine. Convex Sets. McGraw Hill, New York, 1964. [183] F. Y. M. Wan. Introduction to the Calculus of Variations and Its Applications. Chapman & Hall/CRC, 1995. [184] T. J. Willmore. An Introduction to Differential Geometry. Dover Publications, 2012.

Bibliography

537

[185] W. Yang, W. Y. Yang, W. Cao, T. S. Chung, and J. Morris. Applied Numerical Methods Using MATLAB. WileyInterscience, 2005. [186] L. C. Young. Lectures on the Calculus of Variations and Optimal Control Theory. Chelsea, 1969. [187] K. Zhou, J. C. Doyle, and K. Glover. Robust and Optimal Control. Prentice Hall, Englewood Cliffs, NJ, 1996.

Index Control Bang-Bang, 327 Feedback, 331 Nominal, 474 Optimal, 314, 317, 319, 343, 375, 379 Steering, 75 Controllable Completely, 460 Controller Admissible, 314, 317, 318, 342, 374, 378 Min-Max, 511 Corner, 62, 110

Bellman, Richard F., 14 Bernoulli, James, 11 Bernoulli, John, 8, 11 Bliss, Gilbert A., 14 Boltyanskii, Vladimir G., 16 Bolza, Oskar, 13 Boundary Condition Natural, 218, 220 Boundary Value Problem Strong Solution, 77 Weak Solution, 77 Brachistochrone Curve, 7 Bushaw, Donald W., 13 Caratheodory, Constantin, 14 Condition Erdmann Corner, 109, 139, 206 Legendre, 144 Strengthened Legendre, 144 Transversality, 227, 349, 377, 443, 445 Conjugate Point, 151, 210 Conjugate Value, 151, 210 Constraint Control, 314, 341, 374 Finite, 263 Isoperimetric, 255

Darboux, Jean G., 14 Derivative Left-Hand, 31 Partial, 30 Right-Hand, 31 Second Weak, 119 Weak, 118 Differential Weak, 271 Weak Second, 271 Direction Admissible, 68 Right Admissible, 68 Distance Hausdorff, 321 539

540 Point to Set, 320 Du Bois-Reymond, Paul D. G., 12 e.f., 62 Energy Kinetic, 289 Potential, 289 Total, 289 Equation Adjoint, 347 Algebraic Riccati, 506 Continuous Sensitivity, 478 Euler’s Differential Form, 221 Integral Form, 221 Heat, 296 Hilbert’s, 185 Jacobi’s Differential Form, 149 Integral Form, 149, 210 Riccati Differential, 419, 481, 484, 505 State, 318, 342, 374, 378 Stochastic Differential, 502 Estimator State, 490 Euler’s Equation Differential Form, 108, 138 Integral Form, 108, 138 Euler, Leonard, 11 Ewing, George M., 15 Excess Function Expansion, 194

Index Extremal, 109, 205, 221 Non-singular, 112, 206 Regular, 112, 144, 206 Secondary, 149, 210 Field Conservative, 288 of Extremals, 188 Figurative, 110 Filter Kalman, 494, 497, 498, 503 Finite Element Method, 26, 76, 295 Formula Variation of Parameters, 320, 462 Function Constraint, 42, 312 Continuous, 33 Control, 341, 373 Convex, 276 Strictly, 276 Cost, 42, 312 Differentiable, 34 Hat, 79, 298 Locally Piecewise Smooth, 225 Measurable, 499 Piecewise Continuous, 57 Piecewise Smooth, 59 Slope, 187 Smooth, 33 Weierstrass Excess, 155, 192, 206 Function Spaces C k (I; R1 ), 33

541

Index C k (I; Rn ), 34 P W C(t0 , +∞), 63 P W C(t0 , t1 ), 62 P W C(t0 , t1 ; Rn ), 245 P W S(t0 , +∞), 63, 225 P W S(t0 , t1 ), 62 P W S(t0 , t1 ; Rn ), 246 P W S p (t0 , t1 ), 242 P W S0 (t0 , t1 ; Rn ), 246 V0 (t0 , t1 ), 95 V0 (t0 , t1 ; Rn ), 246 V0p (t0 , t1 ), 242 VL (t0 , t1 ), 217 Functional, 93, 216, 315 Cost, 274, 317, 319, 343, 374, 378 Farmer’s Allocation Cost, 395 General High Order, 253 Higher Order, 241 Vector Form, 240, 250 Galilei, Galileo, 11 Gamkrelidze, Revaz V., 16 Graimian Controllability, 462 Observability, 462 Graves, Lawrence M., 13 Hamilton, William R., 12 Hamiltonian, 293 Augmented, 347 Time Dependent Augmented, 441 Hestenes, Magnus R., 13 Hilbert Integral, 190 Hilbert, David, 13

Inequality Gronwall-Reid-Bellman, 15 Triangle, 30 Integral Action, 289 Integrand Regular, 206 Jacobi, Karl G. J., 12, 146 Kneser, Adolf, 12 Lagrange, Joseph L., 11 Lagrangian, 44, 258, 289 LaSalle, Joesph P., 13 Legendre, Adrien-Marie, 12 Leibniz’s Formula, 104, 213 Lemma Du Bois-Reymond, 95 Fundamental Calculus of Variations, 95 Limit Left-Hand, 31 Right-Hand, 31 Matrix Control Gain, 491 Controllability, 468 Covariance, 502 Mass, 82, 300 Non-singular, 84 Observability, 468 Observer Gain, 491 Stable, 471 Stiffness, 82, 300 Transition, 462

542 Mayer, Christian G. A., 15 McShane, Edward J., 13 Measure, 498 Finite, 499 Probability, 499 Sigma-Finite, 499 Metric Strong, 64 Weak, 65, 133 Minimizer Normal, 45, 259 Minimum Global, 35, 134, 205 Local, 35 Proper Local, 35 Strong Local, 134, 205 Weak Local, 134, 205 Mischenko, Evgenii F., 16 Morse, Harold C. M., 15 Necessary Condition Abstract, 267 Euler, 107, 138, 205, 220 Jacobi, 146, 151 Legendre, 143, 144 Second Erdmann Corner, 158 Second Order, 139 Weak Weierstrass, 176 Weierstrass, 155, 207 Neighborhood Delta, 30 Strong, 65, 132, 133 Newton’s Laws, 287 First, 287 Second, 287 Third, 287

Index Newton, Sir Isaac, 11 Noise Sensor, 460, 502 White, 460, 502 Norm p-norm, 29 Euclidean, 29 General, 30 Observable Completely, 461 Observer, 490 Luenberger, 494 Optimal, 498 Output Controlled, 507 Sensed, 460, 491 Perron’s Paradox, 40 Point Conjugate, 151, 210 Core, 270 Internal, 270 Radial, 270 Pontryagin, Lev S., 14, 16 Principle Hamilton’s, 291 Least Action, 288, 289 Maximum, 341, 343, 347, 436 Nonautonomous, 442 Pontryagin Maximum, 347 Nonautonomous, 442 Stationary Action, 291 Weierstrass Maximum, 157, 208 Problem

543

Index Accessory Minimum, 147, 210 Brachistochrone, 17, 69 Bushaw, 333, 351 Double Pendulum, 20 Equality Constrained Optimization, 42, 312 Estimation, 503 Optimal, 503 Farmer’s Allocation, 393 Fixed-Time Optimal Control, 373 Free Endpoint, 215, 216 General Optimization, 265 Higher Order, 238, 239 Isoperimetric, 42, 255 Linear Quadratic Control, 404, 414, 446 Linear Quadratic Gaussian, 509 LQR, 506 Minimum Surface Area, 72 Newton’s Minimum Drag, 3, 8 Non-singular, 110, 206 Point to Curve, 225 Queen Dido, 7 Regular, 110, 206 Relaxed, 41 River Crossing, 73 Rocket Sled, 21, 313 Secondary Minimum, 147 Simplest

Calculus of Variations, 93, 316 Higher Order Form, 241, 253 Optimal Control, 341 Vector Form, 240, 249, 250 Time Optimal, 74 Vector, 238 Process Stochastic, 500 Reid, William T., 15 Riccati Differential Equation, 422 Riccati, Jacopo F., 11 Sensitivity Raw, 475 Sensitivity Equation Method, 9 Sequence Minimizing, 36 Set Attainable, 320 Constraint, 54, 312 Equality, 42 Convex, 275 Initial, 341, 373 Level, 54 Measurable, 499 Terminal, 341, 373 Sigma - algebra, 498 Solution Strong, 77, 296 Weak, 77, 78, 297 Space

544 Event, 499 Measurable, 498 Measure, 499 Probability, 499 Sample, 499 Stability Asymptotic, 471 Exponential, 471 State Adjoint, 347 Augmented, 343 Sufficient Condition Abstract, 275 Sufficient Condition (1), 196, 211 Sufficient Condition (2), 196, 212 Support Plane, 322 System Adjoint, 327 Augmented Control, 345 Autonomous, 475 Closed-Loop, 472 Co-state, 327 Control, 342 Detectable, 472 Disturbance, 502 Dual, 469 Linearized, 475 Stabilizable, 472 Theorem Fundamental Field, 194 Fundamental Sufficiency, 193 Hilbert’s Differentiability, 111,

Index 139, 206 Implicit Function, 229 Inverse Function, 49 Inverse Mapping, 48 Lagrange Multiplier, 42–44, 48, 257 Trajectory Nominal, 474 Valentine, Frederick A., 15 Variable Gaussian Random, 503 Random, 499, 503 Variation Admissible, 270 First, 68, 103, 213, 271 Gˆateaux, 271 Second, 69, 271 Second Gˆateaux, 271 Weierstrass, Karl T. W., 12 Young, Lawrence C., 13 Zermelo, Ernst F. F., 15