Optimal Control: An Introduction to the Theory and Its Applications (Dover Books on Engineering) [Illustrated] 0486453286, 9780486453286

Citation preview

Optimal Control

An Introduction to the

Theory and Its Applications Michael Athans Professor of Electrical Engineering (emeritus) Massachusetts Institute of Technology, Cambridge, Massachusetts and Invited Research Professor, Instituto Superior Tecntco,

Lisbon, Portugal

Peter L. Falb Professor of Applied Mathematics Brown University,

Providence, Rhode Island

Dover Publications, Inc. Mineola, New York

Copyright Copyright © 1966, 1994, 2007 All rights reserved.

by Michael Athans and Peter L. Falb

Bibliographical Note This Dover edition, first published in 2007, is an unabridged republication of

the work published by McGraw-Hill, Inc., New York, in 1966. A new Preface to the Dover Edition has been provided for this edition.

Library of Congress Cataloging-in-Publication Data Athans, Michael. Optimal control : an introduction Michael Athans and Peter L. Falb. p. cm.

to the theory

and

its applications /

New York : Dover Publications, 2007. Originally published: New York,

McGraw-Hill, 1966. ISBN 0-486-45328-6 (pbk.) 1. Automatic control. I. Athans, Michael. II. Falb, Peter L.

TJ213.A8 2007 629.8 22 0701

2006050500 Manufactured in the United States of America

Dover Publications, Inc., 31 East 2nd Street, Mineola, N.Y. 11501

To our children.

PREFACE TO THE DOVER EDITION It has been forty years since the original version of Optimal Control was published. Actually, the book was ready in 1965, but McGraw-Hill delayed its publication to have a 1966 copyright. Its selling price was $19.95, a princely sum in those days. Subsequently, the price rose to over $80. Optimal Control was written between 1962 and 1965 while we were staff members at the MIT Lincoln Laboratory. We thought we could write it in six months, but it took a bit longer. It was the first published textbook in the field, followed by the Bryson & Ho and Marcus & Lee texts a couple of years later.

Looking at the field forty years later, we marvel at the truly phenomenal growth of ideas, theories, methodologies, applications and design software. Optimal control, together with state-space representations and stochastic Kalman filtering concepts, are the foundations of what people still call “Modern Control and Systems Theory.” It would be mmpossible to summarize the exponential growth of discoveries in this exciting field; all of them are based upon rigorous mathematical principles. Surely, nobody ever could have predicted in the early 60s the powerful multivariable feedback design methods we have today,

such as H2, Heo and robust [1-synthesis accompanied by reliable MATLAB software. Advances in microprocessors have helped the application of Modern Control and Systems Theory to a variety of disciplines: MPC in chemical process control; aerial, underwater

and

ground

autonomous

robotic vehicles; numerous

automotive control systems; civilian and defense aerospace and marine vehicles; etc.

It is also very encouraging that, during the last decade, we see optimal! control

being used in novel multi-disciplinary applications such as immunology, biological systems, environmental systems, biochemical processes, communication networks, intelligent transportation systems and socio-economic systems. The Pontryagin Maximum Principle and the Hamilton-Jacobi-Bellman equation remain just as important today as they were forty years ago.

Our book Optimal Control has been described by some as a “classic.” It is for this reason that we have not changed it over the years. Clearly, some of the chapters in the book are “old-fashioned” by today’s standards. When we were writing the book, just after the translation of the Pontryagin et al research monograph,

vii

Vili

Preface to the Dover Edition

we believed that the future of optimal control would be primarily directed

toward nonlinear systems (hence the emphasis on the time-optimal and fuel-

optimal examples in Chapters 7, 8, and 10). The much briefer material in Chapter 9, dealing with the Linear-Quadratic problem, was added for the sake of completeness and at the urging of Prof. R. E. Kalman. In retrospect, it was the linear dynamic optimization methods that have blossomed. Solving nonlinear two-point boundary-value problems and/or computing approximate solutions to the Hamilton-Jacobi-Bellman partial differential equation still remain challenging issues; and let us not forget singular optimal control problems. We hope that, in the decades to come, the field will routinely deal with nonlinear optimal robust feedback control problems with similar unified methodologies and software as It does for their linear counterparts. Michael Athans

Professor

of Electrical

Engineering

(emeritus),

Massachusetts

Institute

Technology, Cambridge, Massachusetts, United States and Invited Research Professor, Instituto Superior Tecnico, Lisbon, Portugal

of

Peter L. Falb Professor of Applied Mathematics, Brown University, Providence, Rhode Island, United States

PREFACE Recent advances in control science have created the need for an introductory account of the theory of optimal control and its applications which will provide both the student and the practicing engineer with the background necessary for a sound understanding of these advances and their implications. This book is directed toward the fulfillment of that need. The book is primarily an introductory text for engineering students at approximately the first-year graduate level.

We hope that the basic mate-

rial provided in this text will enable the student, at an early stage in his career, to evaluate, on the one hand, the engineering implications of theoretical control work, and to judge, on the other hand, the merits and soundness of the numerous papers published in the engineering control literature. We have tried to make this book a bridge between the beginning student and the control literature, and to indicate the inestimable value to the student of careful thinking and disciplined intuition. In view of these hopes and aims, we have included a large number of examples and exercises. Structurally, the book is divided into the following three major parts: 1. Mathematical preliminaries related to the description and analysis of dynamical systems (Chapters 2 to 4)

2. Aspects of the theory of optimal control, including the maximum principle of Pontryagin (Chapters 5 and 6) | 3. Application of the optimal-control theory to the design of optimal feedback systems with respect to several performance criteria (Chapters 7 to 10) More specifically, we review the basic concepts of linear algebra and develop the vector notation used throughout the book in Chapter 2. In Chapter 3, we discuss the elementary topological properties of n-dimensional

Euclidean space, the basic theory of vector functions, and several aspects of

vector differential equations. In Chapter 4, we provide an introduction to the state-space representation of dynamical systems and we define the control problem.

We

derive and

study

the basic

conditions for optimality,

including the maximum principle of Pontryagin, in Chapter 5. The structure and general properties of optimal systems with respect to several specific performance criteria are examined in Chapter 6, which provides a buffer between the theoretical material of previous chapters and the design probix

x

Preface

lems considered in subsequent chapters. Chapter 7 is devoted to the solution of the time-optimal] control problem for a number of specific systems. In Chapter 8, we apply the theory to the design of several minimum-fuel systems. We present the general results available for an important class of optimization problems, namely, the class of control problems involving linear plants and quadratic performance criteria, in Chapter 9. In the final chapter, Chapter 10, we study a class of control problems which are more easily solved by direct methods than by methods based on the maximum principle of Pontryagin. We owe a significant debt of gratitude to many of our colleagues and students for their invaluable assistance in the preparation of this book. In particular, we wish to thank Prof. C. A. Desoer of the University of California at Berkeley, Dean J. G. Truxal of the Polytechnic Institute of Brooklyn, Prof. E. Polak of the University of California at Berkeley, Dr. H. K. Knudsen of the Lincoln Laboratory of the Massachusetts Institute of Technology, and Dr. S. J. Kahue of the Air Force Cambridge Research Laboratories for their careful reading of the manuscript and their numerous helpful suggestions and comments. We also wish to express our appreciation to Dr. H. Halkin of the Bell Telephone Laboratories at Whippany for his comments on Chapters 4 and 5, to Prof. R. W. Brockett of the Massachusetts Institute of Technology for his suggestions relating to Chapter 4, and to Prof. R. E. Kalman of Stanford University for his very helpful comments on Chapter 9. A number of students at the Massachusetts Institute of Technology have aided us with their suggestions and their careful proofreading, and we should like to especially thank D. Gray, D. Kleinman, W. Levine, J. Plant, and H. Witsenhausen for their exceptional assistance. R. A. Carroll of the Lincoln Laboratory provided us with the computer results in Chapter 9, for which we are very grateful. Finally, we should like to thank Miss J. M. Kelley, who did an outstanding job in typing the major

portion of the manuscript, and Mrs. S. M. McKay, who also did an excellent

job of typing for us.

Michael Athans Peter L. Falb

CONTENTSt

Pe — &®— OR

Mathematical Preliminaries : Algebra

Introduction Sets Operations on Sets Functions Vector Spaces Linear Combinations and Bases Linear Algebra: Linear Transformations and Matrices Linear Algebra: Operations on Linear Transformations and Matrices Linear Algebra: Linear Transformations of V into V Linear Algebra: Eigenvectors and Higenvalues Euclidean Spaces: Inner Products Euclidean Spaces: The Schwarz Inequahty Suclidean Spaces: Orthogonality and Norm Euclidean Spaces: Some Properties of the Scalar Product on &,, Euclidean Spaces: Some Properties of Symmetric Matrices

CHAPTER 3 3-1 *3-2 3-3 *3-4 3-5

=—

Introduction The System Design Problem The Control Problem Historical Perspective Aims of This Book Genera] Comments on the Structure of This Book Chapter Description Prerequisites and Study Suggestions

CHAPTER 2 2-1 2-2 2-3 2-4 2-5 *2-6 *% 2-7 % 2-8 *&2-9 *2-10 *2-11 2-12 2-13 2-14 2-15

Introduction

WON

1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8

1

Mathematical Preliminaries: Analysis

Introduction Distance and Distance and Distance and Distance and

Related Related Related Related

Notions: Notions: Notions: Notions:

Definition Spheres and Limits Open and Closed Sets Completeness and Contractions

SESE F

CHAPTER

a Be:

Preface to the Dover Edition Preface

a.

t See page 14 for an explanation of the starred sections.

xii

Contents

Properties of Sets in R,: Compactness 3-6 3-7 - Properties of Sets in #,: Hyperplanes and Cones Properties of Sets in 2,,: Convexity *3-8 Vector Functions: Introductory Remarks 3-9 *3-10 Vector Functions: Continuity 3-11 Vector Functions: Piecewise Continuity *3-12 Vector Functions: Derivatives 3-13 Vector Functions: Smooth Sets in R, Vector Functions: Integrals 3-14 *3-15 Vector Functions: Function Spaces *3-16 Vector Functions: Functionals Differential Equations: Introductory Remarks *3-17 3-18 Differential Equations: The Basic Existence Theorem Linear Differential Equations: Basic Concepts 3-19 *3-20 Linear Differential Equations: The Fundamental Matrix Time-invariant Systems: The Exponential of A 3-21 3-22 Time-invariant Systems: Reduction to Canonical Form Time-invariant Systems: Evaluation of the Fundamental Matrix 3-23 by Means of the Laplace Transform Time-invariant Systems: The nth-order System 3-24 3-25 The Adjoint System 3-26 Stability of Linear Time-invariant Systems

HAPTER 4

4-1 44-2 4-3 4-4 45 +46 A-7 4-8 4-9 *4-10 *% 4-11 %4-12 4-13 *%4-14 %4-15 4-16 %4-17 4-18 4-19 4-20 *% 4-21

Basic Concepts

Introduction

An RL Network A Multivariable System Dynamical Systems: Introductory Remarks Dynamical Systems: Forma! Definition Dynamical Systems: The Systems Considered in This Book Linear Dynamical Systems Input-Output Relationships and the Transfer Function Finding the State (or Dynamical-system) Representation of a Plant Whose Transfer Function Contains Only Poles Finding the State (or Dynamical-system) Representation of a Plant Whose Transfer Function Contains Poles and Zeros The Control Problem: Introductory Remarks The Control Problem: Formal Definition The Control Problem: Important Special Cases The Set of Reachable States Controllability and Observability: Definition Controllability for Linear Time-invariant Systems Observability for Linear Time-invariant Systems Practical Implications: Regulating the Output Practical Implications: The Effect of Canceling a Pole with a Zero Practical Imphcations: An Example Normal Linear Time-invariant Systems

65

74 75 80 82 92 95 100 108 110 115 122 125 132 135 138 143 147 149

161 151 151 157 159 163 168 171 173 175 182 190 191 195 197 202 207 211 215 215 217

Contents

CHAPTER 5 5-1 5-2 5-3 5-4

5-5 *5-6 * 5-7

5-8 5-9 * 5-10 *5-11 *5-12 *% 5-13 *5-14 % 5-15 * 5-16 5-17 *5-18 5-19 * 5-20 5-21

Introduction Ordinary Minima Ordinary Minima with Constraints: A Simple Problem Ordinary Minima with Constraints: Necessary Conditions and the Lagrange Multipliers Some Comments An Example The Variational Approach to Control Problem 1: Necessary Conditions for a Free-end-point Problem The Variational Approach to Control Problem 2: Sufficiency Conditions for the Free-end-point Problem The Variational Approach to Control Problem 3: A Fixed-end-point Problem Comments on the Variational Approach The Minimum Principle of Pontryagin: Introduction The Minimum Principle of Pontryagin: Restatement of the Control Problem The Minimum Principle of Pontryagin The Minimum Principle of Pontryagin: Changes of Variable Proof of the Minimum Principle: Preliminary Remarks A Heuristic Proof of the Minimum Principle Some Comments on the Minimum Principle Sufficient Conditions for Optimality: Introductory Remarks Sufficient Conditions for Optimality: An Equation for the Cost A Sufficient Condition for Optimality Some Comments on the Sufficiency Conditions

CHAPTER 6

6-1 6-2 *6-3 *6-4 *6-5 6-6 6-7

6-9 6-10 *6-11

Conditions for Optimality: The Minimum Principle and the Hamilton-Jacobi Equation

Structure and Properties of Optimal Systems

Introduction Minimum-time Interpretation Minimum-time Principle Minimum-time Minimum-time

Problems 1: Formulation and Geometric Problems 2: Application of the Minimum Problems 3: Comments Problems 4: Linear Time-invariant Systems

Minimum-time Problems 5: Structure of the Time-optimal

Regulator and the Feedback Problem Minimum-time Problems 6: Geometric Properties of the Time-optimal Control Minimum-time Problems 7: The Existence of the Time-optimal Control Minimum-time Problems 8: The Hamilton-Jacobi Equation Minimum-time Problems 9: Comments and Remarks Minimum-fuel Problems 1: Introduction

221 221 222 228 233 236 241 254 267 272 275 284

291 308 347 351 353 355 361

364 364 365 374 393 395 407 411 419 421 426 428

xiv

Contents

6-12 6-13 w6-14 6-15 6-16 6-17 *6-18 * 6-19 6-20 *6-21 6-22 6-23 6-24 6-25

Minimum-fuel Problems 2: Discussion of the Problem and of the Constraints Minimum-fuel Problems 3: Formulation of a Problem and Derivation of the Necessary Conditions Minimum-fuel Problems 4: Linear Time-invariant Systems Minimum-fuel Problems 5: Additional Formulations and Cost Functionals Minimum-fuel Problems 6: Comments Minimum-energy Problems 1: Introduction Minimum-energy Problems 2: The Linear Fixed-end-point, Fixed-time Problem Minimum-energy Problems 3: An Example Minimum-energy Problems 4: Magnitude Constraints on the Control Variables Singular Problems 1: The Hamiltonian a Linear Function of the Control Singular Problems 2: The Hamiltonian a Linear Function of the Control and of Its Absolute Value Some Remarks Concerning the Existence and Uniqueness of the Optimal and Extremal Controls Fixed-boundary-condition vs. Free-boundary-condition Problems Concluding Remarks

CHAPTER 7 *7-1 7-2 7-3 x74 7-5 7-6 *7-7 7-8 7-9 7-10 *7-11 w7-12 7-13 7-14 7-15

428 430 451 457 | 458 466 475 481 493 496 498 503

The Design of Time-optimal Systems

Introduction Time-optimal Control of a Double-integral Plant Time-optimal Control of Plants with Two Time Constants Time-optimal Control of a Third-order Plant with Two Integrators and a Single Time Constant Time-optimal Control of Plants with N Real Poles Some Comments Time-optimal Control of the Harmonic Oscillator Time-optimal Control of a Stable Damped Harmonic Oscillator Time-optimal Control of a Harmonic Oscillator with Two Control Variables Time-optimal Control of First-order Nonlinear Systems Time-optimal Control of a Class of Second-order Nonlinear Systems Time-optimal Control of a Double-integral Plant with a Single Zero Time-optimal Control of a Double-integral Plant with Two Zeros General Results on the Time-optimal Control of Plants with Numerator Dynamics Concluding Remarks

504 507 526 536 551 565 590 595 610 614 622 638 647

Contents

CHAPTER 8 8-1 8-2 8-3 *8-4 *8-5 *8-6 8-7 *8-8 8-9 %8-10 *8-11

Introduction First-order Linear Systems: The Integrator First-order Linear Systems: Single-time-constant Case Fuel-optimal Control of the Double-integral Plant: Formulation Fuel-optimal Control of the Double-integral Plant: Free Response Time Fuel-optimal Control of the Double-integral Plant: Fixed or Bounded Response Time Fuel-optimal Control of the Double-integral Plant: Response Time Bounded by a Multiple of the Minimum Time Minimization of a Linear Combination of Time and Fuel for the Double-integral Plant Minimization of a Linear Combination of Time and Fuel for the Integral-plus-time-constant Plant Minimization of a Linear Combination of Time and Fuel for a Nonlinear Second-order System Comments and Generalizations

CHAPTER 9 *9-1 *9-2 *9-3 9-4 *9-5 9-6 9-7 *9-8 *9-9 9-10 *% 9-11 9-12 *9-13

10-5

The Design of Optimal Linear Systems with Quadratic Criteria

Introduction Formulation of the Problem The State-regulator Problem Discussion of the Results and Examples The State-regulator Problem: Time-invariant Systems; T = Analysis of a First-order System The Output-regulator Problem The Output-regulator Problem for Single-input—Single-output Systems The Tracking Problem Approximate Relations for Time-invariant Systems Tracking Problems Reducible to Output-regulator Problems Analysis of a First-order Tracking System Some Comments

CHAPTER 10 10-1 10-2 10-3 10-4

The Design of Fuel-optimal Systems

Optimal-control Problems when the Control Is Constrained to a Hypersphere

Introduction Discussion of the Constraint ||u(¢t)|| < m Formulation of a Time-optimal Problem Analytical Determination of the Time-optimal Control for a Class of Nonlinear Systems Discussion of the Results

662 662 664 671 675 676 684 693 703 710 726 745

750 750 752 756 766 @71 776 782 788 793 801 804 806 813

815 815 816 817 821 824

xvi

10-6

10-7.

10-8

Contents

The Optimal Control of Norm-invariant Systems

Time-optimal Velocity Control of a Rotating Body with a Single Axis of Symmetry Suggestions for Further Reading

841 855

References Index

867

Chapter 1

INTRODUCTION l-]

Introduction

Considerable interest in optimal control has developed over the past decade, and a broad, general theory based on a combination of variational techniques, conventional servomechanism theory, and high-speed computa-

tion has been the result of this interest.

We feel that, at this point, there is

a need for an introductory account of the theory of optimal control and its applications which will provide both the student and the practicing engineer with the background and foundational material necessary for a sound understanding of recent advances in the theory and practice of control-system design. We attempt to fill this need in this book. We briefly indicate some of our aims and philosophy and describe the contents of the book in this introductory chapter. In particular, we set the context by discussing the system design problem in Sec. 1-2 and by specifying the particular type of system design problem which generates the control problem in Sec. 1-3. We then discuss (Sec. 1-4) the historical development of control theory, thus putting a proper temporal perspective on the book.

Next, we indicate (Sec. 1-5) the aims of the book. Following our statement of purposes, we make some general comments on the structure of the book in

Sec. 1-6, and we present a chapter-by-chapter description of its contents in Sec. 1-7. We conclude this chapter in Sec. 1-8 with a statement of the

necessary prerequisites and some study suggestions based on our teaching experience.

1-2

The System Design Problem

A system design problem begins with the statement (sometimes vague)

of a task to be accomplished either by an existing physical process or by a physical process which is to be constructed. For example, the systems

engineer may be asked to improve the yield of a chemical distillation column or to design a satellite communication system.

As an integral part of this

task statement, the engineer will usually be given: 1. A set of goals or objectives which broadly describe the desired per-

formance of the physical process; for example, the engineer may be asked to 1

2

Introduction

design a rocket which will be able to intercept a specified target in a reasonable length of time.

2. A set of constraints which represent limitations that either are inherent

in the physics of the situation or are artificially imposed; for example, there

are almost always requirements relating to cost, reliability, and size.

The development of a system which accomplishes the desired objectives

and meets the imposed constraints is, in essence, the system design problem. There are basically two ways in which the system design problem can be approached: the direct (or ad hoc) approach and the usual (or standard) approach.

Approaching the system design problem directly, the engineer combines experience, know-how, ingenuity, and the results of experimentation to produce a prototype of the required system. He deals with specific components and does not develop mathematical models or resort to simulation.

In short, assuming the requisite hardware is available or can be constructed, the engineer simply builds a system which does the job. For example, if an engineer is given a specific turntable, audio power amplifier, and loudspeaker and is asked to design a phonograph system meeting certain fidelity specifications, he may, on the basis of direct experimentation and previous experience,

conclude

that the requirements

can

be met

with

a particular

pre-

amplifier, which he orders and subsequently incorporates (“hooks up’’) in the system. The direct approach is, indeed, often suitably referred to as the “art of engineering.” Unfortunately, for complicated systems and stringent requirements, the direct approach is frequently inadequate. Moreover, the risks and costs involved in extensive experimentation may be too great. For example, no one would attempt to control a nuclear reactor simply by experimenting with it. Finally, the direct approach, although it may lead to a sharpening of the engineer’s intuition, rarely provides broad, general design principles which can be applied in a variety of problems. In view of these difficulties, the systems engineer usually proceeds in a rather different way. The usual, or standard, approach to a system design problem begins with the replacement of the real-world problem by a problem involving mathe-

matical relationships. In other words, the first step consists in formulating a suitable model of the physical process, the system objectives, and the imposed constraints. The adequate mathematical description and formulation of a system design problem is an extremely challenging and difficult task. Desirable engineering features such as reliability and simplicity are almost

impossible

to translate

into

mathematical

language.

Moreover,

mathe-

matical models, which are idealizations of and approximations to the real world, are not unique. Having formulated the system design problem in terms of a mathematical

model, the systems engineer then seeks a pencil-and-paper design which represents the solution to the mathematical version of his design problem.

Sec. 1-3

The control problem

3

Simulation of the mathematical relationships on a computer (digital, analog,

or hybrid) often plays a vital role in this search for a solution.

The design

obtained will give the engineer an idea of the number of interconnections required, the type of computations that must be carried out, the mathematical description of subsystems needed, etc. When the mathematical relations that specify the overall system have been derived, the engineer often simulates these relations to obtain valuable insight into the operation of the system and to test the behavior of the model under ideal conditions. Conclusions about whether or not the mathematics will lead to a reasonable physical system can be drawn, and the sensitivity of the model to parameter variations and unpredictable disturbances can be

tested.

Various alternative pencil-and-paper designs can be compared and

evaluated. After

completing

the

mathematical

design

and

evaluating

it through

simulation and experimentation, the systems engineer builds a prototype,

or “‘breadboard.”’ The process of constructing a prototype is, in a sense, the reverse of the process of modeling, since the prototype is a physical sys-

tem which must adequately duplicate the derived mathematical relation-

ships. The prototype is then tested to see whether or not the requirements are met and the constraints satisfied. If the prototype does the job, the

work of the systems engineer is essentially complete. Often, for economic and esthetic reasons, the engineer is not satisfied with a system which simply accomplishes the task, and he will seek to improve or optimize his design.

The process of optimization in the pencil-and-paper

stage is quite useful in providing insight and a basis for comparison, while the process

of optimization

in the prototype

concerned with the choice of best components.

building

stage is primarily

The role of optimization in

the control-system design problem will be examined in the next section. 1-3

The Control Problem

A particular type of system design problem is the problem of ‘‘controlling”’ a system. For example, the engineer may be asked to design an autopilot with certain response characteristics or a ‘‘fast’’ tracking servo or a satellite attitude

control

system

which

does

not

consume

too

much

fuel.

The

translation of control-system design objectives into the mathematical language of the pencil-and-paper design stage gives rise to what will be called the control problem. The essential elements of the control problem are: 1. A mathematical model (system) to be ‘“‘controlled”’

2. A desired output of the system 3. A set of admissible inputs or ‘‘controls”’

4. A performance or cost functional which measures the effectiveness of a

given “control action”’

4

Introduction

How do these essential elements arise out of the physical control-system

design problem?

The mathematical model, which represents the physical system, consists

of a set of relations which describe the response or output of the system for

various inputs. Constraints based upon the physical situation are incorporated in this set of relations. In translating the design problem into a control problem, the engineer is faced with the task of describing desirable physical behavior in mathematical terms. The objective of the system is often translated into a requirement on the output.

For

example,

if a tracking

servo

is being

designed,

the

desired output is the signal being tracked (or something close to it). Since “control” signals in physical systems are usually obtained from equipment which can provide only a limited amount of force or energy, constraints are imposed upon the inputs to the system. These constraints lead to a set of admissible inputs (or ‘control’ signals). Frequently, the desired objective can be attained by many admissible inputs, and so the engineer seeks a measure of performance or cost of control

which will allow him to choose the “‘best’”’ input. The choice of a mathematical performance functional is a highly subjective matter, as the choice of

one design engineer need not be the choice of another. The experience and intuition of the engineer play an important role in his determination of a suitable cost functional for his problem. Moreover, the cost functional will depend upon the desired behavior of the system. For example, if the engineer wishes to limit the oscillation of a system variable, say x(t), he may assign a cost of the form [z(¢)]® to this variable and try to make the integral of this cost over a period of time, say t; < ¢ < te, small. Most of the time, the cost functional chosen will depend upon the input and the pertinent system variables. When a cost functional has been decided upon, the engineer formulates his control problem as follows: Determine the (admissible) inputs which generate the desired output and which, in so doing, minimize (optimize) the

chosen performance measure. At this point, optimal-control theory enters the picture to aid the engineer in finding a solution to his control problem. Such a solution (when it exists) is called an optimal control. To recapitulate, a control problem is the translation of a control-system

design problem into mathematical terms; the solution of a control problem is an (idealized) pencil-and-paper design which serves to guide the engineer in developing the actual working control system. 1-4

Historical Perspective

Before World War II, the design of control systems was primarily an art. During and after the war, considerable effort was expended on the design of closed-loop feedback control systems, and negative feedback was used to

Sec. 1-4

Historical perspective

improve performance and accuracy.

5

The first theoretical tools used were

based upon the work of Bode and Nyquist.

In particular, concepts such as

frequency response, bandwidth, gain (in decibels), and phase margin were

used to design servomechanisms in the frequency domain in a more or less trial-and-error fashion. This was, in a sense, the beginning of modern

automatic-control engineering. The theory of servomechanisms developed rapidly from the end of the war to the beginning of the fifties. Time-domain criteria, such as rise time, settling time, and peak overshoot ratio, were commonly used, and the introduction of the “‘root-locus’”’ method by Evans in 1948 provided both a bridge between the time- and frequency-domain methods and a significant new design tool.

During this period, the primary concern of the control engineer

was the design of linear servomechanisms. Slight nonlinearities in the plant and in the power-amplifying elements could be tolerated since the use of negative feedback made the system response relatively insensitive to variations and disturbances. The competitive era of rapid technological change and aerospace explora-

tion which began around mid-century generated stringent accuracy and cost

requirements as well as an interest in nonlinear control systems, particularly relay (bistable) control systems. This is not surprising, since the relay is an

exceedingly simple and rugged power amplifier.

Two approaches, namely,

the describing-function and phase-space methods, were used to meet the new

design challenge. The describing-function method enabled the engineer to examine the stability of a closed-loop nonlinear system from a frequencydomain point of view, while the phase-space method enabled the engineer to design nonlinear control systems in the time domain. Minimum-time control laws (in terms of switch curves and surfaces) were obtained for a variety of second- and third-order systems in the early fifties. Proofs of optimality

However,

were

more or less heuristic and geometric in nature.

the idea of determining an optimum system with respect to a

specific performance measure, the response time, was very appealing; in addition, the precise formulation of the problem attracted the interest of the mathematician. The time-optimal control problem was extensively studied by mathematicians in the United States and the Soviet Union. In the period from

1953 to 1957, Bellman, Gamkrelidze, Krasovskii, and LaSalle developed the basic theory of minimum-time problems and presented results concerning the existence, uniqueness, and general properties of the time-optimal control. The

recognition

that

control

problems

were

essentially

problems

in the

calculus of variations soon followed. Classical variational theory could not readily handle the “hard” constraints usually imposed in a control problem. This difficulty led Pontryagin to first conjecture his celebrated maximum principle and then, together with Boltyanskii and Gamkrelidze, to provide a proof of it. The

8

Introduction

2.. Aspects of the theory of optimal control, including the maximum principle of Pontryagin, for continuous, finite-dimensional, deterministic systems (Chaps. 5 and 6) 3. Application of the theory to the design of optimal feedback systems with respect to several performance criteria (Chaps. 7 to 10) We have attempted throughout the book to state definitions, theorems, and problems in a careful, precise manner; on the other hand, we have often only sketched proofs, and we have either omitted or left as exercises for the reader many proofs, giving suitable references to the literature. We have found that such a procedure is quite appealing to engineering students. We use the compact notation of set theory and vector algebra to avoid

obscuring general concepts in a cloud of complicated equations. We have found that students, after an initial transition period, adjust to this terminology and become

accustomed

to thinking ‘‘physically”’ in terms of sets,

vectors, and matrices. We attempt to stress general ideas, and we try to illustrate these ideas with a number of examples. We do not, however, attempt to gloss over the difficulties, particularly those of a computational

nature, associated with the analysis and design of systems involving many variables.

Many carefully worked examples are included in the book to provide the reader with concrete illustrations of the general theory. We believe that

these examples serve to deepen intuition. In addition, we have cises fall into three categories: 1. Exercises which are routine. 2. Exercises which involve and well as some of the computational tion of an optimal control.tf

the understanding and strengthen the included numerous exercises. The exer-

illustrate ‘‘fine points” of the theory as difficulties associated with the determina-

3. Exercises which challenge the student to discover new aspects of the

theory and its applications. Several of these are of the ‘almost impossible” variety. We feel that the educational value of exercises is very great. We cross-reference frequently, and in addition, we pinpoint the references

which are relevant to a particular discussion so that the interested student can profitably consult the literature.

with this in mind.

However,

We include an extensive bibliography

we do not claim that our bibliography is

complete, for we have simply included the papers, reports, and books with

which we are familiar.

Since our purpose is to provide an introduction to the theory of optimal

control and its applications, we have not discussed a number of important topics which are of an advanced nature or which require additional preparation. In particular, the following topics are not included: t Several] exercises in Chap. 9 require the use of a digital computer.

Sec. 1-7

Chapter description

9

1. Computational algorithms for the determination of optimal controls

for complex systems

2. Problems with state-space constraints

3. Optimal-control theory for discrete (or sampled-data) systems

4. Problems involving distributed-parameter systems 5. The optimal control of stochastic systems

6. The design of optimal filters, predictors, or smoothers

Briefly, some of our specific reasons for omitting these important topics

are as follows: 1. Although

considerable research effort has been (and is) devoted to the

development of convergent computational algorithms, there have been (with some notable exceptions) few general results which guarantee convergence or contain information regarding the speed of convergence of an algorithm. 2. State-space constraints are more difficult to handle than control con-

straints. Necessary conditions for problems involving state-space constraints are available in the literature but do not, in our opinion, represent material of an introductory nature. 3. Currently, the theory for discrete, distributed-parameter, and stochastic systems is being developed. Since the study of distributed-parameter systems requires a knowledge of partial differential equations and since the study of stochastic systems requires a knowledge of advanced probability theory (including stochastic differential equations), these topics are clearly not suitable for an introductory text. 4. The Wiener-Kalman-Bucy theory can be used to design optimal filters for linear systems with Gaussian noise processes. However, general results pertaining to nonlinear systems and non-Gaussian

noise are not currently

available; moreover, the interested reader can readily follow the WienerKalman-Bucy theory after digesting the deterministic material included in this book.

We shall present a chapter-by-chapter description of the contents of the

book in the next section. 1-7

Chapter Description

We present a brief description of the contents of each chapter, taking care to point out the significant interrelationships between chapters.

Chapter 2.

Mathematical Preliminaries: Algebra

We review the basic concepts of linear algebra and develop the vector and matrix notation used throughout the book in this chapter. After a brief discussion of the familiar notions of set theory, we introduce the con-

cept of a vector space and then consider linear transformations. We treat matrices from the point of view of linear transformations rather than with

10

Introduction

the more familiar array-of-numbers approach. We stress that a matrix is associated with a linear transformation and given coordinate systems, thus

indicating

that

linear transformations

are intmnsic

whereas

matrices

are

not. In the remainder of Chap. 2, we discuss eigenvalues and eigenvectors, similarity of matrices, inner and scalar products, Euclidean vector spaces, and some properties of symmetric matrices. The material in this chapter is used in every other chapter of the book. Chapier 3 We

Mathematical Preliminaries: Analysis

discuss the elementary

topological

(metric)

properties

of n-dimen-

sional Euclidean space, the basic theory of vector functions, and several aspects of vector differential equations in this chapter. We begin by studying the notion of distance and the related concepts of convergence, open and closed sets, and compactness. We then define and examine the notion of convexity. Next, we consider functions of several variables, developing the definitions of continuity, piecewise continuity (or regularity), derivative,

gradient,

and integral for such functions.

We

introduce the important

concepts of a function space and the distance between elements of a function

space. In the remainder of Chap. 3, we are concerned with vector differential equations. We prove the basic existence-and-uniqueness theorem, and we develop the method of solution of linear differential equations by means of the fundamental matrix.

We make frequent use of the material

in this chapter in the remaining chapters of the book. Chapter 4

Basic Concepts

We provide an introduction to the state-space representation of dynamical systems, and we define and discuss the control problem in this chapter. We start with an examination of the concept of state for some simple network systems. After an informal introduction to the mathematical description of physical systems, we present an axiomatic definition of a dynamical system. We then consider finite-dimensional continuous-time differential systems, and we use the material of Chap. 3 to establish the relationship between

the state and

representing the system.

the initial conditions

of the differential equations

We describe the technique for finding a state-

variable representation of single-input-single-output constant systems in Secs. 4-9 and 4-10. We indicate the physical significance of the state

variables by means of an analog-computer simulation of the differential equations. We complete the chapter with the definition of the control problem and a discussion of some of the natural consequences of that definition. In particular, we consider the set of reachable states and the qualitative notions of controllability, observability, and normality. We also develop some

of the important

implications of these concepts.

The

basic material of this chapter is the cornerstone of the theory and examples which we develop in subsequent chapters.

Sec. 1-7

Chapter description

11

Chapter 5 Conditions for Optimality: The Minimum Principlet and the Hamilton-Jacobi Equation We derive and study the basic conditions for optimality in this chapter.

We include a review of the theory of minima for functions defined on n-dimensional Euclidean space, an indication of the application of the techniques of the calculus of variations to control problems, a statement and heuristic proof of the minimum principle of Pontryagin, and a discussion

of the Hamilton-Jacobi equation, based on a combination of Bellman’s “principle of optimality” and a lemma of Caratheodory in a manner suggested by Kalman. To bolster the intuition of the student, we start with ordinary minima (Secs. 5-2 to 5-4), and we review the Lagrange multiplier technique for constrained minimization problems. We then introduce the

variational approach to the control problem

(Secs. 5-5 to 5-10), indicating

the step-by-step procedure that can be used to derive some necessary and some sufficient conditions for optimality. We next develop the necessary conditions for optimality embodied in the minimum principle of Pontryagin (Secs. 5-11 to 5-17). We carefully state several control problems and the

versions of the minimum principle corresponding to each of these problems. We give a heuristic and rather geometric proof of the minimum principle

in Sec. 5-16, and we comment on some of the implications of the minimum principle in Sec. 5-17. We conclude the chapter with a discussion of the Hamilton-Jacobi equation and the sufficiency conditions associated with it. The minimum principle is the primary tool used to find optimal controls in Chaps. 7 to 10. Chapter 6

Structure and Properties of Optimal Systems

We discuss the structure and general properties of optimal systems with respect to several specific performance criteria in this chapter. We indicate the methods which may be used to deduce whether or not optimal and extremal controls are unique, and we briefly consider the question of singular

controls. the

In developing the chapter, we aim to provide a buffer between

theoretical

material

of the

previous

chapters

and

the

more

specific

design problems considered in subsequent chapters. We begin with an examination of the time-optimal control problem for both linear and nonlinear systems (Secs. 6-2 to 6-10). After discussing the time-optimal problem geometrically, we use the minimum principle to derive suitable necessary conditions. We then define the notions of singularity and normality and state the ‘“‘bang-bang’”’ principle for normal time-optimal systems. We present a series of theorems dealing with existence, uniqueness, and number of switchings for linear time-invariant systems in Sec. 6-5. The material t We speak of the minimum principle rather than the maximum principle in this book.

The only difference in formulation is a change of sign which makes no essential difference in the results obtained.

12

Introduction

in this portion of the chapter is used in Chap. 7. cussion

of the

minimum-fuel

control problem

We continue with a dis-

(Secs.

6-11

to 6-16).

We

again use the minimum principle to derive necessary conditions, and we again follow this with a definition of the singular and normal fuel-optimal problems. We establish the ‘‘on-off’” principle for normal fuel-optimal problems, and we develop uniqueness theorems for linear time-invariant systems. We consider several additional problem formulations involving both time and fuel in Sec. 6-15. The material in this part of the chapter is used in Chap. 8. We next study minimum-energy problems (Secs. 6-17 to 6-20).

We derive an analytic solution for the fixed-terminal-time—fixed-

terminal-state problem for time-invariant linear systems, and we illustrate the general results with a simple example. The results of this segment of the chapter are applied and expanded upon in Chap.9. Weconclude the chapter with a brief introduction to singular problems (Secs. 6-21 and 6-22) and some comments on the existence and uniqueness of optimal and extremal controls. Chapter 7

The Design of Time-optimal Systems

We apply the theory of Chaps. 5 and 6 to the time-optimal control problem for a number of specific systems in this chapter. We illustrate the construction of time-optimal controls as functions of the state of the system, and we discuss the structure of time-optimal feedback systems, indicating

the types of nonlinearities required in the feedback loop. We also comment on the design of suboptimal systems. We begin, in Secs. 7-2 to 7-6, with a consideration of plants whose transfer functions have only real poles. The complexity of the time-optimal feedback system is illustrated by comparing the time-optimal systems for second-order plants (Secs. 7-2 and 7-3) with that of a third-order plant (Sec. 7-4). Next, we examine the time-optimal problem for harmonic-oscillator-type plants (Secs. 7-7 to 7-9). We show (Secs. 7-10 and 7-11) how the concepts developed for linear systems can be applied to the time-optimal control problem for a class of nonlinear systems, using experimental

data

and

graphical

constructions.

We

conclude

the

chapter with an introductory discussion of plants having both zeros and poles in their transfer function (Secs. 7-12 to 7-15).

We indicate the effects

of minimum and non-minimum-phase zeros on the character and structure of the time-optimal system. Chapter 8

The Design of Fuel-optimal Systems

We apply the theory of Chaps. 5 and 6 (and some results of Chap. 7) to the design of a number of minimum-fuel systems. We show that conservative systems often have nonunique minimum-fuel controls. We illustrate ways in which the response time can be taken into account in minimum-fuel

problems, indicating the engineering implications of the resultant optimal system. We also point out that nonlinear systems often admit singular fuel-optimal solutions (in marked contrast to the time-optimal case). We

Sec. 1-8

Study suggestions

13

begin the chapter by illustrating the nonuniqueness of fuel-optimal controls (Secs. 8-2 to 8-5). Different methods for including the response time in the performance functional are examined in Secs. 8-6 to 8-10. The possibility of singular solutions is also discussed in Sec. 8-10, and we conclude

with some comments on the graphical methods that are frequently used to determine the optimal control as a function of the state in such minimum-

fuel problems. Chapter 9

The Design of Optimum Linear Systems with Quadratte Criteria

In this chapter, we present the general results available for an important

class of optimization problems, namely, the class of control problems involving linear time-varying plants and quadratic performance criteria. In view of some results of Kalman on the inverse problem, the material in this chapter may be construed as, in a sense, a generalization of conventional control theory. We begin with the state-regulator problem (Secs. 9-2 to

9-6); then we consider the output-regulator problem (Secs. 9-7 and 9-8); and finally, we conclude the chapter with an examination of the tracking problem (Secs. 9-9 to 9-13). Since linear time-invariant systems are studied

in Secs. 9-5, 9-8, and 9-11, we obtain a degree of correlation between the

properties of optimal systems and well-designed servo systems, as conven-

tional servomechanism theory can be applied to the design of linear timeinvariant feedback systems. However, the theoretical results developed

in this chapter can easily be applied to the control of processes (e.g., multi-

variable or time-varying plants) for which the conventional servomechanism theory cannot be used directly. Chapter 10 Optimal-control Problems when the Control Is Constrained to a Hypersphere

In this chapter, we study a certain broad class of control problems which

can be solved more easily by direct methods than by methods based on the minimum

principle.

The

basic ideas are discussed in Secs.

10-2 to 10-6.

We also illustrate, in a specific example, the effect of constraints upon the

design

and

properties

formance criteria. 1-8

of an optimal

system

with

respect

to several

per-

Prerequisites and Study Suggestions

We suppose that the engineering student using this book as a text has, in general, the following background: . 1. A knowledge of basic conventional servomechanism theory, including

such ideas as the system response, the transfer function of a system, feedback, and linear compensation

2. A knowledge of the basics of ordinary differential equations and cal-

culus, including the Laplace transform method for the solution of linear

14

Introduction

differential equations with constant coefficients and the notion of a matrix,

together with some skill in performing matrix manipulation.

We consider this background adequate for an understanding of the major portion of the book. We do not expect the student to be an expert in linear algebra or in vector differential equations; consequently, we incorporate in the text the specific mathematical maternal that we need. The subject matter contained in the book has been covered in a onesemester course in the Department of Electrical Engineering at the Massa-

chusetts Institute of Technology.

Sections indicated with a star in the

table of contents were treated in the classroom, and the remaining sections

were assigned reading. No text should be followed verbatim, and different teachers will expand upon or delete various topics, depending on their own

inclinations and the level of their students’ preparation.

In this regard,

we note that Chaps. 8 to 10 are independent of each other and so can be treated in any order. Also, for students with a more advanced background (say, for example, a course based on the book ‘‘Linear System Theory,”’ by Zadeh and Desoer),

Chaps.

2 to 4 can be omitted from

the classroom

and simply assigned as review reading and the material in Chaps. 5 and 6 developed in greater detail.

Chapter 2

MATHEMATICAL ALGEBRA

2-1

PRELIMINARIES:

Introduction

We shall use a number of mathematical results and techniques throughout this book. In an attempt to make the book reasonably self-contained, we collect, in this chapter and the next, the basic definitions and theorems required for an understanding of the main body of the text. Our treatment will be fairly rapid, as we shall assume that the reader has been exposed to

most of the mathematical

notions we need.

Also, our treatment

will be

incomplete in the sense that we shall present only material which is used in the sequel. We shall discuss sets, functions, vector spaces, linear algebra, and Euclidean spaces in this chapter. In essence, most of our discussion will be aimed at translating the familiar physical notion of linearity into mathematical terms. As a consequence of this aim, we shall view the concepts of a vector space and of a linear transformation rather than the concepts of an n-tuple of numbers and of a matrix as basic. We shall suppose that the reader is familiar with the notion of determinant

and its relation to the solution of linear equations. The particular information that is needed can be found in Ref. B-9,¢ chap. 10, or in Ref. B-6, app. A. The material in this chapter, in whole or in part, together with various

extensions and ramifications of it, can be found, for example, in Refs. B-6,

B-9, D-5, H-4, K-20, R-10, S-4, and Z-1.

2-2

Sets

We consider objects which have various properties related to one another. A collection of objects which distinguishing properties is called a set. We use € -to in a set; in other words, if A is a set, then a C A means

and which can be have common and denote membership that a is a member

¢ Letter-and-number citations are keyed to the References at the back of the book. 15

16

Mathematical preliminaries: algebra

(or element) of A. Two sets are the same if and only if they have the same elements. If A and B are sets, then we shall say that A is contained in B or is a subset of B if every element of A is an element of B. We write this inclusion relationship in the form ACB

(2-1)

If A is a set and if © is a property which elements of A may have, then we let {a: @(a)} stand for the set of elements of A which actually have the property ©. For example, if A is the set of all girls in New York and @ is the property of being a blonde, then {a: @(a)} is the set of all blonde girls in New York.

Example 2-1 If R is the set of real numbers and if © is the property “| | 0) such that kk i=lY Bt = belly

(2-146)

Thus, we have established part a of Theorem 2-5. Example 2-38

Suppose that

is a symmetric 2 X 2 matrix. gu >O0

On Lie an

Then Q is positive definite if and only if and

gigs. — Gis? > 0 Qu

if

Q= |

Gai

Qua

is

az

33

qat

=|

is 8 symmetric 3 X 3 matrix, then Q is positive definite if and only if gi >O0

and

911922 — 912% > O

and

det Q >0

This example illustrates part b of the theorem in the special cases where n = 2andn

= 3.

Chapter 3

MATHEMATICAL ANALYSIS

8-1

PRELIMINARIES:

Introduction

We continue the development of the basic mathematical concepts necessary for an understanding of the main body of the book in this chapter. Previously, we examined ideas which depended upon such algebraic notions as addition and multiplication by a real number. We shall now turn our attention to ideas which depend upon the notions of distance and magnitude. In particular, we shall study distance, convexity, vector functions, differential equations,

and linear systems

in this chapter.

shall, for the sake of completeness, develop some of the what is generally known as advanced calculus. We again note that our treatment will be fairly rapid, the reader has met with most of the ideas we need. We our treatment will be incomplete in that we shall present in the sequel.

Moreover,

we

familiar results of

as we assume that also observe that only material used

Moreover, we do not develop the concepts any further than

is necessary for our purposes. The interested reader will find the material of this chapter, in whole or in part, together with various generalizations of it, in Refs. B-6, B-10, C-7, D-5, E-3, K-20, P-6, R-10, S-4, V-1, and Z-1 (and many other books). 3-2

Distance and Related Notions: Definition

We begin now to study a very basic concept, the notion of distance, and some of its consequences. Let us examine our intuitive idea of distance

and try to abstract the essential elements from it.

First of all, we observe

that we usually speak of the distance between two points as in “‘the distance between New York and Boston is 208 miles.” Secondly, we note that the distance between two points is zero if and only if the points are the

same.

For example, the distance between Boston and Boston is zero miles.

Thirdly, we usually think of the distance between two points as being inde-

pendent of the order in which the points are mentioned; e.g., the distance

between New York and Boston is the same as the distance between Boston

54

Sec. 8-2

Distance

and New York. Finally, sense that if A, B, and C is no greater than the sum between B and C. For

55

our intuitive idea of distance is triangular in the are three points, the distance between A and C of the distance between A and B and the distance example, the distance between New York and

Boston is less than the sum of the distance between New York and Albany

and the distance between Albany and Boston. These observations lead us to the following precise formulation of the notion of distance:

Definition 3-1

If X is a set, then a real-valued function d, which is defined

on all of X XK X, is called a distance function (or distance or metric) on X if

the following conditions are satisfied:

D1 D2 D3 We

d(z, y) > 0 for all x, y in X and d(z, y) = 0 tf and only tf x = y. d(x, y) = d(y, x) for all x, y in X. d(x, z) < d(x, y) + d(y, z) for all x, y, z in X.

often refer to d(x, y) as ‘‘the distance between x and y,” and we call D3

the triangle inequality. A set X together with a given distance function d on X 18 called a metric space. If X is a Euclidean space with P as inner product (see Secs. 2-11 and 2-13), then we may define a distance d on X by setting

d(x, y) = |x —yll>p = V P(e —y, x —y) where ||x — y/|p represents the norm in X. In view of properties N1 and erty IP3 of P (Sec. 2-11), we see that on X. In the particular case where

of x — y N2 (Sec. Eq. (3-1) X is the

product on F, [Eq. (2-86)], we have, for

21 |

Y2

and

L En

or, equivalently,

y=

LYn_

d(x, y) = |x —yll = V(x —y, x — y)

(3-2)

Tax] fu] £2

d}

Sec. 2-13) and x, y are of the norm and propindeed define a distance R#, and P is the scalar

ryt|

X2

x=

(see 2-13) does space

(3-1)

He

Y2

pb]. |} =

(Ltn

.

LYnL

1)

Y (a — w)* = [x - yl

f=

(3-3)

J

We call this distance function the natural, or Euclidean, distance (or metric)

on R,.

|

56

Mathematical preliminaries: analysis Example 8-1

If X = £ is the set of real numbers, then the natural distance between

z and y, dz, y), is simply the absolute value of z — y, that is, d(z, y) = |z — y|. example, d(5, 3) = |5 — 3| = 2. Example 3-3 If

For

“f= fl = 8

then d(x, z) = ||x — z|| = +/5, d(x, y) = |lx — yll = 2 +/6, and d(y, z) = lly — s/| = 3.

We note that +/5 < 2 +/6 + 3, that 2 1/6 < 4/5 + 3, and that3 < 1/5 +2 /6.

Example 3-3 If X is any set and if we set d(x, y) = lifz # yand d(z, z) = Oforz,y in X, then d is a distance function on X. Example

3-4

Suppose that X = R, and that x = [2]

and y = [¥].

Then the

function d(x, y) defined by d(x, y) = |x: — y:| + |z2 — ys| is a distance function on R:. Exercise 8-1

Show that the function d of Example 3-4 is indeed a distance on Re. 2 Generalize this distance to R,. Hint: In Example 3-4, d(x, y) = y lzx — ysl. Exercise 8-2

If we let max

defined by d ({2'], |”)

= max

. . 9: and generalize this distance to R,.

8-3

t=1

{a, b} denote the maximum of a and 5, then the function d {|z; — yal, |z2 — yal} is a distance on Ry.

v1 yi ||) Hunt: . d ({2'],

= max

Prove this

Ye {las . —— ysl}.

Distance and Related Notions: Spheres and Limits

Let us suppose that X is a set, that d is a distance on X, and that x» is a fixed element of X. We are often interested in the points which are within a given distance of the fixed point 2.

S(z0, p) defined by

If p > 0is areal number, then the set

S(ao, p) = {x © X: d(ao, x) < p}

(3-4)

is called the open sphere of radius p about zo. In other words, S(zo, p) is the set of elements of X whose distance from 2 is less than p. For example, if X = R and if we are using the natural distance on R, then S(0,

lr] < 1}.

Example

We note that if p < ¢, then S(20, p) C S(xo, ).

8-5

distance [Eq. Fig. 3-1a.)

Let X be the space FR: and let x, = [3] (3-3)], we see that S(xo, 1) is the set {x

Then,

- | 2]: 20

1) =

{r:

using the natural + 2,3 < i}.

(See

If we use the distance of Exercise 3-2, then S(xo, 1) is the set {x =s [2]:

max {lz:\, [zal} < i}

(See Fig. 3-1b.)

In a similar way, if p > 0 is a real number, then the set S(zo, p) defined by

S(ao,p) = {x © X: d(zo, x) < p}

(3-5)

Sec. 3-3

Spheres and limits

2s called the closed sphere of radius p about x.

57

In other words, S(zo, p) is the

set of elements of X whose distance from 2p is not greater than p.

For exam-

ple, if X = R and if we are using the natural distance on R, then S(0, 1) =

{r: |r] < 1}. The reason for using the terms open and closed will appear in the next section. Finally, we observe that, in the particular case of R, with the

distance

x — y||, we have

S(X0, p) = {x © Ry: |lx — Xoll < p} S(xo, p) = {x E R,: ||x — xol| < p}

and

(3-62) (3-65)

Having defined the notions of open and closed sphere about a point,we are now ready to develop the idea of convergence. So we suppose that z,,

n=1,

2,...,

18 a sequence

of ele-

ments of X. What does it mean to say that the sequence approaches zp or tends to xo or converges to zo? Intuitively, we view the sequence as tending to Xo if the elements z, get closer and

closer to x» as nm increases. More precisely, we have: Definition 3-2 Convergence The

sequence

{t,:

n=

1,

2,...}

is

said

to converge to xo if, given any real num-

ber « > 0, there is. an N(e) such that if-» n> .

N(e),

18, d(xo,

then

ta)


j=l

m

y

7) aij(rj

— 8;)

m™

G2;(r; — 8;)

>»

(Q(v — w), @(v — w)) =

=

a2;(r; — 8;)

,

j

y =i

Gaj(T3

—

8;)

JS

y

Lem!

Gnj(T;

—

83)

-

[using Eq. (2-32)]}, from which it follows that (Q(v — w), @(v — w)) =

y

s=l

{ y

jg=ml

< xt { y

airy — s;)}? (1; —- ;)*}

< lv — wll? { >

t=]

and hence that

@ is continuous.

(y, a:;?}

x a;;*}

j=l

using Eq. (2-91)

Sec. 8-10 Example 8-25 Fig. 3-18). Then setting f(x) = ||x|| Example 3-26

Continuity

79

Let f be the function from RF into itself, defined by f(z) = |z| (see f is continuous. In fact, the function f from R,, into R, defined by for xin Rp, is a continuous function. If

is an element of R,, and if we define a function f from Rp into F by setting

f(x) = (that is, f(x) is the maximum

max —

a=1,2,...,™

{lz}

of the absolute values of the components of x}, then f is

7

continuous, since |z;| < ( »

z;2)} for: = 1,2,

...,m.

j=l

x2

F(x) =[x| ———g~ X

5

Fig. 3-18 The continuous function f(z) = |z]. Example 8-27

X

Fig. 3-19 The set A is closed, but the projection of A on the z, axis is not closed.

If v1 Zo

x=| 2 is an element of R,, and if we define functions ¢:, 1 = 1,2, ..., m, from Re into R by setting ¢:(x) = x:, then the ¢: arecontinuous. In particular, if we are considering A: and if we set ¢, ({*])

= 2, then ¢: is continuous.

If A is the subset of R22, defined by

but ¢:(A) is not A= {[ 21: Z2 = 1/x, m1 > o} (see Fig. 3-19), then A is closed in Re closed in R. Consider the remarks after Theorem 3-6 in the light of this example.

Exercise 8-11 Show that if g and h are continuous functions from R» into Rn, then g¢ + hb is a continuous function from R, into Ry. Hint: If u and v are in Ra, then

(ge + b)(u) — (g + bY(¥)I| < lle(u) — gv)|| + b(u) — b(v)I.

80

Mathematical preliminaries: analysis

Now let us suppose that f is a continuous function from FR, into R, and that C is a compact subset of R,, (see Definition 3-14). We shall soon show

(Theorem 3-7) that f(C) [see Eq. (2-14)] is a compact subset of R,. A particularly important consequence of this is the fact that a real-valued continuous function actually attains both a maximum and a minimum value when

restricted to a compact set.

Theorem 3-7

If f£ is a continuous transformation from R,, into R,, and if

C 18 a compact subset of Rm, then £(C) is a compact subset of Ra.

PROOF We shall use property C4 (see Sec. 3-6) to prove that f(C) is compact.f Suppose that y,, . = 1, 2, ... , 1s a sequence of elements of

f(C); then there are elements x,, 7 = 1,2,

forn = 1, 2,....

...

, of C such that f(x.) = yn

Since C is compact, there is a subsequence Xn,, Xn;

. of the sequence {x,} which has a limit x in C.

However, it follows

from the continuity of f that ya, = f(xn,), Ya, = f(x,,),

. . . converges to

f(x), which is an element of f(C). In other words, yn,, Yn, - - . is the desired convergent subsequence of the sequence {y,}. 3-11

Vector Functions: Piecewise Continuity

We examined the notion of continuity at a point in the previous section. Let us now see what can happen at a point where a transformation is not continuous.

jj _Air)

0

Example 8-28

itself,

|—1

Let f be the function from R into setting f(z) =

—1

(See Fig. 3-20.)

if z 0 for every n, then

Tt—?>

=

by

To begin with,

continuous at 0; however, we note that if z,, n = 1, 2,..., 18 @ sequence converging to 0, with either Zn < 0 for every n or x, > 0 for every n, then the limit of the sequence f(za), 7 = 1, 2,..., exists.

In fact, if tz. < 0 for every n, then f(zs)

defined

f(z) =1lifz>0.

Fig. 3-20 The function f(z) is not continuous at x = 0.

lim

Such a point is called a dzscon-

tinuity of the transformation. let us consider two examples.

@

1.

Example 8-29 Let g be the function from R into itself, defined xz 1s an irrational number and g(z) = 1 if xisarational number. at any z and, in particular, is not continuous at 0. Moreover, sequences Zs, 7 = 1, 2, . . . , converging to 0, with (say) za >

by setting g(r) = —1 if Then gis not continuous we note that there are 0 for every n for which

either lim g(z.) = —lor lim g(z,) = 1 or for which the sequence g(z.),n = 1,2, ..., Tie->

has no limit.

n—

oo

t Recall that property C4 states that a set K is compact if, given any sequence Zn, n=l, 2,..., of elements of K, there is a subsequence 2 t for every n, lim f(tn) = 2; that 1s, x 18 the limit of the sequence {f(t.)}. x=

We often write f(t+)

or

lim f(t) =z

(3-47)

(t+

In a similar way, we say that f has a limit on the left at t (or has a limit from

below at t) if there 1s an element y in X such that, given any sequence ta,n = 1, 2, - » which converges to t with t, < t for every

1 Fie=1, 240

n, lim f(t.) = ys that is, y is the limit of the We often write

sequence {f(t,.)}.

— or

y=ft-)

lim f(t) = y

tt —

For example, the function f of Example 3-28

has the limit

1 from

the right at 0 and the

limit —1 from the left at 0.

We note that

~f

f(0)=0

(3-48) Fig.

3-21

The

left-

and

‘ight-hand limits of f(¢) are

the came

-

but f (f) is

even if z = f(t+) and y = f(t—) exist and are equal, f may still not be continuous at ¢. This is illustrated in Example 3-30. Example 3-30

f(0) =0.

Then

3-21.)

Let

lim

t-0+

Definition 3-25

be the function from RF into itself, defined hy f(t) = lift ~ Oand

f(t) = 1 =

Piecewise

lim f(t), but f is not continuous at 0.

t—0-

Continuity

(See Fig.

If f is a transformation from R

into a set X, with d as distance on X, then we say that f has a simple (or jump) discontinuity at t if (1) f is not continuous at t and (2) f has both a limit from the right and a limit from the left at t, that is, both f(t+) and f(it—) exist. If D(f) = {t: tts a discontinuity of f; that ts, f is not continuous at t} 1s a countable sett [t.e., if the elements of D(f) can be enumerated in a sequence tr, n = 1, 2, . . . (which may terminate)| and consists of only simple discontinutties, then we shall say that f is a piecewise continuous (or regulated)t function. In

other words, f is piecewise continuous if there are only a countable number of

points at which f is not continuous and if, at each of these points, f has a limit both on the left and on the right. t See Sec. 2-2. {See Ref. D-5.

82

Mathematical preliminaries: analysis Example 3-81

Let f be the function from & into itself which is defined by setting

f(t) = 0 if t ¢

try to approximate f(t) by

fle(t)] = f(v).

We shall use this

Sec. 8-12

Derivatives

83

a function f*(£) of the following type:

F(to) + Q(t — to) = fr)

(3-49)

where @ is a linear transformation of RF into itself. the form fot) = f(to) + a(t — to)

We may write (3-49) in (3-50)

where a is an element of R.t It is easy to see that f*(t) represents a line which passes through f(t). (See Fig. 3-23.) We should like to choose a in such a way that f*(¢) is tangent to f(t) at to, that is, in such a way that the limit of |f*(t) — f(t)|/|é — to} as ¢ approaches tf 1s 0.

In other words, we want to choose

aso that

F(t) — f(to) li lt bt tte

_

a|

=

0

(3-51)

t-

If there is an a in F# for which Eq. (3-51) is

!

satisfied, then we say that f is differentiable at to and that a is the derivative of f att. In

7

this case, we often write df

a=)

= Df(to) = f(to)

(3-52)

Fig.

£2(¢)

| {

— t °

3-23

function f«(2).

An

approximating

We say simply that f is differentiable if f has a derivative at every ty in R, and we write

a

or

Df(t)

orf)

(3-53)

for the function from RF into itself whose value at ¢ is the derivative of f at t.

We call this function the derivative of f.

We note that it can be shown that

if f has a derivative, then f is continuous.

Furthermore,

it can be shown

that if f is differentiable and if a, b arein R witha < b, then there is a 6 in the interval (a, 6) with

f(b) — fla) = fo — «) = 2

— a)

This is often called the mean-value theorem. If f is a differentiable function from FR into itself, then we may consider the function f(t) = df/dt and ask whether or not this function has a derivat We note that a is the matrix of @ (which has a 1 X 1 matrix). Furthermore, we may also write (3-50) in the form

Se(t) = f(to) + (a, t — te) where (

,

) denotes the scalar product on R (see Eq. (2-86)].

(See Sec. 2-8.)

84

Mathematical preliminaries: analysis

tive at a given fin R.

If f(t) has a derivative at t, then we say that f(t) is

twice-differentiable at ty or that f(é) has a second derivative at to, and we write af(o)

.

=

fi(ty )

—

oI

We may repeat this procedure to obtain higher-order derivatives of f(t), and we shall write

= f*(4) =

on) d”.

4/2]

;

for these higher-order derivatives. We shall not bother to define the notion of higher derivative, which corresponds to the concepts of derivative soon to be discussed, but rather shall leave this particular matter to the reader. We may repeat this procedure to define the notion of derivative for a vector-valued function f from FR into R,. In other words, if tp is an element of FR and if @ is a linear transformation of R into R,, then we consider functions f*(¢) which are given by or, equivalently, by where a is a vector in R,.t

that

f(t) = f(to) + @(t — to)

(3-54)

f(t) = f(to) + a(t — to)

(3-55)

As in (3-51), we want to choose a in such a way

im | f(t) — £(to) —aj|=0

ea

t — to

(3-56)

If there is an a in #, for which Eq. (3-56) is satisfied, then we say that f is differentiable at to and that a is the derivative of f at to. In this case, we write

a = Df (to) = f(t) We

(3-57)

note that f is differentiable at t) if and only if the components fi, f2, . » Jn of f (see Sec. 3-9) are differentiable at t and that

filto) | alte)

F(t)=]

(3-58) | Salto)

t ais the n-column vector which is @(1) and is the matrix of the linear transformation @ [see (2-31)].

Sec, 8-12

Derivatives

85

We say simply that f is differentiable if f has a derivative at every t in R,

and we write

df

qr)

Sor

f(t)

(3-59)

for the function from R into R, whose value at ¢ is the derivative of f at t. We call this function the derivaiwe of f. We note that

fi =|

T fi(t) hilt)

|

(3-60)

BAGH Furthermore, we can see that if f and g are differentiable, then the sum f + g is differentiable and the product rf is differentiable (r being an element of R), with

ar + Bl and

iw) +80

(3-61)

am) = rt)

(3-62)

An immediate consequence of Eqs. (3-61) and (3-62) is the fact that the set

of all differentiable functions from R into R, forms a vector space which 18 a

subspace of the space of all functions from R into Ra, 5(R, R,).

Finally, we

note that if g is a differentiable function from R into FR and if f is a differentiable function from R into R,, then the composite function fog is differentiable and

aE _ tomo =4|

2

(3-63)

Exercise 3-12 Show that a function f from R into R, is differentiable at to if and only if each component f; of f is differentiable at t) and that Filte) E(t) = Al to)

Hint: Make use of Eq. (3-6c). Example 3-38 Let J be the closed interval with end points 0 and 1, that is, J = [0, 1) [see (3-7)], and let f be a function from J into Ra, with components fi, f2, . . . ,fs. Then we say that f is differentiable on I if, for each t) in J, there is an ain R, (which may depend

86

Mathematical preliminaries: analysis

on ¢,) such that

.

f(t) — f(to) _

im

t—

ter

where the notation

ty

| _ °

0

im |] £2 = He) _ || = g

tte

t — ty

tel

means that, for every sequence t,, n = 1,2, .. . , of elements of J, with t, = ty for all n, which converges to ¢s, the corresponding sequence

f(tn) — f(t) _ te

-_

te

converges to 0. It is easy to see that f is differentiable on J if and only if each component f; of f is differentiable on J. For example, the function f from J into R., defined by

f(t) = [=

"il, is differentiable on J and f(t) = | 7 cos rt |;

COS wt

—fs sin zt

So far, we have considered functions of one

(real) variable; now

let us

suppose that f is a function from R,, into R. There are several senses in which we may speak of the derivative of f: First of all, we may repeat the

procedure

used

to

obtain

the

derivative

for

functions

of one

variable;

secondly, we may view f as a function of the components of vectors in R,, and obtain derivatives of f with respect to those components (i.e., partial derivatives); and finally, we may consider the behavior of f along a given “direction” in R,, and obtain a derivative of f with respect to this direction. We shall now make these different notions of derivative more precise. If uo is an element of R,, and if u is ‘near’ uo, we may try to approximate f(a) by a function f*(u) of the type

f(to) + @(u — uo) = fe(u)

(3-64)

where @ is a linear transformation from R,, into R [that is, @ is an element of the dual space R* of R,, (see Exercises 2-11 and 2-12)]. We may write Eq. (3-64) in the form

fe(u) = f(uo) + (a, u — uo)

(3-65)f

where a is an element of R,,. Again, we should like to choose the m vector a so that f*(u) is tangent to f(u) at uo, that is, in such aWAy that the limit of |fe(u) — f(u)|/|lu — uoll as u approaches uy is 0. In other words, we want

tee | Lu) — flo) _ (@, uw — wh)

u—>te

Iu

— Uo|

lu

— Uo|

= 0

(3-66)

If there is an a in R,, for which Eq. (3-66) is satisfied, then we say that f is

differentiable at uy and that a is the gradient (or derivative) of f atuo. case, we often write

a = Vf(uo) = grad f(uo) = au. t Where (

,

Us

) denotes the scalar product on R.» [see Eq. (2-86)].

In this

(3-67)

Sec. 8-12

Derivatives

37

We say simply that f is differentiable if f has a gradient at every Uy in Rn, and we write

(Vf) (a)

or

grad f(u)

or

ot

(3-68)

for the function from #,, into itself whose value at u is the gradient of f at u. We call this function the gradient of f. We observe that the gradient of f is a vector-valued function, and we shall denote the components of it by

(Df\u)

that is,

(Dif) (u) (Daf)(u)

oa

— aof

or

=

(Dafu)|

3-69 (3-69)

Of /d%1 Of /dus

(3-70)

Laf/a

For example, if f is the function from R, into R, given by

({™'1) = uy? + sin us th en

of — a =

2U

ea

‘|

of i Du,

and

2u1

of We uz = C08 Us

We note, finally, that if f is differentiable and if u; and wu, are points of

Rn, then it can be shown that there is a point u* on the line segment joining u, and uz (see Definition 3-16) such that

f(ui) — f(a) = (a (u*), ui — us) We

(3-70a)

shall use this relation, which is often referred to as the mean-value

theorem, in Sec. 3-18.

Now let us look at f as a function of the components of vectors in Rp.

In other words, if

-

Uy

-

U2

u=

=

y

i=1

Ue;T

[Um

then we may write

f(a) = 710) U:@; ) = f(ui, Us, . . . , Um) t=

{ Where the e; are the elements of the natural basis in R,, (see Eqs. (2-27)].

(3-71)

88

Mathematical preliminaries: analysis

Now let us suppose that uy is a given element of R,, with components Uo1, os,

. . - » Yom, and let us examine the behavior of f as we change only

one of the components of uo—say the first.

In other words, let us consider

the function g; from FR into itself, which is given by gi(u) = f(u, toa, . . - 5 Wom)

(3-72)

If g; has a derivative at uo:, then we say that f has a partzal derivative with respect to u,; at wo. We denote this partial derivative by (D,:f)(uo) or (f/du)

lus) that

1s,

(Do)(ua) = Dif(u) = ZI,

(3-73)

and we note that (D,f)(u)

—

Jim

flu,

Uo2,

+

s+

3 Hon)

+ Lo

Uo2r,

-

+

+

3 Uom)

(3-74)

We may define the partial derivatives of f with respect to the other compo-

nents tg, U3, . . . , um Of u in an entirely similar way; i.e., we say that f has

a partial derwative with respect to u; at uo if there is a number a; such that lim

f(uo1,

2

+

sy

Udi-1,

U,

Uoi43,

-

u—? thei

-

U—

+

thom)

7

f(uo1,

Uo,

-

-

+}

tom)

—

4a,

Uo

=0

(3-75)

and in that case, we write

a: = (Dep)(us) = 2 |

(3-76)

If f has a partial derivative with respect to u; at every point Uo, then we say that f has a partial derivative with respect to u; and we denote the function from F,, into R whose value at uy is (D;f)(uo) by

Dpu)

or

#

(3-77)

But we have used this notion before for the components of the gradient of f [see Eq. (8-70)]. We shall leave it to the reader to verify that we are being consistent in our use of this notation. We are now ready to consider the notion of a derivative of the function f from &,, into #& along a given “direction” in Ry». Let us suppose that up is an element of R,, and that e is a given vector in R,, of norm 1, that is,

(e,e)=1

(3-78)

Sec, 3-12

Derivatives

89

We shall call the unit vector e a direction, and we shall say that f has a

directional derivative at Uy in the direction e if the function g, from & into

itself, defined by

ge(t) = f(uo + te) has a derivative att = 0.

We call the derivative of g. at 0, g.(0), the dzrec-

tional derivative of f at uy in the directione.

of

OU, |do

where @1, @2,

(3-79)

(See Fig. 3-24.)

Weremark that

= 9 e; (0)

(3-80)

. . . , mis the natural basis in R,,; that is, the partial deriva-

tives of f are the directional derivatives of f in the directions e; determined by ra

uo

uy

Fig. 3-24 Limits are taken along the line L to obtain the directional derivative of f at Up in the direction e.

the natural basis in R,, (see Sec. 2-6). shown that

In fact, ife = > a,e;, then it can be inl

#0) = Fv 2 |. t=10

=(

Fan | Qe

(D if) (Uo) " (Def) (Uo)

| Hone

| (D mf ) (Uo) _]

Joby.

(e, grad f(uo))

(3-81) (3-82) (3-83)

90

Mathematical preliminaries: analysis For example, if f is the function from R, into R, given by

i{{i}) - +

and if

e = ha

then the directional derivative of f at 0 in the direction e is

Sal-({ivah t= oho) Finally, let us consider a transformation f of R,, into R, with components ti, fa, . . . » fn and let us suppose that uo is an element of R,. Now, if u is ‘near’ Uo, we may try to approximate f(u) by a function f*(u) of the type

f*(u) = f(uo) + @(u — uo) where @ is a linear transformation of R,, into R,.

(3-84) We should like to choose

@ in such a way that f*(u) is tangent to f(u) at uo, that is, in such a way that

im (fu) — f(u)|l ma uo

_

. 6-86)

In other words, if A = (a,;) is the matrix of @ (see Sec. 2-7), then we want

to choose the a;; so that

f(u) —f(uy)

lim

lj — woll

U-—? Ge

A+u — uy — ————- || = 0

|u — wll

(3-86)

If there are a,; in R for which Eq. (3-86) is satisfied, then we say that f is differentiable at Uy and that the matriz A is the derivative of fatus.

Ais often

called the Jacobian matriz of f at uo, and it can be shown that the n X m matnx A has the form

A

—

(Difi)(to)

(Defi)\(uo)

«+

(D

(Uo)

(D of 2) (uo)

ss

(D mi 2) (uo)

(Difn)(uo)

(Defa)(to)

- °°

(Dmfn) (Uo)

fe)

>

(Defi) (Uo) (3-87)

where fi, fo, . . . , fn are the components of f and the D; represent partial

derivatives. precisely

In other words, the entry in the 7th row and jth column of A is D;f:(uao)

(3-88)

We often write (df/du)|uo for the matrix A. In the case where f is a transformation of R, into itself, we call the determinant of the matrix A the

Sec. 8-12

Derivatives

91

Jacobian of f at uo, and we write

detA = det [D,f;(uo)] =

OFyty

++ td |

O(u1, Ua,

. . . , Un) Ito

det 2

du lu

_

(3-89)

We say simply that f is differentiable if f has a derivative at every Uo in Ra, and we write of bu

or

[D;f.(u)}

(3-90)

for the function from R,, into the set of n X m matrices IN(n, m) (see Sec.

2-8) whose value at u is the Jacobian matrix of fatu.

the derivative of f.

We observe that if

We call this function

Filta, Ue, . - . , Um)

f(u) =

fa(uy, Ue,

. . . , Um)

(3-91)

fn(ty, ta, . - - » Um)

then the derivative of f, of/du, is given by

of:

Of:

|

af | Of: Of2

|.

Ou)

gu

|

Or

OUs

«te

ae af... OUe

| Oty

oft|

OUm

Ofs

dUm

(3-92)

a

OUm_

For example, if f is the transformation of R; into Re, given by

(e)-E then

du

| O

—1

2us;

Example 3-84 Suppose that @ is a linear transformation from FR, into R, with A = (a,;) as matrix. Then we know that @(u) = Au for win 2, and hence we can assert that @@/du = A. In particular, if @ is the linear transformation from FR, into R:, with matrix A given by

then

a@ ou

_ 4, en [

—1

4

92

Mathematical preliminaries: analysis Example 8-85

Suppose that f is a function from Rs into R, which is given by ti? + 422 f(a)

=

—Uitbs | 41 SIN

P

of

and We note that

so that f is not one-one.

3-18

a|

te

2u1

2u:

| SIN te 0 0

«hy, COS Ue

X|,-[0 o| ou ° 10 O

(21) -(fal) - [2]

Vector Functions: Smooth Sets in &,,

We are now going to use the notions of derivative and gradient introduced in the previous section to study the concept of ‘smoothness’ for subsets of R,. Roughly speaking, a subset of FR, is “‘smooth’”’ if it has a tangent plane at every point. Let us now make this idea more precise. We begin with a series of definitions. Definition 3-26 Suppose that f(x) 1s a continuous real-valued function on R,. Then we let S(f) denote the subset of Ra given by

S(f) = {x: f(z) = 0}

(3-93)

In other words, S(f) is the set of points x in R, at which f ts zero. We call S(f) the (continuous) hypersurface in R, determined by f. Definition 3-27 Suppose that S(f) is a hypersurface in R,, and that xo ia an element of S(f) [that ts, that f(xo) = O]. Then we shall say that xo 18 a regular point of S(f) tf the gradient of f with respect to x at Xo exists and 18 not the zero vector. In other words, Xo is a regular point of S(f) if

of We observe that if x, is a regular point of S(f), then the set L(xo) given by L(xo) = {x (2 (xo), x — Xs)

= of

(3-95)

is a well-defined hyperplane (see Definition 3-15) which passes through the point x». This observation leads us to the following definition: Definition 3-28 If xo is a regular point of the hypersurface S(f), then we call L(xo) the tangent hyperplane of S(f) at xo. We also say that a vector

Sec. 3-18

Smooth sets in R,

93

n(Xo) 78 normal to S(f) at Xo tf

n(x,) = ow (x)

¢¥0

(3-96)

In other words, 0(X0) 1s a nonzero vector which 1s collinear unth the gradient vector of f at Xo.

We note that if n(xo) is normal to S(f) at the (regular) point xo, then the

tangent hyperplane L(x») is also given by

L(Xo) = {x: (n(Xo), X — Xo) = 0}

(3-97)

Now let us suppose that f is a real-valued function defined on Ra. Then we shall say that f is smooth if: 1. f is differentiable {i.e., if Af/dx exists; see (3-68)]. 2. The gradient df/dx is a continuous function from FR, into itself. We have: Definition 3-29 If f is a smooth function and if every point Xo of S(f) ts regular, then we say that S(f) 1s a smooth hypersurface.

We observe that a smooth hypersurface has a well-defined tangent hyperplane at every point. Example 8-86 Let f(z) = (a, x), where a is some nonzero vector in R,. a smooth function, the set S(f) is the hyper-

plane L through the origin given by (a, x) = 0, and the tangent hyperplane L(x,) at any point x, of S( f) is precisely the set S(f) (= L) itself. Example 8-87 Let f be the real-valued function zo? — 1.

on

FR; given by s({2])

Then f

= 2,2 +

is smooth, the set S(f) is the

unit circle (see Fig. 3-25), and, for example, the tangent hyperplane L (val)

at the point

vA

1//2

! (—

( 4,4) v2 ' V2 1;

1

0

“1

of S(f) ~

is the line 2: +2: —

4/2 = 0 (as illustrated in Fig. 3-25).

‘

\

Then f is

X2

The

reader should consider this example in the light of Definition 3-22.

Fig. 3-25 3-37.

+X%5o-V2 =0

ane

The set S(f) of Example

We are now ready to define the notion of a “smooth” set in R,. We let fi; fo, - . » ,fn-e be mn — k distinct smooth functions on R,, where 1 < k < n—1. We let S(f:, fo, . . , faz) denote the intersection [see Eqs. (2-10)]

of the hypersurfaces S(f;), S(f2), . . - , S(fn—x), that is,

S(fis fa, . « « yfa-e) = S(ft) OV Sf) OA + +

A S(fr+)

(8-98)

In other words, S(f1, fe, . . - , faz) is the set of points x in R, at which

94

Mathematical preliminaries: analysis

all the f;,,1 = 1,2, ...,n

— k, are zero; thus,

S(fi, fay... » fue) = {x: filx) = 0 and f.(x) = 0 and:

:-: andf,(x)

= 0}

(8-99)

We now have: Definition 3-30 We shall say that S(fi, fe, . . - , fn—z)18. @ smooth k-foldin R,, tf, for every point xoin S(fi, fo, - . - » fn—z), then — k vectors (Of: /0x) (Xo),

(Of2/0x)(Xo), . . - , (Ofn_z/OX)(Xo) are linearly independent [see Eq. (2-26)]. We observe that if S(fi, feo, . . . , faz) is a smooth k-fold, then each of the hypersurfaces S(f;) is smooth. It follows that if x» is an element of S(fi, fe, . . - , fn-z), then the tangent hyperplane L,(xo) of S(f;) at xo is defined fori = 1,2, ...,n—k. We let M(x») denote the intersection of the hyperplanes L;(xo), that is,

M (xo) = D1(X0) (\ La(X0) OO’ + + + CV Ln—e(Xo) =

x: (hs (Xo), xX — x)

= 0 and (2

and: and we call M(x»)

(3-100) (xo), X — x.) = 0

-- and (a

(xo), x — x) = o|

(3-101)

the tangent plane of S(fi, fo, . . - ; fn—z) at Xo.

Thus, a

smooth k-fold has a tangent plane at every point.

Example $-388 Let fi(x) = (a', x) and f:(x) = (a2, x), where a! and a? are linearly independent vectorsin R,. Then f, and f2 are smooth, the set S( fi, f2) is an (n — 2)-fold, xy and the tangent plane M(x,) of S(fi, fs) at any point x, of S(fi, f2) is precisely the set

S(fi, fr) iteelf.

Example 3-39 Let f: function on FR; given by

be

the

real-valued

v1

x

fi

Z2 La

= 21 — sin Z;

and let f, be the real-valued function given by Fig. 3-26 ple 3-39.

The set S(f:, f:) of Exam-

fel

Z1 | 2: Xs

=

on

R;

= T_ — COS Z3

Then f; and f; are smooth, the 1-fold S(f;,, f:) in R; is the helix illustrated in Fig. 3-26, and, for example, the tangent line

of S(fi, fz) at the point

is the intersection of the planes 7, — z; = O and 2; — 1 = 0.

Sec. 8-14

Integrals

95

Finally, we have the following definition:

Definition 3-31

Let S(fi, fo, . . . , fa—z) be @ smooth k-fold in R, and let

Xo be an element of S(fi, fo, . - . , fn—z). Then we shall say that a vector p as transversal to S(fi, fo, . . - »fn—z) at Xo tf (p, x—

Xo)

=

0

for all x in M (Xo)

(3-102)

We observe that p is transversal to S(fi, fe, . . . , faz) at Xo if and only

if p is a linear combination

(Ofe/8x)(x0),

(see Sec. 2-6) of the n — k vectors (0f1/0x) (Xo),

. . . , (Afn—z/@X)(X0).

This notion of transversality will be quite

important in Chap. 5. We also note that a vector p will be transversal to S(fi, fe, . - . ,fn—e) at Xo if and only if p satisfies the k relatzons

(p, X' — Xo) = 0

7=1,2,...,k

(3-103)

where the x/ are elements of M(x.) such that the vectors x’ — x» are linearly independent. ‘This remark will take on greater significance in Chap. 5. 3-14

Vector Functions: Integrals

We are now going to discuss a very important concept: the notion of the

integral of a function.

Loosely speaking,

the integral of a function f is a

function g whose derivative is the function f. We shall shortly make this idea more precise. Let us suppose that f is a real-valued function which is defined on the

closed interval [a, b] of R, and let us suppose that g is a continuous function from [a, b] into R. Then we shall say that g is an antiderivative of f if there is a countable subset (see Sec. 2-2) A of [a, b] such that if ¢ is in [a, b] — A

[see Eq. (2-4)], then g has a derivative at ¢ and g(t) = f(t). We remark that if g and h are two antiderivatives of f, then the difference g — h is a

constant on (a, bj.¢

If f is a piecewise continuous function on [a, b] (see

Sec. 3-11), then it can be shown that there is an antiderivative g of f.

For

example, if f is a step function (see Example 3-31 and Sec. 3-15), that is, if there are a finite number of elements fo, #:, ..., tr. of [a, b] with

a=t fi femlt) dr]

+f.

fame) ae

(3.116)

Sec. 3-14

Integrals

99

We note again that the properties of the integral expressed in Theorem 3-8 have natural analogs for the integral of a matrix-valued function. There are several other properties of this integral which we shall require in the main body of the book. These are given in the next theorem. Theorem 3-9 Let F(t) be a piecewise continuous function from [a, b] into the set of n X m matrices, let P be a given (constant) p X n matriz and Q be

a given (constant) m X q matriz, and let v be a given (constant) vector in Rm. Then a. PF(£)Q ts a piecewise continuous function from |a, 6] into the set of p X q matrices, and

| (PF@)Ql dr = P{ [" FG) ar} Q ts

(3-117)

ts

b. F(r)v is a piecewise continuous function from {a, b] into R., and

(3-118)

[° F@v) ar = {FG ae} v

Let us now turn our attention to some examples which illustrate the

notion of integral and some of its consequences. Example 3-40

Let f be the function from (0, 7] into R defined by f(t) = e'. i

and

ts

ty

e" dr = ef — ef

T

i JO

e'dr = eT — 1

If g is the function from (0, 7] into R defined by g(t) = ¢ + 1, then

Jit1 ate) ae a te 2

th

2

and

I g(r) dr = os +T T

3

It is clear that

f

T

T2

T

T

(f+ (r+ jdm eh -1 +547 = fi cart fi (7 +1)dr

Example 3-41

Let f be the function from {0, 7] into R; defined by et f(t) = | sin? 1

i

Then

f

tz 1

{(r) dr =

ts

ft ts

e” dr ef

—

ef

I, sin rdr | = | — cos f2 + 00s &1 te — th ts fi

1 dr

Then

100

Mathematical preliminaries: analysis

Example 8-42 F(t) =

Let F be the function from (0, 7] into the set of 2 X 2 matrices given by

|.

uP=[_9

Then

fy Pe = [pre]

jfandg=[) o].then Prg =['o,) 5] and f° PF(r)Q dr =

Iftv = [5 |} then Ftv =

MPF eo {rahe

2 “a and

fr Fore = [oor] [ a nell] {fh roa} 3-15

Vector Functions: Function Spaces

We have already come across the idea of a set of functions which have

properties in common,

as in the set of all piecewise continuous functions.

Usually the sets of functions which we considered were vector spaces; now we should like to introduce into these sets of functions notions of distance

compatible with the notions of sum and product. Loosely speaking, then, a vector space of functions with a compatible distance function will be called a

function space. We shall make this concept more precise, and we shall examine several very important examples in the remainder of this section. Definition 3-35 Function Spacet Let & be either a subspace of the vector space S(Rm, Ra) of all functions from R,, into R,, or a subspace of the vector space S([a, b}], Ra) of all functions from the interval [a, b] into Ra, and let d be a distance on § (see Definition 3-1). space if:

FS1

Then we shall say that 5 13 a function

Given elements f and g of ¥ and sequences {f,} and {g.} in F which converge to f and g, respectively [that zs, d(fi, f) — 0 and d(gi, g) — OJ; then f, + gx converges to f + g, that zs,

d(f. + gr, f+ g)— 0 FS2

(3-119)

Given an element f of § and an element r of R and sequences {f,} in § and {rz} in R which converge to f and r, respectively; then the sequence rif, converges to rf, that 1s, d(rifi, rf) —- 0

1 See Refs. 8-4 and V-1.

(3-120)

Actually, this definition is a special case of the definition of

what is known as a topological vector space. quite general.

The concept of a topological vector space is

Sec. 8-15

Function spaces

101

Now let us turn our attention to some important examples of function spaces. We-shall begin by considering the set of all bounded functions from the interval [a, b] into R,, where we have: Definition 3-36 Bounded Function A function f from [a, bj into R, is said to be bounded if there is an M in R, M > 0, such that

\f(t)|| < M

for all t in {a, }]

(3-121)

We let @((a, b], Rn) denote this set; that is, @((a, b], R.) =

{f: f is a bounded function from [a, bj into R,}

(3-122)

It is easy to see that ®((a, b], R,) is a vector space which 1s a subspace of the set ¥([a, 6], R,) of all functions from [a,b] into R,. Weshould now like to introduce a distance into @({a, b], 2.) which makes it into a function space. Definition 3-37 Norm of aFunction Let f be an element of @((a, 6], R.); that is, f 2s a bounded function from [a, b] into R,. Then the (uniform) norm

of f, which we write |lf||, 1s, by definition, the greatest lower bound of the set of numbers M such that ||f(t)|| < M for all t in [a, b].t In particular, of f 28 continuous, then

lIf|| = sup ||f@)|| = max {lf©|| tEla.0) tCla,b)

(3-123)

since |\i(t)|| actually attains its maximum value on [a, b] (see Theorem 3-7). If f and g are elements of @((a, 6], R.), that 1s, of both f and g are bounded, then the distance between f and g, d(f, g), 18 simply the norm of f — g; that 1s,

d(f, g) = |\f — gll = sup |lf@® — gl

(3-124)ft

tE[a,d]

Let us now show that @({a, 6], 2.) with this notion of distance is a function space. We have:

Theorem 3-10 The function d gwen by Eq. (3-124) 1s indeed a distance on the set @((a, b], R,»). The set @([a, bj, R,) 1s a function space unrth respect to this distance. PROOF Weshall first verify properties D1 to D3 of Definition 3-1 for the

function d given by Eq. (3-124). lg(t) — £(¢)|| for all ¢ in [a, b].

Property D2 is clear, since ||f(t) — g(é)|| =

As for the triangle inequality D3, we note

that if f, g, and h are in @([a, b), R,), then

lf) — bOI] < l€O — ell + le® — boll

(3-125)

t ||f|| is often called the supremum of ||f(t)|| for ¢ in [a, bd). t Where sup (f(t) — g(t)|| is to be construed as meaning the greatest lower bound of t€[a,d}

the set of numbers M such that |\f(¢) — g(¢)|| < M for all ¢ in [a, b]; that is, g(t)|| is the supremum of ||f(t) — g(¢)|l.

sup

tE[a,b

|f(e) —

102

Mathematical preliminaries: analysis

for all ¢ in (a, b], which. implies that

SUP. f() — hl

< sup

{lf — g@l| + lle® — bO|}


0 and that d(f, f) = 0.

llg@) —b@®]|

(8-126)t

Now, if d(f, g) = 0, then,

for any given ty in [a, 6), we must have ||f(¢.) — g(to)|| = 0, so that f(t.) = g(to). Thus, we have shown that d is a distance on ®({a, 6], Rp). Now let us verify property FS1 of Definition 3-35. Suppose that we show that if f and g are elements of @((a, 5], R.), then

lf + gll < |lfll + llell

(3-127)

lf + ge — f+ e)ll < lif. — fil + lle — gil

(3-128)

d(f, + eg, f + g) < d(fi, f) + d(gi, g)

(3-129)

It will then follow that

that is, that

Clearly, Eq. (3-129) implies that property FS1 is satisfied.

Eq. (3-127), we have

If + gil = sup. | + g)®||= sup

< sup. \|f(¢)|| + sup

Now, as for

lf) + g(d)||

le|| = lIf|| + llgll

(8-130)

We leave it to the reader to verity that property FS2 is satisfied. We observe that @((a, 5], R,) is complete (see Sec. 3-5) with respect to the

distance d of Eq. (3-124).

Tosee this, we note that if f; is a Cauchy sequence

(see Definition 3-11), then each f,(¢) is a Cauchy sequence in #, and so must converge to an element of R,, which we denote by f(t). If « > 0 1s given, then there is an N(e/2) such that |[f; — f,|| < «/2 if j,k > N(¢/2). We claim that if 7 > N(«/2), then ||f; — f|| N(e/2) so that ||fi (t) — £(O|| < €/2). Now let us denote by C({a, 6}, ®,) the set of all continuous functions from (a, b] into R,,; that is, C((a, d], R.) =

{f: f is continuous on [a, b] into R,}

(3-131)

In view of Theorem 3-7, we observe that the set C((a, 6], 2.) is contained in the set ®((a, b], R,). Moreover, it is actually a subspace. (See Sec. 3-10.) t The basis for Eq. (3-126) is the following type of argument:

If M and N are numbers

such that ||f(¢) — g()l| < M and |lg(t) — h(t)|| < N for all ¢ in [a, b), then ||f(¢) — g(é)|| +

ligt) — h(t)|| < M + N for all ¢ in (a, b] and, consequently, ||f(t) — h(t¢)|| < M + N for all ¢in (a, 6}. The reader should bear this in mind whenever we make statements similar to Eq. (3-126). t Compare this with Eq. (3-126).

Sec. 3-15

Function spaces

103

In other words, continuous functions defined on a compact interval are bounded. The distance d given by Eq. (3-124) is, when restricted to €((a, b], Ra), a distance between continuous functions. Is @((a, 6], R.) a function space with respect to this distance?

Well, we shall show in Theorem

3-11 that if f, is a sequence of continuous functions which converges to f,

then f must be continuous.

This will show that C((a, b], R,) is a function

space. In fact, C([a, b], R.) is a complete function space with respect to this distance. Theorem 3-11 If f, is a sequence of continuous functions which converges to f, then f 1s continuous.

proor

Suppose that to is in [a, b] and that {z,,} is a sequence in [a, }]

which converges to f. We shall show that f(¢,,) converges to f(t). let « > O be given; then there is a K such that k > K implies that

sup

t€[a,b]

Now

é€

||f.(t) — f(d)|| < M implies that

fem) — HUI < 5

(3-133)

It follows from Eq. (3-121) that, for m > M,

\|f(tm) — f(to)|| < « and hence that f(t,,) converges to f(t).

(3-134)

This shows that f is continuous.

Exercise 3-13 Show that Theorem 3-11 implies that the set of continuous functions C((a, b], Ra) is a function space. Hint: Make use of Theorems 3-7, 3-11, and 3-10.

Theorem 3-11 is often stated as: The limit of a uniformly convergent sequence of continuous functions is again continuous. We observe also that

the theorem remains true if ‘‘continuous” is replaced by “‘piecewtse continuous” throughout; that is, we have: Theorem 3-1la If f; is a sequence of piecewise continuous functions which converges to f [with respect to the distance of Eq. (3-124)], then f 1s piecewise continuous. This will again imply that the set of all piecewise continuous functions

from [a, 6] into R, will be a function space, provided that we can show that every piecewise

continuous

function from

[a, b] into R, is bounded.

We.

shall do this by showing that if f is piecewise continuous, then there is a

104

Mathematical preliminaries: analysis

sequence.s,, of step functions (see Sec. 3-14)t for which ||s,, — f|| converges: to 0. In other words, we have: Theorem 3-12 If f 1s a piecewise continuous function from [a, b] into R,,

then there 18 a sequence S», of step functions such that, given any « > 0, there zs

an M(e) such that m > M(e) implies that ||s.(t) — £(t)|| < ¢ for all tin [a, 5). PROOF (The idea of the proof is illustrated in Fig. 3-27.) Let N be an integer. Suppose that ¢ is in (a, b] and ¢ ¥ a or b; then there are elements u(t) and r(t) in [a, 6] such that if ¢, + are in (c(é), é) or in (¢, r(#)), then ||f(¢) — f(r)|| < 1/N. This is so because f(f+) and f(¢—) exist.{ Similarly, there 1 i

4 ol

l

rT

a, —{—

|

l | |

ria)

x

)

| {

\ |

|

|

1

|

ly

_

;

iL

|

| {

| | (

| | |

|

) | |

c(t)

tft r(t)

|

els)

| |

| |

|

\/

!

|

S|

1W.

4

x

rc

|

$

6(b)=r(s)

| ! |

/

{ ! |

Fig. 3-27 Illustration for the proof of Theorem 3-12. The dashed curve is the piecewise continuous function f(t), and the solid curve is the approximating step function sy(t).

are elements r(a) and «(b) such that if, for example, ¢, 7 are in (a, r(a)), then

\|f(c) — f(r)|| < 1/N.

Now pick elements s), 82, . . . , 8p of (a, 6) such that

every ¢ in [a, 6] is in at least one of the intervals (a, r(a)), («(s1), r(si)),

. . - ,

(c(8,), r(8p)), (e(b), 6]. We may do this in view of the Heine-Borel property of compact sets (see Sec. 3-6, property C2). We now consider the set {a, b, 81, 82, . . - , 8p, r(a), (b), (81), r(81), . . - , (Sp), T(Sp)}, and we define a=&

for every solution x(t) of the system.

(3-441)

If the system ts not stable or, in other

words, if there 1s a solution &(t) such that

lim [|£(2)|| = ©

t—~> @

(3-442)

then we say that the system is unstable. Let us denote the eigenvalues of the matrix A by Mi

Je

7=1,2,...,0n

where the )’s and the y’s are real numbers andj] = +/—1.

(3-443)

The following

criterion relates the stability of the system (3-440) to the eigenvalues (3-443) of the system matrix A. Stability Criteriont The system (3-440) is stable if and only if 1.

1 ¥ ult

be

(4-2)

Every self-respecting engineer knows that, as long as the applied voltage as finite, the current through the inductor L cannot change instantaneously.t Bearing this fact in mind, let us perform

ee

“I

oe | te)

ult)

Fig. 4-1

_,___|__., : R | yt) | | | LoL I In this network, the output

voltage y(t) is measured across R.

the following experiment:

During the

time interval (0, 7], we apply a known continuous voltage input w(t), and we suppose that we know y(0) and u(0). To emphasize that the input is known

over the entire semiclosed interval (0, 7], we shall denote it by 40,7]

(4-3)

In other words, v9.7) is the function defined on (0, 7] by the relation uor(t) = u(t) Now

we

measure

the

output

y(t)

for all ¢in (0, T] during

this time

(4-4) interval,

denote by y,0,7 the function defined on (0, T] by the relation

yor(t) = y(t) We shall call the pair

for all¢in (0, T]

(uco.r, Yor)

and

let us

(4-5) (4-6)

an input-output pair on the interval (0, T]. What we would like to do is predict the output voltage y.o,7;. Thus, we are faced with the question: Given uo,r; and the output differential equation (4-2), what additional information is needed to completely spectfy y (0.1)? If we set a = R/L, then Eq. (42) takes the form

y(t) + ay(t) = au(t)

(4-7)

and its solution may be written as

y(t) = yer +e f° exau(r) dr

(4-8)

ft We assume that impulses do not occur in the physical world, so that u(t) cannot be an impulse.

Sec. 4-2 Since

An RL network

U(0,T)

18 known,

153

the term

e—at f . e*au(r) dr is known, and so the only additional quantity that we must know to predict y 0,7) 18 y(0), the value of the output voltage at ¢ = 0.

We

note that we

do not need to know the value of the input u(0) at i= 0. We could have deduced this by the following reasoning: Suppose that u(0+) # u(0), that is, that a jump in the input voltage occurred at ¢ = 0+; then the current in ZL would not change instantaneously, so that

i(0+) = 7(0)

(4-9)

y(O+) = y(0)

(4-10)

and hence we would find that

independent of u(0). To recapitulate: For the network of Fig. 4-1, we have found that knowledge of uco,7) and y(0) was sufficient to completely

specify y0,7}.

If we make the trivial definition

a(t) = y(t)

t€ (0, T)

(4-11)

we can see that 7(0) is the minimum amount of information required to determine y0,7), given to,.m. We shall call z(0) the state of the system at t= 0, and we note that x(t) satisfies ccc the differential equation | R | Z(t) + ax(t) = au(t)

(412)

aay

Furthermore, we observe that if ? is an element of (0, 7), that is, if 0
0. Then we have: Theorem 4-5 The time-invariant linear system of Eqs. (4-264) and (4-265) is completely controllable if and only if the vectors Cas span the entire space Ra, that is, if and only if every element of R, is a linear combination of the vectors Cag (see Sec. 2-5). PROOF We shall prove this theorem under the additional assumption that the system matrix A has no complex eigenvalues (a complete proof can be found in Ref. F-2). First of all, let us suppose that & is a controllable (at 0) state of the system. Then there are a T > 0 and a control u* such that

-f=

I . e-“Bu*(s) ds

(4-270)

Since (see Sec. 3-21)

en~Mt = » (—1)ear & p=0

and since every » Xn matrix Sec. 2-10, Theorem 2-3),

satisfies

its characteristic

equation

n—1l

et = ) ya(s)Ac

(see

(4-271)

where the ¥,(s) are scalar-valued functions of s.

It follows that

n—-l

—E= ) [." va(e)AcBu*(e) ds

(4-272)

ax=@

and hence that n-l

= »

since where “only On Then

u*(s) =

m

r

.

2, [ f 0 Wa(8) ug (8) ds | Abs

y ug(8)es

&=1

(4-273)

(4-274)

the es are the elements of the natural basis of R,. This proves the if” part of the theorem. the other hand, let us suppose that & is an element of the space R,. we shall show that there are a time 7 > 0 and a control u* such that

~E= I, * ¢-“Bu*(s) ds Let us assume for the moment that 7 is some time T >

a real-valued function ¢(s) such that:

(4-275) 0 and that there is

204 1. ath 2. 3.

Basic concepts e@ (0) = »@(T) = 0 fora derivative of ¢(s) and y(s) o(s) > 0 on (0, T). The matrix Q given by

= 0,1, ...,2 = ¢(s).

— 2, where ¢™(s) is the

Q= iA o(s)e* ds

is nonsingular.

(4-276)

Since every element of #, is a linear combination of the egg, we may write

(4-277)

—Q75 = Y asses a,B

where

the agg are real numbers.

components u3(s) given by

We

claim that the control u*(s),

n—tl

ug (8) = », Ouse (8) drives — to the origin in time T. i » e-*Bu*(s) ds = 5 a=0

with

(4-278)

Now S

ep | fl,

yp’) (s)e7As de | bz

(4-279)

§=1

and so it will be enough to show that

(4-280)

Jy" o@()e™ ds = Qas

since Eq. (4-275) would then imply that —-QQ0E = -E=

f

e—“Bu*(s) ds

(4-281)

However, Eq. (4-280) is readily established by using induction on a, Eq. (4-276), and integration by parts [making use of Eq. (3-278)]. | It remains only to show that a suitable function ¢(s) exists.

that the function does the job. that

y(s) = s**(T — 8)"

Clearly, properties 1 and 2 are satisfied.

Q =f, else as

We claim

(4-282) Now let us show

(4-283)

is nonsingular. Let J be the Jordan canonical form of A (see Sec. 2-10). Then there is a nonsingular matrix P such that P-'AP = J, and so eMT-s) = P-Igi(T-s)p

(4-284)

Since the matrix e47 is nonsingular, it will be enough to prove that the matrix

PeaAtop-1 = [" ufs\e(t-» ds

(4-285)

Sec. 4-16

Controllability for linear systems

205

is nonsingular. But (see Sec. 3-22) e?-* is a triangular matrix with entries e(T—») on the diagonal, the \; being the (not necessarily distinct) eigenvalues of A.

It follows that

det | . gp(s)el?-) ds =

1 [ i > g(s)er? de | i=]

(4-286)

and hence that the right-hand side of Eq. (4-285) is nonsingular, as

[, ole ds > 0

(4-287)

for allz. (See Theorem 3-8.) This completes the proof. Some particularly important corollaries of this theorem are: Corollary 4-4 The subspace of controllable states of the system of Eqs. (4-264) and

(4-265) is the same as the span of the vectors @ag of Eq. (4-269).

In other words, Xo 18 controllable if and only if Xo 1s a linear combination of the

vectors

Cas.

Corollary 4-5

Let G be the n X nm matrix, defined by the relation

G + [B: AB:

A’?B:

---:

A 'B]

(4-288)

or, equivalently, by the relation G

+

[eo,

Coz,

-

.

-

» Com

+ C11,

C12,

.

-

-

5 C1m

where the Cag are given by Eq. (4-269). Then the system is completely controllable if and only if the rank of G is n, that is, rank G =n

(4-290)

or, equivalently, if and only if there is a set of n linearly independent column

vectors of G. Corollary 4-6

If Bisann

X 1 matriz b, that is, if the controls are scalar-

valued, then the system is completely controllable if and only tf then Xn

matrix

G=([b:! Ab:

18 nonsingular, or equivalently, setting

---! A™b]

(4-291)

206

Basic concepts

if and only if

:

by

>

aij

«+ -

j=l

.

>

a;

DB,

jul

det G = det| #2, av ba

y

;

(Dp.

2, aH

anf;

+ +-

jul

>

(n~1)h,

P| se 9

(4-292)

cng;

j=l

-

where the a; are the entries in the matriz A*. Corollary 4-7 If the system matriz A is a diagonal matrix with distinct entries, then the system 1s completely controllable if and only if B has no zero rows. These last two corollaries often provide us with a simple test for the complete controllability of a system. Example 4-10

Consider the system

ol LR} + [i]

el-(e2 Is it controllable?

Well, B is the one-column vector b = [7]

(4-293) and Ab is the vector l of

It follows from the theorem that this system is controllable. Example 4-11 Consider the system

[| = lo Is it controllable?

o| [e| + [31 u

(4-294)

Well, B is the one-column vector b = lol and Ab is the vector | a ’

It follows that b, Ab do not span R; and, hence, that the system is not controllable. Example 4-12

Consider the system

led-t all} + to 2} a] Is it controllable? [’

2 |»80

—1 1 | and A is the 2 X 2 matrix 0 2 2 |. Thus, G = [B! AB] is the 2 x 4

Well, B is the 2 X 2 matrix [

that AB is the 2 x 2 matrix

matrix given by

-l1 c=[~)

|

0 ~1

1 2

6

0 —-1

As the rank of G is 2, the system is controllable.

[2 are linearly independent. independent in this case. Exercise 4-18

(4-296)

2 5 Observe that the vectors [ “ol and

Also observe that any two-column vectors are linearly .

Show that it is not possible to find a solution of Eq. (4-294) starting

at the point (1 and terminating at the point [ °|.

Sec. 4-17

Observability for linear systems

Exercise 4-14

207

Which of the following systems are completely controllable?

@{ey=[o R}+lo “ol (3) oy [f We) +to “ol OLet-Lr olla+lo ol la] Exercise 4-15 of the system

What conditions on 6;, b:, and bs ensure the complete controllability Zi

»

Zo}

=]0

£3

Exercise 4-16

0

1

O

Z1

b;

aA

l

Ze}

+ | bel

O

Avr

Z3

b;

u

Consider the nonlinear system Z1 = aiXels + 1 t. @ a2E123 + Ue. Ly = gX1X2 + Us

where a; + a2 + a; = 0.

4-17

Can you show that this system is controllable?

Observability for Linear Time-invariant Systems

Again let us consider a linear time-invariant system of the form x(t) = Ax(t) + Bu(t) y(t) = Cx(t) + Dut)

(4-296) (4-297)

where A, B, C, and D are, respectively, n X n,n Xm, p Xn, and p X m matrices, and let us attempt to see what insight into the notion of observability we can come up with. Now let us suppose that x, is observable at to; then we claim that xp is observable at t= 0. To see this, we let u be a given element of U, and we set

v(t) = u(t — bo)

(4-298)

Then v is an element of U, and in view of the observability of xo at t, we

may conclude that there is a T > 0 such that ve,..47) and yo,.47) Vits.to+7))

determine

Xo.

x(t)

= &(Xo;

However,

=

a(t;

Vito,t]) Xo)

ed(tt) {x0 + I, * e-M-WBv(r) dr}

(4-299)

for ¢ in [to, to + T'], implies that

y(t) = CeM-w {x0 4 |, * ¢-A-WBy(r) dr}+ Dv(t)

(4-300)

208

Basic concepts

for ¢.in (to,to + T]. If we make the change of variables = t — b,o = 7 — lo, then Eq. (4-300) becomes, by virtue of Eq. (4-298),

y() = Cet {x + [" e-'Bu(o) do} + Du(s)

(4-301)

for s in (0, T]. It follows that ior; and yer) = &(X0; Uo,7r)) determine Xp. Thus, it is enough to consider observability at ¢ = 0, for linear time-invariant systems. In other words, a linear time-invariant system 1s observable uf and only tf ut is observable at t = 0.

Now,

if xo is observable at 0, then knowledge of ylye; = &(x0; 0) for

some 7J' > 0 is sufficient to determine xX». We claim that the converse is true; in other words, we claim that if the free motion of the system, starting

from Xo (see Sec. 3-19), is sufficient to determine x», then x» is observable. To verify this claim, we merely note that if u is any element of U, then

y(t) = Ca(t; uc, Xo) + Du(?)

(4-302)

= Celt {x0 + I e“Bu(r) dr} + Dut) = y(t) + Cf e-*"Bu(r) dr + Du(t)

(4-303) (4-304)

It is clear from Eq. (4-304) that if yQ,~,; determines xo, then knowledge of Uco,7) aNd Yo,7) will determine Xp. Now let us suppose that x, and x, are two observable states (at ¢ = 0) and that xp * x,. If we let yo*(t) and y:°(t) denote the outputs of our system, starting from xX» and x;, respectively, and generated by the control u(r) = 0

for all +, then we know that

Yor(t) = Cex,

(4-305)

yi%(t) = Cex,

(4-306)

We claim that there is a value of t > 0 (say 7’) for which

yo(T) ¥ y1°(T)

(4-307)

For, if yo°(t) = y1°(t) for all t > 0, then knowledge of y,° over any interval (0, ¢] could not determine xp and, hence, x» would not be observable. It follows that if every state is observable (at ¢ = 0), then the mapping which

associates

with

each

state x, the corresponding

y(t) = Cex, is one-one (see Sec. 2-4).

(free)

output

function

We have previously noted that the set of controllable states is a subspace

of R,.

Is this true for the set of observable states?

Let us see.

Clearly,

if Xo is observable, then rxo, where r € R, is observable. Now, suppose that x) and x, are observable; then we claim that x» + x: is observable. In order to verify this claim, we shall first have to take a closer look at what observability means for linear time-invariant systems. We have, in particular, the following theorem:

Sec. 4-17

Observability for linear systems

Theorem 4-6

Letc,,1= 1,2,...

209

, p, denote the n-column vector whose

components are the elements of the ith row of C, for example, Fen | C12

c=]

where C = (c;;)

(4-308)

| Cin_)

and let wii, k =

given by

0,1, ...,n—landi=1,2,...,

p, be the n vector,

wii = A’c;

(4-309)

where A’* is the transpose [see Eq. (2-45)] of A*.

and let r,; be the real number, given by

Let x be an observable state

The = (Wai, X)

(4-310)

that 28, tf wes; 18 the jth component of Wi; and x; 18 the jth component of x, then

te =) wansh

(4-311)

j=l

Let X1, Xo, . . . , Xn be n indeterminates and consider the system of equations

t=

Yo wyX;

jel

&=0,1,...,n—-1)¢=1,...,p

(4812)

Then x is the unique solution of the system (4-312). Conversely, if an equation of the form (4-312) (the ri: may be different) has a unique solution X; = y;, j=1,2,...,%, then the vector y with components y; 1s an observable state of the system.

PROOF

Since x is observable, there is a 7 > 0

over [0, 7] determines x uniquely. given by y(t) =

Now

n-~l

y(t) (A’*c, > 9)

k=0 n—1

(4-313)

=D valores ®

(4-314)

=

(4-315)

k=0

where

such that y(t) = CeA4x

the ith component y,(¢) of y(t) is

eAt =

y k=0 n—-l

>

k=0

vi (t) Th

vi (t)A*

210

Basic concepts

in view of the fact that

(see Sec. 3-21) and the fact that every n * n matrix satisfies its characteristic

equation.{ Now x is (by definition) a solution of Eq. (4-312). If there were another solution x’ of Eq. (4-312), then it would follow from Eq. (4-315) that y(t) would also be given by y(t) = Cedtx’

(4-316)

on (0, 7] and, hence, that y,o.7, could not determine x uniquely. Thus, x is the unique solution of Eq. (4-312). We leave it to the reader to verify the truth of the converse part of the theorem.

Corollary 4-8

If xo and x, are observable, then Xo + X; 18 observable.

Exercise 4-17 Prove Corollary 48. Hunt: Let (wai, Xo) = reso, (Wai, X1) ™ Tesi; then (Was, Ko + 2X1) = revo + rici. Of what equation are the components of xo + x, a solution? Why are they the unique solution of that equation? An indirect argument should help in answering the last question.

Corollary 4-9

If x 1s observable, then Cex over any interval (0, T), T' > 0,

determines x uniquely. Corollary 4-10 The system is completely observable if and only wf every vector x 1s a linear combination of the vectors wix (see Sec. 2-5). Corollary 4-11 The subspace of observable states 1s the span of the vectors wi. In other words, Xo is observable if and only if xo 1s a linear combination of the vectors Wi.

Corollary 4-12

Let H be the n X nm matriz, defined by the relation H

+ [C’ ! AC’

! real

(A’)*-10']

(4-317)

Then the system is completely observable if and only if the rank of H 28 n or, equivalently, if and only if there is a set of n linearly independent column vectors of H.

In particular, if C is a1

X n matrix (row vector) c, that 1s, uf

the output is scalar-valued, then the system is completely observable if and only tf then Xn

matriz

H = [c’ !: A’c’ i - - - t (A’)* 1c]

(4-318)

1s nonsingular (cf. Corollaries 4-5 and 4-6). Corollary 4-13 If the system matriz A 7s a diagonal matrix with distinct entries, then the system 1s completely observable if and only if C has no zero columns (cf. Corollary 4-7).

These last two corollaries often provide us with a simple test for the complete observability of a system. t By the Cayley-Hamilton theorem (Theorem 2-3).

Sec. 4-18 Example 4-18

Regulating the output Consider the system

Le}-Co ol [2] +L] yan

Is it observable? Example 4-14

Well, c’ =

ol =]

_h

Hi and A’c’ = Hi

Consider the system

[7] = 2

y=

Well, c’ = H

Ze

and A’c’ = Hi

(4-319)

so that the system is observable.

(a) -[o ol [2] + [1] Is it observable?

211

(320

(4-321)

so that the system is not observable.

If we consider a unit mass moving in a frictionless environment and if we let x:(¢) denote the displacement, z.(t) the velocity, and u(t) the applied force, then z; and 2; are related

by Eq. (4-320).

The output y being the velocity could arise from the situation in which

we observe the system with a Doppler-shift radar that measures only range rate. From the velocity, we can determine the position, then, only to within an unknown constant. This is an implication of the nonobservability of the system. Exercise 4-18 Consider the system Z(t)

= airaxi(t) + aistelt) + dint)

Z(t) = aanxi(t) + Geere(t) + deu(t) y(t) = crai(t) + ests(t) Derive the conditions for noncontrollability and for nonobservability. Is it possible for the system to be both noncontrollable and nonobservable? [Illustrate by an appropriate (nontrivial) choice of numerical values. Construct an analog-computer representation of the system. Explain in terms of the analog-computer block diagram noncontrollability, nonobservability, or both. Exercise 4-19 Consider the system, with w ~ 0, Qi = ati + we + bru ft. = —wt + at. + Dau y=

2

Show that the system is controllable woless b; = b: = 0 and that the system is observable.

4-18

Practical Implications: Regulating the Output

So far, we have paid little attention to the practical implications of the concepts of controllability and observability. We shall remedy this situation by examining some practical aspects of these concepts in this and the following two sections. In this section, we shall consider the problem of regulating the output.

212

Basic concepts

Let us suppose that our dynamical system is linear and time-invariant and that it is given by £(é) = Ax(t) + Bu(?d) (4-322) y(t) = Cx(t) + Du(d) (4-323) In engineering practice, we and holding it there in the we shall speak of regulating 1. In driving the output

are often interested in taking the output y(¢) to 0 absence of subsequent disturbances; thus, when the output, we shall imply that we are interested: to zero

2. In determining the control which will maintain the output at zero

thereafter Let the initial time be é = 0. Woo, 7) such

Suppose that we have found a control

that

y(T) = 0

(4-324)

Suppose that the system is observable; this implies that we can compute the initial state x» at ¢ = 0. Since Eq. (4-322) is linear and time-invariant, Eqs. (4-324) and (4-323) yield the relation

C{etrz, 4 ear f e-4* Bu(r) dr} + Du(T) = 0

(4-325)

Now suppose that the system is also controllable: this means that we can find & Uo,r) Such that x(7) = 0. Moreover, if then

x(t) = 0

and y(t) = 0

u(t?) = 0 foralli > T for alli > T

(4-326) (4-327)

Therefore, if the system 1s completely controllable and completely observable, then we can regulate the output.

We note that controllability and observability provide us with sufficient conditions for the regulation of the output; however, they certainly do not

represent necessary conditions.

To see this, we observe that, in order to

regulate the output, it suffices to find a control fi(é) for t > T such that the solution of x(t) = Ax(t) + B&(t), denoted by &(t) for t > T, satisfies the equation Ci(t) + Dat)

= 0

for allt >

T

(4-328)

Let us denote by A the set of all (7) for which such a control f(t) can be

found.

If the system is controllable, then we are guaranteed{ that we can

force the system from x, to 7. eee to

However, the system may not be control-

and yet it is conceivable that we can find a control that will force xo

t Because controllability implies that any state is acceasible from any other state.

Sec. 4-18 Example 4-16

Regulating the output

213

To illustrate this point, we consider the simple second-order system

(4-320)

exo) Lo -i} xo) + Lo}

(4-330)

y(t) = x(t)

It is easy to see that the system is observable but not controllable. Let &: and & denote the initial values of the state variables z:(t) and z:(¢). Equation (4-329) is easily integrated to yield

zo{t) = Ere

t

(4-331)

zi(t) = 1 — bet + be + I u(r) dr

Since the system is observable, we can compute é: and ¢; from the output y(t). Let T be a given time. We wish to find a control such that (7) = 0. Since y(t) = x(t), we must make z,(7) = 0; clearly, if x:(2) = 0 for all ¢ > T, then y(t) = 0 for all ¢ > T.

These considerations define the set @ in the 22; state plane.

Clearly, the set A is the

0 =2,(T) = & — be? + &s + kT

(4-332)

entire z; axis. Let u(t) = k be a constant; then at ¢ = T we must have, from Eq. (4-331),

which implies that the constant control

u(t) ==

Oh

We

for0 T and, Thus, although the system is not controllable, we examine an analogous situation in Sec. 4-20, except be bounded. Now suppose that we consider the system with Eq.

t>T

(4-334)

hence, z:(t) = y(t) = Oforallt > T. can regulate the output. We shall that we shall require that the control (4-329).but with output y(¢) given by

y(t) = z(t)

(4-335)

Then the system (4-329) and (4-335) is neither controllable nor observable. see that, in this case, we cannot regulate the output.

It is easy to

Let us now examine some other implications of controllability and observability. We shall see that quite often the fact that the system is not observable or not controllable means that there are more state variables than necessary. To illustrate this point, we consider the physical system shown in Fig. 4-12.

The constants k1, ke, kz, and ky are gains, and 2;(£), r2(t), zs(¢),

and 2,(t) are the outputs of the four integrators.

of the x; variables, 7 = 1, 2, 3, 4.

The output y(t) is the sum

It is easy to establish that the vector ry(t)

x(t) =

a x(t)

(4-336)

212

Basic concepts

Let us suppose that our dynamical system is linear and time-invariant and that it is given by z(t) = Ax(t) + Bui?) (4-322)

y(t) = Cx(é) + Du()

(4-323)

In engineering practice, we are often interested in taking the output y(¢) to 0

and holding it there in the absence of subsequent disturbances; thus, when we shall speak of regulating the output, we shall imply that we are interested: 1. In driving the output to zero 2. In determining the control which will maintain the output at zero thereafter Let the initial time be ¢t) = 0. Suppose that we have found a control Uco,r) such that

y(7) = 0

(4-324)

Suppose that the system is observable; this implies that we can compute the

initial state x) at¢ = 0. Since Eq. (4-322) is linear and time-invariant, Eqs. (4-324) and (4-323) yield the relation

Cletx, + ett f" * Bu(r) dr} + Du(T) = 0

(4-325)

Now suppose that the system is also controllable: this means that we can find & Uco,r) Such that x(7) = 0. Moreover, if then

x(t) = 0

and

y(t) = 0

u(t) = 0

for allt >

for allt > T

T

(4-326)

(4-327)

Therefore, zf the system is completely controllable and completely observable, then we can regulate the output. We note that controllability and observability provide us with sufficient conditions for the regulation of the output; however, they certainly do not represent necessary conditions. To see this, we observe that, in order to regulate the output, it suffices to find a control fi(t) for ¢ >

7 such that the

solution of x(¢) = Ax(f) + B&(é), denoted by &(¢) for ¢ > T, satisfies the equation Ci(t) + Da(t) = 0

forall i > T

(4-328)

Let us denote by A the set of all £(7) for which such a control (t) can be

found.

If the system is controllable, then we are guaranteed{ that we can

force the system from x, to H. ey to

However, the system may not be control-

and yet it is conceivable that we can find a control that will force x»

t Because controllability implies that any state is accessible from any other state.

Sec. 4-18 Example 4-15

Regulating the output

213

To illustrate this point, we consider the simple second-order system 41(t)

Level Lo

0

1

x(t)

1

1] [2a] + Lolo

y(t) = x(t)

(4-320)

(4-330)

It is easy to see that the system is observable but not controllable. the initial values of the state variables z:(t) and 2z.(t). Equation grated to yield

z(t) = &se-*

Let £, and & denote (4-329) is easily inte-

,

(4-331)

nit) = bi — beet + Bat fae) a

Since the system is observable, we can compute £: and ¢; from the output y(t).

Let T be

a given time. We wish to find a control such that ¥(7) = 0. Since y(t) = 2z:(t), we must make 2:(7') = 0; clearly, if z(t) = 0 for all ¢ > T, then y(t) = 0 for all ¢ > T.

These considerations define the set A in the 22: state plane.

entire

Clearly, the set A is the

x3 axis.

Let u(t) = k be a constant; then at ¢ = T we must have, from Eq. (4-331),

O = 2(T) = bi — ge? + &2 + kT

(4-332)

which implies that the constant control

uf) =k =

hb

for0 T and, Thus, although the system is not controllable, we examine an analogous situation in Sec. 4-20, except be bounded. Now suppose that we consider the system with Eq.

t>T

(4-334)

hence, 2:(t) = y(t) = Oforallé > T. can regulate the output. We shall that we shall require that the control (4-329). but with output 4(¢) grven by

(4-335)

y(t) = x(t) Then the system (4-329) and (4-335) is neither controllable nor observable. see that, in this case, we cannot regulate the output.

It is easy to

Let us now examine some other implications of controllability and observability. We shall see that quite often the fact that the system is not observable or not controllable means that there are more state variables than necessary. To illustrate this point, we consider the physical system shown in Fig. 4-12.

The constants ky, ke, ks, and k, are gains, and x,(t), z2(t), x(t),

and 2,(t) are the outputs of the four integrators. of the x; variables, 7 = 1, 2, 3, 4. Tt is easy to establish that the vector

The output y(¢) is the sum

x(t)

x(t) =

an a(t)

(4-336)

214

Basic concepts

qualifies as a state vector, and so the physical system of Fig. 4-12 can be represented by the equations ky

£() = Ox) +] 57 | wo

(4-337)

ks y@=f1

1

1

1

(4-338)

It is left to the reader to verify that the system described by Eqs. (4-337) ky

f

|

Xs (¢)

| xo(t)

u(t)

{|

*

Y

t

f 3

°

f

ty (t)

f

xl)

,5

yt)

+

v he

Fig. 4-12.

A physical system simulated on an analog computer.

and (4-338) is neither controllable nor observable. Yet it is easy to establish that the output satisfies the differential equation y(t) = ku(t) where

k=

ky

+

(4-339) ky

+

ks

+

ka

(4-340)

If k + 0, then it is easy to see that the system

2(t) = ku(t) y(t) = 2(é)

(4-341) (4-342)

is both controllable and observable and, hence, that the output can be regulated and that z(t) = y(t) qualifies as a state.

Clearly, from the input-output point of view, the system (4-337) and (4-338) and the system (4-341) and (4-342) are equivalent. This simple

example points out that the choice of the state of the system and especially

the dimensionality of the state space are unimportant from the input-output point of view. If, however, the dimensionality of the state space is larger than necessary, then we usually end up with an uncontrollable and unob-

servable system.

Reference Z-1 contains a rigorous exposition of these ideas; Ref. B-20 provides useful relations for systems treated from the output point of view, using the concept of output controllability.

Sec. 4-20

4-19

Anexample

215

Practical Implications: The Effect of Canceling a Pole with a Zero

We shall show, in this section, that the conventional pole-zero cancellation techniques employed in servomechanism design result in systems which are uncontrollable (i.e., not completely controllable) in the sense of our definition. We consider a system with distinct eigenvalues whose state variables z(t) satisfy the differential equations [see Eq. (4-199)] Z(t)

=

a(t)

+ vu(d)

+=1,2,...,0

(4-343)

where »; is the residue of the transfer function H(s) given by Eq. (4-170) at the ith pole. If we suppose that, in the transfer function H(s) given by Eq. (4-170), a1 > 8

(4-344)

that is, that there are both a pole and zero at s = 8, = o;, then the equation

for the state variable z(t) is

Z(t) = 8:2, (t) + viu(t)

where v; = ,, the residue of H(s) at 8s: = 1. U1

=

p}

=

since \; = 8; = ¢;.

But, by virtue of Eq. (4-202), om) °° (An

bm(A1 — 61)(M1 — 2) . (Aa

—

A2) (Aa

—

da)

(4-345)

se

(Ay

_

rx)

=

0

(4-346)

Thus,

Z(t) = 817: (t)

(4-347)

and so the system is not completely controllable in view of Corollary 4-7. 4-20

Practical Implications: An Example

From a

practical point of view, controlling a system means that the output

can be made to respond in a desired way. With this observation in mind, let us examine the effects of noncontrollability in the very simple example that follows. Consider the system with (scalar) output y(é) and (scalar) input u(t) which are related by the second-order linear differential equation

g(t) + 29) + yf) = ut) + ud)

(4-348)

The transfer function H(s) of this system is given by

vs) ya —2t) at) ua) ~ 2°) = 34 os+1~ @ +1)

3%)

216

Basic concepts

Thus, there are two poles and one zero, all at s = —1. Following the procedure of Sec. 4-10, we define the state variables z;(¢) and z(t) by setting z(t) = y( .

Z(t) | _

7 E

Ba

Then

(4-350)

2a(t) = y(t) — u(t) 0

1

1} J 2:(t)

0] + ||

>|

and we claim that this system is not controllable. form the matrix

G = [b } Ab]= |

(4-351)

u(t)

To verify this claim, we

7

(4-352)

note that its determinant is zero, and apply Corollary 4-6. Let us now apply the similarity transformation to Eq. (4-351), which puts

the system matrix A into its Jordan canonical form (see Sec. 2-10).

J= and let

I~

P = |}

1 1

Let

(4-358)

0

|

(4-354)

Then J is the desired canonical form, and we let x=P¥%z that

21 (t)

18,

=

z(t)

(4-355) =

y(t)

ao(t) = a(t) + a(t) = y(t) + 9

(4-356)

— ult)

Then the new state variables 2,(t) and .z.(t) satisfy the equation

éi(t)|

iol If we

set 7,(0) = &, and

given by

|

-1

1

1] | a(t)

[> -ilLBel + fo | uo 2.(0) = &, then

the solution

(£357) of Eq.

(4-357)

is

i(t) = fret + fate + emt f° eruls) dr

(4-358)

we(t) = fse*

(4-359)

Let us now pose the following problem:

Given initial values : and &:,

determine the control which will drive the output y(t) to zero and cause 1t to remain at zero thereafter.

So we want to find a control such that, for some 7, y(t) = 0

y(t) =0

for alli > T

+ forallt>T

(4-360)

Sec. 4-81

Normal linear systems

217

This requirement on the output is equivalent to the requirement on the state variables 7,(¢) and x(t) of Eq. (4-356) that

2,(t) = 0 r(t) = —u(t)

for alli > T for allt > T

(4-361)

We deduce from Eq. (4-358) that any control u(t) which satisfies the equation

— [f eu) dr =

+ BT

(4-362)

will indeed drive z; to zero at time 7, and we deduce from Eq. (4-359) that application of the control u(t) = —&e*

for allt > T

(4-363)

will ensure that 9(¢) = 0 for allt > 7. Thus, although the system is not controllable, the objective of driving the output and its time derivative to zero can be attained. Now let us impose a constraint on the magnitude of the control. In other words, let us suppose that \u(t)| < M for all ¢ (4-364) (cf. Example 4-6). Again we pose the question of whether or not we can find a control, now satisfying the constraint (4-364), which will drive the output y(f) and its derivative ¥(¢) to zero. If we assume that £, &, and 7

are such that there is a control u(é) with |u(t)| < M such that

— [,

T

(4-365)

u(t) di = fi + &T

then z:(7') will be zero, but it still may be true that & is such that

lée"|> M

forT 0

and

lim

f’(z) < 0f

(5-4)

(5-4a)

If a 1s a minimum of f, then

lim

f(z) >0

(5-5)

lim

f'(z) < 0

(5-6)

and tf b 18 a minimum of f, then

Various examples of this condition are illustrated in Fig. 5-2. t This means that f’(z+) and f’(r—) exist for all z but not necessarily f’(z). t See Definition 3-24.

224

Conditions for optimality

It will be instructive to briefly examine the type of reasoning which leads to necessary condition NC2. Supposing that x* is an interior point of [a, 6] and that x* is a minimum of f, we note that, for ¢ sufficiently small, say for 0 < we have

lt| 0 leads to a contradiction.

Hunt: If f’(z*) > 0, then there is an e > 0 with

f(z*) >e>0. Pick 38 >0, so that |t| < & implies that —& 0, then x* is a (local) minimum of f. Although there are a variety of sufficiency conditions for a point to be a minimum of f, this is the only one which will be used in this book. Now let us turn our attention to the problem of finding the minimum of a function g of several variables. In other words, g is a real-valued function of a vector x with components 2, 2, ..., %. The question of minimizing such functions is a difficult one, and so we shall only indicate those results which we shall use later on. To begin with, we have: Definition 5-3 A point x* in R, is said to be a (local) minimum of g if there 18 an « > O such that |x — x*|| < ef implies that

g(x*) < g(x)

(5-11)

Again, x* 1s a minimum of g tf g(x*) ts no greater than g(x) for all x in a

neighborhood of x*.

We also have:

Definition 5-4

A point x* in R, is said to be an absolute minimum of g if

g(x*) < g(x)

(5-12)

for all xin Ry. We again observe that an absolute minimum of g is also a local minimum of g but that a local minimum of g need not be an absolute minimum of g. Now, we are often interested in determining the minima of g in certain subsets of R, and in studying functions which are not necessarily defined on all of R, but rather on a subset of R,. In particular, we have: Definition 5-5 A bounded open subsel D of R, 1s called a domain if, given any two potnis Xo and x, in D, there ts a precewrse linear function (see Eq. (3-104)]

Polygonal paths

4 from (0, 1] into D such that (0) = xo

and 2(1) = x, (that is, there zs a polygonal path in D which connects x» and Xi, as 78

connecting x_ and 4,

Fig. 5-3

A typical domain.

illustrated in Fig. 5-3). Letting oD denote the boundary of D (see Definition 3-10) and letting D be a subset of R, such that

t Recall that |

DCDBcCDVU a | denotes the Euclidean norm.

(5-13)

440

Uonailions

Ior

opumalty

we say that a point x* of D is a (local) minimum of g in D if there is an « > 0 such that

t€ S(x*, 0 ND

(5-14)t¢

g(x*) < g(%)

(5-15)

implies that

In other words, x* 13 a minimum of g in D if g(x*) is no greater than g(x) for all in D which are close to x*. So we shall now suppose that D is a given domain, that g(x) = g(x, 2s,

. , Za) is continuous on the closure D (= DU aD) of D, and that g has

continuous partial derivatives with respect to each of the components 17, 7s, . ,» 2, 0f xon D. We then have the necessary condition: NC3 If a point x* in D tg a minimum of g, then all the partial derivatives of g vanish at x*, that is,

an

) = 99 om (x*)*\) —=

9g

x) _

to.

= 8G or. (x*)*) e

—

=0

(5-16)

or, equivalently, the gradient of g (see Eqs. (3-67) and (3-68)] 18 the zero vector at x*. We may view this necessary condition as follows: Let a be a unit vector with components a, a2, ... , a» and let f(t) be the function defined by

f(t) = g(&* + ta)

(5-17)

If x* is a minimum of g, then 0 is a minimum of f, and it follows that

70) = Y 22 ai = (Wglz*), «) = 0 Ty

tml

(5-18)

.

Noting that f’(0) is the directional derivative of g in the direction e« at x* [see Eq. (3-80)], we may conclude that necessary condition NC3 is equivalent to the condition that the directional derivative of g vanish at x* for every direction a. Let us carry the reasoning of the previous paragraph a step further. Supposing that g has continuous second partial derivatives, we may write, in a neighborhood of 0, the relation f

()

7

n

on

arg

2

A

OX; Ox;

t=lj

(x* + ta)a,a;

(5-19)

and we may conclude from sufficiency condition SC1 that 0 will be a mint See Eqs. (3-4) and (2-3).

Sec. 6-8

Ordinary minima

227

imum of f if f’(0) = 0 andif

#0) = YY

tml j=zl

(5-20)

sade axa > 0

The result of this line of thought is the following sufficiency condition: SC2_ If the directional derwwative of g vanishes at x* for every direction a and

af the inequality

> Ys= ezg, aay > 0 t=]

(5-21)

j=]

holds for every direction a, then x* is a (local) minimum of g. Now, the term in Eq. (5-21) looks very much like an inner product (see Sec. 2-11). If we let Q be the symmetric n X matrix whose entries are the (0°9/dz; d2z;)(x*), that is, if Q is given by the relation

an

Q=

mn a1

* 3a; (=*) Jere _O2n“0 (x

=a ox2? )

*

oT )

OLn OF? (x*)

5,32, ©)

"9 * O22 On (x*) 02,2 (x*)

(5-22) 5

then we may define an inner product Q on FR, whose matrix is Q [see Eq. (2-83)], and we can assert that sufficiency condition SC2 is equivalent to the following condition: SC3 If the gradient of g is 0 at x* and if the matriz Q of Eq. (5-22) 18 positive definite (see Eq. (2-84)],¢ then x* 1s a minimum of g. In particular, if g is a function on R2, then the conditions

2Og (gs) = 30 (x)a = 0

(5-23)

ih 2") 28 ar)- | PE an} >a

(5-25)

54 (x*) > 0

(5-24)

guarantee that x* is a minimum of g. We observe again that the way in which these conditions were obtained was based upon the evaluation of the effect of small changes in x on g by means of the ‘‘derivative” (partial derivatives here) of g. We leave it to the reader to examine the results from this point of view. Tt Bee Sec. 2-15.

| 228

Conditions for optimality

Example 6-1

the origin 3%g/dx* = matrix Q surface in

Consider the function g defined on R; by g(z, y) = (x? + y?)/2.

Then

(0, 0) is a minimum of g, since ég/dx = x, dg/dy = y, 9°g/axdy = 0, d%g/dy? = 1, 1 imply that the sufficiency condition is satisfied at (0, 0). We note that the of Eq. (5-22) in this case is the identity matrix. The function g, viewed as a Ry, is a paraboloid of revolution and is illustrated in Fig. 5-4.

«

y

Fig. 5-4 ple 5-1.

The paraboloid of Exam-

Fig. 5-5 The hyperbolic paraboloid of Example 5-2. The origin is the “‘saddle’’

point.

Example 6-2 Consider the function g defined on R: by g(z, y) = (x? — y?)/2. Then 0g/dz and dg/dy vanish at the origin, but the origin is neither a minimum nor a maximum of g. Here, the matrix Q of Eq. (5-22) is given by

e=[5

1

0

1]

and is neither positive nor negative definite.t The function g, viewed as a surface in Rs, is a hyperbolic paraboloid and is illustrated in Fig. 5-5. The origin is called a saddle point of g, and the motivation for this name should be apparent from the figure.

5-3

Ordinary Minima with Constraints: A Simple Problem

Iu the previous section, we considered the problem of determining the minima of a function over a subset of the Euclidean space R,; now we are going to examine the problem of determining the minima of a function sub-

ject to certain constraints.

In order to motivate the general results which

we shall develop in the next section, we shall here consider some special illustrative situations involving functions on Rz.

More specifically, let us suppose that f = f(z, y) isa real-valued function defined on R, and that A(z) is a real-valued function defined on R. We may view thé equation

y=h(zt)

(5-26)

xER

as defining a curve in R:,{ and we note that if we define g(z, y) by setting

(5-27)

g(x, y) = y — h(z) t See Sec. 2-15.

|

¢ This curve is the set {(z, y): y = h(x)}, and, of course, R; is the zy plane.

Sec. 5-8

Ordinary minima with constraints

then the equation of our curve becomes

229 ’

g(x,y) = y — h(x) = 0

(5-28)|

Now we can ask the following question: Where are the minima of f(z, y) along the curve y = h(x)? In other words, we want to find the points (x*, y*) on the curve [that is, y* = h(x*)] such that if (2, y) is a point on the curve which is ‘‘near’’ (x*, y*), then

f(z*, y*) < fz, y) Since y* = h(x*) descriptive form We

and

may

write Eq.

(5-29)

fix*, h(x*)) < fiz, h(z)]

call such a point

y — h(x) = 0.

y = A(z), we

(5-29)

(z*, y*) a minimum

in the more

(5-30)

of f subject to the constrain

Now let us use the perturbation idea to obtain a necessary condition fo! a constrained [by y — A(x) = 0] minimum of f. We shall suppose thai (x*, y*) is such a constrained minimum and that f and hk are sufficiently

smooth (i.e., differentiable) to validate our procedures. (z*, y*), then we may write

x= 2* + ef

If (x, y) is ‘near’

= y* + eb

where ¢ is a small, positive number and & = (£, &) is a vector. that

f(x, y) = f(z*, y*) + AVF (2*, y*), E)+o0(€)

(5-31; It follows

(5-32)

where Vf(z*, y*) is the gradient of f at (x*, y*) [see Eq. (3-67)] and o(e) is a

term depending on e, with the property that

lim0 2)€ = 0

(5-33)

If the point (z, y) is not only ‘“‘near’’ (x*, y*) but also lies on our curve, ther

we must have y = h(x), which means, in view of Eqs. (5-31) and of the fact that y* = h(2*), that y = h(x) = h(a* + e&:) = h(x*) + cfs = y* + eke

(5-34

However, since x is ‘near’ z*, we may write

h(a* + ef) = Alar) + eZ (eB + 0f6)

(5-35)

and hence we may conclude that

f= Fee t See Sec. 3-13.

Note that our curve is the set S(g).

(5-36)

230

Conditions for optimality

[as lim o(e)/e = 0). o>

y=

A(z),

Thus,

for perturbations of (z*, y*) along the curve

Eq. (5-32) becomes

f(z, y) = fe", y®) +

=

(z*,y*)

of

ah

Ey +e ay

(ets) Gr

2?

£1 + o(e)

(5-37)

Since (z*, y*) is a minimum of f along the curve, since ¢ is positive, since

lim o(e)/e = 0, and since £, may be either positive or negative, we must have

«0

4}

Oz

, af|

|(x*.y*)

OY

ah

|(2*,y%) a2

z?

= 0

(5-38)

This equation is thus a necessary condition for (x*, y*) to be a minimum of

f subject to the constraint y — A(z) = 0. We observe that in deriving Eq. (5-38) we used perturbations along the curve. In other words, once £, was chosen, then ¢, was determined by Eq. (5-36). Thus, as is consistent with Eq. (5-30), our derivation depended upon only one variable. We see, therefore, that the effect of the constraint was to limit our freedom in choosing a perturbation and, hence, to decrease the dimension of our problem.

Exercise 5-8 Derive Eq. (5-38) by considering the function F(z) = fiz, h(z)) of one variable. Use the perturbation method.

Now let us look at Eq. (5-38) in a somewhat different way by considering the function

g(x,y) = y — h(a)

(5-39)

Our constraint may then be written as and we observe that

g(z, y) = 0

(5-40)

og By—=1

5-4] (5-41)

is not zero at any point along our constraint curve.

a(f, y)9) 9) _ det a(x,

Ox aa9 | or

-

= det

- Ff,

iagdy af

OY

oh

—-— | 2s dx

oah

Oz * Oy dz

t See Sec. 3-12.

Thus, we have (5-42) f

af

ay 1

(5-43) (5-44)

Sec. 5-8

Ordinary minima with constraints

231

where the term on the left is the Jacobian of f and g with respect to x and y [see Eq. (3-89)]. It follows that Eq. (5-38) may be interpreted as

a(f, g) a(x, y) What

(z*,y*)

(5-45)

~

is the significance of this interpretation?

For one thing, it means

that the system of equations in the variable p,

i (z*,y)

20

ay lerwy has a solution p = p*.

(5-46)

? dy leew

In fact, since g(x, y) = y — A(z),

_pe_ p=p*= is one such solution.

(xz*,y*)

_ Of By lcenwe) _ const

(5-47)

We observe that if G(a, y)

=

f(a, y)

+

p*g(2, y)

= f(z, y) + p*ly — A(z))

(5-48)

where p* is a solution of Eqs. (5-46), then we may view Eqs. (5-46) replacing p as a necessary condition for (x*, y*) to be a minimum of In other words, if (x*, y*) is a minimum of G(z, y), then we know partial derivatives of G with respect to z and y must be zero at {see Eq. (5-16)], so that we have

aG _ 8g . Sr lets) ~ ag her) + P I

9)Oz

\(e*.y*)

ype)Ox

of

Thus,

p* y). the y*)

W304 yey \(z*,¥*)

* ByaG lew) _@ = Fy (imu) + PPale, w)) By kee 1?

with G(z, that (x*,

+ 9

8

=

ay enw —

we are led to the following conclusion:

Jn order that (x*, y*) be a

minimum of f subject to the constraint g(x, y) = y — h(x) = O, tt 28 necessary that there be a number p™ such that the equations

of

7

Ox

|(x*,y*) + P

og

7

Ox

\(2*.y*)

—

of « OF _ Oy \(z*y*) +P dy zy) 0 g(x*, y*) = y* — h(z*) = 0 are satisfied.

The number p™ is called a Lagrange multiplier.

(5-49)

(5-50) (5-51)

2352

CUonditions for optimality

Let us now generalize our problem a bit. Supposing that f(z, y) is a real-valued function defined on R. and that g(z, y) is a real-valued function on FR, {not necessarily of the form y — h(x)], we may ask for necessary conditions for a minimum of f along the curve g(z, y) = 0. Well, assuming that f and g are sufficiently smooth (i.e., sufficiently differentiable) and assuming that the norm of the gradient of g is not zero, i.e., that

Vol = (32) + (22)"} >0

(5-52)

we can assert that if (7*, y*) is a minimum of f subject to the constraint g(x, y) = 0, then there is a number p*, called a Lagrange multiplier, such that z*, y*, and p* are a solution of the system of equations

that is, that

ofse

[eeyey +

of dy \t2*y*) +

of,

99 _

of

og

PY « 99Se

g(z, y) = 0 legen =_ 9

:

az + P35, — 9

(5-58)

—3 +p ~~ ay = (0

(5- 4) 5-54

» 99 _ dy let) g(z*, y*) = 0

(5-55) (5-56) (5-57) (5-58)

The verification of this assertion is quite similar to what we have done before, and only the role of the assumption of Eq. (5-52) requires clarifica-

tion.

This assumption

[Eq.

(5-52)] ensures that the equation g(z, y) = 0

can be locally ‘“‘solved’’ for either z or y.

For example, if

og By ltetw

0

(5-59)

then, in a neighborhood of z*, there is a function h(x) such that

y* = h(x*)

0 = gz, h(x)]

dh

—_—-ag/adz

(5-60)

(5-605)

that is, the equations g(x, y) = 0 and y = A(z) define the same curve ‘‘near’”’ (x*, y*). We now leave it to the reader to verify the necessity of Eqs. (5-56) to (5-58).

Example 6-8 Let f(z, y) = —xy and g(z, y) = y + 2xe*. What are the potential candidates for minima of f subject to the constraint g = 0? Here, we can take A(z) = t Actually, we only require that apparent.

||Vgij > 0 on the curve g{z, y) = 0, as will become

Sec. 5-4

Necessary conditions and Lagrange multipliers

—ze* and use Eq. (5-38).

233

Thus, we must solve the equations

—y + (—2z)(—e* — ze*) = 0

vy + ze? = 0

Clearly, (0, 0) and (—2, +2/e*) are the only solutions of these equations and thus are the only candidates for the minima of f subject to the constraint g = 0. Consideration of the function z%e? leads us to the conclusion that (0, 0) is the minimum of f subject to the constraint g = 0 [while (—2, +2/e?) is a maximum of f subject to our constraint]. Example 6-4 Let f(z, y) = —zy and let g(z, y) = x*+y?— 1. What are the potential candidates for minima of f subject to the constraint g = 0? First of all, we note that the assumption of Eq. (5-52) is not satisfied at (0, 0); however, (0, 0) is not a point on the curve g(z, y) = 0, and so we can still apply the necessary conditions of Eqs. (5-56) to (5-58). Thus, setting

Az, y, p) = —zy + p(x? + y? — 1) we find that we must solve the equations

OG _ TY ap =

+ 2pz == 0

—z + 2py = 0

az,y)ez?>+ty?—i=0 It is easy to verify that the solutions of these equations are:

y* = 1/2),

y* = —1/+/2), ve

1

.

(pt =h,

(p* = —},

(1/+/2,

Exercise 6-4

ct = 1/73,

1/+/2),

z* = —-1/+/2, (-1/+V/2,

y* = 1/+/2).

—1/+/2),

(1/+/2,

(p* = }, z* = 1/+/2,

(p* = -§,

2% = 1/2,

Thus, our candidates are —1/4/2),

and

(-—1/+/2,

Show that the points (1/+/2, 1/+/2) and (—1/+/2, —1/+/2) actually

provide the minima in Example 5-4.

5-4

y* = -1/V/2),

Hunt: Consider the function

—|z| 4/1 — |z|?.

Ordinary Minima with Constraints: Necessary Conditions and the Lagrange Multipliers

Having bolstered our intuition in the previous section, we are now prepared to tackle the problem of determining the minima of a function subject to certain constraints. The development of necessary conditions for this type of problem will lead us to the notion of Lagrange multipliers, which will find frequent application in the sequel. We shail begin with a precise statement of the problem, then we shall present the desired necessary conditions in Theorem 5-1, and finally, we shall briefly discuss these necessary conditions. So let us now suppose that D is a given domain in R,, (see Definition 5-5), that f(x) = f(z:, a2, . . . , Zn) is a continuous real-valued function on the closure D of D,f and that gi(x) = gi(a:, x2, ..., In), 92(X) = G2(Xi, Xe, ~~.» Zn), - - - 5 Gm(X) = Gm(Ti1, Zz, - . - , In), M O such that Ix —x*\| ix", p*@, ut, fl

(5-273)

satisfying the boundary conditions

x*(to) = Xo

p*(n) = 2 xe)

(5-274)

b. The function H[x*(t), p*(t), u, i) has a (possibly local) minimum a function of uatu = u*(t) for t in [to, ti]. t See Sec. 2-15.

as

Sec. 6-7

Necessary conditions

265

Let us now make some comments about this theorem. First of all, we note that we have not formally proved it; in point of fact, the formal proof, based upon variational methods, is quite complicated, requires various assumptions (e.g., suitable differentiability), and is, in any case, beyond the scope of this book. Secondly, we observe that, in contrast to Theorem 5-2,

we have

only

shown

that u = u*(é)

is a local minimum

of HA[x*(é),

p*(t), u, t]; 2 78, in fact, true thatu = u*(t) ts an absolute minimum of H{x*(é), p*(t), u, ¢], as we shall see in our discussion of the minimum

principle.

We

can thus begin to see the need for some additional tools beyond the basic variational ones; actually, in a later section, when we consider a fixed-endpoint problem, we shall find it difficult even to show that the optimal control extremizes H and we shall thus be forced to seek a different approach. Finally, we observe that, along an optimal trajectory, the total denvative of H with respect to time is the same as the partial derivative of H with respect to time, 1.e., that

dH|x*(t), p*(), u*@, t

2 (9 | ay \ 80) aqy\ +(F | PO)

a

/OH | + (se +a") 4 ae QS —

(5-275) (5-276)

at |,

ee

in view of Eqs. (5-273) and (5-253); clearly, this implies that 7f H does not depend on t explicitly, then H is constant along an optimal trajectory [compare with Eq. (5-176)]. Example 5-8 As an illustration of the type of complications which arise in developing a formal proof based on variational methods, we shall show that Eq. (5-270) implies the nonnegative definiteness of (d%H /du?) |, in the special case where n = m = 1, that is, where z(#) and u(¢) are scalars. Thus, our system equation is

#(t) = flx(é), u(t), @ and the perturbation equation (5-258) becomes

W(t) = a(t)y(t) + B(t)n(¢)

(5-277)

where we have, for convenience, set a(t) = (af/dz) |, and b(t) = (af/du) |,. assumptions, we may write Eq. (5-270) in the form

an ve) +f [S

=| 9

ox?

for all »(t) + 0.

With these

| di> 0 (5-278)

We wish to show that Eq. 6.278) implies that oH

au? |,

>0

(5-279)

So let us suppose that (82H /du*) |, < Oat é¢ =i; then (by the continuity of (09H /éu*) |,)

we may assume that, on a small neighborhood S of ¢ (that is, on a small interval about 2),

oH du?

|,

< —a

a>O

(5-280)

266

Conditions for optimality

We now claim that b(t) ~ 0 on any subinterval of S. For if (¢) = 0 on a subinterval S: C S, then, letting y(t) = 0 on the complement of S, [that is, n(¢) = 0 for? E [t,, t:] — S,] and letting n{#) =~ 1 (say) on S:, we would find that the left-hand side of Eq. (5-278) was negative (since ¥(¢) would be identically zero for this choice of n{¢)}. Thus, we may suppose that b(t) # 0on S._ It follows from Eq. (5-277) that

n(é) = B-(t y(t) — b-1(t)a(t)y(t)

(5-281)

We now restrict ourselves to functions »(t) which are zero on the complement of S (that is, on [te, t:] — S) and for which the corresponding y is zero at the end points of S. In other words, if S = [f — «, ¢ + ¢}, then n(t) = 0 for t& 0

(5-286)

+ ¢]; and, by continuity, we may also suppose that

max

tEf— kallw||2M() (5-309)

for all vin R,, for all w in Rn», and for t in (to, t:|, and where the equality

may hold only if w = 0; this implies that the matrix (0°H /du’) |, 18 positive definite for t in {to, ti] and that the entire n +m Xn +m matriz

appearing in Eq. (5-309) 1s positive semidefinite. Then 4 is a local minimum of J. PROOF Let us suppose that condition 6 is satisfied whenever

ju — a] = max {|lu@) — a@ll} 895 (G)(u — 4) — lel) 1804) — lel? [alae a

(5-311)

Thus, making use of Eq. (5-305) and conditions c and d, we conclude that

J(u) — J(a) 2 ae t See Sec. 2-15.

tIe(t) |]2B0 (ex) + ka = f° l m(2) ||204 (2) dt

— lel*l4(6s)|#00() — lel? f" IIn@|0L( dt (6-812)

Z2éU)

Uonditions for optimality

and hence that

ea) — 1¢8) > [3 — te tent | eleeneme ||? (2) dt (5-313) e*||n(é)

t Ir E — Iel Imo

However, in view of the fact that ¢(é) is the solution of Eq. (5-297) with initial condition ¥(t.) = 0, we observe that

lel [I4(és)|| < Ne] max

t€Elte,t1)

|ln(|| = Nel ln

for some N > 0 which does not depend on n. there is a 69 such that

0 by

J(u) — J(@) > 0

for all u such that |lju — || < 5).

(5-314) (5-315)

(5-316) (5-317)

Thus, we have shown that @ is a local

minimum of J. Corollary 5-1 Suppose that M(t)= 0. Then, for 0 to be a local mintmum of J, u 18 sufficient that: a. (0H/du) |, = 0 b. (0? /du’) |, be positive definite c. Then+mXn-+m matrix appearing in Eq. (5-309) be postte,e semidefinite

Corollary 5-2

Suppose that the dynamical system 1s linear, 1.¢e., that

x(t) = A@x@) + BY)ul) —s K(to) = Xo

(5-318)

and suppose that the performance functional J is quadratic, 1.e., that

J(u) = Hx(t), Fx(t:)) + 4 [ [(x(é), Q(t)x(é)) + (u(t), R(t)u(d))] dt (5-319) where F, Q(t), and R(t) are, respectively, n

KX n,n Xn, and m X m matrices.

Then, for A to be a local minimum of J, it ts sufficient that:

a. The matrix R(t) be positive definite for t in (to, t:]

b. The mairiz Q(t) be positive semidefinite for t in [to, t:]

c. The matrix F be positive semidefinite d. fi be a solution of the equation Ra)

+ B’(OP@) = 0

(5-320)

where h(t) ts the solution of the linear differential equation

b(t) = —A’()PO — QHL@

(5-321)

OO.

GO

SUICICNCy

CORCGITIVOS

ait

satisfying the boundary condition

(5-322)

f(t:) = FR(t1) £(t) being the solution of Eq. (5-318) corresponding to fa. Corollary 5-2 will be used repeatedly in Chap. 9. Exercise 6-18

Prove Corollary 5-2.

We observed in Sec. 5-7 that if u* was an optimal control, then (0@H/du) |, was zero and the matrix (d?H/du’) |, was positive definite. In this section, we supposed that the control & satisfied these two conditions and the additional conditions b, c, and d of Theorem 5-4, and then we proved that @ had to be a locally optimal control. It is in this sense that we refer to the sufficiency conditions of Theorem 5-4 as being a “‘strengthening’’ of the necessary conditions of Sec. 5-7. Exercise 6-14

Consider the first-order system

2) = -2 -7O eum

20) = 20

4nd the cost functional

Jeu) = 4000) +4 [ fexe) + wxe] at 1

Assume that u*(t) is an optimal control and that z*(t) is the corresponding optimal trajectory. Let

u(t) = w(t) + en(t) u(t) = z*(t) + eg(t) be perturbations on the control and the trajectory. (a) Show that

z(t) = —2*() — g2**@) + ut) — ol

+ 2"*Ol+ a) — Pez"

- =

(b) Identify the terms (af/ax) |,, (df/au) |,, and o(e) of Eq. (5-208) in (a). (c) Verify Eq. (5-215) for this particular system.

(d) Show that the Hamiltonian for this problem is given by

H(c, p, ts t) = fat + fut — 2p — datp + up (e) Show that the terms in Eq. (5-226) are given for this problem by

a oy 7 2°) — PO — 2*(¢)P()

Fe ep 7 UO + 70 o(¢) = 5 (1 — 2e*WpOlle® — s*OP + 5 lu — uO? — LPO — 20? = Fill — 22 OpOlesO + Fen — F pers)

272

Conditions for optimality

(f) Determine the o(«) terms of Eqs. (5-227), (5-228), and (5-230) for this problem. (g) Write Eq. (5-232) explicitly. (h) Show that p(z) must be a solution of the linear differential equation

p(t) = —x2*(t) + [1 + 2*%X(t)Ip(t) Deduce that p*(¢) must be the unique solution of this differential equation, satisfying the boundary condition

p*(1) = z*(1)

(¢) Show that [using Eq. (5-253)] u*(t) and p*(¢) must be related by

u*(t) = —p*(t) (j) Show that, for this problem, Eq. (5-263) becomes

raarnn wave ee fi (MORE AAOMO OHO a

(k) Show that z*(¢) and p*(t) satisfy the system of equations

; * a*() = —2*(t) — a*8(t) = — pel p*(s) = —z*(t) + p*(t) + 2**¢)p*@) with boundary conditions

z*(0)

= 2

p*(1l) = 2*(1)

(l) Suppose that @ is a control for which there is a corresponding # such that @, #, and 2 satisfy the necessary conditions and, in addition, satisfy the relation

2(t)0(t) $$

for t in (0, 1]

Show that 2 must be a local minimum of J. Exercise 6-15 Consider the system

Z(t) = —z(t) £3(t) = and the cost functional

—z(t)t? + u(t)

zi(0) = &

22(0) = &

Hu) = [lero + 20) + wo)

Write the following equations explicitly for this problem: (5-215), (5-225), (5-227), (5-228), (5-230), (5-236), (5-237), (5-263), (6-271), (5-273), (5-303), and (5-309). Exercise 5-16

(5-211),

(5-213),

(5-240) to (5-243),

Repeat Exercise 5-15 for the system

a(t) = —e*"Oz(t) + u(t)a(t) + u(t) and the cost functional

3-9

(5-208),

(5-239),

—-z(0) = ae

Ju) = [Pein + wn

The Variational Approach to Control Problem 3: A Fixed-end-point Problem

We are now going to examine a fixed-end-point problem from the variational point of view. We shall arrive at an impasse in trying to show that

Sec. 5-9

A fixed-end-point problem

273

an optimal control must extremize the Hamiltonian and shall thus have to reorient our thinking toward the control problem. Let us now state the problem. We consider the dynamical system

x(é) = f[x(), u(@), 4

(5-323)

where x(¢) and f are n vectors and u(é) is an m vector withO >

J (#4) J(a*) J (a5) J(@?) J(f*)

for for for for for

all all all all all

@! a? 0° G? G4

“near” + u? “near’ “‘near’ ‘‘near’’

u! ué u? u‘

(5-351)

are satisfied. The control wu? is globally optimal, and the controls u! and w® are called locally optimal. Let us now examine the behavior of the Hamiltonian H for the problem with the cost functional plotted in Fig. 5-10. Weshall first discuss the properties of H from an analytical point of view and then illustrate these properties with “symbolic plots.” The Hamiltonian H for our problem is given by

H(x, p, u, t) = L(x, u, t) + (p, f(, u, 4)

(5-352)

and the canonical system of vector differential equations is given by

£(t) = 5 ix(t), p(t), ud), f

(5-353)

a(t) = — 28 ). a(t). wld. 2

(5-354)

Sec. 6-10

Comments on the variational approach

279

x

*2

P2

(d)

(c)

with boundary conditions X(to) = Xo

(5-355)

We shall be particularly interested in the solutions of these equations for the controls u', u’, . . . , u® which correspond to the extrema of J(u) in Fig. 5-10. We suppose that u!, u*, . . . , u® are plotted in Fig. 5-l1la and that the corresponding responses x!, x’, ... , x® of our system (5-323), starting from the same initial state x(t) = xo, are plotted in Fig. 5-11b. Since

280

Conditions for optimality

0H/dp = f(x, u, ¢), the responses x!, x?, . . . , x* do not depend upon the costate.

In Fig. 5-11c, we plot the solutions p}, p*, . . . , p® of Eq. (5-354)

which correspond to the controls u!, u?, . . . , u®, respectively.

words, p‘ = pj, ,,; is the solution of the (linear) differential equation

pt) = — Sr, po, wl, 6

In other

(5-357)

satisfying the boundary condition p*(t,) = Oforz = 1,2, ... , 6. Since the controls u}, u2, . . . , u® correspond to extrema of J(u), we may deduce that

oF HO, PO, wild, d= 0 fort = 1, 2, ...,

6 and é € [t, t:).

Moreover,

(5-358) in view of the fact that

u!, u?, and u® are minima of J(u), that u? and u‘ are maxima of J(u), and

that u‘ is an inflection of J(u), we may infer that the m XK m matrix 0° /du? has the properties:

1. 3?H /du? is positive definite when evaluated at [x(), p(t), (2), 7

forj = 1, 3, 5

(5-359)

2. 0°H /du? is negative definite when evaluated at [x*(¢), p*(¢), u*(2), @

fork = 2,4

(5-360)

3. 07H /du? is neither positive nor negative definite when evaluated at

[x*(¢), p*(), u®(?), @]

(5-361)

We may also infer that the matrix

anndx? ‘dx| aHau |

eH

eT

du dx}

_

= Q(, p, u, 4)

(5-362)

dw?

has the following properties:

1. Q is positive definite when evaluated at [x(¢), p*(¢), w(2), a

for 7 = 1, 3, 5

(5-363)

2. Q is negative definite when evaluated at

[x¥(¢), p*(Z), u*(é), t]

fork = 2,4

(5-364)

3. Q is neither positive nor negative definite when evaluated at

[x*(¢), p*(¢), u(t), Uf

(5-365)

Sec. 5-10

Comments on the variational approach

These considerations

also allow us to suppose

following inequalities hold:

281

that, for ¢ € [to, é:], the

A[x'(t), p'@®, u'(), 4) < A[x'(@), p(t), us@), 4) «=

for a = 2, 3, 4, 5, 6

H[x*(t), p(t), u(t), 4) < A[x*(d), p*(é), ud),

for 8 = 1, 2, 4, 5, 6

A[x*(t), p°(2), u*(¢), 4] < H[x*(Z), p*@), wr), ) =

for y = 1, 2, 3, 4, 6 (5-368) for 6 = 1, 3, 4, 5, 6 (5-369)

H[x*(t), p*(d), u2(d), ) = A[x*(d), p*(é), wr), ] «=

(5-366)

(5-367)

H[{x‘(z), p(t), u‘(@), t] > Hix‘), p*@), ue), 4} ~—s fore = 1, 2, 3, 5, 6

(5-370)

In Fig. 5-12, we plot H[x!(t), p'(é), u(¢), t] versus u(¢) in order to illustrate the inequality (5-366). The w,u2 plane in Fig. 5-12 represents the space g A(at(t) ple), u(t), 2)

uy

Fig. 5-12 u(t).

Rm. and does In u(é)

The

Hamiltonian H[x'(t), p'(t), u(¢), ¢] has a minimum

at u(t) but not at

We observe that H(x!(t), p'(t), u(é), ] has a minimum at u(t) = w'(é) that, although u? is the global minimum of J(u), A[x1(2), p(t), u(d), é not have a minimum at u(t) = u'(Z). Fig. 5-13, we show that H[x‘(t), p(t), u(é), ¢] has a maximum at = u‘(t) (see the inequality (5-370)], and in Fig. 5-14, we illustrate the

fact that H{x*(t), p*(¢), u(é), ¢] has an inflection at u(t) = u(t).

We hope that these symbolic plots illustrate the point that the necessary conditions of Theorem 5-3 are local in nature (i.e., are satisfied by controls which locally minimize the cost as well as by the globally optimal control)

282

Conditions for optimality

and also the point that corresponding conditions could be derived for con-

trols which locally maximize

the cost.

Furthermore,

we hope that these

symbolic plots emphasize the fact that the necessary conditions do not provide us with enough information to determine the optimal control. For

4 M(x8e), pile), ult), t)

Fig. 5-13

The Hamiltonian H[x‘(t), p‘(t), u(t), ¢] has a maximum at u‘(é).

H(x8(t), p§(t), w(t),¢)

Fig. 5-14

The Hamiltonian H[x*t), p*(t), a(t), ¢) has an inflection at u(t).

example, the three controls u’, u’, and u‘ all satisfy the necessary conditions of Theorem 5-3, and so the fact that wu’ is globally optimal can only be deduced by computing J(u'), J(u*), and J(u°) and then comparing the computed values to find the smallest one.

Sec. 5-10

Comments on the variational approach

Let us make one final observation about Fig. 5-10.

283

If we suppose that u

is constrained to be an element of the closed set Q shown in Fig. 5-10, then we can observe that

J(u’) < J(u)

for all u in 2

However, wu’ is not an extremum of H; and the matrix

2

=

that is, =

(5-371)

[x7(t), p7(t), u7(d), ¢] ¥ 9,

[x7(t), p?(t), u7(Z), t] need not be positive definite.

Thus,

af we were seeking the minimum of J(u) over the set 2, our necessary conditions would not apply and the variational techniques that we have used so far would not be helpful. The reader should note that u’ is a boundary point of Q and should consider this situation in light of the results obtained in Secs. 5-2 and 5-3 and in light of the comments of Sec. 5-5. We have already noted that, for a fixed-end-point problem, difficulties arise in developing suitable necessary conditions on the basis of the variational techniques that we used for free-end-point problems. These difficulties stemmed primarily from the fact that we were not sure that there were “enough” variations, since our admissible variations were constrained. [See Eq. (5-335).] We are now going to indicate some further reasons for adopting a different point of view. First of all, we note that we required L and f to be “sufficiently” differentiable,

and

in particular,

ruled out such criteria as

we

required

that

dL/du

[O tu@| at

be defined.

This

(5-872)

which, as we shall see in Chap. 8, are quite important. So we seek an approach which does not place severe differentiability requirements on L

and f.

Secondly, we observed, when examining the behavior of J(u) over © in

Fig. 5-10, that our necessary conditions did not apply in the case of a closed constraint set.{ Thus, such constraints on the control as

lu(|

OLn+1

(1) #%(t) = > ix*(t), p*(t), u%(t), f = 2*(t) + u*(t} aH p(t) = — = zr [2%(t), pre), u(t), f = —2z*(t) — p*(t)

[x*(t), p*(t), u*(t),

2° s(t), Dealt]

m= — 220 (é) — p*(t)u*(t)

(2) min Hi[r*(t), p*(t), u, ta4i(t), Payi(t)]

|(2) min H[x*(t), p*(t), u, #]

= Hifz*(t), p*(t), w*(t), za4i(t)) Pagal]

= Hi[x*(t), p*(t), u*(t), ¢)

= 2*Xt) + 12 + p*(t)[z*(t) + u*(t)E)

= (x*Xt) + 202,(t) + p*(t)

X [2%(t) + uh(Q)ze CON} + payr(t)

(3) H,lz*(t), p*(t), u*(t), ta4i(t), Peyi(é)] = 0| (3) Al[z*(t), p*(t), w(t), e] that

is,

=

z*%(t) + 232,(t) + p*(t)[z*(t)

A at) = phat) |

—

f°

=

(x*(r),

p*(r),

ts

u*(r),

7] dr

,

7 Tf Or + pteyurtod de

Afz*(t:), p*(t:), u*(t1), i] = 0

(4) paii(ti) = 0 The reader should observe the effect of substituting z Leven

(5-518)

or, equivalently, the property that We then claim that

X(és) = x*(t)

(5-519)

£o(ts) 2 25 (ts)

(5-520)

If Eq. (5-520) were not satisfied, then we would have

#o(ts) < 29 (te)

(5-521)

If (see Fig. 5-20a) 7 is defined by the relation tj=t+r

(5-522)

then Eqs. (5-519) and (5-521) become, respectively,

R(t, + 1) = x*(te)

(5-523)

fo(te + 7) < x9 (te)

(5-524)

Now let us suppose that, at ts, we apply the control

Be erery = Teste cote)

(5-525)

as illustrated in Fig. 5-20a. In effect, fi is the same as the optimal control u* but translated in time; that is,

fifs+7r) =u*(s) The

control fi generates

a trajectory

8 € (és, t*] ¥, which

(5-526)

originates

(£o(te + 7), x*(t2)) and which meets the line S’ at t* +7.

at the point

In other words,

316

Conditions for optimality

if $(s + rt) = (40(8 + 7), ¥(8 + 7)) for s & (és, é*], then £(¢* +7)

However, we have

Fo(t* + 7) = Zol(te +7) +

= x*(é*) = x)

(5-527) |

** Die(s +7), f(s +7r)]d(s+r7)

(5-528)

= fut, + 7) + fg Lie*(e), w*(0)] 4

(5-529)

= 2o(te +7) + xo(t*) — x9 (ts)

(5-530)

= £4(ts) + 2% (t") — 2% (te)

(5-531)

In view of Eq. (5-521), we conclude that £o(t* +1) < 25 (E*)

(5-532)

which contradicts the optimality of u*. Thus, our claim (5-520) is established, and so we have shown that optimality of u* is equivalent to the geometric property that all the y trajectories which meet S’ must lie ‘“‘above”’ the trajectory generated by u*. (See Fig. 5-19.)

This geometric characterization of optimality will help us to construct a

“tube” about the optimal trajectory in which the control u* is approximately optimal. Consideration of this “tube” will enable us to overcome the difficulty associated

with

the requirement

tories must pass through x, (see Sec. 5-9). 3

that our perturbed

trajec-

A Small Change in the Initial Condition and Its Consequences

With a view toward constructing this ‘‘tube” in which u* is approximately optimal, let us now make a very small change in our initial point (0, xo) and see how this change “propagates” along the optimal trajectory. In other words, we assume that our new initial point (in Ra4:) 1s (0,

Xo)

+

e(vo, v)

where ¢ is a small positive number such that

and where,

lim Hl

—0

+ o(e)

€

(5-533)

as usual,

_ 9

o(e) is a vector

(5-534)

We observe that the solution of (5-507), starting from (5-533), generated by the control u* is of the form

(x9 (t), x*(t)) + e(xo(d), x(t) + of€)

(5-535)

where (x0(¢), x(é)) is the solution of the (nm + 1)st-order linear system

iol) = (% er, wr), xO)

x) = (Zt, wl) x0

(5-536)

(5-537)

Sec. 6-16

Proof of the minimum

principle

317

satisfying the initial conditions Xo(to)

=

(5-538) (5-539)

Vo

x(to) = v We also note that the system of Eqs. in the matrix form

st

9 9L| | OF

_

x(t)

Me

(5-536) and (5-537) may be written

9£)

's

OXe

.., ab

OL n

Oxof

0 !

*

allen

(5-540)

where the |, indicates that the function in question is to be evaluated along

the optimal trajectory. If we let Ax(t) denote the n + 1 X n + 1 matrix appearing in Eq. (5-540), so that ( @L| 0:

aL

aL —

:

Ox

|

if we let &(¢) denote the n + 1 vector (xo(t), x(é)), and if we let — denote the » + 1 vector (vo, v), then ¢(t) Cost is the solution of the (n + 1)st-order A %o s’ homogeneous linear system

Ut) = Ax(t)y()

(5-542)

-7 | /vie")

satisfying the initial condition

U(to) = — = "|

(5-543)

This means that our new trajectory in

Rati

is of

the

form

y*(t) + eft) + of)

(5-544)

We further note that if we view the vector — as being “attached”

to our

initial point yo = (0, xo), then, as is

(0, %)

(O, ¥,)

State space

Fig. 5-2] ‘Ihe result of small perturba~tions on the initial conditions. The effect of a small perturbation eE can be approximately described by the propagation of a vector «4(t) attached to the optimal trajectory y*(t). The small perturbations are exaggerated in this figure for the sake of clarity. The vector ¢(¢) is the solution of the linear system (5-542), with the initial condition given by Eq. (5-543).

illustrated in Fig. 5-21, we may view the vector y(é) as being ‘“‘attached”’ to the point y*(t). Thus, $(¢) can be considered the result of moving (or transferring) the vector & along the optimal trajectory. We also note that the o(e) vector in (5-544) depends upon ¢, so that we should write o(e, 4). However, because of our smoothness

318

Conditions for optimality

assumptions, we can assert that

lim 0

sup

{Hote Ou

tte, ti]

= 0

€

(5-545)

which means that the ¢ dependence of o(e, t) can be neglected. The geometric interpretation of this remark appears in Fig. 5-22. As we can see in the figure, the trajectory y*(¢) + e¢(t) + o(e) always les in a narrow band

(bounded

y*(t) + eg(t).

by the “envelope”

The

interpretation

curves

of our

in Fig.

remark

5-22)

is,

about

simply,

the curve

that

the

Cost

h 25

y'\t)te¢(t)

y' (th+ev(t) + ofe)

—

x

State spoce

Fig. 5-22 The “width’’ of the envelope does not change appreciably with time, and it is approximately e’.

“width” of this band does not change very much the order of ¢ along the entire trajectory.

Letting ®,(f, 4.) denote Eq. (5-542) we see that

and that

the

n+1%X2n-+1

d(t) = Delt, tok

Y(i*) = @, (t*, lo)E

with time; i.e., it is of

fundamental

matrix

of

(5-546)

(5-547)

Thus, we may view the system (5-542) as defining a linear transformation of the space of n + 1 vectors “attached” to yo into the space of n + 1 vectors “attached” to y*(¢). Moreover, since the matrix ®¢(t, fo) 18 nonsingular, this linear transformation is nonsingular. Now, in view of the fact that any portion of our optimal trajectory is an

optimal trajectory, we can see that if ¢,; and ¢. are elements of [t, ¢*], then

we can define, in an entirely similar way, a linear transformation of the

space of n + 1 vectors “attached” to y*(t,) into the space of n + 1 vectors

“attached” to y*(t2).

In fact, if &, is an n + 1 vector “attached” to y*(¢1),

Sec. 5-16

Proof of the minimum principle

319

then its image under this linear transformation is precisely the vector

@o (bo, t)Ei

(5-548)

Thus, the homogeneous linear system (5-542) may be viewed as describing the future effect of a small change in the optimal trajectory at a given time. We shall view the small positive number ¢ as a parameter in the sequel. In other words, we shall, when dealing with specific quantities, assume that e« has been chosen small enough to validate our arguments for those specific quantities. For example, if & and — are n + 1 vectors “attached” Cost

s(24

A 40

y(e)eep(t)

fag (t") + ele")

me)

p75 ( 0")

s”

- ye)

poy *(t*)tey(t*) ry *(t") en

'

_ a

hy

‘State

space

Fig. 5-23 The trajectory $(¢) generated by the admissible control u is “approximately above’’ the trajectory y*(t) + «¢(t) because the control u* is optimal to within first order in ¢ in a tube about y*(t).

to yo, then we (tacitly) assume that ¢ is sufficiently small for both y*(¢) +

et(t) and y*(t) + e$(¢) to be “elose” to y*(t), where ¥(¢) and §(¢) are solu-

tions of (5-542) satisfying the initial conditions y(t) = — and (to) = £, respectively.

Since small changes in initial conditions cause correspondingly small changes in the solutions of a differential equation, we can assert that all the trajectories of the system (5-507) which start from the initial point (5-533) and which meet S’ to within the first order in ¢ will lie “approximately above’ the trajectory generated by u*. In other words, if @ is an admissible control which generates a trajectory starting from the point (5-533) with the property that, at some time ?,

where

then

£(f)

=

xX; +

ef;

9) = [£0(t), £(d)]

£o(2) — ap (t*) — ebo(t*) = —|o(6)|

(5-549)

(5-550)

320

Conditions for optimality

Thus, to within higher-order terms in ¢, the control u* is optimal from the

new initial point (5-533). In fact, as is illustrated in Fig. 5-23, the control u* is optimal (to within the first order in ¢«) in a “tube” about the optimal trajectory. The perturbations u of u* which we shall consider will not take us out of this “‘tube,’”’ so that we shall be able to suppose that the cost J(u) is approximately bigger than the cost J(u*) for any point in the “tube.” We note that this “tube” is, of course, a geometric fiction which we simply use as an aid to our intuition. 4

Moving Hyperplanes Now let us examine the adjoint system of the homogeneous linear system

4(t) = Ax(t)¢(t) [see Eq. (5-542)] in an effort to see if it has any useful

geometric interpretation. system.)

(See Sec. 3-25 for the definition of the adjoint

Since the matrix —A,() is given by the relation -AL()

=}

aL | a =

Ox 0

_ (2

|e |

_ a

)

OX

— fi)

021

O21

OZ

a

&

- al OLn

le

_

_ fa)

\s

_|—9f);_ a1)

7

(5-551)

OXn

s

|e

O23

*

_ 9)

mal

_ 9h)

O21

2,

_ OSs

11.

_ obs

_ fa], OLn

+

=

O21

|e

Ore

s

Sn

Of

x

(5-552)

eo)

we can immediately see that the adjoint system of (5-542) is of the form

(s8)-[=, Cay te]

on

po(t) = 0 po - - Z| mo - (Z|) 2@

(5-554) (5-555)

0:

oO

or, equivalently, of the form

We may immediately deduce from Eq. that is, that po(t) = po for all ¢.

(5-554) that po(t) is a constant,

On the other hand, if we recall that the

Hamiltonian H(x, p, u, po) is defined by

H(x, p, U, po) = pol.(x, u) + (p, f(x, u))

(5-556)

Sec. 6-16

Proof of the minimum principle

321

then we can see that Eq. (5-555) may be written in the form pf)

=

oH

—

Ox

[x*(é),

p(t),

u*(), Po}

(5-557)

We also observe that if po is a constant and p(t) is a solution of Eq. (5-557), then the scalar product [see Eq. (3-434)]

(| 2

uo)

= const

(5-558)

for any solution &(¢t) of Eq. (5-542).

Now, if we let Po be a hyperplane passing through yo with equation

(| "| y- yo) = 0 and if we suppose that yo + e€ is an n +1 (5-553) satisfying the initial condition Pollo) = mo

(5-559) t

vector, then the solution of

P(bo) = =

(5-560)

defines a hyperplane P; with equation /

70 |

>

_

\_

-

which passes through y*(é) and has the same scalar product with the vector

u(t) “attached” hyperplane Po.

to y*(t) as the vector & “attached” to yo makes with the In particular, if Po is perpendicular to &, then P; is per-

pendicular to ¥(¢). The hyperplane P, can be viewed as the result of moving (or transferring) the hyperplane P») along the optimal trajectory, as is illustrated in Fig. 5-24. We observe, in passing, that if r» = 0, then the equation of P, is of the form

(p(t), x — x*(t)) = 0

(5-562)

and, consequently, is independent of the ‘‘cost” coordinate in R»+1, in the

sense that P, is parallel to the zo axis.

This observation will provide a clue

to the significance of the pathological situation pj = 0. If we compare Eqs. (5-557) and (5-489), we can see that the costate or adjoint (sic!) variables will correspond to the motion of a particular hyper-

plane along the optimal trajectory. Thus, the crucial question becomes: How can we determine the particular hyperplane that will enable us to establish

the theorem? Well, let us begin to answer this question by asking another: What is a general way of determining hyperplanes passing through a particular t Observe that |

wr z

| is normal to the hyperplane Py.

322

Conditions for optimality

point y? If we suppose that y is a boundary point of a convex set C, then we know that there is a support hyperplane of C passing through y (see Theorem 3-5). So what we propose to do now is to introduce a suitable convex set Cf which has y*(‘*) = (J*, x;) as a boundary point and which has a support hyperplane that will be used to prove the theorem. In fact, we shall determine convex cones C; with vertices at y*(¢) by considering a special class of variations or perturbations of the control u*. These cones will always lie on the same side of the hyperplanes determined by the costate variables and will be called cones of attainability. In a sense, the rays of the cone C' will correspond to the perturbations of u* that we shall consider. Cost

Xo

x;

vi)

\

(7%, pit)

Qa

Fig. 5-24 ILustration of the moving hyperplane P,. The vector (xo, x) is perpendicular to the hyperplane P, at y*(t). The vector & (or e€) represents a perturbation on the initial condition y*(ts). At the time ¢t, the vector (xo, p(¢)) is perpendicular to the hyperplane P, at y*(t). The coordinate system is illustrated at the top of the figure.

So, in the next two subsections, we turn our attention to the description of the class of variations of u* with which we shall work. 3

Temporal Variation of the Optimal Control

The first basic type of variation, which we shall call a consists in merely changing the terminal time {* by a examining the result of applying the optimal control u* interval. To be more precise, if 7 is an arbitrary real temporal variation u[r] of u* is defined as

ulr](t) =

temporal variation, small amount and over the new time number, then the

u*(t)

t 7,

We also

, (2, )C, C C1

(5-610)f

©, (¢*, OC, C Cr

(5-611)

and, in particular, that

t Usually, these relations will be equalities; however, only the indicated inclusion relationship is needed in the proof of the minimum principle.

334

Conditions for optimality

It follows from Eq. (5-611) and from the fact that @, is the fundamental matrix of a linear system [Eq. (5-542)] that

and

that

y(t, t*) Oe (t*, OC: C Delt, t*) Ce C;

C

PM, (t, t*) Cy.

(5-612)

(5-613)

This relation (5-613) can be used to show that the cones of attainability will always lie on the same side of the moving hyperplane determined by the costate variables. 8

Proof of Theorem 5-5P

We are now ready to “prove” Theorem 5-5P. We begin by asserting that the ray in the direction of decreasing cost emanating from y*(¢*) does not meet the interior (see Definition 3-8) of the cone C,. More precisely, we claim that if y is the n + 1 vector given by

P="0 0 .

w=

_

(5-614)

OL

then the ray is, given by

= {y:y = y*(t*) + ey, 6 > 0) does not meet the interior of the cone C,. Fig. 5-34.

The situation is illustrated in :

We shall verify this claim about u by an indirect argument. suppose that the vector

f=y*t) +dy

(5-615)

B>0

So let us

(5-616)

is an interior point (see Definition 3-4) of Cy. This implies that there is a perturbation @ of u* which generates a trajectory 7(t) of (5-507) over the interval [¢), 7] with the property that

$(2) = y*(t*) + By + ofc) Now, recalling that

(5-617)

0-[8] re-[]

X1

ea

Sec. 5-16

Proof of the minimum

principle

335

and that y is given by Eq. (5-614), we can see that Eq. (5-617) may be written in the form

J(@a)=J*-B+oe)

x2) =x, 4+ ofc)

(5-619)

However, for suitably small «, the control @ will not take us out of our “optimal” tube, and so the control u* is optimal to within the first order Cost

+ Pe

*o

aT

af

Po

pit*)

Fig. 5-34

The hyperplane

P,- shown

is a typical support hyperplane to the convex

cone C,», and it separates C,- and the ray

hyperplane P;» contains the ray ». y*(¢*).

Be

Since the ray > is an element of Cs, the *

The vector [ ates) | is normal to PY at the point

The convex cone Cs is on one side (P,+*) of Py», and the ray ip is on the other

side (Pw-).

The ray , points in the direction of decreasing cost.

ine. On the other hand, according to Eq. (5-619), the control @ leads to a reduction of order ¢« in the cost and to a higher-order (in «) change in the termi-

nal point of the system trajectory in R,.

obtain a contradiction in that the tory y*(¢) (to within the terms of ¥ cannot be an interior point of interior of Cz. Since the cone C;+ is convex, Rati.

Thus,

if ¢ is very small, we shall

trajectory $(¢) will lie “below” the trajechigher order).t It follows that the vector C,. Thus, the ray a does not meet the

its interior 7(C;+) is an open convex set in

As the ray a” is a.convex set which does not meet the interior 7(C;-)

t The detailed proof of this key point appears in Ref. P-5 and involves some complicated

topological arguments. See, in particular, lemmas 3 and 4 on pp. 94-99 of Ref. P-5. We also note that the relation (5-613) plays a role here.

336

Conditions for optimality

of Cy, we may deduce from Theorem 3-5 that there is a support hyperplane P;» of C,- which separates C;. and the ray u.t The hyperplane P%., which is illustrated in Fig. 5-34, passes through the point y*(*) and has the property that the ray L lies in the half-space P;-— while the cone C;° is contained in the half-space P,«+ (see Sec. 3-7). In other words, there is a nonzero n + 1 vector which we denote by Re

is defined by the equation

| such that the hyperplane P;-

“wl

(Letim }y 77) =°

020)

(Lpfin} 9) 2°

6922)

(ei

ew

and, moreover, the following conditions are satisfied:

where y is (the direction) given by Eq. (5-614) and 8 is any element of the cone Cy.

We

observe that the veetor|

*(Pe

|i is normal

to the hyper-

plane P;, at y*(é*) and that Eq. (5-621) implies that

po. = 0

(5-623)t

since the vector yp is given by Eq. (5-614).

We are now going to determine the costate variables on the basis of our discussion of moving hyperplanes in subsection 4. We recall that the

system (5-542) is linear and homogeneous and is of the form

(5-624)

Ut) = An (OU where A, (i) is then + 1 X nm + 1 matrix given by [see Eq. (5-541)]

A, (1) =

pf OL)

A Ott

o

9L|

OF

|... ab

(al)

(5-625)

+ According to Theorem 3-5, there is at least one such hyperplane. There may, in fact, be many, and our arguments will apply equally well to any of them. Strictly speaking, this argument is valid only if the interior of C,+ is nonempty. If, however, the interior of

Cis is empty, then the desired result can be obtained by considering the rays y (a) = {y:y = y*(é*) + By, 8 > a}, a > 0, which do not meet C,* and applying a limiting argument or by using the notions of the footnote on pp. 93 and 94 of Ref. P-5. t We

note that if p> = 0, then the hyperplane P,- will contain the ray y and

problem will be, in a sense, independent of the cost.

—

our

Sec. 5-16

Proof of the minimum principle

337

This system describes the motion of a vector along the optimal trajectory. We recall also that the system (5-553) which is the adjoint system of (5-624) is also linear and homogeneous and is of the form Bott)

Ke

| ~

ee

Pol(t)

TAO Ke

|

(5-628)

We have noted in subsection 4 that this system represents the motion of a hyperplane along the optimal trajectory, and we shall use this fact to determine the desired costate. For convenience, we let @,(t, to) denote (as before) the fundamental matrix of the system (5-624), and we let Ws (é, éo)

denote the fundamental matrix of the adjoint system (5-626). We now construct the costate by finding the hyperplane whose motion along the optimal trajectory leads to Py. Since the matrix W,(t*, to) is nonsingular,

we know that there is a nonzero n + 1 vector I

eovofil-[]

If we define an (n + 1)-vector-valued function

a | = W(t, to) "|

ra

such that

| by setting

t E [to, t*]

om

(5-628)

then we can immediately conclude from Eq. (5-554) that

pot) = po =o

for allt

and from Eq. (5-555) that

pro = - 2ax |, ot -(2})ax

|.

vw

(5-629)

(5-630)

Since the Hamiltonian H(x, p, u, po) is given by

H(x, p, u, po) = pol (x, u) + ( [ LixO, wixt, 4, d at

(5-704)

> 0 = S(t, be, ¥)

(5-708)

(5-708)

and the lemma is established. We observe that the control fi of the lemma may not be optimal since there may be controls u? which transfer (Xo, ¢o) to S along trajectories which do not lie entirely in X, as is illustrated in Fig. 5-38. Now, the function D(z, u, t) of our problem need not have the properties of the lemma; however, we shall, under certain assumptions about the

Hamiltonian, be able to replace L(x, u, ¢) with a function L(x, u, t) which

does satisfy the conditions of the lemma and thus obtain the desired sufft Note that the time #, at which (x(t), t) meets S need not be the same as 41.

Sec. 5-20

A sufficient condition

ciency conditions.

First, we shall need some definitions.

we have:

Definition 5-12

357

In particular,

Let H(x, p, u, t) be the Hamiltonian of our problem, so that

H(x, p, u, t) = L(x, u, f) + (p, £G, u, 4)

(5-707)

If, for each point (x, t) in X, the function H(x, p, w, t) has, as a function of o, a unique absolute minimum with respect to all w in 2 al w = fi(x, p, £), then

we shall say that H 1s normal relative to X or that our control problem is normal relative to X.

In that case, we shall call the function &i(x, p, t) the H-minimal

control relative to X.

(x72), ¢)

(19,0)

Ss Fig. 5-38 The trajectory (x*(é), ¢) does not lie entirely in X but does meet S and may be less ‘‘costly’”’ than the trajectory (%(¢), ).

In other words, ii(x, p, t) 1s the H-minimal control relative to X tf, for each

(x, t) in X,

H{x, P, fi(z,

Pp, t),

t]
bilx*(t), pf (t)} does not define the time-optimal cont=]

trol as a function of x*(¢) and p*(¢). We against misunderstanding this statement; time-optimal control does not exist or the defined; it simply states that the necessary < H[{x*(t), p*(), u(t), ¢] does not lead to

u*(t), x*(1), p*(), and t.

would like to caution the reader it does not mean that either the time-optimal control cannot be condition H[x*(¢), p*(t), u*(é), ¢] a well-defined relation between

Singularity interval

uj(t) =?

L —-————

7

t

at Fig. 6-5

|

—

(

i

(

Vo, It, y,

Lanny ut(t)

+f Tp

_---,3

te

H——

f

*(2)~

The function g;(t) shown corresponds to a singular time-optimal problem.

We shall encounter singular problems throughout this chapter. In the remainder of this section, we shall assume that we are dealing with a normal time-optimal problem. We shall thus be able to substitute Eq. (6-75) into Eqs. (6-52) to (6-54), (6-56) to (6-58), (6-60), and (6-61) and, in so doing, eliminate u*(t) from all the necessary conditions. Thus, all the necessary conditions stated in terms of u*(¢) in step 1 will reduce to necessary conditions which

are independent

of u*(t).

As we shall see

in step 3, this fact will enable us to determine the time-optimal control. We shall now state two theorems which summarize these ideas.

Theorem 6-2

The Bang-Bang Principle

Let u*(t) be a time-optimal con-

trol for Problem 6-1b, and let x*(t) and p*(t) be the corresponding state trajec-

tory and costate. If the problem is normal (see Definition 6-4), then the components uz (t), us (t), . . . , ur(é) of u*(t) must be defined by the relation

ur) = —sgn{ ¥ bslx*(, dptO} = 7=1,2,...,7 (679) t=1

for ¢ € (to, T*).¢

Equation (6-79) can also be written in the more compact

t Except at the countable set of switch times ty;.

Sec. 6-3

Application of the minimum principle

form

u*(t) = — SGN {q*(é)}

— SGN {B’[x*(®), tp*®}

383

(6-80)f

Thus, uf the problem 1s normal, the components of the tume-optimal control are

piecewise constant functions of time (t.e., are bang-bang). We shall leave it to the reader to verify the following theorem by direct substitution. Theorem 6-3 Reduced Necessary Conditions Let u*(t) be a timeoptimal control for Problem 6-1b. Let x*(t) be the state on the time-optimal trajectory, and let p*(i) be the corresponding costate. Let T* be the minimum time. If the problem is normal (see Definition 6-4), then it 18 necessary that: a. Theorem 6-2 be sattsfied b. The state x*(t) and costate p*(t) satisfy the reduced canonical equations

at(.) = filx*(, 1 — Y bdx*@, dean |) dulx*@, Apt} j=l

s=1

;

81)

p(t) = — y oflx*(d), 4 p(t) s=1

dx} (t)

+ (y ogn (y buix*(i), dpt(o}) » Pale pt) (6-82)

fork =1,2,...,nand fort E [to, T*). c. The Hamiltonian (6-52) along the time-optimal the equation

trajectory 18 given by

Hix*(), p*®), w*), d= 1+ ¥ fe*O, At t=]

y — 5=1

y bslx*(t), Hpi) |

i=l

¢ E (to, T*]

(6-83)

d. At the terminal time T*, the following relation [see Eq. (6-61)] holds:

+) flx*(P*), T*\pi(T*) — yId bulx*(P*), T*Ip?(T*) _ ye dgelt*(T*), Tg gy aw]

oT*

+ The vector-valued function SGN { _ } is defined as follows: Let ai, as, .. . , ar be the components of the vector a, and let b:, bs, . . . , 6, be the components of the vector b then a = SGN [b!} means that a; = sgn {b;} g=l1,2,...,7

384

Structure and properties of optimal systems

e. At the initial time to,

}

X*(to) = x(to)

(6-85)

At the terminal time. T*, the following relations hold [see Eqs. to (6-66):

glx*(T*), T*7]}=0 p

“(7*) = (T ) ~~

a=1,2,...,n—B8;821

n—B

2

ka

(6-64)

(6-86)

dgalx*(T*), T*] ax*(T*)

(6-87)

We shall now illustrate the implications of Theorem 6-3. Let us suppose that n = 3 and that r = 2. As illustrated in Fig. 6-6, the 2 X 3 matrix B’{x*(¢), ¢] is associated with a transformation that maps the 3-vector p*(é) into the 2-vector q*(#) = B’[x*(¢), ‘Jjp*(t). It is easy to see that, in order

p*( ty )

pt

~~~" Fig.

6-6

Geometric

product (u(é), q*(é)).

interpretation

of the fact that u*(é)

must

if

te)

|

minimize

the scalar

to minimize the scalar product (u*(é), q*(t)), the control vector u*(¢) must attain its maximum magnitude and point in a direction which is roughly opposite to that of the q*(¢) vector. Thus, whenever q*(¢) is in the first quadrant, then u*(f) must point to the corner A of the constraint square; whenever q*(t) is in the second quadrant, u*(t) must point to the corner B of the square, and so on. The reduced necessary conditions provided by Theorems 6-2 and 6-3 lead to a systematic way of deriving the time-optimal control. This exposition will be carried out in step 3. For the time being, we wish to remind the reader that Theorems 6-2 and 6-3 refer to the optimal quantities and represent necessary conditions. Thus, given a control u(t) and the corresponding trajectory x(t), if u(¢) violates any one of the necessary conditions, then we are immediately assured that u(t) is not a time-optimal

control. The following example illustrates the results of Theorems for a specific svstem.

6-2 and 6-3

Sec. 6-8

Application of the minimum principle

Example 6-3

385

Consider the dynamical system with state variables z1(t), a(t), za(t) and

control variables u(t) and u(t), described by the equations Z(t) = —a2,(t) + 21(t)ur(t)

Bet) = 2i(t) — tee%t) + ualt) Z(t) =

—t*7,%t) — 2,(t)

Suppose that the control variables u:(?) and «w.(¢) are constrained in magnitude by the

inequalities

[ui(é)] 1

Juc(t)| S 1

Suppose that at the initial time ¢, = 0, the system is at the initial state 2i(0) =

—1

x3(0) = —1

z3(0) = 10

Let us suppose that the target set S is described by the equations gi(Z1, Za, Ls, t) = 23 = O

gx(%1, Ya, 23, t) = 212? + 2? — t?} —1

= 0

In other words, the target set is the boundary of an expanding circle in the z:z2 plane, whose radius is +/{? + 1.

The problem is to find the time-optimal control from the initial state (—1, —1, 10) to the target set. We confess that we do not know whether the time-optimal control exists and, if it exists, whether it is unique. Let us assume that it exists, that it is unique, and that the problem is normal, so that we can illustrate the necessary conditions. The Hamiltonian for this problem is H

=

H{zi(t),

zo(t),

z(t),

pitt),

p(t),

p(t),

ui(t),

ua(t), t|

m= 1 — xo(t)pi(t) + xr(t)ur(t)pilt) + xi(t)pa(t) — tre%t)po(t) + uslt)pa(t) — t92,%(t)ps(t) — xs(t)pa(t)

The reduced necessary conditions are determined as follows: From Eq. (6-79), we have

ur(t) = — sgn {2;(t)p;(t)} u3(t) = — sgn {p3(t)}

From Eq. (6-81), we have

ii(t) = —23(t) — z:(t) agm {xr(t)py(E)} i3(t) = xi(t) — t3°(t) — sgn {p3(t)] ts(t) = —t*xy%(t) — x3 (2) From Eq. (6-82), we have

Bi(t) = +pr(t) sgn {x7 (t)pr(t)} — p(t) + 3¢2x37(t)p3(t) B3(t) ws tpi (t) + 2tr3(t)p3(t) Bs(t) = +p;(t) From Eq. (6-84), we find that, at the minimum time 7",

1 — 23(T*)pi(T*) — |zi(T*)p3(T*)| + 21(T*)p3(T*) — T*227(T*)p3(T*) — |p3(T*)| — T*2f(T*)p3(T*) — 23(T*)p3(T*) = €:0 + e227 where es is some constant. At the initial time t) = 0, we must have

zi(0) = —-1

23(02)=—1

# 23(0) = 10

386

Structure and properties of optimal systems

At the minimum time T'*, we must have

23(T*) = 0

2° T*) + 23 T*) — Tt -—1 = 0

and {see Eq. (6-87)]

pi(T*)

= k,0 + k22x7(T*)

pi(T*) = ki0 + k223(T'*)

pi(T*) = kil + k:0

where k; and k; are some constants.

Step 3 In steps 1 and 2, we stated relations which must be satisfied by the time-

optimal control u*(t), the resulting state x*(t), the corresponding

costate

p*(t), and the minimum time 7J*. Our problem is, however, to find the time-optimal control, and so the question that arises is the following: How can we use all these theorems to find the time-optimal control for Problem 6-16? We shall answer this question in the remainder of this section. We shall develop a systematic procedure and label each step so that the reader can

follow the logical development. Step 3a

Formation of the Hamiltonian

We start by forming the Hamiltonian function H{x(é), p(t), u(t), 4 for the system £(¢) = f[x(¢), ¢] + B[x(t), ¢ju(t) and the cost functional J(u) = [’ 1 dt.

The Hamiltonian is [see Eq. (6-45) or (6-46)] given by

Hix(), p@), wu, 4 = 1 + Gx@, 4, p®) +

, Bix, Ap)

(6-88)

We write x(t), p(é), u(t) to indicate that eventually these vectors will be functions of time. However, at this step, we do not imply any restrictions

either on the values of the vectors x(t), p(t), and u(é) or on the value of time 2. :

Step 3b The

Minimization of the Hamiltonian Hamiltonian

function

H[x(t), p(t), u(t), ¢] depends

on 2n +7

+ 1

variables. Let us suppose that we hold x(t), p(t), and ¢ constant and that we examine the behavior of the Hamiltonian [which is now a function of u(t) only, since x(t), p(é), and ¢ are held constant] as u(t) varies over the

constraint set 2.

In particular, we wish to find the control which will

absolutely minimize the Hamiltonian. an H-minimal control as follows:

Definition 6-6 H-minimal Control H-minimal tf it satisfies the relation

For this reason,

we shall define

The admissible control u°(t) 2s called

A[x(t), p@), urd), 4] < Alx@®, pd), u@), f] for all u(t) € Q, all x(t), all p(t), and all t.

(6-89)

Sec. 6-3

Application of the minimum principle

387

If we mimic the development presented in step 2, then we find that the H-minimal control u°(t) for the Hamiltonian function of Eq. (6-88) is given by

u(t) = — sen { S dulx, dp} = fF =1,2,..-,7 i=]

(6-90)

or, more compactly, in vector form by

u(t) = — SGN {B'[x(), dp}

(6-91)

If we now substitute the H-minimal control u°(é) into Eq. (6-88), we find that

H{[x(t), p(t), u(t), 4] = 1 + x, 4, pld)) — (SGN {B’[x(¢), tp()}, B’[x@), dp)

and hence that

(6-92)

Hlx(t), p(t), w(t), 1 = 1+ Y fle, Ape) — Y | Ye dle, Apild| t=]

j=1

t=1

(6-93) Note that the right-hand side of Eq. (6-93) is only a function of x(t), p(t),

and ?t.

For this reason, we define the function H°[x(t), p(¢), t] by the relation

H°[x(t), p(t), t] = Once

more

we remind

min A[x(é), p(é), ud), @]

the reader that these definitions and

not explicitly involve trajectories or optimal quantities. Siep 3c

(6-94)

u(jea

equations do

Restricting x(t) and p(?)

Let us now demand that the (as yet unspecified) vectors x(é) and p(é) satisfy the differential equations

x(t) = dH°[x(#), spit) pd), 4 a(t) = —

ie

(6-95) t}

(6-96)

or, equivalently, the differential equations

#x(0) = filx@®, 1— Y bule(o, deen { ) dalx(, dpa(d)} j=l

ut = — i=YEON ay r

n

+ [> sgn » j=l

t=l1

(6-97)

i=1

eon bs[x(Z), tip.(e) } | » {Puzo

lp s(t)

(6-98)

388

Structure and properties of optimal systems

for k = 1, 2,...,7.

Wenote that

dH°[x(t), p(é), t)} — dH[x(t), p(t), u(Z), @] op(t)

7

dp(t)

u(t) = 10°(¢)

(6-99)

u(t) =n9(t)

(6-100)

and that

dH°[x(¢), p(t), 4] _ oH[x(2), pd), ud), f ox(t)

Step 3d

7

ox(t)

An “Experiment”

Our objective is to find the time-optimal control u*(t) that transfers the system x(t) = f[x(t), é] + Bix(é), ¢]u(t) from the given initial state

X(t) to the target Definition 6-4).

set S.

We

assume

that

the

problem

is normal

(see

Let us now suppose that we simulate Eqs. (6-97) and (6-98) on an analog computer. At the known initial time f, we use the known initial values

of the

state variables

the system variables

(6-97).

Pillo),

po(to),

x,(to),

We

xe(to),

. . - , Zn(to)

use some

ee

ey

as initial conditions

guessed values

for

of the initial costate

Dulto).

Let q;(t),7 = 1, 2, . . . , r, be the functions defined by the relations

q(t) = > dilx(d), tlps(d) Let us assume that

qj(to) ¥ 0 Now

Eqs.

(6-101)

i=l

for allj = 1,2,...,7

(6-102)

(6-102), (6-101), and (6-90) imply that the numbers uj°(to) = — sen {q,(to)}

are either +l or —1.

Thus, the solutions of Eqs. (6-97) and (6-98) are well

defined, at least for ¢ near tp.

We shall denote the solutions of Eqs. (6-97)

and (6-98) by

x(t)

=

x(t, to,

X(to),

P(to)]

(6-103)

p(t) = plt, to, x(to), p(to)]

to emphasize

their dependence

guessed initial costate p(to).

on

the known

initial state x(t.)

and

the

The “experiment” proceeds as follows: We measure the signals x(¢) and p(t), and at each instant of time, we also form and measure the signals

a(t) = 3 dedx(), dp) i), GO, GO, --. t=]

f=1,2,...,7 §=1,%-- 247

Hex), pO), 1=1+ ¥ fixO, td) — Ys la! *>1

j=l

(6104) (6-105)

(106)

Sec. 6-3

Application of the minimum principle

gelx(t),) Mele,1

w=1,2,...,n—8 = 1,2,...,0-8

hix(t), 1 = 29824 ox(t)

,i1e....n—B

389

(6-107) (6-108) (6109)

Using the particular (guessed value) p(é), we ask the following questions sequentially for each ¢ in some interval {to, 7}. Question 1 If g;(¢) = 0, is ¢;(t) ~ 0? If no, is g(t) #0? If no, is q;(t) = 0? (And so on.) If the answer to question 1 is yes, then we ask question 2. If the answer is no, then we change the value of p(to) and repeat question 1.

Question 2 the relations

If the answer to question 1 is yes, is there a time 7 such that

(6-110)

galx(T'), T] = 0

are satisfied for all a = 1, 2,...,n—

6?

is no, we change p(to) and start all over again. is yes, we ask question 3.

Question 3

If the answer to question 2 If the answer to question 2

If the answer to question 2 is yes, are there constants ¢1,

€2, . . . » ng Such that the relation

n—8

dg.(x(T), Helx(T), p(T), = J eg Bett Th a=l1

is satisfied?

all over again. Question 4

(6-111)

If the answer to question 3 is no, we change p(t)) and start

If the answer to question 3 is yes, we ask question 4. If the answer to question 3 is yes, are there constants k1,

ke, . . . , kang such that the relation

p(T) = ye

dgalx(T), T]

Ty

(6-112)

is satisfied? If the answer to question 4 is no, we change p(to) and start all over again. If the answer to question 4 is yes, this implies that we have found a p(t.) such that the answer to questions 1 to 4 is yes; we store this p(t.) and start the experiment all over again—until we have found all the

vectors p(ép) with the property that the answers to questions 1 to 4 are all yes. This logical sequence of questions is illustrated in Fig. 6-7.

Step 3e We

The Candidates for Time-optimal Control shall now formalize the results of the ‘‘experiment’’

conducted in

step 3d. Loosely speaking, we define the set @» to be the set of initial costate vectors (to) corresponding to the given initial state x(t), with the property that the answer to questions 1 to 4 is yes. Clearly, @> is a subset

390

Structure and properties of optimal systems

of the n-dimensional space R,. We can think of @» as the “output’’ of the logical process illustrated in Fig. 6-7. More precisely, we define @o as follows:

Question

Yes

Yes

Question

Question

0

|

Question

im

No

pit))

Yes

Yes

Ww

No

No

na

Store P (to)

P, -

No

x(t)

a

Solve

differential

Pi tol

equations

Teiven

cnane P ‘Lo

x (fo)

| eee

Pito) to start

Fig. 6-7 The logical diagram of the “‘experiment’’ which may be used to determine the time-optimal control.

Definition 6-7

properties:

Let @o be the set of initial costates (to) with the following

(1) For each. p(to) € @o, the corresponding solutions of Eqs. (6-97)

and (6-98), denoted by

P(t) = Bilt, to, x(t0), (C0) £(t)

satisfy the relatzon

=

z(t,

to,

X(to),

P(to)]

(0 = ¥ blk, 40 =0 only at a countable set of times.

7 =1,2,...,7

(11H

(2) There is a time T [which depends on x(to)

and f(to)] such that we can find constants e1, 2,

. . . ,@n-p and ki, ke, . . .,

kn-s 80 that the relations

H(g(P), (7), TM = ) fle(h, Mp?) — s=1

j21

S ost8(P), Tip.¢P)

s=1 n—B

_= d Ca dg.(t(T), TI glt(T), T1=0

a=1,2,...,n-8 n—86

p(t) = ) py OD ai

are satisfied.

(6-18)

(6-115) (6-116)

(6-117)

Sec. 6-3

Application of the minimum

principle

391

We now suggest that the reader go over the statement of Theorem 6-3 and compare Eq. (6-115) with Eq. (6-84), Eq. (6-116) with Eq. (6-86), and Eq. (6-117) with Eq. (6-87). In view of the fact that the functions 4;(¢)

are zero only at a countable set of times and in view of the fact that the differential equations (6-97) and (6-98) are the same as the differential

equations (6-81) and (6-82), we deduce the following lemma:

Lemma 6-2 Each solution &(t) and f(t), t © (to, T), generated by an element of the set @o satisfies all the reduced necessary conditions given by Theorem 6-3.

We have found that the H-minimal

given by {see Eq.

(6-91)]

control (see Definition 6-6) u°(¢) is

u(t) = — SGN {B’[x(é), dp} for all x(¢), p(t), andé. If we now evaluate u°(¢), when x(t) = £(£), p(é) = f(d), and ¢t € [to, T], we find that

u(t) | way = O°) = — SGN {B(R®), 160} p(t) = p(t)

+E lt, T}

(6118)

If we compare Eqs. (6-118) and (6-91), we immediately deduce, in view of Lemma

6-2, the following lemma:

Lemma 6-3 Each control &°(t) generated by an element of the set Po satisfies the necessary conditions stated in Theorem 6-2. Observe that

H[&(d), 64), AC), t] < AlkO), H@), uO, @]

(6-119)

for all u(t) E 2 and t E [t, 7).

We shall now discuss the implications of Lemmas 6-2 and 6-3 and, in so doing, explain the usefulness of the necessary conditions in our search for the time-optimal control. For the sake of concreteness, let us suppose that there are three different time-optimal controls from the given initial state x(t.) to the given target

set S; all three time-optimal controls will, by definition, require the same minimum

time 7*.

Let us indicate these (time-optimal) controls by

ur(), u(t), uz)

t E [t, T*)

(6-120)

If we conduct the “experiment” of step 3d, then we shall determine a set ®o; let us suppose that we can find [from Eq. (6-118)] five different controls corresponding to the elements of @o. f1°(¢), fi2°(¢),

We shall denote these controls by

Q;°(t),

4,°(¢),

A5°(t)

(6-121)

and the intervals over which they are defined by (to, Til, (to, Te], (to, Ts], (to, T4], (to, 7s)

(6-122)

respectively. We claim that three of the five controls of Eq. (6-121) will be identical to the three time-optimal controls of Eq. (6-120). Again to

392

Structure and properties of optimal systems

be specific, we suppose that qT, T* T.=T*

4:°(¢) = uf (é)

a.) =u)

=us

T;=T*

¢€ (to, T*]

I

a)

t€ |b, T*] t€ (to, T*I

(6-123)

The question that arises is the following: What is the significance of the controls &,°(t) and 0,°(t)? Well, these two controls must be locally time-

optimal. Since the minimum principle is a local condition, it cannot distinguish between locally optimal and globally optimal controls. For this reason, the set of controls which satisfy all the necessary conditions will consist of both the locally time-optimal and the globally time-optimal controls. The only way to distinguish which of the controls 0,°(), . . . , fi;°(¢) are globally time-optimal is to measure and compare the times

T,, T., . . . , Ts and, in so doing, to find that T

=

T.

T,>T* T;>

=

T;

=

T*

T*

(6-124)

It is for this reason that we say that the necessary conditions provide us with all the controls which are candidates for optimality. In the following section, we shall discuss the results obtained in this section and motivate the subsequent sections of the chapter that deal with the time-optimal control problem. Exercise 6-2 Consider the dynamical system of Example 6-3. Using the appropriate theorems of Secs. 5-13 and 5-14, derive the necessary conditions on the optimal control for the following target sets S:

(a) S is the origin of the three-dimensional state space. (b) S is the gtate

1 X=]

1

1

(c) S is the plane x, = 0. (d) S is the moving point described by the equations m=t

Zo = 0?

Za. = sint

(e) S is the moving plane z,; = ¢ + ¢? + @?. Exercise 6-3 Consider the problem described in Example 6-3.

For this particular

problem, outline the “experiment’”’ you would perform to determine the candidates for

time-optimal control. (Hint: Mimic the development of step 3.) In view of the special form of the equations, can you suggest ways of simplifying the search for the set 0? Exercise 6-4 Consider the problem stated in Example 6-3. Suppose now that the control variables u(t) and w(t) are constrained by the inequalities

—-1 < u(t) < +3 Derive the reduced necessary conditions.

—5 < uxt) < +2

Sec. 6-4

6-4

Minimum-time

Comments

on minimum-time

problems

393

Problems 3: Comments

In the preceding section, we derived necessary conditions for the timeoptimal control and suggested an idealized systematic way of determining the candidates for a time-optimal control. We found (see Theorem 6-2) that if the problem was normal, then the components of the time-optimal control were piecewise constant functions of time. We did not derive any results for the singular problem which will be examined in Sec. 6-21. Since the components of the time-optimal control must be piecewise constant functions of time for normal problems, we can see that one of the necessary conditions—i.e., the condition H{x*(t), p*(t), u*(é), t] < A[x*(), p*(t), u(t), é, u(t) C Q—serves to limit the search for the time-optimal control to the class |u;(¢)| = 1,7 = 1,2, ...,7r. Thisis perhaps the most useful result derived from the minimum principle, as the other necessary conditions merely provide appropriate boundary and transversality conditions. The reader has probably noticedt that the Hamiltonian

H[x(), p(t), u(), ] = 1 + (fx, 4, PO) + (u@), Bx), tp(@))

(6-125)

and the differential equations z(é) =

dH[x(t), 2 p(t), u(t), #]

po = — HzO.opt)00, 00.4

(6-126)

are completely defined by the system and the cost functional and, thus, are independent of the boundary conditions at t) and of the target set S. In addition, the H-minimal control u°() (see Definition 6-6), given by the equation

u(t) = — SGN

{B’[x(2), dp}

(6-127)

is (functionally) independent of the imposed boundary conditions.

Thus,

steps 3a to 3c, outlined in Sec. 6-3, will be the same for any time-optimal

problem. The necessary conditions on the Hamiltonian and the costate variable at the terminal time 7*, together with the given initial state and the equations of the target set, merely provide enough boundary conditions to solve the system of 2n differential equations satisfied by the state and costate variables. We indicated in Sec. 6-3 the step-by-step procedure used to determine the controls €°(¢), the resultant trajectories £(t), and the corresponding costates p(t) satisfying all the necessary conditions. quantities, we make the following definition: t This is also reflected by Table 5-1.

To distinguish these

394

Structure and properties of optimal systems

Definition 6-8

Extremal Variables

A control °(t) 1s called extremal if

fio(t), the resultant trajectory &(t), and the corresponding

costate p(t) satisfy

all of the necessary conditions [1.e., they satisfy Eqs. (6-113) and (6-115) to

(6-119)]. We shall also call 2(t) and p(t) the extremal state and costate trajectories, respectively. As we mentioned in Sec. 6-3, in general, we can have many extremal

controls.

Each extremal control will then generate either a locally time-optimal

or a globally time-optimal

trajectory.t

Since

an extremal

control

all the necessary conditions, we can make the following remarks: Remark 6-7

satisfies

If the time-optimal control u*(t) exists and 1s unique and if

there are no other locally time-optimal controls, then there is one and only one extremal control &°(t) which is, in fact, time-optimal, that is, 4°(t) = u*(?t).

It should be clear that in Remark 6-7, the assumption that there are no other locally optimal controls makes the minimum principle both a necessary

and a sufficient condition. Remark 6-8 If there are exactly m, distinct time-optimal controls and wf there are exactly m, distinct controls which are locally time-optimal (but not globally time-optimal), then there will be exactly m, + m: extremal

controls.

Remark 6-9 If a (global) time-optimal control does not exist and if there are ms distinct locally time-optimal controls, then there will be exactly m2 extremal controls.

This remark illustrates that the existence of extremal controls does not necessarily imply the existence of a global time-optimal control. Remark 6-10

If a time-optimal control exists, we can find u by evaluating

the time T required by each extremal control and by picking the one which minimizes T.

The above remarks lead us to the conclusion that, when dealing with a time-optimal control problem (such as Problem 6-16), we should know the answers to the following questions: 1. Does a time-optimal control exist? 2. Is the time-optimal control unique? 3. Is the time-optimal problem normal? 4. Is there more information hidden in the necessary conditions for a given system and target set?

Unfortunately, at this time, the answers to such questions are not available for arbitrary nonlinear systems and target sets. Results, however, are known for the class of linear systems. As this class of systems is extremely important,

tional

results

which

we shall devote several sections to it, deriving addi-

are important

practical points of view.

t See also the discussion in Sec. 5-17. to saddle points.

from

both

the

theoretical

and

the

Locally time-optimal controls may correspond

Sec. 6-6

6-5

Minimum-time

control of linear systems

395

Minimum-time Problems 4: Linear Time-invariant Systemsf{

In this section, we shall specialize the results derived for Problem 6-16 to the case of linear time-invariant systems. We shall also assume that the target set is the origin of the state space. For this reason, we shall call the problem the linear time-optimal regulator problem. We note also that most of the problems that we shall examine in Chap. 7 are of this type. We shall first state Problem 6-lc so that the reader will know the precise problem that we shall consider. Second, we shall state in Theorem 6-4 the necessary conditions satisfied by a time-optimal control. Third, we shall establish in Theorem 6-6 an explicit test for normality. Fourth, we shall establish in Theorem 6-7 the uniqueness of the time-optimal control for normal problems. Fifth, we shall prove Theorem 6-8 regarding the number of switchings. Finally, we shall establish in Theorem 6-9 the uniqueness of the extremal controls. The particular special problem we shall consider is the following:

Problem 6-Ic invariant System

The Time-optimal Regulator Problem for a Linear TimeGiven the dynamical system x(t) = Ax(t) + Bu(?)

where a. The n-vector x(t) ts the state. b. The system matrix Ais ann

(6-128)

X n constant matriz.

c. The gain matriz B 1s an n X r constant matriz.

d.

The r vector u(t) ts the control.

Assume that the system (6-128) is completely controllable and that the components u(t), us(t), ..., u(t) of u(t) are constrained in magnitude by the

relations

lu(l 0

(6-208)

ie — (f,(t), Ba?(é)) dt > pr ~ (f,(t), Bas?(t)) dt

(6-209)

and that

From Eqs. (6-208) and (6-209), we deduce, for distinct @,°(¢) and &,°(¢), the following relation:

* — (p(t), Bay(t))dé > {" — (pilt), Ba°(é)) dt

(6-210)

But Eq. (6-204) and the inequality (6-210) are contradictory. It follows that f,°() and &,°(t) cannot be distinct, and so 7; = 7. and €,°(t) = 4,°(2). Since the extremal control is unique and since we have assumed that an optimal control u*(é) exists (which is vnique in view of the normality), ' Eqs. (6-193) and (6-194) must hold, and the theorem is established. This theorem states that there is one and only one extremal control.

We

derived the theorem under the assumption that the target set was the origin of the state space. Unfortunately, the theorem is not true if the terminal state is not the origin of the state space. In other words, if the problem is to transfer the system from a given initial state ~ to a terminal state 6 0

(6-213)

Step 2 Compute the costate p(t) from the relation

(6-214)

p(t) = e*'x Step 3

Compute the control variables u,(t), ue(t), . . . , u(t) from the equations

u;(t) = — sgn {(b;, p())}

(6-215)

where the b; are the column vectors of B. Step 4 Solve Eq. (6-211), starting at the initial state & and using the controls eAmMnitead

in

etan

2

Qinna

«2

uninanaley

dafinas

tha

nantral

voetnar

and

sainre

408

Structure and properties of optimal systems

€ uniquely determines the state trajectory (given the control), it is clear that the state, at each instant of time, is a function of both — and x. To emphasize this dependence, we shall denote the solution of Eq. (6-211) by

(6-216)

Xen(t) Step 5 Monitor the solution x,,(t).

If there is a time 7 such that

(6-217)

= 90

X(T)

then the control computed in step 3 is an extremal control (see Definition 6-9) and, by Theorem 6-9, is time-optimal. If Eq. (6-217) cannot be

(t)

7

Fy

-

u;(t)

| ~Gh,,7> fuel

a

Fig. 6-8

Xe elt)

9

7 Chonge

.

Lez

é

___

!

tl

I i]

4

The “open-loop” structure of the time-optimal problem.

satisfied, then change = in step 1 and repeat the procedure until Eq. (6-217) is satisfied. This procedure is iterative because one must change the initial costate = many times to find the time-optimal control. The initial guess and the subsequent values of « must be such that Eq. (6-213) is satisfied; Eq. (6-213) is a direct consequence of Eqs. (6-202), (6-203), and (6-208). The procedure from step 1 to step 5 is illustrated in block-diagram form in Fig. 6-8. We use double arrows to indicate the ‘‘flow’” of vector signals and single arrows to indicate the “flow” of scalar signals. The “relay” symbol

+

-|

is used to indicate the “sgn” operation.f The outputs of the “relays” Ri, ..., &, are the control variables u,(é),..., u(t). The block labeled ‘Change x” monitors x,,(t) and, after a “sufficiently long” time, decides whether or not a new x should be introduced to the system. t Thus, an ideal poJarized relay is the engineering realization of the signum function. This convention will be used throughout the book.

Sec. 6-6

Structure of the time-optimal feedback system

409

There are two questions that arise: 1. How long must one wait before x is changed? 2. How do we decide on a new value of «x based on the previous values used and the ‘‘miss distances” of x,;.(¢) from 0? The answers to these questions are the subject of many investigations ;{ however, we shall not discuss any iterative techniques, as they are beyond the scope of this book. We would like to comment that these procedures are complex and that they require a large-memory, high-speed computer. The procedure suggested is an open-loop technique. In other words, given an initial state —, we find a time-optimal control u*(t) which will depend on the initial state — and on the time ¢; this control will not depend on the instantaneous value of the state x(t) fori > 0. The disadvantages of an open-loop method of control are well known to the engineer. The additional difficulty that is also inherent in this type of computation is the following: Suppose that the eigenvalues of the system matrix A have negative real parts, ie., that the system is stable. Then the adjoint dynamical system p(t) = —A’p(f) is unstable because the eigenvalues of the matrix —A’ have positive real parts. Thus, a small ‘error’ in the initial costate x will be “amplified” as time goes on and may cause the computed control to deviate significantly from the optimal one.{ For this reason, it is worthwhile to investigate whether or not we can find the time-optimal control u*(t) (at time ¢) as a function of the state x*(t) (at time t). If we can do that, then we shall have derived the optimal feedback

system which has the property that, at each instant of time, the state completely determines the value of the control. We claim that there is a “switching function” h[x*(t)] such that the timeoptimal control u*(é) is given by u*(t) = — SGN

{h{x*(é)]}

(6-218)

Equation (6-218) is often called the time-optimal feedback control law. We shall now present a heuristic justification for Eq. (6-218). Let x*(i) be a state on the optimal trajectory from the initial state & to 0, and let p*(t) denote the value of the costate at that time. Clearly, the relation

x*(T*) = 0 = er» [x¥() — [" e-4-OB SGN {B’p*(r)} dr] (6-219) is satisfied.

Since the matrix e4(T*- is nonsingular,.we deduce that

x*(t) = [ "e-Ac—-OB SGN {B'p*(r)} dr p*(r) = ee Op*(t) oo Dt

But t See

Refs. N-6,

B-1, H-9, E-1, P-1, K-19,

(6-220) (6-221)

G-5, and F-1.

{If the adjoint system is simulated on an analog computer, additional] errors due to

the saturation of amplifiers and due to the amplification of the inherent noise will occur.

410

Structure and properties of optimal systems

and, furthermore, the time (7'* — ¢) required to force x*(t) to 0 must be a function of x*(d); that is, T* —t = a(x*(é)] From Eqs.

(6-222)

(6-222) and (6-221), we conclude that Eq.

(6-220) represents a

relation between x*(¢) and p*(t), which we shall write in the form

(6-223)f

p*(t) = p*[x*(4)] Since u*(t) = — SGN

that

{B’p*(t)}, we deduce from Eqs. (6-223) and (6-218)

h[x*(¢)] = B’p*[x*(4)]

(6-224)

and, thus, the existence of the switching function has been heurtstically justified. AJl the problems we shall solve in Chap. 7 will involve the

n(t)

4

R

.

ial t)) op Computer

ok

d

” =hp(a lt)

*

.

r

gle) .

.

:

x=Axr+Bul|

ur (t)

pant)

:

.

lt)

~ alt)Y

Y

-_s —

ante —/

Fig. 6-9

The feedback structure of a time-optimal control system.

determination of this switching function. As we shall see, this switching function will be defined by the so-called ‘“‘switch hypersurfaces” in the

state space. We illustrate

the

block

diagram

of

the

optimal

feedback

system

in

Fig. 6-9. We note that the state variables 2,(f), x(t), . . . , tn({t) are measured at each instant of time and are introduced to the subsystem labeled ‘‘Computer’’; the outputs of the computer are the switching functions h,[x(t)], Ae[x(é)], . . . , A-{x(t)], which are then introduced to the ideal relays Ri, R:, ..., R,- to generate the time-optimal control variables. The determination of the functions h,[x(t)], . . . , A,[x(t)] and their engineering realizations is the “‘heart’’ of a time-optimal control problem. The primary purposes of Chap. 7 are (1) to present some of the techniques available for determining switching functions and (2) to acquaint the reader with the types of nonlinear-function generators required in the engineering implementation of a time-optimal system.

t This relation is not necessarily unique. In other words, for a given x*(?), there may correspond more than one p*(?) which will generate the optimal control (see Ref. K-19).

Sec. 6-7

6-7

Geometry of the time-optimal control

411

Minimum-time Problems 6: Geometric Properties of the Time-optimal Control

In Sec. 6-2, we discussed the geometric nature of the time-optimal problem in terms of the sets of reachable states. We then jumped from geometric considerations to the analytical results which were obtained from the necessary conditions provided by the minimum principle. In this section, we shall attempt to provide a geometric interpretation of the necessary conditions.

We

shall limit our discussion to Problem

6-lc, and

we shall assume that the time-optimal problem is normal according to Definition 6-4. We remark that the material of this section represents a specialization of the comments of Sec. 5-21. We shall examine the minimum-time surface, and we shall interpret the time-optimal controi as the control that causes the state to move in the direction of steepest descent on the minimum-time surface. We shall then be able to establish a correspondence between the costate and the

gradient of the minimum-time surface. Our treatment will be heuristic in nature, as we are interested primarily in geometric interpretations of

the necessary conditions. Let x be a state in our state space, and let us assume that there is a timeoptimal control (which is unique) that drives x to 0.

unique minimum time required to force x to 0 by T*(x)

We shall denote the

(6-225)

We now claim that the minimum time 7'*(x) depends only on the state x and not on the time #, that is, that

aT*(x) _ 0 a”

(6-226) T

This is so because the time invariance of the system x(t) = Ax(¢) + Bu(t)

implies that the minimum time can only be a function of the state. In other words, if x is the state at ¢ = 0 and if the minimum time required to force x to 0 is 7'*(x), then, if x is the state at ¢ = to, it follows that the time-optimal control will force x to 0 at the time t + T*(x). [See Eqs. (4-266) and (4-267).] Since the time required to force 0 to 0 is zero and since we are concerned with positive time solutions, it is evident that 7T'*(x) has the properties T*(x) = 0 T*(x) > 0

ifx = 0 ifx ~0

(6-227) (6-228)

t It has been the experience of the authors that this equation is confusing to students who read it as ‘‘the partial derivative of the time 7'* with respect to time is zero” and who do not understand the real meaning of a partial derivative. We trust that those who read Sec. 3-12 will not fall into the same trap.

412

Structure and properties of optimal systems

We shall use the notation

i T*(x)] 021

oT*(x) _ ox

(6-229) |

aT *(x) OLn |

in the sequel to denote the gradient of the function 7*(x) with respect to x. In the sequel, we shall discuss several properties of the function 7*(x). It is useful to think of 7T*(x) as the T"(x) minimum-cost surface for Problem 6-1c and to visualize 7'*(x) as illustrated in Fig. 6-10.

We now define the notion of a mum isochrone.t Definition 6-10 Minimum chrones Let S(r) be the set of which can be forced to the origin

mini_Isostates 0 in

the same minimum time r,7 20. We shall call S(r) the r-minimum isochrone. S(r) 18 given by the relation

S(r) = {x: T*(x) = 737 >0}

(6-230)

Let S(r) denote the set of states which can

;

Fig.

6-10

The

,

minimum-time

surface T'*(x) as a function of x.

cost

be forced to the origin by time-optimal controls which require a minimum time less than or equal tor. More precisely, we have S(r) = {x: T*(x) 0}

(6-231)

From Eqs. (6-230) and (6-231), we conclude that S(r) is a subset of S(r). We also assert{ that S(r) is the boundary of S(r) and that S(t) zs closed. We shall now prove that the set S(r) is a strictly convex set. Let x, and x: be two distinct states on the r-minimum isochrone; i.e., let

nECSr)

x € S(r)

(6-232)

In view of the normality, we know that there are a unique time-optimal control uf (é) = — SGN {B’pf(t)} that transfers x, to 0 and a unique timeoptimal control uy (#) = — SGN {B’p?}(é)} {distinct from uf (t)] that transt The notion of a minimum isochrone will be used in Chaps. 7, 8, and 10. { For a formal proof, see Ref. S-9.

Sec. 6-7

fers x. to 0.

Geometry of the time-optimal control

415

Thus, the equations

x, = f eB SGN {B’p*(t)} dt x, = f e-MB SGN {B’p3(t)} de

(6-233) (6-234)

must be satisfied.

Let x be a state on the (open) line segment joining x; and Xz, as illustrated in Fig. 6-11; more precisely, let a be such that t 0O

—fr,|z2!

if z1
|b:|/A:, then we cannot find a 17 such that

z,(1)

= 0, and therefore a time-optimal control does not exist.

Now, if all the eigenvalues ); are nonpositive, it 1s easy to show that Eq. (6-263) can be satisfied for all |é;j and alli = 1, 2,... , n, because

we can find values of fT sufficiently large.

This, in turn, implies that a

time-optimal control exists for all initial statest (why?). Example 6-4

Consider the time-optimal control of the system

H(t) maz(t) +ult) If a < 0, all ¢§. If for which [u*(¢)} =

fu(d)|) 0, then the system is unstable. Let us find the set of initial states Vo a time-optimal control exists. If a time-optimal control u*(t) exists, then 1, and so we have

a

I ” e-sty®(t) dt

Thus, we deduce that

ie =| [fT eerur at] < [ feraumion at < fi lel ero) at

= f et dt = ——(or — 1) (6-268) ’

1

~—aT

*n order for the relation (6-268) to hold for some finite positive 7, it is necessary that

lei
1/a, then a time-optimal control to the origin does not exist.

set Yq is an open set containing the origin.

(6-269)

In this case, the

Exercise 6-18 Consider the time-optima! control to the origin (0, 0) of the following second-order systems. In each case, the control u(¢) is constrained in magnitude by

|u(¢)| < 1. (a)

2:(t)

= x1(t) + 2,(t)

Zo(t) =

—21(t) + 22(t) + ult)

(c) 2:(t) = z(t) T(t) = 2(t) + u(t)

(b)

£:(t) = z(t)

Z2(t) = x(t) + u(t)

For each system, find the set of initial states Yo for which a time-optimal control to the origin exists. (Assume that if there is an admissible extremal control, then there is an

optimal control.) 6-9

Minimum-time

Problems 8: The Hamilton-Jacobi

Equation

In Sec. 5-19, we discussed the behavior of the minimum cost along the optimal trajectory. In Sec. 6-7, we discussed the geometric properties of the time-optimal control for Problem 6-lc. In this section, we shall Tt See Ref. P-5. We do not deal with existence proofs in this book, because such proofs fall outside its introductory scope.

422

Structure and properties of optimal systems

combine

these concepts

the time-optimal problem.

and

examine

the Hamilton-Jacobi

equation for

Our objective will be to illustrate the use of

the general results given in Chap. 5 in the time-optimal problem 6-1c. Throughout this section, we shall deal with the time-optimal control of the normal system x(t) = Ax(f) + Bu(t), where the target set is the origin of the state space. We shall use the following notation: If we are given a state x, then we shall let 7*(x) denote the minimum time required to force x to 0 and we shall let u* denote the value of the time-optimal control at the state x. Our specific objectives in this section are the following: 1. To indicate how the Hamilton-Jacobi equation can be used to test whether or not a given function 7(x) can possibly be equal to 7T*(x) after we have solved the time-optimal problem 2. To show that difficulties arise if the assumption that a given control is the time-optimal one turns out to be incorrect 3. To indicate the difficulties associated with the determination of the time-optimal control directly from the Hamilton-Jacobi equation We shall first discuss the use of the Hamilton-Jacobi equation viewed as a necessary condition. From the general theory of Sec. 5-19, we can deduce the following lemma: Lemma 6-6 Let x* be a state on a time-optimal trajectory, and let u* be the value of the time-optimal control at x*. Since dT*(x)/dt = 0 for all x, zt 1s necessary that T'*(x) satisfy the relation

dT *(x) -Ox

x=

) =0

(6270)

provided that (8T*(x)/dx)|,-x« exists. Exercise 6-14

Prove Lemma 6-6.

Hunt: See Eq. (5-695).

This lemma is useful when the time-optimal problem has already been

solved and we seek to determine whether or not a given function 7(x) can possibly be the expression for the minimum time as a function of the state. If a given function 7(x) violates Eq. (6-270) even at a single point, then T(x) can be immediately eliminated as a candidate for the minimum time. We shall illustrate this point by the following example: Example 6-6

rs] =[2 TEE] +[g]ao mois

Suppose that the linear system is described by the equations

We assertt that the time-optimal control is

t See Sec. 7-2.

u* = —1

ifr—x =| 5]

(6-272)

ut=-1

ifx=xm= H

(6-273)

Sec. 6-9

The Hamilton-Jacobi equation

423

Let us suppose that we have a “hunch” that the expreasion

T(x) = T(x1, 21) = $21? + 223

(6-274)

is the expression for the minimum time as a function of the state. Observe that this hunch is not unreasonable, because T(x) > 0 for all x, T(0) = 0, and lim T(x) = ©, x

@

We shall now demonstrate how to prove that our hunch is wrong. First, we calculate the gradient of 7(x); we find, from Eq. (6-274), that this gradient is given by oT(x) oT (x)

Ox)

ax

Zt |

(«| aT (x) |

(6-275)

lon,

OX

21 =f

eam

We evaluate it at x = x. and x = xg to find that

ax

OT (x)

s=z, 7 A

(6-277)

At X = Xa, the left-hand side of Eq. (6-270) becomes

1+ (3

0 | Hi [i] ~1 i"

1] i; =1+2—-4=-1 0

(6278)

We can immediately conclude that the 7(x) given by Eq. (6-274) cannot be the minimum-

time expression, because at x = xq Eq. (6-270) does not hold.

Let us see now what happens if we test T(x) at the point x = xg. hand side of Eq. (6-270) becomes

At x = xg, the left-

[gif e-ati-2-0 © @em

1+((> oflablaly 14°

This means that 7(x) satisfies the necessary condition of Lemma 6-6 at x = xg, and on the basis of this test, we could conclude that 7(x) may be the expression for the minimum time. Nonetheless, the test at x. discards this possibility. Now suppose that we made an error in the determination of the time-optimal control; for example, suppose that we thought that

u* = +1

itr=x=(1|

(6-280)

Then, instead of Eq. (6-279), we would find that 1+([9

oleh

ep

+2{°

Na par+it+2-4%0

(6-281)

and we could eliminate 7'(x) from consideration. Now it is true that 7(x) ~ 7*(x), but the conclusion from Eq. (6-281) cannot be trusted, because the original premise that u° = +1 at xg is wrong. In other words, if we make a mistake in determining the optimal control, then, in all likelihood, we would eliminate the correct minimum-time

expression 7'*(x) by this procedure.

From a practical point of view, however, this lemma is not very useful. The reason is that most of the time the engineer is interested in the timeoptimal feedback

control system rather than in testing whether or not a

4£%

structure and properties of optimal systems

given 7'(x) is the same as 7*(x). Nonetheless, the use of the HamiltonJacobi equation as a necessary condition is of value in theoretical investigations and in checking whether the results obtained using the minimum

principle are correct.

In Chap. 5, we concentrated much more on the Hamilton-Jacobi equation viewed as a sufficient condition. We shall now discuss the problem of solving the Hamilton-Jacobi equation and constructing the time-optimal control. At this point, we fee) that the reader should go over the statement of Theorem 5-12 very carefully. We know that the Hamiltonian for the time-optimal problem 6-1c is

H(x, p, w) = 1 + (Ax, p) + (u, B’p)

(6-282)

Since the system x = Ax + Bu is normal, the Hamiltonian H is normal according to Definition 5-12, and so the H-minimal control is given by |

fi= — SGN {B’p}

(6-283)

From Eqs. (6-283) and (6-282), we have

H(x, p, &) = 1 + (Ax, p) — (SGN {B’p}j, B’p)

=1+ i=1k=1 5 ¥ auem— jelYo] foly bom]

(6-284)

Now let us consider the partial differential equation (the Hamilton-Jacobi equation) 14

y

Y one em

--

y

p,, SES)=)

—0

(6-285)

im] b=

and let us suppose that we were able to find a solution

T(x)

(6-286)

of the partial differential equation (6-285) such that:

1.

T(0) =0

2. The control vector

(6-287)

— — SGN {pr FI ox

(6-288)

substituted in the linear system yields the system X(t) _= Ax(t) - BSGN whose solution has the property

x(t)

{p

, aTax(t) [x()]

with £(0) = & £(7(5)] = 0

(6-289) (6-290) (6-291)

Sec. 6-9

The Hamilton-Jacobi equation

425

In other words, the solution 7(x) of the Hamilton-Jacobi equation (6-285)

defines a control @ [given by Eq. (6-288)] which, in turn, generates a trajec-

tory £(¢) which arrives at the origin at the time 7(£), where E is the given

initial state. If such is the case, then the control f(t) will be time-optimal, at least with respect to neighboring controls; that is, A(¢) will be locally time-optimal. We shall illustrate these ideas by a very simple example. We shall then discuss the shortcomings of this method. Example 6-6

Suppose that we are given the first-order system

o(t) = Qu(t)

fu(t)) 0}

X, = {z:2 < 0}

(6-296)

It is trivial to verify that the function

T(z) = $/2|

(6-297)

is the solution to the partial differential equation

(6-295) for all c © X: U X2, because

aT (2) oT)

=

forz © X1

(6-298)

=-4

forzEGX:

(6-299)

Observe that

ofa) and that

0; by substituting Eq. (6-292), we find that z(t) = —2

and

a(t) = & — 2t

EEX

(6-303) (6-304) (6-305) (6-303) into Eq. (6-306)

(6-307)

426

Structure and properties of optimal systems

Observe that from Eq. (6-297), we have

T(t) = ft

Therefore, we deduce that

géEX

(6-308)

a[T(t)]) = &-—E=0

(6-309)

Moreover, for all ¢ € [0, 7(£)), we have which means that

At) =§&—-2>0

(6-310)

At) © Xi:

(6-311)

We thus immediately conclude that the constant control a= is time-optimal for all z € X:. is time-optimal for all z € X2.

—1

(6-312)

Similar reasoning yields the conclusion that the control

i= +1

(6-313)

Note that we have defined the regions X: and X; in such a way that o7(2z)/dz was well defined.

We had no particular difficulty in determining the time-optimal control for the above simple system using the Hamilton-Jacobi equation. There are several reasons for this: 1. It was extremely easy to guess a solution to the Hamilton-Jacobi equation that satisfied the boundary conditions. 2. It was extremely easy to define the two regions X,; and X».

3. It was extremely easy to verify that x{7(£)] = 0. If we attempt, however,

to determine the time-optimal

control for a

higher-order system, we shall encounter the following difficulties :t

1. It is almost impossible to determine the solution to the HamiltonJacobi equation for systems whose order is higher than 2. 2. For an nth-order system, one must subdivide the state space into at

least 2n regions X;, Xe,

. . . , X2n, which are extremely difficult to specify

6-10

Problems 9: Comments

for systems whose order is higher than 2. It is for these reasons that, at present, the design of time-optimal feedback systems is mostly carried out using the necessary conditions provided by the minimum principle rather than the sufficient conditions provided by the Hamilton-Jacobi equation. Minimum-time

and Remarks

In the preceding seven sections, we analyzed the time-optimal control problem from several points of view. Our objectives were to interlace the mathematical and geometrical aspects of the problem so that the reader would appreciate their implications. t The following comments are the outgrowth of the experience of the authors in trying

to solve several minimum-time problems.

Sec. 6-10

Comments on minimum-time problems

427

As the reader has probably noted, the strongest set of theorems were stated with regard to the linear time-invariant time-optimal regulator

problem (Problem 6-lc). Some of these theorems carry over to the timeoptimal control to the origin for the linear time-varying system

t(t) = A(é)x() + Bud)

(6-314)

provided that the time-optimal problem is normal. In particular, Theorem 6-7 and Exercise 6-7, which state that the time-optimal control is unzque, still applyt for the time-varying system (6-314). In addition, Theorem 6-9 (which states that if the origin is the terminal state, then the extremal controls are unique) is also true{ for the system (6-314). The theorem which

deals with the number

of switchings

(Theorem

6-8)

and the most

useful existence theorem (Theorem 6-10) have no counterpart for the time-

varying system

because they are stated in terms of the (constant)

eigen-

values of the system matrix A. If we turn our attention to the nonlinear time-optimal problem (such as Problem 6-1b), we find, in the current literature, few§ useful results in

terms of general characteristics or theorems.

The difficulty is not a con-

ceptual one; rather, it is due to the fact that we cannot write analytical

expressions for the solutions of nonlinear differential equations.

The theoretical development of the time-optimal control problem will be applied in Chap. 7 to the design of time-optimal systems. Our objective in Chap. 7 will be the determination of the time-optimal control as a function of the state, i.e., the solution of the time-optimal feedback problem. We feel that before the reader can appreciate the general concepts of the time-optimal problem, he should see the explicit solution of some timeoptimal problems. Several exercises in Chap. 7 are designed to illustrate the geometric concepts discussed in the preceding sections. In Chap. 7 we shall, whenever possible, present the engineering realization of the timeoptimal control laws so that the relative complexity of the required “‘hardware” can be evaluated. We suggest, at this point, that the reader who is tired of general concepts and theorems skip the remainder of this chapter

and go over the material of Chap. 7. The following exercises are designed to acquaint the reader with the

time-optimal control of linear time-varying systems.

Exercise 6-16 For each of the following systems, determine the necessary and sufficient conditions for the time-optimal problem to be normal. Assume that all the time-varying coefficients are infinitely differentiable functions of time. First refresh your memory by reading Definitions 6-4 and 6-5 and by going over the material preceding Theorem 6-5.

t See Ref. P-5, p. 186.

t See Ref. P-5, pp. 186 and 187. §Some

references which deal with the nonlinear time-optimal

problem from the

theoretical point of view (and mostly from the point of existence) are H-3, N-5, F-3, L-7, R-8, and W-3.

428

Structure and properties of optimal systems

(a) Z:(¢) = x(t)

ée(t) = +ar(t)ar(t) + as(t)za(t) + dilt)ult) — u(t)| T*

(6-328)

We shall now use the statement of Theorem 5-9 to derive the necessary conditions for our fuel-optimal problem. Statement 1 of Theorem 5-9 guarantees the existence of a costate p*(é) corresponding to u*(é) and x*(¢). Now the Hamiltonian (6-326) evaluated at x*(¢), p*(¢), and u*(¢) is

H[x*(), p*(d), u*(), ] = y ojluz (| + y fix*@), Opt j=l

t=] fr

y b{x*(d), uf (pit) + », 1é=1 Since x*(t) = 9H/dp*(t)

(6-329)

and p*(é) = —dH/dx*(t), we conclude that zy (#)

and p; (i), k = 1,2, ... ,n, must satisfy the canonical equations

at) = fix", 1+ Y dulx*@, dur

(6-330)

j=l

pt) = - > {2450.4 oro- JY wo Y {ale A eco (6-331) fork = 1, 2,...,m. The reader should note that the canonical equations (6-330) and (6-331) for this fuel-optimal problem are identical to the canonical equations (6-53) and (6-54) for the time-optimal problem. The reason is that the difference between the Hamiltonians (6-329) and (6-52) is & function of u*(t) alone. Statement 2 of Theorem 5-9 is

H[x*(t), p*(é), u*@, | < H[{x*@, p*@O, u, @) for all u(t) C2 and t € [t, 7].

Eq. (6-332) reduces to the relation

y csluf (é)| + y fix*(), dp?

j=l

(6-332)

From Eq. (6-329), we then deduce that

+ ye uf (t) { x bidx*(t), tpt}

r

< } alu] + y flx*), dp? + yu u,(t) { y bsix*(t), tpi ()} j=l

+=1

t=}

(6-333)

Sec, 6-13

Derivation of the necessary conditions

435

which, in turn, yields the relation

Y 1 alu Ol + j=1Y af

{ bf Ol + ax*@, p*O) + Bur(o, p*(0)

j=

< ) laf] + Ax"), p*O) + Bar), p*) (6-399) because Eq. (6-372) holds for all u(é) and, in particular, for A*(¢).

We have

assumed that u*(t) and @*(¢) are distinct and that the fuel-optimal problem is normal; the normality assumption implies that the inequality in the

relation (6-399) is strict whenever

u*(t) ~ G*(t)

(6-400)

For this reason, the relation (6-399) reduces to the strict inequality

Y lurol + Bur, p*O) < Y aro] + Bar, p*) (6-401)

jo

whenever Eq. (6-400) holds.

j=

Sec. 6-14

Minimum-fuel control of linear systems

447

If we integrate both sides of Eq. (6-401), we find that [for distinct u*(#) and @*(t)]

p*() at i) lurOl dt + f' Bure,

< hr 2 jar@iat+ [’ wari), p*®)de (6-402)

and that (in view of Eq. (6-393)]

ff" Bu"), p*(D) at < [" Bar), p*®) dt

(6-403)

Now let us examine Eq. (6-397) and form the scalar product

(x, [" e-*Bu*(t) di) = (x, [” eBar(s) at)

(6-404)

where x = p*(0). Since x is a constant vector and since [from Eq. (6-398)] m = etA'tp*(t)) we can see that Eq. (6-404) reduces to the equation

[Bur (o, pro) dt = f/" Bar, p*O) di

(6-405)

Thus, the assumption that u* and * are distinct and fuel-optimal has led to the two relations (6-403) and (6-405), which are contradictory. It follows that u*(¢) = &*(¢) for all ¢ € [0, 7,] (except possibly at the switch times), and so the fuel-optimal control is unique.

The reader should note that it was the assumption that the problem is

normal which allowed us to deduce the strict inequality (6-402); if the problem were not normal, then the relation (6-402) would not be a strict

inequality, even though u* and i* are distinct. Let us now recapitulate: In this section, we stated a fixed-time fueloptimal problem (Problem 6-26), we stated the necessary conditions in Theorem

6-12,

we

derived

the

sufficient

conditions

for the

fuel-optimal

problem to be normal (Theorem 6-13), and we used the normality to derive Theorem 6-14. It would be nice to know whether or not we are dealing with a problem

whose solution is unique (if it exists). However, from the point of view of determining the fuel-optimal control using the necessary conditions, it would be still more useful to know whether or not there is one and only one control that satisfies all the necessary conditions, i.e., a unique extremal control. We discussed the importance of unique extremal controls in the case of time-optimal problems. We shall reiterate these ideas in order to motivate the uniqueness theorem which we shall prove later in this section. Let us assume that the fuel-optimal control is unique. This, of course, means that the fuel surface has a well-defined absolute minimum. How-

448

Structure and properties of optimal systems

ever, we do not know whether or not there are any relative minima. If there are many cantrols which satisfy all of the necessary conditions given in Theorem 6-12, then there may be relative (local) minima in addition to the absolute minimum. In this case, to distinguish the control that corresponds to the absolute minimum, we must compute the cost (fuel) required

by every

control that satisfies the necessary

conditions

and compare

the

costs; naturally, the control with the least cost (fuel) is indeed the required

optimum.

The following theorem states that the assumptions which guarantee the uniqueness of the fuel-optimal control also guarantee that there is one and only one control that satisfies the necessary conditions. Clearly, knowledge of such a theorem (coupled with Theorem 6-12) will be of immense help in the long process required to determine the optimal control. We shall now state the uniqueness theorem in a precise manner. In the statement of the theorem, we shall include for the sake of completeness the necessary conditions. Theorem 6-15 Uniqueness of the Extremal Controls for Problem 6-2b Let T; be a given posttive time, let © be a given initial state, and let @ be a given terminal state, not necessarily equal to 0. Suppose that G(t) and fi(t), t€ (0, T,], are two admissible extremal controls. More precisely, suppose that: a. The solution x(t) of the system

x(!) = Ax(t)+ Ba)

2(0) =£€

(6-406)

and the solution £(t) of the system

x(t) = Ax(t) + Ba)

=—_£(0)

E

(6-407)

are such that

£(T,) = x(T;) = 6

(6-408)

b. There are a costate f(t) corresponding to A(t) and &(t) and a cosiate p(t)

corresponding to ti(t) and &(t) such that

b(t) = —A’p(t) = (0) = & (unknown) f(t) = —A’p()

so that and

~— (0) = # (unknown) p(t) = e-*"tg p(t) = e*"#

(6-409)

(6-410) (6-411) (6-412)

c. The following relations hold (why?) for all u(t) € Q and t € (0, T,):

¥ [a] + BAW, BO) < SY fu) + Baw, 60)

6-413)

y la;(t)| + (BE), BO) < y |uj(t)| + (Bu(t), BY))

(6-414)

=)

j=l

j=}

j=l

Sec. 6-14

Minimum-fuel control of linear systems

449

If the fuel-optimal problem is normal (see Definition 6-11), then a(t) = ai()

for allt € [0, T;}

(6-415)T

which means that the extremal control is unique. PROOF From Eas. (6-406) to (6-408), we find that

0 = 4(T)) = et [e+ [" eBay at| = 3(7,) = et [E+ [" eBa(t at]

and that

(6-416) (6-417)

Since the matrix e47: is nonsingular, we deduce the (vector) equality

[

—-MBA(t) dt = [ *! -MBA(t) dt

(6-418)

Now we form the scalar product of both sides of Eq. (6-418), first with the constant vector # and then with the constant vector #, to derive (how?) the two relations

[ @& e*BaD) dt = [" (a, e-BA) ae

(6-419)

fr (8, e-MBA(L)) dt = tr (&, e “Bi(t)) di

(6-420)

But, in view of Eqs.

(6-411) and (6-412), we see that Eqs.

(6-420) reduce to the relations

(6-419) and

{” @, Baw)dt = [" BO, BAW) at LP @@, Baw) at = [" GW, Ba@)at

6-421)

(6-422)

Now let us suppose that @ and fi are distinct. If the two extremal controls are distinct and if the fuel-optimal problem is normal, then from Eq. (6-413) we deduce that whenever fi(t) ¥ fi(t), the relation

> la] + BAO, ©) < Y la] + BAW, bO)

j=1

j=

—(6-423)

must hold, because Eq. (6-413) holds for all admissible u(t) and, 1n particular, for u(é) = fi(t). Using identical reasoning, we deduce from Eq. (6-415)

that the inequality

¥ la + Baw, BO) < j=lY [al + Baw, pO) — (6-424) j=l Tr

must hold. Adding the two inequalities (6-423) and the inequality

(6-424), we obtain

(Ba(d), b()) + Ba), PO) < BAH, b@) + (Ba), H)) t Except possibly at the switch times.

(6-425)

450

Structure and properties of optimal systems

which is valid whenever the inequality

f(t) ¥ fi(f).

Equation

(6-425), in turn, yields

[ 1Ba®, 6) + Baw, poy at < [” (Baw, pO) + Ba@, p—)) at

(6-426)

Thus, the assumption of normality and the assumption that ft and i are distinct imply the strict inequality (6-426). On the other hand, the necessary conditions require that the relations (6-421) and (6-422) be satisfied. By adding the right-hand side of Eq. (6-422) to the left-hand side of Eq. (6-421) and by adding the right-hand side of Eq. (6-421) to the left-hand side of Eq. (6-422), we obtain the equality

L' (Ba@, 6) + Bae, soy] at = [” (Baw, 9) + Bac, pO] ae (6-427) But

the two

relations

(6-426)

and

(6-427)

are contradictory.

For

this

reason, we conclude that @ and fi cannot be distinct, and so the uniqueness

of the extremal controls has been established. Both of the uniqueness theorems which we have presented were based upon the assumption that the fuel-optimal problem was normal. We have presented in Theorem 6-13 a sufficient condition for the problem to be normal. We wish to reiterate that the condition is not necessary. Thus, it is possible for a problem to be normal even though the sufficiency condition is violated. It may turn out, as we shall see in Chap. 8, that the problem is normal for some initial states but not normal for others. In the next section, we shall formulate some different types of fuel-optimal

problems, which, in a given situation, may represent a superior mathematical formulation of the physical situation. Meanwhile, we invite the reader to examine the following exercises, whose purpose is to illustrate the concepts discussed. Exercise 6-21

In Sec. 6-5, we prove (see Theorem 6-8) that if the eigenvalues of the

matrix A are real, then each function (b;, p*(é)) has at most n — 1 zeros. Use this theorem to derive an upper bound on the number of switchings for the normal fueloptimal problem. Exercise 6-22 Attempt to show that Eq. (6-392) in Theorem 6-13 is also necessary

for Problem 6-2) to be normal.

Why will this procedure fail?

Can you determine any

additional assumptions which will make Eq. (6-392) a necessary condition?

Exercise 6-28

Again consider the linear system i(¢) = Ax(t) + Bu(t).

Suppose that

the given initial state &, the terminal state 6, and the terminal time 7, are related by the equation £ = ¢~47/6. Determine the fuel-optimal control. Describe the physical significance of your answer. Is this fuel-optimal control unique even if A is a singular

matrix?

Explain!

Exercise 6-24 The sufficient conditions stated in Theorem 6-13 were derived for Problem 6-2b. Suppose that in the statement of Problem 6-2), we replace the terminal state © by an arbitrary target set S, and suppose that we consider the fuel-optimal prob-

Sec. 6-15

Additional formulations and cost functionals

451]

lem of transferring & to S in either fixed or free time. Do you think that Eq. (6-392) will still be a sufficient condition for normality? Explain! Exercise 6-25 Consider the linear time-varying second-order system di(t) = x(t)

(6-428)

tot) = a(t)ar(t) + b(¢)ze(t) + c(t)u(t) Suppose that |u(t)|

a(t)

: Xn (t)

“ 3 the minimum energy required

Sec. 6-19

Anexample

469

decreases very slowly. For 7, very large, it is clear that E* ~ 2£,?/T;. In Fig. 6-22, we illustrate the optimal trajectories from the initial state

(1, —1) to (0,0). Observe the severe changes that can occur as the terminal time 7’; increases. From Fig. 6-21, we know that the energy differential

for T; = 3 and 7’, = 10 is very small; however, the trajectories are radically different (with large overshoots occurring when T,, is large). The above discussion brings the sensitivity of optimal problems into focus;

quite often,

smal]

changes

in some

parameter

either large or small changes in the minimum design must take such effects into account.

Fig. 6-22

Next,

cost.

we shall verify that the minimum

energy is positive for all & ¥

product notation as E *

and that

T';) can cause

The behavior of the optimal trajectories originating at (1, —1) as 7’; varies.

f = 0; to do this, we note that we can write Eq.

and we definite If we control

(say

A good engineering

’

’ T;)

=

(Ey & Ty)

{&]

\Lé]

2

1 Th.

TA

3

LST,

(6-536) in the scalar-

ms [&))

3

TPL

be 4/

6-537

(

observe that the matrix appearing in the scalar product is positive for all values of 7, (compare with Exercise 6-34). Thus, E* > 0. are willing to accept large values of T,, then we can accomplish the task with a very small amount of energy. It is easy to see that lim

afob 10 E*(é, &, Ty) = 0

>

u(t) = 0

t€ (0, T,]

(6-538) (6-539)

470

Structure and properties of optimal systems

But if the control u(t) is small, then the trajectories exhibit large overshoots (see Fig. 6-22). We saw in Fig. 6-21 that the #* versus 7, curve had an inflection point. It 1s easy to see by setting 0E*(é;, £2, T,)/dT;

occurs whenever the relation

= 0 that an inflection point

&T, = —3&

(6-540)

is satisfied. Now we shall demonstrate that if we formulate the minimumenergy problem as a free-terminal-time problem, then the necessary condition that the Hamiltonian function be zero at the terminal time yields this inflection point. Recall that the Hamiltonian is given by [see Eq. (6-523)]

H = }ur(t) + ra(t)pild) + u(é)pa(?)

(6-541)

Let T be the unknown terminal time; then it is necessary that

gu*(T) + xe(T)pi(T) + u(T) p(T) = 0

(6-542)

But it is necessary that 2.(T) = 0 and that u(t) = —p.(é) for all ¢ € (0, T); it follows that u(T) = —p.(T), and so Eq. (6-542) reduces to the equation 3u2(") = 0

(6-543)

u(T) = 0

(6-544)

or, equivalently, to the equation

But Eqs. (6-544) and (6-531) yield the relation

—re2 + mT =0

(6-545)

From Eqs. (6-545), (6-533), and (6-534), we can conclude that the initial

state variables ¢, and & and the terminal time T must satisfy the equation

7 (6&: + 2&T)

=0

(6-546)

Equation (6-546) is satisfied if or if

T=

0

iT = —3é

(6-547)

(6-548)

We thus note that the necessary conditions for the free-time problem have led to Eqs. (6-547) and (6-548). We have shown that the case T = corresponds to the absolute minimum of E*, that is, to H* = 0. We also note that Eq. (6-548) is the same as Eq. (6-540), and so Eq. (6-548) yields an inflection point rather than a minimum; this, of course, can happen, as we have noted that (see Sec. 5-17, comment 10) the necessary condition H = 0 corresponds only to extremal points which may be minima, maxima, or

inflection and saddle points.

Sec. 6-19

We

Anexample

471

also remark that the free-time problem does not have an optimal

solution.

The reason is that if T = ©, then u(t) = 0 for all ?, and of course

the zero control does not take the initial state (£1, &) to (0, 0). It is true that we can find optimal solutions for the fixed-time problem for arbitrarily large (but finite) values of 7, and so reduce E* to an arbitrarily small (but nonzero) value. Nonetheless, the limiting solution cannot satisfy the boundary conditions, and so an optimal solution does not exist. However, an «optimal solution does exist in this case.

Let us continue our explicit discussion of specific control problems by considering the optimal control of the system (6-520) with respect to a different performance criterion. Our purpose in the remainder of this section is to exhibit the existence of relative minima in some optimization

problems. To be specific, we pose the following optimization problem. sider the system (6-520), i.e., the system

ii(t) = a(t)

z(t) = u(t)

210) = &

29(0) = €

We

con-

(6-549)

Let us also consider the cost functional

J(u) = kT +4 [wo dt = t [ke + 2u(t)]dt

(6-550)

where T is free and k > 0. We wish to find the control that transfers the system (6-549) from the initial state (£,, &) to (0, 0) and which minimizes the cost functional (6-550). Clearly, we are dealing with a fixed-end-point, free-terminal-time problem. The cost functional J(u) representst a linear combination of the elapsed time and the consumed energy. We can solve this problem in two ways. One method of solution is to develop the necessary conditions and proceed in a straightforward manner. The other method of solution is to solve the posed problem assuming that the time T is fixed, plot the cost J; as a function of the terminal time T, and then determine the particular terminal time for which the cost is smallest. If we attack the problem under the assumption that T is fixed, then the term kT in the cost functional J, is a known constant, and so the minimization of the cost functional (6-550) with T fixed is equivalent to the minimization of the energy 4 f u?(t) dt. For this reason, the control (6-535) is still optimal, and we immediately conclude that the minimum value of the cost functional is [see Eq. (6-536)]

Ti(by, be, T) = kT + 2, Bht + BGT + b'T) ft Compare with Problem 6-2g.

(6-551)

{72

Structure and properties of optimal systems

Clearly, the cost J1 1s a function of the terminal time 7. iicular value of the terminal time 7, say T*, such that

To find the par-

Ji(&1, &2, T*) < Ji(€1, $2, T)

(6-552)

we first determine the extremal points of J;(:, &, 7) by setting dd 1(€1, &2, T) =__ 0 —7T

(6-553)

2 _ 18h? | Wkiks _ 2het _ og

(6-554)

and we find that

where we have set

k=

B> 0

(6-555)

n order to simplify the equations. From Eq. (6-554), we find, after some igebraic manipulations, that the terminal times at which the function y(t, , 7) has an extremum quation

are the four solutions of the fourth-order

T* — p(&?T? + 681827 + 98:7) = 0

(6-556)

Let us denote the four roots of Eq. (6-556) by 7T,, 7s, Ts, and T,. By ‘aking the square root of both sides of Eq. (6-556), we find that these xtremal times are given by Ti = 3(B& + VW AE? + 1288) T;

=

4 (BE

—

V

Bté.?

+

T; = $(—Bé& + VBE?

12B£1)

(6-557)

— 128€,)

T. = 3(—Bk: — VBE? — 1288)

We wish now to isolate the pc.itive terminal times which correspond to the ocal minima of J:(&, &:, T). ‘thus, the requirement that

ee 023 (6,

£2, T)

ind that

> 0

(6-558)

T>0

(6-559)

‘an be used to show that the terminal time 7;, 1 = 1, 2, 3, 4, corresponds © 8 local minimum if and only if 0

- 37 62

2

Es

(6-560)

Sec. 6-19

Anexample

If 7; is such that

o0 (6-566)

or if

f&

ee

(6-567)

474

Structure and properties of optimal systems

then only the time 7’; corresponds to a minimum; it follows that the optimal control is unique and that the terminal time is 71.

2. If

& O0

&: < 0

and

(6-568)

& 2

x,

The y switch curve

Fig. 7-13 The switch curve the two-time-constant plant.

switch) from u =

—1 occurs at they curve.

this, we again consider the four control sequences {+1}, {—1}, {+1, —1}, and

| %2

—1 tou =

for

—1), respec-

tively, without passing through (0, 0); the control sequence {—1, +1} will generate trajectories which do not hit the origin; among all the trajectories generated by the sequence {-+-1, —1}, there is one and only one which arrives at the origin, and this trajectory can be generated if and only if we apply the control u = +1 until the +1 forced trajectory intersects the y— curve, at which point we must apply

u =

—1.

Similarly, for every state

in R_, the time-optemal control sequence 18

{—1, +1} provided that the transition (or +1 occurs at the ys curve. We now summarize

the results in the following time-optimal control law:

Control Law 7-4

Solution to Problem 7-4

a function of the state (x1, x2), ts given by u*

= u*(x1, £2)

u™

=

u*(a,

© 2)

= +

for all

(a,

The time-optimal control, as Ze)

for all (zs, m2)

[where V4; R,, Yor and R_ are given by Egqe.. (7-97), ee

a

=

V+ U

Ry

y-UR_. (7- 101),

‘1 308) (7-98),

and

Sec. 7-8

Plants with two time constants

533

An engineering realization of the above time-optimal control law is shown in Fig. 7-14. The state variables z;(t) and x2(¢) are measured (or computed). The 2,(é) signal is introduced to the nonlinearity N, whose output is

a(t) =

(G+ ele —

(7-104)

The signal x2(¢) is subtracted from a(z) to form the signal b(#) = a(t) — 22(t). If b(é) is positive, then (21, x2) € R,; if b(t) is negative, then (x1, 72) € R-;

therefore, the signal b(t) has the correct polarity, and hence it may be used to drive the ideal relay R whose output u*(¢) is the time-optimal control. a(t)

alt

7

+

N

b(t)

—_

Xo(t)

1

“LT

J,

u(t)

x,(t)

>»

Plant

~

ant | ep fe)

R

’

y

a(t}= ay [t+ ald =] Fig. 7-14

An

engineering realization of the time-optimal

constant plant. We

system

for the two-time-

shall now demonstrate another advantage of using 2,(¢) and z2(t) as

state variables. Recall that we originally defined the state variables y,(t) and y2(t) by Eqs. (7-74), that is,

ynQH@@=y¥H

yt) = 7

(7-105)

In other words, we could use the output y(é) and its derivative y(é) as the state variables. From Eqs. (7-80), (7-82), and (7-84), we find that 7,(é) and z2(¢) are related to yi(t) and y2(t) by

ni) = Z[-ddan(®) + Ol 1 zat) = 7 [—dadayn(d) + Maya)

(7-106)

Figure 7-15 illustrates, in block-diagram form, the linear transformation necessary to obtain x,(¢) and z(t) from y,(t) and y2(t). Now let us substitute Eqs. (7-106) into Eq. (7-100) in order to obtain the equation of the 7 switch curve in the y172 plane; the 7 curve is then given by 1

K (—Arvayi + Asye) 1

| RK (—Arrays + Arya)

1 (—rrdenn

Fe

AT

A

1

+d

1Y2

)| iE

+ Faqs

+ Ary2) | t - |

(7-107)

534

The design of time-optimal systems

It is impossible to manipulate Eq. (7-107) (for arbitrary values of «) to

obtain y2 as an explicit function of y:, or vice versa.

This implies that if we

measure the state (y1, y2) and we wish to test whether or not the state (y1, ys) is on the switch curve, we cannot construct a simple nonlinearity into which

dy yt) or y(t)

|

1

>|

"

>} M

_

1

Alf) ory(t)

a(t) xolt)

—

he

>

Fig. 7-15 Block diagram of the linear transtormation from the y; and y: variables to the 2, and 2x: variables.

*2 L3 2

/

/

fs

X =(4,, XQ)

=(w,,

w>)

!

j7-

/

Fig. 7-16

/

Several time-optimal trajectories for the two-time-constant plant.

we could introduce (say) the signal y; and compare the output with y: to decide what the time-optimal control should be.

Now we shall attempt to evaluate the minimum time ¢* required to force any initial state (1, 22) to the origin (0, 0) using the time-optimal control

law provided by Eqs. (7-103).

Consider an initial state X = (z1, z2), as

shown in Fig. 7-16, and the time-optimal trajectory XWO to the origin, where W = (wi, w2) is on the y curve. Let u = A* = +1 be the control

Sec. 7-3

Plants with two time constants

applied during the trajectory WO. W

to O, and let

Let t: be the time required to go from

t, be the time required to go from X to W.

Eqs. (7-94) and (7-96), we deduce that 1

A*

NB

0=

—A* A

l

Ea

Then, from

A*

8 aoFP A*

7-108)

@

+ (ws + a)* (95)

(7-109)

A* = — sgn {wi} = — sgn {we} _i1 w—A* i= 1 log ™1—-o

1 = Xe log W4i—

WwW.

=

A*

+

(x2

535

_

A*)

(7-110)

we — A* tv. —_— A*® Af A*

(7-111)

a

(2=5) t107 A

(7-112)

because, during the trajectory XW, we have u = —A*.

Let

t* = t) + ts

(7-113)

Clearly, ¢* is given by t*

oo

1

x, og (=

A*

Tat

Wi

A*

zo

)

_ (7-114)

We want to find ¢* as a function of z; and x2 only, and so we must eliminate w, from Eq. (7-114).

Combining Eqs. (7-109) and (7-112), we find that

p= asf es —an(@ZS) (a) as —_—

A*

a

A*

a

which provides us with a relationship among w,, 11, and +2. Unfortunately, for arbitrary values of a, it is impossible to obtain w, as an explicit function

of z; and x, from Eq. (7-115).

sion for t*.

Thus, we cannot obtain a closed-form expres-

However, for specific values of \; and dz, and hence of a, we

could obtain the value of é* from Eqs. computer.

(7-114) and

(7-115) using a digital

Exercise 7-22 Let A; = —1, \2 = —2. Then a = 2 and Eq. (7-115) reduces to a quadratic; solve for w: in terms of z; and 2: and obtain an analytic expression for é*. Then obtain the equations of the minimum isochrones (see Definition 7-6) and plot the

minimum isochrones for the values t* = 0.5, 1.0, 1.5, and 2.0. you have derived

Exercise 7-28

|u(z)| 0}

Definition 7-11

(7-146)

Let {V2} denote the union of {V;+} and {V.-}.

Using

A* = — sgn {z;3}

we find that

(7-147)

{Vs} = {(x1, Ze, 23): 21 = $A*22? — 22,23 = A* — A*e—4*2s} = {Vt}

U

{Ve}

(7-148)

It follows from the above definitions that {V3} +s a curve tn the state space which has the property that any state on {V2} can be forced to the origin by application of the control u = A* = — sgn {23} (7-149) The projections of {V2} onto the x:72 and 227; planes are shown in Figs. 7-20 and 7-21, respectively. 1"3 2

5

4 %2 Projection of

2t

Ve}

—-

-2

-{

17

0

a

-2. —}——-_—_+

1

2

Se

=

> Xx,

~1+ >

-

{v2}

0

—}

|

Projection

of {V 3}

ae

Projection of

i

i

+——_—_»> X

2°

} —{

1-2

Projection of

{v2}

Fig. 7-20 The projection of the set (or curve) {V3} on the 2:22 plane.

Fig. 7-21 The projection of the set (or curve) {V2} on the 2223 plane.

Consider the set of all states which can be forced to the origin by the control sequence {—1, +1}. It is clear that the transition from the control u = —1 to the control wu = +1 must occur on the set {V:+} (since {Vt} is the set of ali states which can be driven to the origin by application of the control u = +1). If a state (z;, x2, 23) can be forced to the origin by the control sequence { —1, +1}, then there is a state (x12, x22, a2) © {Vat} such that the state (x1, x2, 73) lies on a trajectory, generated by u = —1, which terminates at the state (x12, 722, 232) GC {Vat}. We generalize these considerations in the following definition:

Sec, 7-4

_A third-order plant

Definition 7-12

543

Let {V1} denote the set of states (x1, X2, 23) which can be

forced to the origin by the control sequence { —A*, A*}.

Thena

€ {V,} ef and only if 12

=Zi-

230 =

(x22

_

t2) —

4A* (X22?

(43 + A*)e4* (sx)

where

(x12,

%22,

Ls2)

Ee

—

state (x1, x2, 3)

X27)

(7-150)

— A*

(7-151)

{Vo}

(7-152)

that 1s [from Eq. (7-149)], Zig = $A* x22? — Zee Zag = A* — A*e—4"* A* = — sgn {Ze}

and

We form from From

(7-153) (7-154) (7-155)

note that states belonging to the set {V,} are given in parametric in terms of states (212, 222, 232) in {V2}. We can eliminate 213, 222, Xa: Eqs. (7-150) to (7-154) to obtain a relationship between 21, v2, and 23. Eqs. (7-150) and (7-153), we find that Yoo =

+ |

+ A*(x1 + 22) |

(7-156)

To eliminate the ambiguity of sign in Eq. (7-156), we use Eq. (7-155), and we then find that

t= —A* | + A%(a1 + x) |

(7-157)

From Eas. (7-151) and (7-154), we deduce that

Lg = —A* + A*e—4%(e:-3) (2 — e—4*en)

(7-158)

Substituting Eq. (7-157) into Eq. (7-158), we obtain the relation t3 = —A* + A* exp {arn + E

+ A*(x, + ») |}

x (2 — exp |

+ A*(x; + 2) I)

(7-159)

This equation provides a relationship between the variables x;, 22, 73, where (41, T2, 23) € {Vi}, and A* = +1. It remains to determine A* as a func: tion of %;, %2, and x3. Figure 7-22 shows the projection of {V2} on the 112%2 plane; the equation of this projection is

2, = —} sgn {r2}xr2? — xe

(7-160)

We observe in Fig. 7-22 that the trajectories which belong to {V,:} and which terminate at {V.+} are generated by A* = +1. On the other hand the trajectories which belong to {V1} and which terminate at {V.—} are

generated by A* = —1.

This means that if a state (a1, x2, 23) belongs to

544

The design of time-optimal systems

{V3} and if the two coordinates z; and z, satisfy the relation % > —} sgn {x2}22? — 22 then A* = +1. relation

If, however,

then A* = —1.

It follows that

t
0

(7-256)

or, equivalently, that

ws > [2(wits — 1)] is t Since N = 3, the hypersurface

(7-257)

{V;} is a surface in the three-dimensional state space.

0202

The

design

of time-optimal

systems

The form of Eqs. (7-254) and (7-257) suggests the definition of a function

g(w1, 8) = [2(wit — 1))!"

(7-258)

where this function g(w1, 3) can be viewed as a function of s, with w; > 1 as a parameter. By definition, we must have

g(wi, 81) = g(w1, 82)

and so we must show that

g(w1, 8&3) < g(201, 82) It is easy to establish that

for 83 > 82

lim g(w1, 8) =

lim

(7-260)

0

e0 o—?

(7-259)

(7-261)

g(w1, 8) = wi > 1

(7-262)

@

and that

[2(w.t — 1]!

2

£ g(t

8?(w," — 1)

Since w, > 1 and s > 0, the term

positive.

(sw, log wi — (wi — 1) log 2(w:" — 1)]

(2(w:* — 1))!/*/s%w,* — 1) in Eq.

We assert that there exists only one s = § such that

(7-263) is always

$w,4 log wi = (wi? — 1) log 2(wi4 — 1)

which means that

(7-263)

(7-264)

d

(7-265)

qq Iu 8) | _, = 0

The above considerations are sufficient to enable us to obtain the shape of the function g(wi, 8). As illustrated in Fig. 7-28, the function g(w., s) as a function of s starts at zero, 4 glm, s) = (2(wF-1)]

iV

attains a maximum at s = §, and then drops

7

off, approaching w, asymptotically ass— ©. It is clear from this illustration that for any

83 > 83, g(t, 8&3) < g(t, 82). Exercise 7-31 Consider the transfer func-

g (wy, 4) = 9(m, So) = we We

—

W,

ji}

14

7 1 1

0

Ss,

Fig. 7-28

'

tion given by Eq. (7-241), and let 3, = 1, 8. = 2, 8s; = 3. Repeat the steps involved in Example 7-2 and show that w: = 3,

79 mv 53) 4

7 |

! ! boy

4

1

;

$

So

i

53

ws

—~

S$

Plot of the function g(wi, 8)

versus 8.

= 4,

23,

=

—11.

In

this manner,

you

have verified that the point (1, 1, 1) is above the surface {V:}, since the point (1, 1, —11)

belongs to {Vi}.

Plot the function

g(3, 8) = [2(3* — 1)] forgea
w/w.

Fig. 7-42. by a nonlinearity which is simpler to construct. Since the output characteristics of the nonlinearity N of Fig. 7-42 are identical switch curve, one may consider the problem of suboptimal control problem of approximating the switch curve. The two examples illustrate two such approximations. Example 7-5

inputto the as the below

Recall that the definition of the y switch curve given by Eq. (7-325) was y=(U

Let us consider the curve

y=0

vA

ULU

ga {TeUrt wt, = 0

x]

(7-329)

if last 2

(7-331)

The ¥ curve is illustrated in Fig. 7-46, and it approximates the y_/ and y,/ curves, j =1,2,..., by the wz, axis. We use the controlu = —1 for every state above the 4 f @WXo

u

*®

=

{

fo

4

.

Fig. 7-46 The switch curve + for the suboptimal system of Example 7-5. The control igs u = —1 above the switch curve and u = +1 below it. The dashed curves indicate the trajectories generated by this suboptimal control law. curve and the control u = +1 for every erated by this suboptimal contro] law clear that this suboptimal control law words, it is a stable control law. The engineering realization of this Fig. 7-42 except that the nonlinearity

state below the 4 curve. Typical trajectories genare shown as dashed curves in Fig. 7-46. It is will force any initial state to the origin; in other suboptimal contro! law is identical with that of N of Fig. 7-42 is replaced by the nonlinearity V

with the input-output characteristic shown in Fig. 7-47. is easier to construct than the nonlinearity N.

In general, the nonlinearity V

Sec. 7-7

The harmonic oscillator

587

Whether or not this suboptimal system is good can only be decided by plotting the

isochrones using the 4 curve and comparing them with the minimum isochrones of Fig.

7-43.

In order to provide an idea of the difference of the time i required by this suboptimal

design and the minimum time ¢*, we present Fig. 7-48. In Fig. 7-48, the percentage of

increase in the response time when the initial states are on the wx; azis is plotted.

~

©%_.|

As is

é

| A(t)

input-output

charac-

“@ 17

evident, there is no difference when |{w2z,| < 2,

lwr,| = 4, |wo:| = 6, .. . , because in these

N

regions the + curve is identical to the y curve. It is also clear from Fig. 7-48 that the maximum percentage of degradation in time is

Fig.

7-47

The

teristics of the nonlinearity N required to generate the suboptimal Example 7-5.

of the order of 4.3 percent, and for points

on the wz; axis sufficiently far from the origin,

system

of

the time ? approaches the minimum time ¢t*. This is a consequence of the fact that, for initial states far from the origin, both the minimum isochrones and the suboptimal

ms I>

isochrones tend to the same circles. For many engineering systems, 3 maximum increase in response time of 4.3 percent is quite acceptable, and so this suboptimal design is often useful. x100%

5%

O|

0

+ {

Fig. 7-48 Example 7-6 switch curve:

LS v

2]

3

v

4

5

6

= |u|

Plot of (é/t*) * 100 percent versus |wz,|.

Suppose that we use the following curve 7 as an approximation to the 7 ¥=

v42° U y-9 {=.= —1 wre

=

+1

if |wa,| < 1 if wr1 > 1 if wri


way

7 7

/

7

/

P

Fig. 7-49 The switch curve ¥ for the suboptimal system of Example 7-5. The control is u = —1l above 7 and u = +1 below 7. The dashed curves are trajectories generated by this suboptimal control law.

Exercise 7-87 If Zis the time required to force any initial state on the wz, axis by the suboptimal system with the nonlinearity N, plot (Z/t*) 100 percent versus |wz,| and construct a figure like Fig. 7-48.

than the maximum of (#/t*)

Show that the maximum of (f/t*) < 100 percent is larger

100 percent.

Exercise 7-88 Show that, if the y curve is approximated by the wz. axis (in other words, the suboptimal control law is u = — sgn {wz3}), then any initial state cannot be forced to the origin but rather can only be forced to the closed segment |wz:| < 1, wr. = 0.

TY

a

{

|

-

_

- ———-

=a

Exercise 7-89

me — Be

[w2,}} will result ina

limit cycle through the points (0, 1)°and

(0, —1).

Exercise

N

Show that the control law

{wee + sgn

7-40

Given

the

harmonic-oscil-

lator plant with the transfer function

Fig. 7-50 The input-output characteristica of the nonlinearity N required to generate the suboptimal 7 switch curve.

1 G(s) = s?+1

. Let u(t), |u(t)| 0 and |r| 0d

o> 0

(7-334) rr

(7-335)

In servomechanism practice, the system (7-333) is described by the transfer

function

y(s) _ G(s) = or

a

To

The experienced reader will recognize that the poles of G(s) conjugates at s=

—atjw

(7-336) are complex (7-337)

Figure 7-52 shows the s-plane configuration of G(s). The constant w is sometimes called the natural frequency, and the constant a, the damping. t See Refs. B-24; C-3, pp. 230-234; and K-18.

Sec. 7-8

The damped harmonic oscillator

Let

nO=y¥O™

be a set of state variables.

yw) = 9

591

(7-338)

The y;(t) satisfy the vector differential equation

0

yl) | 7 = Bal

yi(t) al

1 Je

+ w?)

0 oa |

+

It is convenient to define a new set of state variables 2,(¢) applying a suitable linear transformation, namely,

(7-339) and 2.(t)

nl@al _«K 1fe|: 0)[ |

zm]

ae |

by

(7-340)

The x variables satisfy the vector differential equation Z(t)

0 Exercise 7-47 Exercise 7-48

w

—a

*

~ =%

21(t)

A

0

(7-341)

+ BA

Verify Eq. (7-341). Show that if the spinning body of Example 7-4 is subjected to linear

friction, then the differential equations of the velocities have the same form as Eq. (7-341).

We note that if in Eq. (7-341) we set a = 0, we obtain the differential

equation for the undamped harmonic oscillator [see Eq. (7-301)]. Let us examine the system matrix in Eq. (7-341); we note that —w

Jefe Je[2 ]

—-a@

Q

-—-—a

—w

0

coe

In other words, the system matrix is the sum of a diagonal matrix (with elements the real part of the complex poles or eigenvalues) and of a skew-symmetric matrix (with elements the imaginary part of the poles or eigenvalues). Now we formulate the time-optimal control problem for the damped harmonic oscillator. Problem 7-8 Given the system (7-341), determine the control u(t), subject

to the constraint |u(t)| 0) the function p,.(t) is the product of an

increasing

exponential

0

7

>

and

a sinusoid.

/

aN

™

/

N

/

\

slew \__UZ

Figure

7-53

\f

\

a

\

\

‘NU

illustrates

\S1r /

a typical

oe

wwe

\

u(t) =—sgn{ pol t)}

Fig. 7-53

. .

A typical function p.(t) and the control generated by Eq. (7-344).

function p2(t) and the control «u(¢), defined by Eq. (7-344) as a function

of p2(t).

Observe that the control u(t) has the following properties:

1. It must be piecewise constant and must switch between the values

u=-+landu

=

—1.

2. It cannot remain constant for more than +/w units of time. 3. There is no upper bound on the number of switchings.

4. There is no possibility of singular control. Note that the above necessary conditions on the time-optimal control are identical to those in Sec. 7-7.

t See Eqs. (2-109) to (2-111).

Sec. 7-8

The damped harmonic oscillator

593

We now proceed to the solution of Eq. (7-341) using the control

uw=A=

If we define

+1

E, = 2,(0)

(7-351)

to = 22(0)

(7-352)

then we find (after extensive algebraic manipulations) that w? +

a

21(t)

=

w? +

cos wi +

a? £6

a” foe!

+

al

sin wt

— a VOT gat sin (wot +9) +A ot

G)

ro(t) =

— a? + a! £,e-* sin wt + ont a

Q

—A Var to where

y=

(7-358)

ot foe cos wt

ew cos (wt + y) + A=

tan-} =

(7-854) (7-355)

We note that lim

t—

z(t)

=

A

on lim 22(é) = A-

t+ ©

The equations

(7-356)

w

(7-353) to (7-355) describe logarithmic spirals which tend

.

a

a?

+

to the point (., A *) of the (ete @

aw?

v1,

a?

+ @

w?

r1) plane.

Unfortunately,

owing to the transcendental nature of Eqs. (7-353) and (7-354), we cannot eliminate the time in order to obtain the equation of the trajectories. Nevertheless, we can proceed to the following definition: Definition 7-28

Let 7° denote the set of states whitch can be forced to the

origin by the control u = A = +1 tn no more than x/w seconds. Let y_° denote the set of states which can be forced to the origin by the control u = A = —1 in no more than x/w seconds. Clearly, the y.° and y—° curves provide paths to the origin which satisfy all the necessary conditions for optimality. The

shape of the y,.° and y_° curves is illustrated in Fig. 7-54.

Using

similar arguments to those of Sec. 7-7, we can indeed demonstrate (by the familiar technique of elimination) that the 7° and y_° curves are time-optimal paths to the origin. We proceed by using reasoning identical to that used in Sec. 7-7 in order

to generate a series of curves y,/ and y-/ forj = 1, 2,.... We define the y_! curve as the set of all points which can be forced by the control u = —1 to a point on the 7,° curve in ezacily +/w seconds. Similarly, we define the +! curve as the set of all points which can be forced by the control

u = +1 to a point on the y_° curve in exactly x/w seconds.

Thus we have:

394

The design of time-optimal systems

Definition 7-29

The vy curves, j =1,2,...

, are the sets of points which

can be forced by the control u = —1 to the y47-! curve in exactly 1/w seconds. The y+’ curves, j = 1, 2, ... , are the sets of points which can be forced by

the control u = +1 to the y_*—' curve in exactly 1/w seconds. The y+’ and y_/ curves are shown in Fig. 7-54. w

#2

u*=-1 } o

1

"e--~ 2

yE -/ S/

S

~Y

7s.

3

a)

\l

fr

_&a

/ |

Fig. 7-54

OF

Zs

a

1

0

/

ae

g

ey,

/

NS

~~”

\

Js

we Way

~s

Ww

+ u*= +1

The y switch curve for the damped harmonic oscillator.

control is u* = switch curve.

Definition

—1

for states above

7-30

The

curves; in other words,

the switch curve and u*

y switch curve is the union

=

The time-optimal

+1 for states below the

of all the y45 and

= | U rs} U U 3} Control Law 7-8

Solution to Problem

7-8

y-!

(7-357)

The time-optimal control as

2 2 2 2 a function of the state (“4Te 1, — t < 21) ts given by: If (= + at If (-t= + a

oe + a 1) as above the y curve, then u* = —1\ az

If (“+ +r On 7

' a 1) ts below the y curve, then u* = =

2)

+1.

belongs to the y+° curve, then u* = +1.

If (ie + at 21, ws . a 2) belongs to the y_° curve, then u* = —1. Let us now compare Figs. 7-54 and 7-40 and examine the significance of the constant a. It is easy to see that if a— 0, then (the angle) 6— 0 in Fig. 7-54 and the y curve of Fig. 7-54 will become identical to the + curve

Sec. 7-9

Harmonic oscillator with two controls

595

of Fig. 7-40. -However, for a > 0, the size of the y,/ and y_/ curves increases with increasing integer values of j; this, of course, is the natural consequence

of the exponential term e~** which governs the sizeof the spiral trajectories.

Exercise 7-49 Construct approximations to the 7 curve for generating suboptimal control policies. (Hint: Look over Examples 7-5 and 7-6.) Consider the specific values w = 5and a = 1, and construct the exact > curve and the approximations you suggested. Evaluate the “goodness” of each of your approximations by plotting the percentage of increase of the response time over the minimum response time. Exercise 7-60 Consider the system described by the differential equation

el

~ |

| a

+ ay

(7-358)

{u(é)| 0,#>0. This system represents an unstable harmonic oscillator because the eigenvalues (or poles) are at a + jwand a — jw. First, determine the set of initial states

for which it is possible to reach the origin (since the system is inherently unstable, there are initial states for which the bounded control cannot provide sufficient force to drive these states to the origin). For this set of controllable states, find the switch curve and the time-optimal control law. H1nr: The 7,/ and y_/ curves for this system diminish in size for increasing values of 7; see also Ref. C-3, fig. 9-13.

7-9

Time-optimal Control of a Harmonic Oscillator with Two Control Variablest

In the preceding sections of this chapter, we examined the time-optimal control problem for dynamical systems with the common property that the control action was accomplished by a single control variable, u(t), subject to the constraint |u(é)| 1 ,21(t)

=

(I.

-

I3)a2(t)x3(t)

+

7 1(t)

T2%2(t)

=

I;

-

I,)z3(t)z,(t)

+

72(t)

I 3%3(t)

=

(I;

-_

I.)21(t)xe(t)

+

73(t)

(7-360)

where 7;(t), 72(t), 73(¢) are the torques about the 1, 2, 3 axes, respectively, and represent the components of a torque vector 71(t)

a(t) =

| 73(t)

(7-361)

Let us now examine the body illustrated in Fig. 7-55. Since the 3-axis is the symmetry axis, it follows that the two moments of inertia 7; and I, are equal; that is, I,

t Reference G-7, chap. 5.

=

q;

=

I

(7-362)

Sec. 7-9

Harmonic

oscillator with two controls

597

Now let J;, J2, and J; be three gas jets mounted on the body, as shown in

Fig. 7-55. It is clear that the gas jet J; produces a torque r;(t) about the 3-axis only, the jet J: produces a torque r2(é) about the 2-axis only, and the jet J: produces a torque 7,(¢) about the l-axis only. We assume that

the three gas jets are fixed on the body and that they can “fire” in both directions. Let c be the distance between the plane defined by the jets J, and J: and the plane of the 1 and 2 axes, and let cs be the distance from J; to the 3-axis (for the cylinder of Fig. 7-55, c represents half of its length and c; its radius). Then, if f:(¢), fe(t), and f3(¢) represent the thrusts produced by the jets Ji, J2, and J3, respectively, it is clear that 71(t)

=

cf :(t)

72(?)

=

cf2(t)

r3(t)

=

Caf s(t)

(7-363)

Using Eqs. (7-363) and (7-362), we find that the equations of motion (7-360) reduce to

tx() = Ft aaa) + FAO elt) = — 2 axdart) + $f(0

(7-364) (7-365)

td) = Fld Let us examine these equations for a moment. the three velocities have the values

xi(0) = &:

x2(0) = &

(7-366) Suppose

z3(0) = &3

that at ¢ = 0,

(7-367)

It is clear that if the thrust f;(¢) is known, then we may integrate Eq. (7-366) to obtain

x(t) = & + [

dar) dr

(7-368)

so that the velocity 2;(¢) can be controlled independently of the velocities

z(t)

and 2z2(t), and so 2,(é) can be treated

as a known

function of time.

However, since the quantity z(t) appears in Eqs. (7-364) and (7-365), the

velocity x3(t) does influence x(t) and z2(t).t

We shall examine a problem of time-optimal velocity control which is motivated by some physical considerations. Suppose that the body represents a reentering capsule and that a constant spin about the symmetry axis is required for aerodynamic stability considerations. We assume that the velocity z3(¢) has been brought to its desired value (say &3) using the jet J; only; but owing to the gyroscopic coupling, the body is now tumbling in space, i.e., the velocities z,(¢) and 2,(t) are not zero, and it is therefore t This is due, of course, to the familiar gyroscopic effect.

598

The design of time-optimal systems

desirable to reduce 2;(¢) and z2(¢) to zero in minimum time while maintain-

ing the z,(¢) velocity at its constant value &,.

Another example is provided

by a manned space station with artificial gravity generated by a constant

angular velocity about the symmetry axis; in such a case, the reduction of the other two angular velocities z,(é) and z(t) to zero in minimum time

might be required.t In both of the above situations, it is desirable to maintain 73(¢) at a constant level and to drive both 2z;(t) and 22(t) to zero. If we set

fs) = 0

(7-369)

in Eqs. (7-366) and (7-367), we find that 23(t)

=

£;

=

const

(7-370)

Now suppose that the thrusts f,(¢) and f2(t) of the jets J; and J; are bounded in magnitude by the relations

A@d| +/2w, are closed, convex, and regular (i.e., they have no corners).

The reader will notice that the isochrones of Fig. 7-63 have

properties similar to those of Fig. 7-40.

Finally, it is easy to establish that once the state has been forced to the

origin and in the absence of future disturbances, the controls u; = u: = 0 will keep the state at the origin.

fe

HIAPMONURIC

OSCLator

WIth

TWO

COoOnTrO!IsS

VUUY

_

("

C= a5

ax

=

TRO

GE,

t*= {7

\

) UT BR POSSI

K - “V * aS

-

> aa

4

2

Fs

/

X

nay

te

4

Fig. 7-63 The minimum isochrones are made of circular arcs. the switch curves.

The dashed curves are

In a manner analogous to that used in the suboptimal system of Exercise 7-538 Example 7-5, consider the following approximation to the 744, v4-, 7-4, and y— — curves:

fae = {TH +t

4

+

,

T+" 4

@r,

™

=

0

=

0

if 2k < wn 2k

={7-

—" "(wr

=0

if 0


2k

< 2k
ft

te

—1

Fig. 7-73 Plot of the time-optimal control u(t) which generates the time-optimal trajectory ABO of Fig. 7-72. Note that for all ¢ > ts the output y(t) and its derivative y(t) are identically sero.

Fig.7-74 Plot of the time-optimal control u(t) which generates the time-optimal trajectory CDEO of Fig. 7-72. Note that for allt > tz the output y(é) and its derivative y(t) are identically zero.

Fig. 7-74; at ¢ = tg, the state is at E = (0,

control is

u(t) = e-(t)

and

43(t) = —e—(t-ts)

—1).

Thus,

for ¢ > ég, the

t> tr

(7-547)

t> tr

(7-548)

so that the control for ¢ > ¢g is a decaying exponential, as shown in Fig. 7-74, and the trajectory for ¢ > tg is EO, which belongs to G. Exercise 7-67

Design an engineering realization of Control Law 7-12.

Case 2 Non-minimum-phase Zero; That Is, B 25} R_ = {(21, zs): tf (21, 23) € vy, then 22 < 24}

(7-552) (7-553)

We now assert that:

Control Law 7-12b If 8 < 0, then the control that forces any given state (x1, Xs), (v1, %2) & G, to the target set G in minimum time ts given by u* = u*(21,

t)

= +1

u* = u*(z1, 22) = —1

for all (x1, 22)

EC RLU

4

= for all (4, 22) C R_-U 7-

(7-554)

and, furthermore, this control ts unique. Exercise 7-68

Prove Control Law 7-128.

Figure 7-76 shows several time-optimal trajectories to the target set G. Up to now, there has been no significant difference between the plant with the minimum-phase zero (8 > 0) and the plant with the non-minimumphase zero (8 t* such that

ju(és)| = 1

(7-556)

The value of ¢; is given by the relation

he tt + 5 log lea(t*)|

(7-557)}

In the time interval t* < ¢ < t, the state variables satisfy the equations 21(t)

=

()

a2(t) = 22(t*)e Fe)

“stsh

(7-558)

{It might appear at first glance that, since 6 ft: is the closed path KPKPKP.-.-.-. m(t*)=0

2z:(t*) ¢, is the closed path EQEQEQ

the points # and Kare specified by E = (0, —1)

If

K = (0, +1)

---:.

Since

(7-562)

it can be shown that for 8 = —1, the points P and Q in Fig. 7-77 are

P=(2,-1)

Q=(-2,

+1)

(7-563)

634

The design of time-optimal systems

Figure 7-78 illustrates the control which causes this limit cycle.

Observe

that for 0 < ¢ < ¢*, the control forces the state to G; in the interval ¢*
¢, the control switches between +1 and —1 every T seconds, thus generating the limit cycles. If 8 = —1, one can calculate the time T in Fig. 7-78 to be exactly 2 seconds. eee) +1

b}--——

—_

-x2(t")

4

~|

Fig. 7-78

yp

—— ee =

-

_-_—

The ahape of the control u(¢) which forces y and ¥ to zero in minimum time

(t*) for the plant G(s) = (s + 8)/s*, 8 < 0.

The control u(t) is an increasing exponential

during the interval [é*, i,]; for ¢ > ¢, the control is piecewise constant and leads to the limit cycle of Fig. 7-77.

We have shown that if |z2(¢*)| + 0, then a limit cycle is unavoidable. Now let us suppose that at ¢ = t*, we have zi(t*) = 23(¢*) = 0

In this case, if we set

u(t) = 0

(7-564)

for all i > t*

(7-565)

then we find that a(t) = 2.(t) = 0

for all t > ¢*

(7-566)

y(t) = x(t) = 0

for all t > t*

(7-567)

which implies that as required by the problem. Now we are faced with the following problem: We want to drive y and y to zero in minimum time and to hold them at zero forever. Control Law 7-126

does indeed drive y and y to zero in minimum time, but owing to the lamit cycle, y and y cannot be held at zero forever; but 1f we are at the origin of the state plane, then the control u=0 will hold y = y = 0 forever. Therefore, the

origin ts the only element of G which simultaneously satisfies both objectives of the control problem. Thus Control Law 7-126 is not optimal in general, and

we must

determine

another

control law which

will force any

initial

state to the origin (0, 0) in minimum time. Next follow the familiar definitions of the switch curves, the sets in the state plane, and the optimal-control law. Definition 7-40 Let T, denote the set of states which can be forced to the state (0, 0) by the control u = +1. Let T_ denote the set of states which can

Sec. 7-12

Double-integral plant with a single zero

be forced to the state (0, 0) by the control u= obtain

—1.

030

From Eq. (7-533), we

Pe = [len 2s):21 = 5 r+ dass an 2 058 < of

(7-568)

r=

(7-569)

(es

Zo): 21 = 5 (3

— $73");2723 SO;B

The YT, and T'_ curves are shown in Fig. 7-79 (using 8 =

< o|

—1).

Let

reT,Ur

(7-570)

be the new switch curve. A +2 or

Ty

Ry

1

u*s+|

+

-2

+

-I

1

,

1

u=-t

Fig. 7-79

Definition 7-41

> x,

2

T_

-2

The switch curves Fr, and r_.

Let R', denote the set of states above the T curve and let Rt

denote the set of states below the T' curve; that is, Ri

=

(cs

X2):

I1 >

; (x2 +

424|22|);

B


{*

(7-685)

VOU

ihe

design

OI

time-optima!l

systems

It follows that the control u(¢) will be of the form

u(t) = > ai;(t*)eF(t-") ‘ t*

(7-686)

where the coefficients a;(#*) will be functions of u(é*), w(t*), . . . , wD (é*). The above considerations lead us to:

Remark 7-1 Given the two systems (7-665) and (7-676). y(é*) = 0. Then, for the system (7-665), the control u(t) =

» a(t*)eF—-) = i=

Suppose that

for all t > t*

(7-687)

and, for the system (7-676), the control

result in

u(t) =0

# forallt> ¢*

(7-688)

yt) =0

#£forallt> t*

(7-689)

We knowyf that a state vector x(é) for the system (7-665) can be defined

In terms of the output vector y(), given by Eq.

(7-678), the control u(t),

and its n — 1 time derivatives as follows:

z(t) = y(t) — Aou(t)

ae(t) = y(t) — how(t) — hyu(t) xa(t) = y(t) — hoti(é) — Aiu(t) — heu(?) oe

e#

®

@

@

@

@

@

e@©

@

@

8©

a(t) = yo-D(t) — hou

@®

@

@

8&8

@®

@

@®

e@

@

(7-690)

@®

@®

(2) — hut)

@

®

—

@

@®

@®

@

@

@®

@®

@®

@

- + + —ha_yu(t)

@

or, in the abbreviated form,

mi(t) = y™Q) — k=0Yu Oba

§=1,2,...,0 (7-691) ’

The differential equations satisfied by the state variables z,(¢) areft £1(t)

t2(t)

0

0

1

0

0

|

Ln—1(t)

0

0

0

n(2)

—~Go

~G@,

G2

-::;

0

8

* °°

0

Z(t)

0.

1

tn_1(t)

“Gag

~GAn-1

tn(t)

0

0

r2(t)

hy he +]... hn—1 he t See Sec. 4-10.

{ See Eq. (4-187).

/u(t)

(7-692)

Sec. 7-14

General results

651

The constants ho, hi, . . . , Ra—1, ha Appearing in Eqs. (7-690) and (7-692) can be foundf from the equations ho

=

ba

hn-m = bn — hn = bo —

n-m—l1

)

hidim

t=0 n—l

y

+=0

m=1,2,...,n—1

(7-693)

ah;

Suppose that at some time ¢ = ¢*, the vector y(é) is zero, that is, y(t*) = 0

(7-694)

and suppose that it is desired to find a control such that y(t) = 0

for all ¢ > ¢*

(7-695)

Substituting Eq. (7-695) into Eqs. (7-690), we find that a(t)

=

—hou(t)

ae(t) = —hot(t) — hiuté)

for all t > ¢*

ta(t) = —hou®-Y(t) — hut)

— + + + — An-ru(t) (7-696)

Since

zi(t) = —hout(t)

for all ¢ > ¢*

(7-697)

we conclude that

u(t) = — > x(t) 0

for all t > ¢*

(7-698)

Substituting Eq. (7-698) into Eq. (7-692), we find that Eq. (7-692) reduces to the form

x(t) = Qx(é)

for allt > ¢*

(7-699)

where Q is an n X n real matrix defined by

Fm_ fs

pg eean

he he

QH=]...

1

0

ee te tt te ee _

i

t See Eqs. (4-189).

0

7

he

ha-1

he

@o

0

0

1

—A1

—@s

On

(7-700)

|

602

The design of time-optimal systems

Note that Q is a known matrix since we can compute fo, fi, . . . , An from Eqs. (7-693) in terms of the known constants do, ai, . . . , Gn—1 Ad do, by, . , bs. Equation (7-699) is a linear homogeneous vector differential

equation, and we may write the solution x(é) in terms of the vector x(¢*)

[since Eq. (7-699) holds for ¢ > ¢*].

The solution is

x(t) = e2#")x(¢*)

for all t > ¢*

(7-701)

where e®@—*) ig the fundamental matrixf of Eq. (7-699).

Let q(é) denote

the first row vector of the fundamental matrix e°¢-* and let qi(é), 92(é), . ; a(t) denote the components of the vector q(é); thus, q,(¢) is the

element of the first row and the ith column of the matrix e@).

for ¢ > t*, we have

a(t) = (a(d, x6*)) = YS gence)

Thus,

(7-702)

t=}

and from Eq. (7-698), we find that

u(t) = —— ly> q(t)2ct*).

t>t ‘

ho

(7-703)

Remark 7-2 Given the system (7-665) or tts equivalent state representation defined by Eq. (7-692). Given that at t = t*, we have

y(t*) = 0

(7-704)

If, for all t > t*, we apply the control

u(t) = -> Y gultdastt*)

(7-705)

y(t)

(7-706)

tm]

then

= 0

for allt >

t*

Let us next investigate briefly a very important property of the matrix Q. Lemma 7-1 The matriz Q of Eq. (7-700) has the property that

det (MI — Q) = (a + tye + ve + Fn +B) (1)

(7-707)

which implies that [see Eqs. (7-672) and (7-673)] the etgenvalues of Q are at

N= -B

im1,2,...,7

(7-708)

ELEMENTS OF PROOF The fact that the eigenvalues of Q are the zeros of the transfer function H(s), given by Eq. (7-671), can be verified by direct computation of det (AI — Q) using Eqs. (7-693). There is still another way to prove that the eigenvalues of Q are —8:,

pose that the eigenvalues of Q are —71, t See Sec. 3-20.

—72,

—B2,

..-,

...,

—Ya-

—Ba.

Sup-

Then, in Kq.

Sec. 7-14

General results

653

(7-702), we can write 7:(¢) as a sum of exponentials of the form

x(t) = ) &(t*)er =k

>

inl

where the constants 6,(t*) are functions of 2,(t*) and known From Eq. (7-698), we can write

u(t) = }) e(t*)eme

tt > tt

i=1

But (see Remark Eq. (7-687)]

(7-709) constants.

(7-710)

7-1) we know that y(t) = 0 for ¢ > &* implies that [see

u(t) = y a,(t*)eAtt-")

st > et

cml

(7-711)

and from Eqs. (7-710) and (7-711), we conclude that vi = B:

¢=1,2,...,”n

(7-712)

Let us now formulate the following time-optimal control problem: Given the system (7-665) and given that the control u(t) ts restricted by the

relation

lu(é)| ¢* requires

vector y(t) ts driven to zero in minimum time, say t*. vector is held at zero thereafter. we have found a control such that y(¢*) = 0, where ¢* is time. From Remark 7-2, we know that y(t) = 0 for all

~ 1S await he 2 qs(é)z;(é*)

u(i) =

= — 7 (a(, x(*)) 0

for allt > t*

(7-714)

But since |u(t)| < 1 for all ¢, we must also have

| - 5. a0, x(t*))| ¢*

(7-715)

Let G denote the set of states x such that

a= |x:

- = a), 2) |

(7-716)

We shall call the set G the target set. Lemma 7-2 The target set G, defined by Eq. (7-716), ts closed and convex, and the originx = 0 ts an element of G.

034

The design of time-optimal systems

ELEMENTS OF PROOF First, we shall prove that G is closed. Recall (Definition 3-7) that a set G is closed if every limit point of G is an element of G. Consider a sequence x, inG,n = 1, 2, .. . , that is,

_ * (q(t), x.)| ¢*

(7-717)

such that x, converges to x, that is, x,—X

(7-718)

Suppose that the limit point x is not an element of G (which implies that G is not closed); then there exists a time ?, ? > ¢*, such that

| _ ~ (qé),x)|>1+e¢ 0

But

forsomee>0O

(7-719)

| — 7, a,x) -|-@@,x-x)+ (- x) (a(), x.) 0,s2>0

(7-725)

We must show that

(7-728)

for allt > ¢*

| 7 = 0 a), Xi + 8X:)) $1 But Eq. (7-726) is true because

— = ato, 0

(7-731)

This means that one of the eigenvalues of the matrix Q is positive; hence the system (7-699) is unstable. Since the solution of (7-699) will involve

656

The design of time-optimal systems

the term e~*:(-*"), we have

lim ||q(é)|| = ~

(7-732)

— ao

which implies that if x + 0, then

lim | — 5- (a), x)| = © {—>

(7-738)

0

If, however, x = 0, then

| -@o, 0)| =0

forallt>?*

0

(7-734)

and Eq. (7-730) follows from Definition 7-43. Now suppose that all the 6; are real and positive, which means that the transfer function H(s) has only minimum-phase zeros; therefore, if 0 ¢*.

a at

— 5a,»

t=?

If?> ¢*, then

= 0

(74738)

Moreover, the maximum value is 1, since x is in 0G, so that

-7,40, | =1 t See Sec. 3-26. f See Sec. 3-10.

(7-739)

Sec. 7-14

General results

657

Equations (7-738) and (7-739) can be used to “eliminate” ? and obtain an

equation for the components of x. Thus: Remark 7-4 If Eq. (7-735) holds, then a point x belongs to 0G if 2 satisfies Eqs. (7-738) and (7-739). The reader should be warned that it is, in general, impossible to solve explicitly for 7 in terms of x from Eq. (7-738) and then substitute in Eq. (7-739). Thus, the determination of the target set G is an extremely difficult affair, and of course one cannot explicitly solve the time-optimal problem without the algebraic equation of the boundary of G (why?). Up to now, we have assumed that the number of zeros is the same as the number of poles, i.e., that b, # 0

(7-740)

In the remainder of this section, we shall define the target set G for the case where bn =

bn

™

°°:

=

bey

=

0

k=0,1,...,n-—1

(7-741)

This means that the system is described by the differential equation {D" + an.D*1 4+ -- + + a,D + anf y(t)

= {b,.D* + bh-sDP3 + - + + +01D + dofu(t)

(7-742)

Substituting Eq. (7-741) into Eqs. (7-693), we find that ho=hi=he= +++ = hati =O An»

= by

hom = bn —ha

=

bo

—

n—-m—1l1

DY

Iidiem

t=n—k n—-1 y t=n—k

m=i,2,...,k—1

(7-743)

ash,

which implies that Eqs. (7-690) reduce to x(t) = y(t) 19(t)

tn+(t)

=

a(t)

= yo--D (i)

(7-744)

Ta—e_1(t) = y(t) — Anu) Tn—z—2(t) = yO FD)

— hypatt(t) — hoarse?)

ta(t) = y*-O(t) — Anau®-Y() —

> + — ha-re(d)

008

The design of time-optimal systems

The state variables satisfy the differential equations £1 (t)

=

x2(t)

Z(t)

=

23(t)

in ait) = 2-a(0) Ln—z(E)

=

Ln—44i(b)

+

ha—-xu(t)

(7-745)

En—nti(t) = Sn_ese(t) + An—eiru(t) n—l

Z(t) = — Setting

WO ==

in Eqs. (7-744), we find that y(t) = z(t) =

s+ =P] =0 - °°

ta-esi(t) = haxu(t)

y a:2i41(t) + hau(t) i=0

td

= rnx(t) = 0

(748)

for all ¢ > ¢*

for alle > e* (7-747)

Thus, if we substitute

u(t) = ~~ anal)

forallt>

ha

(7-748)

into the last k equations of Eqs. (7-744), we find that =

le

-

—

Sn—n+2(t)

_

|

}

-

aai(t)

_ fects te

Z(t)

|

ae

,’

Ce

wh

— fy TO

1

0

0

1

ww

he

Tn

ww

‘es

ee

we

nga

ww

ww

=

0

en aar(t)

0

Ln—e+2(t) we tf

Ont

ww

Zn(t)

for allt > ¢* (7-749)

which is a homogeneous vector differential equation of the form

x(t) = Ox(t) ~—s for¢ > ¢* where

£(¢)

is the

k-dimensional

vector

whose

components

are

the

last

k components of the vector x(t) and O is the k x k matrix appearing in Kq. (7-749). We claim that

det (AT — 6) = (ne Pt yet ge by

4 Pty 4 PO) (aye by

by

so that the k eigenvalues of 0 are the zeros of the system (7-742). as before, we find that

Tn_ezi(t) = (A(t), £(¢*))

for all é > ¢*

(7-750) Working

(7-751)

Sec. 7-14

General results

where 4(¢) is the first row vector of the fundamental matrix e8¢—.

659

From

Eqs. (7-751) and (7-748) and from the relation |u(t)| < 1, we find that the

relation

| — 55 4, £0) | ¢*. Definition 7-45 The target set G for the system (7-745) ts

G = {x - + ao, ®

t|

(7-753) The set G defined above zs also closed and convex. The fact that the dimenstonality of x 18 less than that of x helps somewhat, espectally if k = 1 ork = 2. Example 7-11

If we have k=l

(7-754)

then Eqs. (7-747) reduce to

and

zi(t) = a(t) =

--

+ mw agi(t) = 0

for allt > ¢*

ra(t) = —ha-ru(é)

(7-755)

(7-756)

But if k = 1, Eqs. (7-743) reduce to hye mh,

=

oa

= hag

han = by ho = bo — Gn-ib1

=

0

(7-757)

so that from Eq. (7-756) we obtain the relation

u(t) = — 5 mal)

(7-758)

Let us briefly summarize the results of the last three sections. We found that the time-optimal control of plants with minimum-phase zeros in their transfer function is considerably more complex than the timeoptimal control of plants with no zeros. We also noted that before we can find the switch curves or the switch hypersurfaces, we must find a target set G in the state space and reduce the time-optimal control problem to driving the state to G in minimum time and then applying a control which forces the state to remain in G thereafter. If the system is of high order and if the number of zeros is large, the analytical determination of the boundary of the target set G is an extremely laborious task, if at all possible. If, on the other hand, the plant contains a single minimum-phase zero, the target set G, as indicated by Example 7-11, is simple. If the plant contains any non-minimum-phase zeros, then the target set

G will always consist of a single point, namely, the origin of the state space.

660

The design of time-optimal systems

There will certainly arise circumstances in which the target set G is very complex, and consequently the engineer who is faced with the problem of designing a time-optimal system cannot even determine the boundary of G, much less the time-optimal control law. In such cases, a good suboptimal design is to replace the target set G with a simpler set or even with the origin. The resulting system will not be time-optimal, but it will be much simpler than the time-optimal one. Exercise 7-71

For each of the plants listed below, (1) write the equations of the state

variables in terms of the output variables and the control and its time derivatives,

(2) write the state-variable differential equations, (3) determine the target set G and plot its boundary, (4) discuss whether the system is completely controllable, and (5) determine

the time-optimal control law. (a) H(s) = SGstl ED

(b) H(s) ==-— 3@ *__ +2)

+3

s—1

(d) H(s) =ae +2) &

(m)

H(s)

s+4

GeHatH © FO“ GTHETH s+2

e+3

s+5

s+l = a+

8

O HOmat

For each of the planta below, find the target set G and its boundary.

~ & +1)

+2)

8(s + 3)

+ 2) (c) H(e) = SEVG s(s + 3)(s + 4)

7-15

(f) H(s) = (s + 2)(s + 4)

Cine tD MFHO-GrHerH

Exercise 7-72

(a) H(e)

FO

= ~+2_ ae +2) _#71

(e) H(s) = s(8 + 2)

HO = GepeTs DEO

(c) H(s)

(0) HG)

~ & +13 + 2)

@) He) =

(s + 4)(3 + 3)

—@ tHe +2)

(s + 3)(s + 4)(s + 5)

Concluding Remarks

In this chapter, we presented techniques which are used in the design of time-optimal feedback control systems for various plants described by simple ordinary differential equations.

The approach used was conceptu-

ally the same throughout the chapter; we obtained as much information as possible from the minimum principle, and then we used this information together with the shape of the system trajectories in the state space to obtain and prove the time-optimal control law. Once the time-optimal control law was derived, we knew what control to apply for each state in the state space, and as a result, we could ‘‘design’’ the time-optimal feedback control system. These engineering realizations

of the time-optimal control laws used a relay-type element to generate the piecewise constant time-optimal control. We found that for second-order

systems with a single control variable, we needed to apply a single nonlinear operation to one of the state variables.

The input-output characteristics

of the required nonlinearity were identical to the switch curve.

The exact

Sec. 7-15

Remarks

661

equation of the switch curve was a function of the equations of the system trajectories. In Sec. 7-11, we suggested a procedure which could be used to obtain the switch curve experimentally. We found that the switch curve for oscillatory systems was more complex than the switch curve for nonoscillatory systems. We suggested suboptimal designs based on the optimal design in order to simplify the “hardware.” With the exception of the problem solved in Sec. 7-9, we focused our attention on the time-optimal control of normal systems with a single control variable. We showed that if the system is of second order, then the control law reduces to the problem of finding (analytically or experimentally) a, switch curve which separates the state plane into two regions. The equation of this switch curve dictates the input-output characteristics of the single nonlinearity required in the construction of the time-optimal feedback control system.

The complexity of the feedback controller may increase rapidly with the increase in the order of the control system. The realization of the control law for the third-order system considered in Sec. 7-4 illustrates this point. As a matter of fact, for systems whose order is higher than three, some iterative procedure must be used to solve the system of transcendental equations that describe the switch hypersurface.

This represents a severe

drawback in the application of the theory. However, the reader should note that many high-order systems can be approximated in a satisfactory manner by low-order systems, and one may use a controller of the type shown in Fig. 7-66 for the suboptimal control of a high-order system. Even though the equation of the switch hypersurface is complex, it should be clear that, from a conceptual point of view, the operation of a high-order time-optimal

system

presents

no particular

difficulty.

Quite often,

inti-

mate knowledge of the optimal solution can help the designer to construct an excellent suboptimal system.t The same comments apply to the design of suboptimal systems for plants which contain more than one zero in their transfer function. The material presented in Secs. 7-13 and 7-14 indicates that for such problems, the determination of the optimal-control law is in itself a difficult task. For this reason, we may often design a good suboptimal system by considering a simpler target set or, still more simply, by specifying that the origin be the target set. In either case, the designer will trade response time for simpler equipment, and the relative merits of each point of view will, of course, depend on the application at hand. t References that deal with the design of suboptimal systems are, to mention just a few, F-13, K-18, M-6, P-7, and W-1.

Chapter 8 THE DESIGN OF FUEL -OPTIMAL SYSTEMS

8-1

Introduction

In Secs. 6-11 to 6-16, we considered the general formulation of fuel-optimal problems. In this chapter, we shall derive the optimal-control law for several simple fuel-optimal problems. At this point, the reader should review the pertinent material of Chap. 6 and, especially, the definition of normal fuel-optimal problems (Definition 6-11), the definition of singular fuel-optimal problems (Definition 6-12), the on-off principle (Theorem 6-11), the sufficient conditions for a normal fuel-optimal problem (Theorem 6-13),

the uniqueness theorems (Theorems 6-14 and 6-15), and the material of Secs. 6-21 and 6-22 which deals with singular problems. We remark that the theory of minimum-fuel systems is not so well developed as the theory of minimum-time systems. Also, the design of minimum-fuel systems is more complex than the design of the corresponding minimum-time systems. Additional complications arise because of the formulation of the problem. If we want to design a system which meets reasonable fuel requirements, then we have a wide choice of performance functionals.f In other words, we may want the response time to be included in our performance functional, as a fuel-optimal system which requires large response times is, from an engineering point of view, impractical. The particular choice of a performance functional combining fuel and response time rests with the designer. An inexperienced designer may have to try many performance criteria, solve the optimal problem for each, and then decide upon a system design. Since it is difficult to predict whether or not a solution will be “good” until the problem is completely solved, we shall, in this chapter, formulate several control problems for the same basic system but using different performance criteria. We shall then be able to compare the different optimal systems and, perhaps more importantly, to determine the type of equipment required for the construction of each. Furthermore, we shall be able to examine the structure of the various optimal systems t See Sec. 6-15.

Sec. 8-1

Introduction

663

with a view to determining whether or not this structure is the same for a wide class of systems. To put it another way, we saw in Chap. 7 that the time-optimal control for a wide variety of second-order systems was the same in the sense that it was determined

by a switch curve that divided

the state plane into two regions, and we now ask if similar behavior occurs

for fuel-optimal systems.

We shall, therefore, examine similar second-order

systems with various performance criteria in this chapter. The detailed examples of fuel-optimal systems that we shall consider represent the translational motion of a mass:

1. When there is no friction or drag force 2. When the friction is proportional to the velocity

3. When the friction is a nonlinear function of the velocity

We shall assume that the control force is provided by rockets or gas jets in such a way that the magnitude of the control force is: 1. Bounded 2. Proportional to the rate of flow of fuel or to the rate of flow of gas

Since we shall consider second-order systems (the state variables being

the position and velocity of the mass)

in order to simplify the exposition,

we assume that the mass of fuel, consumed during the control interval, is negligible compared with the mass of the body.

In essence, this assumption

amounts to viewing our system as one of constant mass.f We shall illustrate the following basic points in this chapter: 1. The fuel-optimal] solution does not exist for some systems. Although this may appear somewhat strange, we shall see in Secs. 8-3 and 8-5 that it can happen. Often in such a case, it is possible to find controls whose fuel requirements are arbitrarily close to the ideal minimum é-optimal controls.

fuel, that is,

2. The fuel-optimal control is not unique for some systems. In other words, there are many controls which require the same amount of minimum fuel. However, and 8-4.)

in this

case,

the

response

times

differ.

(See

Secs.

8-2

3. The formulation and solution of problems in which the response time is specified a priori. The fuel-optimal control is then a function of both the initial state and the specified response time and thus cannot be obtained as a function of the (present) state alone.

4. The formulation and solution of problems in which the response time is specified in terms of the minimum time required to force each state to the origin (see Sec. 8-7). 5. The introduction of a cost functional which is a weighted sum of the consumed fuel and the elapsed time in an effort to overcome the unattractive, from an engineering point of view, aspects of the minimum-fuel solutions when the response time is either free or fixed in advance (see Secs. 8-8 t See Secs. 6-11 and 6-12.

664

The design of fuel-optimal systems

to 811). Weshall show that the time-optimal control and the fuel-optimal control (with free response time) can be found by taking suitable limits. 6. A technique of suboptimal control which leads to simplification of

the optimum controller (see Sec. 8-9).

7. The fact that singular controls can be fuel-optimal when the equations of the system are nonlinear (see Secs. 8-10 and 8-11) and that this situation does not occur when the system is linear. Just as in Chap. 7, we shall prove certain lemmas in great detail while omitting the proofs of others. We shall try to lead the reader from analytical proofs and constructions to proofs and constructions which are based upon graphical techniques. In so doing, we shall show that quite often the ‘‘shapes” of functions, rather than explicit formulas, can be used to determine the optimal control graphically. Performance

criteria involving

consumed

fuel arise quite frequently in

aerospace problems. The fact that the motion of missiles, satellites, and other space vehicles is achieved by the burning of fuel or the exhaust of gas from a jet and the fact that weight is at a premium indicate the need for the inclusion of fuel in the cost functional of a given aerospace mission. In fact, many solutions to minimum-fuel problems have been obtained in this context without the use of the minimum principle.t The techniques employed were, for the most part, a mixture of semiclassical variational techniques and physical reasoning. The reader will find that, often, use of the minimum principle will simplify proofs but, in general, will not alleviate the computational difficulties. References which deal with fueloptimal problems using the minimum principle are A-3, A-6 to A-8, A-13

to A-15, B-1, B-14, F-12, F-14, F-17, J-6, K-16, K-17, L-1, M-4 to M-6, and N-3.

8-2

First-order Linear Systems: The Integrator

In this section and the next, we shall examine the fuel-optimal control problem for linear first-order systems. Although first-order systems are more or less trivial, their study reveals certain important characteristics which are common to almost all fuel-optimal systems. Before we formulate the problem in precise mathematical language, we

shall begin with a description of the physical system. Suppose that we are given a mass m which is undergoing translational motion in a frictionless environment. We let y(t) be the velocity of the mass at time ¢, and we let u(t) be the applied force or thrust. Let us assume that the thrust u(?) is generated by a rocket or gas jet; then it is reasonable to expect that the magnitude of the thrust will be bounded, and so we shall assume that

|u(é)| 1 for all é € [0, T] and the control is u(t) = — sgn {x} according to

Eq. (8-12); in this case,

H=k+1—|s|

=0

(8-41)

which yields (since k > 0) the relation lr] =kK+1>1

(8-42)

It follows that the control does not change sign, and so the control

u(t) = — sgn {é} is the extremal control which drives the system (8-33) from & to 0.

(8-43) Since

the extremal control is unique and since an optimal control exists, Eq. (8-43)

must define the optimal control. Note that the control (8-43) is the ézmeoptimal control for the system (8-33). Since |u(t)| = 1 for all ¢ € (0, T], it follows that the minimum value of J, denoted by J*, is given by

J* = [e+

na=G&+0T

(8-44)

But, since the control (8-43) is the time-optimal one, the response time T is the minimum time, and so Therefore, we deduce that We also note that

T = ||

(8-45)

J* = (k + 1)(é|

(8-46)

lim J* = |&| = F* b0

(8-47)

In other words, by taking the limit as k — 0, the minimum cost J* converges to the minimum fuel F*. This implies that the control (8-43) is also a fueloptimal control.

Moreover,

the control (8-43) requires the least response

time—in this case, the minimum time.

We shall see in subsequent sections

of the book{ that we can sometimes obtain the fuel-optimal control which t The word ‘‘optimal”’ is with respect to the cost (8-34). t See Secs. 8-8 to 8-11.

Sec. 8-3

Single-time-constant plant

O71

requires the least response time by taking

lim. {min.

T

k+0 * uf)

[& + u(s)l] at})

(8-48)

To recapitulate: In this section, we have shown that there may be many fuel-optimal controls and, in particular, that the time-optimal control may also be fuel-optimal. Exercise 8-1 Consider minimizing the fuel,

where the response time T

the system

2z(¢) = u(t),

|u(t)| 0O

(8-52)

Clearly, z(t) qualifies as a state for the system (8-52). Note that the transfer function of the plant (8-50) is 1/(s + a), which is the reason for the terminology single-time-constant plant.

The problem we shall solve in this section is: Given the system (8-52) with the control constrained by Eq. (8-49). Given an initial state x(0) = £ and a desired reachable terminal state x(T) = 0 ¥ &. Find the control which will force the system (8-52) from & to 6 and which will minimize the fuel functional

Flu) = F = [" |u(o)| dt where the response time T may or may not be specified.

(8-53)

OIG

ine

design

Of tuci-Ooptimal

systems

The treatment of the basic ideas in this section will be rapid and informal and many of the statements will be offered without proof. The mathematics involved are simple, and so the reader will have no trouble in readily verifying the statements. Our purpose will be to demonstrate that: 1. Quite often, a fuel-optimal solution does not exist; nevertheless, one can find a solution which, for all practical purposes, is almost optimal. 2. Quite often, the fuel-optimal control is identical to the time-optimal control. The Hamiltonian for the system (8-52) and the cost functional (8-53) is

H = ju(t)| — az(t)p(t) + au(t)p(?)

(8-54)

The costate p(t) is the solution of the differential equation

pO = — Zax = ap so that, if

(8-55)

= p(0), then

p(t) = re

(8-56)

The control which absolutely minimizes H is given by u(t) = 0

if |ap(t)| < 1

(8-57)

if ap(t) = —1

(8-59)

u(t) = — sgn {ap(t)} __ if |ap(t)| > 1

0 < u(t)

1

p1(0)

=

71

(8-121)

Substituting Eqs. (8-121) into Eq. (8-85), we find that the Hamiltonian, at t = 0, is given by (8-122) H eo = |—1| + fer — me

682

The design of fuel-optimal systems

But H must be identically zero since this is a free-response-time problem and the system (8-83) is time-invariant. But the relation 1 — 7, O [since (£, 2) © Ra, 2 < 0]; therefore, we must have m1


1

(8-123)

But if 7, < 0 and w, > 1, then we can see that p2(t)

=m.—Tit>72>!1

for all?

>

0

(8-124)

If p2(t) > 1 for all ¢ > 0, then it can only generate [see Eq. (8-87)] the control sequence {—1}; hence, it cannot generate the control sequence { —1, 0, +1}. Thus, we have arrived at a contradiction, and so we conclude that the sequence {—1, 0, +1} cannot be fuel-optimal. It is left as an exercise to the reader to derive Eq. (8-119) and to show that the response time of Eq. (8-119) is the smallest possible. [Hrnt: T= fr dt = t (dx,/dz,) dt = [ dx,/x3.) We have thus shown that any control u(t) = v(t) € Vrt that satisfies Eqs. (8-120) is fuel-optimal and, therefore, that the control correspondA%2 ing to the control sequence {0, +1} is \

also fuel-optimal.

|

Lemma 8-6

If (1, &) € Ru, then

there ts no solution to the fuel-optemal problem. However, given any « > 0,

there exists a control sequence, namely;

“’ \

Fig. 8-5

\\

{—1, 0, +1}, which will drive (£1, £2)

€ R, to (0, 0) and for which the fuel P

1s gwen by

P= F* +e = || +e

Ra

Clearly, limF = F* = |fs|

\ \.

Thus,

The definition of the set Te.

the minitmum-fuel

(8-125)

(8-126)

solution

can

be approached as closely as destred but

meyer

attained.

We

call such

a control

e-fuel-optimal. PROOF Consider the state-plane arrangements of Fig. 8-5. In Fig. 8-5, we have defined the set (line) I, in Ry, at a distance e/2 from the 2x, axis. More precisely, r=

Kc

42): 22 =

Now consider a state (£, 2) in R;.

2

— 5) ry, > s+

(8-127)

Depending on the position of (£1, &2), it

_ Is evident that we can force (£1, £2) to the origin either by the control sequence

{—1,0, +1} or by the control sequence {—1, +1}. If (£, &) is transferred to the origin by the control sequence { —1, 0, +1}, then its fuel requirements

Sec. 8-5

Free response time

683

are |f,| -+ ¢/2 until it reaches the I’, curve, zero until it reaches the 1 curve [at the point (e?/8, —«/2)], and ¢/2 to reach the origin along the y, curve;

thus, the total fuel # consumed is f = |£,| +. trajectory resulting from the control u =

If (1, &) is such that the

—1 intersects the 7, curve between

the points (e#/8, —«/2) and (0, 0), then the fuel required will be less than [Eo] + ¢. In order to show that if « = 0, then there is no fuel-optimal solution, we note that the state (£1, £2) © R, is transferred to the state (a, 0) (see Fig. 8-5) by the control u = —1 using the minimum amount of fuel possible, namely |f:|; any other control requiring the same fuel |£3| will force (£1, £) to some state (8, 0) such that 6 > a, and so no fuel-optimal solution exists. We have derived up to now certain conclusions for states in y, \U R,U Ri. We can certainly use similar arguments for states in y_U R,U R;. We summarize the results in the following control law: Control Law 8-2 Solution to Problem 8-2 A fuel-optimal control (there may be many), as a function of the state (x1, 22), for Problem 8-2 ts given by u* = u*(%, 2) = +1

for all (x1, 2) © v4

u*

for all

u*(x1,

Zs)

=

—]

u* = u*(2, z2) = O

If (a, x2) € Ri VU

(x1, £2)

Ee

+y-

for all (11, x2) C Re U Ry

Rg, then there is no fuel-optimal control.

We remark that the fuel-optimal controls of Control Law 8-2 have the property that they require the least response time and that Control Law 8-2 is a restatement of Lemmas 8-4 and 8-5. We have seen that, depending on the initial state, there may or may not exist a fuel-optimal solution; furthermore, there are sets of states for which

the fuel-optimal control is not unique. In the next section, we shall examine the problem of fuel-optimal control with the response time either fixed or bounded from above. Exercise 6-4

Show that if (§:, &:) € R., then the response time required by the fuel-

optimal control sequence

{0, +1} is

T=

_&

—its

&3

(8-128)

and if (£:, &:) € Rs, then the response time required by the fuel-optimal control sequence {0, —1} is

_ Exercise controls to Exercise v¥+ Vy U Exercise M; let z:(t)

z(t)

Es

8-5 Consider the state (1,—1) € Ry. Find three different fuel-optimal the origin. Plot the trajectories generated by each fuel-optimal control. 8-6 Show that the minimum fuel F*(x1, x2) = |za| required to force (x1, 22) € RB: U R, to (0, 0) is the solution to the Hamilton-Jacobi equation. 8-7 Simplified Soft-landing Problem Consider a body with constant mass denote its altitude from the surface of a planet (without atmosphere) and let

denote

its vertical velocity.

Suppose that g is the (constant) gravity of the

044

The design of fuel-optimal systems

planet. Let f(t) be the thrust, —Fxu < f(t) < Fm, and let

F= [" ixola be the consumed fuel. Assuming that Fu > Mg, determine the fuel-optimal thrust (T is not specified), as a function of the altitude and vertical velocity, which will reduce the altitude and velocity to sero. The equations are

di(t) = zat)

Mix(t) = f(t) — Mg

(8-130)

Are there values of initial altitude and velocity which will cause the body to crash?

Is there more than one fuel-optimal solution? Exercise 8-8 Consider the system (8-83) and the fuel functional (8-84) with 7 free. Find the optimal solution (if it exists) to the target sets

S: = {(z1, ze): 21 = 2a, [zi < 1}

S;

=

{(z1,

Zs): t3

=

—Z2,

laa}

Ss =

{(z1, 2): a1? + 23? < 1}

Sa

{(x1,

=

Ze): v1

=

+1,

z=


= 0}

Exercise 6-9 Consider the system (8-83) and the fuel functional (8-84) with T free. Let (1, 1) be the desired terminal state. Determine the fuel-optimal control law and evaluate the minimum fuel as a function of the state. Comment on existence and uniqueness.

8-6

Fuel-optimal Control of the Double-integral Plant: Fixed or Bounded Response Time

The results we derived in the preceding section were, from a practical point of view, often unsatisfactory in view of the large response times required by the fuel-optimal control whenever the state was near the 2, axis and in view of the almost infinite response time required by the e-fyeloptimal controls whenever the state was in FR, or R3. For this reason, we shall reconsider the problem of fuel-optimal control of the double-integral plant with: 1. Fixed response time 2. Bounded response time The procedure we shall adhere to in this section is as follows: After we state the problem, we shall present Lemma 8-7, which states that the fixedtime problem will have a solution if and only if the specified response time is no less than the corresponding minimum time. However, we wish to caution the reader that we are not offering in this section a rigorous proof

of the existence of an optimal control. ever, the proof involves mathematical

We assert that it does exist; how-

concepts

(such as measurable func-

tions) which we have not discussed. (See, for example, Ref. C-4 for a flavor of the concepts involved.) For this reason, we shall assume that an

optimal solution does exist.

We shall then carry out some computations,

Sec. 8-6

Fixed response time

685

and we shall find the fuel-optimal control as a function of the initial state and of the specified response time. We now state the problem. Problem 8-3

Given the system ti(t)

= x(t)

i(t) = ul)

[u)| 0, that is, a point on the positive x, axis. The minimum time ?¢*(é;, 0) corresponding to this state is (see Eq. (8-134)] given by the relation

t*

=2

Vt

2

Suppose we are given a value for 7',; then there is a state (£1, 0) such that

i= ie

(8-162)

which means that the time 7’, specified is the minimum time needed to force (£1, 0) to (0, 0).

Let us now consider the set of states (#:, 0) such that

o 0. Compute the locus of (z:, 22) and plot the resulting curve for T; = 3. Exercise 8-14

Repeat Exercise 8-13 when

the initial state (£1, &) is such that & = &, f; > 0.

We

(8-168)

\

~"7

rT)~s) £

of

Fig.

switch

8-7

/

re tee

[Illustration

curve;

Re

of the

I(T’)

in this figure, we have

used the value 7; = 3.

conclude this section with the verification of the Hamilton-Jacobi

equationf

for this problem.

The reason for this development

is that, up

to now, we have been concerned primarily with the optimal control of timeinvariant systems with free response time; the system in this section is time-invariant, but the response time is fixed. First of all, let us suppose that we are given an initial state (&1, &) in Ri and that the response time 7’, is fixed; we know that the fuel-optimal control is u*(t) = —l forO 1

(8-194)

for all (&1, 2) and all t € [0, Ty).

Let us explain the physical significance of Eq. (8-194). Equation (8-194) guarantees that the fuel-optimal control has the property that every state (x1, 23) on a fuel-opttmal trajectory can be forced to the origin in some time T, ~ t which is bounded by a multiple of the minimum time t*(x,, x2) corresponding to that state. In a sense, Eq. (8-194) is a time-varying constraint. The significance of this formulation will be clear as we proceed with the solution to Problem 8-4. Suppose that (11, x2) is a state in R, [see Eq. (8-112) and Fig. 8-3]. We know that the minimum time {*(z;, z2) corresponding to that state is [see Eq. (8-134)] given by the equation

t*(1, 22) = ta + V/ 421 + 223?

(8-195)

We also know (see Lemma 8-5) that the state (21, 22) © R, can be forced to (0, 0) by the fuel-optimal control sequence {0, +1}, which requires fuel |z2| and response time [see Eq. (8-119)]

T(x, 42) = —}22 — n

(8-196)

On the other hand, if (21, x2) € Re, then the minimum time corresponding to (21, 22) € R; is [see Eq. (8-135)] given by

t*(21, 22) = —2e + VW —4a1 + 27,

(8-197)

We also know (see Lemma 8-12) that the state (x1, x2) € Rez can be forced to (0, 0) by the fuel-optimal control sequence {0, —1}, which requires fuel

|z2| and response time

-

T (21, 22) = 422 — a

(8-198)

Definition 8-2 Let Q(T) denote the set of states (x1, 2) € Ri VU Ro which can be forced to the origin by the control sequence {0, +1}, af (x1, x2) € Ra, or by the control sequence {0, —1}, tf (21, re) © Re, and which require the same response time T.

From Eqs. (8-196) and (8-198), we find that Q(T) = {(x1, 22): 21 = —aeT' + d$22\20|; (21, x2) C Re U Re} We also set

QH(T) = QATAR Q(T) = Q(T) OR:

(8-199)

(8-200)

Figure 8-9 illustrates several of the Q(T) curves. A useful property of the Q(T) curves is provided by the following lemma:

Sec. 8-7

Bounded response time

Lemma 8-14

Q(T:).

Suppose that T; < Ts.

Let (x1, 22) © Q(T)), (21, 22) €

Then the relation

lzi| > |z:l

18 satisfied.

697

(8-201)

Proor The proof follows by direct computation using Eq. (8-199). Loosely speaking, this lemma means that the Q(7) curves “expand” with increasing T’. Let t* be some positive number; we know that we can find the set of states for which ¢* is the minimum time to the origin and that we have 4%2

‘

‘\

NI

7-

\

\

\ \

—+

~1

\

2

\

\ \

\

\

,

+— >

OF

\

a—Z=

\

*

\

NX

a

\ %

8-9 Some of the Q(T) curves deby Eq. (8-199); the curves shown in figure are for T = 1, T = 1.25, and 1.50.

Ze)

. o(ae')

\

\ y~

(2,

\

o(1.50)

‘

ste

4

011.00) 911.25)

\

-14

N\A

Z'=(2,, 25)

x,

1

\

Fig. fined this T =

x

y

\

‘\

Fig. 8-10 The points Z and Z’ belong to the intersection of S(?*) and Q(gi*).

denoted this set of states by S(t*) (see Definition 7-6). The equation of the minimum isochrone S(t*) is given by the relation [see Eq. (7-27)]

S(t*) = {(x1, 22): 21 = —h202? + $(t* — 22)*, if (21, 2) C Ry U Ri, orz,

=

$22?

-

+(t*

+

L2)?,

if

(x4,

£2)

cE

R,

U

R3}

(8-202)

Figure 8-10 shows an S(¢*) minimum isochrone. We have also plotted in Fig. 8-10 the Q(8t*) curve, where 8 > 1. By setting J = Bt* in Eq. (8-197), we deduce that Q(Bi*) = {(t1, 22): 2%. = —72Bt* + 424|z9|; (21, 22) C Ra

Ra}

(8-203)

As illustrated in Fig. 8-10, the sets Q(8t*) and S(¢*) intersect at two points,

denoted by Z = (2:, 22) and Z’ = (z;, z,), which are symmetric about the

698

The design of fuel-optimal systems

origin.

The point Z = (2, ze) is in R,, that is,

a, 22) = S(t) A QH(BE*) = SU) A Q06t*) AR.

(8-204)

(2, 23) = S(t*) C\ Q-(6t*) = S(t*) CO Q(Bt*) OV Re

(8-205)

and the point Z’ = (z}, 2;) is in Re, that is,

From Eqs. (8-203) and (8-202), we find that the point (2:, 22) is specified

by the equation

4.

=

—2,Bt*

ed $2.7

=

— 42,7

+

4(¢*

—_

22)?

(8-206)

Thus, we conclude that the point (2, 22) can be found from the two equations

Zo = t*[(1 — 28) + 2-+/8(8 — 1)) 2, =

(8-207)

—2e8t* — 42,2

(8-208)

and, owing to the symmetry of (z:, z2) and (z;, 2,), we also have

2, = —t*[(1 — 28) + 2/86 — 1)

(8-209)

2, = —2,Bt* + 42,7

(8-210)

Thus, given 8 and ¢*, the two points Z = (21, 22) and Z’ = (z;, 2,) are com-

pletely specified. We now eliminate the time é* between Eqs. (8-207) between Eqs. (8-209) and (8-210) to obtain the relations 21 = Mp2."

(21, 22) © Re

and

(8-208) and (8-211)

= —mpzz? = (4, ) EC Ra

(8-212)

where mg is given by the equation

ms

-

B

98 —2>/BB-1)—-1

_l

2

(8-213)

We combine Eqs. (8-211) and (8-212) into a single equation, and we define @ curve yg, (replacing the z’s by 2’s) as

vs = {(21, 2): 1 = —mpza|zal}

(8-214)

where mg is given by Eq. (8-213). Figure 8-11 shows the shape of the yg curve for various values of 8 > 1. We shall now establish two lemmas which lead to the control law. Lemma

8-15

Let

(x1, 22) € yg and

let t*(x;, x2) be the minimum

time

associated with (x1, t2). If (21, x2) € Ra, then the control sequence {0, +1} is fuel-optimal; if (21, 12) € Re, then the control sequence {0, —1} 18 fueloptimal; in etther case, the response time T ts given exactly by the relation T = Bt* (21, Xs)

for all (x;, 2) € vp.

(8-215)

Sec. 8-7

PROOF

Bounded response time

699

This lemma is a direct consequence of the construction of the

ve curve. Since the +g curve is the locus of the points (2;, z2) and (2;, 23), as ¢* varies, and since the points (21, z2) and (2, 2;) belong both to the

isochrone S(t*) and to the set Q(ft*), the lemma follows from the definition of the Q(T) curves (Definition 8-2). Lemma 8-16 Given B, < 82; let (x1, 22) € vp, and let (x1, 22) € z,; then \zi| < ta.

PROOF The lemma and (8-213).

Fig.

8-11

Some

follows by direct computation

of the yg curves defined by Eq.

B = 1.2, 6 = 2.0, and 6 = 10.

using Eqs.

(8-214); the curves shown

|

(8-214)

are for

This lemma means that as 8 increases, the ys curves tend to the 2; axis (compare with Fig. 8-11).

The yg curve and the y curvef divide the state plane into four sets G,, G2, Gs, and G,, as illustrated in Fig. 8-12; the precise definition of these sets is G,

=

Gz = G;

=

{ (21,

Z):

112

{(x1, £2): 21 < { (x1,

23): Ys

—+422|2r4|

and

x;

>

—$xe|2a| and 2; > —4$2-|z9|

and

11


—dae|z2| and 21 < —mpzxa|z9|}

(8-218)

(8-219)

Note that the set G; includes the y_ curve and that the set Gs; includes the 7+

curve.

+ Recall that y = 7, U 7- [see Definition 7-4 and Eq. (7-21)).

700

The design of fuel-optimal systems

We shall show that Control Law 8-4 provides the solution to Problem 8-4.

Control Law 8-4 Solution to Problem 8-4 The function of the state (x1, 22), 1s given by the equation

optimal

u* = u*(r1, 22) = —1 u* = 4*(x%1, t2) = +1

for all (x1, x2) € Gy for all (x1, t2) € Gs

u*

for all

=

u*(x1,

Z2)

=

0

(x1,

X2)

control,

as a

(8-220)

EG.U

G,

and, furthermore, is unique. g 72

Go

Gy

oS

-|

+

\

Ysa)

+

°]

o> X.

1

'

8 Gy

A

%

Ga —j+ y

Fig. 8-12 Ya curves,

The division of the state plane into the sets G,, Gz, Gs, and G, by the y.and The trajectory YABO is an optimal one.

PROOF Suppose that X = (2), 22) is a state in G,; let t3 be the minimum time corresponding to X. We must show that the response time 7'x required by the fuel-optimal control sequence {0, +1} is such that Suppose that

Tx < pty

(8-221)

Tx > pty

(8-222)

Then there must be a 6’, 8’ > 8, such that Tx = ptt

(8-223)

and, therefore, a curve y,s such that (x1, 22) € ys, which, in turn, implies

that ys € G.; but this contradicts Lemma 8-16; therefore, if (%, 22) € G,, then Eq. (8-221) holds.

Sec. 8-7

Bounded response time

701

Now suppose that Y = (y1, y2) is a state in G,, as illustrated in Fig. 8-12.

Let t} be the minimum

time associated with Y.

results in the trajectory YABO

the response time T'y is such that

The control of Eqs. (8-220)

shown in Fig. 8-12.

We must show that

Ty < Bey

(8-224)

Let ts be the time at which the trajectory hits the yg curve (at the point A); since the portion YA of the trajectory Y ABO is identical to the time-optimal trajectory to the origin, it follows that the minimum time ¢{ corresponding to the point A is f=t}—ts (8-225) But the time required to go from A to the origin along the trajectory ABO

is, by definition, 82%; hence the response time TJ’ to go from Y to the origin along YABO is given by

Ty = ta + Blip — ta) = BIE + (1 — Bla

(8-226)

Since B > 1, the term (1 — #)és is negative, and therefore we deduce that Ty < ply

(8-227)

If the initial state is in G; \U G3, we can use identical reasoning to establish the control law. a(t)

+1

-1

‘|

[

1

u®(t)|

1

se |

42(t)

ae

Mp

x2 | xe

/

Xo

rh

_

N Fig. 8-13

An engineering realization of Control Law 8-4.

Let us now turn our attention to the engineering realization of Control Law 8-4. The control system must measure the state (x1, x2), decide whether the state belongs to Gi, to Gs, or to Gz U G,, and then deliver the optimal control specified by Eqs. (8-220). The system shown in Fig. 8-13 accomplishes this task. Note that the state variable z; is introduced to the nonlinearity N ; the input-output characteristics of N are such that the output

702

The design of fuel-optimal systems

is z|r2|.{ The signal z2|z,| is attenuated and added to z; to produce the signal a(t). The signal 2.|22| is also multiplied by the constant gain mg and added to the signal z; to produce the signal b(t). It is easy to see that

and that

a(t) > 0

means

(%1, 22) CG,U

Gy

a(t)

means

(x1,

G;

0 b(t) < 0

means means

(t1, 2) CG (41,22) EGZU

b(t) =0

means

EC ¥4 Uy

(2, 22) € ¥p

Gs G,

The polarized relays R; and Rz obtain the sign of a(t) and b(t), since

J

It follows that If

= — sgn {a(t)} — sgn {d(/)}

(x1,

Ls)

E Gi,

then

fi)

=

(8-228)

—2

If (x1, 2) € Gs, then f(t) = +2 If (71, m2) € G, UG, then fi) = 0

(8-229)

The input-output characteristics of Rp are those of a perfect relay with a

dead zone; that is,

u*(t) = +1

u*(t) = 0

if fj) >1

if |f@|

ai

“

y

0

+—

! 5

-3

——j

3

“>

x,

Ye Hs, Ha

Fig. 8-37

The division of the state plane into the six regions Hi, H:, Hs, H., Hs, and Hg.

We claim that the solution to the posed problem is: Control Law 8-7 Solution to Problem 8-7 If 0 0 for all a ¥ 0 and that a real symmetric matrix N is posttive semidefiniie if (b, Nb) > 0 for all b = 0. We note that the zero matrix is clearly positive semidefinite. Let us now consider each term in the cost functional J(u) given by Eq. (9-5) and see in what way it represents a reasonable mathematical translation of the physical requirements and specifications. First, consider the term DL. = }(e(t), O(He(t)), where Q(z) is positive semi-definite. Clearly, this term is nonnegative for al] e(¢) and, in particular, is zero when e(é) = 0. If the error is small for all ¢ € [t, 7], then the integral of LZ, wilt be small. Since

= Heli), Qe) = 4 DY geet 1 sal

where the q,,{f) are the entries in Q(é) and the e,(¢) and e;(2) are the components of e(?), it is clear that the cost L, weighs large errors much more heavily than small errors, and so the system is penalized much more severely for large errors than for small errors.t Next,

we

consider

the

term

L, = 4(u(t),

R(d)u(é)),

with

R(é)

positive

definite. This term weighs the cost of the contro] and penalizes the system much more severely for large controls than for small controls. Since R(Z) is positive definite, the cost of control is always positive for all u(t) = 0. This term L, is often called control power, and f° L, dt is often called the control energy. The reason for this nomenclature is the following: Suppose that u(t) is a scalar proportional to voltage or current; then u*(t) is proportional to power and fe u3(t) dt is proportional to the energy expended in the interval [¢o, J]. The requirement that R(t) be positive definite rather than positive semidefinite is, as we shall see, a condition for the existence of a finite control. We remark that this requirement was also used in Sec. 6-18. Finally, we turn our attention to the term 4(e(7), Fe(7)).

This term is

often called the terminal cost, and its purpose is to guarantee that the error e(T) at the terminal time 7' is small. In other words, this term should be included if the value of e(¢) at the terminal time is particularly important; if this is not the case, then we can set F = 0 and rely upon the rest of the cost

functional to guarantee that the terminal error is not excessively large.

A question that may arise is the following:

Why should the matrices

Q(f and R({é) be time-varying rather than constant? + If Z, were replaced by large or small errors.

If we develop a theory

> lqt)e:(é)|, then the system would be penalized equally for i=l

The

use of a functional like f

T

7

>

i=1

qi(£)e,(¢) dé ts not recom-

mended because cancellation of large positive errors by large negative errors may result in a’ zero cost, although large errors are always present.

Sec. 9-2

Formulation of the problem

755

for time-varying matrices Q(z) and R(¢) rather than for constant matrices Q and R, then we can use the cost functional J(u) for more realistic problems. For example, suppose that at ¢ = to, the state x(t) is given and hence the output y(t)

= C(é)x(to) is known; then it may happen that z(¢,) is such that

the initial error e({¢,) is very large. The fact that e(é)) is large is certainly not the fault of the control system. For this reason, we may want to choose Q(t) in such a way that large intial errors are penalized less severely than subsequent large errors. To do this, we can pick Q(#) in such a way that, given the times

b0)

a{7) (for £=0)

[x10

of all, note that for large values of r,

=1 aire en

the state regulator does a very bad job

r= |

0.57

0

;

0

10

20

—~f

fig. on state (7) o e the optimal regulator when a = —1 (stable case)

and @ = +1 (unstable case).

of regulating the state; the reason is that it is cheaper for the system to have a large error in the state over this finite interval of time than to use large control. As the value ofr decreases, the state x(t) starts to decrease from its initial value z(0), reaches a minimum, and then increases dunng the latter part of the control interval. This be havior is due to the fact that k(T) = 0,

which implies that the optimal control must

go

9-105 shows

to

zero

as . t—> Tf.

Figure

the behavior of the func-

tion k(t) as r varies. Once more, note that k(t) is almost a constant for small values of r at the beginning of the

control interval [0, 1}.

Next, let us examine the effect of the boundary condition k(T)

the terminal value z(7) for both the stable (a = —1) (a = +1) systems. In Fig. 9-11, we plot the ratio 2(T) (for f > 0)

2(T) (for f = 0)

= f upon

and the unstable (9-160)

versus f, using, in both cases, the values 7(0) = q=r=T=1. Frm Fig. 9-8a, we know that, when f = O anda = —1, z({1) = 0.2184; and from Fig. 9-10a, we know that, when f = 0 and a = +1, x(t) = 1.2244. The conclusion that we may draw from Fig. 9-11 is: Inclusion of the terminal cost $fz*(T’) leads to a better regulator in the unstable case than tn the stable case, especially for small values of f. In either case, lim 2(T) = 0, but x(T) 0 for any finite f. Let us now turn our attention to the case T = ©. 9-2, we find that the optimal control is given by

u(t) = — = he where

Using Control Law

(9-161)

& is the positive root of the algebraic equation

~ he ~ 20k —¢ =0

(9-162)

Sec. 9-6

Analysis of a first-order system

T7861

and so & is given by the equation k=

art raif+ at

(9-163)

Comparing Eqs. (9-163) and (9-159), we observe that

k= lim k(t)

forallf >0

(9-164)

Ifa = —l andg =r =—1, then k = —1 + 1/2 = 0.414, which is indeed the “steady-state value” of k(t) in Fig. 9-9. Ifa = +1,¢ = 1, andr = 0.05, then k ~ 0.278, which is the “steady-state value” of k(é) in Fig. 9-100 for r=

(0.05.

Exercise 9-2

Consider the system £(¢) = Ax(?} + Ba(t) and the cost functional

Ji = HAT), FT) +f" Wat, Onl) + (WHO, Rua

(9-165)

Suppose that F is positive definite and that F satisfies the algebraic equation —FA

— A‘F + FBR-'B¥F

— Q=0

(9-166)

Show that the minimum cost is J] = }(x(t), Fx(t)) for ali 2 and 7, either finite or infinite. Show that the optimal control 1s a time-imvariant linear function of the state. Can you explain what Eq. (9-166) implies? Exercise 9-3 Consider the system with input u(t) and output y(t) and with the transfer function

y(s) _

-

t

ene)

ue) ~ 9°) = 3 Pas Fa Let z:(£) = y(t) and z;(f) = 9(t), and consider the cost functional

J, =2 f * Xt) + grXt) + rutidi

g>O:r>0

(9-168)

Determine the optimal control and construct the block diagram (eimilar to the one shown in Fig. 9-5) of the optimal system. In each of the following cases ap = 0 ae = O

a, = 1 a= —!

(9-169a) (9-1695)

Go = —]1 a = 1

a, = 0 a, = 0

(9-169d) (9-169¢e)

ay = 2

a= 3

(9-169¢)

(a) Plot the locus of the poles of the optima] closed-loop system forr = landr = 0.1

as g varies. (6) Plot the locus of the poles of the optimal closed-loop system for g = landg = 0.1 as r Varies. (c) State your conclusions about the character of the system (i.e., oscillatory, sluggish, etc.).

The following exercises are for students who have access to a digital computer and know how to program it.

$624

Uptimal linear systems with quadratic criteria

Exercise

9-4

Consider the first-order system

cost functional 3 f

2

(zt) + «*(t}] dt.

(2) = (2e-' ~— 1)a(#) + u(é) and the

Suppose that z(0) = 0.

Plot the optimal z(t)

and u(2). Repeat for the system 2(f) = (gin 10é)a(?) + u(t), 2(0) = 0. Exercise 9-5 Consider the system Z(t)

= z(t)

ist) = u(t)

(9-170)

J,e% f [2,%() + O.lasXt) + 0.1u%E)] dt

(9-172)

with

(0) =1

2(0) = 0

(9-171)

and the cost functional T

Plot z(t) and z2{z) versus ¢ in the interval0 0,

kitkss

(3-228)

orkis

=

Blez:)?

~ Bis? > 0, we deduce that

(9-230)

orks

=

Bkes

+

kis

(9-229)

/r

=

kis

+

Rar

From Eqs. (9-229), (9-230), and (9-227), we find that £1, is the solution of a fourth-order

algebraic equation, which we shall not attempt to solve here. ee

u(t)

.

s+f

y(t)

= 4{t)

$

rit)

yer

hay * Shy

Fig. 9-16 The optimal function (s -++ 8)/s?.

closed-loop output regulator for the plant with the transfer

An interesting aspect of the problem arises when § = 0, that is, when the system (9-225) is not controllable. If the system is not controllable, then Control Law 9-4 should not be used. To illustrate the nonsenee that arises when we attempt to use Control Law 9-4, we blindly substitute @ = 0 into Eqs. (9-227) to obtain Kis?

=

Rukis ki?

(9-231)

= rhs:

(9-232).

=

(9-233)

2rki:

Since k,, > 0 and r > 0, we see from Eq. (9-231) that kts

Equation

=

/r

>0

(9-234)

(9-232) leads to the conclusion kis=r

(9-235)

and Eq. (9-233) results in the relations

kis =O

or

ky = 2r

Clearly, Eqs. (9-235) and (9-236) are contradictory. This ridiculous answer because we have tried to use Control Law 9-4 for an uncontrollable system.

Exercise 9-9

(9-236) arises

For each of the following systems, y(2) represents the output and u(z)

the control; the cost functional is

f=

[wo + word

(9.237)

Sec. 9-9

The tracking problem

793

For each syatem, determine the optimal control, determine the poles of the optimalcontrol system, and plot the locus of these poles as r varies. (a) #@) +¥@ = ul (6) ¥®) — ¥@) = ull)

(c) 9) + 39) + 2y = ud Ce) 70) + yf

= ul)

2) 7() — y@) =u)

(g) y@) + xe) = u(t)

(f)

¥y@

= ult)

Is there a common characteristic of the loci of the poles of the second-order systems (a) to (e)? Exercise 9-10 Consider the system with output 7(f} and input u(t) related by the differential equation

7(t) + y(t) = aulé) + ulé) and with the cost functione! given by Eq. (9-237).

Determine and plot the locus of

the poles of the optimal closed-loop system as 7 varies and as a varies. whena = 1?

9-9

(9-288) What happens

The Tracking Problemf

In the previous sections, we examined the state- and output-regulator problems. In this section, we shall focus our attention on the tracking problem. Recall that we discussed in detail the physical aspects of the tracking problem in Sec. 9-2 and that we presented its formulation; we shall repeat it here for the sake of completeness. In this section, we shall] proceed in a manner similar to that of Sec. 9-3. We shall use the minimum principle to determine the extremal controls. We shall show that: lL. The H-minimal control ts a linear function of the costate. 2. The reduced canonical equations are linear but not homogeneous (as the desired output will appear as a forcing term). 3. We shall find that the costate p(é) is related to the state x(i) by an expression of the form p(¢) = K(é)x(t}) — g(t); we shall show that the matrix K(é) must be the solution of the Riccati equation and that g(¢) must be the solution of a linear differential equation which involves the desired output. We shall leave the proofs of existence and uniqueness to the reader, and we shall concentrate on the discussion of the properties of the optimal track-

ing system. We suppose that we are given a linear observable system with state x(t), output y(¢), and control u(t), described by the equations X(t) = A@)x() + Bul)

y(t) = C()x(t)

(9-239)

Suppose that the vector z(é) is the desired output; we assume that the dimen-

sion of z(t) is equal

to the dimension

of the output y(t).

Our objective

is to control the system in such a way as to make the output y(t) be “near” t This problem is treated in Ref. K-8.

794

Optimal linear systems with quadratic criteria

z(t) without excessive control-energy define the error vector e(¢) by setting

expenditure.

For this reason, we

e(t) = z(é) — yf)

(9-240)

The cost functional J, which we shall minimize is given by

Js = He(T), Fe(7)) +4 [" (e@, We) + UO, Ru) dt (9-241) with the (usual) assumptions that T' is specified, that R(¢) is positive definite,

and that Q(t) and F are positive semidefinite.

Since the error can be expressed as a function of z(t) and x(é), that is, since e(t) = z(t) — C(x)

(9-242)

we can, by substituting Eq. (9-242) into Eq. (9-241), obtain J as a function of x(t), u(d), and z(é). We are now ready to use the minimum principle and thus obtain the necessary conditions. Our method of approach will be very similar to that used in Sec. 9-3, and many proofs will be omitted or left as exercises for the reader. The Hamiltonian for the tracking problem is given by

H = 3(z2@ — Cx], Q@[z@) — C@x@)) + xu), R@u(d)) + (AMzxO, p©) + Bu,

p@)

(9-243)

The condition dH/du(i) = 0 yields the equation

a

= 0 = Ritu(t) + Bp)

(9-244)

and so we deduce the relation

u(t) = —R-)B’ (Op)

(9-245)

Since R(¢) is positive definite, the control (9-245) minimizes H; that is, the control (9-245) is H-minimail. The condition p(t) = —dH/dx(t) yields

Pt) = —C’*HQMCE)x() — A’Mp@) + C’MQWMz¢)

(9-246)

From Eqs. (9-245) and (9-239), we obtain the relation

X(t) = A()x(4) — BY) R-()B’(é) pd)

(9-247)

We define, for the sake of simplicity, the matrices

S() = BOR-"()B'()

¥V@) = C’OQOCH

WY = COQ)

(9-248)

(9-249)

(9-250)

Sec, 9-9

The tracking problem

795

We now combine Eqs. (9-246) and (9-247) to obtain the reduced canonical equations

ls] ~ [veal lool * Liveawl = 28

This is & system of 2n jinear time-varying differential equations. The vector W(t)z(t) acts as the “forcing function.” At ¢ = %, n boundary conditions are provided by the initial state

X(to) = &

(9-252)

Ati = T,n boundary conditions are provided by the transversality condition

g p(T) = ax(T) [He(T), Fe(T))] = C(T)FC(T)x(T) — C'(T)F2(T)

(9-253)

Let @(t; to) be the 2n X 2n fundamental matrix for the system (9-251).

We can now derive the relation

5 =

---n =
=

>

f

git)

[ A(t) -B(2) R24) Be) K (2) Fig. 9-17 The simulation of the system that produces g(t). and K(f) have been precomputed.

z(z)

Cz) Q(z)

_

2

f

ot

It is assumed that g(te)

II x(Zo)

(rf

yw»

| t g(£o)

Ke

ate) R21 Bit)

F=>(S

62)

x(t)

frre

Clz)

2

(2)

6 (2)

Fig. 9-18 The simulation of the optimal tracking system based on Eqs. (9-274) and (9-276). The matrix G(é) is given by G(t) = AQ) — BOR (OB’@EK@. It is assumed

that g(ts) and K{#) have been precomputed.

Since the mapping defined by Eq. (9-281) is linear, it is easy to establish the following theorem: Theorem 9-2 Let

Belts) = WUT; ACTF za(T) + f° WG; tC’ )QG)z6(7) ar

If

y

z(t) = Y. vetelr) B=1

B=1,2,...,N

7 Ete, TI

(9282)

(9-283)

where ys are any constants, then the insital-condition vector g(ts), corresponding

to the signal Zy,,.r), 18 given by

N

g(t) = > vege(to) Ax=1

(9-284)

Proor The theorem follows upon substitution of Eq. (9-283) into Eq. (9-281) and application of Eq. (9-282).

Sec. 9-10

Approximate relations

The implications of this theorem are as follows:

301

Suppose that we know

that the desired output signals z(#) are always linear combinations of known signals Z(t}; then we can precompute the initial vectors gg(t)) and generate g(t.) using Eq. (9-283), provided that no stability problems amse.

In the

following section, we shal] discuss certain approximate relations

for time-invariant systems as the terminal time 7 becomes increasingly large.

The following exercises, which can be done by mimicking the development of this section, will provide some additional theoretical results. Exercise 8-14

Consider the linear observable system

£() = A(x) + Bi)ult) + of) y(t) = C(é)x(f)

(9-285)

where w(t) is a known disturbance vector. Let z(t) be the desired output, let e(t>) = z(t) — y(t), and consider the cost functional J; of Eq. (9-267). Show that the optimal control is given by u(t) = R-'@)B’(@)(k() — K@)xz(0)) (9-286)

where K(¢) is the solution of the Riccati equation (9-269) with the boundary condition (9-270) and where the vector h(t) is the solution of the linear differential equation

hit) = — [A() — BOROW)B' OKO) hG) — C'OQWOzH) + KOelt)

(9-287)

with the boundary condition

A(T) = C’UT)Q(T)xAP)

(9-288)

Determine, using the Hamilton-Jacobi equation, the equation for the minimum cost. Now suppose that z{é) = 0 for all ¢ G[f, 7]. Can you generate h(¢) without knowing the disturbance «(r), for all r € [f, T]? Exercise 9-16 Consider the linear observable system

x(t) = A@x(t) + Biulé) + ol) y@®) = C&@)x(t) + D@)uld) + vid)

(9-289)

where w(t) and v(t) are known disturbance vectors. Let z(!) be the desired output, let e(t) = z(t) — y(6), and consider the cost J; of Eq. (9-267). Determine the optimal control and find the equation of the minimum cost.

9-10

Approximate Relations for Time-invariant Systems

In this section, we shall examine the tracking problem for linear timeinvariant systems, constant desired output vectors, and increasingly large T. All the results that we shall derive are epprozimate in nature and are valid for very large values of the terminal time T. Unfortunately, at this time, the theory which deals with the limiting case T = © is not available. Consider the observable and controllable Jinear time-invanant system x(t) = Ax(é) + But?)

y(t) = Cx(t)

(9-290)

802

Optimal linear systems with quadratic criteria

Let 2 be a constani vector which represents the desired output, so that the error e(t) is given by e(f) = Z— y@

=z — Crit)

(9-291)

Let us consider the cost functional

Ji =4 f (le, Qe) + (a, Ru(s))] at

(9-202)

and Jet us assume that the constant matrices Q and R are poartive definite. Clearly, the results of Control Law 9-5 can be used for the solution of this special problem, as long as T is specified and finite. We know thatt as 7'— =, the “gain” matrix K(t) of the Riccati equation

(9-269) tends to the constant positive definite matrix K.

Recall that the

matrix RK is the positive definite solution of the algebraic matrix equation —~RA — AK + RBR-'BR — C/QC = 0

(9-293)

Let the terminal time 7' be exceedingly large but finite. be two other very large times such that

Let 7, and T,

0K 0

(10-81)

We shall find the optimal control for each of the above problems. As we mentioned before, we shall derive the optimal controls using the Schwarz inequality. The proofs are relatively easy, because we can examine the first-order differential equation satisfied by the norm of x(¢). Since the system (10-76) is norm-invariant, we deduce that ||x(¢)|| is the solution of the first-order differential equation

5 let)!= (uQ), EET) = of

(10-82)

Since |/u(é)|| < m, we also conclude that v(é) is constrained by the relation

Wl = | (we), ar

0

(10-148)

where

T is fixed and x(T) is not specified.

10-7

Time-optimal Velocity Control of a Rotating Body with a Single Axis of Symmetry}

happens as T’'—>

«?

Determine the optimal control.

What

In this section, we shall apply the theory developed in Secs. 7-7, 7-9, and 10-6 to the problem of controlling the velocity of a spinning space body with a single axis of symmetry. Our purpose is twofold: First, we wish

to

indicate

the

use

of

optimal-control

theory in the preliminary stages

of design, and second, we wish to explicitly show the effects upon the performance of the system of changing the constraint on the control. In Secs. 7-7 and 7-9, we derived the equations of motion for the space body with a single axis of symmetry; we shall repeat the equations in this section for the sake of completeness. Suppose that the body described in Example 10-5 has a single axis of symmetry. Let us assume that the axis of symmetry is the 3-axis.

This means that the moments of inertia J, and

I, about the l-axis and the 2-axis, respectively, are equal, i.e., that T=[,=1,#1;

t The material in this section is based on Refs. A-5 and A-9.

(10-146)

842

Control constrained to a hypersphere

If we substitute Eq. (10-146) into Eqs. (10-134), we find that the angular

velocities y,(t), y2(t), and ys(t) satisfy the differential equations Tyi(t) = Z — Is)ys(é)y2(t)

Iye(t) = Is — Dyslt)yi()

T s3(t)

=

(10-147)

0

Now let us suppose that by some means we generate a torque vector u(t), so that the equations of motion become Iyi(t) = I — Is)ys()y2() + ui) Ty(t) = (Is — D)yslé)yi) + u2(d) Iayz(t) = us(t)

(10-148)

We are now faced with the task of generating the torques by some system. We can generate torques by gas jets, reaction wheels, etc. Let us suppose that we have decided to use gas jets. The next problem that we face is: Where should we place the gas jets on the body? If we examine Kas. (10-148), we observe that the “spin” velocity y3(¢) (which is the angular velocity of the body about its axis of symmetry) ponent w3(¢) because

is only a function of the torque com-

ys(t) = ys(0) + i. [{ ws) a

(10-149)

Clearly, the velocity y;3(t) is not a function of y(t) and y2(t); however, y:(t)

and y:(é) do depend on y;(é).

It is often desirablef to control the spin

velocity y3(t) independent of the other velocities.

agree that:

For this reason, let us

A jet J3, which generates a thrust f;(t), will be placed on the

body in such a manner that the torque vector us(t) produced by the thrust £;(¢)

as of the form

us(z) -|

0 0

|

(10-150)

us(t) In other words, the jet J; must generate a torque about the symmetry azxts only; this implies that the thrust f3(t) must be in a plane perpendicular to the sym-

metry axis (the 3-axt8).

Next, we consider the problem of generating the torques u,(t) and w2(é). We shall propose three different schemes of generating these two torques. Later on in this section, we shall compare the three schemes and draw some interesting conclusions.

t Suppose that the space body is a manned space station; then we can establish a constant artificial gravity by maintaining a constant spin velocity and keeping the other velocities y;(¢) and y:(t) at zero so that the space station will not tumble. If the space body is a reentry vehicle, then a spin velocity about the symmetry axis is desirable because it increases aerodynamic stability.

Sec. 10-7

Velocity control of a rotating body

443

In the following definitions, the components of a vector are measured along the 1-, 2-, 3-axes of the body. The axes 1, 2, 3 define a nght-hand coordinate system, so that all the rules of establishing directions in a righthand coordinate system are followed. Scheme A

Single Fixed Jet

A single gas jet J4 is fixed on the body (see Fig. 10-4a) so that the thrust

vector f4(t) has components

f4(t) =|

fo" O 0

(10-151)

and produces a torque vector u4(t) with components 0

0

uA(t) = | ust(t) | = | afit()

0

(10-152)

0

Thus, the thrust f4(¢) generates a torque u24(t) = af;4(t) about the 2-axis

only.

relation

We also assume that the thrust is constrained in magnitude by the

f4@|| = [f4@| S FA Scheme B

(10-153)

Two Fized Jets

Two gas jets J®! and J? are fixed on the body (see Fig. 10-46) so that their respective thrust vectors f71(t) and f27(#) have components f:?(6)

f21(¢) =

0 0

0

£52(t) = | f.8?(E) 0

(10-154)

and produce torque vectors u41(¢) and u??(t) with components u(t)

0

0

= 7 u2?1(t) | = | af? (2) 0 0

u52(¢) =

u18?(t) 0 0

=

arf 2?(t) 0 0

(10-155) Thus, the thrust f#1(¢) generates a torque u2?!(t) = af,?1(t) about the 2-axis only, and the thrust f27(t) generates a torque u)2*(t) = af:*?(). about the l-axis only. We also assume that the thrusts are constrained in magnitude by the relations

|f2*(¢)|| = |f122(t)| < FF

f7*(¢)|| = |f22*@)| < F?

(10-156)

(a)

(d)

(c)

Fig. 10-4

(a) The physical arrangement of the jets on the body for scheme A.

The

jet J4 is fixed on the body, and it can ‘‘fire’’ in both directions so as to generate a torque about the 2-axis only. (b) The physical arrangement of the jets on the body for scheme B. The jets J/®! and J?? are fixed on the body. The jet /*! generates a torque about the 2-axis only, and the jet J2? generates a torque about the l-axis only. (c) The physical arrangement of the jets on the body for scheme C. The jet J© is gimbaled and is allowed to rotate in a plane parallel to the 1-2 plane. The angle 6(t) specifies the direction of the thrust vector f¢(é).

Sec. 10-7

Scheme C

Velocity control of a rotating body

845

Single Gimbaled Jet

A single jet J© is placed on the body (see Fig. 10-4c) and is allowed to rotate in a plane perpendicular to the 3-axis. The thrust vector f°(¢) produced by the jet J° has components

_ | Ac The torque vector u°(¢) has components ad (

uc(t) = be 0

=

af o“(t

arte

(10-158)

Thus, the gimbaled jet can produce torques about the 1-axis and the 2-axis

simultaneously.

We

assume that the thrust vector f(¢) is constrained in

magnitude by the relation

fC) |] = VFS? + f2°@)? < FP

(10-159)

Let us now discuss the control problem from a physical point of view. Suppose that, prior to ¢ = 0, we have “fired” the jet J; in order to bring the spin velocity to a desired value—say 7; * O—=so that at ¢ = 0 we have

ys(0) = 9s

(10-160)

Owing to the gyroscopic coupling we shall have, in general, some nonzero initial values y,(0) and y2(0) for the other two angular velocities. In the absence of any external torques for t > 0, the body will tumble in space. It is then desired to determine the control torque that will reduce the velocities y,(¢) and y(t) to zero; in other words, it is desired to reduce the tumbling motion of the body to a pure spinning motion (at the desired spin

velocity 93). In order to have a meaningful optimal problem, we must have a performance criterion. We shall consider the minimum-time criterion. Thus, the physical problem is: Reduce the initial velocities y:(0) and y2(0) to zero in minimum time and maintain the spin velocity ys(t) at tts desired value G3, that is,

yx(t) = y3(0) = Js = const _— for allt > O

(10-161)

If we substitute y;(t) = 9; = const into Eqs. (10-148), then we can determine the differential equations of the velocities y,(f) and y2(é). It is more convenient, however, to deal with angular momenta; for this reason, let us define the angular momenta x,(t) and x.(t) by the equations

tilt) =TIyi(t) = a(t) = Tyo?) From

Eqs.

(10-162),

(10-161),

and

(10-148), we

(10-162)

find that the angular

446

Control constrained to a hypersphere

momenta satisfy the equations

%1(t) = waa(t) + ur(é)

(10-163)

Z(t) = —w2i(t) + ur(t) where w is & constant given by o = I TF Is Ys The two-dimensional

radians/second

angular-momentum

(10-164)

vector x(é) =

| is the pro-

jection of the three-dimensional angular-momentum vector : i(t) | r3(t)

Isfs

on the 1-2 plane.

The two-dimensional torque vector u(t) = ju

| is the

projection of the three-dimensional torque vector u(t)

tu2(t) 0

on the 1-2 plane. As we mentioned before, we wish to find the control vector u(t) that will drive the angular-momentum vector x(t) to 0 in minimum tvme. From the definitions of the three schemes of control, we conclude that we must solve for the time-optimal control to the origin for the following systems and constraints (we omit all the superscripts to avoid confusion):

Scheme A_

Single Fixed Jet ti(t)

>

= w2e(d)

io(t) = —ar,(t) + u(t) Scheme B

Two Fized Jets

#(t) = wre(t) + mil?)

te(t) = —wxi(t) + uelt) Scheme C

_—us()| < Fe

(10-165)

lus(t)| < F8a

(10-166)

ual?) < F Fo

Single Gimbaled Jet

y(t) = wae(t) + u(t) £4(t)

=

— w(t)

+

ua(t)

a/ui2(E)

+

Us*(b)