Practical course of the optimal control thery with examples. 9965296693, 1602070000

Citation preview

Al-Farabi Kazakh National University

S. Ya. Serovajsky

PRACTICAL COURSE OF THE OPTIMAL CONTROL THEORY WITH EXAMPLES

Almaty «Қазақ университеті» 2011

УДК 517.9 ББК 22. 1 С 28 Рекомендовано к изданию Ученым советом механико-математического факультета и РИСО КазНУ им. аль-Фараби

Рецензенты: доктор физико-математических наук, профессор М.Т. Дженалиев доктор физико-математических наук, профессор С.И. Кабанихин доктор физико-математических наук, профессор К.К. Шакенов

Серовайский С. Я. Практический курс теории оптимального управления с С 28

примерами: учебное пособие. – Алматы: Қазақ университеті, 2011. – 179 c. ISBN 9965-29-669-3 В учебном пособии на английском языке рассматриваются конкретные достаточно простые примеры задач оптимального управления, на которых осуществляется детальный разбор различных трудностей, возникающих при практическом решении оптимизационных задач. С помощью данного пособия учащиеся смогут не только познакомиться с основными направлениями теории экстремума и ее практическим применением, но и овладеть английской научной терминологией, что позволит им лучше ориентироваться в англоязычной математической литературе и представлять результаты своей работы на английском языке.

С

1602070000 - 150 068 - 10 460(05) - 11

ISBN 9965-29-669-3

УДК 517. 9 ББК 22. 1  Серовайский С.Я., 2011 © КазНУ им. аль-Фараби, 2011.

Contents___________________________ Preface ............................................................................................................... 6 Introduction...................................................................................................... 10 1. Problem formulation .......................................................................... 10 2. The maximum principle ..................................................................... 12 3. Example ............................................................................................. 17 4. Approximate solution of the optimality conditions............................ 21 Summary................................................................................................ 24 Example 1. Insufficiency of the optimality conditions ................................... 25 1.1. Problem formulation ....................................................................... 26 1.2. The maximum principle .................................................................. 26 1.3. Analysis of the optimality conditions ............................................. 28 1.4. Uniqueness of the optimal control .................................................. 32 1.5. Uniqueness of an optimal control in a specific example................. 35 1.6. Further analysis of optimality conditions........................................ 37 1.7. Sufficiency of the optimality conditions ......................................... 40 1.8. Sufficiency of the optimality conditions in a specific example ..... 42 1.9. Conclusion of the analysis of the optimality conditions ................. 45 1.9. Final remarks .................................................................................. 49 Summary................................................................................................ 51 Example 2. The singular control ..................................................................... 53 2.1. Problem formulation ....................................................................... 53 2.2. The maximum principle .................................................................. 54 2.3. Analysis of the optimality conditions ............................................. 55 2.4. Nonoptimality of singular controls ................................................. 60 2.5. Uniqueness of singular controls ...................................................... 62 2.6. The Kelly Condition ....................................................................... 66 Summary................................................................................................ 69 Example 3. Nonexistence of optimal controls................................................. 70 3.1. Problem formulation ....................................................................... 70 3.2. The maximum principle .................................................................. 71 3.3. Analysis of the optimality conditions ............................................. 72 3.4. Unsolvability of the optimization problem ..................................... 76 3

3.5. Existence of optimal controls.......................................................... 81 3.6. The proof of the solvability of an optimization problem ................ 83 3.7. Conclusion of the analysis .............................................................. 86 Summary................................................................................................ 91 Example 4. Nonexistence of optimal controls (Part 2).................................. 92 4.1. Problem formulation ....................................................................... 93 4.2. The maximum principle for systems with fixed final state ............. 94 4.3. Approximate solution of the optimality conditions........................ 96 4.4. The optimality conditions for problem 4......................................... 97 4.5. Direct investigation of problem 4 ................................................... 99 4.6. Revising the problem analysis ...................................................... 101 4.7. Problems with unbounded set of admissible controls ................... 102 4.8. The Cantor function ...................................................................... 106 4.9. Further analysis of the maximum condition.................................. 108 4.10. Conclusion of the problem analysis ............................................ 110 Summary.............................................................................................. 115 Example 5. Ill-posedness in the sense of Tikhonov...................................... 117 5.1. Problem formulation ..................................................................... 118 5.2. Solution of the problem ................................................................ 118 5.3. Ill-posedness in the sense of Tikhonov ......................................... 119 5.4. Analysis of well-posedness in the sense of Tikhonov.................. 124 5.5. The well-posed optimization problem .......................................... 125 5.6. Regularization of optimal control problems.................................. 127 summary .............................................................................................. 128 Example 7. Insufficiency of the optimality conditions (Part 2)..................... 129 7.1. Problem formulation ..................................................................... 129 7.2. The existence of an optimal control .............................................. 130 7.3. Necessary conditions for an extremum ......................................... 133 7.4. Transformation of the optimality conditions................................. 135 7.5. Analysis of the boundary value problem ...................................... 137 7.6. The nonlinear heat conduction equation with infinitely many equilibrium states ................................................................................. 143 7.7. Conclusion of the analysis of the variational problem .................. 144 Summary.............................................................................................. 146 Example 8. Extremal bifurcation................................................................... 147 8.1. Problem formulation ..................................................................... 147 8.2. The necessary condition for an extremum .................................... 148 8.3. Solvability of the Chafee – Infante problem ................................. 149 8.4. The set of solutions of the Chafee – Infante problem ................... 152 8.5. Bifurcation points.......................................................................... 154 4

Summary.............................................................................................. 156 Conclusion ..................................................................................................... 158 Tasks............................................................................................................... 159 Comments....................................................................................................... 164 Bibliography .................................................................................................. 166 Short Russian-English mathematical dictionary ............................................. 173

Preface The theory of optimal control is one of the most important fields of modern mathematics. It is an essential part of the extremum theory and a natural extension of the classical variational calculus. The greatest mathematicians including Fermat, Euler, Lagrange, Weierstrass, and Hilbert laid the foundations of the theory of optimal control. It is connected with a great number of extremely complicated mathematical problems and has many practical applications in various branches of science and technology. It is capable of providing the most beneficial thing mathematics can give to practical applications: the possibility to precisely specify the conditions for a process to achieve the required result. General principles of solving optimal control problems are well known. As a rule, they involve specific iterative procedures based on the gradient methods for minimizing functionals or on the necessary optimality conditions. Formulation of a general algorithm for solving a relatively large class of important optimization problems is not so difficult nowadays even for graduate and postgraduate students, engineers, and researchers familiar with standard courses of differential equations, optimization theory, and approximation methods. Any competent researcher is able to choose an appropriate algorithm for solving a given problem of optimal control and implement it on a computer. Naturally, a computer will produce some output for any reasonable algorithm. Unfortunately, experience suggests that iterative procedures for the optimal control problems usually do not converge. Moreover, if a procedure does converge, the obtained limit does not always represent a solution of the problem or even its approximation. These difficulties are usually rooted in the specific features of optimization problems rather than the imperfection of numerical methods. Most of these difficulties could be avoided by performing a complete qualitative analysis of the problem in question before implementing the algorithm on a computer. Unfortunately, when solving optimal control 6

problems, it is usually impossible to completely analyze all the related problems in the setting because of extreme complexity. In practice, we are often forced to apply certain methods of optimization in a formal way, without rigorous substantiation. To some extent, this approach is justified. Indeed, it is worth trying to formulate an algorithm and implement it on a computer hoping for a satisfactory result rather than investigating extremely complex theoretical problems that may have no practical application. Moreover, this kind of investigation may require sophisticated theoretical skills, assuming that the complete analysis of the problem is possible at all. In some cases, formal application of standard optimization methods turns out to be successful. However, very often it turns out the other way: the iterative process either diverges or converges to a wrong result. This is the price of insufficient theoretical research of the optimization problem. What is the best approach in this situation? First, it is important to be able to determine whether the result is wrong. It is not a good idea to put the reputation of mathematical and computer-aided methods at further risk by giving out a wrong result to the end user. Second, if the result is wrong, it must be properly analyzed and the causes must be established. It would be unreasonable to repeat the iterative process with various initial approximations hoping that it will converge, especially when the extremum problem is actually unsolvable. Revealing the true cause of the wrong result makes it possible to overcome the difficulties by applying the appropriate mathematical methods or correcting the formulation of the problem. Therefore, it is necessary to know the ways in which wrong results can manifest themselves in practice. In this context, the analysis of "good" examples that can be solved using classical optimization methods is hardly useful. Besides, these problems have received extensive coverage in the literature (see Comments). In contrast, the purpose of this book is to present a qualitative analysis of "bad" examples of optimal control problems. All the considered examples are rather simple. This allows us to present a detailed analysis avoiding cumbersome mathematical calculations that may obscure the essence of the problem. At the same time, these examples are far from being trivial. They represent adequate models of some types of unpleasant situations that may arise when solving optimization problems in practice. We shall restrict our consideration to the classical optimal control problems described by ordinary differential equations. In Introduction, we outline the standard procedure for solving these problems based on the necessary optimality condition in the form of the maximum principle with subsequent application of the method of successive approximations. This approach has been chosen because of its popularity and applicability for 7

practical solution as well as for qualitative analysis. The efficiency of the described method is illustrated by an elementary example. Subsequent chapters deal with the optimal control problems for a simple dynamic system (which is the same in all the examples). The extremal problems considered therein differ only in the structure of the set of admissible controls and in the features of the optimality criterion. The analysis of each example begins with formal application of the general solution procedure described above. However, certain difficulties arise during the investigation that reflect the imperfection of the optimization methods being applied. We gradually reveal the causes of these difficulties and describe a class of extremum problems for which they don't arise. Furthermore, we find out specific features of the problem setting that lead to unsatisfactory results. Practical recommendations for overcoming the difficulties are given. The optimality conditions in the first example have too many solutions, of which only two are optimal. Thus, there is no uniqueness of solution and the optimality conditions are not sufficient at the same time. In the second example, the form of the maximum principle is degenerate and its solution is a singular control. In the next two examples, the optimal control does not exist and the optimality conditions have no solutions, although the function H in the fourth example has local maxima. In the fifth and sixth examples, we consider examples of optimal control that are ill-posed in the sense of Tikhonov and Hadamard, respectively. In the case of the fifth example, not every minimizing sequence converges to the optimal control. In the sixth one, the dependence of solutions on the problem parameter is not continuous. In two final examples, the optimality conditions are reduced to nonlinear differential equations with nonunique solutions, the set of solutions being infinite in the seventh example. In the eighth one, the number of solutions depends explicitly on the problem parameter. This is extremal bifurcation phenomenon. This book is based on the author's course in Counterexamples in the Theory of Optimal Control read at the Faculty of Mathematics and Mechanics of the al-Farabi Kazakh National University. It is an improved and updated version of Serovajsky (2001, 2004). The work was inspired to a large extent by the excellent collection of counterexamples in Gelbaum and Olmstead (1967). This book is intended for students, postgraduate students, engineers, and researchers who are interested in the theory of optimal control and its applications. I would like to thank my referees Professors S.A. Aisagaliev, S.I. Kabanihin, and K.K. Shakenov. I am grateful for useful discussions with the 8

specialists in the mathematical theory of optimal control F.A. Aliev, V.G. Boltyanskii, A.G. Butkovskii, F.P. Vassil'ev, M. Gebel, M.T. Djenaliev, A.I. Egorov, A. Griewank, A.D. Iskenderov, V.G. Litvinov, K.A. Lur'e, V.S. Neronov, U.Ya. Raytum, T.K. Sirazetdinov, V.M. Tikhomirov, R.P. Fedorenko, A.V. Fursikov, and V.A. Yakubovich. I also wish to acknowledge the influence of the remarkable books by J.-L. Lions (19721987) on my development as a researcher. I am grateful to Yu.I. Dzibalov, T.Yu. Kabirova, V.V. Monastyreva, and V.L Shcherbak, and for their help in preparing this book for publication. I also wish to thank the students and staff members of the Department of Mathematics and Mechanics of the al-Farabi Kazakh National University for their attention and support.

Introduction Our primary goal is to describe a standard solution procedure for a certain class of optimal control problems. We restrict our consideration to the case where the state of a system is defined by the Cauchy problem for a general first order ordinary differential equation. A function with values in a given interval is the control. An integral functional represents the optimality criterion. We shall deduce the necessary optimality conditions in the form of the maximum principle. As a result, we shall obtain a certain problem of finding a relative extremum, its parameters being represented by the function of the system state and the solution of the adjoint system. We apply the method of successive approximations to find an approximate solution of the problem. As an example, we consider a very simple optimization problem such that the algorithm converges to its unique optimal control. Then this method will be applied to much more complicated problems with fixed final state (see Example 4) and an isoperimetric condition (see Example 7). In the examples below, we shall see various unfavorable situations caused by substantial difficulties in solving the optimization problems described above. In particular, these are situations where application of standard methods does not give immediate results. However, experience and better understanding can be gained only while solving "bad" problems. Easy success in solving "good" problems is hardly beneficial in this respect. 1. PROBLEM FORMULATION First of all, a problem of optimal control involves a mathematical model of the process in question, which is usually represented by some equation or a system of equations for the unknown state functions. Since the process in question is assumed to be controllable, the state equation must include a parameter called the control (which may be a number, a vector, a function, a 10

collection of functions, etc.) selected by the researcher. Thus, the mathematical model of a controllable system in the general case be described by the operator state equation

A[u, x(u )] = 0. Here A is an operator, u is the control, and x (u ) is the function of the system state corresponding to the given control. In practice, the choice of the control is usually denned by certain technical, technological, or economical conditions. For this reason, the control is assumed to belong to a set of admissible controls U. By choosing an admissible control we determine a specific way of process development. Let a functional I = I (u , x ) represent the optimality criterion for the selected control. Then the optimal control problem is written as follows: Problem P0. Find a control u ∈ U that minimizes the functional u → I (u , x (u )) on U. Being formulated in such a general form, this problem obviously cannot be solved. We shall essentially simplify it, but it will still define a sufficiently large class of applied optimal control problems that are far from being trivial, as we shall see below. Consider a system described by the differential equation

x& = f (t , u , x) , t ∈ (0, T ) ,

(0.1)

with the initial condition

x (0) = x0

(0.2)

Here x = x (t ) is the function of the system state, u = u (t ) is the control, f is a known function, x0 is the initial state of the system. We assume that the set of admissible controls is

U = {u u1 ≤ u (t ) ≤ u2 , t ∈ (0, T )} , where u1 and u2 are known values, which may be − ∞ and ∞ , respectively. Solving the Cauchy problem (0.1), (0.2) for some admissible value u, we can determine the evolution of the system, i.e., the function of the system state at every instant. The integral functional

11

T

I = I (u , x) = ∫ g ( t , u (t ), x(t ) ) dt 0

will represent the optimality criterion, where g is a known function. If the system state is uniquely determined by the control, then the optimality criterion will depend only on the control. As a result, we obtain the following formulation. Problem P. Find a function u ∈ U that minimizes the functional f on U. Our purpose is now to develop a method of solution for the above problem. 2. THE MAXIMUM PRINCIPLE In accordance with the method of Lagrange multipliers, we introduce the functional T

L(u , x, р) = I (u , x) + ∫ р(t ) [ x& (t ) − f ( t , u (t ), x(t ) )] dt. 0

Evidently, if the function x satisfies equation (0.1), then the functionals L and I coincide for any function p. In this situation we may try to pass from the original problem of finding a relative extremum to the problem of minimizing the functional L, which contains (in a certain sense) the information about the system conditions in the form (0.1). We set

H (u , x, p ) = f (u , x ) − g (u , x ).

(0.3)

Then the functional L becomes as follows: T

L(u, x, р ) =

∫ [ р(t ) x&(t ) − Н (u (t ), x(t ), p(t )) ] dt. 0

Suppose that u is the optimal control, i.e.,

ΔI = I (v, y ) − I (u, x) ≥ 0, ∀v ∈ U , where x and y are solutions to problem (0.1), (0.2) for the controls u and v, respectively. Since the functionals in question coincide on the set of solutions of equation (0.1), we can reduce inequality (0.4) to the relation 12

ΔL = L(v, y , p ) − L (u , x, p ) ≥ 0 ∀v ∈ U , ∀p.

(0.5)

We now determine the increment of L T

T

0

0

ΔL = ∫ р(t ) [ у& (t ) − x& (t )] dt − ∫ ΔHdt ≥ 0 ∀v ∈ U , ∀р, where

ΔH = H (v, y , p ) − H (u , x, p ). The known functions f and g in the problem formulation are assumed to be sufficiently smooth. Putting Δx = y − x and using the Taylor series expansion, we obtain

H (v, y, p ) = H (v, x + Δx, p ) = H (v, x, p ) + H x (v, x, p ) x + η1 =

= H (v, x, p ) + H x (v, x, p) x + η1 + η 2 . Here H x = ∂H / ∂x, η1 is a term of higher order in Δx , and

η 2 = [ H x (v, x, p ) − H x (u , x, p ) ] Δx. As a result, inequality (0.5) is reduced to the form T

T

0

0

∫ р(t ) Δx& (t ) dt − ∫ [ Δu H + Н х Δx ] dt + η ≥ 0 ∀v ∈U , ∀р,

(0.6)

where Δ u H is the increment of H with respect to the control, i.e.,

Δ u H = H (v, x, p ) − H (u , x, p ). and the remainder term η is defined by the formula T

η = − ∫ (η1 + η 2 )dt.

(0.7)

0

Remark 1. In the following examples, the explicit form of the remainder term will be of particular interest.

Integrating by parts, we find the value of the first integral in (0.6):

13

T

T

∫ р(t )Δx& (t )dt = р(Т )Δx(Т ) − р(0)Δx(0) − ∫ р& (t )Δx(t )dt = 0

0

T

= р (Т ) Δx (Т ) − ∫ р& (t ) Δx (t ) dt. 0

We take into account that the initial state of the system is equal to xo for every control. Thus, inequality (0.6) becomes as follows: T

T

0

0

− ∫ Δ u Hdt − ∫ (H x + p& )Δxdt + p(T )Δx(T ) + η ≥ 0∀v ∈ U , ∀p. (0.8) We now try to make inequality (0.8) as simple as possible by an appropriate choice of the arbitrary function p. The obvious solution is to make the second and the third terms on the left-hand side vanish. Thus, p must satisfy the equation

p& = − H x , t ∈ (0, T )

(0.9)

with the condition

p(T ) = 0.

(0.10)

Relations (0.9), (0.10) are called the adjoint system. As a result, inequality (0.8) takes the form T

− ∫ Δ u Hdt + η ≥ 0 ∀v ∈ U .

(0.11)

0

We take a function v sufficiently close to the optimal control u. If the solution of the problem (0.1), (0.2) continuously depends on the control, then the increment Δx of the function of the system state is sufficiently small. Then the first and the second terms in the left-hand side of (0.11) will be of the first and the second infinitesimal order, respectively. In this case, we can assume that the sign of the left-hand side is defined by the sign of its first term. Thus, for v sufficiently close to the optimal control, we have T

∫ {H [v(t ), x(t ), р(t )] − H [u (t ), x(t ), р(t )]} dt ≤ 0. 0

14

(0.12)

Let τ be a point in the interval (0,T) and w be an admissible control. We now define the following control (see Figure 1):

⎧u (t ), if t ∉ (τ − ε ,τ + ε ), v (t ) = ⎨ ⎩ w(t ), if t ∈ (τ − ε ,τ + ε ). This is the so-called spiky variation of the control u. v v w u u

v

w

t

τ

τ−ε

τ+ε

Figure 1. The spiky variation of the control u

The function v belongs to U and can be as close to u as desired for ε sufficiently small. Then equality (0.12) holds for this control. Taking into account that the controls u and v coincide outside the interval (τ − ε , τ + ε ) , we reduce equality (0.12) to the form τ +ε

∫ {H [ w(t ), x(t ), р(t )] − H [u (t ), x(t ), р(t )]} dt ≤ 0.

τ -ε

Dividing the left-hand side of the foregoing inequality by 2ε, passing to the limit as ε → 0 , and using the Mean-Value Theorem, we obtain

H [w( τ ), x( τ ), ð( τ )] − H [τ , u ( τ ), x( τ ), ð( τ )] ≤ 0 . Since the point τ and the control w are arbitrary, we conclude that

H [ u (t ), x (t ), р (t ) ] = max H [ w, x (t ), р (t ) ] , t ∈ (0, T ), w∈U

15

(0.13)

As a result, we obtain the so-called necessary optimality condition for a control. Theorem 1 [The Maximum Principle]. For the control u to be the optimal control problem, it is necessary that it satisfy the maximum condition (0.13), where x is the corresponding solution of the problem (0.1), (0.2) and p is a solution of the adjoint system (0.9), (0.10). Remark 2. A perfectly rigorous formulation of the hypotheses of this theorem is not our goal. We are now interested only in the general pattern of arguments that lead to the maximum principle and, mainly, in its practical application. Such an approach is justified because it is usually impossible to provide a complete rigorous analysis of an optimization problem in practice. We need to seek a solution in spite of the problems with analytical methods. Indeed, the main causes of various unexpected problems to arise later are the lack of rigor in formulation and the ignored restrictions ensuring that the maximum principle holds. In fact, all the examples considered below represent certain problems caused by formal application of standard optimization methods.

Following the maximum principle, in order to solve the optimization problem, we must find the functions u, x, p from relations (0.1), (0.2), (0.9), (0.10), (0.13). The maximum principle is effective in this case because of our transition from the problem of minimization of the original functional to the problem of finding the relative extremum of the function H, which explicitly depends on the control. The price of this transition is the introduction of the new unknown function p. Remark 3. If the optimality criterion included the final value of the function of the system state, then the boundary condition in the adjoint system would not be homogeneous, and the remainder term would have an additional summand involving the square of the increment of the system state at the final instant. Remark 4. In Examples 4, 7, and 8, we consider optimal control problems with fixed final state, for which the system state is specified in both the initial and final instants. We show that there are no boundary conditions for the adjoint equation in this case. In Example 7, we establish the maximum principle for the optimal control problem with an additional condition imposed on the system state (isoperimetric condition).

The following example illustrates the potential of the maximum principle.

3. EXAMPLE We now consider a simple example that helps to get a better understanding of the maximum principle and demonstrate its effectiveness. Suppose that the state of a system is described by the conditions

x& = u , t ∈ (0,1); х (0) = 0.

(0.14)

Let the set of admissible controls be

U = {u u (t ) ≤ 1, t ∈ (0,1)} , and let

I=

1

∫ (u 2

1

)

+ x 2 dt.

2

0

Consider the following optimization problem. Problem 0. Find a function u that minimizes the functional I on U. To bring this problem to the standard form, we put

f (u , x ) = u , x0 = 0, T = 1, g (u , x ) = (u 2 + x 2 ) / 2, u1 = −1, u2 = 1. Following formula (0.3), we introduce the function

H = H (u ) = pu − (u 2 + x 2 ) / 2. Then the adjoint system (0.9), (0.10) becomes as follows:

р& = х, t ∈ (0,1); р (1) = 0;

(0.15)

and the maximum condition (0.12) is

H (u ) = max H ( w). | w| ≤1

(0.16)

Thus, we have three relations (0.14) – (0.16) for three unknown functions u, x, and p. First, we solve problem (0.16) to find the relative extremum of H. Equating its derivative to zero, we obtain the stationary state condition

17

∂H = p − u = 0, ∂u which implies that H has a unique point of local extremum (the stationary point) u = p. Since the second-order derivative of H is negative, this is a maximum point. Remark 5. Example 4 deals with an interesting situation where the secondorder derivative of H is negative and the stationary point is a local maximum that is not a global one and therefore does not satisfy the maximum condition.

The obtained value corresponds to the absolute extremum of the function in question. However, the position of the point u = p relative to the admissible segment [-1,1] may be arbitrary (see Figure 2).

Н p 1 , H increases on this segment; therefore, its maximum is achieved for the maximum admissible control u = 1 . Finally, for u < 1 , the value u = p is admissible; therefore, it is a solution of the maximum condition. Taking into account that p depends on time, we obtain the formula

⎧ -1, if р(t ) < -1, ⎪ u (t ) = ⎨ р (t ), if -1 < р (t ) < 1, ⎪ 1, if р (t ) > 1. ⎩

(0.17)

Formula (0.17) gives the solution of the maximum condition (0.16) and allows us to find the control if the function p is known {see Figure 3).

р 1

u

u р t р

u

-1

р Figure 3. Solution of the maximum condition for a known value of p

Substituting this value into (0.14), we obtain the system (0.14), (0.15) for the unknown functions x and p. If it turned out to be a Cauchy problem for two (possibly nonlinear) differential equations, then the solution would be easy. There are many simple and reliable numerical algorithms for solving such problems. However, the boundary conditions for x and p are specified at different instants of time. In this case, we can solve the system using an iterative method.

20

4. APPROXIMATE SOLUTION OF THE OPTIMALITY CONDITIONS The differential equations in (0.14) and (0.15) cannot be solved in parallel, since the boundary conditions for the state function and the solution of the adjoint system are specified at the opposite instants of time. Therefore, these relations should be treated sequentially. Using the method of successive approximations, we find the function xk at the k-th iteration solving the Cauchy problem

x&k = uk , t ∈ (0,1); xk (0) = 0

(0.18)

for the known control uk . Then we solve the adjoint system

р& k = хk , t ∈ (0,1); рk (1) = 0

(0.19)

to find the function pk . Finally, the next approximation of the control uk +1 is determined from the formula

⎧ -1, if рk (t ) < -1, ⎪ uk +1 (t ) = ⎨ рk (t ), if -1 < рk (t ) < 1, ⎪ 1, if р (t ) > 1. k ⎩

(0.20)

Remark 6. In addition to the optimality conditions, gradient methods are also used for solving extremum problems. The idea of these methods is that every approximation of the control is obtained from the previous one by the shift in the direction of the gradient of the functional. In the problems with constraints, in addition, the result is projected onto the set of admissible controls. If we use the gradient-projection method in the problem under consideration, then the resulting iteration formula will coincide with (0.20). In particular, as follows from Figure 3, the transition from the known function p to the unknown u is made using the procedure of projection onto the segment of admissible values of the control.

We now try to apply the method of successive approximations directly to the system of optimality conditions. Let u0 be an initial approximation. It must belong to the set of admissible controls, i.e.,

−1 ≤ u0 (t ) ≤ 1, t ∈ [ −1,1]. Integrating this inequality from aero to an arbitrary t and using (0.18), we have 21

t

−t ≤ x0 (t ) = ∫ u0 (τ ) dτ ≤ t. 0

Further integration of the obtained relation from t to unity (the terminal point) yields

−

1 2

t2 −1

≤

2

1− t2

1

≤ p0 (t ) = − ∫ x0 (τ ) dτ ≤

2

t

≤

1 2

.

Since all values of p0 belong to the segment [-1,1], following (0.20), we find the next approximation of the control u1 = p0 . We have

−1/ 2 ≤ u1 (t ) ≤ 1/ 2, t ∈ [ −1,1]. Integrating this inequality, we get t

−

t t ≤ x1 (t ) = ∫ u1 (τ )dτ ≤ . 2 2 0

Further integration of the obtained inequality from t to unity gives

−

1 4

≤

t2 −1 4

1

≤ p1 (t ) = ∫ x1 (τ ) dτ ≤

1− t2

t

4

≤

1 4

.

Then (0.20) implies that the next approximation of the control satisfies the inequality

−1/ 4 ≤ u2 (t ) ≤ 1/ 4, t ∈ [ −1,1]. Repeating the above calculations for the next iteration, we obtain

−1/ 8 ≤ u3 (t ) ≤ 1/ 8, t ∈ [ −1,1]. In the general case, for the kth iteration we have the following estimate:

uk (t ) ≤ 2 − k , t ∈ [ −1,1]. Thus, we have established the convergence u k (t ) → 0 as k → ∞ . The obtained results show that the sequence {u k } constructed using the method of successive approximations converges to the function u*, which is 22

identically equal to zero, for any initial approximation of the control chosen from U. A natural question arises of whether u* will be a solution of the considered optimal control problem? To answer this question, we return to the formulation of the problem. Since the integrand in the functional to be minimized is nonnegative, we have I ≥ 0 0 for every admissible control. Zero value of the functional is achieved if and only if

u (t ) = 0, x(t ) = 0, t ∈ [ −1,1]. The zero control is admissible. Moreover, according to problem (0.14), it defines the state function, which is also identical zero. Thus, it is the admissible control u* that makes the functional vanish; furthermore, the functional does not assume negative values. Therefore, this optimal control problem has a unique solution, which is the one we found as a result of approximate solution of the optimality conditions. Remark 7. The obvious goal in this case was to verify the effectiveness of the maximum principle, rather than solve a specific optimization problem (which is quite simple). Remark 8. Later, we shall return to the problem considered above. Theoretical results established below will be applied to this problem, for it has truly remarkable properties. Further investigation will reveal where these properties stem from.

The method of successive approximations is also applicable for solving the system of optimally conditions (0.1), (0.2), (0.9), (0.10), (0.13) for the general extremum problem. This involves successive solution of the state equation, the adjoint problem, and the optimality condition (the maximum condition). Naturally, the algorithm does not necessarily converge in the general case; however, approximate solution of the problem is still possible. Remark 9. It is not to be supposed that an algorithm that converges very well to the "right solution" can allow the determination of the optimal control with accuracy as high as desired. As a rule, differential equations are solved using approximation methods. Thus, the iteration error introduced by the initial approximation is aggravated by the equations approximation error, which is of entirely different kind. The iteration error is usually the governing factor at the initial stage of the algorithm. As the algorithm converges, the control approximation tends to the optimal control and the iteration error decreases. However, the equations approximation error remains constant unless the method of solution of the differential equations changes in the process of calculations. The approximation error will start to play the decisive role at some point in the solution process, and there is no way to deal with this in our method. Thus, the algorithm will fail sooner or later, as the

23

functional will start to increase unexpectedly. Then each new approximation of the control will be worse than the previous one, causing the iteration error to increase. This error will become predominant again, which will cause the algorithm to minimize the functional until the iteration error becomes less than the approximation error. As a result, the algorithm will be oscillating. In general, it is possible to overcome this obstacle and achieve greater accuracy by gradually improving the approximation of the differential equations.

Summing up the results of our analysis, we see that, on one hand, the solution procedure for any given optimal control problem is hardly simple. On the other hand, in general, it is possible to achieve sufficiently high accuracy with the help of the technique described above. Unfortunately, it is not always the case. The purpose of this book is to demonstrate different kinds of unforeseen obstacles that arise in solving optimal control problems (even not particularly difficult ones). We shall try to understand the nature of those obstacles and find the ways to overcome them. SUMMARY The obtained results lead us to the following conclusions. 1. The optimal control problem consists of the state equation for the control, the set of admissible controls, and the optimality criterion represented by a functional defined on the set of admissible controls. 2. To solve the optimal control problem, the optimality condition in the form of the maximum principle can be used. 3. The maximum principle is the problem of finding a relative extremum for a function H which includes the state function and the solution of the adjoint system as parameters. 4. The first step in the solution of the system of optimality conditions is to express the control from the maximum condition for H in terms of the other unknown quantities. 5. The major difficulty in solving the optimality conditions is that the boundary conditions for the state function and the solution of the adjoint system are specified at the opposite instances of time. 6. The complete system of optimality conditions can be solved using an iterative procedure, namely, the method of successive approximations. 7. With the method of successive approximations used for solving the system of the optimality conditions, it is possible to find the solution of the optimal control problem with sufficiently high accuracy. 24

Example 1. Insufficiency of the optimality conditions We have described the general scheme for solving optimal control problems and considered a specific example that proved our scheme to be highly effective. However, perfect situations like this are by no means frequent. Unfortunately, there are numerous obstacles on the way of solving optimization problems, and we should get prepared to overcome them. The first counterexample considered below is a rather simple optimization problem that differs from the previous one in the type of extremum only. Once again we use the maximum principle in the problem analysis. The resulting system of optimality conditions is easy to solve iteratively, using the method of successive approximations. However, it turns out that different initial approximations correspond to different limits of the sequence of controls. Moreover, we shall see that the values of the optimality condition are not the same for different limit controls. These complications might imply some essential drawbacks in the scheme of solving optimal control problems described above. The optimal control turns out to be not unique in the considered example, while the solution of the problem is not the only function that satisfies the maximum principle. As a consequence, the system of optimality conditions has by no means a unique solution. Thus, even if the iterative process converges, we cannot guarantee that the problem does not have other solutions and that the resulting control is optimal. We must establish the restrictions to be imposed on the system for the optimization problem to have a unique solution and for the optimality conditions in the form of the maximum principle to be necessary and sufficient. These conditions did hold in the previous example, which is why it could be solved successfully. In the next example, these conditions are guaranteed to fail; however, it will not prevent us from finding the solution.

25

1.1. PROBLEM FORMULATION We consider a rather simple example very similar to the previous one. Let the state of the system be described by the Cauchy problem

x& = u , t ∈ (0,1); х(0) = 0.

(1.1)

The control u = u (t ) is chosen from the set

U = {u u (t ) ≤ 1, t ∈ (0,1)} . In this case, the optimality criterion is

I=

1

(u 2∫

1

2

+ x 2 ) dt.

0

We now formulate the following problem of optimal control. Problem 1. Find a control u∈U that maximizes the functional I on U. Remark 1.1. The only difference between Problem 1 and the previous example is in the type of extremum.

Although the scheme of solving optimization problems described above involved minimizing the functional, this is a nonessential restriction. Clearly, a control that minimizes the functional -I will maximize the functional I. While for the minimization of I it is required to find the maximum of the function H, for the maximization of I we shall obviously need to minimize H. Therefore, we may use the above technique to analyze Problem 1. Since this problem is very similar to the previous one, it may seem natural to assume that there will be no unpleasant surprises and hope that the optimal control can be found with the same ease as before. However, it will be amazing to find that Problem 1 differs dramatically from the previous problem, and this fact will have far-reaching implications. 1.2. THE MAXIMUM PRINCIPLE In order to reduce the problem in question to the standard form described before, we introduce the following notation:

f (t , u, x) = u , x0 = 0, T = 1,

26

g (t , u, x) = ( u 2 + x 2 ) / 2, u1 = −1, u2 = 1 This notation is the same as the one used in the analysis of the preceding example. Following formula (0.3), we define the function

H = H (u ) = pu − (u 2 + x 2 ) / 2 . Then the adjoint system (0.9), (0.10) becomes as follows:

p& = х, t ∈ (0,1); p (1) = 0.

(1.2)

In contrast to the previous problem, here we have to maximize the functional. Therefore, the problem that corresponds to the maximum condition (0.13) is

H (u ) = min H ( w). | w| ≤1

(1.3)

Thus, we have three relations (1.1) – (1.3) for three unknown functions u, x, p. These relations differ from the system (0.14) – (0.16) only in the type of extremum of H. As before, we first find the solution of the optimality condition (1.3) by expressing the control in terms of the other unknown functions. Equating the derivative of H to zero, we obtain

∂H ∂u

= p − u = 0.

Hence, H has a unique point of local extremum и = p (obviously, the same as in the previous example). Since the second derivative of H is negative, we have the maximum. This means that H has no local minima at all. Therefore, the relative minimum of H may be achieved only on the boundary of the set of admissible controls. The boundary values are

H (1) = p − (1 + x 2 ) / 2, H ( −1) = − p − (1 + x 2 ) / 2, The solution of the minimum condition (1.3) must correspond to the minimum of these two values. Thus, we have the formula

⎧ 1, р (t ) < 0,

u (t ) = ⎨

⎩ −1, р (t ) > 0. 27

(1.4)

Given the solution of the adjoint system (see Figure 4), formula (1.4) allows us to find the desired control. Thus, (1.4) is similar to the result of (lie analysis of the problem of finding a relative maximum of H.

р u

1

р

0

t

р -1

u

u

Figure 4. The solution of the minimum condition (1.3) for a given p Remark 1.2. As follows from formula (1.4), the desired solution can be discontinuous. However, this should not cause much trouble because examples of discontinuous controls are perfectly possible in practice: suppose a control system with a relay that operates at some point switching the system to a different mode. There is a wide range of practically justified extremum problems whose optimal controls are discontinuous.

So we have the system (1.1), (1.3), (1.4) for three unknown functions u, x, and p. As before, we shall solve the obtained problem using an iterative method. First, we solve the Cauchy problem (1.1) for a given control to find the corresponding system state. Then we solve the adjoint system (1.3) and determine the new approximation of the control using formula (1.4). This process continues up to the point of achieving the desired accuracy of the solution. It is not obvious that the described algorithm converges; therefore, the system of optimality conditions requires additional analysis. 1.3. ANALYSIS OF THE OPTIMALITY CONDITIONS As follows from formula (1.4), the control may assume only two values, the choice being determined by the sign of p. In particular, suppose that

u (t ) = 1, t ∈ (0,1), 28

which holds when p is negative. Substituting this value into (1.1), we find the state t

x(t ) = ∫ u (τ )dτ = t. 0

Integrating this equality from t to unity, we have 1

1

t2 −1 р (t ) = − ∫ x (τ ) dτ = − ∫ τ dτ = . 2 t t Hence, the function p = p (t ) assumes only negative values for t ∈ (0,1). Then (1.4) implies that p corresponds to the control which is identically equal to unity. We now conclude that the three functions

u(t) = 1, x(t) = t, p(t) = (t 2–1)/2, t∈(0,1)

(1.5)

are indeed the solution of the problem (1.1), (1.3), (1.4). In particular, if the control identically equal to unity is chosen to be the initial approximation, then the corresponding iterative process converges in one iteration. This is natural, since the initial approximation is a solution of the problem in this case. We now assume that

u (t ) = −1, t ∈ (0,1), which holds when p is positive. Substituting this function into equation (1.1), we find the state t

x(t ) = ∫ u (τ ) dτ = −t. 0

Integrating this equality from t to 1, we have 1

1

t

t

р (t ) = − ∫ x(τ ) dτ = ∫ τ dτ =

1− t2 . 2

Hence, the function p = p (t ) assumes only positive values for t ∈ (0,1). By formula (1.4), p corresponds to the control identically equal to –1. Thus, the three functions 29

u(t) = –1, x(t) = – t, p(t) = (1– t2)/2, t∈(0,1)

(1.6)

are also the solution to the problem (1.1), (1.3), (1.4). In particular, if the control identically equal to –1 is chosen to be the initial approximation, then the iterative process converges in one iteration step as well. We have determined two solutions of the system (1.1), (1.3), (1.4). Our ultimate goal, though, is to solve the extremum problem rather than the optimality conditions. Therefore, an important question arises of whether the values of the functional I are the same for these solutions. The calculations yield

I (1) =

1

∫ (1 + t ) dt = 2

1

2

1 + 1/3

0

I ( −1) =

1

∫ ( (−1) 2

1

2

)

+ ( −t ) 2 dt =

0

2

=

2 3

1 + 1/3 2

,

2 = . 3

We see that the values of the functional are the same for both controls, i.e., both controls are equally valid solutions of the optimization problem. Conclusion 1.1. The system (1.1), (1.3), (1.4) has two solutions such that the value of the functional is the same for both of them. Note that the solutions (1.5) and (1.6) of the system of optimality conditions differ only in their signs. The question arises of whether this situation is really extraordinary. Suppose that a triple of functions (u , x, p ) is a solution to the system (1.1), (1.3), (1.4). Then the following relations hold:

− x& = −u, t ∈ (0,1); (− х)(0) = 0, − p& = − х, t ∈ (0,1); (− p)(1) = 0. Finally, equality (1.4) yields

⎧ 1, ( − p )(t ) < 0, (−u )(t ) = ⎨ ⎩ −1, ( − p )(t ) > 0. Thus, the triple ( −u , − x, − p ) is also a solution to the system.

30

Conclusion 1.2. If a triple of functions (u , x, p ) is a solution to the system (1.1), (1.3), (1.4), then the triple ( −u , − x, − p ) is also a solution to the system. So it is a property of the system of optimality conditions to have a pair of solutions that differ only in their sign. At the same time, the optimality conditions follow directly from the original formulation of the optimal control problem. Consider an arbitrary admissible control и corresponding to a solution x of problem (1.1). Then the control -u belongs to the set of admissible controls (since the minimum and maximum values of the control differ only in their signs). As noted before, the function -x is a solution to problem (1.1) corresponding to the control -u. As a result, we obtain

I (u ) =

1

∫ (u 2

1

0

2

+x

2

1

) dt = 2 ∫ ⎡⎣( − u ) 1

2

+ (− x) 2 ⎤⎦ dt = I ( −u ).

0

Thus, the functional I assumes the same values for the controls и and -u. Conclusion 1.3. If a control и provides the maximum of the functional I on U, then the control -u also maximizes this functional on U. Thus, the formulation of the optimal control problem already implies the existence of a pair of solutions. Remark 1.3. We shall often use this kind of technique in the situations where solutions are not unique (in particular, see Examples 7 and 8). We shall seek a transformation that takes one solution of the problem into another. Then, having found a solution of the problem, we shall use the transformation to find a new solution. Remark 1.4. The previous example has all the properties pointed out in Conclusions 1.2 and 1.3, but its system of optimality conditions and optimal control problem (minimization of the functional I on U) have a unique solution. The reason is that the corresponding functions u, x, and p are identically equal to zero in that example. Therefore, changing the sign does not lead to a new solution. In the last example, the solution is necessarily nontrivial (since the trivial control provides the minimum of the functional); therefore, by changing the sign we obtain a new solution.

In the last example, the optimal control problem turned out to have more than one solution. In the next section we shall find out the difference between Problem 1 and the previous problem that lead to this situation.

31

1.4. UNIQUENESS OF THE OPTIMAL CONTROL It should not be surprising that some extremum problems have unique solution and some don't. For example, the trivial problem of minimizing the simplest quadratic function

f = f ( x) = x 2 on the interval [–1,1] has a unique solution, while the problem of maximizing it has two solutions (see Figure 5). f

f

x -1

0 f(0) = min f([-1,1])

1

x -1

0 f(1) = f(-1) = max f([-1,1])

1

Figure 5. The parabola has one minimum and two maxima in [-1,1]

The question is: What is the difference between the functions f ( x ) = x

2

2

and g ( x ) = − x (maximizing f is equivalent to minimizing g) that causes one of them to have one minimum on [–1,1] and the other to have two? Evidently, the property of convexity is the key. The function f = f ( x ) is said to be convex on the segment [a, b] if the following inequality holds:

f (α x + (1 − α ) y ) ≤ α f ( x) + (1 − α ) f ( y ) ∀x, y ∈ [ a, b], α ∈ [0,1]. If this inequality turns into equality only if x = y , and α = 0, α = 1, then f

is called strictly convex. Geometrically, the convexity of f means that the segment of the curve f = f ( x ) that connects the points

( x, f ( x ) )

and

( y, f ( y ) ) lies not higher (or, in the case of strict convexity, lower) than the segment of the straight line connecting these points (see Figure 6).

32

f(x)

f(x)

f(x)

f(y)

f(y)

f(y)

x

y

x

y

x

y

Figure 6. Convexity of functions: a) strictly convex function; b) nonconvex function; c) convex, but not strictly convex function

A simple example shows that strict convexity is required for the existence of a unique minimum (see Figure 7).

f ( x1 ) = f ( x2 ) = min f f min f x1 x2 Figure 7. A minimum may be not unique if the function is not strictly convex

This definition of convexity can be naturally generalized to the case of functionals on a vector space (for example, the Euclidean space). Let a functional I be defined on a subset U of a vector space. The set U is called convex if for any two points x, y ∈ U , the set of points z = α x + (1 − α ) y , α ∈ [0,1] (the segment of the straight line connecting x and y) belongs to U (see Figure 8).

33

a

b

Figure 8. Convexity of sets: a) convex set; b) nonconvex set

A functional I defined on a convex set U is called convex if the following inequality holds:

I (α u + (1 − α )v ) ≤ α I (u ) + (1 − α ) I (v) ∀x, y ∈ [u , v], α ∈ [0,1]. The definition of strict convexity for functionals is the same as for functions and requires that the inequality be strict for u ≠ v and α ≠ 0, α ≠ 1. We shall now formulate a simple uniqueness theorem for minima of a functional. Theorem 2. A strictly convex functional defined on a convex set can have at most one point of minimum. Proof. Suppose that a strictly convex functional I on a convex set U has two different points of minimum, x and y. Then the element α x + (1 − α ) y belongs to U for any α ∈ (0,1) .

Since the functional is strictly convex, we have

I [α x − (1 − α ) y ] < α I ( x) + (1 − α ) I ( y ) = = α min I (U ) + (1 − α ) min I (U ) = min I (U ). The value of the functional at the chosen element of U is less than its minimum on U. The assumption that there are two points of minimum lead to a contradiction. □ Remark 1.5. Certainly, a nonconvex functional may have a unique minimum too (see Figure 9). Theorem 2 guarantees uniqueness only in the case of strictly convex functionals. Remark 1.6. Theorem 2 only implies that if a solution of the problem of minimizing a strictly convex functional exists, and then it is the only one. The theorem does not concern the problem of existence of solutions.

34

Figure 9. A nonconvex function may have a unique minimum

1.5. UNIQUENESS OF AN OPTIMAL CONTROL IN A SPECIFIC EXAMPLE

We now show that Theorem 2 can be used to establish the uniqueness of optimal control in the problem of minimizing the functional I defined in the example below; however, this theorem is not applicable in the case of maximizing I. First of all, note that Theorem 2 has to be adapted for optimization problems. The reason is that the dependence of the optimality criterion on the control is not only described explicitly, but is also expressed through the system state related to the control by problem (1.1). This circumstance, however, is not an insuperable obstacle. Let u1 and u2 be two different admissible controls, and let x1 and x2 be the corresponding solutions of system (1.1). Then the following conditions hold:

x&i = ui , t ∈ (0,1), хi (0) = 0; i = 1, 2. Multiplying the first pair of these equalities (with i = 1) by α ∈ (0,1) and the second pair by (1–α) and summing the results, we have

α x&1 + (1 − α ) x&2 = α u1 + (1 − α )u2 , t ∈ (0,1); α х1 (0) + (1 − α ) х2 (0) = 0. At the same time, denoting by x3 the solution of problem (1.1) corresponding to the control u3 = α u1 + (1 − α )u2 , we obtain the conditions

35

x&3 = u3 , t ∈ (0,1), х3 (0) = 0. Consequently,

x3 = α x1 + (1 − α ) x2 ,

(1.7)

We now estimate the quantity

I ( u3 ) =

1

1

∫ ⎡⎣( u ) 2

2

3

+ ( x3 ) 2 ⎤⎦ dt.

0

Since the quadratic function is strictly convex for t ∈ (0,1) , we have

[u (t )] = [α u [ x (t )] = [ xu

1

+ (1 − α )u2 ] < α [u1 (t ) ] + (1 − α ) [u2 (t ) ] ,

1

+ (1 − α ) x2 ] < α [ x1 (t ) ] + (1 − α ) [ x2 (t ) ] .

2

3

2

3

2

2

2

2

2

2

Integrating these expressions, we obtain the inequality

I [α u1 + (1 − α )u 2 ] =

1

1

∫ ⎡⎣(u ) 2 3

2

+ ( x3 ) 2 ⎤⎦ dt
ξ .

u (t ) = ⎨

(1.8)

Evidently, the corresponding solution of problem (1.1) for t < ξ is x(t) = t. In particular, x (ξ ) = ξ . Then, for t > ξ we have t

x(t ) = x(ξ ) − ∫ dτ = 2ξ − t. ξ

Thus, the state of the system for the control (1.8) is given by

⎧ t, t < ξ , ⎩ 2ξ − t , t > ξ .

х (t ) = ⎨

Since the value of the adjoint system state at the final instance is known, we first determine the solution of (1.3) for t > ξ :

37

1

р (t ) = − ∫ (2ξ − τ ) dτ = 2ξ (t − 1) + t

1− t2 2

⎡1 + t ⎤ − 2ξ ⎥ . ⎣ 2 ⎦

= (1 − t ) ⎢

We shall seek a function p that changes its sign at t = ξ , i.e., vanishes at ξ. Thus, we have the condition

p (ξ ) = (1 − ξ )(1 − 3ξ ) = 0. This quadratic equation for ξ that has two solutions, ξ = 1 and ξ = 1 / 3. The first solution is trivial since the function p is known to vanish at the final instance of time. It is the point ξ = 1 / 3 which is of particular interest, since it belongs to the time interval in question. So there exits a unique point ξ = 1 / 3 where the solution of problem (1.3) may vanish. Considering the equation on the interval (t,1/3) for arbitrary t ∈ (0,1 / 3) , we have

( 3)

р (t ) = р 1

1/ 3

∫ τ dτ =

−

t 2 − 1/ 9 2

t

.

Thus, the solution of the adjoint system is given by the formula

⎧ −(1/ 3 − t )(t + 1/ 3)/2, t < 1/ 3,

р (t ) = ⎨

⎩ (1 − t )(t + 1/ 3)/2,

t > 1/ 3.

The function p is obviously negative for t < 1/ 3 and positive for t > 1/ 3 . Then, by (1.4), the control is written as

⎧ 1, t < 1/ 3,

u (t ) = ⎨

⎩ −1, t > 1/ 3,

(1.9)

which coincides with formula (1.8) for ξ = 1/ 3 . From the above analysis, we conclude that the system (1.1), (1.3), (1.4) has one more solution such that the control is discontinuous, the system state is not differentiable, and the solution of the adjoint system change its sign at the point ξ = 1/ 3 . The existence of the third solution of the optimality conditions which differs dramatically from the other two seems to be a bad sign. Moreover, as we know, functions that differ from solutions of

38

(1.1), (1.3), (1.4) only in their signs are also solutions of this system (which means that the fourth solution exists). Conclusion 1.4. The system (1.1), (1.3), (1.4) has at least four solutions.

We know that the values of the functional I for the first two controls (identically equal to 1 and -1) coincide. It is also clear that / assumes the same values on the third and the fourth controls. However, it is not obvious whether the values of the optimality criterion for the first and third controls coincide U0 – the set of optimal controls U 1 – the set of solutions of the necessary optimality condition U1

U0

U0 U1 U0 = U1

necessary optimality condition

sufficient optimality condition

necessary and sufficient optimality condition

Figure 10. Relation between the set of optimal controls and the set of solutions of the optimality condition

We now determine the value of the functional at the control given by (1.9): 1 1 1/ 3 ⎞ 1 ⎛ 1/ 3 ⎞ 1⎛ 1⎛ 1 ⎞ 14 I = ⎜ ∫ u 2 dt + ∫ х 2 dt ⎟ = ⎜1 + ∫ t 2 dt + ∫ (2 / 3 − t ) 2 dt ⎟ = ⎜1 + ⎟ = . 2⎝0 0 0 0 ⎠ 2⎝ ⎠ 2 ⎝ 27 ⎠ 27 This value is different from the one found before.

Conclusion 1.5. Different solutions of the system (1.1), (1.3), (1.4) provide different values of the functional.

If the values of the optimality criterion do not coincide for different controls, this means that some of the obtained solutions are worse than others because they do not provide the maximum of the functional being maximized. Conclusion 1.6. Not every solution of the optimality conditions is optimal, i.e., the optimality conditions are not sufficient.

39

Thus, solving the system of optimality conditions may yield nonoptimal controls. It is interesting to find out why not every solution of the maximum principle is the optimal control and why the optimality conditions appear to be sufficient in the previous example. 1.7. SUFFICIENCY OF THE OPTIMALITY CONDITIONS

The procedure used to derive the optimality conditions in the form of the maximum principle was as follows. If a certain admissible control is optimal, then the corresponding increment of the functional must be nonnegative. Then we transformed the formulas for the increment of the functional to obtain the maximum condition. Thus, the optimality of the control implied that the maximum principle was satisfied, which means that the optimality conditions are necessary (Figure 10). The sufficiency of the optimality condition means that every solution of the condition is an optimal control (Figure 10). If it doesn't hold, then the arguments used to prove that the optimality of a control implies the maximum principle being satisfied are irreversible. The question is: At which stage of our arguments did they become irreversible? We now return to the procedure of deriving the maximum principle. The assumption that a control и is optimal means that

ΔI = I (v, y ) − I (u , x ) ≥ 0 ∀v ∈ U ,

(1.10)

where x and у are the functions of the system state on the controls и and v, respectively. Condition (1.10) is equivalent to

ΔL = L(v, y , λ ) − L(u , x, λ ) ≥ 0 ∀v ∈ U , ∀λ ,

(1.11)

where L is the Lagrange functional. After a series of transformations, inequality (1.11) was reduced to the form T

T

0

0

− ∫ Δ u Hdt − ∫ ( H x + p& )Δxdt +

(1.12)

+[hx + p (T ) + η ]Δx(T ) + η ≥ 0, ∀v ∈ U , ∀p. With p chosen to be a solution of the adjoint system, inequality (1.12) became as follows: T

− ∫ Δ u Hdt + η ≥ 0, ∀v ∈ U . 0

40

(1.13)

Up to this point, all the arguments were reversible. Indeed, if a function и satisfies inequality (1.13), then any increment of the Lagrange functional at this control is nonnegative for the chosen p. However, by definition, the Lagrange functional coincides with the optimality criterion for any p (including the solution of the adjoint system). This implies (1.10) and therefore the optimality of the control u. Our further arguments were as follows. Since the remainder term η is the second order term in the control increment, whereas the integral in the left-hand side of (1.13) is the first order term, it follows that for all controls v sufficiently close to u, the sign of their sum will be defined by the sign of the integral. Hence T

∫ {H [v(t ), x(t ), p(t )] − H [u(t ), x(t ), p(t )]}dt ≤ 0.

(1.14)

0

For a control v, we took the spiky variation

⎧u (t ), t ∉ (τ − ε , τ + ε ), ⎩ w(t ), t ∈ (τ − ε , τ + ε ),

v (t ) = ⎨

where w ∈ U , τ ∈ (0,1) , and ε is a sufficiently small positive parameter. Dividing the left-hand side of inequality (1.14) by 2ε > 0 and passing to the limit as ε → 0 , we obtain the inequality

H [ w(τ ), x (τ ), p (τ )] − H [u (τ ), x (τ ), p (τ )] ≤ 0, which becomes the maximum condition

H [u (τ ), x(τ ), p (τ )] = max[ w, x (τ ), p (τ )], τ ∈ (0, T ), w∈U

(1.15)

since the point τ and the value w are arbitrary. Assume that the control и satisfies the maximum condition (1.15) (and, consequently, the preceding inequality). Integrating the inequality, we establish that (1.14) holds as well (up to the designation of the arbitrary control). Now, if we could obtain inequality (1.13), then (1.10) would follow, which would mean that the control is optimal. Thus, the only "suspicious" stage of our arguments in deriving the maximum principle is the passage from (1.13) to condition (1.14). Conclusion 1.7. The optimality conditions arc not sufficient because relations (1.13) and (1.14) are not equivalent.

41

Note that the optimality conditions were sufficient in the preceding example, which means that (1.13) and (1.14) were equivalent in that case. In this connection, it would be interesting to establish conditions that guarantee the sufficiency of the maximum principle. If a control и provides a minimum of the functional I, then it satisfies (1.10), which can be written in the form. T

− ∫ Δ u Hdt ≥ 0, ∀v ∈ U . 0

Evidently, if we add a nonnegative quantity to the left-hand side of this inequality, then its sign will not change. Assume now that the remainder term η is nonnegative. Then, adding η to the integral in the left-hand side of the foregoing inequality, we obtain (1.13), which implies the optimality of u, as we already know. Thus, we have established that the solution of the maximum principle is optimal and, consequently, the optimality conditions arc sufficient for the previous example. Theorem 3. If the remainder term in the formula for the increment of the functional to be minimized is nonnegative, then the maximum principle is the sufficient optimality condition. Remark 1.8. Obviously, for the problem of maximizing the functional, the sufficiency of the optimality condition is guaranteed by the condition η ≤ 0. Remark 1.9. The obtained results do not imply that the maximum principle is not a sufficient optimality condition when η is negative. We can only assert that the above procedure of proving sufficiency does not work for negative values of η. We emphasize that the sufficiency of the maximum principle, according to Theorem 3, is due to the remainder being of fixed sign rather than being small. The optimality of any solution of the maximum principle can certainly be guaranteed if η is sufficiently small, regardless of its sign.

1.8. SUFFICIENCY OF THE OPTIMALITY CONDITIONS IN A SPECIFIC EXAMPLE

We now try to establish the sufficiency of the maximum principle for the considered example following Theorem 3. For this purpose, we write the remainder term in the explicit form. As is known, in the general case, it is defined by the formula

42

T

η = − ∫ (η1 + η 2 ) dt. 0

Here η1 represents the second order terms in the expansion of H (u , x + Δx, p ) , and η 2 = [ H x (v, x, p ) − H x (u , x, p )]Δx. In our example, we have η 2 = 0 since the derivative Hx is independent of the control. Thus, the only remaining quantity is

η1 = H (v, x + Δx, p ) − H (v, x, p ) − H x (v, x, p ) Δx. The following formula holds:

H = рu – (u2 + x2)/2. Consequently,

η1 = [ −( x + Δx) 2 + x + 2 xΔx] / 2 = x 2 / 2, which implies

η=

1

T

∫ Δх dt. 2 2

0

By Theorem 3, the maximum principle for the example in question guarantees the optimality of the control. Conclusion 1.8. The sufficiency of the optimality conditions in the form of the maximum principle in Problem 0 follows from the condition that the remainder term is nonnegative.

It remains to find out why analogous results cannot be obtained for Problem 1. This problem deals with the maximum of the functional I, which means that all signs in the above inequalities must be reversed. In particular, the optimality condition is equivalent to the inequality T

− ∫ Δ u Hdt + η ≤ 0 ∀v ∈ U , 0

instead of (0.10). Furthermore, the solution of the optimality condition (the problem of minimizing the function H) will satisfy the inequality

43

T

− ∫ Δ u Hdt ≤ 0 ∀v ∈ U , 0

instead of (0.11), where H (and, consequently, η) is denned the same way as before, since the examples differ only in their extremum types. The solution of the optimality condition for Problem 1 must satisfy the foregoing inequality. However, if we add a nonnegative η to the integral in the left-hand side, then, in contrast to the previous case, the sign of the inequality may change. Then the previous condition, which is equivalent to the optimality of the control, may fail. Hence, the optimality condition for Problem 1 is not sufficient. Conclusion 1.9. The fact that the optimality condition in the form of the maximum principle in Problem 1 is not sufficient is due to the sign of the remainder term in the formula of functional increment.

We still have to find out what properties of the formulation of the original problem caused the optimality conditions to be sufficient in one case and not sufficient in the other. For this purpose, we should understand what condition guarantees that η is nonnegative. First of all, for the equality to hold, it is sufficient that the functions f and g that appear in the state equation and the functional being minimized can be written in the form

f(u,x) = f1(u) + f2(x), g(u,x) = g1(u) + g2(x). Under these conditions, we have

η1 = H (v, x + Δx, p ) − H (v, x, p ) − H x (v, x, p ) Δx = = p[ f 2 ( x + x) − f 2 ( x) − f 2 x ( x) Δx] − [ g 2 ( x + Δx) − g 2 ( x) − g 2 x ( x) Δx]. Since the solution p of the adjoint system is not known yet (it will be determined when solving the optimality conditions), it will be possible to determine the sign of η only if the coefficient of p vanishes. This certainly happens if f 2 is linear with respect to x. Under these assumptions, we have T

η = ∫ [ g 2 (x + Δx ) − g 2 (x) − g 2x (x)Δx ] dt. 0

It is easy to see that η is nonnegative if g 2 is convex.

44

Conclusion 1.10. The maximum principle is sufficient if the right-hand side of the state equation and the integrand in the expression for the functional to be minimized can be represented as the sum of a function depending on the control and a function depending on the system state, the equation being linear and the functional being convex with respect to the state. Remark 1.10. This conclusion still holds if the functions of the general form that appear in the problem formulation explicitly depend on the parameter t.

It is evident that Example 0 satisfies the above requirements, whereas Example 1 does not, which explains the results obtained above. Remark 1.11. The maximum principle may still be a sufficient optimality condition if the conditions of the above conclusion do not hold (see Remark 1.9). Remark 1.12. In more complex optimal control problems, it is usually impossible to determine the sign of the remainder term in the formula for the functional increment. The problem of establishing the sufficiency of the maximum principle for nonlinear systems is extremely complicated.

The question arises of whether the sufficiency of the maximum principle is a common or rare case. Since it is guaranteed only when the remainder term is of fixed sign or when it is small, we conclude that it holds, only in exceptional cases. Conclusion 1.11. The optimality conditions in the form of the maximum principle are sufficient only in exceptional cases. Remark 1.13. It should be noted that almost every positive property in mathematics usually appears to be rare.

1.9. CONCLUSION OF THE ANALYSIS OF THE OPTIMALITY CONDITIONS

The analysis of the optimality conditions in the form of the maximum principle is not finished at this point. We already know that the system (1.1), (1.3), (1.4) has at least four solutions, two of them being nonoptimal controls. However, it is still not clear whether we have found all solutions of the system. By (1.4), the control must be piecewise-constant. The first two solutions were obtained under the assumption that the control is constant or, equivalently, that the function p is of constant sign. The assumption that yielded the next two solutions was that the control has one point of discontinuity and p changes sign at one point. However, the control could be 45

discontinuous at two points of the interval (0.1). For example, it may be as follows:

⎧ 1, 0 < t < τ 1 , ⎪ u (t ) = ⎨−1, τ 1 < t < τ 2 , ⎪ 1, τ < t < 1. 2 ⎩ It is possible only when p is positive in the intervals (0, τ 1 ) and (τ 1 ,1) and negative for t ∈ (τ 1 ,τ 2 ) . The corresponding solution of problem (1.1) is given by the formula

t, 0 < t < τ1 , ⎧ ⎪ x(t ) = ⎨ 2τ 1 − t , τ 1 < t < τ 2 , ⎪2τ − 2τ + t , τ < t < 1. 2 2 ⎩ 1 Integrating the adjoint equation over the intervals (τ 1 , τ 2 ) and (τ 2 ,1) and taking into account that p vanishes at their boundaries, we obtain two equalities τ2

∫ ( 2τ

1

− t )dt = 0,

τ1

1

∫ ( 2τ

1

− 2τ 2 + t )dt = 0.

τ2

Consequently,

2τ 1 (τ 2 − τ 1 ) −

τ 22 − τ 12 2

= 0, (2τ 1 − 2τ 2 )(1 − τ 2 ) +

1 − τ 22 2

= 0.

Dismissing the trivial solutions τ 1 = τ 2 and τ 2 = 1 (because they don't provide two points of discontinuity of the control), we obtain

2τ 1 −

τ 2 + τ1 2

= 0,

( 2τ 1 − 2τ 2 ) +

1 + τ 22 2

= 0.

Hence, there exists a unique pair of points τ 1 = 1 / 5 and τ 2 = 3 / 5 with the desired properties. We now determine the sign of p in each of the intervals. For t ∈ (0,1 / 5) , we have 46

1/ 5

p (t ) = − ∫ tdt = t

t 2 − 1/ 25 2

< 0.

For t ∈ (1 / 5,3 / 5) , 3/5

(3 / 5 − t )(t − 1/ 5) ⎛2 ⎞ − t ⎟ dt = > 0. 2 ⎝5 ⎠

p (t ) = − ∫ ⎜ t

Finally, for t ∈ (3 / 5,1) , we have

(1 − t )(t − 3 / 5) ⎛ 4⎞ p (t ) = − ∫ ⎜ t − ⎟ dt = − < 0. 5⎠ 2 t ⎝ 1

Thus, the sign of p is coordinated with the behavior of the control, i.e., equality (1.4) holds. This means that we have obtained the fifth solution of the system of optimality conditions. The sixth solution can be obtained from the fifth by changing its sign. The above arguments can be generalized. For an arbitrary k = 1, 2,..., we divide the segment [0,1] into 2k+1 equal segments. Then we set the control to be equal to 1 on the first segment, –1 on the next two segments, +1 in the next two, and so on. The resulting function is denoted by u k+ . The corresponding solution xk+ of problem (1.1) is piecewise-linear, and the solution pk+ of problem (1.3) is a differentiable function which changes sign at the points of discontinuity of the control (see Figure 11).

47

u1+

u0+

u3+

u2+

t

t

1

1

t

x3+

t

1

p1+

p2+

t

p3+

t

t

t

1

t

t

1

1

p0+

t

x2+

x1+

x0+

1

1

t

1

1

1

Figure 11. Solutions of the system of optimally conditions in Problem 1

The function u k− = uk+ is also a solution. Thus, for any k (the number of points of discontinuity), there are exactly two controls satisfying the optimality conditions. Conclusion 1.12. System (1.1), (1.3), (1.4) has infinitely many solutions.

We now have to find out which of the solutions of the optimality conditions is optimal. To answer this question, we write the values of the functional

I (uk+ ) = I (uk− ) = =

1 2

+

1

1 ⎡ + uk 2 ∫0 ⎣

( ) ( ) 2

+ xk+

2k + 1

1

2

3 ( 2k +1)

3

=

2

1 2

⎤ dt = 1 + 2k + 1 ⎦ 2 2

+

1 6 ( 2k +1)

2

1/( 2 k +1)

∫ (x ) + k

2

dt =

0

, k = 0,1,... .

This formula shows that the value of the functional decreases as k increases. Conclusion 1.13. Problem 1 has only two solutions: the controls identically equal to 1 and -1.

The solution of the problem in question is now complete. Remark 1.14. Note that we have solved the problem completely despite the fact that the maximum principle is not a sufficient optimality condition. In this case, first of all, the maximum principle is used to select the set of all solutions of the

48

optimality conditions. Then the best controls (in the sense of the value of the functional) are selected from the set of solutions of the maximum principle. These controls are optimal.

1.9. FINAL REMARKS

Our analysis of Problem 1 could be considered finished at this point, for its solution has been found. However, we would like to make some important and interesting remarks. First, it is interesting to learn more about the properties of nonoptimal solutions of the maximum principle. The simplest example of a necessary but not sufficient condition for a function to have an extremum is the vanishing of its derivative at a given point (Fermat's theorem on stationary points). All the solutions of the relation in question are either points of local extremum or points of inflection. One would expect that nonoptimal solutions of the maximum principle also have specific properties related to the maximized functional I. In particular, we can verify that the functions u1+ and u1− provide the maximum of I on the set of all functions discontinuous at t = 1/3. Moreover, they provide the extremum of I on the class of all functions with the absolute value of 1 that have a unique point of discontinuity. Indeed, let и=1 for t < ξ and и =-1 for t > ξ . Then the following function is a solution of problem (1.1):

⎧ t, t < ξ , ⎩ 2ξ − t , t > ξ .

х (t ) = ⎨

We now determine the corresponding value of the functional:

I (ξ ) =

1

∫( 2

1

0

)

ξ

1

⎤

2⎣

0

ξ

⎦

⎤ 2 1⎡ 2 2 ⎢1 + ∫ t dt + 4ξ ∫ (ξ − t )dt ⎥ = − 2ξ (ξ − 1) . 2⎣ 0 ξ ⎦ 3 1

=

1⎡

u 2 + x 2 dt = ⎢1 + ∫ t 2 dt + ∫ (2ξ − t ) 2 dt ⎥ = 1

The function I = I(ξ) has two points of extremum ξ = 1 and ξ = 1 / 3 in [0.1]. The first one is associated with the continuous control and corresponds to the solution of Problem 1. The point ξ = 1 / 3 is a point of minimum of the function I =I(ξ) on [0,1] and, consequently, it provides the 49

minimum of the functional on the class of functions identically equal to 1 that have a unique point of discontinuity (see Figure 12). I

ξ 1

1/3

Figure 12. Dependence of the functional on the point of discontinuity of the control

Other nonoptimal solutions of the maximum principle have similar properties. Conclusion 1.14. Nonoptimal solutions of the optimality conditions possess specific properties related to the optimality criterion.

Another interesting conclusion can be made if we rewrite the system (1.1), (1.3), (1.4) in a different form. Excluding и and x from the system, we obtain the boundary value problem

&& p = F ( p ), t ∈ (0,1); p (1) = 0, p& (0) = 0,

(1.16)

where F(p) stands for the right-hand side of (1.4). Since problem (1.4) is equivalent to the original system of optimality conditions, we arrive at the following conclusion on possible properties of boundary value problems for nonlinear differential equations of the second order. Conclusion 1.15. Boundary value problem (1.4) has infinitely many solutions. Remark 1.15. In further examples, we shall consider boundary value problems for nonlinear differential equations of the second order with even more interesting properties (see Examples 3, 7, and 8).

We now consider the nonlinear heat conduction equation

50

∂v ∂t

=

∂ 2v ∂ξ 2

− F (v ), t > 0, 0 < ξ < 1,

(1.17)

with the boundary conditions

∂v ∂ξ

= 0, v ξ =1 = 0, t > 0,

(1.18)

ξ =0

and with certain initial conditions, where the function F has the same form as in (1.16). Evidently, the the equilibrium state for the system (1.17), (1.18) is a solution of the boundary value problem

d 2v dξ

2

= F (v), 0 < ξ < 1;

dv(0) dξ

= 0, v(ξ ) = 0,

which coincides with problem (1.16) up to the notation. Then, based on the results obtained before, we arrive at the following conclusion specifying characteristic features of boundary value problems for nonlinear parabolic equations. Conclusion 1.16. The system (1.17), (1.18) has infinitely many equilibrium states.

Arriving at a specific equilibrium state obviously depends on the initial state of the system. Remark 1.16. The foregoing assertion can be used in an algorithm of finding various solutions of the boundary value problem (1.16) and, consequently, the whole system of optimality conditions. For the given system, this corresponds to the stabilization method.

The analysis of Problem 1 is now complete. Examples 2, 7, and 8 contain the description of other cases where optimality conditions are not sufficient. SUMMARY

The following conclusions can be made based on the analysis of Problem 1. 1.

Optimal controls obtained in the process of solving optimization problems may be nonunique. 51

2.

An optimal control can be unique only in exceptional cases, for example, when the functional to be minimized is convex.

3.

Not every solution of the maximum principle is an optimal control. This means that the optimality condition is not sufficient.

4.

Optimality conditions in the form of the maximum principle are sufficient only in exceptional cases, for example, if the remainder term in the formula for the functional increment is nonnegative.

5.

The remainder term is necessarily nonnegative for linear systems such that the functional to be minimized has an integral form, the integrand being the sum of a function depending only on the control and a convex function depending only on the state.

6.

If the optimality condition .is not sufficient, complete solution of the problem on the basis of this condition still might be possible.

7.

It is possible to find out that a solution of the optimality conditions is nonunique by specifying different initial approximations and observing the convergence of the iterative process to different limits.

8.

Optimality conditions prove to be not sufficient if the value of the functional to be minimized turns out to be smaller than the limit value at some intermediate iteration step.

Example 2. The singular control When solving optimal control problems in the previous chapters, we used the maximum principle as one of the most effective optimization methods. At the same time, we established that not every control satisfying the maximum principle is optimal, which is a consequence of the insufficiency of the optimality condition. Moreover, application of the maximum principle may involve other major difficulties. In both examples considered above (regardless of whether the optimality conditions were sufficient or not), we started the analysis of the maximum condition with expressing the control in terms of the other unknown quantities. However, this method sometimes turned out to be ineffective because the maximum principle appeared to be degenerate, which brings up the notion of a singular control. We shall show that a singular control may be optimal or nonoptimal. There may exist more than one singular control, or even infinitely many singular controls. One of the necessary conditions for a singular control to be optimal is the Kelly condition. 2.1. PROBLEM FORMULATION

Consider a system described by the Cauchy problem

x& = u, t ∈ (0,1); х(0) = 0. The control u = u (t ) belongs to the set

U = {u u (t ) ≤ 1, t ∈ (0,1)} . The optimality criterion is given by the formula

53

(2.1)

I=

1

1

2 ∫0

uxdt.

Problem 2. Find a control u ∈ U that provides the minimum of the functional I on the set U.

Note that both the state equation and the set of admissible controls are very simple and are defined the same ways as in the previous examples. The functional to be minimized is even simpler than before. It is obviously bilinear, i.e., in addition to being linear with respect to the control and the state function, it also includes their product. This problem can hardly be called very complex and therefore one would not expect any surprises. To solve the problem, we shall apply the method based on the maximum principle that was used above. 2.2. THE MAXIMUM PRINCIPLE

To reduce the problem to the standard form, we introduce the following notation:

f (t , u , x) = u , x0 = 0, T = 1, g (t , u , x) = ux, ϕ = 0, u1 = −1, u2 = 1. Following the technique described in the previous chapter, we define the function

H = H (u ) = pu − ux. Then the adjoint system is written as follows:

p& = u, t ∈ (0,1); p(1) = 0.

(2.2)

According to the maximum principle, the optimal control must provide the maximum of H on U, i.e., it must satisfy the equality

H (u ) = max H ( w) | w|≤1

(2.3)

Hence, we have the problem (2.1)-(2.3) for the optimal control, which is similar to the relevant systems considered above. We shall use the previously developed technique to analyze these relations.

54

2.3. ANALYSIS OF THE OPTIMALITY CONDITIONS

First of all, we need to express the control in terms of the other unknown functions from (2.3). Using standard methods, we find the derivative

∂H ∂u

= p − x.

Since this expression does not depend on the control explicitly, we conclude that H (which is a linear function) does not have local extrema. With no stationary points, H may have relative extremum only on the boundary of the set of admissible controls. The boundary values are

H (1) = p − x, H ( −1) = x − p. Then the solution of the maximum principle is given by the formula 1,

⎧ 1, р (t ) − x(t ) > 0,

u (t ) = ⎨

⎩ −1, р (t ) − x(t ) < 0,

(2.4)

i.e., in general, the optimal control must be piecewise-constant. Consider the system (2.1), (2.2), (2.4). The solution of problem (2.1) obviously has the form t

x(t ) = ∫ u (τ )dτ . 0

The solution of problem (2.2) is obtained similarly: 1

р (t ) = − ∫ u (τ ) dτ . t

We now find the difference t 1 ⎡1 ⎤ р (t ) − x(t ) = − ⎢ ∫ u (τ )dτ + ∫ u (τ )dτ ⎥ = − ∫ u (τ ) dτ . ⎣t ⎦ 0 0

Substituting this value into (2.4), we have

55

⎧ ⎪1, ⎪ u (t ) = ⎨ ⎪−1, ⎪⎩

1

∫ u (τ )dτ < 0, 0

1

∫ u (τ )dτ > 0.

(2.5)

0

We have obtained a special equation for the control, which can be called integral in a certain sense. Although this relation is by no means a classical integral equation, we may readily apply the method of successive approximations in order to solve it. According to this method, each new approximation of the control is determined from the right-hand side of equality (2.5), in which the control is taken from the preceding iteration. However, this equation has a specific feature that allows us to analyze this problem directly. The right-hand side of (2.5) obviously does not depend on time. Therefore, the control is constant and, by formula (2.5), it is equal to either 1 or 1. However, if u (t ) = 1 , then the integral of the control is positive, and if u (t ) = −1 , then the integral is negative. Hence, equation (2.5) has no solutions. Conclusion 2.1. The problem (2.1), (2.2), (2.4) has no solutions.

These results may lead to the conclusion that the optimality conditions have no solutions at all. The unsolvability of the necessary optimality conditions has important consequences because the set of solutions of the maximum principle is, in general, larger than the set of optimal controls (as mentioned in the previous example). In what follows, we shall show that Problem 2 is still solvable. It follows that the corresponding optimal control must satisfy the maximum principle (2.3). The question arises of how the unsolvability of the system (2.1), (2.2), (2.4) agrees with the fact that the maximum condition (2.3) must necessarily hold for the optimal control. If the maximum principle has no solutions, the consequences are dramatic, since we shall have to acknowledge that it is not a necessary optimality condition because it does not hold for the optimal control. The only way to overcome this obstacle is to admit that (2.4) and (2.3) are not equivalent. Conclusion 2.2. Relations (2.3) and (2.4) are not equivalent.

Indeed, the stationarity condition for the problem in question is reduced to the equality 56

p (t ) − x (t ) = 0, t ∈ (0,1). Taking into account the values of the functions x and p determined above, we write this equality in the form 1

∫ u(τ )dτ = 0.

(2.6)

0

In this case, the maximum principle (2.3) can be written as follows:

[ p(t ) − x(t )] u (t ) = max [ p (t ) − x(t )] w, t ∈ (0,1). | w| ≤1

Then the following condition holds: 1

1

u (t ) ∫ u (τ ) dτ = max w∫ u (τ ) dτ , t ∈ (0,1). | w| ≤1

0

0

Obviously, both sides of the foregoing equality vanish for any function u from the set of admissible controls that satisfies (2.6). In this case, this maximum principle is said to be degenerate. Every element of the set

⎧

U 0 = ⎨u ∈ U

⎩

1

⎫

∫ u (τ )dτ = 0⎭⎬ 0

is called a singular control and satisfies the maximum principle (2.3) in a specific way. Remark 2.1. The characteristic feature of a singular control is that it makes the function H independent of the control since the coefficient of the control becomes zero. Thus, the function H has the same value for all controls and the maximum condition holds in the trivial (degenerate) form: 0 = 0. Conclusion 2.3. While using the maximum principle, one should take to account that there may exist singular controls.

The question arises of how large is the class U 0 of singular controls. It obviously contains the function identically equal to zero and functions of the form

57

⎧ a, t ∈ (0, 0.5),

f =⎨

⎩− a, t ∈ (0.5,1)

for any a ∈ [−1,1] . Functions of the form a sin 2kπt for natural k also belong to U 0 , as well as many others (see Figure 13). u

u

u

t

t

1

1

t

1

Figure 13. Examples of singular controls in Problem 2 Remark 2.2. Every function such that the area under its curve on the given time interval is equal to zero is a solution of the integral equation (2.6), i.e., a singular control (see Figure 13).

Conclusion 2.4. For Problem 2, there exist infinitely many singular controls that are solutions of the maximum principle. (Moreover, the set of these controls is uncountably infinite.)

Thus, all solutions of the maximum principle for the problem under consideration are singular controls. We know that an optimal control must satisfy the maximum principle. It is required to find out which of the singular controls is optimal. We now try to find the value of the functional at an arbitrary singular control, i.e., at an admissible control satisfying (2.6). This task may seem unrealistic since we cannot even write an explicit formula for an arbitrary singular control. Indeed, the power of the set of singular controls is not less than that of the continuum. Nevertheless, this will not prevent us from solving the problem. Substituting the value of the control written in terms of the system state from equation (2.1) into the formula for the optimally criterion, we have

58

1

1

& I = ∫ uxdt = ∫ хxdt = 0

0

1 2

1

d

∫ dt ( )

x 2 dt =

0

x 2 (1) 2

.

Using the expression for the solution of problem (2.1), we obtain

I (u ) =

1 2

1

∫ udt =0

(2.7)

0

by condition (2.6). We conclude that the value of the functional to be minimized is nonnegative and it may vanish only at the controls that belong to U 0 , i.e., the singular controls. Conclusion 2.5. Every singular control in Problem 2 is optimal. Conclusion 2.6. The optimal control problem in question has infinitely тaпу solutions, the set of solutions being uncountable infinite.

It is interesting to find out why the optimal controls are nonunique. Problem 2 is obviously equivalent to the variational problem of minimizing lire functional defined by formula (2.7) on the set U. Note that this functional is convex (as a result of the linear operation of integration followed by the convex operation of raising to the second power), but it is not strictly convex. In particular, for the functions и and v satisfying the equalities

⎧ 1, t < 1/ 2,

u (t ) = −v (t ) = ⎨

⎩−1, t > 1/ 2,

with α = 1/2, we have

I [α u + (1 − α )v ] = α I (u ) + (1 − α ) I (v ) = 0. Thus, the assumptions of Theorem 2 on the uniqueness of solution of the optimal control problem do not hold. Conclusion 2.7. The convexity of the functional being minimized is not sufficient for the optimal control to be unique. Remark 2.3. A similar result for functions was obtained earlier (see Figure 7). In that case, the number of solutions of the extremum problem was also uncountably infinite.

We already know that every solution of the maximum principle in the present example is a singular control, and every singular control is optimal. 59

Conclusion 2.8. The maximum principle is a necessary and sufficient optimality condition for Problem 2.

The question arises of whether all singular controls are optimal. If this is true, solution of the optimization problem would be reduced to finding a singular control (if any). 2.4. NONOPTIMALITY OF SINGULAR CONTROLS

We now consider an optimal control problem very similar to the previous one. Assume that we have a system described by equation (2.1) and the set of admissible controls and the optimality criterion are of the same form as in the previous example. Problem 2'. Find a control u ∈ U that maximizes the functional I on U.

The maximum condition (2.3) becomes

H (u ) = min H ( w). | w| ≤1

(2.8)

Condition (2.8) obviously holds for all elements of U 0 , which means that we again have infinitely many singular controls. Remark 2.4. A control that is singular for the problem of minimizing the functional will necessarily be singular for the problem of maximizing the functional. It is obvious because in the case of singular controls the function H becomes independent of the control. Thus, the type of extremum is of little importance here.

We proved earlier that every singular control minimizes the given functional. Therefore, singular controls cannot be optimal for the problem under consideration. Conclusion 2.9. Not every singular control is optimal.

Since singular controls that satisfy (2.8) are not optimal for Problem 2', the maximum principle does not guarantee the optimality of controls. Conclusion 2.10. The maximum principle is not a sufficient optimality condition in Problem 2'.

It is required to find out why the optimality conditions are sufficient in Problem 2, but not sufficient in Problem 2'. When analyzing Example 1, we have established that the maximum principle is the sufficient optimality condition for the problem of minimizing the functional if the corresponding

60

remainder term η is nonnegative. For the problem of maximizing the functional, the same assertion holds with η ≤ 0 . As is known, the remainder term is denned by the formula T

η = − ∫ (η1 + η 2 ) dt.

(2.9)

0

where η1 depends on the second derivative of H with respect to the system state and η 2 includes the increment both with respect to the control and to the system state. Equation (2.1) and the optimality criterion are linear with respect to the system state. Hence η1 = 0 and the remainder term depends only on

η 2 = [ H x (t , v, x, p) − H x (t , u , x, p)]Δx Since H = pu - ux, it follows that η 2 = (u − v ) Δx . Then the formula for the remainder term is T

T

0

0

η = − ∫ η 2 dt = ∫ (u − v)Δxdt. Equation (2.1) implies that the increment of the control u − v is equal to the derivative of the state increment ∆x. Hence, the increment of the functional is 1

η = ∫ Δх& Δxdt = 0

1

d Δx 2 (1) 2 Δ x dt = . ( ) 2 ∫0 dt 2

1

Since the functional was being minimized in Problem 2, the nonnegative remainder term guaranteed the sufficiency of the maximum principle and, consequently, the optimality of all its solutions. In Problem 2', it is required to maximize the functional and therefore the sign of the remainder term is opposite to what is desired. It is now easy to see why the optimality condition is not sufficient, i.e., its solution is not optimal. We have established that none of the singular controls in Problem 2' is optimal. Now we have to find a solution to this problem. Note that only the degenerate case of (2.8) was considered. However, the optimality condition may have nonsingular solutions. In this case, instead of the optimality condition (2.5) for nonsingular solutions, we have 61

1 ⎧ − 1, if ⎪ ∫ u (τ )dτ < 0, ⎪ 0 u (t ) = ⎨ 1 ⎪ 1, if u (τ ) dτ > 0. ∫ ⎪⎩ 0

This equation has two solutions identically equal to 1 and -1, respectively. Conclusion 2.11. The maximum principle may have both singular and nonsingular solutions.

According to formula (2.8), every admissible control that maximizes the integral of the squared control is optimal. This assertion obviously holds for the functions identically equal to 1 or -1. Conclusion 2.12. Problem 2' has two solutions which are nonsingular solutions of the maximum principle.

Once again we have solved the problem despite that the optimally condition in the form of the maximum principle was essentially insufficient. Remark 2.5. Singular controls in Problem 2' minimize (rather than maximize) the functional. We proved again that nonoptimal solutions of the maximum principle may contain information on important properties of the optimality criterion.

In both cases considered above, there existed infinitely many singular controls. The question arises of whether it is always the case. 2.5. UNIQUENESS OF SINGULAR CONTROLS

Consider the following system:

x& = u, t ∈ (0,1); х(0) = 0. The control u = u (t ) again belongs to the set

U = {u u (t ) ≤ 1, t ∈ (0,1)} . The optimality criterion is defined by the formula

62

(2.10)

I=

1

1

∫ x dt. 2 2

0

Problem 2". Find a control u ∈ U which minimizes the functional I on the set U.

We put

H = H (u ) = pu − x 2 / 2. Then the adjoint system has the form

p& = х, t ∈ (0,1); p(1) = 0.

(2.11)

The corresponding maximum condition is written as follows:

рu = max ( рw). | w| ≤1

(2.12)

Since the function H is linear, it seems to achieve the maximum value only on the boundary of the set of admissible controls, which implies

⎧ 1, р (t ) > 0,

u (t ) = ⎨

⎩ −1, р (t ) < 0.

Thus, one would conclude that the absolute value of the optimal control is identically equal to unity. However, the problem in question is so simple that its solution may be found directly, without using the maximum principle. Indeed, by definition, the functional to be minimized is nonnegative. In vanishes only when the system state is identically equal to zero, which can happen only for the control и = 0. The latter is an admissible control; therefore, it is optimal. Conclusion 2.13. Problem 2" has a unique solution u = 0.

This result apparently contradicts (2.13). However, equality (2.13) represents only nonsingular solutions of the maximum principle (if they exist). The contradiction between (2.13) and the form of optimal control established above suggests that the maximum principle defines some singular controls. If we don't take into account the controls defined by formula (2.13), the maximum condition (2.12) holds only in the case where the coefficient of the control is equal to zero. As a result, we have p = 0. From problem (2.11) 63

it follows that the solution of the adjoint system vanishes only for x = 0. Substituting this value into (2.10) we find the control u = 0 , which is optimal, as we already know. Conclusion 2.14. Problem 2" has a unique singular control. Conclusion 2.15. The singular control in Problem 2" is optimal.

Thus, in contrast to the problems considered before, we have the case of a unique singular control which turned out to be optimal. It is interesting to find out whether the maximum principle is a sufficient optimality condition in this case. In formula (2.9) for the remainder term, only η1 is nonzero. This quantity includes the squared increment of the system state and is defined by the formula

η1 = H (t , v, x + Δx, p) − H (t , v, x, p) − H x (t , v, x, p)Δx = − x 2 / 2. Then the remainder term is T

1 η= Δx 2 dt. 2

∫ 0

Since the remainder term is nonnegative, the optimality conditions are necessary and sufficient. Conclusion 2.16. The maximum principle in Problem 2" is a necessary and sufficient optimality condition. Remark 2.6. We shall obtain the system (2.10), (2.11), (2.13) again in Example 3 and prove that it has no solutions. Remark 2.7. We shall return to Problem 2" in Example 5. We shall demonstrate one more serious issue in this problem in addition to its singular control.

In parallel with Problem 2", we consider the following optimization problem. Problem 2'". Find a control u ∈ U maximizing the functional I on U.

Condition (2.12) is now replaced by

pu = min ( pw). | w|≤1

64

(2.14)

Here we have the same unique singular control u = 0 , which is not optimal because it minimizes (rather than maximizes) the functional. Therefore, (2.14) must have solutions represented by nonsingular controls. Obviously, (2.14) implies

⎧ 1, p (t ) < 0,

u (t ) = ⎨

⎩−1, p (t ) > 0,

which differs from (2.13) only in its sign. It is interesting that the system of optimality conditions (2.10), (2.11), (2.15) already appeared in Example 1. As is known, there is a countable set of solutions to the problem in Example 1: for any number of discontinuity points, there are two solutions which differ only in their signs. Repeating the calculations of Example 1, we conclude that the optimal controls are the functions identically equal to 1 or -1. Conclusion 2.17. The maximum principle in Problem 2'" is not a sufficient optimality condition. Moreover, the unique singular control is not optimal. Conclusion 2.18. Problem 2'" has two solutions. Remark 2.8. In Problem 2', there were infinitely many nonoptimal singular controls and two nonsingular optimal solutions of the maximum principle. In the present case, on the contrary, there are infinitely many nonoptimal nonsingular solutions of the maximum principle and a unique singular optimal control.

We see that the existence of a singular control in optimization problems is not very unusual. Moreover, there may be as many singular controls as possible and these controls may be optimal or nonoptimal. The question arises of whether there exists a singular control in every problem. We now return to Example 1. The function H was defined by the formula

H (u ) = pu − (u 2 + x 2 ) / 2. Any change in the control will obviously cause a change in the function H. We can make the first term in the right-hand side of this formula vanish by choosing the control so that p vanishes. However, we cannot get rid of the squared control in this expression. Therefore, there is no singular control in this case.

65

Conclusion 2.19. Singular controls do not necessarily exist in every problem.

A singular control obviously may exist if H has the form

H = f1 ( x, p ) f 2 (u ) = f3 ( x, p ). Now if there exits a control such that the functions x and p satisfy the equality f1 ( x, p) = 0 , then this control is singular. Remark 2.9. The foregoing equality may be considered as a problem of finding a singular control.

The last question to answer is how to determine whether a given singular control is optimal. 2.6. THE KELLY CONDITION

In the previous sections, we checked singular controls for optimality by analyzing the sign of the remainder term in the formula for the functional increment. However, as mentioned before, the sign of the remainder in can be determined only in very simple cases. Besides, this method is implied in the case of arbitrary form of solutions of the maximum principle whereas singular controls have some specific properties that make them different from other solutions. Therefore, there may exist some relations that could allow us to establish the optimality criterion specifically for singular controls. Indeed, there exists a series of optimality conditions for singular controls. We shall consider only one of them, possibly the most commonly used. The following statement presents the so-called Kelly condition. Theorem 4. If the functions in the problem formulation are sufficiently smooth, then the optimal singular control satisfies the Kelly condition.

∂ d 2 ∂H ∂u dt 2 ∂u

≥ 0.

Remark 2.10. The proof of this theorem is beyond the scope of this book. Remark 2.11. Relation (2.16) is written for the problem of minimizing the functional. In the case of maximizing the functional, the sign in (2.10) must be reversed.

66

To use the Kelly condition, we should first find a singular control and then determine the value of the expression in the left-hand side of (2.16) at this control. Our purpose now is to find out if the Kelly condition holds for the problems with singular controls considered above. In particular, the following equality holds for Problem 2":

∂H ∂u

= р.

Taking into account the form of the adjoint system (2.11), we find the derivative

d ∂H

= р& = х.

dt ∂u Using equation (2.10), we obtain

d 2 ∂H dt 2 ∂u

= х& = u.

Finally, we have

∂ d 2 ∂H ∂u dt 2 ∂u

= 1,

which leads us to the following conclusion. Conclusion 2.20. The Kelly condition holds for Problem 2".

Hence, the corresponding singular control may be optimal. In fact, as we already know, it is optimal. As far as Problem 2'" is concerned, since the functional therein is being maximized, the sign in (2.16) must be reversed. As a result, the Kelly condition fails. Conclusion 2.21. The Kelly condition does not hold for Problem 2"'.

This means that the corresponding singular control cannot be optimal, as was shown before. Thus, we have verified the effectiveness of the Kelly condition. For Problem 2, we have

67

∂H ∂u

= р − х.

Using equations (2.10), (2.11), we obtain

d ∂H dt ∂u

= р& − х& = u − u = 0.

Differentiating this equality with respect to t and then with respect to u, we see that the Kelly condition in Problem 2 becomes an equality. Therefore, all the corresponding singular controls may be optimal. Indeed, we already know, they are optimal. Conclusion 2.22. The Kelly condition holds for Problem 2".

Problem 2' differs from Problem 2 only in the type of extremum. Therefore, if (2.16) holds in the form of equality for the corresponding function H, then the Kelly condition holds for this problem as well. This implies that singular controls in Problem 2' may be optimal (however, in fact, they are not. Conclusion 2.23. The Kelly condition holds for Problem 2', but its singular controls are not optimal.

The foregoing result is not contradictory. We know that every optimal singular control must satisfy the Kelly condition. Hence, if the Kelly condition fails, it means that the corresponding singular control is not optimal, However, if the Kelly condition holds, it does not guarantee the optimally of the singular control. Conclusion 2.24. The Kelly condition is necessary but not sufficient for the optimality of singular controls. Remark 2.12. For a function f to have a minimum at a point x, it is necessary that f ′( x ) = 0 . It is required to verify the additional condition f ′′( x ) > 0 for the obtained solution of the last equality. If it fails, x is not a point of minimum for the given function. We used the same approach earlier when we studied the maximum condition. In applying the Kelly condition, the situation is completely analogous. First of all, we find singular solutions of the maximum principle and then verify if the Kelly condition holds for them. If it does, then the obtained singular control may be optimal; otherwise, the control is certainly nonoptimal. Remark 2.13. The Kelly condition is an optimality condition of the second order since its definition involves the second derivative of the function H with respect to the control.

68

Remark 2.14. We still have not solved the problem of existence of singular controls, nor have we studied the methods of finding singular controls in the problems of the general form. One of the methods of finding singular controls will be presented in Example 5.

SUMMARY

The following conclusions can be made on the basis of the analysis we have carried out. 1. The maximum principle may be degenerate in some cases, which means that it may have specific solutions called singular controls. 2.

The maximum principle may have singular and nonsingular controls in any combination: either only one kind or both kinds at the same time.

3.

If the functional to be minimized is convex but not strictly convex, then the optimal control problem may have more than one solution,

4.

The set of singular controls can be finite or infinite.

5.

Singular controls (like other solutions of the maximum principle) can be optimal or nonoptimal.

6.

Optimal singular controls satisfy the Kelly condition.

7.

Nonoptimal singular controls may or may not satisfy the Kelly condition.

Example 3. Nonexistence of optimal controls We have shown that substantial difficulties may arise in the process of solving optimal control problems when analyzing the necessary optimality conditions. In particular, in some situations, the optimality conditions may hold for more than one control. This may happen if solutions of the problem are nonunique or if the optimality conditions are not sufficient. However, the opposite situation may occur as well, which is even worse. Necessary optimality conditions in the form of the maximum principle may have no solution at all. If we try to use the method of successive approximations to solve a problem of this kind, then the iterative process will diverge for any initial approximation. The reason is that there are no optimal controls in this case. In what follows, we establish the existence theorem for extremum problems of the general form. The resulting statement will be used to analyze the optimal control problems considered in the previous chapters in the case where an optimal control exists and in the case where it does not exist. 3.1. PROBLEM FORMULATION

As before, let the state of a system he described by the Cauchy problem

x& = u , t ∈ (0,1); х(0) = 0 The control u = u (t ) is again chosen from the set

U = {u u (t ) ≤ 1, t ∈ (0,1)} . The optimality criterion has the form

70

(3.1)

I=

1

∫(x 2

1

2

)

− u 2 dt.

0

Problem 3. Find a control u ∈ U that minimizes the functional I оn the set U.

In the previous chapters, we considered optimal control problems w the same state equation and the same set of admissible controls and simile quadratic optimality criterions. However, both terms in the integrand of the minimized functional had the same sign. If both terms are positive, the problem has a unique solution and the optimality conditions are necessary I and sufficient. If both terms are negative (which corresponds to the problem 3 of maximization from Example 1), then there exist two optimal controls, and the maximum principle is not a sufficient optimality condition. We shall show that the results turn out to be dramatically different from those obtained above if the functional contains terms with different signs. 3.2. THE MAXIMUM PRINCIPLE

Following the well-known method, we define the function

(

)

H = H (u ) = pu − x 2 − u 2 . Then the adjoint system is again described by the relations

p& = х, t ∈ (0,1); p (1) = 0.

(3.2)

For the control и to be optimal, it is necessary that it satisfies the maximum condition

H (u ) = max H ( w). | w| ≤1

(3.3)

From the stationarity condition (which requires that the derivative of H with respect to the control be equal to zero), it follows that и = p. However, the second derivative of H is positive (being equal to unity). Thus, we have the point of minimum rather than maximum of the function under consideration. Remark 3.1. We again point out the close connection between the necessary optimality conditions and the methods of analyzing functions for extrema. First, we find a stationary point, which is a solution of the necessary condition of the first order for an extremum. Then, since the necessary condition of the second order fails

71

(the condition that the second derivative be nonnegative), we dismiss this point. A similar procedure was used in Problem 2'. First, a singular control was found (which was a solution of the maximum principle, i.e., an optimality condition of the first order). Then this solution was dismissed since it did not satisfy the Kelly condition (a necessary condition of the second order).

Under the present conditions, H may achieve its maximum only on the boundary of the set of admissible controls. We have

H (1) = p − ( x 2 − 1) / 2, H ( −1) = − p − ( x 2 − 1) / 2. Taking the maximum of these values, we obtain the formula

⎧ 1, р (t ) > 0,

u (t ) = ⎨

(3.4)

⎩ −1, р (t ) < 0.

Thus, we have relations (3.1), (3.2), and (3.4) for finding an optimal control, which is very similar to the systems of optimality conditions considered before. In order to solve them, we can use, for example, the method of successive approximations. However, we shall analyze this problem using direct methods. 3.3. ANALYSIS OF THE OPTIMALITY CONDITIONS

As follows from (3.4), the desired control must be a piecewise-constant function, its points of discontinuity corresponding to the points where the function p changes sign. For example, assume that the solution of the adjoint system is positive. Then the corresponding control is identically equal to unity, which follows from (3.4). Substituting this value into the state equation, we find x (t ) = t . Therefore, the solution of the adjoint system is 1

1

t

t

р (t ) = − ∫ x(τ ) dτ = − ∫ τ dτ =

t2 −1 2

.

This function assumes only negative values, which contradicts the assumption that p is positive. Assume now that the function p is negative everywhere. Then the corresponding control is identically equal to -1. Solving problem (3.1), we obtain x (t ) = −t . Substituting this function into (3.2), we have 72

1

1

t

t

р (t ) = − ∫ x(τ ) dτ = ∫ τ dτ =

1− t2 2

.

Hence, the solution of the adjoint system assumes only positive values, which contradicts the assumptions again. Conclusion 3.1. A constant control cannot be a solution of the system (3.1), (3.2), (3.4).

However, it is not impossible for the control to have a discontinuity, while the corresponding solution of the adjoint system changes sign at some point. In particular, suppose that there exists a point ξ ∈ (0,1) such that p is positive for t < ξ and negative for t > ξ . Then formula (3.4) provides the control

⎧ 1 , t < ξ, ⎩ −1, t > ξ .

u (t ) = ⎨

The corresponding solution of problem (3.1) for t < ξ is x (t ) = t . Taking into account that x(ξ ) = ξ , for t > ξ we have t

x(t ) = x(ξ ) − ∫ dτ = 2ξ − t. ξ

It follows that the system state is

⎧ t, t < ξ , ⎩ 2ξ − t , t > ξ .

х (t ) = ⎨

We now find the solution of the adjoint system (3.2) for t > ξ : 1

р (t ) = − ∫ (2ξ − τ ) dτ = 2ξ (t − 1) + (1 − t 2 ) / 2 = t

= (1 − t ) [ (1 + t ) / 2 − 2ξ ] .

By assumption, this function must change its sign at t = ξ , which implies 73

p (ξ ) = (1 − ξ )(1 − 3ξ ) / 2 = 0. Two values of the parameter ξ satisfy this equality. Since ξ = 1 is a boundary point for the considered time interval, we conclude that there is a unique point ξ = 1 / 3 at which the solution of the adjoint system can change its sign. We can now determine the solution of problem (3.2) for t < 1 / 3 : 1/ 3 t 2 − 1/ 9 ⎛1⎞ p (t ) = p ⎜ ⎟ − ∫ τ dτ = . 2 ⎝3⎠ t

It is interesting that p is negative for t < 1 / 3 and positive for t > 1 / 3 , although the opposite result was expected. We have to conclude that there are no functions p with the desired properties. On the other hand, if we suppose that the solution of the adjoint system first assumes negative values and then positive values, we shall arrive at a contradiction again. Conclusion 3.2. The system (3.1), (3.2), (3.4) cannot have a solution with a single point of discontinuity.

We can assume that the control has two points of discontinuity and, consequently, that p changes its sign twice. However, if suppose that it is positive on the first segment, we shall again arrive at a contradiction. The same is true for the assumptions of any given number of discontinuity points of the control. Conclusion 3.3. The system (3.1), (3.2), (3.4) has no solutions. Remark 3.2. The above conclusion would not seem surprising if we noticed that problem (3.1), (3.2), (3.4) and the system of optimality conditions (1.1), (1.2), (1.4) differ only the sign of one of the relations. In the first example, after a certain assumption was made about the sign of p, we found a confirmation of this assumption, which yielded a solution of the optimality conditions. In the present situation, every assumption of this kind turns out to be wrong because the relations (1.4) and (3.4) have different signs.

It may first seem that this result is not that unfortunate. We already encountered systems of optimality conditions with no solutions, for example, the system (2.1), (2.2), (2.4). Moreover, the problem (2.11)-(2.13) coincides with the problem (3.1), (3.2), (3.4). In both cases, we were able to find optimal controls, although the corresponding systems were not equivalent to the maximum principle and the optimal controls turned out to be singular. The major difference between problem 3 and the above-mentioned problems is that it does not have singular controls and formula (3.4) is equiva74

lent to the maximum principle (3.3). The reason is that the function H contains the squared control, so that it is impossible to get rid of it and make the maximum principle degenerate. Conclusion 3.4. The maximum principle in problem 3 has no solutions.

We now try to understand what will happen if we formally use the method of successive approximations for solving the systems of optimality conditions. The corresponding iterative process obviously does not converge for any initial approximation. Moreover, converging algorithm of approximate solution cannot exist for this system because there are no possible limits for convergence in this case. Conclusion 3.5. An iterative process for solving the optimality conditions in Problem 3 does not converge for any initial approximation. Remark 3.3. The worst thing here is that in practice we often have to use formal methods to solve a given problem, not knowing in advance whether the system of optimality conditions has a solution or not. In this case, it is hard to establish the real reason for the algorithm to not converge. It remains unclear whether this circumstance is caused by the absence of solutions or by some unfavorable properties of the algorithms itself, for example, the wrong choice of the initial approximations. It is certain, however, that the insolvability of the problem may be one of the min reasons for the algorithm to fail to converge. Remark 3.4. In the next example, we shall establish that in the case where a problem has no optimal control, the algorithm of solution may converge (although not to the solutions of the problem).

We now try to find out the consequences of the insolvability of the maximum principle. It is known that every optimal control must satisfy the maximum principle. Thus, the set of solutions of the necessary optimality conditions, in general, contains the set of optimal controls. However, under the present conditions, the set of solutions of the maximum principle is empty. This may happen only in the case where the optimal problem in question in unsolvable. Conclusion 3.6. Problem 3 is unsolvable. Remark 3.5. The above considerations may cause some doubts. On one hand, we proved that there is no optimal control by establishing the unsolvability of the system of necessary optimality conditions. On the other hand, the derivation of the maximum principle seemed to start from the assumption of existence of an optimal control. It may seem that we are trapped in a vicious circle. First, we obtain the maximum principle assuming that an optimal control exits, and then we conclude that there is no optimal control because the maximum principle is unsolvable. It is easy to verify, though, that our conclusions are not contradictory. If there existed an

75

optimal control, then it would satisfy the maximum principle. However, since the maximum principle has no solutions, our initial assumption that the optimization problem is solvable was false.

We now try to prove without using the maximum principle that there is no optimal control. 3.4. UNSOLVABILITY OF THE OPTIMIZATION PROBLEM

Here we present a direct proof of the unsolvability of the optimization problem under consideration. Prom the definition of the set of admissible controls, we obtain the inequalities

x 2 (t ) ≥ 0, u 2 (t ) ≤ 0, t ∈ (0,1). Then the following formula holds for every admissible control:

I=

1 2

1

2 ∫ x dt − 0

1 2

1

1

2 ∫ u dt ≥ − .

2

0

(3.5)

Thus, the values of the functional to be minimized on the set of admissible controls can be estimated from below. Consider the sequence {uk } defined as follows (see Figure 14):

2j 2j +1 ⎧ 1, ≤t< , ⎪ 2k 2k uk (t ) = ⎨ , k = 1, 2,... . 2j +1 2j + 2 ⎪−1, . ≤t< ⎩ 2k 2k u1

u2

(3.6)

u3

t

t

Figure 14. The sequence of states corresponding to the controls (3.6).

76

t

We now estimate a solution xk of problem (3.1) corresponding to the control uk . It is a continuous piecewise-differentiable function (see Figure 15). х1

х3

х2

t

t

t

Figure 15. The sequence of states corresponding to the controls (3.6)

For 2j/2k ≤ t < (2j+1)/2k , we have t

j −1

⎡ ( 2 i +1) / 2 k

i =0

⎣

xk (t ) = ∫ uk (τ ) dτ = ∑ ⎢ 0

t

+

∫

∫

⎤

(2 i + 2) / 2 k

uk (τ )dτ +

∫

uk (τ )dτ ⎥ +

⎦

( 2 i +1) / 2 k

2i / 2 k

2j ⎛ 1 1 ⎞ ⎛ 2j⎞ − ⎟ + ⎜t − ⎟ = t − . 2k ⎠ ⎝ 2k ⎠ 2k i = 0 ⎝ 2k j −1

uk (τ )dτ = ∑ ⎜

2 j / 2k

Similarly, for (2j + l)/(2k) < t < (2j + 2)/(2k), we have

x k (t ) =

⎛ 2 j +1 ⎞ 2 j + 2 −t⎟ = − t. 2k ⎝ 2k 2k ⎠ 1

−⎜

The obtained relations yield the following inequality (see Figure 16) 0 ≤ хk(t) ≤ 1/2k, t∈(0,1) , k = 1,2, … . uk

uk St

2j/2k

t

t

2j/2k

1/2k

1/2k

Figure 16. The state xk does not exceed 1/(2k)

77

St

Using condition (3.5), we have

−

1 2

≤ Ik =

1

∫( x 2

1

2 k

)

− uk2 dt ≤

0

1 8k

2

−

1 2

, k = 1, 2,... .

Passing to the limit as k → ∞ , we establish that I → −1 / 2 . As follows from (3.5), the value of the functional being minimized is not less than -1/2 at every admissible control. At the same time, for k sufficiently large, the value of the functional at uk given by formula (3.6) is as close to -1/2 as desired. Thus, {uk } is a minimizing sequence, i.e., a sequence of admissible controls such that values of the optimality criterion at these controls converge to its infimum on the set of admissible controls. Conclusion 3.7. The infimum of the functional being minimized on the set of admissible controls is equal to -1/2. Remark 3.6. The sequence xk is associated with an interesting fact which is

irrelevant to the subject of our research. The length l ( xk ) of the curve of every

2 . We have xk → x∞ as k → ∞ in the space of

function xk on [0,1] is equal to

continuous functions, where x∞ ≡ 0 and l ( x∞ ) = 1 . Thus, lim l ( xk ) ≠ l (lim xk ) , i.e., the limit of the sequence of the length of curves is not equal to the length of the limit curve. We conclude that the functional that maps every continuous function on [0,1] to the length of its curve is discontinuous.

From condition (3.5), it follows that the admissible control that provides the infimum of the functional on the set of admissible controls must ensure that the following two equalities hold: 1

1

∫ x dt = 0, ∫ u dt = 1. 2

2

(3.7)

0

0

As is known, the system state is defined by the formula t

x (t ) = ∫ u (τ ) dτ . 0

This equality implies that x is continuous if the control is integrable. Then the first equality in (3.7) implies that x is identically equal to zero. From equation (3.1), it follows that the corresponding control is also identically 78

equal to zero and therefore does not satisfy the second equality in (3.7). Hence, if the first equality in (3.7) holds, then the second one does not. At the same time, the functional can achieve its lower bound only if both of these equalities hold. Therefore, the functional never achieves its lower bound, which means that there is no optimal control in Problem 3. Conclusion 3.8. The functional I never achieves its lower bound on U.

This result may seem particularly surprising. Indeed, for a given control we have a certain value of the optimality criterion. Another control corresponds to another value of the functional being minimized. One of the selected controls must be better than the other according to this criterion. It is naturally to suppose that some control is being better than all the others since the set of admissible controls is bounded. Then how can we explain the unsolvability of the optimization problem? The fact that there is no optimal control means that for any given control there exists an admissible control that provides a smaller value of the functional. In particular, for any admissible control v, there is a number к such that the value of I at u k defined by (3.6) is less than I(v). Remark 3.7. The described situation is completely similar, for example, to finding the minimum in the open interval (0,1). However small a positive number may be, there is a smaller one in (0,1). Although the interval is bounded, it has no minimum element because the smallest positive number does not exist. Since this fact is of no surprise, we should not think it is so much extraordinary that the considered functional has no minimum.

We now return to the sequence of admissible controls {uk } defined by (3.6). We know that this is a minimizing sequence, i. e., the corresponding values of the functional converge to its lower bound on the set of admissible controls. It may seem that the limit of this sequence can be an optimal control. Indeed, if this sequence converges to a function u, it could be true that I (u k ) → I (u ) . We already know that the sequence {I (uk )} converges to the lower bound of the functional on the set of admissible controls; therefore, w could conclude that it coincides with I(u). Thus, the function и would be a solution of the problem which was proved to be unsolvable (using two different methods). These contradictions are resolved by admitting that the sequence {u k } does not converge, i.e., it has no limit in any reasonable class of functions. Indeed, as k increases, the number of discontinuity points of the corresponding control increases (see Figure 14). Moreover, for any time interval, if k is 79

sufficiently large, then u k will have as many points of discontinuity in this interval as desired. Therefore, the minimizing sequence does not converge in all conventional classes of functions and an optimal control cannot be its limit. Remark 3.8. In the sequel, we shall show that the minimizing sequence is not useless even if does not converge.

It would be good to find out a criterion that determines the solvability of optimization problems. First, it is interesting to know whether it is possible to establish such a criterion at all (before obtaining the optimality conditions). In one of the above examples we already established that there is no optimal control without even considering the optimality conditions. In addition, we could use Weierstrass's theorem, which states that every continuous function achieves its minimum on a closed bounded set. 3.5. EXISTENCE OF OPTIMAL CONTROLS

We present a result that states the shows the conditions of solvability for an extremum problem. Suppose that we need to find a function minimizing a functional I on a given set of admissible controls U. If I is bounded from below, then its range I(U) has a lower bound. This means that there exists a minimizing sequence, i.e., a sequence of elements u k ∈ U such that I (u k ) → inf U . However, it is not known whether the sequence {u k } itself converges. Assume that U is a bounded subset of a normed vector space V. Then there exists a positive constant с such that || v ||≤ c for all admissible controls. In this case, the sequence u k is uniformly bounded, i. e., || v ||≤ c for all k. Let V be a Hilbert space, i.e., a complete vector space equipped with a scalar product. By the Banach – Alaoglu theorem (a generalization of the classical Bolzano – Weierstrass theorem to the case of infinitedimensional spaces), { u k } has a subsequence that weakly converges in V. If we denote the subsequence by { u k } again, the convergence u k → u means that the scalar products (u k , λ ) converge to (u, λ ) for every function λ ∈ V . We have established so far that there exists a weak limit of the minimizing sequence. But it is not clear if this limit belongs to the set of admissible controls. Suppose that U is convex and closed (and, consequently, contains the limits of every sequence of its elements that converges in the norm). It is known from the theory of Hilbert spaces that every 80

convex closed subset I of a Hilbert space is weakly closed, i.e., it contains the limits of all weakly converging sequences of its elements. Since the minimizing sequence consists I only of the elements of U and weakly converges, it follows that its weak limit и belongs to U and is therefore an admissible control. However, it is not known whether the functional achieves its lower bound at u. Assume that the functional I is convex and continuous. Every convex continuous functional is weakly lower semi continuous. This means that if u k → u weakly in V, then

I (u ) ≤ inf lim I (uk ).

(3.8)

This inequality implies that the sequence {I(uk)} has converging subsequences, although it does not necessarily converge itself (which follows fro the Bolzano – Weierstrass theorem and the Banach – Alaoglu theorem). As follows from (3.8), since the functional is weakly semi continuous, I(u) does not exceed the lower bound of limits of all subsequences of {I(uk)}. Since the subject of our consideration is not an arbitrary weakly converging sequence, but the one that minimizes the functional on U, {I(uk)} note only has converging subsequences, but converges itself to the lower bound of the functional I on U. Then inequality (3.8) can be written in the form

I (u ) ≤ lim I (u k ) = inf I (U ), which means that the value of the functional I at the element и does not exceed its lower bound on the set U. We established earlier that this element belongs to U. Since none of the elements of a set of numbers can be less than its к bound, the foregoing relation turns out to be an equality. The value of the functional at the element u ∈ U is equal to its lower bound on U. Thus, the admissible control и is a solution to the problem in question. Theorem 5. The problem of minimizing a convex lower semicontinuous functional bounded from below on a convex closed bounded subset of a Hilbert space is solvable. Remark 3.9. In general, the control does not necessarily have to be an element of a Hilbert space. The assertion of Theorem 5 also holds for reflexive Banach spaces of the general form (and even for a more general class of spaces conjugate to Banach spaces) which also satisfy the assumptions of the Banach—Alaoglu theorem. The only difference, although minor, is connected with a more complicated definition of weak convergence: we cannot use the scalar product in the general

81

case. An example of a reflexive Banach space is the space of functions that are Lebesgue integrable with any power greater than unity. Remark 3.10. In the next example, we shall show that the existence of an optimal control can be established without the assumption that the set of admissible controls is bounded. In this case, an additional condition will be required to be imposed on the functional. Remark 3.11. In Example 7, we establish the existence of an optimal control without using the assumption that the set of admissible controls is convex. Remark 3.12. In Theorem 2, we obtained the conditions, under which the problem has no more than one solution, i.e., if a solution exists, then it is unique. Theorem 4 states the existence of a solution rather than its uniqueness. If the assumptions of both theorems hold, then the existence and uniqueness of solution is established. Thus, if we add the assumption that the functional is strictly convex to the hypotheses of Theorem 5, we shall obtain the assertion on the existence and uniqueness of solution.

We shall now try to use Theorem 5 to establish the existence of a solution to Problem 0, which was solved in the previous chapters. 3.6. THE PROOF OF THE SOLVABILITY OF AN OPTIMIZATION PROBLEM

We now return to Problem 0. Its system was described by the relations

x& = u , t ∈ (0,1), x (0) = 0. The optimal control problem consisted in finding a function u = u (t ) from the set

U = {u u (t ) ≤ 1, t ∈ (0,1)} that minimizes the functional

I=

1

∫ (u 2

1

2

)

+ x 2 dt.

0

Before using Theorem 5 for analyzing this problem, we must take into account that the functional depends on the control not only directly (through the first term in the integrand) but also through the system state. This fact was already mentioned when applying the theorem on the uniqueness of an optimal control to this problem. 82

We now find ourselves in an essentially new situation. As follows from the formulation of Theorem 5, in order to establish the existence of an optimal control, it is necessary to specify a function space to which the control function must belong. We choose the space V = L2 (0,1) of the functions Lebesgue square-integrable on a given time interval. Indeed, V I a Hilbert space with the scalar product 1

(u , λ ) = ∫ u (t )λ (t ) dt 0

and the norm defined by the equality

u

2

1

= ∫ u (t ) dt. 2

0

To be able to use Theorem 5, we must first verify whether the set of admissible controls and the functional to be minimized satisfy the corresponding properties. We already showed that the set of admissible controls is convex when establishing the uniqueness of an optimal control. In addition, we shall prove that this set is closed. Assume that there is a converging sequence {u k } of elements of U, i.e., u k → u in V. it is required to prove that u ∈ U . It is known that if the sequence of elements of L2 (0,1) converges in the norm of this space, then there exists a subsequence that converges almost everywhere. Thus, there exists a subsequence of {u k } (which will be denoted by {u k } again) such that u k (t ) → u (t ) for all t ∈ (0,1) except maybe for a set of zero measure. Since u k ∈ U , we have

| u k (t ) |≤ 1, t ∈ (0,1). Passing to the limit in this inequality, we obtain

| u (t ) |≤ 1, t ∈ (0,1). which holds for almost all t ∈ (0,1) . Since the elements of L2 (0,1) are measurable, they can be arbitrarily altered on a set of zero measure. Hence, u ∈U . Conclusion 3.9. The set U is closed and therefore weakly closed.

83

Thus, the set of admissible controls possesses all the necessary properties. We now consider the functional to be minimized. It is obvious that it is bounded from below (by zero), and its convexity was established when proving the uniqueness of solution. It remains to prove that the functional I is continuous. Suppose that u k → u in V. Using the state equation, we obtain t

xk (t ) − x (t ) = ∫ uk (τ ) − u (τ ) dτ ≤ 0

1/ 2

⎛t ⎞ ≤ ⎜ ∫ 12 dτ ⎟ ⎝0 ⎠

⎡t ⎤ 2 ⎢ ∫ uk (τ ) − u (τ ) dτ ⎥ ⎣0 ⎦

1/ 2

≤ t uk − u ,

where xk and x are the system states corresponding to the controls u k and u. Hence, xk → x in the class of continuous functions and therefore in L2(0,1). We have 1 1 ⎤ 1⎡ I (uk ) − I (u ) ≤ ⎢ ∫ (xk − x)(xk + x)dτ + ∫ (xk − x)(xk + x)dτ ⎥ ≤ 2 ⎣0 0 ⎦

≤

1 ⎡ xk − x 2⎣

xk + x + uk − u uk + u ⎦⎤ .

From the definition of the set of admissible controls, it follows that every admissible control v satisfies the estimate

v

2

1

= ∫ v(t ) dt ≤ 1. 2

0

The corresponding system state у satisfies the inequality

у (t ) =

t

t

0

0

∫ v(t )dt ≤ ∫ v(t ) dt ≤ t ,

which implies ||y|| ≤ t2/2. Hence, I (u k ) → I (u ) .

84

Conclusion 3.10. The functional I is continuous and therefore weakly lower semicontinuous. Remark 3.13. We conclude that the property of weak continuity (or semicontinuity) is stronger then the property of strong continuity in terms of the type of convergence. In particular, if the functional is strongly continuous, the convergence of the sequence of controls implies the convergence of the corresponding values of the functional. If the functional is weakly continuous, the same result follows from the weak convergence of the controls. The convergence of the values of the functional follows from the weak convergence of controls not as easily as from their strong convergence. That is why the property of weak continuity is considered to be stronger than strong continuity in this case. For this reason, we need the additional requirement of convexity to obtain the weak semicontinuity of the functional from its strong continuity (semicontinuity). Similarly, we can show that the property of being weakly closed for a set is stronger than the property of being strongly closed. The reason is that a weakly closed set must contain not only strong limits but also weak limits of its subsequences.

All the assumptions of Theorem 5 appear to be hold for the problem under consideration. Therefore, we can use this theorem to establish the existence of an optimal control for the extremum problem without solving it. 3.7. CONCLUSION OF THE ANALYSIS

We have shown that there exist simple and reasonable optimal control problems that have no solutions. We have also obtained a statement which allows us to prove the solvability of extremum problems of the general form. This statement was used to establish the existence of an optimal control for a specific optimization problem that was analyzed in the previous chapters. Now we are interested why the obtained theorem cannot be used to analyze Problem 3. The sets of admissible controls in both examples are the .same. The state equations are also the same. Thus, these problems differ only in their optimality criteria. Since the set of admissible controls is bounded, using the connection between the state and the control in system (3.1), it is easy to prove that the functional in Problem 3 is bounded from below. Its continuity can be established in the same manner as for the functional of Problem 0. However, the squared control in the integrand has negative sign and therefore the functional is not convex. Thus, the assumptions of Theorem 5 do not hold, and it is no surprise that there is no optimal control. Conclusion 3.11. Problem 3 is unsolvable because the functional being minimized is not convex.

85

The question arises of whether the optimal control problem is unsolvable whenever the functional being minimized is not convex. We note that Example 2 dealt with the problem of maximizing a convex functional, which was equivalent to minimizing a concave functional. The assumptions of Theorem 5 fail for this problem as well (since the functional is not convex). Nevertheless, the problem has even more than one solution. Conclusion 3.12. The optimal control problem may be solvable even if the functional being minimized is not convex.

Another question is connected with the minimizing sequence {u k } for Problem 3 (the sequence of piecewise-constant controls with increasing number of discontinuity points). This sequence is certainly bounded. Using the Banach-Alaoglu theorem the way we did in the proof of Theorem 5, we show that there exists a subsequence of {u k } weakly converging in L2 (0,1) . However, we stated that this sequence does not converge. It cannot converge in the norm of L2(0,1). Conclusion 3.13. The sequence {u k } converges weakly and does not

converge strongly in L2 (0,1). Remark 3.14. For the problem of minimizing a continuous functional, if the extremum problem is unsolvable, then every weakly converging minimizing sequence does not converge strongly.

We can prove that the function identically equal to zero is the weak limit of the sequence. It obviously belongs to the set of admissible controls (since this set is convex and closed and therefore weakly closed, it contains all weak limits of its sequences). The question arises of why the weak limit of the minimizing sequence is not an optimal control. We know that the functional is continuous. Unfortunately, this property is not sufficient for the weak convergence of the sequence of controls to imply the convergence of the functional sequence. The additional requirement is the weak continuity (or at least semicontinuity) of the functional. Since the functional is not convex, its strong continuity does not imply weak lower semicontinuity. Nevertheless, in this case we can assert that the functional being minimized is not weakly lower semicontinuous, because otherwise we could use the weak convergence of controls to prove that the corresponding weak limit is optimal. Conclusion 3.14. The functional being minimized in Problem 3 is continuous but is not weakly lower semicontinuous.

86

After all we have learned about the considered example, it may seem that the properties of the remainder terra in the formula for the functional increment are not important for us. We shall find out its sign anyway. As is known, T

η = − ∫ (η1 + η 2 ) dt. 0

Here η 2 = 0 since neither the functional nor the state equations contain the terms depending on the control and the state simultaneously. The value of η1 is given by the formula

η1 =

− ( x + Δx ) 2 + x 2 + 2 x Δx

2

=−

Δx 2

2

.

As a result, the remainder term in the formula of the functional increment is nonnegative. Apparently, this should lead us to conclude that the maximum principle for Problem 3 is a necessary and sufficient optimality condition. It may seem unreasonable to discuss the sufficiency or necessity of optimality conditions in the situation where the problem has no solutions. Nevertheless, as we mentioned before, the necessity of the system of optimality conditions only means that the set of its solutions, in general, contains the set of optimal controls (optimal control certainly satisfies the optimality conditions, see Figure 10). In contrast to this, the sufficiency of the system of optimality conditions means that the set of its solutions, in general, is contained in the set of optimal controls (every solution of the optimality conditions is optimal). If the optimality conditions turn out to be necessary and sufficient, then the above-mentioned sets coincide. In the present example, both of these sets are empty; therefore, they coincide. This means that the result of our analysis of the remainder term in the formula for the functional increment is valid. Conclusion 3.15. The maximum principle in Problem 3 is a necessary and sufficient optimality condition.

It might seem that an unsolvable optimization problem is totally meaningless. Apparently, there is no point in solving a problem with no solutions. However, we now recall that the lower bound of the functional on the set of admissible controls does exist. Then we may try find an admissible control such that the value of the functional at this control is as close to its lower bound as desired. This problem is well posed. For sufficiently large k, 87

an element of the sequence {xk } defined above can serve as an example of such approximate solution. Indeed, any problem formulation reflects the studied phenomenon only to a limited extent. We only have approximate methods to solve equations and optimization problems. In this sense, our attempt to find an approximate solution is justified from both theoretical and practical points of view. Conclusion 3.16. For unsolvable optimization problems, there may exist reasonable approximate solutions, i.e., controls that provide the values of the functional sufficiently close to its lower bound. Remark 3.15. An assertion that a certain mathematical problem has no solutions is not really well defined. It is usually implied that there are no solutions in a particular class of objects. In most cases, however, it is possible to specify a larger class of objects such that the problem is solvable for this class.

The considered optimal control problem may have physical meaning. Suppose that we need to sail into the wind from a point A to a point В (see Figure 17). In this case, the control is represented by the angle of the sail to the wind; the system states are the coordinates of the sailboat. To reach the desired point B, the sailboat must move changing the angle of the sail to the wind from time to time (the trajectory s1 ). If the passage is narrow, the sailboat must tack frequently (the trajectory s2 ). Moving along the straight line AB means that the tacking frequency must be infinitely high. Possible trajectories sk of the sailboat resemble the states x of system (3.1) corresponding to the minimizing sequence considered before (see Figure 15). The segment AB represents the limit state of the system (which is, in a sense, an optimal state). However, no admissible control corresponds to this state. In the control theory, this situation is known as the sliding mode. Conclusion 3.17. Unsolvable problems of optimal control may have physical meaning. Remark 3.16. The fact that unsolvable optimal control problems can be meaningful both from mathematical and physical points of view testifies to the importance of studying such problems. In particular, we can consider the problem of finding minimizing sequences for unsolvable optimal control problems. This problem can be solved in the context of the extension of unsolvable optimal control problems.

88

moving of sailing vessel wind direction

Figure 17. Sailing into the wind Remark 3.17. In the considered example, minimizing sequences from quite a large class. In particular, this class includes the sequences of admissible controls of the unit norm weakly converging to zero (for example, various sets of periodic functions with infinitely frequency).

The system of optimality conditions can be represented in the form of a problem for a single unknown function. If a solution of the adjoint system is chosen to be this unknown function, then we obtain the boundary value problem

&& p = F ( p), t ∈ (0,1); p(1) = 0, p& (0) = 0,

(3.9)

where F(p) is the right-hand side of (3.4)/ since problem (3.9) is equivalent to our system of optimality conditions, we can formulate the following conclusion. Conclusion 3.18. The boundary value problem (3.9) has no solutions.

We now consider the nonlinear heat conduction equation

∂v ∂t

=

∂ 2v ∂ξ 2

− F (v ), t > 0, 0 < ξ < 1

with the boundary conditions

89

(3.10)

∂v ∂ξ

= 0, v ξ =1 = 0, t > 0

(3.11)

ξ =0

and some initial conditions, where the function F is of the same form as in (3.9). It is easy to see that an equilibrium state of the system (3.10), (3.11) is a solution of the boundary value problem

d 2v dξ

2

= F (v ), 0 < ξ < 1;

dv (0) dξ

= 0, v (ξ ) = 0,

which coincides with the problem (3.9) up to the notation. Conclusion 3.19. The system (3.10), (3.11) has no equilibrium states. SUMMARY

The following conclusion can be formulated on the basis of the presented analysis. 1. In the process of solving optimal control problems, the iterative process may fail to converge for every initial approximation, which can be caused by the fact that the optimality conditions have no solutions. 2. The optimality conditions have no solutions because of the unsolvability of the optimization problem. 3. The existence or non existence of an optimal control, in general, can be established without solving the optimization problem. 4. The problem may be unsolvable because the functional being minimized is nonconvex. 5. The optimal control problem may still be solvable if the functional is nonconvex. 6. If the optimization problem is unsolvable, the maximum principle may be a necessary and sufficient optimality condition, which implies that both the set of optimal controls and the set of solutions of the optimality condition are empty. 7. Unsolvable optimal control problems may have physical meaning. 8. If there is no optimal control, it is reasonable to formulate the following problem: Find an admissible control such that the value of the functional at this control is close to its lower bound as desired. 90

Example 4. Nonexistence of optimal controls (Part 2) The specific feature of the previous example was that the optimal control problem was unsolvable as well as the corresponding system of necessary optimality conditions. Although it may seem to be the worst possible situation, we will show that there are problems with even more unfavorable properties. So far we have studied optimization problems with free final state. In the following example, we need to find a control that transfers the system from one fixed state into another while minimizing a certain functional. The maximum principle can be applicable in such problems. In this case, boundary renditions for the adjoint equation are not specified, whereas the state equation has two boundary conditions. Nevertheless, the system of optimality conditions appears to represent a reasonable problem which may be solvable. In the process of analyzing the maximum principle for a specific example, we find what seems to be a unique solution. However, it becomes clear that there exists an admissible control that provides an even smaller value of the functional. Such a result can only be explained by the unsolvability of the optimization problem. One of the possible reasons for the nonexistence of optimal controls may be the fact that the set of admissible controls is unbounded. However, we will show that it is possible to establish the existence of a solution even under this condition. As in the previous example, we will find out that the problem may be unsolvable because the functional being minimized is nonconvex. Another surprise is the vanishing of the remainder term in the formula for the functional increment, which seems to imply that the optimality conditions are sufficient. At the same time, the existence of a nonoptimal solution of the maximum principle means that the optimality conditions are not sufficient. The reason for these surprising results is that the obtained

91

control only provides a local maximum of the function H and is not a solution of the maximum condition. 4.1. PROBLEM FORMULATION

Let the state of the system be described by the Cauchy problem

x& = u , t ∈ (0,1); х(0) = 0.

(4.1)

The set of admissible controls U consists of functions u = u (t ) such that the solution x of problem (4.1) satisfies the equality

x (1) = 1.

(4.2)

The optimality criterion is represented by the functional 1

I = ∫ 4 1 + u 2 dt. 0

Problem 4. Find a control u ∈ U that minimizes the functional I on U. x The main difference between Problem 4 and the previous problems is in condition (4.2), which represents a fixed final state. Thus, only the system states that have predefined initial and final values are admissible (see Figure 18). Remark 4.1. Strange as it may seem, the optimality criterion docs not depend explicitly on the system state. For this reason, 1 system (4.1) may appear to be unrelated to the functional being minimized. Moreover, the minimum value of the functional is equal to tunity, which is achieved only at the control identically equal to zero. However, the corresponding system state, which is a solution of problem (4.1), is also equal to zero and therefore does not satisfy the boundary condition (4.2). Therefore, the zero control is certainly not admissible and cannot be a solution of Problem 4. In this case, the state equation is used to determine the set of admissible controls and thus affects the optimality criterion.

Figure 18. Admissible states.

92

Remark 4.2. The constant function и = 1 is an example of a control tat takes the system from one state to another in a given period of time. The system is said to be controllable if it is possible to find a control that transfers it to a given state. The investigation of this important property is beyond the scope of this book.

For the problems with fixed final state, it is necessary to modify the procedure of deriving the maximum principle. 4.2. THE MAXIMUM PRINCIPLE FOR SYSTEMS WITH FIXED FINAL STATE

In this section, we obtain a necessary optimality condition for a general system with fixed final state. The system is described by the equation

x& = f (u, x), t ∈ (0, T ); x(0) = x0 ,

(4.3)

with the additional condition

x(T ) = x1 .

(4.4)

The control и is chosen from a certain set U. The optimality criterion is represented by the functional T

I = ∫ g ( u (t ), x(t ) ) dt. 0

Problem 4΄. Find a function u ∈ U that minimizes the functional I on U under the condition (4.4). Remark 4.3. We certainly should have specified the form of elements of the set U. In this case, however, our only purpose is to establish the form of the necessary optimality conditions for a problem with fixed final state.

Assume that u is an optimal control, i.e., for every control v ∈ U such that the corresponding state y satisfies (4.3) and (4.4) we have

ΔI = I (v, y ) − I (u , x) ≥ 0,

(4.5)

where x is the optimal system state. Remark 4.4. The problem of finding a control in U that transfers the system from one given state to another will not be considered here.

As before, we introduced the functional 93

T

L(u , x, р ) = I (u , x ) + ∫ р (t ) [ x& (t ) − f ( u (t ), x(t ) )] dt 0

and the function

H (u , x, p ) = pf (u , x ) − g (u , x ). Then condition (4.5) implies the inequality

ΔI = I (v, y , p ) − I (u , x, p ) ≥ 0 ∀p, which is analogous to condition (5). Performing transformations similar to those in the derivation of the maximum principle for the problem with free final state (see Introduction), we obtain T

T

0

0

− ∫ Δ u Hdt − ∫ ( Н х + р& ) Δxdt + η ≥ 0 ∀р, where the notation is the same as before. Since the function p is arbitrary, we will choose it so that the following adjoint equation holds:

р& = − Н х , t ∈ (0, T ). Therefore, we have T

− ∫ Δ u Hdt + η ≥ 0 ∀v, 0

which coincides with condition (10) (up to the properties of the function v). We conclude the derivation of the optimality conditions with the usual procedure and finally obtain the maximum principle

H [u (t ), x (t ), р (t ) ] = max H [ w, x(t ), р (t ) ] , t ∈ (0, T ). w∈U

(4.7)

Theorem 6 [The maximum principle]. For the control и to be a solution of Problem 4', it is necessary that и satisfy the maximum condition (4.7), where x satisfies (4.3), (4.4) and p satisfies equation (4.6). Remark 4.5. We again omit the restrictions imposed on the system which make this result possible.

94

It may first seem that the obtained system of optimality conditions is not really well defined. On one hand, the problem for the state function is overdetermined, since there are two boundary conditions specified for the first-order differential equation. On the other hand, there are no conditions specified for the adjoint equation. However, we actually consider a system of two first-order differential equations with two boundary conditions, which makes the system of optimality conditions reasonable. Conclusion 4.1. The maximum principle is applicable for solving the optimal control problem with fixed final state. 4.3. APPROXIMATE SOLUTION OF THE OPTIMALITY CONDITIONS

Our purpose now is to find out how the obtained system of optimality conditions can be solved in practice. Since there is no boundary condition for the adjoint system, it is not to apply the method of successive approximations the way it was in the optimal control problem with free final state. One of the methods that can be used for the approximate solution of the system of optimality conditions (4.3), (4.4), (4.6), (4.7) is the shooting method, in which the following additional condition is introduced

p (0) = a,

(4.8)

where a is an unknown numeric parameter. Suppose that the control is expressed in terms of x and p from the maximum condition (4.7). Then, specifying a certain value of a and solving the Cauchy problem (4.3), (4.6), (4.8), we can find x and p, which will obviously depend on a. we now define the function

F ( a ) = x (T ) − x1 , where x (T ) is the value of x at t = T for the given value of a. For condition (4.4) to hold, the parameter a must be chosen so that

F ( a ) = 0.

(4.9)

Equation (4.9) can be considered as a nonlinear algebraic equation for the numeric parameter a. It can be solved using an iterative method, for example, the method of simple iteration defined by the formula

ak +1 = ak − θ k F ( ak ), k = 0,1,..., 95

(4.10)

where θ k is an iteration parameter. The algorithm procedure is organized as follows. First, we specify the initial approximations of the control and the parameter a. Then we solve the Cauchy problem (4.3), (4.6), (4.8) to find the functions x and p and, consequently, the function F. The next approximation of the control is determined from the maximum condition (4.7) for the obtained values of x and p. The new approximation of the parameter a is determined from formula (4.10). The calculations are continued until the desired accuracy is achieved. Conclusion 4.2. An algorithm based on the shooting method can be used for solving the system of optimality conditions in an optimal control problem with fixed final state. Remark 4.6. The question arises of whether the described iterative process converges. The same applies to all approximation methods, in particular, for the method of successive approximations in an optimal control problem with free final state. Experience leads us to conclude that the convergence of iterative procedures for the problems with fixed final state is substantially worse than for the problems with free final state. Remark 4.7. Instead of (4.8), we could specify the value of p at the final instance of time, which is more natural for the adjoint system state. In this case, both the original and the adjoint problem are solved successively rather than in parallel at every step of the iterative process. Equation (4.9) and formula (4.10) remain unchanged.

4.4. THE OPTIMALITY CONDITIONS FOR PROBLEM 4

We now derive the necessary optimally conditions for the example in question. Set

H = pu − 4 1 + u 2 . From the maximum principle (Theorem 6), it follows that the optimal control satisfies the equality

(

)

pu − 4 1 + u 2 = max pw − 4 1 + w 2 , w

where p is a solution of the adjoint equation

96

(4.11)

р& = −

∂H ∂x

= 0.

(4.12)

As a result, we have the system of optimality conditions (4.1), (4.2), (4.11), (4.12) for the optimal control. We now determine the control from (4.11). First of all, equating the derivative of H to zero, we obtain

∂H ∂u

= p−

u 2(1 + u 2 )3 / 4

= 0.

(4.13)

From equation (4.12) it follows that p is constant. Then (4.13) may be written in the form

u

(

1+ u2

)

3/ 4

= с,

(4.14)

where с is a constant. This is a nonlinear algebraic equation since its left-hand side is a function of the desired control. Only a constant (not necessary unique) can be a solution of this equation because there is no dependence on time in problem (4.14). Denoting the corresponding constant value of the control by c1, we determine the function x (t ) = c1t from condition (4.1). Setting t = 1 , we have c1 = 1 . Thus, the unique solution of the optimality conditions is the constant

function u0 = 1 . Conclusion 4.3. There exits a unique constant control u0 the derivative of the function H vanishes.

We have obtained the value of u0 from the condition that the derivative of the function H vanishes. However, it is not clear whether this control provides the maximum for the function H. Let us find the second derivative

∂2H ∂ u2

=−

d ⎡

⎤ u 2 − u2 ⎢ ⎥ = − . 3/ 4 2 7/4 du ⎢ 2 ( 1 + u 2 ) ⎥ 4 1 + u ( ) ⎣ ⎦

Since the second derivative is negative for u = u0 , we indeed have the maximum point. Thus, the desired triple of functions is 97

u0 (t ) = 1, x0 (t ) = t , p0 (t ) = 2−7 / 4. The value of p0 is determined from (4.13) for u = u0 . Conclusion 4.4. The system of optimality conditions (4.1), (4.2), (4.8), (4.9) has a unique solution u0, x0, p0. Remark 4.8. Generally speaking, we have some reasons to be doubtful about the foregoing statement, which could put at risk our further arguments. Nevertheless, we will proceed with our analysis until we finally get the answers.

Our purpose now is to find out whether the obtained control is indeed a solution to the problem under consideration, although it might seem obvious that the unique solution of the maximum principle is optimal. 4.5. DIRECT INVESTIGATION OF PROBLEM 4

We now turn to direct investigation of Problem 4. The integrand term of the functional being minimized is obviously not less than unity. Thus, we have the estimate from below for the value of the optimality criterion. We specify the following sequence of controls (see Figure 19):

⎧ 0, 0 < t < k −1 ⎪ k u k (t ) = ⎨ , k = 1, 2,... . k −1 ⎪ k, ≤ t 1 , the sequence {uk} is not bounded and does not converge to any limit (not to mention the nonexistent optimal control). Remark 4.9. The Banach-Alaoglu compactness criterion is applicable not only in Hilbert spaces but also in reflexive Banach spaces such as Lp(0,1) for p>1. The sequence {uk} is bounded in the space L1(0,1), however, in this particular space this does give us any information about the possible convergence of the sequence {uk}.

All previous examples dealt with bounded sequences, which was due to the fact that the set of admissible controls was bounded. In Problem 4, the situation is different. This leads us to suppose that the reason for the nonexistence of an optimal control is that the set of admissible controls is not bounded. However, this is not the only reason. In particular, a function of one variable (for example, the parabola) can achieve its minimum (zero) on the set of real numbers, which is an unbounded domain. However, it is not clear whether it is possible to prove that the optimization problem is solvable if the set of admissible controls is not bounded. This will be the subject of our further analysis. 4.7. PROBLEMS WITH UNBOUNDED SET OF ADMISSIBLE CONTROLS

We now return to the general theorem on the existence of an optimal control and try to prove the same assertion without the assumption that the set of admissible controls is bounded. Note that this property was only used to prove that the minimizing sequence is bounded. Thus, our goal will be

101

achieved if we manage to prove a similar assertion without using the set of admissible controls. We will impose certain additional restrictions on the functional to be minimized. Let the assumptions of Theorem 4 hold, except for the assumption that the set of admissible controls is bounded. Namely, it is required to minimize a convex lower semicontinuous functional I bounded from below on a closed convex (not necessarily bounded) subset U of a Hilbert space V. In addition, we assume that I is coercive. This means that I (u k ) → ∞ for every sequence {u k } such that uk → ∞ . As before, there exists a minimizing sequence, i.e., a sequence of uk ∈ U such that I (u k ) → inf I (u ) . Suppose that this sequence is not bounded, which implies that uk → ∞ . Since I is coercive, it follows that I (u k ) → ∞ . At the same time, the functional on the minimizing sequence

must converge to the corresponding lower bounds rather than infinity. Therefore, our assumption that {uk} is not bounded is wrong. Having proved that the minimizing sequence is bounded, we repeat the arguments of the proof of Theorem 4 to establish the existence of an optimal control. Conclusion 4.10. The existence of an optimal control may be established without the assumption that the set of admissible controls is bounded. Remark 4.10. If the functional is coercive, we can prove that the minimizing sequence is bounded. However, this does not imply that the norm of every admissible control must be bounded. Since the set of admissible controls is not bounded, this is not possible.

The obtained results can be summarized in the following theorem. Theorem 7. The problem of minimizing a convex lower semicontinuous coercive functional bounded from below on a convex closed subset of a Hilbert space is solvable.

To demonstrate the effectiveness of this statement; we consider the following example. Problem 4". Find a control u = u (t ) that minimizes the functional 1

(

)

I = ∫ u 2 + x 2 dt 0

102

in the space of square-integrable functions, where x is a solution of the problem

x& = u , t ∈ (0,1); х(0) = 0. Remark 4.11. The only difference between this problem and Problem 0 is that Problem 4" has no restrictions imposed on the control.

Since the set of admissible functions coincides with the entire space L2(0,1), Theorem 4 cannot be used to establish the existence of an optimal control. We now prove that the functional I is coercive. Indeed, let {uk} be an unbounded sequence, i.e., uk → ∞ . We have 1

(

)

I (uk ) = ∫ uk2 + xk2 dt = uk

2

+ xk

2

,

0

where xk is the system state corresponding to the control uk. Then I (u k ) → ∞ , i.e., the functional I is coercive. Taking into account that all other assumptions of Theorem 7 hold, we conclude that Problem 4" is solvable. Conclusion 4.11. Problem 4" is solvable. Remark 4.12. As we know, Problem 0 has a solution identically equal to zero. The zero control is also a solution to Problem 4" since it is the only control that provides the zero value of the nonnegative functional I.

Thus, the unboundedness of the set of admissible controls is not the reason for the optimization problem to be unsolvable. The unsolvability of Problem 4 may be caused by some unfavourable properties of the set of admissible controls or the optimality criterion. Solving the Cauchy problem (4.1) and using condition (4.2), we can represent the set of admissible controls in the form

⎧

1

⎫

U = ⎨u ∈ L2 (0,1) ∫ u (t ) dt = 1⎬ .

⎩

0

103

⎭

g g2 g(v) g g1 g[(u1+u2)/2] > [g(u1)+ g(u2)]/2 u v

u1

u2

Figure 21. The function g is not convex

For any two elements u and v of the set U and any number a ∈ [0,1] , we have 1

1

1

0

0

0

∫ [ au (t ) + (1 − a)v(t )] dt = a ∫ u (t )dt + (1 − a) ∫ v(t )dt = a + (1 − a) = 1. It follows that the control au + (1 − a )v belongs to U and therefore U is convex. At the same time, it is easy to see that the functional being minimized is not convex. In particular, the integrand function g in the optimality criterion defined by the formula

g (u ) = 4 1 + u 2 is not convex (see Figure 21). The functional is not coercive either. The minimizing sequence defined above is not bounded, although the values of the functional at the elements of this sequence converge to the lower bound of the functional rather than infinity. Conclusion 4.12. Problem 4 is unsolvable because the functional to be minimized is neither convex nor coercive. Remark 4.13. In Example 7, we will establish the existence of an optimal control in the case where the set of admissible controls is not only unbounded, but also nonconvex.

104

4.8. THE CANTOR FUNCTION

Having analyzed the unsolvability of the extremum problem, we now take a step in a different direction to present a remarkable function directly related to the subject of our analysis. First, the time interval [0,1] is divided into three equal parts. We put y(t)=1/2 in the second subinterval (1/3, 2/3). At the second step, we divide each of the other two subintervals into three equal parts and put

⎧1/ 4, t ∈ (1/ 9, 2 / 9), y (t ) = ⎨ ⎩3 / 4, t ∈ (7 / 9, 8 / 9). At the third step, each of the remaining four parts is divided into three equal parts and the values of y on the middle subintervals are set to be 1/8, 3/8, 5/8, and 7/8, respectively. Proceeding with this process indefinitely, we thus define the function y(t) on a subset S of the interval [0,1]. We then extend y to the complement set C= [0,1]\S, which is called the Cantor set, so that the resulting function is continuous. The extension is possible since the variation of y is sufficiently small in any neighborhood of any point of the Cantor set. The resulting function is called the Cantor function (see Figure 22).

3/4

1/2 S 1/4

1/9 2/9

1/3

2/3

7/9 8/9

1

Figure 22. The Cantor function

The Cantor function possesses remarkable properties. It is continuous and monotonic by construction. It is differentiable at every point of S, the derivative being equal to zero. We now determine the measure of S. Summing up the lengths of the intervals on which the Cantor function was defined at each construction step, we have 105

mes ( S ) =

1 3

+

2 9

+

k 1⎛ ∞ ⎛ 2⎞ ⎞ + ... = ⎜ ∑ ⎜ ⎟ ⎟ = 1. 27 3 ⎝ k =0 ⎝ 3 ⎠ ⎠

4

Thus, the measure of the subset S of [0,1] is equal to the measure of [0,1]. Consequently, the complement set C is a set of zero measure. Thus, the derivative of the Cantor function is equal to zero almost everywhere in [0,1]. Note that the values of the Cantor function at the ends of the interval [0, 1] are the same as those specified for the system state function in the formulation of Problem 4. Let w be a control equal to zero on S, i.e., vanishing almost everywhere in the given domain. We can consider to be defined on the whole interval [0,1] because measurable functions are defined up to a set of zero measure. Then the Cantor function is a solution of the Cauchy problem (4.1) corresponding to this control. The control w seems to be admissible because it transfers the system from one state into another in the required time. Taking into account that the derivative of the Cantor function is equal to zero almost everywhere, we have w = 0 almost everywhere in [0,1]. The value of the optimally criterion for this control is equal to unity, which is the lower bound of the functional to be minimized on the set of admissible controls, We seem to have found an admissible control such that the value of the functional at this control is equal to its lower bound. Thus, the control w seems to be optimal. Now, since we proved that the optimization problem in question is unsolvable, we have to find a contradiction in our latest arguments. Consider the boundary value problem

y& = w, t ∈ (0,1), y (0) = 0, y (1) = 1. Integrating the equation over the given time interval and using the boundary conditions, we obtain 1

∫ 0

1

∫

w(t )dt = y& (t )dt = y (1) − y (0) = 1. 0

The integral in the left-hand side of this equality can be written as the sum of the integrals of w over S and C. Since w vanishes on S, it follows that

∫ w(t )dt = 1.

C

106

Thus, the integral of the control w(t) over the Cantor set C is equal to unity, which is a contradiction because C is a set of zero measure. Hence, w is not integrable and therefore does not belong to the set U of admissible controls. Conclusion 4.13. The function w, which transfers the system to a given final state in the specified time, is not an admissible control because of its functional properties. Remark 4.14. The derivative of the Cantor function (and, consequently, the corresponding control) is meaningful only in the context of the theory of

generalized functions. The above analysis shows that the optimal control problem in question could be solvable if the admissible controls were defined as objects of a more general type rather than the usual integrable functions. Conclusion 4.14. Unsolvable extremum problems may become solvable for more general classes of admissible controls.

Summing up, the optimal control problem under consideration is unsolvable in the traditional setting, but the obtained contradiction can be overcome in a more general context. However, the analysis of this problem is far from being finished yet. 4.9. FURTHER ANALYSIS OF THE MAXIMUM CONDITION

We now return to the analysis of the maximum principle in Problem 4. We estimate the remainder term in the formula for the functional increment T

η = − ∫ (η1 + η 2 )dt. 0

Since the state equation and the optimality criterion do not contain nonlinear terms with respect to the state function, we conclude that η1 = 0 . Besides, there are no terms containing both the control and the system state at the same time, which means that η 2 = 0 . Conclusion 4.15. The remainder term in the formula for the functional increment in Problem 4 is equal to zero.

Taking into account Theorem 2, this conclusion seems to imply that the maximum principle is a necessary and sufficient optimality condition for the problem in question. Moreover, since the remainder term vanishes, it 107

follows that there is no information loss in the process of deriving the maximum principle. Thus, the maximum principle certainly does not hold for nonoptimal controls. At the same time, we know that the solution u0 is not optimal. How could we come to a wrong conclusion in the situation where all the arguments in the derivation of the maximum principle are reversible? The only answer is that the maximum principle might have no meaning in this case, which follows directly from the nonexistence of optimal controls. The truth is that we obtained the maximum principle under the assumption that there exists an optimal control, but it turned out that the problem is actually unsolvable. Therefore, all further arguments were groundless. While this may seem to explain the contradiction, the question arises of whether the maximum principle has any meaning under the condition that there is no optimal control and the remainder term in the formula of the functional increment vanishes. Let a function u satisfy the maximum principle (4.11) for Problem 4:

H (u, p) ≥ H (v, p ) ∀v ∈ U , where

H = pu − 4 1 + u 2 and U is the set of controls transferring the system (4.11) to a given final state. Integrating the foregoing inequality, we have 1

∫ [ H (u, p) − H (v, p)] dt ≥ 0 ∀v ∈ U . 0

Denoting by Δx the difference between the system states at the controls v and u and taking into account the adjoint equation (4.12), we obtain 1

∫ {[ H (u, p) − H (v, p)] + Δxp& }dt ≥ 0 ∀v ∈ U . 0

Integrating by parts and using (4.1) and (4.2), we get 1

1

1

0

0

0

& = Δx(1) p (1) − Δx (0) p (0) − ∫ Δxpdt & = ∫ (u − v) pdt. ∫ Δxpdt 108

By the definition of the function H, 1

∫ 0

1

4

1 + u dt ≤ ∫ 4 1 + v 2 dt , ∀v ∈ U . 2

0

Hence, the value of I at the control u is less than its value at any other control transferring the system to the given final state. Consequently, the control u is optimal. Conclusion 4.16. Every solution of the maximum principle for Problem 4 is an optimal control, i.e., the optimality conditions are necessary and sufficient. Remark 4.15. The above conclusion obviously follows from the fact that the remainder term in the formula of the functional increment is equal.

Conclusion 4.17. The existence of an optimal control is not required for the proof of sufficiency of the optimality conditions.

Thus, on one hand, a solution of the maximum principle in Problem 4 must be an optimal control; on the other hand, the solution of the maximum principle obtained above is nonoptimal. This contradiction leads us to revise some premature conclusions made in the process of analysis of the optimality conditions. 4.10. CONCLUSION OF THE PROBLEM ANALYSIS

We now return to considering the maximum condition (4.11):

(

)

pu − 4 1 + u 2 = max pw − 4 1 + w 2 . w

We start with equality (4.13):

p−

u

(

2 1 + u2

)

3/ 4

= 0,

where p is constant, as we already know. Note that u = 0 and p = 0 satisfying the foregoing equality obviously do not satisfy the optimality conditions since the zero control corresponds to the zero system state, which contradicts condition (4.2). Then equality (4.13) can be written in the form 109

cu = z (u ),

(4.15)

where

c = 1/(2 p ), z (u ) = (1 + u 2 )3 / 4 . Depending on the constant c, the algebraic equation (4.15) may be unsolvable or have one or two solutions (see Figure 23).

c1u

-c1u -c2u

c2u c0u

-c0u

u -u2

-u0

-u1

u1

u0

u2

Figure 23. Equation (4.15) may be unsolvable or have one or two solutions.

The constant c is defined by the (constant) solution of the adjoint system and must be such that the corresponding control satisfies condition (4.2). If the absolute value of c is sufficiently small ( c = c0 or c = −c0 in Figure 23), then equation (4.15) and the system of optimality conditions are unsolvable. For two values of c ( c1 < 0 and − c1 < 0 ), equation (4.15) has a unique solution. If c = c1 , we have a constant control u0 with the corresponding solution

x(t) = u0t of problem (4.1), so that u (1) = u0 . Then (4.2) holds only for u0 = 1 . We thus obtained the control u0, which was determined earlier. If c = − c1 , then the corresponding control is negative and therefore x is a decreasing function and (4.2) fails. We now consider the case where the absolute value of c is large enough for equation (4.15) to have two solutions. If we take one of the obtained constants for the control, then (4.2) will either yield nothing (if the control

110

is negative), or again give us u0 (if the control is positive). It may seem that the control u0 determined above corresponds to the unique solution of the optimality conditions, but there is one more possibility that could not be foreseen. The control might be piecewise-constant. It may assume two values u1 and u2 corresponding to two solutions of equation (4.15) and, consequently, to one solution of the adjoint equation. As follows from Figure 23, both values must have the same sign. If the control is negative, then the function x decreases and therefore does not satisfy condition (4.2). For this reason, we only consider the case of positive u1 and u2 . Assume that there exists a point ξ ∈ (0,1) such that

⎧u1 , t < ξ , u (t ) = ⎨ ⎩u2 , t > ξ .

(4.16)

The corresponding solution to problem (4.1) has the form

⎧

t < ξ,

u1t ,

u (t ) = ⎨

⎩(u1 − u2 )ξ + u2t , t > ξ .

(4.17)

In order for the condition (4.2) to hold, we set

x(1) = (u1 − u2 )ξ + u2 = 1. Hence,

ξ=

u2 − 1 u2 − u1

.

This value must belong to (0,1), which implies u1 < 1 < u2 for u1 < u2 and

u2 < 1 < u1 for u2 < u1 . Suppose that u1 is the minimum of the values u1 and u2. Then the ex-

istence of a unique point ξ such that the corresponding control determined from (4.16) satisfies condition (4.2) is possible if and only if u1 < 1 < u2 . As shown in Figure 23, this inequality does hold because u0 = 1 . Hence, the triple of functions consisting of the control, the system state determined from (4.16) and (4.17) for the specified value of ξ , and the corresponding 111

function p is a solution of the system (4.1), (4.2), (4.12), (4.13) (see Figure 24). Note that formula (4.16) requires that the values of u1 and u2 be known. These values are uniquely determined from equation (4.15) for a given value of the parameter c. Evidently, for every c > c1 there is a unique pair (u1,u2) satisfying the requirements. The constant c1 can be determined from (4.15) for u = u0 = 1. As a result, we have c1 = 2

3/ 4

. For every c > 23/4,

there is a unique pair of positive numbers u1 and u2 that define the solution of the problem (4.1), (4.2), (4.12), (4.13) given by formulas (4.16) and (4.17) (see Figure 24). u2

u

u u2

1

1 u1

u1 t x

ξ

t

1

x

1

ξ

1

1

t

ξ

t

ξ

1

1

Figure 24. The solution of the system (4.1),(4.2), (4.12), (4.13) is not unique

Conclusion 4.18. The system (4.1), (4.2), (4.12), (4.13) has infinitely many solutions, the set of solutions being uncountably infinite.

Note that under the assumption that the control assumes only two values u1 and u2 and has two points of discontinuity, we obtain new solutions of the system (4.1), (4.2), (4.12), (4.13). It is important that for any constant 112

c > c1 we can find a unique pair (u1, u2) and, consequently, a new solution of the system. Moreover, having one solution, we can shift the points of discontinuity ξ1 and ξ 2 so that the shifted points belong to the interval (0,1) and the distance between them remains the same. As a result, we would get a new control with two points of discontinuity that also satisfies the system (see Figure 25). The case of three and more points of discontinuity is also possible. u

u

u2

u2

1

1

u1

u1

t

t x

ξ1

ξ2

ξ1

1

ξ2

1

ξ2

1

x

1

1

t

ξ1

ξ2

t

ξ1

1

Figure 25. Solutions of the system (4.1), (4.2), (4.12), (4.15) where the control has two points of discontinuity

Conclusion 4.19. The conclusion on the number of the solutions of the system of optimality conditions for Problem 4 needs to be revised.

So far we have been analyzing the system (4.1), (4.2), (4.12), (4.13). However, it is not obvious that the obtained solutions of this system satisfy the maximum principle (4.11). This means that we must return to analyzing the function

H = pu − 4 1 + u 2 .

113

The function H obviously tends to infinity as u increases. Therefore, H has no global maximum. The solutions of the equation ∂H / ∂u = 0 may only be the points of local extrema or the points of inflection. In particular, the control u0 corresponds to a local maximum of H (since the second derivative of H is negative). Conclusion 4.20. The control u0 provides only a local maximum to the function H.

The most important conclusion follows. Conclusion 4.21. The maximum condition for Problem 4 is unsolvable.

Based on the analysis that has been carried out, we conclude that the optimality conditions in the form of the maximum principle in Problem 4 are unsolvable. Therefore, it turns out that the present example is similar to the previous one, although we have spent much effort to establish this fact. We finally arrive at the following conclusion. Conclusion 4.22. The maximum principle for Problem 4 is a necessary and sufficient optimality condition.

It is not surprising anymore that the value of the optimality criterion at the control u0 turned out to be greater than its lower bound because this control is not a solution of the maximum principle. All the contradictions in our arguments have now been resolved, which completes the analysis of the present example. SUMMARY

The analysis of the present example can be summarized as follows. 1.

The maximum principle is meaningful for the problems with fixed final state.

2.

In the problems with fixed final state, the problem for the system state is overdetermined, whereas the problem for the adjoint system is underdetermined.

3.

An iterative algorithm based on the shooting method can be applied to the corresponding system of optimality conditions.

4.

If the functional to be minimized is coercive, the existence of an optimal control can be established without the assumption that the set of admissible controls is bounded. 114

5.

The existence of an optimal control is not required to prove that the optimality conditions are sufficient.

6.

In the case where there is no optimal control, the maximum principle may be solvable. This means that the optimality conditions are insufficient.

7.

A control may be inadmissible if it does not belong to a specified function class.

8.

Unsolvable extremum problems may be solvable for more general classes of admissible controls.

9.

If controls provide only a local maximum of the function H, then it does not satisfy the maximum principle and therefore is not optimal.

Example 5. Ill-posedness in the sense of Tikhonov So far we have been dealing with extremum problems with quite unfavourable properties. In particular, there were cases where optimal controls did not exist or were nonunique. In contrast to this, the main subject of the present example is an optimal control problem that has a unique solution. In some of the previous examples, the optimality conditions were not sufficient. In the present example, the optimality conditions in the form of the maximum principle are both necessary and sufficient. Another problem with the optimality conditions was that they were either unsolvable or had too many solutions. In the following example, there is a unique optimal control that satisfies the maximum condition. It may seem that no unexpected difficulties can be encountered. Nevertheless, surprises are not over yet. We will show that in the present example it is possible to construct a sequence of admissible controls such that the values of the functional at the elements of this sequence converge to its minimum value, although the sequence itself does not converge to an optimal control. This means that finding a solution of the extremum problem with the desired accuracy is not guaranteed even under such favourable conditions. The described situation is characteristic for optimization problems which are not well-posed in the sense of Tikhonov. In what follows, we present sufficient conditions for the problem to be well-posed. We also describe the regularization methods for ill-posed optimal control problems that allow us to overcome various difficulties in some situations.

116

5.1. PROBLEM FORMULATION

Let the state of the system be described by the Cauchy problem

x& = u , t ∈ (0,1); х(0) = 0.

(5.1)

The control u = u (t ) is assumed to belong to the set

U = {u ∈ L2 (0,1) u (t ) ≤ 1, t ∈ (0,1)} . The optimality criterion is represented by the integral functional

I=

1

1

∫ х dt. 2 2

0

Problem 5. Find a control u ∈ U that minimizes the functional I on U.

Although this problem was already considered when we were studying singular controls (see Problem 2"), we will show that not all of its issues have been clarified. 5.2. SOLUTION OF THE PROBLEM

The solvability of Problem 5 can be easily established using Theorem 5. Indeed, we already established all the necessary properties of the functional to be minimized and the set of admissible controls during the analysis of similar problems. Taking into account the strict convexity of the functional, which can be proved the same way as in Problem 0 (see Example 1). We can also establish the uniqueness of the optimal control using Theorem 2. Besides, similar results can be obtained by analyzing the optimality conditions. We showed in Example 2 that the maximum principle for Problem 5 is defined by the formula

рu = max ( рw), | w| ≤1

(5.2)

where p is a solution of the problem

p& = х, t ∈ (0,1); p (1) = 0.

(5.3)

As we know, the system of optimality conditions (5.1) – (5.3) does not have nonsingular solutions. The maximum principle can only hold if it is degenerate, which corresponds to the case p = 0 . From (5.3) it follows that 117

x = 0 . Finally, using (5.1), we find the unique solution of the maximum principle – the singular control u0 = 0.

Conclusion 5.1. The maximum principle for Problem 5 has a unique solution – the singular control u0 .

At the same time, the functional to be minimized is nonnegative. It vanishes only for x = 0 , i.e., at the control u0. We thus obtain very positive results. Conclusion 5.2. Problem 5 has a unique solution u0. Conclusion 5.3. The maximum principle for Problem 5 is a necessary and sufficient optimality condition. Remark 5.1. The sufficiency of the maximum principle can be established directly by applying Theorem 3, as was done in Example 1.

It may seem that there is no reason to return to such a simple problem that has been analyzed sufficiently well. However, we will show that this problem has more surprises than we expect. Remark 5.2. The existence of a singular control is already an important sign of upcoming difficulties.

5.3. ILL-POSEDNESS IN THE SENSE OF TIKHONOV

Consider the sequence of controls defined by the following formulas (see Figure 26):

uk (t ) = sin π kt , k = 1, 2,... .

(5.4)

These functions are infinitely differentiable and are not greater than unity in absolute value. Hence, these controls are admissible. u1

u2

u3

t

t

Figure 26. The minimizing sequence in Problem 5.

118

t

The corresponding solutions of problem (5.1) are defined as follows (see Figure 27): t

xk (t ) = ∫ sinπ kτ dτ =

1 − cos π kt

πk

0

х2

х1

(5.5)

.

х3

t

t

t

Figure 27. The sequence of states defined by formula (5.5)

The following estimate holds:

0 ≤ xk (t ) ≤ 2 / π k , t ∈ (0,1), k = 1, 2,... . Hence, t

0 ≤ ∫ xk2 dt ≤ 0

4 (π k ) 2

, k = 1, 2,... .

Then the sequence of functional corresponding to the controls uk tends to zero, i.e., the minimum value of the optimality criterion on the set of admissible controls. Conclusion 5.4. The sequence {uk} in Problem 5 is minimizing.

Under these conditions, it may seem natural to take a function uk for sufficiently large k as an approximation of the optimal control. The question arises of whether the minimizing sequence {uk} converges to the optimal control u0. If this is true, the norm of the difference ( uk − u0 ) must tend to zero. We now estimate this norm:

u k − u0

2

1

1

= ∫ uk (t ) − u0 (t ) dt = ∫ sin 2π ktdt = 2

0

0

1 2

.

Thus, the optimal control u0 is not the limit of the minimizing sequence {uk}. 119

Conclusion 5.5. The minimizing sequence {uk} does not converge to the optimal control. Remark 5.3. The above conclusion is rather obvious. It suffices to compare the elements uk (sinusoids with indefinitely increasing oscillation frequency) with the optimal control identically equal to zero. See Figure 26.

The sequence {uk} does not converge in any conventional functional space. This is not surprising since we already encountered the same problem with nonconvergent minimizing sequence in two previous examples. However, there was no optimal control in those examples, i.e., there was no possible limit for the minimizing sequence. Since the existence of an optimal control in the present example is obvious, we arrive at the following unfortunate conclusion. Conclusion 5.6. Not every minimizing sequence converges to the optimal control in the present example.

Summing up these results, we can subdivide all optimal control problems into two classes. The problem of optimal control is called well-posed in the sense of Tikhonov if every minimizing sequence for this problem converges to the optimal control. If there exits a minimizing sequence that does not converge to the optimal control, then the problem is called illposed in the sense of Tikhonov. Conclusion 5.7. Problem 5 is ill-posed in the sense of Tikhonov.

Thus, even though there is a unique optimal control and the optimality conditions are necessary and sufficient, easy solution is not guaranteed. In ill-posed problems, even if we are able to find a control at which the value of the functional being minimized is as close to its lower bound as desired, this control is not guaranteed to be close enough to the optimal control. Remark 5.4. The minimizing sequence defined above weakly converges to the optimal control. For this reason, the problem could be called weakly well-posed in the sense of Tikhonov. However, this positive result is not really meaningful because the elements of the minimizing sequence (frequently oscillating functions) are by no means close to the optimal control, which is constant. Remark 5.5. Note that the weak convergence of the minimizing sequence is of no surprise. The Banach – Alaoglu theorem is still applicable here since the set of admissible controls is bounded.

We now try to explain why the problem in question is not well-posed in the sense of Tikhonov. Apparently, in problems of this kind, the functional is not very sensitive to the variations of the control. Under this condition, a substantial variation of the control has only a weak effect on the optimality 120

criterion. Therefore, the value of the functional at a control considerably different from the optimal one may be relatively close to its minimum value. Conclusion 5.8. The problem is ill-posed in the sense of Tikhonov because the optimality criterion is not sensitive enough to the variation of the control.

The question arises of whether the fact that the problem is not wellposed in the sense of Tikhonov is a really serious obstacle. In applied problems, we always seek approximate solutions. Therefore, it may be reasonable to always seek the approximate solution in the form of an admissible control at which the value of the functional is sufficiently close to its lower bound. With such an approach, the fact that a certain problem is not well-posed seems to be of no concern. Suppose that we have a situation where the present example has physical meaning. As we know, the exact optimal control is the function identically equal to zero, which is very simple and has no problem being represented in practice. At the same time, when we seek an approximate solution, it turns out to be a sinusoid with high frequency of oscillations. Although the corresponding value of the functional is sufficiently small, the obtained control is not satisfactory as far as practical application is concerned. For this reason, determining optimal controls with the required accuracy is a more preferable way, though finding the approximate minimum of the functional may also be satisfactory enough in some cases. Remark 5.6. If a problem is ill-posed in the sense of Tikhonov, numerical algorithms are usually very sensitive to different kinds of errors. This brings up the notion of ill-posedness in the sense of Hadamard, which is the subject of the next example.

The question arises of whether a minimizing sequence that does not converge to the optimal control may converge to any other limit. If the functional is continuous, the convergence of a sequence of controls obviously implies the convergence of the corresponding sequence of the functional values. So if the sequence of controls converges to a limit which is not an optimal control, then the corresponding sequence of the functional values will converge to the value of the functional at this limit. In this case, however, the sequence is not minimizing since the values of the functional at this sequence do not converge to its lower bound. Conclusion 5.9. The minimizing sequence either converges to the optimal control or does not converge at all.

121

It is interesting to know how common is the case where the minimizing sequence in an ill-posed problem does not converge. In the present example, it is easy to see every sequence of admissible controls that weakly converges to zero is minimizing. Since the class of weakly converging sequences is substantially larger than that of strongly converging sequences, we conclude that minimizing sequences usually do not converge in problems that are ill-posed in the sense of Tikhonov. Conclusion 5.10. Minimizing sequences in ill-posed problems usually do not converge.

In unsolvable optimization problems, minimizing sequences obviously do not converge to optimal controls since they don't exist. Conclusion 5.11. Unsolvable optimization problems are ill-posed in the sense of Tikhonov.

This brings up the question: Is it possible for an optimal control problem with more than one solution to be well-posed in the sense of Tikhonov? Consider the case where the optimal control problem has two different solutions и and v. Let {uk} and {vk} be sequences of admissible controls converging to u and v, respectively. If the functional I to be minimized is continuous, we have

I (uk ) → inf I (U ), I (vk ) → inf I (U ) . Let {wk} be a sequence whose elements with odd indices are the elements of {uk} and those with even indices are the elements of {vk}. We have I ( wk ) → inf I (U ) because the same is true for all subsequences of

{I ( wk )} . Hence, {wk} is a minimizing sequence. At the same time, it does not converge because two of its subsequences converge to different limits. It follows that this problem is ill-posed in the sense of Tikhonov. Conclusion 5.12. Optimization problems with more than one solution are ill-posed in the sense of Tikhonov.

We see that the class of optimal control problems well-posed in the sense of Tikhonov is smaller than the class of problems that have a unique solution. In particular, Problem 5 is uniquely solvable, but it is ill-posed in the sense of Tikhonov. For this reason, in order to prove that a problem is wellposed, the restrictions to be imposed on the system must be stronger than those we had when establishing the existence and uniqueness of an optimal 122

control. In what follows, we find the conditions that guarantee the wellposedness of the optimal control problem in the sense of Tikhonov. 5.4. ANALYSIS OF WELL-POSEDNESS IN THE SENSE OF TIKHONOV

In the proof of the solvability of the extremum problem, we have used the convexity of the functional to be minimized. To provide the uniqueness of the optimal control, we needed the stronger property of strict convexity. To establish that the optimal control problem is well-posed in the sense of Tikhonov, we need an even stronger condition – the strict uniform convexity of the functional. A functional I defined on a convex set U is called strictly uniformly convex if there exists a continuous function δ = δ (τ ) such that

δ (0) = 0, δ (τ ) > 0, τ > 0; δ (τ ) < 0, τ < 0; δ (τ ) → ∞, τ → ∞ and for every two elements u,v ∈ U and α ∈ [0,1] the following inequality elements holds:

I [αu + (1 − α )v] ≤ αI (u ) + (1 − α ) I (v) − α (1 − α )δ (|| u − v ||). Obviously, every strictly uniformly convex functional is strictly convex. Suppose that a function и is a solution of the problem of minimizing a strictly uniformly convex functional I on a convex set U. Consider an arbitrary minimizing sequence, i.e., a sequence of uk ∈ U such that I(uk) → inf I(U). Then

I [αuk + (1 − α )u ] ≤ αI (uk ) + (1 − α ) I (u ) − α (1 − α )δ (|| uk − u ||) and therefore

I [αu k + (1 − α )u ] − I (u ) ≤ α [ I (u k ) − I (u )] − α (1 − α )δ (|| u k − u ||). Since the control и is optimal, the left-hand side of the foregoing inequality is nonnegative. As a result, we have

(1 − α )δ (|| uk − u ||) ≤ I (uk ) − I (u ). We now pass to the limit as α→0 since the parameter α is arbitrary. This yields the inequality 123

0 ≤ δ (|| uk − u ||) ≤ I (uk ) − I (u ). Since {uk} is a minimizing sequence, we have

δ (|| uk − u ||) → 0. Assume that the sequence of positive numbers {τk}, where τ k = u k − u , does not tend to zero. This means that it either does not converge or tends to a positive number. From the properties of the function δ, it follows that {δ(τk)} either converges to a nonzero limit or does not converge at all. In any case, it does not converge to zero, which leads to a contradiction. We have thus proved that uk→u. Therefore, every minimizing sequence converges to the optimal control, which means that this extremum problem is well-posed in the sense of Tikhonov. Conclusion 5.13. The strict uniform convexity of the functional is required to prove that the optimization problem is well-posed in the sense of Tikhonov.

Taking into account Theorems 5 and 7 on the existence of a solution of the extremum problem, we arrive at the following conclusion. Theorem 8. The problem of minimizing a lower semicontinuous functional which is bounded from below and is strictly uniformly convex on a convex closed bounded subset of a Hilbert space is well-posed in the sense of Tikhonov. (The condition of boundedness for the subset can be replaced with the coerciveness of the functional.)

This theorem will be used below to prove that the problem considered in Introduction is well-posed in the sense of Tikhonov. 5.5. THE WELL-POSED OPTIMIZATION PROBLEM

Let the set

U = {u ∈ L2 (0, T ) u (t ) ≤ 1, t ∈ (0, T )} . be the domain of definition for the functional

I=

1

∫ (u 2

1

2

)

+ x 2 dt.

0

The system state x is described by the formulas 124

x& = u, t ∈ (0,1); х(0) = 0. . Problem 5'. Find a control u ∈U minimizing the functional I on U.

We are dealing with Problem 0, for which the existence and uniqueness of an optimal control was established earlier. All the assumptions of Theorem 8, except for the property of strict uniform convexity of the functional, were proved to hold. In order to establish that the functional is strictly uniformly convex, we first consider the quadratic function

f = f ( x) = x 2 . We have

f [α x + (1 − α ) y ] − α f ( x ) − (1 − α ) f ( y ) = [α x + (1 − α ) y ]2 = = α 2 x 2 + 2α (1 − α ) xy + (1 − α ) 2 y 2 − α x 2 − (1 − α ) y 2 for all α ∈ (0,1) and all numbers x, y. Therefore,

f [αx + (1 − α ) y ] = αf ( x) + (1 − α ) f ( y ) − 2α (1 − α )( x − y ) 2 . 2

It follows that f is strictly uniformly convex and δ (τ ) = τ . Setting x = u (t ) and y = v (t ) integrating the foregoing equality with respect to t, we obtain 1

1

1

0

0

2 2 2 ∫ [α u + (1 − α )v] dt = α ∫ u dt + (1 − α ) ∫ v dt − 0

1

−2α (1 − α ) ∫ (u − v ) 2 dt , ∀u , v ∈ U , α ∈ (0,1). 0

Hence, the quadratic functional I defined by the formula 1

J (u ) = ∫ u 2 dt 0

is strictly uniformly convex, i.e.,

J [αu − (1 − α )v] = αJ (u ) + (1 − α ) J (v) − 2α (1 − α ) || u − v ||2 . 125

We already established the convexity of the functional 1

K (u ) = ∫ x (u ) 2 dt , 0

(see Example 1), where x(u) is a solution of the Cauchy problem corresponding to the control u. Taking into account that I is the half sum of the functional J and K, we have

I [α u + (1 − α )v ] ≤ α I (u ) + (1 − α ) I (v ) − 2α (1 − α ) || u − v ||2 . Conclusion 5.14. The functional I is strictly uniformly convex.

By Theorem 8, the problem is well-posed in the sense of Tikhonov. Conclusion 5.15. Problem 5’ is well-posed in the sense of Tikhonov.

In Problem 5, the functional to be minimized is strictly convex, but not strictly uniformly convex. Although this is enough to establish the uniqueness of the optimal control it is not enough for the problem to be wellposed. Conclusion 5.16. Problem 5 is not well-posed in the sense of Tikhonov because the functional to be minimized is not strictly uniformly convex. 5.6. REGULARIZATION OF OPTIMAL CONTROL PROBLEMS

Numeric solution of ill-posed extremum problems may involve certain difficulties. In particular, minimization algorithms do not necessarily guarantee the obtaining of optimal controls with the desired accuracy. Various regularization methods can help to deal with these problems. For example, the Tikhonov regularization method for Problem 5 involves the use of the functional 1

I ε = I + ε ∫ u 2 dt , 0

where ε is the regularization parameter. The problem of minimizing this functional coincides with Problem 0 up to a constant multiplier. As Problem 0 was proved to be well-posed, the solution of the regularized problem should not be difficult. After finding a solution, we pass to the limit as ε → 0 . The initial calculations are made for sufficiently large ε, when the problem has good properties (numerical 126

algorithms for this problem converge sufficiently well), although they are not as good as in Problem 5. The next stage of calculations is performed after ε is reduced, the initial approximation being the control from the previous step of the regularization method. The deterioration of convergence of the solution algorithm as the problem is becoming closer to the original ill-posed problem is partially compensated by the gradual refinement of the initial approximation of the control at every step of the regularization method. It is sometimes possible to solve the ill-posed problem using this algorithm. Remark 5.7. The algorithm described above involves an imbedded iterative process.

SUMMARY

The analysis of the present example can be summarized as follows. 1.

The minimizing consequence can diverge. This is caused by ill-posedness in the sense of Tikhonov.

2.

The class of well-posed optimization problems in the sense of Tikhonov is narrower than the class of optimization problems with unique solution.

3.

The well-posedness of optimization problems can be proved with using if strong uniformly convexity of the minimizing functional.

4.

The ill-posed optimization problems can be solving by means of regularization methods.

5.

The regularization methods can be used also for the finding of the singular controls.

Example 7. Insufficiency of the optimality conditions (Part 2) In this example, we consider an optimal control problem for a system with fixed final state and isoperimetric condition. Although we cannot expect the set of admissible controls to be convex under such conditions, we will prove that the problem is solvable. To obtain the necessary optimality conditions, we use the method of Lagrange multipliers, which allows us to generalize the results outlined in Introduction. The system includes two differential equations for the main state and the adjoint state, a relative extremum problem, and a specific algebraic equation for the Lagrange multiplier. After some transformations, the obtained relations are reduced to a rather simple boundary value problem for a nonlinear differential equation of the second order which possesses some remarkable properties. The analysis of the boundary value problem involves finding a nontrivial solution and a sequence of transformations that take one solution into another. As a result, we will show that the boundary value problem has infinitely many solutions, but most of them are nonoptimal. 7.1. PROBLEM FORMULATION

The system is described by the Cauchy problem

x& = u, t ∈ (0,1); х(0) = 0

(7.1)

The control и = u(t) is chosen from the set U of functions that transform the system into the zero final state, i.e., such that

x(1) = 0, with the additional condition 128

(7.2)

1

∫

x(t )

4

dt = 1.

(7.3)

0

The optimality criterion is defined by the formula

I=

1

1

∫ u dt. 2 2

0

Problem 7. Find a control u ∈ U minimizing the functional I on U.

The essential specific feature of this problem is the presence of condition (7.3) which is called isoperimetric. Remark 7.1. It might seem that the minimum of the functional is equal to zero, being achieved at the unique control identically equal to zero. Indeed, the corresponding system state determined from problem (7.1) is also trivial; therefore, it satisfies the boundary condition (7.2). In this case, however, the isoperimetric condition (7.3) fails. Hence, the trivial control is not admissible and cannot be a solution of Problem 7. Thus, although the functional has such a simple form, this problem is far from being trivial. This is explained by the fact that the structure of the set of admissible controls defined by (7.2) and (7.3) is highly complex.

Because of the specific form of the set of admissible controls, we need improved methods for proving the solvability of the problem and obtaining the optimality conditions. 7.2. THE EXISTENCE OF AN OPTIMAL CONTROL

The optimization problem formulated above essentially differs from the previous ones in the structure of the set of admissible controls. For this reason, it is by no means obvious that the problem is solvable, and we will start our analysis with the proof of solvability. Unfortunately, we will not be able to use the existence theorems obtained above because the set U is obviously nonconvex. Indeed, if a control и is admissible, then -u is admissible as well because changing the sign of the control implies the change of sign of the system state function, so that conditions (7.2) and (7.3) remain satisfied. At the same time, the half-sum of these functions is identically equal to zero and therefore does not satisfy (7.3). Note that (7.3) defines a spherical surface in the space of functions whose fourth power is integrable, which is certainly not convex. Nevertheless, we will see that this obstacle does not prevent us from establishing the existence of an optimal control. 129

We now specify the function spaces that allow us to formulate the problem in a more compact form. The space L4 (0,1) consists of functions x = x (t ) whose fourth power is Lebesgue integrable in the interval (0,1), i.e., 1

∫ x (t )

4

dt < ∞.

0

1

The Sobolev space H 0 (0,1) is a space of functions x = x (t ) vanishing at t = 0 and t = 1 and square-integrable over the interval (0,1), their first (generalized) derivatives also being integrable over (0,1): 1

∫

1

2

x (t ) dt < ∞,

0

∫ x& (t )

2

dt < ∞.

0

The space L4 (0,1) is a reflexive Banach space with the norm 1/ 4

⎛1 ⎞ 4 х 4 = ⎜ ∫ x(t ) dt ⎟ . ⎝0 ⎠ 1

The space H 0 (0,1) is a Hilbert space with the norm 1/ 2

⎛1 ⎞ 2 х = ⎜ ∫ x& (t ) dt ⎟ . ⎝0 ⎠ 1

A sequence {xk} weakly converges to an element x in the space H 0 (0,1) if the following condition holds: 1

1

0

0

1 ∫ x&k (t )λ& (t )dt → ∫ x& (t )λ& (t ) dt ∀λ ∈ H 0 (0,1).

1

{

We put X = H 0 (0,1) , V = x ∈ X

x

4

}

2

= 1 , and I = I ( x ) = x .

Then we obtain the problem of minimizing the functional I on the subset V of the space X. This formulation of Problem 7 is more convenient for our analysis. 130

Since the functional I is nonnegative and therefore bounded from below, it has the lower bound in V. This means that there exists a minimizing sequence {xk} such that

xk ∈ V , k = 1, 2,...; I ( xk ) → inf I (V ).

(7.4)

We now use the coerciveness of the functional to prove that the minimizing sequence is bounded. Assume the contrary. If the sequence {xk} is not bounded, i.e., xk → ∞ as k → ∞ , then the definition of the functional implies that I ( xk ) → ∞ . This contradicts the fact that {xk} is a minimizing sequence. It follows that {xk} is bounded. By the Banach – Alaoglu theorem, there is a subsequence of {xk} (which will be again denoted by {xk} for simplicity) such that xk → x weakly in X. By the Rellich – Kondrashov theorem (from the theory of the Sobolev 1

spaces), if xk → x weakly in H 0 (0,1) , then xk → x strongly in L4(0,1). Remark 7.2. In the foregoing statement, the condition that the domain of definition of all the functions is one-dimensional is essential.

Hence, xk

4

→ x 4 , i.e., 1

∫ 0

1

xk (t ) dt → ∫ x(t ) dt. 4

4

0

Conditions (7.1) yield the equality 1

∫ x (t )

4

k

dt = 1.

0

Passing to the limit in this equality, we obtain 1

∫ x(t )

4

dt = 1.

0

We conclude that x ∈V . Remark 7.3. The convexity of the set of admissible controls was used in the proof of the solvability of the extremum problem for establishing that the limit of the minimizing sequence is admissible. As we now see, the latter can be established

131

without using the convexity of the set of admissible controls, which should allow us to achieve our goals. Remark 7.4. In fact, we have proved that the set V is weakly closed, i.e., it contains the limits of all its weakly converging sequences. In the general theorem of existence of a solution of the extremum problem, the convexity of the set of admissible controls was used to prove that it was weakly closed.

Being equal to the squared norm in a Hilbert space, the functional I is convex and continuous. Therefore, it is weakly lower semicontinuous. Then the condition that xk → x weakly in X implies

I ( x) ≤ inf lim I ( xk ). By (7.4), the right-hand side of this inequality is equal to inf I(V). Hence, I(x) < inf I(V). This relation can only hold as an equality because the value of I at any element of V cannot be less than its lower bound. We conclude that

х∈V , I ( x) = inf I (V ) . This means that x is a point of minimum of the functional I on the set V and therefore the problem in question is solvable. Conclusion 7.1. Problem 7 is solvable. Conclusion 7.2. If the set of admissible controls is nonconvex, this is not an insuperable obstacle in the analysis of the optimal control problem. Remark 7.5. Although we have proved that the problem is solvable, the fact that the set of admissible controls is nonconvex will affect further results. Namely, condition (7.3) is the cause of unfavourable properties of the system of optimality conditions to be obtained below.

7.3. NECESSARY CONDITIONS FOR AN EXTREMUM

As in the proof of the existence of an optimal control, additional difficulties arise when deducing the necessary conditions for an extremum, which is caused by the isoperimetric condition. These difficulties can be overcome by means of the method of Lagrange multipliers. We introduce the auxiliary functional

132

1

⎛1

⎞

⎝0

⎠

L (u , x, p, λ ) = I (u ) + ∫ ( x& − u ) pdt + λ ⎜ ∫ x 4 dt − 1 ⎟ = 0

1

1

0

0

& , = − ∫ Н (u , p ) dt + ∫ xpdt where

H(u,x,p,λ) = up – u2/2 – λ (x4 –1). If the functions и and x satisfy (7.1) – (7.3), the functionals I and L coincide. Suppose that a function и is an optimal control, i.e., the following inequality holds for all admissible controls v:

ΔI = I(v,y) – I(u,x) ≥ 0, where x and у are the solutions of (7.1) for the controls и and v respectively. Since I and L coincide, the foregoing inequality is reduced to the form 1

1

0

0

∫ [ Н (u, х, p, λ ) − Н (v, у, p, λ )] dt + ∫ ( y& − x& ) pdt ≥ 0. Further transformations yield 1

1

∫ [ Н (u, х, p, λ ) − Н (v, х, p, λ )] dt + ∫ (4λ х 0

3

− р& ) Δхdt + η ≥ 0,

0

where η is a higher-order term in Δx = y − x. Having determined the adjoint equation 3 р& = 4λ х , t ∈ (0,1),

we arrive at the inequality 1

∫ [ Н (u, х, p, λ ) − Н (v, х, p, λ )] dt + η ≥ 0. 0

This readily implies the maximum principle: 133

(7.5)

H (u , x, p, λ ) = max H (v, y, p, λ ).

(7.6)

Conclusion 7.3. For the control и to be a solution of Problem 7, it is necessary that it satisfy the maximum condition (7.6). Conclusion 7.4. The optimality conditions in the form of the maximum principle can be established even in the presence of isoperimetric conditions.

Thus, for the three unknown functions и, x, p and the number λ, we have the system consisting of two differential equations of the first order, the boundary conditions (7.1), (7.2), (7.5), the global extremum problem (7.6), and equality (7.3). The number and structure of relations in the system of optimality conditions obviously agrees with the number and structure of the unknown quantities. We may now begin the analysis of the problem. 7.4. TRANSFORMATION OF THE OPTIMALITY CONDITIONS

From the system (7.1) - (7.3), (7.5), (7.6), we obtain the problem for the single unknown function x. Equating the derivative of H to zero yields u = p , which allows us to eliminate the control from the system of optimality conditions. Differentiating the state equation (7.1) and using (7.5), we have

&x& = u& = р& = 4λ х 3 .

(7.7)

Multiplying this expression by x and integrating the result, we obtain 1

1

1

1

& 0 = − ∫ х& 2 dt. 4λ ∫ х dt = ∫ && хxdt = − ∫ х& dt + xx 4

0

2

0

1

0

0

In view of the isoperimetric condition (7.3), the foregoing equality is written as 1

4λ = − ∫ х& 2 dt. 0

Substituting this value into (7.7), we arrive at the equality

134

⎛1 2 ⎞ 3 && x + ⎜ ∫ х& dt ⎟ х = 0. ⎝0 ⎠

(7.8)

Thus, the optimal system state satisfies the integro-differential equation (7.8) with the homogeneous boundary conditions

х(0) = 0, х(1) = 0.

(7.9)

Conclusion 7.5. Necessary optimality conditions in optimization problems with isoperimetric conditions may be reduced to an integro-differential equation. Remark 7.6. If a function x is a nonzero solution of the boundary value problem (7.8), (7.9), then it certainly satisfies the isoperimetric condition (7.3). Indeed, after multiplying both sides of equality (7.8) by x, we first integrate the result over the given interval. Then we integrate by parts taking into account the boundary conditions (7.9), cancel the square of the norm of x in the Sobolev space, and finally obtain condition (7.3) as a result. Thus, we will not have to verify the validity of this condition for nonzero solutions of problem (7.8), (7.9) to be obtained in the sequel.

To simplify the obtained relations, we introduce the function

y(t) = ||x|| x(t), t∈(0,1).

(7.10)

We have 3

&& у = х && х = − х х3 = − у3 . It follows that у satisfies the nonlinear ordinary differential equation

&& у + у 3 = 0, t ∈ (0,1)

(7.11)

with the boundary conditions

у(0) = 0, у(1) = 0.

(7.12)

Suppose that a solution у of the problem (7.11), (7.12) has been found. Then condition (7.10) implies that 2

y = x . As a result, we obtain

x (t ) = x

−1

y (t ) = y 135

−1/ 2

y (t ).

(7.13)

Thus, knowing a solution of the problem (7.11), (7.12), it is possible to solve the Euler equation for the variational problem in question using the formula (7.13). Conclusion 7.6. Problem 7 can be reduced to the boundary value problem (7.11), (7.12). Remark 7.7. After finding a solution of the problem (7.11), (7.12), we can determine the state of the original system using formula (7.13). Differentiating this system state, from equation (7.1) we obtain the corresponding control. 7.5. ANALYSIS OF THE BOUNDARY VALUE PROBLEM

Consider the nonlinear boundary value problem (7.11), (7.12), which is of independent interest. The function identically equal to zero is obviously a solution of this problem. At the same time, if x is an optimal state in Problem 7, the isoperimetric condition implies that x is nontrivial and it satisfies the necessary condition for an extremum (7.8). Then the function у defined by formula (7.10) is also nontrivial and is a solution of the boundary value problem (7.11), (7.12). Conclusion 7.7. The boundary value problem (7.5), (7.6) has more than one solution. In addition to the trivial solution, there exists a nontrivial solution related to the optimal state in Problem 7.

The trivial solution of the extremum condition (7.8) cannot be a solution of the variational problem since it does not satisfy the isoperimetric condition. Therefore, the set of solutions of the necessary condition for an extremum is certainly larger than the set of solutions of the original problem. Conclusion 7.8. The necessary condition for an extremum is not sufficient in the present problem.

The question arises of whether the boundary value problem (7.11), (7.12) has only two solutions. If it were true, the variational problem would have a unique solution. However, if a; is a solution of the variational problem, then the function —ж satisfies the isoperimetric condition, the values of the functional / at x and —ж being the same. Since the solution of the problem is nonzero in view of the isoperimetric condition, we come to the following conclusion.

136

Conclusion 7.9. The solution of the isoperimetric problem is not unique: if x is a solution of the problem, so is -x.

Note that the set of solutions of (7.8) and (7.11) is also invariant with respect to changing the sign of solutions: if a function is a solution of these equations with homogeneous boundary conditions of the first order, then changing its sign yields a new solution. Conclusion 7.10. The boundary value problem (7.11), (7.12) has at least three solutions: one solution is trivial and two others correspond to the solutions of the variational problem. у

у(t) = у(1-t)

t 1/2

0

1

у

0

у(t) = -у(1-t) 1

t

1/2

Figure 28. Possible symmetric solutions of problem (7.11), (7.12)

We do not know yet if these problems have any other solutions, but the way we found the last solution (the second for the variational problem and the third for the boundary value problem) provides the clues. We will try to find other transformations that take one solution of the problem into another. Direct verification shows that if a function у is a solution of the boundary value problem (7.11), (7.12), then so are the functions

z1(t) = y(1 – t) , z2(t) = -y(1 – t) , t∈(0,1). 137

у

у = А1 у

у 3 = А3 у

t

у 4 = А4 у

у 2 = А2 у

Figure 29. Possible nonsymmetric solutions of problem (7.11), (7.12)

There are two alternatives. Let у satisfy one of the conditions of being symmetric with respect to the middle of the time interval (see Figure 28)

у(t) = y(1 – t), t∈(0,1),

(7.15)

у(t) = –y(1– t), t∈(0,1),

(7.16)

Then equalities (7.14) do not yield new solutions. If the solution is not symmetric, then we obtain new solutions of the boundary value problem. If a nontrivial solution of problem (7.11), (7.12) does not satisfy (7.15) or (7.16), then the problem has at least five solutions, four of them being nontrivial solutions of the variational problem. Let Y denote the set of nontrivial solutions of the boundary value problem. As we know, there are three transformations that take elements of this set to its other elements (see Figure 29):

А1 y(t) = –y(t), А2 y(t) = y(1– t), А3 y(t) = –y(1– t), t∈(0,1). We can also mention the identity transformation A0. Since the composition of transformations defined on Y is also a transformation on Y, one would expect that the composition of the above transformations must yield new solutions of the problem. However, it is easy to see that composition does not produce transformations other than those already determined (see Figure 30). It follows that the boundary value problem (7.11), (7.12) has at least three solutions if the nontrivial solutions are symmetric and at least five solutions if the nontrivial solutions are not symmetric. It is possible that other solutions exist.

138

А0

А1

А2

А3

А0

А0

А1

А2

А3

А1

А1

А0

А3

А2

А2

А2

А3

А0

А1

А3

А3

А2

А1

А0

Figure 30. Composition of the transformations of Y

Consider the boundary value problem of the form (7.11), (7.12) with the equation

&& z + z 3 = 0, t ∈ (0, a ).

(7.17)

and the boundary conditions

z(0) = 0, z(а) = 0,

(7.18)

where a is an arbitrary positive number. Suppose that у is a solution of problem (7.11), (7.12). We introduce the function

z = z(t) = a-1 у(t/а), t∈(0,а).

(7.19)

Then

{

&& z (t ) + [ z (t )]3 = а −3 && y (t / a ) + а −3 [ y (t / a)]3 = а −3 && y (t / a ) + [ y (t / a)]

3

} = 0.

Thus, we have obtained a solution of problem (7.17), (7.18). Remark 7.8. For every value of the parameter a, transformation (7.19) produces a new solution of equation (7.11) satisfying the first boundary condition (7.12). The second boundary condition essentially restricts the choice of admissible values of the parameter a.

Let y1 denote a nonzero solution of problem (7.11), (7.12). Then the function

z2 = z2(t) = 2у1(2t), t∈(0,1) is a solution of problem (7.17), (7.18) for a = 1/2. We now introduce the following function (see Figure 31):

139

у3 у2 у1 t

у4

Figure 31. Solutions of problem (7.11), (7.12)

⎧ z2 (t ),

y 2 (t ) = ⎨

0 < t < 1/ 2,

⎩− z2 (1 − t ), 1/2 < t < 1.

The boundary condition (7.12) holds for y2. This function also satisfies equation (7.11) on the interval (0,1/2) because z = z 2 (t ) is a solution of the equation

&z& + z 3 = 0 for 0 < t < 1/ 2. At the same time, z = − z 2 (1 − t ) is a solution of the same equation for 1/ 2 < t < 1. Note that y2 is continuously differentiable at t = 1/ 2 if the same is true for y1. As follows from equation (7.11), the second derivative of y2 is equal to the third power of y2 taken with the opposite sign. Then y2 is twice continuously differentiable at t = 1/ 2 . It follows that y2 is a solution of problem (7.11), (7.12) that differs from all previously found solutions. The function -y2 represents another solution. If y1 does not satisfy the symmetry conditions (7.15) or (7.16), then the

140

transformations А2 and A3 can be applied to find two more solutions of the boundary value problem. Now we have an algorithm of constructing new solutions of our boundary value problem. Obviously, the function

z3 = z3(t) = 3у1(3t), t∈(0,1) is a solution of problem (7.17), (7.18) for a = 1/3. Then the function

0 < t < 1/ 3, ⎧ z3 (t ), ⎪ y3 (t ) = ⎨ − z3 (2 / 3 − t ), 1/3 < t < 2 / 3, ⎪ z (t − 2 / 3), 2/3 < t < 1 ⎩ 3 is also a solution of the boundary value problem (see Figure 31). Remark 7.9. Here we take into account that for any constant с the shift Ax (t ) = x (t − c ) transforms solutions of equation (7.11) (without the boundary conditions) into solutions of the same equation. The transformation Ax (t ) = x ( c − t ) , which is the composition of a shift and the operator A2, has the same property.

In the general case, for all natural numbers ft, the function

zk = zk(t) = kу1(kt), t∈(0,1) is a solution of (7.17), (7.18) for a = 1/k. Repeating the arguments used above, we conclude that the function

zk (t ), 0 < t < 1/ k , ⎧ ⎪− z (2 / kt ), 1/k < t < 2 / k , ⎪⎪ k yk (t ) = ⎨ zk (t − 2 / k ), 2/k < t < 3 / k , ⎪− z (4 / k − t ), 3/k < t < 4 / k , ⎪ k ⎪⎩ . . . . . . . . is also a solution of (7.17), (7.18). One or three more solutions can be obtained by applying the above-mentioned transformations to this function. Remark 7.10. Any value of the parameter a can be chosen for constructing solutions that satisfy (7.11). However, the second boundary condition in (7.12) can only be satisfied for a = 1/k, where k = 1,2,....

Conclusion 7.11. Tie boundary value problem (7.11), (7.12) has infinitely many solutions.

141

Remark 7.11. In some of the previous examples, boundary value problems for nonlinear second-order differential equations had infinitely many solutions. It was not unusual since those equations were rather complex. It is surprising that the same situation is observed for such simple nonlinear equations in the present example. Remark 7.12. The next example deals with a boundary value problem similar to (7.11), (7.12) such that the number of its solutions depends on a parameter in the problem formulation. Remark 7.13. It is not certain that there are no more transformations that produce new solutions of the problem. Remark 7.14. If we require that equation (7.11) hold almost everywhere rather than at every point of the specified interval and allow nonsmooth and even discontinuous solutions, then we can obtain an uncountably infinite set of solutions on the basis of those already found. Indeed, if functions x and у are solutions of our boundary value problem, then the discontinuous function coinciding with x on one segment of the interval (0,1) and with у on the other segment will also be a solution of the problem.

7.6. THE NONLINEAR HEAT CONDUCTION EQUATION WITH INFINITELY MANY EQUILIBRIUM STATES

The results obtained above allow us to establish remarkable properties of a nonlinear parabolic equation related to equation (7.11). Consider the nonlinear heat conduction equation

∂v ∂τ

=

∂ 2v ∂ξ

2

+ v 3 , ξ ∈ (0,1), τ > 0,

(7.20)

with the boundary conditions

v(0, τ ) = 0, v(1, τ ) = 0, τ > 0,

(7.21)

and the initial condition

v (ξ , 0) = v0 (ξ ), ξ ∈ (0,1).

(7.22)

We need to investigate the behaviour of this system for τ → ∞ . If v(ξ ,τ ) → y (ξ ) as τ → ∞ , then the function y, which is called an equilibrium state of the system, is a solution of the differential equation

&& у + у 3 = 0, ξ ∈ (0,1) 142

with the boundary condition

y(0) = 0, y(l) = 0. It follows that the equilibrium state is a solution of the boundary value problem (7.11), (7.12). We arrive at the following conclusion. Conclusion 7.12. The system (7.20) – (7.22) has infinitely many equilibrium states.

Realization of a particular equilibrium state depends on the initial state v0. Remark 7.15. It is again quite surprising that such a simple nonlinear heat conduction equation has infinitely many solutions. Similar situation were observed only in the case of rather complex nonlinearity. Remark 7.16. The system (7.20) – (7.22) may be used for finding non-trivial solutions of the corresponding stationary problem on the basis of the stabilization method.

7.7. CONCLUSION OF THE ANALYSIS OF THE VARIATIONAL PROBLEM

We now return to the analysis of the original variational problem. Using formula (7.13), from the known solution yk of problem (7.11), (7.12), we find the solution

xk ( t ) = y k ( t )

−1/ 2

yk (t ), t ∈ (0,1)

of the integro-differential equation (7.8) with homogeneous boundary conditions. As was mentioned before, every nontrivial solution of this problem belongs to the set V. We now estimate the value of the functional at xk: 1

1

I ( xk ) = ∫ x&k (t ) dt = ∫ yk 2

0

= yk

−1

−1/ 2

2

y& k (t ) dt =

0

1

∫ y& (t ) k

2

dt = yk

−1

yk

2

= yk .

0

Using the definition of the function yk (see Figure 31), we have

143

1

∫

2

y& k (t ) dt = yk

2

=

j/k

k

∑ ∫

j =1 ( j −1) / k

0

1/ k

y& k (t ) dt = k ∫ y& k (t ) dt. 2

2

0

From the equality

yk(t) = k y1(kt), 0 < t < 1/k. it follows that j/k

∫ 0

j/k 2

y& k (t ) dt = k

4

∫

1

2

y&1 ( kt ) dt = k

0

3

∫

y&1 (τ )

2

dτ =k 3 y1

2

.

0

As a result, we obtain the formula

I(xk) = k2 ||y1||, k = 1, 2, … . Among all the obtained solutions of the necessary conditions for an extremum, the solution y1 (as well as the functions obtained from y1 using the transformations Ai) provides the minimum value of the functional. To find the optimal system state, we need to apply formula (7.13). Differentiating the resulting function, we find the optimal control from equation (7.1). Remark 7.17. We have not explicitly determined the function y1. We have only established that it exists and possesses certain properties. For the complete solution, we need to explicitly solve the problem (7.11), (7.12). Since the equation is nonlinear, it is necessary to apply some approximation method for solving boundary value problems for differential equations. The determination of y1 may involve considerable difficulties because this problem is not uniquely solvable. Remark 7.18. It is not known whether y1 satisfies the symmetry conditions (7.15) or (7.16), i.e., whether the variational problem has two or four solutions. Apparently, this can be clarified only after finding the function y1. Remark 7.19. It is not certain that we have found all solutions of the boundary value problem (7.11), (7.12) and, consequently, of the necessary condition for an extremum (7.8). There may exist various sets of solutions defined by y1 which are not related to each other by the above transformations. If there is at least one more function y1, then another infinite set of solutions of our boundary value problem can be obtained using the technique described above.

144

SUMMARY

The analysis of the present example yields the following conclusions. 1.

The existence of an optimal control can be established without using the property of convexity of the set of admissible controls.

2.

The maximum principle may be established even for optimization problems with isoperimetric conditions.

3.

If the set of admissible controls is nonconvex, the solution of the extremum problem may be nonunique and the optimality conditions may be insufficient.

4.

The necessary optimality conditions in the optimization problems with isoperimetric conditions may be represented by integrodifferential equations.

5.

Boundary value problems with very simple nonlinearity may have infinitely many solutions.

6.

The heat conduction equation with simple nonlinearity may have infinitely many equilibrium states.

7.

When the problem is not uniquely solvable, it is possible to find transformations that can be used to construct new solutions based on the known solutions.

Example 8. Extremal bifurcation In the previous example, we considered a boundary value problem for a simple nonlinear differential equation of the second order which was associated with a certain extremum problem and had infinitely many solutions. The Chafee – Infante problem considered below has even more curious properties. Again we are dealing with the problem of transferring a system from one state into another while minimizing a relatively simple integral functional. We obtain the system of optimality conditions reduced to a boundary value problem with highly extraordinary properties. It may first seem that this problem is rather close to the one considered in the previous example, since it is the homogeneous first boundary value problem for a second-order differential equation with cubic nonlinearity. However, it turns out that the number of its solutions essentially depends on a parameter in the optimality criterion. A very complex and interesting bifurcation problem arises, which is a special form of ill-posedness in the sense of Hadamard. 8.1. PROBLEM FORMULATION

Consider the system described by the Cauchy problem

x& = u, t ∈ (0, π ); х(0) = 0.

(8.1)

The control u = u (t ) is assumed to belong to the set U of functions that transfer the system into the zero final state, i.e.,

x(π ) = 0. We set 146

(8.2)

π

I = ∫ ( 2u 2 +ν x 4 − 2μ x 2 ) dt , 0

where u and v are positive constant parameters of the problem and x is a solution of problem (8.1) corresponding to the control u. The following optimal control problem is called the Chafee – Infante problem. Problem 8. Find a control u ∈ U minimizing the functional I on U.

This problem is rather close to the problem considered in Example 4. They differ only in the form of the optimality criterion. For this reason, the analysis of Problem 8 is not supposed to be very difficult. However, since the functional I is nonconvex because of a negative term in the integrand, we should expect some obstacles to appear. Remark 8.1. It is possible to prove that Problem 8 is solvable using the technique described in the previous example. This proof will be omitted to avoid repetition.

8.2. THE NECESSARY CONDITION FOR AN EXTREMUM

In order to obtain the optimality conditions in the form of the maximum principle, we use the same procedure as in Example 4. Set

H (u , x, p ) = up − 2u 2 − ν x 4 + 2 μ x 2 . The optimal control must satisfy the maximum condition

H (u , x , р ) = max H ( w, y , р ), w

(8.3)

where p is a solution of the adjoint equation

р& = −∂Н / ∂х = 4ν х 3 +2 х.

(8.4)

We have the system (8.1) – (8.4) for the optimal control. Equating to zero the derivative of H with respect to the control, we find its unique stationarity point

u = p /4.

(8.5)

The second derivative of H with respect to the control is negative (being equal to -4). Hence, the control defined by formula (8.5) maximizes H. 147

Differentiating the state equation with respect to t and taking into account (8.4) and (8.5), we have

&& x = u& = p& / 4 = ν x 3 − μ x. As a result, we obtain the equation

&& x + μ x −ν x 3 = 0, t ∈ (0, π )

(8.6)

with the boundary conditions

x(0) = 0, x(π ) = 0.

(8.7)

The boundary value problem (8.6), (8.7) is called the Chafee – Infante problem. Conclusion 8.1. The optimal system state is a solution of the Chafee – Infante problem. Remark 8.2. As in the previous example, after the transformation of the optimality conditions, we have obtained a boundary value problem for a differential equation with cubic nonlinearity. In this case, however, the nonlinear term has the opposite sign and there is an additional linear term (the second term in the left-hand side of the equation). For this reason, we may expect to get essentially different results. Remark 8.3. In contrast to Example 7, where the nonlinear boundary value problem was a consequence of the isoperimetric condition, the main role in defining the form of the obtained equation in the present example is played by the optimality criterion.

If a solution of problem (8.6), (8.7) is an optimal system state, then, in view of equation (8.1), the optimal control is the derivative of this solution. Thus, to solve the optimization problem, we first have to solve the Chafee— Infante problem and then to determine whether its solutions provide the minimum of the functional I. 8.3. SOLVABILITY OF THE CHAFEE – INFANTE PROBLEM

It is evident that the Chafee—Infante problem has a trivial solution. In order to show that problem (8.6), (8.7) has nontrivial solutions; we will use a certain assertion from the theory of boundary value problems for nonlinear second-order differential equations. Consider the following problem:

&& x = f (t , x), t ∈ (0, T ); x(0) = 0, x(T ) = 0. 148

(8.8)

A function у is called a lower solution of (8.7) if

&& у − f (t , у ) ≥ 0, t ∈ (0, T ); у (0) ≤ 0, у (T ) ≤ 0. A function z is called an upper solution of (8.8) if

&& z − f (t , z ) ≤ 0, t ∈ (0, T ); z (0) ≥ 0, z (T ) ≥ 0. Remark 8.4. A solution of problem (8.8) in the usual sense is always a lower and an upper solution at the same time. In this case, all the relations in the above definition hold in the form of equalities. The converse statement is, in general, not true.

Theorem 10. If for a sufficiently smooth function f there exist a lower solution у and an upper solution z of problem (8.8) such that

y(t) ≤ z(t), t∈(0,T), then this problem has a solution x(t) such that

y(t) ≤ x(t) ≤ z(t), t∈(0,T). Remark 8.5. Lower and upper solutions provide estimates from below and from above, respectively, for a solution of the boundary value problem (hence the terminology).

We will use Theorem 8 to prove that there exists a nontrivial solution of the Chafee – Infante problem. We introduce the function y (t ) = ε sin t , where ε is a positive constant. We have

(

)

&& у + μ у − ν у 3 = ( μ − 1)ε sin t − νε 3 sin 3 t = ε sin t μ − 1 − ε 2ν sin 3 t . For μ > 0 and ε sufficiently small, the right-hand side of the rightmost equality is nonnegative. Then, since y = 0 on the boundaries of the considered domain, it follows that у is a lower solution of problem (8.6), (8.7). Conclusion 8.2. For ε sufficiently small, the function y (t ) = ε sin t is a lower solution of the Chafee – Infante problem.

Let a function z be identically equal to a constant c. Then

&& z + μ z −ν z 3 = c( μ −ν c 2 ) ≤ 0 for c ≥

μ /ν . 149

Conclusion 8.3. For с sufficiently large, the function z(t)=с is an upper solution of the Chafee – Infante problem.

Assuming that

{

}

c = max ε , μ /ν , we have y (t ) ≤ z (t ) for all t. By Theorem 10, problem (8.6), (8.7) has a solution x such that ε sin ε sin t ≤ x (t ) ≤ c for t∈(0,π) (see Figure 32). c

z x

ε

y t π

Figure 32. The solution of the boundary value problem is between its lower and upper solutions

Conclusion 8.4. The Chafee – Infante problem has a positive solution for every μ > 1 and v > 0. Remark 8.6. It can be proved that the Chafee – Infante problem has a unique positive solution. Remark 8.7. If a function x is a solution of the Chafee – Infante problem, then the function y(t)= x(π - t) is also a solution of this problem. Since there is a unique positive solution, new solutions cannot be obtained by any transformation. Therefore, the solution must be symmetric with respect to the middle of the time interval, i.e., х(t) = x(π - t). Remark 8.8. For μ ≤ 1, the above arguments are invalid. It can be proved that the Chafee-Infante problem has only a trivial solution if μ ≤ 1.

Conclusion 8.5. The Chafee – Infante problem has more than one solution if μ >1 and only a trivial solution if μ ≤ 1.

150

8.4. THE SET OF SOLUTIONS OF THE CHAFEE – INFANTE PROBLEM

As in the previous example, instead of directly solving the problem to find all the solutions, we will seek transformations that take one solution into another. In particular, one of such transformations is obvious: if a function x is a solution of the Chafee – Infante problem, then -x is also a solution of this problem. Conclusion 8.6. The Chafee – Infante problem has odd number of solutions for any values of its parameters.

Thus, in addition to the trivial solution x0 and a positive solution x+1, whose existence follows from Theorem 10, we have a negative solution x-1= –x+1. We now consider the equation

&& y + μ y / 4 − vy 3 / 4 = 0, t ∈ (0, π ). with homogeneous boundary conditions. As follows from the above arguments, for μ > 4 this problem has a positive solution, which is denoted by y+. We introduce the function

⎧

х+2 (t ) = ⎨

у+ (2t ),

0 < t < π / 2,

⎩− у+ (2π − 2t ), π /2 < t < π .

For 0 < t < π/2, we have

&& x+2 (t ) = 4 && y+ (2t ) = − μ y+ (2t ) + ν [ y+ (2t )]3 = − μ x+2 (t ) + ν [ x+2 (t ) ]3 . Similarly, for π /2 < t < π we obtain

&& x+2 (t ) = −4 && y+ (2π − 2t ) = μ y+ (2π − 2t ) − −ν [ y+ (2π − 2t )]3 = − μ x+2 (t ) +ν [ x+2 (t )]3 . Hence, the function x+2(t) satisfies (8.6), (8.7). Consequently, for μ> 4 the Chafee-Infante problem has at least five solutions: the trivial solution го, the positive solution x+1, the negative solution x-1= –x+1, and two solutions x+2 and x-1, which change their signs only once. Conclusion 8.7. The Chafee – Infante problem has exactly three solutions if 1 < μ < 4 and at least five solutions if μ > 4.

Similarly, the homogeneous boundary value problem for the equation 151

&& z+

μ 9

z−

ν 9

z 3 = 0, t ∈ (0, π ),

has a positive solution if μ>9. Let us denote this solution by z+. We introduce the function

0 < t < π / 3, ⎧ z+ (3t ), ⎪ х+3 (t ) = ⎨− z+ (2π − 3t ), π / 3 < t < 2π / 3, ⎪ z (3t − 2π ), 2π / 3 < t < π . ⎩ + For 0 < t < π/3, we have

&& x+3 (t ) = 9 && z + (3t ) = − μ z + (3t ) + ν [ z + (3t )]3 = − μ x+3 (t ) + ν [ x+3 (t )]3 . Similarly, for π /3 < t < 2 π/3 we obtain

&& x+3 (t ) = −9 && z+ (2π − 3t ) = μ z+ (2π − 3t ) − −ν [ z+ (2π − 3t )]3 = − μ x+3 (t ) + ν [ x+3 (t )]3 . Finally, if 2π /3 < t < π, then

&& x+3 (t ) = 9 && z + (3t − 2π ) = − μ z + (3t − 2π ) + +ν [ z + (3t − 2π )]3 = − μ x+3 (t ) + ν [ x+3 (t )]3 . Thus, for μ > 9 the problem has two more solutions x+3 and x-3= – x+3 (see Figure 33). х+1

х+3

х+2

х0

t

х-1

t

t

х-2

х-3

Figure 33. The solutions of the Chafee – Infante problem for 4< μ

152

≤9

Remark 8.9. The maximum of the positive solution of the Chafee – Infante problem depends on the parameters μ and v. For this reason, the function graphs shown in Figure 33 are only approximate representations of the corresponding solutions.

Conclusion 8.8. The Chafee – Infante problem has exactly five solutions if 4< μ ≤ 9 and at least seven solutions if μ > 9.

In the general case, for μ > k2, the equation

v&& +

μ k

2

v−

ν k

2

v 3 = 0, t ∈ (0, π )

has a positive solution, which will be denoted by v+. We introduce the function

0 < t < π / k, ⎧ v+ (kt ), ⎪−v (2π − kt ), π /k < t < 2π / k , ⎪⎪ + х+ k (t ) = ⎨ v+ (kt − 2π ), 2π / k < t < 3π / k , ⎪−v (4π − kt ), 3π /k < t < 4π / k , ⎪ + . . . . . . ⎪⎩ . Direct verification shows that this function satisfies (8.6), (8.7) and changes its sign (k–1) times. Conclusion 8.9. The Chafee – Infante problem has (2k–1) solutions for (k –1)2 < μ ≤ k2 and at least 2k+1 solutions for μ> k2.

Such a striking character of the effect of the coefficient μ of the linear term is curious arid amazing. In this situation, we should concentrate on the dependence of the solution on this parameter. 8.5. BIFURCATION POINTS

The analysis in the previous section leads us to the following conclusion. Conclusion 8.10. The number of solutions of the Chafee – Infante problem depends on the parameter μ; therefore, the dependence of the solution on this parameter is not continuous.

153

The dependence of the general behavior of the solution on the positive parameter μ can be described as follows. For small values of this coefficient, the problem has a unique (trivial) solution x0. The initial increase in μ does not affect the solution, which implies that the dependence of the solution on μ is continuous at this stage. When μ achieves the critical value of unity, two more solutions appear (x+1 and x-1), which means that the system properties change discretely at this point. Further increase in μ leads to the gradual changes in the solutions, but their structure remains the same. This goes on until μ achieves the second critical value equal to four. As a result, two new solutions appear (x+2 and x-2). From this point, the dependence of all the nontrivial solutions on у is continuous until μ achieves the value of 9, when two more solutions appear (x+3 and x-3). In the general case, while μ changes within the intervals between k2 and (k + 1)2 for natural k, the number of solutions of the Chafee–Infante problem does not change and their dependence on μ is continuous. But each time μ assumes the values k2, two more solutions appear (x+k and x-k). The value of a problem parameter at which the number of solutions changes is called the bifurcation point. Conclusion 8.11. The values μk = k, к = 1,2,..., are the bifurcation points of the Cbafee – Infante problem.

The process described above is shown in Figure 34, which is called the bifurcation diagram.

X

x+3 x+2

x+1

μ

x0 4

1

9

x-1

x-2 x-3 Figure 34. The bifurcation diagram for the Chafee – Infante problem

In the diagram, the x-axis is for the numeric parameter μ and the y-axis is for the functional space X of solutions of the problem. The bifurcation diagram clearly shows that new solutions emerge from the existing solutions at the bifurcation points. The dependence of each solution on μ is continuous through the entire interval of its existence. These results are also important for the analysis of the nonstationary Chafee – Infante problem, which includes the nonlinear heat conduction equation

∂u ∂τ

=

∂ 2u ∂ξ

2

+ μ u − ν u 3 , τ > 0, 0 < ξ < π

with the boundary conditions

u (0,τ ) = 0, u (π , τ ) = 0 and the initial condition

u (ξ , 0) = u0 (ξ ), 0 < ξ < π .

155

Obviously, the solutions of the boundary value problem (8.6), (8.7) represent equilibrium states of the nonstationary Chafee – Infante problem. We now arrive at the following conclusion. Conclusion 8.12. For (k – 1)2 < u < k2, the nonstationary Chafee – Infante problem has exactly 2k – 1 equilibrium states. Remark 8.10. Certainly not all of these equilibrium states are stable. Remark 8.11. As in the previous example, the nonstationary problem can be considered as a means of finding nontrivial solutions of the corresponding stationary problem.

It is clear that the solution of the Chafee – Infante problem strongly depends on the parameter μ, since a change in the value of μ, may cause a change in the number of solutions. This kind of dependence suggests essential ill-posedness in the sense of Hadamard. Conclusion 8.13. The Chafee – Infante problem is ill-posed in the sense of Hadamard. Remark 8.12. Changing the parameter и does not have such a considerable effect.

We will not try to find out which solution of the boundary value problem (or, equivalently, the optimality conditions) minimizes the functional. This involves certain technical difficulties and does not lead to any new significant results. Our main purpose in this example is to obtain a boundary value problem which is equivalent to the system of optimality conditions and possesses important and remarkable properties. Remark 8.13. It is likely that the optimal controls in this case are the derivatives of the positive and negative solutions of the Chafee—Infante boundary value problem. The solutions that change their signs vary at a greater rate (see Figure 33), which implies a relatively large norm of the control and hence relatively large values of the functional. This supposition, however, needs to be thoroughly substantiated.

SUMMARY

The analysis of this example yields the following conclusions. 1. The notions of upper and lower solutions can be used in solving boundary value problems for nonlinear differential equations. 2. If the functional to be minimized is nonconvex, the number of solutions of the system of optimality conditions may depend on a parameter of 156

the problem, which is associated with essential ill-posedness in the sense of Hadamard. 3. The number of solutions of the boundary value problem for a nonlinear second-order differential equation may depend on a certain parameter of the problem, which implies the bifurcation of solutions. 4. The number of equilibrium states in the boundary value problem for the nonlinear heat conduction equation may depend on a certain parameter of the problem.

Conclusion This monograph does not provide very deep and extensive coverage of its subjects. It contains no complicated theorems and rigorous descriptions of the mathematical aspects of optimal control theory. We confined ourselves to considering a small class of solvable problems. For this reason, the monograph does not represent a fundamental scientific research in the optimization theory and cannot serve as a textbook. Several simple examples were considered which turned out to be very nontrivial. It was shown that formal application of the standard optimization methods in these problems either gives no results or leads to wrong solutions. We have also considered some unfavourable conditions that may arise in solving optimal control problems if they are not subjected to proper and thorough analysis. One of our objectives was to help the reader to avoid possible illusions about the effectiveness of optimization methods in the solution of applied problems. Since such considerable obstacles were encountered in the analysis of very simple examples, we may expect extremely severe difficulties in practical optimization problems, which are much more complex. Note that we have not even considered the complicated and poorly studied problem of convergence of numerical algorithms. We should also keep in mind that before solving an optimization problem, it must be deduced and subjected to quantitative and qualitative analysis. In this connection, we can mention, for example, the problem of identification of a given system, in which it is required to determine how well the model represents the actual process. This only aggravates the difficulties described in the examples. However, the situation is far from being desperate. There are a lot of important practical optimal control problems which have been completely solved. Extremum problems are very interesting as well as very important for applications. They are the subject of research for many scientists. Our goals will be achieved if this book helps the reader to gain better understanding of the nature of extremum problems and the character of various obstacles that may be encountered in the practical solution of optimal control problems. We also hope that the described methods of overcoming these obstacles will prove to be useful. Some readers may be interested in the new methods of solving optimization problems presented here. 158

Tasks Task 1 (Preliminary task). Stationarity condition It is necessary to choose the function with given properties. If some property is impossible, it is necessary to explain this situation variant

example 1

example 2

example 3

1

Stationarity condition has the unique solution, but it is not optimal.

Stationarity condition has three solutions. There are local minimum, local maximum, and absolute maximum.

Stationarity condition has three solutions. There are local minimum, local maximum, and absolute minimum.

2

Stationarity condition Stationarity does not have any condition solution is solutions. sufficient condition of However the optimality. However minimum of the the extremum of the function exists. function does not exist.

3

Stationarity condition has two solutions, but there are not optimal.

4

Stationarity condition Stationarity condition Stationarity is not sufficient has the unique solution, condition has two condition which is solutions. There are of optimality. not optimal. minimum and maximum.

Stationarity condition has two solutions, which are optimal.

159

Stationarity condition is not necessary condition of optimality.

Stationarity condition is sufficient condition of optimality.

5

Stationarity condition is necessary and sufficient condition of optimality.

6

Stationarity condition Stationarity condition Stationarity does not have any solution condition is not solutions. However the has two solutions. sufficient condition maximum of the It is also sufficient of optimality, but the function exists. optimal control condition of exists. optimality.

7

Stationarity condition Stationarity condition has two solutions. One does not have any of them is optimal. solutions.

8

Stationarity condition Stationarity condition Stationarity has an infinite set of has a solution, but the condition has three solutions. function does not have solutions. the minimum. There are two local minimum, and one absolute maximum.

9

Stationarity condition Stationarity condition Stationarity has two solutions. One for a function with two condition is not of them is not optimal. absolute minimums. sufficient condition of optimality, but the optimal control does not exist.

10

Stationarity condition Stationarity condition Stationarity has two solutions. is not applicable. condition has there There are local solutions. One of minimum and absolute them is not optimal. maximum.

Stationarity condition has the unique solution, which is optimal.

Stationarity condition is not applicable.

Stationarity condition has an infinite set of solutions.

List of general examples 2

⎛c 2 2 ⎞ 1) x& = au , x (0) = 0; u (t ) ≤ b; I = ∫ ⎜ u + dx ⎟ dt ⇒ min 4 ⎠ 0⎝ T

(

)

2 2 2) x& = 2u , x (0) = 0; u (t ) ≤ b; I = ∫ −cu + dx dt ⇒ min 0

2

⎛ 0⎝

2 3) x& = − au , x (0) = 0; u (t ) ≤ 2; I = ∫ ⎜ cu + T

1 2

⎞ ⎠

x 2 ⎟ dt ⇒ min

(

)

2 2 4) x& = − au , x (0) = 0; u (t ) ≤ 1/ 2; I = ∫ cu − dx dt ⇒ min 0

1/ 2

∫ ( cu

5) x& = − au , x (0) = 0; u (t ) ≤ 1/ 2; I =

2

)

+ dx 2 dt ⇒ min

0

T

(

)

2 2 6) x& = u / 2, x (0) = 0; u (t ) ≤ b; I = ∫ cu − dx dt ⇒ min 0

7) x& = au , x (0) = 0; u (t ) ≤ b; I =

1 2

T /2

∫ ( cu

2

)

+ dx 2 dt ⇒ min

0

T

(

)

2 2 8) x& = − au , x (0) = 0; u (t ) ≤ 1/ 2; I = ∫ −cu + dx dt ⇒ min 0

2

(

)

2 2 9) x& = u / 2, x (0) = 0; u (t ) ≤ 1/ 2; I = ∫ cu − dx dt ⇒ min 0

10) x& = au , x (0) = 0; 2 u (t ) ≤ b; I = The parameters a, b, c, d are arbitraries.

161

1 2

T /3

∫ ( cu 0

2

)

− dx 2 dt ⇒ min

1. 2. 3. 4. 5.

Task 2. Necessary conditions of optimality for the given example Determination of the function Н for the concrete example. Determination of the conjugate equation. Determination of the maximum principle. Finding of the control from the maximum principle. Finding of the control for the analogical maximization problem.

1. 2. 3.

Task 3. Convergence of the iterative method for the given example Determination of the iterative method for the concrete example. Proving of the iterative method for some class of parameters. Finding of the optimal control.

1. 2. 3. 4.

Task 4. Uniqueness of the optimal control for the given example Proving of the linearity of the control-state mapping. Selection of parameters for the strong convexity of the functional. Proving of the uniqueness of the optimal control. Analysis of the uniqueness for the analogical maximization problem.

Task 5. Sufficiently of the optimality conditions for the given example 1. Determination of the remainder term. 2. Selection of parameters for the assurance of the optimality conditions. 3. Analysis of the sufficiently of the optimality conditions for the analogical maximization problem.

1. 2. 3. 4.

1. 2. 3. 4. 5.

Task 6. Singular control for the given example Selection of parameters for the existence of the singular control. Determination of the singular control. Application of the Kelly condition. Analysis of the singular control for the analogical maximization problem. Task 7. Existence of the optimal control for the given example Proving of the convexity of the admissible control set. Proving of the closure of the admissible control set. Proving of the continuity of the functional. Selection of parameters for the convexity of the functional. Proving of the existence of the optimal control.

162

Task 8. Optimization problem for the system with fixed final state 1. Consideration the given example with fixed final state. 2. Determination of the necessary conditions of optimality. 3. Application of the shooting method. Task 9. Well-posedness in the sense of Tichonov for the given example 1. Selection of parameters for the uniform convexity of the functional. 2. Proving of the well-posedness in the sense of Tichonov. 3. Analysis of the well-posedness in the sense of Tichonov for the analogical maximization problem. Task 10. Well-posedness in the sense of Hadamard 2

1.

Consideration the given example with the replacement of the term x

2. 3.

in the functional by ( x − z ) , where z is a given function. Proving of the continuity of the functional with respect to the parameter z. Proving of the well-posedness in the sense of Hadamard.

1.

Task 11. Optimization problem with isoperimetric condition Consideration the given example with additional constraint

2

1

∫ x(t )

4

dt = 1.

0

2. 3.

1. 2.

Determination of the necessary conditions of optimality. Determination of the iterative method for the necessary conditions of optimality. Task 12. Bifurcation of the extremum Selection of the function, depending from parameter, such as the phenomenon of the bifurcation of the extremum for the stationarity condition. Examination of properties of solutions of the stationarity condition for all value of the parameters.

Comments In this book, we considered a series of optimal control problems with nonstandard properties. These problems are certainly not new. In particular, Example 2 is described in Gabasov and Kirillova (1973), Example 3 in Varga (1977), Example 5 in Vasil'ev (1981), and Example 7 in Lions (1983). Example 4 is based on a variational problem from Olech (1976), and Example 8 is associated with a boundary value problem in Henry (1981). Examples 1 and 6 can be found in Serovajsky (2001, 2004). However, the analysis presented in the present monograph is essentially different from the analysis in the mentioned literature. Modern optimal control theory is essentially based on the fundamental monographs Pontrjagin et al. (1974) and Bellman (1957). At the same time, almost all fundamental ideas of this theory have analogs in classical calculus of variations, see the monographs Bliss (1946), Budylin (2001), El'sgolts (1969), Gel'fand and Fomin (1961), Hestenes (1966); Lavrent'ev and Ljusternik (1950), and Young (1969). The definition and analysis of the maximum principle can be found in Pontrjagin et al. (1974) or any textbook on optimization methods, for example, Alekseev et al. (1979), Bryson and Ho (1975), Gabasov and Kirillova (1973, 1974), Galeev and Tihomirov (2000), Moiseyev (1975), Pshenichny (1969), and Vasil'ev (2002), The study of necessary extremum conditions in the problems with phase restrictions dates back to the Ljusternik theorem (Ljusternik, 1934). See also Arutunov (1997), Boltjanskii (1975), Dmitruk et al. (1980), Dubovitskii and Milyutin (1977), Ioffe and Tikhomirov (1974), Levitin et al. (1978), Neustadt (1966, 1967), Novozhilov and Plotnikov (1982), Stepan’yanz (2002), Yakubovich (1977, 1978, 1979), and others. The applications of convex analysis in the theory of extremum are described in Pshenichny (1980), Rockafellar (1970), and Ekland and Temam (1979). The analysis of nonconvex optimal control problems is presented, for example, in Asplund (1969), Baranger (1974), Ekland and Temam (1979), Kromer (2008), Raymond (1994), whereas nonsmooth extremum problems were analyzed in Clarke (1983, 1989), Dem’yanov and Rubinov (1990), and Ekland and Temam (1979). A systematic account of optimization methods for systems with distributed parameters is given in Lions 164

(1971) (see also Barbu, 1981; Butkovskii, 1975; Wolfersdorf, 1976; Goebel, 1969; Egorov, 1964; Ivanenko and Mel'nik, 1988; Litvinov, 1987; Lur'e, 1975; Raytum, 1989; Sirazetdinov, 1977; Sokolowsky, 1976). Optimization problems for singular infinite-dimensional systems were studied in Lions (1983) and Fursikov (1982). Optimization problems for infinite-dimensional systems in the case where the state function is not differentiable with respect to the control were studied in Serovajsky (2005, 2006). Approximate methods for solving optimal control problems are described, for example, in Cea (1978), Chernous'ko and Banichuk (1973), and Chernous'ko and Kolmanovskii (1977), Fedorenko (1978), Fletcher (1987), Gruver and Sachs (1981), Moiseev (1975), Polak (1974), and Vasil'ev (1980). Simultaneous reduction of various kinds of errors in the process of numerical solution of extremum problems is considered in Adylova and Serovajsky (1988). The theory of singular controls was developed in Gabasov and Kirillova (1973), and degenerate problems of optimal control were considered in Gurman (1977). The problems of existence of optimal controls are formulated in Cesare (1966), Ekland and Temam (1979), Ioffe and Tikhomirov (1974), Lions (1971), Olech (1976), Varga (1977), Vasil'ev (1981), and Young (1969). Examples of unsolvable optimization problems for systems with concentrated parameters are given in Varga (1977), Olech (1976), Fleming and Rishel (1978), and Cesare (1966). Unsolvable problems for systems with distributed parameters are considered in Baranger (1973), Murat (1972), Suryanarayana (1975). The well-posedness of extremum problems and regularization methods for them are discussed, for example, in Vasil'ev (1981), Zolezzi (1981), Lucchetti and Patrone (1982), Tikhonov and Arsenin (1974), Tikhonov and Vasil'ev (1978), and Holmes (1972). Bifurcation of extermals are considered in Darinskii et al. (2004). For the necessary prerequisites, we refer the reader to Kantorovich and Akilov (1977), Reed and Simon (1972), Hutson and Pym (1980), Sobolev (1988), and Kolmogorov and Fomin (1989). The theory of boundary value problems for differential equations is presented in Lefshets (1961), Hartman (1970), Hutson and Pim (1980), and Henry (1981). This bibliography is certainly incomplete. It only refers to the main areas of optimal control theory and related problems. For more answers to any questions associated with optimal control theory, see the bibliography in the mentioned literature.

Bibliography Adylova, I.V. and Serovajsky, S.Ya. (1988). Numerical solution of coefficient inverse problems with correlating approximation. Inzh.-Phys. Journ. NS, 858-859 (in Russian). Alekseev, V.M., Tikhomirov, V.M., and Fomin, S.V. (1979). Optimal Control. Nauka, Moscow (in Russian). Arutunov, A.V. (1997). Conditions of Optimality. Anormal and Singular Problems. Factorial, Moscow (in Russian). Asplund, E. (1969). Topics in the theory of convex functions. Theory and Applications of Monotone Operators. Edizioni "Oderisi", Gubbio, 1-33. Aysagaliev, S.A. (1999). Boundary Value Problems of Optimal Control. Kaz. Univ., Alma-Ata (in Russian). Baranger, J. (1973). Quelques Resultats en Optimisation поп Convexe. These. Grenoble. Barbu, V. (1982). Necessary conditions for control problems governed by nonlinear partial differential equations. Res. Notes in Math.., 60. Pitman, Boston—London, 19-47. Bellman, R. (1957). Dynamic Programming. Princeton University Press, Princeton. Bliss, G. (1946) Lectures on the Calculus of Variations. University of Chicago Press, Chicago. Boltyanskii, V.G. (1975). The tent method in the theory of extremum problems. Uspekhi Mat. Nauk, 30 (3), 3-55 (in Russian). Bryson, A.E. and Ho, Y.C. (1975). Applied Optimal Control. Washington, DC: Hemisphere.

166

Budylin, А.М. (2001) The Calculus of Variations. Sankt-Peterburg, SPbGU. http://www.newlibrary.ru/book/budylin_a_m_/variacionnoe_ischislenie.html . Butkovskii, A.G. (1975). Control Methods for Systems with Distributed Parameters. Nauka, Moscow (in Russian). Cea, J. (1978). Lectures on Optimization — Theory and Algorithms. Springer-Verlag, Berlin. Cesare, L. (1966). Existence theorems for weak and usual optimal solutions in Lagrange problems with unilateral constraint. Trans. Amer. Math. Soc, 369-412, 413-430. Chernous'ko, F. L. and Banichuk, N. V. (1973). Variational Problems of Mechanics and Control. Nauka, Moscow (in Russian). Chernous'ko, F.L. and Kolmanovskii, V.V. (1977). Computational and applied optimization methods. Matem. Analiz, Itogi Nauki i Tekhniki, 14, 101-166 (in Russian). Clarke, F.H. (1983). Optimization and Nonsmooth Analysis. John Wiley and Sons, New York. Clarke F.H. (1989). Methods of Dynamic and Nonsmooth Optimization. SIAM, Philodelphia. Darinskii, B.M., Sapronov, Yu.I., Tsarev, S.L. (2004). Bifurcation of extermals for Fredholm functional. Sovrem. Mathem., 12, 3–140. Dem’yanov, V.F. and Rubinov A.M. (1990). Basis of Nonsmooth Analysis and Quasidifferential Calculus. Nauka, Moscow (in Russian). Dmitruk, A.V., Milyutin, A.A., Osmolovskii, N.P. (1980). The Ljusternik theorem and the extremum theory. Uspekhi Mat. Nauk., 35 (5), 11-46 (in Russian). Dubovitskii, A.D. and Milyutin, A. A. (1977). The Necessary Conditions for a Weak Extremum. Nauka, Moscow (in Russian). Egorov, A.I. (1978). Optimal Control of Thermal and Diffusion Processes. Nauka, Moscow (in Russian). Egorov, Yu.V. (1964). Necessary optimality conditions in a Banach space. Mat. Sbornik., 64(1), 79-101 (in Russian). Ekland, I. and Temam, R. (1979). Convex Analysis and Variational Problems. Mir, Moscow (in Russian).

167

El'sgolts, L.E. (1969). Differential Equations and the Calculus of Variations. Nauka, Moscow (in Russian). Fedorenko, R.P. (1978). Approximate Solution of Optimal Control Problems. Nauka, Moscow (in Russian). Fel'dbaum, A. A. (1966). Fundamentals Of The Theory of Optimal Automatic Systems. Nauka, Moscow (in Russian). Fletcher, R. (1987). Practical optimization methods. John Wiley & Son, Chichester. Fleming, U. and Rishel, R. (1978). Deterministic and Stochastic Optimal Control. Mir, Moscow (in Russian). Fursikov, A.V. (1982). The properties of solutions of certain extremum problems connected with the Navier—Stokes equations. Mat. Sbornik., 118 (3), 323-349 (in Russian). Gabasov, R. and Kirillova, F.M. (1971). Qualitative Theory of Optimal Control. Nauka, Moscow (in Russian). Gabasov, R. and Kirillova, F.M. (1973). Singular Optimal Controls. Nauka, Moscow (in Russian). Gabasov, R. and Kirillova, F.M. (1974). Maximum Principle in Optimal Control Theory. Nauka i Tekhnika, Minsk (in Russian). Galeev, E.M. and Tihomirov, V.M. (2000). Optimization: Theory, Examples, Alghorithms. Editorial, Moskow (in Russian). Gamkrelidze, R. V. (1977). Fundamentals of Optimal Control. Tbilisi State Univ., Tbilisi (in Russian). Gelbaum, B. and Olmsted, J. (1964). Counterexamples in Analysis. HoldenDay, Inc., San-Francisco—London—Amsterdam. Gel'fand, I.M. and Fomin, S.V. (1961). Calculus of Variations. Fizmatgiz, Moscow (in Russian). Goebel, M. (1969). Optimal control of coefficients in linear elliptic equations. Math. Oper. Stand. Optim., 3, 89-97. Gruver, W.A. and Sachs, E. (1981). Algorithmic Methods in Optimal Control. Pitman Res. Math. 47, Pitman, London. Gurman, V.I. (1977). Degenerate Optimal Control Problems. Nauka, Moscow (in Russian). 168

Hartman, F. (1970). Ordinary Differential Equations. Mir, Moscow (in Russian). Henry, D. (1981). Geometric Theory of Semilinear Parabolic Equations. Lecture Notes in Mathematics, 840. Springer-Verlag, Berlin—New York. Hestenes, M.R. (1966). Calculus of Variations and Optimal Control Theory. John Wiley and Sons, New York—London—Sydney. Holmes, R.B. (1972). A Course on Optimization and Best Approximation. Lecture Notes in Mathematics, 257. Springer-Verlag, New York. Hutson, V. and Pym, J.S. (1980). Applications of Functional Analysis and Operator Theory. Academic Press, New York—London. Ioffe, A.D. and Tikhomirov, V.M. (1974). The Theory of Extremum Problems. Nauka, Moscow (in Russian). Ivanenko, V.I. and Mel'nik,V.S. (1988). Variational Methods in Control Problems for Systems with Distributed Parameters. Naukova Dumka, Kiev (in Russian). Kantorovich, L.V. and Akilov, G.P. (1977). Functional Analysis. Nauka, Moscow (in Russian). Kolmogorov, A.N. and Fomin. S.V. (1989). Elements of Function Theory and Functional Analysis. Nauka, Moscow (in Russian). Krasovskii, N.N. (1968). Theory of Motion Control. Nauka, Moscow (in Russian). Kromer, S. (2008). Existence and symmetry of minimizers for nonconvex radially symmetric variational problems. Calculus of Variations and Partial Differential Equations. 32 (2), 219 – 236. Krotov, V.F. and Gurman, V.I. (1973). The Method and Problems of Optimal Control. Nauka, Moscow (in Russian). Lavrent'ev, M.A. and Ljusternik, L.A. (1950). A Course of Calculus of Variations. Nauka, Moscow (in Russian). Levitin, E.S., Milyutin, A.A., and Osmolovskii, N.P. (1978). Conditions of higher order for local minimum in problems with restrictions. Uspekhi Mat. Nauk., 28 (6), 85-148 (in Russian). Lefshets, S. (1961). Geometric Theory of Differential Equations. Izd. Lit., Moscow (in Russian).

169

Li, M.B. and Markus, L. (1972). Fundamentals of Optimal Control Theory. Nauka, Moscow (in Russian). Lions, J.-L. (1971). Optimal Control of Systems Governed by Partial Differential Equations. Die Grundlehren der mathematischen Wissenschaften, 170. Springer-Verlag, New York—Berlin. Lions, J.-L. (1974). Some Methods of Solving Nonlinear Boundary Value Problems. Mir, Moscow (in Russian). Lions, J.-L. (1983). Controle des Systemes Distribues Singuliers. Methodes Mathematiques de V informatique, 13. Gauthier-Villars, Montrouge. Litvinov, V.G. (1987). Optimization in Elliptic Boundary Value Problems with Applications in Mechanics. Nauka, Moscow (in R.ussian). Lucchetti, R. and Patrone, F. (1982). Hadamard and Tyhonov wellposedness of a certain class of convex functions. J. Math. Anal. Appl.. 88, 204-215. Lur'e К.А. (1975). Optimal Control in the Problems of Mathematical Physics. Nauka, Moscow (in Russian). Ljusternik, L.A. (1934). On conditions of extremum of functionals. Mat. Sbornik, 3, 390-401 (in Russian). McShane, E.J. (1940). Generalized curves. Dukes Math. J., 6, 513-536. Moiseev, N.N. (1975). Elements of the Theory of Optimal Systems. Nauka, Moscow (in Russian). Murat, F. (1971). Un contre-exemple pour le probleme du controle dans les coefficients. С R. Acad. Sci. Paris Set. A-B, 273, A708-A711. Murat, F. (1972). Theoremes de non existence pour des problemes de controle dans les coefficients. C. R. Acad. Sci. Paris Sir. A-B, 274, A395A398. Neustadt, L.W. (1966, 1967). An abstract variational theory with applications to a broad class of optimization problems. I. General theory. SIAM J. Control, 4, 505-527. Neustadt, L.W. (1967). An abstract variational theory with applications to a broad class of optimization problems. II. Applications. SIAM J. Control, 1, 90-137.

170

Novozhilov, M.M. and Plotnikov, V.I. (1982). Generalized rule of Lagrange multipliers for distributed systems with phase restrictions. Differents. Uravn., 18(4), 584-592 (in Russian). Olech, C. (1976). Existence theory in optimal control. Topol. Fund. Anal., 1, 291-328. Polak, E. (1974). Numerical Optimization Methods. Mir, Moscow (in RusPontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V., and Mishchenko, E.F. (1974). Mathematical Theory of Optimal Processes. Nauka, Moscow (in Russian). Pshenichny, B.N. (1969). Necessary Extremum Conditions. Nauka, Moscow (in Russian). Pshenichny, B.N. (1980). Convex Analysis and Extremum Problems. Nauka, Moscow (in Russian). Raymond, J. P. (1994). Existence and uniqueness results for minimization problems with nonconvex functionals. J. Optim. Theory Appl. 82 (3), 571592. Raytum, U.E. (1989). Optimal Control Problems for Elliptic Equations. Zinatne, Riga (in Russian). Reed, M. and Simon, B. (1972). Methods of Modern Mathematical Physics. I. Functional Analysis. Academic Press, New York – London. Rockafellar, R. (1970). Convex Analysis. Princeton Mathematical Series, 28. Princeton, N.J. Serovajsky, S.Ya. (2001). Counterexamples in the Optimal Control Theory. Kazakh. Univ., Almaty (in Russian). Serovajsky, S.Ya. (2004). Counterexamples in the Optimal Control Theory. – Brill Academic Press. Netherlands, Utrecht-Boston. Serovajsky, S.Ya. (2005). Calculation of Functional Gradients and Extended Differentiation of Operators. Journal of Inverse and Ill-Posed Problems. 13 (4), 383-396. Serovajsky, S.Ya. (2006). Optimization and Differentiation. Vol. 1. Almaty, Print-S, 2006 (in Russian). Sirazetdinov, Т.К. (1977). Optimization of Systems with Distributed Parameters. Nauka, Moscow (in Russian).

171

Sobolev, S.L. (1988). Some Applications of Functional, Analysis in Mathematical Physics. Nauka, Moscow (in Russian). Sokolovsky, J. (1976). On optimal control of parabolic systems. Lect. Notes Сотр. Sci. 1, P. 623-635. Stepan’yanz, G.A. (2002) Necessary Conditions of Extremum (Dubovitskii – Milyutin Theory). MAI, Moscow (in Russian). Suryanarayana, M.B. (1975). Existence theorems for optimization problems concerning hyperbolic partial differential equations. Existence theory in the calculus of variations and optimal control. J. Optim. Theory Appl., 15, 361392. Tikhonov, A.N. and Arsenin, V.Ya. (1974). Methods of Solution of IllPosed Problems. Nauka, Moscow (in Russian). Tikhonov, A.N. and Vasil'ev, F.P. (1978). Methods for the solution of illposed extremal problems. Banach Center Publ.,3. PWN, Warsaw, 297-342. Varga, Dz. (1977). Optimal Control of Differential and Functional Equations. Nauka, Moscow (in Russian). Vasil'ev, F.P. (in Russian).

(2002).

Optimization

Methods.

Factorial,

Moscow

Vasil'ev, F.P. (1981) Numerical Methods of Solution of Extremum Problems Nauka, Moscow (in Russian). Wolfersdorf, L. (1976). Optimal control for processing governed by mildly nonlinear differential equations of parabolic type. Z. Angew. Math. Mech. 56, 531-538. Yakubovich, V.A. (1977, 1978, 1979). To the abstract theory of optimal control. Sib. Math. Journ. 18(3), 685–707 (1977); 19(2), 436–460 (1978); 20(4), 885–910; 21(5), 1131-1159 (1979) (in Russian). Young, L.C. (1969). Lecture on the Calculus of Variations and Optimal Control Theory. W.B. Saunders Co., Philadelphia–London–Toronto. Zolezzi, T. (1981). A characterization of well–posed optimal control systems. SIAM. J. Control Optim. 19(5), 605–616.

Short Russian-English mathematical dictionary 

– algorithm

      -   

– infinity – bifurcation – greater-than – equal-to-or-greater-than

 -       -  - -      

– variation – spiky variation – conclusion – convexity – convex – strictly convex – strictly uniformly convex – degenerate

  

– smooth – boundary

      

– differentiation – sufficient

   

– uniqueness

     ! -    -   -    -     - "#–$       

– dependence – Cauchy problem – well-posed problem – boundary problem – ill-posed problem – optimal control problem – Chafee–Infante problem – closure – closed 173

    

– remark – sign

   -    $    $   $   

$ 

– integrate – integrate by parts – integrable – interval – iterative process – iteration

    !#  !    

– well-posedness – coercive – optimality criterion

   %   % 

– left-hand side – linear – false

  -    &  &     -     % 

-     -  ) -  ( &   -  '   -  

– less than – equal-to-or-less-than – measure – gradient methods – method of Lagrange multipliers – method of successive approximations – gradient-projection method – regularization method – shooting method – set – set of admissible controls – empty set

    *     *    -     *   * )   * )  *    *

– ill-posedness – continuity – continuous – uniformly continuous – inequality – unsolvability – unsolvable – lower bound – norm

(+      ,  

– sum of sets – constraint

-     ' ( 

174

Ограниченность Ограниченный - снизу Однородный Окрестность Остаточный член Ось Отображение

– boundedness – bounded – lower bound – homogeneous – neighborhood – remainder term – axis – mapping

Параметр Пересечение множеств Переходить к пределу Погрешность Подмножество Подпоследовательность Положение равновесия Полунепрерывность Последовательность - минимизирующая Почти всюду Правая часть Предел Предположение Приближение - начальное Принадлежать Принцип максимума Приращение Причина Произведение скалярное Производная - обобщенная Произвольный Пространство - банахово - гильбертово - нормированное - рефлексивное - Соболева

– parameter – intersection of sets – pass to the limit – error – subset – subsequence – equilibrium state – semicontinuity – sequence – minimizing sequence – nearly everywhere – right-hand side – limit – assumption – approximation – initial approximation – belong to… – maximum principle – increment – cause – scalar products – derivative – generalized derivative – arbitrary – space – Banach space – Hilbert space – normalized space – reflexive space – Sobolev space

Равномерно Разделить Разность

– uniformly – divide – difference 175

Разрешимость Разрешимый Расходиться Решение - единственное - приближенное

– solvability – solvable – diverge – solution – unique solution – approximate solution

Свойство Система сопряженная - билинейная Состояние - конечное - -, фиксированное - начальное Существование Сходиться - слабо

– property – adjoint system – bilinear system – state – final state – fixed final state – initial state – existence – converge – weakly converge

Теорема - о среднем Точка - разрыва - стационарности Требование

– theorem – Mean-Value Theorem – point – point of discontinuity – stationary point – requirement

Угол Управление - допустимое - особое Уравнение - интегро-дифференциальное - обыкновенное дифференциальное - сопряженное Условие изопериметрическое - оптимальности - -, необходимое

- -, достаточное - стационарности Утверждение

– angle – control – admissible control – singular control – equation – integro-differential equation – ordinary differential equation – adjoint equation – isoperimetric condition – optimality condition – necessary optimality condition – sufficient optimality condition – stationarity condition – assertion

Функционал - квадратичный - минимизируемый

– functional – quadratic functional – minimizing functional 176

- полунепрерывный Функция - достаточно гладкая - интегрируемая - интегрируемая с квадратом - кусочно дифференцируемая - кусочно постоянная - разрывная Экстремаль

– semi continuous functional – function – sufficiently smooth function – integrable function – square-integrable function – piecewise-differentiable function – piecewise-constant function – discontinuous function – extremal

Учебное издание Серовайский Семен Яковлевич ПРАКТИЧЕСКИЙ КУРС ТЕОРИИ ОПТИМАЛЬНОГО УПРАВЛЕНИЯ С ПРИМЕРАМИ Учебное пособие Выпускающий редактор В.Н. Сейткулова ИБ № 5170 Подписано в печать 26.05.11. Формат 60х84 1/16. Бумага офсетная. Печать цифровая. Объем 11,18 п.л. Тираж 100 экз. Заказ № 437. Издательство «Қазақ университетi» Казахского национального университета им. аль-Фараби. 050040, г. Алматы, пр. аль-Фараби, 71. КазНУ. Отпечатано в типографии издательства «Қазақ университетi».