Variational Methods in Nonlinear Analysis: With Applications in Optimization and Partial Differential Equations 9783110647389, 9783110647365

Citation preview

Dimitrios C. Kravvaritis, Athanasios N. Yannacopoulos Variational Methods in Nonlinear Analysis

Also of Interest Elementary Functional Analysis Marat V. Markin, 2018 ISBN 978-3-11-061391-9, e-ISBN (PDF) 978-3-11-061403-9, e-ISBN (EPUB) 978-3-11-061409-1

Real Analysis Marat V. Markin, 2019 ISBN 978-3-11-060097-1, e-ISBN (PDF) 978-3-11-060099-5, e-ISBN (EPUB) 978-3-11-059882-7

Applied Nonlinear Functional Analysis. An Introduction Nikolaos S. Papageorgiou, Patrick Winkert, 2018 ISBN 978-3-11-051622-7, e-ISBN (PDF) 978-3-11-053298-2, e-ISBN (EPUB) 978-3-11-053183-1

Functional Analysis. A Terse Introduction Gerardo Chacón, Humberto Rafeiro, Juan Camilo Vallejo, 2016 ISBN 978-3-11-044191-8, e-ISBN (PDF) 978-3-11-044192-5, e-ISBN (EPUB) 978-3-11-043364-7

Dynamics of Solid Structures. Methods using Integrodifferential Relations Georgy Viktorovich Kostin, Vasily V. Saurin ISBN 978-3-11-051623-4, e-ISBN (PDF) 978-3-11-051644-9, e-ISBN (EPUB) 978-3-11-051625-8

Dimitrios C. Kravvaritis, Athanasios N. Yannacopoulos

Variational Methods in Nonlinear Analysis |

With Applications in Optimization and Partial Differential Equations

Mathematics Subject Classification 2010 35-02, 65-02, 65C30, 65C05, 65N35, 65N75, 65N80 Authors Prof. Dr. Dimitrios C. Kravvaritis National Technical University of Athens School of Applied Mathematical and Physical Sciences Heroon Polytechniou 9 Zografou Campus 157 80 Athens, Greece [email protected]

Prof. Dr. Athanasios N. Yannacopoulos Athens University of Economics and Business Department of Statistics Patission 76 104 34 Athens, Greece [email protected]

ISBN 978-3-11-064736-5 e-ISBN (PDF) 978-3-11-064738-9 e-ISBN (EPUB) 978-3-11-064745-7 Library of Congress Control Number: 2020934276 Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.dnb.de. © 2020 Walter de Gruyter GmbH, Berlin/Boston Cover image: your_photo / iStock / Getty Images Typesetting: VTeX UAB, Lithuania Printing and binding: CPI books GmbH, Leck www.degruyter.com

|

DCK: To my wife Katerina and my two sons Christos and Nikos. ANY: To my fixed-point Electra and to Nikos, Jenny and Helen

|

Spent the morning putting in a comma and the afternoon removing it. G. Flaumbert or O. Wilde If you can’t prove your theorem, keep shifting parts of the conclusion to the assumptions, until you can. E. De Giorgi (according to Wikipedia)

Preface Nonlinear analysis is a vast field, which originated from the need of extending classical linear structures in analysis in order to treat nonlinear problems. It can be considered as general term for describing a wide spectrum of abstract mathematical techniques, ranging from geometry, topology and analysis, which nevertheless find important applications in a variety of fields, such as mathematical modeling (e. g., in economics, engineering, biology, decision science, etc.), optimization or in applied mathematics (e. g., integral, differential equations and partial differential equations). It is the aim of this book to select out of this wide spectrum, certain aspects that the authors consider as most useful in a variety of applications, and in particular aspects of the theory related with variational methods, which are most closely related to applications in optimization and PDEs. The book is intended to graduate students, lecturers or independent researchers who wish a concise introduction to nonlinear analysis, in order to learn the material independently, use it as a guideline for the organization of a course, or use it as a reference when trying to master certain techniques to use in their own research. We hope that we have achieved a good balance between theory and applications. All the theoretical results are proved in full detail, emphasizing certain delicate points, and many examples are provided guiding the reader either towards extensions of the fundamental results or towards important applications in a variety of fields. Large sections are devoted to the applications of the theoretical results in optimization, PDEs and variational inequalities, including hints toward a more algorithmic approach. Keeping the balance right, the volume manageable but at the same time striving to provide as wide a spectrum of techniques and subjects as possible has been a very tricky business. We hope that we have managed to present a good variety of these parts of nonlinear analysis which find the majority of applications in the fields which are the within the core interest of this book, i. e., optimization and PDEs. Naturally, we had to draw lines in our indulgence to the beautiful subjects we have decided to present (otherwise we would have results in a nine volume treatise rather than a nine chapter book) but at the same time we have to completely omit important fields such as, e. g., degree theory, and we apologize for this, hoping to make up for this omission in the future! At any rate, we hope that our effort (which by the way was extremely enjoyable) will make itself useful to those wishing a concise introduction to nonlinear analysis with an emphasis toward applications, and will help newcomers of any mathematical age to the field, colleagues in academia to organize their courses, and practitioners or researchers toward obtaining the methodological framework that will help them toward reaching their goal. This book consists of nine chapters, which we think cover a wide spectrum of concepts and techniques of nonlinear analysis and their applications. https://doi.org/10.1515/9783110647389-201

X | Preface Chapter 1 collects, mostly without proofs, some preliminary material from topology, linear functional analysis, Sobolev spaces and multivalued maps, which are considered as necessary in order to proceed further to developing the material covered in this book. Even though proofs are not provided for this introductory material, we try to illustrate and clarify the concepts using numerous examples. Chapter 2 develops calculus in Banach spaces and focuses on convexity as a fundamental property of subsets of Banach spaces as well as on the properties of convex functions on Banach spaces. This material is developed in detail, starting from topological properties of convex sets in Banach space, in particular, their remarkable properties with respect to the weak topology, and then focuses on properties of convex functions with respect to continuity and semicontinuity or with respect to their Gâteaux or Fréchet derivatives. Extended comments on the deep connections of convexity with optimization are made. Chapter 3 is devoted to an important tool of nonlinear analysis, which is used throughout this book, fixed-point theory. One could devote a whole book to the subject, so here we take a minimal approach, of presenting a selection of fixed-point theorems which we find indispensable. This list consists of the Banach contraction theorem, Brouwer’s fixed-point theorem which, although finite dimensional in nature, forms the basis for developing a large number of useful fixed-point theorems in infinite dimensional spaces, Schauder’s fixed-point theorem and Browder’s fixedpoint theorem. Special attention is paid to the construction of iterative schemes for the approximation of fixed points, and in particular the Krasnoselskii–Mann iterative scheme which finds interesting applications in numerical analysis. We also introduce the Caristi fixed-point theorem and its deep connection with a very useful tool, the Ekeland variational principle. The chapter closes with a fixed-point theorem for multivalued maps. Chapter 4 provides an introduction to nonsmooth convex functions and the concept of the subdifferential. The theory of the subdifferential as well as subdifferential calculus is developed gradually, leading to its applications to optimization. The Moreau proximity operator in Hilbert spaces is introduced and its use in a class of popular optimization algorithms, which go under the general name of proximal algorithms is presented. Chapter 5 is devoted to the theory of duality in convex optimization. The chapter begins with an indispensable tool, a version of the minimax theorem which allows us to characterize saddle points for functionals of specific structure in Banach spaces. We then define and provide the properties of the Fenchel–Legendre conjugate as well as the biconjugate, and illustrate them with numerous examples. To further illustrate the properties of these important transforms, we introduce the concept of the inf-convolution, a concept of interest in its own right in optimization. Having developed the necessary machinery, we present the important contribution of the minimax approach in optimization starting with a detailed presentation of Fenchel’s duality

Preface

| XI

theory and then its generalization to a more general scheme that encompasses Lagrangian duality as well or the augmented Lagrangian method. These techniques are illustrated in terms of various examples and applications. The chapter closes by presenting an important class of numerical methods based on the duality approach and discuss their properties. Chapter 6 is an introduction to the calculus of variations and its applications. After a short motivation, we start with a warm up problem, and in particular the variational theory of the Laplacian. This is a nice opportunity to either introduce or remind (depending on the reader) some important properties of the Laplacian, such as the solvability and the properties (regularity, etc.) of Poisson’s equation or the spectral theory related to this operator. Having spent some time on this important case, we return to the general case, presenting results on the lower semicontinuity of integral functionals in Sobolev spaces, the existence of minimizers and their connection with the Euler–Lagrange equation. We then consider the problem of possible extra regularity of the minmizer beyond that guaranteed by their membership in a Sobolev space (already established by existence). To this end, we provide an introduction to the theory of regularity of solutions variational problems, initiated by De Giorgi, and in particular present some fundamental results concerning Hölder continuity of minimizers or higher weak differentiability. The chapter closes with two important examples, the application of the calculus of variations to semilinear elliptic problems and to quasilinear elliptic problems and in particular problems related to the p-Laplace operator. Chapter 7 considers variational inequalities, a subject of great interest in various fields ranging from mechanics to finance or decision science. As a warm-up, we revisit the problem of the Laplacian but now related to the minimization of the Dirichlet functional on convex closed subsets of a Sobolev space, which naturally lead to a variational inequality for the Laplace operator. Then the Lions–Stampacchia theory for existence of solutions to variational inequalities (not necessarily related to minimization problems) is developed, initially in a Hilbert space setting and with generalizations to variational inequalities of second kind. We then proceed to study the problem of approximation of variational inequalities, presenting the penalization method, as well as the internal approximation method. The chapter closes with an important class of applications, the study of variational inequalities for second order elliptic operators. As the Lions–Stampacchia theory covers equations as a special case, we take up the opportunity to start our presentation with second-order elliptic equations and in this way generalize certain properties that we have seen for the Laplacian in Chapter 6, to more general equations. We then move to variational inequalities and discuss various issues such as solvability, comparison principles, etc., and close the chapter with a study of variational inequalities involving the p-Laplace operator. Chapter 8 introduces critical point theory for functionals in Banach spaces. Having understood quite well the properties of minimizers of functionals and their con-

XII | Preface nection with PDEs, it is interesting to see whether extending this analysis to critical points in general may provide some useful insights. The chapter starts with the presentation of an important class of theorems related to the Ambrosseti–Rabinowitz mountain pass theorem and it generalizations (e. g., the saddle point theorem). This theory provides general criteria under which critical points of certain functionals exist. Through the Euler–Lagrange equation such existence results may prove useful to the study of certain PDEs or variational inequalities. Our applications focus on semilinear and quasilinear PDEs, related to the p-Laplacian. The book finishes with Chapter 9 which deals with a very important part of nonlinear analysis, the study of monotone type operators. During the development of the material up to now, we have seen that monotonicity in some form or another has played a key role to our arguments (such as, e. g., the monotonicity of the Gâteaux derivative of convex functions, or the coercivity of bilinear forms in the Lions–Stampacchia theory, etc.). In this chapter, motivated by these observations, we study the monotonicity properties of operators as a primary issue and not as a derivative of other properties, e. g., convexity. As we shall see, this allows us to extend certain interesting results that we have arrived to via minimization or in general variational arguments for operators that may not necessarily enjoy such a structure. We start our study with monotone operators, then move to maximal monotone operators and then introduce pseudomonotone operators. Our primary interest is in obtaining surjectivity results for monotone- type operators, which will subsequently used in the study of certain PDEs or variational inequalities, but along the way encounter certain interesting results such as, e. g., the Yosida approximation, etc. The theory is illustrated with applications in quasilinear PDEs, differential inclusions, variational inequalities and evolution problems. We wish to acknowledge the invaluable support of various people for various reasons. We must start with acknowledging the support of Dr. Apostolos Damialis, at the time the Mathematics Editor at De Gruyter whose contribution made this book a reality. Then we are indebted to Nadja Schedensack at De Gruyter for her superb handling of all the details and her understanding. We thank Ina Talandienė and Monika Pfleghar for their superb technical assistance. As this material has been tested on audiences at our respective institutions over the years, we wish to thank our students who have helped us shape and organize this material through the interaction with them. We both feel compelled to mention specifically George Domazakis, whose help in proof reading and commenting some of this material as well as with technical issues related to the De Gruyter latex style file has been invaluable, and we deeply thank him for that. We are also thankful to Kyriakos Georgiou for his kind offer for proofreading the manuscript and his useful comments. There are many colleagues and friends the interaction and collaboration with which throughout these years have shaped our interests and encouraged us. We need to mention, Professors N. Alikakos, I. Baltas, D. Drivaliaris, N. Papageorgiou, G. Papagiannis, G. Smyrlis, I. G. Stratis, A. Tsekrekos, S. Xanthopoulos, N. A. Yannacopoulos

Preface

| XIII

and A. Xepapadeas. At a personal level, ANY is indebted to Electra Petracou for making all this feasible through her constant support and devotion and helpful advise. Athens, 2019

D. C. Kravvaritis, NTUA, A. N. Yannacopoulos, AUEB

Contents Preface | IX Notation | XXI 1 1.1 1.1.1 1.1.2 1.1.3 1.1.4 1.1.5 1.1.6 1.1.7 1.2 1.2.1 1.2.2 1.3 1.4 1.5 1.6 1.7 1.7.1 1.8 1.8.1 1.8.2 1.8.3 1.8.4 1.8.5 2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.1.5 2.1.6 2.2

Preliminaries | 1 Fundamentals in the theory of Banach spaces | 1 Linear operators and functionals | 1 The dual X ⋆ | 5 The bidual X ⋆⋆ | 6 Different choices of topology on a Banach space: strong and weak topologies | 8 The strong topology on X | 9 The weak⋆ topology on X ⋆ | 10 The weak topology on X | 14 Convex subsets of Banach spaces and their properties | 22 Definitions and elementary properties | 22 Separation of convex sets and its consequences | 24 Compact operators and completely continuous operators | 28 Lebesgue spaces | 33 Sobolev spaces | 39 Lebesgue–Bochner and Sobolev–Bochner spaces | 48 Multivalued maps | 52 Michael’s selection theorem | 55 Appendix | 57 The finite intersection property | 57 Spaces of continuous functions and the Arzelá–Ascoli theorem | 57 Partitions of unity | 58 Hölder continuous functions | 58 Lebesgue points | 59 Differentiability, convexity and optimization in Banach spaces | 61 Differentiability in Banach spaces | 61 The directional derivative | 61 The Gâteaux derivative | 61 The Fréchet derivative | 64 C 1 functionals | 66 Connections between Gâteaux, Fréchet differentiability and continuity | 67 Vector valued maps and higher order derivatives | 69 General results on optimization problems | 71

XVI | Contents 2.3 2.3.1 2.3.2 2.3.3 2.3.4 2.3.5 2.4 2.5 2.6 2.6.1 2.6.2 2.7 2.7.1 2.7.2 3 3.1 3.1.1 3.1.2 3.1.3 3.1.4 3.1.5 3.2 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5 3.3 3.3.1 3.3.2 3.3.3 3.3.4 3.4 3.4.1 3.4.2 3.5 3.6

Convex functions | 75 Basic definitions, properties and examples | 76 Three important examples: the indicator, Minkowski and support functions | 79 Convexity and semicontinuity | 80 Convexity and continuity | 83 Convexity and differentiability | 83 Optimization and convexity | 87 Projections in Hilbert spaces | 89 Geometric properties of Banach spaces related to convexity | 92 Strictly, uniformly and locally uniformly convex Banach spaces | 92 Convexity and the duality map | 97 Appendix | 101 Proof of Proposition 2.3.20 | 101 Proof of Proposition 2.3.21 | 102 Fixed-point theorems and their applications | 105 Banach fixed-point theorem | 105 The Banach fixed-point theorem and generalizations | 105 Solvability of differential equations | 107 Nonlinear integral equations | 108 The inverse and the implicit function theorems | 109 Iterative schemes for the solution of operator equations | 114 The Brouwer fixed-point theorem and its consequences | 116 Some topological notions | 116 Various forms of the Brouwer fixed-point theorem | 117 Brouwer’s theorem and surjectivity of coercive maps | 119 Application of Brouwer’s theorem in mathematical economics | 120 Failure of Brouwer’s theorem in infinite dimensions | 122 Schauder fixed-point theorem and Leray–Schauder alternative | 123 Schauder fixed-point theorem | 124 Application of Schauder fixed-point theorem to the solvability of nonlinear integral equations | 125 The Leray–Schauder principle | 127 Application of the Leray–Schauder alternative to nonlinear integral equations | 128 Fixed-point theorems for nonexpansive maps | 130 The Browder fixed-point theorem | 131 The Krasnoselskii–Mann algorithm | 134 A fixed-point theorem for multivalued maps | 138 The Ekeland variational principle and Caristi’s fixed-point theorem | 140

Contents | XVII

3.6.1 3.6.2 3.6.3 3.7 3.7.1 3.7.2 4 4.1 4.2 4.2.1 4.2.2

The Ekeland variational principle | 141 Caristi’s fixed-point theorem | 143 Applications: approximation of critical points | 144 Appendix | 145 The Gronwall inequality | 145 Composition of averaged operators | 145

4.3 4.4 4.5 4.6 4.7 4.7.1 4.7.2 4.8 4.8.1 4.8.2

Nonsmooth analysis: the subdifferential | 147 The subdifferential: definition and examples | 147 The subdifferential for convex functions | 151 Existence and fundamental properties | 151 The subdifferential and the right-hand side directional derivative | 154 Subdifferential calculus | 157 The subdifferential and the duality map | 162 Approximation of the subdifferential and density of its domain | 166 The subdifferential and optimization | 168 The Moreau proximity operator in Hilbert spaces | 170 Definition and fundamental properties | 170 The Moreau–Yosida approximation | 174 The proximity operator and numerical optimization algorithms | 177 The standard proximal method | 178 The forward-backward and the Douglas–Rachford scheme | 184

5 5.1 5.2 5.2.1 5.2.2 5.2.3 5.3 5.4 5.5 5.5.1 5.5.2 5.6 5.7 5.7.1 5.7.2 5.7.3

Minimax theorems and duality | 191 A minimax theorem | 191 Conjugate functions | 196 The Legendre–Fenchel conjugate | 196 The biconjugate function | 200 The subdifferential and the Legendre–Fenchel transform | 204 The inf-convolution | 207 Duality and optimization: Fenchel duality | 212 Minimax and convex duality methods | 222 A general framework | 223 Applications and examples | 231 Primal dual algorithms | 235 Appendix | 243 Proof of Proposition 5.1.2 | 243 Proof of Lemma 5.5.1 | 245 Proof of relation (5.61) | 247

6

The calculus of variations | 249

XVIII | Contents 6.1 6.2 6.2.1 6.2.2 6.2.3 6.3 6.3.1 6.3.2 6.4 6.5 6.6 6.6.1 6.6.2 6.6.3 6.7 6.7.1 6.7.2 6.7.3 6.8 6.8.1 6.8.2 6.9 6.9.1 6.9.2 6.9.3 6.9.4 7 7.1 7.2 7.3 7.4 7.5 7.5.1 7.5.2 7.6 7.6.1 7.6.2 7.6.3

Motivation | 249 Warm up: variational theory of the Laplacian | 251 The Dirichlet functional and the Poisson equation | 252 Regularity properties for the solutions of Poisson-type equations | 257 Laplacian eigenvalue problems | 261 Semicontinuity of integral functionals | 268 Semicontinuity in Lebesgue spaces | 269 Semicontinuity in Sobolev spaces | 273 A general problem from the calculus of variations | 274 Differentiable functionals and connection with nonlinear PDEs: the Euler–Lagrange equation | 276 Regularity results in the calculus of variations | 279 The De Giorgi class | 281 Hölder continuity of minimizers | 284 Further regularity | 288 A semilinear elliptic problem and its variational formulation | 294 The case where sub and supersolutions exist | 295 Growth conditions on the nonlinearity | 300 Regularity for semilinear problems | 306 A variational formulation of the p-Laplacian | 307 The p-Laplacian Poisson equation | 307 A quasilinear nonlinear elliptic equation involving the p-Laplacian | 309 Appendix | 314 A version of the Riemann–Lebesgue lemma | 314 Proof of Theorem 6.6.3 | 315 Proof of Theorem 6.6.5 | 327 Proof of generalized Caccioppoli estimates | 332 Variational inequalities | 335 Motivation | 335 Warm up: free boundary value problems for the Laplacian | 336 The Lax–Milgram–Stampacchia theory | 341 Variational inequalities of the second kind | 348 Approximation methods and numerical techniques | 351 The penalization method | 351 Internal approximation schemes | 356 Application: boundary and free boundary value problems | 359 An important class of bilinear forms | 359 Boundary value problems | 362 Free boundary value problems | 369

Contents | XIX

7.6.4 7.7 7.7.1 8 8.1 8.2 8.2.1 8.2.2 8.2.3 8.3 8.3.1 8.3.2 8.3.3 8.4 8.4.1 8.4.2

9 9.1 9.2 9.2.1 9.2.2 9.2.3 9.2.4 9.3 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5 9.3.6 9.3.7 9.4 9.4.1 9.4.2 9.5 9.5.1

Semilinear variational inequalities | 376 Appendix | 378 An elementary lemma | 378 Critical point theory | 379 Motivation | 379 The mountain pass and the saddle point theorems | 380 The mountain pass theorem | 380 Generalizations of the mountain pass theorem | 381 The saddle point theorem | 384 Applications in semilinear elliptic problems | 385 Superlinear growth at infinity | 386 Nonresonant semilinear problems with asymptotic linear growth at infinity and the saddle point theorem | 390 Resonant semilinear problems and the saddle point theorem | 394 Applications in quasilinear elliptic problems | 399 The p-Laplacian and the mountain pass theorem | 399 Resonant problems for the p-Laplacian and the saddle point theorem | 404 Monotone-type operators | 409 Motivation | 409 Monotone operators | 410 Monotone operators, definitions and examples | 410 Local boundedness of monotone operators | 411 Hemicontinuity and demicontinuity | 413 Surjectivity of monotone operators and the Minty–Browder theory | 414 Maximal monotone operators | 417 Maximal monotone operators definitions and examples | 417 Properties of maximal monotone operators | 418 Criteria for maximal monotonicity | 421 Surjectivity results | 422 Maximal monotonicity of the subdifferential and the duality map | 427 Yosida approximation and applications | 429 Sum of maximal monotone operators | 439 Pseudomonotone operators | 442 Pseudomonotone operators, definitions and examples | 442 Surjectivity results for pseudomonotone operators | 444 Applications of monotone-type operators | 446 Quasilinear elliptic equations | 446

XX | Contents 9.5.2 9.5.3 9.5.4 9.5.5

Semilinear elliptic inclusions | 448 Variational inequalities with monotone-type operators | 450 Gradient flows in Hilbert spaces | 451 The Cauchy problem in evolution triples | 455

Bibliography | 463 Index | 469

Notation The following notation is used throughout this book.

General notation –

– – – – –

– – – – –

–

– –

We will use the notation A ⊂ B denoting both strict or nonstrict inclusion. If we want to emphasize that an inclusion is strict, we will denote it by A ⊊ B. We will also use the notation A \ B for the difference of two sets. 1A is the indicator function of a set A defined by 1A (s) = 1 if s ∈ A and 1A (s) = 0 otherwise. IA is the convex indicator function of a set A defined by IA (s) = 0 is s ∈ A and IA (s) = +∞ otherwise. The standard notation for the set of real numbers, ℝ and the set of natural numbers ℕ is used. We will also use the notation ℝ+ = {x ∈ ℝ : x ≥ 0}. ℝd = {(x1 , . . . , xd ) : xi ∈ ℝ, i = 1, . . . , d}. x = (x1 , . . . , xd ) or z = (z1 , . . . , zd ) is used to denote the elements of ℝd , while |x| = (|x1 |2 + ⋅ ⋅ ⋅ + |xd |2 )1/2 is used to denote the Euclidean norm for ℝd , and x ⋅ x the usual inner product. By B(xo , R) := {x ∈ ℝd : |x − xo | ≤ R} we denote the ball in ℝd , with center x0 and radius R, and by Q(x0 , R) we denote the cube centered at x0 of side R. 𝒟 ⊂ ℝd , an open domain in ℝd . Unless explicitly stated otherwise, 𝒟 is considered to be bounded. 𝜕𝒟, the boundary of 𝒟, assumed to be sufficiently smooth, e. g., Lipschitz, unless explicitly stated otherwise. 𝕄m×n is the set of m × n matrices. If A ∈ 𝕄m×n , by AT we denote its transpose, and in the case m = n by det(A) we denote its determinant. Throughout the most part of this book, X and Y are used for denoting generic Banach spaces. The elements of the space X are denoted by x or z (using if necessary additional demarcations, e. g., xo , zo , etc.), while the elements of Y are denoted by y (or using if necessary additional demarcations). Similarly the duals of the above are denoted by X ⋆ and Y ⋆ , respectively. The elements of X ⋆ are denoted by x⋆ or z⋆ (using if necessary additional demarcations, e. g., x⋆o , z⋆o , etc.), while the elements of Y ⋆ are denoted by y⋆ (or using if necessary additional demarcations). The duality pairing between two spaces X, X ⋆ is denoted by ⟨⋅, ⋅⟩X ⋆ ,X , with the subscript dropped when no confusion is possible. Hilbert spaces are denoted by H, and the inner product by ⟨⋅, ⋅⟩H (with the subscript dropped when no confusion may arise).

https://doi.org/10.1515/9783110647389-202

XXII | Notation –

Operators, linear or nonlinear, between two spaces X and Y are denoted by sans serif fonts, e. g., A, B, L, etc. The action of a linear operator A on an element x ∈ X is denoted by Ax, while for a nonlinear operator it is denoted by A(x). – f : X → Y is used to denote maps in general between two topological spaces (including linear functionals), whereas F is used to denote possible nonlinear functionals F : X → ℝ. – Df is used to denote the Gâteaux or Fréchet derivative of a map f : X → Y. – If f : 𝒟 ⊂ ℝd → ℝ, we will use the notation ∇f = ( 𝜕x𝜕 f , . . . , 𝜕x𝜕 f ) for its gradient.

–

–

– – –

1

d

If f = (f1 , . . . , fd ) : 𝒟 ⊂ ℝd → ℝd , we will use the notation divf = 𝜕x𝜕 f1 + ⋅ ⋅ ⋅ 𝜕x𝜕 fd for 1 d its divergence. By f 󸀠 , we denote the derivative of a map f : [0, T] ⊂ ℝ → X (the case X = ℝ included). Please note that we denote the derivative by f 󸀠 and NOT f 󸀠 ! The ordinary prime is used to denote an alternative element of any set. C ⊂ X is used to denote a convex subset of X. φ : X → ℝ ∪ {+∞} is used to denote convex functions. arg minz∈C φ(z) := {x ∈ C : φ(x) = infz∈C φ(x)}, is the set of minimizers of the function φ : X → ℝ ∪ {+∞} over C ⊂ X. By cont(φ), we denote the set of points where φ is finite and continuous.

More specialized notation – A linear or nonlinear operator, D(A) it domain, Gr(A) its graph, N(A) its kernel, R(A) its range, Definition 1.1.1, Section 1.1.1. – ℬ(X, Y) the space of linear bounded operators A : X → Y, Definition 1.1.1, Section 1.1.1. – ℒ(X, Y) the space of linear continuous operators A : X → Y, Definition 1.1.1, Section 1.1.1. – ℒ(X) the space of linear continuous operators A : X → X, Definition 1.1.1, Section 1.1.1. – X ≃ Y, isometric spaces, Definition 1.1.3, Section 1.1.1. – X ⋆ , the dual space of a normed space X, Definition 1.1.12, Section 1.1.2. – X ⋆⋆ , the bidual space of a normed space X, Definition 1.1.19, Section 1.1.3. ⋆ – J : X → 2X the duality map, Definition 1.1.16, Section 1.1.2. – j : X → X ⋆⋆ , canonical embedding, Theorem 1.1.22, Section 1.1.3. – BX (0, 1) = BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1}, the closed unit ball of X, Section 1.1.4. – BX (0, 1) = BX (0, 1) = {x ∈ X : ‖x‖X < 1}, the open unit ball of X. – SX (0, 1) = {x ∈ X : ‖x‖X = 1}, the unit sphere of X. – xn → x, strong convergence in a Banach space X, Definition 1.1.28, Section 1.1.5.2. – σ(X ⋆ , X) the weak⋆ topology on X ⋆ , Definition 1.1.32, Section 1.1.6.1. – Xw⋆⋆ is X ⋆ equipped with the weak⋆ topology on X ⋆ , Definition 1.1.32, Section 1.1.6.1. ⋆

– x⋆n ⇀ x⋆ , weak⋆ convergence in X ⋆ , Definition 1.1.38, Section 1.1.6.1.

More specialized notation

– – – – – – – – – – – – – – – – – – – –

| XXIII

σ(X, X ⋆ ) the weak topology on X, Definition 1.1.42, Section 1.1.7.1. Xw is X equipped with the weak topology on X, Definition 1.1.42, Section 1.1.7.1. xn ⇀ x, weak convergence in X, Definition 1.1.50, Section 1.1.7.2. A⋆ : Y ⋆ → X ⋆ , the adjoint of the linear operator A : X → Y, Definition 1.1.18, Section 1.1.2. index(A) = dim(N(A)) − dim(Y/R(A)) the index of a Fredholm operator, Definition 1.3.5, Section 1.3. ρ(A) the resolvent set of a bounded linear operator A, Definition 1.3.11, Section 1.3. σ(A) the spectrum of a bounded linear operator A, Definition 1.3.11, Section 1.3. ℬ(𝒟) the Borel σ-algebra on 𝒟 ⊂ ℝd , Section 1.4. μℒ the Lebesgue measure on ℝd , Section 1.4. The Lebesgue measure of a set 𝒟 ⊂ ℝd is denoted either by μℒ (𝒟) or simply by |𝒟| when there is no risk of confusion. Lp (𝒟) Lebesgue space, Definition 1.4.1, Section 1.4. For any p ∈ ℕ, p > 1 we denote by p∗ the conjugate p1 + p1∗ = 1. If p = 1, then p∗ = ∞, whereas if p = ∞, p∗ = 1. ⋆ (Lp (𝒟))⋆ ≃ Lp (𝒟). osc(xo , r) := supx∈B(xo ,r) f (x) − infx∈B(xo ,r) f (x), the oscillation of a function f : ℝd → ℝ on the ball B(x0 , r), Definition 1.8.7, Section 1.8.4. ffl 1 ´ d 1 A u(x)dx := |A| A u(x)dx, for every A ⊂ ℝ , u ∈ L (A) is the mean value of the function f on A, Definition 1.8.7, Section 1.8.4. ℳ(D) the set of Radon measures on 𝒟 ⊂ ℝd , Definition 1.4.3, Section 1.4. Cc (𝒟) := {ϕ ∈ C(𝒟) : ϕ(x) = 0, ∀ x ∈ 𝒟 \ 𝒟o , 𝒟o ⊂ 𝒟, compact}, Section 1.4. Cc∞ (𝒟) be the set of infinitely continuously differentiable functions on 𝒟 with compact support, Section 1.5. C 0,α (𝒟) the space of Hölder continuous functions f : 𝒟 ⊂ ℝd → ℝ with Hölder exponent α ∈ (0, 1], Definition 1.8.5, Section 1.8.4. W k,p (𝒟) the k-Sobolev space of order p, Definition 1.5.3, Section 1.5. k,p Wloc (𝒟) Definition 1.5.3, Section 1.5.

– W0k,p (𝒟) the k-Sobolev space of order p of functions of vanishing trace on 𝜕𝒟, Definition 1.5.6, Section 1.5. ⋆ – W −k,p (𝒟) = (W k,p (𝒟))⋆ , Definition 1.5.6, Section 1.5. dp is the critical exponent for the Sobolev embedding Lsp (𝒟) 󳨅→ W 1,p (𝒟), – sp := d−p p ≤ d. – f : X → 2Y a multivalued map, Section 1.7. – f u (A) the upper inverse of a set A for a multivalued map f : X → 2Y , Definition 1.7.1, Section 1.7. – f ℓ (A) the lower inverse of a set A for a multivalued map f : X → 2Y , Definition 1.7.1, Section 1.7. – DF(x; h) is the directional derivative of F : X → ℝ at x ∈ X in the direction h ∈ X, Definition 2.1.1, Section 2.1.1.

XXIV | Notation – DF(x) the Gâteaux derivative of F : X → ℝ at x ∈ X, Definition 2.1.2, Section 2.1.2. If it exists, the Fréchet derivative is denoted the same, Definition 2.1.6, Section 2.1.3. – C ⊂ X is in general a convex set, Definition 1.2.1, Section 1.2.1. – core(C) = {x ∈ C : ∀ z ∈ SX , ∃ ϵ > 0, such that x + δz ∈ C, ∀ δ ∈ [0, ϵ]} is the core or algebraic interior of a convex set, Definition 1.2.5, Section 1.2.1. – convA is the set of all convex combinations from A ⊂ X, Definition 1.2.16, Section 1.2.2. – φ : X → ℝ ∪ {+∞} is a convex function, Definition 2.3.2, Section 2.3.1. – dom φ = {x ∈ X : φ(x) < +∞} is the domain of a convex function, Definition 2.3.4, Section 2.3.1. – epi φ = {(x, λ) ∈ X × ℝ : φ(x) ≤ λ} is the epigraph of a convex function, Definition 2.3.4, Section 2.3.1. – IC (x) is the indicator function for a convex set, Definition 2.3.9, Section 2.3.2.1. – σC : X ⋆ → ℝ ∪ {+∞} is the support function of a convex set C ⊂ X, Definition 2.3.12, Section 2.3.2.3. – PC : H → C, the projection operator to the closed convex subset C ⊂ H, Definition 2.5.1, Section 2.5. – E ⊥ is the orthogonal complement of closed linear subspace E ⊂ H, Definition 2.5.4, Section 2.5. ⋆ – 𝜕φ : X → 2X is the subdifferential of a convex function, Definition 4.1.1, Section 4.1. – NC (x) is the normal cone of C ⊂ X at x ∈ X, Definition 4.1.5, Section 4.1. – D+ φ(x, h) is the right-hand side directional derivative of the function φ at x ∈ X in the direction h ∈ X, Definition 4.2.6, Section 4.2.2. – D− φ(x, h) is the left-hand side directional derivative of the function φ at x ∈ X in the direction h ∈ X, Example 4.2.9, Section 4.2.2. – proxφ : H → H the proximity operator related to the convex function φ : H → ℝ ∪ {+∞}, Definition 4.7.1, Section 4.7.1. – φλ : H → ℝ the Moreau–Yosida approximation of a convex function φ, Definition 4.7.3, Section 4.7.2. – φ⋆ : X ⋆ → ℝ ∪ {+∞} the Fenchel–Legendre conjugate of the function φ : X → ℝ ∪ {+∞}, Definition 5.2.1, Section 5.2.1. – φ = φ1 ◻φ2 the inf-convolution of φ1 , φ2 : X → ℝ ∪ {+∞}, Definition 5.3.1, Section 5.3. – F ⋆⋆ : X → ℝ ∪ {+∞} the Fenchel–Legendre biconjugate of φ : X → ℝ ∪ {+∞}, Definition 5.2.11, Section 5.2.2. – 𝔸(F) := {g : X → ℝ : g continuous and affine, g ≤ F}, is the set of continuous affine functions minorizing F, Definition 5.2.13, Section 5.2.2. – F Γ the Γ-regularization of F, Definition 5.2.13, Section 5.2.2. – Rλ , λ > 0 the resolvent operator for the nonlinear operator A, Definition 9.3.21, Section 9.3.5.2.

More specialized notation

| XXV

– Aλ , λ > 0 the resolvent operator for the nonlinear operator A, Definition 9.3.21, Section 9.3.5.2.

1 Preliminaries In this introductory chapter, we collect (mostly without proofs but with a number of illustrative examples) the fundamental notions from linear functional analysis, convexity, Lebesgue, Sobolev, Sobolev–Bochner spaces and multivalued mappings, that are essential in proceeding to the main topic of this book which is nonlinear analysis and its various applications. For detailed presentations of the material covered in this chapter, we refer to, e. g., [1, 5, 4, 28, 48, 109].

1.1 Fundamentals in the theory of Banach spaces Most of this book will concern Banach spaces, i. e., complete normed spaces. In this section, we recall some of their fundamental properties.

1.1.1 Linear operators and functionals Linear mappings between normed spaces and functionals play an important role in (linear) functional analysis. Definition 1.1.1 (Operators). Let X, Y be two normed spaces. (i) A map A : D(A) ⊂ X → Y (not necessarily linear) is called an operator (or functional if Y = ℝ). We further define the sets D(A) := {x ∈ X : A(x) ∈ Y},

Gr(A) := {(x, A(x)) : x ∈ D(A)} ⊂ X × Y, N(A) := {x ∈ X : A(x) = 0},

R(A) := {y ∈ Y : ∃ x ∈ X, for which A(x) = y} = A(X),

called the domain, graph, kernel and range of A, respectively. (ii) If A is such that A(λ1 x1 + λ2 x2 ) = λ1 A(x1 ) + λ2 A(x2 ), it is called a linear operator (or linear functional, denoted by f if Y = ℝ). (iii) The linear operator A : X → Y is called bounded if sup{‖A(x)‖Y : x ∈ X, ‖x‖X ≤ 1} < ∞. The set of bounded linear operators between X and Y is denoted by ℬ(X, Y). (iv) The set of continuous linear operators between X and Y is denoted by ℒ(X, Y). In the special case where Y = X, we will use the simplified notation ℒ(X) instead of ℒ(X, X). The following theorem collects some useful properties of linear operators and functionals over normed spaces. https://doi.org/10.1515/9783110647389-001

2 | 1 Preliminaries Theorem 1.1.2 (Elementary properties of linear operators and functionals). Let A : X → Y be a linear mapping. (i) The following are equivalent: (a) A is continuous at 0, (b) A is continuous, (c) A is uniformly continuous, (d) A is Lipschitz, (e) A is bounded. It therefore holds that ℬ(X, Y) = ℒ(X, Y). (ii) It holds that 󵄩 󵄩 ‖A‖ℒ(X,Y) := sup{󵄩󵄩󵄩A(x)󵄩󵄩󵄩Y : x ∈ X, ‖x‖X ≤ 1} 󵄩 󵄩 = sup{󵄩󵄩󵄩A(x)󵄩󵄩󵄩Y : x ∈ X, ‖x‖X = 1} ‖A(x)‖Y = sup{ : x ∈ X, x ≠ 0} ‖x‖X 󵄩 󵄩 = inf{c > 0 : 󵄩󵄩󵄩A(x)󵄩󵄩󵄩Y ≤ c‖x‖X , x ∈ X}. In fact, ℒ(X, Y) is a normed space with norm ‖ ⋅ ‖ℒ(X,Y) , and if Y is a Banach space, ℒ(X, Y) is a Banach space also. An important special class of continuous linear functionals are isomorphisms and isometries. Definition 1.1.3 (Isomorphisms and isometries). Let X, Y be normed spaces. (i) A linear operator A : X → Y which is 1–1 and onto (bijection), and such that A and A−1 are bounded is called an isomorphism. If an isomorphism A exists between the normed spaces X and Y, the spaces are called isomorphic, and this is denoted by X ≃ Y. (ii) A linear operator A : X → Y which is 1–1 and onto (bijection), and such that ‖A(x)‖Y = ‖x‖X , for every x ∈ X, is called an isometry. If an isometry A exists between the normed spaces X and Y, the spaces are called isometric. Clearly, an isometry is an isomorphism. An isomorphism leaves topological properties between the two spaces invariant, while an isometry leaves distances invariant. One easily sees that if X1 , X2 , X3 are normed spaces and X1 ≃ X2 , and X2 ≃ X3 , then X1 ≃ X3 , with a similar result holding for isometric spaces. This shows that the relation X ≃ Y is an equivalence relation in the class of normed spaces. Furthermore, one can show that a linear 1–1 and onto (bijective) operator between two normed spaces is an isomorphism if and only if there exists c1 , c2 > 0 such that c1 ‖x‖X ≤ ‖A(x)‖Y ≤ c2 ‖x‖X , for every x ∈ X, while an isometry maps the unit ball of X to the unit ball of Y, thus preserving the geometry of the spaces. As we will see quite often, when two normed spaces are isometric we will often “identify” them in terms of this isometry. One of the fundamental results concerning linear spaces and linear functionals is the celebrated Hahn–Banach theorem. We will return to this theorem very often, in any of its multiple versions, especially its versions related to separation of convex

1.1 Fundamentals in the theory of Banach spaces |

3

subsets of normed spaces. Here, we present its analytic form and will return to its equivalent geometric forms (in terms of separation theorems) in Section 1.2, where we study convexity and its properties in detail. In its analytic form, the Hahn–Banach theorem deals with the extension of a linear functional defined on a subspace X0 of a (real) linear space X to the whole space. Theorem 1.1.4 (Hahn–Banach I). Let X0 be a subspace of a (real) linear space X and let p : X → ℝ be a positively homogeneous subadditive functional on X, i. e., a functional with the properties (a) p(λ x) = λ p(x), for every λ > 0 and x ∈ X, and (b) p(x1 + x2 ) ≤ p(x1 )+p(x2 ), for every x1 , x2 ∈ X. If f0 : X0 → ℝ is a linear functional such that f0 (x) ≤ p(x) for every x ∈ X0 , then, there exists a linear functional f : X → ℝ such that f (x) = f0 (x) on X0 and f (x) ≤ p(x) for every x ∈ X. As a corollary of the above, we may obtain the following version of the Hahn– Banach theorem for normed spaces (simply apply the above for the choice p(x) = ‖f0 ‖ℒ(X0 ,ℝ) ‖x‖X ). Theorem 1.1.5 (Hahn–Banach II). If X is a (real) normed linear space, X0 is a subspace and f0 : X0 → ℝ is a continuous linear functional on X0 , then there exists a continuous linear extension f : X → ℝ, of f0 such that ‖f0 ‖ℒ(X0 ,ℝ) = ‖f ‖ℒ(X,ℝ) . A second fundamental theorem concerning bounded linear operators in Banach spaces is the Banach–Steinhaus theorem or principle of uniform boundedness, which provides a connection between the following two different types of boundedness and convergence in the normed space ℒ(X, Y). Definition 1.1.6 (Norm and pointwise bounded families). Let X, Y be normed spaces. (i) A family of operators {Aα : α ∈ ℐ } ⊂ ℒ(X, Y) is called norm bounded if ‖Aα ‖ℒ(X,Y) < ∞ for every α ∈ ℐ , and pointwise bounded if for every x ∈ X (fixed) ‖Aα (x)‖Y < ∞ for every α ∈ ℐ . (ii) A sequence {An : n ∈ ℕ} ⊂ ℒ(X, Y) is called norm convergent to A ∈ ℒ(X, Y) if ‖An − A‖ℒ(X,Y) → 0 as n → ∞, and pointwise convergent to A ∈ ℒ(X, Y) if An (x) → A(x) for every x ∈ X. The Banach–Steinhaus theorem states the following. Theorem 1.1.7 ( Banach–Steinhaus). Let X, Y be Banach spaces.1 Consider a family of continuous linear mappings 𝒜 := {Aα : X → Y : α ∈ ℐ } ⊂ ℒ(X, Y), (not necessarily countable). 𝒜 is norm bounded if and only if it is pointwise bounded. The principle of course remains valid for linear functionals, upon setting Y = ℝ. 1 Y may in fact simply be a normed space; see, e. g., the principle of uniform boundedness Theorem 14.1 [48].

4 | 1 Preliminaries In other words, if we have a family of linear operators {Aα : X → Y : α ∈ ℐ } ⊂ ℒ(X, Y) such that supα∈ℐ ‖Aα x‖Y < ∞ for every x ∈ X, then supα∈ℐ ‖Aα ‖ℒ(X,Y) < ∞, i. e., there exists a constant c > 0 such that ‖Aα x‖Y ≤ c‖x‖X for every x ∈ X and α ∈ ℐ ,

with the constant c being independent of both x and α. The Banach–Steinhaus theorem has important implications for operator theory. The following example shows us how to use it to define linear bounded operators as the pointwise limit of a sequence of bounded operators, a task often fundamental in numerical analysis and approximation theory. Example 1.1.8 (Definition of operators as pointwise limits). Consider a sequence of linear bounded operators {An : n ∈ ℕ} ⊂ ℒ(X, Y) with the property: For every x ∈ X, there exists yx ∈ Y such that An (x) → yx in Y. Then these pointwise limits define an operator A : X → Y by A(x) := yx = limn An (x) for every x ∈ X, which is linear and bounded, and ‖A‖ℒ(X,Y) ≤ lim inf ‖An ‖ℒ(X,Y) .

(1.1)

n→∞

By the Banach–Steinhaus theorem there exists a c > 0 such that ‖An x‖ ≤ c ‖x‖, for all x ∈ X, n ∈ ℕ. We pass to the limit as n → ∞ and since An (x) → A(x) in Y for every x ∈ X we have that ‖An (x)‖Y → ‖A(x)‖Y as n → ∞, therefore, ‖A(x)‖Y ≤ c ‖x‖X , for every x ∈ X from which follows that A ∈ ℒ(X, Y). Furthermore, taking the limit inferior on the inequality ‖An (x)‖Y ≤ ‖An ‖ℒ(X,Y) ‖x‖X for every x, we have that 󵄩󵄩 󵄩 󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩A(x)󵄩󵄩󵄩Y = lim 󵄩A (x)󵄩󵄩󵄩Y = limninf󵄩󵄩󵄩An (x)󵄩󵄩󵄩Y ≤ limninf ‖An ‖ℒ(X,Y) ‖x‖X , n 󵄩 n where upon taking the supremum over all ‖x‖ ≤ 1 we obtain (1.1).

∀ x ∈ X, ◁

The other fundamental result concerning bounded linear operators between Banach spaces is the open mapping theorem, which is again a consequence of the Baire category theorem (for a proof see, e. g., [28]). Theorem 1.1.9 (Open mapping). Let X, Y be Banach spaces and A : X → Y a linear bounded operator, which is surjective (onto). Then A is also open, i. e., the image of every open set in X under A is also open in Y. Example 1.1.10. Let A : X → Y be a linear bounded operator which is onto and one to one (bijective). Then A−1 is a bounded operator. By the open mapping theorem if 𝒪 ⊂ X is open, then A(𝒪) ⊂ Y is open. Since A(𝒪) = (A−1 )−1 (𝒪), it follows that A−1 maps open sets to open sets hence, it is continuous, and being linear it is bounded. ◁ An important consequence of the open mapping theorem is the closed graph theorem. Theorem 1.1.11 (Closed graph). Let X, Y be Banach spaces and A : X → Y be a linear operator (not necessarily bounded). A is continuous (hence, bounded) if and only if Gr(A) is closed.

1.1 Fundamentals in the theory of Banach spaces | 5

Using the open mapping theorem, we can show that if Gr(A) is closed in X × Y, the operator A is continuous, a fact which is not true for nonlinear operators.

1.1.2 The dual X ⋆ An important concept is that of the dual space, which can be defined in terms of continuous linear mappings from X to ℝ (functionals). Definition 1.1.12 (Dual space). Let X be a normed space. The space ℒ(X, ℝ) of all continuous linear functionals from X to ℝ is called the (topological) dual space of X, and is denoted by X ⋆ , its elements by x⋆ and their action on elements x ∈ X, defines a duality pairing between X and X ⋆ by x⋆ (x) =: ⟨x⋆ , x⟩X ⋆ ,X , often denoted for simplicity as ⟨⋅, ⋅⟩. Proposition 1.1.13 (The dual as a normed space). Let X be a normed space. The dual space X ⋆ is a Banach space2 when equipped with the norm3 ‖x⋆ ‖X ⋆ = sup{|⟨x⋆ , x⟩| : ‖x‖X ≤ 1}, where ⟨⋅, ⋅⟩ is the duality pairing between the two spaces, ⟨x⋆ , x⟩ = x⋆ (x), for every x ∈ X, x⋆ ∈ X. Often it is convenient to understand the dual space of a normed linear space in terms of its isometry to a different normed linear space. One of the most important results of this type holds in the special case where X is a Hilbert space H. In this case, one has the celebrated Riesz representation theorem. Theorem 1.1.14 (Riesz). Let H be a Hilbert space with inner product ⟨⋅, ⋅⟩H . For every x⋆ ∈ H ⋆ , there exists a unique xo ∈ H such that x⋆ (x) = ⟨x⋆ , x⟩ = ⟨xo , x⟩H for every x ∈ X. Moreover, ‖x⋆ ‖H ⋆ = ‖xo ‖H . For more general Banach spaces, one may still identify the dual space with a concrete vector space. Example 1.1.15 (The dual space of sequence spaces). Let X = ℓp = {x = {xn : n ∈ ℕ} : xn ∈ ℝ, ∑n∈ℕ |xn |p < ∞}, p ∈ (1, ∞), equipped with the norm ‖x‖ℓp = {∑n∈ℕ |xn |p }1/p . ⋆ Its dual space is identified with the sequence space ℓp with p1 + p1⋆ = 1. ◁ Similar constructions can be done for other spaces; see, e. g., Sections 1.4 and 1.5. Since the dual space X ⋆ is the space of continuous linear functionals on X, the Hahn–Banach theorem plays an important role in the study of the dual space. One of the many instances where this shows up is in the construction of the duality map from X to X ⋆ , the existence of which follows by an application of the Hahn–Banach theorem. 2 even if X is not.

3 or equivalently, ‖x⋆ ‖X ⋆ = supx∈X\{0}

|⟨x⋆ ,x⟩| ‖x‖X

= supx∈X, ‖x‖X =1 |⟨x⋆ , x⟩|.

6 | 1 Preliminaries Definition 1.1.16 (The duality map). The duality map J : X → X ⋆ is the set valued nonlinear map x 󳨃→ J(x), where 󵄩 󵄩2 J(x) := {x⋆ ∈ X ⋆ : ⟨x⋆ , x⟩ = ‖x‖2X = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ }. Example 1.1.17. For every x ∈ X, it holds that 󵄩 󵄩 󵄨 󵄨 ‖x‖X = sup{󵄨󵄨󵄨⟨x⋆ , x⟩󵄨󵄨󵄨 : x⋆ ∈ X ⋆ , 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ≤ 1} 󵄨 󵄨 󵄩 󵄩 = max{󵄨󵄨󵄨⟨x⋆ , x⟩󵄨󵄨󵄨 : x⋆ ∈ X ⋆ , 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ≤ 1}. The fact that the supremum is attained is very important, since in general the supremum in the definition of the norm ‖⋅‖X ⋆ may not be attained (unless certain conditions are satisfied for X, e. g., if X is a reflexive Banach space, see [28]). ◁ At this point, we recall the notion of the adjoint operator. Let X, Y be Banach spaces, with duals X ⋆ , Y ⋆ , respectively. We will first define the concept of the adjoint or dual operator. Definition 1.1.18 (Adjoint of an operator). Let A : X → Y be a linear operator between two Banach spaces. The adjoint operator A⋆ : Y ⋆ → X ⋆ is defined by ⟨A⋆ (y⋆ ), x⟩X ⋆ ,X = ⟨y⋆ , A(x)⟩Y ⋆ ,Y , for every x ∈ X, y⋆ ∈ Y ⋆ . The adjoint or dual operator generalizes in infinite dimensional Banach spaces the concept of the transpose matrix. If X, Y are Hilbert spaces, the term adjoint operator is used, and upon the identifications X ⋆ ≃ X and Y ⋆ ≃ Y the definition simplifies to ⟨A⋆ y, x⟩X = ⟨y, Ax⟩Y for every x ∈ X, y ∈ Y. An operator A : X → X, where X = H is Hilbert space, such that A⋆ = A is called a self-adjoint operator. For such operators, one may show that the operator norm admits the representation4 ‖A‖ = sup‖x‖=1 |⟨Ax, x⟩|. 1.1.3 The bidual X ⋆⋆ Definition 1.1.19 (Bidual of a normed space). The dual space of X ⋆ is called the bidual and is denoted by X ⋆⋆ := (X ⋆ )⋆ . Let us reflect a little bit on the nature of elements of the bidual space X ⋆⋆ . They are functionals from X ⋆ → ℝ, i. e., continuous linear mappings x⋆⋆ : X ⋆ → ℝ that 4 If m = sup‖x‖=1 |⟨Ax, x⟩|, it is straightforward to show that m ≤ ‖A‖. For the reverse inequality, express ⟨Ax, z⟩ = 41 (⟨A(x+z), x+z⟩−⟨A(x−z), x−z⟩) ≤ 41 (|⟨A(x+z), x+z⟩|+|⟨A(x−z), x−z⟩|) ≤ m4 (‖x+z‖2 +‖x−z‖2 ) = m (‖x‖2 2

+ ‖z‖2 ) and choose z =

‖x‖ Ax. ‖Ax‖

1.1 Fundamentals in the theory of Banach spaces | 7

take a functional x⋆ : X → ℝ and map it into a real number x⋆⋆ (x⋆ ). This allows us to define a duality pairing between X ⋆ and X ⋆⋆ by ⟨x⋆⋆ , x⋆ ⟩X ⋆⋆ ,X ⋆ = x⋆⋆ (x⋆ ) for every x⋆⋆ ∈ X ⋆⋆ and every x⋆ ∈ X ⋆ . Clearly, this is a different duality pairing than the one defined between X and X ⋆ . If we were too strict on notation, we should use ⟨⋅, ⋅⟩X ⋆ ,X for the latter and ⟨⋅, ⋅⟩X ⋆⋆ ,X ⋆ for the former, but this would make notation unnecessarily heavy and we will often just use ⟨⋅, ⋅⟩ for both when there is no risk of confusion. For instance, as x, x⋆ , x⋆⋆ will be used for denoting elements of X, X ⋆ , X ⋆⋆ , respectively, it will be clear that ⟨x⋆ , x⟩ corresponds to the duality pairing between X and X ⋆ , whereas ⟨x⋆⋆ , x⋆ ⟩ corresponds to the duality pairing between X ⋆ and X ⋆⋆ . Proposition 1.1.20 (The bidual as a normed space). Let X be a normed space. The bidual space X ⋆⋆ of X, is a Banach space endowed with the norm 󵄩󵄩 ⋆⋆ 󵄩󵄩 󵄩 ⋆󵄩 ⋆⋆ ⋆ 󵄩󵄩x 󵄩󵄩X ⋆⋆ = sup{⟨x , x ⟩X ⋆⋆ ,X ⋆ : 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ ≤ 1}. X

⋆⋆

In the following example, we construct an important linear mapping from X to .

Example 1.1.21. Consider any x ∈ X. For any x⋆ ∈ X ⋆ , define the mapping x⋆ 󳨃→ ⟨x⋆ , x⟩X ⋆ ,X , which is a mapping from X ⋆ to ℝ. Call this mapping fx : X ⋆ → ℝ. Since |fx (x⋆ )| = |⟨x⋆ , x⟩| ≤ ‖x⋆ ‖X ⋆ ‖x‖X it is easy to see that fx is a bounded mapping, and ‖fx ‖ℒ(X ⋆ ,ℝ) ≤ ‖x‖X . In fact, ‖fx ‖ℒ(X ⋆ ,ℝ) = ‖x‖X . ◁ The above example introduces the following important concept. For any x ∈ X, the mapping x⋆ 󳨃→ ⟨x⋆ , x⟩X ⋆ ,X , can be considered as a linear mapping fx : X ⋆ → ℝ, which by the fundamental estimate |⟨x⋆ , x⟩X ⋆ ,X | ≤ ‖x⋆ ‖X ⋆ ‖x‖X is continuous, therefore, it is a linear functional on X ⋆ hence, an element of (X ⋆ )⋆ = X ⋆⋆ . Note that by definition fx (x⋆ ) = ⟨x⋆ , x⟩, and we may interpret fx (x⋆ ) as the duality pairing between X ⋆ and X ⋆⋆ , so that fx (x⋆ ) = ⟨fx , x⋆ ⟩X ⋆⋆ ,X ⋆ ; hence, fx may be interpreted as this element of X ⋆⋆ for which it holds that ⟨fx , x⋆ ⟩X ⋆⋆ ,X ⋆ = ⟨x⋆ , x⟩X ⋆ ,X ,

∀ x⋆ ∈ X ⋆ .

(1.2)

For this mapping, it holds that ‖fx ‖X ⋆⋆ = ‖x‖X . Now define the map j : X → X ⋆⋆ by j(x) = fx for any x ∈ X. Note that by (1.2) for the map j : X → X ⋆⋆ it holds that ⟨j(x), x⋆ ⟩X ⋆⋆ ,X ⋆ = ⟨x⋆ , x⟩X ⋆ ,X , for every x⋆ ∈ X ⋆ . Since ‖fx ‖X ⋆⋆ = ‖j(x)‖X ⋆⋆ = ‖x‖X , the mapping j is one to one and conserves the norm. If we define Y := R(j) = {x⋆⋆ ∈ X ⋆⋆ : ∃ x ∈ X, j(x) = x⋆⋆ }, then obviously j : X → Y is onto and since it is an isometry we may identify X with Y = R(j). Note that in general j : X → X ⋆⋆ is not onto. In terms of the mapping j : X → X ⋆⋆ , the vector space X can be considered as a subspace of its bidual space X ⋆⋆ , and for this reason the mapping is called the canonical embedding of X into X ⋆⋆ . The above discussion leads to the following fundamental theorem. Theorem 1.1.22 (Canonical embedding of X into X ⋆⋆ ). Let X be a normed space. Then there exists a linear operator j : X → X ⋆⋆ , with ⟨j(x), x⋆ ⟩X ⋆⋆ ,X ⋆ = ⟨x⋆ , x⟩X ⋆ ,X , for all

8 | 1 Preliminaries x⋆ ∈ X ⋆ , such that ‖j(x)‖X ⋆⋆ = ‖x‖X for all x ∈ X. This operator is called the canonical embedding of X into X ⋆⋆ and is not in general surjective (onto), but allows the identification of X with a closed subspace of X ⋆⋆ , in terms of an isomorphic isometry. For certain types of normed spaces, the canonical embedding may be surjective (onto). Definition 1.1.23 (Reflexive space). If j : X → X ⋆⋆ is surjective, i. e., j(X) = X ⋆⋆ , then the space X is called reflexive. For a reflexive space, j is an isometry between X and X ⋆⋆ ; hence, X and X ⋆⋆ are isomorphic isometric and can be identified in terms of the equivalent relation X ≃ X ⋆⋆ . By the Riesz isometry, all Hilbert spaces are reflexive. Example 1.1.24 (Biduals of sequence spaces). We may illustrate the above concepts using the example of sequence spaces (see Example 1.1.15). Since in the case p ∈ (1, ∞), ⋆ it holds (ℓp )⋆ ≃ ℓp , for p1 + p1⋆ = 1, following the same arguments as in Example 1.1.15 1 for ℓp we see that (ℓp )⋆⋆ = (ℓp )⋆ ≃ ℓp with p1⋆ + p⋆⋆ = 1. One immediately sees that ⋆⋆ in the case there p ∈ (1, ∞) we have that p = p, so that (ℓp )⋆⋆ ≃ ℓp and the spaces are reflexive. The situation is different in the cases where p = 1 and p = ∞. ◁ ⋆

⋆

⋆⋆

Reflexive Banach spaces satisfy the following fundamental properties. Proposition 1.1.25 (Properties of reflexive spaces). Let X be a Banach space. (i) If X is reflexive, every closed subspace is reflexive. (ii) X is reflexive if and only if X ⋆ is reflexive.

1.1.4 Different choices of topology on a Banach space: strong and weak topologies Often the topology induced by the norm on a Banach space X, the strong topology is too strong. Let us clarify what we imply by that. Compactness, openess, closedness of sets as well as continuity and semicontinuity of mappings are topological properties, and whether they hold or not depends on the topology with which a set is endowed. As the choice of topology changes the choice of open and closed sets, it is clear that a set A ⊂ X may be open (closed) for a particular choice of topology on X and not enjoying these properties for another choice. Similarly, whether or not a set A ⊂ X is compact depends on whether any open covering of the set admits a finite subcover, and in turn whether a cover is open or not depends on the choice of topology. Hence, a subset A ⊂ X may be compact for a particular choice of topology while not being compact for some other choice. Similarly, a mapping f : X → Y is called continuous is f −1 maps open sets of Y to open sets of X and again whether a set is open or not depends on the choice of topology; hence, a map may be continuous for some choice of topology but not for some other choice. Finally, a mapping F : X → ℝ is lower semicontinuous if

1.1 Fundamentals in the theory of Banach spaces |

9

the set {x ∈ X : F(x) ≤ c} is closed for any c ∈ ℝ, and clearly again this depends upon the choice of topology. As a rule of thumb, one can intuitively figure out that as we weaken a topology (i. e., if we choose a topology whose collection of open sets is a subset of the collection of open sets of the original one) then it is easier to make a given set compact; however, at the same time it makes it more difficult for a mapping to be continuous. As an effect of that, if we choose a topology on X which is weaker than the strong topology we may manage to turn certain important subsets of X, which are not compact when the strong topology is used (such as for instance the closed unit ball BX (0, 1) = BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1}), into compact sets. This need arises when we need to guarantee some limiting behavior of sequences in such sets, as is often the case in optimization problems. Since one tries to juggle between compactness and continuity one way to define a weaker topology is by choosing the coarser (weaker) topology that makes an important family of mappings continuous. If this family of mappings is a family of seminorms on X then this topology enjoys some nice properties, e. g., it is a locally convex topology which is Hausdorfff (i. e., enjoys some nice separation properties). When trying this approach with X, for the choice of the family of mappings defined by the duality pairing we see that we may obtain a weak topology, the weak topology on X, which provides important compactness properties, and furthermore complies extremely well with the property of convexity. This approach can be further applied on the Banach space X ⋆ , again for a family of mappings related to the duality pairing, and obtain a weak topology on X ⋆ , called the weak⋆ topology, which again enjoys some very convenient properties with respect to compactness.

1.1.5 The strong topology on X We start by recalling the strong (norm) topology on a Banach space X. 1.1.5.1 Definitions and properties Definition 1.1.26 (Strong topology). Let X be a Banach space. The strong topology on X is the topology generated by the norm.5 The strong topology on X has the following important property, which will lead us to the necessity of endowing X with a different (weaker) topology. Theorem 1.1.27. The closed unit ball BX (0, 1) := {x ∈ X : ‖x‖X ≤ 1} is compact if and only if X is of finite dimension. 5 i. e., the topology of the metric generated by the norm.

10 | 1 Preliminaries 1.1.5.2 Convergence in the strong topology Definition 1.1.28 (Strong convergence in X). A sequence {xn : n ∈ ℕ} ⊂ X converges strongly to x, denoted by xn → x, if and only if limn→∞ ‖xn − x‖X = 0. By the reverse triangle inequality for the norm, according to which | ‖xn ‖X −‖x‖X | ≤ ‖xn −x‖X we see that the norm is continuous with respect to strong convergence, in the sense that if xn → x then ‖xn ‖X → ‖x‖X as n → ∞. The next result, connects completeness with the convergence of series in X. Theorem 1.1.29. Let (X, ‖ ⋅ ‖X ) be a normed linear space. The space (X, ‖ ⋅ ‖X ) is complete if and only if it has the following property: For every sequence {xn : n ∈ ℕ} ⊂ X such that ∑n ‖xn ‖X < ∞ we have that ∑n xn converges strongly.6 1.1.5.3 The strong topology on X ⋆ Since X ⋆ can be considered as a Banach space when endowed with the norm ‖ ⋅ ‖X ⋆ , we can consider X ⋆ as a topological space endowed with the strong topology defined by the norm ‖ ⋅ ‖X ⋆ . This topology enjoys the same properties as the strong topology on X, so in complete analogy with Theorem 1.1.27 we have the following. Theorem 1.1.30. The closed unit ball BX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1} is compact if and only if X ⋆ is of finite dimension. We may further consider the strong convergence in X ⋆ . Definition 1.1.31 (Strong convergence in X ⋆ ). A sequence {x⋆n : n ∈ ℕ} ⊂ X ⋆ converges strongly to x⋆ , denoted by x⋆n → x⋆ , if and only if limn→∞ ‖x⋆n − x⋆ ‖X ⋆ = 0. Clearly, the analogue of Theorem 1.1.29 holds for the strong topology on X ⋆ .

1.1.6 The weak topology on X ⋆ ⋆

The strong topology on X ⋆ is perhaps too strong as Theorem 1.1.30 shows. To regain the compactness of BX ⋆ (0, 1) in infinite dimensional spaces, we need to endow X ⋆ with a weaker topology, called the weak⋆ topology. 1.1.6.1 Definition and properties Let X be a normed space, X ⋆ its dual, and ⟨⋅, ⋅⟩ the duality pairing between them. Definition 1.1.32 (Weak⋆ topology on X ⋆ ). The topology generated on X ⋆ by the family of mappings M ⋆ := {fx : X ⋆ → ℝ : x ∈ X} defined for each x ∈ X by fx (x⋆ ) = ⟨x⋆ , x⟩ 6 i. e., the sequence {sn : n ∈ ℕ} ⊂ X defined by sn = ∑ni=1 xi converges strongly to some limit x ∈ X.

1.1 Fundamentals in the theory of Banach spaces |

11

for every x⋆ ∈ X ⋆ is called the weak⋆ topology on X ⋆ and is the weaker topology on X ⋆ for which all the mappings in the collection M ⋆ are continuous. The weak⋆ topology will be denoted by σ(X ⋆ , X) and the space X ⋆ endowed with the weak⋆ topology is denoted by Xw⋆⋆ . For the construction of topologies by a set of mappings, see [28]. An alternative (and equivalent) way of defining the weak⋆ topology on X ⋆ is by considering the collection of seminorms on X ⋆ defined by the collection of mappings {px : X ⋆ → ℝ+ : x ∈ X} where px (x⋆ ) = |⟨x⋆ , x⟩|, and then defining the weak topology on X as the topology generated by this family of seminorms, in the sense that it is the weakest topology making these seminorms continuous. Recall that such topologies are called locally convex topologies. The construction of a local basis for this topology is presented in the next example. Example 1.1.33 (A local basis for the weak⋆ topology). For any x⋆0 ∈ X ⋆ , fix {x1 , . . . , xn } ⊂ X, finite, ϵ > 0 and consider the sets of the form n

󵄨 ⋆ 󵄨 ⋆ ⋆ ⋆ ⋆ 𝒪 (x0 ; ϵ, x1 , . . . , xn ) := ⋂{x ∈ X : 󵄨󵄨󵄨⟨x − x0 , xi ⟩󵄨󵄨󵄨 < ϵ}, i=1

xi ∈ X,

called (x1 , . . . , xn ) semiball of radius ϵ centered at x⋆0 . The collection of sets B⋆x⋆ := {𝒪⋆ (x⋆0 ; ϵ, x1 , . . . , xn ) : ∀ ϵ > 0, ∀ {x1 , . . . xn } ⊂ X, finite, n ∈ ℕ}, 0

forms a basis of neighborhoods of x⋆0 for the weak⋆ topology σ(X ⋆ , X) and this helps us to characterize weak⋆ open sets in X ⋆ . A subset 𝒪⋆ ⊂ X ⋆ is weak⋆ open if and only if for every x⋆0 ∈ 𝒪⋆ there exists ϵ > 0 and x1 , . . . , xn ∈ X such that the semiball 𝒪⋆ (x⋆0 ; ϵ, x1 , . . . , xn ) ⊆ 𝒪⋆ . ◁ The weak⋆ topology on X ⋆ is a weaker topology than the strong topology on X ⋆ , so the same subsets of X ⋆ may have different topological properties depending on the chosen topology. However, we have the following. Example 1.1.34. A subset A ⊂ X ⋆ is bounded if and only if it is weak⋆ bounded. If A is bounded (in the strong topology on X ⋆ ), then for every strong neighborhood N0 there exists λ > 0 such that A ⊂ λN0 . To show that it is weak⋆ bounded, we need ⋆ to show the above for every weak⋆ neighborhood N0w , for a possibly different λ󸀠 > 0. ⋆ Consider any weak⋆ neighborhood of the origin N0w which by the fact that the weak⋆ topology on X ⋆ is weaker than the strong topology on X ⋆ is also a strong neighborhood, so the result follows by the fact that A is bounded in the strong topology. For the converse assume that A is weak⋆ bounded, i. e., for every weak⋆ neighbor⋆ ⋆ hood N0w of the origin there exists λ󸀠 > 0 such that A ⊂ λ󸀠 N0w . Then, it holds that sup{|⟨x⋆ , x⟩| : x⋆ ∈ A} < ∞, for every x ∈ X, so by the Banach–Steinhaus theorem sup{‖x⋆ ‖X ⋆ : x⋆ ∈ A} < ∞, therefore, A is bounded in the strong topology. ◁

12 | 1 Preliminaries The weak⋆ topology has the following important properties (see, e. g., [28]). Theorem 1.1.35. The weak⋆ topology, σ(X ⋆ , X), has the following properties: (i) The space X ⋆ endowed with the weak⋆ topology is a Hausdorff 7 locally convex space. (ii) The weak⋆ topology is metrizable if and only if X is finite dimensional. (iii) A normed space X is separable if and only if the weak⋆ topology restricted to the closed unit ball BX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1} is metrizable.8 (iv) Every nonempty weak⋆ open subset A ⊂ X ⋆ is unbounded, in infinite dimensional spaces. Note that even though X ⋆ can be turned into a normed space using ‖ ⋅ ‖X ⋆ , when endowed with the weak⋆ (σ(X ⋆ , X)) topology, it is no longer in general a metric space on account of Theorem 1.1.35(ii). In this sense, assertion (iii) is very important since metrizable topologies enjoy some convenient special properties resulting from the fact that they admit a countable basis and are first countable. For example, it allows us to use weak⋆ sequential compactness as equivalent to weak⋆ compactness for (norm) bounded subsets of X ⋆ . However, caution is needed as the metrizability does not hold over the whole of X ⋆ , thus requiring in general the use of nets to replace sequences when dealing with the weak⋆ topology in general. On the other hand, the weak⋆ topology displays some remarkable compactness properties, known as the Alaoglu (or Alaoglu–Banach–Bourbaki) theorem, a fact that makes the use of this topology indispensable. Theorem 1.1.36 (Alaoglu). The closed unit ball in X ⋆ , BX ⋆ (0, 1) := {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1} is weak⋆ compact (i. e., compact for the σ(X ⋆ , X) topology).9 If furthermore, X is separable, then every bounded sequence in X ⋆ has a weak⋆ convergent subsequence. Example 1.1.37 (Weak compactness of the duality map). For any x ∈ X, the duality map J(x) (recall Definition 1.1.16) is a bounded and weak⋆ compact set in X ⋆ . This follows since for any fixed x⋆ ∈ J(x), it holds that ‖x⋆ ‖X ⋆ ≤ ‖x‖X , so that J(x) is a bounded set. The weak⋆ compactness then follows by Alaoglu’s theorem (see Theorem 1.1.36). ◁ Even though, nets provide a complete characterization of the weak⋆ topology, there are still important properties that may be captured using sequences. 7 i. e., a topological space such that any two distinct points have distinct neighborhoods. 8 Clearly, this result extends to any norm bounded and closed (with respect to the strong topology on X ⋆ ) set of the form A = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ c}, for some c ∈ (0, ∞). Note that the weak⋆ topology is never metrizable on the whole of X ⋆ . 9 Hence, also the set cBX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ c}. This result also implies that any bounded subset of X ⋆ is relatively weak⋆ compact in X ⋆ , so that bounded and weak⋆ closed subsets of X ⋆ are weak⋆ compact.

1.1 Fundamentals in the theory of Banach spaces | 13

1.1.6.2 Convergence in the weak topology Definition 1.1.38 (Weak⋆ convergence). A sequence {x⋆n : n ∈ ℕ} ⊂ X ⋆ converges ⋆

weak⋆ to x⋆ , denoted by x⋆n ⇀ x⋆ , if it converges with respect to the weak⋆ topology σ(X ⋆ , X), i. e., if and only if limn→∞ ⟨x⋆n , x⟩ = ⟨x⋆ , x⟩ for every x ∈ X. ⋆

One may similarly define weak⋆ convergence for a net and show that a net {x⋆α : α ∈ ℐ } ⊂ X ⋆ converges weak⋆ to x⋆ if and only if ⟨x⋆α , x⟩ → ⟨x⋆ , x⟩ for every x ∈ X. The weak⋆ convergence has the following useful properties. Proposition 1.1.39. Consider the sequence {x⋆n : n ∈ ℕ} ⊂ X ⋆ . Then:

(i) If x⋆n → x⋆ then x⋆n ⇀ x⋆ . ⋆

(ii) If x⋆n ⇀ x⋆ , then the sequence {‖x⋆n ‖X ⋆ : n ∈ ℕ} ⊂ ℝ is bounded and ‖x⋆ ‖X ⋆ ≤ lim infn ‖x⋆n ‖X ⋆ , i. e., the norm ‖ ⋅ ‖X ⋆ is sequentially lower semicontinuous with respect to weak⋆ convergence. ⋆ (iii) If x⋆n ⇀ x⋆ and xn → x, then ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. ⋆

Property (i) follows easily from the fact that |⟨x⋆n − x⋆ , x⟩| ≤ ‖x⋆n − x⋆ ‖X ⋆ ‖x‖X . Property (iii) follows by a rearrangement of the difference |⟨x⋆n , xn ⟩ − ⟨x⋆ , x⟩| = |⟨x⋆n , xn − x⟩ + ⟨x⋆n − x⋆ , x⟩| and fact that the weak⋆ convergent sequence {x⋆n : n ∈ ℕ} ⊂ X ⋆ is norm bounded, which in turn comes as a consequence of the Banach–Steinhaus theorem (see Theorem 1.1.7). Property (ii) can be seen in a number of ways, we provide one of them in the following example. Example 1.1.40 (Weak⋆ (sequential) lower semicontinuity of the norm ‖ ⋅ ‖X ⋆ ). The function x⋆ 󳨃→ ‖x⋆ ‖X ⋆ is weak⋆ lower semicontinuous and weak⋆ sequentially lower semicontinuous. The weak⋆ lower semicontinuity of the norm ‖ ⋅ ‖X ⋆ follows from the fact that the functions fx : X ⋆ → ℝ, defined by fx (x⋆ ) = ⟨x⋆ , x⟩ for every x⋆ ∈ X ⋆ are (by definition) continuous with respect to the weak⋆ topology, and since ‖x⋆ ‖X ⋆ = sup{|⟨x⋆ , x⟩| : ‖x‖X ≤ 1} = sup{|fx (x⋆ )| : ‖x‖X ≤ 1}, we see that the function ‖ ⋅ ‖X ⋆ is the (pointwise) supremum of a family of continuous functions; hence, it is semicontinuous. The latter can be easily seen as follows: if f (x) = supα∈ℐ fα (x), with fα continuous (or lower semicontinuous) then {x ∈ X : f (x) ≤ c} = ⋂α∈ℐ {x ∈ X : fα (x) ≤ c} which is a closed set as the intersection of the closed sets {x ∈ X : f (x) ≤ c}. The weak⋆ sequential lower semicontinuity can follow by another application of the Banach–Steinhaus Theorem 1.1.7, working as in Example 1.1.8, for the sequence of operators {An : n ∈ ℕ} ⊂ ℒ(X, ℝ) defined by An (x) = ⟨x⋆n , x⟩ for any x ∈ X, which converges pointwise to the operator A ∈ ℒ(X, ℝ) defined by A(x) = ⟨x⋆ , x⟩ for every x ∈ X. ◁ Example 1.1.41 (Does every norm bounded sequence in X ⋆ have a weak⋆ convergent subsequence?). Can we claim that if {x⋆n : n ∈ ℕ} ⊂ X ⋆ satisfies supn∈ℕ ‖x⋆n ‖X ⋆ < c,

14 | 1 Preliminaries for some c > 0 independent of n, there exists x⋆o ∈ X ⋆ and a subsequence {x⋆nk : k ∈ ℕ}

such that x⋆nk ⇀ x⋆o as k → ∞? The answer is no, unless X is separable. Note that since in general (see Theorem 1.1.35) the weak⋆ topology is not metrizable, Alaoglu’s theorem which guarantees weak⋆ compactness of (norm) bounded subsets of X ⋆ , does not guarantee weak⋆ sequential compactness, i. e., that any sequence {x⋆n : n ∈ ℕ} ⊂ X ⋆ , such that there exists a constant c > 0 (independent of n) with the property ‖x⋆n ‖X ⋆ ≤ c, for every n ∈ ℕ, has a convergent subsequence in the sense of convergence in the weak⋆ topology on X ⋆ . This will only be true when X is separable, in which case the weak⋆ topology restricted on this bounded set will be metrizable. ◁ ⋆

1.1.7 The weak topology on X The strong topology on X is perhaps too strong as Theorem 1.1.27 shows. To regain the compactness of the closed unit ball BX (0, 1) in infinite dimensional spaces, we need to endow X with a weaker topology, called the weak topology. 1.1.7.1 Definitions and properties Let X be a normed space, and X ⋆ its dual. Definition 1.1.42 (Weak topology on X). The topology generated on X by the family of mappings M := {fx⋆ : X → ℝ : x⋆ ∈ X ⋆ } = {⟨x⋆ , ⋅⟩ : X → ℝ : x⋆ ∈ X ⋆ } is called the weak topology on X and is the weaker topology on X for which all the mappings in the collection M are continuous. The weak topology will be denoted by σ(X, X ⋆ ), and the space X endowed with the weak topology will be denoted by Xw . For the construction of topologies by a set of mappings, see [28]. An alternative (and equivalent) way to define the weak topology on X is by considering the collection of seminorms on X defined by the collection of mappings {px⋆ : X → ℝ+ : x⋆ ∈ X ⋆ } where px⋆ (x) := |⟨x⋆ , x⟩|, and then defining the weak topology on X as the topology generated by this family of seminorms, in the sense that it is the weakest topology making these seminorms continuous. Recall that such topologies are called locally convex topologies. The construction of a local basis for this topology is presented in the next example. Example 1.1.43 (A local basis for the weak topology). For any x0 ∈ X, fix {x⋆1 , . . . , x⋆n } ⊂ X ⋆ , finite, and ϵ > 0 and consider the sets of the form n

𝒪(x0 ; ϵ, x1 , . . . , xn ) := ⋂{x ∈ X : 󵄨󵄨󵄨⟨xi , x − x0 ⟩X ⋆ ,X 󵄨󵄨󵄨 < ϵ, xi ∈ X }, ⋆

⋆

i=1

󵄨

⋆

󵄨

⋆

⋆

1.1 Fundamentals in the theory of Banach spaces |

15

called (x⋆1 , . . . , x⋆n ) semiball of radius ϵ centered at x0 . The collection of semiballs Bx0 := {𝒪(x0 ; ϵ, x⋆1 , . . . , x⋆n ) : ∀ ϵ > 0, ∀ {x⋆1 , . . . x⋆n } ⊂ X ⋆ , finite, n ∈ ℕ}, forms a basis of neighborhoods of x0 for the weak topology σ(X, X ⋆ ), and this helps us characterize the open sets for the weak topology on X (or weakly open sets). A subset 𝒪 ⊂ X is weakly open if and only if for every x0 ∈ 𝒪 there exists ϵ > 0 and x⋆1 , . . . , x⋆n ∈ X ⋆ such that the semiball 𝒪(x0 ; ϵ, x⋆1 , . . . , x⋆n ) ⊂ 𝒪. ◁ Example 1.1.44. As a simple geometrical illustration of how the semiballs may look like, consider the case where X = ℝ2 , x0 = (0, 0) and x⋆1 = (1, 1), x⋆2 = (1, −1). Then 𝒪(x0 ; ϵ, x1 ) = {(x1 , x2 ) : |x1 + x2 | ≤ ϵ}, ⋆

⋆ ⋆ 𝒪(x0 ; ϵ, x1 , x2 )

and

= {(x1 , x2 ) : |x1 + x2 | ≤ ϵ, |x1 − x2 | ≤ ϵ},

which look like strips and intersection of strips, respectively. However, this geometrical picture which is valid in finite dimensional spaces may be misleading in infinite dimensions (see, e. g., Theorem 1.1.45(v)). ◁ The following theorem (see, e. g., [28]) collects some fundamental results for the weak topology. Theorem 1.1.45. The following are true for the weak topology σ(X, X ⋆ ): (i) The space X endowed with the weak topology σ(X, X ⋆ ) is a Hausdorff locally convex space. (ii) The weak topology is weaker than the norm topology, i. e., every weakly open (closed) set is strongly open (closed). The weak and the strong topology coincide if and only if X is finite dimensional. (iii) The weak topology is metrizable10 if and only if X is finite dimensional. (iv) X ⋆ is separable if and only if the weak topology σ(X, X ⋆ ) on X restricted on the closed unit ball BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1} is metrizable. Clearly, the result extends to any closed norm bounded A ⊂ X. (v) If X is infinite dimensional then for any xo ∈ X, every weak neighborhood N w (xo ) contains an affine space passing through xo . Assertion (iii) indicates that we have to be cautious with the weak topology since it will not enjoy some of the convenient special properties valid for metrizable or first countable topologies. For instance, it is not necessarily true that compactness coincides with sequential compactness, and in general the weak closure of a set is expected to be a larger set than the set of limits of all weakly converging sequences in the set. In this respect, Assertion (iv) is very important since it allows us to reinstate 10 That is, there exists a metric that induces that weak topology.

16 | 1 Preliminaries these convenient characterizations in terms of sequences for the weak topology in special cases. For example, it allows us to use sequential compactness as equivalent to compactness for bounded subsets of X in the weak topology, or it allows the characterization of weak closures of sets in terms of limits of sequences. However, caution is needed as the metrizability does not hold over the whole of X, thus requiring in general the use of nets to replace sequences when dealing with the weak topology in general. An interesting question is what kind of Banach spaces X satisfy the condition that X ⋆ is separable? This would always hold true for instance if X is separable and reflexive, as in such a case X ⋆ is separable. A further example of nonreflexive X with separable X ⋆ is the case where X = c0 and X ⋆ = ℓ1 (see, e. g., [48]). The weak topology on X is weaker than the strong topology on X so concepts like closed and open sets, compact sets, etc. vary with respect the topology chosen on X. The following example shows that as far as the concept of boundedness is concerned the choice of topology does not make a difference. Example 1.1.46 (Weakly bounded set vs. bounded set). A subset A ⊂ X is weakly bounded if and only if it is bounded.11 It is easy to see that if A is bounded then it is also weakly bounded. Since A is bounded for every (strong) neighborhood N0 there exists λ > 0 such that A ⊂ λN0 . To show that A is weakly bounded, we must show that for every weak neighborhood N0w there exists λ󸀠 > 0 such that A ⊂ λ󸀠 N0w . However, any weak neighborhood N0w is also a strong neighborhood so the required property holds for λ󸀠 = λ. For the converse, the argument is slightly more involved. Assume that A is weakly bounded so that for every weak neighborhood N0w there exists λ󸀠 > 0 such that A ⊂ λ󸀠 N0w . This implies that the family of linear maps 𝒜 := {fx : X ⋆ → ℝ : x ∈ A}, defined by fx (x⋆ ) := ⟨x⋆ , x⟩ for every x⋆ ∈ X ⋆ , is pointwise bounded. By the Banach– Steinhaus Theorem 1.1.7 this is also norm bounded, i. e., the elements of 𝒜 are bounded when considered as elements of ℒ(X ⋆ , ℝ) which coincides with X ⋆⋆ . But ‖fx ‖ℒ(X ⋆ ,ℝ) = ‖x‖X (see Example 1.1.21) which means that A is norm bounded; hence, it is strongly bounded. ◁ However, certain topological properties are significantly different if we change topologies on X. Clearly, since the weak topology is weaker than the strong (norm) topology, any weakly open set is open, however, as the following example shows the converse is not true. Similarly, for closed sets (unless these are convex where in this case the concept of weak and strong closedness coincide, as we will prove in Proposition 1.2.12). Example 1.1.47. In infinite dimensions, the open unit ball BX (0, 1) = {x ∈ X : ‖x‖X < 1} has empty weak interior; hence, is not open in the weak topology. In fact (see Theo11 In fact, this result holds in a more general setting than that of Banach spaces and is known as the Mackey theorem (see, e. g., Theorem 6.24 in [5]).

1.1 Fundamentals in the theory of Banach spaces |

17

rem 1.1.45(v)), every nonempty weakly open subset is unbounded in infinite dimensions. Note, however, that the closed unit ball BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1} is also weakly closed (as a result of convexity). Suppose not, and consider a point xo in the weak interior of BX (0, 1). Then there exist ϵ > 0, n ∈ ℕ and x⋆1 , . . . , x⋆n ∈ X ⋆ such that 𝒪(xo ; ϵ, x⋆1 , . . . , x⋆n ) ⊂ BX (0, 1). We claim that we may obtain a point zo ∈ X \ {0}, such that ⟨x⋆i , zo ⟩ = 0 for all i = 1, . . . , n. Indeed, if not ⋂ni=1 N(x⋆i ) = {0}, where by N(x⋆i ) we denote the kernel of the functional x⋆i considered as a linear mapping x⋆i : X → ℝ, and since for any x⋆ ∈ X ⋆ , {0} ⊂ N(x⋆ ) we conclude that ⋂ni=1 N(x⋆i ) ⊂ N(x⋆ ), therefore using a well-known result from linear algebra12 x⋆ ∈ span{x⋆1 , . . . , x⋆n } which contradicts the assumption that dim(X) = ∞. Then, for every λ ∈ ℝ we have that xo + λzo ∈ 𝒪(x0 ; ϵ, x⋆1 , . . . , x⋆n ) ⊂ BX (0, 1), which implies that ‖xo + λzo ‖X ≤ 1 which is clearly a contradiction. ◁ Example 1.1.48 (The closure of the unit sphere). Assume that dim(X) = ∞ and let w SX (0, 1) = {x ∈ X : ‖x‖X = 1} and BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1}. Then SX (0, 1) = BX (0, 1). In general, not every element of BX (0, 1) can be expressed as the limit of a weakly convergent sequence in SX (0, 1), except in special cases. Such is, e. g., the case where X = ℓ1 on account of the Schur property that states the equivalence of the weak and the strong convergence. To check this, if we take for the time being for granted that BX (0, 1) is a weakly w closed set (for a proof see Proposition 1.2.12) so that SX (0, 1) ⊂ BX (0, 1), since SX (0, 1) ⊂ w BX (0, 1) and SX (0, 1) is the smallest weakly closed set containing SX (0, 1). To prove the reverse inclusion, consider any xo ∈ BX (0, 1) and take a weak (open) neighborhood N w (xo ). Then, by the construction of the local basis for the weak topology (see Example 1.1.43) there exists x⋆1 , . . . , x⋆n ∈ X ⋆ and ϵ > 0 such that 𝒪(xo ; ϵ, x⋆1 , . . . , x⋆n ) := {x ∈ X : |⟨x⋆i , x − xo ⟩| < ϵ, i = 1, . . . , n} ⊂ N w (xo ). With the same arguments as in Example 1.1.47, there exists nonzero zo ∈ X such that ⟨x⋆i , zo ⟩ = 0 for all i = 1, . . . , n, so that for any λ ∈ ℝ, xo + λzo ∈ 𝒪(xo ; ϵ, x⋆1 , . . . , x⋆n ) ⊂ N w (xo ). Since ‖xo ‖X ≤ 1, there exists λo ∈ ℝ such that ‖xo +λo zo ‖X = 1, therefore, xo +λo zo ∈ 𝒪(xo ; ϵ, x⋆1 , . . . , x⋆n )∩SX (0, 1) ⊂ N w (xo )∩SX (0, 1) so that N w (xo ) ∩ SX (0, 1) ≠ 0. Since for any xo ∈ BX (0, 1) and any weak neighborhood w N w (xo ) of xo , we have that N w (xo ) ∩ SX (0, 1) ≠ 0 we see that xo ∈ SX (0, 1) , so that since w xo was arbitrary we conclude that BX (0, 1) ⊂ SX (0, 1) , and the proof is complete. ◁ On the other hand, when it comes to the question of continuity of linear operators the choice of topology does not make too much difference ([28]). 12 If on a linear space for the linear maps f , f1 , . . . , fn : X → ℝ, it holds that ⋂ni=1 N(fi ) ⊂ N(f ) then f is a linear combination of the f1 , . . . , fn . This can be proved by induction. It is easy to see that the claim is n true for n = 1. Assuming it holds for n, if ⋂n+1 i=1 N(fi ) ⊂ N(f ) then it holds that ⋂i=1 N(ϕi ) ⊂ N(ϕ) where ϕi = fi |N(fn+1 ) , ϕ = f |N(fn+1 ) , i = 1, . . . , n so applying the result at level n to these functions we have that ϕ ∈ span{ϕ1 , . . . , ϕn }, i. e., for some λ1 , . . . , λn ∈ ℝ we have that N(fn+1 ) ⊂ N(f − ∑ni=1 λi fi ) and applying the result for n = 1 we conclude the existence of λn+1 such that f = ∑n+1 i=1 λi fi , concluding the proof.

18 | 1 Preliminaries Theorem 1.1.49. A linear operator A : X → Y is continuous with respect to the weak topologies on X and Y (weakly continuous) if and only if it is continuous with respect to the strong topologies on X and Y (strong or norm continuous), the same holding of course for linear functionals (Y = ℝ). Even though, nets provide a complete characterization of the weak topology, there are still important properties that may be captured using sequences. 1.1.7.2 Convergence in the weak topology Definition 1.1.50 (Weak convergence). A sequence {xn : n ∈ ℕ} ⊂ X converges weakly to x, denoted by xn ⇀ x, if it converges with respect to the weak topology σ(X, X ⋆ ), i. e., if and only if lim ⟨x⋆ , xn ⟩X ⋆ ,X = ⟨x⋆ , x⟩X ⋆ ,X ,

n→∞

∀ x⋆ ∈ X ⋆ .

One may similarly define weak convergence for a net {xα : α ∈ ℐ } ⊂ X and show that the net converges weakly to x if and only if ⟨x⋆ , xα ⟩X ⋆ ,X → ⟨x⋆ , x⟩X ⋆ ,X for every x⋆ ∈ X ⋆ . Remark 1.1.51 (The Urysohn property). Since X w := (X, σ(X, X ⋆ )) is a Hausdorff topological space, we have uniqueness of weak limits. A sequence weakly converges to a point if each subsequence contains a further subsequence which converges to this point (Urysohn property). The same property naturally holds for the strong topology. Example 1.1.52. Show that the weak convergence enjoys the Urysohn property. Indeed assume by contradiction that every subsequence of {xn : n ∈ ℕ} ⊂ X has a further subsequence that converges weakly to x but xn ⇀̸ x in X. Then there exists ϵ > 0, some x⋆i ∈ X ⋆ , i = 1, . . . , m, and a subsequence {xnk : k ∈ ℕ} with the property |⟨x⋆i , xnk ⟩ − ⟨xi ⋆ , x⟩| ≥ ϵ, i = 1, . . . , m, for every k. However, by assumption this subsequence has a further subsequence {xnk : ℓ ∈ ℕ} such that xnk ⇀ x, which clearly is a ℓ ℓ contradiction. ◁ Proposition 1.1.53 (Properties of weak convergence). Let {xn : n ∈ ℕ} be a sequence in X. Then: (i) If xn → x, then xn ⇀ x. (ii) If xn ⇀ x, then the sequence xn is bounded (i. e., supn∈ℕ ‖xn ‖X < ∞), and ‖x‖X ≤ lim infn ‖xn ‖X , i. e., the norm is sequentially weakly lower semicontinuous. (iii) If xn ⇀ x and x⋆n → x⋆ in X ⋆ , then ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. The first claim is immediate from the fact that the weak topology is weaker (coarser) than the strong topology. The second claim may be proved in various ways; we present one possible proof in Example 1.1.54 below. The third claim follows easily by a rearrangement of ⟨x⋆n , xn ⟩−⟨x⋆ , x⟩ = ⟨x⋆n −x⋆ , xn ⟩+⟨x⋆ , xn −x⟩, using the norm boundedness of the sequence {‖xn ‖X : n ∈ ℕ} ⊂ ℝ.

1.1 Fundamentals in the theory of Banach spaces | 19

Example 1.1.54 (Weak (sequential) lower semicontinuity of the norm). The function x 󳨃→ ‖x‖X is (sequentially) lower semicontinuous for the weak topology. The proof of this claim (stated in Proposition 1.1.53(ii)) follows by the observation that the norm can be represented as ‖x‖X = sup{|⟨x⋆ , x⟩| : x⋆ ∈ X ⋆ , ‖x⋆ ‖X ⋆ ≤ 1} (see Example 1.1.17) and then using the same reasoning as in Example 1.1.40, based on the remark that in the weak topology, for any x⋆ ∈ X ⋆ fixed, the functions fx⋆ : X → ℝ, defined by fx⋆ (x) = ⟨x⋆ , x⟩ are continuous and the supremum of a family of continuous functions is lower semicontinuous (see the argument in Example 1.1.40), we obtain the weak lower semicontinuity of the norm. For the sequential weak lower semicontinuity, we may reason once more as in Example 1.1.40, reversing the roles of x and x⋆ . ◁ Example 1.1.55 (If xn ⇀ x and x⋆n ⇀ x⋆ , it may be that ⟨x⋆n , xn ⟩ ↛ ⟨x⋆ , x⟩). Consider the ⋆ space X = ℓp , p ∈ (0, ∞), with dual space X ⋆ = ℓp , with p1 + p1⋆ = 1, and bidual ⋆

X ⋆⋆ = (ℓp )⋆ = ℓp . Let en = {δin : i ∈ ℕ}, which may be considered either as an element of X, X ⋆ or X ⋆⋆ . Consider the sequences {xn : n ∈ ℕ} = {en : n ∈ ℕ} ⊂ X and ⋆ {x⋆n : n ∈ ℕ} = {en : n ∈ ℕ} ⊂ X ⋆ . One may easily confirm that xn ⇀ 0, and x⋆n ⇀ 0. ⋆ ⋆ p⋆ Indeed, considering any x ∈ X , of the form x = {xi : i ∈ ℕ, ∑i∈ℕ |xi | < ∞}, so ⋆ that ⟨x⋆ , xn ⟩ = xn → 0 as n → ∞, since ∑i∈ℕ |xi |p < ∞, with a similar treatment for x⋆n ⇀ 0. However, ⟨x⋆n , xn ⟩ = 1 for every n ∈ ℕ, so that ⟨x⋆n , xn ⟩ ↛ 0. ◁ ⋆

Even though, the weak topology is not metrizable in general, the celebrated Eberlein–Smulian theorem (see, e. g., [4] or [82]) shows that for the weak topology the concepts of sequential compactness and compactness are equivalent as for metrizable topologies. Theorem 1.1.56 (Eberlein–Smulian). Let X be a Banach space and consider A ⊂ X. The following are equivalent: (i) A is compact in the weak topology σ(X, X ⋆ ) (weakly compact). (ii) A is sequentially compact in the weak topology σ(X, X ⋆ ) (weakly sequentially compact). 1.1.7.3 The weak topology for reflexive spaces For the particular case of reflexive Banach spaces X endowed with the weak topology one may obtain some important information, which essentially follows by the important compactness properties of the weak⋆ topology on X ⋆ . Before proceeding, we present in the next example the possible connections of the weak⋆ topology on X ⋆ with the weak topology on X. Example 1.1.57 (The weak topology on X ⋆ ). Since X ⋆ is a normed space, with dual X ⋆⋆ , what would happen if we tried to repeat the construction of Definition 1.1.42 for X ⋆ ? Replacing X by X ⋆ and X ⋆ by X ⋆⋆ in this construction we create thus the topology σ(X ⋆ , X ⋆⋆ ). This will be another topology on X ⋆ , called the weak topology on X ⋆ . How does this topology compare with the weak⋆ topology σ(X ⋆ , X) on X ⋆ ?

20 | 1 Preliminaries In general, we may consider that X ⊂ X ⋆⋆ (in the sense of the canonical embedding of Theorem 1.1.22) so that the weak⋆ topology σ(X ⋆ , X) is weaker than the weak topology on X ⋆ constructed here σ(X ⋆ , X ⋆⋆ ). However, if X is reflexive, then the canonical embedding is onto and X ≃ X ⋆⋆ , so that the topologies σ(X ⋆ , X) and σ(X ⋆ , X ⋆⋆ ) coincide. This means that in reflexive spaces, we may consider the weak topology on X ⋆ as equivalent to the weak⋆ topology on X ⋆ , thus enjoying the important compactness properties of the weak⋆ topology ensured by Alaoglu’s theorem. ◁ After the above preparation we may state the following result (see, e. g., [28]) or [82]) which provides some important information concerning compact sets in this topology, called weakly compact sets. Theorem 1.1.58. Let X be a Banach space. The following statements are equivalent: (i) X is reflexive. (ii) The closed unit ball of X, BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1} is weakly compact. (ii󸀠 ) The closed unit ball of X ⋆ , BX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1} is compact when X ⋆ is endowed with the weak topology. (iii) Every bounded sequence {xn : n ∈ ℕ} ⊂ X has a subsequence which converges for the topology σ(X, X ⋆ ) (a weakly converging subsequence), i. e., there exists a subsequence {xnk : k ∈ ℕ} and an element xo ∈ X such that xnk ⇀ xo in X. (iv) For every x⋆ ∈ X ⋆ , the supremum in the definition of the dual norm is attained 󵄩󵄩 ⋆ 󵄩󵄩 ⋆ 󵄩󵄩x 󵄩󵄩X ⋆ = max{⟨x , x⟩ : ‖x‖X ≤ 1}. This theorem is very useful as it allows us to extract a weakly convergent subsequence out of every bounded sequence in a reflexive Banach space, and so it will be used very often throughout this book. It plays in reflexive Banach spaces the role that the Bolzano–Weierstrass theorem plays in ℝ. To see how reflexivity leads to weak compactness of the closed unit ball, we may follow the reasoning of the next example. Example 1.1.59 (Weak compactness of the closed unit ball in reflexive spaces). For a reflexive space, we know that the canonical embedding j : X → X ⋆⋆ is onto, so that X ≃ X ⋆⋆ , and we can identify X = (X ⋆ )⋆ . By Proposition 1.1.25, since X is reflexive so is X ⋆ ; hence, by the reasoning of Example 1.1.57, the weak⋆ topology on (X ⋆ )⋆ = X coincides with the weak topology on (X ⋆ )⋆ = X. By Alaoglu’s theorem (see Theorem 1.1.36), the (norm) closed unit ball BX (0, 1) of (X ⋆ )⋆ = X is weak⋆ compact, but since (by reflexivity) the weak⋆ and the weak topology on (X ⋆ )⋆ = X coincide, BX (0, 1) is also weakly compact. But then the Eberlein–Smulian theorem (see Theorem 1.1.56) guarantees weak sequential compactness from which we can conclude that every sequence in BX (0, 1); hence, any (norm) bounded sequence admits a weakly convergent subsequence. ◁

1.1 Fundamentals in the theory of Banach spaces | 21

Example 1.1.60 (The supremum in the definition of the norm is attained for reflexive spaces). We will show that if X is reflexive then for any x⋆ ∈ X ⋆ fixed, there exists xo ∈ X such that ‖x⋆ ‖X ⋆ = ⟨x⋆ , xo ⟩ with ‖xo ‖X ≤ 1. By the reflexivity of X, it holds that the (norm) closed unit ball BX (0, 1) is weakly compact, hence from Eberlein–Smulian theorem (see Theorem 1.1.56) weakly sequentially compact. By definition, ‖x⋆ ‖X ⋆ = sup{⟨x⋆ , x⟩ : x ∈ X, ‖x‖X ≤ 1}, so we construct a sequence {xn : n ∈ X} ⊂ B, with the property13 ⟨x⋆ , xn ⟩ → ‖x⋆ ‖X ⋆ . The weak sequential compactness of BX (0, 1) implies the existence of a subsequence {xnk : k ∈ ℕ} of the above sequence and a xo ∈ BX (0, 1) such that xnk ⇀ xo as k → ∞. This implies that ⟨x⋆ , xnk ⟩X ⋆ ,X → ⟨x⋆ , xo ⟩. On the other hand, since the whole sequence {xn : n ∈ ℕ} satisfies ⟨x⋆ , xn ⟩ → ‖x⋆ ‖X ⋆ , the same holds for the weakly converging subsequence {xnk : k ∈ ℕ}, and we easily conclude that ‖x⋆ ‖X ⋆ = ⟨x⋆ , xo ⟩. The converse statement is known as James theorem and its proof is more subtle (see, e. g., [82]). The reasoning used here will be used repeatedly in this book for providing results concerning the maximization of functionals in reflexive Banach spaces. ◁ The following result (see, e. g., [51] or [108]) will be used frequently throughout this book. w

Proposition 1.1.61. Let X be a reflexive Banach space, A ⊂ X, bounded and xo ∈ A , w where by A we denote the weak closure of A. Then there exists a sequence {xn : n ∈ ℕ} ⊂ A which is weakly convergent to xo in X. w

Proof. Let us fix the point xo ∈ A and let us denote by BX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1} the closed unit ball of X ⋆ . w First, we will show the existence of a countable subset A0 ⊂ A such that xo ∈ A0 . w To this end, we claim that for the fixed xo ∈ A , and for any choice of integers n, m, and every (x⋆1 , . . . , x⋆m ) ∈ (BX ⋆ (0, 1))m = BX ⋆ (0, 1) × ⋅ ⋅ ⋅ × BX ⋆ (0, 1), we can find x ∈ A w such that |⟨x⋆k , x − xo ⟩| < n1 for all k = 1, . . . , m. Indeed, since xo ∈ A , by definition, for any weak neighborhood N w (xo ) it holds that N w (xo ) ∩ A ≠ 0; hence, recalling the construction of weak neighborhoods the claim holds. This can be rephrased as that for any (x⋆1 , . . . , x⋆m ) ∈ (BX ⋆ (0, 1))m it holds that (x⋆1 , . . . , x⋆m ) ∈ ⋃x∈A Vn,m (x), where m

Vn,m (x) = {(x⋆1 , x⋆2 , . . . , x⋆m ) ∈ (BX ⋆ (0, 1)⋆ ) m

󵄨 󵄨 1 : 󵄨󵄨󵄨⟨x⋆k , x − xo ⟩󵄨󵄨󵄨 < , k = 1, . . . , m} n

⊂ (BX ⋆ (0, 1)) , which are clearly open sets (for the weak topology). By this rephrasal, we conclude that {Vn,m (x) : x ∈ A} constitutes an open cover for (BX ⋆ (0, 1))⋆ , which is weakly com13 The construction of this sequence follows by elementary means from the definition of the supremum as the least upper bound; for any n ∈ ℕ, ‖x⋆ ‖X − n1 is not an upper bound of the quantity ⟨x⋆ , x⟩ for x ∈ B, which implies the existence of an xn ∈ B such that ‖x⋆ ‖X − n1 < ⟨x⋆ , xn ⟩ ≤ ‖x⋆ ‖X which has the required properties.

22 | 1 Preliminaries pact by Alaoglu’s theorem; hence, it must admit a finite subcover. Therefore, there exists a finite subset of A, which we denote by Sn,m ⊂ A, such that (BX ⋆ (0, 1)⋆ )m ⊂ ⋃x∈Smn Vm,n (x). As this holds for every n, m, we may define A0 = ⋃n,m∈ℕ Sn,m which is a w

countable subset of A, and has the sought for property that xo ∈ A0 . In order to show w that xo ∈ A0 , we must show that for every weak neighborhood N w (xo ) of xo , it holds that N w (xo ) ∩ A0 ≠ 0. Indeed, let N w (xo ) be any weak neighborhood of xo . By the construction of weak neighborhoods on X, there exists n, m and (x⋆1 , . . . , x⋆m ) ∈ (BX ⋆ (0, 1))m 1 w ⋆ ⋆ m ⋆ such that ⋂m i=1 {x ∈ X : |⟨xi , x − xo ⟩ < n } ⊂ N (xo ). Since (x1 , . . . , xm ) ∈ (BX ⋆ (0, 1)) ⊂ ⋃x∈Sn,m Vn,m (x), there exists x ∈ Sn,m for which (x⋆1 , . . . , x⋆m ) ∈ Vn,m (x), i. e., by the defini-

tion of Vn,m (x), there exists x ∈ Sn,m ⊂ A0 = ⋃n,m Sn,m , with the property |⟨x⋆i , x−xo ⟩| < n1 1 w ⋆ for all i = 1, . . . , m, which means that x ∈ ⋂m i=1 {x ∈ X : |⟨xi , x − xo ⟩| < n } ⊂ N (xo ), 󸀠 therefore x ∈ 𝒩xo . In summary, we have shown that for every weak neighborhood N w (xo ) of xo , there exists a x ∈ A0 such that x ∈ N w (xo ), so that N w (xo ) ∩ A0 ≠ 0. Since N w (xo ) is arbitrary, we have that for every open neighborhood N w (xo ) of xo it holds w that N w (xo ) ∩ A0 ≠ 0, which implies that xo ∈ A0 . Now, let X0 be the smallest closed linear subspace of X which contains A0 and xo . Then X0 is a separable and reflexive Banach space. By the Hahn–Banach theorem, each functional x⋆0 ∈ X0⋆ can be extended to a functional x⋆ ∈ X ⋆ . Therefore, xo lies in the weak closure of A0 with respect to the weak topology on X0 . Since A0 is bounded and X0 is separable and reflexive, the weak X0 topology on A0 is metrizable. Thus, there exists a sequence {xn : n ∈ ℕ} ⊂ A0 such that xn ⇀ xo in X0 . Since X ⋆ ⊂ X0⋆ , we conclude that xn ⇀ x in X. As is natural certain functions which are continuous with respect to the strong topology may no longer be continuous in the weak topology, as for example the function x 󳨃→ ‖x‖X , but this important function remains sequentially lower semicontinuous with respect to the weak topology. Furthermore, the weak topology has some rather convenient properties with respect to convexity, such as for instance that convex sets which are closed under the strong topology are also closed under the weak topology, but those will be treated in detail in Section 1.2.2.

1.2 Convex subsets of Banach spaces and their properties Convexity plays a fundamental role in analysis. Here, we review some of the fundamental concepts that will be very frequently used in this book. For detailed accounts of convexity, we refer to, e. g., [5, 19] or [25].

1.2.1 Definitions and elementary properties We now introduce the concept of a convex subset of X.

1.2 Convex subsets of Banach spaces and their properties | 23

Definition 1.2.1. Let X be a Banach space and C ⊂ X. The set C is called convex if it has the property tx1 + (1 − t)x2 ∈ C,

∀ x1 , x2 ∈ C, ∀ t ∈ [0, 1].

Notation 1.2.2. We will also use the notation [x1 , x2 ] = {tx1 + (1 − t)x2 : t ∈ [0, 1]}, (x1 , x2 ] = {tx1 + (1 − t)x2 : t ∈ (0, 1]} and [x1 , x2 ) = {tx1 + (1 − t)x2 : t ∈ [0, 1)} to simplify the exposition. With this notation, C is convex if for any x1 , x2 ∈ C it holds that [x1 , x2 ] ⊂ C. Example 1.2.3. Let 𝒟 ⊂ ℝd and take X = Lp (𝒟) for p ∈ [1, ∞]. Then each element x ∈ X is identified with a function u : 𝒟 → ℝ such that ‖u‖Lp (𝒟) < ∞. If ψ is any element of X = Lp (𝒟), the set C := {u ∈ Lp (𝒟) : u(x) ≥ ψ(x) a.e.} is a convex subset of X. ◁ The following topological properties of convex sets (see, e. g., [5]) are useful. Proposition 1.2.4 (Intersections, interior and closure of convex sets). In what follows, C ⊂ X will be a convex set. (i) Let {Cα : α ∈ ℐ } be an arbitrary family of convex sets. Then, ⋂α∈ℐ Cα is also convex. (ii) If x ∈ int( C) and z ∈ C, then [x, z) ⊂ int( C). (iii) If x1 ∈ int( C) and x2 ∈ C then [x1 , x2 ) ∈ int( C). (iv) The interior int( C) and closure C are convex sets. (v) If int( C) ≠ 0, then int( C) = int( C) and int( C) = C. A useful concept is that of the algebraic interior. Definition 1.2.5 (Algebraic interior). The algebraic interior or core of a set is defined as core(C) = {x ∈ C : ∀ z ∈ SX , ∃ϵ > 0 such that x + δz ∈ C, ∀ δ ∈ [0, ϵ]}. For a convex set C, the core can equivalently be defined as core(C) = {x ∈ C : X = ⋃ λ(C − x)}. λ>0

Clearly, int( C) ⊂ core(C). However, there are important situations in which int( C) = core(C). As we shall see when studying convex functions, it is important to identify the interior of the domain of certain convex sets (as, e. g., the effective domain of a convex function which has important effects on the continuity of the function in question). On the other hand, it is much easier to check if a point is in the core of a convex set. Therefore, knowing when the two sets coincide may often be useful. However, before doing that we will need another definition. Definition 1.2.6 (Convex series and convex series closed (CS) sets). An infinite sum ∞ of the form ∑∞ n=1 λn xn , where λn ∈ [0, 1] with ∑n=1 λn = 1 is called a convex series. A set

24 | 1 Preliminaries C is called convex series closed (or CS- closed) if for every convex series ∑∞ n=1 λn xn = x, with xn ∈ C, it holds that x ∈ C. It is straightforward for the reader to check that since X is a Banach space any convex series of elements on a bounded subset A ⊂ X (which is by construction Cauchy) is convergent. CS-closedness of C, guarantees that the limit is in C. Clearly, every CSclosed set is convex. Every closed convex set C is CS-closed, but the converse does not always hold true as CS-closed sets may not be closed. Therefore, CS-closedness is often a useful generalization of closedness. The following proposition (see [25]) is very useful. Proposition 1.2.7. Let C ⊂ X be a convex set. Then: (i) If int( C) ≠ 0 then int( C) = core(C). (ii) If C can be expressed as the countable union of closed sets C = ⋃n An , then int( C) = core(C). (iii) If C is CS-closed (see Definition 1.2.6) then core(C) = int( C) = int( C).

1.2.2 Separation of convex sets and its consequences Convex sets enjoy separation properties related to the various forms of the Hahn– Banach theorem (see, e. g., [28]). We state a form of this theorem which will be used very often in this book. Definition 1.2.8 (Closed hyperplanes and half spaces). Let X be a normed linear space. (i) The set Hx⋆ ,α ⊂ X defined by Hx⋆ ,α := {x ∈ X : ⟨x⋆ , x⟩ = α} for some x⋆ ∈ X ⋆ \ {0} and α ∈ ℝ, is called a closed hyperplane. (ii) The set Hx⋆ ≤α := {x ∈ X : ⟨x⋆ , x⟩ ≤ α} is called a closed half-space. Using the above notions, we may present the geometric version of the Hahn–Banach theorem, which essentially offers information concerning the containment of convex sets in half-spaces. Theorem 1.2.9 (Strict separation). Suppose C1 and C2 are two disjoint, convex subsets of a normed space X, where C1 is compact and C2 is closed. Then there exists a closed hyperplane that strictly separates C1 and C2 , i. e., there exists a functional x⋆ ∈ X ⋆ \ {0} and an α ∈ ℝ such that ⟨x⋆ , x1 ⟩ < α,

∀ x1 ∈ C1 ,

and

⟨x⋆ , x2 ⟩ > α

∀ x2 ∈ C2 .

A convenient corollary of the separation theorem arises when one of the two convex sets degenerates to a point.

1.2 Convex subsets of Banach spaces and their properties | 25

Proposition 1.2.10. Let C ⊂ X be a nonempty closed convex subset of a normed linear space. Then each xo ∈ ̸ C can be strictly separated from C by a closed hyperplane, i. e., there exists (x⋆ , α) ∈ (X ⋆ \ {0}) × ℝ, such that ⟨x⋆ , xo ⟩ > α,

and

⟨x⋆ , x⟩ < α,

∀ x ∈ C.

This proposition14 essentially states that a nonempty closed convex set C is contained in a (suitably chosen) closed half-space, whereas any point xo ∈ ̸ C in its complement. In fact, this result can be rephrased as that a nonempty closed convex set can be expressed in terms of the intersection of all the closed half-spaces Hx⋆ ≤α that contain it, C = ⋂C⊂Hx⋆ ≤α Hx⋆ ≤α , an observation which is important in its own right. A final reformulation of the separation theorem which is often useful is the following version. Proposition 1.2.11. Let C1 , C2 be two nonempty disjoint convex subsets of a normed linear space, such that C1 has an interior point. Then C1 and C2 can be separated by a closed linear hyperplane, i. e., there exists (x⋆ , α) ∈ (X ⋆ \ {0}) × ℝ, such that ⟨x⋆ , x1 ⟩ ≤ α,

∀ x1 ∈ C1 ,

and

⟨x⋆ , x2 ⟩ ≥ α

∀ x2 ∈ C2 .

As a result of the separation theorem, convex sets have a number of remarkable and very useful properties. For example, for convex sets, the concepts of weak and strong closedness coincide, meaning that if a convex set C ⊂ X is closed in the strong topology of X it is also closed in the weak topology of X. Clearly, the same is true for closed linear subspaces. Proposition 1.2.12. A closed convex set C is also weakly closed. Proof. Assume that C is strongly closed. We will show that it is also weakly closed. This is equivalent to showing that its complement C c = X \ C is open in the weak topology. Consider any xo ∈ C c . Since C is closed and xo ∈ ̸ C, by an application of the Hahn– Banach theorem (in the form of Proposition 1.2.10) there exists x⋆o ∈ X ⋆ and α ∈ ℝ such that ⟨x⋆o , xo ⟩ < α < ⟨x⋆o , z⟩, for any z ∈ C. Define the set V = {x ∈ X : ⟨x⋆o , x⟩ < α}, which is open in the weak topology. Observe that xo ∈ V so N w (xo ) := V is a (weak) neighborhood of xo . By the strict separation V ∩ C = N w (xo ) ∩ C = 0, which implies V = N w (xo ) ⊂ C c . So for every xo ∈ C c we may find a (weak) neighborhood N w (xo ) such that N w (xo ) ⊂ C c which means that C c is weakly open.15 Remark 1.2.13. An alternative way to phrase the above statement is that for a convex w set C in a Banach space X, the strong and the weak closure of C, denoted by C and C , 14 whose proof follows directly from Theorem 1.2.9 by considering the compact set C1 = {xo } and the closed set C2 = C. 15 Since C c is a weak neighborhood of each of its points, C c = ⋃xo ∈Cc N w (xo ) so C c is weakly open as the arbitrary union of weakly open sets.

26 | 1 Preliminaries respectively, coincide. Since the weak topology is coarser than the strong it is clear that w C ⊂ C . It remains to show the opposite inclusion. Since C is (by definition) strongly w closed and convex, by Proposition 1.2.12 it is also weakly closed. Therefore, C ⊂ C w leading to the stated result C = C . Remark 1.2.14. A weakly closed set is also weakly sequentially closed.16 By that observation, the result stated in Proposition 1.2.12 is very useful and will be employed quite often in the following setting: If a convex set C ⊂ X has the property that for any sequence {xn : n ∈ ℕ} ⊂ C such that xn → x in X it holds that x ∈ C then it also has the property that for any sequence {x̄n : n ∈ ℕ} ⊂ C such that x̄n ⇀ x̄ it holds that x̄ ∈ C. An important side result of Proposition 1.2.12 is the following weak compactness result (the reader may also wish to revisit Example 1.1.47). Proposition 1.2.15. Let X be a reflexive Banach space and C ⊂ X a convex, closed and bounded set. Then C is weakly compact. Proof. By Proposition 1.2.12, since C is convex and closed it is also weakly closed. Since C is bounded, there exists λ > 0 such that C ⊂ λBX (0, 1) where BX (0, 1) is the closed unit ball in X. This implies that C = C ∩ λBX (0, 1). Since X is reflexive by the Eberlein– Šmulian theorem, BX (0, 1) is weakly compact; hence, λBX (0, 1) enjoys the same property. Therefore, since C = C ∩ λBX (0, 1), it is expressed as the intersection of the weakly closed set C and the weakly compact set λBX (0, 1) it is weakly compact. An important corollary of the above proposition is Mazur’s lemma. In order to prove it, we need the concept of the convex hull of a set. Definition 1.2.16 (Convex hull). Let A ⊂ X. The convex hull of A denoted by convA is the set of all convex combinations from A, n

n

i=1

i=1

convA = {x : ∃xi ∈ A, ti ∈ [0, 1], i = 1, . . . , n, ∑ ti = 1, and x = ∑ ti xi }. The convex hull of A, convA, is the smallest convex set including A. The smallest closed convex set including A is called the closed convex hull of A and is denoted by convA. It can be easily seen that convA = convA. Proposition 1.2.17 (Mazur). Let X be a Banach space and consider a sequence {xn : n ∈ ℕ} ⊂ X such that xn ⇀ x in X. Then there exists a sequence {zn : n ∈ ℕ} ⊂ X consisting of (finite) convex combinations of terms of the original sequence such that zn → x (strongly). The sequence {zn : n ∈ ℕ} can be constructed so that for each n ∈ ℕ, the term zn is either (i) a (finite) convex combination of the terms {xn , xn+1 , . . .} or (ii) a (finite) convex combination of the terms {x1 , x2 , . . . , xn }. 16 In fact, a closed set is sequentially closed for any topology.

1.2 Convex subsets of Banach spaces and their properties | 27

Proof. The proof is based on Proposition 1.2.12. Since for every n ∈ ℕ, we have that w xn ∈ C := conv{x1 , x2 , . . .} and because xn ⇀ x we have that x ∈ C . But C is convex, w so that by Proposition 1.2.12 it holds that C = C; hence, x ∈ C, so that there exists a sequence {zn : n ∈ ℕ} ⊂ C, strongly converging to x. To show claim (i), fix any n ∈ ℕ. Clearly, xk ∈ Cn := conv{xn , xn+1 , . . .} for any k ≥ n. Since xk ⇀ x as k → ∞, it holds w w that x ∈ Cn . But Cn is a convex set so that by Proposition 1.2.12, Cn = Cn for every n, therefore, x ∈ Cn . The strong closure is characterized as the set of limits of strongly convergent sequences, so there exists a sequence {zk : k ∈ ℕ} ⊂ Cn such that zk → x as k → ∞. Since our result holds for any n, for each k pick n = n(k) = k which leads to the desired result. Claim (ii) follows similarly. Proposition 1.2.18 (Mazur). If X is a Banach space, A ⊂ X compact, then convA is compact. Proof. We briefly sketch the proof.17 If A = {x1 , . . . , xn } is finite, then we may consider convA as the image of the map (t1 , . . . , tn ) 󳨃→ ∑ni=1 ti xi where (t1 , . . . , tn ) takes values in the compact set {(t1 , . . . , tn ) : ti ∈ [0, 1], ∑ni=1 ti = 1, i = 1, . . . , n}. Since this map is continuous, it follows that convA is compact, therefore, convA is also compact. If A is not finite but compact, for any ϵ > 0, there exists a finite set Aϵ = {x1 , . . . , xn } ⊂ A such that A ⊂ ⋃ni=1 B(xi , ϵ). Note that ⋃ni=1 B(xi , ϵ) ⊂ convAϵ + B(0, ϵ), which by the above argument is closed and convex. Therefore, A ⊂ convAϵ + B(0, ϵ) and since convAϵ +B(0, ϵ) is a closed convex set containing A and convA is the smallest closed convex set containing A it must hold that convA ⊂ convAϵ + B(0, ϵ).

(1.3)

Since Aϵ is a discrete set, we know from the first part of the proof that convAϵ is compact so there exists a finite set A1 := {z1 , . . . , zm } ⊂ convAϵ such that convAϵ ⊂ m ⋃m i=1 B(zi , ϵ). Noting that ⋃i=1 B(zi , ϵ) = A1 + B(0, ϵ) and combining the previous inclusion with (1.3) we see that convA ⊂ A1 + B(0, 2ϵ),

(1.4)

which together with the observation that A1 ⊂ convAϵ ⊂ convA allows us to conclude that convA is totally bounded; hence, compact. Remark 1.2.19. A similar result, known as the Krein–Smulian weak compactness theorem, holds for weak compactness. According to that, if A ⊂ X be a weakly compact w set, then convA is also weakly compact. For a proof see, e. g., [82], page 254 or [48], page 164. 17 The reader can find more details in [108], or in a slightly more general framework in [5] or [75].

28 | 1 Preliminaries A natural question arising at this point is concerning the behavior of convex sets of X ⋆ in the weak⋆ topology. In general, Proposition 1.2.12 is not true, i. e., a convex A ⊂ X ⋆ which is closed in the strong topology of X ⋆ is not necessarily closed in the weak⋆ topology of X ⋆ , unless of course X is reflexive. One may construct counterexamples for the general case where X is a nonreflexive Banach space. A general criterion for weak⋆ closedness of convex subsets A ⊂ X ⋆ , is the Krein–Smulian theorem according to which A is weak⋆ closed if and only if A ∩ rBX ⋆ (0, 1) is weak⋆ closed for every r > 0 (such sets are often called bounded weak⋆ closed). If X is a separable Banach space, then one may show that a convex set A ⊂ X ⋆ is bounded weak⋆ closed if and only if it is weak⋆ sequentially, therefore, for separable X, a convex set A ⊂ X ⋆ is weak⋆ closed if and only if it is weak⋆ sequentially closed. This characterization can be quite useful in a number of cases.

1.3 Compact operators and completely continuous operators Compact operators is an important class of operators that plays a fundamental role in both linear and nonlinear analysis. We review here some of the fundamental properties of compact operators (see, e. g., [48]). Definition 1.3.1 (Compact and completely continuous operators). (i) Let X and Y be Banach spaces. An operator A : X → Y (not necessarily linear) is called compact if it maps bounded sets of X into relatively compact sets of Y (i. e., into sets whose closure is a compact set). (ii) Let X and Y be Banach spaces. An operator A : X → Y (not necessarily linear) is called completely continuous (or strongly continuous) if the image of every weakly convergent sequence in X under A converges in the strong (norm) topology of Y, i. e., if xn ⇀ x in X implies A(xn ) → A(x) in Y, (i. e., ‖A(xn ) − A(x)‖Y → 0). Example 1.3.2 (Finite rank operators are compact). Consider a linear operator A ∈ ℒ(X, Y) such that dim(R(A)) < ∞. Such an operator, called a finite rank operator, is compact. This follows easily from the definition since closed and bounded sets in finite dimensional spaces are compact. ◁ In general, the classes of compact operators and completely continuous operators are not comparable. However, the following general results hold. Theorem 1.3.3. Let X, Y, Z be Banach spaces and A : X → Y, B : Y → Z be linear operators. (i) If A is a compact operator, then it is completely continuous. (ii) If X is reflexive, and A is completely continuous, then A is compact. (iii) (Schauder) A bounded operator A is compact if and only if A⋆ is compact.

1.3 Compact operators and completely continuous operators | 29

(iv) The product BA : X → Z is compact if one of the two operators is compact and the other is bounded. (v) Consider a sequence of compact operators {An : n ∈ ℕ} ⊂ ℒ(X, Y) that converges in the strong operator topology to an operator A : X → Y, such that A ∈ ℒ(X, Y). Then A is compact. An important class of compact operators are integral operators. Example 1.3.4 (Certain integral operators are compact). Consider a compact interval [a, b] ⊂ ℝ and the Banach space X = C([a, b]) = {ϕ : [a, b] → ℝ, ϕ continuous}, endowed with the norm ‖ϕ‖ = supt∈[a,b] |ϕ(t)|. An element x ∈ X is identified with a continuous function ϕ : [a, b] → ℝ. We now consider the integral operator A : C([a, b]) → C([a, b]) defined by Aϕ(t) =

ˆ a

b

K(t, τ)ϕ(τ)dτ,

where K : [a, b]×[a, b] → ℝ is continuous, and we use the simpler notation Aϕ instead of A(ϕ) to denote the element of C([a, b]) which is the image of ϕ ∈ C([a, b]) under A. Then A is a linear and compact operator. The result can be extended for integral operators on X = C(𝒟) in the case where 𝒟 ⊂ ℝd is a suitable domain, defined by its ´ action on any ϕ ∈ C(𝒟) by Aϕ(x) = 𝒟 K(x, z)ϕ(z)dz for any z ∈ 𝒟. The linearity of the operator follows by standard properties of the integral. In order to show compactness, consider the bounded set U ⊂ X, of functions ϕ ∈ X such that supt∈[a,b] |ϕ(t)| < c for some constant c. Then the set A(U) is bounded, as can be seen by the estimate 󵄨󵄨ˆ b 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 ≤ (b − a) max 󵄨󵄨󵄨K(t1 , t2 )󵄨󵄨󵄨‖ϕ‖ < c󸀠 K(t, τ)ϕ(τ)dτ 󵄨󵄨 󵄨󵄨 󵄨 󵄨 t1 ,t2 ∈[a,b] 󵄨󵄨 a 󵄨󵄨 for a suitable constant c󸀠 . We will show that A(U) is equicontinuous; hence, relatively compact in X := C([a, b]) by the Arzelá–Ascoli theorem (see Theorem 1.8.3). Indeed, since K is continuous on the compact set [a, b] × [a, b] it is also uniformly continuous; hence, for every ϵ󸀠 > 0 there exists δ > 0 such that 󵄨󵄨 󵄨 󸀠 󵄨󵄨K(t1 , τ) − K(t2 , τ)󵄨󵄨󵄨 < ϵ

for all t1 , t2 , τ ∈ [a, b] with |t1 − t2 | < δ.

(1.5)

󵄨󵄨 󵄨󵄨 󵄨 󸀠 󵄨󵄨K(t1 , τ) − K(t2 , τ)󵄨󵄨󵄨󵄨󵄨󵄨ϕ(τ)󵄨󵄨󵄨dτ ≤ (b − a)ϵ c,

(1.6)

Then, 󵄨󵄨 󵄨 󵄨󵄨Aϕ(t1 ) − Aϕ(t2 )󵄨󵄨󵄨 ≤

ˆ a

b

30 | 1 Preliminaries ϵ for any ϕ ∈ U. Consider any ϵ > 0. Let ϵ󸀠 = (b−a)c and choose δ > 0 such that (1.5) holds. Then from (1.6) we can conclude that for all t1 , t2 ∈ [a, b] with |t1 − t2 | < δ and all ϕ ∈ U it holds that |Aϕ(t1 ) − Aϕ(t2 )| < ϵ, hence A(U) is equicontinuous and by the Arzelá–Ascoli theorem (see Theorem 1.8.3) we have the compactness of the operator A. ◁

Definition 1.3.5 (Fredholm operators). A bounded linear operator A : X → Y is called a Fredholm operator if (a) R(A) is a closed subset of Y, (b) dim(N(A)) < ∞, and (c) dim(Y/R(A)) < ∞. The quotient space Y/R(A) is often called the cokernel of the operator A, and is denoted by coker(A). The difference index(A) := dim(N(A)) − dim(Y/R(A)) is called the Fredholm index of A. Example 1.3.6 (Finite dimensional operators are Fredholm). If dim(X) < ∞ and dim(Y) < ∞, then any A ∈ ℒ(X, Y) is Fredholm with index(A) = dim(X) − dim(Y). This is a well-known result from linear algebra. Example 1.3.7 (Fredholm operators related to integral operators). In the framework of Example 1.3.4, consider the operator B : X → X defined by B = I − A, which acts ´ on any function ϕ ∈ C(𝒟) by Bϕ(x) = ϕ(x) − 𝒟 K(x, z)ϕ(z)dz. This is a Fredholm operator. Fredholm operators have some interesting properties with respect to duality, as this is reflected in the properties of the adjoint (dual) operator, as well as some useful connections with compact operators. Theorem 1.3.8 (Properties of Fredholm operators). Let A : X → Y be a bounded linear operator between two Banach spaces X, Y. Then: (i) A is Fredholm if and only if A⋆ is Fredholm. (ii) If A is Fredholm then dim(N(A⋆ )) = dim(Y/R(A)) and dim(X ⋆ /R(A⋆ )) = dim(N(A)). (iii) A is a Fredholm operator if and only if there exists a bounded linear operator B : Y → X such that the operators IX − BA : X → X and IY − AB : Y → Y are compact. (iv) If X, Y, Z are Banach spaces and A : X → Y, B : Y → Z are Fredholm operators then their composition BA : X → Z is also a Fredholm operator with index(BA) = index(A) + index(B). (v) If A, B : X → Y are a Fredholm and a compact operator, respectively, then A + B is Fredholm and index(A + B) = index(A). (vi) If A, B : X → Y are a Fredholm and a bounded operator, respectively, then there exists ϵ0 > 0 such that A + ϵB is Fredholm for ϵ < ϵ0 and index(A + B) = index(A). Theorem 1.3.8 in the special case where X = Y can lead to a very useful result, known as the Fredholm alternative, concerning the solvability of linear operator equations, of the form (I − A)x = z, where A : X → X is a linear compact operator and z ∈ X is given.

1.3 Compact operators and completely continuous operators | 31

Theorem 1.3.9 (Fredholm alternative). Let A ∈ ℒ(X) be a compact operator. Then, upon defining the annihilator of A ⊂ X as A⊥ := {x⋆ ∈ X ⋆ : ⟨x⋆ , x⟩ = 0, ∀ x ∈ A}, we have that: dim(N(I − A)) = dim(N(I − A⋆ )) < ∞, and R(I − A) is closed with R(I − A) = N(I − A⋆ )⊥ . In particular N(I − A) = {0} if and only if R(I − A) = X. In particular, the Fredholm alternative states that either the operator equation (I − A)x = z admits a unique solution for any z ∈ X, or the homogeneous equation (I − A)x = 0 admits n linearly independent solutions and then the inhomogeneous equation admits solutions only as long as z ∈ N(I −A⋆ )⊥ . Then the solvability condition of the operator equation (I − A)x = z is related to the existence of nontrivial solutions to problem (I − A)x = 0 or the invertibility of the operator I − A. The latter problem is called an eigenvalue problem. Example 1.3.10 (Solvability of integral equations). In the multidimensional framework of Example 1.3.4, consider the integral equation ˆ K(x, z)ϕ(z)dz = ψ(x), ϕ(x) − λ 𝒟

for any x ∈ 𝒟, where ψ ∈ C(𝒟) is a known function and λ ∈ ℝ is a known constant. Then, on account of the results of Example 1.3.4 we may apply the Fredholm alternative and conclude that this problem admits a unique solution for any ψ ∈ C(𝒟) as long ´ as λ ∈ ℝ is such that the homogeneous problem ϕ(x) − λ 𝒟 K(x, z)ϕ(z)dz = 0 admits only the trivial solution ϕ(x) = 0 for all x ∈ 𝒟. This tells us that the solvability of the ´ problem occurs only for special types of ψ, if λ is such that ϕ(x)−λ 𝒟 K(x, z)ϕ(z)dz = 0 admits nontrivial solutions, i. e., if λ coincides with eigenvalues of the integral operator. A characterization of such ψ requires the study of the dual operator. ◁ As problems of the form (I − A)x = 0 play an important role in linear functional analysis we close this section by recalling some fundamental facts concerning the spectrum of linear operators in Banach spaces. Since it is well known from linear algebra that real matrices may well have complex eigenfunctions, the same is anticipated to hold in infinite dimensions and we therefore in general need to consider complex valued Banach spaces, in which the concept of the norm is generalized so that ‖λx‖ = |λ| ‖x‖ for all x ∈ X and λ ∈ ℂ, so as to treat this problem in its full generality. However, here this is not required for the needs of the material covered, and we will (following [28]) present a selected view of these concepts restricting ourselves to real valued Banach spaces. For a more general viewpoint, we refer, e. g., to [82]. Definition 1.3.11 (The resolvent set and the spectrum). Let A : X → X be a bounded linear operator. (i) The resolvent set of A is the set ρ(A) := {λ ∈ ℝ : (A − λI) : X → X is bijective}.

32 | 1 Preliminaries (ii) The spectrum of A is the set σ(A) := ℝ \ ρ(A). (iii) A real number λ is called an eigenvalue of A if N(A − λI) ≠ {0}. In general, there are elements of σ(A) which are not eigenvalues. As mentioned above, if we worked with complex valued Banach spaces then the spectrum would be defined as σ(A) := {λ ∈ ℂ : λI − A : X → X is not bijective} and this set is expressed as the union of the point spectrum (on which λI − A is not injective), the continuous spectrum (on which λI − A is injective but its image is not dense) and the residual spectrum (where λI − A is injective with dense image but not surjective). Theorem 1.3.12 (Spectrum of compact operators). Let X be an infinite dimensional Banach space and A ∈ ℒ(X) be a compact operator. Then 0 ∈ σ(A) and σ(A) consists either (a) only of 0, or (b) of 0 and a finite set of eigenvalues, or (c) of 0 and a countable set of eigenvalues converging to 0. A lot more can be said when X = H a separable Hilbert space and A : H → H has some rather special properties, in particular that A⋆ = A (see, e. g., [28]). Theorem 1.3.13. Let X = H be a separable infinite dimensional Hilbert space and A : H → H be a linear self-adjoint compact operator. Then there exists a Hilbert basis {xn : n ∈ ℕ}, composed of eigenvectors of A. Example 1.3.14 (Construction of a basis using eigenvectors of an operator). Construct a basis for H consisting of the eigenvectors of the compact self-adjoint operator18 A, assuming for simplicity that A is also positive definite, i. e., ⟨Ax, x⟩ > 0 for every x ∈ H \ {0}. For a self-adjoint operator19 σ(A) ≠ {0} (unless A = 0), so let {λn : n ∈ ℕ} be the countable set of (distinct) eigenvalues of A, setting also λ0 = 0, and define the closed linear subspaces En = N(λn I − A), n = 0, 1, . . ., which can easily be seen to be orthogonal (E0 = N(A) = {0} by the positive definite property20 while En , n = 1, . . . are ⊥ finite dimensional). Let E = span(⋃∞ n=1 En ), and note that both E and E are invariant under the action of A. Consider the operator A0 = A|E ⊥ : E ⊥ → E ⊥ which is in turn a selfadjoint compact operator for which21 σ(A0 ) = {0}, and this in turn shows that A0 = 0, i. e., Ax = 0 for every x ∈ E ⊥ ; hence, E ⊥ ⊂ N(A) = {0}, so E ⊥ = {0}, which implies the density of22 E in H. Since each of the En , n ∈ ℕ is finite dimensional, form a basis on H 18 This is essentially the proof of Theorem 1.3.13. 19 As can be seen from instance by the observation that for self-adjoint operators the spectral norm ρ(A) := sup{|λ| : λ ∈ σ(A)} = ‖A‖, see also [28] for an alternative proof. 20 For a general operator, E0 may be infinite dimensional. 21 Suppose not, then there exists x ∈ E ⊥ \ {0} ⊂ H and λ ≠ 0 such that A0 x = λx, i. e., λ is one of the eigenvalues of A, say λn , and x ∈ E n ⊂ E. Hence, x ∈ E ∩ E ⊥ , therefore, x = 0 contradiction. 22 One could obtain the same result omitting the positive definite property, by defining E = ⋃∞ n=0 En and proceeding as above to conclude that E ⊥ ⊂ N(A) ⊂ E; hence, E ⊥ = {0}.

1.4 Lebesgue spaces |

33

by taking the union of the bases of each En . This basis allows us to express the action ∞ of the operator on any element x = ∑∞ n=1 xn , xn ∈ En as Ax = ∑n=1 λn xn (as the finite rank k k operators Ak : H → ⋃n=1 En , defined by Ak x = ∑n=1 λn xn converge in the operator norm to A since ‖Ak − A‖ ≤ supn≥k+1 |λn | → 0 as k → ∞). It is often convenient, by counting the eigenvalues with multiplicity, so that they are not necessarily distinct anymore, and denoting the new sequence by {λn󸀠 : n ∈ ℕ} to reorganize this basis as a basis {x󸀠n : n ∈ ℕ} with the property that Ax󸀠n = λn󸀠 x󸀠n , allowing for more straightforward eigenvector expansions. ◁

1.4 Lebesgue spaces A fundamental example of Banach spaces of great importance in applications are Lebesgue and Sobolev spaces. As Lebesgue spaces are usually familiar to any one that has had a first course in measure theory or functional analysis we will here only recall some very fundamental facts concerning Lebesgue spaces (for an excellent account see [109]). We also assume familiarity of the reader with the fundamentals of Lebesgue’s theory of integration. Even though, Lebesgue spaces can be defined on general measure spaces and for vector valued functions these extensions will not be needed for most of the material covered in this book, with minor exceptions, so in this section we contain ourselves to an exposition of the theory for functions u : 𝒟 ⊂ ℝd → ℝ, or when needed to functions v : 𝒟 ⊂ ℝd → ℝm , where 𝒟 is an open subset of ℝd . The more general case will be sketched where and if needed. Let 𝒟 ⊂ ℝd be a bounded open subset and let μℒ be the Lebesgue measure on 𝒟. We will denote by ℬ(𝒟) the Borel σ-algebra on 𝒟 (i. e., the smallest σ-algebra containing all open subsets of 𝒟) and from now on we will restrict our attention to Borel measurable functions on 𝒟 (i. e., functions u : 𝒟 → ℝ with the property that for any Borel measurable set I ⊂ ℬ(ℝ) it holds that u−1 (I) ∈ ℬ(𝒟)), hereafter simply called measurable. We will also use the notation |A| for μℒ (A) for every A ∈ ℬ(𝒟). For simplicity, we will denote the Lebesgue integral of such a function over any A ⊂ 𝒟 by ´ ´ A udx = 𝒟 u 1A dx. Definition 1.4.1 (Lebesgue spaces). For 1 ≤ p ≤ ∞, the function spaces Lp (𝒟) := {u : 𝒟 → ℝ : measurable with

ˆ

𝒟

|u|p dx < ∞},

p ∈ [1, ∞),

L∞ (𝒟) := {u : 𝒟 → ℝ : measurable with ess sup𝒟 |u| < ∞}. are23 called the Lebesgue spaces of order p. 23 Recall that for a measurable function u, the essential supremum is defined as ess sup𝒟 u = inf{c ∈ ℝ : μ{x ∈ 𝒟 : u(x) > c} = 0}. In most cases, we will use the simplified notation sup, implicitly meaning the essential supremum.

34 | 1 Preliminaries We will also define the spaces Lploc (𝒟), 1 ≤ p ≤ ∞ as the spaces of all functions u : 𝒟 → ℝ such that u 1K ∈ Lp (𝒟) for every compact subset K ⊂ 𝒟. It is easily seen that these function spaces are vector spaces. In fact, they can be turned into Banach spaces with a suitable norm. Theorem 1.4.2. The spaces Lp (𝒟) equipped with the norms ‖u‖Lp (𝒟)

ˆ 1/p := { |u|p dx} , 𝒟

p ∈ [1, ∞),

‖u‖L∞ (𝒟) := ess sup𝒟 |u|, are Banach spaces, with L2 (𝒟) being a Hilbert space. In particular, the Lp norms for ⋆ p ∈ [1, ∞] satisfy the Hölder inequality according to which if u1 ∈ Lp (𝒟), u2 ∈ Lp (𝒟), with the conjugate exponent p⋆ defined by p1 + p1⋆ = 1, it holds that ‖u1 u2 ‖L1 (𝒟) ≤ ‖u1 ‖Lp (𝒟) ‖u2 ‖Lp⋆ (𝒟) , where the case p = ∞ is included with p⋆ = 1. As a simple consequence of this inequality, we see that if |𝒟| < ∞, then Lq (𝒟) ⊂ Lp (𝒟) if p < q and ‖u‖Lp (𝒟) ≤ |𝒟|1/p−1/q ‖u‖Lq (𝒟) . Using the Hölder inequality, we can see that as long as |𝒟| < ∞ an Lq (𝒟) function can be considered as in Lp (𝒟) if p < q, in the sense that the identity map I : Lq (𝒟) → Lp (𝒟) is continuous. For various reasons, it may be convenient to consider the “outmost” case of L1 (𝒟) as subset of a larger space, the space of Radon measures. Connecting measures with integrable functions is rather natural. One of the ways to envisage that is to recall the Radon–Nikodym theorem according to which if a measure ν on ℬ(𝒟) is absolutely continuous to the Lebesgue measure on24 ℬ(𝒟), then there ex´ ists a function u ∈ L1 (𝒟) such that ν(A) = A udx, for every A ∈ ℬ(𝒟). In this sense one may consider some measures on ℬ(𝒟) (i. e., those absolutely continuous to the Lebesgue measure) as elements of L1 (𝒟) and, therefore, try to embed L1 (𝒟) in a larger space of signed measures. We introduce the following definition. Definition 1.4.3 (Radon measures on ℝd ). An outer measure ν on D ⊂ ℝd is called a Radon measure if it is Borel regular, i. e., if for every A ⊂ D ⊂ ℝd there exists a Borel set B such that ν(A) = ν(B), and it is finite on compact sets. The set of signed Radon measures on D ⊂ ℝd will be denoted by ℳ(D) and can be turned into a Banach space with the total variation norm defined by ‖ν‖ℳ(D) := |ν| := sup{∑i |ν(Di )| : {Di } countable partition of D}. 24 In the sense that for any A ⊂ 𝒟 such that |A| := μℒ (A) = 0, it holds that ν(A) = 0.

1.4 Lebesgue spaces |

35

The space of Radon measures is related via duality with the space of continuous functions as the following theorem asserts (see, e. g., Theorems 19.54 and 19.55 of [109]). Theorem 1.4.4. Let 𝒟 ⊂ ℝd be a bounded open set and consider the Banach space Cc (𝒟) := {ϕ ∈ C(𝒟) : ϕ(x) = 0, ∀ x ∈ 𝒟 \ 𝒟o , 𝒟o ⊂ 𝒟, compact}, the space of continuous functions on 𝒟 with compact support, equipped with the norm ‖ϕ‖Cc (𝒟) = supx∈𝒟 |ϕ(x)|. Then, (Cc (𝒟))⋆ ≃ ℳ(𝒟) is the following sense: (i) For every continuous linear mapping f : Cc (𝒟) → ℝ, there exists a unique signed ´ Radon measure ν ∈ ℳ(𝒟) such that f (ϕ) = 𝒟 ϕdν for every ϕ ∈ Cc (𝒟) and ‖f ‖(Cc (𝒟))⋆ = |ν|, where |ν| is the total variation of the Radon measure ν and (ii) If for each ν ∈ ℳ(𝒟), we consider the bounded linear functional jν : Cc (𝒟) → ℝ ´ defined by jν (ϕ) = 𝒟 ϕdν then the mapping ν 󳨃→ jν is a linear isometry from ℳ(𝒟) to (Cc (𝒟))⋆ . The same results hold upon replacing Cc (𝒟) by C(𝒟) equipped with the usual norm ‖ϕ‖C(𝒟) := supx∈𝒟 |ϕ(x)| and ℳ(𝒟) by ℳ(𝒟). The above theorem provides an alternative definition for the total variation norm ´ in terms of ‖ν‖ℳ(D) = sup{ 𝒟 ϕdν : ϕ ∈ Cc (𝒟), ‖ϕ‖Cc (𝒟) ≤ 1}. Via the connection of the two spaces we may also define a convenient weak⋆ topology on ℳ(D) and a useful notion of weak⋆ convergence. It also allows us to consider the following inclusions Lq (𝒟) ⊂ Lp (𝒟) ⊂ ℳ(𝒟) for 1 ≤ p < q ≤ ∞ (and μ(𝒟) < ∞). This inclusion will be very useful when considering convergence of sequences in Lebesgue spaces. We are now ready to present some fundamental results concerning the characterization of the dual spaces of Lebesgue spaces. Theorem 1.4.5 (Duals of Lebesgue spaces). Let 𝒟 ⊂ ℝd open and bounded. ⋆ (i) If p ∈ (1, ∞), then (Lp (𝒟))⋆ ≃ Lp (𝒟), where p ∈ (1, ∞) and p1 + p1⋆ = 1, in terms of ´ the duality pairing ⟨u, v⟩(Lp (𝒟))⋆ ,Lp (𝒟) = 𝒟 uvdx. In particular, the spaces Lp (𝒟) are reflexive and separable for p ∈ (1, ∞). ´ (ii) (L1 (𝒟))⋆ ≃ L∞ (𝒟) in terms of ⟨u, v⟩(L1 (𝒟))⋆ ,L1 (𝒟) = 𝒟 uvdx, and L1 (𝒟) is separable but (on account of (iii) below) L1 (𝒟) is not reflexive. (iii) (L∞ (𝒟))⋆ ⊂ L1 (𝒟), with the inclusion being strict. L∞ (𝒟) is neither separable nor reflexive. The result that (L∞ (𝒟))⋆ ⊊ L1 (𝒟) (with the inclusion being strict) means that there exist continuous linear mappings f : L∞ (𝒟) → ℝ that may not be represented as ´ f (u) = 𝒟 vudx for some v ∈ L1 (𝒟) (depending only on f ) for every u ∈ L∞ (𝒟). The full characterization of the space (L∞ (𝒟))⋆ requires the space of finite additive bounded variation measures, but this will not be needed here.

36 | 1 Preliminaries Example 1.4.6 (The duality map for Lebesgue spaces). Recall the definition of the duality map (see Definition 1.1.16). Letting X = Lp (𝒟), 1 < p < ∞ and identifying x with ⋆ a function u : 𝒟 → ℝ such that u ∈ Lp (𝒟) we have that J(u) = {v} where v ∈ Lp (𝒟) 2−p and is defined by v(x) = ‖u‖Lp (𝒟) |u(x)|sgn(u)(x) a.e x ∈ 𝒟, where p⋆ is the conjugate exponent of p and sgn(u)(x) = 1u(x)>0 (x) − 1u(x) 0} { { { wo (x) = {w, on {x ∈ 𝒟 : u(x) = 0} { { {−1, on {x ∈ 𝒟 : u(x) < 0}, where w is any element of L∞ (𝒟) such that ‖w‖L∞ (𝒟) < 1. This is a multivalued (or set-valued) map. ◁ Functions in Lebesgue spaces can be approximated by sequences of continuous or even smooth functions as the following density result shows. Theorem 1.4.7 (Density results for Lebesgue spaces). The space Cc (𝒟) := {ϕ ∈ C(𝒟) : ϕ(x) = 0, ∀ x ∈ 𝒟 \ 𝒟o , 𝒟o ⊂ 𝒟, compact} is dense in Lp (𝒟) for p ∈ [1, ∞). The same holds for Cc∞ (𝒟) := {ϕ ∈ C ∞ (𝒟) : ϕ(x) = 0, ∀ x ∈ 𝒟 \ 𝒟o , 𝒟o ⊂ 𝒟, compact}. The characterization of the duals of various Lebesgue spaces allows us to use the general concepts of strong, weak and weak⋆ convergence that have been provided in Sections 1.1.5.3, 1.1.6, 1.1.7. The definitions are straightforward adaptations of the general definitions provided in these sections using the characterization of the dual spaces given in Theorem 1.4.5. Since we consider L1 (𝒟) as a subset of ℳ(𝒟), we will also require a concept of convergence in this space, which will follow from the characterization of ℳ(𝒟) as a dual space (see Theorem 1.4.4), which will correspond to weak⋆ convergence. This discussion leads to the following definitions which are collected in concrete form for the convenience of the reader. Definition 1.4.8 (Various modes of weak and strong convergence). The following modes of weak convergence are frequently used: ´ ´ ⋆ (i) If {νn : n ∈ ℕ} ⊂ ℳ(𝒟), then νn ⇀ ν if 𝒟 ϕdνn → 𝒟 ϕdν for every ϕ ∈ Cc (𝒟). ´ ´ (ii) If {un : n ∈ ℕ} ⊂ Lp (𝒟), p ∈ [1, ∞), then un ⇀ u if 𝒟 un vdx → 𝒟 uvdx for every ⋆ v ∈ Lp (𝒟) where p⋆ is the conjugate exponent of p. ´ ´ ⋆ (iii) If {un : n ∈ ℕ} ⊂ L∞ (𝒟), then un ⇀ u if 𝒟 un vdx → 𝒟 uvdx for every v ∈ L1 (𝒟).

1.4 Lebesgue spaces |

37

(iv) If {un : n ∈ ℕ} ⊂ Lp (𝒟), p ∈ [1, ∞], then un → u in Lp (𝒟) if ‖un − u‖Lp (𝒟) → 0. (v) For any sequence of functions {un : n ∈ ℕ}, then un → u, a. e. if μℒ {x ∈ 𝒟 : un (x) ↛ u(x)} = 0. The following results are classical (see, e. g., [109]) and are collected here for the convenience of the reader. Theorem 1.4.9. The following hold: ´ ´ (i) Fatou’s lemma: If un ≥ 0 and lim infn un = u a. e., then 𝒟 udx ≤ lim infn 𝒟 un dx. (ii) Monotone convergence: If un ≤ un+1 a. e. for every n ∈ ℕ and un → u a. e., then ´ ´ limn 𝒟 un dx = 𝒟 udx. (iii) Lebesgue’s dominated convergence: If un → u a. e. and there exists h ∈ L1 (𝒟) such ´ ´ that ‖un ‖L1 (𝒟) ≤ h for every n ∈ ℕ, then limn 𝒟 un dx = 𝒟 udx. (iv) If un → u in Lp (𝒟), p ∈ [1, ∞), then there exists a subsequence {unk : k ∈ ℕ} such that unk ≤ h, a. e., where h ∈ Lp (𝒟), with unk → u a. e. On the other hand, if {un : n ∈ ℕ} ⊂ Lp (𝒟), u ∈ Lp (𝒟), p ∈ [1, ∞) and un → u a. e. and ‖un ‖Lp (𝒟) → ‖u‖Lp (𝒟) , then un → u in Lp (𝒟). The weak⋆ convergence of measures will turn out to be a very useful tool as it satisfies some very useful lower semicontinuity and compactness properties. To anticipate its use, consider the following line of argumentation. Since Lp (𝒟) are reflexive for p ∈ (1, ∞) we know from abstract arguments based on the use of Theorem 1.1.58 that bounded sequences in Lp (𝒟) admit a subsequence having a weak limit in the same space. Similarly, by using the general abstract arguments for the weak⋆ topology the same is true in L∞ (𝒟) with respect to the weak⋆ convergence. This argument, however, cannot be extended to bounded sequences in L1 (𝒟), where by the nonreflexivity it may well be that bounded sequences in L1 (𝒟) may not admit subsequences with weak limits in the same space. To save the situation, we may always consider a bounded sequence in L1 (𝒟) as a bounded sequence in the larger space ℳ(𝒟) (or ℳ(𝒟)) and then use the weak⋆ compactness of measures in order to obtain a limit which may be a measure rather than an L1 (𝒟) function. In order to guarantee that the limit of a uniformly bounded sequence in L1 (𝒟) is indeed in L1 (𝒟), rather than in the larger space ℳ(𝒟), we need a more restrictive condition, that of uniform integrability (or equiintegrability) which requires that ∀ ϵ > 0, ∃δ > 0 ∀ E ⊂ 𝒟 with |E| < δ it holds that

ˆ E

|un |dx < ϵ, ∀ n ∈ ℕ.

(1.7)

Note that for any sequence {un : n ∈ ℕ} ⊂ L1 (𝒟) such that supn∈ℕ ‖un ‖L1 (𝒟) < ∞, property (1.7) holds but in general δ = δ(ϵ, n), while uniform integrability requires that the δ = δ(ϵ) only, i. e., the same choice of δ will work for the whole family; hence, uniform integrability is a more restrictive condition. In the case of bounded 𝒟 ⊂ ℝd considered

38 | 1 Preliminaries here, we have an alternative, sometimes easier to verify, equivalent criterion for uniform integrability, the De la Vallée–Pousin criterion according to which a sequence {un : n ∈ ℕ} ⊂ L1 (𝒟) is uniformly integrable if and only if there exists an increasing ´ function γ : ℝ+ → ℝ+ with limt→∞ γ(t) = ∞, such that supn 𝒟 γ(|un |)dx < ∞. t In the following theorem, we collect some useful weak compactness results for Lebesgue spaces and measures. Theorem 1.4.10 (Weak⋆ compactness for Lebesgue spaces and measures). Let 𝒟 ⊂ ℝd be a bounded open set. (i) If {μn : n ∈ ℕ} ⊂ ℳ(𝒟) is a bounded sequence (in terms of the total variation norm), then there exists a measure μ ∈ ℳ(𝒟) and a subsequence {μnk : k ∈ ℕ} ⊂ ℳ(𝒟) such that μnk ⇀ μ in ℳ(𝒟) as k → ∞. Furthermore, for any A ⊂ 𝒟 open it ⋆

holds that μ(A) ≤ lim infk μnk (A). (ii) If {un : n ∈ ℕ} ⊂ L1 (𝒟) is a bounded sequence, then there exists a function u ∈ L1 (𝒟) and a subsequence {unk : k ∈ ℕ} such that unk ⇀ u in L1 (𝒟) if and only if {un : n ∈ ℕ} is uniformly integrable or, equivalently, if and only if there exists an increasing ´ function γ : ℝ+ → ℝ+ with limt→∞ γ(t) = ∞, such that supn 𝒟 γ(|un |)dx < ∞. t (iii) If {un : n ∈ ℕ} ⊂ Lp (𝒟), p ∈ (1, ∞), is a bounded sequence, then there exists a function u ∈ Lp (𝒟) and a subsequence {unk : k ∈ ℕ} such that unk ⇀ u in Lp (𝒟). (iv) If {un : n ∈ ℕ} ⊂ L∞ (𝒟) is a bounded sequence, then there exists a function u ∈ ⋆ L∞ (𝒟) and a subsequence {unk : k ∈ ℕ} such that unk ⇀ u in L∞ (𝒟). Assertions (i), (iii) and (iv) of the above theorem follow from the general properties of weak⋆ convergence and weak convergence in reflexive spaces, whereas (ii) is a consequence of the Dunford–Pettis weak compactness criterion for L1 (𝒟) combined with the De la Vallée–Poussin criterion for uniform integrability. The weak compactness results allow us to guarantee, under boundedness conditions, the existence of weakly converging subsequences. Clearly, weak convergence does not imply strong convergence, nor does it guarantee the existence of an a. e. converging subsequence (as does the strong convergence) unless some extra conditions are met. The following proposition (see, e. g., Corollaries 2.49 and 2.58, [66]) summarizes certain such cases. Proposition 1.4.11. Let p ∈ (1, ∞). (i) If {un : n ∈ ℕ} ⊂ Lp (𝒟) and ‖un ‖Lp (𝒟) < c for every n ∈ ℕ, then if un → u a. e. it holds that u ∈ Lp (𝒟) and un ⇀ u in Lp (𝒟). (ii) If un ⇀ u in Lp (𝒟) and ‖un ‖Lp (𝒟) → ‖u‖Lp (𝒟) , then un → u in Lp (𝒟). A very useful result is the following. Proposition 1.4.12 (Vitali). Let p ∈ [1, ∞) and {fn : n ∈ ℕ} be a sequence of measurable functions on 𝒟 ⊂ ℝd (with 𝒟 not necessarily bounded). Then fn → f in Lp (𝒟) if and only if:

1.5 Sobolev spaces | 39

(i) fn → f in measure. (ii) The sequence {|fn |p : n ∈ ℕ} is equiintegrable. ´ (iii) For every ϵ > 0, there exists A ⊂ 𝒟 such that |A| < ∞ and 𝒟\A |fn |p dx < ϵ for every n. Remark 1.4.13. Conditions (ii) and (iii) are always satisfied if there exists a function g ∈ Lp (𝒟) such that |fn | ≤ g a. e. for every n. Condition (iii) is always true if |𝒟| < ∞. Operators between Lebesgue spaces will play an important role in this book. Example 1.4.14 (Integral operators between Lebesgue spaces). Consider the operator A : L2 (𝒟) → L2 (𝒟) acting on any ϕ ∈ L2 (𝒟) by ˆ Aϕ(x) = K(x, z)ϕ(z)dz, a.e., x ∈ 𝒟, 𝒟

where K : 𝒟 × 𝒟 → ℝ is a continuous function. Then A is a compact linear operator. The same result applies if the kernel function K is L2 (𝒟 × 𝒟). ◁ One may easily extend the above definitions and properties to Lebesgue spaces for vector valued function u : 𝒟 ⊂ ℝd → ℝm , by defining the Lebesgue spaces Lp (𝒟; ℝm ) as follows. Since any u : 𝒟 ⊂ ℝd → ℝm can be represented as u = (u1 , . . . , um ), where ui : 𝒟 ⊂ ℝd → ℝ, i = 1, . . . , m, we may define Lp (𝒟; ℝm ) = {u : 𝒟 → ℝm : ui ∈ Lp (𝒟), i = 1, . . . , m},

p ∈ [1, ∞].

p 1/p A suitable norm for such spaces can be ‖u‖Lp (𝒟;ℝm ) = {∑m i=1 ‖ui ‖Lp (𝒟) } , for p ∈ [1, ∞) or ‖u‖L∞ (𝒟);ℝm = maxi=1,...,m ‖ui ‖L∞ (𝒟) , but other suitable choices of equivalent norms are possible. Unless absolutely necessary, we will use the simplified notation Lp (𝒟) for Lp (𝒟; ℝm ).

1.5 Sobolev spaces Sobolev spaces are fundamental in the study of variational problems and in partial differential equations. We recall some important facts concerning Sobolev spaces (for a detailed account see, e. g., [1, 28] or [77]). As before, let 𝒟 ⊆ ℝd be a nonempty open set, and let Cc∞ (𝒟) be the set of infinitely continuously differentiable functions on 𝒟 with compact support. Definition 1.5.1. The Sobolev spaces W 1,p (𝒟). 1 ≤ p < ∞, are defined as W 1,p (𝒟) := {u ∈ Lp (𝒟) : ∃ vi ∈ Lp (𝒟), such that, ∀ ϕ ∈ Cc∞ (𝒟), i = 1, . . . , d}.

ˆ 𝒟

u

𝜕ϕ dx = − 𝜕xi

ˆ 𝒟

vi ϕdx,

40 | 1 Preliminaries The functions vi are called the weak partial derivatives of u in the direction xi , and are 1,p 𝜕u denoted by vi = 𝜕x (𝒟), 1 ≤ p ≤ ∞ as the spaces of . We will also define the spaces Wloc i

all functions u : 𝒟 → ℝ such that u 1K ∈ W 1,p (𝒟) for every compact subset K ⊂ 𝒟.

The weak derivatives are best understood in the sense of distributions, using the concept of duality for the space Cc∞ (𝒟), and can be shown to enjoy a number of convenient properties of the classical derivative such as for instance the Leibnitz rule. One may define higher weak derivatives in the same fashion, i. e., using the integration of parts formula for a function that is infinitely smooth. For example, we may define the ´ ´ 𝜕2 ϕ functions vij in terms of the integration by parts formula 𝒟 vij ϕdx = 𝒟 u 𝜕x 𝜕x dx for j

i

every ϕ ∈ Cc∞ (𝒟) and understand the function vij as the weak second partial derivative

𝜕2 u . Higher weak derivatives will be 𝜕xj 𝜕xi compactly denoted using the convenient multiindex notation α = (α1 , . . . , αd ) ∈ ℕd , |α| and upon defining |α| = ∑di=1 αi , we will denote 𝜕xα u := α1𝜕 αd u. When α = (0, . . . , 0), 𝜕x1 ⋅⋅⋅𝜕xd 𝜕u . we have that 𝜕xα u = u, and when α = ei = (δij : j = 1, . . . , d) we have that 𝜕xα u = 𝜕x i

of u with respect to xi and xj , denoting it by vij =

Example 1.5.2. If u ∈ W 1,2 (𝒟), then u− := max(−u, 0) ∈ W 1,2 (𝒟) also, and ∇u− = −∇u1u0 . ◁ We can then define higher order Sobolev spaces recursively as follows. Definition 1.5.3 (The Sobolev spaces W k,p (𝒟)). The Sobolev spaces W k,p (𝒟), 1 < p ≤ ∞, are the spaces25 W k,p (𝒟) := {u ∈ W k−1,p (𝒟) :

𝜕u ∈ W k−1,p (𝒟), i = 1, . . . , d}. 𝜕xi

k,p We will also define the spaces Wloc (𝒟), 1 ≤ p ≤ ∞ as the spaces of all functions k,p u : 𝒟 → ℝ such that u 1K ∈ W (𝒟) for every compact subset K ⊂ 𝒟.

The Sobolev spaces can be turned into Banach spaces with a suitable norm. Theorem 1.5.4 (W k,p (𝒟) are Banach spaces). Let 𝒟 ⊂ ℝd be open and bounded. (i) The Sobolev spaces W 1,p (𝒟), p ∈ [1, ∞] when endowed with one of the two equivalent norms ‖u‖W 1,p (𝒟) ‖u‖W 1,∞ (𝒟)

1/p

󵄩󵄩 𝜕u 󵄩󵄩p 󵄩 󵄩 := + ∑󵄩󵄩󵄩 󵄩󵄩󵄩 } , p ∈ [1, ∞), 󵄩 𝜕x 󵄩 p i=1 󵄩 i 󵄩L (𝒟) 󵄩󵄩 𝜕u 󵄩󵄩 󵄩󵄩 𝜕u 󵄩󵄩 󵄩 󵄩󵄩 󵄩 󵄩󵄩 , . . . , 󵄩󵄩󵄩 } := max{‖u‖L∞ (𝒟) , 󵄩󵄩󵄩 󵄩󵄩 󵄩 󵄩󵄩 𝜕x1 󵄩󵄩L∞ (𝒟) 󵄩󵄩 𝜕xd 󵄩󵄩󵄩L∞ (𝒟) {‖u‖pLp (𝒟)

d

25 Equivalently, u ∈ W k,p (𝒟) if and only if u ∈ Lp (𝒟) and for every α with |α| ≤ k, there exists ´ ´ vα ∈ Lp (𝒟) such that 𝒟 u𝜕xα ϕdx = (−1)|α| 𝒟 vα ϕdx for every ϕ ∈ Cc∞ (𝒟).

1.5 Sobolev spaces | 41

󵄩 󵄩 = max 󵄩󵄩󵄩𝜕xα u󵄩󵄩󵄩L∞ (𝒟) , α, |α|≤1

p = ∞,

or d 󵄩 󵄩󵄩 𝜕u 󵄩󵄩󵄩 , ‖u‖W 1,p (𝒟) = ‖u‖Lp (𝒟) + ∑󵄩󵄩󵄩 󵄩󵄩󵄩 󵄩 𝜕x 󵄩 p i=1 󵄩 i 󵄩L (𝒟)

p ∈ [1, ∞]

are Banach spaces. In the special case p = 2, the space W 1,2 (𝒟) is a Hilbert space with inner product d

⟨u1 , u2 ⟩W 1,2 (𝒟) = ⟨u1 , u2 ⟩L2 (𝒟) + ∑⟨ i=1

𝜕u1 𝜕u2 , ⟩ . 𝜕xi 𝜕xi L2 (𝒟)

W 1,p (𝒟) are separable for p ∈ [1, ∞) but not for p = ∞. (ii) Similarly, the Sobolev spaces W k,p (𝒟), p ∈ [1, ∞] endowed with any of the equivalent norms 1/p

󵄩 󵄩p ‖u‖W k,p (𝒟) := ( ∑ 󵄩󵄩󵄩𝜕xα u󵄩󵄩󵄩Lp (𝒟) )

,

󵄩 󵄩 ‖u‖W k,p (𝒟) := max 󵄩󵄩󵄩𝜕xα u󵄩󵄩󵄩L∞ (𝒟) , α, |α|≤k

p = ∞,

|α|≤k

p ∈ [1, ∞),

or 󵄩 󵄩 ‖u‖W k,p (𝒟) := ∑ 󵄩󵄩󵄩𝜕xα u󵄩󵄩󵄩Lp (𝒟) ,

p ∈ [1, ∞],

|α|≤k

are Banach spaces. When p = 2, the space W k,p (𝒟) is a Hilbert space with inner product ⟨u1 , u2 ⟩W k,2 (𝒟) = ∑ ⟨𝜕xα u1 , 𝜕xα u2 ⟩L2 (𝒟) . |α|≤k

The spaces W k,p (𝒟) are separable for p ∈ [1, ∞) but not for p = ∞. The following density result clarifies the nature of functions in Sobolev spaces. Theorem 1.5.5 (Meyers–Serrin). If 𝒟 ⊂ ℝd is open, then C ∞ (𝒟) ∩ W k,p (𝒟) is dense in W k,p (𝒟), so that W k,p (𝒟) can be defined as the completion of C ∞ (𝒟) in the norm ‖ ⋅ ‖W k,p (𝒟) . We will also use the following subsets of the Sobolev spaces W k,p (𝒟). Definition 1.5.6. The Sobolev space W01,p (𝒟) is defined as the closure of the set Cc∞ (𝒟) with respect to the norm ‖ ⋅ ‖W 1,p (𝒟) . Similarly, the Sobolev space W0k,p (𝒟) is defined as the closure of the set Cc∞ (𝒟) with respect to the norm ‖ ⋅ ‖W k,p (𝒟) . We also define the spaces W −1,p (𝒟) := (W01,p (𝒟))⋆ and W −k,p (𝒟) := (W0k,p (𝒟))⋆ where ⋆

Theorem 1.5.4 holds for the Sobolev spaces

⋆

W0k,p (𝒟).

1 p

+

1 p⋆

= 1.

42 | 1 Preliminaries The spaces W0k,p (𝒟) can be considered as consisting of these functions in W k,p (𝒟) that “vanish” on 𝜕𝒟, where this can be made precise once the concept of trace for a Sobolev space is defined. The duals of the Sobolev spaces W k,p (𝒟) play an important role in many applications. The elements of these spaces are distributions. The following result sheds some light to the nature of the elements of these spaces. Proposition 1.5.7 (Riesz representation in Sobolev spaces). Let p ∈ [1, ∞) and let p⋆ be so that p1 + p1⋆ = 1. Then: (i) For any u ∈ (W 1,p (𝒟))⋆ there exist v0 , v1 , . . . , vd ∈ Lp (𝒟) such that ⋆

⟨u, w⟩(W 1,p (𝒟))⋆ ,W 1,p (𝒟) =

ˆ

d

𝒟

(v0 w + ∑ vi i=1

𝜕w )dx, 𝜕xi

∀ w ∈ W 1,p (𝒟),

(1.8)

The spaces W 1,p (𝒟) are reflexive for p ∈ (1, ∞). ⋆ (ii) For any u ∈ (W k,p (𝒟))⋆ , there exists a family {vα : |α| ≤ k} ⊂ Lp (𝒟), such that ˆ vα 𝜕xα wdx, ∀ w ∈ W k,p (𝒟), ⟨u, w⟩(W k,p (𝒟))⋆ ,W k,p (𝒟) = ∑ (1.9) 0≤|α|≤k

𝒟

The spaces W k,p (𝒟) are reflexive for p ∈ (1, ∞). It is important to note that the above representation does not imply that ⋆ (W 1,p (𝒟))⋆ ≃ Lp (𝒟; ℝd+1 ) (for a discussion see, e. g., Remark 10.42 in [77])! Although we will be more specific for (W0k,p (𝒟))⋆ , the full characterization of (W k,p (𝒟))⋆ in general will not concern us here (see, e. g., [1, 28] or [77]), but the result of Proposition 1.5.7 allows us to characterize sufficiently weak convergence in W k,p (𝒟). Example 1.5.8 (Weak convergence in W k,p (𝒟)). Consider a sequence {un : n ∈ ℕ} ⊂ W k,p (𝒟) and some u ∈ W k,p (𝒟), p ∈ [1, ∞). Then un ⇀ u in W k,p (𝒟) if and only if un ⇀ u and 𝜕xα un ⇀ 𝜕xα u, 1 ≤ |α| ≤ k, in Lp (𝒟). By allowing 0 ≤ |α| ≤ k, we can simply use the notation 𝜕xα un ⇀ 𝜕xα u, 0 ≤ |α| ≤ k, in Lp (𝒟) to include weak convergence of both the function and its derivatives. Assume that un ⇀ u in W k,p (𝒟) so that ⟨w, un − u⟩(W k,p (𝒟))⋆ ,W k,p (𝒟) → 0 for all w ∈ (W k,p (𝒟))⋆ . Then for any v ∈ Lp (𝒟), and any α such that 0 ≤ |α| ≤ k set vα = v and ´ consider the element wα ∈ (W k,p (𝒟))⋆ generated by vα , i. e., wα (u) = 𝒟 vα 𝜕xα udx, for any u ∈ W k,p (𝒟). Since un ⇀ u in W k,p (𝒟), it holds that ⟨wα , un −u⟩(W k,p (𝒟))⋆ ,W k,p (𝒟) → 0 ´ ´ ⋆ which implies that 𝒟 vα 𝜕xα (un −u)dx = 𝒟 v𝜕xα (un −u)dx → 0 for any v ∈ Lp (𝒟); hence, 𝜕xα un ⇀ 𝜕xα u, |a| ≤ k, in Lp (𝒟). For the converse, consider that un ⇀ u and 𝜕xα un ⇀ 𝜕xα u, 0 ≤ |a| ≤ k, in Lp (𝒟) so ⋆ that for every v ∈ Lp (𝒟) and every α such that 0 ≤ |α| ≤ k it holds that ⟨v, 𝜕xα (un − u)⟩Lp⋆ (𝒟),Lp (𝒟) → 0. Consider now any w ∈ (W k,p (𝒟))⋆ . By Proposition 1.5.7, there ex⋆

ists a collection {vα : 0 ≤ |α| ≤ k} ⊂ Lp (𝒟) such that ⟨w, un − u⟩(W k,p (𝒟))⋆ ,W k,p (𝒟) = ◁ ∑0≤|α|≤k ⟨vα , 𝜕xα (un − u)⟩Lp⋆ (𝒟),Lp (𝒟) → 0. ⋆

1.5 Sobolev spaces | 43

Remark 1.5.9 (What is the nature of the elements of (W k,p (𝒟))⋆ and (W0k,p (𝒟))⋆ = ⋆ ⋆ W −k,p (𝒟)?). For the characterization of W −k,p (𝒟), we need to define the space of distributions D (𝒟)⋆ on 𝒟 which is the set of functionals f : Cc∞ (𝒟) → ℝ, endowed with the weak⋆ topology according to which fn ⇀ f in D (𝒟)⋆ if ⟨fn , ϕ⟩ → ⟨f , ϕ⟩ for every ϕ ∈ Cc∞ (𝒟). We should note that to define the space of distributions properly we need to define the space of test functions D (𝒟) which is the space Cc∞ (𝒟), endowed with a proper topology τ. Distributions can be considered as generalizations of either functions or measures, in the sense that for every μ ∈ ℳ(𝒟) and u ∈ Lp (𝒟) we may ´ ´ define distributions Tμ , Tu : D (𝒟) → ℝ by ⟨Tμ , ϕ⟩ = 𝒟 ϕdμ and ⟨Tu , ϕ⟩ = 𝒟 uϕdx for every ϕ ∈ D (𝒟), respectively. We may also define derivatives of distributions, as the ´ distributions 𝜕α Tvα defined by 𝜕α Tvα (ϕ) = (−1)|α| 𝒟 vα 𝜕xα ϕdx for every ϕ ∈ D (𝒟). With these definitions at hand, by restricting the action of each element u ∈ k,p (W (𝒟))⋆ to D (𝒟) and using Proposition 1.5.7, we have that ⟨Tv , ϕ⟩ = ∑0≤|α|≤k (−1)|α| ×

⟨𝜕α Tvα , ϕ⟩ for every ϕ ∈ D (𝒟), where v = {vα : 0 ≤ |α| ≤ k} ⊂ Lp (𝒟) is the family of functions referred to in Proposition 1.5.7. Conversely, for any family of functions ⋆ v = {vα : 0 ≤ |α| ≤ k} ⊂ Lp (𝒟) the distribution Tv : D (𝒟) → ℝ defined as above can be extended (but not uniquely) to an element u ∈ (W k,p (𝒟))⋆ . On the other hand, its extension to an element u ∈ (W0k,p (𝒟))⋆ is unique, therefore, allowing us to identify ⋆ W −k,p (𝒟) with the subspace of distributions of the form ∑0≤|α|≤k (−1)|α| 𝜕α Tvα for any ⋆

v = {vα : 0 ≤ |α| ≤ k} ⊂ Lp (𝒟). ⋆

The fact that Sobolev functions admit integrable weak derivatives incites useful integrability and continuity properties. These properties can be presented in terms of embedding theorems, which go by the general name of Sobolev embeddings. We will say that X is continuously embedded in Y and use the notation X 󳨅→ Y when X ⊂ Y and the identity mapping I : X → Y is continuous, whereas if I : X → Y is compact we c will call it a compact embedding and use the notation X 󳨅→ Y. Clearly, if X 󳨅→ Y there exists c > 0 such that ‖u‖Y ≤ c‖u‖X for every u ∈ X ⊂ Y. The embedding theorems often require restrictions on 𝜕𝒟 the boundary of the domain 𝒟. Definition 1.5.10 (Lipschitz domain). A domain 𝒟 ⊂ ℝd is called Lipschitz if 𝜕𝒟 is Lipschitz, i. e., it can be expressed (at least locally) in terms of Lipschitz graphs. In particular, a domain with bounded 𝜕𝒟, is called Lipschitz if for every xo ∈ 𝜕𝒟 there exists a neighborhood N(xo ) local coordinates (possibly after relabeling) z = (z 󸀠 , zd ) ∈ ℝd−1 ×ℝ with z = 0 at x = xo , a Lipschitz function f : ℝd−1 → ℝ and r > 0, such that 𝒟 ∩ N(xo ) = {(z , zd ) ∈ 𝒟 ∩ N(xo ) : z ∈ Qd−1 (0, r), zd > f (z )}. 󸀠

󸀠

󸀠

The assumption of Lipschitz domains will be made throughout this book. Theorem 1.5.11 (Sobolev embeddings). Let 𝒟 ⊂ ℝd be a bounded open subset with Lipschitz boundary. Then, for any 0 ≤ k2 ≤ k1 we have the following continuous embeddings:

44 | 1 Preliminaries (i) If (k1 − k2 )p1 < d, then W k1 ,p1 (𝒟) 󳨅→ W k2 ,p2 (𝒟), for 1 ≤ p2 ≤ embedding being compact if 1 W k1 ,p1 (𝒟) 󳨅→ Lp2 (𝒟), for 1 ≤

1 ≤ p2
d with pd ∈ ̸ ℕ and (k1 − k2 − 1)p1 < d < (k1 − k2 )p1 , then W k1 ,p1 (𝒟) 󳨅→ 1

C k2 ,γ (D) for 0 < γ ≤ (k1 − k2 ) −

d , p1

where C k2 ,γ is the space k2 times differentiable

functions with all derivatives up to order k1 in C 0,γ , the space of Hölder continuous functions with exponent γ (see Section 1.8.4). If pd ∈ ̸ ℕ, then W k1 ,p1 (𝒟) 󳨅→ C k2 ,γ (D) 1 for 0 < γ < 1, with the embeddings being compact if (k1 − k2 )p1 > d ≥ (k1 − k2 − 1)p1 and 0 < γ < (k1 − k2 ) − pd . 1

The results remain true for W0k,p (𝒟) for arbitrary domains. Remark 1.5.12. An easy to memorize result is that (under the stated conditions on 𝒟) dp , the critical Sobolev exponent, which if p < d, then W 1,p (𝒟) 󳨅→ Lsp (𝒟), with sp := d−p provides the largest exponent for the embedding to hold. Naturally, the embedding holds for every Lp (𝒟) with p ≤ sp , however, the embedding is compact only for p < sp . Moreover, W 1,p (𝒟) 󳨅→ C(𝒟) for p > d. The compact embeddings guaranteed by the Rellich–Kondrachov theorem may provide useful information, as it allows us to ensure that a bounded sequence in a reflexive Sobolev space, admits a strongly convergent subsequence in a properly selected Lebesgue space (satisfying the assumptions of the Rellich–Kondrachov theorem), and hence a further subsequence converging a. e. The above theorem combines a number of results obtained independently by various authors (including Sobolev himself) into an easily (?) memorable form. The starting point for such results is the so called Gagliardo–Nirenberg–Sobolev inequality, according to which ‖u‖Lsp (ℝd ) ≤ c‖∇u‖Lp (ℝd ) ,

∀ u ∈ Cc1 (ℝd ),

sp :=

dp , d−p

1 ≤ p < d.

(1.10)

This important inequality allows us to obtain embedding theorems for the space of Sobolev functions defined over the whole of ℝd , denoted by W k,p (ℝd ). Then, given a domain 𝒟, using the concept of an extension operator for 𝒟, i. e., a continuous linear operator E : W k,p (𝒟) → W k,p (ℝd ), such that Eu(x) = u(x) a. e. in 𝒟, the embedding is extended to W k,p (𝒟). Naturally, the existence of an extension operator depends on the domain 𝒟, with the domains admitting such an operator called extension domains. Lipschitz domains can be shown to be extension domains (see [1]). The compact embedding results are due to Rellich and Kondrachov whereas the embedding into the

1.5 Sobolev spaces | 45

space of continuous functions is due to Morrey (see, e. g., [1] for details). These embedding results are based on inequalities between the norms of the derivatives of functions, initially obtained over the whole of ℝd and then modified by proper extension (or restriction) arguments. Since most of these inequalities can be inferred from the embeddings, we do not reproduce them explicitly here for the sake of brevity. We only provide an important inequality, the Poincaré inequality that is very useful in applications. Theorem 1.5.13 (Poincaré inequality). (i) Let 𝒟 ⊂ ℝd be an open bounded domain.26 Then there exists c𝒫 > 0 depending only on p and 𝒟 such that c𝒫 ‖u‖Lp (𝒟) ≤ ‖∇u‖Lp (𝒟) ,

∀ u ∈ W01,p (𝒟), p ∈ [1, ∞).

(ii) Let 𝒟 ⊂ ℝd be a connected open subset with Lipschitz boundary, and A ⊂ 𝒟 a Lebesgue measurable set with positive measure, with the possibility that A = 𝒟. ffl 1 ´ Then, if uA = A udx := |A| A udx is the average of u over A, there exists a constant ̂c𝒫 > 0 dependent only on p, 𝒟, A such that ĉ𝒫 ‖u − uA ‖Lp (𝒟) ≤ ‖∇u‖Lp (𝒟) ,

∀ u ∈ W 1,p (𝒟), p ∈ [1, ∞).

The constant in the Poincaré inequality is very important as it carries geometric information on the domain 𝒟. We will also see that it can be characterized in terms of the solution of eigenvalue problems for operators related to the Laplace operator. For the time being, we show that using the Poincaré inequality we may establish an equivalent norm for the Sobolev space W01,p (𝒟) using only the Lp (𝒟) norm of the weak gradient of u. Example 1.5.14 (An equivalent norm for W01,p (𝒟)). If 𝒟 ⊂ ℝd is bounded and open, the norms ‖u‖W 1,p (𝒟) and ‖∇u‖Lp (𝒟) are equivalent norms for W01,p (𝒟), p ∈ [1, ∞). It suffices to find two constants c1 , c2 > 0 such that c1 ‖∇u‖Lp (𝒟) ≤ ‖u‖W 1,p (𝒟) ≤ c2 ‖∇u‖Lp (𝒟) , 0

∀u ∈ W01,p (𝒟).

𝜕u p ‖ p , and Recall the definitions of the norms ‖u‖pW 1,p (𝒟) = ‖u‖pLp (𝒟) + ∑di=1 ‖ 𝜕x i L (𝒟) ´ p 1/p ‖∇u‖Lp (𝒟) = ( 𝒟 |∇u(x)| dx) where |∇u| denotes the Euclidean norm of ∇u. Then, 𝜕u p ‖ 𝜕x ‖Lp (𝒟) ≤ ‖∇u‖pLp (𝒟) for every i = 1, . . . , d from which we can easily see that i

p ‖u‖pW 1,p (𝒟) ≤ ‖u‖pLp (𝒟) + d‖∇u‖pLp (𝒟) . By the Poincaré inequality, ‖u‖pLp (𝒟) ≤ c−p 𝒫 ‖∇u‖Lp (𝒟) so that combining this with the above we conclude that ‖u‖W 1,p (𝒟) ≤ c1 ‖∇u‖Lp (𝒟) for 1/p c1 = (c−p 𝒫 + d) .

0

26 Actually, it suffices that it has finite width, i. e., that it lies between two parallel hyperplanes.

46 | 1 Preliminaries On the other hand, all norms in ℝd are equivalent; hence, there is a constant c󸀠 > 0 such that for all a = (a1 , . . . , ad ) ∈ ℝd it holds that ∑di=1 |ai |p ≥ c󸀠 (∑di=1 |ai |2 )p/2 . Applying this to the gradient ∇u, it follows that ‖u‖p

W01,p (𝒟)

ˆ d 󵄨 󵄨p ˆ d 󵄩 󵄨󵄨 𝜕u 󵄨󵄨 󵄩󵄩 𝜕u 󵄩󵄩󵄩p = |∇u|p dx, ≥ ∑󵄩󵄩󵄩 󵄩󵄩󵄩 ∑󵄨󵄨󵄨 󵄨󵄨󵄨 dx ≥ c󸀠 󵄨󵄨 𝜕xi 󵄨󵄨 󵄩󵄩 𝜕xi 󵄩󵄩Lp (𝒟) 𝒟 𝒟 i=1 i=1

from which we obtain that c1 ‖∇u‖Lp (𝒟) ≤ ‖u‖W 1,p (𝒟) holds for c1 = (c󸀠 )1/p .

◁

0

Example 1.5.15 (A useful inequality [67]). Let 𝒟 be a bounded connected open set with Lipschitz continuous boundary and u ∈ W 1,p (𝒟), p < d, that vanishes on a set Ao of positive measure. Then 1

‖u‖

Lsp (𝒟)

≤ 2ĉ𝒫 (𝒟)(

|𝒟| sp ) ‖∇u‖Lp (𝒟) , |Ao |

where ĉ𝒫 (𝒟) > 0 is a constant independent of Ao . This inequality is useful in estimating the size of level sets of functions. We start by combining the Poincaré inequality of Theorem 1.5.13(ii), for the choice A = Ao with the Sobolev embedding. Defining the function v = u − uAo = u we have by the Sobolev embedding that ‖v‖Lsp (𝒟) ≤ c(‖∇v‖Lp (𝒟) + ‖v‖Lp (𝒟) ), and using the Poincaré inequality on the right-hand side we obtain an inequality of the form ‖v‖Lsp (𝒟) ≤ c󸀠 ‖∇v‖Lp (𝒟) for an appropriate constant c󸀠 > 0 depending on ĉ𝒫 . This may be sufficient for many applications but it would be better if we could get some further information on the dependence of the constant on the measure of set Ao . This can be obtained as follows. We first work as above setting A = 𝒟 in the Poincaré inequality of Theorem 1.5.13(ii) to obtain an inequality of the form (1.11)

‖u − u𝒟 ‖Lsp (𝒟) ≤ ĉ𝒫 (𝒟)‖∇u‖Lp (𝒟) ,

for an appropriate constant ĉ𝒫 (𝒟) > 0. Then, since Ao ⊂ 𝒟 (and is of positive mea1

− s1

sure) clearly ‖u − u𝒟 ‖Lsp (𝒟) ≥ ‖u − u𝒟 ‖Lsp (Ao ) = |u𝒟 | |Ao | sp ; hence, |u𝒟 | ≤ |Ao | u𝒟 ‖Lsp (𝒟) . By the triangle inequality, 1

‖u‖Lsp (𝒟) ≤ ‖u − u𝒟 ‖Lsp (𝒟) + |u𝒟 ||𝒟| sp ≤ 2(

p

‖u −

1

|𝒟| sp ) ‖u − u𝒟 ‖Lsp (𝒟) , |Ao |

where we also used the obvious estimate |A0 | ≤ |𝒟|. The result follows by applying the Poincare-type inequality (1.11) obtained above. ◁ We now consider the behavior of functions in Sobolev spaces on the boundary 𝜕𝒟 of the domain 𝒟 and consider the problem of restricting a Sobolev function on 𝜕𝒟. Functions which are elements of Sobolev spaces are elements of a Lebesgue space; hence, it may be that there does not exist continuous representatives of them. Trace

1.5 Sobolev spaces | 47

theory is concerned with assigning boundary values to such functions. In partial differential equations, there are instances where we need to know the value of a function or of its normal derivative at 𝜕𝒟. This is related to two mappings which are called traces. Definition 1.5.16 (Trace operators). Let 𝒟 ⊂ ℝd be a domain, 𝜕𝒟 its boundary and n(x) the normal vector at x ∈ 𝜕𝒟. Consider a function u : 𝒟 → ℝ. (i) The map γ0 : u 󳨃→ u|𝜕𝒟 is called the trace map. This map “assigns” boundary values to the function u. 𝜕 (ii) The map γ1 : u 󳨃→ 𝜕n u = ∇u ⋅ n|𝜕𝒟 . This map “assigns” to u its normal derivative on the boundary. Clearly, the above is not yet a proper definition since we have not properly defined for which type of functions nor sufficient properties of 𝜕𝒟, such that these maps make sense. For example, γ0 can be defined for functions u ∈ C(𝒟), and γ1 can be defined for functions in C 1 (𝒟), but is this true for generalized functions or elements of Sobolev spaces? Furthermore, the answer to this question depends also on the properties of the domain 𝒟, or rather its boundary 𝜕𝒟. The following proposition asserts that as long as 𝒟 has certain regularity properties and the function belongs to a certain class of Sobolev spaces then there exists a map which can be considered as an extension of the map assigning boundary values to this function in the case where the function is continuous or continuously differentiable. Proposition 1.5.17 (Trace spaces). Let 𝒟 be a bounded domain and consider p ∈ [1, ∞]. (i) If 𝒟 is Lipschitz and sp > 1, then there exists a unique linear and continuous trace operator γ0 : W s,p (𝒟) → Lp (𝜕𝒟), such that γ0 u = u|𝜕𝒟 for any u ∈ D (𝜕𝒟). (ii) If 𝒟 is Lipschitz, the range of the map γ0 , which is called the trace space H 1/2 (𝜕𝒟) := γ0 (W 1,2 (𝒟)), is a Banach space when endowed with the norm ‖u‖2H 1/2 (𝜕𝒟) =

ˆ 𝜕𝒟

󵄨󵄨 󵄨2 󵄨󵄨u(x)󵄨󵄨󵄨 dμ(x) +

ˆ 𝜕𝒟

ˆ 𝜕𝒟

|u(x) − u(z)|2 dμ(x)dμ(z), |x − z|d+1

where μ is the Hausdorff measure on 𝜕𝒟. (iii) If 𝒟 is Lipschitz, there exists a linear continuous map f : H 1/2 (𝜕𝒟) → W 1,2 (𝒟), such that f (γ0 (u)) = u for any u ∈ W 1,2 (𝒟), with the property ‖f (u)‖W 1,2 (𝒟) ≤ c‖u‖H 1/2 (𝜕𝒟) , for every u ∈ H 1/2 (𝜕𝒟), for a constant c = c(𝒟). c (iv) If 𝒟 is Lipschitz, the injection H 1/2 (𝜕𝒟) 󳨅→ L2 (𝜕𝒟) is compact. (v) If 𝒟 is C 1 and sp > 1 + p, then there exists a unique linear and continuous trace 𝜕 operator γ1 : W s,p (𝒟) → Lp (𝜕𝒟), such that γ1 u = 𝜕n u, for any u ∈ D (𝜕𝒟). The Sobolev spaces W0k,p (𝒟) admit alternative characterizations in terms of trace maps. For example, we have the following.

48 | 1 Preliminaries Proposition 1.5.18. Let 𝒟 be a bounded Lipschitz domain. Then W01,p (𝒟) = {u ∈ W 1,p (𝒟) : γ0 u = 0, a.e.}. The above can be interpreted as the statement that the Sobolev space W01,p (𝒟) consists of the functions in W 1,p (𝒟), whose values vanish on 𝒟 (in the sense of traces). A similar characterization can be obtained for other spaces W0k,p (𝒟), where now the normal trace may be involved. For example, W02,p (𝒟) = {u ∈ W 2,p (𝒟) : γ0 u = γ1 u = 0, a.e.}, where of course now the domain will have to be so that the normal trace operator can be defined, i. e., C 1 . Such characterizations are very useful when Sobolev spaces are applied to the solution of boundary value problems.

1.6 Lebesgue–Bochner and Sobolev–Bochner spaces In certain cases (e. g., in evolution equations) the need arises to consider functions from intervals of ℝ to some Banach space X, i. e., mappings t 󳨃→ z = x(t) ∈ X. So as to leave notation as intuitive as possible, we will denote such functions by f = x(⋅) : [0, T] ⊂ ℝ → X (keeping x in the notation as an indicator of the Banach space in which the function takes values). Such functions can be integrated over the Lebesgue measure on ℝ yielding a vector valued integral called the Bochner integral. Since f = x(⋅) takes values on X, measurability issues can be challenging. The theory of the Lebesgue integral, Lebesgue spaces and Sobolev spaces, can be generalized for functions taking values in Banach spaces (see, e. g., [58] or [101]). Definition 1.6.1 (Strong and weak measurability). (i) f : I ⊂ ℝ → X is called strongly measurable if there exists a sequence {sn : n ∈ ℕ} of simple functions (i. e., of functions sn : I ⊂ ℝ → X of the form sn (t) = ∑M(n) i=1 xi,n 1Ai,n (t) for every t ∈ I, where M(n) is finite, xi,n ∈ X and Ai,n ⊂ I measurable sets) such that sn (t) → f (t) in X a. e. (ii) f : I ⊂ ℝ → X is called weakly measurable if for any x⋆ ∈ X ⋆ the function ϕ : I ⊂ ℝ → ℝ, defined by ϕ(t) := ⟨x⋆ , f (t)⟩, for every t ∈ I is measurable. These two concepts are connected with the celebrated Pettis theorem. Theorem 1.6.2 (Pettis). Let X is a separable Banach space. A function f : I ⊂ ℝ → X is strongly measurable if and only if it is weakly measurable. We may now define the Bochner integral which is a useful vector valued extension of the Lebesgue integral.

1.6 Lebesgue–Bochner and Sobolev–Bochner spaces | 49

Definition 1.6.3 (Bochner integral). (i) The Bochner integral for a simple function s : I ⊂ ℝ → X of the form s(t) = ´ M ∑M i=1 xi 1Ai (t) is defined as I s(t)dt = ∑i=1 xi |Ai |, where |Ai | := μℒ (Ai ) is the Lebesgue measure of Ai ⊂ I. (ii) If f : I ⊂ ℝ → X is a strongly measurable function and {sn : n ∈ ℕ} is a sequence of simple functions such that sn (t) → f (t) in X, a. e. in I, we define the Bochner ´ ´ integral I f (t)dt := limn→∞ I sn (t)dt. A strongly measurable function f : I ⊂ ℝ → X is called Bochner integrable if there exists a sequence of simple functions {sn : ´ n ∈ ℕ} such that ‖sn (t) − f (t)‖ → 0 a. e. in I and I ‖sn (t) − f (t)‖dt → 0 with the last integral understood as a Lebesgue integral. If a function f is Bochner integrable, the value of the integral does not depend on the choice of the approximating sequence. Furthermore, the Pettis measurability theorem may be used to extend the class of integrable functions. Clearly, the Bochner integral satisfies the properties of the Lebesgue integral, i. e., linearity, additivity and the fact that null sets for the Lebesgue measure do not contribute to the value of the integral. The following theorem collects the most important properties of the Bochner integral. Theorem 1.6.4 (Properties of Bochner integral). (i) f : I ⊂ ℝ → X is Bochner integrable if and only if the real valued function t 󳨃→ ‖f (t)‖ ´ ´ is Lebesgue integrable and ‖ I f (t)dt‖ ≤ I ‖f (t)‖dt. (ii) If L : X → X is a bounded linear operator and f : I ⊂ ℝ → X is Bochner integrable, ´ ´ then so is L ∘ f : I ⊂ ℝ → X and L( I f (t)dt) = I Lf (t)dt. ´ ⋆ ´ (iii) If f : I ⊂ ℝ → X is Bochner integrable, then I ⟨x , f (t)⟩dt = ⟨x⋆ , I f (t)dt⟩ for every x⋆ ∈ X ⋆ . (iv) If A : D(A) ⊂ X → Y is a closed linear operator and f : I ⊂ ℝ → X is a Bochner integrable function taking values a..e in D(A) and such that A ∘ f : I ⊂ ℝ → Y is ´ ´ ´ Bochner integrable, then I (Af )(t)dt ∈ D(A) and A I f (t)dt = I (Af )(t)dt. ´ (v) If f : I ⊂ ℝ → X is (locally) Bochner integrable, then limh→0 h1 [t,t+h] f (τ)dτ = f (t), a. e. t ∈ I. The Bochner integral can be used for the definition of an important class of Banach spaces, called Lebesgue–Bochner spaces. These play an important role in the study of evolution equations. Definition 1.6.5 (Lebesgue–Bochner spaces). (i) For p ∈ [1, ∞) we define the spaces p

L (I; X) := {f : I ⊂ ℝ → X : f measurable and

ˆ I

󵄩󵄩 󵄩p 󵄩󵄩f (t)󵄩󵄩󵄩X dt < ∞},

which are Banach spaces (called Lebesgue–Bochner spaces) when equipped with ´ 1/p the norm ‖f ‖Lp (I;X) = { I ‖f (t)‖pX dt} .

50 | 1 Preliminaries (ii) In the case p = ∞, we define 󵄩 󵄩 L∞ (I; X) := {f : I ⊂ ℝ → X : f measurable and ess supt∈I 󵄩󵄩󵄩f (t)󵄩󵄩󵄩X < ∞}, which is also a Banach space when equipped with the norm ‖f ‖L∞ (I;X) ess supt∈I ‖f (t)‖X < ∞.

=

Lebesgue–Bochner spaces satisfy versions of the Hölder inequality, in particular, (a) ⋆ ⋆ hf ∈ L1 (I; X) if h ∈ Lp (I; ℝ) and f ∈ Lp (I; X) or (b) ⟨f1 , f2 ⟩ ∈ L1 (I; ℝ) if f1 ∈ Lp (I; X ⋆ ) and f2 ∈ Lp (I; X), with p ∈ [1, ∞] and p1 + p1⋆ = 1. These results follow easily from the standard Hölder inequality and Theorem 1.6.4. One may also provide embedding theorems similar to the standard versions as well as extensions. Theorem 1.6.6 (Lebesgue–Bochner embeddings). If p1 , p2 ∈ [1, ∞] with p1 ≤ p2 and X 󳨅→ Y (with continuous embedding), then Lp2 (I; X) 󳨅→ Lp1 (I; Y) (with continuous embedding). One may also consider the case where X = Y. The characterization of the dual space Lp (I; X) depends on the properties of the Banach space X. For example, if X is reflexive and p ∈ (1, ∞), then (Lp (I; X))⋆ ≃ ⋆ Lp (I; X ⋆ ), where as usual p1 + p1⋆ = 1, the duality pairing being ⟨f1 , f2 ⟩(Lp (I;X))⋆ ,Lp (I;X) = ´ I ⟨f1 (t), f2 (t)⟩X ⋆ ,X dt. One may further define Sobolev spaces in this setting. We start with the definition of the generalized derivative for functions f : I ⊂ ℝ → X. Definition 1.6.7. Let f : I ⊂ ℝ → X be a vector valued function. The distributional derivative f 󸀠 ∈ ℒ(Cc∞ (I; ℝ); Y) is the bounded linear operator defined by the integration ´ ´ by parts formula I f 󸀠 (t)ϕ(t)dt = − I f (t)ϕ󸀠 (t)dt, for every ϕ ∈ Cc∞ (I; ℝ). Note that we may need to choose Y different than X and such that X ⊂ Y, in order for f 󸀠 to become a bounded linear operator. The following concrete example illustrates this phenomenon. Example 1.6.8. Consider the function f : [0, 1] → X := L2 ([−π, π]) defined by f (t) := 2 ψ(t, ⋅) where ψ(t, x) = ∑n∈ℕ an e−n t sin(nx) for every t ∈ [0, 1], x ∈ [−π, π]. This function is well-defined for any {an : n ∈ ℕ} ⊂ ℓ2 . Differentiate formally the Fourier series with respect to t to get f 󸀠 (t) = 𝜕t𝜕 ψ(t, ⋅), which gives for any (t, x) ∈ [0, 1] × [−π, π] that 2

− ∑n∈ℕ n2 an e−n t sin(nx), which for {an : n ∈ ℕ} ⊂ ℓ2 is no longer a function in X = L ([−π, π]) but rather a function in Y = W −1,2 ([−π, π]) = (W01,2 ([−π, π]))⋆ . ◁ 𝜕 ψ(t, x) = 𝜕t 2

We may now define a general class of Sobolev–Bochner spaces. Definition 1.6.9 (Sobolev–Bochner spaces). For the Banach spaces X ⊂ Y we define the Sobolev–Bochner space W 1,p,q (I; X, Y) = {f : I ⊂ ℝ → X, with f ∈ Lp (I; X), f 󸀠 ∈ Lq (I; Y)}.

1.6 Lebesgue–Bochner and Sobolev–Bochner spaces | 51

This is a Banach space when equipped with the norm ‖f ‖W 1,p,q (I;X,Y) = ‖f ‖Lp (I;X) + ‖f 󸀠 ‖Lq (I;Y) . The Sobolev–Bochner spaces enjoy some useful embedding theorems. Theorem 1.6.10 (Sobolev–Bochner embedding). Let p, q ≥ 1 and X 󳨅→ Y (with continuous embedding). Then W 1,p,q (I; X, Y) 󳨅→ C(I; Y) (with continuous embedding). An important special case in Definition 1.6.9 is when Y = X ⋆ , the dual space of X. In this case, one needs a construction called an evolution triple (or Gelfand triple). Definition 1.6.11. An evolution triple (or Gelfand triple) is a triple of spaces X 󳨅→ H 󳨅→ X ⋆ , where X is a separable reflexive Banach space, H is a separable Hilbert space identified with its dual (called the pivot space), and the embedding X 󳨅→ H is continuous and dense. Example 1.6.12. An example of an evolution triple is W0k,p (𝒟) 󳨅→ L2 (𝒟) 󳨅→ W −k,p (𝒟), for 2 ≤ p < ∞. ◁ ⋆

The continuous and dense embedding X 󳨅→ H leads to the continuous and dense embedding H 󳨅→ X ⋆ . Indeed, for every fixed h ∈ H, one may construct x⋆h ∈ X ⋆ , defined by ⟨x⋆h , x⟩X ⋆ ,X = ⟨h, ι(x)⟩H for every x ∈ X, where ι : X → H is the continuous embedding operator of X into H. We then consider the mapping ι⋆ : H ≃ H ⋆ → X ⋆ , defined by h 󳨃→ x⋆h , which is linear, continuous and injective.27 The embedding H 󳨅→ X ⋆ is understood in terms of the mapping ι⋆ . The density of H into X ⋆ follows by the reflexivity of X. ⋆ In such cases we will consider the Sobolev–Bochner spaces W 1,p,p (I; X, X ⋆ ) where as usual p ∈ (1, ∞) and p1 + p1⋆ = 1. The following embeddings hold in an evolution triple. Theorem 1.6.13. Let X 󳨅→ H 󳨅→ X ⋆ be an evolution triple. Then: ⋆ (i) W 1,p,p (I; X, X ⋆ ) 󳨅→ C(I; H) with continuous and dense embedding, for p ∈ (1, ∞). ⋆ Furthermore, for any f1 , f2 ∈ W 1,p,p (I; X, X ⋆ ) and 0 ≤ s ≤ t ≤ T, the following integration by parts formula holds ⟨f1 (t), f2 (t)⟩H − ⟨f1 (s), f2 (s)⟩H = c

ˆ s

t

(⟨f1󸀠 (τ), f2 (τ)⟩X ⋆ ,X + ⟨f2󸀠 (τ), f1 (τ)⟩X ⋆ ,X )dτ. (1.12) c

c

(ii) If X 󳨅→ H, then for any p ∈ (1, ∞) it holds that W 1,p,p (I; X, X ⋆ ) 󳨅→ Lp (I; H) with 󳨅→ denoting compact embedding. ⋆

Proof. We sketch the proof. We start by the integration by parts formula. For any f1 , f2 ∈ C 1 (I; X) formula, (1.12) holds by standard calculus. Then it can be extended to any 27 By the density of X into H; if x⋆h = 0, then by its definition ⟨h, ι(x)⟩H = 0 for every x ∈ X, and since X 󳨅→ H densely we conclude that h = 0.

52 | 1 Preliminaries f1 , f2 ∈ W 1,p (I; X) by noting that C 1 (I; X) ⊂ W 1,p,q (I; X, Y) densely, as long as p, q ≥ 1 and X ⊂ Y continuously, so that C 1 (I; X) is dense in W 1,p (I; X).

1.7 Multivalued maps Given two topological spaces X, Y, a multivalued (or set valued) map is a mapping from X to the power set of Y, i. e., a map from points in X to subsets of Y. Multivalued maps will be denoted by f : X → 2Y . This is in contrast to single valued maps f : X → Y, which map points to points, and can be considered as a special case of the above, where the image of each point is a singleton. We will encounter very often multivalued maps in this book (e. g., we have already introduced the duality map), therefore, we need to introduce some fundamental notions related to them (for more detailed accounts see, e. g., [5] or [70]). We introduce the following definitions.28 Definition 1.7.1 (Multivalued maps). Let X, Y be two topological spaces and f : X → 2Y a multivalued map. We define the following concepts for f : (i) The domain of definition D(f ) := {x ∈ X : f (x) ≠ 0} and the graph Gr(f ) := {(x, y) ∈ X × Y : y ∈ f (x)}. (ii) The image of a set A ⊂ X, denoted by f (A) := ⋃x∈A f (x), and the upper and lower inverse of a set B ⊂ Y denoted by f u (B) := {x ∈ X : f (x) ⊂ B} and f ℓ (B) := {x ∈ X : f (x) ∩ B ≠ 0}, respectively. (iii) A selection from f is a single valued map fs : X → Y satisfying fs (x) ∈ f (x) for all x ∈ X. (iv) f is called closed valued (resp., compact valued) if for every x ∈ X the set f (x) ⊂ Y is closed (resp., compact), and closed if Gr(f ) ⊂ X × Y is a closed set. Definition 1.7.2 (Upper and lower semicontinuity). Let X and Y be topological spaces. A multivalued map f : X → 2Y is called: (i) Upper semicontinuous at x ∈ X if for any open set 𝒪 ⊂ Y such that f (x) ⊂ 𝒪, we can find an open neighborhood N(x) ⊂ X such that f (N(x)) ⊂ 𝒪. We say that f is upper semicontinuous if it is upper semicontinuous at every point x ∈ X. (ii) Lower semicontinuous at x ∈ X if for every open set 𝒪 such that 𝒪 ∩ f (x) ≠ 0 we can find an open neighborhood N(x) ⊂ X such that f (z) ∩ 𝒪 ≠ 0 for all z ∈ N(x). If f is lower semicontinuous at every point x ∈ X it is called lower semicontinuous. 28 Note that the terminology in multivalued maps is not standard, and many authors use different terminology and notation for the same concepts. Here, we mostly follow the terminology in [5] that we find more intuitive.

1.7 Multivalued maps |

53

For single valued maps upper and lower semicontinuity in the sense of Definition 1.7.2 coincide with continuity, so some care has to be taken not to confuse the concept of upper and lower semicontinuity for single valued maps with that for multivalued maps. Example 1.7.3. The function f1 : ℝ → 2ℝ , defined by f (x) = {a} if x < 0 and f (x) = [a, b] if x = 0, is upper semicontinuous everywhere. The function f2 : ℝ → 2ℝ , defined by f (x) = [a, b] if x < 0 and f (x) = {a} if x = 0, is not upper semicontinuous at x = 0. The function f2 is lower semicontinuous everywhere, while f1 is not lower semicontinuous at 0. ◁ We now give some useful equivalent characterizations for upper and lower semicontinuity of multivalued maps in metric spaces. Theorem 1.7.4. Let X, Y be metric spaces and f : X → 2Y be a multivalued (or set valued) map. (i) The following are equivalent: (a) f is upper semicontinuous, (b) f u (𝒪) is open for every open 𝒪 ⊂ Y, (c) f ℓ (𝒞 ) is closed for every closed 𝒞 ⊂ Y, (d) if x ∈ X, {xn : n ∈ ℕ} ⊂ X, xn → x and 𝒪 ⊂ Y is an open set such that f (x) ⊂ 𝒪, then there exists no ∈ ℕ (depending on 𝒪) such that f (xn ) ⊂ 𝒪 for every n ≥ no . (ii) The following are equivalent: (a) f is lower semicontinuous, (b) f ℓ (𝒪) is open for every open 𝒪 ⊂ Y, (c) f u (𝒞 ) is closed for every closed 𝒞 ⊂ Y, (d) if x ∈ X, {xn : n ∈ ℕ} ⊂ X, xn → x and y ∈ f (x) then we can find yn ∈ f (xn ), n ∈ ℕ, such that yn → y in Y. Proof. (i). To show that (a) is equivalent to (b), we reason as follows: If f is upper semicontinuous and 𝒪 ⊂ Y is open, then f u (𝒪) is a neighborhood of every x ∈ f u (𝒪). But a set which is a neighborhood of each of its points is open.29 To show that (b) is equivalent to (c), note that for any B ⊂ Y we have f ℓ (Y \ B) = X \ f u (B). (ii) We only show the equivalence between (a) and (d). Let f be lower semicontinuous at x. Furthermore, let xn → x and y ∈ f (x). For every k ∈ ℕ, let B(y, k1 ) denote the open ball with radius k1 and center y. Since f is lower semicontinuous at x, there exists for every k a neighborhood Vk of x such that for every z ∈ Vk we have f (z) ∩ B(y, k1 ) ≠ 0. Let {nk : k ∈ ℕ} be the subsequence of natural numbers such that nk < nk+1 and xn ∈ Vk if n ≥ nk . For n with nk ≤ n < nk+1 , we choose yn in the set f (xn ) ∩ B(y, k1 ). So, the sequence constructed in this fashion yn → y. Conversely, assume that for every sequence {xn : n ∈ ℕ} such that xn → x and any y ∈ f (x), there exists a sequence yn ∈ f (xn ), n ∈ ℕ, such that yn → y. Suppose that f is not lower semicontinuous at x. Then there exists an open set 𝒪󸀠 with 𝒪󸀠 ∩f (x) ≠ 0 such that every neighborhood V of x contains a point z such that f (z) ∩ 𝒪󸀠 = 0. Therefore, 29 If A is a set which is a neighborhood of each of its points, then A = ⋃ 𝒪󸀠 where 𝒪󸀠 are open sets such that 𝒪󸀠 ⊂ A; hence, its is open.

54 | 1 Preliminaries there exists a sequence {xn : n ∈ ℕ} converging to x with f (xn ) ∩ 𝒪󸀠 = 0 for n ∈ ℕ. Let y ∈ 𝒪󸀠 ∩ f (x). Then there exists a sequence {yn : n ∈ ℕ} with yn ∈ f (xn ) converging to y. For n large enough, we have yn ∈ 𝒪󸀠 . Thus, f (xn ) ∩ 𝒪󸀠 ≠ 0, a contradiction. Remark 1.7.5. In (ii), an equivalent statement could be (d󸀠 ) if x ∈ X, {xn : n ∈ ℕ} ⊂ X, xn → x and 𝒪 ⊂ Y is an open set such that f (x) ∩ 𝒪 ≠ 0, then there exists no ∈ ℕ (depending on 𝒪), such that f (xn ) ∩ 𝒪 ≠ 0 for every n ≥ no . Note that for both (i) and (ii) the equivalence of (a), (b) and (c) above, does not require X, Y to be metric spaces. Additionally, the equivalence of (d) will hold in a more general framework than that of metric spaces as long as sequences are replaced by nets. We collect some useful properties of upper semicontinuous functions. Proposition 1.7.6. Let X, Y be metric spaces and f : X → 2Y a multivalued map. (i) If f is upper semicontinuous and has nonempty closed values (i. e., f (x) ⊂ Y is closed for every x ∈ X), then Gr(f ) ⊂ X × Y is a closed set (i. e., f is closed). (ii) If f is upper semicontinuous and has compact values, then for any compact set K ⊂ X it holds that f (K) ⊂ Y is compact. (iii) If f has nonempty compact valued and for every {(xn , yn ) : n ∈ ℕ} ⊂ Gr(f ) with xn → x, there exists a subsequence {ynk : k ∈ ℕ} with ynk → y ∈ Gr(f ), then it is upper semicontinuous. The converse is also true. (iv) If f has nonempty closed values, is closed and is locally compact, in the sense that for any x ∈ X we may find a neighborhood N(x) such that f (N(x)) ⊂ Y is nonempty and compact, then f is upper semicontinuous. Proof. (i) Suppose not and consider a sequence {(xn , yn ) : n ∈ ℕ} ⊂ Gr(f ) such that (xn , yn ) → (x, y), with y ≠ f (x). Then, since metric spaces are Hausdorff, there are two open sets 𝒪1 , 𝒪2 ⊂ Y, such that y ∈ 𝒪1 , f (x) ⊂ 𝒪2 and 𝒪1 ∩ 𝒪2 = 0. Since yn → y and f is upper semicontinuous, from Theorem 1.7.4(i) (and in particular the equivalence of (a) and (d) using the open set 𝒪2 ), there exists n2 ∈ ℕ such that f (xn ) ⊂ 𝒪2 for all n ≥ n2 . On the other hand, since yn → y, for the open set 𝒪1 ⊂ Y there exists n1 ∈ ℕ such that yn ∈ 𝒪1 for n > n1 . Choosing no = max(n1 , n2 ), we have for n > no that f (xn ) ⊂ 𝒪2 while yn ∈ 𝒪1 while by construction (since (xn , yn ) ∈ Gr(f )) it holds that yn ∈ f (xn ) for all n. This is in contradiction with 𝒪1 ∩ 𝒪2 = 0. (ii) Let {𝒪i , i ∈ I} be an open cover of f (K). If x ∈ K, the set f (x) which is compact, can be covered by a finite number of 𝒪i , i. e., there exists a finite set Ix ⊂ I such that f (x) ⊂ ⋃i∈Ix 𝒪i = 𝒪x . Since 𝒪x is open, by Theorem 1.7.4(ii), f u (𝒪x ) is open and it is obvious that x ∈ f u (𝒪x ). Therefore, {f u (𝒪x )}x∈X is an open cover of K. Since by u assumption K is compact there exist x1 , . . . , xm such that K ⊂ ⋃m n=1 f (𝒪xn ). Clearly, {𝒪i }i∈Ix , n = 1, 2, . . . , m is a finite subcover of {𝒪i }i∈I . n (iii) Let f be upper semicontinuous on X, x ∈ X and {xn : n ∈ ℕ} be any sequence of X such that xn → x. The set K = {x, x1 , x2 , . . .} is compact and the restriction of f on K is upper semicontinuous. Thus, by (ii), the set f (K) is compact, and hence the

1.7 Multivalued maps |

55

sequence {yn : n ∈ ℕ} has a convergent subsequence {ynk : k ∈ ℕ}. Let ynk → y as k → ∞. Assume that y ∉ f (x). Then there exists a closed neighborhood Ū of f (x) with y ∈ ̸ U.̄ Since f is upper semicontinuous at x, for n large enough f (xn ) ⊂ U.̄ Thus, yn ∈ Ū for n large enough, and hence y ∈ U,̄ which is a contradiction. For the converse, assume not. Then there exists an open neighborhood U of f (x) such that every neighborhood V of x contains a point z with f (z) ⊄ U. It then follows that there exists a sequence xn → x and yn ∈ f (xn ) with yn ∈ ̸ U. By assumption, there exists a converging subsequence {ynk : k ∈ ℕ} with limk ynk ∈ f (x). But ynk ∈ ̸ U for all n, therefore, limk ynk ∈ ̸ U, so limk ynk ∈ ̸ f (x), which is a contradiction. (iv) It suffices to show that for any closed 𝒞 ⊂ Y, the set f ℓ (𝒞 ) ⊂ X is closed. Consider a sequence {xn : n ∈ ℕ} ⊂ f ℓ (𝒞 ). Since f is locally compact, we may choose an open neighborhood N(x) ⊂ X such that f (N(x)) is compact; hence, if {yn : n ∈ ℕ} ⊂ Y is such that yn ∈ f (xn ) for every n ∈ ℕ, there exists a subsequence {ynk : k ∈ ℕ} with the property ynk → y and y ∈ f (N(x)). Clearly, (x, y) ∈ Gr(f ) ∩ (X × 𝒞 ); hence, x ∈ f ℓ (𝒞 ) and f ℓ (𝒞 ) ⊂ X is closed. 1.7.1 Michael’s selection theorem The following important theorem due to Ernest Michael [83] (see also [5, 15, 85] or [110]) guarantees the existence of continuous selections. Theorem 1.7.7 (Michael). Let X be a compact metric space,30 Y a Banach space and f : X → 2Y a lower semicontinuous set-valued map with closed and convex values. Then f has a continuous selection. In order to prove this theorem, we need the following lemma. Lemma 1.7.8. Let X be a compact metric space, Y a Banach space and f : X → 2Y a lower semicontinuous set-valued map with convex values. Then, for each ϵ > 0, there exists a continuous function fϵ , such that fϵ (x) ∈ f (x) + BY (0, ϵ) for all x ∈ X. Proof. For every x ∈ X, we choose yx ∈ f (x), and denote by B := BY (0, 1). Since f is lower semicontinuous, the set f ℓ (yx + ϵB) is open and so {f ℓ (yx + ϵB) : x ∈ X} is an open cover of X. Since X is compact, it can be covered by a finite collection of such sets {f ℓ (yxi + ϵB) : i = 1, 2, . . . , n}. Let us consider a partition of unity {ψi : i = 1, 2, . . . , n}, associated with this cover (see Theorem 1.8.4). We claim that our desired map is n

fϵ (x) = ∑ ψi (x)yxi , i=1

∀ x ∈ X.

30 The theorem still holds if X is a paracompact space, i. e., a space where every open cover has a finite open refinement (e. g., a subset of a metric space or a compact subset of a topological space); see [85].

56 | 1 Preliminaries Clearly, fϵ is continuous, since the functions ψi are continuous. Let I(x) = {i = 1, . . . , n : ψi (x) > 0}. It is not empty since ∑ni=1 ψi (x) = 1. When i ∈ I(x), it is easy to see that yxi ∈ f (x) + ϵB. Since the set f (x) + ϵB is convex, we conclude that fϵ (x) ∈ f (x) + ϵB. Proof of Theorem 1.7.7. By induction, we shall construct a sequence of continuous maps gn : X → Y, n ∈ ℕ such that gn (x) ∈ f (x) +

1 B, 2n

∀ x ∈ X, n = 1, 2, . . . ,

(1.13)

and 1 󵄩󵄩 󵄩 󵄩󵄩gn+1 (x) − gn (x)󵄩󵄩󵄩Y < n−1 , 2

∀ x ∈ X, n = 1, 2, . . .

(1.14)

For n = 1, we apply Lemma 1.7.8 with ϵ = 21 , and see that g1 (x) ∈ f (x)+ 21 B. For the induction step, suppose that we have g1 , g2 , . . . , gn , satisfying (1.13) and (1.14). We consider the set valued map ϕn (x) = f (x) ∩ (gn (x) +

1 B), 2n

∀ x ∈ X.

From (1.13), we see that ϕn has nonempty values, which are convex. Moreover, ϕn is also lower semicontinuous. Indeed, if xm → x and if y ∈ ϕn (x), by the lower semicontinuity of f there exists ym ∈ f (xm ) converging to y. By the construction of gn , we have31 ‖ym − gn (xm )‖Y < 2−n , and so ym ∈ ϕn (xm ). Then, from Lemma 1.7.8 we can find gn+1 : X → Y a continuous function such that gn+1 (x) ∈ ϕn (x) +

1

2n+1

B,

∀ x ∈ X;

hence, gn+1 (x) ∈ gn (x) +

1 1 1 B + n+1 B ⊂ gn (x) + n−1 B, 2n 2 2

∀ x ∈ X.

1 This completes the induction. Since ∑∞ n=1 2n−1 < ∞, from (1.14) it follows that {gn : n ∈ ℕ} is a Cauchy sequence in C(X; Y). So, we can find g ∈ C(X; Y) such that gn → g uniformly in X. Because f has closed values from (1.13) we deduce that g(x) ∈ f (x) for all x ∈ X; hence, g is a continuous selection of f .

31 In fact, one can show more generally that if f : X → 2Y is lower semicontinuous and g : X → Y is continuous that x 󳨃→ f (x) ∩ (g(x) + U) where U is an open set is lower semicontinuous; see, e. g., [5].

1.8 Appendix | 57

d

Example 1.7.9 (Differential inclusions). Let f : A ⊂ ℝ × ℝd → 2ℝ be a lower semicontinuous set-valued map into the set of nonempty, closed and convex subsets of ℝd and consider the initial value problem x󸀠 (t) ∈ f (t, x(t)) with initial condition x(t0 ) = x0 . By Michael’s selection theorem (see Theorem 1.7.7), there is a continuous selection fs : A → ℝd . So it is sufficient to consider the classical initial value problem x󸀠 (t) = fs (t, x(t)), with the same initial condition, which by the classical Peano theorem has a C 1 solution x = x(t) in a neighborhood of t0 (local solution). Then, by continuation arguments we may obtain global results. ◁

1.8 Appendix 1.8.1 The finite intersection property A family 𝒜 of sets has the finite intersection property if the intersection of the members of each finite subfamily of 𝒜 is nonempty. Proposition 1.8.1. A metric space X is compact if and only if every family 𝒞 of closed sets with the finite intersection property has a nonempty intersection.

1.8.2 Spaces of continuous functions and the Arzelá–Ascoli theorem Given any two metric spaces X and Y, we may define the space of continuous functions. C(X; Y) := {f : X → Y : f continuous}. An important theorem, which is very useful in applications is the Arzelá–Ascoli theorem. We will first need the following definitions. Definition 1.8.2 (Equicontinuity and pointwise (pre)compactness). Let X, Y be metric spaces and A ⊂ C(X, Y). (i) A is called equicontinuous if for every ϵ > 0 there exists δ > 0 such that for every x1 , x2 ∈ X with dX (x1 , x2 ) < δ we have dY (f (x1 ), f (x2 )) < ϵ for all f ∈ A. (ii) A is called pointwise compact if for every x ∈ X, the set A(x) = {f (x) : f ∈ A} is a compact subset of Y. (iii) A is called pointwise precompact if for every x ∈ X, the set A(x) = {f (x) : f ∈ A} is a compact subset of Y (i. e., A(x) has compact closure). Theorem 1.8.3 (Arzelá–Ascoli). Let (X, dX ) be a compact metric space. A subset A ⊂ C(X, Y) is precompact if and only if it is pointwise precompact and equicontinuous.

58 | 1 Preliminaries If in the above A is closed, then it is compact. In the special case where Y = ℝd , by the Heine–Borel theorem we may exchange pointwise precompactness with boundedness of A.

1.8.3 Partitions of unity Theorem 1.8.4 (Partition of unity). Let (X, τ), a topological space and K ⊂ X a compact subset of X. Let {Ui : i = 1, . . . , n} a finite covering of K with open sets. Then, there exists continuous functions ψi : X → [0, 1], i = 1, . . . , n, with compact support supp(ψi ) ⊂ Ui and such that ∑ni=1 ψi (x) = 1 for every x ∈ K. 1.8.4 Hölder continuous functions Definition 1.8.5 (Hölder continuity). A function f : 𝒟 ⊂ ℝd → ℝ is called Hölder continuous of exponent α ∈ (0, 1] if there exists c > 0 such that |f (x1 ) − f (x2 )| ≤ c|x1 − x2 |α for every x1 , x2 ∈ 𝒟 ⊂ ℝd . The case α = 1 corresponds to Lipschitz continuity. Definition 1.8.6 (Hölder spaces). We define the α-Hölder norm as ‖u‖C0,α (𝒟) := ‖u‖C(𝒟) +

sup

x1 ,x2 ∈𝒟,x1 =x̸ 2

(

|u(x1 ) − u(x2 )| ), |x1 − x2 |α

and denote the corresponding space of all functions such that the above norm is finite by C 0,α (𝒟), the space of Hölder continuous functions of exponent α. The Hölder space C k,α (𝒟) consists of all functions u ∈ C k (𝒟) for which the norm 󵄩 󵄩 ‖u‖Ck,α (𝒟) := ∑ 󵄩󵄩󵄩𝜕k u󵄩󵄩󵄩C(𝒟) + ∑ |m|≤k

sup

|m|=k x1 ,x2 ∈𝒟,x1 =x̸ 2

(

|𝜕k u(x1 ) − 𝜕k u(x2 )| ), |x1 − x2 |α

is finite where we use the multiindex notation. The Hölder spaces equipped with above norms are Banach spaces. Definition 1.8.7 (Oscillation and mean value of a function). Let f : ℝd → ℝ be a given function. For any r > 0, we define the oscillation osc(xo , r) := sup f (x) − x∈B(xo ,r)

inf

x∈B(xo ,r)

f (x),

and if it is locally integrable, the mean value

B(x0 ,r)

f (x)dx :=

1 |B(x0 , r)|

ˆ B(x0 ,r)

f (x)dx.

1.8 Appendix | 59

Theorem 1.8.8. A function f : ℝd → ℝ is Hölder continuous with exponent α ∈ (0, 1] if ffl and only if for every xo ∈ ℝd , r > 0 there exists a constant c > 0 such that B(x ,r) |f (x) − o f (xo )|dx < c r α . Alternatively, if osc(xo , r) < c r α for some α ∈ (0, 1], then f is Hölder continuous with Hölder exponent α. Proof. We sketch the proof. The direct implication is immediate. For the converse implication, assume that the claim is not true. Then, for every c > 0 there exist xo,1 , xo,2 ∈ ℝd such that |f (xo,1 ) − f (xo.2 )| ≥ c|xo,1 − xo,2 |α . Since by the triangle inequality for any x ∈ ℝd , we have that |f (x) − f (xo,1 )| + |f (x) − f (xo,2 )| ≥ |f (xo,1 ) − f (xo,2 )|, calculating the mean value integrals we have upon setting r = 2|xo,1 − xo,2 | that B(xo,1 ,r)

󵄨󵄨 󵄨 󵄨󵄨f (x) − f (xo,1 )󵄨󵄨󵄨dx +

B(xo,2 ,r)

󵄨󵄨 󵄨 󵄨 󵄨 󵄨󵄨f (x) − f (xo,2 )󵄨󵄨󵄨dx ≥ 󵄨󵄨󵄨f (xo,1 ) − f (xo,2 )󵄨󵄨󵄨 ≥ c|xo,1 − xo,2 |α = 2α c r α ,

ffl which implies that B(x ,r) |f (x) − f (xo,i )|dx ≥ 2α−1 c r α , for either i = 1 or i = 2, which o,i leads to a contradiction. The oscillation criterion follows immediately by the mean value criterion.

1.8.5 Lebesgue points Definition 1.8.9 (Lebesgue points). A Lebsegue point of an integrable function f : 𝒟 ⊂ ffl ℝd → ℝ is a point xo ∈ 𝒟 such that limr→0 B(x ,r) |f (x) − f (xo )|dx = 0. o

Any point of continuity is a Lebesgue point. The following theorem is very important. Theorem 1.8.10 (Lebesgue points for L1loc functions). If f ∈ L1loc (ℝd ), then a. e. xo ∈ ℝd is a Lebesgue point for f .

2 Differentiability, convexity and optimization in Banach spaces In this chapter, we consider the various notions of derivatives for mappings between Banach spaces, with special emphasis on the case of mappings taking values in ℝ, connecting derivatives with optimization problems. We then focus on convex functions in Banach spaces, and discuss at certain length their properties with respect to continuity, lower semicontinuity, differentiability and most importantly optimization. Moreover, an important operator, the projection operator on convex and closed subsets of Hilbert spaces is introduced and studied in detail. Finally, we consider various notions of the geometry of Banach spaces related to convexity, introducing the concept of strictly, locally and uniformly convex Banach space and study the properties of the duality map between such spaces and their duals. These important concepts have been covered in [13, 19, 20, 25, 41, 62] where the reader is refered to for further details.

2.1 Differentiability in Banach spaces Let X be a Banach space and F : X → ℝ be a functional (possibly nonlinear). In this section, we are going to introduce and discuss various concepts of differentiability which are generalizations of the usual concept of the derivative and are often used in calculus in Banach spaces (see, e. g., [41]). 2.1.1 The directional derivative Definition 2.1.1 (Directional derivative). The directional derivative of F at x ∈ X along the direction h ∈ X is the limit (whenever it exists) DF(x; h) = lim

ϵ→0

F(x + ϵ h) − F(x) . ϵ

It is not necessary that the operator defined by h 󳨃→ DF(x; h) is a linear operator. In fact, this may not hold even in finite dimensional spaces. As an illustrative example of that consider the functional F : ℝ2 → ℝ, defined by F(x1 , x2 ) =

x22r+1 (x12 +x22 )r

for (x1 , x2 ) ≠

(0, 0) and F(0, 0) = 0, for which the operator defined by h 󳨃→ DF(0; h) =

clearly not a linear operator.

h2r+1 2 , (h21 +h22 )r

is

2.1.2 The Gâteaux derivative If the operator defined by h 󳨃→ DF(x; h) is a linear operator, then we may introduce the concept of Gâteaux (or weak) derivative. https://doi.org/10.1515/9783110647389-002

62 | 2 Differentiability, convexity and optimization in Banach spaces Definition 2.1.2 (Gâteaux derivative). A functional F is called Gâteaux (or weakly) differentiable at x ∈ X if DF(x; h) exists for any direction h ∈ X and the operator h 󳨃→ DF(x; h) is linear and continuous. Then, by the Riesz representation theorem, there exists some DF(x) ∈ X ⋆ such that lim

ϵ→0

F(x + ϵ h) − F(x) =: DF(x; h) = ⟨DF(x), h⟩, ϵ

∀ h ∈ X.

(2.1)

The element DF(x) is called the Gâteaux derivative of F at x. If X = ℝd , then the Gâteaux derivative coincides with the gradient ∇F. It should also be clear by the above definition that if the Gâteaux derivative exists it is unique. It should also be evident that if F is Gâteaux differentiable at x ∈ X with Gâteaux derivative DF(x) ∈ X ⋆ , then 1󵄨 󵄨 lim 󵄨󵄨󵄨F(x + ϵ h) − F(x) − ⟨DF(x), ϵh⟩󵄨󵄨󵄨 = 0, ϵ

∀ h ∈ X.

ϵ→0

(2.2)

For this reason, DF(x; h) = ⟨DF(x), h⟩ is often called the Gâteaux differential. Furthermore, it is straightforward to extend the above definition so as to define Gâteaux differentiability and the Gâteaux derivative for a functional F on an open subset A ⊂ X; we simply have to ensure that the above definition holds for every x ∈ A. If the functional F is not defined over the whole Banach space X but only on a subset A ⊂ X, not necessarily open, then we must take some care when applying the above definitions to make sure that we only consider points x in the interior of A. Example 2.1.3 (Differentiability of norm related functionals in Hilbert space). Let X = H be a Hilbert space with norm ‖⋅‖ and inner product ⟨⋅, ⋅⟩, and consider the functionals F1 , F2 : H → ℝ defined by F1 (x) = ‖x‖ and F2 (x) = 21 ‖x‖2 = 21 ⟨x, x⟩, respectively. Then, using the properties of the inner product we easily find that 1 1 ϵ2 F2 (x + ϵh) − F2 (x) = ⟨x + ϵh, x + ϵh⟩ − ⟨x, x⟩ = ϵ⟨x, h⟩ + ‖h‖2 , 2 2 2 so that dividing by ϵ and passing to the limit as ϵ → 0, yields that DF2 (x; h) = ⟨x, h⟩,

∀ h ∈ X,

and by the Riesz representation theorem (see Theorem 1.1.14) we conclude that DF2 (x) = x. Note that this result holds true even if we choose x = 0. On the other hand, F1 (x + ϵh) − F1 (x) = ‖x + ϵh‖ − ‖x‖

= ⟨x + ϵh, x + ϵh⟩1/2 − ⟨x, x⟩1/2

1/2

= (⟨x, x⟩ + 2ϵ⟨x, h⟩ + ϵ2 ‖h‖2 )

− ⟨x, x⟩1/2

2.1 Differentiability in Banach spaces | 63

= ‖x‖ (1 + 2ϵ

1/2

2 ⟨x, h⟩ 2 ‖h‖ + ϵ ) ‖x‖2 ‖x‖2

− ‖x‖,

so that 1/2

2 1 ⟨x, h⟩ 2 ‖h‖ DF1 (x; h) = lim {‖x‖ (1 + 2ϵ + ϵ ) ϵ→0 ϵ ‖x‖2 ‖x‖2

− ‖x‖} =

1 ⟨x, h⟩, ‖x‖

which leads to the conclusion that DF1 (x) =

1 x, ‖x‖

by using the Riesz isomorphism. If x = 0 then, F1 (ϵh) − F1 (0) = |ϵ|‖h‖ and, therefore, the norm is not differentiable at x = 0. ◁ These results have to be modified in the case where X is a Banach space, where in fact the Gâteaux differentiability of the norm is related to geometric properties of the Banach space, such as for instance strict convexity (see Section 4.4). We now present an example concerning the Gâteaux differentiability of functionals related to the norm in a special, yet very important class of Banach spaces, Lebesgue spaces. Example 2.1.4 (Differentiability in Lp space). Let 𝒟 ⊂ ℝd be an open set and consider the Lebesgue spaces X = Lp (𝒟), 1 < p < ∞, so that any element x ∈ X is identified with a p-integrable function u : 𝒟 → ℝ. We will also use the notation v for any direction h of this function space, where v ∈ Lp (𝒟). Then if F : X → ℝ is defined by 󵄨 󵄨p F(u) = ‖u‖pLp (𝒟) = ∫󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx, 𝒟

we have that DF(u) = p|u|p−2 u = p|u|p−1 sgn(u), which clearly belongs to X ⋆ = Lp (𝒟), where p⋆ is such that 1/p + 1/p⋆ = 1. This result does not hold for p = 1 or p = ∞. To prove the above claim, we need to consider the limit ⋆

∫𝒟 |u(x) + ϵv(x)|p dx − ∫𝒟 |u(x)|p dx F(u + ϵ v) − F(u) lim = lim . ϵ→0 ϵ→0 ϵ ϵ Recall an elementary calculus fact; if ψ(s) = |a + bs|p , a, b ∈ ℝ, then dψ (s) = pb|a + bs|p−1 sgn(a + bs) = pb|a + bs|p−2 (a + bs). ds

64 | 2 Differentiability, convexity and optimization in Banach spaces Using that, along with the dominated convergence theorem1 to interchange the limit with the integral we obtain F(u + ϵ v) − F(u) |u(x) + ϵv(x)|p − |u(x)|p = ∫ lim dx ϵ→0 ϵ→0 ϵ ϵ lim

𝒟

󵄨p−2 󵄨 = ∫ p󵄨󵄨󵄨u(x)󵄨󵄨󵄨 u(x)v(x)dx. 𝒟

The right-hand side is identified as ⟨p|u|p−2 u, v⟩Lp⋆ (𝒟),Lp (𝒟) , which completes the proof of our claim. Note that we have only considered the case where x ≠ 0, i. e., the case where the function u(x) ≠ 0 a. e. in 𝒟. The norm itself is not differentiable at x = 0. ◁ The Gâteaux derivative satisfies some properties similar to the derivative in finite dimensional spaces, such as for instance the mean value theorem. Proposition 2.1.5 (Mean value theorem). Let F : X → ℝ be a Gâteaux differentiable functional. Then 󵄨󵄨 󵄨 󵄩 󵄩 󵄨󵄨F(x1 ) − F(x2 )󵄨󵄨󵄨 ≤ sup 󵄩󵄩󵄩DF(tx1 + (1 − t)x2 )󵄩󵄩󵄩X ⋆ ‖x1 − x2 ‖. t∈[0,1]

(2.3)

Proof. Consider the real valued function ϕ : ℝ → ℝ defined by ϕ(t) := F(x1 +t(x2 −x1 )), t ∈ [0, 1]. A simple calculation (using the Gâteaux differentiability of F at x1 in the direction h = x2 − x1 ) yields dϕ (t) = ⟨DF(x1 + t(x2 − x1 )), x2 − x1 ⟩. dt We now use the mean value theorem for the real valued function ϕ according to which dϕ there exists to ∈ [0, 1] such that ϕ(1) − ϕ(0) = dt (to ) which yields F(x2 ) − F(x1 ) = ⟨DF(x1 + to (x2 − x1 )), x2 − x1 ⟩, from which (2.3) follows.

2.1.3 The Fréchet derivative The Gâteaux differentiability is not the only concept of differentiability available in Banach space. 1 The details are left to the reader; however, note that this step requires the elementary convexity inequality |a|p − |a − b|p ≤ 1t [|a + tb|p − |a|p ] ≤ |a + b|p − |a|p , for |t| ≤ 1 (see, e. g., [78]).

2.1 Differentiability in Banach spaces | 65

Definition 2.1.6 (Fréchet differentiability). The functional F : X → ℝ is called Fréchet (strongly) differentiable at x ∈ X if there exists DF(x) ∈ X ⋆ such that lim

‖h‖→0

|F(x + h) − F(x) − ⟨DF(x), h⟩| = 0. ‖h‖

(2.4)

DF(x) is called the Fréchet derivative of F at x. If a functional is Fréchet differentiable, then its Fréchet derivative is unique. Remark 2.1.7. Note the difference with the Gâteaux derivative: if DF(x) is the Gâteaux derivative of F at x then (2.1) implies (2.2). This is not as strong a statement as (2.4), which in fact requires that the limit exists uniformly for all h ∈ X. To be more precise, (2.2) implies that F(x + ϵh) − F(x) = ϵ⟨DF(x), h⟩ + R(ϵ, h) where R is a remainder term such that limϵ→0 R(ϵ,h) = 0, however, the remainder term R depends on h in general. ϵ On the other hand (2.4) implies that F(x + h) − F(x) = ⟨DF(x), h⟩ + R󸀠 (‖h‖) with R󸀠 being 󸀠 (‖h‖) = 0. A more illustrative way of denoting a remainder term such that lim‖h‖→0 R ‖h‖ R and R󸀠 is using respectively the Landau symbols o(ϵ) and o(‖h‖), in the first case the rate of convergence depends on the choice of direction h, while in the second it is uniform in h as h → 0. We will sometimes use the small-o notation F(x + h) − F(x) − ⟨DF(x), h⟩ = o(‖h‖) to denote that DF(x) is the Fréchet derivative of F at x.

As a result, if a functional is Fréchet differentiable at x it is also Gâteaux differentiable at x but the converse does not necessarily hold, even in finite dimensional spaces, unless the Gâteaux derivative satisfies additional conditions (see Proposition 2.1.14). As an example, consider F : ℝ2 → ℝ defined by F(x1 , x2 ) = xx2 (x12 + x22 ) 1 if x1 ≠ 0 and F(x1 , x2 ) = 0 if x1 = 0, which is Gâteaux differentiable at (0, 0) but not Fréchet differentiable at this point, as we can see by, e. g., choosing the sequences hn = ( n1 , n1 ) and hn = ( n12 , n1 ) (see [14]). Fortunately, for a number of interesting cases we have Fréchet differentiability. Example 2.1.8 (Quadratic functions). Let X = H be a Hilbert space, with norm ‖ ⋅ ‖ and inner product ⟨⋅, ⋅⟩, and let A : H → H be a linear bounded operator. Consider the functional F : H → ℝ defined by F(x) = 21 ⟨Ax, x⟩. In the special case where A = I, the identity operator this functional becomes the square of the norm. We now consider the Gâteaux derivative of F. We see that 1 1 F(x + ϵh) − F(x) = ⟨A(x + ϵh), x + ϵh⟩ − ⟨Ax, x⟩ 2 2 1 1 1 = ϵ⟨Ah, x⟩ + ϵ⟨Ax, h⟩ + ϵ2 ⟨Ah, Ah⟩ 2 2 2 1 1 1 = ϵ⟨h, A⋆ x⟩ + ϵ⟨Ax, h⟩ + ϵ2 ⟨Ah, h⟩ 2 2 2 1 ⋆ 21 = ϵ⟨ (A + A )x, h⟩ + ϵ ⟨Ah, h⟩, 2 2

66 | 2 Differentiability, convexity and optimization in Banach spaces where we used the concept of the adjoint operator and the symmetry of the inner product. Using the fact that A is linear and bounded, we conclude that 1 DF(x; h) = ⟨ (A + A∗ )x, h⟩, 2

∀ h ∈ X,

so that, using the Riesz isomorphism, 1 DF(x) = (A + A∗ )x. 2 Note that the Gâteaux derivative of F is a symmetric operator. We will now show that F is Fréchet differentiable with DF(x) = 21 (A + A∗ )x. Indeed, retracing the steps above, we see that 󵄨󵄨 󵄨 󵄨󵄨F(x + h) − F(x) − ⟨DF(x), h⟩󵄨󵄨󵄨 =

1 󵄨󵄨 󵄨 2 󵄨⟨Ah, h⟩󵄨󵄨󵄨 ≤ c ‖h‖ , 2󵄨

where we used the fact that A is bounded and the constant c > 0 is related to the operator norm of A. Dividing by ‖h‖ and passing to the limit as ‖h‖ → 0 yields the required result (2.4). Note that assuming A is bounded (hence, continuous by linearity) is crucial for the proof of Fréchet differentiability of F. Furthermore, if A = I this result gives the Fréchet differentiability of the square of the norm in Hilbert space. ◁ 2.1.4 C 1 functionals Definition 2.1.9. Let A ⊂ X be an open set. If the Fréchet (or Gâteaux) derivative of a functional F : X → ℝ exists for every x ∈ A ⊂ X, and the mapping x 󳨃→ DF(x) is a continuous map, then we say that the functional F is Fréchet (or Gâteaux) continuously differentiable on A. We will use the notation C 1 (A) for Fréchet continuously differentiable functionals and CG1 (A) for Gâteaux continuously differentiable functionals. Example 2.1.10 (The Dirichlet integral). Let 𝒟 ⊂ ℝd be an open set with sufficiently smooth boundary and X = W01,2 (𝒟), a Sobolev space (see Definition 1.5.6), whose elements are identified with functions u : 𝒟 → ℝ. This space is a Hilbert space when equipped with the inner product ⟨u, v⟩ := ⟨∇u, ∇v⟩L2 (𝒟) (see Theorem 1.5.4 and Example 1.5.14). Consider the functional FD : X → ℝ defined by FD (u) = 21 ∫𝒟 |∇u(x)|2 dx. Then, identifying any direction h ∈ X with any function v ∈ W01,2 (𝒟), we see that 1 ϵ 󵄨 󵄨2 (F (u + ϵv) − FD (u)) = ∫ ∇u(x) ⋅ ∇v(x)dx + ∫󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 dx, ϵ D 2 𝒟

𝒟

taking the limit as ϵ → 0, and using Green’s formula we obtain 1 lim (FD (u + ϵv) − FD (u)) = − ∫ Δu(x)v(x)dx = ⟨−Δu, v⟩, ϵ

ϵ→0

𝒟

2.1 Differentiability in Banach spaces | 67

(using the standard inner product in L2 (𝒟)) where now DFD (u) = −Δu is identified as an element of X ⋆ = (W01,2 (𝒟))∗ . In this manner, we may define a mapping A : X → X ⋆ by u 󳨃→ −Δu, which is the variational definition of the Laplace operator. Clearly, FD is a symmetric bilinear form. The operator DFD (u) = −Δu is in fact the Fréchet derivative of DFD . This can be checked directly from Definition 2.1.6, by noting that for any v ∈ X we have (by Green’s formula) that ⟨DFD (u), v⟩ = ∫𝒟 ∇u(x) ⋅ ∇v(x)dx, so that FD (u + v) − FD (u) − ⟨DFD (u), v⟩ =

1 󵄨󵄨 1 󵄨2 ∫󵄨󵄨∇v(x)󵄨󵄨󵄨 dx = ‖v‖2W 1,2 (𝒟) , 0 2 2 𝒟

where, dividing by ‖v‖W 1,2 (𝒟) and passing to the limit as ‖v‖W 1,2 (𝒟) → 0 yields the re0 0 quired result (2.4). The operator DFD (u) = −Δu is continuous in (W01,2 (𝒟))∗ therefore FD is C 1 . Indeed, consider any u ∈ W01,2 (𝒟) and take w → u in W01,2 (𝒟). Then, for any v ∈ W01,2 (𝒟), 󵄨 󵄨󵄨 󵄨󵄨 󵄨 󵄨󵄨 󵄨 󵄨󵄨⟨DFD (u) − DFD (w), v⟩󵄨󵄨󵄨 = 󵄨󵄨󵄨− ∫ ∇(u − w) ⋅ ∇vdx 󵄨󵄨󵄨 ≤ ‖u − w‖W 1,2 (𝒟) ‖v‖W 1,2 (𝒟) , 0 0 󵄨󵄨 󵄨󵄨 𝒟

so that 󵄩󵄩 󵄩 󵄩󵄩DFD (u) − DFD (w)󵄩󵄩󵄩(W 1,2 (𝒟))∗ = 0

|⟨DFD (u) − DFD (w), v⟩| ‖v‖W 1,2 (𝒟) v∈W 1,2 (𝒟) sup

0

0

≤ ‖u − w‖W 1,2 (𝒟) , 0

hence, DFD (w) → DFD (u) in (W01,2 (𝒟))∗ as w → u in W01,2 (𝒟).

◁

2.1.5 Connections between Gâteaux, Fréchet differentiability and continuity The above definitions show that the C 1 property of a functional at a neighborhood N(x) of x implies the Fréchet differentiability of F at x and this in turn implies the Gâteaux differentiability of F at x. The reverse implications are not true in general. Fréchet differentiability resembles the standard definition of differentiability in the sense that it is connected with continuity. Proposition 2.1.11. Consider a subset A ⊂ X and let F : A → ℝ be Fréchet differentiable at a point x ∈ int(A). Then F is continuous at x. Proof. Since x ∈ int(A) there exists ϵ1 > 0 such that x + h ∈ A as long as ‖h‖ ≤ ϵ1 . Since, by assumption, (2.4) holds, for every ϵ > 0 there exists an ϵ2 > 0 such that 󵄨󵄨 󵄨 󵄨󵄨F(x + h) − F(x) − ⟨DF(x), h⟩󵄨󵄨󵄨 ≤ ϵ ‖h‖,

if ‖h‖ ≤ ϵ2 .

68 | 2 Differentiability, convexity and optimization in Banach spaces By the triangle inequality, 󵄨 󵄨 󵄨 󵄨󵄨 󵄨󵄨F(x + h) − F(x)󵄨󵄨󵄨 = 󵄨󵄨󵄨F(x + h) − F(x) − ⟨DF(x), h⟩ + ⟨DF(x), h⟩󵄨󵄨󵄨 󵄨 󵄨 󵄨 󵄨 ≤ 󵄨󵄨󵄨F(x + h) − F(x) − ⟨DF(x), h⟩󵄨󵄨󵄨 + 󵄨󵄨󵄨⟨DF(x), h⟩󵄨󵄨󵄨 󵄩 󵄨 󵄩 󵄨 ≤ 󵄨󵄨󵄨F(x + h) − F(x) − ⟨DF(x), h⟩󵄨󵄨󵄨 + 󵄩󵄩󵄩DF(x)󵄩󵄩󵄩X ⋆ ‖h‖, where ‖DF(x)‖X ⋆ is the norm of DF(x) ∈ X ⋆ . Let δ = min(ϵ1 , ϵ2 ). Then for any ϵ > 0, the above estimate yields the existence of δ > 0 such that 󵄨󵄨 󵄨 󵄩 󵄩 󵄨󵄨F(x + h) − F(x)󵄨󵄨󵄨 ≤ (ϵ + 󵄩󵄩󵄩DF(x)󵄩󵄩󵄩) ‖h‖,

if ‖h| < δ,

therefore leading to the conclusion that there exists a constant c > 0 such that 󵄨󵄨 󵄨 󵄨󵄨F(x + h) − F(x)󵄨󵄨󵄨 ≤ c ‖h‖,

if ‖h‖ < δ,

from which continuity of F at x follows. In contrast to the assurance of Proposition 2.1.11, the Gâteaux differentiability of a functional at a point does not guarantee continuity at this point for the functional, but rather a weaker property called hemicontinuity. Definition 2.1.12 (Hemicontinuity). The functional F : X → ℝ is called hemicontinuous at x ∈ X if the real valued function λ 󳨃→ F(x + λ h) is continuous at λ = 0 for every h ∈ X, i. e., limϵ→0 F(x + ϵh) = F(x) for every h ∈ X. A hemicontinuous function needs not be continuous even in finite dimensions as the example F(x1 , x2 ) = xx2 (x12 + x22 ) for x1 ≠ x2 , F(x1 , x2 ) = 0, for x1 = 0 indicates (see 1 [14]). Proposition 2.1.13. Consider a subset A ⊂ X and let F : A → ℝ be Gâteaux differentiable at a point x ∈ int(A). Then F is hemicontinuous at x. Proof. Take any h ∈ X and consider the real valued function ϕ : ℝ → ℝ defined by ϕ(λ) := F(x + λ h). The Gâteaux differentiability of F at x implies that the function ϕ is differentiable at λ = 0, therefore it is continuous at this point. This means F is hemicontinuous at x. The following result allows us to pass from Gâteaux to Fréchet differentiability. Proposition 2.1.14. Let F : X → ℝ be Gâteaux differentiable on an open neighborhood N(x) of x and x 󳨃→ DF(x) be continuous. Then F is also Fréchet differentiable at x. In other words, if F is CG1 in a neighborhood of x, then it is Fréchet differentiable at x. Proof. Fix h ∈ X and define g : ℝ → ℝ by g(t) := F(x + th) − F(x) − t⟨DF(x), h⟩. The function g is differentiable and an application of the mean value theorem implies that g(1) − g(0) = dg (t ) for some to ∈ (0, 1). Since dg (t ) = ⟨DF(x + to h) − DF(x), h⟩ we dt o dt o obtain that F(x + h) − F(x) − ⟨DF(x), h⟩ = ⟨DF(x + to h) − DF(x), h⟩ and combined with the continuity of DF leads to the required result.

2.1 Differentiability in Banach spaces | 69

Note that the converse does not hold in general, i. e., if a functional is Fréchet differentiable at x ∈ X, this does not necessarily imply that F is C 1 at x. On the other hand, if F is CG1 in a neighborhood of x, then by the above proposition F is also Fréchet differentiable at x, the Fréchet and the Gâteaux derivative coincide; hence, F is also continuously Fréchet differentiable at x.

2.1.6 Vector valued maps and higher order derivatives The concepts of directional, Gâteaux and Fréchet derivatives for maps f : X → Y where X, Y are two Banach spaces can be defined with the obvious generalizations. Definition 2.1.15. Let f : X → Y be a map between the Banach spaces X, Y. (i) The directional derivative at point x ∈ X along the direction h ∈ X, is defined as Df (x; h) := lim

ϵ→0

f (x + ϵh) − f (x) . ϵ

(ii) If f : A ⊂ X → Y, Df (x; h) is defined for any x ∈ int(A), any direction h ∈ X, and the operator h 󳨃→ Df (x; h), denoted by Df (x), is linear and continuous (Df (x) ∈ ℒ(X; Y)), we will say that f is Gâteaux differentiable at x and define Df (x)h := lim

ϵ→0

f (x + ϵh) − f (x) , ϵ

∀ h ∈ X.

(iii) If the stronger result lim

‖h‖X →0

‖f (x + h) − f (x) − Df (x)h‖Y = 0, ‖h‖X

holds for the operator Df (x) ∈ ℒ(X; Y), defined in (i), then we say that f is Fréchet (or strongly) differentiable at x and Df (x) is called the Fréchet derivative at x. All the results stated above such as, e. g., Proposition 2.1.11 or the mean value theorem, etc, hold subject to the necessary changes for the general case. One furthermore has the following chain rule for vector valued functions. Proposition 2.1.16. Suppose X, Y, Z are Banach spaces and f2 : X → Y is Gâteaux differentiable at x and f1 : Y → Z is Fréchet differentiable at y = f2 (x). Then f = f1 ∘ f2 : X → Z is Gâteaux differentiable at x and D(f1 ∘ f2 )(x) = Df1 (f2 (x)) Df1 (x). If f2 is Fréchet differentiable at x then so is f . Proof. Using Definition 2.1.15, it suffices to show that 1󵄩 󵄩 lim 󵄩󵄩󵄩f1 (f2 (x + ϵh)) − f1 (f2 (x)) − ϵDf1 (f2 (x))Df2 (x)h󵄩󵄩󵄩Z = 0, ϵ

ϵ→0

∀h ∈ X.

70 | 2 Differentiability, convexity and optimization in Banach spaces Adding and subtracting Df1 (f2 (x))(f2 (x+ ϵh) − f2 (x)) to the quantity inside the norm and using the triangle inequality 1 󵄩󵄩 󵄩 󵄩f (f (x + ϵh)) − f1 (f2 (x)) − ϵDf1 (f2 (x))Df2 (x)h)󵄩󵄩󵄩Z ϵ󵄩 1 2 󵄩󵄩 1 󵄩󵄩󵄩 1 󵄩 ≤ 󵄩󵄩󵄩f1 (f2 (x + ϵh)) − f1 (f2 (x)) − ϵDf1 (f2 (x))( (f2 (x + ϵh) − f2 (x)))󵄩󵄩󵄩 󵄩󵄩Z ϵ 󵄩󵄩 ϵ 󵄩󵄩 󵄩 1 󵄩 󵄩󵄩 + 󵄩󵄩󵄩Df1 (f2 (x))( (f2 (x + ϵh) − f2 (x) − ϵDf2 (x)h))󵄩󵄩󵄩 . 󵄩󵄩 󵄩󵄩Z ϵ The last term in the above tends to zero as ϵ → 0 on account of Gâteaux differentiability of f2 . To handle the first term, we multiply and divide by ‖f2 (x + ϵh) − f2 (x)‖Y and ̂ = f (x + ϵh) − f (x) ∈ Y, this term is rearranged as setting y = f2 (x) and h 2 2 ̂ ‖h‖ 1 󵄩󵄩 Y ̂ − f (y) − Df (y)h ̂ 󵄩󵄩󵄩 . 󵄩f (y + h) 1 1 󵄩Z ̂ 󵄩1 ϵ ‖h‖ Y ̂ → 0 as ϵ → 0 so that by the Since f2 is Gâteaux differentiable at x we have that h 1 ̂ ̂ → 0. On the Fréchet differentiability of f1 we have that ̂ ‖f1 (y + h) − f1 (y) − Df1 (y)h‖ Z other hand,

̂ ‖h‖ Y ϵ

‖h‖Y

→ ‖Df2 (x) h‖Y as ϵ → 0, so the result follows.

Higher order derivatives may be defined in a standard fashion, e. g., by considering the derivatives of derivatives. As an example, consider second derivatives. Definition 2.1.17. Let f : A ⊆ X → Y be Gâteaux differentiable at every x ∈ A and apply Definition 2.1.15 on the map Df (x) : A ⊂ X → ℒ(X; Y), defined by x 󳨃→ Df (x). If this map is Gâteaux differentiable at every x ∈ A, its Gâteaux derivative is an operator from X to ℒ(X; Y) denoted by D2 f (x) ∈ ℒ(X; ℒ(X; Y)). If we consider a map f : X → Y which is twice Gâteaux differentiable at x ∈ X, and we take any two directions h1 , h2 ∈ X, then D2 f (x)h1 ∈ ℒ(X; Y) and (D2 f (x)h1 )h2 ∈ Y. We may thus define the map B : X × X → Y by (h1 , h2 ) 󳨃→ (D2 f (x)h1 )h2 which is clearly a bilinear form. The special case Y = ℝ is often of interest. In the case where X = ℝd and Y = ℝ, D2 f (x) coincides with the Hessian matrix which is an element of ℝd×d . Taylor type formulae hold for maps which have higher order Gâteaux derivatives. For example, we have the following. Proposition 2.1.18. If F : X → ℝ is twice Gâteaux differentiable, then there exists an s ∈ (0, 1) such that 1 F(x + h) = F(x) + ⟨DF(x), h⟩ + ⟨D2 F(x + s h)h, h⟩. 2 Proof. If F is twice Gâteaux differentiable then the real valued function t 󳨃→ ϕ(t) := F(x + t h), for every h ∈ X is twice differentiable and application of Taylor’s formula for ϕ yields the required result.

2.2 General results on optimization problems | 71

2.2 General results on optimization problems We recall the following definitions. Definition 2.2.1 (Lower semicontinuity). A function F : X → ℝ ∪ {+∞} is called: (i) Sequentially lower semicontinuous if for every {xn : n ∈ ℕ} ⊂ X such that xn → x it holds that lim infn F(xn ) ≥ F(x). (ii) Lower semicontinuous if the sets Lλ F := {x ∈ X : F(x) ≤ λ} are closed in X for every λ ∈ ℝ. (iii) Weakly sequentially lower semicontinuous if for every {xn : n ∈ ℕ} ⊂ X such that xn ⇀ x it holds that lim infn F(xn ) ≥ F(x). (iv) Weakly lower semicontinuous if the sets Lλ F := {x ∈ X : F(x) ≤ λ} are weakly closed in X for every λ ∈ ℝ. The reader may verify that in the context of Banach space, Definitions 2.2.1(i) and (ii) are equivalent definitions for lower semicontinuity. Furthermore, the concept of lower semicontinuity is robust with respect to the supremum operation and the addition operation. Example 2.2.2 (The pointwise supremum of a family of lower semicontinuous functionals is lower semicontinuous). If {Fα : α ∈ ℐ } is a family of lower semicontinuous functionals, then the functional F defined by F(x) = supα∈ℐ Fα (x) is also lower semicontinuous (note that Lλ F = ⋂α∈ℐ Lλ Fα , so this is a closed set as the intersection of closed sets). Same applies if Fα are continuous for every α ∈ ℐ , in which case F is also lower semicontinuous. ◁ How are the weak and (strong) notions of lower semicontinuity related? In general, the weak notion implies the strong one. Proposition 2.2.3. Let X be a Banach space and F : X → ℝ ∪ {+∞} a proper function (i. e. not taking the value +∞ everywhere). If F is weakly (sequentially) lower semicontinuous, then it is also (sequentially) lower semicontinuous. Proof. For sequential lower semicontinuity, consider any sequence {xn : n ∈ ℕ} ⊂ X such that xn → x. Then it also holds that xn ⇀ x in X and by the weak sequential lower semicontinuity of F we have that F(x) ≤ lim infn F(xn ). That implies the lower sequential semicontinuity. For the lower semicontinuity, it suffices to consider that since any weakly closed subset of X is closed, this assertion holds. The converse of Proposition 2.2.3 is not necessarily true, unless extra conditions hold (see Proposition 2.3.13). We now turn to a discussion of optimization problems.

72 | 2 Differentiability, convexity and optimization in Banach spaces Definition 2.2.4 (Local minima and minima). Let F : A ⊂ X → ℝ, with A ≠ 0. (i) The point xo ∈ A is called a local minimum of F is there exists a neighborhood N(xo ) of xo such that F(xo ) ≤ F(x) for every x ∈ N(xo ) ∩ A. (ii) The point xo ∈ A is called a minimum2 of F in A if F(xo ) ≤ F(x) for every x ∈ A. The following result which is a variant of the Weierstrass theorem, valid for reflexive Banach spaces, is one of the cornerstones in optimization. Theorem 2.2.5 (Weierstrass). Let A ⊂ X be a bounded and weakly sequentially closed subset of a reflexive Banach space X, and F : A ⊂ X → ℝ be a weakly sequentially lower semicontinuous functional. Then F admits a minimum in A, i. e., there exists xo ∈ A such that F(xo ) ≤ F(x) for every x ∈ A. Proof. We will use the so-called direct method. Let {xn : n ∈ ℕ} ⊂ A be a minimizing sequence,3 i. e., a sequence such that F(xn ) → m where m = infx∈A F(x). Since A is bounded, this sequence is bounded. By the reflexivity of X, using Theorem 1.1.58 we conclude that there exists a subsequence {xnk : k ∈ ℕ} and a xo ∈ X such that xnk ⇀ xo in X. Since A is weakly sequentially closed, it follows that xo ∈ A. It remains to show that xo is such that F(xo ) = m, i. e., xo is the minimizer. Indeed, since F is weakly lower sequentially semicontinuous we have that lim infk F(xnk ) ≥ F(xo ) and since {xn : n ∈ ℕ} is a minimizing sequence lim infk F(xnk ) = limk F(xnk ) = m so that m ≥ F(xo ). But m = infx∈A F(x) so it follows that F(xo ) = m. An essential key to the proof of Theorem 2.2.5 was the fact that A was bounded which allowed us to invoke Theorem 1.1.58 to guarantee the existence of a weak limit of a subsequence of the minimizing sequence which is then recognized as the minimizer. Such assumptions of boundedness are often too much to ask for in infinite dimensional spaces, and they are replaced by alternative conditions on the functionals themselves that guarantee the boundedness of the minimizing sequence even if A is unbounded. Such conditions are called coercivity conditions. Definition 2.2.6 (Coercive functional). The map F : A ⊂ X → ℝ is called coercive on A if it satisfies the condition limn F(xn ) = ∞ for every sequence {xn : n ∈ ℕ} ⊂ A such that limn ‖xn ‖ = ∞. Coercivity is related with the boundedness of the level sets of F. Proposition 2.2.7. The map F : A ⊂ X → ℝ is coercive if and only if Lλ F := {x ∈ X : F(x) ≤ λ} is bounded for every λ ∈ ℝ. 2 or sometimes global minimum. 3 This is a standard step: Since m = infx∈A F(x), for any ϵ > 0 there exists x(ϵ) ∈ A such that m ≤ F(x(ϵ)) < m + ϵ. Consider ϵ = n1 , n ∈ ℕ and denote the corresponding x(ϵ) by xn . The resulting sequence has the property m ≤ F(xn ) < m + n1 , and is a minimizing sequence.

2.2 General results on optimization problems | 73

Proof. Suppose that F is coercive and Lλ F is not bounded for every λ ∈ ℝ. That implies that for some λo ∈ ℝ, the set Lλo F is not bounded, therefore, we may find a sequence {xn : n ∈ ℕ} such that F(xn ) ≤ λo and ‖xn ‖ → ∞. By the coercivity of F, we have that limn F(xn ) = ∞, which is in contradiction with limn F(xn ) ≤ λo . For the converse, suppose that Lλ F is bounded for every λ ∈ ℝ but F is not coercive. Then, since F is not coercive, there exists a sequence {xn : n ∈ ℕ} and λo ∈ ℝ such that ‖xn ‖ → ∞ and F(xn ) ≤ λo . This means that xn ∈ Lλo F for every n ∈ ℕ, and ‖xn ‖ → ∞, which is a contradiction since Lλo F is a bounded set. The above proposition connects coercivity with compactness properties of the set Lλ F, as long as X is endowed with an appropriate topology.4 Assuming coercivity of F we have the following variant of Weierstrass’s theorem. Theorem 2.2.8. Let A ⊂ X be a weakly sequentially closed subset of a reflexive Banach space X and F : A ⊂ X → ℝ be a weakly sequentially lower semicontinuous and coercive functional. Then F admits a minimum in A, i. e., there exists xo ∈ A such that F(xo ) ≤ F(x) for every x ∈ A. Proof. Assume that m = infx∈A F(x) finite. If {xn : n ∈ ℕ} ⊂ A is a minimizing sequence. Then by coercivity it is necessarily bounded and then we can follow verbatim the steps of the proof of Theorem 2.2.5. Example 2.2.9. Consider the functional F : X = W01,2 (𝒟) → ℝ, defined by F(u) = ∫𝒟 |∇u(x)|2 dx + ∫𝒟 f (x)u(x)dx, for any u ∈ W01,2 (𝒟) and a given f ∈ L2 (𝒟). The Dirichlet functional FD : X := W01,2 (𝒟) → ℝ defined by FD (u) = ∫𝒟 |∇u(x)|2 dx is coercive on X as easily follows by the Poincaré inequality (see Theorem 1.5.13), which guarantees that ‖∇u‖L2 (𝒟) is an equivalent norm for X := W01,2 (𝒟). This would not be true if we had defined the Dirichlet functional in such a way that did not allow us to use the Poincaré inequality, e. g., on the whole of W 1,2 (𝒟). It is easily seen that F, which is a linear perturbation of the Dirichlet functional is also coercive, as a result of the Sobolev embedding theorem. Thus, there exists a function uo ∈ W01,2 (𝒟) for which F admits its minimum value. This follows by a straightforward application of Theorem 2.2.8. ◁ Note that reflexivity of the Banach space on which the functional is defined plays an important role in all the above proofs. However, there are many functionals that their natural domains of definition are subsets of nonreflexive Banach spaces, e. g., L∞ , L1 or even measure spaces. In these cases, we have to use the concept of weak⋆ convergence and apply the Banach–Alaoglou theorem 1.1.36 concerning the weak⋆ compactness of the unit ball of the dual space. The next theorem is an example of a result in this spirit.

4 Typically, the weak topology if X is reflexive or if X = Y ⋆ the weak⋆ on Y ⋆ .

74 | 2 Differentiability, convexity and optimization in Banach spaces Theorem 2.2.10. Let X ≃ Y ⋆ where Y is a separable Banach space and F : X → ℝ be a proper coercive and sequentially weak⋆ lower semicontinuous functional. Then F attains its minimum on X and the set of minimizers is sequentially weak⋆ compact. The result is also true when F is restricted to A ⊂ X ≃ Y ⋆ as long as A is sequentially weak⋆ closed. Proof. In the spirit of the direct method of the calculus of variations, consider a minimizing sequence {xn : n ∈ ℕ} ⊂ X ≃ Y ⋆ . By coercivity, the minimizing sequence is bounded in X ≃ Y ⋆ therefore by Theorem 1.1.35(iii), there exists a subsequence ⋆ {xnk : k ∈ ℕ} ⊂ X ≃ Y ⋆ and an element xo ∈ X ≃ Y ⋆ such that xnk ⇀ xo . By the sequential weak⋆ lower semicontinuity of F we have that lim infk F(xnk ) ≥ F(xo ) and since {xn : n ∈ ℕ} is a minimizing sequence this implies that infx∈X F(x) ≥ F(xo ) so that xo is a minimizer. Remark 2.2.11. The separability of Y guarantees the metrizability of the weak⋆ topology on bounded sets of Y ⋆ ≃ X (see Theorem 1.1.35), therefore allowing us to use sequential characterizations of the various topological properties and in particular closedness. If Y is not separable, then the result is still true but we must ask for weak⋆ lower semicontinuity of F (i. e., asking that the level sets of F are weak⋆ closed sets5 rather than sequential weak⋆ lower semicontinuity6 ). There are two important cases where we need Theorem 2.2.10 in applications: the case where X ⋆ = ℳB (𝒟), Y = Cc (𝒟) used for studying optimization problems in L1 (𝒟) (understood as a subspace of ℳB (𝒟)) and the case where X = L∞ (𝒟) and Y = L1 (𝒟). Example 2.2.12. Let X = ℳ(𝒟) the space of Radon measures endowed with the total variation norm (see Definition 1.4.3 and Theorem 1.4.4). Consider the functional F : X = ℳ(𝒟) → ℝ defined by F(μ) = ‖T(μ) − z‖2L2 (𝒟) + α‖μ‖ℳ(𝒟) , where T : ℳ(𝒟) → L2 (𝒟) is a linear operator with the property ‖T(μ)‖L2 (𝒟) ≤ c‖μ‖ℳ(𝒟) for some constant c > 0 (independent of the choice of μ ∈ ℳ(𝒟)), α > 0 is a known parameter and z ∈ L2 (𝒟) is a given function. This problem admits a minimum μo ∈ ℳ(𝒟). An example for the map T can be the solution map of the elliptic equation −Δu = μ, for a measure valued d ). source term, which is known to admit (weak) solutions u ∈ W01,p (𝒟) for all p ∈ [1, d−1 c

Since W01,p (𝒟) 󳨅→ L2 (𝒟), the map defined by μ 󳨃→ T(μ) = u, where u is the solution of the above problem, has the desired properties. Problems of this type appear in a variety of applications. Indeed, consider a minimizing sequence {μn : n ∈ ℕ} ⊂ ℳ(𝒟). Since F(μn ) < c for some suitable constant c, it holds that ‖μn ‖ℳ(𝒟) < c, so that by Theorem 1.4.10(i) there exists a subsequence {μnk : k ∈ ℕ} and a μo ∈ ℳ(𝒟), such that μnk ⇀ μo in ⋆

5 and not just weak⋆ sequentially closed. 6 If a functional F is weak⋆ lower semicontinuous, then it is also sequentially weak⋆ lower semicontinuous but the converse is not true.

2.3 Convex functions |

75

ℳ(𝒟) as k → ∞. By the properties of the mapping T and the weak sequential lower semicontinuity of the norms, we conclude the existence of a minimum. ◁

If F is Gâteaux differentiable, local minimizers in open subsets of X can be characterized in terms of first-order conditions related to the Gâteaux derivative DF. Theorem 2.2.13 (First-order conditions). Let F : A ⊂ X → ℝ and xo ∈ int( A) be a local minimum of F. If F is Gâteaux differentiable at xo , then the first-order condition DF(xo ) = 0 holds. Proof. Let h ∈ X be an arbitrary direction. Then there exists ϵo > 0 such that for each 0 < ϵ < ϵo , it holds that F(xo ) ≤ F(xo + ϵ h). A simple manipulation leads to F(xo + ϵ h) − F(xo ) ≥ 0, ϵ and since F is Gâteaux differentiable at x we have that ⟨DF(xo ), h⟩ ≥ 0,

∀ h ∈ X.

Since h ∈ X is arbitrary and X is a vector space we may repeat the above procedure for −h ∈ X so that we finally obtain that at the local minimum xo it holds that ⟨DF(xo ), h⟩ = 0,

∀ h ∈ X,

which is the stated first-order condition. The converse statement is clearly not true even in finite dimensional spaces (e. g., critical points may not be minima). Example 2.2.14. Consider the functional defined in Example 2.1.10 by F(u) = ∫𝒟 |∇u(x)|2 dx + ∫𝒟 f (x)u(x)dx for every u ∈ W01,2 (𝒟) and for a given f ∈ L2 (𝒟). As shown in Example 2.1.10, F admits a minimum. This functional consists of the Dirichlet functional plus a linear form. By Example 2.1.10, one can see that the Gâteaux derivative of F is DF(u) = −Δu + f , so that the first-order condition for the minimum uo reduces to the Poisson equation −Δuo + f = 0 with homogeneous Dirichlet boundary conditions on 𝒟. ◁

2.3 Convex functions We now turn our attention to the study of convex functions from a Banach space to ℝ. As we shall see, convex functions enjoy important and very useful properties with respect to differentiability, continuity, semicontinuity and most importantly optimization (see [19, 25] or [88]).

76 | 2 Differentiability, convexity and optimization in Banach spaces Notation 2.3.1. The notation φ : X → ℝ will be used instead of F : X → ℝ for convex maps. We will also freely move between the terminology map, functional and function. We may also allow convex functions to take the value +∞ denoting that by φ : X → ℝ ∪ {+∞}. 2.3.1 Basic definitions, properties and examples Let X be a Banach space with norm ‖ ⋅ ‖, X ⋆ its dual space and ⟨⋅, ⋅⟩ the duality pairing between them. Definition 2.3.2 (Proper and convex functions). A function φ : X → ℝ∪{+∞} is called proper and convex if it is not identically +∞ and it satisfies φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ),

∀ x1 , x2 ∈ X,

and

t ∈ [0, 1].

If the above inequality is strict for any t ∈ (0, 1), x1 ≠ x2 , the function is called strictly convex. Definition 2.3.3 (Uniformly and strongly convex functions). A function φ : X → ℝ ∪ {+∞} is called uniformly convex if there exists an increasing function ψ : ℝ+ → ℝ+ , with ψ(0) = 0 (and vanishing only at 0) such that φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ) − t(1 − t)ψ(‖x1 − x2 ‖),

∀ x1 , x2 ∈ X,

t ∈ (0, 1),

with the function ψ called the modulus of uniform convexity. In the special case where ψ(s) = c2 s2 , c > 0, the function φ is called strongly convex with modulus of convexity c. We will also need to define the following sets: Definition 2.3.4 (Effective domain and epigraph). Consider a map φ : X → ℝ ∪ {+∞} (not necessarily convex). (i) The effective domain of φ is defined as dom φ = {x ∈ X : φ(x) < +∞}. (ii) The epigraph of φ is defined as epi φ = {(x, λ) ∈ X × ℝ : φ(x) ≤ λ}. (iii) The strict epigraph of φ is defined as epis φ = {(x, λ) ∈ X × ℝ : φ(x) < λ}. The norm of a Banach space is a convex function (as follows directly by the triangle inequality), but it is not strictly convex as one can see choosing x2 = αx1 , for any x1 ≠ 0, 0 < α < 1. On the other hand, if X = H is a Hilbert space, the map x 󳨃→ p1 ‖x‖p , p > 1 is strictly convex, as is also the case for a large family of Banach spaces (see Theorem 2.6.5 and Remark 2.6.6). Another example of convex functional on W01,p (𝒟) is the generalized Dirichlet integral u 󳨃→ p1 ∫𝒟 |∇u(x)|p dx, p > 1 (as can be seen directly by the convexity of the real valued function s 󳨃→ p1 |s|p , or the fact that by the Poincaré inequality it is a norm).

2.3 Convex functions |

77

Example 2.3.5 (Convexity of the maximum functional). Let K be a compact metric space and consider the metric space X = C(K; ℝ) of continuous mappings x : K → ℝ, endowed with the usual sup norm ‖x‖X := maxs∈K |x(s)| (the max taken instead of the sup on account of the compactness of K). Define the functional φ : X → ℝ by φ(x) = maxs∈K x(s) (this is not the norm, as we have omitted the absolute value). This is a convex and continuous functional. Convexity is immediate. Indeed, take any pair x1 , x2 ∈ X = C(K, ℝ) and form the convex combination tx1 + (1 − t)x2 . Then for any s ∈ K, (tx1 + (1 − t)x2 )(s) = tx1 (s) + (1 − t)x2 (s) ≤ t max x1 (s) + (1 − t) max x2 (s), s∈K

s∈K

and taking the maximum over s ∈ K in the left-hand side leads to φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ), which is the convexity property for φ. For proving continuity of φ, consider any pair x1 , x2 ∈ X = C(K, ℝ). Let si ∈ K be the point on which the maximum7 for x1 is attained, i. e., xi (si ) = maxs∈K xi (s), i = 1, 2. By the definition of the functional φ, we have that φ(x1 ) − φ(x2 ) = x1 (s1 ) − max x2 (s) ≤ x1 (s1 ) − x2 (s1 ) ≤ max |x1 (s) − x2 (s)|. s∈K

s∈K

Interchange now the role of x1 , x2 to get φ(x2 ) − φ(x1 ) = x2 (s2 ) − max x1 (s) ≤ x2 (s2 ) − x1 (s2 ) ≤ max |x1 (s) − x2 (s)| s∈K

s∈K

These inequalities imply that 󵄨󵄨 󵄨 󵄨 󵄨 󵄨󵄨φ(x1 ) − φ(x2 )󵄨󵄨󵄨 ≤ max󵄨󵄨󵄨x1 (s) − x2 (s)󵄨󵄨󵄨 = ‖x1 − x2 ‖X , s∈K which is the desired continuity.

◁

Convexity of functions and convexity of sets are two concepts related through the concept of the epigraph. Proposition 2.3.6. A function φ : X → ℝ ∪ {+∞} is convex if and only if its epigraph, epi φ is a convex set. Proof. Let φ be convex and let (x1 , λ1 ), (x2 , λ2 ) ∈ epi φ. Then φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ) ≤ tλ1 + (1 − t)λ2 , 7 Recall that K is compact and xi is continuous so by Weierstrass the point si exists.

78 | 2 Differentiability, convexity and optimization in Banach spaces (since both these pairs are in the epigraph), so (tx1 + (1 − t)x2 , (tλ1 + (1 − t)λ2 ) ∈ epi φ,

∀ t ∈ [0, 1],

i. e., t(x1 , λ1 ) + (1 − t)(x2 , λ2 ) ∈ epi φ,

∀ t ∈ [0, 1],

hence, epi φ is a convex set. Conversely, let epi φ be convex. Since (x1 , φ(x1 )), (x2 , φ(x2 )) ∈ epi φ, it follows that (tx1 + (1 − t)x2 , tφ(x1 ) + (1 − t)φ(x2 )) ∈ epi φ,

∀ t ∈ [0, 1],

so, φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ),

∀ t ∈ [0, 1],

hence, φ is convex. Convex functions have the following useful properties. Proposition 2.3.7 (Properties of convex functions). (i) The sum of convex functions is a convex function. (ii) If φ : X → ℝ ∪ {+∞} is a convex function then λφ is convex for any λ > 0. (iii) The pointwise supremum of a set of convex functions is a convex function, i. e., if φi : X → ℝ ∪ {+∞} are convex then φ = supi∈ℐ φi defined by φ(x) := supi∈ℐ φi (x) is also convex. Proof. (i) and (ii) follow in an elementary fashion from the definition of convex functions. For (iii), note that epi φ = epi(supi∈ℐ φi ) = ⋂i∈ℐ epi φi . Since φi are convex functions, the sets epi φi are convex (Proposition 2.3.6) for every i ∈ ℐ and the intersection of any family of convex sets is also convex (Proposition 1.2.4(i)). Therefore, epi φ is a convex set and so φ is a convex function. We close this general discussion of convex functions with some important properties with respect to composition. Proposition 2.3.8. Let φ : X → ℝ ∪ {+∞} be a convex function. (i) If f : ℝ → ℝ is an increasing convex function, then the composition f ∘ φ : X → ℝ ∪ {+∞} is a convex function. (ii) If A : Y → X be a continuous linear map, where Y is a Banach space, such that φ ∘ A is well defined. Then φ ∘ A : Y → ℝ ∪ {+∞} is a convex function. Proof. The proof is elementary and is left as an exercise.

2.3 Convex functions |

79

2.3.2 Three important examples: the indicator, Minkowski and support functions There are three important examples of functions related to convex sets. 2.3.2.1 The indicator function of a convex set Definition 2.3.9 (Indicator function of a convex set). Let C ⊂ X be a set and define the function φ : X → ℝ ∪ {+∞} by φ(x) = IC (x) where IC is the indicator function8 of C defined as 0, x∈C IC (x) = { +∞, x ∈ X \ C. Since epi φ = C it is clear by Proposition 2.3.6 that φ = IC is a proper convex function if and only if C ⊂ X is a convex set. 2.3.2.2 The Minkowski functional Definition 2.3.10 (Minkowski functional). For any set C ⊂ X define the functional φC : X → ℝ, as φC (x) = inf{t > 0 : t −1 x ∈ C}. This functional is called the Minkowski functional or the gauge of C. The Minkowski functional has some very interesting properties (see e. g. [28]) which allow the characterization of convex sets. Proposition 2.3.11. If C is an open convex set containing 0, it holds that C = {x ∈ X : φC (x) < 1} and φC is sublinear. Proof. To show that C is characterized in terms of φC , let us consider first that x ∈ C. Since C is open, there exists ϵ > 0, sufficiently small, such that B(x, ϵ) ⊂ C, and by convexity of C we also have that (1 + ϵ)x ∈ C; therefore, by the definition of φC we have 1 that φC (x) ≤ 1+ϵ < 1. For the converse, assume that x is such that φC (x) < 1. By the definition of φC , there exists a t ∈ (0, 1) such that t −1 x ∈ C. We express x in terms of a convex combination as x = t(t −1 x) + (1 − t)0 ∈ C, by the convexity of C. To show that it is sublinear, consider two x1 , x2 ∈ X and ϵ > 0. Note that φC (tx) = t φC (x) for every t > 0 and x ∈ C (this is immediate). For the choice (a) x = x1 , t = (φC (x1 ) + ϵ)−1 and (b) x = x2 , t = (φC (x2 ) + ϵ)−1 , we see that φC ((φC (xi ) + ϵ)−1 xi ) < 1 for i = 1, 2 hence, by the above (φC (xi ) + ϵ)−1 xi ∈ C for i = 1, 2. By the convexity of C for any t ∈ [0, 1] we have that t(φC (x1 ) + ϵ)−1 x1 + (1 − t)(φC (x2 ) + ϵ)−1 x2 ∈ C. We choose φC (x1 )+ϵ t = φ (x )+φ , which gives us that (φC (x1 ) + φC (x2 ) + 2ϵ)−1 (x1 + x2 ) ∈ C, so that (x )+2ϵ C

1

C

2

8 Note that the definition of the indicator function employed here is the one commonly used in convex analysis, which differs from the standard definition of the characteristic function of a set (often called indicator function too).

80 | 2 Differentiability, convexity and optimization in Banach spaces φC ((φC (x1 ) + φC (x2 ) + 2ϵ)−1 (x1 + x2 )) < 1, which by the above property is equivalent to (φC (x1 ) + φC (x2 ) + 2ϵ)−1 φC (x1 + x2 ) < 1, which leads to φC (x1 + x2 ) < φC (x1 ) + φC (x2 ) + 2ϵ. This is true for any ϵ > 0, so passing to the limit as ϵ → 0 we obtain that φC (x1 + x2 ) ≤ φC (x1 ) + φC (x2 ). The proof of the geometric form of the Hahn-Banach theorem can be based on the use of the Minkowski functional along with the analytic form (exercise). 2.3.2.3 The support function Definition 2.3.12 (Support function). With any convex set C ⊂ X, we may associate a function σC : X ⋆ → ℝ ∪ {+∞} defined by σC (x⋆ ) = sup⟨x⋆ , x⟩. x∈C

If x⋆o ∈ X ⋆ , x⋆o ≠ 0, is such that σC (x⋆o ) = ⟨x⋆o , xo ⟩ for some xo ∈ X, then x⋆o is said to support C at the point xo ∈ C (or equivalently is called a supporting functional at xo ). Note that we have encountered a particular case of the support function before. In case C = BX (0, 1) then by the definition of the norm for X ⋆ we have that σB (0,1) (x⋆ ) = X ‖x⋆ ‖X ⋆ for every x⋆ ∈ X ⋆ . The support function plays an important role in the theory of duality. The convexity of the support function follows from its definition and Proposition 2.3.7, and it is easy to see that σC (tx⋆ ) = tσC (x⋆ ) for every t > 0. The support function of a closed convex nonempty set C characterizes the set, and allows it to be expressed as the intersection of appropriately chosen halfspaces, in terms of (see, e. g., [13]) C=

⋂

x⋆ ∈X ⋆ ,

x⋆ =0 ̸

Hx⋆ ≤σC (x⋆ ) = {x ∈ X : ⟨x⋆ , x⟩ ≤ σC (x⋆ ), ∀ x⋆ ∈ X ⋆ }.

(2.5)

2.3.3 Convexity and semicontinuity The reasoning of Proposition 2.2.3 cannot be reversed in the general case, so we may need to distinguish between lower semicontinuity and weak lower semicontinuity. However, for convex functions this is not needed, as the two notions coincide. Proposition 2.3.13. Let X be a Banach space and φ : X → ℝ ∪ {+∞} a proper convex function. Then, if φ is lower semicontinuous it is also weakly lower semicontinuous. Proof. It follows since the epigraph of φ is convex and closed; hence, weakly closed (by Proposition 1.2.12), so we have the weak lower semicontinuity.

2.3 Convex functions |

81

Example 2.3.14 (Lower semicontinuity of indicator functions). Consider the function φ = IC where now C ⊂ X is a closed convex set. It is clear that under these conditions φ = IC is proper, convex and lower semicontinuous. ◁ Example 2.3.15 (Lower semicontinuity of the support function). The support function σC of a closed convex set (see Definition 2.3.12) is lower semicontinuous. This can be seen easily, as for any x⋆ ∈ X ⋆ the value of the function can be expressed as the supremum of a family of continuous functions. ◁ Example 2.3.16. A convex functional which is lower semicontinuous is also weakly sequentially lower semicontinuous. One of the possible ways to show this is using the following argument based on Mazur’s lemma. Take any sequence x󸀠n ⇀ x and let c = lim infn φ(zn ). There exists a subsequence, which we relabel as {xn : n ∈ ℕ} such that limn φ(xn ) = lim infn φ(x󸀠n ), where of course xn ⇀ x. We must show that φ(x) ≤ c. By Mazur’s lemma 1.2.17 there exists a sequence {zn : n ∈ ℕ} ⊂ X, such that zn := ∑N(n) a(n)k xk , with ∑N(n) a(n)k = 1, k=n k=n a(n)k ∈ [0, 1], k = n, . . . , N(n), n ∈ ℕ, and zn → x. By the strong semicontinuity of φ we have that φ(x) ≤ lim infn φ(zn ). Convexity of φ implies that N(n)

N(n)

k=n

k=n

φ(zn ) = φ( ∑ a(n)k xk ) ≤ ∑ a(n)k φ(xk ).

(2.6)

By the choice of {xn : n ∈ ℕ} for any ϵ > 0, there exists m ∈ ℕ such that φ(xn ) < c + ϵ for n > m. Using (2.6) for n > m to obtain φ(zn ) ≤ c + ϵ, and taking limit inferior in this, we obtain that lim infn φ(zn ) ≤ c + ϵ. Therefore, φ(x) < c + ϵ, for every ϵ > 0; hence, φ(x) ≤ c. ◁ The argument above implies that as far as convex functions are concerned we do not have to worry too much distinguishing between the various forms of lower semicontinuity, since for convex functions these concepts coincide. Example 2.3.17. The Dirichlet functional FD : X := W01,2 (𝒟) → ℝ (see Example 2.1.10) is convex and (by Fatou’s lemma) lower semicontinuous; hence, (by convexity) also weakly lower semicontinuous. ◁ Convex functions which are lower semicontinuous can be supported from below by affine functions [13, 19]. Proposition 2.3.18. Let φ : X → ℝ ∪ {+∞} be a proper, convex, lower semicontinuous function. Then φ is bounded from below by an affine function, i. e., there exists x⋆ ∈ X ⋆ and μ ∈ ℝ such that φ(x) ≥ ⟨x⋆ , x⟩ + μ,

∀ x ∈ X.

82 | 2 Differentiability, convexity and optimization in Banach spaces Proof. Let xo ∈ X and λo ∈ ℝ such that φ(xo ) > λo . Since (xo , λo ) ∉ epi φ and epi φ is closed and convex, by the separation Theorem 1.2.9, there exists9 a (x⋆o , λo⋆ ) ∈ X ⋆ × ℝ and α ∈ ℝ such that ⟨x⋆o , xo ⟩ + λo⋆ λo < α,

(2.7)

and ⟨x⋆o , x⟩ + λo⋆ λ > α,

∀ (x, λ) ∈ epi φ.

(2.8)

Since (xo , φ(xo )) ∈ epi φ, we have ⟨x⋆o , xo ⟩ + λo⋆ φ(xo ) > α.

(2.9)

It then follows from (2.7) and (2.9) that λo⋆ (φ(xo ) − λo ) > 0 and so λo⋆ > 0. Now applying

(2.8) to (x, λ) = (x, φ(x)) ∈ epi φ, and dividing by λo⋆ > 0, we obtain φ(x) > x⋆

and upon setting x⋆ = − λo⋆ and μ = o

α , λo⋆

the claim is established.

α λo⋆

x⋆

+ ⟨− λo⋆ , x⟩, o

In fact, a little more can be stated in the same spirit as Proposition 2.3.18. The next theorem, which characterizes convex and lower semicontinuous functions in terms of the supremum of a family of affine maps, is very useful in a number of applications, e. g., in providing lower semicontinuity results for integral functionals in the calculus of variations. See, e. g., Proposition 4.75 in [66] or Corollary 2.24 in [19]. Theorem 2.3.19 (Approximation of convex functions). φ : X → ℝ ∪ {+∞} is a proper convex lower semicontinuous function if and only if φ(x) = sup g(x), g∈𝔸(φ)

𝔸(φ) := {g : X → ℝ, affine continuous, g ≤ φ},

and the supremum is taken pointwise. Proof. We sketch the proof. Fix any xo ∈ dom(φ). To show that φ(xo ) = sup{g(xo ) : g ∈ 𝔸, g ≤ φ} it suffices to show that for any ϵ > 0, the quantity φ(xo ) − ϵ is not an upper bound of the set {g(xo ) : g ∈ 𝔸, g ≤ φ}, i. e., that there exists an affine continuous function g ≤ φ such that g(xo ) ≥ φ(xo )−ϵ. The existence of such a function follows from the separation argument, choosing the convex closed set epi φ and the point (xo , φ(xo ) − ϵ) ∈ ̸ epi φ, working similarly as in Proposition 2.3.18 above. In the case where X = ℝd , the supremum can be taken over a countable family of affine functions, (see Proposition 4.78 in [66]), i. e., there exists a family {(bn , an ), : n ∈ ℕ} ⊂ ℝd × ℝ such that φ(x) = sup{bn ⋅ x + an : n ∈ ℕ}. 9 Note that if X is a Banach space, so is the space X × ℝ under the norm ‖(x, λ)‖ = ‖x‖ + |λ|. We note also that its dual, (X × ℝ)∗ can be identified with X ⋆ × ℝ using the duality pairing, ⟨(x⋆ , λ⋆ ), (x, λ)⟩ = ⟨x⋆ , x⟩ + λ⋆ λ.

2.3 Convex functions |

83

2.3.4 Convexity and continuity The next results (see, e. g., [25] or [26]) provide continuity properties for convex functions. Proposition 2.3.20. Let φ : X → ℝ ∪ {+∞} be a proper convex function. If φ is locally bounded at xo ∈ int( dom φ) then φ is locally Lipschitz at xo . In fact, it suffices that φ is locally bounded above at xo . Proof. See Section 2.7.1 in the Appendix. Working as in the proof of Proposition 2.3.20, one can show for some x ∈ dom(φ) the equivalence of the statements (a) φ is continuous at x, (b) φ is Lipschitz on some neighborhood of x, (c) φ is bounded on some neighborhood of x and (d) φ is bounded above on some neighborhood of x. Proposition 2.3.20 essentially connects the nonemptiness of the interior of some level set of φ, Lλ φ := {x ∈ X : φ(x) ≤ λ} with its continuity. On the other hand, the domain of φ can be expressed as dom φ = ⋃λ∈ℝ Lλ φ = ⋃k∈ℤ Lk φ. One could be tempted to jump to the conclusion that if x ∈ int( dom φ) then this automatically implies the continuity of φ at x, but while this holds in finite dimensional spaces, this is not necessarily true in infinite dimensions. This happens because it is not necessarily true that int( dom φ) ≠ 0 implies the existence of some k ∈ ℤ such that int( Lk φ) ≠ 0, unless certain extra assumptions are made on the sets in question. By the Baire category, Theorem 2.7.1 (Section 2.7.2), we see that closedness of the level sets can lead us to the required result. But closedness of the level sets is equivalent to lower semicontinuity of the function. The above comments lead us to an interesting link between lower semicontinuity and continuity for convex functions. Proposition 2.3.21. Let φ : X → ℝ ∪ {+∞} be a proper convex lower semicontinuous function. Then int( dom φ) = core (dom φ) and φ is locally Lipschitz continuous on int( dom φ) = core (dom φ). If X is finite dimensional, then lower semicontinuity of φ is not required. Proof. See Section 2.7.2 in the Appendix at the end of the chapter.

2.3.5 Convexity and differentiability Convexity is related with differentiability in the sense that if a convex function is Gâteaux differentiable, then this imposes important constraints on its derivatives. Theorem 2.3.22. Let φ : X → ℝ be Gâteaux differentiable in a convex open subset C ⊂ X.

84 | 2 Differentiability, convexity and optimization in Banach spaces (i) φ is convex if and only if φ(x2 ) − φ(x1 ) ≥ ⟨Dφ(x1 ), x2 − x1 ⟩,

∀ x1 , x2 ∈ C.

(2.10)

(ii) φ is convex if and only if ⟨Dφ(x2 ) − Dφ(x1 ), x2 − x1 ⟩ ≥ 0,

∀ x1 , x2 ∈ C.

(2.11)

If φ is strictly convex, the above inequalities are strict. Proof. (i) Assume that φ is convex. Consider two points x1 , x2 ∈ C and take the convex combination (1 − t)x1 + tx2 = x1 + t(x2 − x1 ). Convexity of φ implies that φ(x1 + t(x2 − x1 )) ≤ (1 − t)φ(x1 ) + tφ(x2 ) = φ(x1 ) + t(φ(x2 ) − φ(x1 )), for all x1 , x2 ∈ C, t ∈ (0, 1), which leads upon rearrangement to φ(x1 + t(x2 − x1 )) − φ(x1 ) ≤ φ(x2 ) − φ(x1 ). t Since φ is Gâteaux differentiable at x1 ∈ X we may pass to the limit t → 0 and interpreting Dφ(x1 ) as an element of X ⋆ this leads to ⟨Dφ(x1 ), x2 − x1 ⟩ ≤ φ(x2 ) − φ(x1 ). To prove the converse, suppose (2.10) holds for every x1 , x2 ∈ C. Then, for any t ∈ (0, 1), (2.10) holds for the choice x1 , x1 +t(x2 −x1 ) ∈ C as well as the choice x2 , x1 +t(x2 −x1 ) ∈ C. This leads to the inequalities: φ(x1 ) ≥ φ(x1 + t(x2 − x1 )) − t ⟨Dφ(x1 + t(x2 − x1 )), x2 − x1 )⟩,

φ(x2 ) ≥ φ(x1 + t(x2 − x1 )) + (1 − t) ⟨Dφ(x1 + t(x2 − x1 )), x2 − x1 )⟩. We multiply the first by (1 − t) and the second by t and add to obtain convexity. (ii) Assume convexity of φ. Apply (2.10) twice, interchanging x1 and x2 . and add to obtain (2.11). Conversely, let (2.11) hold. An application of the mean value formula (see Proposition 2.1.18) implies that for all x1 , x2 ∈ C there exists t ∈ (0, 1) such that φ(x2 ) − φ(x1 ) = ⟨Dφ(x1 + t(x2 − x1 )) − Dφ(x1 ), x2 − x1 ⟩ + ⟨Dφ(x1 ), x2 − x1 ⟩. We now apply (2.11) for the pair (x1 + t(x2 − x1 ), x1 ) ∈ C × C we obtain ⟨Dφ(x1 + t(x2 − x1 )) − Dφ(x1 ), x2 − x1 ⟩ ≥ 0,

∀ t ∈ (0, 1).

Combining these two inequalities, we obtain (2.10). Therefore, φ is convex.

2.3 Convex functions | 85

If a convex function is twice differentiable, convexity imposes important positivity properties on its second derivative. Theorem 2.3.23. Let C ⊂ X be a convex and open set and φ : C ⊂ X → ℝ be twice Gâteaux differentiable in all directions in C. The function φ is convex if and only if D2 φ is positive definite in the sense that ⟨D2 φ(x)h, h⟩ ≥ 0,

∀ x ∈ C, h ∈ X.

(2.12)

If φ is strictly convex, the above inequality is strict. Proof. Follows immediately by the Taylor expansion (Proposition 2.1.18). Proposition 2.3.24. If a functional φ : X → ℝ is convex and is Gâteaux differentiable at x ∈ X then φ is weakly lower sequentially semicontinuous at x. Proof. Consider a sequence {xn : n ∈ ℕ} ⊂ X such that xn ⇀ x in X. Since φ is convex and Gâteaux differentiable at x, apply (2.10) for the choice x1 = x and x2 = xn to obtain φ(xn ) − φ(x) ≥ ⟨Dφ(x), xn − x⟩,

n ∈ ℕ.

(2.13)

Since xn ⇀ x in X for every x⋆ ∈ X ⋆ it holds that ⟨x⋆ , xn − x⟩ → 0 as n → ∞. Choose x⋆ = Dφ(x) ∈ X ⋆ and go to the limit as n → ∞ in (2.13) to obtain that lim inf φ(xn ) ≥ φ(x) n

which guarantees the weak lower semicontinuity of φ. Convexity and differentiability lead to interesting inequalities for functions. Example 2.3.25 (Convex functions with Lipschitz continuous Fréchet derivative). Assume that X = H is a Hilbert space, identified with its dual, and that φ : H → ℝ∪{∞} is a proper convex function which is Fréchet differentiable with Lipschitz continuous Fréchet derivative of Lipschitz constant L. Then, for any x, z in an open convex set, L ‖z − x‖2 , 2 1 󵄩 󵄩2 (ii) φ(z) − φ(x) ≤ ⟨Dφ(z), z − x⟩ − 󵄩󵄩󵄩Dφ(z) − Dφ(x)󵄩󵄩󵄩 , 2L 1󵄩 󵄩2 (iii) ⟨Dφ(z) − Dφ(x), z − x⟩ ≥ 󵄩󵄩󵄩Dφ(x) − Dφ(z)󵄩󵄩󵄩 (co-coercivity). L (i) φ(z) ≤ φ(x) + ⟨Dφ(x), z − x⟩ +

(2.14)

The first of these properties follows by Theorem 2.3.22, and the observation that 1

φ(z) − φ(x) = ∫⟨z − x, Dφ(x + t(z − x))⟩dt, 0

86 | 2 Differentiability, convexity and optimization in Banach spaces so that 1

φ(z) − φ(x) − ⟨z − x, Dφ(x)⟩ = ∫⟨z − x, Dφ(x + t(z − x)) − Dφ(x))⟩dt ≤ 0

L ‖z − x‖2 , 2

where we used the Lipschitz property of Dφ. The second follows by expressing φ(z) − φ(x) = φ(z) − φ(x󸀠 ) + φ(x󸀠 ) − φ(x) for any x󸀠 ∈ H, and then applying Theorem 2.3.22(i) for φ(z) − φ(x󸀠 ), the first property for φ(x󸀠 ) − φ(x) and minimizing the upper bound over x󸀠 . In particular, φ(z) − φ(x) = φ(z) − φ(x󸀠 ) + φ(x󸀠 ) − φ(x) ≥ ⟨Dφ(z), z − x󸀠 ⟩ + ⟨Dφ(x), x󸀠 − x⟩ +

L 󵄩󵄩 󸀠 󵄩2 󵄩x − x󵄩󵄩󵄩 , 2󵄩

and the minimum of the upper bound is obtained at x󸀠 = x − L1 (Dφ(x) − Dφ(z)), and upon substitution we obtain the stated result. The co-coercivity property follows by using the second, interchanging the role of x and z and adding. The details are left as an exercise. ◁ Example 2.3.26 (The Polyak–Lojasiewicz inequality). If φ : H → ℝ ∪ {+∞} is Fréchet differentiable and strongly convex with modulus of convexity c > 0, then, if xo is a minimizer, we have the Polyak–Lojasiewicz condition 󵄩󵄩 󵄩2 󵄩󵄩Dφ(x)󵄩󵄩󵄩 ≥ 2c(φ(x) − φ(xo )),

∀ x ∈ H.

By simply modifying the proof of 2.3.22(i), using the definition of strong convexity (see Definition 2.3.3), and appropriately passing to the limit taking into account the extra quadratic term, we find that for any x1 , x2 ∈ C, c φ(x2 ) ≥ φ(x1 ) + ⟨Dφ(x1 ), x2 − x1 ⟩ + ‖x2 − x1 ‖2 , 2

∀ x1 , x2 ∈ C.

We rearrange this inequality as 󵄩󵄩2 1 󵄩󵄩󵄩 1 1 󵄩 Dφ(x1 ) − Dφ(x1 )󵄩󵄩󵄩 φ(x2 ) ≥ φ(x1 ) + ⟨Dφ(x1 ), x2 − x1 ⟩ + 󵄩󵄩󵄩√c(x2 − x1 ) + 󵄩󵄩 √c √c 2 󵄩󵄩 2 󵄩 󵄩 󵄩󵄩 1 1󵄩 1󵄩 󵄩2 1 󵄩󵄩 󵄩2 Dφ(x1 )󵄩󵄩󵄩 ≥ φ(x1 ) − 󵄩󵄩󵄩Dφ(x1 )󵄩󵄩󵄩 , = φ(x1 ) − 󵄩󵄩󵄩Dφ(x1 )󵄩󵄩󵄩 + 󵄩󵄩󵄩√c(x2 − x1 ) + 󵄩󵄩 √c 2c 2 󵄩󵄩 2c 󵄩 󵄩2 which rearranged provides a lower bound for 2c1 󵄩󵄩󵄩Dφ(x1 )󵄩󵄩󵄩 . In order to get the sharpest bound, we may choose the value of x2 that minimizes φ, i. e., xo , and for this choice we obtain the stated inequality. This inequality finds important applications in optimization algorithms, where often continuity of Dφ is required. ◁ We will see in Chapter 4 that many of the above properties generalize for nonsmooth convex functions in terms of the subdifferential.

2.4 Optimization and convexity | 87

2.4 Optimization and convexity The assumption of convexity allows us to deduce a number of important results regarding minimization problems (see, e. g., [19] or [88]). Proposition 2.4.1. Let X be a reflexive Banach space and φ : X → ℝ ∪ {+∞} a convex proper and lower semicontinuous function. (i) Let C be a bounded, convex and closed subset of X. Then φ attains its minimum on C. (ii) If C ⊂ X is convex and closed and φ is coercive on C then it attains its minimum on C. We may allow C to be a closed subspace of X or even X itself. Proof. This result is immediate by Theorems 2.2.5 and 2.2.8 and the facts that a closed convex set is also weakly closed (Proposition 1.2.12) and that a convex lower semicontinuous function is also weakly semicontinuous (Proposition 2.3.13). Remark 2.4.2. A convex lower semicontinuous functional is not necessarily weak⋆ lower semicontinuous so Proposition 2.4.1 is not necessarily true if X is nonreflexive. Apart from simplifying existence results, convexity leads to some interesting qualitative properties as far as minimization is concerned. Proposition 2.4.3. (i) A local minimum for a convex functional defined on a convex set C is a global minimum. (ii) If a functional is strictly convex then its minimum is unique. Proof. (i) Assume that x ∈ C is a local minimum, i. e. φ(xo ) ≤ φ(x) for every x ∈ V := B(xo , ϵ)∩C where ϵ > 0 is sufficiently small. For any z ∈ C take the convex combination (1 − t)xo + t z = xo + t(z − xo ), t ∈ [0, 1]. For small enough values of t, xo + t(z − xo ) ∈ V and since xo is a local minimum φ(xo ) ≤ φ(xo + t(z − xo )), and by convexity of φ it follows that φ(xo + t(z − xo )) ≤ (1 − t)φ(xo ) + t φ(z) = φ(xo ) + t(φ(z) − φ(xo )). Combining the above we obtain for all positive and small enough t that φ(xo ) ≤ φ(xo ) + t (φ(z) − φ(xo )), or equivalently that φ(xo ) ≤ φ(z) for every z ∈ C ⊂ X, therefore, xo is a global minimum. (ii) Let xo,1 , xo,2 ∈ C be two global minima of φ such that xo,1 ≠ xo,2 . Consider the point xo = 21 xo,1 + 21 xo,2 ∈ C. By strict convexity of φ it follows that φ(xo ) < φ(xo,1 ) = φ(xo,2 ), which leads to contradiction.

88 | 2 Differentiability, convexity and optimization in Banach spaces Example 2.4.4 (Maximum principle for convex functions). Consider a convex function φ defined on a convex subset C ⊂ X. If φ attains a global maximum at an interior point of C then φ is constant. Assume that φ is not constant but it attains a global maximum at xo ∈ int( C). Choose x ∈ C such that φ(x) < φ(xo ) and t ∈ (0, 1) such that z = xo + t(xo − x) ∈ C (this 1 t is possible since xo is an interior point of C). That implies that xo = 1+t z + 1+t x and by the convexity of φ and the fact that xo is a global maximum we conclude that φ(xo ) ≤

t 1 t 1 φ(z) + φ(x) < φ(xo ) + φ(xo ) = φ(xo ), 1+t 1+t 1+t 1+t

which is a contradiction.

◁

Theorem 2.4.5. Let C ⊂ X, convex and φ : C ⊂ X → ℝ Gâteaux differentiable and convex. Then xo ∈ C is a minimum if and only if ⟨Dφ(xo ), x − xo ⟩ ≥ 0,

∀ x ∈ C.

If C = X the first-order condition reduces to Dφ(xo ) = 0. Proof. Assume xo ∈ C is a minimum. Then, φ(xo ) ≤ φ(z), for every z ∈ C. For any x ∈ C, set z = (1 − t)xo + t x = xo + t (x − xo ) ∈ C for t ∈ (0, 1), and apply this inequality to obtain φ(xo ) ≤ φ(xo + t (x − xo )),

∀ x ∈ C, t > 0,

which yields φ(xo + t (x − xo )) − φ(xo ) ≥ 0, t

t > 0,

and going to the limit as t → 0, ⟨Dφ(xo ), x − xo ⟩ ≥ 0,

∀ x ∈ C.

For the converse, since φ is convex and Gâteaux differentiable by (2.10), we have that φ(x) − φ(xo ) ≥ ⟨Dφ(xo ), x − xo ⟩,

∀ xo , x ∈ C.

Since for xo ∈ C it holds that ⟨Dφ(xo ), x − xo ⟩ ≥ 0, ∀ x ∈ C, we find that φ(xo ) ≤ φ(x) for all x ∈ C so that xo is a minimum. The first-order condition now takes the form of an inequality rather than an equation. Inequalities of this form are called variational inequalities and will be studied in detail in Chapter 7.

2.5 Projections in Hilbert spaces | 89

Example 2.4.6 (The gradient descent method). Let φ : H → ℝ ∪ {+∞} be a proper convex function on a Hilbert space with Lipschitz continuous Fréchet derivative of Lipschitz constant L. Consider the iterative scheme xn+1 = xn − αDφ(xn ), called a gradient descent scheme. If this iterative scheme has a fixed point, i. e., if for some xo it holds that xo = xo − αDφ(xo ) then xo is a minimizer for φ. Moreover, if α = L1 and φ satisfies an inequality ‖Dφ(x)‖2 ≥ 2c(φ(x) − φ(xo )) of the Polyak–Lojasiewicz type for every x ∈ H, we have that φ(xn ) − φ(xo ) ≤ (1 −

n−1

c ) L

(φ(x1 ) − φ(xo )).

It is easy to see that a fixed point xo of the gradient descent scheme (if it exists) is a critical point Dφ(xo ) = 0 hence, by convexity xo is a minimizer. To show the convergence estimate, using inequality (2.14)(i) for xn+1 , xn and taking into account the iteration scheme to express xn+1 − xn , we see that φ(xn+1 ) − φ(xn ) ≤ −(α − 2

Lα2 󵄩󵄩 󵄩2 )󵄩Dφ(xn )󵄩󵄩󵄩 , 2 󵄩

and choosing α so that (α − Lα2 ) > 0, e. g. α = Lojasiewicz inequality to obtain the estimate φ(xn+1 ) − φ(xo ) ≤ (1 −

1 , L

we may further use the Polyak–

c )(φ(xn ) − φ(xo )), L

and the result follows by iteration. Note that we did not use the strong convexity assumption anywhere, but relied only on the Polyak–Lojasiewicz inequality which may also hold for nonstrongly convex functions. An interesting observation is that the step size is inversely proportional to the Lipschitz constant. One may try to use varying step sizes or even choose the step size by an appropriate optimization procedure. One way to allow for unconditional convergence would be to use the explicit version of the gradient scheme, i. e., calculating Dφ in xn+1 rather than xn . This simple numerical scheme will be revisited in its various variants later on in this book to cover for cases where, e. g., smoothness is absent. ◁

2.5 Projections in Hilbert spaces In this section, X = H is a Hilbert space, with inner product ⟨⋅, ⋅⟩ and norm ‖ ⋅ ‖, such that ‖x‖2 = ⟨x, x⟩ for every x ∈ X.

90 | 2 Differentiability, convexity and optimization in Banach spaces Definition 2.5.1. Let C ⊂ H be a convex and closed set and x ∈ H. The operator PC : H → C, defined by PC (x) := xo , where xo is the solution of the problem infz∈C ‖x − z‖ is called the projection operator on C. The projection operator enjoys the following properties, summarized in the following theorem. Theorem 2.5.2 (Projection theorem and properties of projection operators). Let H be a Hilbert space and C ⊂ X be closed and convex. Then: (i) For any x ∈ H the minimization problem minz∈C ‖x−z‖ has a unique solution, xo ∈ C, characterized by the solution of the inequality ⟨x − xo , z − xo ⟩ ≤ 0,

∀ z ∈ C.

(2.15)

(ii) The projection operator PC : H → C is single valued and nonexpansive, i. e., 󵄩󵄩 󵄩 󵄩󵄩PC (x1 ) − PC (x2 )󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖,

∀ x1 , x2 ∈ H.

Proof. (i) The proof is a straightforward application of Theorem 2.4.5, upon noting that the minimization problem defining the projection operator is equivalent to the minimization problem minz∈C ‖x − z‖2 and recalling Example 2.1.3. (ii) The single valuedness of PC follows from the uniqueness of the projection. By the properties of the projection of x1 on C we have that ⟨x1 − PC (x1 ), z − PC (x1 )⟩ ≤ 0,

∀ z ∈ C.

Choose z = PC (x2 ) ∈ C to obtain the inequality ⟨x1 − PC (x1 ), PC (x2 ) − PC (x1 )⟩ ≤ 0.

(2.16)

By the properties of the projection of x2 on C we have that ⟨x2 − PC (x2 ), z − PC (x2 )⟩ ≤ 0,

∀ z ∈ C.

Choose z = PC (x1 ) ∈ C to obtain the inequality ⟨x2 − PC (x2 ), PC (x1 ) − PC (x2 )⟩ ≤ 0. Adding (2.16) and (2.17) yields ⟨x1 − x2 − (PC (x1 ) − PC (x2 )), PC (x1 ) − PC (x2 )⟩ ≥ 0 which gives 󵄩 󵄩2 ⟨x1 − x2 , PC (x1 ) − PC (x2 )⟩ ≥ 󵄩󵄩󵄩PC (x1 ) − PC (x2 )󵄩󵄩󵄩 and an application of the Cauchy–Schwarz inequality yields the desired result.

(2.17)

2.5 Projections in Hilbert spaces | 91

Remark 2.5.3 (The projection operator if X is not identified with X ⋆ ). The form of the variational inequality (2.15) characterizing the projection operator is tacitly assuming that the Hilbert space X is identified with its dual space X ⋆ , so that the inner product (⋅, ⋅) in X is identified with the duality pairing ⟨⋅, ⋅⟩ between X ⋆ and X. In the general case where X is not identified with X ⋆ , we need to modify our arguments, by taking into account that ‖x‖2 = (x, x). This leads to a reformulation of the variational inequality (2.15) as (x − xo , z − xo ) ≤ 0 for all z ∈ C, which using the duality map J : X → X ⋆ can be restated as ⟨J−1 (x − xo ), z − xo ⟩ ≤ 0 for all z ∈ C. The special case where C = E a closed linear subspace of X is also of some importance. Definition 2.5.4. Let E ⊂ H be a closed linear subspace. By E ⊥ , we denote the orthogonal complement of E, E ⊥ := {z ∈ H : ⟨x, z⟩ = 0, ∀ z ∈ E}. Proposition 2.5.5 (The projection operator on a closed subspace). Let E be a linear closed subspace. The projection operator PE : H → E has the following properties: (i) It is a linear nonexpansive operator, i. e., ‖PE x‖ ≤ ‖x‖ for every x ∈ H, satisfying the condition ⟨x − PE x, z⟩ = 0 for all z ∈ E. (ii) For every x ∈ H there exists a unique decomposition x = xo + z where xo = PE x ∈ E and z = (I − PE )x = PE ⊥ x ∈ E ⊥ . (iii) PE is an idempotent operator, i. e., PE PE = PE with the properties N(PE ) = E ⊥ and PE (H) = E. Proof. (i) The projection of x on E, xo , satisfies the inequality ⟨x − xo , z󸀠 − xo ⟩ ≤ 0, for all z󸀠 ∈ E. Take any z ∈ E and first choose z󸀠 = xo + z ∈ E (by the linearity of E) which yields ⟨x − xo , z⟩ ≤ 0, and then choose z󸀠 = xo − z ∈ E (by the linearity of E) which yields ⟨x − xo , z⟩ ≥ 0. Therefore, ⟨x − xo , z⟩ = 0 for every z ∈ E. Consider the linear combination x = λ1 x1 + λ2 x2 ∈ H and the projection xo := PE x = PE (λ1 x1 + λ2 x2 ). Since E is a closed linear subspace, the projection PE x is the unique element of E that satisfies ⟨x − xo , z⟩ = 0,

∀ z ∈ E.

Now, let us calculate ⟨x − (λ1 PE x1 + λ2 PE x2 ), z⟩ = ⟨λ1 x1 + λ2 x2 − (λ1 PE x1 + λ2 PE x2 ), z⟩

= λ1 ⟨x1 − PE x1 , z⟩ + λ2 ⟨x2 − PE x2 , z⟩ = 0.

By the fact that E is a closed linear subspace, λ1 PE x1 + λ2 PE x2 ∈ E and since it satisfies the above equality for any z ∈ E we have (by the uniqueness of projections) that

92 | 2 Differentiability, convexity and optimization in Banach spaces PE x = λ1 PE x1 + λ2 PE x2 , therefore, we have linearity. The nonexpansiveness follows by Proposition 2.5.2. (ii) Let any x ∈ H. The projection xo = PE x satisfies ⟨x − xo , z⟩ = 0 for all z ∈ E, therefore, zo := x − xo ∈ E ⊥ . Clearly, x = xo + zo . It remains to identify zo as PE ⊥ . We express xo as xo = −(zo − x) ∈ E, so that ⟨xo , z⟩ = 0, for all z ∈ E ⊥ and therefore ⟨zo − x, z⟩ = 0 for all z ∈ E ⊥ . But that implies that zo = PE ⊥ x. (iii) The statements follow by the decomposition in (ii). The generalization of the concept of projection in Banach spaces is more limited for a number of reasons, among which is the possible nondifferentiability of the norm (see Example 4.6.4).

2.6 Geometric properties of Banach spaces related to convexity In this section, we consider certain important characterizations of Banach spaces related to convexity (see, e. g., [45, 35] or [75]).

2.6.1 Strictly, uniformly and locally uniformly convex Banach spaces 2.6.1.1 Strict convexity Definition 2.6.1. A Banach space X is called strictly convex if its unit ball is strictly convex, i. e., if for all x1 , x2 ∈ X, x1 ≠ x2 , ‖x1 ‖ = ‖x2 ‖ = 1 it holds that ‖tx1 + (1 − t)x2 ‖ < 1, for all t ∈ (0, 1). One may see that yet another equivalent way of defining strict convexity is using the midpoint, i. e., stating the definition for t = 21 only, a condition which is easier to check. The equivalence is straightforward by the observation that if C is a convex set and x ∈ C, z ∈ int(C) then tx + (1 − t)z ∈ int(C) for every t ∈ (0, 1) (see Proposition 1.2.4). The geometric intuition behind strictly convex Banach spaces is that if X is strictly convex, then the boundary of the unit ball does not contain any line segments. This is understood in the sense that geometrically for any x1 , x2 ∈ SX (0, 1), the elements of the form tx1 + (1 − t)x2 , t ∈ (0, 1), can be considered as line segments, and strict convexity implies that such elements are not on SX (0, 1). Example 2.6.2 (Strict convexity depends on the choice of norm). Whether a Banach space is strictly convex or not depends on the choice of norm. For instance, X = ℝd is strictly convex with the norms ‖x‖p = (∑di=1 |xi |p )1/p for p ∈ (1, ∞) but not for p = 1 or p = ∞ (to convince yourself try sketching the unit ball in ℝ2 under these norms). ◁ Example 2.6.3 (Strict convexity and extreme points). Another way to state strict convexity is using the concept of extreme points. For a convex set C, a point x ∈ C is called an extreme point of C, if whenever x = tx1 + (1 − t)x2 , for some t ∈ (0, 1), and

2.6 Geometric properties of Banach spaces related to convexity | 93

x1 , x2 ∈ C, then x = x1 = x2 . Taking C = BX (0, 1) = {x ∈ X : ‖x‖ ≤ 1}, the closed unit ball of X, we see that if X is strictly convex then the extreme points of BX (0, 1) are on SX (0, 1) = {x ∈ X : ‖x‖X = 1}, the unit sphere of X. First of all, we need to show that BX (0, 1) does have extreme points. Note that any x ∈ BX (0, 1) such that ‖x‖ < 1 cannot be an extreme point of BX (0, 1). To see this, 1 observe that since ‖x‖ < 1, there exists ϵ > 0 such that ‖x‖ < 1+ϵ therefore x1 := (1 + ϵ)x ∈ BX (0, 1). Clearly, x2 := (1 − ϵ)x ∈ BX (0, 1) with x1 ≠ x2 and since x = 21 x1 + 21 x2 , we see that such an x is not an extreme point of BX (0, 1). Consider next x ∈ SX (0, 1). Suppose there exist x1 , x2 ∈ BX (0, 1) such that x1 ≠ x2 and x = tx1 + (1 − t)x2 , t ∈ (0, 1). Note that it is impossible that ‖x1 ‖ < 1 and ‖x2 ‖ < 1, so that it must be ‖x1 ‖ = ‖x2 ‖ = 1. But then it is impossible that x = tx1 + (1 − t)x2 , t ∈ (0, 1) by strict convexity. So any ◁ point x ∈ SX (0, 1) is an extreme point of BX (0, 1). Example 2.6.4. Any Hilbert space H is strictly convex, as can be seen by the Cauchy– Schwarz inequality. The Banach spaces X = Lp (𝒟) for 1 < p < ∞ are strictly convex Banach spaces. The spaces L1 (𝒟) and L∞ (𝒟) are not strictly convex. ◁ The following characterization of strict convexity is often useful (see, e. g., [35]). Theorem 2.6.5. Let X be a Banach space. The following are equivalent: (i) X is strictly convex. (i󸀠 ) ‖x1 + x2 ‖ = ‖x1 ‖ + ‖x2 ‖ for x1 , x2 ≠ 0 if and only if x1 = λx2 for some λ > 0. 2 (ii) For x1 , x2 ∈ X such that x1 ≠ x2 and ||x1 || = ||x2 || = 1 we have that ‖ x1 +x ‖ < 1. 2 (iii) For 1 < p < ∞, and x1 , x2 ∈ X such that x1 ≠ x2 we have that 󵄩󵄩 x + x 󵄩󵄩p ‖x ‖p + ‖x ‖p 󵄩󵄩 1 2󵄩 2 󵄩󵄩 < 1 . 󵄩󵄩 󵄩󵄩 2 󵄩󵄩󵄩 2 (iv) The problem sup{⟨x⋆ , x⟩ : ‖x‖ ≤ 1} achieves a unique solution for any10 x⋆ ∈ X ⋆ , x⋆ ≠ 0. (v) The function φ(x) = ‖x‖2 is strictly convex. Proof. We will assume the equivalence of (i) and (i󸀠 ) and leave the proof last. That (i) implies (ii) is immediate from the definition of strict convexity. To show that (ii) implies (i) (equiv. (i󸀠 )) argue by contradiction; suppose that (ii) holds but X is not strictly convex, so that there exist x1 , x2 ∈ X, (‖x1 ‖ ≤ ‖x2 ‖), nonparallel with ‖x1 + x2 ‖ = ‖x1 ‖ + ‖x2 ‖. Define zi = ‖x1 ‖ xi , i = 1, 2, such that ‖zi ‖ = 1, and z = ‖x1 ‖ x2 . Then i 1 ‖z1 + z2 ‖ = ‖z1 + z − z + z2 ‖ ≥ ‖z1 + z‖ − ‖z2 − z‖ = 2, since ‖x1 + x2 ‖ = ‖x1 ‖ + ‖x2 ‖. But this contradicts (ii). 10 This condition is often phrased as: Each support hyperplane for the unit ball, supports it at only one point (e. g., [82]). Recall that, an x⋆ ∈ X ⋆ , x⋆ ≠ 0, is called a support functional for A ⊂ X at point xo ∈ X ⋆ if ⟨x⋆ , xo ⟩ = sup{⟨x⋆ , x⟩ : x ∈ A}. The point xo is called a support point for A, while the hyperplane Hx⋆ ,⟨x⋆ ,xo ⟩ = {x ∈ X : ⟨x⋆ , x⟩ = ⟨x⋆ , xo ⟩} is called the support hyperplane for A.

94 | 2 Differentiability, convexity and optimization in Banach spaces That (i) implies (iii) follows by the properties of the real function t 󳨃→ |t|p , 1 < p < ∞, applied to t = 21 (‖x1 ‖+‖x2 ‖). Indeed, since X is strictly convex ‖x1 +x2 ‖ < ‖x1 ‖+‖x2 ‖ so that ( 21 (‖x1 +x2 ‖))p < ( 21 ‖x1 ‖+ 21 ‖x2 ‖)p and the result follows. To show that (iii) implies (i) consider x1 , x2 such that ‖x1 ‖ = ‖x2 ‖ = 1. Then (iii) implies (ii) which in turn implies (i). To show that (i) implies (iv) assume on the contrary that there exist x1 ≠ x2 , ‖x1 ‖ = ‖x2 ‖ = 1, such that ⟨x⋆ , x1 ⟩ = ⟨x⋆ , x2 ⟩ = sup{⟨x⋆ , x⟩ : ‖x‖ ≤ 1} = ‖x⋆ ‖X ⋆ . Then ⟨x⋆ , x1 + x2 ⟩ = ⟨x⋆ , x1 ⟩ + ⟨x⋆ , x2 ⟩ = 2‖x⋆ ‖X ⋆ . On the other hand, ⟨x⋆ , x1 + x2 ⟩ ≤ |⟨x⋆ , x1 + x2 ⟩| ≤ ‖x⋆ ‖X ⋆ ‖x1 +x2 ‖, so combining that with the previous inequality (and dividing by ‖x⋆ ‖X ⋆ ) yields that 2 ≤ ‖x1 + x2 ‖. But by the strict convexity of X, it follows that ‖x1 + x2 ‖ < ‖x1 ‖ + ‖x2 ‖ = 2, which yields a contradiction. For the converse, assume that (iv) holds but (ii) not; there exist x1 , x2 ∈ X with ‖x1 ‖ = ‖x2 ‖ = 1 such that ‖x1 + x2 ‖ = 2. Then ‖ 21 (x1 + x2 )‖ = 1, and by the Hahn–Banach theorem (see Example 1.1.17) there exists x⋆ ∈ X ⋆ , such that ⟨x⋆ , 21 (x1 +x2 )⟩ = ‖ 21 (x1 +x2 )‖ = 1, from which it follows that ⟨x⋆ , x1 ⟩ + ⟨x⋆ , x2 ⟩ = 2. However, since ⟨x⋆ , xi ⟩ ≤ 1, i = 1, 2, the previous equality implies that ⟨x⋆ , x1 ⟩ = ⟨x⋆ , x2 ⟩ = ‖x⋆ ‖ = 1, which contradicts (iv). We have thus shown that (iv) implies (ii), which is equivalent to (i). To show that (i) implies (v) suppose that φ is not strictly convex, i. e., there exists x1 , x2 ∈ X, x1 ≠ x2 such that ‖to x1 + (1 − to )x2 ‖2 = to ‖x1 ‖2 + (1 − to )‖x2 ‖2 for some to ∈ (0, 1). On the other hand, using the triangle inequality, we have that ‖to x1 + (1 − to )x2 ‖ ≤ to ‖x1 ‖ + (1 − to )‖x2 ‖, so squaring we have that 󵄩󵄩 󵄩2 󵄩󵄩to x1 + (1 − to )x2 󵄩󵄩󵄩 ≤ t02 ‖x1 ‖2 + 2to (1 − to )‖x1 ‖ ‖x2 ‖ + (1 − to )2 ‖x2 ‖2

≤ to2 ‖x1 ‖2 + to (1 − to )‖x1 ‖2 + to (1 − to )‖x2 ‖2 + (1 − to )2 ‖x2 ‖2 = to ‖x1 ‖2 + (1 − to )‖x2 ‖2 ,

using the elementary inequality 2‖x1 ‖ ‖x2 ‖ ≤ ‖x1 ‖2 + ‖x2 ‖2 . Comparing the outmost left and the outmost right parts of the above inequality which are equal, we conclude that ‖x1 ‖2 − 2‖x1 ‖ ‖x2 ‖ + ‖x2 ‖2 = 0 hence, ‖x1 ‖ = ‖x2 ‖. Defining zi = ‖x1 ‖ xi , such that ‖zi ‖ = 1, i i = 1, 2, we see that ‖to z1 + (1 − to )z2 ‖ = 1, which is a contradiction with (i). To show that (v) implies (i), assume that φ is strictly convex but X is not. Then there exist x1 , x2 ∈ X, x1 ≠ x2 , ‖x1 ‖ = ‖x2 ‖ = 1, and to ∈ (0, 1) such that ‖to x1 + (1 − to )x2 ‖ = 1, which implies that ‖to x1 + (1 − to )x2 ‖2 = 1 = to ‖x1 ‖2 + (1 − to )‖x2 ‖2 , which contradicts (v). We complete the proof by showing that (i) implies (i󸀠 ). We have already shown that (i) implies (ii). If x1 , x2 are such that ‖x1 + x2 ‖ = ‖x1 ‖ + ‖x2 ‖, with ‖x2 ‖ ≥ ‖x1 ‖, x1 , x2 ≠ 0, then by the reverse triangle inequality 󵄩󵄩 󵄩󵄩 1 󵄩󵄩 󵄩󵄩 1 󵄩󵄩 󵄩󵄩 1 1 1 1 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩󵄩 x1 + x2 󵄩󵄩󵄩 ≥ 󵄩󵄩󵄩 x1 + x2 󵄩󵄩󵄩 − 󵄩󵄩󵄩 x2 − x2 󵄩󵄩󵄩 󵄩󵄩 󵄩󵄩 ‖x1 ‖ ‖x2 ‖ 󵄩󵄩 󵄩󵄩 ‖x1 ‖ ‖x1 ‖ 󵄩󵄩 󵄩󵄩 ‖x1 ‖ ‖x2 ‖ 󵄩󵄩 ‖x ‖ − ‖x1 ‖ ‖x ‖ + ‖x2 ‖ ‖x2 ‖ − ‖x1 ‖ 1 = ‖x + x2 ‖ − 2 ‖x ‖ = 1 − = 2, ‖x1 ‖ 1 ‖x1 ‖ ‖x2 ‖ 2 ‖x1 ‖ ‖x1 ‖

2.6 Geometric properties of Banach spaces related to convexity | 95

so that by (ii) it follows that

1 x ‖x1 ‖ 1 󸀠

=

1 x; ‖x2 ‖ 2

therefore, there exists λ > 0 for which

x1 = λx2 and (i ) follows. That (i ) implies (i) follows by contradiction. Assume x1 ≠ x2 such that ‖tx1 + (1 − t)x2 ‖ = 1 for some t ∈ (0, 1). Then defining z1 = tx1 , z2 = (1 − t)x2 , and using (i󸀠 ) there exists λ > 0 such that z1 = λz2 . Taking the norm on both sides we x2 we conclude that x1 = x2 have that t = λ(1 − t), and rearranging z1 = λz2 as x1 = λ(1−t) t which is a contradiction. The proof is complete. 󸀠

Remark 2.6.6. One may in fact replace (v) by (v󸀠 ): the function φ(x) = ‖x‖p , p ∈ (1, ∞) is strictly convex. Indeed, assume not, so that there exists x1 ≠ x2 and to ∈ (0, 1) such that ‖to x1 + (1 − to )x2 ‖p = to ‖x1 ‖p + (1 − to )‖x2 ‖p . The triangle inequality and the monotonicity and convexity of the real valued function s 󳨃→ sp , p ∈ (1, ∞) yields that p 󵄩󵄩 󵄩p p p 󵄩󵄩to x1 + (1 − to )x2 󵄩󵄩󵄩 ≤ (to ‖x1 ‖ + (1 − to )‖x2 ‖) ≤ to ‖x1 ‖ + (1 − to )‖x2 ‖ ,

and since by assumption the left-hand side equals the right-hand side of this inequal‖x1 ‖ , ity we see that (to ‖x1 ‖ + (1 − to )‖x2 ‖)p = to ‖x1 ‖p + (1 − to )‖x2 ‖p so that defining s = ‖x ‖ 2

we have that (to s + (1 − to ))p − to sp − (1 − to ) = 0. By elementary calculus,11 we see that the only solution is s = 1; hence, ‖x1 ‖ = ‖x2 ‖ and our assumption implies that ‖to x1 + (1 − to )x2 ‖p = ‖x1 ‖p = ‖x2 ‖p , and proceeding as in the proof above defining zi = ‖x1 ‖ xi , such that ‖zi ‖ = 1, i = 1, 2, we see that ‖to z1 + (1 − to )z2 ‖ = 1, which is in i contradiction with (i). 2.6.1.2 Uniform convexity Definition 2.6.7. A Banach space X is called uniformly convex if for each ϵ ∈ (0, 2] 2 there exists δ(ϵ) > 0 such that ‖ x1 +x ‖ ≤ 1 − δ for ‖x1 ‖ ≤ 1, ‖x2 ‖ ≤ 1, ‖x1 − x2 ‖ ≥ ϵ. 2 One may immediately see from the definition that if X is uniformly convex then it is also strictly convex; however, the converse is not necessarily true. Example 2.6.8 (Uniform convexity depends on the choice of norm). The space X = ℝd is not uniformly convex when endowed with the ‖ ⋅ ‖1 or ‖ ⋅ ‖∞ norm (see also Example 2.6.2). ◁ Example 2.6.9 (Hilbert spaces are uniformly convex). If X = H a Hilbert space then it is uniformly convex. This follows directly by the parallelogram identity for Hilbert 2 spaces, from which it can be shown that for given ϵ > 0, δ(ϵ) = 1 − √1 − ϵ . ◁ 4

p

Example 2.6.10. The Lebesgue spaces X = L (𝒟) for 1 < p < ∞ are uniformly convex Banach spaces (Clarkson’s theorem see, e. g., [35] for a proof). The spaces L1 (𝒟) and L∞ (𝒟) are not uniformly convex. The same result holds for any Lebesgue space on a more general measure μ. Similarly, for the Sobolev spaces W 1,p (𝒟) and W01,p (𝒟), which are uniformly convex for 1 < p < ∞. ◁ 11 The function ψ(s) = (to s + (1 − to ))p − to sp − (1 − to ) has a single maximum at s = 1 and ψ(0) = 0.

96 | 2 Differentiability, convexity and optimization in Banach spaces An alternative definition of uniform convexity is given by the following proposition which is very useful in a number of applications. Proposition 2.6.11. A Banach space is uniformly convex if and only if for any sequences {xn : n ∈ ℕ} ⊂ X and {zn : n ∈ ℕ} ⊂ X such that ‖xn ‖ ≤ 1 and ‖zn ‖ ≤ 1 the property x +z ‖ n 2 n ‖ → 1 implies ‖xn − zn ‖ → 0. x +z

Proof. Assume that X is uniformly convex and that ‖ n 2 n ‖ → 1. Suppose that ‖xn − zn ‖ ↛ 0. This means that there exists ϵ > 0 and N ∈ ℕ such that ‖xn − zn ‖ ≥ ϵ for n > N. But the uniform convexity of X implies that there exists δ = δ(ϵ) such that x +z x +z ‖ n 2 n ‖ < 1 − δ, which contradicts the assumption ‖ n 2 n ‖ → 1. To prove the converse, suppose that for any sequences {xn : n ∈ ℕ}, {zn : n ∈ x +z ℕ} inside the unit ball of X, such that ‖ n 2 n ‖ → 1 it holds that ‖xn − zn ‖ → 0. That x +z implies that there exists an N ∈ ℕ such that for all n > N, 1 − δ < ‖ n 2 n ‖ < 1 + δ and −ϵ < ‖xn − zn ‖ < ϵ. That implies that for all ϵ > 0, there exists a δ = δ(ϵ) such that x +z ‖xn − zn ‖ ≥ ϵ implies ‖ n 2 n ‖ ≤ 1 − δ, which is the uniform convexity. Example 2.6.12 (A criterion for Cauchy sequences in uniformly convex spaces). If X is uniformly convex, a sequence {xn : n ∈ ℕ} ⊂ X with the properties that ‖xn ‖ ≤ 1 and ‖ 21 (xn + xm )‖ → 1 as n, m → ∞ is a Cauchy sequence. Simply apply Proposition 2.6.11 for {xn : n ∈ ℕ} and {x̄n : n ∈ ℕ} chosen so that x̄n = xm . ◁ Example 2.6.13 (In uniformly convex spaces weak convergence and convergence of norms imply strong convergence). Let X be uniformly convex and consider a sequence {xn : n ∈ ℕ} such that xn ⇀ x in X and ‖xn ‖ → ‖x‖ as n → ∞. Then xn → x. This property is often called the Radon–Riesz property. 1 We define zn = ‖x1 ‖ , z = ‖x‖ x, which are on SX (0, 1). Consider any x⋆ ∈ X ⋆ , such that n

‖x⋆ ‖X ⋆ = 1 and ⟨x⋆ , z⟩ = 1 (this is always possible). Then ⟨x⋆ , 21 (zn +zm )⟩ ≤ ‖ 21 (zn +zm )‖ ≤ 1, while by weak convergence ⟨x⋆ , zn ⟩ → ⟨x⋆ , z⟩, so that the above inequality implies that ‖ 21 (zn + zm )‖ → 1 as n, m → ∞. Then, by Example 2.6.12 we have that {zn : n ∈ ℕ} is Cauchy; hence, zn → z (since we already have that zn ⇀ z). ◁ Uniform convexity is related to reflexivity, as the following theorem shows. Theorem 2.6.14 (Milman, Pettis). A uniformly convex Banach space X is reflexive.

Proof. We only sketch the proof here (see, e. g., [35] or [82] for a complete proof). By Goldstine’s theorem, BX (0, 1) is weak⋆ dense in BX ⋆⋆ (0, 1), i. e., for each x⋆⋆ ∈ SX ⋆⋆ , ⋆ there exists a net {xα : α ∈ ℐ } ⊂ BX (0, 1), such that jxα ⇀ x⋆⋆ . We consider the net {j( 21 (xα + xβ )) : (α, β) ∈ ℐ × ℐ }, for which we may show that ‖ 21 (xα + xβ )‖ → 1, so that by an argument based on Example 2.6.12, appropriately modified for nets, we conclude that the net {xα : α ∈ ℐ } ⊂ BX (0, 1) is Cauchy; hence, converging to a limit xo , for which jxα → jxo . This implies the existence of a xo ∈ X for which x⋆⋆ = jxo , therefore, X is reflexive.

2.6 Geometric properties of Banach spaces related to convexity | 97

This theorem is very useful in proving the reflexivity of classic Banach spaces such as the Lebesgue spaces Lp , p > 1 (see, e. g., [28]). 2.6.1.3 Local uniform convexity The concept of uniformly convex Banach spaces has been weakened in [81], where the concept of local uniform convexity has been introduced. Definition 2.6.15 (Local uniform convexity). A Banach space X is called locally uniformly convex if for any ϵ > 0 and any x ∈ X with ‖x‖ = 1, there exists δ(ϵ, x) > 0 such that for all z with ‖z‖ ≤ 1 such that ‖z − x‖ ≥ ϵ it holds that ‖ x+z ‖ ≤ 1 − δ(ϵ, x). 2 As can be seen from the definition, a uniformly convex Banach space is also locally uniformly convex. Summarizing, Uniform convexity 󳨐⇒ Local Uniform convexity 󳨐⇒ Strict convexity.

(2.18)

It can also be proved, using similar arguments, that Proposition 2.6.11 and the assertion of Example 2.6.13 holds for locally uniformly convex Banach spaces. As mentioned above, properties such as strict convexity and uniform convexity of a Banach space, depend on the choice of norm. An important result is that, using an appropriate renorming, one may turn a Banach space either into a strictly convex or into a locally uniformly convex space. An important result of this type has been proved in [107]. Theorem 2.6.16 (Troyanski). Let X be a reflexive Banach space. There exists a locally uniform convex equivalent norm ‖⋅‖1 such that X and X ⋆ (equipped with the corresponding dual norm) are locally uniformly convex. Proof. The proof is outside the scope of the present book. We refer the interested reader to [107], or Theorems 2.9 and 2.10 in [45]. Geometrical properties of Banach spaces related to convexity such as strict or uniform convexity, have deep links with properties such as the differentiability of the norm (see Chapter 4), or the properties of the duality map. 2.6.2 Convexity and the duality map Let X be a Banach space, X ⋆ its dual, ⟨⋅, ⋅⟩ the duality pairing between them, and J : ⋆ X → 2X defined by 󵄩 󵄩 󵄩 󵄩 J(x) = {x⋆ ∈ X ⋆ : ⟨x⋆ , x⟩ = ‖x‖ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ , ‖x‖ = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ }, the duality map (see Definition 1.1.16). This map plays a basic role in the study of the geometry of Banach spaces and in nonlinear functional analysis [45]. In the next theorem, we give some important properties of J.

98 | 2 Differentiability, convexity and optimization in Banach spaces Theorem 2.6.17. Let X be a reflexive Banach space and let X ⋆ be strictly convex. Then, (i) J is (a) single valued, (b) demicontinuous, i. e., for every xn → x in X it holds that12 ‖J(x)‖ ⋆ J(xn ) ⇀ J(x), (c) coercive, i. e., ‖x‖X → ∞ as ‖x‖ → ∞, (d) monotone, i. e., ⟨J(x1 ) − J(x2 ), x1 − x2 ⟩ ≥ 0 for every x1 , x2 ∈ X, and (e) surjective. (ii) If X is strictly convex, then J is bijective and strictly monotone, i. e., ⟨J(x1 ) − J(x2 ), x1 − x2 ⟩ = 0,

implies x1 = x2 ,

Moreover, if ⟨J(xn ) − J(x), xn − x⟩ → 0,

as n → ∞

(2.19)

then ‖xn ‖ → ‖x‖ and xn ⇀ x in X. If in addition, X is locally uniformly convex, then (2.19) implies that xn → x as n → ∞. (iii) If X ⋆ is locally uniformly convex, then J is continuous. Proof. (i) We first show that J is single valued. Let x⋆i ∈ J(x), i = 1, 2. By the definition of the duality map, we have 󵄩 󵄩2 ⟨x⋆i , x⟩ = ‖x‖2 = 󵄩󵄩󵄩x⋆i 󵄩󵄩󵄩X ⋆ ,

i = 1, 2,

so that ‖x⋆1 ‖ = ‖x⋆2 ‖, and without loss of generality we assume that ‖x⋆1 ‖X ⋆ = ‖x⋆2 ‖X ⋆ = 1. Hence, 󵄩 󵄩 󵄩 󵄩2 󵄩 󵄩2 󵄩 󵄩 2 󵄩󵄩󵄩x⋆1 󵄩󵄩󵄩X ⋆ ‖x‖ ≤ 󵄩󵄩󵄩x⋆1 󵄩󵄩󵄩X ⋆ + 󵄩󵄩󵄩x⋆2 󵄩󵄩󵄩X ⋆ = ⟨x⋆1 + x⋆2 , x⟩ ≤ 󵄩󵄩󵄩x⋆1 + x⋆2 󵄩󵄩󵄩X ⋆ ‖x‖,

∀x ∈ X.

This implies that 1󵄩 󵄩 󵄩 󵄩 1 = 󵄩󵄩󵄩x⋆1 󵄩󵄩󵄩X ⋆ ≤ 󵄩󵄩󵄩x⋆1 + x⋆2 󵄩󵄩󵄩X ⋆ , 2 which by the strict convexity of X ⋆ implies that x⋆1 = x⋆2 (see Theorem 2.6.5(ii)) so that J is single valued. We now prove the demicontinuity of J. To this end, consider a sequence {xn : n ∈ ℕ} ⊂ X such that xn → x in X as n → ∞. We will show that J(xn ) ⇀ J(x) in X ⋆ . Observe that by the definition of the duality map and the properties of the sequence {xn : n ∈ ℕ}, we have that 󵄩󵄩 󵄩 󵄩󵄩J(xn )󵄩󵄩󵄩X ⋆ = ‖xn ‖ → ‖x‖,

as n → ∞.

(2.20)

Since {J(xn ) : n ∈ ℕ} is bounded and X ⋆ is reflexive, there exists a subsequence, denoted the same for simplicity, and an element x⋆ ∈ X ⋆ such that J(xn ) ⇀ x⋆ ,

in X ⋆

as n → ∞.

12 If X were not reflexive, demicontinuity would require J(xn ) ⇀ J(x); see Definition 9.2.8. ⋆

2.6 Geometric properties of Banach spaces related to convexity | 99

By (2.20), and using the weak lower semicontinuity of the norm (see Proposition 1.1.53) we obtain 󵄩 󵄩󵄩 󵄩 󵄩 󵄩󵄩 ⋆ 󵄩󵄩 󵄩J(xn )󵄩󵄩󵄩X ⋆ = ‖x‖. 󵄩󵄩x 󵄩󵄩X ⋆ ≤ limninf󵄩󵄩󵄩J(xn )󵄩󵄩󵄩X ⋆ = lim n 󵄩

(2.21)

Since J(xn ) ⇀ x⋆ and xn → x, we have that ⟨J(xn ), xn ⟩ → ⟨x⋆ , x⟩, so that from ‖xn ‖2 = ⟨J(xn ), xn ⟩ → ⟨x⋆ , x⟩, and (2.20) we have, by the uniqueness of limits, that ‖x‖2 = ⟨x⋆ , x⟩.

(2.22)

This implies that 󵄨 󵄨 󵄩 󵄩 ‖x‖2 = ⟨x⋆ , x⟩ = 󵄨󵄨󵄨⟨x⋆ , x⟩󵄨󵄨󵄨 ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖x‖, from which, dividing through by ‖x‖, follows that (2.23)

󵄩 󵄩 ‖x‖ ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ,

therefore, by combining (2.21) and (2.23), ‖x‖ = ‖x⋆ ‖. Since we also have (by (2.22)) that ⟨x⋆ , x⟩ = ‖x‖2 , we conclude by the definition of the duality map that J(x) = x⋆ . Therefore, for the chosen subsequence we have shown that J(xn ) ⇀ J(x). By the Urysohn property (see Remark 1.1.51), it finally follows that the entire sequence {J(xn ) : n ∈ ℕ} converges weakly to x⋆ = J(x); hence, the demicontinuity of J is established. Since by definition ⟨J(x), x⟩ = ‖x‖2 coercivity of J is immediate. The monotonicity of J follows by (2.25) below. We defer the proof of surjectivity to Section 9.3.4. (ii) We now assume further that X is strictly convex and we prove that J is strictly monotone. Let ⟨J(x1 ) − J(x2 ), x1 − x2 ⟩ = 0.

(2.24)

By the definition of the duality map, ⟨J(x1 ) − J(x2 ), x1 − x2 ⟩ = ‖x1 ‖2 − ⟨J(x2 ), x1 ⟩ − ⟨J(x1 ), x2 ⟩ + ‖x2 ‖2 2

≥ (‖x1 ‖ − ‖x2 ‖) ,

∀ x1 , x2 ∈ X,

so, we express (2.24) as x1 + x2 x1 − x2 x + x2 x − x2 ), ⟩ + ⟨J( 1 ) − J(x2 ), 1 ⟩ 2 2 2 2 x + x2 x + x2 x + x2 x + x2 = ⟨J(x1 ) − J( 1 ), x1 − 1 ⟩ + ⟨J( 1 ) − J(x2 ), 1 − x2 ⟩ 2 2 2 2

0 = ⟨J(x1 ) − J(

(2.25)

100 | 2 Differentiability, convexity and optimization in Banach spaces 2 󵄩󵄩 x + x 󵄩󵄩 2 󵄩󵄩 󵄩󵄩 󵄩 2󵄩 󵄩󵄩) + (󵄩󵄩󵄩 x1 + x2 󵄩󵄩󵄩 − ‖x2 ‖) , ≥ (‖x1 ‖ − 󵄩󵄩󵄩 1 󵄩󵄩 2 󵄩󵄩 󵄩󵄩 2 󵄩󵄩󵄩 󵄩 󵄩

where in the last estimate we have used (2.25). Hence, 󵄩󵄩 x + x 󵄩󵄩 󵄩 2󵄩 󵄩󵄩 = ‖x2 ‖. ‖x1 ‖ = 󵄩󵄩󵄩 1 󵄩󵄩 2 󵄩󵄩󵄩 Since X is strictly convex, x1 = x2 . Therefore, J is strictly monotone. Since J is strictly monotone and surjective, it is also bijective. Now, consider a sequence {xn : n ∈ ℕ} ⊂ X such that ⟨J(xn ) − J(x), xn − x⟩ → 0,

as n → ∞.

We will show that xn ⇀ x in X. We rewrite ⟨J(xn ) − J(x), xn − x⟩ as ⟨J(xn ) − J(x), xn − x⟩

= ‖xn ‖2 − ⟨J(xn ), x⟩ − ⟨J(x), xn ⟩ + ‖x‖2

(2.26)

2

= (‖xn ‖ − ‖x‖) + (‖xn ‖ ‖x‖ − ⟨J(xn ), x⟩) + (‖xn ‖ ‖x‖ − ⟨J(x), xn ⟩), and observe that since |⟨J(xn ), x⟩| ≤ ‖J(xn )‖ ‖x‖ = ‖xn ‖ ‖x‖ and |⟨J(x), xn ⟩| ≤ ‖J(x)‖ ‖xn ‖ = ‖xn ‖ ‖x‖ the second and third term on the right-hand side of (2.26) are nonnegative. Therefore, the right-hand side of (2.26) as a whole is nonnegative and since by assumption the left-hand side tends to 0 as n → ∞ we must have ‖xn ‖ → ‖x‖,

⟨J(xn ), x⟩ → ‖x‖2 ,

⟨J(x), xn ⟩ → ‖x‖2

and

as n → ∞.

Since X is reflexive, by the boundedness of the sequence {xn : n ∈ ℕ}, we may assume the existence of a subsequence (denoted the same for simplicity) and an element z ∈ X such that xn ⇀ z in X and, therefore, ⟨J(x), xn ⟩ → ⟨J(x), z⟩,

as n → ∞.

It then follows by the uniqueness of the limit that ⟨J(x), z⟩ = ‖x‖2 , so that since ‖x‖2 = ⟨J(x), z⟩ = |⟨J(x), z⟩| ≤ ‖x‖ ‖z‖, upon dividing both sides with ‖x‖, we obtain that ‖x‖ ≤ ‖z‖.

(2.27)

On the other hand, by the weak lower semicontinuity of the norm (see Proposition 1.1.53) we obtain ‖z‖ ≤ lim inf ‖xn ‖ = lim ‖xn ‖ = ‖x‖. n

n

(2.28)

2.7 Appendix | 101

Combining (2.27) with (2.28), if follows that ‖x‖ = ‖z‖, therefore, by the strict convexity of X we deduce that x = z. Therefore, for the chosen subsequence, we have that xn ⇀ x. By the Urysohn property, we may deduce convergence for the whole sequence. Finally, let (2.19) hold and assume furthermore that X is locally uniformly convex. Since (2.19) implies that ‖xn ‖ → ‖x‖ and xn ⇀ x in X, it follows from the local uniform convexity of X that xn → x as n → ∞ (see Example 2.6.13). (iii) Let X ⋆ be locally uniformly convex. We show that J is continuous. Consider a sequence {xn : n ∈ ℕ} ⊂ X such that xn → x as n → ∞. Then ‖xn ‖ → ‖x‖. By the definition of the duality map ‖J(xn )‖ = ‖xn ‖ for any n ∈ ℕ and ‖J(x)‖ = ‖x‖, therefore, ‖J(xn )‖ → ‖J(x)‖. By (i), J is demicontinuous, so J(xn ) ⇀ J(x) in X ⋆ . It then follows by the local uniform convexity of X ⋆ (see Example 2.6.13) that J(xn ) → J(x) as n → ∞.

2.7 Appendix 2.7.1 Proof of Proposition 2.3.20 Suppose that |φ| < c, on B(xo , 2δ) ⊂ int( dom φ). Consider x1 , x2 ∈ B(xo , δ), x1 ≠ x2 and δ (x2 − x1 ), where clearly z ∈ B(xo , 2δ). We express x2 in terms of x1 and set z = x2 + ‖x −x 1 2‖ z in terms of the convex combination x2 =

‖x1 − x2 ‖ δ z+ x, ‖x1 − x2 ‖ + δ ‖x1 − x2 ‖ + δ 1

and using the convexity of φ we have that φ(x2 ) ≤

‖x1 − x2 ‖ δ φ(z) + φ(x1 ). ‖x1 − x2 ‖ + δ ‖x1 − x2 ‖ + δ

Subtracting φ(x1 ) from both sides, we have φ(x2 ) − φ(x1 ) ≤

2c‖x1 − x2 ‖ ‖x1 − x2 ‖ 2c (φ(z) − φ(x1 )) ≤ ≤ ‖x1 − x2 ‖, ‖x1 − x2 ‖ + δ ‖x1 − x2 ‖ + δ δ

(2.29)

where we used the fact that z, x1 ∈ B(xo , 2δ) for that both |φ(z)| < c, |φ(x1 )| < c, and then the obvious fact that ‖x1 −x2 ‖+δ > δ. Reversing the role of x1 and x2 and combining the result with (2.29) we deduce that |φ(x2 ) − φ(x1 )| ≤ 2c ‖x1 − x2 ‖, which is the required δ Lipschitz continuity property. To show that it suffices that φ is locally bounded above at xo , note that if φ < c on B(xo , δ) ⊂ int( dom φ) then for any z ∈ B(xo , δ) we may define 2x − z ∈ B(xo , δ), and by convexity (upon expressing x = 21 z + 21 (2x − z)) 1 1 1 φ(x) ≤ φ(z) + φ(2x − z) ≤ (φ(z) + c), 2 2 2

which upon rearrangement yields that φ(z) ≥ 2φ(x) − c; hence, φ is bounded below at z. Since z was arbitrary, we conclude that if φ is locally bounded above then it is also locally bounded below; hence, locally bounded.

102 | 2 Differentiability, convexity and optimization in Banach spaces 2.7.2 Proof of Proposition 2.3.21 Proof. We first show that φ is locally Lipschitz continuous at any x ∈ core(dom φ). Suppose that x ∈ core(dom φ). Then, setting Lλ φ := {x ∈ X : φ(x) ≤ λ}, for any λ ∈ ℝ, X = ⋃ n(dom φ − x) = ⋃ n ⋃ (Lk φ − x) = ⋃ n(Lk φ − x). n∈ℕ

n∈ℕ

k∈ℤ

(n,k)

(2.30)

Since φ is lower semicontinuous, the sets n(Lk φ−x) are closed, by Baire’s theorem (see Theorem 2.7.1(i)) there exists (n0 , k0 ) ∈ ℕ × ℤ such that int( n0 (Lk0 φ − x)) ≠ 0. Hence, x ∈ int( Lk0 φ), therefore, φ is bounded above by k0 on an open neighborhood of x and by Proposition 2.3.20 we conclude that φ is locally Lipschitz continuous at x. We now show that int( dom φ) = core(dom φ). Since for any convex set C ⊂ X it holds that int( C) ⊂ core(C) (see Section 1.2.1), it is always true that int( dom φ) ⊂ core(dom φ). Hence, it suffices to show that core(dom φ) ⊂ int( dom φ). Consider any x ∈ core(dom φ). Then by (2.30) and essentially by the same arguments we can show that x ∈ int( dom φ). Hence, core(dom φ) = int( dom φ), and since we have already proved that φ is continuous in core(dom φ) it is also continuous in int( dom φ). An alternative approach to prove that φ is continuous in int( dom φ) is to consider any point x ∈ int( dom φ) and then for any z ∈ X define the function φz : ℝ → ℝ by φz (t) := φ(t(z − x)). This is a convex function. Also since x ∈ int( dom φ) it is easy to see that 0 ∈ int( domφz ), so, by Proposition 2.3.20, φz is continuous at t = 0. That implies for any ϵ > 0 there exists δ > 0 such that |φz (t) − φz (0)| ≤ ϵ for |t| < δ hence, φ(t(z − x)) < φ(0) + ϵ for |t| < δ. Choosing λ > φ(0) we see by this argument that there exists t > 0 such that tz ∈ Lλ φ + x or equivalently z ∈ 1t (Lλ φ + x), with t in general depending on z. Since z ∈ X is arbitrary, repeating the above procedure for every z ∈ X, we conclude that X = ⋃t>0 1t (Lλ φ + x) = ⋃n n(Lλ φ + x) and using once more the closedness of Lλ φ and the Baire category theorem we conclude continuity of φ at x using Proposition 2.3.20. If X ⊂ ℝd , then it is easy to see that int( dom φ) = core(dom φ). Indeed, let e = (ei ), i = 1, . . . , d be the standard basis of ℝd and consider any x ∈ core(dom φ). Then, for every i = 1, . . . , d there exists ϵi > 0 such that zi = x + δi ϵi ∈ dom φ for every δi ∈ [0, ϵi ], i = 1, . . . , d. Take ϵ = min{ϵi : i = 1, . . . , d}. Then z = x + δe ∈ dom φ for every δ ∈ (0, ϵ) hence, B(x, ϵ) ⊂ dom φ and x ∈ int( dom φ). Hence, core(dom φ) ⊂ int( dom φ) and therefore core(dom φ) = int( dom φ). To show that φ is continuous at int( dom φ) = core(dom φ), we can show almost verbatim as in the infinite dimensional case that φ is continuous in coredom φ and then deduce the continuity at int( dom φ) from their equality. Alternatively, take any x ∈ int( dom φ), pick a δ > 0 so that x + δei ∈ dom φ for every i = 1, . . . , d and then note that the convex hull conv{x + δe1 , . . . , x + δed } is a neighborhood of x. By the convexity of φ for any z ∈ conv{x + δe1 , . . . , x + δed } we have that d

d

i=1

i=1

φ(z) = φ(∑ λi (x + δei )) ≤ ∑ λi φ(x + δei ) ≤ max{φ(x + δei : i = 1, . . . , d}.

2.7 Appendix | 103

Hence, there exists a neighborhood of x in which φ is bounded above by max{φ(x + δei ) : i = 1, . . . , d}, so we may conclude continuity of φ at x using Proposition 2.3.20(i). Theorem 2.7.1 (Baire). Let (X, d) be a complete metric space. (i) If {𝒞n }n∈ℕ is a sequence of closed sets such that ⋃∞ n=1 𝒞n = X, there exists n0 ∈ ℕ such that13 int( 𝒞n0 ) ≠ 0. (ii) If {𝒪n }n∈N is a sequence of open dense subsets of X, then ⋂n 𝒪n ≠ 0 (countable intersections of open dense sets are nonempty).

13 At least one of these closed sets has nonempty interior, i. e., there exists an x ∈ 𝒞n0 and an ϵ > 0 such that BX (x, ϵ) ⊂ 𝒞n0 .

3 Fixed-point theorems and their applications A very important part of nonlinear analysis is fixed-point theory. This theory provides answers to the question of whether a map from a (subset) of a Banach space (or sometimes a complete metric space) to itself admits points which are left invariant under the action of the map and is an indispensable toolbox in a variety of fields, finding many important applications ranging from their use in abstract existence results to the construction of numerical schemes. In this chapter, we present some of the most common fixed-point theorems starting from Banach’s contraction mapping principle which guarantees existence of fixed points for strict contractions and then gradually start removing hypotheses on the map, working our way toward more general fixed-point theorems. In this short sojourn in fixed-point theory, we present the Brower fixed-point theorem (a deep topological finite dimensional result which nevertheless forms the basis for many infinite dimensional fixed-point theorems). Then the Schauder fixed-point theorem (which only assumes continuity of the map and convexity and compactness of the set it acts on) and its important extension, the Leray– Schauder alternative, and then move to Browder’s fixed-point theorem for nonexpansive maps and the related Mann–Krasnoselskii iterative scheme for approximation of fixed points for such maps. We then provide a useful fixed-point theorem for multivalued maps and close the chapter with a very general fixed-point theorem due to Caristi which is equivalent to the extremely useful Ekeland variational principle. All fixedpoint theorems presented are illustrated with extensive examples from differential and integral equations, PDEs and optimization. There are various excellent textbooks and monographs dedicated solely to fixed-point theorems (see, e. g., [2] or [110]).

3.1 Banach fixed-point theorem The Banach fixed-point theorem is a fundamental result in analysis with various important applications [2, 41].

3.1.1 The Banach fixed-point theorem and generalizations Definition 3.1.1. Let (X, d) be a metric space and1 A ⊂ X. The map f : A → X is said to be a contraction if there exists ϱ ∈ (0, 1) such that d(f (x1 ), f (x2 )) ≤ ϱ d(x1 , x2 ),

1 It is possible that A = X. https://doi.org/10.1515/9783110647389-003

∀ x1 , x2 ∈ A.

106 | 3 Fixed-point theorems and their applications Theorem 3.1.2 (Banach). Let A ⊂ X be a nonempty, closed subset of a complete metric space X and f : A → A a contraction. Then f has a unique fixed point xo . Moreover, if x1 is an arbitrary point in A and {xn : n ∈ ℕ}, a sequence defined by xn+1 = f (xn ),

n ∈ ℕ,

(3.1)

then, limn→∞ xn = xo and d(xn , xo ) ≤

ϱn−1 d(x1 , x2 ). 1−ϱ

(3.2)

Proof. Let x ∈ A. Then, from (3.1) we have d(xn , xn+1 ) = d(f (xn−1 ), f (xn )) ≤ ϱ d(xn−1 , xn ),

n > 1,

which upon iteration yields d(xn , xn+1 ) ≤ ϱn−1 d(x1 , x2 ). It then follows that d(xn , xn+m ) ≤ d(xn , xn+1 ) + d(xn+1 , xn+2 ) + ⋅ ⋅ ⋅ + d(xn+m−1 , xn+m ) ≤ (ϱn−1 + ϱn + ⋅ ⋅ ⋅ + ϱn+m−2 ) d(x1 , x2 )

= ϱn−1 (1 + ϱ + ⋅ ⋅ ⋅ + ϱm−1 ) d(x1 , x2 ) ≤

ϱn−1 d(x1 , x2 ). 1−ϱ

Hence, {xn : n ∈ ℕ} is a Cauchy sequence and since X is complete, there exists xo ∈ X such that xn → xo . Furthermore, since A is closed, xo ∈ A. Letting m → ∞, we obtain the estimate (3.2). We now show the fixed-point property of xo . We have d(xo , f (xo )) ≤ d(x, xn ) + d(xn , f (xo )) = d(xo , xn ) + d(f (xn−1 ), f (xo )) ≤ d(xo , xn ) + ϱd(xn−1 , xo ).

For n → ∞, we obtain that d(xo , f (xo )) = 0, i. e., f (xo ) = xo . Finally, uniqueness follows from the observation that if there exist two fixed points xo,1 , xo,2 , then d(xo,1 , xo,2 ) = d(f (xo,1 ), f (xo,2 )) ≤ ϱ d(xo,1 , xo,2 ) and the fact that ϱ ∈ (0, 1). Remark 3.1.3. All assumptions of the theorem are important. As a simple example, consider f : ℝ → ℝ defined by f (x) = π2 + x − Arctanx, for every x ∈ ℝ. Then f 󸀠 (x) = 1 1 − 1+x 2 < 1 for every x ∈ ℝ, so that from the mean value theorem, for any x1 , x2 ∈ ℝ, there exists ξ ∈ (x1 , x2 ) such that ‖f (x1 ) − f (x2 )‖ = ‖f 󸀠 (ξ )‖ ‖x1 − x2 ‖ ≤ ‖x1 − x2 ‖. However, f has no fixed point.

3.1 Banach fixed-point theorem | 107

The previous remark motivates the following extension of the Banach fixed-point theorem which is due to Edelstein. Theorem 3.1.4 (Edelstein). Let (X, d) be a compact metric space and consider f : X → X such that d(f (x1 ), f (x2 )) < d(x1 , x2 ), for every x1 , x2 ∈ X, with x1 ≠ x2 . Then f has a unique fixed point in X. Proof. The existence of a fixed point follows by the existence of a minimum for the function x 󳨃→ d(x, f (x)), which coincides with a solution of x = f (x), i. e., a fixed point for f . The Banach fixed-point theorem is one of the fundamental fixed-point theorems in analysis and finds applications in practically all fields of mathematics. We briefly review here some of its important applications in nonlinear analysis.

3.1.2 Solvability of differential equations Theorem 3.1.5 (Cauchy, Lipschitz, Picard). Let X a Banach space and fo : X → X a Lipcshitz map, i. e., a map such that there exists L ≥ 0, for which ‖fo (x1 ) − fo (x2 )‖ ≤ L ‖x1 − x2 ‖, for every x1 , x2 ∈ X. Then for each x0 ∈ X the problem x󸀠 (t) = fo (x(t)),

t ∈ [0, ∞),

x(0) = x0

(3.3)

has a unique solution x ∈ C 1 ([0, ∞), X). Proof. The problem (3.3) is equivalent to the following: Find x ∈ C([0, ∞), X) such that t x(t) = x0 + ∫0 fo (x(s))ds, where the integral is understood in the Bochner sense (see Section 1.6). For ϱ > 0, we consider the space E = {x ∈ C([0, ∞), X) : supt≥0 e−ϱt ‖x(t)‖ < ∞}. It is easy to verify the following properties: (i) E endowed with the norm ‖x‖E = supt≥0 e−ϱt ‖x(t)‖ is a Banach space. t

(ii) Define the mapping f : E → X by (f (x))(t) = x0 + ∫0 fo (x(s))ds for any x ∈ E. Then f : E → E. (iii) It holds that ‖f (x1 ) − f (x2 )‖E ≤ Lϱ ‖x1 − x2 ‖E for every x1 , x2 ∈ E.

Therefore, for ϱ > L the mapping f : E → E is a contraction and by the Banach fixedpoint theorem it has a fixed point, which is a solution of (3.3). Concerning the uniqueness, let x1 and x2 be two solutions of problem (3.3). Setting t p(t) := ‖x1 (t) − x2 (t)‖, we have, using the Lipschitz property that p(t) ≤ L ∫0 p(s)ds, for t ≥ 0, and by a standard application of Gronwall’s inequality (see Section 3.7.1) we have that p(t) = 0 for every t > 0.

108 | 3 Fixed-point theorems and their applications 3.1.3 Nonlinear integral equations We consider the nonlinear integral equation b

x(t) = λ ∫ K(t, s, x(s))ds + fo (t),

(3.4)

a

where the unknown is a continuous function x : [a, b] → ℝ and fo : [a, b] → ℝ is a known continuous function. Using the Banach fixed-point theorem, we can show that under certain conditions (3.4) admits a unique solution. Theorem 3.1.6. Suppose that fo : [a, b] → ℝ is continuous, K : [a, b] × [a, b] × [−R, R] → ℝ is continuous, satisfies in its domain the Lipschitz condition |K(t, s, x1 ) − K(t, s, x2 )| ≤ L |x1 − x2 | where L does not depend on t, s, x1 , x2 , and letting M = maxt,s,x |K(t, s, x)| suppose that 󵄨 󵄨 R max 󵄨󵄨󵄨fo (t)󵄨󵄨󵄨 ≤ , 2 t∈[a,b]

and,

|λ| < min{

R 1 , }. 2M(b − a) L(b − a)

(3.5)

Then the integral equation (3.4) admits a unique continuous solution. Proof. Consider X = C([a, b]), with elements x = x(⋅) : [a, b] → ℝ, continuous. This is a Banach space endowed with the norm ‖x‖ = maxt∈[a,b] |x(t)|, and let A = BX (0, R), the closed ball of C([a, b]) of radius R. We prove that the mapping f : X → X defined by b

(f (x))(t) = λ ∫ K(t, s, x(s))ds + fo (t),

∀ t ∈ [a, b],

a

maps BX (0, R) into BX (0, R) for a choice of R as above. Let x ∈ BX (0, R), so that |x(t)| ≤ R. We have 󵄨󵄨 b 󵄨󵄨 󵄨󵄨 󵄨󵄨 d(0, f (x)) = max 󵄨󵄨󵄨λ ∫ K(t, s, x(s))ds + fo (t)󵄨󵄨󵄨 󵄨󵄨 t∈[a,b]󵄨󵄨 󵄨 a 󵄨 󵄨󵄨 b 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨 󵄨 ≤ max 󵄨󵄨󵄨λ ∫ K(t, s, x(s))ds󵄨󵄨󵄨 + max 󵄨󵄨󵄨fo (t)󵄨󵄨󵄨 󵄨󵄨 t∈[a,b] t∈[a,b]󵄨󵄨 󵄨 a 󵄨 R R(b − a)M R ≤ |λ| (b − a) M + ≤ + = R, 2 2M(b − a) 2 i. e., f (x) ∈ BX (0, R). Now, from the Lipschitz condition on K we have 󵄨󵄨 b 󵄨󵄨 󵄨󵄨 󵄨󵄨 d(f (x1 ), f (x2 )) = max 󵄨󵄨󵄨λ ∫[K(t, s, x1 (s)) − K(t, s, x2 (s))]ds󵄨󵄨󵄨 󵄨 󵄨󵄨 t∈[a,b]󵄨 󵄨 a 󵄨

3.1 Banach fixed-point theorem |

109

b

󵄨 󵄨 󵄨 󵄨 ≤ |λ| L ∫󵄨󵄨󵄨x1 (s) − x2 (s)󵄨󵄨󵄨ds ≤ |λ| L (b − a) max 󵄨󵄨󵄨x1 (s) − x2 (s)󵄨󵄨󵄨 s∈[a,b] a

= ϱ d(x1 , x2 ), where, because of (3.5), ϱ = |λ| L (b − a)
0 there exists an δ1 > 0 such that if ‖x̂‖X < δ1 then ‖Dgy (x̂)‖X ⋆ ≤ ϵ1 . We fix a ϱ < 1, set ϵ1 = ϱ and then select the corresponding δ1 > 0 in the above. Then, applying the mean value theorem (see Proposition 2.1.5) to g0 we have that for any x̂1 , x̂2 ∈ BX (δ1 ) it holds that 󵄩󵄩 ̂ 󵄩 󵄩󵄩g0 (x1 ) − g0 (x̂2 )󵄩󵄩󵄩X ≤ ϱ‖x̂1 − x̂2 ‖X ,

(3.6)

so that ĝy : BX (δ1 ) → X is a contraction for every ŷ for the same contraction constant (because ĝy = g0 +A−1 ŷ). If for particular values of ŷ we show that ĝy : BX (δ1 ) → BX (δ1 ), then we may apply the Banach fixed-point theorem and we are done. It therefore remains to find the proper values of y = ŷ +yo for which gy maps the ball BX (δ1 ) into itself. Setting x̂1 = x̂ and x̂2 = 0 in (3.6) and since g0 (0) = 0 (because f (xo ) = yo ), we see that for any x̂ ∈ BX (δ1 ) we have that ‖g0 (x)‖X ≤ ϱ‖x̂‖X by (3.6). Noting that ĝy = g0 (x̂) + A−1 ŷ, by the triangle inequality and the above observation we have that 󵄩󵄩 󵄩 󵄩 −1 󵄩 󵄩 −1 󵄩 󵄩󵄩gy (x)󵄩󵄩󵄩X ≤ ϱ‖x‖X + 󵄩󵄩󵄩A ŷ󵄩󵄩󵄩X ≤ ϱδ1 + 󵄩󵄩󵄩A 󵄩󵄩󵄩ℒ(Y,X) ‖ŷ‖Y . Choosing ŷ ∈ BY (δ2 ) for δ2 :=

1 δ (1 ‖A−1 ‖ℒ(Y,X) 1

− ϱ), we see that ‖ĝy (x̂)‖X ≤ δ1 . Hence for

ŷ ∈ BY (δ2 ) the mappings gy (x) : BX (δ1 ) → BX (δ1 ) are contractions so that by the Banach contraction theorem gy has a fixed point which corresponds to the solution of f (x) = y for y ∈ BY (yo , δ2 ). That implies that f −1 : BY (yo , δ2 ) → BX (xo , δ1 ) is well-defined. It remains to prove that f −1 is C 1 . We first show that f −1 is Lipschitz. Consider yi ∈ BY (yo , δ2 ), i = 1, 2 and note that if xi = f −1 (yi ) then by using the local coordinates x̂i = xi − xo and ŷi = yi − yo as before these are fixed points of ĝyi , so that x̂i = ĝyi (x̂i ) = x̂i − A−1 (f (x̂i + xo ) − (ŷi + yo )), i = 1, 2. Starting from the fixed-point equation for x2 = x̂2 + xo , and adding and subtracting A−1 y1 = A−1 (ŷ1 + y0 ) to the right-hand side of this fixed-point equation we note that x2 = xo + ĝy1 − A−1 (y1 − y2 ), and subtracting the fixed-point equation x1 = xo + ĝy1 (x̂1 ) we have that 󵄩 󵄩 ‖x1 − x2 ‖X = 󵄩󵄩󵄩ĝy1 (x̂1 ) − ĝy1 (x̂2 ) + A−1 (y1 − y2 )󵄩󵄩󵄩X 󵄩 󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩ĝy1 (x̂1 ) − ĝy1 (x̂2 )󵄩󵄩󵄩X + 󵄩󵄩󵄩A−1 (y1 − y2 )󵄩󵄩󵄩X 3 as we will frequently do in the future, but here we choose not to.

3.1 Banach fixed-point theorem

| 111

󵄩 󵄩 ≤ ϱ‖x1 − x2 ‖X + 󵄩󵄩󵄩A−1 󵄩󵄩󵄩ℒ(Y,X) ‖y1 − y2 ‖Y where we used (3.6) and the facts that x̂1 − x̂2 = x1 − x2 and A has bounded inverse. This can be rearranged as ‖A−1 ‖ℒ(Y,X) 󵄩 󵄩󵄩 −1 −1 ‖y1 − y2 ‖Y =: L‖y1 − y2 ‖Y , 󵄩󵄩f (y1 ) − f (y2 )󵄩󵄩󵄩X := ‖x1 − x2 ‖X ≤ 1−ϱ which is the Lipschitz continuity for the inverse function. For the differentiability of f −1 observe that from the differentiability of f we have that for every ϵ > 0 there exists δ > 0 such that for ‖x − xo ‖ < δ it holds that ‖f (x) − f (xo ) − Df (xo )(x − xo )‖Y < ϵ‖x − xo ‖X . We use again the simplified notation A = Df (xo ), and the observation that yo = f (xo ) implies that xo = f −1 (yo ) (resp., y = f (x) implies x = f −1 (y)) to rewrite the above as 󵄩󵄩 󵄩 󵄩 −1 󵄩 −1 −1 −1 󵄩󵄩y − yo − A(f (y) − f (yo ))󵄩󵄩󵄩Y ≤ ϵ󵄩󵄩󵄩f (y) − f (yo )󵄩󵄩󵄩X ≤ ϵL‖y − yo ‖Y ,

(3.7)

where for the last estimate we used the Lipschitz property of f −1 . Recalling that A has bounded inverse by assumption, we now note that f −1 (y) − f −1 (yo )) − A−1 (y − yo ) = −A−1 [y − yo − A(f −1 (y) − f −1 (yo ))], so that 󵄩󵄩 −1 󵄩 −1 −1 󵄩󵄩f (y) − f (yo )) − A (y − yo )󵄩󵄩󵄩X 󵄩 󵄩 = 󵄩󵄩󵄩−A−1 [y − yo − A(f −1 (y) − f −1 (yo ))]󵄩󵄩󵄩X 󵄩 󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩A−1 󵄩󵄩󵄩ℒ(Y,X) 󵄩󵄩󵄩y − yo − A(f −1 (y) − f −1 (yo ))󵄩󵄩󵄩Y ≤ ϵ‖y − yo ‖Y , where we used the fact that A−1 is bounded and (3.7). This last inequality implies the differentiability of f −1 at yo with Fréchet derivative A−1 = (Df (xo ))−1 . Upon employing a continuity argument for the invertibility of Df (x) in a neighborhood of xo and repeating the above steps we can prove the stated result. We may now provide a theorem concerning solvability of operator equations of a more complex form than f (x) = y, in the sense that we now consider the solvability with respect to y of functions of the form f (x, y) = 0, where x ∈ X is considered as a parameter. Clearly, one could ask the relevant question of whether we could solve f (x, y) = 0 for x as a function of the parameter y. We will see that the answer depends on the differentiability properties of f with respect to either one of the variables. This is the content of the implicit function theorem of Hildenbrandt and Graves ([69] see also [41] or [110]).

112 | 3 Fixed-point theorems and their applications Theorem 3.1.8 (Implicit function theorem). Let X, Y, Z Banach spaces and U ⊂ X × Y be an open set. Let the function f : U ⊂ X × Y → Z be continuous in U and Fréchet differentiable with respect to the variable y, with continuous partial Fréchet derivative Dy f ∈ C(U, ℒ(Y, Z)). Suppose that there exists (xo , yo ) ∈ U such that f (xo , yo ) = 0, and that the partial Fréchet derivative Dy f is invertible, with bounded inverse at (xo , yo ), i. e., (Dy f (xo , yo ))−1 ∈ ℒ(Z, Y). Then: (i) There exist r1 , r2 > 0 and a unique continuous function ψ : BX (xo , r1 ) → BY (yo , r2 ) such that BX (xo , r1 ) × BY (yo , r2 ) ⊂ U with the properties ψ(xo ) = yo and f (x, ψ(x)) = 0 for every x ∈ BX (xo , r1 ). (ii) If furthermore f ∈ C 1 (U, Z), then ψ ∈ C 1 (BX (xo , r1 ), Y) and Dψ(x) = −(Dy f (x, ψ(x)))−1 ∘ Dx (x, ψ(x)) for every x ∈ BX (xo , r1 ). Proof. We will work in terms of local variables (x̂, ŷ) again around the point (xo , yo ), use the notation A := Dy f (xo , yo ), which by assumption is bounded and invertible and note that upon defining the mapping ŷ 󳨃→ ĝx (ŷ) = ŷ − A−1 f (x̂ + xo , ŷ + yo ), for suitable choice of x̂, a fixed point ŷ of ĝx will correspond to a solution (x, y) of f (x, y) = 0 near (xo , yo ). In terms of these local variables, U corresponds to an open set U0 of (0, 0). For every x̂, it holds that D̂y ĝx (ŷ) = I − A−1 Dy f (x̂ + xo , ŷ + yo ), which is continuous as a mapping from U0 to ℒ(Y, Z). (i) We can see that for every x̂ the mappings ŷ 󳨃→ ĝx (ŷ) are contractions. Indeed, working similarly as in the proof of Theorem 3.1.7 we have that 1

ĝx (ŷ1 ) − ĝx (ŷ2 ) = ∫ D̂y ĝx (t ŷ1 + (1 − t)ŷ2 )(ŷ1 − ŷ2 )dt 0

(3.8)

1

= ∫(I − A−1 Dy f (x̂ + xo , (t ŷ1 + (1 − t)ŷ2 + yo ))(ŷ1 − ŷ2 )dt. 0

By the continuity of (x, y) 󳨃→ Dy f (x, y) on U, we have the continuity of (x̂, ŷ) 󳨃→ Dy f (x̂ + xo , ŷ + yo ) on U0 . Since A−1 Dy (xo , yo ) = I, by continuity for a chosen ϱ ∈ (0, 1) there exist δ1 , δ2 > 0 such that 󵄩 󵄩 sup 󵄩󵄩󵄩I − A−1 Dy f (x̂ + xo , t ŷ1 + (1 − t)ŷ2 + yo )󵄩󵄩󵄩ℒ(Y) ≤ ϱ,

t∈[0,1]

as long as (x̂, ŷ) ∈ BX (0, δ1 ) × BY (0, δ2 ). We therefore obtain by (3.8) that 󵄩󵄩 ̂ 󵄩 󵄩󵄩ĝx (y1 ) − ĝx (ŷ2 )󵄩󵄩󵄩Y ≤ ϱ‖ŷ1 − ŷ2 ‖Y .

(3.9)

The mapping ŷ 󳨃→ ĝx (ŷ) is therefore a contraction. We need to find suitable values for x̂ such that gx maps BY (0, r2 ) ⊂ BY (0, δ2 ) to itself, so that we may apply the Banach

3.1 Banach fixed-point theorem

| 113

contraction mapping theorem. As in the proof of Theorem 3.1.7, we have 󵄩 󵄩 −1 󵄩 󵄩 󵄩 󵄩 󵄩󵄩 ̂ 󵄩󵄩 󵄩󵄩ĝx (y)󵄩󵄩Y ≤ 󵄩󵄩󵄩ĝx (0)󵄩󵄩󵄩Y + 󵄩󵄩󵄩ĝx (ŷ) − ĝx (0)󵄩󵄩󵄩Y ≤ 󵄩󵄩󵄩A f (x̂ + xo , yo )󵄩󵄩󵄩Y + ϱ‖ŷ‖Y ,

(3.10)

where we used the triangle inequality, the definition of gx and (3.9). By the continuity of f and since f (xo , yo ) = 0, we may find a r1 such that 󵄩 󵄩󵄩 −1 ̂ 󵄩󵄩A f (x + xo , yo )󵄩󵄩󵄩Y ≤ (1 − ϱ)r2 ,

∀x̂ ∈ B(0, r1 ).

Hence, by (3.10) we have that ‖ĝx (ŷ)‖Y ≤ r2 , therefore, ĝx maps B(0, r2 ) to itself. By the Banach contraction theorem, for every x̂ ∈ BX (0, r1 ), the mapping ĝx admits a fixed-point ŷ; hence, by the discussion above for every x ∈ BX (xo , r1 ) there exists a y ∈ BY (yo , r2 ) such that f (x, y) = 0. We define the mapping x 󳨃→ y =: ψ(x) where y is the above fixed point, and clearly f (x, ψ(x)) = 0. In order to prove the continuity of the function ψ, for any xi ∈ BX (xo , r1 ), expressed as xi = x̂i + xo , by definition (using the construction of ψ(xi ) as a fixed point we have that ψ(xi ) = yo + ĝxi (ψ(xi )), i = 1, 2. Noting that f (xi , ψ(xi )) = 0, i = 1, 2, we express ψ(x1 ) − ψ(x2 ) = ĝx1 (ψ(x1 )) − ĝx1 (ψ(x2 )) + ĝx1 (ψ(x2 )) − ĝx2 (ψ(x2 )) and using the triangle inequality and the contraction estimate (3.9) we obtain 󵄩󵄩 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩󵄩ψ(x1 ) − ψ(x2 )󵄩󵄩󵄩Y ≤ ϱ󵄩󵄩󵄩ψ(x1 ) − ψ(x2 )󵄩󵄩󵄩Y + 󵄩󵄩󵄩ĝx1 (ψ(x2 )) − ĝx2 (ψ(x2 ))󵄩󵄩󵄩Y , which rearranged yields that 1 󵄩󵄩 󵄩󵄩 󵄩 󵄩 󵄩󵄩ψ(x1 ) − ψ(x2 )󵄩󵄩󵄩Y ≤ 󵄩ĝ (ψ(x2 )) − ĝx2 (ψ(x2 ))󵄩󵄩󵄩Y , 1 − ϱ 󵄩 x1

(3.11)

and since ĝx has by construction continuous dependence on the parameter x̂, taking x̂1 → x̂2 (and consequently x1 → x2 ) by (3.11) we obtain the continuity of ψ. (ii) The differentiability result uses similar arguments as in the proof of Theorem 3.1.7 so it is only sketched. Since f ∈ C 1 (U, Z), together with the observations that f (x + h, ψ(x + h)) = f (x + ψ(x)) = 0 for x and h so that we are within the domain of definition of ψ, and f (x + h, ψ(x + h)) − f (x, ψ(x)) − Dx f (x, ψ(x))h − Dy f (x, ψ(x))(ψ(x + h) − ψ(x)) = o(‖h‖) (using the small-o notation for simplicity) first at xo , we obtain that ψ(xo + h) − ψ(xo ) + (Dy f (xo , ψ(xo ))) Dx f (x, ψ(x))h = o(‖h‖) −1

from which the claim follows at xo . We then proceed by a continuity argument. There are a couple of interesting remarks to be made. First of all, clearly, by applying the implicit function theorem to f (x, y) = x − h(y), we retrieve the inverse function theorem for the function y 󳨃→ h(y). Second, by iteration of the arguments above, if f ∈ C m (U, Z), 1 ≤ m ≤ ∞ on a neighborhood of (xo , yo ) then ψ ∈ C m (N(xo )) on a neighborhood of xo (see [110]). Third, the proof is constructive, leading to an iterative scheme for obtaining the function ψ (or the inverse of a function in the case of the inverse function theorem), in terms of the fixed-point scheme.

114 | 3 Fixed-point theorems and their applications 3.1.5 Iterative schemes for the solution of operator equations In many cases, we need to solve operator equations of the form f (x) = 0 for some nonlinear operator f : X → Y, where X, Y are Banach spaces. Banach’s contraction theorem can prove very useful not only for guaranteeing existence of a solution, but also in constructing iterative schemes for approximating this solution. The following example provides an illustration of such schemes. Example 3.1.9 (A Newton-like method for the solution of f (x) = 0). Let f : BX (xo , ro ) ⊂ X → Y for some xo ∈ X and ro > 0, continuously differentiable and such that Df (xo )−1 ∈ ℒ(Y, X), with ‖Df (x1 ) − Df (x2 )‖ℒ(X,Y) ≤ L‖x1 − x2 ‖X , for every x1 , x2 ∈ BX (xo , ro ). Assuming that f (x) = 0 has a unique zero x̄ ∈ BX (xo , ro ) we may, under certain conditions, approximate it with the Newton- like iterative method xn+1 = xn − (Df (xo ))−1 f (xn ). Can you find a set of conditions such that the above iterative method converges to x?̄ Consider the mapping g : BX (xo , r) → BX (xo , r) defined by ∀x ∈ BX (xo , r),

g(x) := x − (Df (xo )) f (x), −1

for a properly chosen r > 0 (to be specified shortly). If we show that g has a fixed point, x̄ ∈ BX (xo , r) then x̄ is the zero of f we seek, as long as4 r ≤ ro . We will look for a set of conditions such that g can be a contraction map, which maps BX (xo , r) into itself. To check the contraction property, pick any x1 , x2 ∈ BX (xo , r), express g(x1 ) − g(x2 ) = x1 − x2 − (Df (xo )) (f (x1 ) − f (x2 )), −1

and using the differentiability of f we can write 1

1

0

0

d f (x1 ) − f (x2 ) = ∫ f (x2 + t(x1 − x2 ))dt = ∫ Df (x2 + t(x1 − x2 ))(x1 − x2 )dt, dt with the integral defined in the Bochner sense. Rearrange g(x1 ) − g(x2 ) as g(x1 ) − g(x2 ) = (Df (xo )) Df (xo )(x1 − x2 ) − (Df (xo )) (f (x1 ) − f (x2 )) −1

−1

= (Df (xo )) [Df (xo )(x1 − x2 ) − (f (x1 ) − f (x2 ))] −1

1

= (Df (xo )) [Df (xo )(x1 − x2 ) − ∫ Df (x2 + t(x1 − x2 ))(x1 − x2 )dt] −1

1

0

= (Df (xo )) ∫[Df (xo ) − Df (x2 + t(x1 − x2 ))](x1 − x2 )dt, −1

0

4 By assumption, the zero of f is unique in BX (xo , ro ). So since a fixed point in BX (xo , r) is a zero of f it must be the zero we are seeking. If the zero is not unique, it converges to one of the zeros of f in the chosen domain.

3.1 Banach fixed-point theorem

| 115

so that now by the Lipschitz continuity of Df we may control the size of the difference g(x1 ) − g(x2 ). Indeed, we have that 󵄩 󵄩󵄩 󵄩󵄩g(x1 ) − g(x2 )󵄩󵄩󵄩Y

󵄩󵄩 󵄩󵄩 1 󵄩󵄩 󵄩󵄩 −1 󵄩 󵄩󵄩 󵄩 ≤ 󵄩󵄩(Df (xo )) 󵄩󵄩ℒ(Y,X) 󵄩󵄩󵄩∫[Df (xo ) − Df (x2 + t(x1 − x2 ))](x1 − x2 )dt 󵄩󵄩󵄩 󵄩󵄩 󵄩󵄩 󵄩Y 󵄩0 1

−1 󵄩 󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩(Df (xo )) 󵄩󵄩󵄩ℒ(Y,X) ∫󵄩󵄩󵄩[Df (xo ) − Df (x2 + t(x1 − x2 ))](x1 − x2 )󵄩󵄩󵄩Y dt 0

1

−1 󵄩 󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩(Df (xo )) 󵄩󵄩󵄩ℒ(Y,X) ∫󵄩󵄩󵄩Df (xo ) − Df (x2 + t(x1 − x2 ))󵄩󵄩󵄩ℒ(X,Y) ‖x1 − x2 ‖X dt 0

1

−1 󵄩 󵄩 󵄩 󵄩 ≤ L󵄩󵄩󵄩(Df (xo )) 󵄩󵄩󵄩ℒ(Y,X) ‖x1 − x2 ‖X ∫󵄩󵄩󵄩xo − x2 − t(x1 − x2 )󵄩󵄩󵄩X dt, 0

where we used the properties of the Bochner integral and the Lipschitz continuity of Df . Since x1 , x2 ∈ BX (xo , r), by convexity x2 + t(x1 −x2 ) ∈ BX (xo , r); hence, ‖xo −x2 − t(x1 − x2 )‖X ≤ r, and setting c = ‖(Df (xo ))−1 ‖ℒ(Y,X) to simplify notation we see that 󵄩󵄩 󵄩 󵄩󵄩g(x1 ) − g(x2 )󵄩󵄩󵄩Y ≤ crL‖x1 − x2 ‖X , which is a contraction as long as crL < 1. It remains to make sure that g maps BX (xo , r) to itself. Consider any x ∈ BX (xo , r). Then, expressing g(x) − xo = x − xo − (Df (xo )) f (x) −1

= (Df (xo )) [Df (xo )(x − xo ) − f (x) + f (xo ) − f (xo )] −1

= (Df (xo )) [Df (xo )(x − xo ) − (f (x) − f (xo ))] − (Df (xo )) f (xo ) −1

−1

and observing that 1

1

0

0

d f (x) − f (xo ) = ∫ f (x + t(x − xo ))dt = ∫ Df (x + t(x − xo ))(x − xo )dt, dt so that 1

Df (xo )(x − xo ) − (f (x) − f (xo )) = ∫[Df (xo ) − Df (x + t(x − xo ))](x − xo )dt, 0

and we managed once more to bring into the game the difference of the Fréchet derivative at two different points, which we can control by the Lipschitz property of Df .

116 | 3 Fixed-point theorems and their applications Hence, by the above considerations, and with similar arguments as the ones we used for the contraction property we have that 1

−1 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩󵄩 󵄩󵄩g(x) − xo 󵄩󵄩󵄩X ≤ L󵄩󵄩󵄩(Df (xo )) 󵄩󵄩󵄩ℒ(Y,X) (∫󵄩󵄩󵄩xo − x − t(x − xo )󵄩󵄩󵄩X dt)‖x − xo ‖X 0

−1 󵄩 󵄩 󵄩 󵄩 + 󵄩󵄩󵄩(Df (xo )) 󵄩󵄩󵄩ℒ(Y,X) 󵄩󵄩󵄩f (xo )󵄩󵄩󵄩Y

from which it follows that 󵄩󵄩 󵄩 2 󵄩󵄩g(x) − xo 󵄩󵄩󵄩X ≤ cr L + cco , where co = ‖f (xo )‖Y . In order that g maps BX (xo , r) to itself, it must hold that cr 2 L + cco ≤ r. Summarizing, since ro , c, co , L > 0 are given by the problem, we need to make sure that ro , c, co are chosen so as there exists an r ≤ ro such that crL < 1,

and cr 2 L + co ≤ r.

This problem reduces to solving a set of linear and quadratic inequalities, which is of minor importance here. To illustrate the possibility of convergence of the scheme, consider the possibility that r = ro and choose α < 1 so that cro L = α. Then the second inequality becomes αro + cco ≤ r which holds as long as cco ≤ (1 − a)ro . ◁

3.2 The Brouwer fixed-point theorem and its consequences Banach’s contraction theorem is a very useful tool but rather restrictive, since the condition that f is a contraction will not hold for most cases of interest. This introduces the need for fixed-point theorems which do not require that f is a contraction but only that it is continuous. Clearly, this introduces the need of further restrictions on the domain of definition of the function such as for instance compactness. One of the first results in this direction, valid for finite dimensional spaces, and upon which most, if not all, the infinite dimensional counterparts of such results are based, is the celebrated Brouwer fixed-point theorem [2]. We will introduce here the Brouwer fixed-point theorem and illustrate it with various examples and extensions. Throughout this section, we will either focus on X = ℝd , whose elements will be denoted by x = (x1 , . . . , xd ), endowed with the Euclidean metric | ⋅ |, deriving from the standard inner product ⟨x, z⟩ = x ⋅ z (in this section we will use the former), or more general finite dimensional spaces. 3.2.1 Some topological notions Before introducing the Brouwer’s fixed-point theorem, we need to recall some definitions.

3.2 The Brouwer fixed-point theorem and its consequences | 117

Definition 3.2.1 (Retractions and retracts). Let X be a topological space, A ⊂ X and r : X → A a continuous map. The map r is called a retraction of X onto A if r(x) = x 󵄨 for all x ∈ A, or equivalently, if the identity map I = r󵄨󵄨󵄨A : A → A, admits a continuous extension on X. The set A is called a retract of X if a retraction of X onto A exists. Example 3.2.2. Let X = ℝd , endowed with the Euclidean norm | ⋅ |, and A = B(0, R). A retraction for A is given by x, r(x) = { R |x|

if |x| ≤ R x,

if |x| > R;

hence, A is a retract of X.

◁

A very important result is the following “no retraction” theorem for the unit ball in ℝd . For the proof, using simplicial topology, we refer to [59]. Theorem 3.2.3 (No retraction). The boundary 𝜕B(0, 1) := {x ∈ ℝd : |x| = 1} = S(0, 1) of a d-dimensional closed ball B(0, 1) of ℝd is not a retract of B(0, 1), for any d ≥ 1. Clearly, the result holds for the d-dimensional closed ball of any radius R > 0 and its corresponding boundary centered at any xo ∈ ℝd .

3.2.2 Various forms of the Brouwer fixed-point theorem Using the “no retraction” theorem, we may obtain Brouwer’s fixed-point theorem. Theorem 3.2.4 (Brouwer fixed-point theorem I). Let B(0, 1) ⊂ ℝd be the unit closed ball and f : B(0, 1) → B(0, 1) a continuous map. Then f has a fixed point. Proof. Suppose that f has no fixed point. Then f (x) − x ≠ 0 for all x ∈ B(0, 1). We will show that then we may construct a retraction r : B(0, 1) → 𝜕B(0, 1) = S(0, 1) which contradicts Theorem 3.2.3. Indeed, consider the map r : B(0, 1) → S(0, 1) defined by r(x) = x + λ(x)(x − f (x)), for every x ∈ B(0, 1), with λ(x) > 0 chosen so that r(x) ∈ S(0, 1) for any |x| ≤ 1. One may easily calculate the value of λ(x) > 0 so that r maps the ball to its boundary, by requiring that λ(x) ≥ 0 is the solution of the equation 󵄨󵄨 󵄨2 󵄨 󵄨2 󵄨󵄨r(x)󵄨󵄨󵄨 = 󵄨󵄨󵄨x + λ(x)(x − f (x))󵄨󵄨󵄨 = 1.

(3.12)

We must show that this equation has a unique solution λ(x) ≥ 0. From (3.12), we get 󵄨 󵄨2 |x|2 + 2λ(x)⟨x, x − f (x)⟩ + λ2 (x)󵄨󵄨󵄨x − f (x)󵄨󵄨󵄨 = 1.

118 | 3 Fixed-point theorems and their applications where ⟨⋅, ⋅⟩ is the inner product in ℝd . This is a quadratic equation with respect to λ(x), so λ(x) =

−⟨x, x − f (x)⟩ ± √⟨x, x − f (x)⟩2 + (1 − |x|2 )|x − f (x)|2 |x − f (x)|2

,

and keeping in mind that |x|2 ≤ 1 we see that the root with the plus sign satisfies λ(x) ≥ 0. It is easy to see that r is continuous with this choice and that if |x| = 1 it holds that λ(x) = 0; so that r maps any x ∈ S(0, 1) to itself, i. e., it is a retraction. This completes the proof. Corollary 3.2.5. Let B(0, R) ⊂ ℝd be the closed ball of ℝd of radius R. If f : B(0, R) → B(0, R) is continuous, then f has a fixed point. Proof. We consider the mapping g : B(0, 1) → B(0, 1) defined by g(x) = f (Rx) . Clearly, R g is continuous so by Brouwer’s fixed-point theorem it has a fixed-point xo , which implies that Rxo is a fixed point for f . Corollary 3.2.6. Let f : ℝd → ℝd be continuous such that ⟨f (x), x⟩ ≥ 0 for all x such that |x| = R > 0. Then there exists xo with |xo | ≤ R such that f (xo ) = 0. Proof. Suppose f (x) ≠ 0 for all x ∈ B(0, R). Define g : B(0, R) → B(0, R) by g(x) = R − |f (x)| f (x). This map is continuous, therefore, by Corollary 3.2.5 there exists xo such that xo = g(xo ). We have |xo | = |g(x0 )| = R. It then follows that 0 < R2 = |xo |2 = ⟨xo , xo ⟩ = ⟨xo , g(xo )⟩ = −

R ⟨x , f (xo )⟩ ≤ 0 |f (xo )| o

which is a contradiction. Remark 3.2.7. The above result is also true if ⟨f (x), x⟩ ≤ 0 for all x ∈ ℝd such that |x| = R > 0. The power of Brouwer’s fixed-point theorem lies on the fact that it generalizes for subsets of ℝd , which are far more general than the closed ball, and in particular to compact and convex sets. A particular special case of that is the simplex, which was in fact Brouwer’s original version of the theorem. Theorem 3.2.8 (Brouwer fixed-point theorem II). Let C ⊂ ℝd be a compact and convex set and f : C → C a continuous map. Then f has a fixed point. Proof. Let PC : ℝd → C be the projection on C. Since C is compact, there exists R > 0 such that C ⊂ B(0, R). Define g : B(0, R) → B(0, R) by g(x) = f (PC (x)). Since PC is continuous, g is continuous. Therefore by Corollary 3.2.5, it has a fixed- point xo ∈ B(0, R), i. e., xo = g(xo ) ∈ C. But PC (xo ) = xo and so xo = f (xo ). Remark 3.2.9. The conclusion of Theorem 3.2.8 holds if f : A → A is continuous where A ⊂ ℝd is homeomorphic to a compact convex set C.

3.2 The Brouwer fixed-point theorem and its consequences | 119

Brouwer’s theorem has many applications that we will see shortly. As a first application, we provide a proof of Perron–Frobenius theorem using Brouwer’s theorem (see, e. g., [55]). Example 3.2.10 (Perron–Frobenius theorem as a consequence of Brouwer’s theorem). Let A = (aij )di,j=1 be an d×d matrix with aij ≥ 0, i, j = 1, 2, . . . , d. Then A has an eigenvalue λ ≥ 0 with corresponding nonnegative5 eigenvector xo = (xo,1 , . . . , xo,d ). Let C = {x ∈ ℝd : xi ≥ 0, i = 1, . . . , d, ∑di=1 xi = 1}, which is convex and compact. If Ax = 0 for some x ∈ C, then λ = 0 is an eigenvalue and the theorem holds. If Ax ≠ 0 for every x ∈ C, then we may define the mapping f : C → C, by f (x) = d Ax (which is d ∑i=1 ∑j=1 aij xj

well-defined by our assumption) and continuous; hence, by Brouwer’s theorem 3.2.8, admits a fixed point xo = (xo,1 , . . . , xo,d ) ∈ C. Since f (xo ) = xo , this implies that Axo = λxo , with λ = ∑di=1 ∑dj=1 aij xo,j > 0. ◁ We now give a result which is equivalent to Brouwer’s fixed-point theorem (see [110]). Theorem 3.2.11 (Knaster–Kuratowski–Mazurkiewicz lemma). Let Σd = conv{e1 , . . . , ed } be the d − 1-dimensional simplex generated by the elements ei , i = 1, . . . , d, and let A1 , A2 , ⋅ ⋅ ⋅ , Ad ⊆ Σd , be closed sets with the property that, for each set I ⊆ {1, 2, . . . , d} the inclusion conv{ei , i ∈ I} ⊂ ⋃ Ai , i∈I

(3.13)

holds. Then A1 ∩ A2 ∩ ⋅ ⋅ ⋅ ∩ Ad ≠ 0. Proof. For i = 1, 2, . . . , d, we set ψi (x) = dist(x, Ai ), the distance between x and Ai , and fi (x) :=

xi + ψi (x) , 1 + ψ1 (x) + ⋅ ⋅ ⋅ + ψd (x)

x = (x1 , x2 , . . . , xd ) ∈ Σd .

(3.14)

Since the mapping x 󳨃→ ψi (x) := dist(x, Ai ) is Lipschitz for any i = 1, . . . , d, it is obvious that f = (f1 , . . . , fd ) : Σd → Σd is continuous. By Brouwer’s fixed-point theorem (see also Remark 3.2.9) f has a fixed point xo ∈ Σd . From (3.13), we have that xo ∈ A1 ∪ A2 ∪ ⋅ ⋅ ⋅ ∪ Ad , which implies that ψi (xo ) = 0 for one i ∈ {1, 2, . . . , d}. Since fi (xo ) = xo,i , the denominator in (3.14) must be 1. It then follows that ψ1 (xo ) = ⋅ ⋅ ⋅ = ψd (xo ) = 0, i. e., xo ∈ A1 ∩ ⋅ ⋅ ⋅ ∩ Ad . 3.2.3 Brouwer’s theorem and surjectivity of coercive maps We have already encountered the notion of a coercive functional in Section 2.2, and have seen its connection with optimization problems. We will recall this notion here 5 That is, satisfying xo,i ≥ 0, i = 1, . . . , d.

120 | 3 Fixed-point theorems and their applications (in the special context of maps f : ℝd → ℝd ) and investigate its connection to surjectivity properties. This theme will be encountered later on in this book, in the context of nonlinear operators between Banach spaces (see Chapter 9). Definition 3.2.12 (Coercive map). A mapping f : ℝd → ℝd is called coercive if lim|x|→∞ ⟨f (x),x⟩ = +∞. |x| Coercive maps enjoy important surjectivity properties. Proposition 3.2.13. Let f : ℝd → ℝd be a continuous and coercive mapping. Then f is surjective, i. e., the equation f (x) = y has a solution for every y ∈ ℝd . Proof. For y ∈ ℝd , we define g(x) = f (x) − y. Since f is coercive, for every M > 0 there exists R > 0 such that if |x| > R, then ⟨f (x), x⟩ > M |x|. Choosing M = |y|, we obtain for |x| = R, ⟨g(x), x⟩ = ⟨f (x) − y, x⟩ ≥ ⟨f (x), x⟩ − |y| |x| > 0. From Corollary 3.2.6, there exists x such that g(x) = 0, i. e., f (x) = y. 3.2.4 Application of Brouwer’s theorem in mathematical economics Brouwer’s theorem and the KKM lemma has found important applications in mathematical economics and in particular to the theory of general equilibrium and the theory of games (see, e. g., [54, 87]). A dominant model in mathematical economics is that of general equilibrium, in which a number of goods or assets are available in a market, and their prices are to be determined by the forces of demand and supply. Assume a very simple model of this form, the so-called Walras exchange economy in which a number of agents (who are not big enough economic units to affect prices directly) meet in the market place with initial endowments of d goods. These agents will exchange goods at a particular price (yet to be determined) so that the demand for each good matches its total supply (as determined by the total initial endowment of the agents in these goods). There is no production in this simplified economy, simply exchange of already produced goods, and the basic idea behind this simplified model is that agents may be endowed with bundles of goods which are not matching their needs, and they try to meet their needs by exchanging goods with others who may have different endowments and different needs. The rate of exchange, in this simplified economy, defines the prices of the goods. The crucial question is whether such a decentralized scheme may function and produce a price in which total demand matches the total supply for every good. Such a price will be called an equilibrium price. We will show how the existence of such an equilibrium price can be obtained with the use of the KKM lemma. This simplified model finds important applications in economics and in finance (where it is often referred to as the Arrow–Debreu model for asset pricing).

3.2 The Brouwer fixed-point theorem and its consequences | 121

To make the above conceptual model more concrete assume that there are d different goods in the market and J agents each one endowed with a quantity of the d goods wj = (wj,1 , . . . , wj,d ) ∈ ℝd+ , j = 1, . . . , J. Each agent’s actual needs in these goods are xj = (xj,1 , . . . , xj,d ) ∈ ℝd+ . In order to reach this goal, agent j must exchange part of his/her initial endowment wj with some other agent j󸀠 whose initial endowment is not necessarily in accordance to his/her goals. It is not necessary that goods are exchanged at an one to one basis, i. e., one unit of good i is not necessarily exchanged with one unit of good j, a fact that allows us to introduce a price pi for good i in terms of the exchange rate with a fixed good (called the numeraire). The quantity xj is the demand of agent j for the goods, whereas the quantity wj is the supply of agent j for the goods. The total demand vector for the goods is D = ∑Jj=1 xj , whereas the total sup-

ply vector for the goods is S = ∑Jj=1 wj . Since there is no production in the economy, it must be that D ≤ S (meaning that this inequality holds componentwise for the vectors). If this inequality is strict, then certain goods in this economy are left unwanted. Since the total demand and supply depend on the prices of the goods, the important question is whether there exists a price vector p = (p1 , . . . , pd ) ∈ ℝd+ for the goods so that D(p) = S(p). Then we say that the market clears and the corresponding price is called an equilibrium price. An important aspect of this theory is the assumption that both the demand as well as the supply function are determined in a self consistent fashion in terms of a constrained preferences and profit maximization problem, respectively. For example, the preference maximization problem (the solution of which determines the preferred individual demand of each agent) must be solved under the individual constraints that ∑di=1 pi xj,i = ∑di=1 pi ⋅ wj,i where wj is the initial endowment and p = (p1 , . . . , pd ) is the price vector (common to all agents). Since the constraints are positively homogeneous, we may consider the problem as restricted to the simp plex Σd , by rescaling all prices pi to p󸀠i = d i . As the total demand is the sum of all ∑k=1 pk

individual demands the above remark applies to the total demand as well. Moreover, under sufficient conditions on the preferences, the demand function can be shown to be continuous, similarly for the supply function. In what follows, we will assume the price as restricted to the simplex Σd . Before moving to the proof of existence of such an equilibrium price, we must introduce an important identity called Walras’ law, upon which the existence is based. Define the function g : Σd ⊂ ℝd+ → ℝd by g(p) = D(p) − S(p). This function is called the excess demand function and yields the excess total demand for each good as a function of price (note that excess demand may be negative if a good is not sufficiently wanted). It can be shown that under fairly weak assumptions, this function is continuous. Walras’ law states that ⟨g(p), p⟩ := g(p) ⋅ p = 0,

∀ p ∈ Σd ⊂ ℝd+ .

(3.15)

This law comes from the very simple observation that typically agent j will choose his/her demand by maximizing some utility function, describing his/her preferences,

122 | 3 Fixed-point theorems and their applications but subject to the individual constraint ⟨p, xj ⟩ = ⟨p, wj ⟩ which states that the agent will choose a new consumption bundle xj ∈ ℝd+ to satisfy his/her needs, which comes at a price ⟨p, xj ⟩ that must be affordable to the agent who only has his/her initial endowment wj ∈ ℝd+ to trade, which is of market value ⟨p, wj ⟩. Behavioral assumptions on the agents (nonsatiation) imply that the agent will spend all his/her initial endowment in order to reach his/her consumption goal. Adding the individual constraints over all the agents, leads to (3.15). Here, we illustrate an answer to the following related question: If the excess demand function is such that ∑di=1 pi gi (p) ≤ 0 for all p = (p1 , . . . , pd ) ∈ Σd ⊂ ℝd+ (which clearly holds since D ≤ S) does there exist a po = (po,1 , . . . , po,d ) ∈ Σd ⊂ ℝd+ such that gi (po ) ≤ 0 for all i = 1, . . . , d? One can easily see that by Walras’ law (3.15), ⟨g(po ), po ⟩ = ∑di=1 gi (po )po,i = 0, and since gi (po ) ≤ 0 for every i = 1, . . . , d it holds that gi (po ) = 0 for every i = 1, . . . , d; hence, the market clears at this price. The following theorem, because of its applicability in mathematical economics is known as the existence theorem of general equilibrium, and is due in various versions to Gale, Nikaido and Debreu. Theorem 3.2.14. Let g : Σd → ℝd continuous with ⟨p, g(p)⟩ ≤ 0 for all p ∈ Σd . Then there exists po ∈ Σd with gi (po ) ≤ 0 for all i = 1, . . . , d. Proof. The sets Ai = {p ∈ Σd : gi (p) ≤ 0}, i = 1, . . . , d are closed and satisfy (3.13). Indeed, denoting by {ei : i = 1, . . . d} the standard basis of ℝd , if there exists I ⊂ {1, . . . , d} and a p ∈ conv{ei , i ∈ I} with gi (p) > 0 for all i ∈ I, then for this p, ⟨p, g(p)⟩ = ∑ pi gi (p) > 0, i∈I

which contradicts our assumption for g. Therefore, from the KKM lemma (see Theorem 3.2.11) there exists po ∈ A1 ∩ ⋅ ⋅ ⋅ ∩ Ad . Note that this theorem requires that g is defined on Σd , the unit simplex of ℝd , rather than ℝd+ , so it may not be applied directly to the excess demand function. However, one may perform a scaling argument, using p󸀠 = d1 p ∈ Σd noting that the de∑i=1 pi

mand function is homogeneous in p of degree 0 (as follows directly by its definition in terms of an optimization problem). Upon this transformation, we apply Theorem 3.2.14 and obtain the required result. Apart from the theory of general equilibrium, Brouwer’ fixed-point theorem finds other fundamental applications in mathematical economics, such as for instance in the theory of games (see, e. g., [87]). 3.2.5 Failure of Brouwer’s theorem in infinite dimensions The following proposition shows that an extension of the Brower fixed-point theorem in infinite dimensional spaces is not possible.

3.3 Schauder fixed-point theorem and Leray–Schauder alternative

| 123

Proposition 3.2.15 (Kakutani). Let H be an infinite dimensional separable Hilbert space. Then there exists a continuous mapping f : BH (0, 1) → BH (0, 1) that has no fixed point. Proof. Let {zn : n ∈ ℕ ∪ {0}} be an orthonormal basis in H. For any x ∈ H, admitting ∞ an expansion as x = ∑∞ i=0 ai zi define p(x) = ∑i=1 ai−1 zi and the map f by 1 f (x) = (1 − ‖x‖)z0 + p(x). 2 This map is continuous and maps BH (0, 1) into itself. Indeed, for ‖x‖ ≤ 1 we have 󵄩󵄩 󵄩 1 󵄩󵄩f (x)󵄩󵄩󵄩 ≤ (1 − ‖x‖)‖z0 ‖ + ‖x‖ ≤ 1, 2 ∞ 2 2 2 where we used the observation that ‖x‖2 = ∑∞ i=0 |ai | and ‖p(x)‖ = ∑i=1 |ai−1 | . Suppose that f has a fixed-point xo ∈ BH (0, 1), i. e., there exists xo = ∑∞ i=0 ai zi ∈ BH (0, 1), such that f (xo ) = xo . Then 21 (1 − ‖xo ‖)z0 + p(xo ) = xo , i. e.,

1 xo − p(xo ) = (1 − ‖xo ‖)z0 . 2

(3.16)

We have to examine three cases: (i) xo = 0. Then, from (3.16) we have z0 = 0 which is a contradiction. ∞ (ii) ‖xo ‖ = 1. Then from (3.16) we have xo = p(xo ), i. e., ∑∞ i=0 ai zi = ∑i=1 ai−1 zi , which implies that ai = 0 for all i = 0, 1, . . ., therefore, xo = 0 which by (ii) is a contradiction. ∞ 2 (iii) 0 < ‖xo ‖ < 1. Let xo = ∑∞ i=1 ai zi , where ∑i=0 |ai | < 1. Then, from (3.16) we have ∞ ∞ 1 ∑ ai zi − ∑ ai−1 zi = (1 − ‖xo ‖)z0 , 2 i=0 i=1 2 which implies that ai = ai−1 for all i = 1, 2, . . ., therefore, ‖xo ‖2 = ∑∞ i=0 |ai | = ∞ which is a contradiction.

Therefore, all three possible cases result to a contradiction and the claim of the proposition is proved.

3.3 Schauder fixed-point theorem and Leray–Schauder alternative Even though, Brouwer’s fixed-point theorem may fail in infinite dimensional spaces, one can recover similar results by imposing certain constraints on the subsets of the infinite dimensional space on which we require the existence of a fixed point. One of the first results in this direction was proved by Julius Schauder in the 1930s, and requires convexity and compactness of the subset A ⊂ X we wish to work in [2, 110].

124 | 3 Fixed-point theorems and their applications 3.3.1 Schauder fixed-point theorem We need the following lemma. Lemma 3.3.1. Let X be a Banach space and C ⊂ X a compact set. For every ϵ > 0, there exists a finite dimensional subspace Xϵ ⊂ X and a continuous map gϵ : C → Xϵ , such that 󵄩󵄩 󵄩 󵄩󵄩gϵ (x) − x󵄩󵄩󵄩 < ϵ,

∀ x ∈ C.

(3.17)

If, in addition, C is convex, then gϵ (x) ∈ C for all x ∈ C. Proof. Let ϵ > 0. Since C is compact, there exists a finite set {x1 , . . . , xn } ⊂ C, (n = n(ϵ)) such that C ⊂ ⋃ni=1 BX (xi , ϵ). Let Xϵ = span{x1 , . . . , xn } ⊂ X. Then dimXϵ = n < ∞. Define for any i = 1, . . . , n, ϵ − ‖x − xi ‖, if x ∈ BX (xi , ϵ) ψi (x) = { 0, otherwise. We can define the map gϵ : C → Xϵ by gϵ (x) =

ψ1 (x)x1 + ⋅ ⋅ ⋅ + ψn (x)xn . ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x)

Note that, by compactness, for each x ∈ C there exists an i ∈ {1, . . . , n} with ψi (x) > 0. It is clear that gϵ is continuous, since each ψi is continuous. For x ∈ C, we have 󵄩 󵄩 󵄩󵄩 󵄩 󵄩󵄩 ψ (x)x1 + ⋅ ⋅ ⋅ + ψn (x)xn ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x) 󵄩󵄩󵄩 − x󵄩󵄩 󵄩󵄩gϵ (x) − x󵄩󵄩󵄩 = 󵄩󵄩󵄩 1 ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x) 󵄩󵄩 󵄩󵄩 ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x) 󵄩󵄩 ψ (x)(x − x ) + ⋅ ⋅ ⋅ + ψ (x)(x − x ) 󵄩󵄩 󵄩 1 n n 󵄩 = 󵄩󵄩󵄩 1 󵄩󵄩󵄩 ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x) 󵄩󵄩 󵄩󵄩 ≤

ψ1 (x)‖x − x1 ‖ + ⋅ ⋅ ⋅ + ψn (x)‖x − xn ‖ < ϵ. ψ1 (x) + ⋅ ⋅ ⋅ + ψn (x)

This completes the proof. We are now ready to state and prove the Schauder fixed-point theorem. Theorem 3.3.2 (Schauder). Let X be a Banach space and let C ⊂ X be a convex compact set. If f : C → C is continuous, then f has a fixed point. Proof. For any ϵ > 0, let Xϵ , gϵ be as in Lemma 3.3.1. Since C is convex, gϵ (C) ⊂ C. Let Cϵ = conv{x1 , . . . , xn }. Then Cϵ ⊂ C. Now define fϵ : Cϵ → Cϵ by fϵ (x) = gϵ (f (x)).

3.3 Schauder fixed-point theorem and Leray–Schauder alternative

| 125

Since Cϵ is a compact convex set in the finite dimensional space Xϵ , it follows from Brouwer’s theorem (see also Remark 3.2.9) that fϵ has a fixed-point xϵ ∈ Cϵ ⊂ C. Thus, xϵ = fϵ (xϵ ) = gϵ (f (xϵ )). Since C is compact, we can suppose that xϵ has a convergent subsequence (denoted the same for simplicity), i. e., that there exists xo ∈ C such that xϵ → xo as ϵ → 0. Now, observe that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩󵄩xo − f (xo )󵄩󵄩󵄩 ≤ ‖xo − xϵ ‖ + 󵄩󵄩󵄩xϵ − f (xϵ )󵄩󵄩󵄩 + 󵄩󵄩󵄩f (xϵ ) − f (xo )󵄩󵄩󵄩. The first term on the right-hand side tends to 0 as ϵ → 0. The same holds for the last term, by the continuity of f . In addition, we have 󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩xϵ − f (xϵ )󵄩󵄩󵄩 = 󵄩󵄩󵄩gϵ (f (xϵ )) − f (xϵ )󵄩󵄩󵄩 < ϵ, since f (xϵ ) ∈ C by (3.17). Hence, xo − f (xo ) = 0, i. e., xo is a fixed point for f . In many applications, C is not compact. Instead, we may have f to be a compact map. The following corollary of the Schauder fixed-point theorem is useful. Corollary 3.3.3. Let C be a closed, bounded and convex set in a Banach space X and f : C → C be continuous and compact. Then f has a fixed point. Proof. Since f (C) is compact, so is C1 = convf (C). It is clear that C1 ⊂ convC = C, which implies that f maps C1 into itself. So, f has a fixed point. 3.3.2 Application of Schauder fixed-point theorem to the solvability of nonlinear integral equations Schauder’s fixed-point theorem has numerous applications in nonlinear analysis. One of its most important applications is in the theory of nonlinear PDEs or variational inequalities. For the time being to illustrate its use and potential, we prove some applications in terms of the solvability of nonlinear integral equations. Example 3.3.4 (Solvability of a nonlinear integral Volterra equation). Consider following integral equation:

the

t

x(t) = ∫ K(t, τ)fo (τ, x(τ))dτ + g(t),

t ∈ [0, 1].

(3.18)

0

Here, K : [0, 1] × [0, 1] → ℝ, fo : [0, 1] × ℝ → ℝ and g : [0, 1] → ℝ are continuous functions. Suppose furthermore that 󵄨󵄨 󵄨 󵄨󵄨fo (t, x)󵄨󵄨󵄨 ≤ a(t) + b(t)|x|,

t ∈ [0, 1], x ∈ ℝ,

126 | 3 Fixed-point theorems and their applications where a, b : [0, 1] → ℝ+ are continuous functions. Then equation (3.18) has a solution x(⋅) in C([0, 1]). We will work on the space X = C([0, 1]), with elements x = x(⋅) : [0, 1] → ℝ, and consider two equivalent norms which turn it into a Banach space, the usual supremum norm ‖x‖ := supt∈[0,1] |x(t)| and the weighted norm ‖x‖λ := supt∈[0,1] (e−λt |x(t)|), for a suitable λ > 0 to be specified shortly.6 In order to specify λ, fix M ∈ ℝ such that M > ‖g‖ and choose λ > 0 such that (3.19)

(M − ‖g‖)λ > (1 − e−λ )(‖a‖ + ‖b‖M)‖K‖. By the choice of M, it is always possible to find such λ. We now consider the set 󵄨 󵄨 C = {x ∈ C([0, 1]) : ‖x‖λ := max e−λt 󵄨󵄨󵄨x(t)󵄨󵄨󵄨 ≤ M}. t∈[0,1]

Since the norm ‖ ⋅ ‖λ is equivalent on C([0, 1]) with the norm ‖ ⋅ ‖, the set C is convex, bounded and closed. Let f be the nonlinear operator defined by the right-hand side of (3.18). Since ‖x‖λ ≤ ‖x‖, we have the estimate t

󵄨󵄨 󵄩󵄩 󵄩 󵄨 −λt 󵄨 󵄩󵄩f (x)󵄩󵄩󵄩λ ≤ ‖g‖ + max e ∫󵄨󵄨󵄨K(τ, t)󵄨󵄨󵄨 󵄨󵄨󵄨fo (τ, x(τ))󵄨󵄨󵄨dτ t∈[0,1]

0

t

󵄨 󵄨 󵄨 󵄨 ≤ ‖g‖ + max e−λt ∫󵄨󵄨󵄨K(τ, t)󵄨󵄨󵄨eλτ e−λτ (a(τ) + b(τ))󵄨󵄨󵄨x(τ)󵄨󵄨󵄨)dτ t∈[0,1]

0

t

󵄩 󵄩 󵄨 󵄨 ≤ ‖g‖ + 󵄩󵄩󵄩a + b|x| 󵄩󵄩󵄩λ max e−λt ∫󵄨󵄨󵄨K(τ, t)󵄨󵄨󵄨eλτ dτ t∈[0,1]

0

≤ ‖g‖ + (‖a‖ + ‖b‖ ‖x‖λ ) ‖K‖ max e−λt t∈[0,1]

= ‖g‖ + (‖a‖ + ‖b‖ ‖x‖λ ) ‖K‖

eλt − 1 λ

1 − e−λ . λ

By (3.19), f maps C into C. On the other hand f = ANfo , where A : C([0, 1]) → C([0, 1]) t

is the linear operator, ϕ 󳨃→ Aϕ for every ϕ ∈ C([0, 1]), with Aϕ(t) = ∫0 K(t, τ)ϕ(τ)dτ, for every t ∈ [0, 1], and Nfo : C([0, 1]) → C([0, 1]) is the Nemytskii operator, determined by fo , defined by (Nfo u)(t) = fo (t, u(t)), for every t ∈ [0, 1] and every u ∈ C([0, 1]). The operator A : C([0, 1]) → C([0, 1]) is compact (see, e. g., Example 1.3.4), whereas Nfo : C([0, 1]) → C([0, 1]) is a continuous operator (by the conditions imposed on fo ). It then follows that f : C → C is compact. By Schauder’s fixed-point Theorem 3.3.2, f has a fixed point x in C which is a solution of the nonlinear integral equation (3.18).

6 The suprema are actually maxima.

3.3 Schauder fixed-point theorem and Leray–Schauder alternative

| 127

The special case where K(τ, t) = 1 for every τ, t and g(t) = x0 , for all t, allows to obtain a solution to the initial value problem x󸀠 (t) = fo (t, x(t)) with initial condition x(0) = x0 on the interval [0, 1]. The reader can check that the approach using the Schauder fixed-point theorem provides less strict conditions on the parameters than the approach using Banach’s fixed-point theorem. ◁

3.3.3 The Leray–Schauder principle Sometimes it is not easy to find an invariant ball for a given compact operator f : X → X. In such cases, the so-called Leray–Schauder principle or Leray–Schauder alternative, which allows us to obtain a fixed point by using a priori bounds for the solution of an operator equation will be useful. We present below a version of the Leray–Schauder principle following [2, 92]. Theorem 3.3.5 (Leray–Schauder principle). Let X be a Banach space, C ⊂ X be a closed convex set and A ⊂ C an open bounded set and consider a point xA ∈ A. Suppose that f : A → C is a continuous compact map. Then, if there are no x ∈ 𝜕A , λ ∈ (0, 1) such that x = λf (x) + (1 − λ)xA , the map f has a fixed point in A. Proof. One can easily see that we may extend the condition to include λ = 0, 1. Define the set B := {x ∈ A : ∃ t ∈ [0, 1], x = tf (x) + (1 − t)xA }, which is nonempty (xA ∈ A) and closed (since f is continuous). By construction, B ∩ 𝜕A = 0, so that by Urysohn’s theorem (see [59]) we may find a continuous function ϕ ∈ C(A; [0, 1]), which is equal to 1 on B and vanishes on 𝜕A. Using this function, define fC : C → C by fC (x) = [ϕ(x)f (x) + (1 − ϕ(x))xA ]1Ā (x) + xA 1C\Ā (x), which by construction is continuous. Furthermore, again by construction fC (C) ⊂ B1 := conv({xA } ∪ f (A)), and since f (A) is relatively compact (by the compactness of f ) by Mazur’s lemma (see Proposition 1.2.18) its closure, B1 is also compact. By construction fC is continuous and maps B1 to itself, so that we may apply Schauder’s fixed-point Theorem 3.3.2 to show that fC : B1 → B1 admits a fixed-point xo ∈ C, which since xA ∈ A is also in A. This is also a fixed point for f . Indeed, since xo = fC (xo ) = (1 − ϕ(xo ))xA + ϕ(xo )f (xo ) ∈ B (recall the definition of the set B), and as ϕ(xo ) = 1 for xo ∈ B, the same equation yields xo = f (xo ) so that f admits a fixed point. A common use for the Leray–Schauder principle is to derive a strict a priori bound ‖x−xA ‖ < R for any solution of the equation x = λf (x)+(1−λ)xA in C, for λ ∈ (0, 1). Then applying the Leray–Schauder principle for the choice A󸀠 = BX (xA , R) ∩ A we obtain the existence of a fixed point for f . A related result is Schaeffer’s fixed- point theorem.

128 | 3 Fixed-point theorems and their applications Theorem 3.3.6 (Schaeffer). Let X be a Banach space and f : X → X be a continuous and compact map. If the set S := {x : ∃λ ∈ [0, 1], x = λf (x)} is bounded then a fixed point for f exists. Proof. Assume the existence of an R > 0 such that ‖x‖ < R for any x ∈ X that satisfies x = λf (x) for some λ ∈ [0, 1]. Define the continuous mapping r : X → BX (0, R) by R r(x) = x1‖x‖≤R (x) + ‖x‖ x1‖x‖>R (x), which is a retraction of X onto BX (0, R), and define

g : BX (0, R) → BX (0, R) by g = r ∘ f . This map is continuous and compact,7 hence, by Schauder’s fixed-point Theorem 3.3.2, it admits a fixed-point xo ∈ BX (0, R). It remains to show that xo is actually a fixed point of f . Note that it is not possible that ‖f (xo )‖ > R. If it did, then by the fact that xo is a fixed point of g and its definition, xo = g(xo ) = R R f (xo ) = λf (xo ), for λ = ‖f (x)‖ f (xo ) ∈ (0, 1). But then xo is a solution for xo = ‖f (x )‖ o

λf (xo ) for some λ ∈ (0, 1), such that ‖xo ‖ = ‖ ‖f (xR )‖ f (xo )‖ = R, which contradicts our o assumption that ‖x‖ < R (with strict inequality). Therefore, ‖f (xo )‖ ≤ R; hence, xo = g(xo ) = f (xo ) so that xo is a fixed point for f .

Example 3.3.7. Let f : X → X be a continuous compact operator such that supx∈X ‖f (x)‖ < +∞. Then this operator admits a fixed point. We will use Schaeffer’s fixed-point Theorem 3.3.6. Let supx∈X ‖f (x)‖ = M < +∞. For any solution of x = λf (x) for λ ∈ [0, 1] it holds that ‖x‖ = λ‖f (x)‖ ≤ λM, so choosing any R > λM and applying Schaeffer’s fixed-point theorem the claim holds. ◁

3.3.4 Application of the Leray–Schauder alternative to nonlinear integral equations In this section, we illustrate the use for the Leray–Schauder alternative by applying it to the solvability of nonlinear integral equations. For the sake of economy of space, we only present an application to the study of Hammerstein integral equations (see [92]). There are numerous applications of the Leray–Schauder alternative also to other types of integral equations such as Volterra-type equations. For a detailed account, we refer the reader to the monograph of [92] and references therein. 7 To check that, we need to show that it maps bounded subsets of BX (0, R) to precompact sets of BX (0, R) or equivalently that for a bounded sequence {xn : n ∈ ℕ} ⊂ BX (0, R) the sequence {g(xn ) : n ∈ ℕ} ⊂ BX (0, R) has a convergent subsequence. Then for the sequence {xn : n ∈ ℕ} ⊂ BX (0, R) one of the two holds, either (i) {f (xn ) : n ∈ ℕ} has a subsequence in BX (0, R) or (ii) {f (xn ) : n ∈ ℕ} has a subsequence in X \ BX (0, R). In the first case, working with this subsequence (not relabeled) {g(xn ) : n ∈ ℕ} = {f (xn ) : n ∈ ℕ} which does have a convergent subsequence by the compactness of f , while in the second case for any n ∈ ℕ we have that g(xn ) = ‖f (xR )‖ f (xn ) and moving along a

subsequence {xnk : k ∈ ℕ} such that

1 ‖f (xn )‖ k

n

→ c and f (xnk ) → z for suitable c and z, we conclude

that g(xnk ) → Rc z as k → ∞. So in either case a bounded sequence is mapped onto a sequence with a converging subsequence by g so that f ̄ is compact.

3.3 Schauder fixed-point theorem and Leray–Schauder alternative

| 129

Example 3.3.8 (Hammerstein equations). Let 𝒟 ⊂ ℝd be bounded open set, and consider a continuous vector valued function u : 𝒟 → ℝm , such that u(x) = ∫ k(x, z)fo (u(z))dz,

x ∈ 𝒟,

(3.20)

𝒟

where k : 𝒟 × 𝒟 → ℝd is a given continuous function and fo : ℝm → ℝm is a given function which is continuous if restricted to small enough values of its variable. Equation (3.20) is a nonlinear Fredholm integral equation (a special case of the general class of Hammerstein integral equations) with u ∈ C(𝒟; ℝm ) the unknown. Since u admits a representation as u = (u1 , . . . , um ), in terms of ui : 𝒟 → ℝ, we may consider (3.20) as a system of m integral equations. To apply the Leray–Schauder alternative we need first of all to ensure that the operator in question is continuous and compact. As such arguments will be useful in a number of situations, we will provide a positive answer to this question for a wider class of nonlinear integral operators. Let us define the more general nonlinear operator f : C(𝒟; ℝm ) → C(𝒟; ℝm ) by (f (u))(x) = ∫𝒟 K(x, z, u(z))dz, for every x ∈ 𝒟, for K : 𝒟 × 𝒟 × Bℝm (0, R) → ℝd for a suitable R > 0. In the special case where K is of separable form K(x, z, s) = k(x, z)fo (s), with k and fo as above, equation (3.20) reduces to the operator equation x = f (x), for x = u(⋅) ∈ X = C(𝒟; ℝm ). This operator, f , in its more general form, is continuous and compact. Continuity follows easily by the following argument: Consider any uo ∈ C(𝒟; ℝm ), and let B(0, R) be the ball of ℝm , or radius R, centered at 0. Choose any R > 0 such that ‖uo ‖ < R, and fix an arbitrary ϵ > 0. By continuity, we have that, when restricted to the compact set 𝒟 × 𝒟 × B(0, R), K is uniformly continuous. Hence, for the chosen ϵ > 0 there exists δ > 0 such that for every u ∈ C(𝒟; ℝm ) with ‖u − uo ‖ ≤ δ, it holds that as long as |u(z)| ≤ R, then |K(x, z, u(z))−K(x, z, uo (z))| ≤ ϵ, for every x, z ∈ 𝒟. Then one easily sees that |f (u)(x)−f (uo )(x)| ≤ ϵ|𝒟| for every x ∈ 𝒟, (where |𝒟| is the Lebesgue measure of 𝒟) which implies that ‖f (u)−f (uo )‖ ≤ ϵ |𝒟|, therefore, passing to the limit as ϵ → 0, we obtain the continuity of f at uo . For compactness, we have to resort to the Ascoli–Arzela Theorem 1.8.3. Consider any bounded subset A ⊂ C(𝒟; ℝm ); to show that f (A) is relatively compact in C(𝒟; ℝm ) it suffices to show that it is bounded and equicontinuous. Assume that for any u ∈ A it holds that ‖u‖ ≤ c for some c > 0. Boundedness is easy since for any u, such that ‖u‖ ≤ c, it holds that ‖f (u)‖ ≤ max𝒟×𝒟×B(0,c) |K(x, z, v)| |𝒟|. To show the equicontinuity, we will need once more to invoke the uniform continuity of K on the compact set 𝒟 × 𝒟 × B(0, c), to note that for any ϵ > 0, there exists δ > 0 such that |K(x1 , z, u(z)) − K(x2 , z, u(z))| ≤ ϵ for any x1 , x2 , z ∈ 𝒟 with |x1 − x2 | < δ and u ∈ A, so that |f (u)(x1 ) − f (u)(x2 )| ≤ ϵ |𝒟|, leading to the equicontinuity of f (A). We now turn to the solvability of (3.20) using the Leray–Schauder alternative (Theorem 3.3.5). We make the following supplementary assumptions on the data of problem (3.20), (i) |k(x, z)| < c1 , for every x.z ∈ 𝒟, and that (ii) there exists a continuous

130 | 3 Fixed-point theorems and their applications nondecreasing function ψ : [0, R] → ℝ+ such that |fo (s)| ≤ ψ(|s|) for every s ∈ ℝm , R where by | ⋅ | we denote the Euclidean norm in ℝm . Then, assuming that c1 ≤ ψ(R)|𝒟| we can conclude the existence of a solution to (3.20) with ‖u‖ ≤ R. The proof of this claim follows by the Leray–Schauder principle in the following form: We restrict our attention to solutions of (3.20) in C(𝒟; Bℝm (0, R)), with R > 0 chosen as above. We make the following claim: If we manage to show that for each λ ∈ (0, 1) and any solution uλ ∈ C(𝒟; B(0, R)), of uλ (x) = λ ∫ k(x, z)fo (uλ (z)))dz,

x ∈ 𝒟,

(3.21)

𝒟

it holds that ‖uλ ‖ < R (with strict inequality) then a solution u ∈ C(𝒟; B(0, R)) of (3.20) exists. We emphasize the fact that we only require a priori bounds for the solutions of (3.21), and not an actual existence proof, and these a priori bounds should be in terms of the strict inequality ‖uλ ‖ < R. The proof of this claim is immediate by the Leray–Schauder alternative (Theorem 3.3.5), or the Schaeffer fixed-point theorem (see Theorem 3.3.6 and its proof). To conclude the proof, we only need to check whether ‖uλ ‖ < R for any solution of (3.21) holds. Consider any solution of (3.21) in C(𝒟; B(0, R)). Then 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨 󵄨 󵄨 󵄨󵄨uλ (x)󵄨󵄨󵄨 = λ 󵄨󵄨󵄨∫ k(x, z)fo (uλ (z))dz 󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨 𝒟

󵄨 󵄨 ≤ λc1 ∫ ψ(󵄨󵄨󵄨uλ (z)󵄨󵄨󵄨)dz ≤ λc1 ψ(R)|𝒟| ≤ λR < R, 𝒟

so that by taking the supremum over all x ∈ 𝒟 we obtain the required a priori bound ‖uλ ‖ < R. Therefore, by using the above claim we can guarantee the existence of a solution of (3.20) in C(𝒟; B(0, R)). ◁

3.4 Fixed-point theorems for nonexpansive maps We have so far seen that we can get fixed points for maps which are either contractive (Banach’s fixed-point theorem) or if not, continuous and compact8 (Schauder’s fixed-point theorem). It would be interesting to further challenge these assumptions. For instance, what would happen if we relax compactness and contractivity simultaneously? An answer to this question can be given for maps which may not be strict contractions but on the other hand do not increase distances, the so-called family of nonexpansive maps. We start with the definition of nonexpansive maps. 8 or equivalently continuous and defined on compact subsets of a Banach space.

3.4 Fixed-point theorems for nonexpansive maps | 131

Definition 3.4.1 (Nonexpansive map). Let X and Y be two Banach spaces and A ⊂ X. A mapping f : A → Y is called nonexpansive if 󵄩 󵄩󵄩 󵄩󵄩f (x1 ) − f (x2 )󵄩󵄩󵄩Y ≤ ‖x1 − x2 ‖X ,

∀ x1 , x2 ∈ A.

Example 3.4.2 (Projections are nonexpansive maps). If C ⊂ H is a closed convex subset of a Hilbert space, the metric projection PC is a nonexpansive map (see Theorem 2.5.2). ◁ Example 3.4.3 (Nonexpansive maps and derivatives of convex functions). Assume φ : H → ℝ ∪ {+∞} a convex function on a Hilbert space with Lipschitz continuous Fréchet derivative with Lipschitz constant L. Then the map x 󳨃→ x − τDφ(x) is nonexpansive if τ ≤ L2 . Recall the properties of Lipschitz continuous Fréchet derivatives for convex maps (Example 2.3.25) and in particular the co-coercivity estimate (2.14)(iii), ⟨Dφ(z) − Dφ(x), z − x⟩ ≥

1 󵄩󵄩 󵄩2 󵄩Dφ(x) − Dφ(z)󵄩󵄩󵄩 . L󵄩

We have that 󵄩󵄩 󵄩2 󵄩 󵄩2 󵄩󵄩f (x1 ) − f (x2 )󵄩󵄩󵄩 = 󵄩󵄩󵄩(x1 − x2 ) − τ(Dφ(x1 ) − Dφ(x2 ))󵄩󵄩󵄩 󵄩 󵄩2 = ‖x1 − x2 ‖2 + τ2 󵄩󵄩󵄩Dφ(x1 ) − Dφ(x2 )󵄩󵄩󵄩 − 2τ⟨x1 − x2 , Dφ(x1 ) − Dφ(x2 )⟩ 2 󵄩 󵄩2 ≤ ‖x1 − x2 ‖2 + τ(τ − )󵄩󵄩󵄩Dφ(x1 ) − Dφ(x2 )󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖2 , L as long as τ ≤ L2 .

◁

3.4.1 The Browder fixed-point theorem For nonexpansive maps acting on uniformly convex Banach spaces, Browder has proved an important fixed-point theorem (see [31]). A related result was proved independently by Kirk, under the assumption that the Banach space is reflexive and satisfies certain normality conditions (see [74]). Importantly, these results hold for Hilbert spaces, which are known to be examples of uniformly convex Banach spaces (see Example 2.6.9). Theorem 3.4.4 (Browder). Let X be a uniformly convex Banach space, C ⊂ X a nonempty bounded closed convex set, and f : C → C be a nonexpansive map. Then: (i) f has a fixed point. (ii) Fix(f ), the set of fixed points for f is closed and convex. Proof. We provide here for pedagogical reasons a proof in the special case where X = H a Hilbert space (see also [31]).

132 | 3 Fixed-point theorems and their applications (i) Let X = H be a Hilbert space with inner product ⟨⋅, ⋅⟩H =: ⟨⋅, ⋅⟩ and norm ‖ ⋅ ‖ such that ‖x‖2 = ⟨x, x⟩ for every x ∈ X. Fix any zo ∈ C and λ > 1. We define the mapping fλ : C → C by fλ (x) :=

1 1 z + (1 − )f (x). λ o λ

Since 1 󵄩 1 󵄩󵄩 󵄩 󵄩 󵄩󵄩fλ (x1 ) − fλ (x2 )󵄩󵄩󵄩 ≤ (1 − )󵄩󵄩󵄩f (x1 ) − f (x2 )󵄩󵄩󵄩 ≤ (1 − )‖x1 − x2 ‖, λ λ the maps fλ are contractions. Then, by Banach’s contraction mapping theorem, for every λ > 1 the map fλ has a fixed point in C, i. e., there exists xλ ∈ C such that xλ =

1 1 z + (1 − )f (xλ ), λ o λ

which upon rearrangement yields 󵄩󵄩 󵄩 1󵄩 󵄩 2 󵄩󵄩xλ − f (xλ )󵄩󵄩󵄩 = 󵄩󵄩󵄩zo − f (xλ )󵄩󵄩󵄩 ≤ sup ‖x‖ < c, λ λ x∈C with the last estimate arising from the observation that C is bounded and f (xλ ) ∈ C. We select a sequence {λn : n ∈ ℕ} ⊂ (1, ∞) such that λn → ∞, and moving along this sequence the above estimate leads us 󵄩󵄩

lim 󵄩x n→∞󵄩 λn

󵄩 − f (xλn )󵄩󵄩󵄩 = 0.

(3.22)

Consider now the sequence of fixed points {xλn : n ∈ ℕ} ⊂ C, which is bounded, and since by convexity C is relatively weakly compact, there exists a subsequence of elements {xλn : k ∈ ℕ} ⊂ {xλn : n ∈ ℕ} that converges weakly to some xo ∈ C as k k → ∞. We set xt = (1 − t)xo + tf (xo ),

t ∈ (0, 1).

(3.23)

Since f is nonexpansive, and keeping in mind that since X = H is a Hilbert space its norm is expressed in terms of the inner product, we deduce that ⟨xt − f (xt ) − (xλn − f (xλn )), xt − xλn ⟩ k k k 󵄩 󵄩 ≥ ‖xt − xλn ‖2 − 󵄩󵄩󵄩f (xt ) − f (xλn )󵄩󵄩󵄩 ‖xt − xλn ‖ ≥ 0. k k k Letting xλn ⇀ xo , (and recalling that f (xλn ) → xλn , strongly, on account of (3.22)) this k k k inequality implies that ⟨xt − f (xt ), xt − xo ⟩ ≥ 0.

(3.24)

3.4 Fixed-point theorems for nonexpansive maps | 133

Substituting (3.23) in (3.24) and dividing by t > 0 ⟨(1 − t)xo + t f (xo ) − f ((1 − t)xo + t f (xo )), f (xo ) − xo ⟩ ≥ 0, whereby letting t → 0 we deduce that ‖f (xo ) − xo ‖2 ≤ 0, i. e., f (xo ) = xo . Hence, there exists at least a fixed point of f . (ii) That the set of fixed points of f is closed follows from the fact that f is continuous (since it is nonexpansive). To show the convexity of the set of fixed points, we reason as follows: Let x1 , x2 be two fixed points of f and let xλ = (1−λ)x1 +λ x2 , λ ∈ [0, 1]. We will show that xλ is also a fixed point of f . Indeed, we have that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩f (xλ ) − x1 󵄩󵄩󵄩 = 󵄩󵄩󵄩f (xλ ) − f (x1 )󵄩󵄩󵄩 ≤ ‖xλ − x1 ‖ = λ ‖x1 − x2 ‖,

(3.25)

󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩f (xλ ) − x2 󵄩󵄩󵄩 = 󵄩󵄩󵄩f (xλ ) − f (x2 󵄩󵄩󵄩 ≤ ‖xλ − x2 ‖ = (1 − λ) ‖x1 − x2 ‖.

(3.26)

and

It follows that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩󵄩f (xλ ) − x1 󵄩󵄩󵄩 + 󵄩󵄩󵄩f (xλ ) − x2 󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖ = 󵄩󵄩󵄩x1 − f (xλ ) + f (xλ ) − x2 󵄩󵄩󵄩 󵄩 󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩f (xλ ) − x1 󵄩󵄩󵄩 + 󵄩󵄩󵄩f (xλ ) − x2 󵄩󵄩󵄩, so that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩x1 − f (xλ )󵄩󵄩󵄩 + 󵄩󵄩󵄩f (xλ ) − x2 󵄩󵄩󵄩 = ‖x1 − x2 ‖. Since H is a Hilbert space (hence, strictly convex), it follows9 that x1 − f (xλ ) = k(f (xλ ) − x2 ) for some k > 0, therefore, upon rearrangement, f (xλ ) = (1 − μ)x1 + μx2 ,

μ=

k . k+1

(3.27)

Combining (3.27) with (3.25), we conclude that μ ≤ λ while combining (3.27) with (3.26) we conclude that μ ≥ λ; hence, μ = λ. Therefore, (3.27) yields f (xλ ) = (1 − λ)x1 + λx2 from which it follows that xλ = f (xλ ), i. e., xλ is also a fixed point of f . The following example shows that Theorem 3.4.4 fails in a general Banach space. Example 3.4.5. Let Bc0 (0, 1) be the closed unit ball of c0 , the space of all null sequences x = {xn : n ∈ ℕ}, limn xn = 0, endowed with the norm ‖x‖ = supi |xi |. We define f : Bc0 (0, 1) → Bc0 (0, 1), by f (x) = f (x1 , x2 , . . .) = (1, x1 , x2 , . . .). Then ‖f (x) − f (z)‖ = ‖x − z‖, for every x, z ∈ c0 , however, the equation f (x) = x is satisfied only if x = (1, 1, . . .) which is not in c0 . ◁ 9 See Theorem 2.6.5(i󸀠 ) and apply it to x1 − f (xλ ) and x2 − f (xλ ).

134 | 3 Fixed-point theorems and their applications 3.4.2 The Krasnoselskii–Mann algorithm Given that a fixed point exists for a nonexpansive operator, how can we approximate it? The Krasnoselskii–Mann algorithm provides an iterative scheme that allows this, in the special case where X = H is a Hilbert space (see, e. g., [20]). The crucial observation behind this scheme is that even though the iterative procedure for a nonexpansive operator f may not converge to a fixed point of this operator,10 nevertheless the operator fλ (x) = (1 − λ)x + λf (x), for any λ ∈ (0, 1) when iterated, weakly converges to a fixed point.11 In particular, we introduce the following definition. Definition 3.4.6 (Averaged operators). A map fλ : H → H which can be expressed as fλ = (1 − λ)I + λf , for some λ ∈ (0, 1), where f is a nonexpansive operator is called a λ-averaged operator.12 Example 3.4.7 (Projections are averaged operators). The projection operator PC on a closed and convex subset C ⊂ H, which is 1/2-averaged since 2PC − I is nonexpansive. ◁ Example 3.4.8 (Derivatives of convex maps and averaged operators). If φ : H → ℝ ∪ {+∞} is a convex map with Lipschitz continuous Fréchet derivative Dφ, with Lipschitz constant L, then the map f : H → H defined by f (x) = x − τDφ(x), is τL -averaged as 2 τL long as 2 < 1. Indeed, express I − τDφ = (1 − ν)I + νf so that f = I − τν Dφ, which is nonexpansive as long as τν ≤ L2 , so ν ≥ τL and since ν must be in (0, 1) we must choose τL < 1 (see 2 2 also Example 3.4.3). ◁ Averaged operators have many important properties, such as, e. g., that finite compositions of averaged operators are also averaged (a proof of this useful result can be found in Section 3.7.2, Lemma 3.7.2). More details concerning averaged operators and their applications in the study of various numerical algorithms will be given in Chapter 4 (see, e. g., extensions of proximal optimization methods in Section 4.8.2). At this point, we are going to state and prove an important result concerning averaged operators, and in particular that iterations of averaged operators converge weakly to a fixed point. Theorem 3.4.9 (Krasnoselskii–Mann). Let C ⊂ H a closed convex subset of a Hilbert space and f : C → C a nonexpansive operator such that Fix(f ) ≠ 0, where Fix(f ) is the set of fixed points of f . Consider a sequence {λn : n ∈ ℕ} ⊂ [0, 1] such that ∑n∈ℕ λn (1 − λn ) = 10 even if such a fixed point exists. 11 Note that fλ and f share the same fixed points. 12 Note that nonexpansive operators need not be averaged operators; as an example, consider f = −I.

3.4 Fixed-point theorems for nonexpansive maps | 135

+∞, and construct the sequence {xn : n ∈ ℕ} such that xn+1 = xn + λn (f (xn ) − xn ),

n ∈ ℕ,

(3.28)

with x1 ∈ C, arbitrary. Then f (xn ) − xn → 0 and xn ⇀ xo , for some xo ∈ Fix(f ), for any initial condition x1 . Proof. The proof follows in four steps: 1. We first show that the sequence defined in (3.28) converges weakly up to subsequences to some x ∈ C. We pick any (arbitrary) xo ∈ Fix(f ) and consider the evolution of xn − xo . We easily see that xn+1 − xo = (1 − λn )(xn − xo ) + λn (f (xn ) − xo )

= (1 − λn )(xn − xo ) + λn (f (xn ) − f (xo )),

(3.29)

where we used the fact that f (xo ) = xo . We further observe that in a Hilbert space for any elements z1 , z2 ∈ H it holds13 that 󵄩󵄩 󵄩2 2 2 2 󵄩󵄩(1 − λ)z1 + λz2 󵄩󵄩󵄩 = (1 − λ)‖z1 ‖ + λ‖z2 ‖ − λ(1 − λ)‖z1 − z2 ‖ , which applied to λ = λn , z1 = xn − xo , z2 = f (xn ) − f (xo ) and combined with (3.29) yields that 󵄩 󵄩2 ‖xn+1 − xo ‖2 = 󵄩󵄩󵄩(1 − λn )(xn − xo ) + λn (f ((xn ) − f (xo ))󵄩󵄩󵄩 󵄩 󵄩2 󵄩 󵄩2 = (1 − λn )‖xn − xo ‖2 + λn 󵄩󵄩󵄩f (xn ) − f (xo )󵄩󵄩󵄩 − λn (1 − λn )󵄩󵄩󵄩xn − f (xn )󵄩󵄩󵄩 󵄩 󵄩2 ≤ ‖xn − xo ‖2 − λn (1 − λn )󵄩󵄩󵄩xn − f (xn )󵄩󵄩󵄩 ,

(3.30)

where for the last estimate we used the fact that f is nonexpansive. This estimate shows that ‖xn+1 − xo ‖2 ≤ ‖xn − xo ‖2 ,

for some xo ∈ Fix(f ).

(3.31)

A sequence {xn : n ∈ ℕ} ⊂ C satisfying (3.31) is a called a Fejér sequence with respect to Fix(f ). It can easily be seen that this sequence is bounded, and thus admits a subsequence {xnk : k ∈ ℕ} ⊂ C such that xnk ⇀ x for some x ∈ H. Since C is closed and convex, it is also weakly sequentially closed; hence, x ∈ C. 2. We will show that x ∈ Fix(f ). We claim (to be proved in step 3) that f (xn ) − xn → 0. 13 by an elementary expansion of the inner products.

(3.32)

136 | 3 Fixed-point theorems and their applications If claim (3.32) holds then, 󵄩2 󵄩2 󵄩 󵄩󵄩 2 󵄩󵄩f (x) − x󵄩󵄩󵄩 = 󵄩󵄩󵄩xnk − f (x)󵄩󵄩󵄩 − ‖xnk − x‖ − 2⟨xnk − x, x − f (x)⟩ 󵄩2 󵄩2 󵄩 󵄩 = 󵄩󵄩󵄩xnk − f (xnk )󵄩󵄩󵄩 + 󵄩󵄩󵄩f (xnk ) − f (x)󵄩󵄩󵄩 + 2⟨xnk − f (xnk ), f (xnk ) − f (x)⟩ − ‖xnk − x‖2 − 2⟨xnk − x, x − f (x)⟩ ≤ ‖xnk − f (xnk )‖2 + ‖xnk − x‖2 + 2⟨xnk − f (xnk ), f (xnk ) − f (x)⟩

− ‖xnk − x‖2 − 2⟨xnk − x, x − f (x)⟩ 󵄩 󵄩2 = 󵄩󵄩󵄩xnk − f (xnk )󵄩󵄩󵄩 + 2⟨xnk − f (xnk ), f (xnk ) − f (x)⟩ − 2⟨xnk − x, x − f (x)⟩ → 0, as k → ∞, since(by claim (3.32)) xnk − f (xnk ) → 0, xnk − x ⇀ 0 and f (xnk ) − f (x) = f (xnk ) − xnk + xnk − f (x) ⇀ 0 + x − f (x). Hence, f (x) = x and x ∈ Fix(f ). 3. We now show claim (3.32). To show that f (xn ) − xn → 0, we proceed as follows: Adding the estimates (3.30) over n, we conclude that n−1

n−1

n−1

k=1

k=1

k=1

󵄩 󵄩2 ∑ ‖xk+1 − xo ‖2 ≤ ∑ ‖xk − xo ‖2 − ∑ λk (1 − λk )󵄩󵄩󵄩f (xk ) − xk 󵄩󵄩󵄩 ,

which upon rearrangement leads to n−1

󵄩 󵄩2 ∑ λk (1 − λk )󵄩󵄩󵄩f (xk ) − xk 󵄩󵄩󵄩 ≤ ‖x1 − xo ‖2 − ‖xn − xo ‖2 ≤ ‖x1 − xo ‖2 .

k=1

2 2 Upon passing to the limit n → ∞ yields ∑∞ k=1 λk (1 − λk )‖f (xk ) − xk ‖ ≤ ‖x1 − xo ‖ . On the other hand, it is easy to see that the sequence {‖f (xn ) − xn ‖ : ∈ ℕ} is decreasing since (using (3.28)),

󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩f (xn+1 ) − xn+1 󵄩󵄩󵄩 = 󵄩󵄩󵄩f (xn+1 ) − f (xn ) + (1 − λn )(f (xn ) − xn )󵄩󵄩󵄩 󵄩 󵄩 ≤ ‖xn+1 − xn ‖ + (1 − λn )󵄩󵄩󵄩f (xn ) − xn 󵄩󵄩󵄩 󵄩 󵄩 󵄩 󵄩 = 󵄩󵄩󵄩λn (f (xn ) − xn )󵄩󵄩󵄩 + (1 − λn )󵄩󵄩󵄩f (xn ) − xn 󵄩󵄩󵄩 󵄩 󵄩 = 󵄩󵄩󵄩f (xn ) − xn 󵄩󵄩󵄩. Combining this with the fact that ∑∞ n=1 λn (1 − λn ) = ∞ we conclude that limn ‖f (xn ) − xn ‖ = 0; hence, f (xn ) − xn → 0. 4. Furthermore, one can show that the whole sequence weakly converges to some x ∈ C. For that, it suffices to show that every weakly convergent subsequence has the same limit (recall the Urysohn property, Remark 1.1.51). Indeed, assume the existence of two subsequences {xnk : k ∈ ℕ} ⊂ C and {xmk : k ∈ ℕ} ⊂ C such that xnk ⇀ z1 ∈ C, xmk ⇀ z2 ∈ C, with z1 ≠ z2 . We have already shown that it must hold that z1 , z2 ∈ Fix(f ).

3.4 Fixed-point theorems for nonexpansive maps | 137

By the Fejér property, it holds that ‖xn+1 −xo ‖ ≤ ‖xn −xo ‖ for every n ∈ ℕ and xo ∈ Fix(f ), so that {‖xn − xo ‖2 : n ∈ ℕ} converges. Expressing ‖xn − xo ‖2 = ‖xn ‖2 − 2⟨xn , xo ⟩ + ‖xo ‖2 , we see that {‖xn ‖2 − 2⟨xn , xo ⟩ : n ∈ ℕ} converges for every xo ∈ Fix(f ). We apply this for xo = z1 and xo = z2 , so that the sequences {‖xn ‖2 − 2⟨xn , zi ⟩ : n ∈ ℕ}, i = 1, 2 converge and, therefore, their difference {⟨xn , z1 − z2 ⟩ : n ∈ ℕ} converges also. Assume that ⟨xn , z1 − z2 ⟩ → L for some L ∈ ℝ as n → ∞. Passing to the subsequences {xnk : k ∈ ℕ} and {xmk : k ∈ ℕ} and subtracting, we have that ⟨xnk − xmk , z1 − z2 ⟩ → 0 as k → ∞, and noting that xnk − xmk ⇀ z1 − z2 , passing to the limit as k → ∞, we conclude that ⟨z1 − z2 , z1 − z2 ⟩ = 0, so that z1 = z2 . Example 3.4.10 (Krasnoselskii–Mann scheme for averaged operators). The Krasnoselskii–Mann (KM) scheme shows that iterates of averaged operators converge to a fixed point of the operator (if any). This is particularly important by noting that the composition of (finitely many) averaged operators remains an averaged operator, of course, with a different averaging constant. For example, if fi : H → H are 2 −2ν1 ν2 (for a proof of this fact, νi -averaged, i = 1, 2, then f1 ∘ f2 is ν-averaged for ν = ν1 +ν 1−ν1 ν2 see Lemma 3.7.2 in the Appendix of this chapter). Since averaged operators are also nonexpansive, one may consider a version of the Krasnoselskii–Mann scheme for averaged operators, which allows for a modification of the convergence criterion. In particular, let f be a ν-averaged operator and consider the Krasnoselskii–Mann iteration xn+1 = xn +ρn (f (xn )−xn ) for some ρn ∈ (0, 1). Since by definition f = (1 − ν)I + νf0 , where f0 is a nonexpansive operator, the iteration scheme can be expressed as xn+1 = xn + νρn (f0 (xn ) − xn ), which is the standard KM scheme for 1 λn = νρn . This scheme converges as long as ∑∞ ◁ n=1 ρn (1 − νρn ) = ∞ for ρn ∈ (0, ν ). Example 3.4.11 (Convergence of the gradient descent method). A nice application of the Krasnoselskii–Mann scheme is to the convergence of the explicit gradient method xn+1 = xn − τDφ(xn ) for the minimization of a Fréchet differentiable convex function with Lipschitz derivative. We start by noting that the mapping x 󳨃→ x − τDφ(x) is nonexpansive if τ ≤ 2 (see Example 3.4.3). In the interest of faster convergence of the proposed numerL ical scheme, we choose the higher value τ = L2 . However, even then the mapping x 󳨃→ x − L2 Dφ(x) is not a contraction; hence, the convergence of its iterations cannot be guaranteed by Banach’s contraction theorem. It is exactly to this point that the Krasnoselskii–Mann iteration scheme can be applied for a suitable mapping, we will call f0 . Consider now any τ ∈ (0, L2 ), and the operator f : H → H defined by f (x) = x − τDφ(x). Choosing λ = τL , we can express f (x) = (1 − λ)x + λf0 (x), for 2 λ ∈ (0, 1); hence, f is an averaged operator and an application of the Krasnoselskii– Mann scheme provides the weak convergence of its iterates. The same argument covers the weak convergence of the explicit gradient method for variable step sizes τn . The implicit gradient method, xn+1 = xn − τDφ(xn+1 ), displays unconditional convergence properties, even in the absence of Lipschitz continuity of the gradient, and impor-

138 | 3 Fixed-point theorems and their applications tantly even in the absence of smoothness (by replacing the gradient by the subdifferential). This theme will be discussed in detail in Section 4.8. ◁ The Krasnoselskii–Mann iterative algorithm can be extended in the case where X is a Banach space which is uniformly convex and either has a Fréchet differentiable norm ([96]), or satisfies the so-called Opial condition, according to which if xn ⇀ x then lim supn ‖xn − x‖ ≤ lim supn ‖xn − z‖, for every z ∈ X. The strong convergence requires stronger conditions on the operator f even in Hilbert spaces.

3.5 A fixed-point theorem for multivalued maps A fundamental result concerning monotone single or multivalued operators (not necessarily maximal) is the Debrunner-Flor extension theorem [86]. This theorem finds important applications in the theory of monotone type operators, and in particular in the proof of surjectivity results (see Chapter 9). Theorem 3.5.1 (Debrunner-Flor Lemma). Let X be a Banach space, C ⊂ X a nonempty compact and convex set and f : C → X ⋆ a continuous map. Let M ⊂ C × X ⋆ be a monotone subset, i. e., such that ⟨x⋆ − z⋆ , x − z⟩ ≥ 0,

∀ (x, x⋆ ), (z, z⋆ ) ∈ M.

(3.33)

Then, there exists xo ∈ C such that ⟨x⋆ − f (xo ), x − xo ⟩ ≥ 0

∀(x, x⋆ ) ∈ M.

(3.34)

Proof. Suppose that no such xo ∈ C exists, i. e., (3.34) has no solution. We define the set U(x, x⋆ ) = {z ∈ C : ⟨x⋆ − f (z), x − z⟩ < 0}. The mapping z 󳨃→ ⟨x⋆ − f (z), x − z⟩ is continuous, so the set U(x, x⋆ ) is relatively open in C. Since (3.34) has no solution, the collection of U(x, x⋆ ), for all (x, x⋆ ) ∈ M, forms a cover of the set C. Since C is compact, there exists a finite subcover, i. e., there exists ⋆ an m ∈ ℕ, and (xi , x⋆i ) ∈ M, i = 1, . . . , m such that C ⊂ ⋃m i=1 U(xi , xi ). Let ψ1 , . . . , ψm be a partition of unity, subordinate to this cover of C, that is ψi is continuous on C with supp( ψi ) ⊂ U(xi , x⋆i ), and m

∑ ψi (z) = 1, i=1

0 ≤ ψi (z) ≤ 1,

∀ z ∈ C, i = 1, 2, . . . , m.

(3.35)

Let C1 = conv({x1 , x2 , . . . , xm }), the convex hull of {x1 , x2 , . . . , xm }. For z ∈ C1 we consider m ⋆ the maps defined by z 󳨃→ p(z) := ∑m i=1 ψi (z)xi and z 󳨃→ q(z) := ∑i=1 ψi (z)xi . The map p

3.5 A fixed-point theorem for multivalued maps | 139

is continuous and by (3.35) maps the set C1 into itself. Note that C1 is a finite dimensional compact convex set which is homeomorphic to a finite dimensional ball. Thus, by Brouwer’s fixed-point theorem p has a fixed point, i. e., there exists zo ∈ C1 such that p(zo ) = zo . Letting aij = ⟨x⋆i − f (zo ), xj − zo ⟩ we easily see, using also (3.33) that aij + aji = aii + ajj − ⟨x⋆i − x⋆j , xi − xj ⟩ ≤ aii + ajj .

(3.36)

Now, since p(zo ) = zo , we have14 m

m

0 = ⟨q(zo ) − f (zo ), p(zo ) − zo ⟩ = ⟨∑ ψi (zo )x⋆i − f (zo ), ∑ ψj (zo )xj − zo ⟩ m

m

i,j=1

i,j=1

i=1

= ∑ ψi (zo )ψj (zo )aij = ∑ ψi (zo )ψj (zo )

aij + aji 2

j=1

(3.37)

.

From (3.36) it follows that m

0 ≤ ∑ ψi (zo )ψj (zo ) i,j=1

aii + ajj 2

.

(3.38)

If ψi (zo )ψj (zo ) ≠ 0, then zo ∈ U(xi , x⋆i ) ∩ U(xj , x⋆j ). By the construction of U(x, x⋆ ), this means that aii < 0 and ajj < 0, which contradicts (3.38). Therefore, from (3.37), we conclude that ψi (zo ) = 0 for all 1, 2, . . . , m, however, since zo ∈ C1 ⊂ C which contradicts (3.35). This concludes the proof. Theorem 3.5.2 (Kakutani). Let X be a reflexive Banach space and C ⊂ X nonempty compact and convex. Let f : C → 2C be an upper semicontinuous mapping such that f (x) is a nonempty, closed and convex set for all x ∈ C. Then, there exists xo ∈ C such that xo ∈ f (xo ). Proof. Suppose that 0 ∉ x − f (x) for all x ∈ C. From the Hahn-Banach separation theorem there exists x⋆ ∈ X ⋆ with ‖x⋆ ‖X ⋆ = 1, and δ > 0 such that ⟨x⋆ , z⟩ > δ for all z ∈ x − f (x). For any x⋆ ∈ X ⋆ we define W(x⋆ ) = {x ∈ C : ⟨x⋆ , z⟩ > 0, ∀ z ∈ x − f (x)}. For each x⋆ ∈ X ⋆ such that ‖x⋆ ‖X ⋆ = 1, let U(x⋆ ) = {z ∈ X : ⟨x⋆ , z⟩ > 0}. So, x ∈ W(x⋆ ) if and only if x ∈ C and x − f (x) ⊂ U(x⋆ ). m ⋆ ⋆ 14 Note that ∑m i=1 ψi (zo )xi − f (zo ) = ∑i=1 ψi (zo )(xi − f (zo )) by the properties of ψi , i = 1, . . . , m, and similarly for the other sum.

140 | 3 Fixed-point theorems and their applications We rephrase the result of the Hahn-Banach separation theorem obtained above, using this notation: For any x ∈ C, there exists15 x⋆ = x⋆ [x] ∈ X, with ‖x⋆ ‖X ⋆ = 1 and δ > 0 such that x − f (x) + δ B(0, 1) ⊂ U(x⋆ ). Since, f is upper semicontinuous, so is −f , therefore, for 0 < ϵ < δ2 we have −f (z) ⊂ −f (x) + δ2 B(0, 1) for each z ∈ (x + ϵ B(0, 1)) ∩ C. For each such z we have δ δ z − f (z) ⊂ x + B(0, 1) − f (x) + B(0, 1) ⊂ U(x⋆ ), 2 2 that is, (x + ϵ B(0, 1)) ∩ C ⊂ W(x⋆ (x)). So every point x ∈ C lies in the interior of some W(x⋆ ). It then follows, that the sets {int(W(x⋆ (x))) : x ∈ C} form an open cover of C. By the compactness of C there exists a finite sub-cover {int(W(x⋆i )) : i = 1, . . . , n}, and a partition of unity {ψi : i = 1, . . . , n} subordinate to this cover. Define, n

g(x) = ∑ ψi (x)x⋆i ,

x ∈ C.

i=1

This function is continuous from C into X ⋆ and n

⟨g(x), x󸀠 ⟩ = ∑ ψi (x) ⟨x⋆i , x󸀠 ⟩ > 0, i=1

∀ x ∈ C, and x󸀠 ∈ x − f (x).

(3.39)

Applying the Debrunner-Flor Lemma (see Theorem 3.5.1) for the monotone set M = C × {0} and for the function −g(x), there exists xo ∈ C such that ⟨g(xo ), x − xo ⟩ ≥ 0,

∀ x ∈ C.

In particular, this is true for x = x󸀠o where x󸀠o is any element of f (xo ) ⊂ C, so the element zo = xo −x󸀠o ∈ xo −f (xo ) satisfies the inequality ⟨g(xo ), zo ⟩ ≤ 0, which contradicts (3.39). This completes the proof.

3.6 The Ekeland variational principle and Caristi’s fixed-point theorem Caristi’s fixed point theorem [33] is one of the most general fixed point theorems available, in the sense that it requires practically no properties on the mapping apart from a very general contraction like condition, called the Caristi condition. As such it is very useful in a number of applications. Furthermore, it is equivalent to an extremely useful result (produced independently) called the Ekeland variational principle [60, 61] which we will need in a number of occasions. These important results have been proved in various ways, our approach is inspired by [41] and [85]. 15 Note that by x⋆ [x] here we denote the dependence of x⋆ on x and not the action of the functional x⋆ on x; x⋆ [x] ∈ X ⋆ !

3.6 The Ekeland variational principle and Caristi’s fixed-point theorem

| 141

3.6.1 The Ekeland variational principle The Ekeland variational principle is an important tool in nonlinear analysis, which provides approximations to minima in the absense of the conditions required for applying the Weierstrass theorem. While derived independently of Caristi’s theorem, it is in fact equivalent to it (see e. g. [65]). Here we decide to prove first the Ekeland variational principle (following the approach of [41]) and then use the Ekeland variational principle to show the Caristi fixed point theorem. Theorem 3.6.1 (Ekeland variational principle). Let X be a complete metric space and F : X → ℝ be a lower semicontinuous functional which is bounded below. For every ϵ, δ > 0, if xδ ∈ X satisfies F(xδ ) ≤ infx∈X F(x) + δ then, there exists xϵ,δ ∈ X such that F(xϵ,δ ) ≤ F(xδ ), 1 d(xδ , xϵ,δ ) ≤ , ϵ F(xϵ,δ ) < F(z) + ϵδd(z, xϵ,δ ),

(3.40) ∀ z ∈ X, z ≠ xϵ,δ .

Proof. The proof consists in constructing a sequence of approximations of the required point xϵ,δ , that converges to the required point. We proceed in 2 steps. 1. Starting from x0 = xδ , we construct the following sequence of sets Sn ⊂ X and elements xn+1 ∈ Sn : Sn = {z ∈ X : F(z) ≤ F(xn ) − ϵδd(z, xn )}, 1 xn+1 ∈ Sn : F(xn+1 ) − inf F ≤ ϵn , ϵn = (F(xn ) − inf F). Sn Sn 2

(3.41)

The above sequence is well defined. Clearly Sn ≠ 0 since xn ∈ Sn , for every n ∈ ℕ. Moreover, the existence of xn+1 arises from the very definition of the infimum, such a point exists for every choice of ϵn ≥ 0, the particular choice is made to guarantee the desirable property that {xn : n ∈ ℕ} is a Cauchy sequence. To see that the proposed ϵn satisfies ϵn ≥ 0, simply note that by the definition of Sn for every z ∈ Sn , we have F(z) ≤ F(xn ) − ϵδd(z, xn ) ≤ F(xn ) hence, infz∈Sn F(z) ≤ F(xn ). If it happens that infz∈Sn F(z) = F(xn ), then we simply set xn+1 = xn and the iteration stops. The following observations hold for {xn : n ∈ ℕ}: (a) The sequence {F(xn ) : n ∈ ℕ} is decreasing. Indeed, by construction since xn+1 ∈ Sn , we have by the definition of Sn that 0 ≤ ϵδd(xn+1 , xn ) ≤ F(xn ) − F(xn+1 ) hence, the monotonicity claim. Since by assumption {F(xn ) : n ∈ ℕ} is bounded below it is convergent. Therefore F(xn ) → γ, for a suitable γ. (b) By the definition of Sn , once more, we have that ϵδd(xn+1 , xn ) ≤ F(xn ) − F(xn+1 ),

∀ n ∈ ℕ.

Adding over all n, n + 1, . . . , m, and using the triangle inequality we obtain that ϵδd(xm , xn ) ≤ F(xn ) − F(xm ),

∀ m > n,

(3.42)

142 | 3 Fixed-point theorems and their applications and passing to the limit as n, m → ∞, since F(xn ) → γ, and F(xm ) → γ we conclude that d(xn , xm ) → 0 hence, {xn : n ∈ ℕ} is Cauchy. By the completeness of X, there exists zo ∈ X such that xn → zo . We claim that zo ∈ X is the required point, xϵ,δ . 2. Note that since F is lower semicontinuous it holds that F(zo ) ≤ lim infn F(xn ). We now take the limit inferior of the second relation in (3.41) as n → ∞, and setting βn := infSn F we conclude that F(zo ) ≤ γ = lim inf F(xn ) ≤ lim inf βn . n

n

(3.43)

We check one by one that zo satisfies the stated properties in (3.40). (a) Since {F(xn ) : n ∈ ℕ} is decreasing and x0 = xδ , clearly F(xn ) ≤ F(xδ ) and taking the limit as n → ∞, using the lower semicontinuity of F yields F(zo ) ≤ F(xδ ), which is (3.40)(a). (b) By (3.42) setting n = 0, bringing the −F(xm ) term onto the left-hand side, taking the limit inferior as m → ∞, using the lower semicontinuity of F and rearranging, we have that ϵδd(zo , xδ ) ≤ F(xδ ) − F(zo ) ≤ F(xδ ) − inf F ≤ δ, X

where the second inequality follows by the trivial estimate F(zo ) ≥ infX F and the third by the choice of xδ . This proves (3.40)(b). (c) Suppose that zo does not satisfy the third condition in (3.40). Then there exists ̂z ∈ X, ̂z ≠ zo , such that F(zo ) ≥ F(ẑ) + ϵδd(̂z, zo ).

(3.44)

On the other hand by (3.42) fixing n arbitrary, rearranging and taking the limit inferior as m → ∞, using also the lower semicontinuity of F, yields ϵδd(zo , xn ) ≤ F(xn ) − F(zo ).

(3.45)

Combining (3.44) with (3.45), we obtain that F(̂z) + ϵδd(̂z, zo ) ≤ F(zo ) ≤ F(xn ) − ϵδd(zo , xn ), so that F(̂z) ≤ F(xn ) − ϵδ(d(zo , xn ) + d(̂z, zo )) ≤ F(xn ) − ϵδd(̂z, xn ),

(3.46)

where for the last estimate we used the triangle inequality. Since n ∈ ℕ is arbitrary, (3.46) implies that ̂z ∈ Sn for every n ∈ ℕ, therefore, ̂z ∈ ⋂n∈ℕ Sn . Since for any n ∈ ℕ, ̂z ∈ Sn , clearly F(̂z) ≥ infSn F = βn , and taking the limit inferior over n, we have that F(̂z) ≥ lim infn βn , which combined with (3.43) yields F(zo ) ≤ F(̂z) which is in

3.6 The Ekeland variational principle and Caristi’s fixed-point theorem

| 143

contradiction with (3.44) (since we have that ̂z ≠ zo ). Therefore, zo satisfies also the third condition in (3.40). Combining (a), (b) and (c) above, we conclude that zo has the required properties and the proof is complete. A very common choice when applying the Ekeland variational principle is ϵ =

1 . √δ

3.6.2 Caristi’s fixed-point theorem We now, in the spirit of [85], use the Ekeland variational principle to prove a very general fixed-point theorem, the Caristi fixed-point theorem. Theorem 3.6.2 (Caristi). Let (X, d) be a complete metric space, ϕ : X → ℝ ∪ {+∞} a proper lower semicontinuous function bounded from below and f : X → 2X \ {0} a multivalued map that satisfies the Caristi property,16 d(x, z) ≤ ϕ(x) − ϕ(z),

∀ x ∈ X, ∀ z ∈ f (x).

(3.47)

Then f has a fixed point in X, i. e., there exists xo ∈ X such that xo ∈ f (xo ). Proof. We will use the Ekeland variational principle on ϕ, with ϵ = δ = 1. This provides the existence of an xo ∈ X, such that ϕ(xo ) < ϕ(x) + d(x, xo ),

∀ x ≠ xo .

(3.48)

This is the required fixed point of f . Indeed, suppose that xo ∈ ̸ f (xo ). Consider any z ∈ f (xo ) (which clearly satisfies z ≠ xo ). Since f satisfies the Caristi property, applying it for z ∈ f (xo ) and x = xo ∈ X, we have that ϕ(z) ≤ ϕ(xo ) − d(xo , z).

(3.49)

Combining (3.48) (applied for x = z ≠ xo ) and (3.49), we deduce that ϕ(z) ≤ ϕ(xo ) − d(xo , z) < ϕ(z), which is a contradiction. 16 In the single valued case, a contraction map f , with contraction constant ϱ < 1 satisfies the Caristi 1 property upon choosing ϕ(x) = 1−ϱ d(x, f (x)). In this respect, the Caristi fixed-point theorem may be considered as a generalization to the Banach fixed-point theorem.

144 | 3 Fixed-point theorems and their applications 3.6.3 Applications: approximation of critical points One of the important applications of the Ekeland variational principle is in obtaining approximate critical points for F, as can be seen by rearranging the third relation in (3.40) while also assuming the X is a Banach space and F is differentiable. We must introduce the notion of Palais–Smale sequences. Definition 3.6.3 (Palais–Smale sequences and Palais–Smale condition). Let F : X → ℝ be a C 1 (X; ℝ) functional.17 (i) A sequence {xn : n ∈ ℕ} is a Palais–Smale sequence for the functional if {F(xn ) : n ∈ ℕ} is bounded and DF(xn ) → 0 (strongly). (ii) The functional F satisfies the Palais–Smale condition if any Palais–Smale sequence has a (strongly) convergent subsequence. The next proposition shows the construction of approximate critical points using the Ekeland variational principle. Proposition 3.6.4. Assume that X is a Banach space and F ∈ C 1 (X; ℝ), bounded below that satisfies the Palais–Smale condition. Then F has a minimizer xo ∈ X such that DF(xo ) = 0. Proof. We apply the Ekeland variational principle (EVI) for the choice ϵ = n and δ = n12 , n ∈ ℕ and denote the corresponding sequences xδ = zn and xϵ,δ = xn . The sequence {zn : n ∈ ℕ} is a minimizing sequence and since by the first two conditions of the EVI, F(xn ) ≤ F(zn ) and ‖zn − xn ‖ ≤ n1 , so is {xn : n ∈ ℕ} (hence, {F(xn ) : n ∈ ℕ} ⊂ ℝ is bounded), and if any of the two converge they should have the same limit. By the third condition in (3.40), choosing z = xn + th, for t > 0 and h ∈ X arbitrary, since X is a Banach space and F ∈ C 1 (X; ℝ), passing to the limit as t → 0+ , we conclude that ⟨DF(xn ), h⟩ < n1 ‖h‖. As h ∈ X is arbitrary, setting −h ∈ X in the place of h next, we also have −⟨DF(xn ), h⟩ < n1 ‖h‖, which allows us to conclude that |⟨DF(xn ), h⟩| < n1 ‖h‖ for every h ∈ X, so upon dividing by ‖h‖ and taking the supremum over all ‖h‖ ≤ 1 we obtain that ‖DF(xn )‖X ⋆ < n1 , from which we find that DF(xn ) → 0 in X ⋆ . We have thus constructed using EVI a minimizing sequence {xn : n ∈ ℕ}, i. e., F(xn ) → γ = infx∈X F(x) with the extra property DF(xn ) → 0. At this point, there is no guarantee that the minimizing sequence admits a convergent subsequence on order to proceed with the direct method. However, {xn : n ∈ ℕ} is a Palais–Smale sequence, and since by assumption it satisfies the Palais–Smale condition we conclude that there exists a subsequence {xnk : k ∈ ℕ} and a xo ∈ X such that xnk → xo in X. Clearly, xo is a minimizer for F, while DF(xnk ) → DF(xo ) = 0, so xo is a critical point of F. 17 that is, Fréchet differentiable with continuous derivative.

3.7 Appendix | 145

3.7 Appendix 3.7.1 The Gronwall inequality The Gronwall inequality is a very useful tool in the study of differential equations. In a fairly general form, it can be stated as the following. Proposition 3.7.1. Let g be a function such that t

g(t) ≤ c + ∫(a(τ)g(τ) + b(τ))dτ,

t ∈ [0, T],

0

for some positive valued integrable functions a, b. Then, t

τ

t

g(t) ≤ (c + ∫ b(τ)e− ∫0 a(s)ds dτ) e∫0 a(τ)dτ . 0

3.7.2 Composition of averaged operators We show that the class of ν-averaged operators is stable under compositions (see [47]). Lemma 3.7.2. Let H be a Hilbert space. Suppose that fi : H → H are νi -averaged, i = 1, 2. Then f1 ∘ f2 is ν-averaged for ν =

ν1 +ν2 −2ν1 ν2 . 1−ν1 ν2

Proof. An operator f is ν-averaged if and only if 1 − ν 󵄩󵄩 󵄩󵄩 󵄩2 󵄩2 2 󵄩󵄩f (x1 ) − f (x2 )󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖ − 󵄩(I − f )(x1 ) − (I − f )(x2 )󵄩󵄩󵄩 , ν 󵄩

∀ x1 , x2 ∈ H.

(3.50)

To see that, observe that f is ν-averaged if and only if (1 − ν1 )I + ν1 f is nonexpansive. To

obtain the direction of the equivalence we need, it suffices to show that if (3.50) holds, then (1 − ν1 )I + ν1 f is nonexpansive, i. e.,

󵄩󵄩 󵄩󵄩󵄩2 1 1 󵄩󵄩 2 󵄩󵄩(1 − )(x1 − x2 ) + (f (x1 ) − f (x2 ))󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖ , 󵄩󵄩 󵄩󵄩 ν ν holds for any x1 , x2 ∈ H. This follows easily by straightforward expansion of the two

conditions using the fact that the norm is generated by the inner product.

146 | 3 Fixed-point theorems and their applications We now try to check that (3.50) holds for f = f1 ∘ f2 and ν = (3.50) twice, once for f1 and once for f2 to obtain

ν1 +ν2 −2ν1 ν2 . 1−ν1 ν2

We apply

󵄩2 󵄩󵄩 󵄩󵄩f1 ∘ f2 (x1 ) − f1 ∘ f2 (x2 )󵄩󵄩󵄩 ((3.50) for f1 )

≤

1 − ν2 󵄩󵄩 󵄩2 󵄩󵄩(I − f2 )(x1 ) − (I − f2 )(x2 )󵄩󵄩󵄩 ν2 1 − ν1 󵄩󵄩 󵄩2 − 󵄩(I − f1 )(f2 (x1 )) − (I − f1 )(f2 (x2 ))󵄩󵄩󵄩 . ν 󵄩

((3.50) for f2 )

≤

󵄩2 󵄩2 1 − ν1 󵄩󵄩 󵄩󵄩 󵄩(I − f1 )(f2 (x1 )) − (I − f1 )(f2 (x2 ))󵄩󵄩󵄩 󵄩󵄩f2 (x1 ) − f2 (x2 )󵄩󵄩󵄩 − ν1 󵄩 ‖x1 − x2 ‖2 −

(3.51)

1

Moreover, observe that for any λ ∈ ℝ, z1 , z2 ∈ H, 󵄩󵄩 󵄩2 2 2 2 󵄩󵄩λz1 + (1 − λ)z2 󵄩󵄩󵄩 + λ(1 − λ)‖z1 − z2 ‖ = λ‖z1 ‖ + (1 − λ)‖z2 ‖ .

(3.52)

We would like to apply that to the right-hand side of the above inequality for z1 = 2 1 + 1−ν ≠ 1 (I − f1 )(f2 (x1 )) − (I − f1 )(f2 (x2 )) and z2 = (I − f2 )(x1 ) − (I − f2 )(x2 ), but since 1−ν ν ν

we multiply both sides by τ1 , where τ = yields

1−ν1 ν1

+

1−ν2 , ν2

1

and apply (3.52) for λ =

2

1−ν1 . τν1

This

1 1 − ν1 󵄩󵄩 󵄩2 1 − ν2 󵄩󵄩 󵄩2 ( 󵄩󵄩(I − f1 )(f2 (x1 )) − (I − f1 )(f2 (x2 ))󵄩󵄩󵄩 + 󵄩󵄩(I − f2 )(x1 ) − (I − f2 )(x2 )󵄩󵄩󵄩 ) τ ν1 ν2 󵄩 󵄩2 2 󵄩 󵄩 = 󵄩󵄩λz1 + (1 − λ)z2 󵄩󵄩 + λ(1 − λ)‖z1 − z2 ‖ ≥ λ(1 − λ)‖z1 − z2 ‖2 1 (1 − ν1 )(1 − ν2 ) 󵄩󵄩 󵄩2 = 󵄩󵄩(I − f1 ∘ f2 )(x1 ) − (I − f1 ∘ f2 )(x2 )󵄩󵄩󵄩 , τ τν1 ν2 which combined with (3.51), leads to (1 − ν1 )(1 − ν2 ) 󵄩󵄩 󵄩󵄩 󵄩2 󵄩2 2 󵄩󵄩f1 ∘ f2 (x1 ) − f1 ∘ f2 (x2 )󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖ − 󵄩󵄩(I − f1 ∘ f2 )(x1 ) − (I − f1 ∘ f2 )(x2 )󵄩󵄩󵄩 , τν ν 1 2

2) = and noting further that (1−ντν1 )(1−ν ν 1 2

1−ν ν

for ν =

(3.53)

ν1 +ν2 −2ν1 ν2 , we have the stated result. 1−ν1 ν2

4 Nonsmooth analysis: the subdifferential Many interesting convex functions are not smooth. This chapter focuses on the study of such functions, and in particular to the concept of the subdifferential, which is a multivalued mapping generalizing the derivative. We will develop some of its important properties, as well as its calculus rules, and in particular those related to nonsmooth optimization problems. We will close the chapter with the treatment of the Moreau proximity operator in Hilbert spaces and the study of a popular class of optimization algorithms based on the concept of the subdifferential and the proximity operator called proximal methods. These important concepts have been covered in [13, 14, 19, 20, 25, 88], where the reader is referred for further details.

4.1 The subdifferential: definition and examples The subdifferential is a generalization of the derivative for functions which are not necessarily smooth. While the subdifferential can be defined for any function (even though, it may not exist), we only treat here the case of convex functions, for which the subdifferential enjoys a number of useful properties. Definition 4.1.1 (Subdifferential). Let X be a Banach space. The subdifferential of a function φ : X → ℝ ∪ {+∞} at x ∈ X, denoted by 𝜕φ(x) is the set of all subgradients of φ, i. e., 𝜕φ(x) = {x⋆ ∈ X ⋆ : φ(z) − φ(x) ≥ ⟨x⋆ , z − x⟩, ∀z ∈ X}. The subdifferential is in general a multivalued map, 𝜕φ : X → 2X , where 2X is the set of all possible subsets of X ⋆ . The domain of the subdifferential is D(𝜕φ) := {x ∈ X : 𝜕φ(x) ≠ 0}. It can be seen from the definition that for x ∈ X such that φ(x) = +∞, 𝜕φ(x) = 0. Redressed in the language of convex analysis, this means that if x ∈ ̸ dom φ then x ∈ ̸ D(𝜕φ), so clearly the two sets are related. However, they are not equal, rather as we will prove in Theorem 4.5.3 the domain of the subdifferential D(𝜕φ) is dense in dom φ. To motivate the definition of the subdifferential, we provide the following proposition, which shows that if φ is Gâteaux differentiable, then the subdifferential 𝜕φ(x) is a singleton consisting only of Dφ(x). ⋆

⋆

Proposition 4.1.2 (The subdifferential of a differentiable function). Let φ : X → ℝ ∪ {+∞} be a convex function, which is Gâteaux differentiable at x ∈ X. Then 𝜕φ(x) = {Dφ(x)}, and to simplify the notation in such cases we will simply write 𝜕φ(x) = Dφ(x).

https://doi.org/10.1515/9783110647389-004

148 | 4 Nonsmooth analysis: the subdifferential Proof. If φ is differentiable in the Gâteaux sense at x ∈ X, by the convexity of φ it holds that (see Theorem 2.3.22) φ(z) − φ(x) ≥ ⟨Dφ(x), z − x⟩,

∀ z ∈ X,

so clearly Dφ(x) ∈ 𝜕φ(x), i. e., {Dφ(x)} ⊆ 𝜕φ(x). It remains to show that 𝜕φ(x) ⊆ {Dφ(x)}. To this end, take any element x∗ ∈ 𝜕φ(x), and any element xo ∈ X of the form xo = x + tz for any t > 0, z ∈ X. Then, by the definition of the subdifferential it holds that φ(x + t z) − φ(x) ≥ t ⟨x⋆ , z⟩,

∀ x ∈ X, t > 0.

Divide by t and pass to the limit as t → 0, so that by the differentiability of φ we obtain that ⟨Dφ(x) − x⋆ , z⟩ ≥ 0,

∀ z ∈ X.

Since X is a vector space the same inequality holds for −z ∈ X so that ⟨Dφ(x) − x⋆ , z⟩ = 0,

∀ z ∈ X,

therefore, Dφ(x) − x⋆ = 0 in X ⋆ and 𝜕φ(x) ⊆ {Dφ(x)}. The proof is complete. A partial converse of Proposition 4.1.2 holds, according to which if 𝜕φ(x) is a singleton and φ is continuous at x, then φ is Gâteaux differentiable at x ∈ X. For the proof of the converse, we need some more properties of the subdifferential, which will be provided later on, so this is postponed until Section 4.2.2 (see Example 4.2.10). Before proving existence of the subdifferential (Proposition 4.2.1), we consider some examples. Example 4.1.3 (Subdifferential of |x|). Let X = ℝ and consider the convex function φ : ℝ → ℝ defined by φ(x) = |x|. Then 1 { { { 𝜕φ(x) = {[−1, 1] { { {−1

if x > 0,

if x = 0, if x < 0,

where e. g. 𝜕φ(x) = 1 means that 𝜕φ(x) = {1}. This can follow either directly from the definition1 or by applying Proposition 4.1.2. An alternative way to express this is by 1 We are looking for x ∗ ∈ ℝ such that |z| − |x| ≥ x∗ (z − x) for every z ∈ ℝ. If x > 0, this inequality becomes |z| − x ≥ x ∗ (z − x) for every z ∈ ℝ, so choosing z = x ± ϵ, for any ϵ > 0 we have (respectively) that x∗ ≤ 1 and x ∗ ≥ 1 so that x ∗ = 1. If x < 0, then this inequality becomes |z| + x ≥ x ∗ (z − x) for every z ∈ ℝ, so upon the choices z = 0 and z = 2x we have respectively that x ∗ ≤ −1 and x ∗ ≥ −1 so that x∗ = −1. For x = 0, this inequality becomes |z| ≥ x ∗ z for every z ∈ ℝ which holds for any x ∗ ∈ [−1, 1].

4.1 The subdifferential: definition and examples | 149

x stating that for x ≠ 0, the function φ is differentiable with Dφ(x) = |x| , while for x = 0, it is not differentiable but rather subdifferentiable with 𝜕φ(0) = [−1, 1]. Of course, the fact that for x ≠ 0, φ is differentiable, can also be interpreted as that φ is subdifferenx }. Note that as a set valued tiable but the set 𝜕φ(x) for x ≠ 0 is a singleton, 𝜕φ(x) = { |x| map x 󳨃→ 𝜕φ(x) is upper semicontinuous. ◁

The above result can be generalized for the norm of any normed space. This will be done in Section 4.4, where it will be shown that the norm is subdifferentiable and its subdifferential coincides with the duality map for this space. Before turning to the general case, we present another example related to the norm of a Hilbert space. Example 4.1.4 (Subdifferential of the norm in Hilbert space). Let X = H be a Hilbert space, with its dual identified with the space itself X ⋆ ≃ H, and define the functional φ : X → ℝ by φ(x) = ‖x‖ for every x ∈ X. Then 1 x x ≠ 0, 𝜕φ(x) = { ‖x‖ BH (0, 1) x = 0,

where as before 𝜕(x) =

1 x ‖x‖

1 means that 𝜕φ(x) = { ‖x‖ x}. Indeed, for x ≠ 0, φ is Gâteaux

x x , so that by Proposition 4.1.2, 𝜕φ(x) = { ‖x‖ } for x ≠ 0. At differentiable with2 Dφ(x) = ‖x‖ x = 0, the function φ is not differentiable; however, the subdifferential exists. On the other hand, by the definition, at x = 0 we have that for any x⋆ ∈ 𝜕φ(0), it must hold that ‖z‖ ≥ ⟨x⋆ , z⟩H , for every z ∈ H. By the Cauchy–Schwarz inequality, this holds for ◁ any x⋆ ∈ H such that ‖x⋆ ‖ ≤ 1, therefore, 𝜕φ(0) = BH (0, 1).

The subdifferential can be given a geometrical meaning in terms of the concept of the normal cone. Definition 4.1.5 (Normal cone). Let C ⊂ X be a convex set, and consider a point x ∈ C. The set NC (x) := {x⋆ ∈ X ⋆ : ⟨x⋆ , z − x⟩ ≤ 0, ∀ z ∈ C} is called the normal cone of C at x. The convention NC (x) = 0 if x ∈ ̸ C is used. Note that if X = H is a Hilbert space, x ∈ H and xo ∈ C ⊂ H with C closed and convex, then x − xo ∈ NC (xo ) is equivalent to xo = PC (x). The normal cone of a linear 2 Simply consider the function ψ : ℝ → ℝ defined by ψ(t) = ‖x + th‖ for any x, h ∈ H, with x ≠ 0. It holds that ψ(t)2 = ‖x + th‖2 , and the right-hand side differentiable at t = 0, with derivative 2⟨x, h⟩, so that taking into account as well the continuity of ψ the result holds.

150 | 4 Nonsmooth analysis: the subdifferential subspace E ⊂ X of a Banach space is NE (x) = E ⊥ if x ∈ E and NE (x) = 0 if x ∈ X \ E (as one may easily see by setting z = x ± xo in the definition for arbitrary xo ∈ E). Example 4.1.6 (Subdifferential of the indicator function of a convex set). Consider a convex set C ⊂ X and the indicator function of C, IC : X → ℝ ∪ {+∞} defined as 0,

IC (x) = {

x∈C

+∞, x ∈ X \ C.

It is clear that IC is a proper convex function. If C is closed, then it is also lower semicontinuous (see Example 2.3.14). Then, as follows by Definition 4.1.1, the subdifferential of IC at x is the normal cone of C at x, i. e., 𝜕IC (x) = NC (x), ∀ x ∈ C. ◁ Example 4.1.7 (Subdifferential of indicator function (equiv. normal cone) of affine subspace). Let X, Y be two Banach spaces and consider the bounded linear operator L : X → Y. For a given y ∈ R(L), let C = {z ∈ X : Lz = y}. Then, for any x ∈ C, we have that 𝜕IC (x) = NC (x) = (N(L))⊥ . For example, let x⋆ ∈ NC (x). Then, by definition ⟨x⋆ , z−x⟩ ≤ 0 for every z ∈ C. Since x ∈ C, let us consider z ∈ C of the form z = x ± ϵzo , where zo ∈ N(L) (i. e., Lzo = 0) and ϵ > 0 arbitrary. Using these z ∈ C for the variational inequality defining the normal cone, we conclude that ⟨x⋆ , zo ⟩ = 0, for every zo ∈ N(L), so that x⋆ ∈ (N(L))⊥ . ◁ Example 4.1.8 (Subdifferential of an integral functional). Let 𝒟 ⊂ ℝd be an open bounded set and X = Lp (𝒟), so that each x ∈ X is identified with a function u : 𝒟 → ℝ. Let φo : ℝ → ℝ be a convex subdifferentiable function and consider the functional φ : X → ℝ ∪ {+∞} defined by ∫𝒟 φo (u(x))dx,

φ(u) = {

+∞,

if φo (u) ∈ L1 (𝒟), otherwise.

Clearly, the choice of p depends on growth conditions on the function φo , but we prefer to pass upon this for the time being, and assume that p is chosen so that φ(u) is finite for any u ∈ Lp (𝒟). The functional φ defined above is a convex functional, which is subdifferentiable and 𝜕φ(u) = {v ∈ Lp (𝒟) : v(x) ∈ 𝜕φo (u(x)), a.e. x ∈ 𝒟}. ⋆

Indeed, it is straightforward to see that A := {w ∈ Lp (𝒟) : w(x) ∈ 𝜕φo (u(x)), a.e. x ∈ 𝒟} ⊂ 𝜕φ(u), ⋆

since for any w ∈ A, φo (v(x))−φo (u(x)) ≥ w(x)(v(x)−u(x)) a. e. x ∈ 𝒟, for any v ∈ Lp (𝒟), where upon integrating over all x ∈ 𝒟 we conclude that φ(v) − φ(u) ≥ ⟨w, v − u⟩ for any v ∈ Lp (𝒟), which implies that w ∈ 𝜕φ(u).

4.2 Subdifferential for convex functions | 151

For the reverse inclusion, consider any w ∈ 𝜕φ(u), i. e., any w such that φ(v) − φ(u) ≥ ⟨w, v − u⟩ for any v ∈ Lp (𝒟), which can be redressed as ∫(φo (v(x)) − φo (u(x)) − w(x)(v(x) − u(x))dx ≥ 0,

∀ v ∈ Lp (𝒟),

(4.1)

𝒟

from which it follows that3 w ∈ A.

◁

4.2 The subdifferential for convex functions We now discuss the general question of existence of the subdifferential for convex functions. 4.2.1 Existence and fundamental properties The existence of the subdifferential of a convex function4 at its points of continuity follows by construction in terms of a separation argument [19]. Proposition 4.2.1. Let X be a Banach space and let φ : X → ℝ ∪ {+∞} be a proper convex lower semicontinuous function. Then φ is subdifferentiable on the interior of the domain of φ, i. e., D(𝜕φ) = int(dom φ), and ⟨x⋆1 − x⋆2 , x1 − x2 ⟩ ≥ 0,

∀ xi ∈ D(𝜕φ), x⋆i ∈ 𝜕φ(xi ), i = 1, 2.

(4.2)

Proof. Let xo ∈ int(dom φ). By Proposition 2.3.21, φ is continuous at xo and so5 int (epi φ) ≠ 0. Moreover, (xo , φ(xo )) ∈ ̸ int (epi φ). Then (see Proposition 1.2.11) the point (xo , φ(xo )) and the set epi φ can be separated, i. e., there exist (z⋆ , β) ∈ X ⋆ × ℝ and α ∈ ℝ such that ⟨z⋆ , xo ⟩ + βφ(xo ) ≥ α ≥ ⟨z⋆ , x⟩ + βλ,

∀ (x, λ) ∈ epi φ.

(4.3)

̄ 3 To see this, consider any measurable 𝒟0 ⊂ 𝒟 and for any v ∈ Lp (𝒟) define v(x) = v(x)1𝒟0 (x) + u(x)1𝒟c (x), and apply (4.1) for the choice v.̄ This gives 0

∫ (φo (v(x)) − φo (u(x)) − w(x)(v(x) − u(x))dx ≥ 0,

∀ v ∈ Lp (𝒟), ∀ 𝒟0 ⊂ 𝒟,

𝒟0

which by the arbitrariness of 𝒟0 leads to φo (v(x)) − φo (u(x)) − w(x)(v(x) − u(x)) ≥ 0, a. e., x ∈ 𝒟, and hence, w(x) ∈ 𝜕φo (u(x)), a. e., x ∈ 𝒟. 4 In principle, one may define the subdifferential using Definition 4.1.1 even for functions which are not convex. 5 By continuity of φ at xo for any ϵ > 0, there exists δ > 0 such that −ϵ < φ(x) − φ(xo ) < ϵ for all x ∈ X such that ‖x − xo ‖ < δ. This means that N(xo ) = {x ∈ X : ‖x − xo ‖ < δ} is an open neighborhood of xo ∈ dom φ, in which φ(x) < φ(xo )+ϵ, which implies that the open set N(xo )×{r : r > φ(xo )+ϵ} ⊂ epi φ; hence, int (epi φ) ≠ 0.

152 | 4 Nonsmooth analysis: the subdifferential Since for every ϵ > 0, (xo , φ(xo )+ϵ) ∈ epi φ, (4.3) yields βϵ ≤ 0, so that β ≤ 0. But β ≠ 0, since if not, then (4.3) implies that ⟨z⋆ , x − xo ⟩ ≤ 0 for every (x, λ) ∈ epi φ, which is turn implies that ⟨z⋆ , z⟩ = 0 for every z ∈ dom φ, therefore, z⋆ = 0, which is a contradiction. Therefore, β < 0. On the other hand, for any (x, λ) ∈ epi φ, it holds that λ ≥ φ(x), so that (4.3) yields, ⟨z⋆ , xo ⟩ + βφ(xo ) ≥ α ≥ ⟨z⋆ , x⟩ + βφ(x),

∀ x ∈ dom φ.

(4.4)

Dividing (4.4) by −β > 0 and setting x⋆o = − zβ yields ⋆

⟨x⋆o , xo ⟩ − φ(xo ) ≥ ⟨x⋆o , xo ⟩ − φ(x),

∀ x ∈ dom φ,

which can be rearranged as φ(x) ≥ φ(xo ) + ⟨x⋆o , x − xo ⟩,

∀ x ∈ dom φ,

i. e., x⋆o ∈ 𝜕φ(xo ). This concludes the proof of the first statement. To show (4.2), take xi ∈ D(𝜕φ) and consider x⋆i ∈ 𝜕φ(xi ), i = 1, 2. Then, by the definition of the subdifferential, we have that φ(z) − φ(x1 ) ≥ ⟨x⋆1 , z − x1 ⟩,

φ(z) − φ(x2 ) ≥

⟨x⋆2 , z

− x2 ⟩,

∀ z ∈ X,

∀ z ∈ X.

Setting z = x2 in the first inequality and z = x1 in the second yields φ(x2 ) − φ(x1 ) ≥ ⟨x⋆1 , x2 − x1 ⟩, φ(x1 ) − φ(x2 ) ≥ ⟨x⋆2 , x1 − x2 ⟩, and upon addition we conclude (4.2), which is the monotonicity property of the subdifferential. Remark 4.2.2. The subdifferential of a convex function is nonempty at the points where φ is finite and continuous, i. e., on cont(φ). This does not require lower semicontinuity of φ. Remark 4.2.3. Note that in infinite dimensional spaces, the interior of the domain of φ may be empty. Furthermore, even in finite dimensional spaces the subdifferential may be empty at points on the boundary of the domain of 6 φ. This calls for approximations of the subdifferential (see Section 4.5). Through this approximation procedure, it can also be shown that the domain of the subdifferential D(𝜕φ) is dense in dom φ (see the Brøndsted–Rockafellar Theorem 4.5.3). 6 For example, φ(x) = −xγ for x ∈ (−1, 1) and φ(x) = +∞ for |x| ≥ 1, with γ < 1, where it is easily seen that 𝜕φ(x) = 0 for every |x| ≥ 1.

4.2 Subdifferential for convex functions | 153

As already discussed and shown in Proposition 4.1.2, the subdifferential is single valued and coincides with the Gâteaux derivative for differentiable functions. In this sense, the inequality (4.2) is a monotonicity property, which generalizes the relevant monotonicity property of the Gâteaux derivative for convex functions (see (2.11), Theorem 2.3.22). Finally, the subdifferential of a convex function enjoys convexity and compactness properties [19, 89]. Proposition 4.2.4. Let φ : X → ℝ ∪ {+∞} be a convex proper lower semicontinuous function. The subdifferential 𝜕φ(x), at any x ∈ X, at which φ is continuous, is a convex, bounded, weak⋆ closed (hence, weak⋆ compact) subset of X ⋆ . Proof. Convexity and weak⋆ -closedeness follow easily by definition. We prove that 𝜕φ(x) is bounded. Since φ is finite and continuous at x there exists an open neighborhood of x, N(x) = {x󸀠 ∈ X : ‖x − x󸀠 ‖ < δ} such that |φ(x) − φ(z)| < ϵ for all z ∈ N(x) (see Proposition 2.3.20). Let xo ∈ X such that ‖xo ‖ = 1 and take x󸀠 = x + t xo , for |t| < δ. Clearly, x󸀠 ∈ N(x). For any x⋆ ∈ 𝜕φ(x), applying Definition 4.1.1 for the pair (x, x󸀠 ), we have that φ(x + txo ) ≥ φ(x) + ⟨x⋆ , t xo ⟩.

(4.5)

Since x + t xo ∈ N(x), by the continuity of φ, we also have that 󵄨󵄨 󵄨 󵄨󵄨φ(x + t xo ) − φ(x)󵄨󵄨󵄨 ≤ ϵ.

(4.6)

ϵ Therefore, combining (4.5) and (4.6), we deduce that |⟨x⋆ , xo ⟩| ≤ |t| . This implies, by ϵ ⋆ the definition of the dual norm ‖ ⋅ ‖X ⋆ , that ‖x ‖X ⋆ ≤ |t| so that the set 𝜕φ(x) ⊂ X ⋆ is bounded. The weak⋆ compactness follows from the Banach–Alaoglou theorem (see Theorem 1.1.36).

Example 4.2.5 (Subdifferential for strongly convex functions). For uniformly convex or strongly convex functions (see Definition 2.3.3), the monotonicity properties of the subdifferential may be further enhanced. Assume, e. g., that φ : X → ℝ ∪ {+∞} is strongly convex with modulus of convexity c. Then it holds that c ⟨x⋆ , z − x⟩ ≤ φ(z) − φ(x) − ‖z − x‖2 , ∀ x⋆ ∈ 𝜕φ(x), z ∈ X, 2 ⋆ ⋆ (ii) ⟨x1 − x2 , x1 − x2 ⟩ ≥ c‖x1 − x2 ‖2 , ∀ xi ∈ X, x⋆i ∈ 𝜕φ(xi ), i = 1, 2, 󵄩 󵄩 (iii) 󵄩󵄩󵄩x⋆1 − x⋆2 󵄩󵄩󵄩X ⋆ ≥ c‖x1 − x2 ‖, ∀ xi ∈ X, x⋆i ∈ 𝜕φ(xi ), i = 1, 2.

(i)

(4.7)

To prove (i), consider any x ∈ X and any x⋆ ∈ 𝜕φ(x) so that by definition φ(z󸀠 ) − φ(x) ≥ ⟨x⋆ , z󸀠 − x⟩ for every z󸀠 ∈ X. Consider z󸀠 = x + t(z − x) = (1 − t)x + tz, for arbitrary z ∈ X, t ∈ (0, 1) in the above definition, which along with the fact that φ is strongly convex yields t⟨x⋆ , z − x⟩ ≤ φ((1 − t)x + tz) − φ(x)

154 | 4 Nonsmooth analysis: the subdifferential c ≤ (1 − t)φ(x) + tφ(z) − t(t − 1)‖x − z‖2 − φ(x) 2 c = t(φ(z) − φ(x)) − t(t − 1)‖x − z‖2 , 2 where upon dividing by t and passing to the limit as t → 1 leads to (i). To get (ii), we apply (i) first for x⋆1 ∈ 𝜕φ(x1 ) and z = x2 and then, for x⋆2 ∈ 𝜕φ(x2 ) and z = x1 and add, while (iii) follows by the observation that |⟨x⋆1 − x⋆2 , x1 − x2 ⟩| ≤ ‖x⋆1 − x⋆2 ‖X ⋆ ‖x1 − x2 ‖, and (ii). ◁

4.2.2 The subdifferential and the right-hand side directional derivative We have seen in Proposition 4.1.2 that if a convex function φ is differentiable, its Gâteaux derivative coincides with the subdifferential. However, even if φ is not differentiable, by convexity we may show that the right-hand side directional derivative exists and is related to the subdifferential. Definition 4.2.6. Let φ : X → ℝ. The limit 1 D+ φ(x, h) := lim+ (φ(x + t h) − φ(x)) t→0 t

(4.8)

is called the right-hand side directional derivative of the function φ at x ∈ X in the direction h ∈ X. The right-hand side directional derivative D+ φ(x, h) is very similar (and related) to the Gâteaux directional derivative Dφ(x, h) = limt→0 1t (φ(x + t h) − φ(x)), (see Sections 2.1.1 and 2.1.2) but with a very important difference. While if φ is Gâteaux differentiable at x the mapping h 󳨃→ Dφ(x, h) is linear in h for every h ∈ X, in general the mapping h 󳨃→ D+ φ(x, h) is sublinear in h for every h ∈ X. In fact, if at a point x ∈ X, the mapping h 󳨃→ D+ φ(x, h) is linear in h for every h ∈ X, then φ is Gâteaux differentiable at x ∈ X. The following proposition (see e. g. [89]) connects the right-hand side directional derivative with the subdifferential. Proposition 4.2.7 (The right-hand side directional derivative and the subdifferential). If φ : X → ℝ is convex, then D+ φ(x, h) exists (but may not be finite), and the map h 󳨃→ D+ φ(x, h) is sublinear. At points x where φ is continuous, for any h ∈ X it holds that D+ φ(x, h) = sup ⟨x⋆ , h⟩, x⋆ ∈𝜕φ(x)

(4.9)

and the supremum is attained at some x⋆ ∈ 𝜕φ(x). Proof. To show the existence of the limit defining D+ φ(x, h), we simply have to note that convexity of φ leads to monotonicity of the mapping t 󳨃→ 1t (φ(x + t h) − φ(x)) for

4.2 Subdifferential for convex functions | 155

all x, h ∈ X. To see that, take 0 < t1 ≤ t2 and write t t1 )x + 1 (x + t2 h), t2 t2

x + t1 h = (1 −

which is clearly a convex combination since φ(x + t1 h) ≤ (1 −

t1 t2

∈ (0, 1]. Convexity of φ implies that

t t1 )φ(x) + 1 φ(x + t2 h), t2 t2

which leads to 1 1 (φ(x + t1 h) − φ(x)) ≤ (φ(x + t2 h) − φ(x)). t1 t2 This implies the monotonicity of the mapping t 󳨃→ 1t (φ(x+t h)−φ(x)) for every x, h ∈ X, which leads to the existence of the limit (not necessarily finite) as t → 0+ . To prove that the map h 󳨃→ D+ φ(x, h) is sublinear, it is enough to note that this map is convex and positively homogeneous. Convexity follows by the observation that x + t(λh1 + (1 − λ)h2 ) = λ(x + t h1 ) + (1 − λ)(x + t h2 ), so that by convexity of φ we have that φ(x + t(λh1 + (1 − λ)h2 )) ≤ λφ(x + t h1 ) + (1 − λ)φ(x + t h2 ), where upon subtracting φ(x) from both sides, dividing by t and passing to the limit as t → 0+ provides the required convexity. Positive homogeneity follows by a similar argument (and is left as an exercise). To show the connection of D+ φ(x, h) with the subdifferential, observe first that by the definition of the subdifferential it follows that if x⋆ ∈ 𝜕φ(x) then for all t > 0 and h ∈ X, φ(x + th) − φ(x) ≥ t⟨x⋆ , h⟩, so that dividing by t and passing to the limit as t → 0+ yields that for all h and every x⋆ ∈ 𝜕φ(x) it holds that D+ φ(x, h) ≥ ⟨x⋆ , h⟩,

∀ x⋆ ∈ 𝜕φ(x),

which in turn implies that D+ φ(x, h) ≥

sup ⟨x⋆ , h⟩.

x⋆ ∈𝜕φ(x)

If we manage to show that the reverse inequality holds as well, then we have the stated result. For that, it is enough to show that there exists a x⋆o ∈ 𝜕φ(x) such that7 7 This is easy to see since obviously ⟨x⋆o , h⟩ ≤ supx⋆ ∈𝜕φ(x) ⟨x⋆ , h⟩.

156 | 4 Nonsmooth analysis: the subdifferential D+ φ(x, h) ≤ ⟨x⋆o , h⟩. To show the existence of such an element x⋆o ∈ 𝜕φ(x), we will use the Hahn–Banach theorem. We first note that if x is a point of continuity of φ then then the mapping h 󳨃→ + D φ(x, h), denoted from now on for simplicity as D+ φ(x) and its action by D+ φ(x)(h) = D+ φ(x, h), is a continuous, sublinear functional. Sublinearity has already been established, so we only need to show continuity. By the Lipschitz continuity properties of convex functions, and in particular by the fact that if φ is continuous at x then it is locally Lipschitz at this point, we see that there exists a neighborhood N(x) such that as long as t is chosen small enough for x + th to be in this neighborhood, then φ(x + th) − φ(x) ≤ Lt‖h‖, which upon dividing by t and passing to the limit yields that D+ φ(x, h) ≤ L‖h‖, from which we deduce continuity. By the Hahn-Banach theorem, for any ho ∈ X \ {0} fixed, there exists a linear functional8 x⋆o ∈ X ⋆ such that ⟨x⋆o , ho ⟩ = D+ φ(x)(ho ) = D+ φ(xo , ho ), and ⟨x⋆o , h⟩ ≤ D+ φ(x)(h) = D+ φ(x, h),

∀ h ∈ X,

which implies that x⋆o ∈ 𝜕φ(x). This can be seen by setting h = z − x for arbitrary z ∈ X, noting that for any t > 0 it holds that D+ φ(x, h) ≤ 1t (φ(x + th) − φ(x) and letting t → 1 to conclude that ⟨x⋆o , z − x⟩ ≤ φ(z) − φ(x),

∀z ∈ X,

which leads to the conclusion x⋆o ∈ 𝜕φ(x). This completes the proof. Remark 4.2.8. The claim of Proposition 4.2.7 (the max formula) holds at any point where φ is finite and continuous, i. e., on cont(φ). This does not require lower semicontinuity of φ. If φ is lower semicontinuous, it holds on int(domφ). Example 4.2.9. If at a point x ∈ X, the map h 󳨃→ D+ φ(x, h) is linear, then φ is Gâteaux differentiable at x and D+ φ(x, h) = Dφ(x; h) = ⟨Dφ(x), h⟩, for every h ∈ X. The converse is also true. To see the above, define the left-hand sided directional derivative D− φ(x, h) := limt→0− 1t (φ(x + th) − φ(x)), and note that Dφ(x, h) exists at a point x ∈ X if D+ φ(x, h) = D− φ(x, h). A simple scaling argument shows that for every h ∈ X, it holds that D− φ(x, h) = −D+ φ(x, −h). This implies that Dφ(x, h) exists if D− φ(x, h) = D+ φ(x, −h) = D+ φ(x, h),

(4.10)

8 This functional can be defined on span(ho ) as x⋆o (λho ) = ⟨x⋆o , λho ⟩ = λD+ φ(x)(ho ) for any λ ∈ ℝ and then extended by the standard Hahn-Banach theorem (see [75]).

4.3 Subdifferential calculus | 157

and, using the fact that the functional h 󳨃→ D+ φ(x, h) is sublinear,9 we conclude that the mapping h 󳨃→ D+ φ(x, h) is linear. The converse follows from the observation that if the mapping h 󳨃→ D+ φ(x, h) is linear, then D+ φ(x, −h) = −D+ φ(x, h) which implies that D− φ(x, h) = D+ φ(x, h) and the Gâteaux differentiability of φ at x follows. ◁ Example 4.2.10. If the set 𝜕φ(x) is a singleton at some point x ∈ X, at which φ is continuous, then φ is Gâteaux differentiable at x. This is the converse of Proposition 4.1.2. Suppose that φ is continuous at x and 𝜕φ(x) = {z⋆ } for some z⋆ ∈ X ⋆ . Then, by Proposition 4.2.7, D+ φ(x, h) = ⋆max ⟨x⋆ , h⟩ = ⟨z⋆ , h⟩, x ∈𝜕φ(x)

so that h 󳨃→ D+ φ(x, h) is linear and by Example 4.2.9, φ is Gâteaux differentiable at x, whereas by Proposition 4.1.2, z⋆ = Dφ(x). ◁

4.3 Subdifferential calculus The subdifferential has the following properties, which are usually referred to under the name subdifferential calculus [13, 19]. Proposition 4.3.1 (Subdifferential calculus). (i) Let φ : X → ℝ ∪ {+∞} be a convex function. Then 𝜕(λφ)(x) = λ𝜕φ(x), for every λ > 0. (ii) Let φ1 , φ2 : X → ℝ ∪ {+∞} be convex functions. Then 𝜕φ1 (x) + 𝜕φ2 (x) ⊂ 𝜕(φ1 + φ2 )(x),

∀ x ∈ X.

(iii) Let φ : Y → ℝ ∪ {+∞} be a convex function and L : X → Y a continuous linear operator. Then L∗ 𝜕φ(Lx) ⊂ 𝜕(φ ∘ L)(x),

∀ x ∈ X,

where L∗ : Y ⋆ → X ⋆ is the adjoint (dual) operator of L. Proof. (i) Suppose that x⋆ ∈ 𝜕φ(x). Then φ(z) − φ(x) ≥ ⟨x⋆ , z − x⟩, for all z ∈ X, so that multiplying by λ > 0, we see that λx⋆ ∈ 𝜕(λφ)(x). This implies that λ𝜕φ(x) ⊂ 𝜕(λφ)(x). To prove the opposite inclusion, let x⋆ ∈ 𝜕(λφ)(x). Then λφ(z) − λφ(x) ≥ ⟨x⋆ , z − x⟩,

∀ z ∈ X,

9 If for a sublinear map f it holds that −f (−h) = f (h), then f is linear. Indeed expressing f (x1 ) = f (x1 + x2 − x2 ) ≤ f (x1 + x2 ) + f (−x2 ) = f (x1 + x2 ) − f (x2 ), we see that f (x1 ) + f (x2 ) ≤ f (x1 + x2 ), which combined with f (x1 + x2 ) ≤ f (x1 ) + f (x2 ) leads to f (x1 + x2 ) = f (x1 ) + f (x2 ). That f (λx) = λf (x) for every λ ∈ ℝ, follows by positive homogeneity and the property that −f (−h) = f (h).

158 | 4 Nonsmooth analysis: the subdifferential so that dividing by λ > 0 yields, xλ ∈ 𝜕φ(x), which means that x⋆ ∈ λ𝜕φ(x). Therefore, 𝜕(λφ)(x) ⊂ λ𝜕φ(x), and the claim is proved. (ii) Let z⋆1 ∈ 𝜕φ1 (x) and z⋆2 ∈ 𝜕φ2 (x). Then ⋆

φi (z) − φi (x) ≥ ⟨z⋆i , z − x⟩,

∀ z ∈ X, i = 1, 2.

Adding these inequalities, we obtain that (φ1 + φ2 )(z) − (φ1 + φ2 )(x) ≥ ⟨z⋆1 + z⋆2 , z − x⟩,

∀ z ∈ X,

hence, z⋆1 + z⋆2 ∈ 𝜕(φ1 + φ2 )(x), which implies the stated result. (iii) We will denote by ⟨⋅, ⋅⟩ and ⟨⋅, ⋅⟩Y ⋆ ,Y the duality pairings of X ⋆ , X and Y ⋆ , Y respectively. Take any x⋆o ∈ L∗ 𝜕φ(Lx) ⊂ X ⋆ . Then there exists y⋆o ∈ 𝜕φ(Lx) ⊂ Y ⋆ , i. e., a y⋆o ∈ Y ⋆ with the property φ(Lz) − φ(Lx) ≥ ⟨y⋆o , Lz − Lx⟩Y ⋆ ,Y ,

∀z ∈ X

(4.11)

such that x⋆o = L∗ y⋆o . Using the definition of the adjoint operator we can express (4.11) in the equivalent form φ(Lz) − φ(Lx) ≥ ⟨L∗ y⋆o , z − x⟩ = ⟨x⋆o , z − x⟩,

∀ z ∈ X.

(4.12)

By the definition of 𝜕(φ ∘ L)(x), it is evident that x⋆o ∈ 𝜕(φ ∘ L)(x). Indeed, 𝜕(φ ∘ L)(x) = {x⋆ ∈ X ⋆ : (φ ∘ L)(z) − (φ ∘ L)(x) ≥ ⟨x⋆ , Lz − Lx⟩Y ⋆ ,Y , ∀ z ∈ X} = {x⋆ ∈ X ⋆ : (φ ∘ L)(z) − (φ ∘ L)(x) ≥ ⟨L∗ x⋆ , z − x⟩, ∀ z ∈ X}.

which combined with (4.12) yields that x⋆o ∈ 𝜕(φ ∘ L)(x), therefore, the stated inclusion holds true. It is interesting to find conditions under which the inclusions in the addition and the chain rule for the subdifferential (Proposition 4.3.1(ii) and (iii)) can be turned into equalities. We will see that continuity of the functions involved plays an important role in this respect, and in view of Proposition 2.3.20, where continuity at a single point for a convex function implies continuity at every point in the interior of its domain, we anticipate that continuity at a single point will suffice. Theorem 4.3.2 (Moreau–Rockafellar). Let φ1 , φ2 : X → ℝ ∪ {+∞} be convex proper lower semicontinuous functions such that int (dom φ1 ) ∩ dom φ2 ≠ 0. Then 𝜕(φ1 + φ2 ) = 𝜕φ1 + 𝜕φ2 . The condition int (dom φ1 )∩dom φ2 ≠ 0 can also be interpreted as that there exists at least one point xo ∈ dom φ2 , at which φ1 is finite and continuous.

4.3 Subdifferential calculus | 159

Proof. By Proposition 4.3.1(ii), we have that 𝜕φ1 (x) + 𝜕φ2 (x) ⊂ 𝜕(φ1 + φ2 )(x),

∀ x ∈ X.

We will now show that under the extra condition imposed, the reverse inclusion also holds. To this end, let xo ∈ int (dom φ1 ) ∩ dom φ2 , fix any x ∈ X, consider any x⋆ ∈ 𝜕(φ1 + φ2 )(x), and we shall show that x⋆ ∈ 𝜕φ1 (x) + 𝜕φ2 (x), i. e., x⋆ = x⋆1 + x⋆2 , with x⋆1 ∈ 𝜕φ1 (x) and x⋆2 ∈ 𝜕φ2 (x). Recall, by the properties of proper lower semicontinuous convex functions (see Proposition 2.3.21), that since xo ∈ int(dom φ1 ) the function φ1 is continuous at xo . Since x⋆ ∈ 𝜕(φ1 + φ2 )(x), we have that φ1 (z) + φ2 (z) ≥ φ1 (x) + φ2 (x) + ⟨x⋆ , z − x⟩ for every z ∈ X, i. e., φ1 (z) − φ1 (x) − ⟨x⋆ , z − x⟩ ≥ φ2 (x) − φ2 (z),

∀ z ∈ X.

(4.13)

For fixed x, define the functions φ and ψ, by z 󳨃→ φ(z) and z 󳨃→ ψ(z), where φ(z) = φ1 (z) − φ1 (x) − ⟨x⋆ , z − x⟩,

and,

ψ(z) = φ2 (x) − φ2 (z).

Clearly, φ is convex and continuous at xo , while ψ is a concave function. In terms of φ and ψ, (4.13) becomes φ(z) ≥ ψ(z),

∀ z ∈ X.

(4.14)

Consider the sets A = {(z, r) ∈ X × ℝ : φ(z) ≤ r} = {(z, r) ∈ X × ℝ : φ1 (z) − φ1 (x) − ⟨x⋆ , z − x⟩ ≤ r}, B = {(z, r) ∈ X × ℝ : ψ(z) ≥ r} = {(z, r) ∈ X × ℝ : φ2 (x) − φ2 (z) ≥ r}.

Since A is the epigraph of the convex function φ which is continuous at xo , it follows that10 int(A) = int (epi φ) ≠ 0. Furthermore, by (4.14), it is clear that int(A) ∩ B = 0. Therefore, by the separation theorem int(A) and B can be separated by a hyperplane. It then follows that there exists (z⋆o , α) ∈ X ⋆ \ {0} × ℝ and ρ ∈ ℝ such that ⟨z⋆o , x1 ⟩ + α λ1 ≤ ρ ≤ ⟨z⋆o , x2 ⟩ + α λ2 ,

∀ (x1 , λ1 ) ∈ int(A), ∀ (x2 , λ2 ) ∈ B.

(4.15)

By the definition of the sets A and B, it can be seen that (x, ϵ) ∈ int(A), for every ϵ > 0, while (x, 0) ∈ B. An application of (4.15) for this choice leads to the observation 10 Consider any λ > φ(xo ). Then (xo , λ) ∈ int (epi φ). Indeed, by the continuity of φ at xo , for any ϵ > 0 there exists δ > 0 such that |φ(x) − φ(xo )| < ϵ for every x such that ‖x − xo ‖ < δ, which implies that φ(x) < φ(xo ) + ϵ for all x ∈ B(xo , δ). By choosing ϵ appropriately, we can make sure that φ(xo ) + ϵ may take any value in (φ(xo ), λ). This means that for every μ ∈ (φ(xo ), λ) we may find a ball B(xo , δ) such that for every x ∈ B(xo , δ) it holds that φ(x) < μ. That of course means that (x, μ) ∈ epi φ for every x ∈ B(xo , δ), μ ∈ (φ(xo ), λ), therefore, (xo , λ) ∈ int (epi φ).

160 | 4 Nonsmooth analysis: the subdifferential that ⟨z⋆o , x⟩ ≤ ρ ≤ ⟨z⋆o , x⟩ + αϵ, for every ϵ > 0 and passing to the limit as ϵ → 0 we conclude that ρ = ⟨z⋆o , x⟩. Furthermore, picking (x, 1) ∈ int(A) and (x, 0) ∈ B and applying (4.15), we obtain that α ≤ 0. We may also observe that α may not11 take the value 0; hence, α < 0. We multiply both sides of (4.15) by − α1 > 0 and upon defining x⋆o = − α1 z⋆o , we obtain that ⟨x⋆o , x1 ⟩ − λ1 ≤ ⟨x⋆o , x⟩ ≤ ⟨x⋆o , x2 ⟩ − λ2 ,

∀ (x1 , λ1 ) ∈ int(A), ∀ (x2 , λ2 ) ∈ B,

(4.16)

where we also used the fact that ρ = ⟨z⋆o , x⟩. Inequality (4.16) guarantees that for any (x2 , λ2 ) ∈ B, i. e., for any (x2 , λ2 ) such that φ2 (x) − φ2 (x2 ) ≥ λ2 , it holds that ⟨x⋆o , x⟩ ≤ ⟨x⋆o , x2 ⟩ − λ2 . Fix any z ∈ X and set λo,2 = φ2 (x) − φ2 (z). Clearly, the pair (z, λo,2 ) ∈ B, so that (4.16) yields that ⟨x⋆o , x⟩ ≤ ⟨x⋆o , z⟩ + φ2 (z) − φ2 (x), which can be reformulated as ⟨−x⋆o , z − x⟩ + φ2 (x) ≤ φ2 (z),

∀ z ∈ X,

hence, x⋆2 := −x⋆o ∈ 𝜕φ2 (x). Furthermore, inequality (4.16) guarantees that for any (x1 , λ1 ) ∈ int(A), i. e., for any (x1 , λ1 ) such that φ1 (x1 ) − φ1 (x) − ⟨x⋆ , x1 − x⟩ < λ1 , it holds that ⟨x⋆o , x1 ⟩ − λ1 ≤ ⟨x⋆o , x⟩. Fix any z ∈ X and set λo,1 = φ1 (z) − φ1 (x) − ⟨x⋆ , z − x⟩ + ϵ. For any ϵ > 0, the pair (z, λo,1 ) ∈ int(A), so that (4.16) yields that ⟨x⋆ + x⋆o , z − x⟩ + φ1 (x) − ϵ ≤ φ1 (z),

∀ z ∈ X, ϵ > 0,

and passing to the limit as ϵ → 0 we conclude that x⋆1 := x⋆ + x⋆o ∈ 𝜕φ1 (x). Combining x⋆ +x⋆o ∈ 𝜕φ1 (x) and −x⋆o ∈ 𝜕φ2 (x) we conclude that x⋆ = x⋆ +x⋆o +(−x⋆o ) ∈ 𝜕φ1 (x) + 𝜕φ2 (x). This concludes the proof. Remark 4.3.3. The subdifferenial sum rule holds, even if φ1 , φ2 are not lower semicontinuous if, e. g., domφ1 ∩ cont(φ2 ) ≠ 0, i. e., if there exists a point where one of the functions is finite and the other is continuous. Proposition 4.3.4. Let L : X → Y be a continuous linear operator and φ : Y → ℝ∪{+∞} be a convex function which is continuous at a point of dom φ ∩ R(L). Then L⋆ 𝜕φ(Lx) = 𝜕(φ ∘ L)(x),

∀ x ∈ X,

where L⋆ is the adjoint operator of L. 11 If it did, observing that since xo ∈ domφ2 , xo ∈ B and since xo is a point of continuity of φ1 , there exists an open ball B(xo , δ) such that B(xo , δ) ∈ int(A), then applying (4.15) for xo ∈ B and for any B(xo , δ) ⊂ intA we conclude that ⟨z⋆o , xo − z󸀠 ⟩ ≤ 0 for every z󸀠 ∈ B(xo , δ) which implies that z⋆o = 0 which contradicts z⋆o ≠ 0.

4.3 Subdifferential calculus | 161

Proof. Consider the function ψ : X × Y → ℝ ∪ {+∞} defined by ψ(x, y) := φ(y) + IGr(L) (x, y), where Gr(L) is the graph of the operator L. Clearly, IGr(L) (x, y) = 0 if (x, y) ∈ Gr(L) and +∞ otherwise. From the definition of the subdifferential, it follows that z⋆ ∈ 𝜕φ(Lx) if and only if (z⋆ , 0) ∈ 𝜕ψ(x, Lx). Let yo ∈ dom φ ∩ R(L) be a point of continuity of φ. Then there exists xo ∈ X such that yo = Lxo and (xo , Lxo ) is a point of continuity of ψ. We may then apply Theorem 4.3.2 to ψ. Since 𝜕ψ(x, y) = 𝜕φ(y) + 𝜕IGr(L) (x, y), it follows that any (z⋆ , 0) ∈ 𝜕ψ(x, Lx) can be expressed as (z⋆ , 0) = (0, y⋆1 ) + (x⋆2 , y⋆2 ), with x⋆2 = z⋆ and y⋆1 + y⋆2 = 0, where y⋆1 ∈ 𝜕φ(Lx) and (x⋆2 , y⋆2 ) are such that ⟨x⋆2 , z − x⟩ + ⟨y⋆2 , Lz − Lx⟩Y ⋆ ,Y ≤ 0,

∀ z ∈ X,

(4.17)

and we have used the notation ⟨⋅, ⋅⟩ and ⟨⋅, ⋅⟩Y ⋆ ,Y to clarify that the first duality pairing is the duality pairing between X and X ⋆ , whereas the second is between Y and Y ⋆ . A shorthand notation for that is (x⋆2 , y⋆2 ) ∈ NGr(L) (x, Lx), the normal cone of the graph of the operator L at the point (x, Lx). Using the linearity of L and the definition of the adjoint operator, (4.17) yields ⟨x⋆2 , z − x⟩ + ⟨L⋆ y⋆2 , z − x⟩ ≤ 0,

∀ z ∈ X,

from which choosing z = x ± zo ∈ X for arbitrary zo ∈ X we conclude that x⋆2 = −L⋆ y⋆2 . Therefore, z⋆ = L⋆ y⋆1 ∈ L⋆ 𝜕φ(Lx). This implies L⋆ 𝜕φ(Lx) ⊃ 𝜕(φ ∘ L)(x),

∀ x ∈ X,

which when combined with Proposition 4.3.1(iii) yields the desired equality. Remark 4.3.5. Since by Corollary 2.3.21 convex functions defined on open subsets of finite dimensional spaces are continuous, subdifferential calculus simplifies considerably for the subdifferentials of finite dimensional convex functions. Example 4.3.6 (Subdifferential of the ℓ1 norm in ℝd ). Consider X = ℝd , endowed with the norm ‖x‖1 = |x1 | + ⋅ ⋅ ⋅ |xd |, where x = (x1 , . . . , xd ) and X ⋆ ≃ ℝd . This is a norm which finds many applications in various fields such as optimization with sparse constraints, machine learning, etc. The subdifferential of the function φ : X = ℝd → ℝ, defined by φ(x) = ‖x‖1 is 𝜕φ(x) = {x⋆ ∈ ℝd : xi⋆ = sgn(xi ), if xi ≠ 0, xi⋆ ∈ [−1, 1] if xi = 0}, or equivalently the cartesian product 𝜕φ(x) = A1 (x1 ) × ⋅ ⋅ ⋅ × Ad (xd ), where Ak (xk ) = 𝜕ϕk (xk ), where ϕk : ℝ → ℝ is defined as ϕk (xk ) = |xk |.

162 | 4 Nonsmooth analysis: the subdifferential There are several ways to prove this result. We will use the following observation: x⋆ = (x1⋆ , . . . xd⋆ ) ∈ 𝜕φ(x) if and only if xi⋆ ∈ 𝜕ϕi (xi ), for every i = 1, . . . , d where ϕi : ℝ → ℝ is defined by ϕi (xi ) = |xi |. Indeed, assume that x ⋆ = (x1⋆ , . . . xd⋆ ) ∈ 𝜕φ(x), so that ‖z‖1 − ‖x‖1 ≥ ⟨x⋆ , z − x⟩,

z ∈ ℝd ,

(4.18)

where ⟨x, z⟩ = x ⋅ z, the usual inner product in ℝd . For any i = 1, . . . , d, choose a zi󸀠 , and select z = x + (zi󸀠 − xi )ei , where ei is the unit vector in the ith direction. Clearly, for this choice of z, we have that ‖z‖1 − ‖x‖1 = |zi󸀠 | − |xi | and ⟨x⋆ , z − x⟩ = xi⋆ (zi󸀠 − xi ), so that (4.18) yields |zi󸀠 | − |xi | ≥ xi⋆ (zi󸀠 − xi ), and since zi󸀠 ∈ ℝ is arbitrary we conclude that xi⋆ ∈ 𝜕ϕi (xi ). For the converse, let x = (x1 , . . . , xd ) ∈ ℝd , x⋆ = (x1⋆ , . . . , xd⋆ ) ∈ ℝd and assume that xi⋆ ∈ 𝜕ϕ(xi ), for any i = 1, . . . , d, so that |zi | − |xi | ≥ xi⋆ (zi − xi ) for every xi ∈ ℝ. Adding over all i = 1, . . . , d, we recover (4.18), therefore, x⋆ ∈ 𝜕φ(x). Our claim now follows easily for the above observation and Example 4.1.3. ◁

4.4 The subdifferential and the duality map Throughout this section, let X be a Banach space with norm ‖ ⋅ ‖, X ⋆ its dual with norm ‖ ⋅ ‖X ⋆ , and ⟨⋅, ⋅⟩ the duality pairing between them. We will study the connection of ⋆ the subdifferential of a norm related functional with the duality map J : X → 2X (see Definition 1.1.16) and explore its connections with the geometry of Banach spaces [45]. We start by defining the set valued mapping {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ = 1, ⟨x⋆ , x⟩ = ‖x‖}, if x ≠ 0, J1 (x) = { BX ⋆ (0, 1), if x = 0,

(4.19)

and the family of set valued maps Jp : X → 2X defined by ⋆

󵄩 󵄩 Jp (x) = {x⋆ ∈ X ⋆ : 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ = ‖x‖p−1 , ⟨x⋆ , x⟩ = ‖x‖p },

p > 1,

(4.20)

where it clearly holds that Jp (0) = {0}, p > 1. For p = 2, J2 coincides with the duality map (see Definition 1.1.16), 󵄩 󵄩 󵄩 󵄩 J2 (x) = J(x) = {x⋆ ∈ X ⋆ : ⟨x⋆ , x⟩ = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖x‖, ‖x‖ = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ }, while Jp (x) = ‖x‖p−1 J1 (x),

∀ x ∈ X, p > 1.

With the use of the maps Jp , p ≥ 1, we can generalize Examples 4.1.3 and 4.1.4 in the general setting of Banach spaces.

4.4 The subdifferential and the duality map

| 163

Proposition 4.4.1 (The subdifferential of powers of the norm). Define the maps φp : X → ℝ by φp (x) = p1 ‖x‖p , p ≥ 1, where (as already stated) X is a Banach space. Then 𝜕φp (x) = Jp (x), p ≥ 1. Proof. We break the proof into two steps: in step 1, we consider the case p = 1 and in step 2 the case p > 1. 1. We let p = 1, and noting the definition of J1 we need to consider the cases x ≠ 0 and x = 0 separately, starting with the former. We first establish that J1 (x) ⊂ 𝜕φ1 (x). Indeed, for any x⋆ ∈ J1 (x), x ≠ 0, using (4.19), it holds that for any z ∈ X, ‖z‖ − ‖x‖ = ‖z‖ − ⟨x⋆ , x⟩ = ‖z‖ − ⟨x⋆ , z⟩ + ⟨x⋆ , z − x⟩ 󵄨 󵄨 󵄩 󵄩 ≥ ‖z‖ − 󵄨󵄨󵄨⟨x⋆ , z⟩󵄨󵄨󵄨 + ⟨x⋆ , z − x⟩ ≥ ‖z‖ − 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖z‖ + ⟨x⋆ , z − x⟩ 󵄩 󵄩 = (1 − 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ )‖z‖ + ⟨x⋆ , z − x⟩ = ⟨x⋆ , z − x⟩, where in the last inequality we used once more (4.19). Since z ∈ X is arbitrary, we conclude that ‖z‖ − ‖x‖ ≥ ⟨x⋆ , z − x⟩ for every z ∈ X; hence, x⋆ ∈ 𝜕φ1 (x). For the reverse inclusion, 𝜕φ1 (x) ⊂ J1 (x), consider any x⋆ ∈ 𝜕φ1 (x). By the definition of the subdifferential, ‖z‖ ≥ ‖x‖ + ⟨x⋆ , z − x⟩,

∀ z ∈ X.

(4.21)

Setting first z = 0 and second z = 2x in (4.21), we obtain respectively that ⟨x⋆ , x⟩ ≥ ‖x‖ and ⟨x⋆ , x⟩ ≤ ‖x‖, from which follows that ⟨x⋆ , x⟩ = ‖x‖. It remains to show that ‖x⋆ ‖X ⋆ = 1. Combining (4.21) with ⟨x⋆ , x⟩ = ‖x‖, we see that ‖z‖ ≥ ⟨x⋆ , z⟩ for every z ∈ X. This implies that,12 ‖x⋆ ‖X ⋆ = 1 from which it follows that x⋆ ∈ J1 (x) thus proving the reverse inclusion. Let x = 0. If x⋆ ∈ J1 (0) = BX ⋆ (0, 1), then clearly ⟨x⋆ , z⟩ ≤ ‖z‖ for every z ∈ X ⋆ ; hence, x⋆ ∈ 𝜕φ(0). For the reverse inclusion, x⋆ ∈ 𝜕φ1 (0) implies that ‖z‖ ≥ ⟨x⋆ , z⟩ for 1 z ∈ BX (0, 1) we see that 1 ≥ ⟨x⋆ , z󸀠 ⟩, every z ∈ X, and dividing by ‖z‖ and setting z󸀠 = ‖z‖ which is true for any x⋆ ∈ BX ⋆ (0, 1). 2. We now let p > 1. As in step 1, we will consider the cases x ≠ 0 and x = 0 separately, starting with the former. Let x ≠ 0 and assume that x⋆ ∈ 𝜕φp (x). Then by definition 1 p 1 p ‖z‖ − ‖x‖ ≥ ⟨x⋆ , z − x⟩, p p

∀ z ∈ X.

(4.22)

12 Dividing both sides of ‖z‖ ≥ ⟨x⋆ , z⟩ by ‖z‖, and taking the supremum, we conclude that x ‖x⋆ ‖X ⋆ = supz∈B (0,1) ⟨x⋆ , z⟩ ≤ 1. On the other hand x̄ = ‖x‖ ∈ BX (0, 1) and ⟨x⋆ , x⟩̄ = 1; hence, X ⋆ supz∈B (0,1) ⟨x , z⟩ = 1. X

164 | 4 Nonsmooth analysis: the subdifferential Choosing z = (1 + ϵ)x, for ϵ > 0, (4.22) yields 1 (1 + ϵ)p − 1 p ‖x‖ ≥ ⟨x⋆ , x⟩, p ϵ and passing to the limit as ϵ → 0+ we conclude that ‖x‖p ≥ ⟨x⋆ , x⟩. Choosing next z = (1 − ϵ)x, ϵ > 0 in (4.22) and rearranging gives 1 (1 − ϵ)p − 1 p ‖x‖ ≥ −⟨x⋆ , x⟩, p ϵ and passing to the limit as ϵ → 0+ we conclude that ‖x‖p ≤ ⟨x⋆ , x⟩. Combining the two above inequalities, we get that ‖x‖p = ⟨x⋆ , x⟩. Then, clearly ‖x‖p = ⟨x⋆ , x⟩ = |⟨x⋆ , x⟩| ≤ ‖x⋆ ‖X ⋆ ‖x‖, and dividing both sides by ‖x‖, we obtain that ‖x‖p−1 ≤ ‖x⋆ ‖X ⋆ . We claim that the reverse inequality also holds, i. e., that ‖x⋆ ‖X ⋆ ≤ ‖x‖p−1 , therefore, ‖x‖p−1 = ‖x⋆ ‖X ⋆ . To prove the claim, consider any z ∈ X, such that ‖z‖ ≤ ‖x‖ and apply (4.22). We see that ⟨x⋆ , z − x⟩ ≤ 0, and recalling that ⟨x⋆ , x⟩ = ‖x‖p , we obtain that ⟨x⋆ , z⟩ ≤ 1 ‖x‖p and dividing both sides with ‖x‖, setting z󸀠 = ‖x‖ z, we conclude that ⟨x⋆ , z󸀠 ⟩ ≤ ‖x‖p−1 . Since z ∈ X is arbitrary (with ‖z‖ ≤ ‖x‖), z󸀠 an arbitrary element z󸀠 ∈ BX (0, 1). Furthermore, repeating the same argument with −z in the place of z, we conclude that |⟨x⋆ , z󸀠 ⟩| ≤ ‖x‖p−1 , for every z󸀠 ∈ BX (0, 1); hence, taking the supremum over all such z󸀠 and recalling the definition of ‖x⋆ ‖X ⋆ , we conclude that ‖x⋆ ‖X ⋆ ≤ ‖x‖p−1 , which is the required claim. Hence, for any x⋆ ∈ 𝜕φp (x), it holds that ⟨x⋆ , x⟩ = ‖x‖p and ‖x⋆ ‖X ⋆ = ‖x‖p−1 , which implies that x⋆ ∈ Jp (x), and we have that 𝜕φp (x) ⊂ Jp (x). To prove the reverse inclusion, recall the elementary inequality p−1 ap + p1 bp ≥ p

ap−1 b, which is valid13 for any a, b > 0 and p > 1. Consider any x⋆ ∈ Jp (x), i. e., ⟨x⋆ , x⟩ = ‖x‖p and ‖x⋆ ‖X ⋆ = ‖x‖p−1 . Rearrange p1 ‖z‖p − p1 ‖x‖p = p1 ‖z‖p + p−1 ‖x‖p − ‖x‖p , and using p p p the above inequality for a = ‖x‖ , b = ‖z‖ we obtain 1 p 1 p ‖z‖ − ‖x‖ ≥ ‖x‖p−1 ‖z‖ − ‖x‖p p p 󵄩 󵄩 = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖z‖ − ⟨x⋆ , x⟩ ≥ ⟨x⋆ , z⟩ − ⟨x⋆ , x⟩ = ⟨x⋆ , z − x⟩, where in the second line we used the fact that x⋆ ∈ Jp (x) and ⟨x⋆ , z⟩ ≤ ‖x⋆ ‖X ⋆ ‖z‖. Hence, by the fact that z is arbitrary we conclude that for any x⋆ ∈ Jp (x) it holds that 1 ‖z‖p − p1 ‖x‖p ≥ ⟨x⋆ , z − x⟩ for every z ∈ X, i. e., x⋆ ∈ 𝜕φp (x) and Jp (x) ⊂ 𝜕φp (x). p In the case where x = 0, if x⋆ ∈ 𝜕φp (0), then p1 ‖z‖p ≥ ⟨x⋆ , z⟩ for every z ∈ X.

Consider any x ∈ X and choosing first z = ϵx for any ϵ > 0, we get whereas choosing z = −ϵx for any ϵ > 0 we get

ϵ

p−1

p

ϵp−1 ‖x‖p p

≥ ⟨x⋆ , x⟩,

‖x‖p ≥ −⟨x⋆ , x⟩, from which we

13 This follows by a simple convexity argument. Since φ(s) = − ln(s) is a convex function, we have that − ln( p−1 ap + p1 bp ) ≤ − p−1 ln ap − p1 ln bp , which upon rearrangement and exponentiation leads to p p the stated inequality.

4.4 The subdifferential and the duality map

| 165

p−1

conclude that |⟨x⋆ , x⟩| ≤ ϵ p ‖x‖p . Passing to the limit as ϵ → 0+ and since x ∈ X is arbitrary we conclude that ⟨x⋆ , x⟩ = 0 for every x ∈ X, therefore, x⋆ = 0. Hence, 𝜕φp (0) = {0} = Jp (0). By the converse of Proposition 4.1.2 (see Example 4.2.10), we may remark that since J1 (x) = 𝜕‖x‖, if J1 is a single valued mapping on X \ {0} then the norm will be a Gâteaux differentiable function at any point except the origin. With a similar reasoning, if Jp , p > 1, is a single valued mapping on X, then the function φp (x) = p1 ‖x‖p , will be Gâteaux differentiable everywhere. On the other hand, the geometry of the Banach space X (in the sense of Section 2.6) is reflected on the properties of the duality mapping, as the following proposition indicates, therefore, leading to connections between the geometry of the space and the differentiability of the norm. Proposition 4.4.2. Let X be a Banach space, with strictly convex dual X ⋆ . Then for any p > 1, the mappings Jp are single valued, while J1 is single valued in X \ {0}. Proof. Consider the case p > 1 first. Since Jp (0) = {0}, we only need to focus on x ≠ 0. Assume, on the contrary that x⋆1 , x⋆2 ∈ Jp (x), x⋆1 ≠ x⋆2 . By the definition of Jp , it holds that ‖x⋆1 ‖X ⋆ = ‖x⋆2 ‖X ⋆ = ‖x‖p−1 and ⟨x⋆1 , x⟩ = ⟨x⋆2 , x⟩ = ‖x‖p , and combining these we obtain that 󵄩 󵄩 ⟨x⋆1 + x⋆2 , x⟩ = ⟨x⋆1 , x⟩ + ⟨x⋆2 , x⟩ = 2‖x‖p = 2‖x‖p−1 ‖x‖ = 2󵄩󵄩󵄩x⋆1 󵄩󵄩󵄩X ⋆ ‖x‖. Hence, 󵄩 󵄩 󵄩 󵄩 2󵄩󵄩󵄩x⋆1 󵄩󵄩󵄩X ⋆ ‖x‖ = ⟨x⋆1 + x⋆2 , x⟩ ≤ 󵄩󵄩󵄩x⋆1 + x⋆2 󵄩󵄩󵄩X ⋆ ‖x‖, and dividing both sides by ‖x‖ we obtain that 󵄩󵄩 ⋆ 󵄩󵄩 󵄩 ⋆󵄩 󵄩 ⋆󵄩 󵄩 ⋆ ⋆󵄩 󵄩󵄩x1 󵄩󵄩X ⋆ + 󵄩󵄩󵄩x2 󵄩󵄩󵄩X ⋆ = 2󵄩󵄩󵄩x1 󵄩󵄩󵄩X ⋆ ≤ 󵄩󵄩󵄩x1 + x2 󵄩󵄩󵄩X ⋆ . Combining this with the triangle inequality, we immediately see that ‖x⋆1 + x⋆2 ‖X ⋆ = ‖x⋆1 ‖X ⋆ + ‖x⋆2 ‖X ⋆ , which by the strict convexity of X ⋆ , using Theorem 2.6.5(i󸀠 ), implies that x⋆1 = λx⋆2 for some λ > 0, and since ‖x⋆1 ‖X ⋆ = ‖x⋆2 ‖X ⋆ it must hold that x⋆1 = x⋆2 , which is a contradiction. A similar argument applies for J1 . We introduce the concept of a smooth Banach space. Definition 4.4.3 (Smoothness). A normed space X is called smooth at a point xo ∈ SX (0, 1) if there exists a unique x⋆o ∈ SX ⋆ (0, 1) such that ⟨x⋆o , xo ⟩ = 1. It is called smooth if this property holds for every x ∈ SX (0, 1). The Gâteaux differentiability of the norm is equivalent to the smoothness of the space. Proposition 4.4.4. Let X be a Banach space. Then the norm is Gâteaux differentiable on X \ {0} if and only if X is smooth.

166 | 4 Nonsmooth analysis: the subdifferential Proof. Note that by the positive homogeneity of the norm it suffices to restrict attention to SX (0, 1). We start with the observation that an equivalent way to express smoothness is that x⋆ 󳨃→ ⟨x⋆ , x⟩ achieves its maximum value at a unique x⋆ ∈ SX ⋆ (0, 1). Assume that X is smooth. Let φ(x) = ‖x‖, and considering x ∈ SX (0, 1), we note that by Proposition 4.2.7, 𝜕φ(x) = J1 (x) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ = 1, ⟨x⋆ , x⟩ = 1}, so that by the smoothness of X, 𝜕φ(x) is a singleton and by the converse of Proposition 4.1.2 (see Example 4.2.10) the norm is Gâteaux differentiable. For the converse, we proceed in the same fashion. Using Proposition 4.4.2, we may conclude the following proposition which connects the smoothness of X with the geometry of X ⋆ . Proposition 4.4.5. Let X be a Banach space with strictly convex dual X ⋆ . Then the norm is Gâteaux differentiable on X \ {0} and X is smooth. Proof. Since X ⋆ is strictly convex, by Proposition 4.4.2, J1 is single valued on X \ {0}, hence, by Proposition 4.4.1 the subdifferential of the function φ(x) = ‖x‖ is a singleton, therefore, by the converse of Proposition 4.1.2 (see Example 4.2.10), the norm is Gâteaux differentiable at every x ≠ 0; hence, by Proposition 4.4.4, X is smooth. Remark 4.4.6 (Fréchet differentiability of the norm). Under stronger conditions, we may prove that the norm is Fréchet differentiable. In particular, if X is a Banach space with locally uniformly convex dual X ⋆ , then X has a Fréchet differentiable norm (see, e. g., [57]).

4.5 Approximation of the subdifferential and density of its domain We have seen in Proposition 4.2.1 that a proper convex lower semicontinuous function is subdifferentiable at its points of continuity. Two interesting questions are (a) what happens at the points where continuity does not hold, and (b) how is the set of points in which φ is subdifferentiable, i. e., D(𝜕φ), related with the set of points in which φ is bounded, i. e., dom φ (which of course by the general theory of convex functions is related to the set of points in which φ is continuous). The answer to these two important questions can be addressed via an approximation procedure for the subdifferential, which requires only lower semicontinuity. Definition 4.5.1 (ϵ-subdifferential). Let φ : X → ℝ ∪ {+∞} be a proper convex lower semicontinuous function, and consider x ∈ dom φ. For any ϵ > 0, the ϵ-subdifferential of φ at x, denoted by 𝜕ϵ φ(x) is defined as the set 𝜕ϵ φ(x) := {z⋆ ∈ X ⋆ : φ(z) − φ(x) + ϵ ≥ ⟨z⋆ , z − x⟩, ∀ z ∈ X}. Example 4.5.2 (ϵ-subdifferential of the indicator function of a convex set). Consider a convex set C ⊂ X and the indicator function of C, IC : X → ℝ ∪ {+∞}. Then

4.5 Approximation of the subdifferential and density of its domain

| 167

𝜕ϵ IC (x) = Nϵ,C (x) := {x⋆ ∈ X ⋆ : ⟨x⋆ , z − x⟩ ≤ ϵ, ∀ z ∈ C}, for any x ∈ C. This set is a generalization of the normal cone NC (x) (see Definition 4.1.5 and compare with Example 4.1.6). ◁ How is the ϵ-subdifferential related to the subdifferential of a function? It can easily be seen that for any x ∈ X, it holds that 𝜕φ(x) = ⋂ϵ>0 𝜕ϵ φ(x), a fact easily proved by the definitions. Furthermore, the nonemptiness of 𝜕ϵ φ(x) for any x ∈ dom φ follows by the Hahn–Banach separation theorem,14 and working similarly as for the subdifferential, one may prove that 𝜕ϵ φ(x) is a weak⋆ closed and convex set (details are left as an exercise). Finally, for any ϵ1 < ϵ2 it holds that 𝜕ϵ1 φ(x) ⊂ 𝜕ϵ2 φ(x), while t𝜕ϵ1 (x) + (1 − t)𝜕ϵ2 (x) ⊂ 𝜕tϵ1 +(1−t)ϵ2 φ(x), for every ϵ1 , ϵ2 > 0, t ∈ [0, 1]. The ϵ-subdifferential of φ provides an approximation of the subdifferential in the sense of Theorem 4.5.3 below, which also guarantees the density of the domain of the subdifferential in dom φ. This result was first proved in [30]. The proof presented here is the one proposed in [89] (Theorem 3.17) and slightly different than the original proof of Brøndsted and Rockafellar (which does not use the Ekeland variational principle). Theorem 4.5.3. Let X be a Banach space and suppose φ : X → ℝ ∪ {+∞} is proper convex and lower semicontinuous. Given any x ∈ dom φ, any ϵ > 0, any x⋆ ∈ 𝜕ϵ φ(x) then, for any λ > 0, there exists xλ ∈ X and x⋆λ ∈ 𝜕φ(xλ ) such that ϵ , λ ≤ λ.

‖xλ − x‖ ≤ 󵄩󵄩 ⋆ ⋆󵄩 󵄩󵄩xλ − x 󵄩󵄩󵄩X ⋆

(4.23)

This approximation guarantees the density of D(𝜕φ) in dom φ. Proof. Since x⋆ ∈ 𝜕ϵ φ(x) it holds that φ(z) − φ(x) + ϵ ≥ ⟨x⋆ , z − x⟩ for every z ∈ X, which we rearrange as φ(z) − ⟨x⋆ , z⟩ + ϵ ≥ φ(x) − ⟨x⋆ , x⟩. Upon defining the (proper lower semicontinuous convex) function ψ : X → ℝ ∪ {+∞}, by ψ(z) := φ(z) − ⟨x⋆ , z⟩ for every z ∈ X, we express this inequality as ψ(z) + ϵ ≥ ψ(x) for every z ∈ X; hence, this is true also for the infimum over all z ∈ X, so that ψ(x) ≤ infz∈X ψ(z) + ϵ. We now use the Ekeland variational principle (EVP),15 on ψ (taking into account the Banach space structure of X) to guarantee the existence of a xλ ∈ dom φ such that λ‖xλ − x‖ < ϵ,

ψ(xλ ) ≤ ψ(z) + λ‖xλ − z‖,

∀ z ∈ X.

(4.24)

We will show that this xλ is the one we seek, by constructing an appropriate x⋆λ ∈ 𝜕φ(xλ ). 14 Apply the separation theorem to the set epi φ and the point (x, φ(x) − ϵ) ∈ ̸ epi φ. 15 If δ󸀠 , ϵ󸀠 are the parameters in Theorem 3.6.1, we use the scaling δ󸀠 = ϵ and ϵ1󸀠 = λϵ , so that ϵ󸀠 δ󸀠 = λ. As ϵ is considered given, we relabel the point xϵ󸀠 ,δ󸀠 , chosen by EVP, as xλ .

168 | 4 Nonsmooth analysis: the subdifferential Define the function ϕ : X → ℝ ∪ {+∞} by ϕ(z) := λ‖z − xλ ‖, for every z ∈ X, and since ϕ(xλ ) = 0, the second inequality of (4.24) implies that 0 ∈ 𝜕(ψ + ϕ)(xλ ). However, since ϕ is continuous, by Theorem 4.3.2, we have that 𝜕(ψ + ϕ)(xλ ) = 𝜕ψ(xλ ) + 𝜕ϕ(xλ ); hence, (4.25)

0 ∈ 𝜕ψ(xλ ) + 𝜕ϕ(xλ ).

By the definition of ψ, it follows that 𝜕ψ(zλ ) = 𝜕φ(zλ ) − x⋆ . Furthermore, by subdifferential calculus and Proposition 4.4.1, we have that 󵄩 󵄩 𝜕ϕ(zλ ) = {z⋆ ∈ X ⋆ : 󵄩󵄩󵄩z⋆ 󵄩󵄩󵄩 ≤ λ} = BX ⋆ (0, λ). We can therefore express (4.25) as 0 ∈ 𝜕φ(xλ ) − x⋆ + BX ⋆ (0, λ).

(4.26)

Hence, there exists x⋆λ ∈ 𝜕φ(xλ ) and z⋆ ∈ BX ⋆ (0, λ) such that 0 = x⋆λ − x⋆ + z⋆ , which implies that x⋆ − x⋆λ = z⋆ , therefore, ‖x⋆ − x⋆λ ‖X ⋆ ≤ λ. This x⋆λ is the one we seek and completes the proof of the first claim. The second claim follows by the first one and a proper choice of a sequence of λ’s.

4.6 The subdifferential and optimization The first-order conditions may be generalized in terms of the subdifferential. Proposition 4.6.1. Let X be Banach space and φ : X → ℝ∪{+∞} a proper convex lower semicontinuous function. Then, xo ∈ X is a minimum of φ if and only if 0 ∈ 𝜕φ(xo ). Proof. If xo is a minimum of φ then φ(xo ) ≤ φ(z) for every z ∈ X, and this can trivially be expressed as φ(z) − φ(xo ) ≥ 0 = ⟨0, z − xo ⟩,

∀ z ∈ X,

which implies that 0 ∈ 𝜕φ(xo ). The converse follows directly from the definition of 𝜕φ. On the other hand, by replacing the subdifferential with the ϵ-subdifferential, we obtain an “approximate” minimization result. Proposition 4.6.2. Let φ : X → ℝ ∪ {+∞} be a proper convex lower semicontinuous function. Then 0 ∈ 𝜕ϵ φ(xo ) if and only if inf φ(x) ≥ φ(xo ) − ϵ.

x∈X

(4.27)

Furthermore, if xo is an approximate minimum satisfying (4.27), then for any ϵ1 , ϵ2 > 0, such that ϵ = ϵ1 ϵ2 , there exist x ∈ X, x⋆ ∈ 𝜕φ(x) with ‖x − xo ‖ < ϵ2 and ‖x⋆ ‖X ⋆ ≤ ϵ1 .

4.6 The subdifferential and optimization

| 169

Proof. If 0 ∈ 𝜕ϵ φ(xo ), then φ(z) − φ(xo ) + ϵ ≥ 0, for every z ∈ X, so upon rearranging and taking the infimum over all z ∈ X, (4.27) is proved. For the converse, since (4.27) holds, φ(z) − φ(xo ) + ϵ ≥ 0, for every z ∈ X, so that 0 ∈ 𝜕ϵ φ(xo ). For the second claim, choose ϵo > 0 such that φ(xo ) − infx∈X φ(x) ≤ ϵo ≤ ϵ1 ϵ2 , ϵ and choose λ so that ϵo ≤ λ ≤ ϵ1 . For this choice of ϵo , it holds that 0 ∈ 𝜕ϵo φ(xo ), so 2 that applying the Brøndsted–Rockafellar Theorem 4.5.3 we conclude the existence of ϵ x ∈ X, x⋆ ∈ 𝜕φ(x) such that ‖x − xo ‖ ≤ λo ≤ ϵ2 and ‖x⋆ ‖X ⋆ ≤ λ ≤ ϵ1 . Consider now a constrained convex minimization problem of the form infx∈C φ(x) where C ⊂ X is a closed and convex subset of the Banach space X, and φ : X → ℝ is a convex function. Instead of the original constrained problem, we may consider the unconstrained problem infx∈X (φ(x) + IC (x)) where the original function φ is now perturbed by the indicator function of the convex set C. By the definition of the indicator function, if x ∈ C, then the minimizer of the original and the perturbed problem coincide, whereas if x ∈ ̸ C then the perturbed problem does not have a solution. Therefore, in some sense, the perturbed problem is a penalized version of the original one, with the indicator function playing the role of the penalty function. The following proposition offers a generalization of the Karusch–Kuhn–Tucker conditions for the optimization of nonsmooth convex functions under convex constraints in Banach space. Proposition 4.6.3. Let C ⊂ X be a closed convex set, φ : X → ℝ ∪ {+∞} be a lower semicontinuous convex function such that φ is continuous at some point of C (cont(φ) ∩ C ≠ 0 or dom(φ) ∩ int(C) ≠ 0) and consider the problem inf φ(x).

x∈C

(4.28)

Then xo is a solution of (4.28) if and only if there exist z⋆1 ∈ 𝜕φ(xo ) such that ⟨z⋆1 , z−xo ⟩ ≥ 0, for every z ∈ C, or stated differently if and only if there exists z⋆1 ∈ 𝜕φ(xo ) and z⋆2 such that ⟨z⋆2 , z − xo ⟩ ≤ 0 for every z ∈ C, i. e., z⋆2 ∈ NC (xo ), where NC (xo ) is the normal cone of C at xo , with z⋆1 + z⋆2 = 0. Proof. Let ψ : X → ℝ be defined as ψ(x) = φ(x) + IC (x) for any x ∈ X. Then, by subdifferential calculus, 𝜕ψ(x) = 𝜕φ(x)+𝜕IC (x) and 𝜕IC (x) = NC (x) = {x⋆ ∈ X ⋆ : ⟨x⋆ , z− x⟩ ≤ 0, ∀ z ∈ C}. By Proposition 4.6.1, xo is a solution of the constrained problem if and only if 0 ∈ 𝜕ψ(xo ), therefore, if 0 ∈ 𝜕φ(xo ) + 𝜕IC (xo ). This implies that there exist z⋆1 ∈ 𝜕φ(xo ) and z⋆2 ∈ 𝜕IC (xo ), i. e., z⋆2 ∈ X ⋆ such that ⟨z⋆2 , z − xo ⟩ ≤ 0, ∀ z ∈ C such that z⋆1 + z⋆2 = 0. Example 4.6.4 (The projection operator revisited). Let X = H, a Hilbert space, whose dual X ⋆ ≃ X, C ⊂ X be a convex closed subset of X, and consider the following approximation problem: Given x ∈ X find xo ∈ C such that ‖x − xo ‖ = infz∈C ‖x − z‖. This is the problem of best approximation of any element x ∈ X by an element xo ∈ C. We have

170 | 4 Nonsmooth analysis: the subdifferential already encountered this problem before and recognized its solution as the projection of x on C. We will now revisit it using Proposition 4.6.3. The approximation problem is equivalent to minz∈C 21 ‖x − z‖2 . Let us define the convex function φ : X → ℝ, by z 󳨃→ 21 ‖x − z‖2 . By direct calculation, Dφ(z) = z − x for any z ∈ X and a straightforward application of Proposition 4.6.3 leads us to the conclusion that there must be a z⋆1 = xo −x such that ⟨z⋆1 , z −xo ⟩ ≥ 0, ∀ z ∈ C. This leads us to a representation of xo in terms of the variational inequality ⟨x − xo , z − xo ⟩ ≤ 0 for every z ∈ C, which allows us to deduce that xo = PC (x) (see Theorem 2.5.2). The projection operator can be generalized in a Banach space context under restrictive conditions on X (e. g., if X is reflexive and X and X ⋆ are uniformly convex) in which case the operator assigning to each x ∈ X the element PC (x) = xo = arg minz∈C ‖z − x‖ is well posed and single valued, and is characterized by the variational inequality ⟨−J(xo − x), xo − z⟩ ≥ 0 for every z ∈ C or in equivalent form ⟨J(x − xo ), xo − z⟩ ≥ 0 for every z ∈ C, where J : X → X ⋆ is the duality map. However, this operator does not share the nice properties of the projection in Hilbert space (e. g., it may not be a linear operator even when C = E, a closed linear subspace). Note that this formulation is also useful in the case where X is a Hilbert space but not identified with its dual. ◁ Example 4.6.5 (Optimization problems with affine constraints). Let X, Y be Banach spaces, L : X → Y be a linear continuous operator, φ : X → ℝ ∪ {+∞} be a proper lower semicontinuous function that is continuous at some point in C, and consider convex minimization problems with affine constraints of the form min φ(x), x∈C

C = {x ∈ X : Lx = y}.

(4.29)

Let xo ∈ C be a minimizer of (4.29). In this case, the normal cone of C is NC (x) = (N(L))⊥ for every x ∈ C (see Example 4.1.7) so that the first- order condition becomes, 0 ∈ 𝜕φ(xo ) + NC (xo ), where the additivity of the subdifferential follows by the conditions on φ and L. This condition implies that xo ∈ C solves (4.29), if and only if there exists z⋆o ∈ (N(L))⊥ , such that −z⋆o ∈ 𝜕φ(xo ). If we assume furthermore that L has closed range, then by the general theory of linear operators we know that (N(L))⊥ = R(L⋆ ), where L⋆ : Y ⋆ → X ⋆ is the adjoint operator. In this case, the solvability condition reduces to the existence of some y⋆o ∈ Y ⋆ such that −L⋆ y⋆o ∈ 𝜕φ(xo ). ◁

4.7 The Moreau proximity operator in Hilbert spaces 4.7.1 Definition and fundamental properties For this entire section, we let X = H be a Hilbert space and φ : H → ℝ ∪ {+∞} be a proper convex lower semicontinuous function.

4.7 The Moreau proximity operator in Hilbert spaces | 171

Fix a x ∈ H and consider the minimization problem min φx (z), z∈H

where,

1 φx (z) := ‖z − x‖2 + φ(z), 2

which by the strict convexity of φx admits a unique solution xo . In terms of this solution, we may define the single valued operator proxφ : H → H by proxφ x := xo . This operator is called the proximity operator or Moreau proximity operator after the fundamental contribution of [84]. Definition 4.7.1. The proximity operator proxφ : H → H is defined by 1 proxφ x := xo = arg min( ‖z − x‖2 + φ(z)). z∈H 2 The proximity operator is a generalization of the projection operator. Indeed, in the special case where φ = IC , the indicator function of a closed, convex set C, the proximity operator proxφ coincides with the projection operator PC . Furthermore, the family of operators {Jλ : λ > 0} := {proxλφ : λ > 0}, often called resolvent of the subdifferential of φ, plays an important role in an approximation procedure of proper convex lower semicontinuous functions, called the Moreau–Yosida approximation. We state some fundamental properties of the operator proxφ (see e. g. [20]). Proposition 4.7.2 (Properties of the proximity operator). Let H be a Hilbert space with inner product ⟨⋅, ⋅⟩, consider φ : H → ℝ ∪ {+∞} a proper convex lower semicontinuous function and let proxφ : H → H be the corresponding proximity operator. Then: (i) proxφ is characterized by the variational inequality ⟨x − proxφ x, z − proxφ x⟩ ≤ φ(z) − φ(proxφ x),

∀ z ∈ H.

(4.30)

or its equivalent form ⟨proxφ x − x, z − proxφ x⟩ + φ(z) − φ(proxφ x) ≥ 0,

∀ z ∈ H.

(4.31)

(ii) proxφ is single valued, enjoys the monotonicity property ⟨proxφ x1 − proxφ x2 , x1 − x2 ⟩ ≥ 0,

∀ x1 , x2 ∈ H,

(4.32)

∀ x1 , x2 ∈ H.

(4.33)

is nonexpansive, i. e., ‖proxφ x1 − proxφ x2 ‖ ≤ ‖x1 − x2 ‖, and in particular firmly nonexpansive,16 i. e., ‖proxφ x1 − proxφ x2 ‖2 ≤ ⟨x1 − x2 , proxφ x1 − proxφ x2 ⟩,

∀ x1 , x2 ∈ H.

(4.34)

16 This property reminds us of similar properties for the Fréchet derivative of convex functions which are Lipschitz continuous; see Example 2.3.25.

172 | 4 Nonsmooth analysis: the subdifferential (iii) The proximity operator proxφ is characterized in terms of the subdifferential of φ, and in particular xo = proxφ x

if and only if x − xo ∈ 𝜕φ(xo ).

This condition is often expressed as proxφ (x) = (I + 𝜕φ)−1 (x). An equivalent way of stating this, is that Fix(proxφ ) = arg minx∈H φ(x), where by Fix(proxφ ) we denote the set of fixed points of the proximity operator and by arg minx∈H φ(x) we denote the set of minimizers of the function φ. (iv) The operator I − proxφ is also firmly nonexpansive, i. e., 󵄩󵄩 󵄩2 󵄩󵄩(I − proxφ )(x1 ) − (I − proxφ )(x2 )󵄩󵄩󵄩 ≤ ⟨(I − proxφ )(x1 ) − (I − proxφ )(x2 ), x1 − x2 ⟩,

∀x1 , x2 ∈ H,

(4.35)

and, furthermore, ⟨proxφ (x1 ) − proxφ (x2 ), (I − proxφ )(x1 ) − (I − proxφ )(x2 )⟩ ≥ 0, ∀ x1 , x2 ∈ H.

(4.36)

Proof. (i) We will use the simplified notation xo = proxφ x. Assume first that xo satisfies the variational inequality (4.31). Then,17 for any z ∈ H, 1 1 φx (z) − φx (xo ) = ‖z − x‖2 − ‖xo − x‖2 + φ(z) − φ(xo ) 2 2 1 = ‖xo − z‖2 + ⟨xo − x, z − xo ⟩ + φ(z) − φ(xo ) 2 ≥ ⟨xo − x, z − xo ⟩ + φ(z) − φ(xo ) ≥ 0, by (4.31). Therefore, for any z ∈ H, it holds that φx (z) ≥ φx (xo ); hence, xo is the minimizer of φx , which is unique by strict convexity. For the converse, assume that xo is the minimizer of φx on H. Consider any z ∈ H; for any t ∈ (0, 1), it holds that φx ((1 − t)xo + tz) − φx (xo ) ≥ 0. By the definition of φx , this implies that 1 󵄩󵄩 󵄩2 1 2 󵄩(1 − t)xo + tz − x󵄩󵄩󵄩 − ‖xo − x‖ + φ((1 − t)xo + tz) − φ(xo ) ≥ 0, 2󵄩 2 which upon rearrangement yields 1 0 ≤ t 2 ‖z − xo ‖2 + t ⟨xo − x, z − xo ⟩ + φ((1 − t)xo + tz) − φ(xo ). 2

(4.37)

17 Using the identity 21 ‖x1 ‖2 − 21 ‖x2 ‖2 = 21 ‖x1 − x2 ‖2 + ⟨x1 − x2 , x2 ⟩, for every x1 , x2 ∈ H, for the choice x1 = z − x and x2 = xo − x.

4.7 The Moreau proximity operator in Hilbert spaces | 173

The convexity of φ implies that φ((1 − t)xo + tz) ≤ (1 − t)φ(xo ) + tφ(z), hence, φ((1 − t)xo + tz) − φ(xo ) ≤ −tφ(xo ) + tφ(z), which when combined with (4.37) yields 1 0 ≤ t 2 ‖z − xo ‖2 + t ⟨xo − x, z − xo ⟩ − tφ(xo ) + tφ(z). 2 Dividing by t and passing to the limit as t → 0+ leads to (4.31). (ii) Consider any x1 , x2 ∈ H and let zi = proxφ xi , i = 1, 2. Apply the variational inequality (4.31) first for the choice x = x1 , z = z2 and second for the choice x = x2 , z = z1 and add. The terms involving the function φ cancel thus leading to ⟨z1 − x1 , z2 − z1 ⟩ + ⟨z2 − x2 , z1 − z2 ⟩ ≥ 0, which is rearranged as 0 ≤ ‖z2 − z1 ‖2 ≤ ⟨z2 − z1 , x2 − x1 ⟩.

(4.38)

This proves (4.34) and (4.32). We further estimate the right-hand side of (4.38) using the Cauchy–Schwarz inequality, and divide both sides of the resulting inequality by ‖z2 − z1 ‖ to obtain (4.33). (iii) This can either be seen directly from the definition of the subdifferential and the characterization of xo = proxφ x, in terms of the variational inequality (4.31) or by considering φx : H → ℝ∪{+∞} and applying in a straightforward fashion the standard rules of subdifferential calculus. (iv) We use the notation in (ii), and note that 󵄩󵄩 󵄩2 󵄩󵄩(x1 − z1 ) − (x2 − z2 )󵄩󵄩󵄩 = ⟨x1 − x2 , x1 − x2 ⟩ − 2⟨x1 − x2 , z1 − z2 ⟩ + ‖z1 − z2 ‖2 (4.34)

≤ ⟨x1 − x2 , x1 − x2 ⟩ − 2⟨x1 − x2 , z1 − z2 ⟩ + ⟨x1 − x2 , z1 − z2 ⟩

= ⟨x1 − x2 , x1 − x2 ⟩ − ⟨x1 − x2 , z1 − z2 ⟩ = ⟨(x1 − z1 ) − (x1 − z2 ), x1 − x2 ⟩, (where we also used symmetry) which is the strict nonexpansive property for I −proxφ , i. e., (4.35). Similarly, (4.34)

⟨(x1 − z1 ) − (x2 − z2 ), z1 − z2 ⟩ = ⟨x1 − x2 , z1 − z2 ⟩ − ‖z1 − z2 ‖2 ≥ 0, which proves (4.36).

174 | 4 Nonsmooth analysis: the subdifferential 4.7.2 The Moreau–Yosida approximation A closely related notion to the Moreau proximity operator corresponding to some nonsmooth convex function φ, is its Moreau–Yosida approximation φλ , λ > 0, which is a family of smooth approximations to φ. This approximation is very useful in various applications including of course optimization. For its extension in Banach spaces, see Section 9.3.6. Definition 4.7.3 (Moreau–Yosida approximation). The function φλ : H → ℝ, defined by φλ (x) = min φx,λ (z) = φx,λ (proxλφ (x)), z∈H

where,

φx,λ (z) :=

1 ‖x − z‖2 + φ(z), 2λ

is called the Moreau–Yosida approximation (or envelope) of φ. The notation Jλ = proxλφ = (I + λ𝜕φ)−1 , the resolvent of 𝜕φ, is often used. The following question is interesting. Example 4.7.4 (Behavior of proxλφ as λ → 0+ ). It holds that ‖proxλφ x − x‖H → 0 as λ → 0+ . Indeed, for any λ > 0, upon defining xλ = proxλφ x we have that x − xλ ∈ λ𝜕φ(xλ ), which in turn implies that λφ(z) − λφ(xλ ) ≥ ⟨x − xλ , z − xλ ⟩H , for every z ∈ H. Choosing z = x and passing to the limit as λ → 0+ we obtain the required result. ◁ Example 4.7.5. It is straightforward to check that in the case where H = ℝ, φ(x) = |x|,

Then

x−λ { { { proxλφ x := xλ,o = {0 { { {x + λ λ

x− { { { x2 2 φλ (x) = { 2λ { { λ {−x − 2

if x > λ,

if − λ ≤ x ≤ λ, if x < −λ.

if x > λ,

if − λ ≤ x ≤ λ, if x < −λ.

This example can be generalized in the case where H = ℓ2 and φ : H → ℝ is defined by φ(x) = ‖x‖ℓ1 . Then working componentwise we obtain that if xo = proxφ,λ x, then xn − λ { { { (proxλφ x)n := (xλ,o )n = {0 { { {xn + λ

if xn > λ,

if − λ ≤ xn ≤ λ, if xn < −λ,

and it can be easily seen that xo is well-defined as an element of the sequence space. In this particular case, the proximity operation is often called the shrinkage or soft threshold operation, Sλ defined by Sλ (x) := proxλ‖⋅‖1 (x). ◁

4.7 The Moreau proximity operator in Hilbert spaces | 175

The Moreau envelope φλ , λ > 0 of a proper convex lower semicontinuous function φ is a convex function that enjoys some nice differentiability properties (see e. g. [20]), and because of that finds important applications in nonsmooth optimization. Proposition 4.7.6 (Properties of the Moreau envelope). Let φ : H → ℝ ∪ {+∞} be a proper convex lower semicontinuous function, and let φλ be its Moreau envelope. Then (i) The function φλ is convex for every λ > 0, with inf φ(z) ≤ φλ (x) ≤ φ(x),

z∈H

x ∈ H, λ > 0

(4.39)

and, therefore, infz∈H φ(z) = infz∈H φλ (z) for every λ > 0. In particular, it holds that xo ∈ argminz∈H φ(z) if and only if xo ∈ argminz∈H φλ (z) for every λ > 0, while limλ→0+ φλ = φ, pointwise. (ii) For each λ > 0, the function φλ is Fréchet differentiable, with Lipschitz continuous Fréchet derivative Dφλ , at any x ∈ H, and Dφλ (x) =

1 1 (x − proxλφ (x)) = (I − proxλφ )(x). λ λ

Proof. (i) By definition φλ (x) = infz∈H φx,λ (z) ≤ φx,λ (x) = φ(x). Furthermore, for any x, z󸀠 ∈ H, λ > 0, it holds that infz∈H φ(z) ≤ φ(z󸀠 ) ≤ φx,λ (z󸀠 ), and taking the infimum over all z󸀠 ∈ H in this inequality we conclude that infz∈H φ(z) ≤ φλ (x) for every x ∈ H, λ > 0. Thus (4.39) holds. Since infz∈H φ(z) ≤ φλ (x) for every x ∈ H, λ > 0, taking the infimum over all x ∈ H we have that inf φ(z) ≤ inf φλ (x) = inf φλ (z),

z∈H

x∈H

z∈H

∀ λ > 0.

(4.40)

On the other hand, since by (4.39), infz∈H φλ (z) ≤ φλ (x) ≤ φ(x) for every x ∈ H, λ > 0, taking the infimum over all x ∈ H we have that inf φλ (z) ≤ inf φ(x) = inf φ(z),

z∈H

x∈H

z∈H

∀ λ > 0,

(4.41)

therefore, by (4.40) and (4.41) we conclude that infz∈H φλ (z) = infz∈H φ(z) for every λ > 0. Consider now xo ∈ argminx∈H φ(x). By (4.39), we have inf φ(z) ≤ φλ (xo ) ≤ φ(xo ) = inf φ(z),

z∈H

z∈H

so φλ (xo ) = infz∈H φ(z) = infz∈H φλ (z) for every λ > 0; hence, xo is a minimizer of φλ for every λ > 0. Convexity follows from standard arguments on the pointwise infimum of families of convex functions, or by an argument based on (ii) of the present proposition. Finally, for any x, z ∈ H and any λ1 ≤ λ2 it holds that φλ2 (x) = infz󸀠 ∈H φx,λ2 (z󸀠 ) ≤ φx,λ2 (z) ≤ φx,λ1 (z); hence, φλ2 (x) ≤ φx,λ1 (z), for every z ∈ H, and taking the infimum

176 | 4 Nonsmooth analysis: the subdifferential over all z ∈ H, we conclude that φλ2 (x) ≤ φλ1 (x) for all x ∈ H. Combining that with (4.39), we have that for every x ∈ H it holds that φλ2 (x) ≤ φλ1 (x) ≤ φ(x), and passing to the limit as λ → 0 we obtain the desired result. (ii) We will use the simplified notation zi = proxλφ (xi ), i = 1, 2. By the definition of φλ , we have that φλ (xi ) = 2λ1 ‖xi − zi ‖2 + φ(zi ), for any xi ∈ H, i = 1, 2, so that subtracting we obtain φλ (x2 ) − φλ (x1 ) =

1 (‖x − z2 ‖2 − ‖x1 − z1 ‖2 ) + φ(z2 ) − φ(z1 ). 2λ 2

(4.42)

We now recall that (see (4.30)) that for any x, z ∈ H, ⟨x − proxλφ (x), z − proxλφ (x)⟩ ≤ λφ(z) − λφ(proxλφ (x)),

(4.43)

so that setting x = x1 and z = z2 we obtain upon rearranging, λ(φ(z2 ) − φ(z1 )) ≥ ⟨x1 − z1 , z2 − z1 ⟩.

(4.44)

Combining (4.42) and (4.44) we obtain that φλ (x2 ) − φλ (x1 ) ≥

1 (‖x − z2 ‖2 − ‖x1 − z1 ‖2 + 2⟨x1 − z1 , z2 − z1 ⟩). 2λ 2

(4.45)

We would like to further enhance this inequality by noting the identity ‖h2 ‖2 − ‖h1 ‖2 = ‖h2 − h1 ‖2 + 2⟨h1 , h2 − h1 ⟩, which applied for hi = xi − zi , i = 1, 2 yields that 󵄩 󵄩2 ‖x2 − z2 ‖2 − ‖x1 − z1 ‖2 + 2⟨x1 − z1 , z2 − z1 ⟩ = 󵄩󵄩󵄩z2 − x2 − (z1 − x1 )󵄩󵄩󵄩 + 2⟨x2 − x1 , x1 − z1 ⟩.

(4.46)

Combining (4.45) and (4.46), we get 1 󵄩󵄩 󵄩2 (󵄩z − x2 − (z1 − x2 )󵄩󵄩󵄩 + 2⟨x2 − x1 , x1 − z1 ⟩) 2λ 󵄩 2 1 ≥ ⟨x2 − x1 , x1 − z1 ⟩. λ

φλ (x2 ) − φλ (x1 ) ≥

(4.47)

Hence, rearranging (4.47), 1 0 ≤ φλ (x2 ) − φλ (x1 ) − ⟨x2 − x1 , x1 − z1 ⟩. λ

(4.48)

We now return to (4.43) once more, setting x = x2 and z = z1 , and proceed as above to obtain (after multiplication by −1) that φλ (x2 ) − φλ (x1 ) ≤

1 ⟨x − x1 , x2 − z2 ⟩. λ 2

(4.49)

4.8 The proximity operator and numerical optimization algorithms | 177

Combining (4.48) with (4.49), we obtain 1 0 ≤ φλ (x2 ) − φλ (x1 ) − ⟨x2 − x1 , x1 − z1 ⟩ λ (4.49) 1 (⟨x − x1 , x2 − z2 ⟩ − ⟨x2 − x1 , x1 − z1 ⟩) ≤ λ 2 1 1 = (‖x2 − x1 ‖2 − ⟨x2 − x1 , z2 − z1 ⟩) ≤ ‖x2 − x1 ‖2 , λ λ

(4.50)

since by the monotonicity of the proximity operator ⟨x2 − x1 , z2 − z1 ⟩ ≥ 0. We now pick arbitrary x, h ∈ H and we set in (4.50), x1 = x, z1 = proxλφ (x), x2 = x + h and z2 = proxλφ (x + h). We divide by ‖h‖ and pass to the limit as ‖h‖ → 0 to obtain that 1 1 |φλ (x + h) − φλ (x) − ⟨h, (x − proxλφ (x))⟩ = 0, ‖h‖→0 ‖h‖ λ lim

which proves that φλ is Fréchet differentiable with Dφλ (x) = λ1 (x−proxλφ (x)). To obtain the Lipschitz continuity of Dφλ , we note that ⟨Dφλ (x1 ) − Dφλ (x2 ), x1 − x2 ⟩ 1 = ⟨(x1 − x2 ) − (proxλφ x1 − proxλφ x2 ), x1 − x2 ⟩ λ 1 1 = (‖x1 − x2 ‖2 − ⟨proxλφ x1 − proxλφ x2 , x1 − x2 ⟩) ≤ ‖x1 − x2 ‖2 . λ λ The above estimate, combined with the fact that proxλφ is nonexpansive, provides both the monotonicity of Dφλ (hence, also the convexity of φλ ) as well as its Lipschitz continuity.

4.8 The proximity operator and numerical optimization algorithms The proximity operators proxλφ are widely used in a class of numerical minimization algorithms in Hilbert spaces called proximal point algorithms. The fundamental idea behind such methods is to replace the original nonsmooth optimization problem infx∈X φ(x) by a smooth approximation, in particular infx∈X φλ (x), where φλ is the Moreau–Yosida approximation of φ for some appropriate value of the regularization parameter λ > 0. As shown in Proposition 4.7.6, this approximation is Fréchet differentiable with a Lipschitz continuous Fréchet derivative; hence, a minimizer for φλ satisfies the first-order equation Dφλ (xo ) = 0 which, again by Proposition 4.7.6, reduces to 0 = λ1 (xo − proxλφ (xo )). Since λ > 0, a minimizer of φλ turns out to be a fixed point of the proximal operator xo = proxλφ (xo ); hence, by Proposition 4.7.2(iii) a minimizer of λφ, therefore, a minimizer of φ. However, the mere fact that instead of dealing with the original nonsmooth problem we deal with

178 | 4 Nonsmooth analysis: the subdifferential a smooth approximation has huge advantages, as the smoothness allows for better convergence results. Proximal methods are based upon the above observation. In this section, we present the standard proximal method and its convergence, as well as a number of variants, designed for solving minimization problems of the form infx∈X (φ1 (x)+φ2 (x)), consisting of a smooth and a nonsmooth part, φ1 and φ2 , respectively. While these methods can be extended to Banach spaces, we restrict our attention here in a Hilbert space setting for simplicity.

4.8.1 The standard proximal method As already mentioned, the main idea behind proximal point algorithms is based on the important observation that Fix(proxφ ) = arg minx∈H φ(x) (see Proposition 4.7.2(iii)) and consists in trying to approximate a minimizer of φ (and the minimum value) by a sequence {xn : n ∈ ℕ} defined by the iterative procedure xn+1 = proxλφ xn ,

n ∈ ℕ,

(4.51)

for some appropriate choice of the parameter λ. The sequence defined in (4.51) is called a proximal sequence, and under certain circumstances, can converge to a fixed point, which if φ has a minimum belongs to the set of minimizers of φ. This observation leads to the development of a class of numerical algorithms that are very popular in optimization, called proximal methods. One interesting question that arises is concerning the choice of the parameter λ > 0. One immediately sees that λ cannot be too small, since proxλφ (x) → x as λ → 0 (see Example 4.7.4), which means that for small λ, the proximal sequence will essentially not go anywhere. On the other hand, large λ provides higher speed, but one must recall that the first-order conditions are of a local nature and λ too large may create problems with that or with the calculation of the proximal operator. It is also conceivable that one may not require a uniform λ throughout the whole iteration procedure, and choose a varying λ, depending on the neighborhood of the domain of the function that the iteration has led us (similar to the smooth gradient method) leading thus to an adjustment of the “speed” of the method as we proceed. One may also view the proximal sequence (4.51) as a natural generalization of the smooth gradient method in case no smoothness is available. If φ was smooth, one could use the implicit gradient method xn+1 = xn − λDφ(xn+1 ) for some appropriate18 18 Called implicit since we calculate the gradient at xn+1 rather than xn (explicit gradient scheme), thus requiring the solution of a nonlinear equation in order to retrieve xn+1 from xn . The reason we decide to undertake this cost is because the implicit scheme enjoys better convergence properties than the explicit one.

4.8 The proximity operator and numerical optimization algorithms | 179

λ > 0, which may be rearranged as 0 = λ1 (xn+1 − xn ) + Dφ(xn+1 ), or equivalently choose xn+1 as the minimizer of the function x 󳨃→ φ(x) + 2λ1 ‖x − xn ‖2 . In the case of nonsmooth φ, this leads directly to the proximal sequence (4.51). As an alternative motivation for the proximal sequence, consider the explicit gradient method for the minimization of the smooth problem infx∈H φλ (x), for some λ > 0. Choosing step size λ this would yield the iterative scheme xn+1 = xn − λDφλ (xn ) = proxλφ (xn ) which is exactly (4.51). The fundamental version of such an algorithm is the following. Algorithm 4.8.1 (Proximal point algorithm). To find an element xo ∈ arg minx∈H φ(x), we make the following steps: 1. Choose a sequence {λn : n ∈ ℕ}, λn > 0, ∑∞ n=1 λn = ∞ and an accuracy ϵ > 0. 2. For an appropriate initial point x1 , iterate (until a convergence criterion is met) xn+1 = proxλn φ xn := arg min(φ(z) + z∈X

1 ‖x − z‖2 ). 2λn n

In the case where φ is smooth, the proximal point algorithm reduces to the implicit gradient descent scheme with variable step size. Constrained optimization may be treated by appropriately modifying φ. The convergence of the fixed-point scheme is based on the fact that the proximity operator is a nonexpansive operator (see Proposition 4.7.2(ii)). The following theorem provides a weak convergence result for the proximal algorithm (see [20] or [88]). Even though we could provide a proof using the fact that proxλφ is firmly nonexpansive involving some variant of the Krasnoselskii–Mann iteration scheme, we prefer to provide a direct proof for the basic proximal point algorithm, which we feel is more intuitive. Proposition 4.8.2. Let φ : H → ℝ be a proper lower semicontinuous convex function and consider the proximal point algorithm xn+1 = proxλn φ xn ,

n ∈ ℕ.

(4.52)

with ∑∞ n=1 λn = ∞. Then {xn : n ∈ ℕ} is a minimizing sequence for φ, i. e., φ(xn ) → infx∈H φ(x), which converges weakly to a point in the set of minimizers of φ. Proof. The strategy of the proof is to show that the proximal sequence enjoys the socalled Fejér property with respect to the set of minimizers of φ. Given a set A ⊂ H and a sequence {zn : n ∈ ℕ}, we say that it is Fejér with respect to A if ‖zn+1 − z‖ ≤ ‖zn − z‖,

∀ n ∈ ℕ, z ∈ A.

(4.53)

As we will see, Fejér sequences with respect to weakly closed sets A converge weakly to a point zo ∈ A. Since the set of minimizers of φ is a closed convex set, if we manage to show that the proximal sequence is Fejér with respect to arg min φ we are done. We

180 | 4 Nonsmooth analysis: the subdifferential therefore try to show that if xo ∈ arg minx∈H φ(x), then the proximal sequence (4.52) satisfies ‖xn+1 − xo ‖ ≤ ‖xn − xo ‖,

∀ n ∈ ℕ, xo ∈ arg min φ(x). x∈H

(4.54)

We proceed in 3 steps: 1. Using Proposition 4.7.2(iii), we see that xn+1 = proxλn φ (xn ) implies xn − xn+1 ∈ λn 𝜕φ(xn+1 ),

(4.55)

n ∈ ℕ.

By the definition of the subdifferential (Definition 4.1.1), this means that λn φ(z) − λn φ(xn+1 ) ≥ ⟨xn − xn+1 , z − xn+1 ⟩,

∀ z ∈ H.

(4.56)

Setting z = xn in (4.56), we have that 1 ⟨x − xn+1 , xn − xn+1 ⟩ λn n 1 = ‖xn − xn+1 ‖2 ≥ 0, λn

φ(xn ) − φ(xn+1 ) ≥

(4.57)

hence, the sequence {φ(xn+1 ) : n ∈ ℕ} is decreasing. Consider now any xo ∈ A := arg minx∈H φ(x), and apply (4.56) setting z = xo , to obtain 1 ⟨x − xn+1 , xo − xn+1 ⟩ ≤ φ(xo ) − φ(xn+1 ). λn n

(4.58)

We now rearrange ‖xn+1 − xo ‖2 as follows: ‖xn+1 − xo ‖2 = ‖xn+1 − xn + xn − xo ‖2

= ‖xn − xo ‖2 + 2⟨xn+1 − xn , xn − xo ⟩ + ‖xn+1 − xn ‖2

= ‖xn − xo ‖2 + 2⟨xn+1 − xn , xn − xn+1 + xn+1 − xo ⟩ + ‖xn+1 − xn ‖2 = ‖xn − xo ‖2 − ‖xn+1 − xn ‖2 + 2⟨xn+1 − xn , xn+1 − xo ⟩

(4.59)

≤ ‖xn − xo ‖2 + 2⟨xn − xn+1 , xo − xn+1 ⟩ ≤ ‖xn − xo ‖2 + 2λn (φ(xo ) − φ(xn+1 )),

where in the last estimate we used (4.58). Since xo ∈ arg minx∈H φ(x), clearly φ(xo ) − φ(xn+1 ) is negative, while setting γ = infx∈H φ(x) we have that φ(xo ) = γ. In view of this notation, the estimate (4.59) becomes ‖xn+1 − xo ‖2 ≤ ‖xn − xo ‖2 − 2λn (φ(xn+1 ) − γ),

(4.60)

where φ(xn+1 ) − γ > 0. We therefore conclude that the sequence {xn : n ∈ ℕ} satisfies ‖xn+1 − xo ‖ ≤ ‖xn − xo ‖,

∀ n ∈ ℕ;

hence, it enjoys the Fejér property (4.54) with respect to arg min φ.

(4.61)

4.8 The proximity operator and numerical optimization algorithms | 181

2. Rearranging (4.60), we get that 1 λn (φ(xn+1 ) − γ) ≤ (‖xn − xo ‖2 − ‖xn+1 − xo ‖2 ), 2

∀ n ∈ ℕ,

and adding over all n ∈ ℕ we get that m 1 ∑ λn (φ(xn+1 ) − γ) ≤ (‖x1 − xo ‖2 − ‖xm+1 − xo ‖2 ) 2 n=1

1 ≤ ‖x1 − xo ‖2 , 2

(4.62)

∀ m ∈ ℕ.

As mentioned above (see (4.57)), the sequence {φ(xn ) − γ : n ∈ ℕ} is positive and nonincreasing; hence, 0 ≤ φ(xm+1 ) − γ ≤ φ(xn+1 ) − γ, for every n ≤ m. Using this estimate in (4.62), we get that m 1 (φ(xm+1 ) − γ) ∑ λn ≤ ‖x1 − xo ‖2 , 2 n=1

∀ m ∈ ℕ,

(4.63)

and taking the limit as m → ∞, combined with the assumption that ∑n λn = ∞ leads to the result that limm φ(xm ) = γ, so that {xn : n ∈ ℕ} is a minimizing sequence. By a standard lower semicontinuity argument, any weak accumulation point of the proximal sequence (4.52) is in arg minx∈H φ(x). 3. It remains to show that the proximal sequence has weak accumulation points. To this end, recall the Fejér monotonicity property of the proximal sequence with respect to A = arg minx∈H φ(x), shown in (4.61). A Fejér sequence {xn : n ∈ ℕ} is bounded, since by the monotonicity of {‖xn −xo ‖ : n ∈ ℕ}, we have setting R = ‖x1 −xo ‖, that xn ∈ BH (xo , R) for every n ∈ ℕ; hence, by a weak compactness argument there exists a subsequence {xnk : k ∈ ℕ} and a x󸀠o ∈ H, such that xnk ⇀ x󸀠o . From step 2, we already know that x󸀠o ∈ arg minx∈H φ(x). We will show that the whole sequence (and not just the subsequence) weakly converges to this x󸀠o . Indeed, consider any two weakly converging subsequences {xnk : k ∈ ℕ} and {xℓk : k ∈ ℕ} of the proximal sequence (4.52), such that xnk ⇀ xA ∈ arg minx∈H φ(x), xℓk ⇀ xB ∈ arg minx∈H φ(x) as k → ∞. Clearly, by (4.61) (where xo was an arbitrary point of arg minx∈X φ(x)) we have that both {‖xn − xA ‖ : n ∈ ℕ} and {‖xn − xB ‖ : n ∈ ℕ} are decreasing and bounded below hence, convergent. Let us assume that ‖xn − xA ‖ → KA , ‖xn −xB ‖ → KB . We first express ‖xn −xA ‖2 = ‖xn −xB ‖2 +‖xB −xA ‖2 +2⟨xn −xB , xB −xA ⟩, choose the subsequence n = ℓk and pass to the limit to obtain KA = KB + ‖xB − xA ‖2 . We then repeat the above interchanging the role of A and B, choose the subsequence n = nk and pass to the limit to obtain KB = KA +‖xA −xB ‖2 . Combining the above, we get that xA = xB ; hence, by the Urysohn property (see Remark 1.1.51), the whole sequence weakly converges to the same limit which is a minimizer for φ. Example 4.8.3 (When would the proximal algorithm converge strongly to a minimizer?). For the strong convergence of the proximal algorithm to a minimizer, we

182 | 4 Nonsmooth analysis: the subdifferential need to add the extra condition that φ is uniformly convex with modulus of convexity ψ (see Definition 2.3.3). In this case, we have a unique minimizer xo and furthermore the proximal sequence xn → xo (strongly). The uniqueness of the minimizer follows by strict convexity. Consider now the proximal sequence for which we already know by Proposition 4.8.2 that xn ⇀ xo . By the uniform convexity of φ, φ(tx1 + (1 − t)x2 ) ≤ tφ(x1 ) + (1 − t)φ(x2 ) − t(1 − t)ψ(‖x1 − x2 ‖X ), ∀ x1 , x2 ∈ X,

t ∈ (0, 1).

Setting x1 = xn , x2 = xo and t =

1 2

we obtain that

1 1 1 1 1 φ( xn + xo ) + ψ(‖xn − xo ‖) ≤ φ(xn ) + φ(xo ). 2 2 4 2 2

(4.64)

Note that 21 xn + 21 xo ⇀ xo so that by lower semicontinuity φ(xo ) ≤ lim infn φ( 21 xn + 1 x ). We also have that φ(xn ) → φ(xo ) = infx∈H φ(x). We rearrange (4.64) to keep the 2 o modulus on the left-hand side and take limit superior on both sides of the inequality to obtain 1 1 1 1 1 lim sup ψ(‖xn − xo ‖) ≤ lim sup( φ(xn ) + φ(xo ) − φ( xn + xo )) 4 2 2 2 2 n n 1 1 = φ(xo ) − lim inf φ( xn + xo ) ≤ 0, n 2 2 where from the properties of ψ (increasing and vanishing only at 0) we conclude that ψ(‖xn − xo ‖) → 0, and hence, xn → xo (strongly). Example 4.8.4 (Rate of convergence for strongly convex functions). If the function φ is strongly convex with modulus of convexity c (see Definition 2.3.3), then we have strong convergence of the proximal scheme (4.52) and may also obtain the rate of con−1 vergence as, e. g., ‖xo −xN ‖ ≤ ∏N−1 k=1 (1+2cλk ) ‖xo −x1 ‖, where c is the modulus of strong convexity and λk is the variable step size. Compare with the relevant estimates for the gradient descent method. To see this, the reader may wish to recall the properties of the subdifferential for strongly convex functions (4.7) in Example 4.2.5, and in particular that ⟨x⋆1 − x⋆2 , x1 − x2 ⟩ ≥ c‖x1 − x2 ‖2 ,

∀ x⋆1 ∈ 𝜕φ(x1 ), x⋆2 ∈ 𝜕φ(x2 ).

(4.65)

Since xo is a minimizer so that 0 ∈ 𝜕φ(xo ) and xn+1 = proxλn φ (xn ) which is equivalent to − λ1 (xn+1 − xn ) ∈ 𝜕φ(xn+1 ) applying (4.65) for the choice n

x⋆1 = −

1 (x − xn ), λn n+1

x1 = xn+1 ,

and

x⋆2 = 0,

x2 = xo ,

4.8 The proximity operator and numerical optimization algorithms | 183

we obtain ⟨xn+1 − xn , xo − xn+1 ⟩ ≥ cλn ‖xo − xn+1 ‖2 .

(4.66)

Being in Hilbert space, we may express the left-hand side in terms of the sum and difference of the squares of the norms as ‖xn − xo ‖2 = ‖xn − xn+1 + xn+1 − xo ‖2

= ‖xn+1 − xn ‖2 + ‖xn+1 − xo ‖2 + 2⟨xn+1 − xn , xo − xn+1 ⟩,

which upon solving for 2⟨xn+1 − xn , xo − xn+1 ⟩ and combined with (4.66) gives ‖xn − xo ‖2 − ‖xn+1 − xn ‖2 − ‖xn+1 − xo ‖2 ≥ 2cλn ‖xo − xn+1 ‖2 .

(4.67)

Rerranging (4.67), we obtain the inequality (1 + 2cλn )‖xo − xn+1 ‖2 ≤ ‖xn − xo ‖2 − ‖xn+1 − xn ‖2 ≤ ‖xn − xo ‖2 .

(4.68)

−1/2 Iterating, we obtain ‖xo −xN ‖ ≤ ∏N−1 ‖xo −x1 ‖, which is the case of constant k=1 (1+2cλk ) stepsize λk = λ reduces to ‖xo − xN ‖ ≤ (1 + 2cλ)−(N−1)/2 ‖xo − x1 ‖, which are the stated convergence estimates. In fact, we may do a little better using again the properties of the subdifferential for strongly convex functions, and in particular (4.7)(iii)

󵄩󵄩 ⋆ ⋆󵄩 󵄩󵄩x1 − x2 󵄩󵄩󵄩 ≥ c‖x1 − x2 ‖,

∀ x⋆1 ∈ 𝜕φ(x1 ), x⋆2 ∈ 𝜕φ(x2 ),

(4.69)

to estimate the term ‖xn+1 − xn ‖ in (4.68) rather than discard it. In fact, since 0 ∈ 𝜕φ(xo ) and − λ1 (xn+1 − xn ) ∈ 𝜕φ(xn+1 ), relation (4.69) yields ‖xn+1 − xo ‖ ≤ cλ1 ‖xn+1 − xn ‖ so that n

n

−‖xn+1 −xn ‖2 ≤ −c2 λn2 ‖xn+1 −xo ‖2 . Combining this with (4.68) leads to the better estimate (1+cλn )2 ‖xo −xn+1 ‖2 ≤ ‖xn −xo ‖2 , so that ‖xo −xn+1 ‖ ≤ (1+cλn )−1 ‖xn −xo ‖, and by iteration −1 we obtain the improved convergence estimate ‖xo −xN ‖ ≤ ∏N−1 k=1 (1+cλk ) ‖xo −x1 ‖, which −(N−1) if λk = λ reduces to ‖xo − xN ‖ ≤ (1 + cλ) ‖xo − x1 ‖. ◁ Example 4.8.5. Assume that we wish to minimize the quadratic function φ defined by φ(x) = 21 ⟨x, Ax⟩ − ⟨b, x⟩, where A is symmetric. Of course, this is a smooth functional, the solution of which is given by the solution of the operator equation Ax = b. Let us now calculate the proximal operator for φ. The solution to the minimization problem minz∈H 2λ1 ‖x − z‖2 + 21 ⟨z, Az⟩ − ⟨b, z⟩ is expressed as 1 1 proxλφ x = (A + I) (b + x), λ λ −1

so that the proximal algorithm provides the iterative algorithm 1 1 xk+1 = (A + I) (b + xk ), λ λ −1

184 | 4 Nonsmooth analysis: the subdifferential which can be expressed in the equivalent form 1 xk+1 = xk + (A + I) (b − Axk ). λ −1

This will converge to a solution of Ax = b. In the special case where H = ℝn in which case A corresponds to a matrix A ∈ ℝn×n , the above procedure provides an iterative algorithm for the solution of the system of linear equations Ax = b, in which at every step we have to invert the perturbed matrix A + λ1 I, which is often preferable from the numerical point of view, especially if we have to deal with ill posed matrices. Also, it may be that A is not invertible, a situation often arising in the treatment of ill posed problems and their regularization. ◁ Other versions of the proximal point algorithm are possible, such as for instance the inertial version xn+1 = (1 + θ)proxλφ (xn ) − θxn ,

θ ∈ (−1, 1),

(4.70)

which by similar techniques can be shown to converge. For useful information on extensions of this algorithm as well as information on its concrete applications see, e. g., [39]. 4.8.2 The forward-backward and the Douglas–Rachford scheme The proximal point algorithm, though enjoying the extremely important and desirable property of unconditional convergence, suffers a serious drawback. Often the calculation of the proximal operator is a problem as difficult as the original problem we started with. So even though it will work nicely for functions whose proximal operators are easily or even analytically obtained (as for instance for the ℓ1 norm), in many practical situations it is not easily applicable. One way around such problems is to split the function we wish to minimize to a sum of more than one contributions, with the criterion that at least some of the constituents have easy to calculate proximal maps. This is the basic idea behind splitting methods, which is the subject of this section. 4.8.2.1 Forward-backward algorithms An interesting extension, is related to minimization problems of the form minx∈H (φ1 (x) + φ2 (x)) in which both φ1 and φ2 are convex lower semicontinuous functions, but one of them, say φ1 , is Fréchet differentiable with Lipschitz continuous Fréchet derivative. Starting from the first order condition 0 ∈ Dφ1 (x) + 𝜕φ2 (x) for a minimizer x, we split 0 as 0 = (x − z) + (z − x) for some auxiliary z ∈ H, chosen such that x − z = αDφ1 (x),

and,

z − x ∈ α𝜕φ2 (x),

4.8 The proximity operator and numerical optimization algorithms | 185

where α > 0 can be considered as a regularization parameter. Bringing −x to the other side on the second relation, the above splitting becomes x = z + αDφ1 (x),

x = (I + α𝜕φ2 )−1 (z) = proxαφ2 (z).

This shows that the minimizer x must satisfy the above compatibility conditions, and is expressed in terms of the auxiliary variable z as z = x − αDφ1 (x) (this equation may be feasible to solve to obtain x in terms of z since Dφ1 is Lipschitz), where x must solve (by eliminating z) the operator equation x = f (x) := proxαφ2 (x − αDφ1 (x)), i. e., it is a fixed point of the operator f : H → H defined above. We may try to approximate the solution to the operator equation x = f (x) in terms of a (relaxed) iterative scheme of the form xn+1 = xn + λn (f (xn ) − xn ), where λn is a relaxation parameter (λn ∈ (1, 2) corresponding to over-relaxation and λn ∈ (0, 1) corresponding to under-relaxation). This may be also expressed as the two step procedure: zn = xn − αDφ1 (xn ),

xn+1 = xn + λn (proxαφ2 (zn ) − xn ) = (1 − λn )xn + λn proxαφ2 (zn ). Alternatively, the first step can be understood as an explicit gradient scheme for the minimization of the smooth part (which enjoys nice convergence properties), while the second step as the inertial version of the proximal algorithm (4.70) for the nonsmooth part. Summarizing, in the proposed scheme we essentially split the function φ1 + φ2 into its smooth and its nonsmooth part, treat them separately and then combine them in terms of a fixed-point iteration. This minor modification to the standard proximal scheme has certain advantages, one of which may be that the proximal operator for φ2 may be easier to compute that the proximal operator for the sum φ1 + φ2 . It also leads to small changes in the convergence properties of the scheme, but at the same time simplifies other aspects by turning the scheme into a semi-implicit (or forwardbackward) one.19 The above considerations lead to the following algorithm. 19 For instance, the assumption that 𝜕φ = 𝜕φ1 + 𝜕φ2 = Dφ1 + 𝜕φ2 is no longer needed, whereas is some cases it is simpler to invert I + λn 𝜕φ2 than I + λn Dφ1 + λn 𝜕φ2 . As for the terminology, the step zn := xn − λn Dφ1 (xn ) is called the forward step, while the step proxλn φ zn is called the backward one.

186 | 4 Nonsmooth analysis: the subdifferential Algorithm 4.8.6 (Forward-backward algorithm). Let φi , i = 1, 2, be convex and lower semicontinuous with φ1 Fréchet differentiable such that Dφ1 is Lipschitz continuous with Lipschitz constant L. To find an xo ∈ arg minx∈H (φ1 (x) + φ2 (x)): 1. Choose α > 0 sufficiently small and a sequence {λn : n ∈ ℕ} ⊂ [0, β], such that ∑∞ n=1 λn (β − λn ) = +∞ for a suitably chosen β > 1/2, and an accuracy ϵ > 0. 2. For an appropriate starting point (x1 , y1 ) ∈ H, iterate zn = xn − αDφ1 (xn ),

xn+1 = xn + λn (proxαφ2 (zn ) − xn ) = (1 − λn )xn + λn proxαφ2 (zn ),

(4.71)

until a convergence criterion is met. The following proposition provides convergence results for the forward-backward algorithm (see [20] or [46]). For details on extensions of the forward-backward algorithm as well as its practical application, the reader may consult [39]. Proposition 4.8.7 (Convergence of forward-backward algorithm). Assume that 0 < α < L2 , and λn ∈ (0, 1), ∑∞ n=1 λn (1 − λn ) = +∞. Then the forward-backward algorithm (4.71) converges weakly to a minimizer of φ1 + φ2 . Proof. Observe that the forward-backward scheme can be expressed in one equation as xn+1 = (1 − λn )xn + λn f (xn ),

(4.72)

where f = f1 ∘ f2 =: f1 f2 , with f2 : x 󳨃→ x − αDφ1 (x) and f1 : x 󳨃→ proxαφ2 (x). f1 is nonexpansive and f2 is also nonexpansive as long as 0 < α < L2 (see Example 3.4.3), so that f is nonexpansive. Then (4.72) reduces to the Krasnoselskii–Mann iteration scheme and the convergence follows from Theorem 3.4.9. Remark 4.8.8. Slightly better parameter values may be obtained by noting that f is a ν-averaged operator, as a composition of two ν-averaged operators (see Lemma 3.7.2) and working as in Example 3.4.10. Indeed, working exactly as in Example 3.4.7 we may prove that f1 is 21 -averaged while from Example 3.4.8 we know that f2 is αL -averaged as 2 ν1 +ν2 −2ν1 ν2 2 αL = 4−αL long as 2 < 1. Using Lemma 3.7.2, we have that f is ν-averaged for ν = 1−ν ν 1 2 so using the iterative scheme proposed in Example 3.4.10 we may choose the sequence 1 αL {λn : n ∈ ℕ} so that ∑∞ n=1 λn (β − λn ) = ∞ for β = ν = 2 − 2 . Example 4.8.9 (The projected gradient method). Consider the constrained minimization problem infx∈C φ(x) where φ : H → ℝ ∪ {+∞} is a convex lower semicontinuous function, with Lipschitz continuous Fréchet derivative and C ⊂ H is a closed convex set. We may define φ1 = φ, φ2 = IC and express the constrained minimization problem as the unconstrained problem infx∈H (φ1 (x) + φ2 (x)). Applying the forward-backward

4.8 The proximity operator and numerical optimization algorithms | 187

scheme to this problem, and recalling that for this choice proxφ2 = PC , the projection operation on C, we retrieve the projected gradient scheme xn+1 = PC (xn − τDφ1 (xn )),

(4.73)

by setting the parameters in (4.71) to α = τ, λn = 1. An application of Proposition 4.8.7 shows the weak convergence of the scheme (4.73) for τ < L2 , where L is the Lipschitz constant for Dφ. Example 4.8.10 (Lasso problems). An important use of the proximal point algorithm is in the so-called Lasso optimization problems. These are minimization problems in the Hilbert space H = ℝn , endowed with the Euclidean | ⋅ | := ‖ ⋅ ‖2 norm, of the form 1 min( ‖Ax − b‖22 + λ‖x‖1 ), x∈H 2 where ‖ ⋅ ‖1 , denotes the ℓ1 norm, A ∈ ℝm×n , b ∈ ℝm and λ > 0. Problems of this type find many applications in high dimensional statistics, machine learning or image processing, and their interpretation is as finding sparse solutions to least squares problems. The proximal gradient method can be directly applied to this problem, by setting φ1 = 21 ‖Ax − b‖22 and φ2 = λ‖x‖1 and noting that Dφ1 (x) = AT (Ax − b) and proxλφ2 (x) = Sλ (x), the soft thresholding operator we have defined in Example 4.7.5. The Lasso problem can be generalized in sequence spaces, in terms of the ℓ2 and the ℓ1 norm in the place of ‖ ⋅ ‖2 and ‖ ⋅ ‖1 . ◁ 4.8.2.2 Douglas–Rachford algorithm A final modification of the proximal gradient algorithm, again in the spirit of a forward-backward scheme, is the Douglas–Ratchford algorithm, which is often used to minimize sums of convex lower semicontinuous functions if neither of them is differentiable. To motivate the algorithm, consider the Fermat rule for the minimization of φ = φ1 +φ2 , which under sufficient assumptions for the Moreau–Rockafellar Theorem 4.3.2 to hold, leads to 0 ∈ 𝜕φ1 (x) + 𝜕φ2 (x). We multiply this by λ > 0, and split 0 in terms of an auxiliary z ∈ H, as 0 = −(z − x) + (z − x) = (2x − z − x) + (z − x). We now choose 2x − z − x ∈ λ𝜕φ1 (x),

and,

z − x ∈ λ𝜕φ2 (x),

and immediately (bringing the −x term to the other side in both, starting from the second) recognize them as x = (I + λφ2 )−1 (z) = proxλφ2 (z),

x = (I + λφ1 )−1 (2x − z) = proxλφ1 (2x − z).

188 | 4 Nonsmooth analysis: the subdifferential This shows that a minimizer x must satisfy the above compatibility conditions, and is expressed in terms of the auxiliary variable z as x = proxλφ2 (z) where z must solve (by eliminating x) the operator equation 0 = proxλφ1 (2 proxλφ2 (z) − z) − proxλφ2 (z). Adding z on each side of the above, we express z as a fixed point of the map f : H → H, z = f (z) := z + proxλφ1 (2 proxλφ2 (z) − z) − proxλφ2 (z). This fixed-point problem can be approximated by an iterative scheme, for instance by a fixed-point iteration with relaxation of the form zn+1 = zn + λn (f (zn ) − zn ), where λn is a relaxation parameter, with λn ∈ (1, 2) corresponding to overrelaxation and λn ∈ (0, 1) corresponding to underrelaxation. The above scheme may be expressed as the two step iterative procedure: xn = proxλφ2 (zn ),

zn+1 = zn + λn (proxλφ1 (2xn − zn ) − xn ).

(4.74)

The above discussion leads to the Douglas–Rachford minimization algorithm. Algorithm 4.8.11 (Douglas–Rachford). To find a minimizer of φ1 + φ2 , we make the following steps: 1. Choose λ > 0 and a sequence {λn : n ∈ ℕ}, such that λn ∈ (0, 2] for every n ∈ ℕ, with ∑n∈ℕ λn (2 − λn ) = ∞ and an accuracy ϵ > 0. 2. For an appropriate (x1 , z1 ), iterate (until a convergence criterion is met), xn = proxλφ2 (zn ),

zn+1 = zn + λn (proxλφ1 (2xn − zn ) − xn ).

(4.75)

The convergence properties of this algorithm can be proved using the reformulation of the above scheme in terms of z (after the elimination of x), using the nonexpansive properties of the proximity operator. Proposition 4.8.12 (Convergence of the Douglas–Rachford algorithm). Under the assumption that λn ∈ (0, 2] for every n ∈ ℕ, with ∑n∈ℕ λn (2 − λn ) = ∞, the Douglas– Rachford iterative scheme converges weakly to a minimizer xo ∈ arg minx∈H (φ1 (x) + φ2 (x)).

4.8 The proximity operator and numerical optimization algorithms | 189

Proof. Define the mappings fi = 2 proxλφi − I, i = 1, 2, and note that 2xn − zn = f2 (zn ), so that the second step in (4.75) becomes 1 zn+1 = zn + λn (proxλφ1 (f2 (zn )) − (f2 (zn ) + zn )) 2 λn = zn + (f1 (f2 (zn )) − zn ). 2

(4.76)

Note that this reduces to the “averaged” like form zn+1 = (1 − ρn )zn + ρn f (zn ), where ρn = λ22 and f = f1 ∘ f2 =: f1 f2 . The operators fi , i = 1, 2, are nonexpansive operators hence, so is f1 f2 . The weak convergence of the iterative scheme to a fixed point of the operator f1 f2 follows from the general theory of the Krasnolneskii–Mann iterative scheme (see Theorem 3.4.9). In λ λ fact, choosing {λn : n ∈ ℕ} such that ∑n∈ℕ 2n (1 − 2n ) = ∞ leads to the equivalent condition ∑n∈ℕ λn (2 − λn ) = ∞. It is easily seen (retracing the steps in the beginning of this subsection in reverse order) that a fixed-point z of f1 f2 can be used in the construction of a minimizer for φ1 + φ2 , in terms of x = proxλφ2 (z). Example 4.8.13 (Convex feasibility problems). A useful application of the Douglas– Rachford scheme is in convex feasibility problems, i. e., the problem, given two closed and convex sets C1 , C2 ⊂ H, of finding a point x ∈ C1 ∩ C2 . Defining φi = ICi , i = 1, 2, this problem reduces to the problem of infx∈H (φ1 (x) + φ2 (x)) which has a solution as long as C1 ∩ C2 ≠ 0. One may use the Douglas–Rachford algorithm to find such a point x ∈ C1 ∩ C2 . Recalling that proxλφi = PCi , i = 1, 2, the operators fi = 2PCi − I, i = 1, 2, (called reflectors in this case) and the Douglas–Rachford scheme becomes (choosing λn = 1 for simplicity) 1 zn+1 = (I + f1 f2 )(zn ), 2 xn+1 = PC2 zn+1 , and the second step can only be performed once, when a stopping criterion for the z iteration is met. By the general theory of the Douglas–Rachford scheme, if C1 ∩ C2 ≠ 0, the iteration for {zn : n ∈ ℕ} weakly converges to some z such that PC2 (z) ∈ C1 ∩ C2 , while if C1 ∩ C2 = 0, then ‖zn ‖ → ∞. Convex feasibility problems involving more than two sets, can be reduced to the above problem with 2 sets using the so-called Pierra product space formulation consisting of the sets C1󸀠 = C1 × ⋅ ⋅ ⋅ × Ck and C2󸀠 = {(x, . . . x) ∈ H k : x ∈ H} (called the diagonal). Then x ∈ C1 × ⋅ ⋅ ⋅ × Ck ⊂ H k if and only if x ∈ C1󸀠 ∩ C2󸀠 which is a problem of the above form. Convex feasibility problems have multiple applications ranging from medical imaging to combinatorics. One common application is in the socalled, matrix completion problem, which consists, given a partial matrix, of finding

190 | 4 Nonsmooth analysis: the subdifferential a completion having desirable properties, e. g., positive semidefinite. In such cases, the space of n × m matrices is turned into a Hilbert space using the inner product ⟨A1 , A2 ⟩ = Tr(AT1 A2 ), and the convex sets Ci , i = 1, . . . , k correspond to the desired constraints, e. g., C1 = {x = A ∈ ℝn×n : z T Az ≥ ϵ |z|2 , ∀ z ∈ ℝn } for positive definite matrices, etc. For more details on the matrix completion problem, the reader may consult [12]. ◁

5 Minimax theorems and duality In this chapter, we develop the theory of convex duality, which finds many interesting applications in the theory and practice of convex optimization. We begin our development of this theory by introducing a rather general version of the minimax theorem which allows us to answer the question of whether a functional admits a saddle point. We then move to a detailed study of the Fenchel–Legendre conjugate and the biconjugate for convex functions and their properties. With these tools at hand, we proceed to the main aim of this chapter, the study of duality methods in optimization, which allows us to redress constrained minimization problems in saddle point form and connect with each such problem a related maximization problem called its dual (that in many cases is easier to treat than the original problem, called the primal). We first present the theory of Fenchel duality and then a more general framework developed by Ekeland, Temam and others which allows for the treatment of a wide class of problems arising in various applications including data analysis, economics and finance, signal and image processing, etc. These duality techniques find important applications in numerical analysis, and the chapter closes with a treatment of numerical methods for optimization problems based on such concepts. These important issues have been treated in [13, 20, 25, 62, 88], to which we refer for further details.

5.1 A minimax theorem Let F : X × Y → ℝ be a function defined on the Cartesian product of the Banach spaces X, Y. It is elementary to note that it is always true1 that sup inf F(x, y) ≤ inf sup F(x, y). y∈Y x∈X

x∈X y∈Y

(5.1)

The interval (supy∈Y infx∈X F(x, y), infx∈X supy∈Y F(x, y)) is called the duality gap for F. An interesting question is: when does the opposite inequality hold? An affirmative answer to this question would lead to equality of the two sides, i. e., sup inf F(x, y) = inf sup F(x, y). y∈Y x∈X

x∈X y∈Y

(5.2)

We then say that the function F satisfies the minimax equality with the common value called a saddle value for F. This would essentially mean that the order by which we minimize and maximize does not alter the result, a fact that can be very useful as we shall see for a number of optimization problems. The positive or negative answer to this question is directly related to the notion of a saddle point. 1 First, fix x ∈ X and observe that infx∈X F(x, y) ≤ F(x, y), for any x, y, then take the supremum over y ∈ Y to obtain supy∈Y infx∈X F(x, y) ≤ supy∈Y F(x, y) for any x and finally take the infimum over x ∈ X to obtain the stated inequality. https://doi.org/10.1515/9783110647389-005

192 | 5 Minimax theorems and duality Definition 5.1.1. The point (xo , yo ) ∈ X × Y is called a saddle point of F if F(xo , y) ≤ F(xo , yo ) ≤ F(x, yo ),

∀ x ∈ X, y ∈ Y.

Upon defining the sets, m(y) := arg min F(x, y), x∈X

M(x) := arg max F(x, y), y∈Y

i. e., the set of minimizers of F(⋅, y) and the set of maximizers of F(x, ⋅), respectively, we may provide an alternative characterization of a saddle point as follows. If (xo , yo ) is a saddle point of F then xo minimizes F(⋅, yo ) whereas yo maximizes F(xo , ⋅), i. e., xo ∈ m(yo ) = {x ∈ X : F(x, yo ) = inf F(x, yo )}, x∈X

yo ∈ M(xo ) = {y ∈ Y : F(xo , y) = sup F(xo , y)}.

(5.3)

y∈Y

As such, a saddle point has a convenient interpretation in terms of the fixed point of the set valued map Φ, defined by (x, y) 󳨃→ m(y) × M(x). In view of (5.3), a saddle point (xo , yo ) of F has the property that (xo , yo ) ∈ Φ(xo , yo ), therefore, it is a fixed point of the map Φ. This connection of a saddle point with the fixed point of a set valued map allows us to show the existence of saddle points, through the use of fixed-point theorems such as, for instance, the Knaster–Kuratowski–Mazurkiewicz fixed-point Theorem 3.2.11. The following characterization of a saddle point is useful. Proposition 5.1.2. The function F : X × Y → ℝ admits a saddle point (xo , yo ) if and only if F satisfies the minimax equality (5.2), with the supremum in the first expression and the infimum on the second being attained at xo and yo , respectively. Proof. The proof is simple but is included in the Appendix of the chapter (Section 5.7.1) for completeness. Remark 5.1.3. An alternative way of stating the above is saying that the function F2 : Y → ℝ defined by F2 (y) = infx∈X F(x, y) attains its supremum over Y, and the function F1 : X → ℝ, defined by F1 (x) = supy∈Y F(x, y) attains its infimum over X. In fact, it is related to the existence of a xo ∈ X such that supy∈Y F(xo , y) = infx∈X supy∈Y F(x, y) = infx∈X F1 (x), and to the existence of a yo ∈ Y such that infx∈X F(x, yo ) = supy∈Y infx∈X F(x, y) = supy∈Y F2 (y). We may therefore find the common value (saddle value) either by fixing x = xo and then maximizing the section F(xo , ⋅) : Y → ℝ over Y, or by fixing y = yo and then minimizing the section F(⋅, yo ) : X → ℝ over X.

5.1 A minimax theorem

| 193

The problem of existence of saddle points for a given function F : X × Y → ℝ has attracted a lot of attention since Von Neumann’s pioneering work on functions defined on finite sets, and its connection with the foundations of game theory and, in particular, bimatrix zero sum games. Since then, there have been various generalizations and reformulations. We present here a general version of the minimax theorem due to Sion ([103] and present its proof in the spirit of [85]. The theorem requires a condition less stringent than convexity. Definition 5.1.4 (Quasi-convex and quasi-concave function). A function φ : A ⊂ X → ℝ ∪ {+∞} is called: (i) Quasi-convex, if for every λ ∈ ℝ the sets {x ∈ A : φ(x) ≤ λ} are convex. (ii) Quasi-concave, if for every λ ∈ ℝ the sets {x ∈ A : φ(x) ≥ λ} are convex. A convex function is clearly quasi-convex but the converse is not necessarily true; likewise with concave functions. We now state and prove a version of the minimax theorem ([85]) which will suffice for our needs. Theorem 5.1.5 (Minimax). Let CX ⊂ X and CY ⊂ Y be nonempty, compact and convex sets. Consider the function F : CX × CY → ℝ such that: (i) F(⋅, y) : CX → ℝ is lower semicontinuous and quasi-convex2 for any y ∈ CY . (ii) F(x, ⋅) : CY → ℝ is upper semicontinuous and quasi-concave3 for any x ∈ CX . Then the function F has a saddle point. Proof. The proof which makes use of the Knaster–Kuratowski–Mazurkiewicz lemma (Theorem 3.2.11) is broken into 3 steps: 1. By Proposition 5.1.2, it is sufficient to show that min max F(x, y) = max min F(x, y). x∈CX y∈CY

y∈CY x∈CX

(5.4)

Fix an x ∈ CX . Since F(x, ⋅) : CY → ℝ is upper semicontinuous and CY is compact, by the Weierstrass theorem, the maximum exists, i. e., there exists yo ∈ CY such that maxy∈CY F(x, y) = F(x, yo ). The function h : CX → ℝ defined by h(x) = maxy∈CY F(x, y) is lower semicontinuous; hence, by applying the Weierstrass theorem once more, minx∈CX h(x) = minx∈CX maxy∈CY F(x, y) exists. By similar arguments, maxy∈CY minx∈CX F(x, y) exists. It remains to show that these two values are equal. 2. To this end, by (5.1) it is true that max min F(x, y) ≤ min max F(x, y). y∈CY x∈CX

x∈CX y∈CY

2 i. e., for every y ∈ CY and for every λ ∈ ℝ the sets {x ∈ CX : F(x, y) ≤ λ} are convex. 3 i. e., for every x ∈ CX and for every λ ∈ ℝ the sets {y ∈ CY : F(x, y) ≥ λ} are convex.

194 | 5 Minimax theorems and duality We will show that the strict inequality cannot hold hence, the two values are necessarily equal. Assume per contra that it did. Then, there exists λ ∈ ℝ such that max min F(x, y) < λ < min max F(x, y). y∈CY x∈CX

x∈CX y∈CY

(5.5)

Consider the set valued mappings F1 , F2 : CX → 2CY defined by x 󳨃→ F1 (x) = {y ∈ CY : F(x, y) < λ},

x 󳨃→ F2 (x) = {y ∈ CY : F(x, y) > λ},

whose inverse are defined by y 󳨃→ F1−1 (y) = {x ∈ CX : F(x, y) < λ},

y 󳨃→ F2−1 (y) = {x ∈ CX : F(x, y) > λ}.

By the assumptions on F, we see that for every x ∈ CX it holds that F1 (x) is open in CY and F2 (x) is convex and nonempty (by (5.5)).4 On the other hand, for every y ∈ CY it holds that F1−1 (y) is convex and F2−1 (y) is open.5 We claim that because of these properties there exists x̄ ∈ CX such that F1 (x)̄ ∩ F2 (x)̄ ≠ 0.

(5.6)

Accepting this claim for the time being, we see that this leads us to a contradiction. ̄ Then, since ȳ ∈ F1 (x)̄ by the definition of F1 it holds that Indeed, let ȳ ∈ F1 (x)̄ ∩ F2 (x). ̄ ̄ ̄ F(x, y) < λ, while since y ∈ F2 (x)̄ by the definition of F2 it holds that F(x,̄ y)̄ > λ, which is clearly a contradiction. Hence, (5.5) does not hold and, therefore, (5.4) is true. 3. It remains to verify the crucial claim (5.6). This can be done considering the set valued map Φ : CX × CY → 2CX ×CY defined by c

Φ(x, y) = (CX × CY ) ∩ (F2 (y)−1 × F1 (x)) . 3(a) Observe that for the map Φ it holds that6 ⋂ (x,y)∈CX ×CY

Φ(x, y) = 0.

(5.7)

4 Since λ < minx∈CX maxy∈CY F(x, y) ≤ maxy∈CY F(x, y) for every x ∈ CX , and the maximum of F(x, y) is attained in CY , it follows that for every x ∈ CX there exists a yo = yo (x) (a maximizer of F(x, ⋅)), such that λ < F(x, yo ), i. e., yo ∈ F2 (x). 5 F1 (x) is open since F(x, ⋅) is upper semicontinuous, F2 (x) is convex by the quasi-concavity of F(x, ⋅), F1−1 (y) is convex by the quasi-convexity of F(⋅, y) and F2−1 (y) is open by the lower semicontinuity of F(⋅, y) (recall Definition 2.2.1). 6 Suppose not. There exists (x󸀠o , y󸀠o ) ∈ ⋂(x,y)∈CX ×CY Φ(x, y), i. e., x󸀠o ∈ (F2−1 (y))c for every y ∈ CY and y󸀠o ∈ F1 (x) for every x ∈ CX . The first one implies that y ∈ ̸ F2 (x󸀠o ) for every y ∈ CY , which means that F2 (x󸀠o ) = 0 which is in contradiction with (5.5).

5.1 A minimax theorem

| 195

By (5.7), there exists a finite set {(xi , yi ) ∈ CX × CY : i = 1, . . . , n} such that conv{(xi , yi ) : i = 1, . . . , n} ⊄ ⋃nk=1 Φ(xi , yi ). The existence of such a set arises from the following argument: if such a set did not exist then for any finite set {(x󸀠i , y󸀠i ) ∈ CX × CY : i = 1, . . . , n} we would have that n

conv{(x󸀠i , y󸀠i ) : i = 1, . . . , n} ⊂ ⋃ Φ((x󸀠i , y󸀠i ); k=1

hence, by the KKM lemma (Theorem 3.2.11), for any such finite set we would have that ⋂ni=1 Φ(x󸀠i , y󸀠i ) ≠ 0. But this implies that the family {Φ(x, y) : (x, y) ∈ CX × CY } has the finite intersection property. However, recall (see Proposition 1.8.1) that a metric space is compact if and only if every collection of closed sets with the finite intersection property has a non empty intersection. Since CX × CY is by assumption compact, this would imply that ⋂(x,y)∈CX ×CY Φ(x, y) ≠ 0 which clearly contradicts our observation that ⋂(x,y)∈CX ×CY Φ(x, y) = 0. 3(b) We now consider the finite set {(xi , yi ) ∈ CX ×CY : i = 1, . . . , n} of step (a). Since conv{(xi , yi ) : i = 1 ⋅ ⋅ ⋅ n} ⊄ ⋃nk=1 Φ(xi , yi ), there exist (λ1 , . . . , λn ), λi ∈ [0, 1], ∑ni=1 λi = 1 such that n

n

i=1 n

i=1 n

i=1

i=1

c

x̄ := ∑ λi xi ∈ ̸ ⋃((CX × CY ) ∩ (F2−1 (yi )) ), c

ȳ := ∑ λi yi ∈ ̸ ⋃((CX × CY ) ∩ (F1 (xi )) ), which in turn (upon taking complements) implies that n

x̄ ∈ ⋂ F2−1 (yi ), i=1

n

ȳ ∈ ⋂ F1 (xi ), i=1

or equivalently x̄ ∈ F2−1 (yi ), ȳ ∈ F1 (xi ) for every i = 1, . . . , n, which in turn implies that ̄ yi ∈ F2 (x),

̄ xi ∈ F1−1 (y),

i = 1, . . . , n.

̄ we conclude that ȳ ∈ F2 (x)̄ and x̄ ∈ F1−1 (y)̄ By the convexity of the sets F2 (x)̄ and F1−1 (y), so that ȳ ∈ F1 (x)̄ and ȳ ∈ F2 (x)̄ are required. This concludes the proof. There are numerous extensions to this fundamental minimax theorem, either relaxing some of the compactness assumptions or some of the convexity assumptions. Furthermore, there is a wide variety of applications of minimax theorems, ranging from game theory and decision making to optimization, PDEs or statistics and machine learning. Example 5.1.6 (Game theory). Consider the interaction of two players A and E. In this setting, let F(x, y) be the payoff of A if he plays strategy x ∈ CX while E plays strategy y ∈ CY , and −F(x, y) be the payoff for E for the same choice. Then any element of

196 | 5 Minimax theorems and duality arg maxx∈CX F(x, y) is a strategy for A which is a best reply to the action y chosen by E; hence, as rational agent he would choose among these. Likewise, any element of arg maxy∈CY F(x, y) is a strategy for E, which is a best reply to the action x chosen by A; hence, as rational agent she would choose among them. As a result of the interaction between A and E, the outcome of the game would be strategies (xo , yo ) that satisfy the fixed-point condition xo ∈ arg maxx∈CX F(x, yo ) and yo ∈ arg maxy∈CY F(xo , y), which correspond to saddle points of the payoff function F. It is important to note here that via the restatement of a saddle point as the fixed point of a set valued map, one may generalize the notion of saddle point for functions F in more than two variables, with important consequences for the theory of noncooperative games for more than two players (see e. g. [87]). ◁

5.2 Conjugate functions Conjugate functions play a very important role in nonlinear analysis (see e. g. [13, 19, 25, 49, 62]). 5.2.1 The Legendre–Fenchel conjugate We will adopt the convention of denoting by F any real valued function, and by φ any real valued function which is convex. Definition 5.2.1 (Fenchel–Legendre conjugate or transform). Let X be a Banach space with dual X ⋆ and denote by ⟨⋅, ⋅⟩ the duality pairing between X and X ⋆ . For a proper function F : X → ℝ ∪ {+∞}, its Legendre–Fenchel conjugate (or transform) F ⋆ : X ⋆ → ℝ ∪ {+∞} is defined by F ⋆ (x⋆ ) := sup(⟨x⋆ , x⟩ − F(x)), x∈X

∀ x⋆ ∈ X ⋆ .

The properness of F rules out the possibility of F ⋆ admitting the value −∞, thus guaranteeing that F ⋆ is well-defined. The Legendre–Fenchel transform satisfies the following elementary property. Proposition 5.2.2 (Young–Fenchel inequality). Let F : X → ℝ ∪ {+∞} be a proper function and F ⋆ : X ⋆ → ℝ ∪ {+∞} its Legendre–Fenchel transform. Then F(x) + F ⋆ (x⋆ ) ≥ ⟨x⋆ , x⟩,

∀ (x, x⋆ ) ∈ X × X ⋆ .

Proof. By the definition of F ⋆ we have that F ⋆ (x⋆ ) ≥ ⟨x⋆ , x⟩ − F(x),

∀ (x, x⋆ ) ∈ X × X ⋆ ,

from which the Young–Fenchel inequality follows.

(5.8)

5.2 Conjugate functions | 197

Example 5.2.3. Let X be a Banach space and X ⋆ its dual. Define φ : X → ℝ as φ(x) := 1 ‖x‖pX for some p ∈ (1, ∞). Then φ⋆ : X ⋆ → ℝ is the function defined by φ⋆ (x⋆ ) := p ⋆ 1 ‖x⋆ ‖pX ⋆ , p⋆

for p⋆ such that

1 p

+

1 p⋆

= 1.

Let ψ : ℝ → ℝ be defined as ψ(x) := p1 |x|p so that, using straightforward calculus,

ψ⋆ (y) =

⋆ 1 |y|p . p⋆

Notice also that ψ and ψ⋆ are even functions. By definition,

φ⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − ψ(‖x‖X )) = sup = sup

sup (⟨x⋆ , x⟩ − ψ(‖x‖X ))

r∈ℝ+ x∈X,‖x‖X =r

x∈X

󵄩 󵄩 sup (r⟨x⋆ , x⟩ − ψ(r)) = sup (r 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X − ψ(r))

r∈ℝ+ x∈X,‖x‖X =1

r∈ℝ+

󵄩 󵄩 󵄩 󵄩 = sup(r 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X − ψ(r)) = ψ⋆ (󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ), r∈ℝ

where we have used the fact that ψ is even to pass from the optimization problem over ℝ+ to the optimization problem over the whole of ℝ. Note also that in the proof above we have used nothing else but the fact that ψ is even. ◁ Example 5.2.4 (The Legendre–Fenchel conjugate of the norm). What happens in Example 5.2.3 if p = 1? Consider the function φ : X → ℝ defined by φ(x) := ‖x‖X . Then φ⋆ (x⋆ ) = IB⋆ , the convex indicator function of the dual unit ball B⋆ := BX ⋆ (0, 1) = {x⋆ ∈ X ⋆ : ‖x⋆ ‖X ⋆ ≤ 1}. Consider first any x⋆ ∈ B⋆ . Then since |⟨x⋆ , x⟩| ≤ ‖x⋆ ‖X ⋆ ‖x‖X for x⋆ ∈ B⋆ it holds that ⟨x⋆ , x⟩ ≤ ‖x‖X ; hence, ⟨x⋆ , x⟩ − ‖x‖X ≤ 0, attaining the upper bound for x = 0. Therefore, if x⋆ ∈ B⋆ , φ⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − ‖x‖X ) = 0. x∈X

Consider now any x⋆ ∈ X ⋆ \ B⋆ . Then, since ‖x⋆ ‖X ⋆ > 1, there exists xo ∈ X ⋆ such that ρ := ⟨x⋆ , xo ⟩ − ‖xo ‖X > 0.7 Then for any r ∈ ℝ+ consider the element x = rxo and observe that ⟨x⋆ , x⟩ − ‖x‖X = rρ. By the definition of φ⋆ we see that φ⋆ (x⋆ ) ≥ rρ for any x⋆ ∈ X ⋆ \ B⋆ . Letting r → ∞, we conclude that in this case φ⋆ (x⋆ ) = +∞. Therefore, φ⋆ = IB⋆ . ◁ Example 5.2.5. Let ψ : ℝ → ℝ be an even function and ψ⋆ : ℝ → ℝ its Legendre– Fenchel conjugate. Furthermore, let X be a Banach space with X ⋆ its dual, and define the function φ : X → ℝ by φ(x) := ψ(‖x‖X ) for every x ∈ X. Then φ⋆ : X ⋆ → ℝ is the function φ⋆ (x⋆ ) = ψ⋆ (‖x⋆ ‖X ⋆ ) for every x⋆ ∈ X ⋆ . This can be shown using verbatim the approach in Example 5.2.3. ◁ The following examples show that indicator functions and support functions are connected through the Fenchel–Legendre conjugate. 7 Indeed, by the definition of the dual norm ‖x⋆ ‖X ⋆ = sup‖x‖≤1 ⟨x⋆ , x⟩ > 1 there exists xo ∈ X with ‖xo ‖X ≤ 1, so that ⟨x⋆ , xo ⟩ > 1 and the claim follows.

198 | 5 Minimax theorems and duality Example 5.2.6. The Legendre–Fenchel conjugate of the indicator function of the closed unit ball of X is related to the norm of its dual space, X ⋆ . Let B := BX (0, 1) = {x ∈ X : ‖x‖X ≤ 1}, the closed unit ball of X and consider its indicator function φ := IB . Then φ⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − IB (x)) = sup⟨x⋆ , x⟩ = x∈X

x∈B

󵄩 󵄩 sup ⟨x⋆ , x⟩ = 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ .

x∈X,‖x‖X =1

◁

Example 5.2.7. The Legendre–Fenchel conjugate of the indicator function of a closed convex set is its support function. Let C ⊂ X be a closed convex set and consider its indicator function φ := IC . Then φ⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − IC (x)) = sup⟨x⋆ , x⟩ = σC (x⋆ ), x∈X

x∈C

where σC : X ⋆ → ℝ is the support function of C (see Definition 2.3.12).

◁

Example 5.2.8 (The epigraph of the Legendre–Fenchel transform). Let F be any proper function (not necessarily convex) and F ⋆ be its conjugate function. Then, upon defining, for every x ∈ X, the family of functions Fx : X ⋆ → ℝ ∪ {+∞}, by x⋆ 󳨃→ Fx (x⋆ ) := ⟨x⋆ , x⟩ − F(x), it holds that epi F ⋆ = {(x⋆ , λ) ∈ X ⋆ × ℝ : F(x) ≥ ⟨x⋆ , x⟩ − λ, ∀x ∈ X} = ⋂ epi Fx , x∈X

i. e., the points of the epigraph of F ⋆ parametrize the set of affine functions minorizing F, or equivalently, the epigraph of F ⋆ is the intersection over all x ∈ X of the epigraphs of the family of functions Fx : X ⋆ → ℝ. This follows easily by the definition of the epigraph. Take any (x⋆ , λ) ∈ epi F ⋆ . Then F ⋆ (x⋆ ) ≤ λ and by the definition of F ⋆ we have that for every x ∈ X it holds that ⟨x⋆ , x⟩ − F(x) ≤ λ; hence, (x⋆ , λ) ∈ {(x⋆ , λ) : F(x) ≥ ⟨x⋆ , x⟩ − λ, ∀x ∈ X}, therefore, epi F ⋆ ⊂ {(x⋆ , λ) : F(x) ≥ ⟨x⋆ , x⟩ − λ, ∀x ∈ X}. For the reverse inclusion, consider any (x⋆ , λ) ∈ {(x⋆ , λ) : F(x) ≥ ⟨x⋆ , x⟩ − λ, ∀x ∈ X}. Then ⟨x⋆ , x⟩ − F(x) ≤ λ for every x ∈ X and taking the supremum over all x ∈ X we conclude that F ⋆ (x⋆ ) ≤ λ; hence, (x⋆ , λ) ∈ epi F ⋆ . The second claim follows by observing that {(x⋆ , λ) : F(x) ≥ ⟨x⋆ , x⟩ − λ, ∀x ∈ X} = {(x⋆ , λ) : Fx (x⋆ ) ≤ λ} = ⋂x∈X epi Fx . ◁ Example 5.2.9 (Fenchel–Legendre transform of the Moreau–Yosida regularization). Let X = H be a Hilbert space, identified with its dual, let φ : H → ℝ ∪ {+∞} be a proper lower semicontinuous convex function and ϕλ its Moreau–Yosida regularization defined by ϕλ (x) = infz∈H (φ(z) + 2λ1 ‖z − x‖2H ) (see Definition 4.7.3). Then φ⋆λ (x⋆ ) = φ⋆ (x⋆ ) + λ2 ‖x⋆ ‖2H . Using the definitions, and the fact that we may interchange the order of suprema, φ⋆λ (x⋆ ) = sup(⟨x⋆ , x⟩H − φλ (x)) = sup(⟨x⋆ , x⟩H − inf ( x∈H

x∈H

z∈H

1 ‖z − x‖2H + φ(z))) 2λ

5.2 Conjugate functions | 199

= sup(⟨x⋆ , x⟩H + sup(− x∈H

z∈H

1 ‖z − x‖2H − φ(z))) 2λ

= sup sup(⟨x⋆ , z⟩H − φ(z) + ⟨x⋆ , x − z⟩H − z∈H x∈H

1 ‖z − x‖2H ) 2λ

= sup(⟨x⋆ , z⟩H − φ(z) + sup(⟨x⋆ , x − z⟩H − z∈H

x∈H

= sup(⟨x⋆ , z⟩H − φ(z) + sup(⟨x⋆ , x󸀠 ⟩H − z∈H

x󸀠 ∈H

1 ‖x − z‖2H )) 2λ

1 󵄩󵄩 󸀠 󵄩󵄩2 ⋆ ⋆ 󵄩x 󵄩 )) = φ (x ) + 2λ 󵄩 󵄩H

and our claim is proved.

λ 󵄩󵄩 ⋆ 󵄩󵄩2 󵄩x 󵄩 , 2 󵄩 󵄩H ◁

The following proposition collects some useful properties of Fenchel–Legendre transforms. Proposition 5.2.10 (Properties of Legendre–Fenchel transform). (i) F ⋆ is convex and lower semicontinuous (even if F may not be convex). ⋆ (ii) (λF)⋆ (x⋆ ) = λF ⋆ ( xλ ) for every λ > 0. (iii) Let z ∈ X and define the translation of F by z, Fz : X → ℝ ∪ {+∞} as Fz (x) := F(x − z) for every x ∈ X. Then Fz⋆ (x⋆ ) = F ⋆ (x⋆ ) + ⟨x⋆ , z⟩. (iv) If F1 ≤ F2 , then F1⋆ ≥ F2⋆ . Proof. (i) We define the functional ψx : X ⋆ → ℝ, by ψx (x⋆ ) := ⟨x⋆ , x⟩ − F(x), and note that by definition, F ⋆ (x⋆ ) = supx∈X ψx (x⋆ ) for any x⋆ ∈ X ⋆ . Since F ⋆ is the pointwise supremum over the family of affine functions x⋆ 󳨃→ ψx (x⋆ ), it is convex and lower semicontinuous (see Proposition 2.3.7(iii) and Example 2.2.2, resp.). (ii) To calculate (λF)⋆ we need to solve the optimization problem (λF)⋆ (x⋆ ) = sup (⟨x⋆ , x⟩ − (λF)(x)). x∈X ⋆

We express the quantity to be optimized as ⟨x⋆ , x⟩ − (λF)(x) = λ(⟨

x⋆ , x⟩ − F(x)), λ

so that sup(⟨x⋆ , x⟩ − (λF)(x)) = λ sup(⟨ x∈X

x∈X

x⋆ x⋆ , x⟩ − F(x)) = λF ⋆ ( ). λ λ

(iii) By definition, Fz⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − Fz (x)) = sup(⟨x⋆ , x⟩ − F(x − z)). x∈X

x∈X

We express the quantity to be maximized as ⟨x⋆ , x⟩ − F(x − z) = ⟨x⋆ , x − z⟩ − F(x − z) + ⟨x⋆ , z⟩,

200 | 5 Minimax theorems and duality so that sup(⟨x⋆ , x⟩ − F(x − z)) = sup(⟨x⋆ , x − z⟩ − F(x − z) + ⟨x⋆ , z⟩) x∈X

x∈X

= sup(⟨x⋆ , x − z⟩ − F(x − z)) + ⟨x⋆ , z⟩ = F ⋆ (x⋆ ) + ⟨x⋆ , z⟩. x∈X

(iv) Since F1 ≤ F2 , we have that ⟨x⋆ , x⟩ − F2 (x) ≤ ⟨x⋆ , x⟩ − F1 (x),

∀ (x, x⋆ ) ∈ X × X ⋆ .

We first estimate the right-hand side by its supremum so that ⟨x⋆ , x⟩ − F2 (x) ≤ ⟨x⋆ , x⟩ − F1 (x) ≤ sup(⟨x⋆ , x⟩ − F1 (x)) = F1⋆ (x⋆ ), x∈X

∀ (x, x⋆ ) ∈ X × X ⋆ ,

and since for any fixed x⋆ ∈ X ⋆ ⟨x⋆ , x⟩ − F2 (x) ≤ F1⋆ (x⋆ ),

∀ x ∈ X,

the inequality holds also for the supremum of the left-hand side over X, therefore, F2⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − F2 (x)) ≤ F1⋆ (x⋆ ). x∈X

Since this is true for any x⋆ ∈ X ⋆ , it follows that F2⋆ ≤ F1⋆ . It is clear from the definition of the Legendre–Fenchel transform that if we consider it as an operator from X to X ⋆ , it is not a linear operator, i. e., in general (φ1 + φ2 )⋆ ≠ φ⋆1 + φ⋆2 . An interesting question that arises is whether φ⋆1 + φ⋆2 can actually be expressed as the Legendre–Fenchel transform of some function. This leads us to the important notion of the inf-convolution (see Section 5.3).

5.2.2 The biconjugate function We continue adopting the convention of denoting by F any real valued function, and by φ any real valued function which is convex. Definition 5.2.11 (Biconjugate). Let F : X → ℝ ∪ {+∞} be a proper function. The biconjugate of F is the function F ⋆⋆ : X → ℝ ∪ {+∞} defined by F ⋆⋆ (x) = sup (⟨x⋆ , x⟩ − F ⋆ (x⋆ )). x⋆ ∈X ⋆

Remark 5.2.12. In general, we may consider F ⋆⋆ as a function from X ∗∗ to ℝ, which is the Fenchel conjugate of F ⋆ , i. e., as (F ⋆ )⋆ . What we have defined in Definition 5.2.11

5.2 Conjugate functions | 201

can then be understood as the restriction of (F ⋆ )⋆ on X (which can be seen as a subspace of X ∗∗ using the canonical embedding j : X → X ∗∗ defined by ⟨j(x), x⋆ ⟩X ∗∗ ,X ⋆ = ⟨x⋆ , x⟩X ⋆ ,X ; see Theorem 1.1.22). Under this perspective, we could define F ⋆⋆ (j(x)) = sup (⟨j(x), x⋆ ⟩X ∗∗ ,X ⋆ − F ⋆ (x⋆ )) = sup (⟨x⋆ , x⟩X ⋆ ,X − F ⋆ (x⋆ )). x⋆ ∈X ⋆

x⋆ ∈X ⋆

Clearly, if X is a reflexive space these two concepts coincide. The next proposition shows that the biconjugate of a function can be understood as a convexification of a function from below. We have already seen (Theorem 2.3.19) that a proper convex lower semicontinuous function can be expressed as the supremum over the family of all affine functions that minorize it. The obvious question is what happens if we drop the assumptions of convexity and lower semicontinuity. To this end, we must first define the concept of Γ-regularization. Definition 5.2.13 (Γ-Regularization). Let F : X → ℝ ∪ {+∞} be a proper function, and set 𝔸(F) := {g : X → ℝ : g continuous and affine, g ≤ F}. The Γ-regularization of F, is defined as the function F Γ : X → ℝ ∪ {+∞} such that8 F Γ (x) := sup g(x), g∈𝔸(F)

∀ x ∈ X.

It can be seen by a straightforward application of Proposition 2.3.7(iii) that F Γ is a convex function even if F is not. Furthermore, as the supremum of a family of affine functionals, F Γ enjoys lower semicontinuity properties. In this respect, F Γ can be considered as a convex regularization of F from below. Proposition 5.2.14. Let F : X → ℝ ∪ {+∞} be a proper function. Then it holds that F ⋆⋆ = F Γ ≤ F. Proof. The fact that F Γ ≤ F follows from the definition of F Γ : Fixing any x ∈ X then for any g ∈ 𝔸(F), it holds that g(x) ≤ F(x); hence, taking the supremum over all g ∈ 𝔸(F) we conclude that F Γ ≤ F. It remains to show that F ⋆⋆ = F Γ . To this end, consider any affine function g ∈ 𝔸(F). Since g is affine, it is completely characterized by an element x⋆ ∈ X ⋆ and a real number c ∈ ℝ such that ⟨x⋆ , x⟩ + c ≤ F(x), for every x ∈ X. Therefore, to characterize F Γ (x) as the supremum of the quantity g(x) when g ∈ 𝔸(F) it is equivalent to characterize the supremum of the quantity ⟨x⋆ , x⟩ + c, over all x⋆ ∈ X ⋆ and over all c ∈ ℝ such that ⟨x⋆ , x⟩ + c ≤ F(x). But this last condition implies that ⟨x⋆ , x⟩ − F(x) ≤ −c, for all 8 We use the convention that sup 0 = −∞.

202 | 5 Minimax theorems and duality x ∈ X, so that F ⋆ (x⋆ ) := supx∈X (⟨x⋆ , x⟩ − F(x)) ≤ −c. This means that given an x⋆ ∈ X ⋆ (which characterizes the chosen affine function g) then the real number c cannot be arbitrary, but rather must satisfy the constraint c ≤ −F ⋆ (x⋆ ). Therefore, F Γ (x) =

x⋆ ∈X ⋆ ,

sup

⟨x⋆ , x⟩ + c.

c≤−F ⋆ (x⋆ )

Let us consider the constraints a bit more carefully. If F ⋆ (x⋆ ) > −∞, for every x⋆ ∈ X ⋆ , then given x⋆ ∈ X ⋆ , c, can go up to the maximum value cmax = −F ⋆ (x⋆ ), so the maximum over c, of the quantity ⟨x⋆ , x⟩ + c, for fixed x⋆ will be ⟨x⋆ , x⟩ − F ⋆ (x⋆ ). We then vary x⋆ over X ⋆ and take the supremum of the resulting quantity. This problem is simply the problem supx⋆ ∈X ⋆ (⟨x⋆ , x⟩ − F ⋆ (x⋆ )), which is nothing else but F ⋆⋆ (x). Summarizing the above, if x⋆ is such that F ⋆ (x⋆ ) > −∞, then F Γ (x) =

x⋆ ∈X ⋆ ,

sup

(⟨x⋆ , x⟩ + c) = sup (⟨x⋆ , x⟩ − F ⋆ (x⋆ )) = F ⋆⋆ (x).

c≤−F ⋆ (x⋆ )

x⋆ ∈X ⋆

If F ⋆ (x⋆ ) = −∞ for some x⋆ ∈ X ⋆ then we can see that F Γ (x) = +∞ = F ⋆⋆ (x) for all x ∈ X. Proposition 5.2.15 (Fenchel–Moreau–Rockafellar). Let F : X → ℝ ∪ {+∞} be a proper function. Then F ⋆⋆ = F if and only if F is convex and lower semicontinuous. Proof. We only prove that if F is convex and lower semicontinuous that F = F ⋆⋆ , as the other direction is easy. Assume that F is convex and lower semicontinuous. We will show that F = F ⋆⋆ . To this end, we will use the representation of F ⋆⋆ as F ⋆⋆ = F Γ (see Proposition 5.2.14). By the definition of F Γ , it is clear that F Γ ≤ F. To show that they are equal, it is enough to show that there does not exist any x ∈ X for which F Γ (x) < γ < F(x) for some γ ∈ ℝ. Assume the contrary, and let z ∈ X be such a point, i. e., F Γ (z) < γ < F(z). We will then show that there exists an affine function g ∈ 𝔸(F) satisfying the property g(z) > γ, therefore, F Γ (z) > γ (by the definition of F Γ (z) as the supremum of ĝ (z) over all ĝ ∈ 𝔸(F)) which contradicts the choice of z. To establish the contradiction, we will apply the strict separation Theorem 1.2.9 to the sets C1 := epi F and C2 := {(z, γ)}, both considered as subsets of Y := X × ℝ, which is obviously a Banach space with dual9 Y ⋆ = X ⋆ × ℝ. Since F is convex, by Proposition 2.3.6 C1 is a convex set and since F is lower semicontinuous, it is also closed. C2 consists of a single point so it is trivially convex and closed. Furthermore, C1 ∩ C2 = 0 by the choice of z. Therefore, by the strict separation theorem there exists y⋆ := (x⋆ , c) ∈ Y ⋆ and α ∈ ℝ such that ⟨x⋆ , z⟩ + cγ > α and ⟨x⋆ , x⟩ + cλ < α for every 9 Any element of y ∈ Y is identified at the pair (x, c1 ) ∈ X × ℝ, any element y⋆ ∈ Y ⋆ is identified as a pair (x⋆ , c2 ) ∈ X ⋆ × ℝ and the duality pairing between the two spaces is expressed as ⟨y⋆ , y⟩Y ⋆ ,Y := ⟨x⋆ , x⟩X ⋆ X + c1 c2 .

5.2 Conjugate functions | 203

(x, λ) ∈ C1 := epi F, i. e., for every (x, λ) such that F(x) ≤ λ. Clearly, if (x, λ) ∈ epi F, then for any μ > λ, (x, μ) ∈ epi F. Applying the separation result for the new point (x, μ), we have that ⟨x⋆ , x⟩ + cμ < α for any μ > λ and letting μ → ∞ leads to the conclusion c ≤ 0. Consider now the value of F at the point z. It can either hold that (a) F(z) < +∞ or (b) F(z) = +∞. We will show that either case leads to a contradiction. (a) Let us consider the first case, F(z) < +∞. This implies that the point (z, F(z)) ∈ C1 := epi F, therefore, by the separation result we have that ⟨x⋆ , z⟩ + cF(z) < α < ⟨x⋆ , z⟩ + cγ,

(5.9)

and since γ < F(z) this leads to the conclusion that c < 0. Consider the function g : X → ℝ defined by g(x) = αc − c1 ⟨x⋆ , x⟩, which is an affine function. However, it also satisfies the property that g(x) ≤ F(x), for every x ∈ X, therefore,10 g ∈ 𝔸(F). Furthermore, g(z) = αc − c1 ⟨x⋆ , z⟩ > γ, by (5.9). (b) In the second case, i. e., if F(z) = +∞, and since c ≤ 0, it may either hold that c < 0 or that c = 0. If c < 0, we may define g : X → ℝ as above and show that g ∈ 𝔸(F) and g(z) > γ. If c = 0, the construction of g has to be a little different. Since F is proper, there exists xo ∈ dom(F), i. e., xo ∈ X such that F(xo ) < ∞. Then observe that (xo , F(xo )) ∈ C1 = epi F and apply the separation result to obtain that ⟨x⋆ , xo ⟩ + cF(xo ) < α < ⟨x⋆ , z⟩ + cγ, which since c = 0 simplifies to ⟨x⋆ , xo ⟩ < α < ⟨x⋆ , z⟩. We may then define an affine function g1 : X → ℝ such that g1 ∈ 𝔸(F), working as in case (a) above with xo in lieu of z. We may further define g : X → ℝ by g(x) := g1 (x) + |γ−g1 (z)| ⋆ ( ⟨x ⋆ ,z⟩−α + 1)(⟨x , x⟩ − α) and observe that g is affine and it satisfies g(x) ≤ g1 (x) ≤ F(x) for every x ∈ X so that g ∈ 𝔸(F). Furthermore, g(z) = g1 (z) + |γ − g1 (z)| + (⟨x⋆ , z⟩ − α) > γ. Therefore, in all cases we may construct a function g ∈ 𝔸(F) such that g(z) > γ. This establishes the contradiction and the proof is complete. Example 5.2.16. Let C be a closed convex set and let φ = IC be the indicator function of C as defined in Example 2.3.9. We have seen in Example 5.2.7 that IC⋆ = σC , the support function of C. Since C is closed and convex, all the requirements of the Fenchel– Moreau–Rockafellar theorem hold, so that IC = IC⋆⋆ = (IC⋆ )⋆ = σC⋆ , which is of course related to the dual representation of convex sets in terms of the support function (see (2.5), Section 2.3.2). ◁ Remark 5.2.17 (Can we go beyond the biconjugate?). An interesting question that arises is whether we need to go beyond the biconjucate function. The answer is no, as if we did, then φ⋆⋆⋆ = φ⋆ , so any further Legendre–Fenchel transform beyond the second will coincide with either the conjugate or the biconjugate function. This 10 Indeed, if x ∈ X is such that F(x) < +∞, then since (x, F(x)) ∈ epi F the separation result yields ⟨x⋆ , x⟩ + cF(x) < α which can be restated as g(x) := αc − c1 ⟨x⋆ , x⟩ < F(x) (recall that c < 0). If on the contrary x ∈ X is such that F(x) = +∞, then g(x) := αc − c1 ⟨x⋆ , x⟩ < +∞ = F(x).

204 | 5 Minimax theorems and duality is easy to see, as φ⋆⋆ ≤ φ, upon conjugation leads to φ⋆ ≤ φ⋆⋆⋆ . On the other hand, by definition φ⋆⋆⋆ (x⋆ ) = supx∈X (⟨x⋆ , x⟩ − φ⋆⋆ (x)), and since φ⋆ (x⋆ ) ≥ ⟨x⋆ , x⟩ − φ⋆⋆ (x) for every x ∈ X ⋆ , taking the supremum over all x ∈ X leads to the reverse inequality φ⋆⋆⋆ ≤ φ⋆ , and the desired result follows. 5.2.3 The subdifferential and the Legendre–Fenchel transform Recall the Young–Fenchel inequality (see Proposition 5.2.2) according to which φ(x) + φ⋆ (x⋆ ) ≥ ⟨x⋆ , x⟩ for every (x, x⋆ ) ∈ X × X ⋆ . The following result shows that the subdifferentials of φ and φ⋆ can be characterized as the pairs (x, x⋆ ) ∈ X × X ⋆ for which equality is attained in the Young–Fenchel inequality. Proposition 5.2.18. Let φ : X → ℝ ∪ {+∞} be a proper and convex function. Then: (i) φ(x) + φ⋆ (x⋆ ) = ⟨x⋆ , x⟩ if and only if x⋆ ∈ 𝜕φ(x). (ii) If x⋆ ∈ 𝜕φ(x), then x ∈ 𝜕φ⋆ (x⋆ ) (interpreted as in Remark 5.2.20 if X is non reflexive). (iii) If furthermore, X is reflexive and φ is lower semicontinuous, then x⋆ ∈ 𝜕φ(x) if and only if x ∈ 𝜕φ⋆ (x⋆ ). Proof. (i) Assume that φ(x) + φ⋆ (x⋆ ) = ⟨x⋆ , x⟩. We will show that x⋆ ∈ 𝜕φ(x). By definition φ⋆ (x⋆ ) = supx∈X (⟨x⋆ , x⟩ − φ(x)), so that φ⋆ (x⋆ ) ≥ ⟨x⋆ , z⟩ − φ(z),

∀ z ∈ X.

(5.10)

Since φ⋆ (x⋆ ) = ⟨x⋆ , x⟩ − φ(x), we substitute that into (5.10) and this yields ⟨x⋆ , x⟩ − φ(x) ≥ ⟨x⋆ , z⟩ − φ(z),

∀ z ∈ X,

which is expressed as φ(z) − φ(x) ≥ ⟨x⋆ , z − x⟩,

∀ z ∈ X,

therefore, x⋆ ∈ 𝜕φ(x). Conversely, assume that x⋆ ∈ 𝜕φ(x). Then, by the definition of the subdifferential φ(z) − φ(x) ≥ ⟨x⋆ , z − x⟩,

∀ z ∈ X.

This is rearranged as ⟨x⋆ , z⟩ − φ(z) ≤ ⟨x⋆ , x⟩ − φ(x),

∀ z ∈ X,

therefore, taking the supremum over z ∈ X, sup(⟨x⋆ , z⟩ − φ(z)) ≤ ⟨x⋆ , x⟩ − φ(x), z∈X

5.2 Conjugate functions | 205

and hence, by the definition of the Legendre–Fenchel conjugate of φ, the above becomes φ⋆ (x⋆ ) + φ(x) ≤ ⟨x⋆ , x⟩.

(5.11)

On the other hand, the Young–Fenchel inequality yields φ(x) + φ⋆ (x⋆ ) ≥ ⟨x⋆ , x⟩, which when combined with (5.11) gives us the equality φ(x) + φ⋆ (x⋆ ) = ⟨x⋆ , x⟩. (ii) For simplicity let X be reflexive (see Remark 5.2.20 for the general case). Since x⋆ ∈ 𝜕φ(x) by (i), it holds that φ(x) + φ⋆ (x⋆ ) = ⟨x⋆ , x⟩.

(5.12)

We now pick any z⋆ ∈ X ⋆ and apply the Young–Fenchel inequality to the pair (x, z⋆ ) ∈ X × X⋆, φ(x) + φ⋆ (z⋆ ) ≥ ⟨z⋆ , x⟩,

(5.13)

and subtracting (5.12) from (5.13) yields φ⋆ (z⋆ ) − φ⋆ (x⋆ ) ≥ ⟨z⋆ − x⋆ , x⟩,

∀ z⋆ ∈ X ⋆ .

This yields that x ∈ 𝜕φ⋆ (x⋆ ). (iii) By the lower semicontinuity of φ we have that φ⋆⋆ = φ (see Proposition 5.2.15). Hence, applying (i) for φ⋆ noting that since X is reflexive X ⋆⋆ ≃ X while by lower semicontinuity φ⋆⋆ = φ, the result follows. Remark 5.2.19. The Young–Fenchel inequality yields that φ(x) + φ⋆ (x⋆ ) − ⟨x⋆ , x⟩ ≥ 0, while φ(x) + φ⋆ (x⋆ ) − ⟨x⋆ , x⟩ = 0 if and only if x⋆ ∈ 𝜕φ(x). Therefore any point (x, x⋆ ) ∈ Gr(𝜕φ) is a minimizer of the function Φ : X × X ⋆ → ℝ defined by Φ(x, x⋆ ) := φ(x) + φ⋆ (x⋆ ) − ⟨x⋆ , x⟩. Remark 5.2.20. In the general case where X is not reflexive, 𝜕φ⋆ ⊂ X ⋆⋆ . We may then interpret the condition x ∈ φ⋆ (x⋆ ) in Proposition 5.2.18 in terms of the canonical embedding j : X → X ⋆⋆ , as j(x) ∈ 𝜕φ⋆ (x⋆ ) (recall that in the reflexive case j(X) = X ⋆⋆ ). The result of Proposition 5.2.18 can then be generalized with analogous arguments.

206 | 5 Minimax theorems and duality Example 5.2.21. Let X = H be a Hilbert space (identified with its dual) and C ⊂ H be a closed convex set. Define the function φ : H → ℝ ∪ {∞} by φ(x) = 21 ‖x‖2 + IC (x). Then φ⋆ (x⋆ ) = ⟨x⋆ , PC (x⋆ )⟩ − 21 ‖PC (x⋆ )‖2 , while11 𝜕φ⋆ (x⋆ ) = PC (x⋆ ). Indeed, by definition, we have that 1 1 (IC (x) + ‖x‖2 − ⟨x⋆ , x⟩H ), φ⋆ (x⋆ ) = sup (⟨x⋆ , x⟩H − ‖x‖2 − IC (x)) = − inf ⋆ ∈H ⋆ x 2 2 x ∈H and xo = arg minz∈C φ(z) satisfies the first-order condition −(xo − x⋆ ) ∈ NC (xo ), (recall that 𝜕IC (x) = NC (x) for every x ∈ C), which is in turn equivalent (by the definition of the normal cone) to ⟨x⋆ − xo , z − xo ⟩H ≤ 0, for every z ∈ C, i. e., xo = PC (x⋆ ), and the result follows. Furthermore, since x ∈ 𝜕φ⋆ (x⋆ ) if and only if x⋆ ∈ 𝜕φ(x) and as 𝜕φ(x) = x + NC (x) we have that for x⋆ − x ∈ NC (x) which implies that x = PC (x⋆ ). ◁ Example 5.2.22 (Moreau decomposition). Let X = H be a Hilbert space (identified with its dual) and let φ : H → ℝ ∪ {+∞} be a proper convex lower semicontinuous function with convex conjugate φ⋆ . Recall the definition of the Moreau proximity operator (see Definition 4.7.1) related to the function φ, 1 proxφ (x) := arg min( ‖z − x‖2 + φ(z)), z∈H 2

(5.14)

and the corresponding proximity operator for φ⋆ , defined by 1 proxφ⋆ (x) := arg min( ‖z − x‖2 + φ⋆ (z)). z∈H 2

(5.15)

These proximity operators satisfy the Moreau decomposition property proxφ (x) + proxφ⋆ (x) = x.

(5.16)

This follows by the definition of the proximal operator and Proposition 5.2.18. Indeed, if z = proxφ (x), then by the first-order condition for problem (5.2.22) (see also Proposition 4.7.2(iii)) we have that x − z ∈ 𝜕φ(z). This by Proposition 5.2.18 is equivalent to z ∈ 𝜕φ⋆ (x − z). By the definition of proxφ⋆ we have that z󸀠 = proxφ⋆ (x󸀠 ) is equivalent to x󸀠 − z󸀠 ∈ 𝜕φ⋆ (z󸀠 ), and setting x󸀠 = x and z󸀠 = x − z, leads to the equivalence z ∈ 𝜕φ⋆ (x − z). Combining the above, we have that z = proxφ (x) is equivalent to x − z = proxφ⋆ (x), which leads to (5.16). In the particular case where φ = IE for E a closed subspace of H, so that φ⋆ = IE ⊥ , Moreau’s decomposition (5.16) reduces to the standard decomposition for orthogonal projection on subspaces, i. e., x = PE x + PE ⊥ x. Moreau’s decomposition finds useful applications in numerical algorithms for optimization. ◁ 11 In the sense that it is a singleton containing only the element PC (x⋆ ).

5.3 The inf-convolution

| 207

Example 5.2.23 (Subdifferential of the support function in Hilbert space). Let X = H be a Hilbert space (identified with its dual), C ⊂ X be a closed convex set and NC its normal cone (see Definition 4.1.5). Consider the support function of this set σC : X ⋆ → ℝ, defined by σC (x⋆ ) = supx∈C ⟨x⋆ , x⟩ which of course now, since X ⋆ ≃ X, can be considered as a function of X. It holds that x⋆ ∈ 𝜕σC (x) if and only if x ∈ NC (x⋆ ).

(5.17)

This follows by an application of Proposition 5.2.18(iii), by noting that IC⋆ = σC , that X is reflexive and that 𝜕IC (x⋆ ) = NC (x⋆ ) (see Example 4.1.6). ◁

5.3 The inf-convolution An important and useful notion is the infimal convolution (or epi sum) of functions. Definition 5.3.1 (Inf-convolution). Let φ1 , φ2 : X → ℝ ∪ {+∞} be two proper lower semicontinuous convex functions. The inf-convolution of these functions,12 denoted by φ1 ◻φ2 : X → ℝ ∪ {+∞} is the function defined by (φ1 ◻φ2 )(x) = inf [φ1 (z) + φ2 (x − z)] = inf [φ1 (x1 ) + φ2 (x2 )]. z∈X

x1 +x2 =x x1 ,x2 ∈X

The inf-convolution φ1 ◻φ2 is said to be exact at x, if there exists zo ∈ X such that (φ1 ◻φ2 )(x) = φ1 (zo ) + φ2 (x − zo ). Example 5.3.2 (inf-convolution of the indicator function). It is straightforward to see that IC1 ◻ IC2 = IC1 +C2 , where C1 , C2 are arbitrary closed convex sets. ◁ Example 5.3.3. An important special case of the inf-convolution is the case where φ2 (x − z) = 21 ‖x − z‖2 . This leads to the Moreau envelope, introduced in Section 4.7, which when X = H is a Hilbert space can be considered as a smooth approximation for a nonsmooth convex function φ1 . The choice φ2 (x−z) = ‖x−z‖, in a general Banach space, and regardless of the convexity of φ1 leads to a Lipschitz regularization of φ1 (see Section 9.3.6). ◁ The inf-convolution enjoys some very interesting properties, one of which is that it is turned into a sum under the Legendre–Fenchel transform. In this respect it has a similar property as the standard convolution under the Fourier transform. In the following proposition, we collect some useful properties of the inf-convolution [13]. Proposition 5.3.4. The following properties hold for the inf-convolution: (i) epis (φ1 ◻φ2 ) = epis φ1 + epis φ2 ; hence, the alternative term epi sum. 12 The definition may still make sense even if the functions are not convex or lower semicontinuous but then the inf-convolution may not enjoy all the properties shown in this section.

208 | 5 Minimax theorems and duality (ii) epi φ1 + epi φ2 ⊂ epi (φ1 ◻φ2 ) with equality if and only if φ1 ◻φ2 is exact in dom (φ1 ◻φ2 ). (iii) φ1 ◻φ2 is also a convex function. (iv) (φ1 ◻φ2 )⋆ = φ⋆1 + φ⋆2 . (v) If int (domφ1 )∩domφ2 ≠ 0, then (φ1 +φ2 )⋆ = φ⋆1 ◻φ⋆2 and the inf-convolution φ⋆1 ◻φ⋆2 is exact.13 Proof. (i) Consider any pair (x, λ) ∈ X × ℝ such that (x, λ) ∈ epis (φ1 ◻φ2 ). This is equivalent to stating that (φ1 ◻φ2 )(x) = inf [φ1 (x1 ) + φ2 (x2 )] < λ, x1 +x2 =x x1 ,x2 ∈X

which in turn is equivalent to the existence of x1 , x2 ∈ X such that x1 + x2 = x and φ1 (x1 ) + φ2 (x2 ) < λ, which is equivalent to the existence of λ1 , λ2 ∈ ℝ such that φ1 (x1 ) ≤ λ1 and φ2 (x2 ) < λ2 while x1 + x2 = x and λ1 + λ2 = λ. As this is equivalent to the existence of (x1 , λ1 ) ∈ epis φ1 and (x2 , λ2 ) ∈ epis φ2 such that x1 + x2 = x and λ1 + λ2 = λ, the conclusion follows. (ii) The inclusion epi φ1 + epi φ2 ⊂ epi (φ1 ◻φ2 ) follows by the same reasoning as above. For the opposite inclusion we need the exactness14 of φ1 ◻φ2 . Consider any (x, λ) ∈ epi (φ1 ◻φ2 ), i. e., any (x, λ) with the property (φ1 ◻φ2 )(x) ≤ λ. Since φ1 ◻φ2 is exact, there exists zo ∈ X such that (φ1 ◻φ2 )(x) = φ1 (zo ) + φ2 (x − zo ). This provides the existence of λ1 , λ2 ∈ ℝ such that λ = λ1 + λ2 , and x1 = zo , x2 = x − zo , such that (x1 , λ1 ) ∈ epi φ1 and (x2 , λ2 ) ∈ epi φ2 , therefore, epi (φ1 ◻φ2 ) ⊂ epi φ1 + epi φ2 . Assume now that epi φ1 + epi φ2 = epi (φ1 ◻φ2 ). That means we can express any (x, λ) ∈ epi (φ1 ◻φ2 ) in terms of the pairs (zo , λo ) ∈ epi φ1 and (x − zo , λ − λo ) ∈ epi φ2 . Apply the above for λ = (φ1 ◻φ2 )(x). This yields φ1 (zo ) ≤ λo and φ2 (x−zo ) ≤ (φ1 ◻φ2 )(x)− λo , so that adding the two inequalities we obtain that φ1 (zo ) + φ2 (x − zo ) ≤ (φ1 ◻φ2 )(x). But recall that by definition (φ1 ◻φ2 )(x) = infz∈X (φ1 (z) + φ2 (x − z)), so that it must be that φ1 (zo ) + φ2 (x − zo ) = (φ1 ◻φ2 )(x); hence, φ1 ◻φ2 is exact. (iii) This is immediate from (i) by noting that epis φ1 and epis φ2 are convex sets. (iv) The proof follows by the definition. Indeed, setting φ = φ1 ◻φ2 , φ⋆ (x⋆ ) = sup(⟨x⋆ , x⟩ − φ(x)) = sup(⟨x⋆ , x⟩ − inf [φ1 (z) + φ2 (x − z)]) x∈X

x∈X

z∈X

= sup(⟨x , x⟩ + sup[−φ1 (z) − φ2 (x − z)]) = sup (⟨x⋆ , x⟩ − φ1 (z) − φ2 (x − z)) ⋆

x∈X

z∈X

x,z∈X

13 That is, as follows by a trivial restatement of Definition 5.3.1, there exist x⋆1 , x⋆2 ∈ X ⋆ such that x⋆1 + x⋆2 = x⋆ and (φ⋆1 ◻φ⋆2 )(x⋆ ) = φ⋆1 (x⋆1 ) + φ⋆2 (x⋆2 ). 14 Note the subtle difference between infz∈X (φ1 (z) + φ2 (x − z)) < λ and infz∈X (φ1 (z) + φ2 (x − z)) ≤ λ. The first one implies the existence of a zo ∈ X such that φ1 (zo ) + φ2 (x − zo ) < λ. On the other hand, if infz∈X (φ1 (z) + φ2 (x − z)) = λ, the existence of a point zo ∈ X such that φ1 (zo ) + φ2 (x − zo ) ≤ λ is not guaranteed, unless the infimum is attained, i. e., the inf-convolution is exact.

5.3 The inf-convolution

| 209

= sup (⟨x⋆ , x − z⟩ + ⟨x⋆ , z⟩ − φ1 (z) − φ2 (x − z)) x,z∈X

= sup(⟨x⋆ , z⟩ − φ1 (z) + sup(⟨x⋆ , x − z⟩ − φ2 (x − z))) z∈X

= sup(⟨x , z⟩ − φ1 (z) + ⋆

z∈X

x∈X φ⋆2 (x⋆ ))

= φ⋆1 (x⋆ ) + φ⋆2 (x⋆ ),

where we first performed the inner supremum operation, for fixed z, which yields φ⋆2 (x⋆ ) and then continued the calculation. (v) We have seen in (iv) that for the inf-convolution of two convex functions φ1 , φ2 it holds that (φ1 ◻φ2 )⋆ = φ⋆1 + φ⋆2 . Assume that φ1 , φ2 are lower semicontinuous. Consider now φ⋆1 and φ⋆2 in the place of φ1 and φ2 , and applying this result once more ⋆⋆ to the inf-convolution φ⋆1 ◻φ⋆2 we obtain that (φ⋆1 ◻φ⋆2 )⋆ = φ⋆⋆ 1 + φ2 = φ1 + φ2 , where for the last step we used the Fenchel–Moreau–Rockafellar Theorem 5.2.15. We now take once more the Legendre–Fenchel transform on the last formula to obtain that (φ⋆1 ◻φ⋆2 )⋆⋆ = (φ1 +φ2 )⋆ , thus providing a formula for the Legendre–Fenchel transform of the sum of two convex functions. Assume now that int (domφ1 ) ∩ domφ2 ≠ 0. Then, by the Moreau–Rockafellar Theorem 4.3.2, 𝜕(φ1 + φ2 ) = 𝜕φ1 + 𝜕φ2 . Take any x ∈ X and consider any x⋆ ∈ 𝜕(φ1 + φ2 )(x). Since 𝜕(φ1 + φ2 ) = 𝜕φ1 + 𝜕φ2 , it follows that x⋆ ∈ (𝜕φ1 (x) + 𝜕φ2 (x)) so that there exist x⋆1 ∈ 𝜕φ1 (x) and x⋆2 ∈ 𝜕φ2 (x) with the property x⋆ = x⋆1 + x⋆2 . By Proposition 5.2.18, since x⋆ ∈ 𝜕(φ1 + φ2 )(x) it holds that (φ1 + φ2 )(x) + (φ1 + φ2 )⋆ (x⋆ ) − ⟨x⋆ , x⟩ = 0.

(5.18)

Similarly, since x⋆i ∈ 𝜕φi (x), i = 1, 2, we have that φ1 (x) + φ⋆1 (x⋆1 ) − ⟨x⋆1 , x⟩ = 0,

φ2 (x) + φ⋆2 (x⋆2 ) − ⟨x⋆2 , x⟩ = 0, which upon addition and keeping in mind that x⋆1 + x⋆2 = x⋆ leads to φ1 (x) + φ2 (x) + φ⋆1 (x⋆1 ) + φ⋆2 (x⋆2 ) − ⟨x⋆ , x⟩ = 0, that upon rearrangement becomes φ⋆1 (x⋆1 ) + φ⋆2 (x⋆2 ) = ⟨x⋆ , x⟩ − (φ1 + φ2 )(x).

(5.19)

Combining (5.19) with (5.18), we obtain that φ⋆1 (x⋆1 ) + φ⋆2 (x⋆2 ) = (φ1 + φ2 )⋆ (x⋆ ).

(5.20)

This holds for any x ∈ X, x⋆ ∈ 𝜕(φ1 + φ2 )(x), x⋆1 ∈ 𝜕φ1 (x) and x⋆2 ∈ 𝜕φ2 (x), with the constraint x⋆1 + x⋆2 = x⋆ . Taking the infimum of the left-hand side of (5.20) over all x⋆1 ,

210 | 5 Minimax theorems and duality x⋆2 satisfying the above properties and recalling the definition of the inf-convolution, we have that (5.21)

(φ⋆1 ◻φ⋆2 )(x⋆ ) = (φ1 + φ2 )⋆ (x⋆ ).

There are two subtle points to be settled before closing the proof. The first one is that the inf-convolution is defined as the infimum over all x⋆1 , x⋆2 ∈ X ⋆ such that x⋆1 +x⋆2 = x⋆ and not only over x⋆i ∈ 𝜕φi (x) satisfying the extra constraint. The second one is that we would like (5.21) to hold for any x⋆ ∈ X ⋆ and not just for any x⋆ ∈ 𝜕(φ1 + φ2 )(x) where x ∈ X. These two points can be addressed recalling the density of the domain of the subdifferential of any convex function in its domain (see Theorem 4.5.3). The details are left to the reader. The differentiability properties of the inf-convolution are quite interesting. Proposition 5.3.5. If φ1 ◻φ2 is exact at x ∈ X, i. e., there exists zo ∈ X such that (φ1 ◻φ2 )(x) = φ1 (zo ) + φ2 (x − zo ). Then 𝜕(φ1 ◻φ2 )(x) = 𝜕φ1 (zo ) ∩ 𝜕φ1 (x − zo ). Proof. Since φ1 ◻φ2 is exact at x it is expressed as (φ1 ◻φ2 )(x) = φ1 (zo ) + φ2 (x − zo ) for some zo ∈ X, while by Proposition 5.3.4(iv), for any x⋆ ∈ X it holds that (φ1 ◻φ2 )⋆ (x⋆ ) = φ⋆1 (x⋆ ) + φ⋆2 (x⋆ ). Consider any x⋆ ∈ 𝜕(φ1 ◻φ2 )(x). By Proposition 5.2.18, this is equivalent to (φ1 ◻φ2 )(x) + (φ1 ◻φ2 )⋆ (x⋆ ) = ⟨x⋆ , x⟩, which upon using the above observations, is equivalent to φ1 (zo ) + φ2 (x − zo ) + φ⋆1 (x⋆ ) + φ⋆2 (x⋆ ) = ⟨x⋆ , x⟩, that can be reformulated as φ1 (zo ) + φ⋆1 (x⋆ ) + φ2 (x − zo ) + φ⋆2 (x⋆ ) = ⟨x⋆ , zo ⟩ + ⟨x⋆ , x − zo ⟩.

(5.22)

Young’s inequality implies that φ1 (z) + φ⋆1 (x⋆ ) ≥ ⟨x⋆ , z⟩,

φ2 (x − z) +

φ⋆2 (x⋆ )

∀ z ∈ X,

≥ ⟨x , x − z⟩, ⋆

(5.23)

∀ z ∈ X,

with the equality holding if and only if x⋆ ∈ 𝜕φ1 (z), and x⋆ ∈ 𝜕φ2 (x − z), respectively. Adding (5.23), we obtain φ1 (z) + φ2 (x − z) + φ⋆1 (x⋆ ) + φ⋆2 (x⋆ ) ≥ ⟨x⋆ , z⟩ + ⟨x⋆ , x − z⟩,

∀ z ∈ X,

and combining (5.22) with (5.24) we see that φ1 (zo ) + φ⋆1 (x⋆ ) ≥ ⟨x⋆ , zo ⟩, and φ2 (x − zo ) + φ⋆1 (x⋆ ) ≥ ⟨x⋆ , x − zo ⟩ therefore, x⋆ ∈ 𝜕φ1 (zo ) ∩ 𝜕φ2 (x − zo ).

(5.24)

5.3 The inf-convolution

| 211

Example 5.3.6 (Moreau–Yosida approximation of the support function in Hilbert space). Let X = H be a Hilbert space (identified with its dual), C ⊂ X a closed and convex subset and σC : X ⋆ → ℝ the support function of C defined by σC (x⋆ ) = supx∈C ⟨x⋆ , x⟩. Then the inf-convolution of σC with the function 21 ‖⋅‖2 , (i. e., the Moreau– Yosida approximation of σC ) satisfies 1 1 (σC ◻ ‖ ⋅ ‖2 )(x) = inf (σC (z) + ‖x − z‖2 ) = ⟨x, PC (x)⟩ − z∈X 2 2

1 󵄩󵄩 󵄩2 󵄩P (x)󵄩󵄩 , 2󵄩 C 󵄩

(5.25)

and is attained at zo := proxσC (x) = x − PC (x), where PC (x) is the orthogonal projection of x on the set C, and proxσC is the proximal operator of the support function C. To prove that, we first note that zo , being the minimizer of the function ψ(z) := σC (z) + 21 ‖x − z‖2 should satisfy the first-order condition 0 ∈ 𝜕ψ(zo ). This implies, by the Moreau–Rockafellar Theorem 4.3.2, that 1 0 ∈ 𝜕σC (zo ) + 𝜕( ‖x − z‖2 )(zo ) = 𝜕σC (zo ) + zo − x, 2 or equivalently that x− zo ∈ 𝜕σC (zo ). But by Example 5.2.23 (see (5.17)) this implies that (5.26)

zo ∈ NC (x − zo ),

where NC (x − zo ) is the normal cone of C at point x − zo . Recalling the definition of the normal cone (see Definition 4.1.5), we see that (5.26) is equivalent to the variational inequality, ⟨zo , z − (x − zo )⟩ ≤ 0,

∀ z ∈ C.

(5.27)

If we define the new variable z󸀠o := x − zo , we can express (5.27) as ⟨x − z󸀠o , z − z󸀠o ⟩ ≤ 0,

∀ z ∈ C.

(5.28)

Recall (see Theorem 2.5.1) that the orthogonal projection of x onto C, PC (x), solves the variational inequality ⟨x − PC (x), z − PC (x)⟩ ≤ 0,

∀ z ∈ C,

(5.29)

and it is unique. Comparing (5.28) and (5.29), we see that z󸀠o = PC (x), therefore, zo := proxσC (x) = x − PC (x) as required. Substituting zo , we see that 1 1 (σC ◻ ‖ ⋅ ‖2 )(x) = σC (zo ) + ‖x − zo ‖2 2 2 1󵄩 󵄩2 = σC (x − PC (x)) + 󵄩󵄩󵄩PC (x)󵄩󵄩󵄩 . 2

(5.30)

212 | 5 Minimax theorems and duality It remains to calculate σC (x − PC (x)). By definition, σC (x − PC (x)) = sup⟨x − PC (x), z⟩. z∈C

We rearrange (5.29) as ⟨x − PC (x), z⟩ ≤ ⟨x − PC (x), PC (x)⟩,

∀ z ∈ C,

and note that for z = PC (x) ∈ C the equality is attained. This means that σC (x − PC (x)) = sup⟨x − PC (x), z⟩ = ⟨x − PC (x), PC (x)⟩. z∈C

Substituting this result in (5.30), we obtain (5.25).

◁

Example 5.3.7 (Subdifferential of the Moreau–Yosida approximation of the support function in Hilbert space). In the context of Example 5.3.6 above, we show that 1 𝜕(σC ◻ ‖ ⋅ ‖2 )(x) = {PC (x)}, 2 where PC (x) is the projection of x on C. As we have seen in Example 5.3.6, this inf-convolution is exact and is attained at zo = x − PC (x). By Proposition 5.3.5, we have that 1 1 𝜕(σC ◻ ‖ ⋅ ‖2 )(x) = 𝜕σC (zo ) ∩ 𝜕 ‖ ⋅ ‖2 )(x − zo ) = 𝜕σC (zo ) ∩ {x − zo }. 2 2 Consider any xo ∈ 𝜕(σC ◻ 21 ‖ ⋅ ‖2 )(x). Then xo = x − zo = PC (x) and xo ∈ 𝜕σC (zo ). In Example 5.2.23, we have shown that xo ∈ 𝜕σC (zo ) if and only if zo ∈ NC (xo ), which is expressed as x − PC (x) ∈ NC (PC (x)). By the definition of the normal cone, this is equivalent to the variational inequality ⟨x − PC (x), z − PC (x)⟩ ≤ 0 for every z ∈ C, which is of course true (this is just the variational inequality characterizing the orthogonal projection on C; see (2.15)). From the above discussion, we conclude that any xo ∈ 𝜕(σC ◻ 21 ‖ ⋅ ‖2 )(x) is of the form xo = PC (x), so our claim follows. ◁

5.4 Duality and optimization: Fenchel duality An important application of the theory of the Fenchel–Legendre transform is in the theory of duality in optimization. Let us consider two Banach spaces X, Y, a bounded linear operator L : X → Y, two proper convex lower semicontinuous functions φ1 : X → ℝ ∪ {+∞}, φ2 : Y → ℝ ∪ {+∞}, and the optimization problem min(φ1 (x) + φ2 (Lx)). x∈X

5.4 Duality and optimization: Fenchel duality |

213

A large number of interesting real world applications (see examples in this section) can be redressed in this general form. Importantly, optimization problems with linear constraints can be brought in the form above using the appropriate convex indicator function in the place of φ2 to express the linear constraint. The fundamental observation leading to Fenchel duality is that, by Proposition 5.2.15, for a convex lower semicontinuous function φ, we have that φ⋆⋆ = φ. Then, using the definition of φ⋆⋆ , we have that inf (φ1 (x) + φ2 (Lx)) = inf (φ1 (x) + φ⋆⋆ 2 (Lx))

x∈X

x∈X

= inf (φ1 (x) + sup (⟨y⋆ , Lx⟩Y ⋆ ,Y − φ⋆2 (y⋆ )) x∈X

y⋆ ∈Y ⋆

= inf sup (φ1 (x) + ⟨L⋆ y⋆ , x⟩X ⋆ ,X − φ⋆2 (y⋆ )), x∈X y⋆ ∈Y ⋆

with the last problem being in a saddle point form. This version of the problem is interesting in its own right, leading to interesting numerical optimization algorithms, however, one may proceed a little further under the assumption that the order of the inf and sup in the above problem may be interchanged. Under sufficient conditions for interchanging the order of the inf and the sup, one may continue writing inf (φ1 (x) + φ2 (Lx)) = inf sup (φ1 (x) + ⟨L⋆ y⋆ , x⟩X ⋆ ,X − φ⋆2 (y⋆ ))

x∈X

x∈X y⋆ ∈Y ⋆

= sup inf (φ1 (x) + ⟨L⋆ y⋆ , x⟩X ⋆ ,X − φ⋆2 (y⋆ )) y⋆ ∈Y ⋆ x∈X

= sup (− sup(⟨−L⋆ y⋆ , x⟩X ⋆ ,X − φ1 (x)) + φ⋆2 (y⋆ )) y⋆ ∈Y ⋆

x∈X

= sup (−φ⋆1 (−Ly⋆ ) − φ⋆2 (y⋆ )) = sup (−φ⋆1 (Ly⋆ ) − φ⋆2 (−y⋆ )), y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

where we have used the definition of φ⋆1 and the vector space structure of Y ⋆ . If the minimax step can be justified, then there may be a connection between the optimization problem infx∈X (φ1 (x) + φ2 (Lx)) (called the primal problem) and the optimization problem supy⋆ ∈Y ⋆ (−φ⋆1 (Ly⋆ )−φ⋆2 (−y⋆ )) (called the dual problem). It may happen in certain cases of interest that the dual problem is easier to handle both analytically and numerically (for instance, if φ2 is strongly convex then φ⋆2 is C 1 with Lipschitz gradient). Thus, being able to guarantee a solution to the dual problem which allows us to obtain a solution to the primal one is certainly an appealing prospect. The following proposition (see, e. g., [25]) provides a rigorous answer to the above considerations. Proposition 5.4.1 (Fenchel duality). Let φ1 : X → ℝ ∪ {+∞}, φ2 : Y → ℝ ∪ {+∞} be proper convex lower semicontinuous functions, L : X → Y a continuous linear operator, and consider the optimization problems: P := inf (φ1 (x) + φ2 (Lx)) (Primal), x∈X

214 | 5 Minimax theorems and duality and D := sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )) (Dual). y⋆ ∈Y ⋆

Then, in general, P := inf (φ1 (x) + φ2 (Lx)) ≥ D := sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )), x∈X

y⋆ ∈Y ⋆

(weak duality).

If, moreover, (5.31)

0 ∈ core( dom φ2 − L dom φ1 ), then P := inf (φ1 (x) + φ2 (Lx)) = D := sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )), x∈X

y⋆ ∈Y ⋆

(strong duality), (5.32)

and if the supremum D is finite, it is attained so that the dual problem admits a solution y⋆o . In this case, the point xo ∈ X is optimal for the primal problem if and only if the first-order conditions hold L⋆ y⋆o ∈ 𝜕φ1 (xo ),

(5.33)

−y⋆o ∈ 𝜕φ2 (Lxo ), where y⋆o ∈ Y ⋆ is optimal for the dual problem.

Proof. The proof proceeds in 4 steps: 1. Weak duality is a consequence of the Young–Fenchel inequality (5.8). According to that, using the notation ⟨ , ⟩, and ⟨ , ⟩Y ⋆ ,Y for the duality pairings, we have φ1 (z) + φ⋆1 (z⋆ ) ≥ ⟨z⋆ , z⟩,

∀ (z, z⋆ ) ∈ X × X ⋆ ,

φ2 (y)̄ + φ⋆2 (ȳ⋆ ) ≥ ⟨ȳ⋆ , y⟩̄ Y ⋆ ,Y ,

∀ (y,̄ ȳ⋆ ) ∈ Y × Y ⋆ .

Pick any (x, y⋆ ) ∈ X × Y ⋆ , and set (z, z⋆ ) = (x, L⋆ y⋆ ) in the first inequality and (y,̄ ȳ⋆ ) = (Lx, −y⋆ ) in the second. This yields φ1 (x) + φ⋆1 (L⋆ y⋆ ) ≥ ⟨L⋆ y⋆ , x⟩,

∀ (x, y⋆ ) ∈ X × Y ⋆ ,

φ2 (Lx) + φ⋆2 (−y⋆ ) ≥ ⟨−y⋆ , Lx⟩Y ⋆ ,Y = −⟨L⋆ y⋆ , x⟩,

∀ (x, y⋆ ) ∈ X × Y ⋆ .

By adding the two and rearranging terms, we obtain that φ1 (x) + φ2 (Lx) ≥ −φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ ),

∀ (x, y⋆ ) ∈ X × Y ⋆ ,

and taking the infimum over all x ∈ X on the left-hand side and the supremum over all y⋆ ∈ Y ⋆ on the right-hand side yields the required weak duality result.

215

5.4 Duality and optimization: Fenchel duality |

2. Consider now the function φ : Y → ℝ, defined by φ(y) = infx∈X (φ1 (x) + φ2 (Lx + y)). Clearly, φ(0) = infx∈X (φ1 (x) + φ2 (Lx)) = P, the value of the primal problem. This function is a convex function. We claim that if condition (5.31) holds (see also Remark 5.4.2), then (5.34)

φ is continuous at 0.

Let us proceed by accepting the validity of this point (which is proved in step 4). By the continuity of φ at 0 (claim (5.34)) 𝜕φ(0) ≠ 0. Let −y⋆o ∈ 𝜕φ(0). By the definition of the subdifferential, we have that φ(y) − φ(0) ≥ ⟨−y⋆o , y − 0⟩Y ⋆ ,Y which yields φ(0) ≤ φ(y) + ⟨y⋆o , y⟩Y ⋆ ,Y

≤ φ1 (x) + φ2 (Lx + y) + ⟨y⋆o , y⟩Y ⋆ ,Y

= (φ1 (x) − ⟨y⋆o , Lx⟩Y ⋆ ,Y ) + (φ2 (Lx + y) − ⟨−y⋆o , Lx + y⟩Y ⋆ ,Y ) = (φ1 (x) − ⟨L⋆ y⋆o , x⟩) + (φ2 (Lx + y) − ⟨−y⋆o , Lx + y⟩Y ⋆ ,Y ),

(5.35) ∀ (x, y) ∈ X × Y.

where in the first line we used the definition of φ as an infimum to enhance the second inequality. We first rewrite (5.35) as ⟨−y⋆o , Lx + y⟩Y ⋆ ,Y − φ2 (Lx + y) ≤ (φ1 (x) − ⟨L⋆ y⋆o , x⟩) − φ(0),

∀ (x, y) ∈ X × Y,

where by keeping x ∈ X fixed and taking the supremum over y ∈ Y we obtain that φ⋆2 (−y⋆o ) ≤ (φ1 (x) − ⟨L⋆ y⋆o , x⟩) − φ(0),

∀ x ∈ X,

and rearranging once more as ⟨L⋆ y⋆o , x⟩ − φ1 (x) ≤ −φ⋆2 (−y⋆o ) − φ(0),

∀ x ∈ X,

and taking the supremum over all x ∈ X yields (after a final rearrangement) φ(0) ≤ −φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o ). Clearly, P = φ(0) ≤ −φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o ) ≤ sup (−φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o )) = D y⋆ ∈Y ⋆

≤ inf (φ1 (x) + φ2 (Lx)) = φ(0) = P,

(5.36)

x∈X

where we used the weak duality result, D ≤ P. Inequality (5.36) implies that −φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o ) = sup (−φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o )) = inf (φ1 (x) + φ2 (Lx)), y⋆ ∈Y ⋆

x∈X

i. e., the supremum in the dual problem (if it is finite) is attained at y⋆o and the strong duality, P = D, holds.

216 | 5 Minimax theorems and duality 3. We now prove the validity of the optimality conditions (5.33). Recalling Proposition 5.2.18, condition (5.33) implies that φ1 (xo ) + φ⋆1 (L⋆ y⋆o ) = ⟨L⋆ y⋆o , xo ⟩,

φ2 (Lxo ) + φ⋆2 (−y⋆o ) = ⟨−y⋆o , Lxo ⟩Y ⋆ ,Y = −⟨L⋆ y⋆o , xo ⟩, so that adding and rearranging we have that φ1 (xo ) + φ2 (Lxo ) = −φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o ).

(5.37)

But since y⋆o is the solution of the dual problem, − φ⋆1 (L⋆ y⋆o ) − φ⋆2 (−y⋆o )

= D = sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )) = max (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )), ⋆ ⋆ y ∈Y

y⋆ ∈Y ⋆

and by the strong duality it is also true that P = inf (φ1 (x) + φ2 (Lx)) x∈X

= D = sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )) = max (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )). ⋆ ⋆ y ∈Y

y⋆ ∈Y ⋆

Combining the above with (5.37), we see that φ1 (xo ) + φ2 (Lxo ) = inf (φ1 (x) + φ2 (Lx)); x∈X

hence, xo is a solution of the primal problem. 4. It only remains to verify our claim (5.34), i. e., that under condition (5.31) (see also Remark 5.4.2) the function φ : Y → ℝ, defined by φ(y) = infx∈X (φ1 (x) + φ2 (Lx + y)) is continuous at 0. Assume that 0 ∈ core(domφ2 − L domφ1 ) holds, as stated in the assumptions of the theorem. Assuming moreover without loss of generality and to ease notation that15 φ1 (0) = φ2 (0) = 0, we define the set C = {y ∈ Y : ∃x ∈ BX (0, 1) s. t. φ1 (x) + φ2 (Lx + y) ≤ 1}. It is easy to check this is a convex set. Our strategy is to show that Y = ⋃λ>0 λC so that 0 ∈ core(C), while C is CS-closed, therefore, by Proposition 1.2.7, core(C) = int( C) = int( C); hence, since 0 ∈ core(C) it also holds that 0 ∈ int( C). Since by the definition of C, this implies that φ is bounded above by 1 for all y in a neighborhood of 0, Proposition 2.3.20 provides the continuity of φ at 0. 15 Otherwise, we simply work with φ̄ i (z) = φi (z) − φi (0).

5.4 Duality and optimization: Fenchel duality |

217

We first show that Y = ⋃λ∈ℝ λC, i. e., that C is absorbing. Since 0 ∈ core(domφ2 − L domφ1 ), recalling the definition of the core (see Definition 1.2.5) we have that for every y ∈ Y there exists δ > 0 such that δy ∈ domφ2 − L domφ1 , which means that we may choose an x̄ ∈ domφ1 such that Lx̄ + δy ∈ domφ2 , which in turn implies that the sum φ1 (x)̄ + φ2 (Lx̄ + δy) is finite, i. e., there exists some c ∈ ℝ such that (5.38)

φ1 (x)̄ + φ2 (Lx̄ + δy) = c.

For any k > 1, we may write φ1 ( k1 x)̄ = φ1 ( k1 x̄ + (1 − k1 )0) ≤ k1 φ1 (x)̄ + (1 − k1 )φ1 (0) (by convexity) and similarly for φ2 ( k1 (Lx̄ + δy)) ≤ k1 φ2 (Lx̄ + δy) + (1 − k1 )φ2 (0), so that using the assumption that φ1 (0) = φ2 (0) = 0, and adding the above we get that 1 1 c 1 ̄ + φ2 ( (Lx̄ + δy)) ≤ (φ1 (x)̄ + φ2 (Lx̄ + δy)) ≤ . φ1 ( x) k k k k If kc ≤ 1 and k1 ‖x‖̄ X ≤ 1, then the above relation implies that there exists some x = k1 x̄ ∈ BX (0, 1), such that y󸀠 := kδ y satisfies φ1 (x) + φ2 (Lx + y󸀠 ) ≤ 1, i. e., y󸀠 := kδ y ∈ C. Such a k can always be chosen as k = max(‖x‖̄ X , |c|, 1). We conclude that for any y ∈ Y, there exists a kδ y ∈ C for the appropriate choice of k > 0, so that Y ⊂ ⋃λ>0 λC and since trivially ⋃λ>0 λC ⊂ Y, it follows that Y = ⋃λ>0 λC, i. e., C is absorbing. The absorbing property of C implies that 0 ∈ core(C). We now show that C is CS-closed. Consider any sequence {yn : n ∈ ℕ} ∈ C, such ∞ that ∑∞ n=1 λn yn = y, λn ≥ 0, ∑n=1 λn = 1. We intend to show that y ∈ C. Since yn ∈ C, for every n ∈ ℕ, there exists a xn ∈ BX (0, 1) such that (5.39)

φ1 (xn ) + φ2 (Lxn + yn ) ≤ 1. m m Rewrite the partial sum in the series ∑∞ n=1 λn yn = y as ȳ m = ∑n=1 λn yn = Λm (∑n=1 m where Λm = ∑m n=1 λn , and consider the sequence x̄ m = ∑n=1

λn x . Λm n

λn y ), Λm n

This sequence is

Cauchy and converges to some x ∈ BX (0, 1). Multiplying (5.39) for each n = 1, . . . , m, λ with Λn and adding over all such n, using also the convexity of φ1 and φ2 we have that m

φ1 (x̄m ) + φ2 (Lx̄m + ȳm ) ≤ 1, and passing to the limit as m → ∞ and using the lower semicontinuity of φ1 and φ2 , we conclude that φ1 (x) + φ2 (Lx + y) ≤ 1; hence, y ∈ C. The proof is complete. Remark 5.4.2. One may replace condition (5.31) in Proposition 5.4.1 by the alternative condition L dom φ1 ∩ cont(φ2 ) ≠ 0,

218 | 5 Minimax theorems and duality where cont(φ2 ) is the set of points of continuity of φ2 , in which case lower semicontinuity for φ1 , φ2 is not required (see [25] or [26]). To see this, we only need to modify step 4 of the above proof, by using the following reasoning. If L dom φ1 ∩ cont φ2 ≠ 0, then we may choose some ȳ ∈ L dom φ1 ∩ cont φ2 . Since ȳ is a point of continuity for φ2 , then for every ϵ > 0 there exists δ > 0 such that for every y󸀠 ∈ δBY (0, 1), φ2 (ȳ + y󸀠 ) ≤ φ2 (y)̄ + ϵ. Since ȳ ∈ L dom φ1 as well, there exists x ∈ dom φ1 , such that ȳ = Lx. Using that in the estimate we have derived from continuity of φ2 , we conclude that for every y󸀠 ∈ δBY (0, 1), φ2 (Lx + y󸀠 ) ≤ φ2 (Lx) + ϵ, which by the boundedness of φ1 (x) (since x ∈ domφ1 ) leads to the result that φ is bounded above in a neighborhood of 0; hence, by Proposition 2.3.20 continuous at 0. An important remark here is that when strong duality holds and the dual problem has a solution y⋆o ∈ Y ⋆ , we can try to construct a solution to the dual problem by trying a solution xo as indicated by (5.33). If an xo ∈ X satisfying (5.33) exists, then we know that this must be a solution to the primal problem. However, there is no guarantee in general that such a xo ∈ X exists. This requires extra conditions on the functions φ⋆1 and φ⋆2 and the mapping L that will guarantee solvability of (5.33) (as an inclusion for the unknown xo ); hence, the existence of a xo ∈ X which is a solution of the primal problem. The Fenchel duality is an important result with many implications and applications. An interesting application of Fenchel duality is that it may be used as an alternative proof of the subdifferential sum rule (see, e. g., [19]). Example 5.4.3 (The subdifferential sum rule as a result of Fenchel duality). Let φ1 , φ2 be two convex and lower semicontinuous functions, and consider a point x ∈ int(dom(φ1 )) ∩ dom(φ2 ). Then 𝜕(φ1 + φ2 )(x) = 𝜕φ1 (x) + 𝜕φ2 (x). Consider any x⋆ ∈ X ⋆ . We will show the existence of a x⋆o ∈ X ⋆ , such that (φ1 + φ2 )⋆ (x⋆ ) = φ⋆1 (x⋆ − x⋆o ) + φ2 (x⋆o ).

(5.40)

Once the existence of such a point x⋆o is established, then if x⋆ ∈ 𝜕(φ1 + φ2 )(x) by Proposition 5.2.18 we have that (φ1 + φ2 )(x) + (φ1 + φ2 )⋆ (x⋆ ) = ⟨x⋆ , x⟩, which using (5.40) yields φ1 (x) + φ2 (x) + φ⋆1 (x⋆ − x⋆o ) + φ2 (x⋆o ) = ⟨x⋆ − x⋆o , x⟩ + ⟨x⋆o , x⟩. Using Proposition 5.2.18 again for φ1 , φ⋆1 and φ2 , φ⋆2 , respectively, we conclude that x⋆ − x⋆o ∈ 𝜕φ1 (x) and x⋆o ∈ 𝜕φ2 (x); hence, x⋆ ∈ 𝜕φ1 (x) + 𝜕φ2 (x). By the arbitrariness of x⋆ , we conclude that 𝜕(φ1 + φ2 )(x) ⊂ 𝜕φ1 (x) + 𝜕φ2 (x), and since the reverse inclusion is always true, the equality holds.

219

5.4 Duality and optimization: Fenchel duality |

̂1 , The Fenchel duality can be used to show the claim (5.40). Define the functions φ ̂2 , by φ ̂2 (x) = φ2 (x) and φ ̂1 (x) = −⟨x⋆ , x⟩ + φ1 (x). An easy computation shows that φ ̂⋆2 (z⋆ ) = φ⋆2 (z⋆ ) and φ ̂⋆1 (z⋆ ) = φ⋆1 (x⋆ + z⋆ ). Applying the for any z⋆ ∈ X ⋆ , we have φ ̂1 + φ ̂2 (note that condition (5.31) holds) and observing that Fenchel duality result for φ ̂1 (x) + φ ̂2 (x)) = −(φ1 + φ2 )⋆ (x⋆ ), we obtain the claim (5.40), for x⋆o = −z⋆o , with infx∈X (φ ̂2 ⋆ (z⋆ ) − φ ̂1 (−z⋆ ) = −φ⋆1 (x⋆ + z⋆ ) − φ⋆2 (−z⋆ ). z⋆o being a maximizer of −φ ◁ Example 5.4.4. Let X be a Banach space and consider the minimization problem inf φ(x), where C ⊂ X, closed and convex.

x∈C

(5.41)

Problem (5.41) can be brought into the general form of Fenchel duality in two different ways. (a) Upon defining φ1 = φ and φ2 = IC , problem (5.41) can be expressed as infx∈X (φ1 (x) + φ2 (x)). Using the Fenchel duality theorem in the special case where L = I, we see that the dual problem assumes the form supx⋆ ∈X ⋆ (−φ⋆1 (x⋆ ) − σC (−x⋆ )), where σC is the support function of the set C defined as σC = supx∈C (⟨x⋆ , x⟩) (see Example 5.2.7). (b) Upon defining φ1 = φ + IC and φ2 = 0. We also choose X = Y and L = I. Then the dual problem is supx⋆ ∈X ⋆ (−φ⋆1 (x⋆ )). We must now calculate the function φ⋆1 . One way forward would be to recall that under appropriate assumptions (see Proposition 5.3.4(v)), the Legendre–Fenchel transform of the sum can be expressed as the inf-convolution of the Legendre–Fenchel transforms. Under such conditions, (φ + IC )⋆ = φ⋆ ◻ IC⋆ = φ⋆ ◻σC , where σC is the support function of C. The dual problem thus becomes supx⋆ ∈X ⋆ (−(φ⋆ ◻σC )(x⋆ )). There are certain cases, depending either on the choice of C or the choice of φ, where this inf-convolution can be explicitly calculated, leading to a well-defined and manageable dual problem. Then under strong duality, once a solution to the dual problem has been obtained using (5.33) a solution to the primal problem can be found. Two explicit such examples will follow. ◁ Example 5.4.5 (Linear programming). Let X = ℝn , Y = ℝm and L : X → Y a bounded linear map which in this setting can be understood as a m × n real matrix A ∈ 𝕄m×n and L⋆ = AT , the transpose of A. Clearly, in this finite dimensional setting we will use the identifications16 X ⋆ ≃ X ≃ ℝn and Y ⋆ ≃ Y ≃ ℝm . Given two vectors, c ∈ ℝn , b ∈ ℝm , the canonical form of a linear programming problem is inf ⟨c, x⟩,

x∈X≃ℝn

subject to

Ax ≥ b,

x ≥ 0,

(5.42)

where the notation x ≥ 0 means that every component of the vector x satisfies the inequality. We will apply the Fenchel duality theory to this problem and show that the 16 But retain for convenience of the reader the notation x, y for their elements instead of x, y.

220 | 5 Minimax theorems and duality dual form of (5.42) is sup ⟨y, b⟩,

y∈Y≃ℝm

subject to AT y ≤ c,

y ≥ 0.

(5.43)

To bring the primal problem to the standard form for which the Fenchel duality approach can be applied, we define the convex sets C1 = {x ∈ ℝn : x ≥ 0},

C2 (b) = {y ∈ ℝm : y ≥ b},

and,

C3 = {c},

and the convex functions φ1 : ℝn → ℝ, m

φ2 : ℝ → ℝ,

φ1 (x) = ⟨c, x⟩ + IC1 (x), φ2 (y) = IC2 (b) (y).

With the use of these two functions the primal problem (5.42) can be expressed as infx∈ℝn (φ1 (x) + φ2 (Ax)). According to the general theory of the Fenchel duality, the dual problem will be of the form17 supy∈ℝm (−φ⋆1 (AT y) − φ⋆2 (−y)). To calculate φ⋆1 , we note that φ1 = φ + ψ, where φ(x) = ⟨c, x⟩ and ψ(x) = IC1 (x). Under the conditions of Proposition 5.3.4(v), φ⋆1 = φ⋆ ◻ψ⋆ = φ⋆ ◻σC1 , where we used the result that the Legendre–Fenchel transform of the indicator function of a set is the support function (see Example 5.2.7). In order to calculate φ⋆1 , note that for any x⋆ ∈ ℝn , φ⋆ (x⋆ ) = sup (⟨x⋆ , x⟩ − ⟨c, x⟩) = sup ⟨x⋆ − c, x⟩ = IC3 (x⋆ ), x∈ℝn

x∈ℝn

the indicator function of the set C3 = {c}. Introducing that into the above, we obtain φ⋆1 (x⋆ ) = (IC3 ◻σC1 )(x⋆ ) = infn (IC3 (z) + σC1 (x⋆ − z)) = σC1 (x⋆ − c). z∈ℝ

However, because of the special form of the set C1 , we can take our calculation a little further. By definition, +∞ if x⋆ − c > 0

σC1 (x⋆ − c) = sup⟨x⋆ − c, x⟩ = sup⟨x⋆ − c, x⟩ = { x∈C1

x≥0

0

if x − c ≤ 0 ⋆

= IC1 (c − x⋆ ).

We therefore conclude that φ⋆1 (x⋆ ) = IC1 (c − x⋆ ). We can similarly see that φ⋆2 (y⋆ ) = σC2 (b) (y⋆ ) = sup ⟨y⋆ , y⟩ = sup⟨y⋆ , y⟩ y∈C2 (b)

y≥b

17 Where by the choice of the spaces involved and to simplify notation, we use y in the place of y⋆ .

5.4 Duality and optimization: Fenchel duality |

+∞

= sup⟨y⋆ , y − b⟩ + ⟨y⋆ , b⟩ = {

221

if y⋆ > 0

⟨y⋆ , b⟩ if y⋆ ≤ 0,

y≥b

which can be conveniently rephrased as φ⋆2 (y⋆ ) = ⟨y⋆ , b⟩ + IC2 (0) (−y⋆ ). The Fenchel dual problem is therefore supy⋆ ∈ℝm (−IC1 (c−AT y⋆ )+⟨y⋆ , b⟩−IC2 (0) (y⋆ )), which (setting y⋆ to y for simplicity) coincides with problem (5.43). ◁ Example 5.4.6 (Minimization of convex functions with linear constraints ([24])). Another general class of problems in which the Fenchel dual problem can be explicitly expressed are problems of the form inf

x∈C⊂X

1 2 ‖x‖ 2

subject to

Lx = c,

(5.44)

where X = H, Y are Hilbert spaces identified with their duals, (with Y possibly finite dimensional), C ⊂ X is closed and convex, L : X → Y is a bounded linear operator and c ∈ Y is known. Problems of this type arise quite often, e. g., in signal processing, where x represents typically a signal (modelled as an element of some Hilbert space X) which has to be matched to some measurements c for this signal. The operator L is the so-called measurement operator, and it is often the case that the space Y is a finite dimensional space. For example, the operator L may model the situation of taking a finite number of measurements from the function x. As a concrete example, one may consider X as an L2 space. Then, by the Riesz representation, any set of N continuous measurement operators {Mi : X = H → ℝ : i = 1, . . . , N}, may be represented in terms of the inner product with N elements zi ∈ X = H, as Mi (x) = ⟨zi , x⟩H for every i = 1, . . . N. Hence, given a set of measurements Mi (x) = yi , i = 1, . . . , N collected in the vector c = (c1 , . . . , cN )T ∈ Y ≃ ℝN , we may define the operator L : X = H → Y ≃ ℝN by Lx = (M1 (x), . . . , MN (x))T , with its adjoint L⋆ : ℝN → H defined by L⋆ y = ∑Ni=1 yi zi , for every y = (y1 , . . . , yN )T ∈ Y ≃ ℝN . Then the (inverse) problem of recovering a signal x, as close as possible (in the sense of minimal norm) to a reference signal xref (taken as xref = 0 without loss of generality) compatible with the given measurements Lx = c, as well some additional convex constraints (e. g., positivity, etc.) described by the set C is exactly of the general form (5.44). To apply Fenchel duality, we rewrite the primal problem in the form infx∈X (φ1 (x) + φ2 (Lx)), where φ1 (x) = 21 ‖x‖2 + IC (x), and φ2 (x) = IC1 (x), where C1 = {c} (a convex set). The Fenchel dual is then supy⋆ ∈Y ⋆ (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )). Note that since Y is a finite dimensional space, then the dual problem is a finite dimensional optimization problem which is typically a lot easier to handle than the primal infinite dimensional problem. In order to calculate φ⋆1 and its subdifferential, we proceed as in Example 5.2.21 to find that φ⋆1 (x⋆ ) = ⟨x⋆ , PC (x⋆ )⟩ − 21 ‖PC (x⋆ )‖2 , and 𝜕φ⋆1 (x⋆ ) = PC (x⋆ ) (i. e., it is a singleton). We can also calculate explicitly φ⋆2 (y⋆ ) = supy∈Y (⟨y⋆ , y⟩ − IC1 (y)) = ⟨y⋆ , c⟩ so that

222 | 5 Minimax theorems and duality 𝜕φ⋆2 (y⋆ ) = c (also a singleton). We therefore conclude that the Fenchel dual problem is 1󵄩 󵄩2 sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )) = sup (−⟨L⋆ y⋆ , PC (L⋆ y⋆ )⟩ + 󵄩󵄩󵄩PC (L⋆ y⋆ )󵄩󵄩󵄩 + ⟨y⋆ , c⟩). ⋆ ⋆ ⋆ ⋆ 2 y ∈Y y ∈Y The condition for strong duality to hold is 0 ∈ core(c − Ldom φ1 ) or equivalently c ∈ core(Ldom φ1 ). Under this assumption, the first-order condition for a solution of the dual problem y⋆o ∈ Y ⋆ is 0 ∈ 𝜕(φ⋆1 (L⋆ y⋆o ) + φ⋆2 (−y⋆o )), which using subdifferential calculus yields 0 ∈ L𝜕φ1 (L⋆ y⋆o ) − c, and by the results stated above we have c = LPC (L⋆ y⋆o ).

(5.45)

The solution of the primal problem xo , which is what we are really after, is connected to the solution of the dual problem through the inclusions L⋆ y⋆o ∈ 𝜕φ1 (xo ), −y⋆o ∈ 𝜕φ2 (Lxo ).

(5.46)

We can easily calculate 𝜕φ1 (xo ) = xo + NC (xo ), so the first of (5.46) gives that L⋆ y⋆o − xo ∈ NC (x), which is equivalent to the variational inequality ⟨L⋆ y⋆o − xo , z − xo ⟩ ≤ 0 for every z ∈ C, from which follows that xo = PC (L⋆ y⋆o ).

(5.47)

Hence, once we have found the solution to the dual problem y⋆o , the solution to the primal problem can be represented in terms of (5.47). Upon combining (5.47) and (5.45), we conclude that given a y⋆o ∈ Y ⋆ satisfying c = LPC (L⋆ y⋆o ), the solution of the primal problem can be expressed as xo = PC (L⋆ y⋆o ). One can verify in a straightforward manner that this is indeed the unique solution of the primal problem (see Theorem 1, in [91]). This parametrization of the solution in terms of the solution y⋆o of the nonlinear system of equations c = LPC (L⋆ y⋆o ), is very useful both from the theoretical as well as from the computational point of view. This system requires information on the range of the nonlinear operator A := LPC L⋆ : Y ⋆ → Y, which satisfies the property ⟨y⋆1 − y⋆2 , A(y⋆1 ) − A(y⋆2 )⟩ ≥ 0 for every y⋆1 , y⋆2 ∈ Y ⋆ . Many of the useful properties regarding the solvability of this system stem from this property (called monotonicity). We shall return to this concept in Chapter 9. ◁

5.5 Minimax and convex duality methods The minimax theory, introduced in Section 5.1, is closely related to convex duality methods in optimization such as for instance Fenchel duality. In fact, in the introduction of Section 5.4 the role of the minimax theorem was highlighted in the transition

5.5 Minimax and convex duality methods | 223

from the primal to the dual problem, even though it was not explicitly mentioned in the proof of the Fenchel duality theorem. Looking at the proof of this result, we note that a key role was played by a function φ : Y ⋆ → ℝ ∪ {∞}, related to a perturbed minimization problem of the form φ(y) = infx∈X (φ1 (x) + φ2 (Lx + y)), which coincided with the value of the primal problem for y = 0. We have shown that 𝜕φ(0) ≠ 0 and concluded that any element −y⋆o ∈ 𝜕φ(0), is a maximizer for the dual problem and, furthermore, D = P. Note that −y⋆o ∈ 𝜕φ(0) is equivalent to 0 ∈ 𝜕φ⋆ (−y⋆o ), and as a result of that −y⋆o must be a minimizer of φ⋆ , which implies that the dual problem can be expressed in terms of the Fenchel dual of the perturbation function φ of the primal problem. These considerations were not fully addressed in Section 5.4, although they may provide an equivalent, if not more general, view on the role of duality in optimization theory, which builds upon this idea of treating a general parametric perturbation of the original (primal) problem and involving the minimax approach to determine a related dual problem. In this section, we consider the connection between the minimax theory and convex duality methods in optimization, once more, by presenting a general framework which as a special case encompasses also Fenchel duality (see, e. g., [13] or [62]).

5.5.1 A general framework To aim for more generality than in Section 5.4, let us consider the general optimization problem inf Fo (x),

x∈X

(Primal)

where X is a Banach space and Fo : X → ℝ ∪ {+∞} is a given function. Motivated by the treatment of Fenchel duality in Section 5.4, the key idea in this framework is to embed the original problem minx∈X Fo (x) to a whole family of parametric problems by considering a parametric family of perturbations of the original problem F : X × Y → ℝ ∪ {+∞} where Y is an appropriate Banach space, which can be thought of as a parameter space and F is chosen such that F(x, 0) = Fo (x) for every x ∈ X. As a particular example, one may have in mind the case where Fo (x) = φ1 (x) + φ2 (Lx) so that F(x, y) = φ1 (x) + φ2 (Lx + y) is an appropriate choice, which corresponds to the perturbation chosen in the study of Fenchel duality. In terms of the perturbation function, we may express the primal problem as inf Fo (x) = inf F(x, 0),

x∈X

x∈X

(Primal)

(5.48)

We may now define the perturbed optimization problem inf F(x, y),

x∈X

y ∈ Y,

(5.49)

224 | 5 Minimax theorems and duality and the corresponding value function V : Y → ℝ by V(y) := inf F(x, y), x∈X

y ∈ Y,

(5.50)

with V(0) =: P being the value of the primal problem. Again, in the case considered in Section 5.4, V(y) = φ(y) = infx∈X (φ1 (x) + φ2 (Lx + y)). It turns out that the parametric family of problems, has some remarkable structure: We shall see that the Fenchel conjugates F ⋆ : X ⋆ × Y ⋆ → ℝ ∪ {+∞}, V ⋆ : Y ⋆ → ℝ ∪ {+∞}, of the perturbation function F and the value function V, can be used to define another class of optimization problems, the so-called dual problems sup (−F ⋆ (0, y⋆ )),

y⋆ ∈Y ⋆

(Dual),

the value D of which is related to the value P of the original (primal) problem through the relations V(0) = P and V ⋆⋆ (0) = D. It is important to mention that often the dual problems are much better behaved that the original problem. Furthermore, minimax theorems can be used to connect the value functions of the original and the dual problems, and through the solution of the dual problem, one can then pass to the solution of the original problem in a very elegant manner. The theory of convex conjugates plays a very important role in this task, since a careful use of this theory allows one to show that as long as the value function is continuous at 0, V ⋆⋆ (0) = V(0), leading to P = D (strong duality), whereas an element y⋆o of 𝜕V(0), is a maximizer for the dual problem. In the case of strong duality the maximizer y⋆o of the dual problem is connected with, and may be used to construct, the minimizer xo of the primal problem. Moreover, an interpretation of the pair (xo , y⋆o ) as the saddle point of the function L : X × Y ⋆ → ℝ ∪ {+∞}, defined as L(x, y⋆ ) = − sup(⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y)), y∈Y

(5.51)

is available. The connection with the minimax theorem becomes more apparent by noting that for L defined as above F(x, 0) = sup L(x, y⋆ ), y⋆ ∈Y ⋆

and

− F ⋆ (0, y⋆ ) = inf L(x, y⋆ ), x∈X

so that strong duality can be expressed in minimax form as inf sup L(x, y⋆ ) = sup inf L(x, y⋆ ).

x∈X y⋆ ∈Y ⋆

y⋆ ∈Y ⋆ x∈X

Finally, we should note that as the choice of parametric families F in which the original problem is embedded is not unique, the general theory we are about to present is very flexible and may contain as special cases Fenchel duality or Lagrange duality. In building up the theory summarized above, let us first consider some elementary properties of the parametric family of minimization problems (5.49) and the value function V.

5.5 Minimax and convex duality methods | 225

Lemma 5.5.1 (Properties of the value function and the Lagrangian). Consider a function F : X × Y → ℝ ∪ {+∞}, such that F(x, 0) = Fo (x), for every x ∈ X, the primal problem infx∈X Fo (x) = infx∈X F(x, 0), the value function V : Y → ℝ ∪ {+∞} defined as V(y) := infx∈X F(x, y), as well as its conjugates V ⋆ : Y ⋆ → ℝ ∪ {+∞} and V ⋆⋆ : Y → ℝ ∪ {+∞}. Define further, the Lagrangian function L : X × Y ⋆ → ℝ ∪ {+∞} by L(x, y⋆ ) = − sup(⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y) ). y∈Y

Then the following hold: (i) If F is a convex function (in both variables), then the value function V is convex while (regardless of the convexity of F), the conjugate functions F ⋆ , V ⋆ , V ⋆⋆ are connected by V ⋆ (y⋆ ) = F ⋆ (0, y⋆ ), V

⋆⋆

∀ y⋆ ∈ Y ⋆ ,

(0) = sup (−F (0, y )) = D ⋆

⋆

y⋆ ∈Y ⋆

and (Dual).

(5.52)

(ii) If F is a proper convex lower semicontinuous function, defining the Lagrangian function L : X × Y ⋆ → ℝ as in (5.51) it holds that F(x, 0) = sup L(x, y⋆ ), y⋆ ∈Y ⋆

and

− F ⋆ (0, y⋆ ) = inf L(x, y⋆ ), x∈X

(5.53)

so that the primal and dual problems can be reformulated in terms of inf Fo (x) = inf sup L(x, y⋆ ),

x∈X

x∈X y⋆ ∈Y ⋆

sup (−F ⋆ (0, y⋆ )) = sup inf L(x, y⋆ ),

y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

x∈X

(Primal), (Dual).

(5.54)

Proof. See Section 5.7.2 in the Appendix of the chapter. At this point, we recall the general properties of biconjugate functions (Propositions 5.2.14 and 5.2.15), according to which for any proper function V it holds that V ⋆⋆ ≤ V and, if V is convex, with equality at the points where V is lower semicontinuous. Therefore, Lemma 5.5.1 already reveals an interesting fact, concerning the primal and the dual problem, which can be further strengthened if V is subdifferentiable at 0, as stated in the following proposition (see, e. g., [13] or [62]). Proposition 5.5.2 (Weak and strong duality). Assume that F : X × Y → ℝ ∪ {+∞} is convex and lower semicontinuous. (i) It holds that the value of the dual problem is less or equal than the value of the primal problem, D = sup (−F ⋆ (0, y⋆ )) ≤ inf F(x, 0) = inf Fo (x) = P, y⋆ ∈Y ⋆

x∈X

x∈X

(weak duality),

226 | 5 Minimax theorems and duality or in terms of the Lagrangian, D = sup inf L(x, y⋆ ) ≤ inf sup L(x, y⋆ ) = P, y⋆ ∈Y ⋆ x∈X

(weak duality),

x∈X y⋆ ∈Y ⋆

and the difference P − D ≥ 0 is called the duality gap. (ii) If 𝜕V(0) ≠ 0 (as for instance if V continuous at y = 0), then any y⋆o ∈ 𝜕V(0) is a solution of the dual problem and D = sup (−F ⋆ (0, y⋆ )) = inf F(x, 0) = inf Fo (x) = P, x∈X

y⋆ ∈Y ⋆

x∈X

(strong duality),

or in terms of the Lagrangian, D = sup inf L(x, y⋆ ) = inf sup L(x, y⋆ ) = P, y⋆ ∈Y ⋆ x∈X

x∈X y⋆ ∈Y ⋆

(strong duality),

and the duality gap P−D = 0. If furthermore, xo is any solution of the primal problem, then, (xo , y⋆o ) is a saddle point of the Lagrangian and F(xo , 0) + F ⋆ (0, y⋆o ) = 0.

(5.55)

(iii) The following are equivalent: (a) (xo , y⋆o ) is a saddle point of the Lagrangian L, (b) xo is a solution of the primal problem, y⋆o is a solution of the dual problem and there is no duality gap, and (c) the extremality condition (5.55) holds. (iv) If the (generalized Slater) condition ∃ x̄ ∈ X such that y 󳨃→ F(x,̄ y) is finite and continuous at y = 0

(5.56)

holds, then V is continuous at 0 hence, 𝜕V(0) ≠ 0. Proof. (i) Since F is a convex function, V is convex (Lemma 5.5.1(i)) and by Proposition 5.2.14 it holds that V ⋆⋆ (0) ≤ V(0), a fact which using Lemma 5.5.1(ii)–(iii) leads to weak duality. (ii) Consider any y⋆o ∈ 𝜕V(0). Then, by the Fenchel–Young inequality (see Proposition 5.2.18) 0 ∈ 𝜕V ⋆ (y⋆o ), so that by the first equation in (5.52) it holds that F ⋆ (0, y⋆o ) = ⋆inf ⋆ F ⋆ (0, y⋆ ) = − sup (−F ⋆ (0, y⋆ )), y ∈Y

y⋆ ∈Y ⋆

(5.57)

hence, y⋆o is a solution of the dual problem. By the same argument, V(0) + V ⋆ (y⋆o ) = ⟨y⋆o , 0⟩Y ⋆ ,Y = 0, and by (5.52) this leads to inf F(x, 0) − sup (−F ⋆ (0, y⋆ )) = P − D = 0,

x∈X

y⋆ ∈Y ⋆

(5.58)

5.5 Minimax and convex duality methods | 227

hence, the strong duality result. If now the primal problem admits a solution, so that there exists xo such that F(xo , 0) = infx∈X F(x, 0) = P, then combining (5.58) with (5.57) we obtain (5.55). Recalling (5.54), we have that if xo is a solution of the primal problem and y⋆o is a solution of the dual problem then P = inf sup L(x, y⋆ ) = sup L(xo , y⋆ ) ≥ L(xo , y⋆ ), x∈X y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

∀ y⋆ ∈ Y ⋆ ,

(5.59)

and D = sup inf L(x, y⋆ ) = inf L(x, y⋆o ) ≤ L(x, y⋆o ), y⋆ ∈Y x∈X

x∈X

∀ x ∈ X.

(5.60)

As shown above, if the dual problem admits a solution, then there is no duality gap, P = D. Hence, setting y⋆ = y⋆o in (5.59) and x = xo in (5.60), we see that D ≤ L(xo , y⋆o ) ≤ P = D, therefore, L(xo , y⋆o ) = P = D. Then (5.59) and (5.60) can be reinterpreted as L(xo , y⋆ ) ≤ P = L(xo , y⋆o ) = D ≤ L(x, y⋆o ),

∀ (x, y⋆ ) ∈ X × Y ⋆ ,

i. e., (xo , y⋆o ) is a saddle point for L. Note that if V is continuous at y = 0 then 𝜕V(0) ≠ 0. (iii) We have already shown that (b) implies (c) in (ii). We show next that (c) implies (a): Assume that (5.55) holds for some point (xo , y⋆o ). Since by the Fenchel–Young inequality F(x, y) + F ⋆ (x⋆ , y⋆ ) ≥ ⟨x⋆ , x⟩X ⋆ ,X + ⟨y⋆ , y⟩Y ⋆ ,Y , for every (x, y) ∈ X × Y and (x⋆ , y⋆ ) ∈ X ⋆ × Y ⋆ , choosing x⋆ = y = 0, we see that F(x, 0) + F ⋆ (0, y⋆ ) ≥ 0 for every (x, y⋆ ) ∈ X × Y ⋆ , with equality at (xo , y⋆o ), which combined with (5.53) leads to L(xo , y⋆ ) ≤ sup L(xo , y⋆ ) = inf L(x, y⋆o ) ≤ L(x, y⋆o ), y⋆ ∈Y ⋆

x∈X

∀ (x, y⋆ ) ∈ X × Y ⋆ .

Choosing y⋆ = y⋆o in the above, we have L(xo , y⋆o ) ≤ L(x, y⋆o ), for every x ∈ X, while choosing x = xo in the above leads to L(xo , y⋆ ) ≤ L(xo , y⋆o ) for every y⋆ ∈ Y ⋆ , so combining these two we have L(xo , y⋆ ) ≤ L(xo , y⋆o ) ≤ L(x, y⋆o ),

∀ (x, y⋆ ) ∈ X × Y ⋆ ,

so that (xo , y⋆o ) is a saddle point for L. It remains to show that (a) implies (b): Recall Proposition 5.1.2. Since (xo , y⋆o ) is a saddle point of L, xo minimizes the mapping x 󳨃→ supy∈Y ⋆ L(x, y⋆ ) = F(x, 0) = Fo (x) (where we used (5.53)) so it is a solution of the primal problem, whereas y⋆o maximizes the mapping y⋆ 󳨃→ infx∈X L(x, y⋆ ) = −F ⋆ (0, y⋆ ) (where we used once more (5.53)) so it is a solution of the dual problem. Again by Proposition 5.1.2, no duality gap exists. (iv) If the generalized Slater condition (5.56) holds, then V is continuous at y = 0; hence, Proposition 5.5.2(iii) can be applied. Indeed, since F is a convex function, being continuous at y = 0 guarantees that F(x,̄ y) is bounded in a neighborhood of y = 0, let us say BY (0, δ), i. e., there exists c ∈ ℝ such that F(x,̄ y) ≤ c for every y ∈ BY (0, δ). But then V(y) := infx∈X F(x, y) ≤ F(x,̄ y) ≤ c for every y ∈ BY (0, δ). Since V is a convex function, this implies continuity of V at y = 0 and also the fact that 𝜕V(0) ≠ 0.

228 | 5 Minimax theorems and duality Example 5.5.3 (A first-order condition for the saddle point formulation). Condition (5.55) can be interpreted as an extremality condition for the saddle point formulation of the Lagrangian. In particular, by the Fenchel–Young inequality (inequality (5.8) applied to the function F and its convex conjugate F ⋆ ) for any pairs (x, y) ∈ X×Y and (x⋆ , y⋆ ) ∈ X ⋆ × Y ⋆ , we have that F(x, y) + F ⋆ (x⋆ , y⋆ ) ≥ ⟨x⋆ , x⟩X ⋆ ,X + ⟨y⋆ , y⟩Y ⋆ ,Y so choosing any pair of the form (x, 0) ∈ X × Y, (0, y⋆ ) ∈ X ⋆ × Y ⋆ we have by the Fenchel–Young inequality that F(x, 0) + F ⋆ (0, y⋆ ) ≥ 0, for every (x, y⋆ ) ∈ X × Y ⋆ . In this sense if a point (xo , y⋆o ) satisfies condition (5.55), then it is a minimizer for the function (x, y⋆ ) 󳨃→ F(x, 0) + F ⋆ (0, y⋆ ). However by (5.53), this function coincides with (x, y⋆ ) 󳨃→ Φ1 (x) − Φ2 (y⋆ ), where Φ1 (x) = supy⋆ ∈Y ⋆ L(x, y⋆ ), and Φ2 (y⋆ ) = infx∈X L(x, y⋆ ), so that a point (xo , y⋆o ) satisfying condition (5.55) minimizes Φ1 over X and maximizes Φ2 over Y ⋆ ; hence, is a saddle point for the Lagrangian. Note furthermore, that once more by the Fenchel–Young inequality, condition (5.55) can be interpreted as (0, y⋆o ) ∈ 𝜕F(xo , 0). ◁ We will show in Examples 5.5.4 and 5.5.5 below, that two well-known examples of duality, Fenchel duality and Lagrange duality fit within the general framework presented in this section. Example 5.5.4 (Fenchel duality as a special case). Assume that Fo = φ1 + φ2 ∘ L where φ1 : X → ℝ ∪ {+∞}, φ2 : Y → ℝ ∪ {+∞} are convex proper lower semicontinuous functions and L : X → Y is a bounded linear operator. Consider the family of perturbations F : X × Y → ℝ ∪ {+∞}, defined by18 F(x, y) = φ1 (x) + φ2 (Lx − y). For this choice, we retrieve the standard Fenchel duality result and a saddle point formulation. ◁ Example 5.5.5 (Lagrangian duality as a special case). Another important class of perturbations can be associated with Lagrangian duality, which corresponds to the problem min φ0 (x), x∈C

C = {x ∈ X : φi ≤ 0, i = 1, . . . , n},

where the function φ0 : X → ℝ ∪ {+∞} is convex proper lower semicontinuous and the functions φi : X → ℝ, i = 1, . . . , n are convex continuous functions, so that the set C is a convex and closed subset of X. We consider the following perturbation of the original problem: Choose as parameter space Y = ℝn so that y = (y1 , . . . , yn ), define the sets C(yi ) = {x ∈ X : φi (x) + yi ≤ 0} and the function F : X × Y → ℝ, by n

F(x, y) = φ0 (x) + ∑ IC(yi ) (x). i=1

Clearly, F(x, 0) = φ0 (x) + IC (x). 18 Since Y is a vector space, we may equivalently use the perturbation F(x, y) = φ1 (x) + φ2 (Lx + y).

5.5 Minimax and convex duality methods | 229

We now calculate the Lagrangian function. We have that −L(x, y⋆ ) = sup(⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y)) y∈Y

n

= −φ0 (x) + sup(⟨y⋆ , y⟩Y ⋆ ,Y − ∑ IC(yi ) (x)) y∈Y

i=1

n

= −φ0 (x) + ∑ sup(yi yi⋆ − IC(yi ) (x)), i=1 yi ∈ℝ

which by a careful consideration of the supremum leads to the equivalent form φ0 (x) + ∑ni=1 yi⋆ φi (x), y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ ℝn+ , L(x, y⋆ ) = { −∞ y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ ̸ ℝn+ , with the vector y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ ℝn+ usually called the vector of Lagrange multipliers. The reader can easily verify19 that n

sup L(x, y⋆ ) = sup (φ0 (x) + ∑ yi⋆ φi (x)) = φ0 (x) + IC (x),

y⋆ ∈Y ⋆

y⋆ ∈ℝn+

i=1

(5.61)

so that the primal problem can be expressed as infx∈X supy⋆ ∈ℝn+ L(x, y⋆ ), while the dual problem is of the form n

sup inf L(x, y⋆ ) = sup inf (φ0 (x) + ∑ yi⋆ φi (x)),

y⋆ ∈Y ⋆ x∈X

y⋆ ∈ℝn+ x∈X

i=1

which is probably easier to handle as it is finite dimensional. An interesting interpretation of the dual problem is to first solve the inner minimization problem, which is a parametric (parameterized by the Lagrange multipliers) unconstrained minimization problem, over all x ∈ X, obtaining a solution xp (y⋆ ), and then try to solve the problem of maximizing φ0 (xp (y⋆ )) + ∑ni=1 yi⋆ φi (xp (y⋆ )) over y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ ℝn+ . By the form of the problem, especially the part ∑ni=1 yi⋆ φi (xp (y⋆ )), we note that unless y⋆ is chosen so that φi (xp (y⋆ )) ≤ 0, for every i = 1, . . . , n, then this last problem is not well posed. This may be interpreted as that we first solve the parametric problem for any chosen value of the Lagrange multipliers and then choose out of these only the solutions corresponding to the values of the Lagrange multipliers which satisfy the constraints. The above comments can be made more precise, upon assuming that the Slater condition (5.56) holds,20 considering the saddle point formulation, and looking for a 19 This just requires a careful rearrangement of the suprema; see Section 5.7.3 in the Appendix. 20 For this particular case, this reduces to the Slater condition that there exists x̄ such that φo (x)̄ < +∞ and φi (x)̄ < 0, i = 1, . . . , n.

230 | 5 Minimax theorems and duality saddle point (xo , y⋆o ) for L. Since xo minimizes L(x, y⋆o ) and y⋆o maximizes L(xo , y⋆ ) (and minimizes −L(xo , y⋆ )), the first-order conditions become n

⋆ 0 ∈ 𝜕x φ0 (xo ) + ∑ yi,o 𝜕x φi (xo ), i=1

n

⋆ 0 ∈ 𝜕y⋆ (−φ0 (xo ) − ∑ yi,o 𝜕φi (xo )), i=1

where we use the subscript in the subdifferential to denote the variable with respect to which we take the variation, and in the first one we used subdifferential calculus (along with the Slater condition). We focus on the second condition which we express (using the definition of the subdifferential) as n

⋆ − yi⋆ )φi (xo ) ≥ 0, ∑(yi,o

∀ yi⋆ ≥ 0, i = 1, . . . , n.

i=1

⋆ Choosing each time yj⋆ = yj,0 for all j ≠ i, we see that the above gives ⋆ (yi,o − yi⋆ )φi (xo ) ≥ 0,

∀ yi⋆ ≥ 0, i = 1, . . . , n,

(5.62)

which is clearly impossible if φi (xo ) > 0 (since this would clearly not hold for any ⋆ choice yi⋆ = k yi,o , k > 1). So, for the above to hold it clearly must be that φi (xo ) ≤ 0 for all i = 1, . . . , n. In fact, with a similar argument as above, one may show that (5.62) ⋆ ⋆ leads to yi,o φi (xo ) = 0 for any i = 1, . . . , n. If φi (xo ) < 0, then yi,o − yi⋆ ≤ 0 for all yi⋆ ≥ 0, ⋆ ⋆ ⋆ ⋆ so that 0 ≤ yi,o ≤ yi for every yi ≥ 0, which leads to the conclusion that yi,o = 0 if ⋆ φi (xo ) < 0. We therefore conclude that the conditions for (xo , yo ) to be a saddle point for the Lagrangian would be n

⋆ 0 ∈ 𝜕x φ0 (xo ) + ∑ yi,o 𝜕x φi (xo ), i=1

⋆ yi,o ≥ 0, ⋆ yi,o φi (xo )

φi (xo ) ≤ 0,

= 0,

i = 1, . . . , n,

(5.63)

i = 1, . . . , n.

The optimality conditions (5.63) are called the Kuhn–Tucker optimality conditions, and have the following important implication: If xμ is a solution of the unconstrained parametric minimization problem Pμ : infx∈X (φ0 (x) + ∑ni=1 μi φi (x)) for some choice μ = (μ1 , . . . , μn ) ∈ ℝn+ , such that φi (xμ ) ≤ 0 and μi φi (xμ ) = 0 for all i = 1, . . . , n, then xμ = xo , where xo is a solution of the original constrained (primal problem). This allows us to choose the solution of the constrained minimization problem out of these solutions of the unconstrained parametric problem which satisfy the complementarity conditions. ◁

5.5 Minimax and convex duality methods | 231

5.5.2 Applications and examples We close this section by providing a number of examples where the general framework developed in the previous subsection may prove itself useful. Out of the numerous applications, we select two basic themes. One is the theme of using convex duality and the minimax approach in order to obtain error bounds for the approximation of the solution of a minimization problem. This is developed here in Example 5.5.6, for a simple case, in terms of Fenchel duality, however, the results can be extended to more general cases (see, e. g., [98]). The second theme, developed in Example 5.5.7, provides a first introduction to an important method called the augmented Lagrangian method, which finds useful applications in the development of numerical algorithms for the solution of optimization problems (see, e. g., [71]). Example 5.5.6 (Error bounds using duality). Consider the minimization problem infx∈X (φ1 (x) + φ2 (Lx)), in the standard setup used for Fenchel duality, but with the extra assumptions that φ2 is a uniformly convex function with modulus of convexity ϕ : ℝ+ → ℝ+ (see Definition 2.3.3), and L : X → Y is coercive in the sense that there exists c > 0, such that ‖Lx‖Y ≥ c‖x‖X , for every x ∈ X, so that one may define an equivalent norm ‖ ⋅ ‖L , on X, by ‖x‖L = ‖Lx‖X . We will show that under the above assumptions, if xo is the solution to the above minimization problem, and x ∈ X is any approximation of it, then we have the following error bound: 2ϕ(‖x − xo ‖L ) ≤ φ1 (x) + φ2 (Lx) + φ⋆1 (L⋆ y⋆ ) + φ⋆2 (−y⋆ ),

∀ y⋆ ∈ Y ⋆ .

For any λ ∈ (0, 1), consider the element z = λxo + (1 − λ)x ∈ X. By the uniform convexity of φ2 , and the convexity of φ1 , we have φ2 (L(λxo + (1 − λ)x)) ≤ λφ2 (Lxo ) + (1 − λ)φ2 (Lx) − 2λ(1 − λ)ϕ(‖x − xo ‖L ), φ1 (λxo + (1 − λ)x) ≤ λφ1 (xo ) + (1 − λ)φ1 (x),

so that upon adding the two and rearranging we have 2λ(1 − λ)ϕ(‖x − xo ‖L )

≤ λ(φ1 (xo ) + φ2 (Lxo )) + (1 − λ)(φ1 (x) + φ2 (Lx)) − (φ1 (z) + φ2 (Lz)),

and since φ1 (z) + φ2 (Lz) ≥ φ1 (xo ) + φ2 (Lxo ) for any choice of λ, the above inequality simplifies, upon dividing by 1 − λ, and passing to the limit λ → 1, to 2ϕ(‖x − xo ‖L ) ≤ φ1 (x) + φ2 (Lx) − (φ1 (xo ) + φ2 (Lxo )) = φ1 (x) + φ2 (Lx) − P.

(5.64)

This is already a bound for the error of the approximation, nevertheless not a very convenient one since it depends on the exact value of P = infx∈X (φ1 (x)+φ2 (Lx)), which is in principle unknown. As such, it will not serve as a good a priori bound.

232 | 5 Minimax theorems and duality However, one may use weak (or strong) duality in order to bring this bound into a more convenient form. Indeed, by weak duality, P ≥ D = sup (−φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ )) ≥ −φ⋆1 (L⋆ y⋆ ) − φ⋆2 (−y⋆ ), y⋆ ∈Y ⋆

for every y⋆ ∈ Y ⋆ . Combining that with (5.64), we obtain a more convenient error bound 2ϕ(‖x − xo ‖L ) ≤ φ1 (x) + φ2 (Lx) + φ⋆1 (L⋆ y⋆ ) + φ⋆2 (−y⋆ ),

∀ y⋆ ∈ Y ⋆ ,

(5.65)

which upon judicious choice of candidates for y⋆ may provide easier to obtain error bounds. Note that upon the observation that −⟨y⋆ , Lx⟩Y ⋆ ,Y − ⟨−L⋆ y⋆ , x⟩X ⋆ ,X = 0 (by the definition of the adjoint operator), the error bound (5.65) can be expressed in the equivalent but more symmetric form 2ϕ(‖x − xo ‖L ) ≤ [φ1 (x) + φ⋆1 (L⋆ y⋆ ) + ⟨L⋆ y⋆ , x⟩X ⋆ ,X ]

+ [φ2 (Lx) + φ⋆2 (−y⋆ ) + ⟨−y⋆ , Lx⟩Y ⋆ ,Y ].

A stricter upper bound may be obtained by minimizing the right-hand side of the above inequality over y⋆ ∈ Y ⋆ , which nevertheless may be a problem as demanding as the original one. However, the fact that since φ2 is uniformly convex implies differentiability properties for φ⋆2 which may be used for further refinements for the error estimates (see [98] for further details). ◁ Example 5.5.7 (The augmented Lagrangian method). Consider the perturbation function γ Fγ (x, y) = φ1 (x) + φ2 (Lx + y) + ‖y‖2Y , 2

γ > 0.

Clearly, this is an admissible perturbation since Fγ (x, 0) = φ1 (x) + φ2 (Lx). The Lagrangian that corresponds to this perturbation provides an interesting method, which also leads to a class of popular numerical algorithms, for the solution of the minimization problem infx∈X (φ1 (x) + φ2 (Lx)), called the augmented Lagrangian method. This method has a long history, originating to the (independent) contributions of Hestenes and Powel in the late 1960s who used this method to treat nonlinear programming problems with equality constraints, and leading later to the work of Fortin and of Ito and Kunisch who used it to treat problems related to the standard Fenchel problem, presented in this example (see [71] and references therein). We will assume for simplicity that we are in a Hilbert space setting, and assume that φ1 : X → ℝ ∪ {+∞} is a convex function with Lipschitz continuous Fréchet derivative, φ2 : Y → ℝ ∪ {+∞} is a proper lower semicontinuous function (not necessarily smooth) and L : X → Y is a bounded linear operator. We will also assume that Y ⋆ ≃ Y,

5.5 Minimax and convex duality methods | 233

denoting the inner product by ⟨ , ⟩Y , while this is not necessarily so for X. With this information at hand, one may proceed to calculate the Lagrangian. We have that L(x, y⋆ ) = − sup(⟨y⋆ , y⟩Y − Fγ (x, y) ) y∈Y

γ = φ1 (x) + inf (−⟨y⋆ , y⟩Y + φ2 (Lx + y) + ‖y‖2Y ) y∈Y 2

γ󵄩 󵄩2 = φ1 (x) + inf (φ2 ( y󸀠 ) + ⟨y⋆ , Lx − y󸀠 ⟩Y + 󵄩󵄩󵄩Lx − y󸀠 󵄩󵄩󵄩Y ), 2 y󸀠 ∈Y

where for the last line we take the infimum over y󸀠 = Lx + y. We observe that, being in Hilbert space, 󵄩2 γ 󵄩󵄩󵄩 γ 󵄩󵄩 1 ⋆ 1 󵄩 ⋆ 󵄩2 󵄩 󸀠 󵄩2 󸀠󵄩 ⋆ 󸀠 󵄩󵄩Lx − y 󵄩󵄩󵄩Y + ⟨y , Lx − y ⟩Y = 󵄩󵄩󵄩Lx + y − y 󵄩󵄩󵄩 − 󵄩󵄩󵄩y 󵄩󵄩󵄩Y , 󵄩󵄩Y 2γ 2 2 󵄩󵄩 γ so that 󵄩2 γ 󵄩󵄩󵄩󵄩 1 ⋆ 1 󵄩 ⋆ 󵄩2 󵄩󵄩 󸀠󵄩 󸀠 y − y L(x, y⋆ ) = φ1 (x) + inf (φ ( y ) + Lx + 󵄩 󵄩󵄩 ) − 󵄩󵄩󵄩y 󵄩󵄩󵄩Y 2 󵄩󵄩Y 2 󵄩󵄩󵄩 γ 2γ y󸀠 ∈Y 1 󵄩 󵄩2 1 = φ1 (x) + φ2,γ (Lx + y⋆ ) − 󵄩󵄩󵄩y⋆ 󵄩󵄩󵄩Y , γ 2γ where by φ2,γ we denote the Moreau–Yosida envelope of φ2 (see Definition 4.7.3). By the properties of the Moreau–Yosida envelope (see Proposition 4.7.6), the function φ2,γ is Lipschitz continuously Fréchet differentiable (while φ2 is not necessarily smooth) so that the Lagrangian is smooth in both variables. The above Lagrangian depends on the regularization parameter γ > 0, and is called the augmented Lagrangian. We will use the parameterization λ = γ −1 , and denote the augmented Lagrangian by λ 󵄩 󵄩2 Lλ (x, y⋆ ) = φ1 (x) + φ2,λ (Lx + λy⋆ ) − 󵄩󵄩󵄩y⋆ 󵄩󵄩󵄩Y . 2 Using the properties of the Moreau–Yosida envelope (Proposition 4.7.6), it is easy to see that Lλ is Fréchet differentiable with respect to both variables, and in particular 1 Dx Lλ (x, y⋆ ) = Dx φ1 (x) + L⋆ (Lx + λy⋆ − proxλφ2 (Lx + λy⋆ )), λ Dy⋆ Lλ (x, y⋆ ) = (Lx + λy⋆ − proxλφ2 (Lx + λy⋆ )) − λy⋆

(5.66)

= Lx − proxλφ2 (Lx + λy⋆ ),

or in more compact form 1 Dx Lλ (x, y⋆ ) = Dx φ1 (x) + L⋆ (I − proxλφ2 )(Lx + λy⋆ ), λ Dy⋆ Lλ (x, y⋆ ) = (I − proxλx2 )(Lx + λy⋆ ) − λy⋆ . One may show the equivalence of the following three statements:

(5.67)

234 | 5 Minimax theorems and duality (a) (xo , y⋆o ) satisfies Dx Lλ (xo , y⋆o ) = 0,

Dy⋆ Lλ (xo , y⋆o ) = 0,

(5.68)

(b) (xo , y⋆o ) satisfies Dx φ1 (xo ) + L⋆ y⋆o = 0,

y⋆o ∈ 𝜕φ2 (Lxo ),

(5.69)

(c) (xo , y⋆o ) is a saddle point of the augmented Lagrangian Lλ . The equivalence of (a) and (b) can easily be seen by recalling that proxλφ2 = (I +λφ2 )−1 . In particular, if (b) holds then the second of (5.69) yields that for every λ > 0, Lxo +λy⋆o ∈ (I + λ𝜕φ2 )(Lxo ) or equivalently that Lxo = proxλφ2 (Lxo + λy⋆o ) which, by (5.66), is the second of (5.68). On the other hand, the first of (5.69) can be expressed as 1 0 = Dx φ1 (xo ) + L⋆ (λy⋆o + 0) λ 1 = Dx φ1 (xo ) + L⋆ (λy⋆o + Lxo − proxλφ2 (Lxo + λy⋆o )), λ which by (5.66)) is the first of (5.68). Suppose that (a) holds, i. e., for some λ > 0, (xo , y⋆o ) satisfies (5.68) with these Fréchet derivatives given by (5.66). The second of (5.68) implies that Lxo + λy⋆o ∈ (I + λ𝜕φ2 )(Lxo ) which in turn implies that y⋆o ∈ 𝜕φ2 (Lxo ), the second relation in (5.69). Furthermore, since the second of (5.68) implies by (5.66) that 0 = Lxo − proxλφ2 (Lxo + λy⋆o ), combining the first of (5.68) with this and using the first of (5.66) we obtain the first of (5.69). Thus, the equivalence of (a) and (b) is complete. To show the equivalence of (a) and (c), it is enough to recall that (xo , y⋆o ) is a saddle point for Lλ if and only if xo ∈ arg min Lλ (x, y⋆o ), x∈X

y⋆o ∈ arg min L (xo , y⋆ ). ⋆ ⋆ λ y ∈Y

This is equivalent to 0 = Dx Lλ (xo , y⋆o ) and 0 = Dy⋆ Lλ (xo , y⋆o ); hence, the stated equivalence. We now show that if (xo , y⋆o ) satisfies any of the equivalent conditions (a), (b) or (c) above, then xo is a solution of the original optimization problem infx∈X (φ1 (x)+φ2 (Lx)). Consider condition (b), i. e., that xo satisfies Dx φ1 (xo ) + L⋆ y⋆o = 0,

for some y⋆o ∈ 𝜕φ2 (Lxo ).

(5.70)

5.6 Primal dual algorithms | 235

This can be expressed as Dx φ1 (xo ) = −L⋆ y⋆o ∈ −L⋆ 𝜕φ2 (Lxo ) which is in turn equivalent to 0 ∈ Dx φ1 (xo ) + L⋆ 𝜕φ2 (Lxo ) whereby using standard subdifferential calculus we conclude that 0 ∈ 𝜕(φ1 + φ2 ∘ L)(xo ); hence, xo is a minimizer of the original problem. The augmented Lagrangian is to be compared with the standard Lagrangian we would get in the case where γ = 0 (equiv. λ = ∞). This would lead to the Lagrangian function L(x, y⋆ ) = − sup(⟨y⋆ , y⟩Y − F0 (x, y)) y∈Y

= φ1 (x) − sup(⟨y⋆ , y⟩Y − φ2 (Lx + y)) y∈Y

= φ1 (x) − sup(⟨y⋆ , y󸀠 ⟩Y − φ2 (y󸀠 ) − ⟨y⋆ , Lx⟩Y ) = φ1 (x) −

y󸀠 ∈Y φ⋆2 (y⋆ )

+ ⟨y⋆ , Lx⟩Y .

We claim that any saddle point of the augmented Lagrangian is also a saddle point for the standard Lagrangian, which is no longer necessarily smooth in y⋆ , with the converse being also true. Indeed, if (xo , yo ) is a saddle point for L, then Dx L(xo , y⋆o ) = 0,

0 ∈ 𝜕y⋆ L(xo , y⋆o ).

(5.71)

The first equation of (5.71) reduces to Dx φ1 (xo ) + L⋆ y⋆o = 0 which is the first equation in (5.69). The second inclusion of (5.71) reduces to 0 ∈ Lxo − 𝜕φ⋆2 (y⋆o ), which is equivalent to Lxo ∈ 𝜕φ⋆2 (y⋆o ), so that by the Fenchel–Young inequality we have that y⋆o ∈ 𝜕φ2 (Lxo ), which is the second equation in (5.69). Hence, any saddle point of L is a saddle point of Lλ for any λ > 0. The reverse implication can be also easily checked. How does the above construction help us in the treatment of the original problem? The important difference between the standard Lagrangian approach and the augmented Lagrangian approach, is that even though a saddle point (xo , y⋆o ) of the augmented Lagrangian is also a saddle point of the standard Lagrangian, with xo being a solution of the original problem, the smoothness of the augmented Lagrangian in both variables makes the numerical calculation of the saddle point a much easier and efficient task than the original problem of retrieving a saddle point for the original Lagrangian. ◁

5.6 Primal dual algorithms Convex duality methods often form the basis for a number of interesting numerical methods for the resolution of optimization problems. For simplicity, we assume that X, Y are Hilbert spaces, with respective inner products ⟨⋅, ⋅⟩X , ⟨⋅, ⋅⟩Y , in this section.

236 | 5 Minimax theorems and duality Primal dual algorithms are based on the observation that a minimization problem minx∈X f (x) can be brought into a saddle point form as P = inf sup L(x, y⋆ ) = sup inf L(x, y⋆ ) = D, y⋆ ∈Y ⋆ x∈X

x∈X y⋆ ∈Y ⋆

so that a solution xo of the original (primal) problem can be retrieved from a saddle point (xo , y⋆o ) of the Lagrangian L, with y⋆o being a solution of the dual problem. Primal dual algorithms try to address simultaneously the primal and the dual problem by treating xo ∈ arg min L(x, y⋆o ), y⋆o

x∈X

∈ arg max⋆ L(xo , y⋆ ), y∈Y

usually considering the first-order conditions for the above problems, 0 ∈ 𝜕x L(xo , y⋆o ),

0 ∈ 𝜕y⋆ L(xo , y⋆o ),

and treating them in some sort of iterative scheme, the fixed point of which (if it exists) corresponds to the desired saddle point (xo , y⋆o ). Since in many cases of interest the problems under consideration do not involve smooth functions, some regularization procedure involving, e. g., the Moreau–Yosida envelope and leading thus to proximal methods can be employed. Such methods are often employed for optimization problems of the standard form min(φ1 (x) + φ2 (Lx)) x∈X

which under convexity assumptions on φ2 , taking into account that φ2 = φ⋆⋆ 2 , can be expressed in the saddle point form (see Example 5.5.4) inf max{⟨y⋆ , Lx⟩Y − φ⋆2 (y⋆ ) + φ1 (x)},

x∈X y⋆ ∈Y

(5.72)

where we assume, being in Hilbert space that Y ⋆ ≃ Y. The first-order conditions for a saddle point (xo , y⋆o ) are 0 ∈ L⋆ y⋆o + 𝜕φ1 (xo ),

0 ∈ Lxo − 𝜕φ⋆2 (y⋆ ),

which may be further interpreted as a fixed point scheme for an appropriately chosen operator. One way to numerically treat this problem is within the class of the Uzawatype algorithms, where a proximal descent method is used for the x variable and a standard ascend method is used for the dual y⋆ variable, in terms of xn+1 ∈ xn − λ(L⋆ y⋆̄ n + 𝜕φ1 (xn+1 )),

5.6 Primal dual algorithms | 237

y⋆n+1 ∈ y⋆n + μ(Lx̄n − 𝜕φ⋆2 (y⋆n+1 )), where the interaction between the primal and the dual problem is in terms of the coupling of the first with the second inclusion through the terms x̄n and ȳ⋆n . These are in general functions of xn , xn+1 , y⋆n , y⋆n+1 , which may be chosen in a variety of ways, each leading to different versions of the algorithm, as long as the conditions xn+1 = xn and y⋆n+1 = y⋆n imply that x̄n = xn and y⋆̄ n = y⋆n , respectively.21 Recalling the definition of the proximal operators proxλφ1 = (I + λ𝜕φ1 )−1 , and proxμφ⋆2 = (I + μ𝜕φ⋆2 )−1 , we may express the above scheme in proximal form as xn+1 = proxλφ1 (xn − λL⋆ y⋆̄ n ), y⋆n+1 = proxμφ⋆2 (y⋆n + μLx̄n ).

In fact one may reverse the order of the primal and the dual step and rewrite the above algorithm as y⋆n+1 = proxμφ⋆2 (y⋆n + μLx̄n ),

xn+1 = proxλφ1 (xk − λL⋆ y⋆n+1 ), x̄n = xn + θ(xn − xn−1 ),

where we have also chosen y⋆̄ n = y⋆n+1 , and θ ∈ [0, 1] is a parameter which is used in order to interpolate x̄k between xn−1 and xn . In the special case θ = 0, x̄n = xn and we recover a classic algorithm proposed by Kenneth Arrow and Hurwicz in the late 1950s, as modified for the specific problem at hand. As it will turn out, other choices for θ as, e. g., θ = 1, (considered as an overrelaxation step) have superior convergence properties. Note also the similarity with the Douglas–Rachford scheme (in the special case L = I, see also Remark 5.6.3). We therefore, have the following algorithm, due to Chambolle and Pock (see [38]) for the resolution of minimization problems Algorithm 5.6.1 (Chambolle–Pock PDHG). 1. Choose λ, μ, θ ∈ [0, 1]. 2. Choose initial condition (x1 , y⋆1 ) ∈ X × Y and set x̄1 = x1 . 3. Iterate y⋆n+1 = proxμφ⋆2 (y⋆n + μLx̄n ),

xn+1 = proxλφ1 (xn − λL⋆ y⋆n+1 ),

(5.73)

x̄n+1 = xn+1 + θ(xn+1 − xn ),

until a convergence criterion is met. 21 This is needed for compatibility with the condition that a saddle point (xo , y⋆o ) is a fixed point of the iteration scheme.

238 | 5 Minimax theorems and duality While the formal similarity of this algorithm with the Douglas–Rachford scheme can be used to obtain convergence results for the above algorithm we prefer to present here a more direct approach. The following proposition due to [38] provides some results concerning the convergence of this algorithm for the choice θ = 1. Our approach follows closely the steps in [38], while in [40], the interested reader may find a revision and generalization of some of the arguments in [38]. Proposition 5.6.2. Assume that λμ‖L‖ < 1. Then the sequence {(xn , y⋆n ) : n ∈ ℕ} generated by (5.73) for θ = 1, is bounded in X × Y ⋆ and the sequence {(x̄N , ȳ⋆N ) : N ∈ ℕ}, where x̄N = N1 ∑Nn=1 xi and y⋆̄ N = N1 ∑Nn=1 y⋆i , converges weakly to a saddle point of (5.72). Proof. The strategy of the proof is to show that the distance of the sequence {(xn , y⋆n ) : n ∈ ℕ} generated by (5.73) with respect to any saddle point is bounded; hence, weakly convergent to some point w̄ o := (x̄o , ȳ⋆o ) in X × Y ⋆ . We will then identify this weak limit as a saddle point. We will assume X ⋆ ≃ X and Y ⋆ ≃ Y, but retain the notation Y ⋆ and y⋆ in compliance to our general notation concerning the dual formulation of optimization problems. The proof proceeds in 3 steps. 1. In the first step, we establish a generalized monotonicity result for {(xn , y⋆n ) : n ∈ ℕ} in the form of c(

1 1 󵄩 1 1 󵄩 󵄩2 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩Y ) ≥ ‖x − xN ‖2X + 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y 2λ 2μ 2λ 2μ N−1

+ c ∑ [⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 )] n=1

N−1

− c ∑ [⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x)], n=1

for every (x, y ) ∈ X × Y for an appropriate constant c > 0. We start with the general form of the algorithm as ⋆

⋆

̄ y⋆n+1 = proxμφ⋆2 (y⋆n + μLx),

xn+1 = proxλφ1 (xn − λL⋆ y⋆̄ ), for an appropriate choice x̄ and y⋆̄ , to be specified during the course of the proof. By the definition of the proximal map, we have that 1 (x − xn+1 ) − L⋆ ȳ ∈ 𝜕φ1 (xn+1 ), λ n 1 ⋆ (y − y⋆n+1 ) + Lx̄ ∈ 𝜕φ⋆2 (y⋆n+1 ), μ n so that by the definition of the subdifferential we have that 1 φ1 (x) ≥ φ1 (xn+1 ) + ⟨xn − xn+1 , x − xn+1 ⟩X − ⟨L(x − xn+1 ), y⋆̄ ⟩Y , ∀ x ∈ X, λ 1 φ⋆2 (y⋆ ) ≥ φ⋆2 (y⋆n+1 ) + ⟨y⋆n − y⋆n+1 , y⋆ − y⋆n+1 ⟩Y + ⟨Lx,̄ y⋆ − y⋆n+1 ⟩Y , ∀ y⋆ ∈ Y ⋆ , μ

(5.74)

5.6 Primal dual algorithms | 239

where we used ⟨x − xn+1 , L⋆ y⋆̄ ⟩Y = ⟨L(x − xn+1 ), y⋆̄ ⟩X . We first note that the term ⟨xn − xn+1 , x − xn+1 ⟩ can be expressed as 1 1 1 ⟨xn − xn+1 , x − xn+1 ⟩X = ‖xn − xn+1 ‖2X − ‖x − xn ‖2X + ‖x − xn+1 ‖2X , 2 2 2

(5.75)

with a similar expression for ⟨y⋆n − y⋆n+1 , y⋆ − y⋆n+1 ⟩. This can be verified by direct calculation of the norms, recalling that we are in a Hilbert space setting. Substituting (5.75) into (5.74) and adding the two inequalities, we obtain 0 ≥ [ ⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 ) ] − ⟨Lxn+1 , y⋆ ⟩Y + ⟨Lx,̄ y⋆ − y⋆n+1 ⟩Y − [ ⟨Lx, y⋆ ⟩ − φ⋆ (y⋆ ) + φ1 (x) ] + ⟨Lx, y⋆ ⟩ − ⟨L(x − xn+1 ), y⋆̄ ⟩ n+1 Y

2

n+1

n+1 Y

1 1 1 + ‖xn − xn+1 ‖2X − ‖x − xn ‖2X + ‖x − xn+1 ‖2X 2λ 2λ 2λ 1 󵄩 1 󵄩 1 󵄩 󵄩2 󵄩2 󵄩2 + 󵄩󵄩󵄩y⋆n − y⋆n+1 󵄩󵄩󵄩Y − 󵄩󵄩󵄩y⋆ − y⋆n 󵄩󵄩󵄩Y + 󵄩󵄩󵄩y⋆ − y⋆n+1 󵄩󵄩󵄩Y , 2μ 2μ 2μ

Y

which is further rearranged as 1 󵄩 1 󵄩2 ‖x − xn ‖2X + 󵄩󵄩󵄩y⋆ − y⋆n 󵄩󵄩󵄩Y 2λ 2μ ≥ [ ⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 ) ] − [ ⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x) ] 1 1 󵄩 1 󵄩 1 󵄩2 󵄩2 + ‖x − xn+1 ‖2X + ‖xn − xn+1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆n+1 󵄩󵄩󵄩Y + 󵄩󵄩󵄩y⋆n − y⋆n+1 󵄩󵄩󵄩Y 2λ 2λ 2μ 2μ ̄ y⋆ − y⋆ ⟩ − ⟨L(xn+1 − x), y⋆ − y⋆̄ ⟩ . + ⟨L(xn+1 − x), n+1

Y

n+1

(5.76)

Y

In (5.76), we may control the sign of the first two terms upon choosing (x, y⋆ ) to be a saddle point of the Lagrangian formulation of the optimization problem. We would like to have a similar control over the last two terms. We note that upon choosing x,̄ y⋆̄ in an appropriate manner, the last two terms of (5.76) can be simplified. Choosing them as in the PDHG algorithm, i. e., x̄ = 2xn − xn−1 and ȳ⋆ = y⋆n+1 , we have for the last two terms of (5.76) that ̄ y⋆n+1 − y⋆ ⟩Y − ⟨L(xn+1 − x), y⋆n+1 − y⋆̄ ⟩Y I := ⟨L(xn+1 − x),

≥ ⟨L(xn+1 − xn−1 ), y⋆n+1 − y⋆ ⟩Y − ⟨L(xn − xn−1 ), y⋆n − y⋆ ⟩Y 󵄨 󵄨 − 󵄨󵄨󵄨⟨L(xn − xn−1 ), y⋆n+1 − y⋆n ⟩Y 󵄨󵄨󵄨 ≥ ⟨L(xn+1 − xn−1 ), y⋆n+1 − y⋆ ⟩Y − ⟨L(xn − xn−1 ), y⋆n − y⋆ ⟩Y 󵄩 󵄩 − ‖L‖ ‖xn − xn−1 ‖X 󵄩󵄩󵄩y⋆n+1 − y⋆n 󵄩󵄩󵄩Y ≥ ⟨L(xn+1 − xn−1 ), y⋆n+1 − y⋆ ⟩Y − ⟨L(xn − xn−1 ), y⋆n − y⋆ ⟩Y α 1 󵄩 󵄩2 − ‖L‖ ‖xn − xn−1 ‖2X − ‖L‖ 󵄩󵄩󵄩y⋆n+1 − y⋆n 󵄩󵄩󵄩Y , 2 2α

(5.77)

240 | 5 Minimax theorems and duality for an appropriately chosen α (to be specified shortly). Combining (5.77) with (5.76), we see that 1 1 󵄩 󵄩2 ‖x − xn ‖2X + 󵄩󵄩󵄩y⋆ − y⋆n 󵄩󵄩󵄩Y 2λ 2μ ≥ [⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 )] − [⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x)] 1 1 α‖L‖ ‖x − xn+1 ‖2X + ‖xn − xn+1 ‖2X − ‖xn − xn−1 ‖2X 2λ 2λ 2 1 ‖L‖ 󵄩󵄩 ⋆ 1 󵄩 󵄩2 󵄩2 )󵄩y − y⋆n+1 󵄩󵄩󵄩Y + 󵄩󵄩󵄩y⋆ − y⋆n+1 󵄩󵄩󵄩Y + ( − 2μ 2μ 2α 󵄩 n

+

(5.78)

+ ⟨L(xn+1 − xn ), y⋆n+1 − y⋆ ⟩Y − ⟨L(xn − xn−1 ), y⋆n − y⋆ ⟩Y . Our aim is to turn (5.78) to a comparison between ‖xN − x‖2X + ‖y⋆N − y⋆ ‖2Y and ‖x1 − x‖2X + ‖y⋆1 − y⋆ ‖2Y , showing that ‖xN − x‖2X + ‖y⋆N − y⋆ ‖2Y is decreasing if (x, y⋆ ) is chosen appropriately (e. g., if (x, y⋆ ) = (xo , y⋆o ) is a saddle point), so that the sequence {(xN , y⋆N ) : N ∈ ℕ} generated by the PDHG scheme is a Fejér sequence with respect to the set of saddle points of the problem, therefore, leading to weak convergence of the scheme to a saddle point (xo , y⋆o ). To this end, observe that we could turn certain parts of (5.78) into telescopic sums, leading thus to a number of simplifications. Adding (5.78) from n = 1 to N − 1, and setting x0 = x1 to eliminate certain terms, we see that after simplification, for any (x, y⋆ ) ∈ X × Y ⋆ it holds that 1 1 󵄩 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩Y 2λ 2μ 1 󵄩 1 󵄩2 ≥ ‖x − xN ‖2X + 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y 2λ 2μ N−1

+ ∑ [ ⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 ) ] n=1

N−1

− ∑ [ ⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x) ] n=1

+

1 ‖x − xN−1 ‖2X + ⟨L(xN − xN−1 ), y⋆N − y⋆ ⟩Y 2λ N

+( ≥

1 α‖L‖ N−2 1 ‖L‖ N−2󵄩󵄩 ⋆ 󵄩2 − ) ∑ ‖xn+1 − xn ‖2X + ( − ) ∑ 󵄩󵄩yn+1 − y⋆n 󵄩󵄩󵄩Y 2λ 2 2μ 2α n=1 n=1

1 󵄩 1 󵄩2 ‖x − xN ‖2X + 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y 2λ 2μ N−1

+ ∑ [ ⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 ) ] n=1

N−1

− ∑ [ ⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x) ] n=1

(5.79)

5.6 Primal dual algorithms | 241

+ ‖xN − xN−1 ‖2X +

1 ‖x − xN−1 ‖2X + ⟨L(xN − xN−1 ), y⋆N − y⋆ ⟩Y , 2λ N

> 0 and as long as α is chosen so that 2λ1 − α‖L‖ 2 conclude our estimates by further estimating

1 2μ

−

‖L‖ 2α

> 0. We are now in position to

󵄩 󵄩 ⟨L(xN − xN−1 ), y⋆N − y⋆ ⟩Y ≥ −‖L‖ ‖xN − xN−1 ‖X 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y ≥−

β‖L‖ ‖L‖ 󵄩󵄩 ⋆ ⋆ 󵄩2 ‖xN − xN−1 ‖2X − 󵄩y − yN 󵄩󵄩󵄩Y , 2 2β 󵄩

(5.80)

for a suitable choice for β. Substituting (5.80) into (5.79), we obtain 1 1 󵄩 1 1 ‖L‖ 󵄩󵄩 ⋆ 󵄩2 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩Y ≥ ‖x − xN ‖2X + ( − )󵄩y − y⋆N 󵄩󵄩󵄩Y 2λ 2μ 2λ 2μ 2β 󵄩 N−1

+ ∑ [ ⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 ) ] n=1

N−1

− ∑ [ ⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x) ] n=1

+( which upon the choice β =

1 λ‖L‖

β ‖L‖ 1 − )‖xN − xN−1 ‖2X , 2λ 2

reduces to

1 1 󵄩 1 󵄩 1 󵄩2 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩Y ≥ ‖x − xN ‖2X + (1 − λμ‖L‖2 ) 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y 2λ 2μ 2λ 2μ N−1

+ ∑ [⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 )] n=1

(5.81)

N−1

− ∑ [⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x)]. n=1

Since, trivially, ‖x − xN ‖2X ≥ (1 − λμ‖L‖2 )‖x − xN ‖2X as long as 1 − λμ‖L‖2 > 0 for such a choice of parameters (5.81) yields the estimate c(

1 󵄩 1 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩Y ) 2λ 2μ 1 󵄩 1 󵄩2 ≥ ‖x − xN ‖2X + 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩Y 2λ 2μ

(5.82)

N−1

+ c ∑ [⟨Lxn+1 , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (xn+1 )] n=1

N−1

− c ∑ [⟨Lx, y⋆n+1 ⟩Y − φ⋆2 (y⋆n+1 ) + φ1 (x)], n=1

for c = (1 − λμ‖L‖2 )−1 .

∀ (x, y⋆ ) ∈ X × Y ⋆ ,

242 | 5 Minimax theorems and duality 2. We now use (5.82) to show that the sequence {(xn , y⋆n ) : n ∈ ℕ} is weakly convergent. This requires choosing (x, y⋆ ) ∈ X × Y ⋆ so as to further simplify (5.82) and obtain a boundedness result for the sequence. Choosing (x, y⋆ ) = (xo , y⋆o ) a saddle point for the Lagrangian (see (5.72) and assuming that the saddle value is So ), so that [⟨Lxn+1 , y⋆o ⟩Y − φ⋆2 (y⋆o ) + φ1 (xn+1 )] ≥ So , [⟨Lxo , y⋆n+1 ⟩Y

−

φ⋆2 (y⋆n+1 )

+ φ1 (xo )] ≤ So ,

∀ xn+1 ∈ X,

∀ y⋆n+1 ∈ Y ⋆ .

(5.83)

Adding (5.83) from n = 1, . . . , N − 1, we have N−1

∑ [⟨Lxn+1 , y⋆o ⟩Y − φ⋆2 (y⋆o ) + φ1 (xn+1 )]

n=1

−

N−1

∑ [⟨Lxo , y⋆n+1 ⟩Y n=1

−

φ⋆2 (y⋆n+1 )

(5.84)

+ φ1 (xo )] ≥ 0.

Setting now (x, y⋆ ) = (xo , y⋆o ) (a saddle point) in (5.82), and combining the result with (5.84) we conclude that 1 1 󵄩 1 󵄩 1 󵄩2 󵄩2 ‖x − x1 ‖2X + 󵄩󵄩󵄩y⋆o − y⋆1 󵄩󵄩󵄩Y ≥ (1 − λμ‖L‖2 )( ‖xo − xN ‖2X + 󵄩󵄩󵄩y⋆o − y⋆N 󵄩󵄩󵄩Y ). 2λ o 2μ 2λ 2μ This is true for any N; hence, the sequence {wN := (xN , y⋆N ) : N ∈ ℕ} is bounded in X × Y ⋆ , therefore, there exists w̄ o := (x̄o , ȳ⋆o ) ∈ X × Y such that wn ⇀ w̄ o in X × Y ⋆ . We will identify w̄ o as a saddle point. 3. In order to identify the limit, we will use once more (5.82) for a general choice of (x, y⋆ ) ∈ X × Y ⋆ . Consider the sequence {(x̄N , ȳ⋆N ) : N ∈ ℕ} defined by x̄N = 1 ∑N x , ȳ⋆N = N1 ∑Ni=1 y⋆i . Clearly, (x̄N , ȳ⋆N ) ⇀ (x̄o , ȳ⋆o ) in X × Y ⋆ . One easily sees that N i=1 i ∑Nn=1 ⟨Lxn , y⋆ ⟩Y = N⟨Lx̄N , y⋆ ⟩Y , and ∑Nn=1 ⟨Lx, y⋆n ⟩Y = N⟨Lx, ȳ⋆N ⟩Y , while by convexity N

∑ φ1 (xn ) ≥ Nφ1 (x̄N ),

n=1

and

N

∑ φ⋆2 (yn ) ≥ Nφ⋆2 (ȳ⋆N ).

n=1

Substituting these into (5.82), we obtain [⟨Lx̄N , y⋆ ⟩Y − φ⋆2 (y⋆ ) + φ1 (x̄N )] − [⟨Lx, ȳ⋆N ⟩Y − φ⋆2 (ȳ⋆N ) + φ1 (x)] ≤

1 1 1 󵄩 󵄩2 󵄩 󵄩2 [ (‖x − x1 ‖2X − ‖x − xN ‖2X ) + (󵄩󵄩󵄩y⋆ − y⋆1 󵄩󵄩󵄩X − 󵄩󵄩󵄩y⋆ − y⋆N 󵄩󵄩󵄩X )], cN 2λ 2μ

(5.85)

∀ (x, y⋆ ) ∈ X × Y ⋆ , and passing to the limit as N → ∞ (using also the weak lower semicontinuity of the convex functions φ1 , φ⋆2 ), we conclude that [⟨Lx̄o , y⋆ ⟩Y − φ⋆2 (x) + φ1 (x̄o )] − [⟨Lx, ȳ⋆o ⟩Y − φ⋆2 (ȳ⋆o ) + φ1 (x)] ≤ 0, ∀ (x, y⋆ ) ∈ X × Y ⋆ ,

so that (x̄o , ȳ⋆o ) is a saddle point of ⟨Lx, y⋆ ⟩Y + φ1 (x) − φ⋆2 (y⋆ ).

5.7 Appendix | 243

As seen in the proof of the above proposition (see estimate (5.85)) the rate of convergence is O(1/N) (ergodic convergence). Convergence may be faster (e. g., O(1/N 2 )) if either φ1 or φ⋆2 is uniformly convex. In such cases, acceleration techniques may be employed (see, e. g., [38]). In finite dimensional spaces, one may show (strong) convergence of the whole sequence. The Chambolle–Pock algorithm may be further extended in various directions, i. e., in the study of minimization problems of the form min(φ1 (x) + φ2 (x) + φ3 (Lx)), x∈X

whose minimax (saddle point) formulation is min max (⟨Lx, y⋆ ⟩Y + φ1 (x) + φ2 (x) − φ⋆3 (y⋆ )), ⋆ x∈X y ∈Y

with φ1 proper convex, lower semicontinuous with Dφ1 Lipschitz continuous and with φ2 , φ3 proper lower semicontinuous convex, such that their corresponding proximal maps are easy to compute (or even in cases where φ3 is a convex lower semicontinuous function defined by an inf-convolution; see Definition 5.3.1). Furthermore, one may consider generalized proximal maps in which the quadratic regularization term λ1 ‖x − z‖2 is replaced by a regularization term involving the Bregman function Dψ (x, z) := ψ(x) − ψ(z) − ⟨Dψ(z), x − z⟩ for an appropriate continuously differentiable function (a popular choice being the entropy function). Remark 5.6.3 (Connections with the Douglas–Rachford scheme). As already mentioned, the algorithm (5.73) can be considered as an extension of the Douglas– Rachford scheme. Indeed, recalling the definition of the Moreau proximity operator the above scheme can be expressed, upon defining the operators A, B acting on z = (x, y⋆ )T by 1 I x A ( ⋆) = ( λ −L y

−L⋆ x 1 ) ( ⋆) , I y μ

x 𝜕φ (x) 0 B ( ⋆ ) = ( ⋆1 ⋆ ) + ( y 𝜕φ2 (y ) −L

L⋆ x ) ( ⋆) , 0 y

the Uzawa scheme can be expressed as 0 ∈ A(zk+1 − zk ) + Bzk+1 .

5.7 Appendix 5.7.1 Proof of Proposition 5.1.2 Assume that a saddle point (xo , yo ) exists for F. By the definition of a saddle point, sup F(xo , y) = F(xo , yo ), y∈Y

(5.86)

244 | 5 Minimax theorems and duality and inf F(x, yo ) = F(xo , yo ).

(5.87)

x∈X

Consider the function F1 : X → ℝ, defined by F1 (x) = supy∈Y F(x, y). Since infx∈X F1 (x) ≤ F1 (x) for every x ∈ X, choosing x = xo in this inequality yields that infx∈X F1 (x) ≤ F1 (xo ), so that using the definition of F1 we see that inf sup F(x, y) ≤ sup F(xo , y).

x∈X y∈Y

(5.88)

y∈Y

Similarly, considering the function F2 : Y → ℝ defined by F2 (y) = infx∈X F(x, y), we note that, trivially, F2 (y) ≤ supy∈Y F2 (y) for every y ∈ Y and setting y = yo yields that F2 (yo ) ≤ supy∈Y F2 (y), so that using the definition of F2 we see that inf F(x, yo ) ≤ sup inf F(x, y).

x∈X

(5.89)

y∈Y x∈X

Combining (5.86)–(5.89) leads to the inequality, inf sup F(x, y) ≤ sup inf F(x, y),

x∈X y∈Y

y∈Y x∈X

which when combined with (5.1) leads to the equality inf sup F(x, y) = sup inf F(x, y).

x∈X y∈Y

y∈Y x∈X

Restating all these inequalities in a single line yields (5.1)

F(xo , yo ) = inf F(x, yo ) ≤ sup inf F(x, y) ≤ inf sup F(x, y) x∈X

y∈Y x∈X

≤ sup F(xo , y) = F(xo , yo ),

x∈X y∈Y

y∈Y

from which it follows that all these inequalities are in fact equalities, F(xo , yo ) = inf F(x, yo ) = sup inf F(x, y) = inf sup F(x, y) = sup F(xo , y). x∈X

y∈Y x∈X

x∈X y∈Y

y∈Y

That means sup inf F(x, y) = inf F(x, yo ) = F(xo , yo ), y∈Y x∈X

x∈X

so that the supremum of the function F2 : Y → ℝ defined by F2 (y) = infx∈X F(x, y) is attained at yo , and inf sup F(x, y) = sup F(xo , y) = F(xo , yo ),

x∈X y∈Y

y∈Y

5.7 Appendix | 245

so that the infimum of the function F1 : X → ℝ defined by F1 (x) = supy∈Y F(x, y) is attained at xo . For the converse, assume that sup inf F(x, y) = inf sup F(x, y) = F(xo , yo ), y∈Y x∈X

(5.90)

x∈X y∈Y

with the supremum in the first expression and the infimum on the second being attained at yo at xo , respectively. Then inf sup F(x, y) = sup F(xo , y) ≥ F(xo , yo )

x∈X y∈Y

(5.91)

y∈Y

where we used successively that supy∈Y F(⋅, y) attains the infimum at xo and then the fact that the supremum is an upper limit, and sup inf F(x, y) = inf F(x, yo ) ≤ F(xo , yo ), y∈Y x∈X

(5.92)

x∈X

where we used successively that infx∈X F(x, ⋅) attains the supremum at yo and then the fact that the infimum is a lower limit. Combining (5.91) and (5.92) into a single inequality, inf sup F(x, y) = sup F(xo , y) ≥ F(xo , yo ) ≥ inf F(x, yo ) = sup inf F(x, y),

x∈X y∈Y

x∈X

y∈Y

y∈Y x∈X

and using the fact that by (5.90) the far left and the far right limit are equal, we see that these are, in fact, all equalities; hence, sup F(xo , y) = F(xo , yo ) = inf F(x, yo ), x∈X

y∈Y

which implies that F(xo , y) ≤ sup F(xo , y) = F(xo , yo ) = inf F(x, yo ) ≤ F(x, yo ), x∈X

y∈Y

∀ x ∈ X, y ∈ Y,

which means that (xo , yo ) is a saddle point of F. 5.7.2 Proof of Lemma 5.5.1 (i) For the convexity of the value function, assume that F is convex in both variables x and y. Consider yi ∈ Y, i = 1, 2, and their convex combination λy1 + (1 − λ)y2 , λ ∈ [0, 1]. By the definition of V(yi ) in terms of an infimum (see (5.50)), we see that for every ϵ > 0 there exists a xi ∈ X such that V(yi ) ≤ F(xi , yi ) < V(yi ) + ϵ,

i = 1, 2.

(5.93)

246 | 5 Minimax theorems and duality Since F is convex in both variables, F(λx1 + (1 − λ)x2 , λy1 + (1 − λ)y2 ) ≤ λF(x1 , y1 ) + (1 − λ)F(x2 , y2 ) ≤ λV(y1 ) + (1 − λ)V(y2 ) + ϵ

(5.94)

where for the last inequality we used (5.93). Since V(y) = inf F(x, y) ≤ F(x, y), x∈X

for every (x, y) ∈ X × Y, clearly V(λy1 + (1 − λ)y2 ) ≤ F(λx1 + (1 − λ)x2 , λy1 + (1 − λ)y2 ), therefore, combining this with (5.94) leads to V(λy1 + (1 − λ)y2 ) ≤ λV(y1 ) + (1 − λ)V(y2 ) + ϵ, for every ϵ > 0 and passing to the limit as ϵ → 0+ yields the required convexity result. To prove the first of (5.52), note that by definition, V ⋆ (y⋆ ) = sup(⟨y⋆ , y⟩Y ⋆ ,Y − V(y)) = sup(⟨y⋆ , y⟩Y ⋆ ,Y − inf F(x, y)) y∈Y

=

x∈X

y∈Y

sup (⟨0, x⟩X ⋆ ,X + ⟨y , y⟩Y ⋆ ,Y − F(x, y)) = F ⋆ (0, y⋆ ), ⋆

(x,y)∈X×Y

where we also used the definition of F ⋆ (x⋆ , y⋆ ) =

sup (⟨x⋆ , x⟩X ⋆ ,X + ⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y)),

(x,y)∈X×Y

at x⋆ = 0. To prove the second of (5.52), note that by definition (see Definition 5.2.11 and Remark 5.2.12), V ⋆⋆ (0) = sup (⟨y⋆ , 0⟩Y ⋆ ,Y − V ⋆ (y⋆ )) = sup (−V ⋆ (y⋆ )) = sup (−F ⋆ (0, y⋆ )) y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

where for the last equality we used the first of (5.52). (ii) We will use the notation Fx for the function Fx : Y → ℝ ∪ {+∞} defined by fixing the value of x, i. e., by Fx (y) = F(x, y)., and Fx⋆ : Y ⋆ → ℝ ∪ {+∞}, for its convex conjugate function. Note that by the definition of the Lagrangian function Fx⋆ (y⋆ ) = sup(⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y)) =: −L(x, y⋆ ), y∈Y

while the biconjugate Fx⋆⋆ : Y → ℝ at y = 0 satisfies (using the definition of the biconjugate), Fx⋆⋆ (0) = sup (−Fx⋆ (y⋆ )) = sup (L(x, y⋆ )). y⋆ ∈Y ⋆

y⋆ ∈Y ⋆

5.7 Appendix | 247

It is straightforward to see that the function Fx : X → ℝ ∪ {+∞} is proper convex and lower semicontinuous. Therefore, by Proposition 5.2.15 we have that Fx⋆⋆ (0) = Fx (0), which leads to the conclusion that sup (L(x, y⋆ )) = F(x, 0) = F(x).

y⋆ ∈Y ⋆

We now recall the definition of F ⋆ : X ⋆ × Y ⋆ → ℝ ∪ {+∞} and set x⋆ = 0 to obtain F ⋆ (0, y⋆ ) = sup sup(⟨y⋆ , y⟩Y ⋆ ,Y − F(x, y)) x∈X ⋆ y∈Y

= sup sup(⟨y⋆ , y⟩Y ⋆ ,Y − Fx (y)) = sup(−L(x, y⋆ )), x∈X ⋆ y∈Y

x∈X

from which it follows that −F ⋆ (0, y⋆ ) = inf L(x, y⋆ ). x∈X

5.7.3 Proof of relation (5.61) Consider first an x ∈ C, i. e., φi (x) ≤ 0 for every i = 1, . . . , n. If yi⋆ > 0 for at least one i = 1, . . . , n, then the term yi yi⋆ is going to become large for yi > 0 in which case if definitely holds that φi (x) ≤ 0 < yi and the term IC(yi ) (x) = 0 so that supyi ∈ℝ (yi yi⋆ − IC(yi ) (x)) = +∞ in this case. If on the contrary yi⋆ < 0 for every i = 1, . . . , n, then each of the terms yi yi⋆ are going to become large for yi < 0. We have that φi (x) ≤ 0, but if we take yi too negative then φi (x) ≤ yi will not hold, leading to IC(yi ) (x) = +∞. If on the other hand yi is in the interval φi (x) ≤ yi ≤ 0, then IC(yi ) (x) = 0 and the major role in the maximization problem is played by yi⋆ yi . We therefore conclude that supyi ∈ℝ (yi yi⋆ −IC(yi ) (x)) = y⋆i φi (x) if yi⋆ < 0. Collecting all possible cases, we see that if x ∈ C, then −L(x, y⋆ ) = +∞; hence, L(x, y⋆ ) = −∞ for y⋆ ∈ ℝn+ , whereas −L(x, y⋆ ) = −φ0 (x) + ∑ni=1 yi⋆ φi (x) for y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ −ℝn+ . By changing variables to −y⋆ ∈ ℝn+ , we conclude that L(x, y⋆ ) = φ0 (x) + ∑i=1 yi⋆ φi (x) for y⋆ = (y1⋆ , . . . , yn⋆ ) ∈ ℝn+ , whereas L(x, y⋆ ) = −∞ for y⋆ ∈ ̸ ℝn+ . Consider next an x ∈ ̸ C. Then φi (x) > 0 for every i = 1, . . . , n. If yi⋆ > 0 for at least one i = 1, . . . , n, then the corresponding term yi⋆ yi will grow for positive yi . However, for large yi it is definitely true that φi (x) < yi so that IC(yi ) (x) = 0, and the dominating term is yi⋆ yi so that supyi ∈ℝ (yi yi⋆ − IC(yi ) (x)) = +∞. Suppose that yi⋆ < 0 for every i = 1, . . . , n. The term yi⋆ yi is going to become larger as yi decreases. As, φi (x) > 0 it is never true that φi (x) ≤ yi for yi < 0 so the term −IC(yi ) (x) = −∞, so we must restrict ourselves to the smallest possible positive value allowed for yi so that φi (x) ≤ yi holds. This is clearly φi (x) so that supyi ∈ℝ (yi yi⋆ −IC(yi ) (x)) = yi⋆ φi (x). With arguments similar as above (i. e., changing variables to −y⋆ ), we find exactly the same result as in the case where x ∈ C.

6 The calculus of variations An important part of nonlinear analysis is dedicated to the calculus of variations. The calculus of variations studies problems related to the minimization of integral functionals, defined in appropriate function spaces. It is a field which has been associated historically with many important applications in the physical sciences. Philosophical aspects aside, the calculus of variations is very useful, as important partial differential equations (PDEs) can be expressed as the first- order minimization conditions for suitable integral functionals. We may then infer qualitative or quantitative information on the behavior of the solutions of these equations from the theory of minimization of the corresponding functional. On the other hand, many important models ranging from economics or decision science to engineering, signal and image processing, etc., can be expressed in terms of integral functionals (see, e. g., [37] or [42]) so the study of minimization of integral functionals is a problem of interest in its own right (irrespectively of its connections with PDEs). In this chapter, we present certain important aspects of the theory of the calculus of variations, starting with an investigation of the Dirichlet functional and its connection with the Poisson equation, then providing results concerning lower semicontinuity of general integral functionals, existence of minimizers, their connection with the solution of the Euler–Lagrange PDE and the further regularity of these minimizers. Having developed a general theory for integral functionals, we present some applications of the calculus of variations to the study of semilinear elliptic PDEs, involving the Laplace operator and quasilinear elliptic PDEs, involving the p-Laplace operator. There are many authoritative monographs dedicated to the calculus of variations (see, e. g., [50, 66] or [67]).

6.1 Motivation One of the main motivations for the study of the calculus of variations is its connection with partial differential equations (PDEs). An important class of PDEs can be expressed as the first-order conditions for the minimization of integral functionals in Sobolev spaces. therefore, using the well-established theory for minimization of functionals in Banach spaces, one may infer useful information concerning the solvability and the properties of the solutions of these PDEs. This sums up the general strategy of utilization of the methodology of the calculus of variations to the study of PDEs. Example 6.1.1 (Laplace equation). Let 𝒟 ⊂ ℝd be a bounded domain with sufficiently smooth boundary 𝜕𝒟, let f : 𝒟 → ℝ be a given function and for an unknown function u : 𝒟 → ℝ, consider the Poisson equation −Δu(x) = f (x), u(x) = 0,

https://doi.org/10.1515/9783110647389-006

x ∈ 𝒟,

x ∈ 𝜕𝒟.

(6.1)

250 | 6 The calculus of variations Let us also define the functional F : X → ℝ, where X = W01,2 (𝒟) as ˆ 1󵄨 󵄨2 F(u) := ( 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 − f (x) u(x))dx, for every u ∈ W01,2 (𝒟). 2 𝒟

(6.2)

Using a simple extension of the arguments in Example 2.1.10, one can easily recognize (6.1) as the first-order condition for the minimization of the convex functional (6.2) in W01,2 (𝒟) (see also Example 2.2.14). Having at this point enough technical tools to treat the problem of minimizing F, and given the connection between solutions of the Poisson equation (6.1) and the minimizers of F, one is tempted to transfer this experience to the solvability and the properties of solutions for the PDE (6.1). Naturally, we may reverse our reasoning and given knowledge concerning the solvability of the PDE (6.1) try to infer knowledge on the properties of minima of the functional F. ◁ The discussion in Example 6.1.1 can be extended to other equations, and importantly to nonlinear elliptic equations. The following example motivates this. Example 6.1.2 (Nonlinear elliptic equations). Within the general framework of Example 6.1.1, we now start from the opposite end and consider a functional F : X → ℝ, where X is an appropriate Sobolev space on 𝒟, defined as ˆ f (x, u(x), ∇u(x))dx, (6.3) F(u) := 𝒟

for every u ∈ X, where f : 𝒟 × ℝ × ℝd → ℝ is a function assumed to be sufficiently smooth in all its variables. We will use the notation f (x, y, z) to denote the values of the function f , where x ∈ 𝒟 ⊂ ℝd , y = u(x) is the value of the function u : 𝒟 → ℝ at the selected point x ∈ 𝒟 and z = ∇u(x) ∈ ℝd , the gradient of u at the selected point 𝜕 𝜕 x ∈ 𝒟. We will also denote by 𝜕y f : 𝒟 × ℝ × ℝd → ℝ and 𝜕z f = ∇z f : 𝒟 × ℝ × ℝd → ℝd , the derivatives of the function f with respect to the variables y and z, respectively. We now perform formal calculations to calculate the Gâteaux derivative of F and try to characterize a critical point of F. Choosing any function v ∈ X, standard calculations yield ˆ F(u + ϵv) − F(u) 𝜕 𝜕 = ( f (x, u(x), ∇u(x)) v(x) + f (x, u(x), ∇u(x)) ⋅ ∇v(x))dx + O(ϵ), ϵ 𝜕y 𝜕z 𝒟 where we have used the Taylor expansion of the function f and by O(ϵ) we denote terms of order ϵ or higher. Integrating by parts and assuming that v vanishes on 𝜕𝒟 we obtain ˆ F(u + ϵv) − F(u) 𝜕 𝜕 = [ f (x, u(x), ∇u(x)) − ∇ ⋅ ( f (x, u(x), ∇u(x)))]v(x)dx + O(ϵ). ϵ 𝜕z 𝒟 𝜕y Taking the limit as ϵ → 0, we formally obtain ˆ 𝜕 𝜕 DF(u; v) := [ f (x, u(x), ∇u(x)) − ∇ ⋅ ( f (x, u(x), ∇u(x)))]v(x)dx, 𝜕z 𝒟 𝜕y

(6.4)

6.2 Warm up: variational theory of the Laplacian

| 251

so that by the argumentation of Theorem 2.2.13, and the fact that v is arbitrary the critical point(s) u of F satisfy the equation 𝜕 𝜕 f (x, u(x), ∇u(x)) − ∇ ⋅ ( f (x, u(x), ∇u(x))) = 0. 𝜕y 𝜕z

(6.5)

This is a PDE, called the Euler–Lagrange equation. Depending on the choice of the function f , we may obtain a large variety of nonlinear PDEs as the Euler–Lagrange equation. Of course, the above formal calculations should be made rigorous, especially the parts in which interchange of limits and integrals are involved, and this requires specific smoothness conditions on the function f and the proper choice for the function space X. Then, once the PDE in consideration has been recognized as the Euler–Lagrange equation for a functional F it remains to check whether the functional in question indeed acquires a minimum uo , which must be the solution of the relevant Euler–Lagrange equation; hence, of the original PDE. ◁ This chapter addresses these questions, but in a manner which is rather more general in scope, than the original PDE motivation. In particular, it addresses the problem of well posedness of minimization problems for a wide class of integral functionals F : X → ℝ of the form F(u) :=

ˆ 𝒟

f (x, u(x), ∇u(x))dx,

(6.6)

where 𝒟 ⊂ ℝd is an open bounded domain of sufficiently smooth boundary, u : 𝒟 → ℝ is a function such that the integral (6.6) is well-defined, ∇u = ( 𝜕x𝜕 u, . . . , 𝜕x𝜕 u) its gradi1

d

ent, f : 𝒟 × ℝ × ℝd → ℝ is a given function, focusing on the case where X = W 1,p (𝒟) or X = W01,p (𝒟). As already mentioned, functionals of the form (6.6) are connected with nonlinear PDEs, but their study is of interest in its own right, as integral functionals of this kind arise in a number of interesting applications.

6.2 Warm up: variational theory of the Laplacian We start our treatment of the calculus of variations with the study of the Dirichlet and the Rayleigh quotient functionals which are closely connected to the Laplacian and with linear problems such as the Poisson equation and the Laplace eigenvalue problem. Even though, these problems are linear problems they are not to be frowned upon in a book on nonlinear analysis, as they provide an important background and an indispensable toolbox in the study of nonlinear problems.

252 | 6 The calculus of variations 6.2.1 The Dirichlet functional and the Poisson equation Let 𝒟 ⊂ ℝd be an open bounded and connected set with smooth enough boundary.1 Proposition 6.2.1. The Dirichlet functional FD : W01,2 (𝒟) → ℝ defined by FD (u) = 1 ´ |∇u(x)|2 dx is strictly convex and weakly sequentially lower semicontinuous. 2 𝒟 Proof. Convexity follows easily by the convexity of the function z 󳨃→ |z|2 , z ∈ ℝd , and strict convexity follows by the identity 󵄨󵄨 󵄨2 󵄨 󵄨2 󵄨󵄨t∇u1 (x) + (1 − t)∇u2 (x)󵄨󵄨󵄨 + t(1 − t)󵄨󵄨󵄨∇u1 (x) − ∇u2 (x)󵄨󵄨󵄨 󵄨 󵄨2 󵄨 󵄨2 = t 󵄨󵄨󵄨∇u1 (x)󵄨󵄨󵄨 + (1 − t)󵄨󵄨󵄨∇u2 (x)󵄨󵄨󵄨 , which is valid a. e. in 𝒟 for every t ∈ [0, 1], upon integration. Weak lower semicontinuity is extremely simple for this case (because of its special form), so one could argue directly by noting that if un ⇀ u in W01,2 (𝒟), then it is bounded, and since W01,2 (𝒟) is a Hilbert space there exists a subsequence (denoted the same) such that un ⇀ u and ∇un ⇀ ∇u in L2 (𝒟), for some u ∈ W01,2 (𝒟). Along this sequence write (for a. e. x ∈ 𝒟), 󵄨󵄨 󵄨2 󵄨 󵄨2 󵄨󵄨∇un (x)󵄨󵄨󵄨 = 󵄨󵄨󵄨∇un (x) − ∇u(x) + ∇u(x)󵄨󵄨󵄨 󵄨 󵄨2 󵄨 󵄨2 = 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 + 2(∇un (x) − ∇u(x)) ⋅ ∇u(x) + 󵄨󵄨󵄨∇un (x) − ∇u(x)󵄨󵄨󵄨 󵄨 󵄨2 ≥ 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 + 2(∇un (x) − ∇u(x)) ⋅ ∇u(x), so that, upon integration, FD (un ) ≥ FD (u) +

ˆ 𝒟

(∇un (x) − ∇u(x)) ⋅ ∇u(x)dx.

(6.7)

´ Since ∇un ⇀ ∇u in L2 (𝒟), we have that limn 𝒟 (∇un (x) − ∇u(x)) ⋅ ∇u(x)dx → 0, so that taking the limit inferior on both sides of (6.7) we conclude that lim infn FD (un ) ≥ FD (u); hence, the required lower semicontinuity result. Remark 6.2.2. Clearly, the strategy used for the above lower semicontinuity result cannot be easily extended to more general integral functionals; however, convexity may prove helpful. In this general approach, we will employ the Fatou lemma for proving lower semicontinuity, first passing from weak to strong convergence using the Mazur lemma and then extracting an appropriate subsequence which converges a. e. We will study this approach in detail in Section 6.3. 1 Typically Lipschitz boundary is sufficient, unless more regularity is required, which will be stated explicitly.

6.2 Warm up: variational theory of the Laplacian

| 253

We now consider the problem of minimization of perturbations of the Dirichlet functional over X = W01,2 (𝒟). Proposition 6.2.3. Consider any f ∈ L2 (𝒟) and define the functional F : W01,2 (𝒟) → ℝ, by ˆ F(u) := FD (u) − f (x)u(x)dx. (6.8) 𝒟

This functional has a unique minimizer in W01,2 (𝒟), which can be obtained as the solution of the Poisson equation

−Δu = f ,

u = 0,

in 𝒟,

understood in the weak sense, i. e., ˆ ˆ f (x)v(x)dx, ∇u(x) ⋅ ∇v(x)dx = 𝒟

𝒟

(6.9)

on 𝜕𝒟,

∀ v ∈ W01,2 (𝒟).

(6.10)

Proof. The existence of a minimizer follows by the arguments of the direct method of the calculus of variations (Theorem 2.2.5). Consider a minimizing sequence {un : n ∈ ℕ} for F, which is bounded in W01,2 (𝒟) because of the Poincaré inequality. Indeed, along the minimizing sequence, ˆ ˆ 󵄨󵄨 󵄨󵄨2 F(un ) = f (x)un (x)dx ≤ c, (6.11) 󵄨󵄨∇un (x)󵄨󵄨 dx − 𝒟

𝒟

for some appropriate constant c, and since ˆ 󵄨󵄨ˆ 󵄨󵄨 󵄨󵄨 󵄨2 󵄨 󵄨 F(un ) ≥ 󵄨󵄨∇un (x)󵄨󵄨󵄨 dx − 󵄨󵄨󵄨 f (x)un (x)dx󵄨󵄨󵄨 󵄨󵄨 𝒟 󵄨󵄨 𝒟 ˆ ˆ 1/2 ˆ 1/2 󵄨󵄨 󵄨󵄨2 󵄨󵄨 󵄨󵄨2 󵄨󵄨 󵄨󵄨2 ≥ 󵄨󵄨∇un (x)󵄨󵄨 dx − ( 󵄨󵄨un (x)󵄨󵄨 dx) ( 󵄨󵄨f (x)󵄨󵄨 dx) 𝒟 𝒟 𝒟 ˆ ˆ ˆ ϵ2 1 󵄨󵄨 󵄨󵄨2 󵄨󵄨 󵄨󵄨 󵄨󵄨2 󵄨󵄨2 ≥ ∇u (x) dx − u (x) dx + 󵄨󵄨 n 󵄨󵄨 󵄨󵄨 n 󵄨󵄨 󵄨󵄨f (x)󵄨󵄨 dx 2 2 2ϵ 𝒟 𝒟 𝒟 ˆ ˆ ϵ2 1 󵄨󵄨 󵄨󵄨2 󵄨󵄨 󵄨󵄨2 2 ≥ (c𝒫 − ) 󵄨󵄨un (x)󵄨󵄨 dx + 2 󵄨f (x)󵄨󵄨 dx, 2 2ϵ 𝒟 󵄨 𝒟

(6.12)

where ϵ > 0 is arbitrary, and we used subsequently the Cauchy–Schwarz inequality and the Poincaré inequality. Combining (6.11) and (6.12), we see that ˆ ˆ 1 ϵ2 󵄨 󵄨2 󵄨󵄨 󵄨󵄨2 (c2𝒫 − ) 󵄨󵄨󵄨un (x)󵄨󵄨󵄨 dx ≤ c − 2 󵄨󵄨f (x)󵄨󵄨 dx, 2 2ϵ 𝒟 𝒟 2

and choosing ϵ so that ϵ2 < c2𝒫 , we have that {un : n ∈ ℕ} is bounded in L2 (𝒟), and returning to (6.11) we conclude that {∇un : n ∈ ℕ} is also bounded in L2 (𝒟), therefore, {un : n ∈ ℕ} is bounded in W01,2 (𝒟).

254 | 6 The calculus of variations Since {un : n ∈ ℕ} is bounded in W01,2 (𝒟), which is a Hilbert space, there exists a subsequence {unk : k ∈ ℕ}, and a u ∈ W01,2 (𝒟), such that unk ⇀ u in W01,2 (𝒟). This weak limit is the required minimizer, by the weak lower sequential semicontinuity of the functional F (which is a linear perturbation of FD whose weak lower sequential semicontinuity has been established in Proposition 6.2.1). The Gâteaux derivative of the functional F is DF(u) = −Δu − f , (see, e. g., Example 2.1.10), so that the first-order condition becomes (6.9) or its equivalent weak form (6.10). The equivalence between the Poisson equation and its weak form arises by multiplying (6.9) with a sufficiently smooth test function, integrating by parts and recalling the density of smooth functions in the Sobolev space W01,2 (𝒟) (recall Definition 1.5.6). Uniqueness of the minimizer follows by strict convexity, however, this does not necessarily imply the uniqueness of solutions of the Poisson equation (as a critical point is not necessarily a minimum). Having obtained the results concerning the existence of a unique minimizer to the functional F defined in (6.8) and its connection with the Poisson equation (6.9), we may obtain important information on the Poisson PDE. Proposition 6.2.4. The Poisson equation (6.9) admits a unique solution u ∈ W01,2 (𝒟), for every f ∈ L2 (𝒟). Upon defining the linear continuous operator, A := −Δ : W01,2 (𝒟) → (W01,2 (𝒟))⋆ = W −1,2 (𝒟), by ⟨Au, v⟩L2 (𝒟) = ⟨−Δu, v⟩L2 (𝒟) := ⟨∇u, ∇v⟩L2 (𝒟) ,

∀ v ∈ W01,2 (𝒟),

it holds that the inverse operator (−Δ)−1 : L2 (𝒟) → L2 (𝒟), is well-defined, single valued, self-adjoint, positive definite and compact. Proof. The existence of solutions to the Poisson equation (6.9) has already been shown in Proposition 6.2.3. To show uniqueness, assume two solutions u1 , u2 of (6.9) and note that w := u1 − u2 solves Δw = 0 in 𝒟 with homogeneous Dirichlet conditions. Take the weak form of (6.9) with w as a test function. An integration by parts implies that ‖∇w‖L2 (𝒟) = 0, and using the Poincaré inequality and the argumentation of Example 1.5.14 we conclude that w = 0; hence, we have uniqueness. The inverse operator T := (−Δ)−1 is connected to the variational solution of the Poisson problem (6.9), in the sense that for any f ∈ L2 (𝒟), we define Tf = u where u is the solution of problem (6.9) for the right-hand side f . By the above arguments, c T is a single valued linear operator from L2 (𝒟) to W01,2 (𝒟) 󳨅→ L2 (𝒟). This operator is bounded, as one can easily infer by using the weak form (6.10) setting as test function v = Tf and using the Cauchy–Schwarz inequality on the right-hand side and the Poincaré inequality on the left-hand side (details are left as an exercise). Using the c boundedness of this operator and the compact embedding of W01,2 (𝒟) 󳨅→ L2 (𝒟) (guaranteed by the Rellich–Kondrachov embedding, see Theorem 1.5.11), we conclude that T : L2 (𝒟) → L2 (𝒟) is a compact operator. The remaining properties follow easily from the weak form of (6.9).

6.2 Warm up: variational theory of the Laplacian

| 255

The compactness of the inverse operator (−Δ)−1 : L2 (𝒟) → L2 (𝒟) is of fundamental importance, as one may now resort to the well-understood Fredholm theory for compact operators in Hilbert spaces and their spectral properties (see Theorem 1.3.13, in conjunction with Theorem 1.3.12) in order to fully specify the spectral properties of the Laplacian operator −Δ. This follows from the simple observation that the eigenvalues of the operator A := −Δ are clearly related to the eigenvalues of T := (−Δ)−1 , since any λ ∈ ℝ is an eigenvalue of A if and only if λ−1 is an eigenvalue of T, and since T is a compact operator there exists abundant useful information concerning its eigenvalues. We close this section with some important properties of the solution of the Poisson equation, related to the maximum principle for the Laplace operator (see, e. g., [36] or [63]). Proposition 6.2.5 (Weak maximum principle). (i) Suppose u ∈ W 1,2 (𝒟) satisfies −Δu ≥ 0 in the weak sense, i. e., that ⟨(−Δ)u, v⟩(W 1,2 (𝒟))⋆ ,W 1,2 (𝒟) ≥ 0, 0

0

∀ v ∈ W01,2 (𝒟), v ≥ 0 a. e.

If u− := max(−u, 0) ∈ W01,2 (𝒟), then u ≥ 0 a. e. (ii) Suppose u ∈ W 1,2 (𝒟) satisfies −Δu ≤ 0 in the weak sense, i. e., that ⟨(−Δ)u, v⟩(W 1,2 (𝒟))⋆ ,W 1,2 (𝒟) ≤ 0, 0

0

∀ v ∈ W01,2 (𝒟), v ≥ 0 a. e.

If u+ := max(u, 0) ∈ W01,2 (𝒟), then u ≤ 0 a. e. Proof. We only prove (i) as (ii) follows from (i) by changing u to −u. Recall (see Example 1.5.2) that if u ∈ W 1,2 (𝒟) then u− ∈ W 1,2 (𝒟) also, and that ´ − ∇u = −∇u1u 0. (ii) Suppose u ∈ W 1,2 (𝒟) ∩ C(𝒟) satisfies −Δu ≤ 0 in the weak sense, i. e., that ⟨(−Δ)u, v⟩(W 1,2 (𝒟))⋆ ,W 1,2 (𝒟) ≤ 0, 0

0

∀ v ∈ W01,2 (𝒟), v ≥ 0.

If u+ = max(u, 0) ∈ W01,2 (𝒟) and u ≢ 0, then u < 0. Proof. We only prove (i) as (ii) follows from (i) be changing u to −u. The assumption that 𝒟 is connected plays an important role in the proof. We recall the fundamental topological notion of connectedness: Since 𝒟 is connected, the only subset of 𝒟 which is at the same time open and closed is 𝒟 itself (see, e. g., [59]). By the weak maximum principle (Proposition 6.2.5), it holds that u ≥ 0 in 𝒟. Define the set A = {x ∈ 𝒟 : u(x) > 0} which since u ∈ C(𝒟) and u ≢ 0 is open. If we show that is it also closed, then A = 𝒟 and we are done. Consider the function v : 𝒟 → ℝ, defined by v(x) = |x|−β −R−β . It is straightforward (but requires tedius algebra) to show that choosing the parameter β appropriately, the function v satisfies −Δv ≤ 0 in the annulus A0,ρ,R := {x ∈ ℝd : ρ < |x| < R}, and vanishes on its outer boundary. To show that A is closed, consider any sequence {xn : n ∈ ℕ} ⊂ A such that xn → xo ∈ 𝒟. Choose R such that B(xo , R) ⊂ 𝒟 and choose N large enough so that |xN − xo | < R. Since xN ∈ A, we have that u(xN ) > 0, which implies that we may find ρ ∈ (0, R) and ϵ > 0 such that u(x) ≥ ϵ for |x − xN | = ρ. Consider the annulus AxN ,ρ,R = {x ∈ ℝd : ρ < |x − xN | < R}, centered at xN and using the function v defined above shifted by xN , define the function w : AxN ,ρ,R → ℝ by w(x) = u(x) − ϵρβ v(x − xN ), which satisfies w ∈ W 1,2 (AxN ,ρ,R ) ∩ C(AxN ,ρ,R ) and −Δw ≥ 0. By the properties of v and since u ≥ 0, we see that w(x) ≥ 0 for |x − xN | = R; hence, w− (x) = 0 on the outer boundary of AxN ,ρ,R , while w ≥ 0 on AxN ,ρ,R , similarly, for the inner boundary. Applying the weak maximum principle for w in AxN ,ρ,R , we conclude that w(x) ≥ 0 for every x ∈ AxN ,ρ,R , therefore, u(xo ) ≥ ϵρβ v(xo − xN ) > 0 so that x ∈ A. Example 6.2.8. Minor modifications of the above arguments can be used to establish similar results for the equation −Δu + λu = f ,

in 𝒟,

6.2 Warm up: variational theory of the Laplacian

u = 0,

| 257

on 𝜕𝒟,

for λ > 0 or even for λ > −λ1 where λ1 := infu∈W 1,2 0

‖∇u‖2 2

L (𝒟)

‖u‖2 2

L (𝒟)

which are related to the

minimization of the integral functional ˆ ˆ ˆ λ 1 󵄨2 󵄨󵄨 u(x)2 dx − f (x)u(x)dx, F(u) = 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + 2 𝒟 2 𝒟 𝒟 on W01,2 (𝒟). The details are left as an exercise.

◁

The strong maximum principle provides important information on solutions of either the Poisson equation or of inequalities related to the Poisson equation but requires the additional property that u ∈ C(𝒟) (or even in some cases that u ∈ C(𝒟)) which we cannot know a priori as all we have established so far, concerning the solvability of the problem in Proposition 6.2.4 is that u ∈ W01,2 (𝒟). However, as we shall see in the next section solutions of Poisson-type equations enjoy important regularity properties. 6.2.2 Regularity properties for the solutions of Poisson-type equations Using the variational formulation, we have shown the existence of weak solutions for equations of the form −Δu + λu = f ,

(6.13)

with Dirichlet boundary conditions for suitable values of λ (see Example 6.2.8). We now consider the question of regularity, i. e., can u get any better than W01,2 (𝒟)? The answer is yes, as long as f enjoys sufficient regularity itself. We start with the following important property of the Laplace operator, which essentially states that if the Laplacian of a function is k times differentiable (in the weak sense) the function itself gains two more weak derivatives, i. e., is k + 2 times differentiable. 1,2 Proposition 6.2.9. Suppose that 𝒟 ⊂ ℝd is open and bounded and u ∈ Wloc (𝒟). k,2 k+2,2 (i) If Δu ∈ Wloc (𝒟) for some k > 0, then u ∈ Wloc (𝒟). (ii) If Δu ∈ C ∞ (𝒟), then u ∈ C ∞ (𝒟).

Proof. The idea of the proof is sketched. To prove (i), we first note that this is true in the whole of ℝd , a fact that can be easily shown using the Fourier transform. We then use a localization (or rather extension) argument. To this end, consider any choice of sets 𝒟o and 𝒟o󸀠 such that 𝒟o󸀠 ⊂⊂ 𝒟o ⊂⊂ 𝒟, and consider a function ψ ∈ Cc∞ (ℝd ) such that ψ ≡ 1 on 𝒟o󸀠 and supp ψ ⊂ 𝒟o . Define the function v = ψu which is a function k,2 on ℝd . One may calculate Δv = Δ(ψu) = ψΔu + uΔψ + 2∇u ⋅ ∇ψ ∈ Wloc (𝒟) by the

258 | 6 The calculus of variations properties of Δu, u and ψ (and an integration by parts argument). But then by the first step v = ψu ∈ W k+2,2 (ℝd ), and that combined with the fact that 𝒟o and 𝒟o󸀠 are arbitrary, k+2,2 (𝒟). To prove (ii), we use an induction argument and the fact that we get that u ∈ Wloc ∞ m,2 C (𝒟) = ⋂m>0 W (𝒟). k,2 The above result can be readily used to show, e. g., that if u ∈ Wloc (𝒟) is a solution k+2,2 k,2 of (6.13) for f ∈ Wloc (𝒟), then u ∈ Wloc (𝒟). A similar argument as the above allows m,p us to conclude that for any solution of (6.13), if f ∈ Wloc (𝒟) (f ∈ C ∞ (𝒟), resp.) and n,p m+2,p u ∈ Wloc (𝒟) for some m ≥ 0, n ∈ ℤ, p ∈ (1, ∞) then u ∈ Wloc (𝒟) (u ∈ C ∞ (𝒟), resp.). The following theorem, in the spirit of a more general result due to Stampacchia (see, e. g., [104]) provides Lp estimates for the solution of (6.13).

Theorem 6.2.10 (Lp estimates). Assume that d > 2 and that f ∈ Lp (𝒟) for some p > d2 . Then any weak solution of (6.13), with λ > 0, is in L∞ (𝒟) and ‖u‖L∞ (𝒟) ≤ c‖f ‖Lp (𝒟) for some constant c (which does not depend on u and f ). Proof. To show that u ∈ L∞ (𝒟), it suffices to show that |u| ≤ c, a. e., for some appropriate constant c > 0. This in turn is equivalent to showing that there exists a constant c > 0 such that μℒd ({x ∈ 𝒟 : |u(x)| > c}) = 0. The proof proceeds in 6 steps: 1. For any k ∈ ℝ, let us define Ak := {x ∈ 𝒟 : |u(x)| > k}, and set ψ(k) := μℒd (Ak ) =: |Ak |. Clearly, for any k1 ≤ k2 , we have that Ak2 ⊂ Ak1 so that ψ(k2 ) ≤ ψ(k1 ), i. e., ψ is nonincreasing. We will show that there exists a k ∗ such that ψ(k ∗ ) = 0. 2. We recall an inequality due to Stampacchia, according to which if ϕ : ℝ+ → ℝ+ is a nonincreasing function such that ϕ(k2 ) ≤ c

ϕ(k1 )δ , (k2 − k1 )γ

for every 0 < k1 < k2 ,

(6.14)

where δ > 1, γ > 0, then there exists k ⋆ > 0 such that ϕ(k ⋆ ) = 0, and in fact, ϕ(k ⋆ ) = 0,

1

for k ⋆ = c γ ϕ(0)

δ−1 γ

δ

2 δ−1 .

(6.15)

The proof of this inequality proceeds by considering the sequence kn = k ⋆ (1 − 2−n ) γ for which one can show by induction that ϕ(kn ) ≤ ϕ(0)2− (δ−1) n , and then using monotonicity we get γ

0 ≤ ϕ(k ⋆ ) ≤ lim inf ϕ(kn ) ≤ lim ϕ(0)δ−1 2− (δ−1) n = 0, n

n

and the result follows. 3. We will try to show that the function ψ defined in step 1 satisfies a Stampacchialike inequalty of the form (6.14) so that applying the results of step 2 we obtain by (6.15) that ψ vanishes for some sufficiently large k ∗ . To show that ψ satisfies an inequality of the form (6.14), we proceed as follows: We define the cutoff function Gk (s) = (s − k)1s≥k + (s + k)1s≤−k and consider the composition vk = Gk ∘ u, which vanishes on Ack . This function is the composition of a globally

6.2 Warm up: variational theory of the Laplacian

| 259

Lipschitz function with a function in W01,2 (𝒟); hence, vk ∈ W01,2 (𝒟) with ∇vk = 1Ak ∇u. We will use vk as test function in the weak formulation of (6.13), in order to gain information on the behavior of the solution u on the set Ak . Since vk ≡ 0 on Ack and ˆ ˆ ˆ ∇u(x) ⋅ ∇vk (x)dx = ∇u(x) ⋅ ∇u(x)dx = ∇vk (x) ⋅ ∇vk (x)dx, Ak

𝒟

Ak

the weak form of (6.13) with the chosen test function yields ˆ ˆ ˆ 󵄨󵄨 󵄨2 u(x)vk (x)dx = f (x)vk (x)dx. 󵄨󵄨∇vk (x)󵄨󵄨󵄨 dx + λ Ak

Ak

We observe that ˆ ˆ u(x)vk (x)dx = Ak

{u≤−k}

Ak

u(x)(u(x) + k)dx +

ˆ {u≥k}

u(x)(u(x) − k)dx ≥ 0,

so that since λ > 0, the weak form (6.16) implies ˆ ˆ 󵄨󵄨 󵄨2 f (x)vk (x)dx. 󵄨󵄨∇vk (x)󵄨󵄨󵄨 dx ≤ Ak

(6.16)

(6.17)

Ak

We will use the estimate (6.17) to obtain the required estimates for the measure of the set Ak . 4. Since we have information on the value of vk on Ak but not on ∇vk , we will try to substitute the term involving the integral of |∇vk |2 on Ak in (6.17) with a term involving the integral of a suitable power of vk on the same set. To this end, we will use the 2d Sobolev embedding Theorem 1.5.11 according to which Ls2 (𝒟) 󳨅→ W01,2 (𝒟) for s2 = d−2 . Hence, there exists a constant c1 > 0 such that ‖v‖Ls2 (𝒟) ≤ c1 ‖v‖W 1,2 (𝒟) , which upon 0 rearrangement yields ˆ

ˆ

Ak

󵄨󵄨 󵄨2 󵄨󵄨∇vk (x)󵄨󵄨󵄨 dx ≥ c2 (

Ak

2/s2

󵄨󵄨 󵄨s2 󵄨󵄨vk (x)󵄨󵄨󵄨 dx)

.

(6.18)

for a suitable constant c2 > 0. Combining (6.18) with (6.17) we have that ˆ (

Ak

2/s2

󵄨󵄨 󵄨s2 󵄨󵄨vk (x)󵄨󵄨󵄨 dx)

≤ c3

ˆ Ak

f (x)vk (x)dx,

(6.19)

for a suitable constant c3 > 0. Estimate (6.19) involves only vk , but it would be better suited for our purposes if it involved only the Ls2 (Ak ) norm of vk . To manage that, we use Hölder’s inequality on the right-hand side of (6.19) to retrieve the Ls2 (Ak ) norm for 2d the conjugate exponent for vk . Using this strategy in (6.19) and denoting by s⋆2 = d+2 s2 , we have that ˆ ˆ 2/s2 1/s⋆2 ˆ 1/s2 󵄨 󵄨󵄨s2 󵄨 󵄨󵄨s∗2 󵄨 󵄨s 󵄨 󵄨 ( 󵄨󵄨vk (x)󵄨󵄨 dx) ≤ c3 ( 󵄨󵄨f (x)󵄨󵄨 ) ( 󵄨󵄨󵄨vk (x)󵄨󵄨󵄨 2 dx) , Ak

Ak

Ak

260 | 6 The calculus of variations where upon dividing both sides with the Ls2 (Ak ) norm for vk , and raising to the power s2 we obtain the estimate ˆ Ak

s2 ˆ s⋆ 2 󵄨󵄨s∗2 󵄨 󵄨s2 󵄨󵄨 󵄨 󵄨 󵄨󵄨vk (x)󵄨󵄨 dx ≤ c4 ( 󵄨󵄨f (x)󵄨󵄨 dx) ,

(6.20)

Ak

for an appropriate constant c4 > 0, where

s2 s⋆2

=

d+2 . d−2

Note that the right-hand side of estimate (6.20) allows us to involve |Ak |, the measure of the set Ak into the game. Expressing |f |s2 = 1 |f |s2 , and using the Hölder inp p ⋆ equality once more with exponents p1 = p−s ⋆ and p1 = s⋆ , we have 2

ˆ Ak

ˆ

⋆

p−s2 󵄨󵄨 󵄨s⋆ 󵄨󵄨f (x)󵄨󵄨󵄨 2 dx ≤ |Ak | p (

≤ |Ak |

Ak

ˆ

p−s⋆ 2 p

(

𝒟

2

󵄨󵄨 󵄨p 󵄨󵄨f (x)󵄨󵄨󵄨 dx) 󵄨󵄨 󵄨p 󵄨󵄨f (x)󵄨󵄨󵄨 dx)

s⋆ 2 p

(6.21)

s⋆ 2 p

= |Ak |

p−s⋆ 2 p

s⋆

‖f ‖Lp2 (𝒟) .

Combining (6.20) and (6.21), we obtain the estimate ˆ Ak

󵄨󵄨 󵄨s2 󵄨󵄨vk (x)󵄨󵄨󵄨 dx ≤ c5 |Ak |

s2 (p−s⋆ 2) s⋆ p 2

,

(6.22)

where c5 > 0 is a suitable constant depending on ‖f ‖Lp (𝒟) (which by assumption is finite). 5. We now show that we may use the estimate (6.22) to obtain a Stampacchia like inequality for the function ψ, defined by ψ(k) = |Ak | for every k ∈ ℝ. Consider any k1 < k2 . Clearly, Ak2 ⊂ Ak1 and |vk1 | := |Gk1 ∘ u| ≥ k2 − k1 on Ak2 . This allows for the estimate ˆ ˆ |vk1 |s2 dx ≥ |vk1 |s2 dx ≥ (k2 − k1 )s2 |Ak2 |. (6.23) Ak1

Ak2

Writing (6.22) for the level k = k1 and combining the result with (6.23), we obtain the following comparison between the measures of the superlevel sets at two different values of k, as (k2 − k1 ) |Ak2 | ≤ c5 |Ak1 | s2

s2 (p−s⋆ 2) s⋆ p 2

.

(6.24)

We therefore conclude that the measure of the superlevel sets of the solution of (6.13) at two different levels k1 < k2 satisfies the inequality (6.24). s (p−s⋆ ) 2d 6. Upon defining δ := 2 s⋆ p 2 = p(d+2)−2d , γ = s2 = d−2 and ψ(k) := |Ak |, we express (d−2)p (6.24) as

2

ψ(k2 ) ≤ c5

ψ(k1 )δ , (k2 − k1 )γ

∀ k1 < k2 ,

| 261

6.2 Warm up: variational theory of the Laplacian

which is a Stampacchia like inequality. Since by the choice of p > d2 and d > 2, we have δ > 1 and γ > 0, by step 1 we have the existence of a k ⋆ such that ψ(k) = |Ak | = 0 for every k > k ⋆ , which is the required result that allows us to obtain the L∞ bounds. Note that the L∞ bound depends on the constant c5 which in turn depends on ‖f ‖Lp (𝒟) . Note that since we are in a bounded domain 𝒟, the L∞ (𝒟) bound implies Lr (𝒟) bounds for every r > 0. This remark leads to the following proposition. Proposition 6.2.11. Let d > 2 and assume that f ∈ Lp (𝒟) for some p > weak solution u of (6.13), with λ > 0, satisfies u ∈ L∞ (𝒟) ∩ C(𝒟).

d . 2

Then any

Proof. We approximate any f ∈ Lp (𝒟) with a sequence {fn : n ∈ ℕ} ⊂ Cc∞ (𝒟) such that fn → f in Lp (𝒟) (see Theorem 1.4.7), and we define un to be the solution of −Δun + λun = fn . By Proposition 6.2.9, we have that un ∈ Cc∞ (𝒟) and by linearity setting vn = u − un and fn̄ = f −fn ∈ Lp (𝒟), we see that vn satisfies −Δvn +λvn = fn̄ . Applying Theorem 6.2.10 to the last equation, we see that ‖vn ‖L∞ (𝒟) ≤ c‖f − fn ‖Lp (𝒟) so that passing to the limit as n → ∞ we have that ‖u − un ‖L∞ (𝒟) → 0 and the result follows. Similar estimates can also be obtained for unbounded domains 𝒟, at the expense of more technical arguments (see, e. g., [36] and references therein).

6.2.3 Laplacian eigenvalue problems We now consider two related minimization problems: min

ˆ

u∈W01,2 (𝒟)

𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx,

ˆ

subject to

𝒟

󵄨󵄨 󵄨2 󵄨󵄨u(x)󵄨󵄨󵄨 dx = 1,

(6.25)

and ´ min

u∈W01,2 (𝒟)\{0}

FR (u) :=

min

u∈W01,2 (𝒟)\{0}

´𝒟

|∇u(x)|2 dx

𝒟

|u(x)|2 dx

.

(6.26)

The first problem is a constrained optimization problem, which is related to the minimization of the Dirichlet functional on the surface of the unit ball in L2 (𝒟), denoted by SL2 (𝒟) . The second problem is an unconstrained optimization problem over the whole of W01,2 (𝒟)\{0}, of the functional FR : W01,2 (𝒟)\{0} → ℝ, defined by FR (u) =

´ |∇u(x)|2 dx ´𝒟 , 2 𝒟 |u(x)| dx

called the Rayleigh–Courant–Fisher functional. To see the connection between the two problems, note that for any u ∈ W01,2 (𝒟) \ ´ {0}, it holds that w := ‖u‖ u2 ∈ W01,2 (𝒟) ∩ SL2 (𝒟) and FR (u) = FR (w) = 𝒟 |∇w(x)|2 dx. L (𝒟)

This means that (6.25) and (6.26) share the same solutions. Furthermore, it is interesting to note that the first-order condition for local minima of any of these problems

262 | 6 The calculus of variations reduces to an eigenvalue problem for the Laplacian, −Δu = λu, u = 0,

in 𝒟,

or its equivalent weak form ˆ ˆ ∇u(x) ⋅ ∇v(x)dx = λ u(x)v(x)dx, 𝒟

(6.27)

on 𝜕𝒟,

𝒟

∀ v ∈ W01,2 (𝒟),

(6.28)

where both the value of λ ∈ ℝ (eigenvalue) and the function u ∈ W01,2 (𝒟) \ {0} (eigenfunction) are to be determined. Based on our observation in the previous section that (−Δ)−1 : L2 (𝒟) → L2 (𝒟) is a compact operator (Proposition 6.2.4) and that, if μn is an eigenvalue of (−Δ), then λn = μ−1 n is an eigenvalue of −Δ, one can, resorting to the spectral theory of compact operators on Hilbert spaces (see Theorems 1.3.12 and 1.3.12), show that the spectrum is countable so that there exists a sequence of eigenvalues λn → ∞. Importantly, recalling formulation (6.26) we see that the first eigenvalue λ1 > 0 has a variational representation as ´ λ1 =

min

u∈W01,2 (𝒟)\{0}

´𝒟

|∇u(x)|2 dx

𝒟

|u(x)|2 dx

=

min

u∈W01,2 (𝒟)\{0}

FR (u),

(6.29)

i. e., λ1 corresponds to the minimum of the Rayleigh quotient. Another interesting observation is that this result is connected with the Poincaré inequality, in the sense that the first eigenvalue of the Laplacian provides the best estimate for the Poincaré constant. We collect these observations, along with some other fundamental properties of the eigenvalues and eigenfunctions of the Laplacian, in the following proposition (see, e. g., [13] or [63]). Proposition 6.2.12. Assume that 𝒟 ⊂ ℝd is a bounded simply connected and regular open set. Problems (6.25) and (6.26) are equivalent and admit (nontrivial) local minima u ∈ W01,2 (𝒟). These are solutions of the eigenvalue problem (6.27) or its equivalent weak form (6.28), which admits a countable set of solutions {(un , λn ) : n ∈ ℕ} with 0 < λ1 ≤ λ2 ≤ ⋅ ⋅ ⋅, and λn → ∞ (the eigenvalues counted with their multiplicity2 ) with {un : n ∈ ℕ} constituting an (orthogonal) basis for L2 (𝒟), and {λn−1/2 un : n ∈ ℕ} being a basis for the ´ Hilbert space W01,2 (𝒟), equipped with the inner product ⟨u, v⟩W 1,2 (𝒟) = 𝒟 ∇u(x)⋅∇v(x)dx. 0 Moreover, the global minimum is attained at the eigenfunction u1 corresponding to the first eigenvalue λ1 , which is simple and strictly positive, λ1 > 0, and characterized in terms of the Rayleigh–Courant–Fisher variational formula (6.29), while u1 > 0 on 𝒟. 2 Each eigenvalue is recorded in the sequence as many times as the dimension of the corresponding eigenspace, N(λI − A) which is finite.

6.2 Warm up: variational theory of the Laplacian

| 263

Proof. We are only interested in nontrivial solutions so to ease notation we will replace W01,2 (𝒟) \ {0} with W01,2 (𝒟) in problem (6.26), assuming that u ≠ 0. The proof is broken up into 4 steps: 1. The connection between the two problems has already been established. Any local minimum will satisfy the first-order condition of either (6.25) or (6.26). Concerning the first problem, one may use the methodology of Lagrange multipliers, according to which, upon rewriting problem (6.25) as minu∈W 1,2 (𝒟) F(u) subject to 0

F1 (u) = 0, for F(u) = ‖∇u‖2L2 (𝒟) , F1 (u) = ‖u‖2L2 (𝒟) , for any local minimum u there exists a Lagrange multiplier λ ∈ ℝ such that DF(u) + λDF1 (u) = 0. A simple calculation reduces this first-order condition to the eigenvalue problem (6.27). In order to calculate the first-order conditions for the Rayleigh functional, we perturb around a local minimum u and expanding ϵ 󳨃→ ϕ(ϵ) = FR (u+ϵv) around ϵ = 0 (exercise) we obtain the first-order condition −Δu = FR (u)u, which upon calling λ = FR (u) reduces again to the eigenvalue problem (6.27). Concerning the existence of a global minimum for problem (6.25), a standard application of the direct method suffices, using the convexity of the constraint set and the Poincaré inequality, tracing more or less the steps we followed in Proposition 6.2.3. 2. Concerning the eigenvalue problem (6.27), we use the observation that λ is an eigenvalue of the Laplacian if and only if λ−1 is an eigenvalue of (−Δ)−1 : L2 (𝒟) → L2 (𝒟), which being a self-adjoint positive definite and compact operator admits a countable set of eigenvalues and eigenfunctions {(μn , un ) : n ∈ ℕ} such that μ1 > μ2 > ⋅ ⋅ ⋅ > μn → 0 (if counted without multiplicities) with the eigenfunctions {un : n ∈ ℕ} forming a basis for the Hilbert space L2 (𝒟), assumed without loss of generality to be normalized (see Theorems 1.3.12 and 1.3.13). Upon defining λn = μ−1 n , we obtain the required result (in its form without counting multiplicities). One can easily see using the weak form (6.28) that ‖∇un ‖2L2 (𝒟) = λn ‖un ‖2L2 (𝒟) = λn , so that upon defin-

ing wn = λn−1/2 un we have ‖wn ‖W 1,2 (𝒟) = 1. Orthogonality follows by the weak form of 0

the eigenvalue problem. For orthogonality of un , um , n ≠ m in L2 (𝒟), it suffices to first use um as a test function in the weak form for un , then use un as a test function in the weak form for um , and use symmetry to deduce that (λn − λm )⟨un , um ⟩L2 (𝒟) = 0, so that λn ≠ λm implies that ⟨un , um ⟩L2 (𝒟) = 0. A similar argument leads to the orthogonality of {wn : n ∈ ℕ}, i. e., ⟨wn , wm ⟩W 1,2 (𝒟) = δn,m . We now show that span(wn : n ∈ ℕ) is 0

dense in W01,2 (𝒟), or equivalently if v ∈ W01,2 (𝒟) is such that ⟨v, wn ⟩W 1,2 (𝒟) = 0 for every 0 n ∈ ℕ, then v = 0. Indeed, upon integration by parts, and using the fact that un are eigenfunctions of −Δ, we have that ⟨v, wn ⟩W 1,2 (𝒟) = √λn ⟨v, un ⟩L2 (𝒟) so, since λn > 0, 0 ⟨v, wn ⟩W 1,2 (𝒟) = 0 for every n ∈ ℕ is equivalent to ⟨v, un ⟩L2 (𝒟) = 0 for every n ∈ ℕ, 0

which by the fact that {un : n ∈ ℕ} is a basis for L2 (𝒟) guarantees that v = 0 as required. Extending the distinct sequence of eigenvalues by counting multiplicities and reorganizing the above bases of eigenfunctions to a new basis as in Example 1.3.14, we obtain the required result.

264 | 6 The calculus of variations 3. We now show the variational formula (6.29). There are several ways around this. The Lagrange multiplier formulation of the problem leads directly to this result. Another way to see this is the following: Since for any eigenfunction u with eigenvalue λ, it holds that FR (u) = λ. It is immediate to see that λ1 ≥ infu∈W 1,2 (𝒟) FR (u), so we only 0

need to establish that λ1 = FR (u1 ) ≤ infu∈W 1,2 (𝒟) FR (u), i. e., that for every u ∈ W01,2 (𝒟), 0

u ≠ 0, it holds that FR (u) ≥ λ1 . To this end, expand any u ∈ W01,2 (𝒟) in the orthonormal basis {wn : n ∈ ℕ} of W01,2 (𝒟), as ∞

∞

∞

n=1

n=1

u = ∑ ⟨u, wn ⟩W 1,2 (𝒟) wn = ∑ ⟨∇u, ∇wn ⟩L2 (𝒟) = ∑ ⟨u, un ⟩L2 (𝒟) un , 0

n=1

where we used integration by parts, the definition of wn , and the fact that un are eigenfunctions of the Laplacian. Using this expansion, we calculate ˆ

∞ ∞

𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx = ⟨∇u, ∇u⟩L2 (𝒟) = ∑ ∑ ⟨u, un ⟩L2 (𝒟) ⟨u, um ⟩L2 (𝒟) ⟨∇un , ∇um ⟩L2 (𝒟) m=1 n=1

∞

= ∑ λn ⟨u, un ⟩2L2 (𝒟) ,

(6.30)

n=1

where we used the orthogonality of {un : n ∈ ℕ} in L2 (𝒟), and ∞

‖u‖2L2 (𝒟) = ∑ ⟨u, un ⟩2L2 (𝒟) . n=1

(6.31)

Since λn ≥ λ1 for every n ∈ ℕ, it clearly holds that ˆ 𝒟

∞

∞

n=1

n=1

󵄨󵄨 󵄨2 2 2 2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx = ∑ λn ⟨u, un ⟩L2 (𝒟) ≥ λ1 ∑ ⟨u, un ⟩L2 (𝒟) = λ1 ‖u‖L2 (𝒟) ,

so that FR (u) ≥ λ1 for every u ∈ W01,2 (𝒟) as required. In fact, these steps can be used to show that for any u ∈ W01,2 (𝒟) with ‖u‖L2 (𝒟) = 1, −Δu = λ1 u,

if and only if ‖∇u‖2L2 (𝒟) = λ1 .

(6.32)

Indeed, assuming that ‖∇u‖2L2 (𝒟) = 1, then (6.30) implies that λ1 = ∑∞ n=1 λn ⟨u,

2 un ⟩2L2 (𝒟) while assuming that ‖u‖L2 (𝒟) = 1, (6.31) implies that ∑∞ n=1 ⟨u, un ⟩L2 (𝒟) = 1.

2 Combining these ∑∞ n=1 (λn − λ1 )⟨u, un ⟩L2 (𝒟) = 0 and as λn > λ1 , this is an infinite sum of positive terms; hence, ⟨u, un ⟩L2 (𝒟) = 0 for all n such that λn > λ1 . Let m be the multiplicity of λ1 (which is finite). Then in our previous observation we see that a u ∈ W01,2 (𝒟) such that ‖u‖L2 (𝒟) = 1 and ‖∇u‖2L2 (𝒟) = λ1 , admits a finite expansion of the form u = ∑m n=1 ⟨u, un ⟩L2 (𝒟) un , where un , n = 1, . . . , m, solve −Δun = λ1 un . By the linearity of the problem, u solves −Δu = λ1 u. 4. Finally, we prove that λ1 > 0 and simple and that u1 > 0 in 𝒟.

6.2 Warm up: variational theory of the Laplacian

| 265

Consider any nontrivial weak solution u of −Δu = λ1 u in W01,2 (𝒟). We will show that either u > 0 in 𝒟 or u < 0 in 𝒟. Consider u+ = max(u, 0) and u− = max(−u, 0) and note that u± ∈ W01,2 (𝒟), while ⟨u+ , u− ⟩L2 (𝒟) = ⟨∇u+ , ∇u− ⟩L2 (𝒟) = 0, so that ‖u‖2L2 (𝒟) = ‖u+ ‖2L2 (𝒟) + ‖u− ‖2L2 (𝒟) and ‖∇u‖2L2 (𝒟) = ‖∇u+ ‖2L2 (𝒟) + ‖∇u− ‖2L2 (𝒟) . By the variational rep-

resentation (6.29) for any w ∈ W01,2 (𝒟), w ≠ 0 it holds that ‖∇w‖2L2 (𝒟) ≥ λ1 ‖w‖2L2 (𝒟) . Setting w = u± in this, we obtain that 󵄩 ± 󵄩2 󵄩󵄩 ± 󵄩󵄩2 󵄩󵄩∇u 󵄩󵄩L2 (𝒟) ≥ λ1 󵄩󵄩󵄩u 󵄩󵄩󵄩L2 (𝒟) .

(6.33)

We have that for any u ∈ W01,2 (𝒟) such that ‖u‖2L2 (𝒟) = 1, and −Δu = λ1 u, it holds that ‖∇u‖2L2 (𝒟) = λ1 , so that using the above observations,

󵄩 󵄩2 󵄩 󵄩2 λ1 = ‖∇u‖2L2 (𝒟) = 󵄩󵄩󵄩∇u+ 󵄩󵄩󵄩L2 (𝒟) + 󵄩󵄩󵄩∇u− 󵄩󵄩󵄩L2 (𝒟) 󵄩 󵄩2 󵄩 󵄩2 ≥ λ1 (󵄩󵄩󵄩u+ 󵄩󵄩󵄩L2 (𝒟) + 󵄩󵄩󵄩u− 󵄩󵄩󵄩L2 (𝒟) ) = λ1 ;

(6.34)

hence, combining (6.33) with (6.34) we conclude that 󵄩 ± 󵄩2 󵄩󵄩 ± 󵄩󵄩2 󵄩󵄩∇u 󵄩󵄩L2 (𝒟) = λ1 󵄩󵄩󵄩u 󵄩󵄩󵄩L2 (𝒟) .

(6.35)

Combining (6.35) with the observation (6.32) toward the end of step 3, we conclude that u± are solutions of −Δu± = λ1 u± with Dirichlet boundary conditions. We may show using a body of arguments, which constitute the regularity theory for the solutions of the Laplacian eigenvalue problem, that any solution of the Laplacian eigenvalue problem in W01,2 (𝒟), subject to sufficient regularity of the domain 𝒟 is in fact a function3 u ∈ W01,2 (𝒟)∩C(𝒟). Since u± satisfy −Δu± = λ1 u± ≥ 0, the strong maximum principle for the Laplacian (Theorem 6.2.7) indicates that either u± > 0 in 𝒟 or u± = 0 (identically) in 𝒟, therefore, since u is nontrivial either u > 0 in 𝒟 or u < 0 in 𝒟. To show that λ1 is a simple eigenvalue, assume on the contrary that there exist two ´ ̂ nontrivial solutions u, û of −Δu = λ1 u in W01,2 (𝒟). By the above result, 𝒟 u(x)dx ≠ 0 so ´ ̂ that there exists μ ∈ ℝ such that 𝒟 (u(x)−μu(x))dx = 0. By the linearity of the problem, w := u−μû is also a solution of −Δw = λ1 w, so that it also enjoys the property that either ´ ̂ u − μû > 0 or u − μû < 0 in 𝒟 which combined with the fact that 𝒟 (u(x) − μu(x))dx = 0, leads to the conclusion that u = μû in 𝒟, therefore, the eigenvalue λ1 is simple. Example 6.2.13 (Variational formulation of higher eigenvalues). A variational formulation exists not for only the first eigenvalue of the Laplacian but also for the other eigenvalues. Define the finite dimensional subspace Vn = span(u1 , . . . , un ) and its 3 There are certain ways around this result. One way is to argue as follows: Since u ∈ W01,2 (𝒟), we have that Δu = λu ∈ W01,2 (𝒟) ⊂ L2 (𝒟); hence, by Proposition 6.2.9, u ∈ W 2,2 (𝒟) and iterating this scheme we end up with u ∈ C ∞ (𝒟).

266 | 6 The calculus of variations orthogonal complement Vn⊥ . By an argument similar as the one used in Proposition 6.2.12, we see that for any u ∈ Vn⊥ it holds that ˆ

𝒟

∞

∞

i=n+1

i=n+1

󵄨2 󵄨󵄨 2 2 2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx = ∑ λi ⟨u, ui ⟩L2 (𝒟) ≥ λn+1 ∑ ⟨u, ui ⟩L2 (𝒟) = λn+1 ‖u‖L2 (𝒟) ;

hence, λn+1 = min{FR (u) : u ∈ Vn⊥ , u ≠ 0}.

(6.36)

This extends the variational formulation to other eigenvalues than the first as well as the definition of eigenvalues of the Laplacian in terms of constrained optimization problems on the surface of the unit ball of W01,2 (𝒟). ◁ We close our discussion of the eigenvalue problem for the Laplace operator with the celebrated Courant–Fisher minimax formula. Theorem 6.2.14 (Courant–Fisher minimax formula). Let En be the class of all ndimensional linear subspaces of W01,2 (𝒟). Then λn = min max FR (u). E∈En u∈E,u=0 ̸

(6.37)

Proof. Let Vn = span(u1 , . . . , un ) ∈ En . Before starting with the proof, we note that the sup and the inf are attained in the above formulae by choosing E = Vn and u = un . To show the formula, we work as follows: For any subspace E ⊂ W01,2 (𝒟), E ∈ En it ⊥ holds that E ∩ Vn−1 ≠ {0}. Indeed, consider the linear mapping T := PVn−1 : E → Vn−1 , and recall that for any finite dimensional linear map it holds that dim(E) = dim(N(T))+ dim(R(T)). By definition, dim(E) = n, while since R(T) ⊂ Vn−1 we have that dim(R(T)) ≤ n − 1; hence, dim(N(T)) ≥ n − (n − 1) = 1, so there exists a nonzero element w ∈ E such ⊥ that PVn−1 w = 0, i. e. w ∈ E ∩ Vn−1 . By (6.36), λn ≤ FR (w) ≤ max{FR (u) : u ∈ E, u ≠ 0}, and since E is an arbitrary n-dimensional subspace of W01,2 (𝒟), we have that λn ≤ FR (w) ≤ min max FR (u). E∈En u∈E,u=0 ̸

(6.38)

On the other hand, one may choose E = Vn so that for any u ∈ E = Vn , using similar arguments as in Proposition 6.2.12 we have that FR (u) =

∑ni=1 λi ⟨u, ui ⟩2L2 (𝒟) ∑ni=1 ⟨u, ui ⟩2L2 (𝒟)

≤ λn ,

since λ1 ≤ ⋅ ⋅ ⋅ ≤ λn . The above holds for any u ∈ E = Vn ; hence, max FR (u) ≤ λn .

u∈Vn ,u=0 ̸

6.2 Warm up: variational theory of the Laplacian

| 267

Since the above choice for E is simply one choice for E ∈ En , it is easy to conclude that min max FR (u) ≤ λn . E∈En u∈Vn ,u=0 ̸

(6.39)

Combining (6.38) with (6.39), we obtain the required result. Other similar minimax or maximin formulae for the eigenvalues can be obtained (see, e. g., [13]) which may further be extended to more general operators than the Laplacian or discrete versions of the Laplacian appearing in graph theory and find applications in various fields such as, for instance, image processing, etc. They also allow us to obtain important insight on the properties of the eigenfunctions and in many cases lead to approximation algorithms (either analytical or numerical). We provide some examples. Example 6.2.15 (Comparison of eigenvalues in different domains). Consider two different domains 𝒟, 𝒟󸀠 ⊂ ℝd such that 𝒟 ⊂ 𝒟󸀠 . Then for every eigenvalue of the Laplacian with Dirichlet boundary conditions, it holds that λn (𝒟󸀠 ) ≤ λn (𝒟). Since 𝒟 ⊂ 𝒟󸀠 , any function u ∈ W01,2 (𝒟) can be extended to a function u󸀠 ∈ 1,2 W0 (𝒟󸀠 ) by defining u󸀠 = u1𝒟 + 01𝒟󸀠 \𝒟 . Clearly, ‖u‖L2 (𝒟) = ‖u󸀠 ‖L2 (𝒟󸀠 ) and ‖∇u‖L2 (𝒟) = ‖∇u󸀠 ‖L2 (𝒟󸀠 ) , so that FR (u, 𝒟) = FR (u󸀠 , 𝒟󸀠 ), where by FR (v, A) we denote the Rayleigh quotient keeping track of both the function v and the domain A. Naturally, the set of all n-dimensional subspaces of W01,2 (𝒟) is a subset of the corresponding set concerning W01,2 (𝒟󸀠 ), and the stated result comes by an application of the variational formula (6.37), by noting that the minimum when the formula is applied for 𝒟 is taken over a smaller set than when applied for 𝒟󸀠 . ◁ Example 6.2.16 (Faber–Krahn inequality). The Faber–Krahn inequality is an important result stating that among all bounded measurable sets 𝒟 ⊂ ℝd , of the same measure, the ball has the smallest possible first eigenvalue λ1 , for the Dirichlet–Laplacian, i. e., λ1 (𝒟) ≥ λ1 (𝒟∗ ) with equality if and only if 𝒟 = 𝒟∗ , where for any set 𝒟 we denote by 𝒟∗ the ball in ℝd with the same volume as 𝒟. The proof of the above result is based on the variational representation formula (6.29), upon using the important concept of spherical rearrangement. Given any function u : 𝒟 → ℝ we define its spherical (or Schwarz) rearrangement u∗ : 𝒟∗ → ℝ as the spherically symmetric radially nonincreasing function of |x|, constructed by arranging the level sets of u in balls of the same volume, u∗ (x) := sup{γ : x ∈ Lγ }, where by Lγ we denote the superlevel set Lγ = {x ∈ 𝒟 : u(x) ≥ γ}. Two important results are that ‖u‖L2 (𝒟) = ‖u∗ ‖L2 (𝒟∗ ) and the Polya–Szego4 inequality ‖∇u‖L2 (𝒟) ≥ ‖∇u∗ ‖L2 (𝒟∗ ) . Using these, we see that FR (u∗ , 𝒟∗ ) ≤ FR (u, 𝒟), for every function u : 𝒟 → ℝ and every 4 This is an important result, with deep connections with isoperimetric inequalities; for details see, e. g., [68] and references therein.

268 | 6 The calculus of variations domain 𝒟 ⊂ ℝd and applying the variational representation formula (6.29) we obtain the required result. ◁ As we will see, in this chapter and the following one, these very interesting properties of the Laplacian are not restricted to the particular operator but rather are general properties of a wider class of variational problems related to self-adjoint operators.

6.3 Semicontinuity of integral functionals Before studying the general problem of minimization of integral functionals, using the direct method, we must first establish some semicontinuity properties. In this section, we collect some fundamental results in this direction, concerning the lower semicontinuity of integral functionals F : X = W 1,p (𝒟) → ℝ of the form ˆ f (x, u(x), ∇u(x))dx, F(u) := (6.40) 𝒟

where 𝒟 ⊂ ℝd is an open bounded set with sufficiently smooth boundary, u is a function from 𝒟 to ℝ, ∇u = ( 𝜕x𝜕 u, . . . , 𝜕x𝜕 u) its gradient and f : 𝒟 × ℝ × ℝd → ℝ is a given 1 d function. These results highlight the deep connection of lower semicontinuity for F with the convexity of the function z 󳨃→ f (x, y, z). It is interesting to note that the functional F can also be defined when u is a vector valued function u : 𝒟 → ℝm , in which case ∇u must be replaced by the derivative matrix Du ∈ ℝm×d (or its transpose) and f : 𝒟 × ℝ × ℝm×d → ℝ. The semicontinuity problem can be treated in this general case with modification of the techniques developed in this section. For an excellent account as well as more detailed results on the subject, the reader may consult [66] or [50]. Our starting point will be the study of semicontinuity of a related integral functional in Lebesgue spaces, and in particular of the functional Fo : Lp (𝒟) × Lq (𝒟; ℝs ) → ℝ defined by ˆ Fo (u, v) = f (u(x), v(x))dx, (6.41) 𝒟

for every u ∈ Lp (𝒟), v ∈ Lq (𝒟; Rs ) for appropriate p, q, s ∈ ℕ. Clearly, if u ∈ W 1,p (𝒟) then setting v = ∇u, and s = d, we have that Fo (u, v) = F(u), where F is the functional defined in (6.40). Hence, establishing semicontinuity results for the auxiliary functional Fo , defined in (6.41), in Lebesgue spaces can lead us to semicontinuity results in Sobolev spaces for the original functional F, defined in (6.40). Of course, some care will have to be taken as to the topologies with which W 1,p (𝒟) and Lp (𝒟) × Lq (𝒟; ℝs ) are endowed for F and Fo , respectively, to be lower semicontinuous but as we shall see shortly these subtleties can be handled with the use of the Rellich–Kondrachov compact embedding theorem for Sobolev spaces into a suitable Lebesgue space.

6.3 Semicontinuity of integral functionals | 269

Following [66], we will study first the sequential lower semicontinuity of general functionals of the form (6.41), with respect to strong and weak convergence in Lebesgue spaces, and then apply this general result in providing weak lower semicontinuity results for functionals of the form (6.40) in Sobolev spaces. These are classic results in the calculus of variations, in the legacy of De Giorgi (see, e. g., [52]) revisited by many authors in subsequent years (for a detailed bibliographical exposition, see [50]). Our exposition follows closely the proof of Theorem 7.5 in [66] (see also Theorem 3.26 in [50]).

6.3.1 Semicontinuity in Lebesgue spaces We start by considering weak lower semicontinuity of integral functionals in Lebesgue spaces. Before moving to the more general problem (6.41), we will first consider the problem of semicontinuity of integral functionals FL : Lp (𝒟; ℝm ) → ℝ, defined by FL (v) :=

ˆ 𝒟

f (x, v(x))dx,

(6.42)

where f : 𝒟 × ℝm → ℝ ∪ {+∞} is a suitable function and v : 𝒟 → ℝm is a vector valued function such that v ∈ Lp (𝒟; ℝm ). We impose the following conditions on the function f . Assumption 6.3.1. Consider the function f : 𝒟 × ℝm → ℝ ∪ {+∞} with the following properties: (i) x 󳨃→ f (x, z) is measurable for every z ∈ ℝm . (ii) z 󳨃→ f (x, z) is lower semicontinuous and convex in ℝm , a. e. in x ∈ 𝒟, ⋆ (iii) There exist α ∈ Lp (𝒟; ℝm ) and β ∈ L1 (𝒟) such that f (x, z) ≥ α(x) ⋅ z + β(x),

a. e. x ∈ 𝒟, ∀ z ∈ ℝm .

Theorem 6.3.2. If Assumption 6.3.1 holds, the functional FL , defined in (6.42) is weakly sequentially lower semicontinuous in Lp (𝒟; ℝm ), p > 1. Proof. Let us assume first that f ≥ 0. We would like to show that if vn ⇀ v in Lp (𝒟; ℝm ), then L := lim infn FL (vn ) ≥ FL (v). If f ≥ 0, our basic tool would be the Fatou lemma, which requires a subsequence of the original sequence converging a. e. However, the convergence vn ⇀ v in Lp (𝒟; ℝm ) is weak, and in general, we cannot guarantee the existence of a subsequence which converges a. e. To overcome this difficulty, we recall Mazur’s lemma (Proposition 1.2.17) according to which, we may construct a new sequence {wn : n ∈ ℕ} out of the original one with the property that wn → v in Lp (𝒟; ℝm ), and apply the above reasoning upon extraction of a further a. e. convergent subsequence. The proof is broken up into 3 steps:

270 | 6 The calculus of variations 1. Assume that f ≥ 0. We will first show an auxiliary strong lower semicontinuity result. Let vn → v in Lp (𝒟; ℝm ). Select a subsequence {vnℓ : ℓ ∈ ℕ} such that limℓ FL (vnℓ ) = L := lim infn FL (vn ). We work along this subsequence and select a further subsequence {vnℓ : k ∈ ℕ} such that vnℓ → v a. e. We may now use the lower k k semicontinuity of f , which yields that lim infk f (x, vnℓ (x)) ≥ f (x, v(x)) a. e. and Fatou’s k lemma to obtain ˆ ˆ ˆ f (x, v(x))dx, lim inf f (x, vnℓ (x))dx ≥ L = lim inf f (x, vnℓ (x))dx ≥ k

𝒟

k

k

𝒟

k

𝒟

which is the required strong lower semicontinuity result. Note that for the strong lower semicontinuity we do not require convexity of f . 2. We now let vn ⇀ v. We first consider a subsequence {vnℓ : ℓ ∈ ℕ} such that L := lim infn FL (vn ) = limℓ FL (vnℓ ). On account of that, for any ϵ > 0 there exists M(ϵ) ∈ ℕ such that L ≤ FL (vnℓ ) ≤ L + ϵ,

∀ ℓ ≥ M(ϵ).

(6.43)

Fix ϵ > 0, and apply Mazur’s lemma to the sequence {vnℓ : ℓ ≥ M(ϵ)}, to construct a new sequence {wℓ : ℓ ∈ ℕ} (or rather ℓ > M(ϵ)) such that wℓ → v in Lp (𝒟; ℝm ). The new sequence consists of convex combinations of the elements in the tail of the original sequence, which implies that for every ℓ, there exists K(ℓ) ≥ M(ϵ) and λ(ℓ, i) > K(ℓ) p m 0 with ∑K(ℓ) i=M(ϵ) λ(ℓ, i) = 1, such that if wℓ = ∑i=M(ϵ) λ(ℓ, i)vni , then wℓ → v in L (𝒟 ; ℝ ). Since z 󳨃→ f (x, z) is convex, for every ℓ it holds that K(ℓ)

K(ℓ)

i=M(ϵ)

i=M(ϵ)

f (x, wℓ (x)) = f (x, ∑ λ(ℓ, i)vni ) ≤ ∑ λ(ℓ, i)f (x, vni ).

(6.44)

As wℓ → v in Lp (𝒟; ℝm ), we can extract a subsequence, denoted the same for simplicity, such that wℓ → v a. e. Clearly, (6.44) holds for the chosen subsequence, so passing (6.44) to this subsequence and integrating over 𝒟 using Fatou’s lemma we obtain that K(ℓ)

FL (wℓ ) ≤ ∑ λ(ℓ, i)FL (vni ). i=M(ϵ)

(6.45)

For all i ≥ M(ϵ), by (6.43), FL (vni ) ≤ L + ϵ, therefore, (6.45) implies that FL (wℓ ) ≤ L + ϵ for all ℓ; hence, lim infℓ FL (wℓ ) ≤ L + ϵ. Since wℓ → u in Lp (𝒟; ℝm ), by the strong lower semicontinuity result we have shown in step 1, it holds that FL (v) ≤ lim infℓ FL (wℓ ). Therefore, FL (v) ≤ L + ϵ for every ϵ > 0, so that taking the limit as ϵ → 0, leads to the desired result FL (v) ≤ L. 3. If f does not satisfy the property f ≥ 0, then we cannot apply Fatou’s lemma which was crucial in our argument. However, by Assumption 6.3.1(iii) we can consider the function f ̄ defined by f ̄(x, z) = f (x, z) − α(x) ⋅ z − β(x) ≥ 0. By our re´ sults so far, the functional FL̄ defined by FL̄ (v) = 𝒟 f ̄(x, v(x))dx is weakly sequentially lower semicontinuous in Lp (𝒟; ℝm ). We can express FL as FL = FL̄ + F1 , where

6.3 Semicontinuity of integral functionals | 271

´ ´ F (v) = 𝒟 (α(x) ⋅ v(x) + β(x))dx. If vn ⇀ v in Lp (𝒟; ℝm ), clearly 𝒟 α(x) ⋅ vn (x)dx → ´1 p⋆ m 𝒟 α(x) ⋅ v(x)dx (by the definition of weak convergence and since α ∈ L (𝒟 ; ℝ )) so that F1 (vn ) → F1 (v) and F1 is weakly continuous. Therefore, FL is weakly lower semicontinuous as the sum of a weakly lower semicontinuous and a continuous functional. Convexity played a crucial role in the proof of weak lower semicontinuity of the integral functional F, when using the Mazur lemma to pass from weak convergence to strong convergence and from that to a. e. convergence. One could think that convexity was imposed as a condition to facilitate our analysis, however, its role is much more fundamental, as the following proposition shows. Theorem 6.3.3. Consider the integral functional FL : Lp (𝒟; ℝm ) → ℝ ∪ {+∞} defined ´ by FL (v) = 𝒟 f (v(x))dx for some f : ℝm → ℝ which is lower semicontinuous. If FL is weakly sequentially lower semicontinuous in Lp (𝒟; ℝm ), then f is a convex function. Proof. Assume that FL is weakly sequentially lower semicontinuous. We will show the convexity of z 󳨃→ f (z). Fix any z1 , z2 ∈ ℝm , t ∈ (0, 1) and h ∈ Sd , where Sd is the unit sphere of ℝd . Define the sequence of functions {wt,n : n ∈ ℕ} by wt,n (x) = z2 + wt (n (h ⋅ x) )(z1 − z2 ) for every x ∈ ℝd , where wt : ℝ → ℝ is the periodic extension (with period 1) of the indicator function 1[0,t] , of the interval [0, t] ⊂ [0, 1]. Using the sequence {wt,n : n ∈ ℕ}, we construct the sequence of functions {wt,n : n ∈ ℕ} defined by wt,n (x) := wt (n (h ⋅ x) ) for every x ∈ ℝd . As can be shown by an application of the Riemann–Lebesgue lemma (see Lemma 6.9.1 in Section 6.9.1), for every t ∈ [0, 1] ⋆ it holds that wt,n ⇀ ψt in L∞ (𝒟; ℝm ), where ψt is the constant function ψt (x) = t for every x ∈ ℝd . Hence, for every t ∈ [0, 1] it holds that wt,n ⇀ w̄ t in L∞ (𝒟; ℝm ) where w̄ t is the constant function w̄ t (x) = z2 + t(z1 − z2 ) a. e. x ∈ 𝒟 and, therefore, wt,n ⇀ w̄ t in Lp (𝒟; ℝm ). Since FL is weakly sequentially lower semicontinuous, we have that ˆ FL (w̄ t ) ≤ lim inf FL (wt,n ) = lim inf f (z2 + wt (n (h ⋅ x) )(z1 − z2 ) )dx. (6.46) ⋆

n

n

𝒟

Clearly, FL (w̄ t ) = |𝒟| f (z2 + t(z1 − z2 ) ) = |𝒟| f (tz1 + (1 − t)z2 ), where |𝒟| is the Lebesgue measure of 𝒟, so that (6.46) yields, ˆ |𝒟| f (tz1 + (1 − t)z2 ) ≤ lim inf f (z2 + wt (n (h ⋅ x) )(z1 − z2 ) )dx. n

𝒟

(6.47)

We now calculate the right-hand side of (6.47). First of all, since wt (n (h ⋅ x) ) takes only two values 0 and 1, it can be seen that f (z2 + wt (n (h ⋅ x) )(z1 − z2 ) ) = wt (n (h ⋅ x) ) f (z1 ) + (1 − wt (n (h ⋅ x) )) f (z2 ),

272 | 6 The calculus of variations and since wt,n (x) := wt (n(h ⋅ x) ) has the property wt,n ⇀ ψt (where ψt is the constant function ψt (x) = t, for every x ∈ ℝd ) we see that ˆ

f (z2 + wt (n (h ⋅ x) )(z1 − z2 ) )dx ˆ = (wt (n (h ⋅ x) ) f (z1 ) + (1 − wt (n (h ⋅ x) )) f (z2 ) )dx

𝒟

𝒟

→ |𝒟| (t f (z1 ) + (1 − t) f (z2 ) ). Substituting the above in (6.47) yields f ( t z1 + (1 − t) z2 ) ≤ t f (z1 ) + (1 − t) f (z2 ), which is the required convexity of f . Remark 6.3.4. The theorem is also valid when f : 𝒟 × ℝm → ℝ has explicit space dependence, in which case weak lower semicontinuity implies the convexity of the function z 󳨃→ f (x, z) a. e. in 𝒟. For a very detailed account of lower semicontinuity and well-posedness results for integral functionals in Lebesgue spaces and their connection with convexity as well-boundeness properties of f the reader can consult [66] or [50]. We now provide a more general result, concerning functionals of two variables u and v of the general form (6.41) involving strong convergence with respect to one variable and weak convergence with respect to the other. Concerning the function f , we make the following assumption. Assumption 6.3.5. Consider the function f : 𝒟 ×ℝs ×ℝm → ℝ∪{+∞} and p, q ∈ [1, ∞), such that: (i) f is Carathéodory, i. e., (y, z) 󳨃→ f (x, y, z) continuous in ℝs × ℝm , a. e. x ∈ 𝒟 and x 󳨃→ f (x, y, z) is measurable for every (y, z) ∈ ℝs × ℝm . ⋆ (ii) There exist functions α ∈ Lp (𝒟; ℝm ), β ∈ L1 (𝒟) and a constant c ∈ ℝ such that f (x, y, z) ≥ α(x) ⋅ z + β(x) + c|y|q ,

a. e. x ∈ 𝒟, ∀ y ∈ ℝs , ∀ z ∈ ℝm .

(iii) The function z 󳨃→ f (x, y, z) is convex a. e. in 𝒟 and for every y ∈ ℝs . Theorem 6.3.6. If Assumption 6.3.5 holds for p, q ∈ [1, ∞), the functional Fo : Lq (𝒟; ℝs ) × Lp (𝒟; ℝm ) → ℝ defined by Fo (u, v) :=

ˆ

𝒟

f (x, u(x), v(x))dx,

(6.48)

is lower sequentially semicontinuous with respect to strong convergence in Lq (𝒟; ℝs ) and weak convergence in Lp (𝒟; ℝm ).

6.3 Semicontinuity of integral functionals | 273

Proof. We only sketch the proof which is based upon the observation that if un → u in Lq (𝒟; ℝs ), then the functional Fō : Lp (𝒟; ℝm ) → ℝ ∪ {+∞}, defined by freezing the u coordinate at the limit, v 󳨃→ Fō (v) := Fo (u, v) is weakly lower semicontinuous, so that if vn ⇀ v in Lp (𝒟; ℝm ) then lim infn Fō (vn ) ≥ Fō (v) = Fo (v, u). Then, for any n ∈ ℕ, Fo (vn , un ) = Fo (vn , u) + Fo (vn , un ) − Fo (vn , u) = Fō (vn ) + Fo (vn , un ) − Fo (vn , u), with the remainder term being as small as we wish by letting n become large, so that lower semicontinuity follows by the lower semicontinuity of Fō . The detailed argument which requires a careful control of the remainder term involves certain technicalities of a measure theoretic nature (see [66]). Remark 6.3.7. Theorem 6.3.6 will still hold if in Assumption 6.3.5 instead of assuming that f is Carathéodory we assume that f is a normal integrand, i. e., that x 󳨃→ f (x, y, z) is measurable for every (y, z) ∈ ℝm × ℝs and (y, z) 󳨃→ f (x, y, z) is a. e. in 𝒟 lower semicontinuous (rather than continuous) at the expense of some technical issues in the proof. The reader could consult, e. g., [66] for such versions of the lower semicontinuity result. Convexity of the function z 󳨃→ f (x, y, z) is fundamental for the validity of lower semicontinuity result. In fact, an analogue of Theorem 6.3.3 can be shown for the more general class of integral functionals treated in Theorem 6.3.6 both in the case where f is Carathéodory (see, e. g., Theorem 3.15 in [50]) and in the case where f is a normal integrand (see, e. g., Theorem 7.5 in [66]). Example 6.3.8. The semicontinuity result of Theorem 6.3.6 is interesting in its own right. Consider, e. g., the integral functional F : Lq (𝒟; ℝs ) → ℝ ∪ {+∞}, defined by ´ F(u) := 𝒟 f (x, u(x), (Ku)(x))dx, where f is a function satisfying Assumption 6.3.5 and K : Lp (𝒟; ℝm ) → Lq (𝒟; ℝs ) is a compact operator. Then the functional F is weakly lower semicontinuous. Such functionals are called nonlocal functionals and find interesting applications, e. g., in imaging. ◁

6.3.2 Semicontinuity in Sobolev spaces As an important application of Theorem 6.3.2 (or of its more general form Theorem 6.3.6), we may provide a useful result concerning lower semicontinuity of integral functionals in Sobolev spaces. Consider a function f : 𝒟 ×ℝ×ℝd and a function u : 𝒟 → ℝ such that u ∈ W 1,p (𝒟). Define the functional F : W 1,p (𝒟) → ℝ by ˆ F(u) := f (x, u(x), ∇u(x))dx. (6.49) 𝒟

We make the following assumption on the function f .

274 | 6 The calculus of variations Assumption 6.3.9. f : 𝒟 × ℝ × ℝd → ℝ ∪ {+∞} satisfies the following: (i) f is Carathéodory, i. e., the function x 󳨃→ f (x, y, z) is measurable for every (y, z) ∈ ℝ × ℝd , while the function (y, z) 󳨃→ f (x, y, z) is continuous in ℝ × ℝd , a. e. x ∈ 𝒟. ⋆ (ii) There exist functions α ∈ Lp (𝒟; ℝd ), β ∈ L1 (𝒟) and a constant c ∈ ℝ such that f (x, y, z) ≥ α(x) ⋅ z + β(x) + c|y|q ,

a. e. x ∈ 𝒟, ∀ y ∈ ℝ, ∀ z ∈ ℝd ,

dp where q ∈ [1, sp ) = [1, d−p ) if p < d and q ∈ [1, ∞) if p ≥ d. (iii) The function z 󳨃→ f (x, y, z) is convex a. e. in 𝒟 and for every y ∈ ℝ.

Theorem 6.3.10. Assume that f satisfies Assumption 6.3.9. Then the functional F defined in (6.49) is sequentially weakly lower semicontinuous in W 1,p (𝒟). Proof. Let {un : n ∈ ℕ} ⊂ W 1,p (𝒟) such that un ⇀ u in W 1,p (𝒟). This implies that un ⇀ u in Lp (𝒟) and vn := ∇un ⇀ ∇u in Lp (𝒟). By the compact Sobolev embedding theorem (see Theorem 1.5.11), there exists a subsequence {unk : k ∈ ℕ} such that unk → u in

dp ) if p < d and q ∈ [1, ∞) if p ≥ d. Assumption 6.3.9 allows us Lq (𝒟), for any q ∈ [1, d−p to use Theorem 6.3.6 from which we deduce the sequential weak lower semicontinuity of the functional F.

Remark 6.3.11. The continuity hypothesis in Assumption 6.3.9(i) may be replaced by lower semicontinuity (see also Remark 6.3.7).

6.4 A general problem from the calculus of variations We now consider the problem of minimization of integral functionals F : W 1,p (𝒟) → ℝ ∪ {+∞} of the form ˆ F(u) := f (x, u(x), ∇u(x))dx, (6.50) 𝒟

using the direct method of Weierstrass (Theorem 2.2.5). Our approach follows [50]. Assumption 6.4.1. f : 𝒟 × ℝ × ℝd → ℝ ∪ {+∞} satisfies: (i) the growth condition f (x, y, z) ≤ a1 |z|p + β1 (x) + c1 |y|r ,

a. e. x ∈ 𝒟, ∀ (y, z) ∈ ℝ × ℝd ,

where β1 ∈ L1 (𝒟), a1 > 0, c1 > 0 and 1 ≤ r < sp = of r is necessary if p ≥ d. (ii) the coercivity condition f (x, y, z) ≥ a2 |z|p + β2 (x) + c2 |y|q ,

dp d−p

if 1 < p < d and no condition

a. e. x ∈ 𝒟, ∀ (y, z) ∈ ℝ × ℝd ,

where β2 ∈ L1 (𝒟), a2 > 0, c2 ∈ ℝ, and p > q ≥ 1.

6.4 A general problem from the calculus of variations | 275

Theorem 6.4.2. Let Assumptions 6.3.9 and 6.4.1 hold for p > 1. Then for any v ∈ W 1,p (𝒟) the problem inf{F(u) : u ∈ v + W01,p (𝒟)},

(6.51)

where F is defined as in (6.50), attains its minimum. If the function (y, z) 󳨃→ f (x, y, z) is strictly convex, this minimum is unique. Proof. We will use the direct method of the calculus of variations (see the abstract presentation of the Weierstrass Theorem 2.2.5). Let m = infu∈v+W 1,p (𝒟) F(u). By Assump0

tion 6.4.1(i), for any u ∈ W 1,p (𝒟), it holds that F(u) < ∞, therefore, m < ∞. This follows by the Sobolev embedding theorem according to which if u ∈ W 1,p (𝒟) then u ∈ Lsp (𝒟). Furthermore, by Assumption 6.4.1(ii), for any u ∈ W 1,p (𝒟), it holds that5 F(u) > −∞, therefore, m > −∞. We now consider a minimizing sequence, i. e., a sequence {un : n ∈ ℕ} ⊂ 1,p W (𝒟), such that limn F(un ) = m. By the coercivity condition, Assumption 6.4.1(ii), this sequence is uniformly bounded in W 1,p (𝒟). This is immediate to see in the case where v = 0, i. e., when u ∈ W01,p (𝒟), by an application of the Lebesgue embedding theorem and the Poincaré inequality: Since {un : n ∈ ℕ} ⊂ W01,p (𝒟) is a minimizing sequence, by Assumption 6.4.1(ii) for any ϵ > 0, there exists N such that for all n > N, a2 ‖∇un ‖pLp (𝒟;ℝd ) − |c2 | ‖un ‖qLq (𝒟) + ‖β2 ‖L1 (𝒟)

≤ a2 ‖∇un ‖pLp (𝒟;ℝd ) + c2 ‖un ‖qLq (𝒟) + ‖β2 ‖L1 (𝒟) ≤ F(un ) < m + ϵ.

Since 1 ≤ q < p by the Lebesgue embedding ‖un ‖Lq (𝒟) ≤ c‖un ‖Lp (𝒟) , so that a2 ‖∇un ‖pLp (𝒟;ℝd ) − |c2 |c ‖un ‖qLp (𝒟) + ‖β2 ‖L1 (𝒟) ≤ m + ϵ, where c is a generic constant which may vary from line to line. Recall the Poincaré inequality (Theorem 1.5.13) according to which for every un ∈ W01,p (𝒟), ‖∇un ‖Lp (𝒟;ℝd ) ≥ c𝒫 ‖un ‖Lp (𝒟) where c𝒫 is the Poincaré constant, so that substituting in the above we obtain a2 ‖∇un ‖pLp (𝒟;ℝd ) − |c2 |ccP−q ‖∇un ‖qLp (𝒟;ℝd ) + ‖β2 ‖L1 (𝒟) ≤ m + ϵ. Since 1 ≤ q < p, this implies the existence of a constant c such that ‖∇un ‖Lp (𝒟;ℝd ) < c for all n > N; hence, the sequence {∇un : n ∈ ℕ} is uniformly bounded in Lp (𝒟; ℝd ). 5 To see this, we need to use the Sobolev embeddings. For instance, if p < d, then by the Nirenberg– dp Gagliardo inequality ‖∇u‖Lp (𝒟;ℝd ) ≥ c‖u‖Lsp (𝒟) , for sp = d−p . Since p > q ≥ 1 it holds that q < sp , and combining the Nirenberg–Gagliardo inequality with the Lebesgue embedding we get that ‖∇u‖Lp (𝒟;ℝd ) ≥ c‖u‖Lq (𝒟) . Using this inequality along with the boundedness Assumption 6.4.1(ii), we can see that F is bounded below.

276 | 6 The calculus of variations As a further consequence of the Poincaré inequality, we know that ‖∇u‖Lp (𝒟;ℝd ) is an equivalent norm of W01,p (𝒟), wherefore {un : n ∈ ℕ} ⊂ W01,p (𝒟) is uniformly bounded and by the reflexivity of W01,p (𝒟) (since p > 1) there exists a weakly convergent subsequence {unk : k ∈ ℕ} ⊂ {un : n ∈ ℕ}, such that unk ⇀ uo for some uo ∈ W01,p (𝒟). By Assumption 6.3.9, we may apply Theorem 6.3.10 to infer the lower semicontinuity of F in W01,p (𝒟), therefore, following the abstract arguments of the Weierstrass theorem uo is a minimizer. If v ≠ 0, then the minimizing sequence can be expressed as un = v + ū n where {ū n : n ∈ ℕ} ⊂ W01,p (𝒟), and our arguments can be applied to ū n . We leave the details as an exercise. Assume now that (y, z) 󳨃→ f (x, y, z) is strictly convex and assume the existence of two minimizers uo and vo . Then w = 21 (uo + vo ) is also a minimizer (since by convexity F(w) ≤ 21 (F(uo ) + F(vo )) = m) and this yields ˆ 1 1 { f (x, uo (x), ∇uo (x)) + f (x, vo (x), ∇vo (x)) 2 2 𝒟 (6.52) 1 1 1 − f (x, (uo (x) + vo (x)), ∇uo (x) + ∇vo (x))}dx = 0, 2 2 2 and since by convexity of (y, z) 󳨃→ f (x, y, z), a. e. x ∈ 𝒟, it holds that 1 1 f (x, uo (x), ∇uo (x)) + f (x, vo (x), ∇vo (x)) 2 2 1 1 1 − f (x, (uo (x) + vo (x)), ∇uo (x) + ∇vo (x)) ≥ 0, 2 2 2

a. e. x ∈ 𝒟,

(6.53)

by combining (6.52) and (6.53) it holds that 1 1 f (x, uo (x), ∇uo (x)) + f (x, vo (x), ∇vo (x)) 2 2 1 1 1 − f (x, (uo (x) + vo (x)), ∇uo (x) + ∇vo (x)) = 0, 2 2 2

a. e. x ∈ 𝒟.

But then, the strict convexity of f implies that uo (x) = vo (x), ∇uo (x) = ∇vo (x), a. e. x ∈ 𝒟 hence, uo = vo as elements of v + W01,p (𝒟). Remark 6.4.3. The assumption p > 1 is important. If, e. g., p = 1 then, even though, the gradients of the minimizing sequence are bounded in L1 (𝒟; ℝd ), this does not guarantee the existence of a weak limit as an element of L1 (𝒟; ℝd ), but rather as a Radon measure. This case requires the use the functional setting of BV spaces.

6.5 Differentiable functionals and connection with nonlinear PDEs: the Euler–Lagrange equation Under certain conditions, we will show that the functional F defined in (6.49) is differentiable, therefore, the first-order condition leads to a nonlinear PDE which is the

6.5 Connection with nonlinear PDEs: the Euler–Lagrange equation

277

|

Euler–Lagrange equation for F. It is important to realize that this is true under certain restrictions on f . One may construct examples in which this is not true even in one spatial dimension (see [17]). Assumption 6.5.1 (Growth condition). The function f : 𝒟 × ℝ × ℝd → ℝ satisfies the following conditions: 𝜕 (i) f is continuously differentiable with respect to y and z and the functions f1 := 𝜕y f, f2 :=

𝜕 f 𝜕z

= ∇z f (taking values in ℝ and ℝd resp.) are Carathéodory functions.

(ii) There exist functions ai ∈ Lp (𝒟) and constants ci > 0, i = 1, 2, such that the partial derivatives of f with respect to y and z, denoted by f1 and f2 , respectively, satisfy the growth conditions ⋆

󵄨󵄨 󵄨 p−1 p−1 󵄨󵄨fi (x, y, z)󵄨󵄨󵄨 < ai (x) + ci (|y| + |z| ),

i = 1, 2,

p > 1.

The following theorem connects the problem of minimization of the functional F with the Euler–Lagrange PDE. Theorem 6.5.2. Suppose Assumption 6.5.1 holds. The minimizer uo of the functional F defined in (6.49) satisfies the Euler–Lagrange equation ˆ 𝜕 ( f (x, uo (x), ∇uo (x))v(x) 𝜕y 𝒟 (6.54) 𝜕 1,p + f (x, uo (x), ∇uo (x)) ⋅ ∇v(x))dx = 0, ∀ v ∈ W0 (𝒟), 𝜕z which is interpreted as the weak form of the elliptic differential equation −div(

𝜕 𝜕 f (x, uo (x), ∇uo (x)) ⋅ ∇v(x)) + f (x, uo (x), ∇u(x)) = 0. 𝜕z 𝜕y

(6.55)

If furthermore, the function (y, z) 󳨃→ f (x, y, z) is convex then any solution of the Euler– Lagrange equation (6.54) is a minimizer of F. Proof. The strategy of proof is to show that F is Gâteaux differentiable and recognize the Euler–Lagrange equation as the first-order condition for the minimum. The key to the existence of the Gâteaux derivative is the existence of the limit 1 L := lim (F(u + ϵv) − F(u)) ϵ→0 ϵ ˆ 1 = lim (f (x, u(x) + ϵv(x), ∇u(x) + ϵ∇v(x)) − f (x, u(x), ∇u(x)))dx. ϵ→0 ϵ 𝒟 Since f is differentiable, we have that f (x, u(x) + ϵv(x), ∇u(x) + ϵ∇v(x)) − f (x, u(x), ∇u(x)) ˆ ϵ d = f (x, u(x) + s v(x), ∇u(x) + s ∇v(x))ds 0 ds

278 | 6 The calculus of variations ˆ =

1

0

d f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x))dt dt

a. e. in 𝒟, where the last equality holds by the simple change of variable of integration s = ϵt. By the properties of the function f , d f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x)) dt 𝜕 = f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x))ϵv(x) 𝜕y 𝜕 + f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x)) ⋅ ϵ∇v(x), 𝜕z

so that substituting all these into the above we obtain ˆ 1 wϵ (x)dx (F(u + ϵv) − F(u)) = ϵ 𝒟 where wϵ (x) :=

ˆ

1

0

+

(

(6.56)

𝜕 f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x))v(x) 𝜕y

𝜕 f (x, u(x) + ϵ t v(x), ∇u(x) + ϵ t ∇v(x)) ⋅ ∇v(x))dt. 𝜕z

Clearly, by Assumption 6.5.1(i), wϵ (x) → w a. e. in 𝒟 where w(x) :=

𝜕 𝜕 f (x, u(x), ∇u(x))v(x) + f (x, u(x), ∇u(x)) ⋅ ∇v(x), 𝜕y 𝜕z

so if we are allowed to invoke the Lebesgue dominated convergence theorem, (6.56) will lead us to the result that ˆ L= w(x)dx 𝒟 ˆ 𝜕 𝜕 = ( f (x, u(x), ∇u(x))v(x) + f (x, u(x), ∇u(x)) ⋅ ∇v(x))dx. 𝜕y 𝜕z 𝒟 It thus remains to check the existence of a function h ∈ L1 (𝒟) such that |wϵ (x)| ≤ h(x), a. e. in 𝒟. To this end, we change variables of integration once more and express ˆ 1 ϵ 𝜕 ( f (x, u(x) + sv(x), ∇u(x) + s∇v(x))v(x) wϵ (x) = ϵ 0 𝜕y +

𝜕 f (x, u(x) + sv(x), ∇u(x) + s∇v(x)) ⋅ ∇v(x))ds, 𝜕z

which in turn implies upon using the growth conditions of Assumption 6.5.1(ii) that a. e. x ∈ 𝒟, ˆ ϵ 󵄨 󵄨p−1 󵄨 󵄨p−1 󵄨󵄨 󵄨 1 (α (x) + c1 (󵄨󵄨󵄨u(x) + sv(x)󵄨󵄨󵄨 + 󵄨󵄨󵄨∇u(x) + s∇v(x)󵄨󵄨󵄨 ))|v|(x)ds 󵄨󵄨wϵ (x)󵄨󵄨󵄨 ≤ ϵ 0 1

6.6 Regularity results in the calculus of variations | 279

+

1 ϵ

ˆ 0

ϵ

󵄨 󵄨p−1 󵄨 󵄨p−1 󵄨 󵄨 (α2 (x) + c2 (󵄨󵄨󵄨u(x) + sv(x)󵄨󵄨󵄨 + 󵄨󵄨󵄨∇u(x) + s∇v(x)󵄨󵄨󵄨 ))󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨ds

󵄨 󵄨 p−1 󵄨 󵄨 p−1 󵄨 󵄨 󵄨 󵄨 ≤ c + c1 ( 󵄨󵄨󵄨u(x)󵄨󵄨󵄨 + 󵄨󵄨󵄨v(x)󵄨󵄨󵄨) + (󵄨󵄨󵄨∇u(x) + ∇v(x)󵄨󵄨󵄨) ) 󵄨󵄨󵄨v(x)󵄨󵄨󵄨 󵄨 󵄨 p−1 󵄨 󵄨p−1 󵄨 󵄨 + c2 ( |u(x) + 󵄨󵄨󵄨v(x)󵄨󵄨󵄨 + (󵄨󵄨󵄨∇u(x) + ∇v(x)󵄨󵄨󵄨) ) 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 =: h(x). The above estimate is obtained by majorization of the integrand above, setting s = 1. We observe, that since v ∈ W01,p (𝒟), both v and |∇v| are in Lp and by standard use of Hölder’ s inequality we may obtain that h ∈ L1 (𝒟). We thus conclude that, for every v ∈ W01,p (𝒟), DF(u; v) := ⟨DF(u), v⟩ ˆ 𝜕 𝜕 ( f (x, u(x), ∇u(x))v(x) + f (x, u(x), ∇u(x)) ⋅ ∇v(x))dx, = 𝜕y 𝜕z 𝒟 where DF(u) is now understood as an element of (W01,p (𝒟))∗ = W −1,p (𝒟). Therefore, if uo is a minimizer then it must hold that ⋆

0 = ⟨DF(uo ), v⟩ ˆ 𝜕 𝜕 = ( f (x, uo (x), ∇uo (x))v(x) + f (x, uo (x), ∇u(x)) ⋅ ∇v(x))dx, 𝜕y 𝜕z 𝒟 for all v ∈ W01,p (𝒟), which is (6.54). A further integration by parts leads to the weak form (6.55) of the Euler–Lagrange equation. Assume now that (y, z) 󳨃→ f (x, y, z) is convex and consider any solution uo of the Euler–Lagrange equation (6.54). By the convexity and differentiability of f for any u we have that f (x, u(x), ∇u(x)) − f (x, uo (x), ∇uo (x)) ≥

𝜕 f (x, uo (x), ∇uo (x))(u(x) − uo (x)) 𝜕y 𝜕 + f (x, uo (x), ∇uo (x)) ⋅ ∇(u − uo )(x). 𝜕z

We integrate over all x ∈ 𝒟, and using u − uo ∈ W01,p as a test function in the Euler– Lagrange equation (6.54), we conclude that F(u) − F(uo ) ≥ 0; hence, uo is a minimizer for F. Remark 6.5.3. Using more sophisticated techniques, one may obtain related results under more relaxed assumptions. We will not enter into details here but we refer the interested reader to the detailed discussion in Section 3.4.2 in [50].

6.6 Regularity results in the calculus of variations We now consider the problem of regularity of minimizers of integral functionals. As we will see under certain conditions on the function f , the minimizer of the integral

280 | 6 The calculus of variations functional F, is not just in W 1,p (𝒟) but enjoys further regularity properties. We have already seen that in Section 6.2.2, within the simpler framework of the Poisson equation, focusing not so much on the properties of the corresponding functional but rather on the related PDE corresponding to the first-order condition. In this section, we consider a more general problem concerning regularity of minimizers, focusing on the properties of the functional which is minimized. We present an introduction to a beautiful theory, initiated by De Giorgi and taken up and further developed by many important researchers which through some delicate analytical estimates allows us to establish Hölder continuity of the minimizer. Then, using an important technique, the difference quotient technique introduced by Nirenberg, which approximates derivatives of minimizers in terms of difference quotients, combined with the Euler–Lagrange equation, we may obtain results on the higher differentiability of the minimizer (e. g., show 2,p that u ∈ Wloc (𝒟)). The problem of regularity is a long standing, highly interesting and important part of the calculus of variations. As a matter of fact, Hilbert stressed its importance in the formulation of his 19th problem. This short introduction is only skimming the surface of this deep subject, we refer the reader to the monograph of Giusti (see [67]) as well as the recent lecture notes of Beck (see [21]), on which the presentation of this section is based. The proofs are long and technical so for the faint hearted we summarize the main strategy here in case one wishes to skip the next section which contains the details, and move directly to Section 6.6.2. The key point to proving the Hölder continuity of minimizers is the observation (see Proposition 6.6.7) that minimizers u satisfy an important class of inequalities called Caccioppoli-type inequalities, ˆ A+ (k,x

ˆ

0 ,r)

󵄨󵄨 󵄨p −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 ((R − r)

A− (k,x

󵄨p

0 ,r)

󵄨󵄨 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 ((R − r)

ˆ A+ (k,x

ˆ

0 ,R)

A− (k,x

0 ,R)

p 󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A+ (k, x0 , R)󵄨󵄨󵄨), p 󵄨 󵄨 (k − u(x)) dx + 󵄨󵄨󵄨A− (k, x0 , R)󵄨󵄨󵄨),

where A+ (k, x0 , R) := {x ∈ B(x0 , R) : u(x) > k}, and A− (k, x0 , R) := {x ∈ B(x0 , R) : u(x) < k} are the local (within the ball B(x0 , R)) super and sublevel sets for u respectively at level k, using the notation |A| for the Lebesgue measure of any set A. It turns out that if a function u satisfies Caccioppoli-type inequalities for all levels k, (the class of such functions is called the De Giorgi class) then it is locally Hölder continuous with an appropriate exponent α (see Theorem 6.6.3). This can be proved, by noting that the Caccioppoli inequalities for all levels k (or for all levels above a critical one) provide an important estimate for the oscillation of the function u, defined by osc(x0 , R) :=

sup u(x) −

x∈B(x0 ,R)

inf

x∈B(x0 ,R)

u(x),

6.6 Regularity results in the calculus of variations | 281

of the form ρ ρ osc(x0 , ) ≤ c + λ osc(x0 , ρ), 4 4

∀ρ such that 0 < ρ ≤ R0 ,

which eventually leads to an estimate of the form α

r osc(x0 , r) ≤ c[( ) + r α ], R for appropriate c > 0 and α ∈ (0, 1). This last estimate guarantees the local Hölder continuity of u, with exponent α (see Theorem 1.8.8). 6.6.1 The De Giorgi class We begin by introducing an important class of functions, the De Giorgi class [21, 67], whose properties play a crucial role in the regularity theory in the calculus of variations. Definition 6.6.1 (The De Giorgi class). Consider a function u ∈ W 1,p (𝒟), 1 < p < ∞, and for any ball B(x0 , R) ⊂ 𝒟 define the superlevel set A+ (k, x0 , R) := {x ∈ B(x0 , R) : u(x) > k}, and the sublevel set A− (k, x0 , R) := {x ∈ B(x0 , R) : u(x) < k}. Using the notation |A| for the Lebesgue measure of any set A, we introduce the Caccioppoli-type inequalities ˆ (C+)

A+ (k,x0 ,r)

ˆ (C−)

A− (k,x0 ,r)

󵄨󵄨 󵄨p −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 ((R − r) 󵄨󵄨 󵄨p −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 ((R − r)

ˆ A+ (k,x0 ,R)

ˆ

A− (k,x0 ,R)

p 󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A+ (k, x0 , R)󵄨󵄨󵄨),

(6.57) p 󵄨 󵄨 (k − u(x)) dx + 󵄨󵄨󵄨A− (k, x0 , R)󵄨󵄨󵄨),

and define as 1,p DGp± (k0 , R0 , 𝒟) := {u ∈ Wloc (𝒟) : ∃c0 , k0 , R0 such that

∀ B(x0 , r) ⊂⊂ B(x0 , R) ⊂ 𝒟, with R ≤ R0 , and ∀ ± k ≥ k0 , (6.57)(C±) hold}.

1,p (i) The set DGp (k0 , R0 , 𝒟) = DGp+ (k0 , R0 , 𝒟) ∩ DGp− (k0 , R0 , 𝒟) ⊂ Wloc (𝒟) is called the De Giorgi class at level k0 . (ii) If a function u satisfies the Caccioppoli inequalities (6.57) for all k ∈ ℝ, i. e., u ∈ ⋂k0 ∈ℝ DGp (k0 , R0 , 𝒟) =: DGp (R0 , 𝒟), we say that u belongs to the De Giorgi class.

282 | 6 The calculus of variations (iii) If in the Caccioppoli inequalities (6.57) the second term on the right-hand side is missing the corresponding De Giorgi class is called homogeneous and is denoted by DGHp (𝒟). The motivation for defining the De Giorgi class arises from the fact that minimizers of integral functionals satisfying certain growth conditions can be shown to satisfy the above Caccioppoli inequalities and, therefore, belong to the De Giorgi class. An exam´ ple is the minimizers of integral functionals of the form F(u) = 𝒟 f (x, u(x), ∇u(x))dx if the Carathéodory function f satisfies the growth condition |z|p ≤ f (x, y, z) ≤ L(1 + |z|p ) for every y ∈ ℝ, z ∈ ℝd , and a. e. x ∈ 𝒟 for a suitable constant L > 0 (see Section 6.6.2). Remark 6.6.2. The De Giorgi class is often defined in terms of more general Caccioppoli-type inequalities. For instance, in [67], the Caccioppoli inequalities are stated as ˆ A+ (k,r)

󵄨󵄨 󵄨p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 (

c (R − r)p

ˆ

p 󵄨 󵄨1− p +ϵ (u(x) − k) dx + (cNH + k p R−dϵ )󵄨󵄨󵄨A+ (k, R)󵄨󵄨󵄨 d ),

A+ (k,r)

(6.58) for 0 < ϵ ≤ dp , and c0 , cNH positive constants. The reason for generalizing the De Giorgi class as above, is because minimizers or quasi-minimizers for functionals satisfying more general growth conditions than |z|p ≤ f (x, y, z) ≤ L(1 + |z|p ), e. g., L0 |z|p − a(x)|y|q − b(x) ≤ f (x, y, z) ≤ L1 |z|p + a(x)|y|q + b(x),

(6.59)

for suitable exponent q and positive functions a, b, can be shown to satisfy Caccioppoli-type inequalities of the above form (see Remark 6.6.9 below as well as Section 6.9.4 in the Appendix of the chapter). Importantly, if a = 0 in (6.59) then we may take cNH = 0 in the generalized Caccioppoli inequality (6.58). The results concerning local boundedness for functions in the De Giorgi class can be extended for the above generalization using very similar arguments (see [67] for detailed proofs). In particular, using similar techniques as the one used in Theorem 6.6.3, it can be shown (see Theorem 7.2 in [67]) that any function in DGp+ (𝒟) is locally bounded from above in 𝒟 and for any x0 ∈ 𝒟 and R sufficiently small, with an estimate given in terms of cubes Q(x0 , R) rather than balls B(x0 , R) as sup u ≤ c2 ((

Q(x0 , R2 )

1 p

+ p

Q(x0 ,R)

dϵ

(u ) dx) + k0 + cNH R p ),

(6.60)

with cNH being the same constant as in (6.58). In fact, under the same assumptions (see Theorem 7.3 in [67] or Theorem 1 in [56]) estimate (6.60) may be generalized for any ρ < R (with R sufficiently small) and any q > 0, as 1 sup u ≤ c3 (( (R − ρ)d Q(x0 ,ρ)

ˆ Q(x0 ,R)

+ q

1 q

dϵ

(u ) dx) + k0 + cNH R p ).

(6.61)

6.6 Regularity results in the calculus of variations | 283

An important result of De Giorgi (see, e. g., [21] or [67]) is that membership of functions in the De Giorgi classes DGp± (k0 , R0 , 𝒟) implies boundedness for the function above and below, respectively, while membership of functions in DGp (R0 , 𝒟) implies local Hölder continuity for this function. This result is stated in the following theorem, whose proof is rather long and technical and may be omitted at first reading. Theorem 6.6.3 (De Giorgi). Let u ∈ DGp (k0 , R0 , 𝒟). (i) Then, for every ball B(x0 , R) ⊂ 𝒟, with R ≤ R0 , there exists a constant cDG (depending only on d, p and the constant c0 ) such that ˆ

󵄨 − −d p+1 p 󵄨

− k0 − cDG (R + R ≤

inf

x∈B(x0 , R2 )

u(x) ≤

󵄨 󵄨󵄨A (−k0 , x0 , R)󵄨󵄨󵄨(

A− (−k0 ,x0 ,R)

sup u(x)

x∈B(x0 , R2 )

ˆ

󵄨 + −d p+1 p 󵄨

≤ k0 + cDG (R + R

p

󵄨 󵄨󵄨A (k0 , x0 , R)󵄨󵄨󵄨(

1/p

(−u(x) − k0 ) dx)

p

A+ (k0 ,x0 ,R)

)

1/p

(+u(x) − k0 ) dx)

).

(ii) If p > 1 and u satisfies the Caccioppoli inequalities (6.57) for all k ∈ ℝ and R0 = 1, then u is locally Hölder continuous in 𝒟, i. e., u ∈ C 0,α (𝒟) for some α > 0 depending on d, p and c0 . Proof. See Section 6.9.2. Remark 6.6.4. The estimate for the upper bound in Theorem 6.6.3 can be given in terms of the mean value integral of the excess function on B(x0 , R). This can be seen by expressing ˆ

󵄨 + −d p+1 p 󵄨

R

󵄨 󵄨󵄨A (k0 , x0 , R)󵄨󵄨󵄨( ˆ

= (R−d ≤ c1 (

p

A+ (k0 ,x0 ,R)

(u(x) − k0 ) dx) + p

B(x0 ,R)

1/p

p

A+ (k0 ,x0 ,R)

1/p

(u(x) − k0 ) dx)

󵄨 󵄨 (R−d 󵄨󵄨󵄨A+ (k0 , x0 , R)󵄨󵄨󵄨)

1/p

((u(x) − k0 ) ) dx)

,

for an appropriate constant c2 > 0, independent of R, since A+ (k0 , x0 , R) ⊂ B(x0 , R) and |B(x0 , R)| = ωd Rd , where ωd is the volume of the unit ball in ℝd . In fact, as shown by Di Benedetto and Trudinger (see Theorem 3 in [56]) positive functions that belong to the De Giorgi class satisfy a Harnack-type inequality. We present the result in its form for functions satisfying the generalized Caccioppoli inequalities (6.58).

284 | 6 The calculus of variations Theorem 6.6.5 (Harnack inequality [56]). If u ≥ 0 and u ∈ DGp (𝒟), (with cNH = 0 and k0 = 0) then for every σ ∈ (0, 1) there exists a constant cH = cH (p, d, σ, c0 ) such that6 sup u ≤ cH

B(0,σR)

inf u.

B(0,σR)

(6.62)

Proof. See [56], see also Section 6.9.3 for a sketch. The Harnack inequality is an important result which provides useful information for solutions (or subsolutions) of elliptic-type PDEs. From this, one may derive the maximum principle7 as well as other useful qualitative and quantitative properties of solutions (or sub and supersolutions). While the Harnack inequality can be derived from the PDE directly, it is useful to note that the important contribution of the approach of [56] is that it derives the Harnack inequality directly from the variational integral and not using at all the corresponding Euler–Lagrange equation; hence, this approach is general and does not require any additional smoothness properties on f . 6.6.2 Hölder continuity of minimizers We now show that minimizers belong to the De Giorgi class, and hence, by Theorem 6.6.3(ii) enjoy Hölder continuity properties. Definition 6.6.6 (Q-minimizers and minimizers). Consider any open subset 𝒟󸀠 ⊂ 𝒟 and a Q ≥ 1. A function u ∈ W 1,p (𝒟) such that F(u; 𝒟󸀠 ) ≤ QF(u + ϕ; 𝒟󸀠 ) for any 𝒟󸀠 ⊂ 𝒟 and any ϕ ∈ W01,p (𝒟), is called a Q-minimizer for F. If the above property holds for every ϕ ≤ 0 (resp., ϕ ≥ 0), it is called a sub (resp., super) Q-minimizer. When Q = 1, we recover the standard notion of minimizer (or sub and superminimizer, resp.). The notion of Q-minimizers and sub/superminimizers is related to solutions and sub/supersolutions of PDEs, a connection that can be made using the Euler–Lagrange equation for the variational problem (if this exists). The following proposition provides a very important step toward the regularity of solutions to variational problems. Proposition 6.6.7. Assume that F : W 1,p (𝒟) → ℝ, p > 1, is defined as in (6.40), with f satisfying the growth conditions |z|p ≤ f (x, u, z) ≤ L(1 + |z|p ),

(6.63)

for some L > 0. dϵ

6 In general, it satisfies the generalized Harnack inequality supB(x0 ,σR) u ≤ cH (infB(x0 ,σR) u + cNH R p ). 7 This can be seen as follows: Assume u is in the homogeneous De Giorgi class, so it is continuous. Let its minimum value be 0 without loss of generality. The set Ao := {x ∈ 𝒟 : u(x) = 0} is closed (by continuity). Furthermore, using the Harnack inequality (6.62) we can see that is is also open. If 𝒟 is connected, then Ao = 𝒟 hence, u = 0 over the whole of 𝒟 (see, e. g., [67]).

6.6 Regularity results in the calculus of variations | 285

(i) If u ∈ W 1,p (𝒟) is a sub-Q-minimizer for F, there exists c = c(p, L, Q) such that for every k ∈ ℝ and every pair B(x0 , r) ⊂⊂ B(x0 , R) we have that u satisfies the Caccioppolitype inequality, ˆ A+ (k,x

0 ,r)

ˆ

󵄨p 󵄨󵄨 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c((R − r)

A+ (k,x

0 ,R)

p 󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A+ (k, x0 , R)󵄨󵄨󵄨). (6.64)

(ii) If u ∈ W 1,p (𝒟) is a super-Q-minimizer for F, there exists c󸀠 = c󸀠 (p, L, Q) such that for every k ∈ ℝ and every pair B(x0 , r) ⊂⊂ B(x0 , R) we have that u satisfies the Caccioppoli-type inequality, ˆ A− (k,x

0 ,r)

󵄨󵄨 󵄨p 󸀠 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c ((R − r)

ˆ A− (k,x

0 ,R)

p 󵄨 󵄨 (k − u(x)) dx + 󵄨󵄨󵄨A− (k, x0 , R)󵄨󵄨󵄨). (6.65)

Proof. (i) The proof proceeds in 3 steps. 1. For any r1 , r2 such that r ≤ r1 < r2 ≤ R and k ∈ ℝ we will show that any sub-Qminimizer of F, for a suitable θ < 1, satisfies the estimate ‖∇u‖pLp (A+ (k,x

0 ,r1 ))

≤ θ‖∇u‖pLp (A+ (k,x

0 ,r2 ))

󵄩 󵄩p + 󵄩󵄩󵄩(u − k)󵄩󵄩󵄩Lp (A+ (k,x

0 ,r2 ))

󵄨 󵄨 + 󵄨󵄨󵄨A+ (k, x0 , r2 )󵄨󵄨󵄨.

(6.66)

Consider any r1 , r2 such that r ≤ r1 < r2 ≤ R and choose a cut off function 2 . ψ ∈ Cc∞ (B(x0 , r2 )), such that 0 ≤ ψ ≤ 1 and ψ ≡ 1 in B(x0 , r1 ) with |∇ ψ| ≤ r −r 2

1

Since u is a sub-Q-minimizer we choose as test function ϕ = −ψ(u − k)+ ≤ 0 and F(u; A+ (k, x0 , r2 )) ≤ Q F(u + ϕ; A+ (k, x0 , r2 )), where the set8 A+ (k, x0 , r2 ) is the intersection of the k superlevel set for u with the ball B(x0 , r2 ). To simplify the notation in the intermediate calculations, we will use the notation Ai := A+ (k, x0 , ri ), i = 1, 2 and note that A1 ⊂ A2 . Using the growth condition for the functional, we see that ˆ

ˆ 󵄨󵄨 󵄨p (6.63) f (x, u(x), ∇ u(x))dx 󵄨󵄨∇ u(x)󵄨󵄨󵄨 dx ≤ A2 A ˆ 2 ≤Q f (x, u(x) + ϕ(x), ∇ (u + ϕ)(x))dx A2

(6.63)

≤ QL

ˆ A2

(6.67)

󵄨 󵄨p (1 + 󵄨󵄨󵄨∇ (u + ϕ)(x)󵄨󵄨󵄨 )dx.

Since ∇ (u − k)+ = ∇ u1{u>k} , by a standard algebraic inequality and the Leibnitz rule, using the properties of the cutoff function ψ, we have that p󵄨 p 󵄨󵄨 󵄨p 󵄨p −p 󵄨󵄨∇ (u + ϕ)(x)󵄨󵄨󵄨 ≤ c[(1 − ψ(x)) 󵄨󵄨󵄨∇ u(x)󵄨󵄨󵄨 + (r2 − r1 ) (u(x) − k) ],

8 This is not necessarily an open set, but we may approximate it by an open set.

on A2 ,

286 | 6 The calculus of variations for an appropriate constant c. Since ψ ≡ 1 on B(x0 , r1 ), we have that 1 − ψ ≡ 0 on this set; hence, integrating the above inequality over A2 := A+ (k, x0 , r2 ) we see that the first term contributes to the integral only over A+ (k, x0 , r2 ) ∩ (B(x0 , r2 ) \ B(x0 , r1 )) = A+ (k, x0 , r2 ) \ A+ (k, x0 , r1 ) = A2 \ A1 . This observation leads to the estimate ˆ

󵄨p 󵄨󵄨 󵄨󵄨∇(u + ϕ)(x)󵄨󵄨󵄨 dx A2 ˆ ˆ p 󵄨󵄨 󵄨󵄨p ≤ c[ (1 + (r2 − r1 )−p (u(x) − k) )dx] 󵄨󵄨∇u(x)󵄨󵄨 dx + A2 \A1

A2

ˆ ≤ c[

󵄨󵄨 󵄨p −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + (r2 − r1 )

A2 \A1

ˆ

(6.68)

p

A2

(u(x) − k) )dx + |A2 |].

Combining (6.67) with (6.68) along with the trivial observation that A1 := A+ (k, x0 , r1 ) ⊂ A2 := A+ (k, x0 , r2 ), since r1 < r2 , we obtain the estimate ˆ A1

|∇ u|p dx ≤ c[

ˆ A2 \A1

󵄨󵄨 󵄨p −p 󵄨󵄨∇ u(x)󵄨󵄨󵄨 dx + (r2 − r1 )

ˆ

p

A2

(u(x) − k) )dx + |A2 |].

(6.69)

Note that (6.69) is very close to (6.66), but not quite so. The fact that on the right-hand side of (6.69) we have the integral of |∇ u|p on A2 \ A1 := A+ (k, x0 , r2 ) \ A+ (k, x0 , r1 ) rather than on A2 := A+ (k, x0 , r2 ) (as it appears on the left-hand side) is annoying. As the integral on the right-hand side is on an annulus type region (i. e., on A2 := A+ (k, x0 , r2 ) with a hole A1 := A+ (k, x0 , r1 )), we employ the so-called hole-filling technique of Wilder, ´ which consists of adding the term c A |∇ u|p dx on both sides of (6.69), we obtain upon 1 dividing by 1 + c and redefining the constant c (using the same notation) and setting c θ = c+1 < 1, that ˆ A1

ˆ

󵄨󵄨 󵄨p 󵄨󵄨∇ u(x)󵄨󵄨󵄨 dx ≤ θ

A2

󵄨󵄨 󵄨p −p 󵄨󵄨∇ u(x)󵄨󵄨󵄨 dx + (r2 − r1 )

ˆ A2

p

(u(x) − k) )dx + |A2 |,

(6.70)

c < c. This is (6.66). where we also used the elementary estimate c+1 2. We will show that the Cacciopoli-type inequality

‖∇u‖pLp (A+ (k,x

0 ,r))

≤ c((R − r)−p ‖u − k‖pLp (A+ (k,x

0 ,R))

󵄨 󵄨 + 󵄨󵄨󵄨A+ (k, x0 , R)󵄨󵄨󵄨),

holds, by iterating appropriately (6.66) (equiv. (6.70)). We define the real valued functions ˆ p ̂1 (ρ) := ̂2 (ρ) := 󵄨󵄨󵄨󵄨A+ (k, x0 , ρ)󵄨󵄨󵄨󵄨, φ (u(x) − k) dx, φ + A (k,x0 ,ρ) ˆ 󵄨󵄨 󵄨p ̂(ρ1 , ρ2 ) := ρ−p ̂ ̂ φ(ρ) := 󵄨󵄨∇ u(x)󵄨󵄨󵄨 dx, φ 1 φ1 (ρ2 ) + φ2 (ρ2 ),

(6.71)

(6.72)

A+ (k,x0 ,ρ)

̂ is nondecreasing in the second variable. and note that φ is nondecreasing, while φ

6.6 Regularity results in the calculus of variations | 287

Since r1 , r2 in step 1 were arbitrary we pick any ρ, ρ󸀠 such that r ≤ ρ ≤ ρ󸀠 ≤ R, set r1 = ρ and r2 = ρ󸀠 and we redress the estimate (6.66) (equiv. (6.70)), using the functions defined in (6.72) as ̂(ρ󸀠 − ρ, ρ󸀠 ), φ(ρ) ≤ θφ(ρ󸀠 ) + φ

∀ r ≤ ρ ≤ ρ󸀠 ≤ R.

(6.73)

We claim that (6.73) may be iterated to yield (6.74)

̂(r, R), φ(r) ≤ cφ

for an appropriate constant c (depending on θ) which is the required estimate (6.71). 3. It remains to prove that (6.73) implies (6.74). To see this, consider the increasing sequence ρn = r + (1 − λn )(R − r), n = 0, 1, . . ., for an appropriate 0 < λ < 1, such that λ−p θ < 1, and note that ρn − ρn−1 = λn−1 (1 − λ)(R − r). We write the inequality choosing each time the pair ρ = ρn , ρ󸀠 = ρn+1 to get ̂1 (ρn+1 ) + φ ̂2 (ρn+1 ), φ(ρn ) ≤ θφ(ρn+1 ) + KM n−1 φ

n = 0, . . . , N − 1,

where M := λ−p ,

K := (1 − λ)−p (R − r)−p ,

and upon multiplying each one by θn and adding we obtain that N−1

N−1

m=0

m=0

̂1 (ρm+1 ) + ∑ θm φ ̂2 (ρm+1 ) φ(r) ≤ θN φ(ρN ) + KM −1 ∑ (θM)m φ N−1

N−1

m=0

m=0

̂1 (R) + ( ∑ θm )φ ̂2 (R) ≤ θN φ(ρN ) + KM −1 ( ∑ (θM)m )φ ̂1 (R) + c2 φ ̂2 (R), ≤ θN φ(ρN ) + Kc1 φ

̂i (ρm ) ≤ φ ̂i (R), for every m = where for the second estimate we used the fact that φ ̂i are nondecreasing and ρm is increasing), and we have set 1, . . . , N − 1, i = 1, 2 (since φ ∞

c1 := M −1 ∑ (θM)m = m=0

∞

c2 := ∑ θm = m=0

M −1 , 1 − θM

1 . 1−θ

Passing to the limit as N → ∞ and choosing c = max(c1 , c2 ), we deduce (6.74). The proof is complete. (ii) If u is a super-Q-minimizer, then −u is a sub-Q-minimizer, and the proof follows by (i) upon replacing u with −u and k by −k. We are now ready to conclude the local Hölder continuity of minimizers.

288 | 6 The calculus of variations Theorem 6.6.8 (Hölder continuity of Q-minimizers). Assume that F : W 1,p (𝒟) → ℝ, p > 1, is defined as in (6.40) with f satisfying the growth conditions |z|p ≤ f (x, u, z) ≤ L(1 + |z|p ),

(6.75)

for some L > 0. Then any Q-minimizer is locally Hölder continuous. Proof. Any Q-minimizer is in DGp (R0 , 𝒟) by Proposition 6.6.7. Then by Theorem 6.6.3 u is locally Hölder continuous with a suitable exponent α. Remark 6.6.9. The result of Proposition 6.6.7 can be generalized in order to deal with more general structure conditions (see, e. g., [67]). Assume that f is a Carathéodory function satisfying the structure condition L0 |z|p − b(x)|y|q − a(x) ≤ f (x, y, z) ≤ L1 |z|p + b(x)|y|q + a(x), where 1 < p ≤ q < sp =

pd d−p

(6.76)

and a, b are positive functions in Ls (𝒟) and Lr (𝒟), respec-

tively, with s > dp and r > Then, for every x0 ∈ 𝒟 there exists a R0 depending on a and b such that for every 0 < r < R < min(R0 , dist(x0 , 𝜕𝒟)), and every k ≥ 0 we have the Caccioppoli-like inequality ˆ ˆ c p 󵄨 󵄨1− p +ϵ 󵄨󵄨 󵄨p (u(x) − k) dx + c(‖a‖Ls (𝒟) + k p R−dϵ )󵄨󵄨󵄨A+ (k, R)󵄨󵄨󵄨 d , 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ p (R − r) A+ (k,r) A+ (k,r) where ϵ > 0 is such that

sp . sp −q

1 s

=

p d

− ϵ and

1 r

= 1−

q sp

− ϵ. Note that if s = r = ∞, then p

we may choose ϵ = dp , in which case k p R−dϵ |A+ (k, R)|1− n +ϵ = ( Rk )p |A+ (k, R)|, leading to a Caccioppoli inequality very similar to (6.64). This is a local result, as the radius of the ball in which it holds depends on the data of the problem near x0 . The proof of this result follows along the same lines as in Proposition 6.6.7, with a little extra complications (see Section 6.9.4 for a simplified case or [67] for the result in its full generality). One may also note that if local boundedness for the minimizer has been shown (see Remark 6.6.2) then the structure condition (6.76) can be replaced by a local structure condition of the form |z|p − ac (x) ≤ f (x, y, z) ≤ L|z|p + ac (x), with the positive function ac depending on the local upper bound. Therefore, by Proposition 6.6.7, a quasi-minimizer will satisfy Caccioppoli-type inequalities of the form (6.57). Moreover, Q-minimizers for functionals satisfying the structure condition (6.76) satisfy Harnacktype inequalities, in particular an inequality of the form given in Theorem 6.6.5, as long as a = 0. 6.6.3 Further regularity Under more restrictive assumptions on the data of the problem, one may show that the minimizer enjoys further regularity properties.

6.6 Regularity results in the calculus of variations | 289

In order to proceed, we will need to define the difference and the translation operators in the direction i, denoted by Δh,i and τh,i , respectively. Definition 6.6.10 (Difference and translation operators). For any function u : 𝒟 → ℝ, any direction vector ei , and h ∈ ℝ, we may define (i) The translation operator τh,i u(x) := u(x + hei ), τ u−u u(x+hei )−u(x) (ii) The partial difference operator Δh,i (x) := = h,ih (x). h Strictly speaking the above operators are well defined for functions on 𝒟h := {x ∈ 𝒟 : dist(x, 𝜕𝒟) > |h|}. The idea behind introducing the partial difference operator is that in the limit as h → 0, under certain conditions, we may approximate the partial derivatives of a function, in terms of this operator. In fact, as will be shown shortly, uniform Lp bounds for the Δh,i u for some p > 1 guarantee membership of u ∈ W 1,p . John Nirenberg used this idea in producing a theory for the regularity of elliptic problems. Here, we adapt some of these ideas, following mainly the approach of [63] to discuss further regularity results for the solution of variational problems. We start by collecting a number of useful results concerning the difference operator. Proposition 6.6.11 (Properties of the difference operator). (i) The Leibnitz property Δh,i (u1 u2 ) = (τh,i u1 )(Δh,i u2 ) + (Δh,i u1 )u2 is satisfied for any u1 , u2 . (ii) Any function u ∈ L1 (𝒟) satisfies the integration by parts formula ˆ 𝒟

ϕ(x)Δh,i u(x)dx = −

ˆ 𝒟

u(x)Δ−h,i ϕ(x)dx,

(iii) If u ∈ W 1,p (𝒟), then Δh,i u ∈ W 1,p (𝒟) and

∀ ϕ ∈ Cc1 (𝒟),

𝜕 (Δ u) 𝜕xj h,i ∗

|h| < dist(supp ϕ, 𝜕𝒟).

= Δh,i ( 𝜕x𝜕 u). j

(iv) Consider u ∈ Lploc (𝒟) with 1 < p ≤ ∞ and let p be the conjugate exponent p1 + p1∗ = 1. Then for any i ∈ {1, . . . , n} the partial derivative

𝜕u 𝜕xi

∈ Lploc (𝒟) if and only if for every

𝒟󸀠 ⊂⊂ 𝒟 there exists a constant c (depending on 𝒟󸀠 ) such that

󵄨󵄨ˆ 󵄨󵄨 󵄨󵄨 󵄨 (Δh,i u)(x)ϕ(x)dx󵄨󵄨󵄨 ≤ c‖ϕ‖Lp∗ (𝒟󸀠 ) , 󵄨󵄨 󵄨󵄨 𝒟󸀠 󵄨󵄨

∀ ϕ ∈ Cc1 (𝒟󸀠 ).

Proof. Claims (i)–(iii) follow by elementary algebraic manipulations, integration by parts using the translation invariance and the fact that W 1,p is a vector space, respec𝜕u ∈ tively, and are left as exercises to the reader. We only sketch the proof of (iv). If 𝜕x i ´ 1 𝜕u p Lloc (𝒟), then we express (τh,i u−u)(x) = 0 𝜕x (x +hei t)dt and estimate ‖τh,i u−u‖Lp (𝒟󸀠 ) ≤ i ´ 𝜕u |h| ‖ 𝜕x ‖Lp (𝒟󸀠 ) . We then estimate | 𝒟󸀠 (Δh,i u)(x)ϕ(x)dx| with the use of the Hölder ini

equality. For the reverse implication, fix 𝒟󸀠 ⊂⊂ 𝒟 and note that for ϕ ∈ Cc1 (𝒟󸀠 ) we

290 | 6 The calculus of variations have that limh→0 Δ−h,i ϕ(x) = integration by parts formula

𝜕 ϕ(x), 𝜕xi

so that using dominated convergence and the

ˆ ˆ 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨󵄨 𝜕 󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 lim u(x)(Δ−h,i ϕ)(x)󵄨󵄨󵄨 = ϕ(x)dx u(x) 󵄨 󵄨 󵄨 󵄨 󵄨󵄨 󸀠 󵄨󵄨 󵄨󵄨 󵄨󵄨h→0 𝒟󸀠 𝜕xi 󵄨 𝒟 ˆ 󵄨󵄨 󵄨󵄨 󵄨 󵄨 (Δ u)(x)ϕ(x)󵄨󵄨󵄨 ≤ c‖ϕ‖Lp∗ (𝒟󸀠 ) , = 󵄨󵄨󵄨− lim 󵄨󵄨 󵄨󵄨 h→0 𝒟󸀠 h,i with the last estimate following by duality since we have assumed that Δh,i u ∈ Lp (𝒟󸀠 ). Statement (iv) in Proposition 6.6.11 is crucial and will be used both in the direct and in the converse direction. In particular, it is important to stress that if u ∈ W 1,p (𝒟|h| ) then for every i ∈ {1, . . . , d}, ‖Δh,i u‖Lp (𝒟) ≤ ‖∇u‖Lp (𝒟|h| ) , where 𝒟|h| is the |h|-neighborhood of 𝒟, as well as that for small enough h (h < h0 𝜕u ‖Lp (𝒟) . with h0 depending on the d and dist(𝒟󸀠 , 𝜕𝒟)) we have that ‖Δh,i u‖Lp (𝒟󸀠 ) ≤ c‖ 𝜕x i On the converse side, if ∀ 𝒟󸀠 ⊂⊂ 𝒟 ∃ c such that ∀ h < dist(𝒟󸀠 , 𝒟) we have ‖Δh,i u‖Lp (𝒟󸀠 ) ≤ c, 󵄩󵄩󵄩 𝜕u 󵄩󵄩󵄩 𝜕u then ∈ Lp (𝒟) and 󵄩󵄩󵄩 󵄩󵄩󵄩 ≤ c. 󵄩󵄩 𝜕xi 󵄩󵄩Lp (𝒟) 𝜕xi We present here a simple example that illustrates how the use of the partial dif2,2 ference operator as a test function may lead to Wloc (𝒟) estimates for the solution of a variational problem, or even more general elliptic problems. Starting from this estimate, one may employ more sophisticated techniques and obtain Hölder continuity of the derivatives of the solution. Following Evans (see [63]), let A = (A1 , . . . , Ad ) : ℝd → ℝd be a C 1 function and consider the nonlinear elliptic system −div ⋅ (A(∇u)) = g, u=0

In the special case where Ai =

𝜕f , 𝜕zi

in 𝒟,

(6.77)

on 𝜕𝒟.

i = 1, . . . , d, for some C 2 scalar valued function

f : ℝd → ℝ, the system (6.77) can be recognized as the Euler–Lagrange equation for ´ the minimization of the functional F(u) := 𝒟 f (∇u)dx. We will adopt the following assumptions on the vector field A.

Assumption 6.6.12. The vector field A : ℝd → ℝd satisfies the following: 𝜕A (i) There exists c0 > 0 such that for any z ∈ ℝd , it holds that ∑di,j=1 𝜕z i (z)ξi ξj ≥ c0 |ξ |2 , for every ξ = (ξ1 , . . . , ξd ) ∈ ℝd .

j

6.6 Regularity results in the calculus of variations | 291

(ii) There exists a constant c1 > 0 such that for every i ∈ {1, . . . , d} and z ∈ ℝd , it holds 𝜕A that | 𝜕z i (z)| < c1 . j

The above assumptions in the special case where A = ∇f (variational case) may be expressed in terms of the Hessian of the function f and can be related to the convexity properties of the function f . Furthermore, for the purposes of this section we have made rather heavy assumptions on A, similar results as the ones stated here can be proved under weaker assumptions, using more elaborate and technical arguments. Proposition 6.6.13. Under Assumption 6.6.12, on the vector field A, any solution u ∈ 2,2 W01,2 (𝒟) of (6.77) is also in Wloc (𝒟), and in particular, for any 𝒟󸀠 ⊂⊂ 𝒟 it holds that 󵄩󵄩 2 󵄩󵄩 󵄩󵄩D u󵄩󵄩L2 (𝒟󸀠 ) ≤ c(‖g‖L2 (𝒟) + ‖∇u‖L2 (𝒟) ), for an appropriate constant c (depending on 𝒟󸀠 ). Proof. Our aim is to show that for any open set 𝒟󸀠 ⊂⊂ 𝒟, it holds that ˆ ˆ ˆ 󵄨󵄨 2 󵄨󵄨 󵄨󵄨2 󵄨2 2 D u(x) g(x) dx + dx ≤ c( 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx), 󵄨󵄨 󵄨󵄨 𝒟󸀠

𝒟

𝒟

2,2 Wloc (𝒟). 󸀠

which implies that u ∈ The proof follows in 3 steps. 1. Fix an open set 𝒟 ⊂⊂ 𝒟 and choose an open set 𝒟󸀠󸀠 such that 𝒟󸀠 ⊂⊂ 𝒟󸀠󸀠 ⊂⊂ 𝒟. We also consider a cutoff function ψ ∈ Cc∞ (𝒟󸀠󸀠 ) such that ψ ≡ 1 on 𝒟󸀠 and 0 ≤ ψ ≤ 1. For a suitably small h, we fix i ∈ {1, . . . , d} and consider as test function for the weak formulation of the problem ϕ = −Δ−h,i (ψ2 Δh,i u). Using the integration by parts formula and the properties of the difference operator, the weak form yields d

ˆ

∑ j=1

𝒟

Δh,i (Aj (∇u))(x)

𝜕 (ψ2 Δh,i u)(x)dx = − 𝜕xj

ˆ 𝒟

g(x) Δ−h,i (ψ2 Δh,i u)(x)dx.

Using the more convenient notation v = ∇u (resp., componentwise vj =

𝜕u ) 𝜕xj

(6.78) and the

fact that the difference and the partial derivative operator commute, (6.78) can be expressed as d

∑ j=1

ˆ 𝒟

[ψ(x)2 Δh,i (Aj (∇u))(x)Δh,i vj (x) + 2ψ(x)

=−

ˆ 𝒟

𝜕ψ (x)Δh,i (Aj (∇u))(x)Δh,i u(x)]dx 𝜕xj

g(x) Δ−h,i (ψ2 Δh,i u)(x)dx.

Since Aj are differentiable, we have for any j = 1, . . . , d that for every pair wℓ = (wℓ,1 , . . . , wℓ,d ) ∈ ℝd , ℓ = 1, 2, Aj (w2 ) − Aj (w1 ) =

ˆ 0

1

d A (sw2 + (1 − s)w1 )ds ds j

292 | 6 The calculus of variations ˆ =

0

1 d

𝜕 A (sw2 + (1 − s)w1 )(w2,k − w1,k )ds 𝜕zk j k=1 ∑

ˆ 1 =( Dz Aj (sw2 + (1 − s)w1 )ds) ⋅ (w2 − w1 ). 0

We apply the above for the choice w2 = ∇u(x + hei ) and w1 = ∇u(x) for any x ∈ 𝒟, and recalling the definition of Δh,i (Aj (∇u)) = Δh,i (Aj (v)), we obtain Δh,i (Aj (∇u)) = Δh,i (Aj (v)) = d

= ∑( k=1

ˆ 0

1

1 (A (v(x + hei ) − Aj (v(x)) h j

𝜕 1 A (s∇u(x + hei ) + (1 − s)∇u(x))ds) (vk (x + hei ) − vk (x)) 𝜕zk j h

d

= ∑ ajk (x)Δh,i vk (x), k=1

where ajk (x) =

ˆ 0

1

𝜕 A (s∇u(x + hei ) + (1 − s)∇u(x))ds. 𝜕zk j

By the assumption on Aj , we have that the matrix A = (ajk )dj,k=1,... satisfies the uniform ellipticity condition d

∑ ajk zj zk ≥ c0 |z|2 ,

j,k=1

for some c0 > 0. We substitute the above in the weak formulation of the equation to get ˆ

d

∑ ψ(x)2 ajk (x)Δh,i vk (x)Δh,i vj (x)dx + 2

𝒟 j,k=1

ˆ

=−

𝒟

ˆ

d

∑ ψ(x)

𝒟 j,k=1

𝜕ψ (x)ajk (x)Δh,i vk (x)Δh,i u(x)dx 𝜕xj

g(x) Δ−h,i (ψ2 Δh,i u)(x)dx,

which lead to the following inequality: ˆ

d

∑ ψ(x)2 ajk (x)Δh,i vk (x)Δh,i vj (x)dx

𝒟 j,k=1

󵄨󵄨ˆ d 󵄨󵄨 󵄨ˆ 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 𝜕ψ 󵄨 ≤ 2󵄨󵄨󵄨 ∑ ψ(x) (x)ajk (x)Δh,i vk (x)Δh,i u(x)dx󵄨󵄨󵄨 + 󵄨󵄨󵄨 g(x) Δ−h,i (ψ2 Δh,i u(x))dx󵄨󵄨󵄨. 󵄨󵄨 𝒟 󵄨󵄨 󵄨󵄨 𝒟 󵄨󵄨 𝜕x j 󵄨 j,k=1 󵄨 (6.79)

6.6 Regularity results in the calculus of variations | 293

2. We now estimate each of the above terms separately as follows: For the first one, ˆ ˆ 󵄨2 󵄨 󵄨2 󵄨 ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ≤ c0 c0 𝒟 󸀠󸀠

ˆ

≤

𝒟 d

∑ ψ(x)2 ajk (x)Δh,i vk (x)Δh,i vj (x)dx,

(6.80)

𝒟 j,k=1

where starting from the right to the left we have used the ellipticity condition, and the fact that 𝒟󸀠󸀠 ⊂ 𝒟. For the second one, we estimate using the Cauchy–Schwarz inequality (weighted by an arbitrary constant ϵ1 ) as 󵄨󵄨󵄨ˆ d 󵄨󵄨󵄨 𝜕ψ 󵄨󵄨 󵄨 ∑ ψ(x) (x)ajk (x)Δh,i vk (x)Δh,i u(x)dx󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨 𝒟 󵄨󵄨 𝜕x j 󵄨 j,k=1 󵄨 ˆ 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 (6.81) ≤ c1 ψ(x)󵄨󵄨Δh,i v(x)󵄨󵄨 󵄨󵄨Δh,i u(x)󵄨󵄨dx 󸀠󸀠 𝒟 ˆ ˆ c 󵄨󵄨 󵄨2 󵄨 󵄨2 ≤ c1 ϵ1 ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx + 1 󵄨󵄨Δh,i u(x)󵄨󵄨󵄨 dx. 󸀠󸀠 󸀠󸀠 4ϵ1 𝒟 𝒟 For the third one by similar arguments (weighting by an arbitrary constant ϵ2 ), ˆ ˆ 󵄨󵄨ˆ 󵄨󵄨 1 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨2 2 2 2 g(x) dx + ϵ2 󵄨󵄨 g(x) Δ−h,i (ψ Δh,i u)(x)dx 󵄨󵄨 ≤ 󵄨󵄨Δ−h,i (ψ Δh,i u)(x)󵄨󵄨󵄨 dx. (6.82) 󵄨󵄨 𝒟 󵄨󵄨 4ϵ2 𝒟 𝒟

We must somehow estimate the second integral which consists of the differences of u. We use the notation w = Δh,i u and note that since ψ2 Δh,i u ∈ W 1,2 (𝒟), we have that ˆ ˆ ˆ 󵄨󵄨 󵄨2 󵄨󵄨 󵄨2 󵄨󵄨 󵄨2 2 2 󸀠 2 󵄨󵄨Δ−h,i (ψ Δh,i u)(x)󵄨󵄨󵄨 dx = 󵄨󵄨Δ−h,i (ψ w)(x)󵄨󵄨󵄨 dx ≤ c 󵄨󵄨∇(ψ w)(x)󵄨󵄨󵄨 dx 𝒟

𝒟

≤ c󸀠󸀠 ≤ c󸀠󸀠 =c

𝒟|h|

ˆ

𝒟|h|

ˆ

𝒟 󸀠󸀠

ˆ

󸀠󸀠 𝒟

󵄨 󵄨2 ψ(x)4 󵄨󵄨󵄨∇w(x)󵄨󵄨󵄨 dx + c󸀠󸀠 ψ(x)2 |∇w(x)|2 dx + c󸀠󸀠󸀠

ˆ

ˆ

󵄨2 ψ(x) 󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx + c󸀠󸀠󸀠 󸀠󸀠 2 󵄨󵄨

𝒟|h| 𝒟 󸀠󸀠

ˆ

󵄨 󵄨2 2ψ(x)2 󵄨󵄨󵄨∇ψ(x)󵄨󵄨󵄨 w(x)2 dx w(x)2 dx

𝒟 󸀠󸀠

w(x)2 dx,

where for the first integral we used the fact that ψ4 ≤ ψ2 (since 0 ≤ ψ ≤ 1), for both the fact that supp ψ = 𝒟󸀠󸀠 ⊂ 𝒟 and finally the observation that ∇w = ∇Δh,i u = Δh,i (∇u) = Δh,i v as the difference operator and the gradient operator commute. Substituting the above estimate in (6.82), we have for the third term that ˆ 󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨 g(x) Δ−h,i (ψ2 Δh,i u)(x)dx 󵄨󵄨󵄨 󵄨󵄨󵄨 𝒟 󵄨󵄨󵄨 (6.83) ˆ ˆ ˆ 1 󵄨 󵄨2 󵄨󵄨 󵄨󵄨2 g(x)2 dx + ϵ2 c󸀠󸀠 ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx + ϵ2 c󸀠󸀠󸀠 Δ u(x) dx, ≤ 󵄨󵄨 h,i 󵄨󵄨 4ϵ2 𝒟 𝒟 󸀠󸀠 𝒟 󸀠󸀠

upon substituting w = Δh,i u once more.

294 | 6 The calculus of variations 3. We use (6.80), (6.81) and (6.83) in the weak form estimate (6.79) to obtain upon combining like terms that ˆ 󵄨2 󵄨 󸀠󸀠 (c0 − ϵ1 c1 − ϵ2 c ) ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx 󸀠󸀠 𝒟 ˆ ˆ c1 1 󵄨󵄨2 󵄨󵄨 󸀠󸀠󸀠 ≤( dx + Δ u(x) g(x)2 dx, + ϵ2 c ) 󵄨󵄨 󵄨󵄨 h,i 󸀠󸀠 4 ϵ1 4 ϵ 𝒟 2 𝒟 c0 2

hence, choosing ϵ1 , ϵ2 so that c0 − ϵ1 c1 − ϵ2 c󸀠󸀠 ≥ redefining the constant c we end up with ˆ 𝒟 󸀠󸀠

ˆ

󵄨 󵄨2 ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ≤ c(

𝒟

≤ c(

ˆ

g(x)2 dx + 2

𝒟

g(x) dx +

> 0 and dividing with ˆ 𝒟 󸀠󸀠

ˆ

𝒟

󵄨󵄨 󵄨2 󵄨󵄨Δh,i u(x)󵄨󵄨󵄨 dx)

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx),

c0 2

and

(6.84)

where for the last estimate we used Proposition 6.6.11. Since 𝒟󸀠 ⊂ 𝒟󸀠󸀠 and ψ ≡ 1 on 𝒟󸀠 we have that ˆ ˆ ˆ ˆ 󵄨󵄨 󵄨2 󵄨 󵄨󵄨 󵄨2 󵄨2 ψ(x)2 󵄨󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ≤ c( g(x)2 dx + 󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ≤ 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx), 𝒟󸀠

𝒟 󸀠󸀠

𝒟

𝒟

and by the arbitrary nature of 𝒟󸀠󸀠 and the fact that inequality (6.84) holds for every h sufficiently small we conclude using once more Proposition 6.6.11 that ˆ 𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇v(x)󵄨󵄨󵄨 dx ≤ 󸀠

ˆ 𝒟

ˆ

󵄨2 ψ(x) 󵄨󵄨Δh,i v(x)󵄨󵄨󵄨 dx ≤ c( 󸀠󸀠 2 󵄨󵄨

𝒟

2

g(x) dx +

ˆ 𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx),

1,2 which implies that u ∈ Wloc (𝒟).

Since the constant c in the above estimate may depend on 𝒟󸀠 , the estimates in Proposition 6.6.13 are interior estimates and cannot be continued up to the boundary without further complications. We will present such a result for a more general problem in the next chapter, in Section 7.6.2. Furthermore, the method can be extended for systems of elliptic equations.

6.7 A semilinear elliptic problem and its variational formulation Consider the following semilinear elliptic problem: −Δu = f (x, u), u=0

in 𝒟,

on 𝜕𝒟

(6.85)

where f : 𝒟 × ℝ → ℝ is a continuous function, which under extra conditions on the function f (that will be discussed in detail shortly) is associated with the functional

6.7 A semilinear elliptic problem and its variational formulation

| 295

F : X → ℝ, F(u) =

1 2

ˆ 𝒟

󵄨2 󵄨󵄨 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx −

ˆ 𝒟

ˆ (

u(x)

0

f (x, s)ds)dx,

(6.86)

where X is an appropriately chosen Banach space, a possible choice for which being X := W01,2 (𝒟). Note that this is a special case of the general class of functionals studied in Sec´ tions 6.4 and 6.5; using the slightly modified notation F(u) = 𝒟 f ̄(x, u(x), ∇u(x))dx for the general functional of the above mentioned sections we have that f ̄(x, y, z) = ´y 1 |z|2 + 0 f (x, s)ds. The general theory developed in Sections 6.4 and 6.5, naturally 2 applies to this specific case, however, this simpler special and separable form, which often appears in various important applications, will serve as a nice example of how the general conditions may be sharpened, and on how the specific form of the functional may inherit to the corresponding Euler–Lagrange equation certain rather special properties, not necessarily valid in the general case. 6.7.1 The case where sub and supersolutions exist We start our study of semilinear problems considering first the case where the nonlinear function is such that a weak sub and supersolution exists. This allows the treatment of (6.85) without growth conditions on the nonlinearity f , as well as obtaining detailed and localized information on its solution. Definition 6.7.1 (Weak sub and supersolutions). Let C + := {v ∈ W01,2 (𝒟) : v(x) ≥ 0 a. e.}. (i) The function u ∈ W 1,2 (𝒟) is a weak subsolution of (6.85) if it is a weak solution of the inequality −Δu − f (x, u) ≤ 0, u ≤ 0,

in 𝒟,

or equivalently that ˆ ˆ ∇u(x) ⋅ ∇v(x)dx − f (x, u(x))v(x)dx ≤ 0, 𝒟

(6.87)

on 𝜕𝒟,

𝒟

∀ v ∈ C+ .

(ii) The function ū ∈ W 1,2 (𝒟) is a weak supersolution of (6.85) if it is a weak solution of the inequality −Δū − f (x, u)̄ ≥ 0,

ū ≥ 0,

in 𝒟,

on 𝜕𝒟,

(6.88)

296 | 6 The calculus of variations or equivalently that ˆ 𝒟

̄ ⋅ ∇v(x)dx − ∇u(x)

ˆ 𝒟

̄ f (x, u(x))v(x)dx ≥ 0,

∀ v ∈ C+ .

We now show (following [95]) how the information of existence of a sub and a supersolution for (6.85) in the weak sense, can be combined with the calculus of variations in order to provide existence of weak solutions. Proposition 6.7.2. Assume that f : 𝒟 ×ℝ → ℝ is continuous and such that a (weak) subsolution u and a (weak) supersolution ū exist for (6.85) (in the sense of Definition 6.7.1), with the property that u ≤ ū a. e. in 𝒟. Then there exists a weak solution u ∈ W01,2 (𝒟) of (6.85) with the property u ≤ u ≤ ū a. e. in 𝒟. Proof. Consider the modified function

f ̃(x, s) :=

f (x, u(x)) if s ≤ u(x), x ∈ 𝒟, { { { ̄ f (x, s) if u(x) ≤ s ≤ u(x), x ∈ 𝒟, { { { ̄ ̄ if s ≥ u(x), x ∈ 𝒟, {f (x, u(x))

´ ̃ s) = s f ̃(x, σ)dσ. It is easy to observe that the function Φ̃ is and its primitive Φ(x, 0 sublinear. We define the functional F̃ : X := W01,2 (𝒟) by 1 ̃ F(u) = 2

ˆ 𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx −

ˆ 𝒟

̃ u(x)) dx. Φ(x,

This functional is easily seen to be sequentialy weakly lower semicontinuous, and coercive by the sublinearity of Φ.̃ Note that the weak semicontinuity comes from the convexity of the functional in ∇u (see Theorem 6.3.10). Therefore, by a standard application of the direct method of the calculus of variations a minimizer uo ∈ X := W01,2 (𝒟) exists. We cannot at this point say anything about uniqueness of the minimizer since there are no conditions on f that guarantee strict convexity. It is also possible to show (see Section 6.7.2 for a more general case) that the minimizer uo satisfies an Euler– Lagrange equation of the form (6.85), with f replaced by f ̃. We claim that the minimizer uo has the property u ≤ uo ≤ u.̄ One way to prove this claim is by using the Euler–Lagrange equation for the functional F̃ which is the weak form of the semilinear elliptic equation −Δuo − f ̃(x, uo ) = 0, uo = 0

in 𝒟,

on 𝜕𝒟.

(6.89)

6.7 A semilinear elliptic problem and its variational formulation

| 297

Since u is a weak subsolution of (6.85) it satisfies the inequality (6.87) and from that we observe that the function v = u − uo satisfies the inequality9 −Δv − (f (x, u) − f ̃(x, uo )) = −Δv − (f (x, v + uo ) − f ̃(x, uo )) ≤ 0. We multiply by v+ = (u − uo )+ and integrate over 𝒟 to obtain ˆ ˆ 󵄨󵄨 + 󵄨󵄨2 dx − (f (x, u(x)) − f ̃(x, uo (x)))v+ (x) dx ≤ 0, ∇v (x) 󵄨󵄨 󵄨󵄨 𝒟

𝒟

(6.90)

where we have used the fact that ˆ ˆ 󵄨󵄨 + 󵄨󵄨2 + Δv(x)v (x) dx = − 󵄨󵄨∇v (x)󵄨󵄨 dx. 𝒟

𝒟

By the definition of f ̃ we note that if v ≠ 0, i. e., when we consider a x ∈ 𝒟 such that u(x) ≥ uo (x), then f (x, u(x)) = f ̃(x, uo (x)) so the contribution of the subset 𝒟+ := {x ∈ 𝒟 : u(x) ≥ uo (x)} to the second integral in (6.90) is zero. Obviously, the contribution to the same integral of the subset 𝒟− := {x ∈ 𝒟 : u(x) ≤ uo (x)} is also vanishing since v+ (x) = 0 for every x ∈ 𝒟− . Therefore, (6.90) becomes ˆ 󵄨󵄨 + 󵄨󵄨2 󵄨󵄨∇v (x)󵄨󵄨 dx ≤ 0, +

𝒟

which by the positivity of this term immediately yields that ˆ 󵄨󵄨 + 󵄨󵄨2 󵄨󵄨∇v (x)󵄨󵄨 dx = 0, 𝒟

so that ∇v+ (x) = 0, a. e., x ∈ 𝒟. That means that v+ = c (constant) a. e. By the properties of uo and u we see that v+ = 0 on 𝜕𝒟, therefore, v+ = 0 a. e. in 𝒟. That implies that u ≤ uo a. e. in 𝒟. To obtain the other inequality repeat the same steps, this time using (6.88) and multiplying with w+ = (uo − u)̄ + . We conclude the proof by noting that since u ≤ uo ≤ u,̄ by the definition of f ̃ it follows that f ̃(x, uo ) = f (x, uo ), therefore, (6.89) coincides with (6.85). The method of super and subsolutions, when applicable, is an important method allowing us to obtain sharp a priori estimates on the solutions of variational problems and the associated Euler–Lagrange PDE, and will be revisited in Chapter 7 (see Section 7.6.3). The above scheme may be combined with an appropriate fixed-point scheme which allows us to obtain more information on solutions of (6.85). The following theorem (see [43]) shows how the existence of sub and supersolutions guarantees the existence of solutions for (6.85). 9 Weakly and for a positive test function.

298 | 6 The calculus of variations Theorem 6.7.3. Suppose that f (x, u) = fL (u)+f0 (x) with fL Lipchitz and f0 ∈ (W01,2 (𝒟))∗ = W −1,2 (𝒟) and that there exist a sub and supersolution to (6.85), u and ū respectively, such that u ≤ u.̄ Then there exist two solutions w and w to (6.85), such that u ≤ w ≤ w ≤ u.̄ Furthermore, w and w are the minimal and maximal (resp.) solutions between u and ū is the sense that any other solution u of (6.85) satisfies u ≤ w ≤ u ≤ w ≤ u.̄ Proof. Let L be the Lipschitz constant of fL , fix ρ > L and for a given v ∈ L2 (𝒟) consider the weak solution u ∈ W01,2 (𝒟) to the problem −Δu + ρu = ρv + fL (v) + f0 .

(6.91)

By the standard arguments of the calculus of variations, this problem admits a unique solution and let Tρ : L2 (𝒟) → L2 (𝒟) be the map defined in terms of the solution of this problem, by Tρ (v) = u. Clearly, a fixed point of the map Tρ is a solution to (6.85). The continuity of Tρ : L2 (𝒟) → L2 (𝒟) can be seen as follows: upon considering (6.91) with a given right-hand side g ∈ (W01,2 (𝒟))⋆ , by taking the weak form of the equation (6.91) using as test function v, then using the Poincaré inequalilty on one side and the Cauchy–Schwarz inequality on the other, and dividing by the norm of v we can obtain that ‖v‖W 1,2 (𝒟) ≤ c‖g‖(W 1,2 (𝒟))⋆ for an appropriate constant c; combining this 0 0 observation with the Lipschitz property of fL we conclude. It is important for the proof to note that for the choice of ρ as ρ > L this map is also monotone, i. e., for any v1 ≥ v2 it follows that Tρ (v1 ) ≥ Tρ (v2 ). This essentially follows from the comparison principle for elliptic problems. Indeed, set fi = ρvi + fL (vi ) + f0 , i = 1, 2 and observe that for any x∈𝒟 f1 (x) − f2 (x) = ρ(v1 (x) − v2 (x)) + (fL (v1 (x)) − fL (v2 (x))) 󵄨 󵄨 󵄨 󵄨 󵄨 󵄨 ≥ ρ󵄨󵄨󵄨v1 (x) − v2 (x)󵄨󵄨󵄨 − 󵄨󵄨󵄨fL (v1 (x)) − fL (v2 (x))󵄨󵄨󵄨 ≥ (ρ − L)󵄨󵄨󵄨v1 (x) − v2 (x)󵄨󵄨󵄨 ≥ 0, where we used the Lipschitz continuity of fL and the fact that ρ > L. We then see that ui = Tρ (vi ) solve in W01,2 (𝒟) the elliptic problem −Δui + ρui = fi ,

i = 1, 2,

and since f1 ≥ f2 , the comparison principle (see Proposition 6.2.6 and Example 6.2.8) yields that u1 ≥ u2 ; hence, the monotonicity of Tρ . We now construct the solutions w and w using the following iterative schemes: Consider the sequence {un : ∈ ℕ} defined as un = Tnρ u and the sequence {ū n : n ∈ ℕ}, defined as ū n = Tnρ u,̄ n = 0, 1, . . ., with the convention that T0ρ = I, the identity map. It

6.7 A semilinear elliptic problem and its variational formulation

| 299

is our aim to show that the sequences {un : n ∈ ℕ} and {ū n : n ∈ ℕ} are convergent, and the respective limits are the solutions w and w we are seeking for. We claim that by construction the sequence {un : n ∈ ℕ} is increasing. Note that for the first two terms it holds that u0 = u ≤ u1 = Tρ u. Indeed, u1 = Tρ u is the solution to the elliptic problem (we omit the explicit statement of the boundary conditions which are always homogeneous Dirichlet) −Δu1 + ρu1 = ρu0 + fL (u0 ) + f0 = ρu + fL (u) + f0 .

(6.92)

Since u is a subsolution we have by rearranging that −Δu ≤ fL (u) + f0 , which combined with (6.92) yields, −Δu1 + ρu1 ≥ ρu0 − Δu0 , and by the comparison principle (see Proposition 6.2.6 and Example 6.2.8), we see that u1 ≥ u0 . Now recall that Tρ is monotone, therefore, applying Tρ recursively, yields that un−1 ≤ un for any n ∈ ℕ, so that {un : n ∈ ℕ} is an increasing sequence. We also claim that by construction the sequence {ū n : n ∈ ℕ} is decreasing. The proof is similar but we need to use instead that the sequence is constructed by applying the monotone operator Tρ to a supersolution, ū rather than a subsolution. The details are left to the reader. On the other hand, since by assumption, u ≤ u,̄ we have that u0 ≤ ū 0 and applying iteratively the monotone operator Tρ leads to the conclusion that un ≤ ū n for every n ∈ ℕ. The above considerations allow us to conclude that u ≤ un−1 ≤ un ≤ ū n ≤ ū n−1 ≤ u,̄

∀ n ∈ ℕ.

Since {un : n ∈ ℕ} is an increasing sequence bounded above by u,̄ it converges pointwise to some function w, which by a straightforward application of the Lebesgue dominated convergence theorem is such that un → w in L2 (𝒟). Similarly, since {ū n : n ∈ ℕ} is a decreasing sequence bounded below by u, there exists a function w such that ū n → w in L2 (𝒟). Since un+1 = Tρ (un ) for any n, taking the limit as n → ∞ in both sides, and using the continuity of Tρ in L2 (𝒟), we conclude that w = Tρ (w), so that w is a fixed point for Tρ ; hence, a solution for (6.85). Similarly, by taking the limit as n → ∞ in ū n+1 = Tρ (ū n ), we conclude that w = Tρ (w), so that w is a fixed point for Tρ ; hence, a solution for (6.85). For the minimal and maximal nature of w and w, respectively, let u be any solution of (6.85) such that u ≤ u ≤ u.̄ Apply the monotone operator Tρ for n times, and taking into account that u as a solution of (6.85) is a fixed point of Tρ we see that un ≤ u ≤ ū n for any n ∈ ℕ and passing to the limit as n → ∞ we obtain the required result.

300 | 6 The calculus of variations Example 6.7.4. Consider the logistic equation −Δu = u(λ − u) with homogeneous Dirichlet boundary conditions. If λ > λ1 , the first eigenvalue of −Δ with Dirichlet conditions, with eigenfunction ϕ1 > 0, then this equation admits a solution u ∈ W01,2 (𝒟) satisfying 0 < u < λ. This can be shown by noting that u = λ is a supersolution, whereas for ϵ > 0 sufficiently small, u = ϵϕ1 is a subsolution, such that u ≤ u. This follows easily by the properties of ϕ1 , which is an L∞ (𝒟) function. Then a straightforward application of Theorem 6.7.3 yields the required result. ◁

6.7.2 Growth conditions on the nonlinearity We now consider the general case where we do not have information concerning sub or supersolutions but we impose appropriate growth conditions on the nonlinear function f instead (see, e. g., [16] or [65]). Assumption 6.7.5. The Carathéodory function f : 𝒟 × ℝ × ℝ satisfies the following: (i) Growth condition, |f (x, s)| ≤ c|s|r + α1 (x) for every (x, s) ∈ 𝒟 × ℝ where 0 ≤ r < ⋆ 2d s2 −1 = d+2 and α1 ∈ Lr (𝒟), where s2 = d−2 if d ≥ 3. In the cases d = 1, 2, no growth d−2 constraint on f is required. ≤ λ, uni(ii) Nonresonant condition, there exists λ < λ1 so that lim sup|s|→∞ f (x,s) s formly in x ∈ 𝒟, where λ1 > 0 is the first eigenvalue of the Dirichlet Laplacian. As we will see, Assumption 6.7.5(i) guarantees the differentiability of the functional F, Assumption 6.7.5(ii) that it is bounded below and coercive, so the semilinear equation (6.85) corresponds to the first-order condition for the minimization problem of the functional F, which is well posed. Proposition 6.7.6. If f satisfies Assumption 6.7.5, then the semilinear equation (6.85) admits a solution in u ∈ W01,2 (𝒟). Proof. As mentioned above, the strategy of the proof is to show that F : X := W01,2 (𝒟) → ℝ is differentiable, coercive and sequentially weakly lower semicontinuous; hence, the minimization problem is well posed, and the minimum satisfies the first-order condition, which is (6.85). The proof is broken up into 4 steps: 1. F is C 1 (X; ℝ) (i. e., continuously Fréchet differentiable). As F consists of the Dirichlet functional, for which we have already established this result (see Example 2.1.10), plus perturbation by the integral functional FΦ (u) = ´ ´s 𝒟 Φ(x, u(x))dx, where Φ(x, s) := 0 f (x, r)dr, we only have to check the continuous Fréchet differentiability for the latter. This requires the growth condition of Assumption 6.7.5(i). We claim that the Gâteaux derivative of FΦ can be expressed as ˆ 𝒟

(DFΦ (u))(x)v(x) dx =

ˆ 𝒟

f (x, u(x))v(x) dx,

∀ v ∈ W01,2 (𝒟),

6.7 A semilinear elliptic problem and its variational formulation

| 301

or in more compact notation, using the duality pairing ⟨⋅, ⋅⟩ between (W01,2 (𝒟))∗ = W −1,2 (𝒟) and W01,2 (𝒟), as ⟨DFΦ (u), v⟩ = ⟨f (⋅, u), v⟩. Indeed, for any ϵ > 0, 1 (F (u + ϵv) − FΦ (u)) = ϵ Φ

ˆ 𝒟

1 (Φ(x, u(x) + ϵv(x)) − Φ(x, u(x))) dx, ϵ

and considering x as fixed define the real valued function ψ : ℝ → ℝ by t 󳨃→ ψ(t) := Φ(x, u(x) + tv(x)) which by the definition of Φ is a C 1 function and such that ψ󸀠 (t) = f (x, u(x) + tv(x))v(x). Furthermore, for fixed x ∈ 𝒟, consider the sequence of functions {wϵ : ϵ > 0} defined by wϵ (x) :=

1 1 (Φ(x, u(x) + ϵv(x)) − Φ(x, u(x))) = (ψ(ϵ) − ψ(0)). ϵ ϵ

By the properties of the function ψ, we have that for each x ∈ 𝒟 (fixed) it holds that wϵ (x) → ψ󸀠 (0) = f (x, u(x))v(x) as ϵ → 0+ , therefore, the sequence of functions wϵ → w a. e. in 𝒟 as ϵ → 0+ , where the function w : 𝒟 → ℝ is defined by w(x) := f (x, u(x))v(x). We thus conclude that ˆ 1 wϵ (x) dx. (FΦ (u + ϵv) − FΦ (u)) = ϵ 𝒟 The remaining step is passing this a. e. convergence inside an integral, which can be accomplished using the Lebesgue dominated convergence theorem. This requires establishing that |wϵ | ≤ h for every ϵ > 0, where h ∈ L1 (𝒟). To this end, we note that for every x ∈ 𝒟, 1 1 wϵ (x) = (ψ(ϵ) − ψ(0)) = ϵ ϵ

ˆ 0

ϵ

1 ψ (t)dt = ϵ

ˆ

󸀠

0

ϵ

f (x, u(x) + tv(x))v(x)dt,

so that ˆ ϵ 󵄨󵄨 󵄨 󵄨󵄨 󵄨 1 󵄨f (x, u(x) + tv(x))󵄨󵄨󵄨v(x)dt 󵄨󵄨wϵ (x)󵄨󵄨󵄨 ≤ ϵ 0 󵄨 ˆ 1 ϵ 󵄨󵄨 󵄨r 󵄨 󵄨 (c󵄨󵄨u(x) + tv(x)󵄨󵄨󵄨 + α1 (x))󵄨󵄨󵄨v(x)󵄨󵄨󵄨dt ≤ ϵ 0 󵄨 󵄨r ≤ (c󵄨󵄨󵄨u(x) + |v(x)|󵄨󵄨󵄨 + α1 (x))v(x) =: h(x), where we have first used the growth condition on f , and then the observation that since for r > 0 the function t 󳨃→ |u(x) + t|v(x)||r is increasing, we may majorize the integral by using the upper estimate |u(x) + tv(x)|r < |u(x) + |v(x)||r (which is of course independent of t therefore the upper bound of the integral is obtained trivially). All

302 | 6 The calculus of variations that is left to do is to check whether h ∈ L1 (𝒟). Clearly, this requires careful choice of the exponent r; a task in which the fact that u, v ∈ W01,2 (𝒟) and the Sobolev embedding theorem proves very helpful. We observe that the growth condition in conjunction with the Sobolev embedding is sufficient for our claim to hold. We will allow ourselves a little discussion around this last point (consider this as some sort of reverse engineering that motivates our choice of assumptions; readers more familiar with such calculations may skip this paragraph). Our immediate reaction is to say that since u, v ∈ W01,2 (𝒟), then they are definitely L2 (𝒟) functions, therefore, r = 1 would work. However, we can do much better than that recalling that since u, v ∈ W01,2 (𝒟) they in fact enjoy much better integrability properties that square integrability as a consequence of the Sobolev embedding theorem (see Theorem 1.5.11) according to which W01,2 (𝒟) 󳨅→ Lp (𝒟) for appropriate values of p (1 ≤ p < s2 , 2d where s2 = d−2 for d > 3 and s2 = ∞ for d = 1, 2). That tells us that we may consider p u, v ∈ L (𝒟) for some p > 2, thus allowing for h to be L1 (𝒟) even for values of r greater than 1. Since the highest value of p we are allowed to consider when applying the Sobolev embedding theorem depends on the dimension, we leave it as p for the time being, in order to obtain an estimate for r. This can be done through the use of Hölder’s inequality. Consider, e. g., the term |u|r v where both u, v ∈ Lp (𝒟). In order to show that |u|r v ∈ L1 (𝒟), we estimate 󵄩󵄩 r 󵄩󵄩 󵄩󵄩|u| v󵄩󵄩L1 (𝒟) =

ˆ 𝒟

ˆ 1/ℓ ˆ 1/ℓ∗ 󵄨󵄨 󵄨󵄨r 󵄨󵄨 󵄨󵄨rℓ 󵄨󵄨 󵄨󵄨ℓ∗ 󵄨󵄨u(x)󵄨󵄨 |v(x)| dx ≤ { 󵄨󵄨u(x)󵄨󵄨 dx} { 󵄨󵄨v(x)󵄨󵄨 dx} . 𝒟

𝒟

ℓ = p and To keep the right-hand side bounded, we need to make the choice ℓ∗ = ℓ−1 rℓ = p, which leads to the conclusion that the largest value that may be assigned to r is r = p − 1, where p is the largest p allowed in order to ensure that W01,2 (𝒟) 󳨅→ Lp (𝒟). 2d This can be found by Theorem 1.5.11 to be p = s2 = d−2 if d ≥ 3 and p = ∞ (thus leading to unrestricted growth for f ) if d = 1, 2. A similar argument allows us to estimate the term α1 v and find out the exact integrability conditions that must be imposed on the function α1 , given that v ∈ Lp (𝒟). We have so far proved the claim that FΦ is Gâteaux differentiable. In order to prove that it is Fréchet differentiable, by Proposition 2.1.14, it suffices to show that for u fixed, the mapping u 󳨃→ DFΦ (u) considered as a mapping W01,2 (𝒟) → (W01,2 (𝒟))∗ is continuous, i. e., if we consider a sequence {un : n ∈ ℕ} ⊂ W01,2 (𝒟) such that un → u in W01,2 (𝒟), then we must show DFΦ (un ) → DFΦ (u) in (W01,2 (𝒟))∗ . For a sequence {un : n ∈ ℕ} such that un → u in W01,2 (𝒟) we have by the Sobolev embedding theorem that un → u in Ls (𝒟) for s ∈ [1, s2 ] (note that at this point we do not require compactness of the embedding, so s = s2 is allowed). The best possible choice for s is to take s = s2 (since if un → u in Ls2 (𝒟), then this convergence holds for 󸀠 any Ls (𝒟) with s󸀠 < s2 ). We may also choose a subsequence {unk : k ∈ ℕ} such that unk → u a. e. in 𝒟 and |unk (x)| ≤ w(x) for some w ∈ Ls2 (𝒟). We will work along this subsequence, which we relabel as {uk : k ∈ ℕ} for convenience, and note that, by an

6.7 A semilinear elliptic problem and its variational formulation

| 303

application of Hölder’s inequality, 󵄨󵄨 󵄨ˆ 󵄨 󵄨󵄨 󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨⟨DFΦ (uk ) − DFΦ (u), v⟩󵄨󵄨 = 󵄨󵄨 (f (x, uk (x)) − f (x, u(x)))v(x)dx󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨 𝒟 󵄩󵄩1/s∗∗ 󵄩󵄩 ≤ 󵄩󵄩f (⋅, uk ) − f (⋅, u)󵄩󵄩Ls (𝒟) ‖v‖1/s Ls (𝒟) .

(6.93)

We claim that ‖f (⋅, uk ) − f (⋅, u)‖Ls∗ (𝒟) → 0 as k → ∞. Define the sequence of functions {wk : k ∈ ℕ}, by wk (x) = f (x, uk (x)) − f (x, u(x)) and note that wk → 0 a. e. in 𝒟. Our claim reduces to ‖wk ‖Ls∗ (𝒟) → 0 as k → ∞, which will follow by a straightforward application of Lebesgue’s dominated convergence theorem if |wk (x)|s < h(x), a. e. in 𝒟, for some h ∈ L1 (𝒟). By the growth condition for f in Assumption 6.7.5(i), we see that ∗

󵄨󵄨 󵄨s∗ 󵄨 󵄨 󵄨r 󵄨 󵄨r 󵄨s∗ 󵄨 󵄨 󵄨r 󵄨 󵄨r 󵄨s∗ 󵄨󵄨wk (x)󵄨󵄨󵄨 ≤ 󵄨󵄨󵄨2a1 (x) + c 󵄨󵄨󵄨uk (x)󵄨󵄨󵄨 + c 󵄨󵄨󵄨u(x)󵄨󵄨󵄨 󵄨󵄨󵄨 ≤ 󵄨󵄨󵄨2a1 (x) + c 󵄨󵄨󵄨w(x)󵄨󵄨󵄨 + c 󵄨󵄨󵄨u(x)󵄨󵄨󵄨 󵄨󵄨󵄨 ∗ 󵄨 󵄨r s ∗ 󵄨 󵄨r s∗ ≤ c󸀠 (a1 (x)s + 󵄨󵄨󵄨w(x)󵄨󵄨󵄨 + 󵄨󵄨󵄨w(x)󵄨󵄨󵄨 ), for some c󸀠 > 0. If we define h by h(x) := c󸀠 (a1 (x)s + |w(x)|r s + |u(x)|r s ) for every x, then, by the proper choice of r, we can guarantee that h ∈ L1 (𝒟). Since w, u ∈ Ls (𝒟), we ∗ ∗ can see that |w|r s , |v|r s ∈ L1 (𝒟) as long as r s∗ = s, or equivalently r = ss∗ = s−1 = s2 −1. Similar arguments lead us to the conclusion that if a1 ∈ Lr1 (𝒟), then r1 must be chosen ∗ s 2 so that s∗ = r1 , i. e., r1 = s−1 = ss−1 , in order for as1 ∈ L1 (𝒟). Hence, under these 2 assumptions we may guarantee that ‖wk ‖Ls∗ (𝒟) → 0 as k → ∞. Therefore, dividing (6.93) by ‖v‖W 1,2 (𝒟) (and noting that by the Sobolev embedding 0 ‖v‖Ls (𝒟) ≤ cs ‖v‖W 1,2 (𝒟) for some constant cs depending on the domain 𝒟 and the di0 mension), we conclude that upon taking the supremum over all ‖v‖W 1,2 (𝒟) ≤ 1, we have 0 ‖DFΦ (uk ) − DFΦ (u)‖(W 1,2 (𝒟))∗ → 0 as k → ∞. 0 Note, that we have shown the required result along the selected subsequence {uk : k ∈ ℕ} and in order to guarantee continuity we must show that this holds for the whole sequence. This can be achieved by resorting to the Urysohn property for the strong convergence (see, e. g., Remark 1.1.51) for the sequence {DFΦ (un ) : n ∈ ℕ} ⊂ (W01,2 (𝒟))∗ , in the standard fashion. This concludes the proof that FΦ is Fréchet differentiable at u. Note that the same steps, also lead to the conclusion that FΦ is C 1 . 2. F is sequentially weakly lower semicontinuous on X. This follows directly by Tonelli’s general result Theorem 6.3.10. An alternative way to see it without the use of this general result is to note that F := FD + FΦ i. e., it is a perturbation of the Dirichlet functional (that we proved its weak lower semicontinuity in Proposition 6.2.1) plus perturbation by the integral functional FΦ (u) = ´ ´s 𝒟 Φ(x, u(x))dx, where Φ(x, s) := 0 f (x, r)dr. The weak continuity of the functional FΦ : X = W01,2 (𝒟) → ℝ follows again by an application of the Sobolev embedding theorem. Consider a sequence {un : n ∈ ℕ} ∈ X such that un ⇀ u in X = W01,2 (𝒟). ∗

c

∗

∗

By the compact embedding W01,2 (𝒟) 󳨅→ Lp (𝒟) where p < s2 , we see that there exists a subsequence {unk : k ∈ ℕ} such that unk → u in Lp (𝒟) and a further subsequence {unk : ℓ ∈ ℕ} such that unk → u a. e. in 𝒟. By the growth condition on f , we have ℓ

ℓ

304 | 6 The calculus of variations the growth condition |Φ(x, s)| ≤ c|s|r+1 + α2 on the primitive of f . Consider now the Nemitskii operator ΦN defined by (ΦN (u))(x) = Φ(x, u(x)) for every x ∈ 𝒟, which from p the theory of the Nemitskii operator maps Lp (𝒟) into L r+1 (𝒟) continuously (see, e. g., p [86] or [111]). Therefore, ΦN (unk ) → ΦN (u) in L r+1 (𝒟) and by the Lebesgue embedding ℓ

theorem also in L1 (𝒟). This implies FΦ (unk ) → FΦ (u) (see, e. g., [49]). ℓ

3. The functional F : X := W01,2 (𝒟) → ℝ is bounded below and coercive. This requires the no resonance condition of Assumption 6.7.5(ii). This condition on f implies that there exists a constant c such that Φ(x, s) < c + λ2 s2 for every s ∈ ℝ and a. e. in 𝒟. Then ˆ ˆ λ 1 󵄨󵄨 󵄨2 u(x)2 dx, F(u) ≥ (6.94) 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx − c |𝒟| − 2 𝒟 2 𝒟 where we use the simplified notation |𝒟| = μℒd (𝒟). By the variational characterization of eigenvalues of the Laplacian (see Proposition 6.2.12, and in particular (6.29)), we have that ´ |∇u(x)|2 dx 𝒟 ´ λ1 = min , 2 u∈W01,2 (𝒟) 𝒟 u(x) dx hence, for any u ∈ W01,2 (𝒟) we have the Poincaré inequality λ1

ˆ

𝒟

u(x)2 dx ≤

ˆ

𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx,

(6.95)

and by the choice of λ ≤ λ1 , we have that ˆ ˆ λ 1 󵄨󵄨 󵄨󵄨2 u(x)2 dx ≥ 0, 󵄨∇u(x)󵄨󵄨 dx − 2 𝒟󵄨 2 𝒟 which when combined with (6.94), leads to the result that F(u) ≥ −c |𝒟|, hence, the functional is bounded below. For coercivity, we need to assume that λ < λ1 . By Poincaré’s inequality (6.95), the estimate (6.94) yields λ 1 F(u) ≥ (1 − ) 2 λ1

ˆ 𝒟

󵄨󵄨 󵄨2 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx − c |𝒟|,

and coercivity follows. 4. Combining the results of steps 1, 2 and 3 above, the claim follows. Example 6.7.7 (Nontrivial solutions). If the function f is such that f (x, 0) = 0 a. e. in 𝒟, then the semilinear problem (6.85) always admits the trivial solution u = 0, which

6.7 A semilinear elliptic problem and its variational formulation

| 305

in this case is definitely a local minimizer of the functional F. Under what conditions may Proposition 6.7.6 yield minimizers (hence, solutions of (6.85)) other than the trivial solution? That clearly depends on the rate by which f (x, s) approaches 0 as s → 0. A condition that will ensure that a nontrivial solution will exist is to assume that lim sups→0+ f (s) > λ1 . One way to see why this condition is sufficient, is to note that s this condition guarantees that there exist co > λ1 and δ > 0 such that, as long as s ∈ [0, δ], the function f satisfies the lower bound f (s) > co s for s ∈ [0, δ], which c in turn implies that its primitive Φ satisfies Φ(s) > 2o s2 , for such s. It is then possible to choose ϵ > 0 small enough so that the function v : 𝒟 → ℝ, defined by v(x) = ϵϕ1 (x), where ϕ1 is the eigenfunction corresponding to the eigenvalue λ1 , satisfies v(x) ∈ [0, δ] a. e. in 𝒟 (we have shown that ϕ1 > 0 and ϕ1 ∈ L∞ (𝒟)). That means ´ co ´ 2 𝒟 Φ(v(x))dx ≥ 2 𝒟 v(x) dx and, using that, we conclude by the properties of c that F(v) ≤

ϵ2 (λ − co ) 2 1

ˆ 𝒟

ϕ1 (x)2 dx < 0 = F(0).

This means that u = 0 is not a global minimum and, therefore, the minimizers provided by Proposition 6.7.6 cannot be the trivial solution of (6.85). An alternative (but equivalent) way of seeing that, is to note that v = ϵϕ1 for ϵ > 0 small enough, is a subsolution of (6.85), the result may follow by an application of Proposition 6.7.2. ◁ Example 6.7.8 (Uniqueness). There is no information provided by Proposition 6.7.6 regarding the uniqueness of the solution. This requires extra conditions on f . Typically such conditions are monotonicity conditions, e. g., the extra assumption that (f (s1 ) − f (s2 ))(s1 − s2 ) ≤ 0 for every s1 , s2 ∈ ℝ. Uniqueness can be easily checked by assuming two solutions u1 , u2 of the PDE, deriving the corresponding PDE for their difference w = u1 − u2 and then using w as test function in the weak form of this PDE (details are left as an exercise). Possibility of multiple solutions in semilinear elliptic problems, and the study of the qualitative behavior of solutions as one or more parameters of the system vary is a very active field of study in nonlinear analysis called bifurcation theory. ◁ Example 6.7.9. Certain variants of Proposition 6.7.6 may be easily constructed, in which a linear part can be included, at the cost of having to handle in certain cases a more complicated eigenvalue problem. For example, one may consider nonlinearities of the form f (x, s) = −a(x)s + f0 (x, s) where a ∈ L∞ (𝒟), a(x) ≥ 0, a. e. in 𝒟. Then we may treat the eigenvalue problem for the perturbed operator −Δ + a(x)I on W01,2 (𝒟), whose first eigenvalue λ1,a > 0 satisfies the modified variational representation λ1,a = infv∈W 1,2 (𝒟) FR,a (v) where 0

´ FR,a (v) :=

𝒟

´ |∇u(x)|2 dx + 𝒟 a(x)u(x)2 dx ´ . 2 𝒟 u(x) dx

306 | 6 The calculus of variations Then an extension of Proposition 6.7.6, where now Assumption 6.7.5 is posed for f0 , with λ1 replaced by λ1,a , can be formulated. The coercivity estimates can be simplified working in the equivalent norm ‖ ⋅ ‖1 , defined by ‖u‖21 = FD (u) + ⟨au, u⟩L2 (𝒟) . The details are left to the reader (see also [16]). ◁ 6.7.3 Regularity for semilinear problems We close our study of the semilinear problem (6.85) with a brief sojourn on the regularity properties of its weak solutions. It will be shown that under certain conditions on the nonlinearity the weak solutions can be more regular than W01,2 (𝒟) functions, in particular they can be continuous functions. Our approach, the bootstrap method, builds on the regularity properties of the Laplacian and the linear Poisson equation (see [36]). Assumption 6.7.10. The function f is Carathéodory and satisfies the growth condition 󵄨󵄨 󵄨 p 󵄨󵄨f (x, s)󵄨󵄨󵄨 ≤ c(1 + |s| ), (a) for some p ≥ 1 if d ≤ 2 or (b) for p ≤

4 d−2

a. e. x ∈ 𝒟, ∀ s ∈ ℝ, if d ≥ 3.

Proposition 6.7.11. Let Assumption 6.7.10 hold. If u ∈ W01,2 (𝒟) satisfies (6.85) and, furthermore, is such that f (⋅, u(⋅)) ∈ (W01,2 (𝒟))⋆ then u ∈ L∞ (𝒟) ∩ C(𝒟). Proof. Let us consider the case d > 2. Since by Proposition 6.2.11 we have regularity results for the linear equation −Δu + λu = f0 for some f0 known and λ > 0, our aim is to break up (6.85) into subproblems which are of this form and then build the regularity result on that. To this end, let us consider the auxiliary problems −Δu1 + u1 = g1 (t, u),

u1 ∈ W01,2 (𝒟),

(6.96)

−Δu2 + u2 = g2 (t, u),

u2 ∈ W01,2 (𝒟),

(6.97)

and where the functions g1 , g2 are chosen so as to satisfy |g1 (x, s)| ≤ c, |g2 (x, s)| ≤ c|s|p and g(x, s) = g1 (x, s) + g2 (x, s) − s, a. e. x ∈ 𝒟 and for all s ∈ ℝ. By adding (6.96) and (6.97), we see that u = u1 + u2 . If we can show the required regularity for u1 and u2 separately then we have the stated result. Since |g1 (x, s)| ≤ c, and the domain is bounded g1 (t, u) ∈ L2 (𝒟); hence, any weak solution u1 of (6.96) satisfies u1 ∈ L∞ (𝒟) ∩ C(𝒟), by Proposition 6.2.11. Thus, it remains to show that u2 ∈ L∞ (𝒟) ∩ C(𝒟). 2d , Since u ∈ W01,2 (𝒟) by the Sobolev embedding, we have that u ∈ Lq (𝒟) for q ≤ d−2 q

and since |g2 (x, s)| ≤ c|s|p it follows that g2 (x, u) ∈ L p (𝒟). If pq > d2 , we may immediately

apply Proposition 6.2.11 for (6.97) to yield u2 ∈ L∞ (𝒟) ∩ C(𝒟). Choosing q = have the required condition on p.

2d , d−2

we

6.8 A variational formulation of the p-Laplacian

| 307

By more refined estimates, one can improve the range of values of p for which the stated regularity result holds. For more details, the reader may consult, e. g., [36].

6.8 A variational formulation of the p-Laplacian 6.8.1 The p-Laplacian Poisson equation In this section, we are going to apply the general formulation of the calculus of variations to study the problem of the minimization of the functional F : W01,p (𝒟) → ℝ, where 𝒟 bounded and of sufficiently smooth boundary, defined by F(u) :=

1 p

ˆ 𝒟

󵄨󵄨 󵄨p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx −

ˆ 𝒟

f (x)u(x) dx,

∀u ∈ W01,p (𝒟), p > 1,

(6.98)

and its connection with the quasilinear elliptic partial differential equation −∇ ⋅ (|∇u|p−2 ∇u) = f , u=0

in 𝒟,

(6.99)

on 𝜕𝒟.

Recalling that (W01,p (𝒟))⋆ = W −1,p (𝒟) with p1 + p1⋆ = 1, and denoting by ⟨⋅, ⋅⟩ the duality ⋆

pairing between W −1,p (𝒟) and W01,p (𝒟), one may define the nonlinear operator A := ⋆ (−Δp ) : W01,p (𝒟) → W −1,p (𝒟), by ⋆

⟨A(u), v⟩ := ⟨−Δp u, v⟩ :=

ˆ 𝒟

󵄨󵄨 󵄨p−2 󵄨󵄨∇u(x)󵄨󵄨󵄨 ∇u(x) ⋅ ∇v(x)dx,

∀ u, v ∈ W01,p (𝒟),

(6.100)

which is called the p-Laplace operator. For p = 2, this is a linear operator which coincides with the Laplacian. ⋆

Proposition 6.8.1. Let p > 1. For every f ∈ W −1,p (𝒟), there exists a unique solution of the quasilinear elliptic problem (6.99). This solution is the minimizer of the functional F : X → ℝ defined by (6.98). Proof. Consider the Banach space X := W01,p (𝒟), its dual X ⋆ := W −1,p (𝒟), and let ⟨⋅, ⋅⟩ be the duality pairing among them. The proof consists in showing the minimization problem for the functional F is well posed on the Banach space X := W01,p (𝒟), and identifying (6.99) as the first-order condition for this problem. We proceed in 4 steps: 1. F is strictly convex, weakly sequentially lower semicontinuous and coercive. The convexity of the functional F : X = W 1,p (𝒟) → ℝ, follows easily by the convexity of the function g : ℝd → ℝ, defined by g(z) = p1 |z|p , for p > 1. In fact, the functional is strictly convex. The weak lower sequential semicontinuity of F follows by a direct application of Theorem 6.3.10 by taking into account the convexity of the ⋆

308 | 6 The calculus of variations function φ(z) = p1 |z|p . Coercivity of F on W01,p (𝒟) follows by the use of Poincaré inequality. Indeed observe that ˆ ˆ 1 1 󵄨 󵄨󵄨 󵄨󵄨 󵄨p 󵄨󵄨 p F(u) ≥ 󵄨󵄨f (x)󵄨󵄨󵄨 󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx ≥ ‖∇u‖Lp (𝒟) − c‖u‖Lp (𝒟) , 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx − p 𝒟 p 𝒟 where for the last term we used a Hölder estimate for the term ‖fu‖L1 (𝒟) (the constant c depends on ‖f ‖Lp⋆ (𝒟) ). Using the Poincaré inequality and in particular Proposi-

tion 1.5.14 that guarantees that ‖∇u‖Lp (𝒟) is an equivalent norm for X = W01,p (𝒟) along with the fact that p > 1 leads us to the coercivity result. 2. By step 1 and using the standard arguments of the direct method, the minimization problem for F is well posed and admits a unique minimizer. 3. F is Fréchet differentiable on X = W01,p (𝒟) with derivative ˆ ˆ 󵄨󵄨 󵄨p−2 ⟨DF(u), v⟩ = f (x)v(x) dx, ∀ v ∈ W01,p (𝒟). 󵄨󵄨∇u(x)󵄨󵄨󵄨 ∇u(x) ⋅ ∇v(x) dx − 𝒟

𝒟

To show this, take any v ∈ X and consider the difference 1 (F(u + ϵv) − F(u)) ϵ ˆ ˆ 1 󵄨󵄨 󵄨p 󵄨 󵄨p = (󵄨󵄨∇u(x) + ϵ∇v(x)󵄨󵄨󵄨 − 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 ) dx − f (x)v(x) dx. 𝒟 ϵp 𝒟

(6.101)

Consider next the real valued function t 󳨃→ ϕ(t) := p1 |∇u(x) + t∇v(x)|p , which is differ-

entiable with derivative ϕ󸀠 (t) = |∇u(x) + t∇v(x)|p−2 (∇u(x) + t∇v(x)) ⋅ ∇v(x) for any fixed x. That means, the sequence of functions {wϵ : ϵ > 0} defined by wϵ := ϵ1 (|∇u + ϵ∇v|p − |∇v|p ) converges a. e. in 𝒟 to the function w := |∇u|p−2 ∇u ⋅ ∇v as ϵ → 0. Using the above sequence, we express (6.101) as ˆ ˆ 1 (F(u + ϵv) − F(u)) = wϵ (x) dx − f (x)v(x) dx, ϵ 𝒟 𝒟 and since wϵ → |∇u|p−2 ∇u ⋅ ∇v a. e. in 𝒟 as ϵ → 0, if we can exchange the limit with the integral over 𝒟 we can identify the Gâteaux derivative in the form above. For this step, we need to invoke a Lebesgue dominated convergence argument, i. e., we need to ensure that |wϵ | < h for every ϵ > 0 where h ∈ L1 (𝒟). Note that for every x ∈ 𝒟, ˆ 1 󵄨󵄨 1 ϵ 󸀠 1 󵄨p 󵄨 󵄨p wϵ (x) = (󵄨󵄨∇u(x) + ϵ∇v(x)󵄨󵄨󵄨 − 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 ) = (ϕ(ϵ) − ϕ(0)) = ϕ (t)dt ϵp ϵ ϵ 0 ˆ ϵ 1 󵄨󵄨 󵄨p−2 = 󵄨∇u(x) + t∇v(x)󵄨󵄨󵄨 (∇u(x) + t∇v(x)) ⋅ ∇v(x)dt, ϵ 0 󵄨 therefore, for some appropriate constant c > 0, ˆ ϵ 󵄨󵄨 󵄨 1 | |∇u(x) + t∇v(x)|p−2 (∇u(x) + t∇v(x)) ⋅ ∇v(x)|dt 󵄨󵄨wϵ (x)󵄨󵄨󵄨 ≤ ϵ 0

6.8 A variational formulation of the p-Laplacian

=

1 ϵ

ˆ 0

ϵ

| 309

󵄨 󵄨 p−1 󵄨 󵄨 󵄨 󵄨 󵄨 󵄨p−1 󵄨 󵄨󵄨 󵄨󵄨∇u(x) + t∇v(x)󵄨󵄨󵄨 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨dt ≤ c(󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 + 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨) 󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 =: h(x).

This last estimate follows easily from the fact that for every ξ , ξ 󸀠 ∈ ℝd we have by the triangle inequality that |ξ + ξ 󸀠 | ≤ |ξ | + |ξ 󸀠 |, so that since p > 1, it follows that |ξ + ξ 󸀠 |p−1 ≤ (|ξ | + |ξ 󸀠 |)p−1 . Apply this for ξ = ∇u(x) and ξ 󸀠 = t∇v(x) and since t ∈ (0, 1) majorize the resulting expression by setting t = 1, obtaining like that an upper bound for the integral. Since u, v ∈ W01,p (𝒟), we have that ∇u, ∇v ∈ Lp (𝒟), therefore, (|∇u| + ⋆ |∇v|)p−1 ∈ Lp (𝒟) and by Hölder’s inequality h := (|∇u| + |∇v|)p−1 |∇v| ∈ L1 (𝒟). We have then established the fact that |wϵ | ≤ h where h ∈ L1 (𝒟) and the proof concludes. To show Fréchet differentiability, we will use Proposition 2.1.14 along with the elementary (but not so easy to prove) inequality | |z1 |p−2 z1 − |z2 |p−2 z2 | ≤ c [|z1 | + |z2 |]p−2 |z1 − z2 | for any z1 , z2 ∈ ℝd , applied to z1 = ∇un (x), z2 = ∇u(x), for any sequence {un : n ∈ ℕ} such that un → u in W 1,p (𝒟) (the details are left as exercise). 4. By step 3, we recognize the p-Laplace Poisson equation as the first-order condition for the minimization of F which shows existence of solutions of (6.99). To show uniqueness of solutions, note that the existence of a weak minimizer is not enough as critical points are not necessarily minima. We may use the elementary10 (and easier to prove) inequality c1 [|z1 | + |z2 |]

p−2

|z1 − z2 |2 < (|z1 |p−2 z1 − |z2 |p−2 z2 ) ⋅ (z1 − z2 ),

for some constant c1 > 0. Assume two solutions u1 , u2 ∈ W01,p (𝒟) for (6.99), subtract and then take the weak form for the difference using as test function u1 − u2 . By applying the above inequality for zi = ∇ui , a. e. in 𝒟, we guarantee that u1 = u2 and the proof is complete. 6.8.2 A quasilinear nonlinear elliptic equation involving the p-Laplacian In this section, we are going to apply the general formulation of the calculus of variations to study the problem of the minimization of the functional F : W01,p (𝒟) → ℝ defined by ˆ ˆ 1 󵄨󵄨 󵄨p F(u) := Φ(x, u(x)) dx, p ∈ (1, ∞), 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx − p 𝒟 𝒟 (6.102) ˆ s Φ(x, s) = f (x, σ)dσ, 0

where f : 𝒟 × ℝ → ℝ is a given nonlinear function, and its connection with the quasilinear elliptic partial differential equation −∇ ⋅ (|∇u|p−2 ∇u) = f (x, u), u=0

10 see, e. g., [43].

in 𝒟,

on 𝜕𝒟.

(6.103)

310 | 6 The calculus of variations This problem may be considered as a generalization of the p-Laplacian Poisson equation we have studied in Section 6.8.1. In the particular case where p = 2, the problem reduces to a semilinear elliptic equation of the form studied in Section 6.7. Example 6.8.2 (The case f (x, s) = −μ|s|p−2 s + f0 (x), μ ≥ 0). We start our study of the general class of equation (6.103) with the particular but very interesting case where f (x, s) = −μ|s|p−2 s + f0 (x), where μ > 0 is a constant and f0 : 𝒟 → ℝ a given function. One can easily extend the result of Proposition 6.8.1 for this choice, in the case where μ ≥ 0. Moreover, the solution is unique, as one can see by assuming the existence of two solutions u1 , u2 , taking the difference w = u1 − u2 , the corresponding PDE for w and using v = w as test function in its weak form, to obtain ˆ (|∇u1 |p−2 ∇u1 − |∇u2 |p−2 ∇u2 ) ⋅ (∇u1 − ∇u2 )dx 𝒟 ˆ (6.104) + μ (|u1 |p−2 u1 − |u2 |p−2 )(u1 − u2 )dx = 0. 𝒟

Using the inequality, (|ξ1 |p−2 ξ1 − |ξ2 |p−2 ξ2 ) ⋅ (ξ1 − ξ2 ) ≥ (|ξ1 |p−1 − |ξ2 |p−1 )(|ξ1 | − ξ2 |) ≥ 0, which is valid for any ξ1 , ξ2 ∈ ℝd (but also for any real numbers ξ1 , ξ2 ∈ ℝ), in (6.104) we obtain, since μ ≥ 0 that u1 = u2 and ∇u1 = ∇u2 a. e. in 𝒟, hence the uniqueness of the solution. The quasilinear equation in this case enjoys also a weak maximum principle stating that if f ≥ 0, then u ≥ 0 in 𝒟. This can be shown by taking the weak form of the equation with u− = max(−u, 0), and recalling that ∇u− = −∇u1u≤0 , we obtain that ˆ ˆ ˆ 󵄨󵄨 − 󵄨󵄨p 󵄨 − 󵄨p fu− dx. 󵄨󵄨∇u 󵄨󵄨 dx + μ 󵄨󵄨󵄨u 󵄨󵄨󵄨 dx = − 𝒟

𝒟

𝒟

Since the left-hand side is positive and the right-hand side is negative we conclude that |∇u− |p + μ|u− |p = 0, which leads to the conclusion that u− = 0; hence, u ≥ 0 in 𝒟. A similar argument yields a comparison principle, according to which if u1 , u2 are two solutions of (6.103) for f0,1 ≥ f0,2 , then u1 ≥ u2 . ◁ Example 6.8.3 (Nonlinear eigenvalue problems). The nonlinearity treated in Example 6.8.2 becomes particularly interesting if μ < 0 and f0 = 0. In this case, expressing μ = −λ, for λ > 0, we are led to a nonlinear eigenvalue problem −Δp u = λ|u|p−2 u, u = 0,

in 𝒟,

on 𝜕𝒟.

(6.105)

In the special case where p = 2, this reduces to the standard eigenvalue problem for the Laplacian which was studied in Section 6.2.3. Problem (6.105) has been heavily studied. Here, we will be only interested in the smaller λ > 0, for which problem (6.105)

6.8 A variational formulation of the p-Laplacian

| 311

admits a nontrivial solution. This value, denoted by λ1,p > 0 is called the first eigenvalue of the p-Laplacian operator, and the corresponding solution ϕ1,p , the first eigenfunction. Importantly, ϕ1,p does not vanish anywhere on 𝒟 (if 𝒟 is bounded simply connected and of sufficiently smooth boundary) and can be chosen so that ϕ1,p > 0. One may further show that λ1,p satisfies a variational principle in terms of a generalized Rayleigh quotient λ1,p =

‖∇u‖pLp (𝒟)

inf

W01,p (𝒟)\{0}

‖u‖pLp (𝒟)

.

This infimum is attained, a fact that can be shown, using the direct method on the ‖∇u‖pLp (𝒟)

generalized Rayleigh–Fisher functional FR,p (u) :=

‖u‖pLp (𝒟)

, or rather, noting (by ho-

mogeneity) the equivalence of the above problem with the problem of minimizing ‖∇u‖pLp (𝒟) on {u ∈ W01,p (𝒟) : ‖u‖Lp (𝒟) = 1}. The strict positivity of ϕ1,p is a more delicate result requiring extensions of the maximum principle (or the Harnack inequality) for the p-Laplacian operator (see Remark 6.6.9, see also [9] or [79]). Moreover, any function u such that ‖u‖Lp (𝒟) = 1 for which ‖∇u‖Lp (𝒟) = λ1,p necessarily satisfies (6.105) for λ = λ1,p and is equal to u = ±ϕ1,p . This follows by the uniqueness property for positive solutions of (6.105) on the unit ball of Lp (𝒟). An easy proof of this remark has been given in [22]. Consider any two positive solutions u1 , u2 of (6.105) on the unit ball of Lp (𝒟) and define v = 21 (up1 + up2 ), and w = v1/p . One can easily see that w is on the unit ball itself. A straightforward yet rather tedious calculation yields that, upon defining h =

up1 p u1 +up2

,

󵄨p 󵄨 |∇w|p = v󵄨󵄨󵄨h ∇ ln u1 + (1 − h) ∇ ln u2 󵄨󵄨󵄨 ≤ v[h |∇ ln u1 |p + (1 − h)|∇ ln u2 |p ] 1 = (|∇u1 |p + |∇u2 |p ), 2 where we used the convexity of the function ξ 󳨃→ |ξ |p . Upon integration, we have that ˆ

ˆ

1 |∇w| dx ≤ ( 2 𝒟 p

𝒟

p

|∇u1 | dx +

ˆ 𝒟

|∇u2 |p dx) = λ1 ,

(6.106)

´ Since w is admissible, it must be a minimizer of w 󳨃→ 𝒟 |∇u|p dx, on the unit ball of Lp (𝒟), therefore, (6.106) implies that |∇w|p = 21 |∇u1 |p + 21 |∇u2 |p a. e. in 𝒟, which in turn implies that |h ∇ ln u1 + (1 − h) ∇ ln u2 |p ≤ v[h |∇ ln u1 |p + (1 − h)|∇ ln u2 |p ], therefore, ∇ ln u1 = ∇ ln u2 a. e. so that u1 = cu2 for some constant c > 0. ◁ We will impose the following conditions on f . Assumption 6.8.4. f : 𝒟 × ℝ → ℝ is a Carathéodory function such that satisfies: (i) The growth condition |f (x, s)| ≤ c|s|r−1 + a(x), a. e. x ∈ 𝒟, and all s ∈ ℝ, for 1 < r < ∗ dp if p < d (sp = ∞ if p ≥ d) and a ∈ Lp (𝒟), a ≥ 0, with p1 + p1∗ = 1. sp , with sp = d−p

312 | 6 The calculus of variations (ii) The asymptotic condition lim sup|s|→∞ is defined as λ1,p = infW 1,p (𝒟)\{0} 0

‖∇u‖pLp (𝒟) ‖u‖pLp (𝒟)

pF(x,s) |s|p

< λ1,p uniformly in x, where λ1,p > 0

.

The following proposition provides an existence result for the quasilinear equation (6.103), Proposition 6.8.5. Let the function f satisfy Assumption 6.8.4. The functional F defined in (6.102) admits a minimum which corresponds to a solution of the nonlinear PDE problem (6.103). Proof. We will first show the existence of a minimum for F and then associate it through the use of the Euler–Lagrange equation with a solution of (6.103). The existence of the minimum uses the direct method of the calculus of variations which requires sequential weak lower semicontinuity for F and coercivity. It is conveλ nient to express Φ as Φ(x, s) = p1,p |s|p + Φ1 (x, s) with the function Φ1 satisfying

lim sups→±∞ Φ1|s|(x,s) < 0 uniformly in x ∈ 𝒟. The proof proceeds in 3 steps: p 1. Sequential weak lower semicontinuity follows easily (the first part of the functional enjoys this property by convexity as already seen in the proof of Proposition 6.8.1 and the second part enjoys this property as a result of the compactness dp of the embedding W01,p (𝒟) 󳨅→ Lr (𝒟) for 1 ≤ r < d−p =: sp ). 2. We now check the coercivity of the functional. Suppose that it is not. Then there exists a sequence {un : n ∈ ℕ} ⊂ W01,p (𝒟) such that F(un ) < c for some c > 0 while ‖un ‖W 1,p (𝒟) → ∞. Note that by the form of the functional if such a sequence exists it 0 must also hold that (at least for some subsequence) |un | → ∞ a. e. Consider the noru malized sequence {wn : n ∈ ℕ} defined by wn (x) = ‖u ‖ n1,p , which is clearly bounded in

W01,p (𝒟);

n W

0

(𝒟)

hence, by reflexivity there exists w0 ∈ W01,p (𝒟) with ‖w0 ‖W 1,p (𝒟) ≤ 1 such 0

that wn ⇀ w0 in W01,p (𝒟), while by the Rellich–Kondrachov compact embedding (up to a subsequence) wn → w0 in Lp (𝒟). We now divide F(un ) by ‖un ‖p 1,p and we obtain 1

F(un ) ‖un ‖p 1,p W0 (𝒟)

=

1 p

ˆ

λ1,p 󵄨󵄨 󵄨p 󵄨󵄨∇wn (x)󵄨󵄨󵄨 dx − p 𝒟

ˆ 𝒟

󵄨󵄨 󵄨p 󵄨󵄨wn (x)󵄨󵄨󵄨 dx −

ˆ 𝒟

W0 (𝒟)

Φ1 (x, un (x)) 󵄨󵄨 󵄨p 󵄨w (x)󵄨󵄨 dx, |un (x)|p 󵄨 n 󵄨

Φ(x,u (x))

p n where we have trivially expressed Φ(x, un (x)) = |u (x)| p |un (x)| . Rearranging so as to n ´ bring p1 𝒟 |∇wn (x)|p dx to the left-hand side and keeping in mind the properties of Φ1 and the sequence {wn : n ∈ ℕ}, after taking the limit superior we obtain that ˆ ˆ 󵄨 󵄨p 󵄨󵄨 󵄨󵄨p lim sup 󵄨󵄨󵄨∇wn (x)󵄨󵄨󵄨 dx ≤ λ1,p (6.107) 󵄨󵄨w(x)󵄨󵄨 dx. n→∞

𝒟

𝒟

By the sequential weak lower semicontinuity of the norm in W01,p (𝒟), ˆ ˆ ˆ ˆ 󵄨󵄨 󵄨p 󵄨 󵄨p 󵄨 󵄨p 󵄨󵄨 󵄨p inf 󵄨󵄨󵄨∇wn (x)󵄨󵄨󵄨 dx ≤ lim sup 󵄨󵄨󵄨∇wn (x)󵄨󵄨󵄨 dx ≤ λ1,p 󵄨󵄨∇wo (x)󵄨󵄨󵄨 dx ≤ lim 󵄨󵄨w0 (x)󵄨󵄨󵄨 dx, n→∞ n→∞ 𝒟

𝒟

𝒟

𝒟

6.8 A variational formulation of the p-Laplacian

| 313

whereas by the Poincaré inequality (or in fact by the definition of λ1,p ) we also have ˆ ˆ 󵄨p 󵄨󵄨 󵄨p 󵄨󵄨 λ1,p 󵄨󵄨∇wo (x)󵄨󵄨󵄨 dx. 󵄨󵄨w(x)󵄨󵄨󵄨 dx ≤ 𝒟

𝒟

Combining the above two, we have that ˆ ˆ 󵄨p 󵄨󵄨 󵄨p 󵄨󵄨 󵄨󵄨w0 (x)󵄨󵄨󵄨 dx. 󵄨󵄨∇wo (x)󵄨󵄨󵄨 dx = λ1,p 𝒟

𝒟

This implies that w0 = ±ϕ1,p > 0 where ϕ1,p > 0 is the first eigenfunction of the p-Laplacian (see Example 6.8.3). Without loss of generality let us assume that w0 = ϕ1,p > 0. Moreover, by the uniform convexity of W01,p (𝒟), the sequence {wn : n ∈ ℕ} also satisfies wn → w0 in W01,p (𝒟). To reach a contradiction, recall F(un ) < c, and note that by the definition of λ1,p it holds that ˆ ˆ λ1,p 1 󵄨󵄨 󵄨p 󵄨󵄨 󵄨p 󵄨󵄨∇un (x)󵄨󵄨󵄨 dx − 󵄨󵄨un (x)󵄨󵄨󵄨 dx ≥ 0, p 𝒟 p 𝒟 so that F(un ) < c implies that

ˆ 𝒟

and dividing again by ‖un ‖p ˆ 𝒟

W01,p (𝒟)

Φ1 (x, un (x))dx ≥ −c, → ∞, we obtain using a similar rearrangement that

Φ1 (x, un (x)) |un (x)|p dx = |un (x)|p ‖un ‖p 1,p W0 (𝒟)

ˆ 𝒟

Φ1 (x, un (x)) 󵄨󵄨 c 󵄨p 󵄨󵄨wn (x)󵄨󵄨󵄨 dx ≥ − p p |un (x)| ‖un ‖ 1,p

W0 (𝒟)

.

Passing to the limit, noting that w0 = ϕ1,p > 0, we see that the left-hand side is negative while the right-hand side converges to 0, which leads to a contradiction. 3. We now show that the minimizer is a solution of the related quasilinear PDE. Under Assumption 6.8.4(i), the functional F : W01,p (𝒟) → ℝ is Fréchet differentiable ⋆ and its Fréchet derivative DF : W01,p (𝒟) → (W01,p (𝒟))∗ = W −1,p (𝒟) satisfies ˆ 󵄨 󵄨p−2 ⟨DF(u), v⟩ = (󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 ∇u(x) ⋅ ∇v(x) − f (x, u(x))v(x)) dx, ∀ v ∈ W01,p. (𝒟). 𝒟

Indeed, the functional F consists of two parts, F0 (u) = p1 ‖∇u‖p

W01,p (𝒟)

which was studied

in the previous section where it was shown that it enjoys the stated properties (see the ´ proof of Proposition 6.8.1) and the second part F1 (u) := 𝒟 Φ(x, u) dx which deserves further consideration. With arguments similar as to those used in Proposition 6.7.6, we see that DF1 (u) = f (⋅, u), which can be expressed as DF1 (u) = Nf (u) where Nf is the Nemitskii operator generated by the function f defined by (Nf (u))(x) = f (x, u(x)) a. e. in 𝒟. The continuity of the Nemitskii operator Nf : W01,p (𝒟) → (W01,p (𝒟))∗ follows dp from the compactness of the embedding W01,p (𝒟) 󳨅→ Lr (𝒟) for 1 ≤ r ≤ d−p =: sp (see Theorem 1.5.11) and repeating almost verbatim the arguments in the proof of Proposition 6.7.6. The proof is now complete.

314 | 6 The calculus of variations Remark 6.8.6. One may generalize the nonresonance condition of Assumption 6.8.4(ii) in various ways (see, e. g., [10]). For example, we may assume that Assumption 6.8.4(ii) holds for sets of strictly positive measure. Another set of conditions are the so-called Landesman–Lazer-type conditions. Assume that the nonlinλ earity Φ is of the more general form Φ(x, s) = p1,p |s|p + Φ1 (x, s) + h(x), and define

G± (x) := lim sups→±∞ tion, ˆ 𝒟

Φ1 (x,s) s

(the uniform limit in x ∈ 𝒟). Then, under the assump-

G+ (x)ϕ1,p (x)dx ≤

ˆ 𝒟

h(x)ϕ1,p (x)dx ≤

ˆ 𝒟

G− (x)ϕ1,p (x)dx,

(6.108)

problem (6.103) admits a solution. The proof proceeds along the same lines as the proof of Proposition 6.8.5, and is left as an exercise (see also [10]). Condition (6.108) is called a Landesman–Lazer-type condition. These conditions will also play an important role when trying to identify solutions of (6.103) which are not minimizers of the functional (6.102) (see Section 8.4, see also [11]).

6.9 Appendix 6.9.1 A version of the Riemann–Lebesgue lemma Lemma 6.9.1. Let u ∈ L∞ loc (ℝ), be a periodic function of period 1. For every ϵ > 0, define ⋆ the family of functions {uϵ : ϵ > 0}, by uϵ (x) = u( xϵ ) for every x ∈ ℝ. Then uϵ ⇀ ū := ´1 ∞ 0 u(z)dz in L (I), for every I ⊂ ℝ bounded and measurable. Proof. Since u ∈ L∞ loc (ℝ), the sequence {uϵ : ϵ > 0} is bounded; hence, there exists a

subsequence (which we do not relabel) such that uϵ ⇀ v, as ϵ → 0+ , where v is some function in L∞ (ℝ). We need to identify the limit, and in particular show that v(x) = u,̄ a. e. in (0, 1). To this end, consider any Lebesgue point x0 ∈ (0, 1) of v, and any interval Iδ = (x0 − δ2 , x0 + δ2 ). We note that ⋆

󵄨󵄨ˆ 󵄨󵄨 󵄨󵄨 󵄨󵄨 ˆ 󵄨󵄨ˆ 󵄨󵄨 x 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 ̄ ̄ ̄ 󵄨󵄨󵄨󵄨, ) − u)dx (v(x) − u)dx = lim = lim ϵ (u( (u(y) − u)dy 󵄨󵄨 󵄨 󵄨󵄨 󵄨 󵄨 󵄨 󵄨 󵄨 󵄨󵄨 Iδ 󵄨󵄨 ϵ→0+ 󵄨󵄨 1 Iδ 󵄨󵄨 ϵ→0+ 󵄨󵄨 Iδ 󵄨󵄨 ϵ ϵ

(6.109)

where we first used the fact that uϵ ⇀ v, as ϵ → 0+ , and then we changed variables in the integral. Let k(ϵ) = [ δϵ ] (so that δϵ − 1 < k(ϵ) ≤ δϵ ), and we express the interval ⋆

1 1 Iδ as ϵ1 Iδ = ⋃k(ϵ) i=1 Ii ∪ Aϵ , where Ii = (xi − 2 , xi + 2 ) are unit intervals centered at some points xi , i = 1, . . . , k(ϵ) and Aϵ is a remainder set of Lebesgue measure |Aϵ | ≤ δϵ − k(ϵ) ≤ δ − ( δϵ − 1) = 1. In other words, we have broken up the interval ϵ1 Iδ , which is centered ϵ at a Lebesgue point x0 and has length δϵ into k(ϵ) intervals of unit length, centered

6.9 Appendix | 315

at points xi , plus a small remainder Aϵ of Lebesgue measure less than 1. Using the additivity of the integral, we see that ˆ

k(ϵ) ˆ

1 I ϵ δ

̄ (u(y) − u)dy = ∑ i=1

= k(ϵ)

Ii

̄ (u(y) − u)dy +

ˆ I1

ˆ

̄ (u(y) − u)dy +

Aϵ

̄ (u(y) − u)dy

ˆ

Aϵ

̄ (u(y) − u)dy =

ˆ Aϵ

(6.110) ̄ (u(y) − u)dy,

where we used the 1-periodicity of u and the definition of u,̄ and we may obtain the estimate 󵄨󵄨ˆ 󵄨󵄨 󵄨󵄨 ̄ 󵄨󵄨󵄨󵄨 ≤ 2‖u‖L∞ (0,1) |Aϵ | ≤ 2‖u‖L∞ (0,1) . 󵄨󵄨 (u(y) − u)dy 󵄨󵄨 Aϵ 󵄨󵄨

(6.111)

Substituting (6.110) in (6.109) and using estimate (6.111), we obtain ˆ 󵄨󵄨 󵄨󵄨 ˆ 󵄨󵄨 󵄨󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨 (v(x) − u)dx ̄ ̄ 󵄨󵄨󵄨󵄨 = 0. = lim (u(y) − u)dy 󵄨 󵄨󵄨 󵄨󵄨 ϵ→0+ 󵄨󵄨󵄨ϵ 󵄨󵄨 󵄨 Iδ 󵄨 󵄨 Aϵ

(6.112)

Recall that we have chosen x0 ∈ (0, 1) to be a Lebesgue point for v, so that ´ ´ ̄ = limδ→0+ δ1 I v(x)dx = v(x0 ). This clearly implies that limδ→0+ δ1 I (v(x) − u)dx δ δ 1 ´ ̄ v(x0 ) − u.̄ On the other hand, (6.112) yields that limδ→0+ δ I (v(x) − u)dx = 0, so that δ ̄ v(x0 ) = u. Since x0 is a Lebesgue point, this property holds a. e. The above lemma, which finds important applications in fields like, e. g., homogenization theory, holds in any dimension as well as for the weak convergence in Lp . For details and the general proof, the reader may consult, e. g., [66] Lemma 2.85 or [44] Theorem 2.6.

6.9.2 Proof of Theorem 6.6.3 (i) The idea of the proof is based upon the observation that a function is bounded above in some ball B(x0 , ρ) if the measure of the superlevel set |A+ (k, x0 , ρ)| vanishes for some k ∗ sufficiently large. More precisely, to use the Caccioppoli- type inequalities in conjunction with a technical inequality due to Stampacchia, in order to guarantee that |A+ (k, x0 , ρ)| vanishes for k large enough. The proof is broken up into 4 steps: 1. We first provide the technical inequality upon which the proof will be based. This is a Stampacchia-like inequality (see [104]) which states that if ϕ : [k0 , ∞) × [R1 , R2 ] → ℝ+ , is nonincreasing and nondecreasing in the first and second variable,

316 | 6 The calculus of variations respectively, and satisfies an inequality of the form ϕ(k2 , r1 ) ≤ cS (k2 − k1 )−α [(r2 − r1 )−β + (k2 − k1 )−β ]ϕ(k1 , r2 )θ , for every r1 < r2 ,

(6.113)

and k1 < k2 ,

with α > 0, β > 0 and θ > 1, then there exists k ∗ such that ϕ(k ∗ , R1 ) = 0. This k ∗ can be expressed as k ∗ = k0 + k̄ with k̄ α = cS󸀠 (R2 − R1 )−β ϕ(k0 , R2 )θ−1 + (R2 − R1 )α ,

(6.114)

θ(α+β)+θ−1

where cS󸀠 = 2 θ−1 cS . To show that (6.113) implies that ϕ(k ⋆ ) = 0 for k ⋆ = k0 + k̄ with k̄ as in (6.114), con̄ − 2−n ) and ̂r = r + 2−n (R − R ), apply the inequality (6.113) sider the sequences k̂n = k(1 n 2 1 for the choice k1 = k̂n−1 , k2 = k̂n , r1 = ̂rn , r2 = ̂rn−1 and use this to prove inductively that α+β ϕ(k̂n , ̂rn ) ≤ 2−n θ−1 ϕ(k̂0 , ̂r0 ) for any n ∈ ℕ. Then use the monotonicity properties of ϕ to see that ϕ(k0 + k,̄ r) ≤ ϕ(k̂n , ̂rn ) and pass to the limit as n → ∞. 2. We now assume that u ∈ DGp (k0 , R0 , 𝒟), and we shall show that upon fixing a radius R ≤ R0 such that B(x0 , R) ⊂ 𝒟, and a k ≥ k0 , and for any r1󸀠 , r3󸀠 (not to be

confused with the above sequence) with the property any k0 ≤ m < k, we have the simultaneous estimates, 󵄨󵄨 + 󸀠 󵄨 −p 󵄨󵄨A (k, x0 , r1 )󵄨󵄨󵄨 ≤ (k − m)

ˆ

R 2

< r1󸀠 < r2󸀠 :=

r1󸀠 +r3󸀠 2

< r3󸀠 < R, for

p

A+ (m,x0 ,r3󸀠 )

(u(x) − m) dx,

(6.115)

and ˆ

p

A+ (k,x0 ,r1󸀠 )

(u(x) − k) dx

−p 󵄨 󵄨p ≤ c󵄨󵄨󵄨A+ (m, x0 , r3󸀠 )󵄨󵄨󵄨 d [(r3󸀠 − r1󸀠 ) + (k − m)−p ]

ˆ

p

A+ (m,x0 ,r3󸀠 )

(u(x) − m) dx.

(6.116)

Note that these are Caccioppoli-type estimates, connecting the Lp norms of the excess function over superlevel sets at different levels (and involving intersections with balls of different radii). We will simplify that notation by setting Ai (k) := A+ (k, x0 , ri󸀠 ), i = 1, 2, 3, and clearly A1 (k) ⊂ A2 (k) ⊂ A3 (k),

(6.117)

∀ k > k0 .

We will use a cutoff function technique. Consider a cutoff function ψ ∈ Cc∞ (B(xo , r2󸀠 ), [0, 1]) : ψ ≡ 1

on B(x0 , r1󸀠 ) and

|∇ψ| ≤

r3󸀠

4 . − r1󸀠

(6.118)

6.9 Appendix | 317

Assume for simplicity11 that p < d. By the properties of ψ, we have that ˆ

p

A1 (k)

(u(x) − k) dx

(6.118)

ˆ

=

(6.117)

≤

ˆ

p

A1 (k)

p

A2 (k)

󵄨 󵄨 ≤ 󵄨󵄨󵄨A2 (k)󵄨󵄨󵄨

ψ(x)p (u(x) − k) dx ψ(x)p (u(x) − k) dx

sp −p p

p d

ˆ (

󵄨 󵄨 ≤ 󵄨󵄨󵄨A2 (k)󵄨󵄨󵄨 (

ˆ

ψ(x) (u(x) − k) dx) sp

sp

A2 (k)

ψ(x) (u(x) − k) dx) sp

sp

A2 (k)

p sp

p sp

(6.119)

,

where we also used the Hölder inequality to create the Lsp norm for v = ψ(u−k), where dp sp = d−p is the critical exponent in the Sobolev embedding (recall the Gagliardo– Niremberg–Sobolev inequalities), and the fact that R can be chosen sufficiently small. This last step is done in an attempt to bring into the game the gradient of the norm and involve in this way the Caccioppoli inequalities. We will continue our estimates using the Sobolev inequality, for v = ψu, raised to the power p. Bearing in mind that on A2 (k) it holds that ∇v = ψ∇u + (u − k)∇ψ, we have that ˆ (

p/sp

ψ(x)sp (u(x) − k) p dx) A2 (k) ˆ 󵄨󵄨 󵄨p ≤ c𝒮 󵄨󵄨∇(ψ(u − k))(x)󵄨󵄨󵄨 dx A2 (k) ˆ 󵄨󵄨 󵄨p = c𝒮 󵄨󵄨ψ(x)∇u(x) + (u(x) − k)∇ψ(x)󵄨󵄨󵄨 dx A (k) ˆ 2 p󵄨 󵄨 󵄨p 󵄨p ≤ c1 (ψ(x)p 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 + (u(x) − k) 󵄨󵄨󵄨∇ψ(x)󵄨󵄨󵄨 )dx s

A2 (k)

(6.118)

≤

ˆ c2 (

A2 (k)

󵄨󵄨 󵄨p p 󸀠 󸀠 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + 4 (r3 − r1 )

ˆ

(6.120)

p

A2 (k)

(u(x) − k) dx),

where we also used the properties of the cutoff function ψ. Combining (6.119) with (6.120), we have that ˆ p (u(x) − k) dx A1 (k)

ˆ

󵄨 󵄨d ≤ c3 󵄨󵄨󵄨A2 (k)󵄨󵄨󵄨 p (

A2 (k)

󵄨󵄨 󵄨p p 󸀠 󸀠 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + 4 (r3 − r1 )

ˆ A2 (k)

p

(u(x) − k) dx).

(6.121)

11 If p > d, the continuity of minimizers follows from the Sobolev embedding theorem. The case p = d requires some proof which is along the lines of the case p > d presented here if not a little simpler.

318 | 6 The calculus of variations ´ If u ∈ DGp+ (𝒟), then we may further estimate A (k) |Du(x)|p dx by the Lp norm of u − k 2 on the larger domain A3 (k) and the measure of this set. Using the relevant Caccioppoli inequality (6.57) for the choice r = r2󸀠 , R = r3󸀠 and bearing in mind that r3󸀠 −r2󸀠 = 21 (r3󸀠 −r1󸀠 ), we have that ˆ A2 (k)

󵄨p 󵄨󵄨 󸀠 󸀠 −p 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx ≤ c0 ((r3 − r2 ) = c0 (2

p

(r3󸀠

−

ˆ A3 (k)

−p r1󸀠 )

ˆ

p 󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A3 (k)󵄨󵄨󵄨)

(6.122)

p

A3 (k)

󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A3 (k)󵄨󵄨󵄨).

´ ´ By (6.122) and (6.121) (as A (k) (u(x) − k)p dx ≤ A (k) (u(x) − k)p dx since A2 (k) ⊂ A3 (k) 2 3 and u > k on these sets) leads to the estimate ˆ

p

A1 (k)

(u(x) − k) dx

−p 󵄨 󵄨d ≤ c4 󵄨󵄨󵄨A2 (k)󵄨󵄨󵄨 p ((r3󸀠 − r1󸀠 )

ˆ A3 (k)

(6.123)

p 󵄨 󵄨 (u(x) − k) dx + 󵄨󵄨󵄨A3 (k)󵄨󵄨󵄨),

for an appropriate constant c4 . So far, we only have estimates for the same level k, but varying the radius of the ball with which we take the intersection of the superlevel set. We now try to assess the effect of trying to vary the level at which the superlevel set is taken, in the above estimate. Consider a lower level m < k, (m ∈ [k0 , k)) and the corresponding superlevel sets A3 (m) = {x ∈ B(x0 , r3󸀠 ) : u(x) > m}. We first estimate |A3 (k)| in terms of relevant quantities on A3 (m). Clearly, u(x) > k implies that u(x) > m so that A3 (k) ⊂ A3 (m). This ´ ´ leads to the estimate A (k) (u(x) − m)p dx ≤ A (m) (u(x) − m)p dx and multiplying both 3

3

sides with (k − m)−p , while since on A3 (k) we have that u(x) > k, so that easily conclude that 󵄨󵄨 󵄨 󵄨 󵄨 −p 󵄨󵄨A1 (k)󵄨󵄨󵄨 ≤ 󵄨󵄨󵄨A3 (k)󵄨󵄨󵄨 ≤ (k − m) ≤ (k − m)

ˆ

−p A3 (m)

ˆ

u(x)−m k−m

> 1 we

p

A3 (k)

(u(x) − m) dx

(6.124)

p

(u(x) − m) dx,

where the first inequality arises trivially from the fact that as r1󸀠 < r3󸀠 , A1 (k) ⊂ A3 (k). Note that (6.124) is the first of the simultaneous estimates, i. e., (6.115). Our second estimate is that ˆ A3 (k)

p

(u(x) − k) dx ≤

ˆ A3 (k)

p

(u(x) − m) dx ≤

ˆ A3 (m)

p

(u(x) − m) dx,

(6.125)

where we first used the fact that on A3 (k), 0 ≤ u−k ≤ u−m and then that A3 (k) ⊂ A3 (m).

6.9 Appendix | 319

Using (6.124) in (6.123) to estimate the term |A3 (k)| inside the bracket, we obtain the symmetric form for (6.123) as ˆ

p

A1 (k)

(u(x) − k) dx p

−p 󵄨 󵄨 ≤ c4 󵄨󵄨󵄨A3 (m)󵄨󵄨󵄨 d [(r3󸀠 − r1󸀠 ) + (k − m)−p ]

ˆ

(6.126)

p

A3 (m)

(u(x) − m) dx,

which is the second of the simultaneous estimates, i. e., (6.116). 3. We will now try to combine the simultaneous estimates (6.115)–(6.116) into a single Stampacchia-like inequality, which will guarantee that |A(k, x0 , ρ)| vanishes for k large enough. Since r1󸀠 < r3󸀠 and m < k are arbitrary, we relabel them as r1 < r2 and k1 < k2 respectively, and defining the functions ϕ1 (k, r) :=

ˆ

p

A+ (k,x

0 ,r)

(u(x) − k) dx,

󵄨 󵄨 ϕ2 (k, r) = 󵄨󵄨󵄨A+ (k, x0 , r)󵄨󵄨󵄨,

−p G(s1 , s2 ; p) := s−p 1 + s2 ,

we express the simultaneous estimates (6.115)–(6.116) as p

ϕ1 (k2 , r1 ) ≤ c G(r2 − r1 , k2 − k1 ; p)ϕ1 (k1 , r2 )ϕ2 (k1 , r2 ) d ,

(6.127)

ϕ2 (k2 , r1 ) ≤ G(0, k2 − k1 ; p)ϕ1 (k1 , r2 ).

We raise the first one to the power q1 and the second to the power q2 (for q1 , q2 to be specified shortly), and multiply to get ϕ1 (k2 , r1 )q1 ϕ2 (k2 , r1 )q2 ≤ c5 G(r2 − r1 , k2 − k1 ; p)q1 G(0, k2 − k1 ; p)q2 ϕ1 (k1 , r2 )q1 +q2 ϕ2 (k1 , r2 )

q1 p d

.

(6.128)

We define the function ϕ(k, r) = ϕ1 (k, r)q1 ϕ2 (k, r)q2 , and note that if ϕ vanishes for values of k above a certain threshold, then |A+ (k, x0 , r)| also vanishes, which is fact is the type of estimate we seek. We now see that if q1 , q2 are chosen so that there exists a θ > 1 such that q1 + q2 = θq1 ,

and

q1 p = θq2 , d

(6.129)

then inequality (6.128) becomes ϕ(k2 , r1 ) ≤ c5 G(0, k2 − k1 ; p)q2 G(r2 − r1 , k2 − k1 ; p)q1 ϕ(k1 , r2 )θ ,

θ > 1.

(6.130)

320 | 6 The calculus of variations Noting further that (x + y)r ≤ cr (xr + yr ) for any x, y ≥ 0 and an appropriate constant cr , inequality (6.130) becomes (for an appropriate constant c6 ) ϕ(k2 , r1 ) ≤ c6 G(0, k2 − k1 ; pq2 )G(r2 − r1 , k2 − k1 ; pq1 )ϕ(k1 , r2 )θ ,

θ > 1,

(6.131)

which is in the general form of the Stampacchia inequality (6.113). In order to proceed further, we express the equations (6.129) in terms of q = θ =1+

1 , q

and

θ=

q1 q2

as

p q, d

which shows that we may find an infinity of such choices as long as q satisfies the above algebraic system, the solution of which yields θ=

1 1 p + √ + > 1, 2 4 d

θ=

p q, d

where we note that θ(θ − 1) = dp . To simplify the resulting inequality we choose q2 = 1, so that q1 = q, which brings (6.131) to the form ϕ(k2 , r1 ) ≤ c󸀠 (k2 − k1 )−p ((r2 − r1 )−pq + (k2 − k1 )−pq ) ϕ(k1 , r2 )θ ,

θ > 1,

(6.132)

which for c󸀠 = c6 is in the exact form (6.113) for R1 = R2 , R2 = R and the exponents α = p, β = pq and θ > 1, as given above. We therefore have that ϕ(k0 + k,̄ R1 ) = ϕ(k0 + k,̄ R2 ) = 0 for k̄ satisfying R k̄ p = c󸀠 ( ) 2

−pq

p

R ϕ(k0 , R)θ−1 + ( ) , 2

which implies that for an appropriate constant c7 , k̄ ≤ c7 (R−q ϕ(k0 , R)

(θ−1) p

+ R)

ˆ

󵄨 θ−1 = c7 (R 󵄨󵄨A (k0 , x0 , R)󵄨󵄨󵄨 p ( −q 󵄨󵄨 +

|A+ (k0 , x0 , R)| ) Rd

A+ (k0 ,x0 ,R)

ˆ

󵄨 θ−1 = c7 (R 󵄨󵄨A (k0 , x0 , R)󵄨󵄨󵄨 p ( −q 󵄨󵄨 +

= c7 ((

p

θ−1 p

(

p

A+ (k0 ,x0 ,R)

1 Rd

(u(x) − k0 ) )

(θ−1)q p

+ R)

1 p

(u(x) − k0 ) ) + R)

ˆ

A+ (k0 ,x0 ,R)

p

1 p

(u(x) − k0 ) ) + R),

where we used the fact that (θ−1)q = d(θ−1)θ = p1 . This leads to the stated upper bound p p2 for cDG = c7 . 4. Since u ∈ DGp (k0 , R0 , 𝒟), we also have that −u ∈ DGp (−k0 , R0 , 𝒟) so that working with −u in the place of u we also obtain the stated lower bound.

6.9 Appendix | 321

(ii) To show the local Hölder continuity of u, we first note that since u ∈ DGp (R0 , 𝒟) from (i) we have that the stated bounds hold for every k0 ∈ ℝ, hence, ‖u‖L∞ ≤ c‖u‖Lp + c󸀠 , i. e., we have an L∞ bound from a Lp bound (recall that we assume that u ∈ W 1,p (𝒟)). We therefore have to estimate the Hölder norm for C 0,α for an appropriate choice of α > 0. For notational convenience, we redefine all constants except c0 and cDG . We introduce the notion of the oscillation of a locally bounded function u over a ball B(x0 , R) ⊂ 𝒟, as osc(x0 , R) := M(x0 , R) − m(x0 , R), M(x0 , R) :=

sup u(x),

x∈B(x0 ,R)

where

and m(x0 , R) :=

inf

x∈B(x0 ,R)

u(x).

(6.133)

The oscillation is a quantity that may provide important information on the local continuity properties of u. In fact, if for any r < R < R0 α

r osc(x0 , r) ≤ ĉ[( ) + r α ], R

(6.134)

for an appropriate constant ĉ, then u is locally Hölder continuous with Hölder exponent α (see Theorem 1.8.8). Our aim is to show that for a function in the De Giorgi class there exists an α > 0 such that (6.134) holds, hence, u is locally Hölder continuous. We break the proof in 4 steps: 1. By assumption, we have that u ∈ DGp+ (R0 , 𝒟) = ⋂k0 ∈ℝ DGp (k0 , R0 , 𝒟). We now consider some R > 0 such that B(x0 , 2R) ⊂⊂ 𝒟 with 2R ≤ R0 and define 1 k̄ = (M(x0 , 2R) + m(x0 , 2R)). 2 Clearly, u ∈ DGp+ (k,̄ R0 , 𝒟). Without loss of generality, we may assume that A+ (k,̄ x0 , R) has the property12 that 󵄨󵄨 + ̄ 󵄨 󵄨󵄨A (k, x0 , R)󵄨󵄨󵄨 ≤

1 󵄨󵄨 󵄨 1 d 󵄨B(x , R)󵄨󵄨 = ω R , 2󵄨 0 󵄨 2 d

(6.135)

where ωd is the volume of the unit ball in ℝd . We now consider the sequence kn = M(x0 , 2R) − 2−n−1 osc(x0 , 2R),

n = 0, 1, . . . ,

(6.136)

12 To see why we can make the above assumption, we have for any k ∈ ℝ that |A+ (k, x0 , R)| + |A− (k, x0 , R)| = |B(x0 , R)|, so either |A+ (k, x0 , R)| ≤ 21 |B(x0 , R)|, or |A− (k, x0 , R)| ≤ 21 |B(x0 , R)|. If the first inequality holds, we are ok. If the second hold that our assumption is valid for −u (which by assumption satisfies −u ∈ DGp+ (R0 , 𝒟) = ⋂k0 ∈ℝ DGp (−k0 , R0 , 𝒟), and work with −u instead. The choice of 1/2 is indicative, one may choose any β < 1.

322 | 6 The calculus of variations which clearly satisfies k0 = k̄ and kn ↑ M(x0 , 2R) and as by assumption u ∈ DGp+ (kn , R0 , 𝒟) for every n ∈ ℕ, apply the L∞ bound (upper bound) for every kn to get a series of estimates on the supremum of u in the ball B(x0 , R2 ), in terms of Lp bounds in B(x0 , R). Indeed, applying part (i) of the present theorem we have that for every n ∈ ℕ, M(x0 ,

R ) 2

= sup u(x)

(6.137)

B(x0 , R2 )

|A+ (kn , x0 , R)| ) ≤ kn + cDG (( Rd We estimate ˆ

p

A+ (kn ,x0 ,R)

θ−1 p

(u(x) − kn ) dx ≤ [

1 ( d R

ˆ

1 p

p

A+ (kn ,x0 ,R)

(u(x) − kn ) dx) + R).

p 󵄨 󵄨 (u(x) − kn )] 󵄨󵄨󵄨A+ (kn , x0 , R)󵄨󵄨󵄨

sup

A+ (k

n ,x0 ,R)

p 󵄨 󵄨 ≤ [ sup (u(x) − kn )] 󵄨󵄨󵄨A+ (kn , x0 , R)󵄨󵄨󵄨

(6.138)

B(x0 ,2R)

p󵄨 󵄨 = ( M(x0 , 2R) − kn ) 󵄨󵄨󵄨A+ (kn , x0 , R)󵄨󵄨󵄨,

and use (6.138) to enhance inequality (6.137) in order to obtain M(x0 ,

R ) = sup u(x) 2 B(x , R ) 0 2

(6.139)

θ

|A+ (kn , x0 , R)| p ) (M(x0 , 2R) − kn ) + R], ≤ kn + cDG [( Rd

∀ n ∈ ℕ. |A+ (k ,x ,R)|

θ

n 0 2. We now require some decay estimates for the sequence of the terms ( )p , Rd ∗ that ensure that there exists an n ∈ ℕ (to be specified shortly in step 4, see (6.170)), such that for all n ≥ n∗ we may guarantee that θ

|A+ (kn , x0 , R)| p 1 ( ) < . 2cDG Rd

(6.140)

We defer the proof of this claim until step 4. Accepting estimate (6.140) for the time being we proceed with the bounds (6.139) fixing n = n∗ to get that M(x0 ,

R 1 ) ≤ kn∗ + (M(x0 , 2R) − kn∗ ) + cDG R 2 2 −(n∗ +2)

= M(x0 , 2R) − 2

osc(x0 , 2R) + cDG R,

(6.141)

where we used the definition of the sequence kn (see (6.136)). Clearly, m(x0 ,

R ) = inf u ≥ inf u = M(x0 , 2R), B(x0 ,2R) 2 B(x0 , R2 )

(6.142)

6.9 Appendix | 323

so that multiplying (6.142) by −1 and adding the result to (6.141) we obtain an estimate for the oscillation in terms of osc(x0 ,

∗ R ) ≤ cDG R + (1 − 2−(n +2) ) osc(x0 , 2R). 2

(6.143)

Recall that R was arbitrary and such that 2R ≤ R0 , so that we may rephrase our estimate as (upon setting ρ = R/2), ρ ρ osc(x0 , ) ≤ c + λ osc(x0 , ρ), 4 4

∀ρ ∈ (0, R0 ], where

1 < λ < 1, 4

(6.144)

∗

where we have redefined the constant c = 8cDG and set λ = 1 − 2−(n +2) < 1. 3. We claim that the estimate (6.144) guarantees that for any r < R < R0 α

r osc(x0 , r) ≤ ĉ[( ) + r α ], R

(6.145)

for an appropriate constant ĉ, which is an estimate of the form (6.134). As stated in the beginning of the proof this is a crucial estimate that guarantees the local Hölder continuity of u with Hölder exponent α. To prove the claim (6.145), consider the nonnegative (and nondecreasing) function ϕ defined by ϕ(r) = osc(x0 , r) for any r ∈ [0, R0 ], and express (6.144) as ρ ρ ϕ( ) ≤ c + λϕ(ρ), 4 4

1 < λ < 1, ∀ ρ ∈ (0, R0 ). 4

with

(6.146)

Fix any pair r, R such that 0 < r < R < R0 and choose N such that We set ρn = ϕ(

R , 4n+1

R . 4N

(6.147)

1 < λ < 1, n = 0, . . . , N − 1. 4

(6.148)

R

4N+1

1. From the choice of N and the monotonicity of the function ϕ, from (6.149) we conclude that

ϕ(r) ≤ ϕ(

R ) ≤ λN (ϕ(R) + cc3 R). 4N

(6.150)

324 | 6 The calculus of variations Recalling that ϕ(r) = osc(x0 , r), we observe that (6.150) is very close to our claim (6.145), if we could connect λN with an appropriate power of Rr .

λ=

Indeed, by the choice of N in (6.147) we have that

( 41 )α

(for α =

4 − ln ) ln λ

and rewrite

λN = λ−1 λN+1 = λ−1 (

1

α

) 4N+1

(6.147)


0, there exists a constant c = c(γ, T, R, z0 ) > 0, such that infQ(z0 ,TR) u ≥ c(γ, T, R)θ. This is an interesting and useful result, which allows us to obtain lower bounds on the infimum of a positive function in the De Giorgi class, on a larger cube, by estimates on the measure of the sublevel sets at a particular level θ. We can see that it suffices to show this result for R = 1 and z0 = 0. We will use the simplified notation Q(R) for the resulting cubes. 1(a). We start with a useful observation: Assume that for some θ > 0 and γ > 0 we have that |A− (θ, 1)| ≤ γ|Q(1)| and consider any T > 1/2. Then the condition |A− (θ, 1)| ≤ 1−γ γ|Q(1)| implies that13 |A− (θ, 2T)| ≤ (1 − (2T) d )|Q(2T)|. This means that if the stated condition holds for a level θ on a cube of size 1 then it also holds for the same level (but a different constant γ) for a cube of size 2T for any T > 1/2. Consider R󸀠 = 2T as a new 1−γ 󸀠 cube side. Then the condition is a condition of the form |A− (θ, R󸀠 )| ≤ (1 − (2T) d )|Q(R )|, which we wish to connect with a result on the infQ(R󸀠 /2) u, and upon rescaling R󸀠 = 1 the result we seek must essentially connect a condition of the form |A− (θ, 1)| ≤ γ|Q(1)| with a lower bound of the form infQ( 1 ) u ≥ λθ. 2

1(b). It therefore suffices to show that: If u ≥ 0 on Q(2) and as long as for some γ ∈ (0, 1), θ > 0 it holds that |A− (θ, 1)| ≤ γ|Q(1)|, then infQ( 1 ) u ≥ λ(γ)θ, for some 2 constant λ(γ) > 0. Note that there is no restriction on the size of γ in this claim. This is Lemma 7.5 in [67]. To prove this claim, we first note that for a positive function u ∈ DGp− (𝒟), and for any k1 < k2 < θ, it holds that ˆ

󵄨 󵄨 p−1 󵄨 󵄨 d−1 (k2 − k1 )󵄨󵄨󵄨A− (k1 , ρ)󵄨󵄨󵄨 d ≤ c󵄨󵄨󵄨A− (k2 , ρ) \ A− (k1 , ρ)󵄨󵄨󵄨 p (

A− (k2 ,ρ)

p

1/p

|∇u| )

󵄨 󵄨 p−1 󵄨 󵄨1/p ≤ c󵄨󵄨󵄨A− (k2 , ρ) − A− (k1 , ρ)󵄨󵄨󵄨 p k1 (R − ρ)−1 󵄨󵄨󵄨A− (k2 , R)󵄨󵄨󵄨 .

(6.171)

To show (6.171), we need to work as follows: The first two inequalities in (6.171) follow by a Poincaré-type inequality (see Example 1.5.15) applied to the cutoff function v(x) = 01u≥k2 + (k − u)1k1 0, and assume that it is chosen so that |A− (θ󸀠 , 1)| ≤ 󸀠 γ |Q(1)|. We will show that as long as γ 󸀠 > 0 is small enough, then a positive u satisfies 󸀠 infQ( 1 ) u ≥ θ2 . Our starting point is (6.171), which we now treat inside the unit cube 2

(i. e., ρ < R < 1) and we further estimate as

󵄨 󵄨 p−1 k 󵄨󵄨 − 󵄨 󵄨1 󵄨 d−1 (k2 − k1 )󵄨󵄨󵄨A− (k1 , ρ)󵄨󵄨󵄨 d ≤ c󵄨󵄨󵄨A− (k2 , ρ) \ A− (k1 , ρ)󵄨󵄨󵄨 p 󵄨󵄨A (k2 , R)󵄨󵄨󵄨 p (R − ρ) p−1 k c 󵄨󵄨 − 󵄨 󵄨 󵄨󵄨 − 󵄨1 󵄨 ≤ c󵄨󵄨󵄨A− (k2 , ρ)󵄨󵄨󵄨 p 󵄨󵄨A (k2 , R)󵄨󵄨󵄨 p ≤ 󵄨A (k2 , R)󵄨󵄨󵄨, (R − ρ) (R − ρ) 󵄨 where we used the obvious inequality |A− (k2 , ρ)| ≤ |A− (k2 , R)|. Rearranging the last inequality, we obtain − d 󵄨 − 󵄨󵄨 − 󵄨 󵄨 d 󵄨󵄨A (k1 , ρ)󵄨󵄨󵄨 ≤ c((k2 − k1 )(R − ρ)) d−1 󵄨󵄨󵄨A (k2 , R)󵄨󵄨󵄨 d−1 .

(6.177)

This is true for any ρ, R ∈ [0, 1]. We now consider the sequences ri =

1 (1 2

+ 2−i ) and

󸀠 , ki󸀠 = θ2 (1 + 2−i ) and apply (6.177) consecutively for ρ = ri+1 , R = ri and k2 = ki󸀠 , k1 = ki+1 for every i ∈ N, to obtain the iterative scheme 󸀠

2d d i󵄨 − 󸀠 󵄨󵄨 − 󸀠 󵄨 󵄨 d 󵄨󵄨A (ki+1 , ri+1 )󵄨󵄨󵄨 ≤ 4 d−1 (4 d−1 ) 󵄨󵄨󵄨A (ki , ri )󵄨󵄨󵄨 d−1 . d

(6.178) 2d

d

1 d Noting that d−1 > 1 and since 4 d−1 > 1, setting α = d−1 , c0 = 4 d−1 and c1 = 4 d−1 , − 󸀠 the inequality (6.178) assumes the form |A (ki+1 , ri+1 )| ≤ c0 c1i |A− (ki󸀠 , ri )|1+α . We will −

1 2

show that as long as |A− (k0󸀠 , r0 )| ≤ c0−1/α c1 α , then |A− (ki󸀠 , ri )| ≤ c1−i/α |A− (k0󸀠 , r0 )|, so that |A− (ki󸀠 , ri )| → 0. This can be proved easily by induction. To guarantee the condition on the initial value of the sequence, since we know that |A− (θ󸀠 , 1)| ≤ γ 󸀠 |Q(1)|, −

1 2

it suffices to have γ 󸀠 ≤ c0−1/α c1 α . As long a γ 󸀠 is sufficiently small (as given above) then since |A− (ki󸀠 , ri )| → 0., and because ki󸀠 ↓

u≥

θ󸀠 2

θ󸀠 2

and ri ↓

1 , 2

this result implies that

on Q(1/2), hence the required result. By (6.176), we apply this result for the level ⋆

θ󸀠 = 2−n θ, choosing n⋆ sufficiently large so that γ(n⋆ ) satisfies the smallness condition for γ 󸀠 above, so that the bound infQ(1/2) u ≥ 21 θ󸀠 , which provides us with the required lower bound infQ(1/2) u ≥ 21 2−n θ. 2. We will now show that ⋆

u(x0 ) ≤ c inf u. Q(x0 ,R)

It suffices to show that w(x) :=

u(x) u(x0 )

≥ c > 0 in Q(x0 , R).

(6.179)

6.9 Appendix | 331

1 2

Clearly, w is in the same De Giorgi class as u, so that for every x ∈ 𝒟 and ρ < R < dist(x, 𝜕𝒟), it satisfies the oscillation criterion (see (6.145) and Remark 6.9.2) α

α

ρ ρ oscQ(x,ρ) w ≤ c ( ) oscQ(x,R) w ≤ c ( ) ‖w‖L∞ (Q(x,R)) . R R

(6.180)

We now consider the function w in a cube centered at x0 of varying side Q(x0 , τ), and fixing a lower bound K for ‖w‖L∞ (Q(x0 ,τ)) we will look for the largest value of τ such that ‖w‖L∞ (Q(x0 ,τ)) ≥ K. Clearly, as K → ∞ it must hold that τ → 0. We will adopt the scaling

K = K(τ) := (1 − τ)−δ , for τ ∈ [0, 1) and some δ > 0 to be chosen shortly, and note that there must exist a τ0 ∈ [0, 1), the largest value of τ with the property we seek, i. e., such that ‖w‖L∞ (Q(x0 ,τ0 )) ≥ (1 − τ0 )−δ . For any τ > τ0 , it must hold that ‖w‖L∞ (x0 ,τ) < (1 − τ)−δ . By the continuity of w, there exists a x̂ ∈ Q(x0 , τ0 ), such that w(x̂) = ‖w‖L∞ (Q(x0 ,τ0 )) . 1−τ 1+τ 1+τ As Q(x̂, 2 0 ) ⊂ Q(x0 , 2 0 ), and 2 0 > τ0 , we have that ‖w‖L∞ (Q(̂x, 1−τ0 )) ≤ ‖w‖L∞ (Q(̂x, 1+τ0 )) < K( 2

2

1 + τ0 ) = 2δ (1 − τ0 )−δ . 2

(6.181)

We apply the oscillation criterion (6.180) for the cubes Q(x̂, ρ) and Q(x̂, R), for R = and ρ = ϵα R, for ϵ < 1 to be chosen shortly. This combined with the estimate (6.181) yields that 1−τ0 2

oscQ(̂x, 1−τ0 ϵ) w ≤ cϵα 2δ (1 − τ0 )−δ , 2

and since for every x ∈ Q(x̂, w(x) ≥

inf

1−τ Q(̂x, 2 0

ϵ)

w=

1−τ0 ϵ) 2

sup

1−τ Q(̂x, 2 0

it clearly holds that ϵ)

w − oscQ(̂x, 1−τ0 ϵ) w ≥ (1 − τ0 )−δ (1 − c2δ ϵα ), 2

by the choice of x̂. Choosing ϵ appropriately (e. g., such that 1 − c2δ ϵα = 21 ), we obtain that 1 w(x) ≥ (1 − τ0 )−δ , 2

on Q(x̂,

1 − τ0 ϵ). 2 1−τ

That means |A− (θ0 , R0 )| = 0 ≤ γ|Q(R0 )| for θ0 = 21 (1 − τ0 )−δ and R0 = 2 0 ϵ so that we may apply the result of step 1 for the choice T = 2, and γ = 0 to obtain that 1 w(x) ≥ μθ0 = μ (1 − τ0 )−δ , 2

on Q(x̂, 2R0 ) = Q(x̂, (1 − τ0 )ϵ).

But setting θ1 = μ 21 (1 − τ0 )−δ and R1 = (1 − τ0 )ϵ this implies that |A− (θ1 , R1 )| = 0 ≤ γ|Q(R1 )|, so using once more the result of step 1 for the choice T = 2 we obtain that 1 w(x) ≥ μθ1 = μ2 (1 − τ0 )−δ , 2

on Q(x̂, 2R1 ) = Q(x̂, 2(1 − τ0 )ϵ).

332 | 6 The calculus of variations We will keep iterating the above argument up to level n⋆ , so that we obtain the estimate w(x) ≥ μn

⋆

1 (1 − τ0 )−δ , 2

on Q(x̂, 2R1 ) = Q(x̂, 2n

⋆

−1

(1 − τ0 )ϵ).

We will choose n⋆ so that the corresponding cube Q(x̂, 2n −1 (1−τ0 )ϵ) contains the cube Q(x0 , 1). For this choice, we see that there exists a constant c > 0 such that w(x) ≥ c > 0, therefore, u(x0 ) ≤ c infQ(x0 ,1) u(x). Clearly, upon scaling, this result is true for any Q(x0 , R), for an appropriate choice of c > 0. To conclude the proof, let us choose a cube Q(x1 , ρ) ⊂ 𝒟 for a suitable x1 ∈ 𝒟 and ρ > 0 and consider a point x0 ∈ Q(x1 , ρ) such that u(x0 ) = supQ(x1 ,ρ) u. Since (6.179) holds for any R, we choose an R sufficiently larger than ρ, i. e., R = 3ρ, so that ⋆

u(x0 ) = sup u ≤ c inf u ≤ c inf u(x), Q(x1 ,ρ)

Q(x0 ,R)

Q(x1 ,ρ)

which is the required result on Q(x1 , ρ). The general result follows by a covering argument. Remark 6.9.3. The proof presented here follows the alternative approach of Giusti, [67], which uses the assumption of Hölder continuity, which follows as long as Caccioppoli-type inequalities of the form (6.57) hold (see also Remark 6.6.2). The proof of Di Benedetto and Trudinger in [56] uses a slightly different approach, using the upper bound (6.61) together with a lower bound derived by the estimates of step 1 of the proof of Theorem 6.6.5 combined with a covering result due to Krylov and Safonov (see [76], see also Proposition 7.2 in [67]), according to which for positive functions in the De Giorgi class DGp (𝒟), there exists a q > 0 such that inf u ≥ c((

Q(x0 , R2 )

1 q

Q(x0 ,R)

uq dx) − c2 Rα ),

with c2 as in Remark 6.6.9. 6.9.4 Proof of generalized Caccioppoli estimates In this section, we provide a proof of the generalized Caccioppoli estimates of Remark 6.6.9. For simplicity, we provide the proof in the case where q = p, s = r = ∞ and ϵ = dp , referring to Theorem 7.1 in [67] for the general case. In particular, using the same arguments and bringing the extra terms resulting from the modified growth condition on the left-hand side to the right, (6.67) can be modified as ˆ ˆ 󵄨󵄨 󵄨󵄨p 󵄨󵄨 󵄨󵄨p 󵄨󵄨∇u(x)󵄨󵄨 dx ≤ c( 󵄨󵄨∇(u + ϕ)(x)󵄨󵄨 dx A2

ˆ

+

A2

A2

󵄨 󵄨p b(x)󵄨󵄨󵄨u(x) + ϕ(x)󵄨󵄨󵄨 dx +

ˆ

A2

󵄨 󵄨p b(x)󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx +

ˆ

A2

a(x)dx),

(6.182)

6.9 Appendix | 333

where we have chosen the constant c > 0 large enough.14 Working as above, by the properties of ϕ and ψ, this reduces to ˆ ˆ p 󵄨󵄨p 󵄨󵄨 󵄨󵄨p 󵄨󵄨 −p p (u(x) − k) dx 󵄨󵄨∇u(x)󵄨󵄨 dx ≤ c1 ( 󵄨󵄨(1 − ψ(x)) ∇u(x)󵄨󵄨 dx + (r2 − r1 ) A2 A2 A2 (6.183) ˆ ˆ ˆ 󵄨󵄨p 󵄨󵄨 󵄨󵄨p 󵄨󵄨 a(x)dx), b(x)󵄨󵄨u(x)󵄨󵄨 dx + + b(x)󵄨󵄨(u + ϕ)(x)󵄨󵄨 dx +

ˆ

A2

A2

A2

which now has 3 extra terms that we must control. In order to do this, we add ´ p A2 b(x)|u(x)| dx on both sides of this inequality, and trying to obtain a term of ´ the form A b(x)(1 − ψ(x))2 |u(x)|p dx out of this addition on the right-hand side we 2 express u on A2 as u = (1 − ψ)u + ψ((u − k)+ + k), noting that for an appropriate constant c(p) > 0 we have (dropping the explicit dependence on x for convenience) that |u|p ≤ c(p)(1 − ψ)p |u|p + ψp |(u − k)+ |p + ψp k p . We see that the first contribution in the above estimate, combined with the similar contributions that arise from the third and fourth terms on the right-hand side of (6.183) upon expressing u on A2 as u + ϕ = (1 − ψ)u + kψ, and using the estimate |u + ϕ|p ≤ c(p)((1 − ψ)p |u|p + k p ψp ), ´ will provide the term A b(x)(1 − ψ(x))2 |u(x)|p dx we seek for on the right-hand side. 2 Combining the above, and using an appropriate constant c2 , we have an estimate of the form ˆ ˆ 󵄨󵄨 󵄨p 󵄨 󵄨p b(x)󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + A2

A2

ˆ

ˆ 󵄨 󵄨p 󵄨 󵄨p (1 − ψ(x))p 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + (1 − ψ(x))p 󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx A2 A2 ˆ ˆ p + (r2 − r1 )−p (u(x) − k) dx + b(x)ψ(x)p (u(x) − k)p dx

≤ c2 (

+ kp

ˆ A2

A2

b(x)ψ(x)p dx +

ˆ A2

A2

a(x)dx).

The last two terms can be estimated using |A2 | and the relevant Lebesgue norm of a and b, as ˆ ˆ p p k b(x)ψ(x) dx + a(x)dx ≤ (‖b‖L∞ (B(R)) k p + ‖a‖L∞ (B(R)) )|A2 |. A2

A2

´ The term which still is problematic is A bψp (u − k)p dx. As we would like to connect 2 this term with estimates of the gradient of u, our strategy is to first use the Hölder inequality to create the relevant norm for which the Sobolev inequality holds, and then apply the Sobolev inequality in order to introduce the gradient into the game. To 14 Larger than all individual constants which appear.

334 | 6 The calculus of variations this end, we estimate (dropping explicit x dependence for convenience) ˆ A2

p

ˆ

p

bψ (u − k) dx ≤ (

≤ cR

ψ 󵄨󵄨(u − k) 󵄨󵄨 dx) sp 󵄨󵄨

A

ˆ2

A2

ˆ

≤ cR󸀠 (

+ 󵄨󵄨sp

p sp

󵄨󵄨 + 󵄨p 󵄨󵄨∇(ψ(u − k) )󵄨󵄨󵄨 dx

A2

p

‖b‖L∞ (B(R)) |A2 | d

|∇u|p dx + (r2 − r1 )−p

ˆ A2

p

((u − k)+ ) dx),

where for the third inequality we used the Sobolev inequality and then our standard ´ estimate for A |∇ψ(u − k)+ |p dx. The constants cR and cR󸀠 depend on R, and are propor2 tional to p 󵄨 󵄨p cR = ‖b‖L∞ (B(R)) |A2 | d ≤ ‖b‖L∞ (B(R)) 󵄨󵄨󵄨B(R)󵄨󵄨󵄨 d

Choosing R sufficiently small (depending on the data of the problem), we may take ´ cR󸀠 < 21 , and hence, bring the term A |∇u|p dx to the left-hand side with a positive sign. 2 Combining the above, and observing that 1 − ψ = 1A2 \A1 , so that the integral of any function multiplied by (1 − ψ) reduces to the integral over A2 \ A1 , we have for a new constant c3 > 0 that ˆ

ˆ 󵄨󵄨 󵄨p 󵄨 󵄨p b󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + A2 A2 ˆ ˆ 󵄨󵄨 󵄨󵄨p ≤ c3 ( 󵄨󵄨∇u(x)󵄨󵄨 dx + A2 \A1

+ (r2 − r1 )−p

ˆ

A2

A2 \A1

󵄨󵄨 󵄨p 󵄨󵄨u(x)󵄨󵄨󵄨 dx

p

(u(x) − k) dx + (‖b‖L∞ (B(R)) k p + ‖a‖L∞ (B(R)) )|A2 |),

which is trivially enhanced by noting that A1 ⊂ A2 as ˆ

ˆ 󵄨󵄨 󵄨p 󵄨 󵄨p b󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + A1 A1 ˆ ˆ 󵄨󵄨 󵄨p ≤ c3 ( 󵄨󵄨∇u(x)󵄨󵄨󵄨 dx + A2 \A1

+ (r2 − r1 )−p

ˆ

A2

A2 \A1

󵄨󵄨 󵄨p 󵄨󵄨u(x)󵄨󵄨󵄨 dx

p

(u(x) − k) dx + (‖b‖L∞ (B(R)) k p + ‖a‖L∞ (B(R)) )|A2 |),

We can now use the hole filling technique for the positive quantity |∇u|p +b|u|p , adding the integral of this quantity over A1 multiplied by c3 > 0 on both sides to get upon rearranging the same inequality but now with a new constant c4 < 1. The proof may now proceed as in step 2 of Proposition 6.6.7.

7 Variational inequalities The aim of this chapter is the study of variational inequalities. These are important counterparts to PDEs and involve inequalities rather than equalities. One way to derive variational inequalities is to study minimization problems for certain functionals on closed and convex sets rather than on a linear subspace. As a first example in a long series of related applications, one may consider the so-called obstacle problem that consists of the equilibration of an elastic membrane over an obstacle which results to a differential inequality involving the Laplacian and the given obstacle. In order to set the ideas, we start our treatment of variational inequalities with this problem, establish its connections with free boundary value problems and consider certain regularity issues. We then study a general class of variational inequalities, not necessarily related to minimization problems, and present a general theoretical framework developed by Lions, Stampacchia, Lax and Milgram for their treatment. Certain approximation methods for problems of this type, e. g., the penalization method or internal approximation schemes that may give rise to numerical algorithms are presented. The chapter closes with application in a general class of variational inequalities involving elliptic operators. Importantly, this framework allows also the treatment of a general class of PDEs, uniformly elliptic boundary value problems a direction which is pursued here as well. Variational inequalities find important applications in various fields including decision theory, engineering and mechanics or mathematical finance (see, e. g., [23, 32]). Many monographs have been dedicated to the theoretical and numerical aspects of variational inequalities (see, e. g., [23, 32, 73, 100, 106]).

7.1 Motivation We will first motivate the material of this chapter through some examples. Example 7.1.1 (Minimization in ℝd ). When treating the minimization problem of a C 1 function on an interval [a, b], then the minimum xo is located either at an interior point or at its end points. In any case it holds that f 󸀠 (xo ) (x − xo ) ≥ 0 for every x ∈ [a, b]. Now, let C be a closed convex and bounded subset in ℝd , consider f : C → ℝ continuously differentiable, and denote by ∇f := Df its gradient. By Weierstrass’ theorem, there exists xo ∈ C such that f (xo ) = minx∈C f (x). Given any z ∈ C, set zt = (1 − t)xo + tz, t ∈ [0, 1]. Consider the function ϕ : [0, 1] → ℝ, defined by ϕ(t) = f ((1 − t)xo + tz). Clearly, ϕ has a minimum at to = 0. Since ϕ is differentiable, and denoting Dϕ =: ϕ󸀠 , by our previous observation it holds that ϕ󸀠 (0)t ≥ 0 for every t ∈ [0, 1], or equivalently ϕ󸀠 (0) ≥ 0. But, ϕ󸀠 (0) = ∇f (xo ) ⋅ (z − xo ), where ⋅ denotes the inner product in ℝd . Hence, ∇f (xo ) ⋅ (z − xo ) ≥ 0, for every z ∈ C. ◁ Example 7.1.2 (Laplace equation). Recall (see Example 6.1.1) that the minimization of ´ ´ the Dirichlet functional F : W01,2 (𝒟) → ℝ, defined by F(u) = 21 𝒟 |∇u|2 dx − 𝒟 fu dx https://doi.org/10.1515/9783110647389-007

336 | 7 Variational inequalities over the whole of W01,2 (𝒟), leads to a first-order condition which can be expressed in the form of the Laplace equation −Δu = f on 𝒟 complemented with homogeneous Dirichlet boundary conditions on 𝒟. Motivated by the previous examples and in particular by Example 7.1.1, it would be interesting to ask the question: What would happen if instead of minimizing the functional F over the whole of W01,2 (𝒟) we minimized over a closed convex and bounded set C ⊂ W01,2 (𝒟) instead? A useful example of such a set is C0 := {u ∈ W01,2 (𝒟) : u(x) ≥ ϕ(x) a. e. x ∈ 𝒟} for a suitable choice of a given function ϕ with sufficient regularity. As already seen in Proposition 2.4.5, for an abstract formulation of this problem, the first-order condition will be of the form ⟨DF(uo ), u − uo ⟩ − ⟨f , u − uo ⟩ ≥ 0 for every u ∈ C0 , which recalling that ⟨DF(u), v⟩ = ⟨−Δu, v⟩ for every v ∈ X, allows us to interpret the first-order condition for the problem as the differential inequality find uo ∈ C0 : ⟨−Δuo − f , u − uo ⟩ ≥ 0,

(7.1)

∀ u ∈ C0 ,

instead of a differential equation. Such inequalities are called variational inequalities, and can be expressed in an equivalent form as free boundary value problems. ◁ In all the above examples, which we used as motivation, variational inequalities were introduced as first order conditions for minimization problems in convex subsets of a Banach space. This is of course not the general case, as one may obtain interesting examples of variational inequalities in a more general context, not necessarily related to minimization problems. The basic aim of this chapter therefore is to develop a theory for the treatment of problems of the general form find xo ∈ C : ⟨A(xo ), x − xo ⟩ ≥ 0,

(7.2)

for all x ∈ C,

where C ⊂ X is a closed convex subset of a Banach space X and A : X → X is an operator, possibly nonlinear and not related to a derivative or subdifferential, so that problem (7.2) is not necessarily the first-order condition of an optimization problem. This requires the development of an elegant theory initiated by Lax–Milgram–Lions and Stampacchia that has also led to important developments to the theory of monotonetype operators. Note furthermore that in the case where C = X (or a linear susbspace), the variational inequality (7.2) reduces to an equation.1 ⋆

7.2 Warm up: free boundary value problems for the Laplacian To provide a feeling for variational inequalities, we first start with a simple problem, related to Example 7.1.2 above and in particular the inequality find uo ∈ C : ⟨−Δuo , u − uo ⟩ ≥ 0,

for all u ∈ C,

(7.3)

1 For any z ∈ X one may choose the test function x ∈ X so that x − xo = ±z, and so that (7.2) yields ⟨A(xo ), z⟩ = 0 for all z ∈ X, which leads to the equation A(xo ) = 0.

7.2 Warm up: free boundary value problems for the Laplacian | 337

for C = {u ∈ W 1,2 (𝒟) : u = f0 , on 𝜕𝒟, u ≥ ϕ a. e. in 𝒟}, where ϕ is a given function (called the obstacle) and f0 is a given function specifying the boundary data. Clearly, the equality u = f0 on 𝜕𝒟 is to be understood in the sense of traces, i. e., that γ0 u = f0 (see Definition 1.5.16, see also [1]). Furthermore, if one may find a sufficiently smooth function f1 defined on the whole of 𝒟 such that γ0 f1 = f0 , setting ū = u − f1 , ϕ̄ = ϕ − f1 and f = Δf1 we see that problem (7.3) reduces to find ū o ∈ C0 , : ⟨−Δū o − f , ū − ū o ⟩ ≥ 0,

for all ū ∈ C0 ,

(7.4)

for C0 = {ū ∈ W01,2 (𝒟) : ū ≥ ϕ̄ a. e. in 𝒟}, and the above choice of f . Naturally, one has to be very careful on the properties of the extension function f1 (i. e., membership in the right function space) as well as to what sense is Δf1 defined. Problem (7.3) is called an obstacle problem and may be considered as modeling an elastic membrane whose shape is described by the function u without any force exerted on it, apart from the fact that it has to lie over an obstacle which is described by the function ϕ. Naturally, even though, no external forces are applied to the membrane, the underlying obstacle exerts some force on it and this may serve as a physical understanding of the connection between (7.3) and (7.4). Obstacle problems find interesting applications in other fields apart from mechanics as, for example, in decision theory, mathematical finance, etc. By the symmetry of the operator A = −Δ, both variational inequalities (7.3) and (7.4) can be understood as the first-order condition for some minimization problem, and in particular ˆ 1 min FD (u), FD (u) := |∇u|2 dx, (7.5) u∈C 2 𝒟 and min F0 (u), u∈C0

F0 (u) :=

1 2

ˆ 𝒟

|∇u|2 dx +

ˆ 𝒟

fudx,

(7.6)

respectively. For simplicity in this section, we may assume that the obstacle ϕ is a smooth function. By the connection of problem (7.3) (resp., (7.4)) with the minimization problem (7.5) (resp., (7.6)) using techniques from the calculus of variations, we may show some solvability results for these variational inequalities. Furthermore, one may show, concerning problem (7.3), that its solution satisfies −Δuo ≥ 0 with −Δuo = 0, −Δuo > 0

uo ≥ ϕ,

if uo > ϕ,

if uo = ϕ,

uo |𝜕𝒟 = f0 ,

(7.7)

338 | 7 Variational inequalities so that, upon defining the coincidence set 𝒞 := {x ∈ 𝒟 : u(x) = ϕ(x)},

the solution is in general a superharmonic function2 which satisfies the Laplace equation on the complement of 𝒞 . It is interesting to note that, apart from the fixed boundary 𝜕𝒟 where the solution is prescribed and equal to the known function f0 , the coincidence set defines another boundary (on which the solution coincides with the obstacle function ϕ) which is not known a priori but rather depends on the solution of the problem. For that reason, problems of this type are often called free boundary value problems. Note that (7.7) can be rephrased in the more compact complementarity form3 uo ≥ ϕ,

−Δuo ≥ 0,

(uo − ϕ)Δuo = 0,

uo |𝜕𝒟 = f0 .

Similarly, for problem (7.4) we may have an analogous specification for the solution with 0 replaced by f on the first two equations in (7.7) and uo |𝜕𝒟 = 0. The next proposition (see, e. g., [64] and references therein) collects some fundamental properties of the free boundary problem for the Laplacian. Proposition 7.2.1. Assume that 𝒟 as well as the functions ϕ : 𝒟 → ℝ and f0 : 𝜕𝒟 → ℝ are of class C 1 , while ϕ ≤ f0 on 𝜕𝒟. Then: (i) The variational inequality (7.3) admits a unique solution uo ∈ C. (ii) The solution uo also satisfies (7.7). Proof. (i) For this symmetric problem, the solvability of (7.3) follows from its connection with the variational problem (7.5) so we will use the direct method for the calculus of variations, essentially following the standard procedure in Theorem 6.4.2, which we briefly repeat here to refresh our memory. We sketch it in 3 steps: 1. By standard arguments, we select a minimizing sequence {un : n ∈ ℕ} ⊂ C. Since FD (un ) → m := inf{FD (u) : u ∈ C}, by the form of FD , we have that ‖∇un ‖L2 (𝒟) ≤ c for some c > 0 and n sufficiently large. Note that since we are not in W01,2 (𝒟), hence, ‖∇u‖L2 (𝒟) does not constitute an equivalent norm, we need to establish boundedness of ‖un ‖W 1,2 (𝒟) from the above estimate before we are allowed to proceed in the selection of a weakly convergent subsequence and passage to the limit. ̂ n : n ∈ ℕ} ⊂ To this end, we need to modify {un : n ∈ ℕ} to a new sequence {u ̂ := max{ϕ, ̂f }. Then, by W01,2 (𝒟). Let ̂f0 ∈ C 1 (𝒟), be an extension of f0 and define ϕ 0 ̂ ∈ W 1,2 (𝒟) for any n ∈ ̂ n := un − ϕ the assumptions imposed on f0 and ϕ, setting u 0 ℕ and, therefore, Poincaré’s inequality holds for any element of the new sequence, 2 i. e., assuming sufficient smoothness a function such that Δu ≤ 0. 3 If uo > ϕ, then (uo − ϕ)Δuo = 0 implies that Δuo = 0. If −Δuo > 0, then (uo − ϕ)Δuo = 0 implies that uo = ϕ.

7.2 Warm up: free boundary value problems for the Laplacian | 339

−1 ̂ 2 ̂ hence, ‖un − ϕ‖ L (𝒟) ≤ c𝒫 ‖∇un − ∇ϕ‖L2 (𝒟) . Then we see that by combining the triangle inequality and the Poincaré inequality

‖un ‖W 1,2 (𝒟) = ‖un ‖L2 (𝒟) + ‖∇un ‖L2 (𝒟) ̂ 2 ̂ 2 ≤ ‖u − ϕ‖ + ‖ϕ‖ ≤

n L (𝒟) L (𝒟) + ‖∇un ‖L2 (𝒟) −1 ̂ 2 ̂ c𝒫 ‖∇un − ∇ϕ‖ L (𝒟) + ‖ϕ‖L2 (𝒟) + ‖∇un ‖L2 (𝒟)

so that the quantity on the left-hand side is estimated by ‖∇un ‖L2 (𝒟) which is bounded. Therefore, ‖un ‖W 1,2 (𝒟) < c󸀠 for some c󸀠 > 0; hence, by reflexivity there exists a uo ∈ W 1,2 (𝒟) and a subsequence unk ⇀ uo in W 1,2 (𝒟). If uo ∈ C, then by the weak lower semicontinuity of FD we are done. 2. The fact that the limit uo ∈ C follows by the convexity of C and by recalling the fact that for convex sets the weak and the strong closures coincide (see Proposition 1.2.12). To check that C is closed consider a sequence {vn : n ∈ ℕ} ⊂ C, such that vn → v in W 1,2 (𝒟). We will show that v ∈ C. Indeed, since vn → v in W01,2 (𝒟), by c

the compact embedding W01,2 (𝒟) 󳨅→ L2 (𝒟) it follows that there exists a subsequence vnk → v in L2 (𝒟); hence, a further subsequence (denoted the same) such that vnk → v a. e. in 𝒟. Since vnk ∈ C it follows that vnk (x) ≥ ϕ(x), a. e. in 𝒟 and passing to the limit in this subsequence the inequality remains valid, so that v(x) ≥ ϕ(x), a. e. in 𝒟. Therefore, v ∈ C hence, C is closed. 3. Steps 1 and 2 complete the existence of a minimizer uo ∈ C, while derivation of the Euler–Lagrange equation (first-order condition) yields that uo solves (7.3). Uniqueness follows by a strict convexity argument, based on the observation that if two such solutions exist then their gradients coincide and by the Poincaré inequality, applied to their difference, so do they. (ii) We prove our claim in 3 steps. 1. We first prove that −Δuo ≥ 0 in 𝒟. Let ψ ∈ Cc∞ (𝒟) be a test function, such that ψ ≥ 0. Then, if uo ∈ C it is clear that v = uo +ψ ∈ C. Setting u = v in the variational inequality ´ (7.3) we obtain 𝒟 ∇uo (x)⋅∇ψ(x)dx ≥ 0 which is the weak form for the inequality −Δuo ≥ 0 (as a simple integration by parts argument, plus a density argument of Cc∞ (𝒟) in W01,2 (𝒟) shows). 2. We show next that −Δuo = 0 in {uo > ϕ} ∩ 𝒟. We claim that uo − ϕ is lower semicontinuous in 𝒟 (or equivalently by the continuity of ϕ, that uo is lower semicontinuous). For the time being, we accept this claim that will be proved in step 3. Then we can argue as follows: By the lower semicontinuity of uo − ϕ in 𝒟, we have that the set 𝒟+ := {x ∈ 𝒟 : uo (x) − ϕ(x) > 0} is open, so that for every x ∈ 𝒟+ we can find an r > 0 with the property B(x, r) ⊂⊂ 𝒟+ and consider a test function ψ ∈ Cc∞ (B(x, r)). Therefore, if ϵ > 0 is small enough, we have that both uo +ϵψ−ϕ > 0 and u−ϵψ−ϕ > 0. ´ That means v1 = uo + ϵψ ∈ C, so letting u = v1 in (7.4) we get 𝒟 ∇uo (x) ⋅ ∇ψ(x)dx ≥ 0. Moreover, by the same arguments v2 = uo − ϵψ ∈ C, so letting u = v2 into (7.4) we get ´ 𝒟 ∇uo (x) ⋅ ∇ψ(x)dx ≤ 0. Combining these two (along with the arbitrary choice of the

340 | 7 Variational inequalities ball B(x, r) ⊂⊂ {uo > ϕ}), we conclude that ˆ ∇uo (x) ⋅ ∇ψ(x)dx = 0, 𝒟

∀ ψ ∈ Cc∞ (𝒟 \ 𝒞 ),

which is the weak form of −Δuo = 0 on 𝒟 \ 𝒞 . 3. We now show that under the assumptions on the data of the problem uo is lower semicontinuous. To show this, we may argue as follows: 3(a). We have already shown in step 1 that −Δuo ≥ 0 in 𝒟 in the weak sense, so that the mapping ˆ 1 r 󳨃→ uo (z)dz =: w(r), u (z)dz =: |B(x, r)| B(x,r) o B(x,r) is decreasing on (0, dist(x, 𝜕𝒟)). This can be easily seen by using the function r vR (x) := R2−d [( ) R

2

−(d−2)

+

d−2 r d ( ) − ] 1B(0,R) , 2 R 2

r = |x|,

which is a C 1,1 function except at the origin, while a simple calculation in spherical d(d−2)ω coordinates yields ΔvR = |B(0,R)|d 1B(0,R) where ωd is the volume of the d-dimensional

d unit ball (|B(0, R)| = ωd Rd ). We may also note that dR vR (x) ≥ 0 for every x ∈ B(0, R); hence, if R1 < R2 it holds that vR1 (x) ≤ vR2 (x) for any x ∈ B(0, R2 ), so that ψ := vR2 − vR1 ≥ 0 is a suitable test function, in C 1,1 which vanishes outside B(0, R2 ) and such that

Δψ = d(d − 2)ωd (

1 1 1 − 1 ). |B(0, R2 )| B(0,R2 ) |B(0, R1 )| B(0,R1 )

(7.8)

Since −Δuo ≥ 0 in the weak sense, we have for the test function ψ = vR2 −vR1 that (using (7.8)) ˆ ˆ 0≤ (−Δuo )(x)ψ(x)dx = uo (x)(−Δψ)(x)dx = d(d − 2)ωd (w(R1 ) − w(R2 )), 𝒟

𝒟

ffl

i. e., the function R 󳨃→ w(R) = B(0,R) uo dx is decreasing. By appropriately shifting the ffl origin to any point x ∈ 𝒟 we have that the function R 󳨃→ B(x,R) uo (z)dz is decreasing. 3(b). By the monotonicity obtained in step 3(a), the function ū defined by ̄ u(x) := lim

R→0 B(x,R)

uo (z)dz,

∀ x ∈ 𝒟,

is well-defined, and again by the same arguments satisfies the lower bound, B(x,R)

̄ uo (z)dz ≤ u(x) := lim

R→0 B(x,R)

uo (z)dz,

∀ R > 0.

̄ Moreover, u(x) = uo (x) for any Lebesgue point of uo , so that ū = uo , a. e. in 𝒟.

(7.9)

7.3 The Lax–Milgram–Stampacchia theory | 341

3(c). We will show that, by construction, the function ū is lower semicontinuous hence, uo has a lower semicontinuous version. We will identify uo with ū so that it is pointwise defined at every point. Then (7.9) is a version of the mean value property for super harmonic functions. To check the lower semicontinuity of u,̄ consider any sequence {xn : n ∈ ℕ} ⊂ 𝒟, such that xn → x and note that for any fixed radius R > 0 it holds that ˆ 1 uo (z)dz = u (z)1B(xn ,R) (z)dz |B(xn , R)| 𝒟 o B(xn ,R) ˆ 1 = uo (z)1B(xn ,R) (z)dz, ωd Rd 𝒟 so noting that uo 1B(xn ,R) → uo 1B(x,R) a. e., and passing to the limit as n → ∞ using the Lebesgue dominated convergence theorem we get ˆ 1 lim uo (z)1B(xn ,R) (z)dz lim u (z)dz = n→∞ B(x ,R) o ωd Rd n→∞ 𝒟 n ˆ (7.10) 1 ̄ u (z)dz = u(x), u (z)1 (z)dz = = o o B(x,R) ωd Rd 𝒟 B(x,R) since for any choice of centers x, xn , it holds that |B(xn , R)| = |B(x, R)| = ωd Rd . Furthermore, applying (7.9) for every xn , taking the limit inferior and combining this with (7.10) we get that ̄ u(x) = lim

n→∞ B(x ,R) n

̄ n ), uo (z)dz ≤ lim inf u(x n→∞

which is the required lower semicontinuity result. The proof is complete.

7.3 The Lax–Milgram–Stampacchia theory We now present a first approach toward variational inequalities, which is essentially centered or inspired by results related to minimization of quadratic functionals over convex and closed subsets of Hilbert spaces, but importantly leads to results for variational inequality problems of a more general nature, not directly related to minimization problems. These considerations lead to a celebrated theory, pioneered by Lax, Milgram, Lions and Stampacchia (see, e. g., [32] or [73]). In this chapter, we will focus on the Hilbert space case. The more general problem of variational inequalities in Banach spaces will be treated later on in Chapter 9. Throughout this chapter, let X := H be a Hilbert space, with dual X ⋆ not necessarily identified with X. Let (⋅, ⋅) be the inner product in X, ‖ ⋅ ‖ its norm defined by ‖x‖2 = (x, x), and let us denote by ⟨⋅, ⋅⟩ the duality pairing between4 X ⋆ and X. Recall that in 4 If an example is required as motivation, consider X = W01,2 (𝒟) and X ⋆ = (W01,2 (𝒟))∗ = W −1,2 (𝒟).

342 | 7 Variational inequalities such cases, we may define a surjective isometry j : X → X ⋆ , by ⟨j(x1 ), x2 ⟩ = (x1 , x2 ), for every x1 , x2 ∈ X. The inverse of this map j−1 : X ⋆ → X is the map associated with the Riesz–Fréchet representation theorem (see Theorem 1.1.14). We recall further that the dual space X ⋆ can be turned into a Hilbert space with norm ‖x⋆ ‖2X ⋆ = (j−1 (x⋆ ), j−1 (x⋆ )) = ‖j−1 (x⋆ )‖2 . The Lax–Milgram–Stampacchia theory deals with variational inequalities in Hilbert space, involving a bilinear mapping a : X × X → ℝ, and in particular problems of the form: Given a closed convex subset C of the Hilbert space X and x⋆ ∈ X ⋆ , find xo ∈ C such that a(xo , x − xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

If a was symmetric (i. e., such that a(x, z) = a(z, x) for every x, z ∈ X), we may associate the bilinear form a with the Gâteaux derivative of the quadratic functional F : X → ℝ, defined by F(x) = 21 a(x, x) (see Example 2.1.8), connect the variational inequality with a minimization problem over C, and by employing the general theory of convex minimization (Theorem 2.4.5) obtain well posedness results. However, the Lax–Milgram– Stampacchia theory goes beyond that, and allows us to treat variational inequalities even in the nonsymmetric case, which is not directly related to minimization problems. Before presenting the Lax–Milgram–Stampacchia theory, we need to introduce the notion of continuous and coercive bilinear forms. Definition 7.3.1 (Continuous and coercive bilinear form). Let X = H be a Hilbert space. The bilinear form a : X × X → ℝ is called: (i) continuous, if there exists a constant c > 0 such that |a(x1 , x2 )| ≤ c ‖x1 ‖ ‖x2 ‖, for every x1 , x2 ∈ X. (ii) coercive, if there exists a constant cE > 0 such that a(x, x) ≥ cE ‖x‖2 , for every x ∈ X. Remark 7.3.2. If a continuous and coercive bilinear form a is also symmetric5 it can be used to define a new inner product on X by ⟨x1 , x2 ⟩o := a(x1 , x2 ) for every x1 , x2 ∈ X. We may then renorm X using the norm defined by this new inner product (see [28]). A bilinear form which is continuous can be used to define a linear and bounded operator. Proposition 7.3.3. Let X be a Hilbert space and a : X ×X → ℝ a continuous and coercive bilinear form. The operator A : X → X ⋆ , defined by the relation a(x, z) = ⟨Ax, z⟩ for every z ∈ X, is linear, bounded and satisfies the (strong) monotonicity property ⟨Ax1 − Ax2 , x1 − x2 ⟩ ≥ cE ‖x1 − x2 ‖2 . 5 i. e., a(x1 , x2 ) = a(x2 , x1 ) for every x1 , x2 ∈ X.

(7.11)

7.3 The Lax–Milgram–Stampacchia theory | 343

Proof. By the continuity of the bilinear form a, for fixed z ∈ X, by the Riesz representation, there exists a unique xo ∈ X ⋆ such that a(x, z) = ⟨xo , z⟩. We will denote xo = Ax and this will define an operator A : X → X ⋆ . The operator A is bounded, since 󵄨 󵄨 󵄨 󵄨󵄨 󵄨󵄨⟨Ax, z⟩󵄨󵄨󵄨 = 󵄨󵄨󵄨a(x, z)󵄨󵄨󵄨 ≤ c ‖x‖ ‖z‖,

∀ x ∈ X, z ∈ X ⋆ ,

which leads to ‖A‖ℒ(X,X ⋆ ) ≤ c. The monotonicity property (7.11), follows by the linearity of the operator A and the coercivity property of the bilinear form. We are now ready to present the celebrated Lax–Milgram–Stampacchia theory of variational inequalities (see, e. g., [42] or [73]). Theorem 7.3.4 (Lax–Milgram–Stampacchia I). Let C ⊂ X be a closed and convex subset of the Hilbert space X, a : X × X → ℝ a continuous and coercive bilinear form and L : X → ℝ a continuous linear form, or equivalently, an x⋆ ∈ X ⋆ such that L(x) = ⟨x⋆ , x⟩ for every x ∈ X. (i) There exists a unique xo ∈ C such that a(xo , x − xo ) ≥ L(x − xo ),

∀ x ∈ C,

(7.12)

or equivalently, using Proposition 7.3.3 ⟨Axo , x − xo ⟩ ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

(7.13)

(ii) If furthermore, a is symmetric then xo is the unique minimizer on C of the functional F : X → ℝ, defined by 1 F(x) = a(x, x) − ⟨x⋆ , x⟩. 2 Proof. (i) Let PC : X → C be the projection operator from X to the closed convex C ⊂ X, defined by PC (x) = arg minz∈C ‖x − z‖. This map is nonexpansive (pseudo-contraction) and is characterized by the variational inequality (x − PC (x), z − PC (x)) ≤ 0 for every z ∈ C (see Theorem 2.5.2), where (⋅, ⋅) is the inner product with which X is endowed.6 Note that (7.13) can be expressed as ⟨x⋆ − Axo , x−xo ⟩ ≤ 0 for every x ∈ C. Fix a ρ > 0 (to be determined shortly) and upon multiplying the variational inequality with ρ and observing that trivially ρ(x⋆ − Axo ) = ρ(x⋆ − Axo ) + xo − xo , we obtain the equivalent form ⟨ρ(x⋆ − Axo ) + xo − xo , x − xo ⟩ ≤ 0,

∀ x ∈ C,

which using the map j : X → X ⋆ can be expressed as (ρj−1 (x⋆ − Axo ) + xo − xo , x − xo ) ≤ 0,

∀ x ∈ C.

6 Note that here we have not necessarily identified X with its dual X ⋆ .

(7.14)

344 | 7 Variational inequalities However, recalling the characterization of projection in terms of variational inequalities, it can be seen that (7.14) is equivalent to xo = PC (ρj−1 (x⋆ − Axo ) + xo ),

(7.15)

for some ρ > 0. It thus suffices to show that (7.15) holds for some ρ > 0. We define the family of maps Tρ : C → C by Tρ (x) = PC (ρ(x⋆ − Ax) + x) for every x ∈ C, and we claim that for the proper choice of ρ the map Tρ is a strict contraction, so that by the Banach fixed-point theorem (see Theorem 3.1.2) the claim follows. To this end, consider x1 , x2 ∈ X and observe that 󵄩󵄩 󵄩 󵄩󵄩Tρ (x1 ) − Tρ (x2 )󵄩󵄩󵄩 󵄩 󵄩 = 󵄩󵄩󵄩PC (ρj−1 (x⋆ − Ax1 ) + x1 ) − PC (ρj−1 (x⋆ − Ax2 ) + x2 )󵄩󵄩󵄩 󵄩 󵄩 ≤ 󵄩󵄩󵄩ρj−1 (x⋆ − Ax1 ) + x1 − ρj−1 (x⋆ − Ax2 ) − x2 󵄩󵄩󵄩 󵄩 󵄩 = 󵄩󵄩󵄩x1 − x2 − ρj−1 (Ax1 − Ax2 )󵄩󵄩󵄩,

(7.16)

where we used the properties of projection operators and the fact that X is a Hilbert space. Moreover, 󵄩󵄩 󵄩2 −1 󵄩󵄩x1 − x2 − ρj (Ax1 − Ax2 )󵄩󵄩󵄩 = (x1 − x2 − ρj−1 (Ax1 − Ax2 ), x1 − x2 − ρj−1 (Ax1 − Ax2 )) 󵄩 󵄩2 = ‖x1 − x2 ‖2 + ρ2 󵄩󵄩󵄩j−1 (Ax1 − Ax2 )󵄩󵄩󵄩 − 2ρ(x1 − x2 , j−1 (Ax1 − Ax2 )).

(7.17)

By the linearity of A, setting z = x1 − x2 to simplify notation, we see that 󵄩󵄩 −1 󵄩2 2 2 2 󵄩󵄩j (Ax1 − Ax2 )󵄩󵄩󵄩 = ‖Ax1 − Ax2 ‖X ⋆ = ‖Az‖X ⋆ ≤ c ‖z‖ , since the operator A is bounded, and (x1 − x2 , j−1 (Ax1 − Ax2 )) = ⟨Az, z⟩ = a(z, z) ≥ cE ‖z‖2 , where we used the definition of the operator A and the coercivity of the bilinear form a (Assumption 7.3.1(ii)). Therefore, (7.17) can be estimated as 󵄩󵄩 󵄩2 −1 2 2 󵄩󵄩x1 − x2 − ρj (Ax1 − Ax2 )󵄩󵄩󵄩 ≤ (1 + cρ − 2ρcE )‖x1 − x2 ‖ , and choosing ρ > 0 such that 0 < ϱ2 := 1 + c2 ρ2 − 2ρcE < 1 we have 󵄩󵄩 󵄩 −1 󵄩󵄩x1 − x2 − ρj (Ax1 − Ax2 )󵄩󵄩󵄩 < ϱ‖x1 − x2 ‖, which when combined with (7.16) leads to 󵄩󵄩 󵄩 󵄩󵄩Tρ (x1 ) − Tρ (x2 )󵄩󵄩󵄩 < ϱ‖x1 − x2 ‖, therefore, Tρ is a strict contraction when ρ ∈ (0, 2cc2E ).

7.3 The Lax–Milgram–Stampacchia theory | 345

(ii) Even though, this follows directly by the abstract results of Theorem 2.4.5, upon noting that in this case the variational inequality coincides with the first-order condition for the minimization problem in question, we provide an elementary proof, which highlights the importance of symmetry.7 By the definition of F, we see that for all x ∈ C, F(x) = F(xo + (x − xo )) 1 = a(xo + (x − xo ), xo + (x − xo )) − ⟨x⋆ , xo + (x − xo )⟩ 2 1 1 = a(xo , xo ) − ⟨x⋆ , xo ⟩ + a(x − xo , xo ) + a(x − xo , x − xo ) − ⟨x⋆ , x − xo ⟩ 2 2 1 = F(xo ) + a(x − xo , x − xo ) + a(x − xo , xo ) − ⟨x⋆ , x − xo ⟩. 2

(7.18)

By coercivity of a it follows that a(x−xo , x−xo ) ≥ 0. Now, let xo be the unique solution of (7.12) (equivalently (7.13)). The symmetry of a implies that8 a(x−xo , xo )−⟨x⋆ , x−xo ⟩ ≥ 0, for any x ∈ C, therefore, (7.18) implies that F(x) ≥ F(xo ) for every x ∈ C, so that xo is the minimizer of F in C. In the special case where C = X we recover the well known Lax–Milgram theorem, which forms the basis for the study of linear PDEs. Theorem 7.3.5 (Lax–Milgram). Let X be a Hilbert space and a : X × X → ℝ a continuous and coercive bilinear form. Then for any x⋆ ∈ X ⋆ , there exists a unique solution to the equation a(x⋆ , x) = ⟨x⋆ , x⟩,

∀ x ∈ X.

Proof. We will use the Lax–Milgram–Stampacchia Theorem 7.3.4 in the special case where C = X. Let xo ∈ X be the unique solution of (7.13) and consider any z ∈ X. Then, since we now work on the whole of the vector space and not only on a closed convex subset, x1 = xo + z ∈ X and x2 = xo − z ∈ X. Applying (7.13) with x = x1 we see that a(xo , z) ≥ ⟨x⋆ , z⟩. Applying (7.13) once more with x = x2 , we see that we see that a(xo , z) ≤ ⟨x⋆ , z⟩. Combining these two inequalities, we obtain the desired result. Furthermore, in the case of symmetry this solution has minimizing properties. The following equivalent form of the variational inequality (7.13) is often useful. Theorem 7.3.6 (Minty). The variational inequality find xo ∈ C : a(xo , x − xo ) = ⟨Axo , x − xo ⟩ ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C,

(7.19)

is equivalent to find xo ∈ C : a(x, x − xo ) = ⟨Ax, x − xo ⟩ ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

7 A nice alternative way to look at the symmetric case (see [28]) is to use Remark 7.3.2. 8 where we have expressed the linear form L in terms of x⋆ .

(7.20)

346 | 7 Variational inequalities Proof. Suppose xo ∈ C solves (7.19). Then, for any x ∈ C a(x, x − xo ) = a(x − xo , x − xo ) + a(xo , x − xo ) ≥ a(xo , x − xo ) ≥ ⟨x⋆ , x − xo ⟩, where we used first the linearity of a with respect to the first argument and then the coercivity to deduce that a(x − xo , x − xo ) > 0. Therefore, a solution of (7.19) is a solution of (7.20). Conversely, suppose now that xo ∈ C solves (7.20). For any x ∈ C and ϵ ∈ (0, 1), define z = xo + ϵ(x − xo ) ∈ C and apply (7.20) for the pair (xo , z) ∈ C × C. This yields a(xo + ϵ(x − xo ), ϵ(x − xo )) ≥ ⟨x⋆ , ϵ(x − xo )⟩,

∀x ∈ C

which, dividing by ϵ > 0 and taking the limit as ϵ → 0+ leads to (7.19). Note that Minty’s reformulation (7.20) differs from the original formulation of the variational inequality (7.13), in that while in the original problem (7.13) the operator A acts on the solution xo of the inequality, in the reformulated version (7.20) it acts on any element x ∈ C. This is important, since in (7.20) we may choose such elements of C, so that Ax has desirable properties thus leading to convenient schemes for the study of qualitative properties of the solution of the variational inequality or even numerical schemes. The reformulation by Minty allows us to obtain a saddle point reformulation of variational inequalities. Proposition 7.3.7. Consider the map L : X × X → ℝ defined by L(x1 , x2 ) = a(x1 , x1 − x2 ) − ⟨x⋆ , x1 − x2 ⟩,

∀ x1 , x2 ∈ X.

Then xo ∈ C is a solution of the variational inequality a(xo , x − xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C,

(7.21)

if and only if (xo , xo ) is a saddle point for L over C × C in the sense that L(xo , x) ≤ L(xo , xo ) ≤ L(z, xo ),

∀ x, z ∈ C.

(7.22)

Proof. Suppose that xo ∈ C is a solution of (7.21). Then, setting x1 = xo and x2 = x we see that (7.21) implies that L(xo , x) ≤ 0 for every x ∈ C. On the other hand, if xo ∈ C is a solution of (7.21), by Minty’s trick we also have that a(x, x − xo ) ≥ ⟨x⋆ , x − xo ⟩ for every x ∈ C, which, setting x1 = x and x2 = xo leads to L(x, xo ) ≥ 0 for every x ∈ C. Since L(xo , xo ) = 0, (7.22) is immediate. For the converse assume that 0 = L(xo , xo ) ≤ L(x, xo ) for every x ∈ C. By the definition of L, this is equivalent to 0 ≤ a(x, x − xo ) − ⟨x⋆ , x − xo ⟩ for every x ∈ C which is Minty’s form for (7.21). Similarly, if L(xo , x) ≤ L(xo , xo ) = 0 for every x ∈ C, by the definition of L we obtain (7.21).

7.3 The Lax–Milgram–Stampacchia theory | 347

As can be seen from the proof of the Lions–Stampacchia Theorem 7.3.4 linearity of the operator A : X → X ⋆ did not play a major role in the actual existence proof, which was based on the Banach contraction theorem. We may then extend the validity of this theorem to a class of nonlinear operators, satisfying a similar coercivity condition. Assumption 7.3.8. The (possibly nonlinear) operator A : X → X ⋆ satisfies the following properties: (i) A is Lipschitz, i. e., there exists c > 0, such that ‖A(x1 ) − A(x2 )‖X ⋆ ≤ c‖x1 − x2 ‖X for every x1 , x2 ∈ X, (ii) A is coercive (strongly monotone), i. e., there exists cE > 0 such that ⟨A(x1 ) − A(x2 ), x1 − x2 ⟩ ≥ cE ‖x1 − x2 ‖2 ,

∀ x1 , x2 ∈ X.

We then have the following. Theorem 7.3.9 (Lions–Stampacchia II). Let C ⊂ X be a closed convex set. If the (nonlinear) operator A : X → X ⋆ satisfies Assumption 7.3.8 then the variational inequality find xo ∈ C : ⟨A(xo ), x − xo ⟩ ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C,

(7.23)

admits a unique solution for every x⋆ ∈ X ⋆ . Moreover, the solution map x⋆ 󳨃→ xo is Lipschitz with constant c1 . E

Proof. The proof follows very closely that of Theorem 7.3.4. Following exactly the same steps, we observe that a solution of (7.23) is a fixed point of the map Tρ : C → C defined by Tρ (x) = PC (ρ(x⋆ − A(x)) + x) for every x ∈ C, for some ρ > 0, while (7.16) and (7.17) are formally the same, so 󵄩󵄩 󵄩2 󵄩2 2 󵄩 󵄩󵄩Tρ (x1 ) − Tρ (x2 )󵄩󵄩󵄩 ≤ ‖x1 − x2 ‖ + 󵄩󵄩󵄩A(x1 ) − A(x2 )󵄩󵄩󵄩 − 2ρ⟨A(x1 ) − A(x2 ), x1 − x2 ⟩ ≤ (1 + ρ2 c2 − 2ρcE ) ‖x1 − x2 ‖2 , where we used Lipschitz continuity for the second term and coercivity (strong monotonicity) for the third term. From the above estimate, we can find ρ > 0 such that Tρ is a contraction and the proof proceeds as in Theorem 7.3.4. Example 7.3.10. Minty’s trick can be reformulated for equations. Consider the (possibly nonlinear) continuous operator A : X → X ⋆ , and the variational inequality ⟨A(z) − x⋆ , z − x⟩ ≥ 0 for every z ∈ X. Then A(x) = x⋆ in X ⋆ . Conversely, if A satisfies the monotonicity property ⟨A(x1 ) − A(x2 ), x1 − x2 ⟩ ≥ 0 for every x1 , x2 ∈ X and A(x) = x⋆ in X ⋆ , then ⟨A(z) − x⋆ , z − x⟩ ≥ 0 for every z ∈ X. To show the first direction set z = z󸀠 + tx ∈ X for arbitrary z󸀠 ∈ X and t > 0, substitute into the variational inequality and pass to the limit as t → 0+ using continuity to obtain ⟨A(x), z󸀠 ⟩ ≥ 0. Repeat the same with −z󸀠 to obtain ⟨A(x), z󸀠 ⟩ ≤ 0, so that ⟨A(x), z󸀠 ⟩ = 0 for every z󸀠 ∈ X and the result follows. For the reverse direction, by monotonicity ⟨A(z) − A(x), z − x⟩ ≥ 0 for every z ∈ X, so since A(x) = x⋆ we get the variational inequality. ◁

348 | 7 Variational inequalities There are many examples where the abstract setting of the Lax–Milgram– Stampacchia theory can be applied. Its application to elliptic PDEs, variational inequalities and free boundary value problems will be studied in Section 7.6.

7.4 Variational inequalities of the second kind An interesting class of variational inequalities are variational inequalities of the form, Given a closed convex subset C of a Hilbert space X = H, a proper lower semicontinuous convex function φ : C → ℝ ∪ {+∞} and an element x⋆ ∈ X ⋆ = H ⋆ , find xo ∈ C : a(xo , x − xo ) + φ(x) − φ(xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

Variational inequalities of this general type are called variational inequalities of the second kind. In the special case where φ = 0, we obtain the familiar case of a variational inequality,9 treated in Section 7.3 in terms of the Lax–Milgram–Stampacchia theory. In the case where the bilinear form a is symmetric, this variational inequality can be interpreted as the first-order condition for the minimization on C of a convex functional, therefore, the existence of a solution to the variational inequality can be obtained by the standard theory for convex optimization presented in Section 2.4. However, this problem is well posed for the general case where a is a continuous and coercive bilinear form, not necessarily symmetric, by an appropriate extension of the Lax–Milgram–Stampacchia Theorem 7.3.4 (see e. g. [32]). Theorem 7.4.1. Let X be a Hilbert space, C ⊂ X closed and convex, a : X × X → ℝ a continuous and coercive bilinear form and φ : C → ℝ ∪ {+∞} a proper convex and lower semicontinuous function. (i) For any x⋆ ∈ X ⋆ there exists a unique xo ∈ C such that a(xo , x − xo ) + φ(x) − φ(xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

(7.24)

(ii) If furthermore, a is symmetric, then xo is the minimizer in C of the convex functional F : X → ℝ ∪ {+∞} defined by 1 F(x) = a(x, x) + φ(x) − ⟨x⋆ , x⟩, 2

∀ x ∈ X.

Proof. As X ⋆ is not necessarily identified with X, the setting is exactly the same as in Section 7.3. (i) Define the function φC : X → ℝ ∪ {+∞} by φC (x) = φ(x)(1 + IC (x)) which is equal to φ for any x ∈ C and equal to +∞ for any x ∈ ̸ C. It can be seen that the variational 9 sometimes called variational inequality of the first kind.

7.4 Variational inequalities of the second kind | 349

inequality (7.24) is related to the following formulation: Find xo ∈ C : a(xo , x − xo ) + φC (x) − φC (xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ X,

where now we assume that x takes values over the whole Hilbert space X rather than simply on C (as is the original formulation of the problem). We further use the definition of the operator A to express the variational inequality as: Find xo ∈ C : ⟨Axo , x − xo ⟩ + φC (x) − φC (xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ X,

which using the duality map j : X → X ⋆ is further expressed as Find xo ∈ C : (j−1 (Axo − x⋆ ), x − xo ) + φC (x) − φC (xo ) ≥ 0,

∀ x ∈ X.

We fix a ρ > 0 (to be determined shortly), multiply the variational inequality with ρ and observing the trivial fact that ρj−1 (Axo − x⋆ ) = xo − (xo − ρj−1 (Axo − x⋆ )) we obtain the equivalent form for the variational inequality as (xo − (xo − ρj−1 (Axo − x⋆ )), x − xo ) + ρφC (x) − ρφC (xo ) ≥ 0,

∀ x ∈ X.

(7.25)

The function ρφC : X → ℝ∪{+∞} is convex, proper and lower semicontinuous, so that the proximity operator proxρφC is well-defined. Recall (see Proposition 4.7.2(i), and in particular inequality (4.31) appropriately modified for the case where X ⋆ is not identified with X), that if zo = proxρφC x then, zo satisfies the variational inequality (zo − x, z − zo ) + ρφC (z) − ρφC (zo ) ≥ 0,

∀ z ∈ X.

(7.26)

Comparing (7.26) with (7.25) (in particular setting z → x, zo → xo , x → xo −ρj−1 (Axo −x⋆ ) in (7.26)) we see that the solution of the variational inequality should be such that xo = proxρφC (xo − ρj−1 (Axo − x⋆ )).

(7.27)

Note that if this is true, then we automatically have that xo ∈ C by the definition of the function10 φC . It thus suffices to show that (7.27) holds for some ρ > 0. We therefore, consider the family of maps Tρ : C → C defined by Tρ (x) := proxρφC (x − ρj−1 (Ax − x⋆ )),

for every x ∈ C.

We will show that there exists ρ > 0 such that Tρ has a fixed point, by using Banach’s contraction theorem, so that by (7.27), this fixed point is the solution we seek. To show that Tρ is a strict contraction for the proper choice of ρ > 0 we need to use the fact that 10 which is infinite if its argument is not in C and by the fact that proxρφC minimizes a perturbation of this function.

350 | 7 Variational inequalities the proximity operator proxρφC associated with the convex proper and lower semicontinuous function ρφC is a nonexpansive operator (see Proposition 4.7.2(ii)). From this point onward, the proof follows almost verbatim the steps of the proof of Theorem 7.3.4 by simply replacing the projection operator PC by the proximity operator proxρφC . (ii) Suppose that a is symmetric and xo ∈ C satisfies (7.24). Then, for any x ∈ C we see that 1 1 F(x) − F(xo ) = a(x, x) − a(xo , xo ) + φ(x) − φ(xo ) − ⟨x⋆ , x − xo ⟩ 2 2 1 = a(xo , x − xo ) + a(xo − x, xo − x) + φ(x) − φ(xo ) − ⟨x⋆ , x − xo ⟩ ≥ 0, 2 where we have explicitly used the symmetry property for a, the observation that by coercivity a(xo −x, xo −x) ≥ 0 and the fact that xo ∈ C satisfies (7.24). Hence, F(x) ≥ F(xo ) for every x ∈ C, therefore, xo is the minimizer of F in C. For the converse, assume that a is symmetric and xo ∈ C is the minimizer of F on C. By the convexity of C for any x ∈ C, and any t ∈ (0, 1), we have that z = (1 − t)xo + tx ∈ C so that since xo is the minimizer of F it holds that F(xo ) ≤ F(z). Using the definition of F and the fact that a is bilinear and symmetric, this yields 1 2 t a(xo − x, xo − x) + t a(xo , x − xo ) + φ((1 − t)xo + tx) − φ(xo ) 2 ≥ t ⟨x⋆ , x − xo ⟩.

(7.28)

By the convexity of φ, we have that φ((1 − t)xo + tx) − φ(xo ) ≤ tφ(x) − tφ(xo ), which when combined with (7.28) yields the inequality 1 2 t a(xo − x, xo − x) + t a(xo , x − xo ) + t(φ(x) − φ(xo )) ≥ t ⟨x⋆ , x − xo ⟩. 2 Dividing by t and passing to the limit as t → 0+ we obtain (7.24). A Minty-type equivalent reformulation (similar to the one presented in Theorem 7.3.6 for variational inequalities of the first type) is also valid for variational inequalities of the second type. Theorem 7.4.2 (Minty). The variational inequality (7.24) is equivalent to solving for xo ∈ C such that a(x, x − xo ) + φ(x) − φ(xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C,

or in terms of the operator A as ⟨Ax, x − xo ⟩ + φ(x) − φ(xo ) ≥ ⟨x⋆ , x − xo ⟩,

∀ x ∈ C.

Proof. The proof follows closely the proof of Theorem 7.3.6. For the converse, we need to take into account the convexity of the function φ. The details are left to the reader.

7.5 Approximation methods and numerical techniques | 351

7.5 Approximation methods and numerical techniques 7.5.1 The penalization method The penalization method is a useful method when dealing with variational inequalities. The essence of the method is trying to introduce the constraints involved in the variational inequality into the operator defining the variational inequality and then reduce the problem to one which considers the whole space X rather than the constraint set C. To fix ideas let X be a Hilbert space, C ⊂ X a nonempty closed convex set, a : X × X → ℝ a bilinear continuous and coercive form and consider the variational inequality xo ∈ C : a(xo , x − xo ) ≥ ⟨x⋆ , x − xo ⟩,

(7.29)

∀ x ∈ C.

We next introduce a convex function φ : X → [0, ∞] with the property that φ(x) = 0 if and only if x ∈ C. An example of such a function is the indicator function of C, i. e., φ = IC . However, note that while this can be used for theoretical arguments of existence, for practical purposes (e. g., numerical treatment) it may not be such a good choice. Using the function φ, for any ϵ > 0 we may define a family of variational inequalities of the form 1 1 xo,ϵ ∈ X : a(xo,ϵ , x − xo,ϵ ) + φ(x) − φ(xo,ϵ ) ≥ ⟨x⋆ , x − xo,ϵ ⟩, ϵ ϵ

x ∈ X,

(7.30)

or 1 xo,ϵ ∈ X : a(xo,ϵ , x) + ⟨β(xo,ϵ ), x⟩ = ⟨x⋆ , x⟩, ϵ

x ∈ X,

(7.31)

for a suitable penalty function β, which must satisfy a monotonicity property. The family of variational inequalities (7.30) or (7.31) is called the penalized version of (7.29), and the function φ is called the penalty function. Note the important difference between (7.29) and (7.30) (or (7.31)), which consists in that the former is solved over the closed convex subset C (modelling the constraints) whereas the latter is solved over the whole space X, turning it into an equation rather than an inequality. The solution is however effectively constrained in C, through the introduction of the penalty terms in (7.30). The unconstrained problem is definitely easier to handle than the original constrained version. The important contribution of the penalization method is the fact that it can be shown that as ϵ → 0 the family of solutions {xo,ϵ : ϵ > 0}, of (7.29) has a limit, which in fact is the solution xo of the original problem (7.29). The penalty method may be used for theoretical purposes (e. g., showing existence or regularity of solutions) or for numerical approximation. Before providing the convergence result for the penalty method, we provide some examples. It is worth noting that the choice of penalty function may not be unique, so that one has the liberty of choosing the particular penalty function that better suits the problem at hand.

352 | 7 Variational inequalities Example 7.5.1. A possible penalty term may be β(x) = ϵ1 (x − PC (x)) where PC is the projection operator to the closed convex set C ⊂ X. Note the monotonicity property of the penalty term, i. e., that ⟨β(x1 ) − β(x2 ), x1 − x2 ⟩ ≥ 0 for every x1 , x2 ∈ X which in this example follows by the properties of the projection operator. ◁ Example 7.5.2. Let 𝒟 ⊂ ℝd a bounded and smooth domain, X = W01,2 (𝒟), and identify elements of X with functions u : 𝒟 → ℝ, and elements of X ⋆ = W −1,2 (𝒟) with functions f : 𝒟 → ℝ. Consider the bilinear form a related to the elliptic operator A = −div(A∇) + a ⋅ ∇ + a0 I, where A = (aij )di,j=1 , a = (a1 , . . . , ad ) with aij , ai , a0 ∈ L∞ (𝒟), i, j = 1, . . . , d given functions, and the closed convex sets C1 = {u ∈ W01,2 (𝒟) : u ≥ ψ a. e.} and C2 = {u ∈ W01,2 (𝒟) : u ≤ ψ a. e.} for given obstacle ψ. Then one may check by direct substitution in the inequality defining the projection that PC1 (u) = sup{u, ψ}; hence, u − PC1 (u) = −(ψ − u)+ , whereas PC2 (u) = inf{u, ψ} hence, u − PC2 (u) = (u − ψ)+ . This leads to the possible penalization schemes for the variational inequalities find u ∈ C1 : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩, ∀ v ∈ C1 , 1 ⟨Auϵ , v⟩ − ⟨(ψ − uϵ )+ , v⟩ = ⟨f , v⟩, ∀v ∈ X, (Pϵ ) ϵ

(7.32)

find u ∈ C2 : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩, ∀ v ∈ C2 , 1 ⟨Auϵ , v⟩ + ⟨(uϵ − ψ)+ , v⟩ = ⟨f , v⟩, ∀v ∈ X, (Pϵ󸀠 ) ϵ

(7.33)

and

in the limit ϵ → 0+ . Indeed, one may informally see that in this limit (Pϵ ) will make sense if (ψ − u)+ = 0 so that u ≥ ψ while (Pϵ󸀠 ) will make sense if (u − ψ)+ = 0 so that u ≤ ψ. Smooth approximations of the above functions are also possible. ◁ The following proposition (see, e. g., [23] or [106]) proves the convergence of such approximation schemes. Proposition 7.5.3. Assume that the bilinear form a : X × X → R is continuous and coercive and that the penalty function φ : X → [0, ∞] is such that φ(x) = 0 if and only if x ∈ C. Let xo,ϵ be the solution of problem (7.30) and xo the solution of problem (7.29). Then limϵ→0 xo,ϵ = xo in X and limϵ→0 ϵ1 φ(xo,ϵ ) = 0. Proof. For each ϵ > 0 fixed, the penalized problem (7.30) is a variational inequality of the second kind, and the existence of a unique solution xo,ϵ is guaranteed, e. g., by Theorem 7.4.1, considering the case C = X. Alternatively, one could bypass this result and show existence directly by using finite dimensional approximations, Brouwer’s fixed-point theorem and passage to the limit, thus turning the current proposition into a full existence result (see Example 7.5.4). In order to pass to the limit as ϵ → 0+ , we need some a priori estimates on the solution. Since for any x ∈ X it holds that 1 φ(x) ≥ 0 with equality if and only if x ∈ C, choosing x ∈ C as test element in (7.30), ϵ

7.5 Approximation methods and numerical techniques | 353

the contribution of ϵ1 φ(x) vanishes and we obtain upon rearrangement 1 a(xo,ϵ , xo,ϵ ) + φ(xo,ϵ ) ≤ a(xo,ϵ , x) − ⟨x⋆ , x − xo,ϵ ⟩, ϵ

∀ x ∈ C.

(7.34)

Since ϵ1 φ(xo,ϵ ) ≥ 0 and a is coercive, we estimate the left-hand side of (7.34) as 1 cE ‖xo,ϵ ‖2 ≤ a(xo,ϵ , xo,ϵ ) ≤ a(xo,ϵ , xo,ϵ ) + φ(xo,ϵ ). ϵ On the other hand, by continuity the right-hand side of (7.34) is estimated by 󵄨 󵄨 a(xo,ϵ , x) − ⟨x⋆ , x − xo,ϵ ) ≤ 󵄨󵄨󵄨a(xo,ϵ , x) − ⟨x⋆ , x − xo,ϵ )󵄨󵄨󵄨 󵄩 󵄩 ≤ c‖xo,ϵ ‖ ‖x‖ + 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ (‖x‖ + ‖xo,ϵ ‖) ≤ c1 ‖xo,ϵ ‖ + c2 , for some finite constants c1 , c2 related to ‖x‖ and ‖x⋆ ‖X ⋆ , but independent of ϵ. Combining the above estimates with (7.34), we obtain the a priori estimate cE ‖xo,ϵ ‖2 ≤ c1 ‖xo,ϵ ‖ + c2 ,

∀ ϵ > 0,

which leads to a uniform bound ‖xo,ϵ ‖ ≤ c󸀠 , for all ϵ > 0. That implies the existence of a subsequence of {xo,ϵ : ϵ > 0} (denoted the same for simplicity) and an x̄ ∈ X such that xo,ϵ ⇀ x̄ as ϵ → 0+ . We will show that this convergence is in fact strong, but before doing so we first need to consider the behavior of the penalty term in the limit. For that, consider (7.34) again but now estimate the left-hand side, since a(xo,ϵ , xo,ϵ ) ≥ 0, as 1 1 φ(xo,ϵ ) ≤ a(xo,ϵ , xo,ϵ ) + φ(xo,ϵ ), ϵ ϵ and by a similar estimation of the right-hand side, we obtain 1 φ(xo,ϵ ) ≤ c1 ‖xo,ϵ ‖ + c2 ≤ c3 , ϵ

∀ ϵ > 0,

where c3 is independent of ϵ. For the last inequality, we used the fact that ‖xo,ϵ ‖ is uniformly bounded (which was our previous estimate). This implies upon multiplying with ϵ, that φ(xo,ϵ ) ≤ c3 ϵ,

∀ ϵ > 0,

and since φ is a positive function, passing to the limit (along the chosen subsequence) as ϵ → 0+ and using the weak lower semicontinuity of φ we conclude that φ(x)̄ = 0 therefore x̄ ∈ C. By the positivity of φ, (7.34) implies that a(xo,ϵ , xo,ϵ ) ≤ a(xo,ϵ , x) − ⟨x⋆ , x − xo,ϵ ),

∀ x ∈ C, ∀ ϵ > 0,

354 | 7 Variational inequalities so taking the limit superior as ϵ → 0+ along the chosen subsequence and using the fact that xo,ϵ ⇀ x̄ we see that ̄ lim sup a(xo,ϵ , xo,ϵ ) ≤ a(x,̄ x) − ⟨x⋆ , x − x⟩, ϵ→0+

∀ x ∈ C.

Note that the weak convergence for xo,ϵ does not guarantee the existence of limit for a(xo,ϵ , xo,ϵ ), and this is the reason for taking the limit superior. By the coercivity of a it holds that a(xo,ϵ − x,̄ xo,ϵ − x)̄ ≥ 0 for any ϵ > 0 which when rearranged using the fact that a is bilinear yields a(xo,ϵ , x)̄ + a(x,̄ xo,ϵ ) − a(x,̄ x)̄ ≤ a(xo,ϵ , xo,ϵ ),

∀ ϵ > 0,

which upon taking the limit inferior as ϵ → 0+ along the chosen subsequence and using the fact that xo,ϵ ⇀ x̄ to handle the linear terms leads to a(x,̄ x)̄ + a(x,̄ x)̄ − a(x,̄ x)̄ = a(x,̄ x)̄ ≤ lim inf a(xo,ϵ , xo,ϵ ). + ϵ→0

Since lim infϵ→0+ a(xo,ϵ , xo,ϵ ) ≤ lim supϵ→0+ a(xo,ϵ , xo,ϵ ) combining the above inequalities leads to the conclusion that ̄ a(x,̄ x)̄ ≤ a(x,̄ x) − ⟨x⋆ , x − x⟩,

∀ x ∈ C,

therefore, x̄ ∈ C is a solution of (7.30), and by the uniqueness of solutions to problem (7.30) we conclude that x̄ = xo . Therefore, we have obtained that xo,ϵ ⇀ xo in X. By the standard argument (based on the Urysohn property, see Remark 1.1.51), we have convergence for the whole sequence and not just for the chosen subsequence. It remains to show that the convergence is strong. To this end, note that by the positivity of φ and the coercivity of a it holds that 1 1 0 ≤ cE ‖xo,ϵ − xo ‖2 + φ(xo,ϵ ) ≤ a(xo,ϵ − xo , xo,ϵ − xo ) + φ(xo,ϵ ). ϵ ϵ By the fact that a is bilinear, a(xo,ϵ − xo , xo,ϵ − xo ) = a(xo,ϵ , xo,ϵ ) − a(xo , xo,ϵ ) − a(xo,ϵ , xo ) + a(xo , xo ), and since xo,ϵ solves the penalized problem (7.34), setting x = xo as test element we obtain that 1 a(xo,ϵ , xo,ϵ ) − a(xo,ϵ , xo ) + φ(xo,ϵ ) ≤ −⟨x⋆ , xo − xo,ϵ ⟩. ϵ Combining the above three estimates, 1 0 ≤ cE ‖xo,ϵ − xo ‖2 + φ(xo,ϵ ) ϵ ≤ −⟨x⋆ , xo − xo,ϵ ⟩ − a(xo , xo,ϵ ) + a(xo , xo ),

∀ ϵ > 0,

7.5 Approximation methods and numerical techniques | 355

and as above, passing first to the limit inferior and then to the limit superior as ϵ → 0+ , and keeping in mind that xo,ϵ ⇀ xo , we see that 1 1 (cE ‖xo,ϵ − xo ‖2 + φ(xo,ϵ )) ≤ lim sup(cE ‖xo,ϵ − xo ‖2 + φ(xo,ϵ )) ≤ 0, 0 ≤ lim inf + ϵ ϵ ϵ→0+ ϵ→0 so that 1 lim+ cE ‖xo,ϵ − xo ‖2 + φ(xo,ϵ ) = 0, ϵ ϵ→0 and since ϵ1 φ(xo,ϵ ) ≥ 0 we conclude that ‖xo,ϵ − xo ‖ → 0, therefore, xo,ϵ → x. Example 7.5.4 (An existence proof based on penalization). As already mentioned, penalization may be used to provide an alternative existence proof for the variational inequality (7.29) (see, e. g., [23]). To this end, instead of using, e. g., Theorem 7.4.1 to show the existence for the penalized problem (7.30) or (7.31), we may do so directly by using a finite dimensional approximation instead whose solvability is guaranteed by the Brouwer fixed-point theorem and passing to the limit as the dimensions go to infinity (Galerkin method). For concreteness, let us consider (7.31) and to be more precise the situation encountered in Example 7.5.2 and in particular (7.32). Consider a basis {em : m ∈ ℕ} for X, let Xm = span{ek , k = 1, . . . , m} and consider the approximate (finite dimensional) problems 1 a(um , vm ) + ⟨β(um ), vm ⟩ = ⟨f , vm ⟩, ϵ

∀ vm ∈ Xm ,

(7.35)

with the explicit dependence of um on ϵ omitted for easing the notation. Then by an application of the Brouwer fixed-point theorem (and in particular Proposition 3.2.13) problem (7.35) admits, for every ϵ > 0 fixed, a unique um ∈ Xm , for any m ∈ ℕ. The family of solutions {um : m ∈ ℕ} is uniformly bounded in m ∈ ℕ, as can be shown by using as test function vm = um − vo for any vo ∈ C in (7.35) where by noting that β(vo ) = 0, (7.35) yields 1 a(um , um − vo ) + (β(um ) − β(vo ), um − vo ⟩ = ⟨f , um − vo ⟩, ϵ

(7.36)

which by the monotonicity of the penalty term β provides the inequality a(um − vo , um − vo ) ≤ ⟨f , um − vo ⟩ − a(vo , um − vo ), where by the coercivity and continuity of a we get that cE ‖um − vo ‖2 ≤ (‖f ‖X ⋆ + c‖vo ‖)‖um − vo ‖, from which we have that ‖um − vo ‖ ≤ c󸀠 and, therefore, ‖um ‖ ≤ c󸀠󸀠 for some c󸀠 , c󸀠󸀠 independent of m. Then, by reflexivity we may extract a subsequence (denoted the same) and a uϵ ∈ X such that um ⇀ uϵ in X = W01,2 (𝒟) as m → ∞, and by the compact embedding of W01,2 (𝒟) in L2 (𝒟), a further subsequence (still denoted the

356 | 7 Variational inequalities same) such that um → uϵ in L2 (𝒟). Passing to the limit as m → ∞, we may show that xϵ satisfies the penalized form of the equation. To pass to the limit, we first note that by (7.36) and since we have already established the fact that xm is bounded, we see that ⟨ ϵ1 β(um ), um − vo ⟩ ≤ c1 for some c1 independent of m. Recalling the special form of β(u) = −(ψ − u)+ , we write 󵄩󵄩 + + + 󵄩2 󵄩󵄩(ψ − um ) 󵄩󵄩󵄩L2 (𝒟) = ⟨(ψ − um ) , ψ − um ⟩ = ⟨−(ψ − um ) , um − ψ⟩ = ⟨β(um ), um − ψ⟩ = ⟨β(um ), um − v0 ⟩ + ⟨β(um ), vo − ψ⟩ ≤ ⟨β(um ), um − vo ⟩ ≤ c󸀠 ,

as ⟨β(um ), vo −ψ⟩ = −⟨(ψ−u)+ , vo −ψ⟩ ≤ 0, since vo ≥ ψ. This implies that ϵ1 ‖β(um )‖ < c2 with c2 independent of m, so that again by reflexivity there exists β̄ ∈ L2 (𝒟) such that β(um ) ⇀ β.̄ With this information at hand, we now take an arbitrary v ∈ X and consider a {vm : m ∈ ℕ}, with vm ∈ Xm such that vm → v in X. Then for any m ∈ ℕ we take vm as test function in (7.35) so that passing to the limit as m → ∞ we have 1 a(uϵ , v) + ⟨β,̄ v⟩ = ⟨f , v⟩, ϵ so that if we can show that β̄ = β(uϵ ) we are done. But this can be seen by passing to the subsequence for which um → uϵ in L2 (𝒟). For more general types of penalty functions, one could obtain this result by monotonicity arguments, which will be discussed in detail in Chapter 9. In fact, one could obtain an existence result for a general penalized equation in terms of finite dimensional approximations using the arguments leading to the celebrated Browder–Minty surjectivity theorem for monotone operators (see Theorem 9.2.12) ◁

7.5.2 Internal approximation schemes In this section, we consider the approximation of first kind variational inequalities of the type find xo ∈ C : ⟨Axo , x − xo ⟩ ≥ L(x − xo ),

∀ x ∈ C,

(7.37)

where C ⊂ X is a closed convex set, A : X → X ⋆ is a linear operator associated with a continuous and coercive bilinear form a : X × X → ℝ and L : X → ℝ is a continuous linear form. We assume for simplicity that X is a Hilbert space. By the standard Lax– Milgram–Stampacchia theory, this admits a unique solution xo ∈ C. A key element in such schemes are the families of closed subspaces {Xh : h > 0}, Xh ⊂ X, and closed convex sets {Ch : h > 0}, parameterized with h > 0, where h is some parameter converging to 0. Usually these approximations are chosen to be finite dimensional, for instance in a Hilbert space with an orthonormal basis we may

7.5 Approximation methods and numerical techniques | 357

choose as this approximation the sequence of finite dimensional projections to the basis (Fourier expansion). We then replace the variational inequality (7.37) with its approximate version for any11 h > 0, find xh ∈ Ch : ⟨Axh , x − xh ⟩ ≥ L(x − xh ),

∀ x ∈ Ch .

(7.38)

By the standard Lax–Milgram–Stampacchia theory, the approximation (7.38) admits a unique solution xh for every h > 0. Our aim here is, assuming that we have a solution for (7.38), to show that under certain conditions xh → xo as h → 0. We need the following assumption on the family of approximations (see [106]). Assumption 7.5.5 (The family {Ch : h > 0}). The family of approximations {Ch : h > 0} is such that: (i) For any {xh : h > 0} ⊂ X, with xh ∈ Ch for every h > 0, if xh ⇀ x then12 x ∈ C. (ii) For every z ∈ C, there exists ̂zh ∈ Ch such that ̂zh → z (strongly) in X as h → 0. The following proposition (see [106]) shows convergence of the approximation scheme (7.38) to solutions of the variational inequality (7.37). Proposition 7.5.6. Let a : X × X → ℝ be a continuous and coercive bilinear form and L : X → ℝ a continuous linear form. Consider also a family of approximations {Ch : h > 0} for the closed convex set C ⊂ X, that satisfies Assumption 7.5.5. Then, for the solution xh of (7.38) we have that xh → xo as h → 0. Proof. The solution of (7.38) satisfies the uniform bound ‖xh ‖ < c󸀠 , h > 0, for some c󸀠 independent of h. Indeed, using the bilinear form version of (7.38) we have that a(xh , xh ) ≤ a(xh , x) − L(x − xh ) for every x ∈ Ch , which by the coercivity and continuity of a yields the estimate cE ‖xh ‖2 ≤ c‖x‖ ‖xh ‖ + c1 (‖x‖ + ‖xh ‖),

∀ x ∈ Ch .

Choose an arbitrary z ∈ C and consider the approximating sequence ̂zh ∈ Ch (which exists by Assumption 7.5.5(ii)) as test function x = ̂zh . Since ̂zh → z we have that ̂zh is uniformly bounded, i. e., ‖̂zh ‖ < c1 for some constant independent of h > 0, hence, cE ‖xh ‖2 ≤ c3 ‖x‖ + c4 for appropriate constants c3 , c4 independent of h > 0, and this implies that ‖xh ‖ < c󸀠 , for every h > 0. By the uniform boundedness of {xh : h > 0} and reflexivity, there exists a subsequence {xhn : n ∈ ℕ}, hn → 0, and a x̄ ∈ X such that xhn ⇀ x.̄ By Assumption 7.5.5(i), 11 Note that we use here the notation L for the linear form (rather than ⟨x⋆ , x − xh ⟩ for some x⋆ ∈ X ⋆ ), as in general one may also approximate the linear form as well by finite dimensional approximation, for example use x⋆h in the duals of finite dimensional spaces so that x⋆h converges in some appropriate sense to x⋆ . This procedure can be formalized by the notation L(x − xh ). 12 This always holds if Ch ⊂ C for every h > 0.

358 | 7 Variational inequalities we have that x̄ ∈ C. We will show that in fact, x̄ = xo , the solution of (7.37). Moving along the subsequence {hn : n ∈ ℕ} in (7.38) and adopting the simpler notation xn := xhn and Cn := Chn , we have that for every n ∈ ℕ, a(xn , z − xn ) ≥ L(z − xn ),

∀ z ∈ Cn .

(7.39)

Choose an arbitrary z ∈ C and consider the approximating sequence ̂zh ∈ Ch , ̂zh → z (which exists by Assumption 7.5.5(ii)), pass to the subsequence {hn : n ∈ ℕ}, adopt the notation zn := ̂zhn , and then for every n, use as test function z = zn ∈ Cn in (7.39) to obtain a(xn , zn − xn ) ≥ L(zn − xn ),

∀ n ∈ ℕ.

(7.40)

We rearrange (7.40) as a(xn , xn ) ≤ a(xn , zn ) − L(zn − xn ),

∀ n ∈ ℕ,

(7.41)

and taking the limit inferior on both sides of the above inequality, keeping in mind that L(zn − xn ) → L(z − x)̄ and a(xn , zn ) → a(x,̄ z), since xn ⇀ x̄ and zn → z, we have ̄ lim inf a(xn , xn ) ≤ a(x,̄ z) − L(z − x).

(7.42)

a(x,̄ x)̄ ≤ lim inf a(xn , xn ).

(7.43)

n→∞

We claim that n→∞

Assuming the claim to hold for the time being and combining (7.43) with (7.42) yields ̄ which upon rearrangement becomes a(x,̄ x)̄ ≤ a(x,̄ z) − L(z − x), ̄ a(x,̄ z − x)̄ ≥ L(z − x), and since z ∈ C was arbitrary, we conclude (by the uniqueness of the solution of (7.37)) that x̄ = xo . Performing the same steps for any subsequence of {xh : h > 0} and using the Urysohn property (Remark 1.1.51), we conclude that the whole sequence xh ⇀ xo . To complete weak convergence, it remains to prove claim (7.43). By coercivity, a(xn − x,̄ xn − x)̄ ≥ cE ‖xn − x‖̄ 2 ≥ 0, for every n ∈ ℕ, so that upon rearrangement a(xn , x)̄ + a(x,̄ xn ) − a(x,̄ x)̄ ≤ a(xn , xn ),

∀ n ∈ ℕ.

Hence, taking the limit inferior on both sides and recalling that xn ⇀ x,̄ we obtain the claim.

7.6 Application: boundary and free boundary value problems | 359

To show the strong convergence, we will use coercivity once more to estimate (upon replacing x̄ = xo now that the equality has been established) cE ‖xn − xo ‖2 ≤ a(xn − xo , xn − xo ) = a(xn , xn ) − a(xn , xo ) − a(xo , xn ) + a(xo , xo ), where, upon taking limit superior on both sides and recalling that xn ⇀ xo we have that cE lim sup ‖xn − xo ‖2 ≤ lim sup a(xn , xn ) − a(xo , xo ). n→∞

n→∞

(7.44)

Since xn solves (7.39) and xo ∈ C, by Assumption 7.5.5(ii) we may choose a sequence x̂o,n → xo , x̂o,n ∈ Cn , set z = x̂o,n as test function in (7.39) to obtain the analogue for (7.40), and then taking the limit superior in that (upon rearrangement and noting that xn ⇀ xo , and x̂o,n → xo ), to obtain lim sup a(xn , xn ) ≤ a(xo , xo ). n→∞

Combining this result with (7.44) yields cE lim supn ‖xn − xo ‖2 ≤ 0 so that xn → xo , and then by standard arguments using the Urysohn property xh → xo .

7.6 Application: boundary and free boundary value problems 7.6.1 An important class of bilinear forms Let 𝒟 ⊂ ℝd , bounded with sufficiently smooth boundary 𝜕𝒟, and consider the functions aij , ai , a0 ∈ L∞ (𝒟), i, j = 1, . . . , d. For any u, v : 𝒟 → ℝ, such that u, v ∈ X := W01,2 (𝒟), define the bilinear form a : X × X → ℝ, by a(u, v) :=

ˆ 𝒟

d

( ∑ aij (x) i,j=1

d 𝜕u 𝜕v 𝜕u (x) (x) + ∑ ai (x) (x)v(x) + a0 (x) u(x) v(x))dx. (7.45) 𝜕xi 𝜕xj 𝜕x i i=1

A simple integration by parts exercise shows that the operator A : X → X ⋆ defined by this bilinear form through ⟨Au, v⟩ = a(u, v) for every u, v ∈ X = W01,2 (𝒟), admits the form of the second-order differential operator d

d 𝜕 𝜕u 𝜕u (aij ) + ∑ ai + a0 u, 𝜕x 𝜕x 𝜕x i j i i,j=1 i=1

Au = − ∑

(7.46)

or in more compact form Au = −div(A∇u) + a ⋅ ∇u + a0 u,

(7.47)

upon defining the matrix and the vector valued functions A = (aij )di,j=1 , and a = (a1 , . . . , ad ), respectively.

360 | 7 Variational inequalities If ai = 0 and aij = aji for every i, j = 1, . . . , d, the bilinear form a is symmetric and the operator A is related to the Gâteaux derivative of the functional F : X → ℝ defined by F(u) = 21 a(u, u). In the special case where the matrix A = Id×d , ai = 0 for every i = 1, . . . , d and a0 = 0, the operator A = −Δ, where Δ is the Laplacian. The following will be standing assumptions in the remaining part of this chapter. Assumption 7.6.1. The functions aij , ai , a0 ∈ L∞ (𝒟) for i, j = 1, . . . , d. Assumption 7.6.2 (Uniform ellipticity). The matrix valued function A = (aij )di,j=1 satisfies the uniform ellipticity condition, i. e., there exists a constant cE > 0 such that d

∑ aij (x)zi zj ≥ cE |z|2 ,

i,j=1

a. e. x ∈ 𝒟, ∀ z ∈ ℝd .

The corresponding operator A is called uniformly elliptic. We will assume in what follows that aij = aji so that the uniform ellipticity assumptions can be easily checked in terms of the spectral properties of A. Some of the above assumptions may be too restrictive and can be relaxed at the expense of having to employ more sophisticated arguments. Proposition 7.6.3. Let Assumption 7.6.1 hold and suppose that A = (aij )di,j=1 satisfies the uniform ellipticity condition (Assumption 7.6.2). Moreover, assume that ‖a‖L∞ (𝒟) ≤

cE , 4√cp

‖a0 ‖L∞ (𝒟) ≤

cE , 4cp

(7.48)

with cp = c−2 𝒫 , where c𝒫 is the constant appearing in Poincaré’s inequality (see Theorem 1.5.13). Then the bilinear form a : X × X → ℝ, with X = W01,2 (𝒟) is continuous and coercive. Proof. Continuity follows by a simple application of the Hölder inequality, since 󵄨󵄨 󵄨 󵄨󵄨a(u, v)󵄨󵄨󵄨 ≤ max ‖aij ‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖∇v‖L2 (𝒟) i,j + max ‖ai ‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖v‖L2 (𝒟) + ‖a0 ‖L∞ (𝒟) ‖u‖L2 (𝒟) ‖v‖L2 (𝒟) , i

from which it easily follows that there exists a c1 > 0 such that 󵄨󵄨 󵄨 󵄨󵄨a(u, v)󵄨󵄨󵄨 ≤ c1 ‖u‖W 1,2 (𝒟) ‖v‖W 1,2 (𝒟) . 0 0 Note that continuity holds over the whole of W 1,2 (𝒟) and not only on W01,2 (𝒟). To show the coercivity property, we first consider the bilinear form a0 : X × X → ℝ defined by a0 (u, v) =

ˆ

d

∑ aij (x)

𝒟 i,j=1

𝜕u 𝜕v (x) (x)dx. 𝜕xi 𝜕xj

7.6 Application: boundary and free boundary value problems |

361

Clearly, a(u, v) = a0 (u, v) +

ˆ 𝒟

d

(∑ ai (x) i=1

𝜕u (x)v(x) + a0 (x) u(x) v(x))dx. 𝜕xi

(7.49)

By the uniform ellipticity Assumption 7.6.2, we see that a0 (u, u) ≥ cE ‖∇u‖2L2 (𝒟) , so that a0 (u, u) always keeps a positive sign. We now estimate, using (7.49) and Hölder’s inequality, a(u, u) ≥ a0 (u, u) − ‖a‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖u‖L2 (𝒟) − ‖a0 ‖L∞ (𝒟) ‖u‖2L2 (𝒟)

≥ cE ‖∇u‖2L2 (𝒟) − ‖a‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖u‖L2 (𝒟) − ‖a0 ‖L∞ (𝒟) ‖u‖2L2 (𝒟) .

(7.50)

If a = a0 = 0, then by the Poincaré inequality (which in fact guarantees that ‖∇u‖L2 (𝒟) is an equivalent norm for W01,2 (𝒟)) we conclude that a is coercive on X = W01,2 (𝒟). If a ≠ 0 and a0 ≠ 0, we need to subordinate the resulting terms to the leading contribution of a0 (u, u) so that coercivity is guaranteed. To this end, estimate ‖∇u‖L2 (𝒟) ‖u‖L2 (𝒟) ≤ ϵ‖∇u‖2L2 (𝒟) + so that

1 ‖u‖2L2 (𝒟) , 4ϵ

−‖a‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖u‖L2 (𝒟) − ‖a0 ‖L∞ (𝒟) ‖u‖2L2 (𝒟) ‖a‖L∞ (𝒟) + ‖a0 ‖L∞ (𝒟) ) ‖u‖2L2 (𝒟) 4ϵ ‖a‖L∞ (𝒟) + ‖a0 ‖L∞ (𝒟) ) cp ‖∇u‖2L2 (𝒟) , ≥ −ϵ‖a‖L∞ (𝒟) ‖∇u‖2L2 (𝒟) − ( 4ϵ

≥ −ϵ‖a‖L∞ (𝒟) ‖∇u‖2L2 (𝒟) − (

(7.51)

where in the last estimate we used the Poincaré inequality (see Theorem 1.5.13) and cp := c−2 𝒫 , where by c𝒫 we denote the Poincaré constant. Suppose that ϵ is chosen such that cp

‖a‖L∞ (𝒟) 4ϵ

≤

cE , 8

this leading to the condition ‖a‖L∞ (𝒟) ≤ ϵ 2ccE . Then ϵ‖a‖L∞ (𝒟) ≤

p

√cp . 2

This choice leads

i. e., that ‖a0 ‖L∞ (𝒟) ≤

cE . 4cp

Then we may esti-

p

cE , 8

to a condition for ‖a‖L∞ (𝒟) of the form ‖a‖L∞ (𝒟) ≤ ‖a0 ‖L∞ (𝒟) is such that ‖a0 ‖L∞ (𝒟) cp ≤ mate the right-hand side of (7.51) as

cE , 4

p

i. e., ϵ =

ϵ2 2ccE , and we now choose ϵ such that ϵ2 2ccE =

cE . 4√cp

Suppose furthermore that

−‖a‖L∞ (𝒟) ‖∇u‖L2 (𝒟) ‖u‖L2 (𝒟) − ‖a0 ‖L∞ (𝒟) ‖u‖2L2 (𝒟) c c c c ≥ − E ‖∇u‖2L2 (𝒟) − E ‖∇u‖2L2 (𝒟) − E ‖∇u‖2L2 (𝒟) = − E ‖∇u‖2L2 (𝒟) , 8 8 4 2

which when combined with (7.50) leads to c a(u, u) ≥ E ‖∇u‖2L2 (𝒟) , 2

and using the fact that ‖∇u‖L2 (𝒟) is an equivalent norm for W01,2 (𝒟) (see Example 1.5.14) we are led to the required coercivity result.

362 | 7 Variational inequalities 7.6.2 Boundary value problems Within the framework of Section 7.6.1 (and using the same notation), we first consider the application of the Lax–Milgram Theorem 7.3.5 to the solvability of linear boundary value problems of the form −div(A∇u) + a ⋅ ∇u + a0 u = f , u=0

in 𝒟,

on 𝜕𝒟.

(7.52)

We consider the Hilbert spaces X := W01,2 (𝒟), X ⋆ = W −1,2 (𝒟), and upon defining the elliptic operator A : X → X ⋆ , associated to the bilinear form a : X × X → ℝ defined in (7.45), by A = −div(A∇) + a ⋅ ∇ + a0 I, equation (7.52) can be expressed in the abstract form Au = f , assuming f ∈ X ⋆ = W −1,2 (𝒟). A variational formulation of equation (7.52), leading to the so-called concept of weak solutions for the boundary value problem (7.52), will prove very useful in the treatment of this equation. Definition 7.6.4 (Weak solutions). A function u ∈ X := W01,2 (𝒟) is a weak solution of the boundary value problem (7.52), for a given f ∈ X ⋆ = W −1,2 (𝒟), if a(u, v) = ⟨f , v⟩,

∀ v ∈ X = W01,2 (𝒟),

where a : X ×X → ℝ is the bilinear form defined in (7.45), and ⟨⋅, ⋅⟩ is the duality pairing between W01,2 (𝒟) and W −1,2 (𝒟). In certain special cases, in which (7.52) could be identified as the Euler–Lagrange equation for the minimization of an integral functional, the existence of weak solutions has been studied using techniques from the calculus of variations in Chapter 6. However, the Lax–Milgram theory (developed in Section 7.3 and in particular Theorem 7.3.5) can be used (see, e. g., [63]) to study more general cases. Proposition 7.6.5. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that a, a0 satisfy condition (7.48). Then, for any f ∈ (W01,2 (𝒟))∗ = W −1,2 (𝒟), the boundary value problem (7.52) admits a unique weak solution u ∈ W01,2 (𝒟), satisfying the bound ‖u‖W 1,2 (𝒟) ≤ c‖f ‖(W 1,2 (𝒟))∗ 0

0

for some constant c > 0 which depends on the ellipticity constant cE and the domain 𝒟. Moreover, if f ∈ L2 (𝒟), then the operator T : L2 (𝒟) → L2 (𝒟), defined by Tf = u where u is the solution of (7.52), is a compact operator. Proof. Let X = W01,2 (𝒟) and X ⋆ = W −1,2 (𝒟), which are Hilbert spaces. Since a, a0 satisfy condition (7.48) by Proposition 7.6.3 the bilinear form a is continuous and coercive on

7.6 Application: boundary and free boundary value problems | 363

X = W01,2 (𝒟). Then, by a straightforward application of the Lax–Milgram Theorem 7.3.5 we conclude the existence of a unique solution of the boundary value problem (7.52). The bound for the solution follows by the coercivity of the bilinear form, according to which setting v = u in the weak form yields cE ‖u‖2W 1,2 (𝒟) ≤ a(u, u) = ⟨f , u⟩ ≤ ‖f ‖(W 1,2 (𝒟))∗ ‖u‖W 1,2 (𝒟) , 0

0

0

and the result follows upon dividing both sides with ‖u‖W 1,2 (𝒟) . The compactness of 0

c

the solution operator T follows from the compact embedding of W01,2 (𝒟) 󳨅→ L2 (𝒟) (see Theorem 1.5.11).

Of course, the conditions of Proposition 7.6.5 can be rather restrictive. One may treat the solvability of the boundary value problem (7.52) even when a, a0 do not satisfy condition (7.48) using a combination of the Lax–Milgram Theorem 7.3.5 and the Fredholm theory of compact operators (see Theorem 1.3.9). The idea (see, e. g., [63]) is based on the observation that the bilinear form a(u, v) = ⟨Au, v⟩L2 (𝒟) + γ⟨u, v⟩L2 (𝒟) is coercive for large enough values of γ, so that the corresponding operator A + γI is c invertible, and by the compact embedding W01,2 (𝒟) 󳨅→ L2 (𝒟) its inverse T = (A + γI)−1 : L2 (𝒟) → L2 (𝒟) is compact. We may then treat the original problem in terms of a perturbation of T and use the well-developed theory of the solvability of equations involving compact operators to proceed. Theorem 7.6.6. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Then either problem (7.52) admits a unique solution for any f ∈ L2 (𝒟) or the homogeneous problem Aw = 0 admits a nontrivial weak solution in W01,2 (𝒟). Proof. It is easy to see that for large enough γ > 0 the boundary value problem Au + γu = f1 , u=0

in 𝒟,

on 𝜕𝒟,

(7.53)

admits a unique weak solution u ∈ W01,2 (𝒟) for every f1 ∈ W01,2 (𝒟). This follows by a straightforward application of the Lax–Milgram theorem to the bilinear, continuous and coercive form aγ : X × X → ℝ defined by aγ (u, v) = a(u, v) + γ⟨u, v⟩, for every u, v ∈ X, which is associated to the linear operator Aγ := A + γI : X → X ⋆ . One may therefore define the solution mapping f1 󳨃→ u, which in fact coincides with the operator (A + γI)−1 : (W01,2 (𝒟))∗ → W01,2 (𝒟) that, by the Lax–Milgram theorem, is linear and continuous. If we restrict the data of the problem to f1 ∈ L2 (𝒟), by the Rellich–Kondrachov compact embedding theorem we see that the solution mapping (A + γI)−1 : L2 (𝒟) → L2 (𝒟) is linear and compact. We may then define the operator Aγ = A+γI, and express the original problem (7.52) as Aγ u = γu + f . By the properties of the solution map, we may express the boundary

364 | 7 Variational inequalities value problem (7.52) in operator form as u = Tγ u + fγ ,

(7.54)

−1 where Tγ = γA−1 γ , and fγ := Aγ f is given. By the compactness of the operator Tγ : L2 (𝒟) → L2 (𝒟), equation (7.54) is a linear Fredholm operator equation of the second type, which can then be treated by the standard theory of such equations. In particular, a straightforward application of the Fredholm alternative (Theorem 1.3.9 applied to the compact operator Tγ ) yields that either (7.52) has a unique solution for any f ∈ L2 (𝒟) or the eigenvalue problem Au = 0 with homogeneous Dirichlet conditions, admits a weak solution.

Remark 7.6.7. The adjoint operator A⋆ plays an important role in the study of linear problems of the form (7.52). In particular, one may easily see that if A⋆ w = 0 admits other solutions in W01,2 (𝒟) apart from the trivial, then Au = f admits a solution if and only if ⟨f , w⟩ = 0 for any solution w of A⋆ w = 0. This simple observation finds interesting applications in e. g. homogenization theory [44]. Moreover, if Aw = 0 admits nontrivial solutions, or equivalently N(A) ≠ {0}, then by the Fredholm alternative (Theorem 1.3.9 applied to the compact operator Tγ ) one may conclude that dim(N(A)) = dim(N(A∗ )) < ∞. For more details see, e. g., [63]. Elliptic operators of the type studied here have a spectral theory similar to that developed in Section 6.2.3 for the Laplace operator. Theorem 7.6.8. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Consider the eigenvalue problem Au = λu in W01,2 (𝒟). (i) In general, the set of eigenvalues is an at most countable set which, if infinite, is of the form {λn : n ∈ ℕ} with λn → ∞ as n → ∞. (ii) If the bilinear form a is symmetric, then all eigenvalues are real, and satisfy 0 < λ1 ≤ λ2 ≤ λ3 ≤ ⋅ ⋅ ⋅, with λn → ∞ as n → ∞, while the first eigenvalue admits a variational representation as λ1 = min{a(u, u) : u ∈ W01,2 (𝒟), ‖u‖L2 (𝒟) = 1}, with the minimum attained for a function u1 , satisfying the eigenvalue problem for λ = λ1 . Moreover, there exists an orthornormal basis {un : n ∈ ℕ} of L2 (𝒟) consisting of eigenfunctions, i. e., solutions of the problem Aun = λn un , un ∈ W01,2 (𝒟), n ∈ ℕ. Proof. (i) As above choose γ ∈ ℝ large enough so that the bilinear form aγ (u, v) = a(u, v) + γ⟨u, v⟩L2 (𝒟) is coercive in W01,2 (𝒟), and express the eigenvalue problem in the equivalent form (A + γI)u = (γ + λ)u which using the inverse of Aγ := A + γI can be

expressed as u = γ+λ T u where Tγ = γ(A + γI)−1 . The operator Tγ : L2 (𝒟) → L2 (𝒟) γ γ is bounded, linear, bounded and compact, by our standard argument based on the

7.6 Application: boundary and free boundary value problems |

365

Rellich–Kondrachov–Sobolev embedding. We therefore conclude that λ is an eigenγ value of A if γ+λ is an eigenvalue of the linear compact operator Tγ . By Theorem 1.3.12, the eigenvalues of Tγ either form a finite set or a countable set {μn : n ∈ ℕ} with μn → 0 as n → ∞, from which the result follows. (ii) The proof follows similar arguments as the ones used for the study of the eigenvalues of the Laplacian, only that here we can resort directly to the Lax–Milgram lemma for the bilinear form a associated to the operator A, combined with the compact Sobolev embeddings. The result then follows by the abstract theory for the eigenvalue problem for compact operators, as in the case of the Laplacian. The variational representation for λ1 follows by repeating the steps in Proposition 6.2.12 for the bilinear form a instead of the Dirichlet integral ⟨∇u, ∇u⟩L2 (𝒟) . The existence of the basis follows by Theorem 1.3.13. Apart from spetral theory, many of the properties that we have seen for the Laplacian and the Poisson equation hold for more general elliptic equations of the form studied in this section. For instance, under restrictive coefficients on the data of the problem one may obtain analogues of the maximum principle or the method of sub and supersolutions. For lack of space, we do not present these extensions here but we will present such results for the more general case of variational inequalities (which reduces to elliptic equations in the special case C = X) in Section 7.6.3. Working similarly as in Proposition 6.6.13 we may obtain higher regularity for the weak solutions of (7.52). Proposition 7.6.9. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Assume, moreover, that aij ∈ C 1 (𝒟), i, j = 1, . . . , d, and f ∈ L2 (𝒟). 2,2 If u ∈ W01,2 (𝒟) is a solution of (7.52) then u ∈ Wloc (𝒟). The same result follows if 1,2 u ∈ W (𝒟) and Au ≤ f in the sense of distributions. Proof. The proof follows essentially in the same fashion as for Proposition 6.6.13 by 𝜕u noting that (using the same notation) Aj (∇u) = ∑di=1 aij (x) 𝜕x , j = 1, . . . , d, is linear i but there is an explicit spatial dependence on the coefficients aij which will affect the action of the difference operator. The lower order terms of the operator A will be moved to the right-hand side, replacing the term g in Proposition 6.6.13 by f − a ⋅ ∇u − a0 u. These modifications lead to a weak form d

∑

ˆ

j,k=1 𝒟

(τh,i ajk )(x)(Δh,i d

ˆ

+ ∑

j,k=1 𝒟 d

+ ∑

ˆ

j,k=1 𝒟

𝜕u 𝜕u )(x)(Δh,i )(x)ψ(x)2 dx 𝜕xj 𝜕xk

𝜕ψ2 𝜕u (x)(τh,i ajk )(x)(Δh,i )(x)(Δh,i u)(x)dx 𝜕xk 𝜕xj ψ(x)2 (Δh,i ajk )(x)

𝜕u 𝜕u (x)(Δh,i )(x)dx 𝜕xj 𝜕xk

366 | 7 Variational inequalities d

+ ∑ ˆ =

ˆ

j,k=1 𝒟

𝒟

𝜕ψ2 𝜕u (x)(Δh,i ajk )(x) (x)(Δh,i u)(x)dx 𝜕xk 𝜕xj d

(f (x) − ∑ aj (x) j=1

𝜕u (x) − a0 (x)u(x))(−Δ−h,i (ψ2 Δh,i u)(x))dx. 𝜕xj

The estimates proceed as in Proposition 6.6.13 with the first two terms being essentially the same, the next two can be bounded by noting that ajk ∈ C 1 (𝒟) so that the norms of the term Δh,i ajk are bounded, whereas the right-hand side can be handled by (discrete) integration by parts and the fact that aj , a0 ∈ L∞ (𝒟), so that the norms of the resulting terms Δh,i aj are bounded. Details are left to the reader (see also [63]). For weak solutions under the extra assumption that aij ∈ C 1 (𝒟), i, j = 1, . . . d, and if the domain satisfies extra regularity properties we may prove regularity up to the boundary, i. e., that u ∈ W 2,2 (𝒟). Definition 7.6.10 (C 2 domains). A domain 𝒟 ⊂ ℝd is said to be of class C 2 if for every xo ∈ 𝜕𝒟 there exists a neighborhood N(xo ) ⊂ 𝒟 and a C 2 -diffeomorphism g : B+ → N(xo ) where B+ = {x ∈ ℝd : |x| < 1, xd > 0}, is a half-ball. Since a domain of class C 2 can be mapped locally through a diffeomorphism g to a half-ball, if we manage to show the required estimates concerning regularity on half balls, then we may transfer these estimates to any compact domain. This is essentially the strategy of the proof of global regularity results (see, e. g., [63]). Proposition 7.6.11. Let 𝒟 be compact with 𝜕𝒟 of class C 2 . Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that aij ∈ C 1 (𝒟), i, j = 1, . . . , d and f ∈ L2 (𝒟). If u ∈ W01,2 (𝒟) is a weak solution of (7.52), then u ∈ W 2,2 (𝒟) and there is a constant c > 0 depending on 𝒟 and ‖aij ‖C1 (𝒟) , i, j = 1, . . . , d, such that ‖u‖W 2,2 (𝒟) ≤ c‖f ‖L2 (𝒟) . Proof. Our aim is to show that for any xo ∈ 𝜕𝒟 we can obtain local estimates of the form established in Proposition 7.6.9. The proof proceeds in 4 steps: 1. Since we consider 𝒟 to be compact, by a covering argument we can focus our attention to a finite number of neighborhoods, so that it suffices to obtain the relevant estimate for any neighborhood N(xo ) of a point xo ∈ 𝜕𝒟. 2. By step 1, consider any neighborhood N(xo ) of a point xo ∈ 𝜕𝒟, assumed without loss of generality to be xo = 0, and a diffeomorphism g : B+ → N(xo ), which is essentially a change of variables z → x that transforms the half-ball B+ to the neighborhood N(xo ). It will be convenient to adopt the convention of expressing the transformation g as z = (z1 , . . . , zd ) 󳨃→ g(z) = x = (x1 , . . . , xd ), so that z = g −1 (x) and of expressing its 𝜕Z 𝜕z inverse in coordinate form as g −1 = (Z1 , . . . , Zd ) so that wki := 𝜕xk = 𝜕xk (x). i

i

7.6 Application: boundary and free boundary value problems |

367

Recall the change of variables formula from multivariate calculus ˆ ˆ 󵄨 󵄨 Ψ(x)dx = Ψ(g(z))󵄨󵄨󵄨det(Dg(z))󵄨󵄨󵄨dz, N(xo )

g −1 (N(xo ))=B+

for any suitable Ψ : N(xo ) → ℝ. Since we deal with weak formulations of elliptic problems, we will apply this formula to d

Ψ = ∑ aij i,j=1

𝜕u 𝜕ϕ d 𝜕u + ∑a ϕ + a0 ϕ − fϕ, 𝜕xi 𝜕xj i=1 i 𝜕xi

for a test function ϕ localized in N(xo ), so that the right-hand side of the above will provide the weak form in the new variables z, which in fact will be a weak form on the half-ball B+ . After some algebra involving elementary arguments based on the chain rule, and taking care in properly transforming the partial derivatives 𝜕x𝜕 we conclude i

that in the new variables z, the elliptic equation becomes an elliptic equation in B+ of the form ˆ

d

̂ ij (z) [∑ a

B+ i,j=1

ˆ d ̂ ̂ ̂ ̂ 𝜕ϕ 𝜕u 𝜕u ̂ ̂ ̂f (z)ϕ(z)dz, ̂ ̂ ̂ (z) (z) + ∑ ai (z) (z)ϕ(z) + a0 (z)u(z)ϕ(z)]dz = 𝜕zi 𝜕zj 𝜕zi B+ i=1

(7.55)

̂ is a test function localized on B+ , and where ϕ ̂ (z) = u(g(z)), u

󵄨 󵄨 J(z) = 󵄨󵄨󵄨det(Dg(z))󵄨󵄨󵄨, 𝜕z 𝜕zj ̂ ij (z) = J(z) ∑ akℓ (g(z)) i , a 𝜕x k 𝜕xℓ k,ℓ ̂ i (z) = J(z) ∑ ak (g(z)) a k

â0 (z) = J(z)a0 (g(z),

𝜕zi , 𝜕xk

̂f (z) = J(z)f (g(z)).

̂ := (a ̂ ij )di,j=1 satisfies a uniform ellipticity condition One can see that the matrix A so that the transformed system is an elliptic equation on B+ . Indeed, observing that 𝜕z for any ξ 󸀠 = (ξ1󸀠 , . . . , ξd󸀠 ) ∈ ℝd , we can write, upon defining wik = √J 𝜕x i , that k

d

d

d

d

̂ ij ξi󸀠 ξj󸀠 = ∑ ∑ akℓ wik wjℓ ξi󸀠 ξj󸀠 = ∑ akℓ ξk ξℓ ≥ cE |ξ |2 , ∑a

i,j=1

k,ℓ=1 i,j=1

k,ℓ=1

(7.56)

where ξk = ∑di=1 wik ξi󸀠 and for the last inequality we used the uniform ellipticity for the matrix A = (akℓ )dk,ℓ=1 . Note that in vector notation ξ = Dg −1 ξ 󸀠 and since g ∘ g −1 = Id, the identity transformation, it holds that DgDg −1 = Id×d ; hence, there exists a constant c󸀠 such that |ξ | ≥ c󸀠 |ξ 󸀠 |. This, combined with (7.56) leads to ∑di,j=1 a󸀠ij ξi󸀠 ξj󸀠 ≥ cE󸀠 |ξ 󸀠 |2 which ̂ = (a ̂ ij )di,j=1 . is the required uniform ellipticity for A

368 | 7 Variational inequalities ̂ ∈ W 2,2 (B+ ) if and only if u ∈ W 2,2 (N(xo )). So it suffices to establish the Moreover, u required regularity for elliptic equations on half-balls. 3. We now consider the elliptic equation (7.55) on half-balls B+ . As stated before, ̂ ∈ C ∞ (B+ ) choosing any k ∈ {1, . . . , d − 1} it holds that v̂ = ̂ ∈ W 1,2 (B+ ) and ψ if u c 1,2 + 󸀠2 ̂ ) ∈ W0 (B ), so that it can be used as a test function in the weak form Δ−h,k (ψ Δh,k u (7.55). Then, by essentially repeating the same steps as in Proposition 6.6.13, we can 2 show that for all the partial derivatives 𝜕z𝜕 ûz , k, ℓ = 1, . . . , d, k + ℓ < 2d it holds that ℓ k

upon defining U 󸀠 to be the original half-ball and V 󸀠 one with half the radius, d 󵄩 󵄩󵄩 𝜕2 u 󵄩󵄩󵄩 󵄩󵄩 ̂ ‖W 1,2 (U 󸀠 ) ). ≤ c(‖̂f ‖L2 (U 󸀠 ) + ‖u ∑ 󵄩󵄩󵄩 󵄩󵄩 𝜕zℓ zk 󵄩󵄩󵄩L2 (V) k,ℓ=1

(7.57)

k+ℓ 0 so that that a d 󵄨󵄨 2 󵄨󵄨 𝜕2 u 󵄨 ̂ 󵄨󵄨󵄨󵄨 󵄨󵄨 ̂ 󵄨󵄨󵄨 󵄨 𝜕u ̂ | + |u ̂ | + |̂f |) 󵄨󵄨 2 󵄨󵄨 ≤ c( ∑ 󵄨󵄨󵄨 󵄨 + |∇u 󵄨󵄨 𝜕z 󵄨󵄨 󵄨󵄨 𝜕zi 𝜕zj 󵄨󵄨󵄨 k,ℓ=1 d

a. e.,

k+ℓ γ(x1 , . . . , xd−1 )},

for some r > 0 and a function γ : ℝd−1 → ℝ which is C 2 . Denoting by g −1 the transformation that takes 𝒟 to a half-ball, we may assume (without loss of generality) that g −1 (xo ) = 0, and we choose U 󸀠 to be a half-ball of radius ρ > 0 such that U ∈ g −1 (𝒟 ∩ ρ N(xo )), i. e., U 󸀠 = {z ∈ ℝd : |z| < ρ, zd > 0}. We also set V 󸀠 = {z ∈ ℝd : |z| < 2 , zd > 󸀠 󸀠 0}. Then the estimates of step 3 are valid for V and U . Undoing the transformation, this estimate is transferred to a similar local estimate for the solution of the elliptic problem u in the original domain 𝒟, and a covering argument based on step 1 can be used to complete the proof.

7.6 Application: boundary and free boundary value problems | 369

Remark 7.6.12. The result of Proposition 7.6.11 can be generalized for nonhomogeneous Dirichlet data of sufficient smoothness, i. e., if there exists g ∈ W 2,2 (𝒟) such that the weak solution of (7.52) satisfies u − g ∈ W01,2 (𝒟), with the estimate for the solution modified as ‖u‖W 2,2 (𝒟) ≤ c(‖f ‖L2 (𝒟) + ‖g‖W 2,2 (𝒟) ). 7.6.3 Free boundary value problems We now turn our attention to applications of the Lax–Milgram–Stampacchia Theorem 7.3.4. We will consider the application of this theorem to the general class of bilinear forms a defined in (7.45) and the corresponding operator A, and consider the problem find u ∈ C : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩, C := {v ∈

W01,2 (𝒟)

∀ v ∈ C,

: v(x) ≥ ψ(x) a. e. x ∈ 𝒟},

(7.58)

where ψ : 𝒟 → ℝ is a suitable given function. This is a generalization of the problem treated in Section 7.2 for the special case of the Laplacian. Theorem 7.6.13. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that a, a0 satisfy condition (7.48). Then, for any f ∈ W −1,2 (𝒟), the variational inequality (7.58) admits a unique solution. Proof. Let X := W01,2 (𝒟) and X ⋆ = W −1,2 (𝒟). By the assumptions on the data of the problem applying Proposition 7.6.3, we see that the corresponding bilinear form a : X × X → ℝ is continuous and coercive. Furthermore, C is a closed convex subset of X (see step 2 in the proof of Proposition 7.2.1). Therefore, by a straightforward application of Theorem 7.3.4 we conclude. If the obstacle ψ and the function f satisfy certain smoothness conditions, then working similarly as in Proposition 7.2.1 we may show that the solution of the variational inequality (7.58) satisfies the differential inequalities of the form Au = f ,

Au > f ,

if u > ψ,

if u = ψ.

(7.59)

An alternative formulation, is the so-called complementarity form, u ≥ ψ,

Au − f ≥ 0,

and ⟨Au − f , u − ψ⟩ = 0,

(7.60)

where by Au−f ≥ 0 we mean that ⟨Au−f , w⟩ ≥ 0 for every w ∈ W01,2 (𝒟) such that w ≥ 0. This is the complementarity form of the variational inequality which is equivalent to the original formulation. The equivalence can be seen by choosing appropriate test functions, e. g., v = u + w for arbitrary w ≥ 0, v = ψ and v = 2u − ψ.

370 | 7 Variational inequalities Problems of the general type (7.59) are called free boundary value problems or ob-

stacle problems and find important applications in mechanics, image processing, the

theory of stochastic processes and mathematical finance. Unlike the boundary value problems treated in the previous section, where the values of the unknown function u are specified on a known domain (e. g., for the homogeneous Dirichlet problem u = 0

on 𝜕𝒟, with the set 𝜕𝒟 being a priori specified) here the coincidence set 𝒞 ⊂ 𝒟 on

which u = ϕ is unspecified and will only be determined after problem (7.59) is solved, therefore, leading to an unknown (free) boundary.

Remark 7.6.14. The result of Theorem 7.6.13 can be generalized in various ways. For example, we may consider the double obstacle problem, by using the convex set C = {u ∈ W 1,2 (𝒟) : ψ1 (x) ≤ u(x) ≤ ψ2 (x), a. e. x ∈ 𝒟}, for suitable obstacle function

ψ1 , ψ2 . Another possible generalization is to include nonhomogeneous boundary data

g : 𝜕𝒟 → ℝ, g ∈ H 1/2 (𝜕𝒟) (see Proposition 1.5.17) satisfying ψ ≤ g on 𝜕𝒟. In this case,

we need to work in W 1,2 (𝒟) and consider the closed convex set C ⊂ W 1,2 (𝒟) defined by C = {u ∈ W 1,2 (𝒟) : u(x) ≥ ψ(x), a. e. x ∈ 𝒟, u(x) = g(x), x ∈ 𝜕𝒟}.

Remark 7.6.15. If the bilinear form a is noncoercive, then we may resort to a pertur-

bation type argument similar to that used in Theorem 7.6.6, combined with the Leray– Schauder alternative (Theorem 3.3.5). In particular, assuming for simplicity that we consider homogeneous Dirichlet boundary conditions, we may replace a with aγ defined as aγ (u, v) = a(u, v) + γ⟨u, v⟩, for every u, v ∈ W01,2 (𝒟), with γ > 0 chosen so

that aγ is coercive. Then, for any w ∈ L2 (𝒟), we consider the variational inequality

aγ (u, v − u) ≥ ⟨f , v − u⟩ + γ⟨w, v − u⟩ for every v ∈ C, and define the map w 󳨃→ Tγ w := u,

where u is the solution of the above variational inequality. This is a continuous and

compact map from L2 (𝒟) onto itself, a fixed point of which is a solution of the original

inequality. The existence of the fixed point can be obtained by the Leray–Schauder alternative. This requires to show that for any λ ∈ (0, 1) any solution of the equation

w = λTγ w admits an a priori bound. By the definition of Tγ , we can see that any solution

of w = λTγ w will satisfy the variational inequality aγ(1−λ) ⟨Tγ w, v − Tγ w⟩ ≥ ⟨f , v − Tγ w⟩, ̂, v − w ̂ ⟩ ≥ ⟨f , v − w ̂ ⟩, for any v ∈ C. Hence, deriving the required or equiv. aγ(1−λ) ⟨w

bounds requires some rather delicate a priori estimates on a family of related varia-

tional inequalities, which in turn require certain restrictive conditions on the data of the problem (see [100]).

The solution of variational inequalities associated to the bilinear form (7.45) en-

joy important and useful qualitative properties which are related to comparison and maximum principles, under certain restrictive assumptions on the coefficients of the

operator A. Without striving for full generality, we will restrict ourselves to the following simple case.

7.6 Application: boundary and free boundary value problems | 371

Consider the variational inequality find u ∈ C : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩, 1,2

∀ v ∈ C,

C = {u ∈ W (𝒟) : u(x) ≥ ψ(x), a. e. x ∈ 𝒟, u(x) = g(x), x ∈ 𝜕𝒟}.

(7.61)

for given obstacle ψ : 𝒟 → ℝ and boundary data g : 𝜕𝒟 → ℝ, g ∈ H 1/2 (𝜕𝒟) satisfying ψ ≤ g on 𝒟. We will also use the notation Ci , for the set defined in (7.61) for suitable obstacle and boundary data ψi , gi , and denote by ui the solution of (7.61) in Ci with f replaced by fi , i = 1, 2. We are now ready to state and prove a comparison result and a maximum principle for the variational inequality (7.61). Proposition 7.6.16 (Comparison result and maximum principle). Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that ai = a0 = 0, i = 1, . . . , d. Consider suitable obstacle, boundary data and forcing terms ψ, ψi , g, gi , f , fi , respectively, satisfying ψ ≤ g, ψi ≤ gi , and let u, ui , be the corresponding solutions to the variational inequalities (7.61), i = 1, 2. (i) Suppose that ψ1 ≥ ψ2 , g1 ≥ g2 and f1 ≥ f2 . Then, u1 ≥ u2 . (ii) u satisfies the bounds u ≥ m := min(0, inf g) 𝜕𝒟

a. e. in 𝒟

if f ≥ 0

u ≤ M := max(0, sup g, sup ψ) a. e. in 𝒟 𝜕𝒟

𝒟

and if f ≤ 0.

Proof. Note that under the stated assumptions, by Proposition 7.6.3, the corresponding bilinear form is continuous and coercive in W01,2 (𝒟). (i) Since ψ1 ≥ ψ2 and g1 ≥ g2 , the sets C1 , C2 are such that u1 ∨ u2 := max(u1 , u2 ) ∈ C1 , u1 ∧ u2 := min(u1 , u2 ) ∈ C2 ,

and ∀ u1 ∈ C1 , u2 ∈ C2 .

(7.62)

This is elementary to check but for the convenience of the reader the proof is presented in Lemma 7.7.1 in the Appendix of the chapter. For the variational inequality (7.61) for u1 , use as test function v = max(u1 , u2 ) ∈ C1 (by (7.62)). By the identity max(u1 , u2 ) = u1 + (u2 − u1 )+ , this yields ⟨Au1 , (u2 − u1 )+ ⟩ ≥ ⟨f1 , (u2 − u1 )+ ⟩.

(7.63)

For the variational inequality (7.61) for u2 , use as test function v = min(u1 , u2 ) ∈ C2 (by (7.62)). By the identity min(u1 , u2 ) = u2 − (u2 − u1 )+ , this yields ⟨Au2 , (u2 − u1 )+ ⟩ ≤ ⟨f2 , (u2 − u1 )+ ⟩.

(7.64)

372 | 7 Variational inequalities Multiplying (7.63) by −1, rearranging and summing with (7.64) we obtain that ⟨Au2 − Au1 , (u2 − u1 )+ ⟩ + ⟨f1 − f2 , (u2 − u1 )+ ⟩ ≤ 0, which since f1 ≥ f2 leads to ⟨Au2 − Au1 , (u2 − u1 )+ ⟩ ≤ 0.

(7.65)

Since g1 ≥ g2 , and ui ∈ Ci , i = 1, 2, it is clear the (u2 − u1 )+ vanishes on 𝜕𝒟, so that (u2 − u1 )+ ∈ W01,2 (𝒟). This combined with (7.65) leads to (u2 − u1 )+ = 0. Indeed, a quick calculation yields that ˆ A∇(u2 − u1 ) ⋅ ∇(u2 − u1 )+ dx 0 ≥ ⟨Au2 − Au1 , (u2 − u1 )+ ⟩ = 𝒟 ˆ 󵄩 󵄩2 󵄩 󵄩2 + A∇(u2 − u1 ) ⋅ ∇(u2 − u1 )+ dx ≥ cE 󵄩󵄩󵄩∇(u2 − u1 )+ 󵄩󵄩󵄩L2 (𝒟) ≥ cE c2𝒫 󵄩󵄩󵄩(u2 − u1 )+ 󵄩󵄩󵄩L2 (𝒟) , = 𝒟

where we used subsequently the uniform ellipticity and the Poincaré inequality. Hence, (u2 − u1 )+ = 0 and u1 ≥ u2 follows. (ii) For the weak maximum principle, let us first consider the case f ≥ 0. Let m = min(0, inf𝜕𝒟 g) and consider as test function v = max(u, m) = u + (m − u)+ . Since v ≥ u, it is easy to check that v ∈ C, so it is an acceptable test function. The weak form of the variational inequality (7.61) for this choice yields ˆ ˆ + f (m − u)+ dx. A∇u ⋅ ∇(m − u) dx ≥ (7.66) 𝒟

𝒟

Noting that

ˆ 𝒟

A∇u ⋅ ∇(m − u)+ dx = − =−

ˆ 𝒟

ˆ

𝒟

A∇(m − u) ⋅ ∇(m − u)+ dx A∇(m − u)+ ⋅ ∇(m − u)+ dx,

and substituting this in (7.66) we obtain ˆ ˆ − A∇(m − u)+ ⋅ ∇(m − u)+ dx ≥ f (m − u)+ dx ≥ 0, 𝒟

𝒟

(7.67)

where for the last inequality we used the fact that f ≥ 0. Therefore, combining (7.67) and the uniform ellipticity condition for A we obtain ˆ 󵄩 󵄩2 cE 󵄩󵄩󵄩∇(m − u)+ 󵄩󵄩󵄩L2 (𝒟;ℝd ) ≤ A∇(m − u)+ ⋅ ∇(m − u)+ dx ≤ 0, 𝒟

which leads to (m − u) being a constant a. e. in 𝒟, and since it vanishes on 𝒟 (by the choice of m) it holds that (m − u)+ = 0 a. e. in 𝒟 therefore u ≥ m a. e. in 𝒟. If f ≤ 0, let M = max(0, sup𝜕𝒟 g, sup𝒟 ψ), consider as test function v = min(u, M) = u − (u − M)+ and proceed accordingly. +

7.6 Application: boundary and free boundary value problems | 373

Remark 7.6.17. In the case where ψi = −∞, i = 1, 2 or ψ = −∞, we are essentially working with Ci = X or C = X and the above comparison results and maximum principles apply to the corresponding elliptic equation. The above comparison principle motivates the useful concept of super and subsolutions for variational inequalities. Definition 7.6.18 (Super and subsolutions). (i) An element ū ∈ X = W 1,2 (𝒟) is called a supersolution of the variational inequality (7.61) if ū ≥ ψ in 𝒟, ū ≥ g on 𝜕𝒟 and Aū − f ≥ 0 (meaning that ⟨Aū − f , w⟩ ≥ 0 for every w ∈ W01,2 (𝒟), w ≥ 0). (ii) An element u ∈ X = W 1,2 (𝒟) is called a subsolution of the variational inequality (7.61) if u ≥ ψ in 𝒟, u ≤ g on 𝜕𝒟 and Au − f ≤ 0 (meaning that ⟨Aū − f , w⟩ ≤ 0 for every w ∈ W01,2 (𝒟), w ≥ 0). Proposition 7.6.19. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that ai = a0 = 0, i = 1, . . . , d. (i) Let u and ū be any solution and any supersolution of (7.61), respectively. Then, u ≤ u.̄ (ii) Let u and u be any solution and any subsolution of (7.61), respectively. Then u ≤ u. Proof. (i) Since u ≥ ψ and ū ≥ ψ it also holds that min(u, u)̄ ≥ ψ so that we may use it as a test function. Setting v = min(u, u)̄ = u − (u − u)̄ + in (7.61), we obtain ⟨Au − f , (u − u)̄ + ⟩ ≤ 0.

(7.68)

Since ū is a supersolution, Aū − f ≥ 0 and since (u − u)̄ + ≥ 0, ⟨Aū − f , (u − u)̄ + ⟩ ≥ 0.

(7.69)

Combining (7.68) and (7.69), we obtain that ⟨Au − Au,̄ (u − u)̄ + ⟩ ≤ 0, therefore, (u − u)̄ + = 0 and u ≤ u.̄ (ii) Use the test function v = u ∨ u = max(u, u) = u + (u − u)+ and proceed as above. Example 7.6.20. The a priori bounds provided by Proposition 7.6.19 can be refined since if any two supersolutions ū 1 , ū 2 are given, ū = ū 1 ∧ ū 2 = min(ū 1 , ū 2 ) is also a supersolution, and hence, we may apply Proposition 7.6.19 for u.̄ Clearly, this procedure can be repeated for a finite number of iterations, so that if ū i , i = 1, . . . , n, are supersolutions then ū = min(ū 1 , . . . , ū n ) is also a supersolution. A similar construction is possible with subsolutions; if u1 , u2 are two subsolutions then u = u1 ∨u2 = max(u1 , u2 ) is also a subsolution.

374 | 7 Variational inequalities To check this claim, we reason as follows: Clearly, since ū i ≥ ψ on 𝒟 and ū i ≥ g on 𝜕𝒟, i = 1, 2 it holds that ū ≥ ψ on 𝒟 and ū ≥ g on 𝜕𝒟, so it only remains to check that Aū − f ≥ 0. To this end consider the closed convex set C 󸀠 := {u ∈ W 1,2 (𝒟) : u ≥ ū in 𝒟, u = g on 𝜕𝒟} and the corresponding variational inequality find u ∈ C 󸀠 : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩,

∀ v ∈ C󸀠 ,

(7.70)

which admits the equivalent complementarity form (ū plays the role of the obstacle) u ∈ C󸀠

Au − f ≥ 0,

⟨Au − f , u − u⟩̄ = 0.

(7.71)

By the Lions–Stampacchia theory, (7.70) admits a unique solution u ∈ C 󸀠 ; hence, u ≥ u.̄ On the other hand ū i are supersolutions of (7.70), since ū i ≥ u,̄ by Proposition 7.6.19, u ≤ ū i , i = 1, 2; hence, u ≤ ū 1 ∧ ū 2 = u.̄ By the uniqueness of u since u ≥ ū and u ≤ ū we have that u = u.̄ That, by the complementarity form (7.71) leads to Aū − f ≥ 0 so that ū is a supersolution of (7.61) ◁ One may generalize the above constructions to more general problems involving more general possibly nonlinear operators (see, e. g., [34]). We close our treatment of free boundary value problems with a regularity result (see [23]). To avoid unnecessary technicalities, let us consider problem (7.61) under the extra assumptions that ai = a0 = g = 0, i = 1, . . . , d, i. e., find u ∈ C : ⟨Au, v − u⟩ ≥ ⟨f , v − u⟩, C = {u ∈

W01,2 (𝒟)

∀ v ∈ C,

: u(x) ≥ ψ(x), a. e. x ∈ 𝒟}.

(7.72)

for given obstacle ψ : 𝒟 → ℝ, such that ψ ≤ 0 on 𝜕𝒟. Proposition 7.6.21. Let Assumption 7.6.1 hold and A = (aij )di,j=1 satisfy the uniform ellipticity condition of Assumption 7.6.2. Moreover, assume that ai = a0 = 0, i = 1, . . . , d. Under the additional assumptions that aij ∈ C 1 (𝒟), i, j = 1, . . . , d, f ∈ L2 (𝒟) and Aψ ∈ L2 (𝒟), the solution u of the variational inequality (7.72) satisfies u ∈ W 2,2 (𝒟). Proof. We sketch the main argument of the proof. First of all, we note that under the stated assumptions, by Proposition 7.6.3, the corresponding bilinear form is continuous and coercive in W01,2 (𝒟). The proof uses a penalization argument, the regularity results for boundary value problems we have developed in Section 7.6.2, and a passage to the limit argument which requires some delicate estimates. The penalized version (in the sense of Section 7.5.1) of (7.72), is equation ˆ ˆ 1 (uϵ − PC (uϵ ))vdx = fvdx, ∀ v ∈ W01,2 (𝒟), ϵ > 0. a(uϵ , v) + (7.73) ϵ 𝒟 𝒟 Recall (see Example 7.5.2) that for the specific C we consider here PC (u) = max(u, ψ) so that u − PC (u) = −(ψ − u)+ ∈ W01,2 (𝒟) (since ψ ≤ 0 on 𝜕𝒟).

7.6 Application: boundary and free boundary value problems | 375

We proceed in 4 steps: 1. We will first obtain a uniform bound in ϵ for the term ϵ1 ‖(ψ − uϵ )+ ‖L2 (𝒟) . We will use as test function v = uϵ − PC (uϵ ) = −(ψ − uϵ )+ ∈ W01,2 (𝒟) in (7.73). We have the important estimate a(uϵ , uϵ − PC (uϵ )) = a(−uϵ , (ψ − uϵ )+ )

= a(ψ − uϵ , (ψ − uϵ )+ ) − a(ψ, (ψ − uϵ )+ )

= a((ψ − uϵ )+ , (ψ − uϵ )+ ) − a(ψ, (ψ − uϵ )+ )

(7.74)

= a((ψ − uϵ )+ , (ψ − uϵ )+ ) − ⟨Aψ, (ψ − uϵ )+ ⟩, where we used the fact that ∇(ψ − uϵ )+ = ∇(ψ − uϵ ) 1ψ−uϵ >0 . Combining (7.73) (for v = uϵ − PC (uϵ )) with (7.74), we have that 1 a((ψ − uϵ )+ , (ψ − uϵ )+ ) − ⟨Aψ, (ψ − uϵ )+ ⟩ + ‖(ψ − uϵ )+ ‖2L2 (𝒟) = − ϵ

ˆ 𝒟

f (ψ − uϵ )+ dx.

(7.75) Since a((ψ − uϵ )+ , (ψ − uϵ )+ ) ≥ 0 by coercivity, (7.75) implies that for all ϵ > 0, we have 1 󵄩󵄩 + 󵄩2 + 󵄩(ψ − uϵ ) 󵄩󵄩󵄩L2 (𝒟) ≤ ⟨Aψ, (ψ − uϵ ) ⟩ − ϵ󵄩

ˆ

𝒟

f (ψ − uϵ )+ dx

󵄩 󵄩 ≤ (‖Aψ‖L2 (𝒟) + ‖f ‖L2 (𝒟) )󵄩󵄩󵄩(ψ − uϵ )+ 󵄩󵄩󵄩L2 (𝒟) ,

which upon dividing yields the bound ϵ1 ‖(ψ − uϵ )+ ‖L2 (𝒟) ≤ (‖Aψ‖L2 (𝒟) + ‖f ‖L2 (𝒟) ). 2. We now return to (7.73), which in operator form implies that Auϵ − ϵ1 (ψ−uϵ )+ = f , and solving with respect to Auϵ , since both ϵ1 (ψ − uϵ )+ and f are bounded in L2 (𝒟), so is Auϵ (with a bound independent of ϵ). Hence, Auϵ ∈ L2 (𝒟) so that, by arguments similar to those employed in Propositions 7.6.9 and 7.6.11, we have that uϵ ∈ W 2,2 (𝒟) (uniformly bounded in ϵ). By reflexivity, there exists a weakly convergent subsequence ̂ in W 2,2 (𝒟). uϵ ⇀ u 3. Working as in Proposition 7.5.3 or Example 7.5.4 we can show that uϵ → u in W01,2 (𝒟) as ϵ → 0, with u = PC (u) ∈ C. In particular, we express u − uϵ = (u − ψ) − (ψ − uϵ )− + (ψ − uϵ )+ , so that defining wϵ := (u − ψ) − (ψ − uϵ )− we have u − uϵ = wϵ + (ψ − uϵ )+ . In what follows, ci , i = 1, . . . , 5, is used to denote positive constants, which we do not bother to specify. We have already established a bound of the form ‖(ψ − uϵ )+ ‖L2 (𝒟) < c1 ϵ in step 1 above. Inserting this estimate in (7.75), we have that a((ψ−uϵ )+ , (ψ−uϵ )+ ) ≤ c2 ϵ, and by coercivity of a, we conclude that ‖(ψ−uϵ )+ ‖W 1,2 (𝒟) ≤ 0 c3 √ϵ. Since u − uϵ = wϵ + (ψ − uϵ )+ , if we show that ‖wϵ ‖W 1,2 (𝒟) ≤ c4 √ϵ then we have 0

that ‖u − uϵ ‖W 1,2 (𝒟) ≤ c5 √ϵ, hence, the stated convergence result in W01,2 (𝒟). To show 0

the bound for wϵ , use as test function v = −wϵ in (7.73) and v = ψ + (ψ − uϵ )− (so that v − u = −wϵ ) as test function in (7.72). We add the resulting inequalities to obtain 1 a(u − uϵ , wϵ ) + ⟨(ψ − uϵ )+ , wϵ ⟩ ≤ 0. ϵ

(7.76)

376 | 7 Variational inequalities Clearly, by the definition of wϵ we have that ⟨(ψ − uϵ )+ , wϵ ⟩ = ⟨(ψ − uϵ )+ , u − ψ⟩ ≥ 0 (since u ∈ C); hence, (7.76) implies that a(u − uϵ , wϵ ) ≤ 0. This is equivalent with a(wϵ + (ψ − uϵ )+ , wϵ ) ≤ 0, which is rearranged as a(wϵ , wϵ ) ≤ −a((ψ − uϵ )+ , wϵ ) and using the facts that a is continuous and coercive we have that 󵄩 󵄩 cE ‖wϵ ‖2W 1,2 (𝒟) ≤ c󵄩󵄩󵄩(ψ − uϵ )+ 󵄩󵄩󵄩W 1,2 (𝒟) ‖wϵ ‖W 1,2 (𝒟) , 0 0 0 from which follows the required bound on ‖wϵ ‖W 1,2 (𝒟) . 0

̂ in W 2,2 (𝒟) 4. Combining steps 2 and 3, we pass to the limit and since uϵ ⇀ u 1,2 and uϵ → u in W0 (𝒟), where u is a solution of (7.72) we obtain the stated regularity result.

Remark 7.6.22. From estimate (7.74), and using the elementary inequality Aψ ≤ (Aψ)+ , may further estimate a(uϵ , uϵ − PC (uϵ )) ≥ a((ψ − uϵ )+ , (ψ − uϵ )+ ) + ⟨(Aψ)+ , uϵ − PC (uϵ )⟩. Proceeding with the proof, using this modified estimate instead of (7.74), we see that it is sufficient to assume that (Aψ)+ ∈ L2 (𝒟). Furthermore, one could also include the terms ai , a0 , i = 1, . . . , d, as long as coercivity of a holds, with a little further elaboration. 7.6.4 Semilinear variational inequalities The above results may be extended to semilinear elliptic variational inequalities as well. A convenient class of such problems is the class of problems for which the nonlinearity is such that the resulting nonlinear operator A enjoys the continuity and monotonicity properties of Assumption 7.3.8. Given a matrix valued function A ∈ L∞ (𝒟; ℝd×d ) and a function f : ℝ → ℝ consider the semilinear operator A : X → X ⋆ defined by ˆ ˆ ˆ ⟨A(u), v⟩ = A(x)∇u(x) ⋅ ∇v(x)dx + a0 (x)u(x)v(x)dx − f (u(x))v(x)dx 𝒟

𝒟

𝒟

and the associated semilinear variational inequality u ∈ C : ⟨A(u), v − u⟩ ≥ 0, C = {v ∈

W01,2 (𝒟)

∀ v ∈ C,

: v(x) ≥ ψ(x), a. e. x ∈ 𝒟}.

(7.77)

As the linear part of the operator, under the uniform ellipticity Assumption 7.6.2, and appropriate conditions on a0 , definitely satisfies the properties of Assumption 7.3.8, it remains to choose f in such a way that it does not upset monotonicity and continuity. To simplify the exposition, we focus on the case of homogeneous Dirichlet conditions. Proposition 7.6.23. Assume A ∈ L∞ (𝒟; ℝd×d ) satisfies the uniform ellipticity Assumption 7.6.2, a0 ∈ L∞ (𝒟; ℝ+ ) and that f : ℝ → ℝ is Lipschitz and nonincreasing. Then the semilinear variational inequality (7.77) admits a unique solution.

7.6 Application: boundary and free boundary value problems | 377

Proof. We simply need to check whether A satisfies the conditions of Assumption 7.3.8. We break the operator into two contributions A = A0 + A1 , where A0 is the elliptic operator associated with the bilinear form (i. e., when we omit the term related to the nonlinear term f ) and A1 is the Nemitsky operator related to the function −f . The linear part A0 is (trivially) Lipschitz continuous, so we simply have to check the nonlinear part A1 . Recalling the definition of the norm of the dual space X ⋆ , we consider any v, u, w ∈ X = W01,2 (𝒟). By the Sobolev embedding theorem, v, u, w ∈ L2 (𝒟); hence, 󵄨󵄨 󵄨 󵄨󵄨⟨A1 (u) − A1 (w), v⟩󵄨󵄨󵄨 ≤

ˆ 𝒟

󵄨󵄨 󵄨󵄨 󵄨 󵄨󵄨f (u(x)) − f (w(x))󵄨󵄨󵄨 󵄨󵄨󵄨v(x)󵄨󵄨󵄨dx

≤ L‖u − w‖L2 (𝒟) ‖v‖L2 (𝒟) ≤ Lc‖u − w‖W 1,2 (𝒟) ‖v‖W 1,2 (𝒟) , 0

0

for some appropriate constant c > 0 (related to the best constant in the Poincaré inequality). This leads to 󵄩󵄩 󵄩 󵄩󵄩A1 (u) − A1 (w)󵄩󵄩󵄩(W 1,2 (𝒟))∗ = 0

sup

‖v‖W 1,2 (𝒟) =1

󵄨󵄨 󵄨 󵄨󵄨⟨A1 (u) − A1 (w), v⟩󵄨󵄨󵄨 ≤ Lc ‖u − w‖W 1,2 (𝒟) , 0

0

which is the required Lipschitz continuity result. For the monotonicity, it suffices to note that ⟨A1 (u) − A1 (w), u − w⟩ = −

ˆ 𝒟

(f (u(x) − f (w(x))(u(x) − w(x))dx ≥ 0,

since f (u(x)) − f (w(x))(u(x) − w(x)) ≤ 0, a. e. x ∈ 𝒟 by the fact that f is nonincreasing. Therefore, ⟨A(u) − A(w), u − w⟩ = ⟨A0 u − A0 w, u − w⟩ + ⟨A1 (u) − A1 (w), u − w⟩ ≥ ⟨A0 u − A0 w, u − w⟩ ≥ cE ‖u − w‖2W 1,2 (𝒟) , 0

which is the strict monotonicity condition. Since A satisfies the conditions of Assumption 7.3.8, applying the Lions–Stampacchia Theorem 7.3.9 we obtain the stated result. Example 7.6.24. The method of sub and supersolutions can be extended to the semilinear case to provide bounds for the solutions of (7.77). In particular, if a0 > cE > 0 we may show that the solution of (7.77) satisfies m := min(0,

f (0) f (0) ) ≤ u ≤ M := max(sup ψ, ). cE cE x∈𝒟

To show this, use v = max(u, m) and v = min(u, M) = u − (u − M)+ , respectively, which are both in W01,2 (𝒟), as test functions in (7.77), working essentially as in the proof of Proposition 7.6.16 along with the monotonicity of f . To illustrate the technique,

378 | 7 Variational inequalities consider the upper bound. Using the test function v = min(u, M) = u − (u − M)+ in the weak formulation we obtain ˆ ˆ A∇u∇(u − M)+ dx ≤ (f (u) − a0 u)(u − M)+ dx 𝒟 𝒟 ˆ = (f (u) − a0 u)(u − M)dx ≤ (f (0) − cE M)|𝒟| ≤ 0, {u>M}

where we used the fact that since 0 < M < u, we have cE M < cE u ≤ a0 u and since f is not increasing, that f (u) ≤ f (0), whereas by the Poincaré inequality there exists a ´ constant cp > 0 such that cp ‖(u − M)+ ‖L2 (𝒟) ≤ 𝒟 A∇u∇(u − M)+ dx, so that (u − M)+ = 0 a. e. The special case where ψ = −∞ generalizes the results of Section 6.7 on semilinear elliptic equations to operators more general than the Laplacian. ◁

7.7 Appendix 7.7.1 An elementary lemma Lemma 7.7.1. Consider the functions ψi : 𝒟̄ → ℝ, gi : 𝜕𝒟 → ℝ, such that ψi ≤ gi on 𝜕𝒟, i = 1, 2, and the sets Ci = {u ∈ W 1,2 (𝒟) : u(x) ≥ ψi (x), a. e. x ∈ 𝒟, u(x) = gi (x), x ∈ 𝜕𝒟},

i = 1, 2.

If ψ1 ≥ ψ2 and g1 ≥ g2 , then u1 ∨ u2 := max(u1 , u2 ) ∈ C1 ,

and

u1 ∧ u2 := min(u1 , u2 ) ∈ C2 ,

∀ u1 ∈ C1 , u2 ∈ C2 .

Proof. Consider any ui ∈ Ci , i = 1, 2. Letting w = max(u1 , u2 ), we claim that w ∈ C1 . For any x ∈ 𝒟 either u1 (x) ≥ u2 (x) or u1 (x) ≤ u2 (x). If x ∈ 𝒟 such that u1 (x) ≥ u2 (x), then w(x) = u1 (x) and since u1 ∈ C1 , we conclude that w(x) ≥ ψ1 . If, on the other hand, x ∈ 𝒟 such that u1 (x) ≤ u2 (x), then w(x) = u2 (x). Recalling the fact that u1 ∈ C1 , clearly, ψ1 (x) ≤ u1 (x) ≤ u2 (x) = w(x). Therefore, in any case w ≥ ψ1 . For any x ∈ 𝜕𝒟 either u1 (x) ≥ u2 (x) or u1 (x) ≤ u2 (x). In the first case, w(x) = max(u1 (x), u2 (x)) = u1 (x) = g1 (x), where the last equality comes from the fact that u1 ∈ C1 . In the second case, we have on the one hand that w(x) = max(u1 (x), u2 (x)) = u2 (x) = g2 (x) ≤ g1 (x),

(7.78)

where we used the fact that u2 ∈ C2 and that g2 ≤ g1 . On the other hand, since u1 ∈ C1 , we have that g1 (x) = u1 (x) ≤ u2 (x) = w(x); hence, w(x) ≥ g1 (x). Combining this with (7.78), we get that w(x) = g1 (x). We conclude that w ≥ ψ1 a. e. on 𝒟 and w = g1 a. e. on 𝜕𝒟, therefore, w ∈ C1 . The other claim follows in a similar fashion.

8 Critical point theory In this chapter, we focus on a deeper study of critical points for nonlinear functionals in Banach spaces, i. e., on points x ∈ X such that the Fréchet derivative of the functional vanishes. Such points may correspond to minima, maxima or saddle points of the functional. Since the Fréchet derivative of a functional can be interpreted as a nonlinear operator equation, the answer to whether a given functional admits a critical point or not has important applications to questions related to the solvability of certain nonlinear operator equations. Furthermore, if the functionals in question are integral functionals, then the study of their critical points is related to the study of nonlinear PDEs. There is a well-developed theory for the study of critical points of nonlinear functionals, with important results such as the mountain pass or the saddle point theorems, which provide conditions for the existence of critical points for a wide class of nonlinear functionals (including integral functionals). In this chapter, we provide a brief introduction to this theory, focusing on applications to integral functionals and nonlinear PDEs. Critical point theory is an important part of nonlinear analysis and there are several books or lecture notes devoted to it (see, e. g., [65, 72, 94] or [105]).

8.1 Motivation The general aim of these results is to identify a critical point for a given functional F : X → ℝ. To obtain a geometric intuition, assume that the functional can be understood as a depiction of a surface with level sets {x ∈ X : F(x) = c}. Consider two points on the surface, (x0 , F(x0 )) and (x1 , F(x1 )), as well as paths on the surface connecting them of the form (γ(⋅), F(γ(⋅))) where γ : [0, 1] → X is a continuous curve such that γ(0) = x0 and γ(1) = x1 . The function F(γ(⋅)) measures the elevation of the path on the surface from the “plane” X, whereas maxt∈[0,1] F(γ(t)) is the maximal elevation from the plane along a given path γ. On a more picturesque mood, consider the functional F as representing the surface of a mountain and a hiker wishing to go on a walk on the mountain, from point (x0 , F(x0 )) to (x1 , F(x1 )); then maxt∈[0,1] F(γ(t)) is the highest point on the mountain she has reached during her hike. Now consider scanning all possible paths γ between these two points (collected in a set of paths Γ) looking for the one such path that will minimize the maximal elevation, i. e., searching for a solution to the problem infγ∈Γ maxt∈[0,1] F(γ(t)). Quoting [93], such a path will be the optimal path between the two points since the energy required by the hiker would be minimal. Roughly speaking, if the initial and final points are two low points in the valleys of the mountain then this path must definitely (unless the mountain is flat) pass through the ridge of the mountain; hence, will pass through a critical point of the functional F. This intuitive result is in fact the basis of the majority of results in this chapter, which not surprisingly go under the general name of mountain pass theorems. Such https://doi.org/10.1515/9783110647389-008

380 | 8 Critical point theory results were obtained in the 1940s and 1950s in finite dimensions by a number of authors such as, for instance, Morse or Courant. We will not enter into details for the finite dimensional case but mention a result by Courant that essentially states that a coercive function f ∈ C 1 (ℝd ; ℝ) that possesses two distinct strict relative minima x0 , x1 possesses a third critical point x2 (distinct from the other two and not a relative minimizer) which in fact can be characterized by f (x2 ) = infA∈K maxx∈A f (x) where K = {A ⊂ ℝd : A compact and connected, x0 , x1 ∈ A}. Note that the critical point is characterised via a minimax approach.

8.2 The mountain pass and the saddle point theorems 8.2.1 The mountain pass theorem The mountain pass theorem allows us to show the existence of a critical point zo for a functional that we know it admits a strict local minimum at some point x0 , i. e., if there exists some ϵ > 0 such that F(x0 ) < F(x) for all x ∈ B(x0 , ϵ). There are various formulations and variants of the mountain pass theorem; here, we present its fundamental version as first introduced by Ambrosetti and Rabinowitz [8] (see also [65, 72]). Before stating the theorem for the convenience of the reader, we recall the Palais– Smale conditions (see Section 3.6.3). Definition 8.2.1 (Palais–Smale conditions). A functional1 F ∈ C 1 (X; ℝ) satisfies the Palais–Smale condition if any sequence {xn : n ∈ ℕ} ⊂ X such that F(xn ) is bounded and DF(xn ) → 0 has a (strongly) convergent subsequence. Theorem 8.2.2 (Ambrosetti–Rabinowitz). Let X be a Banach space and consider F ∈ C 1 (X; ℝ) satisfying the Palais–Smale condition. Assume there exists an open neighborhood 2 U of x0 and a point x1 ∈ ̸ U such that max(F(x0 ), F(x1 )) < c0 ≤ inf F(x). x∈𝜕U

Consider the set of paths connecting the points x0 , x1 ∈ X, Γ = {γ ∈ C([0, 1]; X) : γ(0) = x0 , γ(1) = x1 }. Then c := inf max F(γ(t)) ≥ c0 Γ t∈[0,1]

1 i. e., Frechet differentiable with continuous derivative. 2 For example, U = BX (x0 , ρ), for some ρ ∈ (0, ‖x1 − x0 ‖).

(8.1)

8.2 The mountain pass and the saddle point theorems | 381

is a critical value for the functional F, i. e., there exists zo ∈ X, such that DF(zo ) = 0 and F(zo ) = c. Proof. The set of paths Y = C([0, 1]; X), endowed with the norm ‖γ‖Y = maxt∈[0,1] ‖γ(t)‖X is a Banach space. Then the set Γ ⊂ Y can be turned into a complete metric space with the metric d(γ1 , γ2 ) = ‖γ1 − γ2 ‖Y . Consider the functional Ψ : Γ → ℝ defined by Ψ(γ) = maxt∈[0,1] F(γ(t)), which by the assumptions is lower semicontinuous (see also Example 2.2.2). For any γ ∈ Γ, clearly Ψ(γ) = maxt∈[0,1] F(γ(t)) ≥ max(F(x0 ), F(x1 )) so that taking the infimum over all paths we conclude that c := infΓ Ψ ≥ max(F(x0 ), F(x1 )) and the functional Ψ is bounded below. We now apply the Ekeland variational principle (Theorem 3.6.1) to Ψ. For every ϵ > 0, there exists an element of Γ ⊂ Y, let us denote it by γϵ , with the properties Ψ(γϵ ) ≤ c + ϵ,

Ψ(γ) ≥ Ψ(γϵ ) − ϵ‖γ − γϵ ‖Y ,

∀ γ ∈ Γ.

(8.2)

By standard arguments, for the function t 󳨃→ F(γϵ (t)) there exists at least a point in [0, 1] where it achieves its maximum and let Mϵ = arg maxt∈[0,1] (F ∘ γϵ )(t) be the set of all such points (depending on ϵ). Clearly, Ψ(γϵ ) = F(γϵ (tϵ )) for some tϵ ∈ Mϵ . Choosing γ(t) = γϵ (t) + δhϕ(t) for arbitrary δ > 0, h ∈ X and for some smooth function ϕ such that ϕ(0) = ϕ(1) = 0 with ϕ(t) ≡ 1 for t ∈ [k, 1 − k] for some suitable k > 0, so that ϕ is sufficiently localized around tϵ , using the continuity properties of F and DF , and working in a similar fashion as in Proposition 3.6.4 we conclude from the second condition of (8.2) that ‖DF(γϵ (tϵ ))‖X ⋆ < ϵ. We then use the Palais–Smale condition and pass to an appropriate subsequence which converges to a critical point. Remark 8.2.3. The mountain pass theorem has originally been proved by a different technique which uses the so called deformation lemma (see [72]). Example 8.2.4 (Mountain pass geometry). An example where the mountain pass theorem can be used is if there is a local minimum at x0 and another (distant) point x1 such that F(x0 ) > F(x1 ). Then the theorem guarantees the existence of a critical point for the functional. Since c ≥ c0 > max{F(x0 ), F(x1 )}, this point is different from the points x0 and x1 . ◁ Remark 8.2.5. The claim of Theorem 8.2.2 remains valid even if the strict inequality in (8.1) is replaced by the weaker condition infx : ‖x‖=ρ {F(x)} ≥ max{F(x0 ), F(x1 )}, but the proof requires a more careful consideration of the case of equality (the strict inequality case reduces to Theorem 8.2.2). For details see e. g., [65]. 8.2.2 Generalizations of the mountain pass theorem The classical mountain pass theorem admits many important generalizations in various directions. One such direction is, by replacing the paths γ : [0, 1] → X along which

382 | 8 Critical point theory we monitor the value of the functional F : X → ℝ, with more general mappings of the form γ : K → X where K is a general compact metric space. In this setup, we replace the two points {γ(0), γ(1)} on which we have information concerning the value of the functional, by information on the value of the functional on γ(K0 ) ⊂ X for a closed subset K0 ⊂ K. Then, extending the arguments used in the proof of Theorem 8.2.2, following [65] or [72], we have an interesting generalization of the mountain pass theorem. Alternative proofs of the generalized mountain pass theorem may be found. An interesting proof which still uses the Ekeland variational principle, but bypasses the machinery of convex analysis, was proposed in [29], and is the one presented below. Theorem 8.2.6. Let X be a Banach space, X ⋆ its dual, ⟨⋅, ⋅⟩ the duality pairing between X ⋆ and X, and consider F ∈ C 1 (X; ℝ). Let K be a compact metric space and K0 ⊂ K be a closed subset of K. Finally, for a given function γ0 ∈ C(K0 ; X), let Γ := {γ ∈ C(K; X) : γ = γ0 on K0 },

and

c := inf max F(γ(s)). Γ

s∈K

Suppose that max F(γ(s)) > max F(γ(s)), s∈K

s∈K0

∀ γ ∈ Γ.

(8.3)

Then there exists a sequence {xn : n ∈ ℕ} such that F(xn ) → c and ‖DF(xn )‖X ⋆ → 0. If in addition F satisfies the Palais–Smale condition then c is a critical value. Proof. We sketch the proof, which uses a perturbation argument and the Ekeland variational principle to construct a sequence of approximate critical points which converge to the required critical point. We proceed in 5 steps. 1. For arbitrary ϵ > 0, we define the perturbed functional Fϵ : Γ × K → ℝ ∪ {+∞}, by (γ, s) 󳨃→ Fϵ (γ, s) := F(γ(s)) + ϵd(s),

where d(s) = min{dist(s, K0 ), 1},

and the functional Ψϵ : Γ → ℝ ∪ {+∞}, by γ 󳨃→ Ψϵ (γ) := max Fϵ (γ(s), s). s∈K

It can be seen that by the stated assumptions Ψϵ , satisfies the conditions of the Ekeland variational principle. 2. We apply the Ekeland variatonal principle to Ψϵ to guarantee the existence of a γϵ ∈ Γ such that Ψϵ (γ) − Ψϵ (γϵ ) + ϵd(γϵ , γ) ≥ 0,

∀ γ ∈ Γ,

cϵ ≤ Ψϵ (γϵ ) ≤ cϵ + ϵ,

(8.4)

where cϵ = infγ∈Γ Ψϵ (γ). Since clearly c ≤ cϵ ≤ c + ϵ, the second inequality becomes c ≤ Ψϵ (γϵ ) ≤ c + 2ϵ.

8.2 The mountain pass and the saddle point theorems | 383

(ϵ) 3. We now define Kmax (γ) to be the set of maximizers of the perturbed functional Fϵ ∘ γ for a given γ ∈ Γ, i. e., (ϵ) Kmax (γ) := {s ∈ K : Fϵ (γ(s), s) = Ψϵ (s)} = arg max Fϵ (γ(s), s). s∈K

(ϵ) (ϵ) By assumption, Kmax (γ) ⊂ K \ K0 . We claim the existence of a sϵ ∈ Kmax (γϵ ) such that γϵ (sϵ ) is almost a critical point. In particular, we claim that

(8.5)

󵄩 󵄩 (ϵ) ∃ sϵ ∈ Kmax (γϵ ) : 󵄩󵄩󵄩DF(γϵ (sϵ ))󵄩󵄩󵄩X ⋆ ≤ 2ϵ.

Recalling that ϵ is arbitrary, we make the choice ϵ = n1 , n ∈ ℕ, to construct a sequence xn = γ1/n (s1/n ) for which F(xn ) → c and ‖DF(xn )‖X ⋆ → 0. If the Palais–Smale condition holds, then c is a critical value for F. 4. The proof of claim (8.5) follows from another claim, that of the existence of a pseudo-gradient vector field for the continuous map f := DF ∘γ : K → X ⋆ , for any fixed ϵ > 0. More precisely, we claim that for any ϵ > 0, there exists a locally Lipschitz map vϵ : K → X with the property that 󵄩󵄩 󵄩 󵄩󵄩vϵ (s)󵄩󵄩󵄩X ≤ 1,

󵄩 󵄩 and ⟨f (s), vϵ (s)⟩ ≥ 󵄩󵄩󵄩f (s)󵄩󵄩󵄩X ⋆ − ϵ,

∀ s ∈ K.

(8.6)

Assume for the time being that the claim in (8.6) holds. For any δ > 0, sufficiently small, consider γδ,ϵ ∈ Γ : γδ,ϵ (s) = γϵ (s) − δϕϵ (s)vϵ (s), where vϵ is the pseudo-gradient vector field of (8.6) and ϕϵ ∈ C(K; [0, 1]) : ϕ ≡ 1

(ϵ) on Kmax (γϵ ),

and ϕϵ ≡ 0

on K0 .

Set γ = γδ,ϵ in (8.4), and calculate the result at a point sδ,ϵ ∈ arg maxs∈K Ψϵ (γδ,ϵ ) ≠ 0, so that Ψϵ (γδ,ϵ (sδ,ϵ )) = Fϵ (γδ,ϵ (sδ,ϵ )) to obtain F(γϵ (sδ,ϵ ) − δϕϵ (sδ,ϵ )vϵ (sδ,ϵ )) + ϵd(sδ,ϵ ) − Ψϵ (γϵ ) + ϵδ ≥ 0.

(8.7)

Using the fact that F ∈ C 1 (K; ℝ), we have (γϵ (sδ,ϵ ) − δϕϵ (sδ,ϵ )vϵ (sδ,ϵ )) = F(γϵ (sδ,ϵ )) − δ⟨DF(γϵ (sδ,ϵ )), ϕϵ (sδ,ϵ )vϵ (sδ,ϵ )⟩ + o(δ), while, clearly, F(γϵ (sδ,ϵ )) + ϵd(sδ,ϵ ) = Fϵ (γϵ (sδ,ϵ ), sδ,ϵ ) ≤ max Fϵ (γϵ (s), s) = Ψϵ (γϵ ), s∈K

we conclude by (8.7) that ⟨DF(γϵ (sδ,ϵ )), δϕϵ (sδ,ϵ )vϵ (sδ,ϵ )⟩ ≤ ϵ + o(1).

(8.8)

384 | 8 Critical point theory Recall that K is compact. We choose a subsequence δn → 0 such that sδn ,ϵ → sϵ for (ϵ) some sϵ ∈ Kmax and pass to the limit in (8.8) using the properties (8.6) of the pseudogradient vector field vϵ to prove our initial claim (8.5). 5. It remains to prove that a pseudo-gradient vector field can be constructed. This can be done via a partition of unity approach as follows. Consider any continuous map f : K → X ⋆ and fix any s ∈ K. By the definition of ‖f (s)‖X ⋆ = sup{⟨f (s), x⟩ : ‖x‖X ≤ 1} for any ϵ > 0, we may find xϵ (s) ∈ X, with ‖xϵ (s)‖X ≤ 1 such that ⟨f (s), xϵ (s)⟩ ≥ ‖f (s)‖X ⋆ − ϵ. By the continuity of f , there exists a neighborhood N(s) of s ∈ K such that this inequality holds in the whole neighborhood, i. e., ⟨f (s󸀠 ), xϵ (s)⟩ ≥ ‖f (s󸀠 )‖X ⋆ − ϵ for every s󸀠 ∈ N(s). Consider the open cover C = {N(s) : s ∈ K} of K which by compactness admits a locally finite refinement {Ni : i = 1, . . . M} such that for any i = 1, . . . , M there exists a set {si : i = 1, . . . , M} ⊂ K with the corresponding neighborhoods N(si ) having the property Ni ⊂ N(si ). For each (fixed) si ∈ K, we can work as above and obtain a local pseudo-gradient vector xi which works for every s󸀠 ∈ Ni . Then we may construct the pseudo-gradient vector field by patching up these local vectors, as follows. Define d (s) the functions, ρi : K → [0, 1], by ρi (s) = M i , where di (s) = dist(s, K \ Ni ), which ∑j=1 dj (s)

are locally Lipshitz continuous and the vector field v : K → X by v(s) = ∑M i=1 ρi (s)xi which essentially assigns to any s the vector xi as long as s ∈ Ni . This mapping has the desired properties.

8.2.3 The saddle point theorem An interesting extension of the mountain pass theorem is the saddle point theorem. Theorem 8.2.7. Let X be a Banach space, and F ∈ C 1 (X; ℝ) a functional satisfying the Palais–Smale condition. Let X1 ⊂ X be a finite dimensional subspace and X2 the complement of X1 , i. e., X = X1 ⊕ X2 . Assume that there exists r > 0, such that max F(x) ≤ a < b ≤ inf F(x) x∈X2

x∈𝜕X1 (r)

where X1 (r) := X1 ∩ B(0, r) and 𝜕X1 (r) = {x ∈ X1 , ||x|| = r}. Let Γ = {γ ∈ C(X1 (r); X) : γ(x) = x, ∀ x ∈ 𝜕X1 (r)}, and define c := inf sup F(γ(x)). γ∈Γ

x∈X1 (r)

Then, c > −∞ and is a critical value of the functional F.

(8.9)

8.3 Applications in semilinear elliptic problems | 385

Proof. Let K = X1 (r) and K0 = 𝜕X1 (r). These are compact sets, since X1 is finite dimensional. To apply the general minimax Theorem 8.2.6, we need to show that max F(γ(t)) > max F(γ(t)), t∈K

t∈K0

∀γ ∈ Γ.

Note that now t is interpreted as x ∈ X1 , i. e., as a vector in ℝm , where m = dim(X1 ). With our choice for K and K0 , we have that max F(γ(t)) = max F(γ(t)) ≤ a. t∈K0

x∈𝜕X1 (r)

It suffices to prove that there exists xo ∈ X1 (r) such that γ(xo ) ∈ X2 , since by assumption infX2 F ≥ b, and evidently maxt∈K F(γ(t)) ≥ infX2 F ≥ b. Let PX1 be the linear projection onto X1 . In order to show that there exists xo ∈ X1 (r) such that γ(xo ) ∈ X2 , it is equivalent to show that PX1 γ : X1 (r) → X1 has a zero.3 Note also that the map PX1 γ|𝜕X1 (r) = I, the identity map, since by construction of Γ, γ(x) = x for x ∈ 𝜕X1 (r), therefore, ⟨PX1 γ(x), x⟩ ≥ 0 for all ‖x‖ = r. Then, by Brouwer’s fixed-point theorem (and in particular Corollary 3.2.6) there exists a point xo , ‖xo ‖ ≤ r such that PX1 γ(xo ) = 0.

8.3 Applications in semilinear elliptic problems As usual, let 𝒟 ⊂ ℝd be a sufficiently smooth bounded domain, and consider the nonlinear elliptic PDE −Δu(x) = λu + f (x, u), u(x) = 0

x ∈ 𝜕𝒟.

x∈𝒟

(8.10)

where λ ∈ ℝ is an appropriate parameter and f : 𝒟 × ℝ → ℝ is a given function. As already seen in Chapter 6 and in particular Section 6.7, under certain conditions, equation (8.10) can be understood as the Euler–Lagrange equation of the functional F : X := W01,2 (𝒟) → ℝ where 1󵄨 󵄨2 λ F(u) = ∫( 󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 − u(x)2 − Φ(x, u(x))) dx, 2 2 𝒟 u

(8.11)

Φ(x, u) = ∫ f (x, s)ds. 0

therefore, the problem of solvability for equation (8.10) can be understood as equivalent to searching for critical points for the functional F. 3 Recall that y ∈ X2 is equivalent to PX1 y = 0.

386 | 8 Critical point theory Using the direct method of the calculus of variations under appropriate conditions on f , we have shown in Section 6.7 the existence of minima for F, which in turn are obtained in terms of the Euler–Lagrange equation as solutions of the PDE (8.10). However, not all solutions to (8.10) can be characterized as minima of F. Therefore in this chapter, having obtained some familiarity with the study of critical points for functionals, we will take a complementary route, and try to show how we can obtain a critical point of the functional (8.11) thus providing a solvability result for (8.10). The general tools developed in the previous section will provide valuable help in this direction, thus leading to a better understanding of solutions of equations of the form (8.10). We will show that by varying the conditions on the nonlinearity f different tools such as the mountain pass or the saddle point theorem will become handy. As we will see the value of the parameter λ ∈ ℝ will play an important role in our study and in particular its value compared with the eigenvalues of the Laplacian operator −Δ, in the aforementioned Sobolev space setting which correspond to the discrete set of solutions of the problem −Δu = λu in W01,2 (𝒟). Recalling the discussion in Section 6.2.3, this problem admits a set of solutions {λn : n ∈ ℕ}, 0 < λ1 ≤ λ2 ≤ ⋅ ⋅ ⋅ ≤ λn ≤ ⋅ ⋅ ⋅ and λn → ∞ as n → ∞, with corresponding eigenfunctions {un : n ∈ ℕ}. Of special interest is the comparison of λ with the first eigenvalue of the Laplacian on 𝒟 with Dirichlet boundary conditions, λ1 > 0. The reader should note that the importance of the spectrum of the Laplacian in the treatment of (8.10) arises from the fact that if λ = λn for some n ∈ ℕ, then the operator A = −Δ − λn I is not boundedly invertible, so that the approach of expressing (8.10) in terms of the compact operator T = (−Δ − λI)−1 and using a fixed-point scheme to construct solutions (as done for instance in Section 7.6) cannot be applied at least in a straightforward fashion. Problems of the type (8.10) when λ = λn for some n ∈ ℕ will be called resonant problems. The application of critical point theory in semilinear problems is a vast field, and such techniques are now classical. Our approach is mainly based on results in [7, 8, 10, 16, 36, 49, 72, 94, 105].

8.3.1 Superlinear growth at infinity We first study the case where f is a nonlinear function, satisfying a superlinear growth condition at infinity and slower than linear growth at 0 (so that for small values of the solution the linear problem is a good approximation to the semilinear problem). These conditions are formulated in the assumption below (in the case where the dimension of the domain is d ≥ 3).

8.3 Applications in semilinear elliptic problems | 387

Assumption 8.3.1. (i) f : 𝒟 × ℝ → ℝ is a Carathéodory function which for every (x, s) ∈ 𝒟 × ℝ satisfies 2d the growth condition |f (x, s)| ≤ c|s|r + α1 (x) where 1 < r < s2 − 1 with4 s2 = d−2 and α1 ∈ Lr (𝒟), with 1r + ⋆

1 r⋆

= 1.

= 0 uniformly in x ∈ 𝒟. (ii) f has the following behavior for small s: lims→0 f (x,s) |s| (iii) f is superquadratic for large s: There exist a > 2 and R > 0 such that 0 < aΦ(x, s) ≤ sf (x, s) for |s| ≥ R, uniformly in x. A function satisfying this assumption is, e. g., f (x, s) = s3 . In the case where d = 1, 2, the growth condition in Assumption 8.3.1(i) is not needed. Proposition 8.3.2. Under Assumption 8.3.1, if λ < λ1 , where λ1 > 0 is the first eigenvalue of the Dirichlet–Laplacian on 𝒟, the nonlinear elliptic problem (8.10) admits a positive nontrivial solution. Proof. We will apply the mountain pass theorem to show that F admits a critical point which is the solution we seek. For that, we need to check that F satisfies the conditions for the mountain pass theorem to hold. In particular, we need to show that F is C 1 , satisfies the Palais–Smale condition, as well as the existence of a u1 ∈ X and a ρ > 0 such that 0 = max{F(0), F(u1 )} < infu∈X, ‖u‖=ρ F(u). In particular, we will show that 0 is a strict local minimum for F (see Example 8.2.4). Then an application of the mountain pass theorem (see Theorem 8.2.2) will guarantee the existence of a critical point, different than 0; hence, a nontrivial solution. We proceed in 4 steps, leaving the verification of the Palais–Smale condition last. 1. On account of the growth assumption 8.3.1(i), the functional F : W01,2 (𝒟) → ℝ is C 1 (see the proof of Proposition 6.7.6). 2. We now identify u = 0 as a strict local minimum of F. To see this, we first note the conditions on the nonlinearity f imply that for every ϵ > 0 there exists δ such that |f (x, s)| < ϵ|s| for |s| < δ which upon integration implies that |Φ(x, s)| ≤ 21 ϵ|s|2 for |s| < δ, which provides information on the Φ for small s. In particular, this estimate yields F(0) = 0. For the large s behavior, we use Assumption 8.3.1(i) to remark that |Φ(x, s)| < c󸀠 |s|r+1 for some c󸀠 > 0, and for |s| > δ. Therefore, for any s ∈ ℝ it holds that |Φ(x, s)| < 21 ϵ|s|2 + c󸀠 |s|r+1 . Using this estimate for Φ allows us to provide a lower bound for the functional F as follows: 1 λ+ϵ F(u) ≥ ‖∇u‖2L2 (𝒟) − ‖u‖2L2 (𝒟) − c󸀠 ‖u‖r+1 Lr+1 (𝒟) . 2 2 Note that the second and the third term are subordinate to the first term. Recall the Poincaré inequality according to which ‖∇u‖2L2 (𝒟) ≥ λ1 ‖u‖2L2 (𝒟) , where λ1 > 0 is the first 4 s2 is the critical Sobolev exponent for the embedding W01,2 (𝒟) 󳨅→ Ls2 (𝒟) to hold, see Theorem 1.5.11(i).

388 | 8 Critical point theory eigenvalue of the operator −Δ with homogeneous Dirichlet boundary conditions on 𝒟, and the Sobolev embedding according to which for the choice of r there exists c1 such that ‖u‖Lr+1 (𝒟) ≤ c1 ‖∇u‖L2 (𝒟) . This leads to a lower bound for F of the form 1 λ+ϵ F(u) ≥ (1 − )‖∇u‖2L2 (𝒟) − c2 ‖∇u‖r+1 L2 (𝒟) . 2 λ1

(8.12)

Since λ < λ1 , ϵ > 0 is arbitrarily small, and r > 1 there exists a ϱ > 0 such that for any u satisfying ‖∇u‖L2 (𝒟) < ϱ, F(u) > 0 = F(0); hence, 0 is a strict local minimum. 3. We now try to show the existence of a u1 ∈ X such that F(u1 ) < F(0). It suffices to show that for any u0 ≠ 0, it holds that F(ρu0 ) → −∞ as ρ → ∞, thus (moving along this radial direction) guaranteeing the existence of a u1 with the required properties. Indeed, by Assumption 8.3.1(iii) and upon integrating5 we see that for large enough |s| > R, Φ(x, s) ≥ |s|a + c3 , for some constant c3 . On the other hand, by the growth Assumption 8.3.1(i) and upon integrating, we see that for any s it holds that Φ(x, s) ≥ cr |s|r+1 + a1 (x)s, so that for any s ∈ ℝ, there exist two constants c4 > 0 and c5 such that Φ(x, s) ≥ c4 |s|a + c5 . Using this estimate, we obtain the upper bound for the functional λ 1 F(u) ≤ ‖∇u‖2L2 (𝒟) − ‖u‖2L2 (𝒟) − c4 ‖u‖aLa (𝒟) − c5 |𝒟|. 2 2

(8.13)

Let us choose a u0 such that ‖∇u0 ‖L2 (𝒟) = 1. By the Sobolev embedding ‖u0 ‖La (𝒟) ≤ c󸀠 ‖∇u0 ‖L2 (𝒟) and ‖u0 ‖L2 (𝒟) ≤ c󸀠 ‖∇u0 ‖L2 (𝒟) , which implies that β := ‖u0 ‖La (𝒟) < ∞ and γ := ‖u0 ‖L2 (𝒟) < ∞. Now consider u = ρ u0 . The upper bound for F implies that 1 λ F(ρu0 ) ≤ ρ2 − ρ2 γ 2 − c4 ρa βa − c5 |𝒟| → −∞, 2 2

as ρ → ∞,

since a > 2, as claimed. By the estimates above, we have the existence of a u1 ∈ X and a ρ > 0 such that 0 = max{F(0), F(u1 )} < infu∈X, ‖u‖=ρ F(u). If F satisfies the Palais–Smale condition, then an application of the mountain pass theorem (see Theorem 8.2.2) will guarantee the existence of a critical point, different than 0; hence, a nontrivial solution. 4. It therefore remains to check the Palais–Smale condition. Consider a Palais– Smale sequence un . We first note that ‖un ‖X = ‖∇un ‖L2 (𝒟) is bounded for any n. Indeed, assume that the sequence un satisfies |F(un )| ≤ c󸀠 and DF(un ) → 0. Then, for large n, it holds that 󵄨󵄨 󵄨 󵄩 󵄩 󵄨󵄨⟨DF(un ), un ⟩󵄨󵄨󵄨 ≤ 󵄩󵄩󵄩DF(un )󵄩󵄩󵄩X ⋆ ‖un ‖X ≤ ϵ‖un ‖X , for large enough n. Therefore, −ϵ‖∇un ‖L2 (𝒟) ≤ ⟨DF(un ), un ⟩ 5 Rewrite by Assumption 8.3.1(iii) as 0 < aΦ ≤ sΦ󸀠 .

8.3 Applications in semilinear elliptic problems | 389

= ‖∇un ‖2L2 (𝒟) − λ‖un ‖2L2 (𝒟) − ∫ f (x, un (x)) un (x)dx ≤ ϵ‖∇un ‖L2 (𝒟) , 𝒟

so that using the lower bound, − ∫ f (x, un (x)) un (x)dx ≥ −‖∇un ‖2L2 (𝒟) − ϵ‖∇un ‖L2 (𝒟) + λ‖un ‖2L2 (𝒟) .

(8.14)

𝒟

On the other hand, 1 λ F(un ) = ‖∇un ‖2L2 (𝒟) − ‖un ‖2L2 (𝒟) − ∫ Φ(x, un (x))dx ≤ c󸀠 . 2 2 𝒟

We rearrange λ 1 F(un ) = ‖∇un ‖2L2 (𝒟) − ‖un ‖2L2 (𝒟) 2 2 1 1 − ∫(Φ(x, un (x)) − f (x, un (x))un (x))dx − ∫ f (x, un (x))un (x)dx a a 𝒟

𝒟

1 1 1 1 ≥ ( − )‖un ‖2X − λ( − )‖un ‖2L2 (𝒟) − ϵ‖un ‖X 2 a 2 a 1 − ∫(Φ(x, un (x)) − f (x, un (x))un (x))dx, a

(8.15)

𝒟

where we used the estimate (8.14). Recalling Assumption 8.3.1(iii), upon defining for any fixed n ∈ ℕ, 𝒟n,+ = {x ∈ 𝒟 : 󵄨󵄨󵄨un (x)󵄨󵄨󵄨 > R},

󵄨

󵄨

𝒟n,− = {x ∈ 𝒟 : 󵄨󵄨󵄨un (x)󵄨󵄨󵄨 ≤ R},

󵄨

󵄨

we see that − ∫(Φ(x, un (x)) − 𝒟

1 f (x, un (x))un (x))dx a

≥ − ∫ (Φ(x, un (x)) − 𝒟n,−

1 f (x, un (x))un (x))dx ≥ c0 , a

for some constant c0 independent of n, where for the first estimate we used Assumption 8.3.1(iii) and for the second we used the fact that on 𝒟n,− , |un | < R (which is independent of n) and, therefore, both f and Φ are also bounded by a bound independent of n. Using this last estimate in (8.15), along with the facts that F(un ) ≤ c󸀠 , we see that 1 1 1 1 c󸀠 ≥ F(un ) ≥ ( − )‖un ‖2X − λ( − )‖un ‖2L2 (𝒟) − ϵ‖un ‖X + c0 . 2 a 2 a

390 | 8 Critical point theory Since a > 2, this implies using the Poincaré inequality that, if λ > 0, λ 1 1 c󸀠 ≥ ( − )(1 − )‖un ‖2X − ϵ‖un ‖X + c0 , 2 a λ1 or if λ < 0 that c󸀠 ≥ (

1 1 λ − )(1 + )‖un ‖2X − ϵ‖un ‖X + c0 . α 2 λ1

so, in any case, since ϵ > 0 is arbitrary, a > 2, and λ < λ1 , we conclude the existence of a constant c1 (independent of n) such that ‖un ‖X < c1 . By the reflexivity of X, there exists a weakly converging subsequence unk ⇀ ū for some ū ∈ X. By the compact embedding of X = W01,2 (𝒟) into Lr+1 (𝒟), there exists a further subsequence (denoted the same) so that unk → ū in Lr+1 (𝒟). This (taking into account the growth conditions on f ) implies that the operator K : X → X ⋆ defined by u 󳨃→ f (x, u) is compact. We express DF as DF(u) = Lu + Ku, with L = −Δ − λI : X → X ⋆ being boundedly invertible for λ < λ1 . Consider a Palais–Smale sequence {un : n ∈ ℕ}, i. e., Lun + Kun → 0. This is expressed as un + L−1 K(un ) → 0, so that since L−1 K is compact and {un : n ∈ ℕ} is bounded, then {L−1 K(un ) : n ∈ ℕ} has a convergent subsequence (denoted the same) and since un + L−1 K(un ) → 0 we conclude that {un : n ∈ ℕ} has a convergent subsequence as well, hence, the Palais–Smale property holds. 8.3.2 Nonresonant semilinear problems with asymptotic linear growth at infinity and the saddle point theorem We now consider again the elliptic problem (8.10), dropping the condition of superlinear growth for the nonlinearity at infinity, assuming instead that f has asymptotically linear growth at infinity (with respect to s). To simplify the exposition and the arguments, we assume a slightly modified version of (8.10) in the form −Δu = λu + f0 (u) + h(x), u = 0,

x ∈ 𝜕𝒟,

x ∈ 𝒟,

(8.16)

where λ ∈ ℝ is a given parameter. We will focus first to the case where λ ≠ λn where {λn : n ∈ ℕ} are the eigenvalues of the Dirichlet–Laplacian. We assume that d ≥ 3. We impose the following conditions on f0 and h. Assumption 8.3.3. (i) The function f0 : ℝ → ℝ satisfies the growth condition |f0 (s)| ≤ c|s|r + α1 for every ⋆ 2d and h ∈ Lr (𝒟). s ∈ ℝ where 1 < r < s2 − 1 with6 s2 = d−2 (ii) lims→±∞

f (x,s) s

= γ± .

6 s2 is the critical Sobolev exponent for the embedding W01,2 (𝒟) 󳨅→ Ls2 (𝒟) to hold, see Theorem 1.5.11(i).

8.3 Applications in semilinear elliptic problems | 391

In the cases d = 1, 2, the growth condition in Assumption 8.3.3 is not required. Proposition 8.3.4. Let f0 , h, satisfy Assumption 8.3.3 and assume that λk < λ+γ± < λk+1 , for some k ≥ 1 where {λn : n ∈ ℕ} are the eigenvalues of the Dirichlet–Laplacian on 𝒟. ⋆ Then (8.16) has a nontrivial weak solution, for every h ∈ Lr (𝒟). Proof. We will look for weak solutions of 8.16 which will be identified as critical points of the integral functional F : X = W01,2 (𝒟) → ℝ defined in (8.11) for f (x, s) = f0 (s) + h(x). We will use the saddle point Theorem 8.2.7, identifying a splitting of X = X1 ⊕ X2 , with X1 finite dimensional such that the condition (8.9) required by this theorem is satisfied. Since λk < λ + γ± < λk+1 , we propose to choose X1 := span{ϕ1 , . . . , ϕk },

X2 := span{ϕi : i ≥ k + 1},

(8.17)

where ϕi are the eigenfunctions of the Dirichlet–Laplacian normalized so that ‖∇ϕn ‖2L2 (𝒟) = 1,

and,

‖ϕn ‖2L2 (𝒟) =

1 , λn

n ∈ ℕ.

(8.18)

These are orthogonal with respect to the inner product in L2 (𝒟) (see Proposition 6.2.12), so that X = X1 ⊕ X2 . We intend to show that this choice actually satisfies (8.9). We proceed in 6 steps, leaving the verification of the Palais–Smale condition last. 1. By the growth condition in Assumption 8.3.3(i), this functional is C 1 (X; ℝ) (see the proof of Proposition 6.7.6). 2. Since λk < λ+γ± < λk+1 , let us choose μ1 , μ2 such that λk < μ1 < λ+γ± < μ2 < λk+1 , f (s) so that by Assumption 8.3.3(ii), there exists R > 0 such that μ1 < λ + 0s < μ2 for λ 2 1 1 2 󸀠 2 󸀠 |s| > R,. Upon integration 2 μ1 s + c1 ≤ 2 s + Φ0 (s) ≤ 2 μ2 s + c2 for every s ∈ ℝ, where Φ0 is the antiderivative of f0 and c1󸀠 , c2󸀠 are two appropriate constants. 3. Consider now any v ∈ X1 . This can be expressed as v = ∑ki=1 ci ϕi for some ci ∈ ℝ, so that by linearity −Δv = ∑ki=1 λi ci ϕi . Multiplying by v, integrating by parts and using the orthogonality of the ϕi in L2 (𝒟), we conclude that k

󵄨 󵄨2 ∫󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 dx = ∑ λi ci2 ‖ϕi ‖2L2 (𝒟) ≤ λk ‖v‖2L2 (𝒟) , 𝒟

i=1

where for the last estimate we used the fact that λi ≤ λk for i = 1, . . . , k. With this estimate at hand, we see that for every u ∈ X1 , 1 1 F(v) ≤ ‖∇v‖2L2 (𝒟) − μ1 ‖v‖2L2 (𝒟) − c1󸀠 |𝒟| + ‖h‖L2 (𝒟) ‖v‖L2 (𝒟) 2 2 μ1 1 ≤ (1 − )‖∇v‖2L2 (𝒟) − c1󸀠 |𝒟| + ‖h‖L2 (𝒟) ‖v‖L2 (𝒟) 2 λk μ 1 1 ≤ (1 − 1 )‖∇v‖2L2 (𝒟) − c1󸀠 |𝒟| + ‖h‖L2 (𝒟) ‖∇v‖L2 (𝒟) , 2 λk λ1

392 | 8 Critical point theory where for the last estimate we used the Poincaré inequality ‖∇v‖2L2 (𝒟) ≥ λ1 ‖v‖2L2 (𝒟) ,

which is valid for any v ∈ W01,2 (𝒟). Since μ1 > λk the above estimate implies that

F(v) → −∞ for any v ∈ X1 with ‖v‖X = ‖∇v‖L2 (𝒟) → ∞.

4. Now consider any w ∈ X2 . This can be expressed as w = ∑∞ i=k+1 ci ϕi and by

similar arguments as above we can see that for any w ∈ X2 , it holds that ‖∇w‖2L2 (𝒟) ≥ λk+1 ‖w‖2L2 (𝒟) . We then estimate F from below as

1 1 F(w) ≥ ‖∇w‖2L2 (𝒟) − μ2 ‖w‖2L2 (𝒟) − c2󸀠 |𝒟| − ‖h‖L2 (𝒟) ‖w‖L2 (𝒟) 2 2 μ 1 ≥ (1 − 2 )‖∇w‖2L2 (𝒟) − c2󸀠 |𝒟| − ‖h‖L2 (𝒟) ‖w‖L2 (𝒟) 2 λk+1

μ 1 1 ≥ (1 − 2 )‖∇w‖2L2 (𝒟) − c2󸀠 |𝒟| − ‖h‖L2 (𝒟) ‖∇w‖L2 (𝒟) , 2 λk+1 λ1

where for the last estimate we used the Poincaré inequality. Since μ2 < λk+1 the above estimate implies that F(w) → ∞ for any w ∈ X2 with ‖w‖X = ‖∇w‖L2 (𝒟) → ∞, and also provides a finite lower bound for F(w) for any w ∈ X2 .

5. The facts that F(v) → −∞ for any v ∈ X1 with ‖v‖X = ‖∇v‖L2 (𝒟) → ∞, while

F(w) has a finite lower bound for any w ∈ X2 , implies the existence of a ρ > 0 such

that maxv∈X1 ,

‖v‖=ρ F(v)

< infw∈X2 F(w), so that if the Palais–Smale condition holds, we

may apply the saddle point Theorem 8.2.7 and conclude the existence of a nontrivial critical point for F, which corresponds to the solution we seek.

6. It only remains to check the validity of the Palais–Smale condition. By similar

arguments as in the proof of Proposition 8.4.2, it suffices to check that any Palais– Smale sequence is bounded in X = W01,2 (𝒟). We argue by contradiction. Consider

any Palais–Smale sequence {un : n ∈ ℕ}, i. e., a sequence such that F(un ) < c󸀠 and ‖DF(un )‖X ⋆ → 0 while at the same time ‖un ‖ → ∞ as n → ∞. Define ψn =

un ‖un ‖X

which

satisfies ‖ψn ‖X = 1 for every n. Since ‖DF(un )‖X ⋆ → 0, we have that for any ϵ > 0 and

any ϕ ∈ X = W01,2 (𝒟),

−ϵ‖ϕ‖X ≤ ∫(∇un (x) ⋅ ∇ϕ(x)dx − λun (x)ϕ(x)dx − f0 (un )ϕ(x) − h(x)ϕ(x))dx ≤ ϵ‖ϕ‖X , 𝒟

for n large enough. Dividing through by ‖un ‖X this leads to the conclusion that −ϵ

‖ϕ‖X ≤ ∫(∇ψn (x) ⋅ ∇ϕ(x) − λψn (x)ϕ(x) ‖un ‖X 𝒟

−

‖ϕ‖X 1 [f (u (x))ϕ(x) + h(x)ϕ(x)])dx ≤ ϵ , ‖un ‖X 0 n ‖un ‖X

(8.19)

8.3 Applications in semilinear elliptic problems | 393

so that lim ∫(∇ψn (x) ⋅ ∇ϕ(x)dx − λψn (x)ϕ(x)dx

n→∞

(8.20)

𝒟

−

1 ‖un ‖X

f0 (un (x))ϕ(x) −

1 ‖un ‖X

h(x)ϕ(x))dx = 0.

Since ‖ψn ‖X = 1 for every n, the sequence {ψn : n ∈ ℕ} is uniformly bounded in X with respect n, so by reflexivity of X there exists a subsequence (denoted the same for simplicity) such that ψn ⇀ ψ in X, by the compact embedding of X = W01,2 (𝒟) a further subsequence such that ψn → ψ in L2 (𝒟) and a further subsequence such that f (u (x)) ψn → ψ a. e. in 𝒟. We express the term ‖u1‖ f0 (un (x)) = ψn (x) 0u n(x) and note that in n X

n

the limit as n → ∞ we have that7 ‖u1‖ f0 (un ) → γ+ ψ+ +γ − ψ− a. e. in 𝒟. By the Lebesgue n X dominated convergence theorem passing to the limit in (8.20), we conclude that ∫(∇ψ(x) ⋅ ∇ϕ(x)dx − (λψ(x) + γ+ ψ+ (x) + γ− (x)ψ− (x))ϕ(x)dx = 0

∀ ϕ ∈ W01,2 (𝒟),

𝒟

which is the weak form for the elliptic problem −Δψ = (λ + γ+ )ψ+ + (λ + γ− )ψ− ψ = 0,

on 𝜕𝒟.

in 𝒟,

We claim that since λk < λ + γ± < λk+1 this implies that ψ = 0. This can be seen as follows: The elliptic problems −Δu = (λ + γ± )u admit only the trivial solution u = 0. The function m(x, ψ) defined by m(x, ψ(x)) = (λ + γ+ )ψ+ (x) + (λ + γ− )ψ− (x) satisfies the inequality min(λ + γ− , λ + γ+ )ψ(x) ≤ m(x, ψ(x)) ≤ max(λ + γ− , λ + γ+ )ψ(x) a. e. x ∈ 𝒟, so that by the comparison principle for elliptic problems (see, e. g., Proposition (6.2.6)) we see that ψ = 0. We now set ϕ = ψn in (8.19) and dividing by ‖un ‖X we obtain that 󵄨󵄨 󵄨󵄨 ‖ψ ‖ 1 1 󵄨 󵄨󵄨 󵄨󵄨 󵄨2 2 f0 (un (x))ψn (x) − h(x)ψn (x))dx 󵄨󵄨󵄨 ≤ ϵ n X , 󵄨󵄨∫(󵄨󵄨∇ψn (x)󵄨󵄨󵄨 − λψn (x) − 󵄨 󵄨󵄨 ‖un ‖X ‖un ‖X ‖un ‖X 󵄨 𝒟

which, recalling that ‖∇ψn ‖L2 (𝒟) = ‖ψn ‖X = 1, implies that 󵄨󵄨 󵄨󵄨 ‖ψ ‖ 1 1 󵄨 󵄨󵄨 2 f0 (un (x))ψn (x) + h(x)ψn (x))dx 󵄨󵄨󵄨 ≤ ϵ n X , 󵄨󵄨1 − ∫(λψn (x) + 󵄨󵄨 󵄨󵄨 ‖un ‖X ‖un ‖X ‖un ‖X 𝒟

7 To see this, simply note that since ‖un ‖X → ∞ either un (x) → ∞ in which case ψn (x) ≥ 0 and 1 f (u (x)) → γ+ ψ+ (x) or un (x) → −∞ in which case ψn (x) ≤ 0 and ‖u 1‖ f0 (un (x)) → γ− ψ− (x). ‖u ‖ 0 n n X

n X

394 | 8 Critical point theory which is further rearranged as 󵄨󵄨 󵄨󵄨 f (u (x)) ‖ψ ‖ 1 󵄨󵄨 󵄨 )ψn (x)2 + h(x)ψn (x)]dx 󵄨󵄨󵄨 ≤ ϵ n X . 󵄨󵄨1 − ∫[(λ + 0 n 󵄨󵄨 󵄨󵄨 un (x) ‖un ‖X ‖un ‖X 𝒟

f (u (x)

n → λ + γ± a. e. Passing to the limit as n → ∞ and keeping in mind that λ + 0u (x)) n depending on whether un (x) → +∞ or un (x) → −∞, while ψn (x) → 0 a. e., we conclude that 1 ≤ 0 which is a contradiction. Hence, any Palais–Smale sequence must be bounded. This concludes the proof.

8.3.3 Resonant semilinear problems and the saddle point theorem We now turn our attention to the resonant case λ = λk . We know, even in the linear case, that resonant problems may not have a solution unless additional conditions hold (see Section 7.6.2). As a quick illustration of that, consider the linear problem −Δu(x) = λk u(x) + h(x), u(x) = 0,

on 𝜕𝒟,

in 𝒟,

where λk is an eigenvalue of the operator −Δ with homogeneous Dirichlet boundary conditions and h is a known function. As one can see very easily, by multiplying with the corresponding eigenfunction ϕk and integrating over all 𝒟, the above problem admits a solution only for these functions h that satisfy the compatibility condition ∫𝒟 h(x)ϕk (x)dx = 0, or put otherwise are orthogonal in terms of the standard inner product of L2 (𝒟) to the k eigenfunction ϕk . Such compatibility conditions are expected to be needed also for the nonlinear resonant case, and in fact they do as we will see in the following discussion and the proposition that follows. Consider again the elliptic problem (8.16) in the resonant case λ = λk , −Δu = λk u + f (x, u), u = 0,

x ∈ 𝜕𝒟,

x ∈ 𝒟,

(8.21)

where λk is an eigenvalue of the Laplacian on 𝒟 with homogeneous Dirichlet boundary conditions. We need to impose the following assumptions on f . Assumption 8.3.5. We assume that f : 𝒟 × ℝ → ℝ satisfies the following properties: (i) f is continuous and there exists a constant c1󸀠 ≥ 0 such that |f (x, s)| ≤ c1󸀠 for every x ∈ 𝒟, s ∈ ℝ. s (ii) ∫𝒟 Φ(x, s)dx → ∞ as |s| → ∞. where Φ(x, s) = ∫0 f (x, σ)dσ. Proposition 8.3.6. Under Assumption 8.3.5, the resonant problem (8.21) admits a nontrivial weak solution.

8.3 Applications in semilinear elliptic problems | 395

Proof. The proof uses the saddle point Theorem 8.2.7. It follows upon the steps of the proof of Proposition 8.3.4, and we use the same notation as there. We will also use the same splitting as in (8.17), i. e., X1 := span{ϕ1 , . . . , ϕk },

X2 := span{ϕi : i ≥ k + 1},

where ϕi are the eigenfunctions of the Dirichlet–Laplacian, properly normalized as in (8.18). We proceed in 3 steps leaving the verification of the Palais–Smale condition last. 1. It is straightforward to show that F is bounded below on X2 . Indeed, for any ∞ w = ∑∞ i=k+1 ci ϕi ∈ X2 we have by linearity that −Δw − λk w = ∑i=k+1 (λi − λk )ci ϕi , and multiplying with w and integrating by parts, using the orthogonality of the eigenfunctions ϕi in L2 (𝒟), we obtain that 󵄨 󵄨2 ∫(󵄨󵄨󵄨∇w(x)󵄨󵄨󵄨 − λk w(x)2 )dx 𝒟 ∞

= ∑ ci2 (λi − λk ) ∫ ϕi (x)2 dx i=k+1 ∞

= ∑ (1 − i=k+1

(8.22)

𝒟 ∞ λk 2 λ λ )ci ≥ (1 − k ) ∑ ci2 = (1 − k )‖w‖2X , λi λk+1 i=k+1 λk+1

where for the second equality we used the fact that8 ‖ϕi ‖2L2 (𝒟) = 9

that λi ≥ λk+1 for all i = k + 1, . . .. We now estimate F(w) as

1 , λi

and then the fact

1󵄨 󵄨2 λ F(w) = ∫( 󵄨󵄨󵄨∇w(x)󵄨󵄨󵄨 − k w(x)2 − Φ(x, w(x)))dx 2 2 𝒟

λ 1 󵄨 󵄨 ≥ (1 − k )‖w‖2X − ∫󵄨󵄨󵄨Φ(x, w(x))󵄨󵄨󵄨dx. 2 λk+1 𝒟

By Assumption 8.3.5(i), there exists c1󸀠 = supx∈𝒟,s∈ℝ f (x, s) such that |Φ(x, s)| < c1󸀠 |s| for ̄ every x ∈ 𝒟̄ and s ∈ ℝ, so that ∫𝒟 |Φ(x, w(x))|dx ≤ c1󸀠 ‖w‖L1 (𝒟) , therefore, obtaining the lower bound, λ 1 F(w) ≥ (1 − k )‖w‖2X − c1󸀠 ‖w‖L1 (𝒟) . 2 λk+1

(8.23)

8 Since the ϕi are normalized so that ‖∇ϕi ‖L2 (𝒟) = ‖ϕi ‖X = 1 and −Δϕi = λi ϕi , by multiplying by ϕi and integrating by parts it follows easily that ‖ϕi ‖2L2 (𝒟) =

1 . λi

9 For the last part, we used the observation that for the eigenfunctions ϕi , the fact that ∫𝒟 ϕi (x)ϕj (x)dx = δij implies also that ∫𝒟 ∇ϕi (x) ⋅ ∇ϕj (x)dx = δij , by a simple integration by parts argument.

396 | 8 Critical point theory By a combination of the Hölder and Poincaré inequalities, ‖w‖L1 (𝒟) ≤ |𝒟|1/2 ‖w‖L2 (𝒟) ≤

|𝒟|1/2 |𝒟|1/2 ‖∇w‖L2 (𝒟) = ‖w‖X , λ1 λ1

and combining that with (8.23) we obtain the lower bound λ 1 F(w) ≥ (1 − k )‖w‖2X − c2󸀠 ‖w‖X , 2 λk+1

∀ w ∈ X2 ,

for an appropriate constant c2󸀠 . This finite lower bound guarantees that infw∈X2 F(w) > −∞. 2. We now show that F(v) → −∞ for v ∈ X1 such that ‖v‖X → ∞. As before, for any v = ∑ki=1 ci ϕi ∈ X1 , we have that k k λ 󵄨 󵄨2 ∫(󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 − λk v(x)2 )dx = ∑(λi − λk )ci2 ∫ ϕi (x)2 dx = ∑(1 − k )ci2 . λi i=1 i=1

𝒟

𝒟

For any of the eigenvalues λi for which λi < λk , this term can be estimated above by a negative quadratic term in ‖v‖X , but a problem arises for the term i = k (or if the eigenvalue λk is not simple with all the other terms i such that λi = λk ). We therefore decompose X1 into two orthogonal components X1(k) = span{ϕi : λi = λk } and X1(−,k) = span{ϕi : λi < λk }, and express any v ∈ X1 as v = vk +v−,k with vk ∈ X1(k) and v−,k ∈ X1(−,k) . With this decomposition in mind, k λ 󵄨 󵄨2 ∫(󵄨󵄨󵄨∇v(x)󵄨󵄨󵄨 − λk v(x)2 )dx = ∑(1 − k )ci2 λi i=1

𝒟

≤ (1 −

k λ λk ) ∑ ci2 = (1 − k )‖v−,k ‖2X , λk−1 i=1 λk−1

with all the sums taken over indices such that λi < λk , so that λi ≤ λk−1 . We therefore have that F(v) =

1 󵄨󵄨 󵄨2 ∫(󵄨∇v(x)󵄨󵄨󵄨 − λk v(x)2 )dx − ∫ Φ(x, v(x))dx 2 󵄨 𝒟

𝒟

λ 1 ≤ (1 − k )‖v−,k ‖2X − ∫ Φ(x, v(x))dx ≤ − ∫ Φ(x, v(x))dx, 2 λk−1 𝒟

∀ v ∈ X1 ,

𝒟

and by Assumption 8.3.5(ii), F(v) → −∞ as ‖v‖X → ∞. 3. It remains to prove that F satisfies the Palais–Smale condition, for which it is enough to show that any Palais–Smale sequence is bounded in X = W01,2 (𝒟). Consider any Palais–Smale sequence {un : n ∈ ℕ} and for any n, decompose un = u−,n +u0n +u+,n , where u−,n ∈ X1(−,k) , u0n ∈ X1(k) and u+,n ∈ X2 . Since {un : n ∈ ℕ} is a Palais–Smale

8.3 Applications in semilinear elliptic problems | 397

sequence it holds that DF(un ) → 0 in X ⋆ , therefore, for any ϵ > 0 and ϕ ∈ X, there exists N such that for all n > N, it holds that 󵄨󵄨 󵄨 󵄨 󵄨 󵄨󵄨 󵄨󵄨 󵄨󵄨⟨DF(un ), ϕ⟩󵄨󵄨󵄨 = 󵄨󵄨󵄨∫(∇un (x) ⋅ ∇ϕ(x) − λk un (x)ϕ(x) − f (x, un (x))ϕ(x))dx 󵄨󵄨󵄨 < ϵ‖ϕ‖X . (8.24) 󵄨󵄨 󵄨󵄨 𝒟

To prove the boundedness of the Palais–Smale sequence we will have to choose ϕ properly in order to show the boundeness of the norm of each component separately. We first choose ϕ = −u−,n and note that by orthogonality and similar arguments as those employed above, ∫(∇un (x) ⋅ (−∇u−,n (x)) − λk un (x)(−u−,n (x) ))dx 𝒟

= − ∫(∇u−,n (x) ⋅ ∇u−,n (x) − λk u−,n (x)u−,n (x))dx ≥ (λk − λk−1 )‖u−,n ‖2L2 (𝒟) , 𝒟

while using Assumption 8.3.5(i) 󵄨 󵄨 − ∫ f (x, un (x))(−u−,n (x))dx ≥ − ∫󵄨󵄨󵄨f (x, un (x))u−,n (x)󵄨󵄨󵄨dx ≥ −c1󸀠 ‖un ‖L1 (𝒟) 𝒟

≥

𝒟 −c1󸀠 |𝒟|1/2 ‖un ‖L2 (𝒟) ,

where in the last estimate we used Hölder’s inequality. Inserting these estimates in (8.24), we conclude that (λk − λk−1 )‖u−,n ‖2L2 (𝒟) − c1󸀠 |𝒟|1/2 ‖un ‖L2 (𝒟) ≤ ⟨DF(un ), (−u−,n )⟩ < ϵ‖u−,n ‖X , which since ϵ is arbitrary proves the boundedness of ‖u−,n ‖2L2 (𝒟) . However, since u−,n ∈

X1(k) which is a finite dimensional space, all norms are equivalent10 and the boundedness of ‖u−,n ‖X is concluded. We next, set ϕ = u+,n . Using orthogonality and the estimate (8.23), which holds for any w ∈ X2 , we obtain the estimate ⟨DF(un ), u+,n ⟩ = ∫(∇u+,n (x) ⋅ ∇u+,n (x) − λk u+,n (x))dx − ∫ f (x, un (x))u+,n (x)dx 𝒟

𝒟

≥ (1 −

λk 󵄨 󵄨 )‖u+,n ‖2X − ∫󵄨󵄨󵄨f (x, un (x)󵄨󵄨󵄨|u+,n (x)|dx λk+1 𝒟

λ ≥ (1 − k )‖u+,n ‖2X − c1󸀠 ‖u+,n ‖L1 (𝒟) λk+1 k−1 2 2 2 2 10 For any u−,n = ∑k−1 i=1 ci ϕi , it holds that ‖u−,n ‖X = ‖∇u−,n ‖L2 (𝒟) ‖ = ∑i=1 ci , while ‖u−,n ‖L2 (𝒟) ‖ = k−1 2 2 ∑k−1 i=1 ci ‖ϕi ‖L2 (𝒟) = ∑i=1

1 2 c , λi i

from which it easily follows that

1 λk

‖vn− ‖2X ≤ ‖u−,n ‖L2 (𝒟) ‖2 ≤

1 ‖v− ‖2 . λ1 n X

398 | 8 Critical point theory

≥ (1 −

λk 1 )‖u+,n ‖2X − c1󸀠 |𝒟|1/2 ‖u+,n ‖X , λk+1 λ1

where for the final step we used a combination of the Hölder and Poincaré inequalities. Combining the above estimate with (8.24), we conclude that (1 −

λk 1 )‖u+,n ‖2X − c1󸀠 |𝒟|1/2 ‖u+,n ‖X ≤ ⟨DF(un ), (u+,n )⟩ < ϵ‖u+,n ‖X , λk+1 λ1

from which the boundedness of ‖u+,n ‖X follows. It remains to verify that ‖u0n ‖X is bounded. For this term, the above strategy cannot be used, as the quadratic term in ‖u0n ‖X will vanish since u0n ∈ X1(k) . We will thus use the condition |F(un )| < c󸀠 which holds since {un : n ∈ ℕ} is a Palais–Smale sequence. Using the decomposition un = u−,n + u0n + u+,n , we obtain that 1 F(un ) = (‖u+,n ‖2X + ‖u−,n ‖2X − λk (‖u+,n ‖2L2 (𝒟) + ‖u−,n ‖2L2 (𝒟) ) − ∫ Φ(x, un (x))dx 2 1 = (‖u+,n ‖2X + ‖u−,n ‖2X − λk (‖u+,n ‖2L2 (𝒟) + ‖u−,n ‖2L2 (𝒟) ) 2

𝒟

− ∫(Φ(x, un (x)) − Φ(x, u0n (x)))dx + ∫ Φ(x, u0n (x))dx. 𝒟

𝒟

Note that Φ(x, un (x)) −

Φ(x, u0n (x))

un (x)

u0n (x)

0

0

un (x)

= ∫ f (x, s)ds − ∫ f (x, s)ds = ∫ f (x, s)ds, u0n (x)

so that by Assumption 8.3.5(i), 󵄨󵄨 󵄨 󵄨 󵄨 0 󸀠󵄨 0 󸀠󵄨 󵄨󵄨Φ(x, un (x)) − Φ(x, un (x))󵄨󵄨󵄨 < c1 󵄨󵄨󵄨un (x) − un (x)󵄨󵄨󵄨 = c1 󵄨󵄨󵄨u+,n (x) + u−,n (x)󵄨󵄨󵄨.

This allows us to estimate 1 F(un ) ≥ (‖u+,n ‖2X + ‖u−,n ‖2X − λk (‖u+,n ‖2L2 (𝒟) + ‖u−,n ‖2L2 (𝒟) ) 2 󵄨 󵄨 − ∫󵄨󵄨󵄨Φ(x, un (x)) − Φ(x, u0n (x))󵄨󵄨󵄨dx + ∫ Φ(x, u0n (x))dx 𝒟

𝒟

1 ≥ (‖u+,n ‖2X + ‖u−,n ‖2X − λk (‖u+,n ‖2L2 (𝒟) + ‖u−,n ‖2L2 (𝒟) ) 2 − c1󸀠 ‖u+,n + u−,n ‖L1 (𝒟) + ∫ Φ(x, u0n (x))dx. 𝒟

All the terms on the right-hand side of the above estimate depend on u+,n , u−,n and more precisely on norms of these quantities that we have already proved that are bounded. This observation combined with the fact that F(un ) < c, for some constant c, leads to the conclusion that there exists a constant c3󸀠 such that ∫𝒟 Φ(x, u0n (x))dx < c3󸀠 , which by Assumption 8.3.5(ii) leads to the boundedness of ‖u0n ‖X . The proof is complete.

8.4 Applications in quasilinear elliptic problems | 399

8.4 Applications in quasilinear elliptic problems In this section, we present applications of critical point theory to the functional F(u) :=

1 󵄨󵄨 󵄨p ∫󵄨∇u(x)󵄨󵄨󵄨 dx − ∫ Φ(x, u(x)) dx p 󵄨

(8.25)

𝒟

𝒟

and its connection with the quasilinear elliptic partial differential equation −∇ ⋅ (|∇u|p−2 ∇u) = f (x, u), u=0

in 𝒟,

(8.26)

on 𝜕𝒟.

The existence of minimizers to this functional, for 𝒟 bounded and of sufficiently smooth boundary, and under appropriate conditions on the nonlinearity f was studied in Section 6.8.2 using the direct method of the calculus of variations. The condition that guaranteed the coercivity of the functional F and hence, the existence of a mini< λ1,p mizer was essentially Assumption 6.8.4 according to which lim sup|s|→∞ pΦ(x,s) |s|p where λ1,p > 0 is the first eigenvalue of the p-Laplacian operator defined as λ1,p =

inf

u∈W01,p (𝒟)\{0}

‖∇u‖pLp (𝒟) ‖u‖pLp (𝒟)

(8.27)

,

which is clearly related to the best constant in the relevant Poincaré inequality. The infimum is attained for a function ϕ1,p > 0 called the first eigenfunction of the p-Laplacian operator (see e. g. [9]). Recall that λ1,p was also related to the existence of nontrivial solutions to the quasilinear problem −∇ ⋅ (|∇u|p−2 ∇u) = λ|u|p−2 u, u=0

on 𝜕𝒟.

in 𝒟,

(8.28)

with λ1,p > 0 being the smallest value of λ for which this equation admits such a solution. We will see that by relaxing this assumption we may obtain other critical points of the functional F using the mountain pass or the saddle point theorem. 8.4.1 The p-Laplacian and the mountain pass theorem We will see that by relaxing Assumption 6.8.4 on f we may obtain other critical points of the functional F related to solutions of the quasilinear elliptic equation (8.26) using the mountain pass theorem. Our approach follows [53]. We will impose the following conditions on f . Assumption 8.4.1. f : 𝒟 × ℝ → ℝ is a Carathéodory function such that it satisfies: (i) The growth condition |f (x, s)| ≤ c|s|r + a(x) a. e. in 𝒟, for p − 1 < r < sp − 1, where sp =

dp d−p

if d > p and p − 1 < r < ∞ if d ≤ p and a ∈ Lp (𝒟), with ∗

1 p

+

1 p∗

= 1.

400 | 8 Critical point theory (ii) The asymptotic condition that there exists a θ > p and a R > 0 such that 0 < θΦ(x, s) ≤ sf (x, s),

x ∈ 𝒟,

(iii) The condition for small values of s that lim sups→0 x ∈ 𝒟, where λ1,p > 0 is defined as in (8.27).

|s| ≥ R. pf (x,s) |s|p−2 s

≤ μ < λ1,p uniformly in

Using the mountain pass theorem, we will show the existence of a positive solution for (8.26), where the positivity arises from the asymptotic positivity of Φ, while the remainder of Assumption 8.4.1(ii) is needed in order to guarantee the validity of the Palais–Smale property for F. At the same time, this condition makes the functional unbounded from below, so that the critical point that will be obtained cannot correspond to a minimum. On the other hand, Assumption 8.4.1(iii) will guarantee the existence of a ρ > 0 and an α > 0 such that F(u) ≥ α for every u ∈ W01,p (𝒟) such that ‖u‖W 1,p (𝒟) = ρ, 0 a condition which is essential for the application of the mountain pass theorem. Proposition 8.4.2. Under Assumption 8.4.1, problem (8.26) admits a nontrivial solution. If furthermore f (x, 0) = 0, it is a positive solution u ≥ 0. Proof. Concerning the properties of the functional F the reader may consult Section 6.8 in order to show that F ∈ C 1 (X; ℝ) for X = W01,p (𝒟). We will concentrate in showing that this functional has the right structure for the mountain pass theorem to apply. The proof follows in 5 steps. 1. We will show first that F is unbounded below. By integrating Assumption 8.4.1(ii) we conclude the existence of a function γ ∈ L1 (𝒟), such that Φ(x, s) ≥ γ(x)sθ for s ≥ R (for a suitable R > 0), which allows us to get a control of the growth rate of the functional when u takes large values, whereas the growth condition (Assumption 8.4.1(i)) provides sufficient information when u takes values below R. We therefore have, using the notation 𝒟+ = {x ∈ 𝒟 : u(x) ≥ R} and 𝒟− = 𝒟 \ 𝒟+ , that F(u) =

1 ‖u‖p 1,p − ∫ Φ(x, u(x)) dx p W0 (𝒟) 𝒟

1 = ‖u‖p 1,p − ∫ Φ(x, u(x))dx − ∫ Φ(x, u(x))dx p W0 (𝒟) D+

≤

D−

(8.29)

1 󵄨 󵄨θ ‖u‖p 1,p − ∫ γ(x)󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx + c1 , W (𝒟) p 0 D+

where c1 is a bound for the last integral.11 Since we are interested in showing that F is unbounded below, consider any u ∈ W01,p (𝒟) (bounded) and take w = ϱu, for ϱ → ∞. 11 On {x : u(x) < R} by the growth condition, the nonlinearity is bounded by a constant and so the term can be estimated as above.

8.4 Applications in quasilinear elliptic problems | 401

Substituting that into (8.29), we see that F(ϱu) ≤

ϱp 󵄨θ 󵄨 ‖u‖p 1,p − ϱθ ∫ γ(x)󵄨󵄨󵄨u(x)󵄨󵄨󵄨 dx − c1 , W0 (𝒟) p D+

which shows that F(ϱu) → −∞ as ϱ → ∞. 2. We now show the existence of ρ > 0 and α > 0 such that F(u) ≥ α for every u ∈ W01,p (𝒟) such that ‖u‖W 1,p (𝒟) = ρ. Combining Assumption 8.4.1(iii) with the growth 0 condition, we have that there exist μ ∈ (0, λ1,p ) and c2 > 0 such that μ p |s| + c2 |s|q1 , p

Φ(x, s) ≤

x ∈ 𝒟,

s ∈ ℝ,

(8.30)

for q1 ∈ {max(p, r + 1), sp }. Indeed, by Assumption 8.4.1(iii), there exists μ ∈ (0, λ1,p )

such that lim sups→0 pΦ(x,s) |s|p−2 s

pf (x,s) |s|p−2 s

< μ uniformly in x ∈ 𝒟, and a δ (depending on μ) such that

≤ μ for |s| < δ, and integrating (under the assumption that f (x, 0) = 0) we obtain μ

that Φ(x, s) ≤ p |s|p for |s| < δ. On the other hand, integrating the growth condition we obtain that |Φ(x, s)| ≤ c3 (|s|q + 1) for all s ∈ ℝ, and the claim easily follows. Having obtained (8.30), we estimate F as follows: F(u) =

1 ‖u‖p 1,p − ∫ Φ(x, u(x)) dx p W0 (𝒟) 𝒟

μ 1 q − ‖u‖pLp (𝒟) − c2 ‖u‖L1q1 (𝒟) . ≥ ‖u‖p 1,p W (𝒟) p p 0

(8.31)

By the choice of q1 < sp we have that W01,p (𝒟) 󳨅→ Lq1 (𝒟) so that there exists a constant c3 such that ‖u‖Lq1 (𝒟) ‖ ≤ c3 ‖u‖W 1,p (𝒟) , whereas by the definition of λ1,p (or the Poincaré 0

inequality in fact) gives us that λ1,p ‖u‖pLp (𝒟) ≤ ‖u‖p

W01,p (𝒟)

we obtain

F(u) ≥ ‖u‖p

W01,p (𝒟)

. Using these estimates in (8.31),

μ 1 q −p ) − c4 ‖u‖ 1 1,p ). ( (1 − W0 (𝒟) p λ1,p

Setting ρ > 0 such that 1 − c4 ρq1 −p , and since μ < λ1,p we can see that there exists α > 0 for which our claim follows. 3. It remains to verify that F satisfies the Palais–Smale condition. This is done in two substeps. 3(a). We first show that any Palais–Smale sequence is bounded, i. e., for any sequence {un : n ∈ ℕ} such that ‖F(un )‖X < c󸀠 for some c󸀠 > 0 and DF(un ) → 0 in X ⋆ , it holds that12 ‖un ‖X ≤ c for an appropriate constant c > 0. We express Φ(x, s) = 12 DF(un ) → 0 implies that ‖DF(un )‖X ⋆ = sup{|⟨DF(un ), v⟩| : ‖v‖X ≤ 1} → 0, and the claim follows u from that choosing v = ‖u n‖ . n X

402 | 8 Critical point theory Φ(x, s) − θ1 f (x, s) + θ1 f (x, s) in order to employ the asymptotic growth condition of Assumption 8.4.1(ii) on f . Using the notation Φ0 (x, s) := Φ(x, s) − θ1 f (x, s) so as to express Φ(x, s) = Φ0 (x, s) + θ1 f (x, s) and noting that by Assumption 8.4.1(ii) Φ0 (x, s) ≤ 0 for |s| ≥ R, so that upon setting 𝒟n,+ := {x ∈ 𝒟 : |un (x)| ≥ R} with 𝒟n,− = 𝒟 \ 𝒟n,+ we have that ∫ Φ0 (x, un (x))dx = ∫ Φ0 (x, un (x))dx + ∫ Φ0 (x, un (x))dx 𝒟

𝒟n,+

𝒟n,−

(8.32)

≤ ∫ Φ0 (x, un (x))dx ≤ c1 , 𝒟n,−

with the last estimate arising from the growth condition Assumption 8.4.1(i). Since our information on F(un ) and DF(un ) guarantee that F(un ) is bounded while |⟨DF(un ), un ⟩| ≤ ϵ‖un ‖X for any ϵ > 0 for large enough n (since DF(un ) → 0 in X ⋆ ) we may try a linear combination of F(un ) and ⟨DF(un ), un ⟩ so as to form the integral of Φ0 whose sign we may control. Combining the above, we have that 1 1 1 F(un ) − ⟨DF(un ), un ⟩ = ( − ) ∫ |∇un |p dx − ∫ Φ0 (un )dx θ p θ 𝒟

𝒟

1 1 ≥ ( − ) ∫ |∇un |p dx − c1 , p θ 𝒟

using (8.32), so that by rearranging and using the boundedness assumptions provided we get an estimate of the form c2 ‖un ‖2X ≤ ϵ‖un ‖X + c3 with c2 = p1 − θ1 > 0, from which we conclude that {un : n ∈ ℕ} is bounded. 3(b). Having established that a Palais–Smale sequence is bounded by the reflexivity of X = W01,p (𝒟) there exists a subsequence {unk : k ∈ ℕ} and a u ∈ X such that unk ⇀ u in X. We will show that this convergence is strong, i. e., un → u in X, thus establishing the Palais–Smale property. We first observe that, upon defining φ(u) =

1 1 ‖u‖pX = ‖∇u‖pLp (𝒟) , p p

and noting that ⟨Dφ(u), v⟩ = ∫𝒟 |∇u|p−2 ∇u ⋅ ∇vdx for every v ∈ X, we have ⟨DF(unk ), unk − u⟩ + ∫ f (x, unk )(unk − u)dx = ⟨Dφ(unk ), ∇unk − ∇u⟩. 𝒟

Since DF(unk ) → 0 (strong) and unk −u is bounded, we have that ⟨DF(unk ), unk −u⟩ → 0. c

By the compact embedding of X = W01,p (𝒟) 󳨅→ Lr+1 (𝒟), there exists a subsequence of {unk : k ∈ ℕ} (denoted the same) such that unk → u in Lq (𝒟), for q = r + 1; hence,

8.4 Applications in quasilinear elliptic problems | 403

moving to this subsequence and taking also into account the growth condition which ⋆ provides a uniform bound for f (x, unk ) in Lq (𝒟) we have ∫𝒟 f (x, unk )(unk − u)dx → 0. This leads to the fact that ⟨Dφ(unk ), ∇unk − ∇u⟩ → 0, which combined with the fact that unk ⇀ u in X will lead us to the conclusion that unk → u in X. To see the last claim note that φ is a convex functional; hence, φ(u) ≥ φ(unk ) + ⟨Dφ(unk ), u − unk ⟩ and taking the limit superior (combined with the fact that the last term converges to 0) we have that φ(u) ≥ lim supk φ(unk ). On the other hand by the weak lower semicontinuity of φ (it is in fact a norm), we have that φ(u) ≤ lim infk φ(unk ), so that lim inf φ(unk ) = lim sup φ(unk ) = lim φ(unk ) = φ(u). k

k

k

Hence, unk ⇀ u, and ‖unk ‖X → ‖u‖X and by the uniform convexity of X = W01,p (𝒟) (see Example 2.6.10) and the Radon–Riesz property (see Example 2.6.13) we conclude that unk → u in X. Using the standard trick, we may show the result for the whole sequence. 4. Using the results of steps 1, 2 and 3, we therefore conclude the existence of a critical point u for F by an application of the mountain pass theorem. 5. It remains to show that if f (x, 0) = 0 then this critical point satisfies u ≥ 0. To see that, define the function f+ by f+ (x, s) = f (x, s+|s| ). It is easy to see that f+ (x, s) = 0 2 for s ≤ 0, while f+ (x, s) = f (x, s) for s > 0. It is also easy to see that all the assumptions on f are still true for f+ , so that repeating all the above we may find a critical point u+ for the functional F+ which is obtained by exchanging f with f+ . This critical point is a weak solution of the quasilinear PDE: −Δp u+ = f+ (x, u+ ), u+ = 0

on 𝜕𝒟,

in 𝒟,

and using v = max(−u+ , 0) ∈ W01,p (𝒟) as a test function, defining 𝒟− = {x ∈ 𝒟 : u+ < 0} we obtain that − ∫𝒟 |∇u+ (x)|p dx = − ∫𝒟 f+ (x, u+ (x))v(x)dx ≥ 0 so that ∇u+ (x) = 0 a. e. − − in 𝒟− . Since, ∇v(x) = −∇u+ (x) for x ∈ 𝒟− and ∇v(x) = 0 for x ∈ 𝒟 \ 𝒟− , we see that ∇u+ (x) = 0 a. e. in 𝒟− implies that ∇v(x) = 0 a. e. in 𝒟, therefore, ‖v‖W 1,p (𝒟) = 0 and 0 v = 0 a. e. in 𝒟 which in turn leads to the conclusion that u ≥ 0 a. e. in 𝒟. Remark 8.4.3. The condition in Assumption 8.4.1(iii) can be generalized as (x,s) ≤ b(x) < λ1,p where b ∈ L∞ (𝒟) is a function such that |{x ∈ 𝒟 : lim sups→0 pf |s|p−2 s b(x) < λ1,p }| > 0 where λ1,p > 0 is defined as in (8.27). Remark 8.4.4. In the case where f (x, 0) = 0, one may also find a negative solution u < 0, by defining the function f− as f− (x, s) = f (x, s−|s| ), which has the property f− (x, s) = 0 2 for s ≥ 0. Then applying Proposition 8.4.2 for the functional F− in which f is replaced by f− one may find a nontrivial critical point which is a weak solution for problem (8.26) with f replaced by f− . The negativity of u is shown similarly as in the proof of Proposition 8.4.2 by using as test function v = max(u, 0).

404 | 8 Critical point theory 8.4.2 Resonant problems for the p-Laplacian and the saddle point theorem We now consider the quasilinear problem (8.26) but for the case where Φ(x, s) = λ1,p |s|p + Φ0 (x, s) (or equivalently f (x, s) = λ1 |s|p−2 s + f0 (x, s) where λ1,p > 0 is the first p eigenvalue of the p-Laplacian (given in (8.27)). This choice leads to the quasilinear problem −∇ ⋅ (|∇u|p−2 ∇u) = λ1,p |u|p−2 u + f0 (x, u), u=0

on 𝜕𝒟.

in 𝒟,

(8.33)

which is called resonant since if f0 = 0 it reduces to the eigenvalue problem (8.28) that admits nontrivial solutions which are scalar multiples of the eigenfunction ϕ1,p > 0 and in analogy with the standard Laplacian operator case (p = 2) leads to noninvertibility of the operator A(u) := −Δp u − λ1,p |u|p−2 u. An interesting question is whether problem (8.33) continues to admit nontrivial solutions when f0 ≠ 0. This problem can be treated using the saddle point theorem. Our approach follows [11]. We impose the following assumption on f0 (whose antiderivative will be denoted by Φ0 ). Assumption 8.4.5. The Caratheodory function f0 : 𝒟 × ℝ → ℝ satisfies: (i) f0 is bounded and satisfies lims→±∞ f (x, s) = γ± (x), a. e. (ii) ∫𝒟 γ+ (x)ϕ1,p (x)dx < 0 < ∫𝒟 γ− (x)ϕ1,p (x)dx (or ∫𝒟 γ− (x)ϕ1,p (x)dx < 0 < ∫𝒟 γ+ (x)× ϕ1,p (x)dx). The condition of Assumption 8.4.5(ii) is a Landesman–Lazer-type condition. Proposition 8.4.6. Under Assumption 8.4.5, problem (8.33) admits a nontrivial solution. Proof. Let X1 = span(ϕ1 ) which is an one-dimensional space and X2 its orthogonal complement, so that W01,p (𝒟) = X1 ⊕ X2 . In order to show that F satisfies the necessary geometry to apply the saddle point Theorem 8.2.7 we need to show that F is unbounded below on X1 , while it is bounded below on X2 . The fact that F ∈ C 1 (X; ℝ) for X = W01,p (𝒟) has already been established. The proof proceeds in 4 steps: 1. To show that F is unbounded below on X1 consider any u = λϕ1,p , and note that since ϕ1,p satisfies λ1,p ‖ϕ1,p ‖pLp (𝒟) = ‖ϕ1,p ‖p 1,p , it holds that W0 (𝒟)

F(λϕ1,p ) =

λ1,p 1 ‖λϕ1,p ‖p 1,p − ‖λϕ1,p ‖pLp (𝒟) − ∫ Φ0 (x, λϕ1,p (x))dx W0 (𝒟) p p 𝒟

= − ∫ Φ0 (x, λϕ1,p (x))dx = −λ ∫ 𝒟

𝒟

Φ0 (x, λϕ1,p (x)) λϕ1,p (x)

ϕ1,p (x)dx.

8.4 Applications in quasilinear elliptic problems | 405

Since ϕ1,p > 0, by Assumption 8.4.5(i),

Φ0 (x,λϕ1,p (x)) λϕ1,p (x)

Φ (x,λϕ1,p (x)) ϕ1,p (x)dx dominated convergence ∫𝒟 0 λϕ (x) 1,p

→ γ+ (x) as λ → ∞, so by Lebesgue’s

→ ∫𝒟 γ+ (x)ϕ1,p (x)dx and by Assump-

tion 8.4.5(ii) F(λϕ1,p ) → −∞ as λ → ∞ (with a similar reasoning if the alternative condition holds). 2. We now consider u ∈ X2 . By the definition of λ1,p (see (8.27)) for any u ∈ X2 there exists a λ > λ1,p such that λ‖u‖pLp (𝒟) < ‖u‖p 1,p . We then see that W0 (𝒟)

F(u) =

λ1,p 1 ‖u‖p 1,p − ‖u‖pLp (𝒟) − ∫ Φ0 (x, u(x))dx p W0 (𝒟) p 𝒟

λ1,p 1 )‖u‖p 1,p − ∫ Φ0 (x, u(x))dx ≥ (1 − W0 (𝒟) p λ ≥

𝒟

λ1,p

1 (1 − )‖u‖p 1,p − c1󸀠 ‖u‖L1 (𝒟) , W0 (𝒟) p λ

where we used Assumption 8.4.5(i) and c1󸀠 is an appropriate constant. By the Hölder inequality ‖u‖L1 (𝒟) ≤ c2󸀠 ‖u‖Lp (𝒟) and using once more the Poincaré inequality, we obtain the lower bound F(u) ≥

λ1,p 1 (1 − )‖u‖p 1,p − c3󸀠 ‖u‖W 1,p (𝒟) , W0 (𝒟) 0 p λ

∀ u ∈ X2 ,

from which we conclude that minu∈X2 F(u) > −∞. 3. It remains to show that F satisfies the Palais–Smale condition. As in the proof of Proposition 8.4.2, it suffices to show that any Palais–Smale sequence is bounded in X = W01,p (𝒟). 3(a). We will show that if {un : n ∈ ℕ} is a Palais–Smale sequence then it is bounded. We may either follow the route in step 3(a) in Proposition 8.4.2 or the following alternative route by contradiction. Assume not, and consider a Palais–Smale u sequence for F, {un : n ∈ ℕ} such that ‖un ‖W 1,p (𝒟) → ∞. Define wn = ‖u ‖ n1,p , which n W

0

0

W01,p (𝒟)

1,p

(𝒟)

is bounded and by the reflexivity of W (𝒟) there exists a w0 ∈ and a subsequence (denoted the same) such that wn ⇀ w0 in W01,p (𝒟), while by the compact embedding W01,p (𝒟) 󳨅→ Lp (𝒟), there exists a further subsequence (still denoted the same) such that wn → w0 in Lp (𝒟). We will show that since {un : n ∈ ℕ} is a Palais– Smale sequence wn → w0 in W01,p (𝒟). Indeed, since F(un ) < c, dividing by ‖un ‖p 1,p W0 (𝒟)

we have that

λ1,p Φ(x, un ) 1 c ‖wn ‖p 1,p − ‖wn ‖pLp (𝒟) − ∫ dx ≤ W0 (𝒟) p p ‖un ‖p 1,p ‖un ‖p 1,p 𝒟

W0 (𝒟)

W0 (𝒟)

.

(8.34)

Note that on account of Assumption 8.4.5 (i), the asymptotic behavior of Φ(x, u) is γ+ (x)u(x) for u(x) → ∞ and γ− (x)u(x) for u(x) → −∞, it is straightforward to see

406 | 8 Critical point theory that ∫𝒟

Φ(x,un ) ‖un ‖p 1,p W

0

it holds that that

(𝒟)

dx → 0 as ‖un ‖W 1,p (𝒟) → ∞. Furthermore, since wn → w0 in Lp (𝒟), 0

‖wn ‖pLp (𝒟)

→

‖w0 ‖pLp (𝒟) .

Taking the limit superior in (8.34), we conclude

lim sup ‖wn ‖p

W01,p (𝒟)

n

≤ λ1,p ‖w0 ‖pLp (𝒟) .

(8.35)

On the other hand by the weak lower semicontinuity of the norm, it holds that ‖w0 ‖p 1,p ≤ lim infn ‖wn ‖p 1,p which combined with the Poincaré inequality (or W0 (𝒟)

W0 (𝒟)

equivalently the definition of λ1,p ), yields λ1,p ‖w0 ‖pLp (𝒟) ≤ ‖w0 ‖p

W01,p (𝒟)

the conclusion that

λ1,p ‖w0 ‖pLp (𝒟) ≤ lim inf ‖wn ‖p

W01,p (𝒟)

n

which leads to

(8.36)

.

Combining (8.35) with (8.36), we conclude the facts that (a) limn ‖wn ‖p

=

W01,p (𝒟) p p p λ1,p ‖w0 ‖ 1,p while (b) λ1,p ‖w0 ‖Lp (𝒟) = ‖w0 ‖ 1,p . By the uniform convexity of W0 (𝒟) W0 (𝒟) 1,p W0 (𝒟) (see Example 2.6.10) and the Radon–Riesz property (Example 2.6.13), we conclude from (a) that wn → w (strong) in W01,p (𝒟). Fact (b) implies that w0 is a minimizer

of problem (8.27) so that w0 is a multiple of ϕ1,p (either ϕ1,p or −ϕ1,p since it is of norm 1), let us assume without loss of generality that w0 = ϕ1,p > 0. Hence, wn → ϕ1,p in W01,p (𝒟). We now consider the condition that DF(un ) → 0 in (W01,p (𝒟))∗ which allows us to conclude that for any ϵ > 0 there exists an N ∈ ℕ such that for any n > N, −ϵ‖un ‖W 1,p (𝒟) < ‖un ‖p

W01,p (𝒟)

0

− λ1,p ‖un ‖pLp (𝒟) − ∫ f (x, un (x))un (x)dx < −ϵ‖un ‖W 1,p (𝒟) , 0

𝒟

(8.37)

while |F(un )| < c gives that −c ≤

λ1,p 1 ‖u ‖p 1,p − ‖un ‖pLp (𝒟) − ∫ Φ0 (x, un (x))dx ≤ c. p n W0 (𝒟) p

(8.38)

𝒟

Multiplying (8.38) by −p and adding to (8.37) we can get rid of the terms involving ‖un ‖p 1,p and ‖un ‖pLp (𝒟) and conclude that W0 (𝒟)

󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨 󵄨󵄨∫(f0 (x, un (x))un (x) − pΦ0 (x, un (x)))dx󵄨󵄨󵄨 ≤ c p + ϵ‖un ‖W 1,p (𝒟) , 󵄨󵄨 󵄨󵄨 0 𝒟

which upon dividing with ‖un ‖W 1,p (𝒟) leads to 0

󵄨󵄨 󵄨󵄨 Φ (x, un (x)) c 󵄨 󵄨󵄨 wn (x))dx󵄨󵄨󵄨 ≤ + ϵ. 󵄨󵄨∫(f0 (x, un (x))wn (x) − p 0 󵄨󵄨 ‖un ‖W 1,p (𝒟) 󵄨󵄨 un (x) 0 𝒟

(8.39)

8.4 Applications in quasilinear elliptic problems | 407

Φ (x,u (x))

Since wn → ϕ1 > 0, we have that un → ∞ so that f (x, un (x)) → γ∞ (x) and 0 u (x)n → n γ+ (x), therefore, passing to the limit in (8.39) and using the Lebesgue dominated convergence theorem, we conclude that ∫𝒟 γ+ (x)ϕ1 (x)dx = 0, which is in contradiction with Assumption 8.4.5(ii). Therefore, {un : n ∈ ℕ} is bounded. If w0 → −ϕ1 , then we proceed similarly. 3(b) To show that any Palais–Smale sequence converges, we proceed exactly as in step 3(b) of Proposition 8.4.2, using boundedness of {un : n ∈ ℕ} to guarantee the existence of a weak limit and then uniform convexity of W01,p (𝒟) to show that this limit is strong. The details are left to the reader. 4. Using steps 1, 2 and 3 using the saddle point theorem, we conclude the proof.

9 Monotone-type operators The final chapter of this book is dedicated to the theory and applications of monotonetype operators. We will introduce various notions of monotonicity (i. e., monotonicity, maximal monotonicity and pseudomonotonicity) and study the remarkable properties of these operators in particular with respect to surjectivity. The theory of monotonetype operators plays an important role in applications. In closing the chapter, we try to present a variety of applications including quasilinear elliptic PDEs, variational inequalities with nonlinear operators as well as gradient flows in Hilbert spaces or evolution equations in Gelfand triples. Monotone-type operators and their applications are covered in many books; see, e. g., [18, 27, 85, 86] or [101].

9.1 Motivation The motivation for monotone operators is closely connected with the theory of convexity. Recall for instance that a Gâteaux differentiable function is convex if and only if ⟨Dφ(x2 ) − Dφ(x1 ), x2 − x1 ⟩ ≥ 0 holds for any x1 , x2 ∈ C for a suitable convex subset (see Theorem 2.3.22). Recall also that a similar property holds even for nonsmooth convex functions in terms of the subdifferential, since for any x⋆i ∈ 𝜕φ(xi ), i = 1, 2 it we have that ⟨x⋆1 − x⋆2 , x1 − x2 ⟩ ≥ 0 (see Proposition 4.2.1). These properties have proved to be extremely useful in a number of applications, e. g., in the convergence of iterative schemes for the minimization of convex functions, either smooth or not (see, e. g., Section 4.8). A similar property was also encountered when considering linear operators related to coercive bilinear forms in Chapter 7, and has played a crucial role in the development of the Lions–Stampacchia and Lax–Milgram theory for the treatment of variational inequalities and linear elliptic equations, respectively. Important linear or nonlinear operators, single valued or multivalued, related to derivatives of convex functionals or not, such as the Laplacian or the p-Laplacian, enjoy the property ⟨A(x1 )−A(x2 ), x1 −x2 ⟩ ≥ 0, and we have already seen, in Chapters 6 and 7, how this property can be used appropriately in order to obtain important conclusions concerning solvability of operator equations, properties of eigenvalues or approximation results for these operators. It is the aim of this chapter to present the elegant theory of monotone operators, as well as its various extensions, and how monotonicity may be used to provide important and detailed information concerning properties of such operators, which turn out to be natural extensions of the properties that the Gâteaux derivatives of convex functions enjoy, and also how this abstract and general theory may be used in applications in order to provide solvability results for nonlinear operator equations which are related to nonlinear PDEs, variational inequalities or evolution equations. https://doi.org/10.1515/9783110647389-009

410 | 9 Monotone-type operators

9.2 Monotone operators 9.2.1 Monotone operators, definitions and examples Let X be a Banach space, X ⋆ its topological dual and by ⟨⋅, ⋅⟩ we denote the duality pairing between X and X ⋆ . Definition 9.2.1 (Monotone operators). Let A : D(A) ⊂ X → 2X be a (possibly multivalued) operator. (i) A is called monotone if ⋆

⟨x⋆1 − x⋆2 , x1 − x2 ⟩ ≥ 0,

∀ x⋆i ∈ A(xi ),

xi ∈ D(A) ⊂ X,

i = 1, 2.

(9.1)

(ii) A is called strictly monotone if it is monotone and ⟨x⋆1 − x⋆2 , x1 − x2 ⟩ = 0 for some (xi , x⋆i ) ∈ Gr(A), i = 1, 2, implies x1 = x2 . If A is single valued, then the monotonicity condition (9.1) simplifies to ⟨A(x1 ) − A(x1 ), x1 − x2 ⟩ ≥ 0,

∀ x1 , x2 ∈ D(A).

Increasing functions and their corresponding Nemitskii operators can be considered as monotone operators. The same holds for linear operators defined by coercive bilinear forms. Also according to Proposition 4.2.1, the subdifferential of a convex function is a monotone operator; hence, also the Gâteaux derivative of a differentiable ⋆ convex function. As a final example, we mention the duality map J : X → 2X , which is a monotone (possibly multivalued) operator, as the subdifferential of the norm (see Proposition 4.4.1). Example 9.2.2. Let H be a Hilbert space. A (possibly multivalued) operator A : D(A) ⊂ H → 2H is monotone if and only if 󵄩 󵄩 ‖x1 − x2 ‖ ≤ 󵄩󵄩󵄩(x1 − x2 ) + λ(z1 − z2 )󵄩󵄩󵄩,

∀ xi ∈ D(A), zi ∈ A(xi ), λ > 0, i = 1, 2.

(9.2)

Inequality (9.2), implies that for each λ > 0 the operator (I+λA)−1 is defined on R(I+λA). Indeed, the direct assertion is immediate by expanding ‖(x1 − x2 ) + λ(z1 − z2 )‖2 in terms of the inner product and using monotonicity, while for the converse note that (9.2) implies that 2λ⟨z1 − z2 , x1 − x2 ⟩ + λ2 ‖z1 − z2 ‖2 ≥ 0, so dividing by λ and passing to the limit λ → 0+ yields the monotonicity of A. ◁ Example 9.2.3. Let 𝒟 ⊂ ℝd be a bounded domain with sufficiently smooth boundary. ⋆ The p-Laplace operator A = −Δp : W01,p (𝒟) → W −1,p (𝒟), defined by 󵄨 󵄨p−2 ⟨A(u), v⟩ := ⟨−Δp u, v⟩ := ∫󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 ∇u(x) ⋅ ∇v(x)dx, 𝒟

∀ u, v ∈ W01,p (𝒟),

9.2 Monotone operators | 411

where p > 1 (see Section 6.8.1) is monotone. The special case p = 2 corresponds to a linear operator, which is easily identified with the Laplacian. To show monotonicity, we will equip W01,p (𝒟) with the equivalent norm ‖u‖1,p = 1

(∑di=1 ‖Di u‖pLp (𝒟) ) p , where by Di u we denote the partial derivative as Di u = 1, . . . , d. Noting that, using Hölder’s inequality, d

𝜕u , 𝜕xi

i =

d

⟨A(u1 ), u2 )⟩ = ∑⟨|Di u1 |p−2 Di u1 , Di u2 ⟩ ≤ ∑ ‖Di u1 ‖p−1 ‖Di u2 ‖Lp (𝒟) Lp (𝒟) i=1

d

≤ (∑ ‖Di u1 ‖pLp (𝒟) )

1/p⋆

i=1

i=1

(‖Di u2 ‖pLp (𝒟) )

1/p

we obtain, using Young’s inequality (and the notation p⋆ =

≤ ‖u1 ‖p−1 1,p ‖u2 ‖1,p , p ) p−1

that

⟨A(u1 ) − A(u2 ), u1 − u2 ⟩

= ⟨A(u1 ), u1 ⟩ − ⟨A(u1 ), u2 ⟩ − ⟨A(u2 ), u1 ⟩ + ⟨A(u2 ), u2 ⟩

p−1 p ≥ ‖u1 ‖p1,p − ‖u1 ‖p−1 1,p ‖u2 ‖1,p − ‖u2 ‖1,p ‖u1 ‖1,p + ‖u2 ‖1,p

≥ ‖u1 ‖p1,p − (

1 1 1 1 ‖u ‖p + ‖u ‖p ) − ( ⋆ ‖u2 ‖p1,p + ‖u1 ‖p1,p ) + ‖u2 ‖p1,p = 0. p⋆ 1 1,p p 2 1,p p p

Using arguments based on more elaborate elementary inequalities one may show strict monotonicity of the p-Laplace operator under suitable conditions (see e. g. [43]). ◁ In this section, for reasons of simplicity and clarity of exposition we will focus on the case of single valued monotone operators. The general case where A is multivalued will be treated in detail in the general context of maximal monotone and pseudomonotone operators in Sections 9.3 and 9.4, respectively. 9.2.2 Local boundedness of monotone operators A fundamental property of monotone operators is local boundedness (see, e. g., [3]). Definition 9.2.4. An operator A : D(A) ⊂ X → X ⋆ is called: (i) Locally bounded at x ∈ X if there exists a neighborhood N(x) of x such that the set A(N(x)) = {A(y) : y ∈ N(x) ∩ D(A)} is bounded in X ⋆ . (ii) Bounded if it maps bounded sets of X to bounded sets of X ⋆ . Remark 9.2.5. If A were a linear operator, then continuity is equivalent to boundedness. However, this is not true for nonlinear operators in general. Theorem 9.2.6. Let A : D(A) ⊂ X → 2X be a monotone operator. Then A is locally bounded on int(D(A)). ⋆

412 | 9 Monotone-type operators Proof. For simplicity, let A be single valued. Assume the assertion of the theorem is not true. Then there exists xo ∈ int(D(A)) and a sequence {xn : n ∈ ℕ} ⊂ D(A) such that xn → xo in X but ‖A(xn )‖X ⋆ → ∞. We define the sequences an := ‖xn − x‖ → 0 and −1 bn := ‖A(xn )‖X ⋆ → ∞ as well as the sequence cn := max(a1/2 n , bn ) which satisfies the 2 properties cn → 0, and of course bn cn ≥ 1 and an ≤ cn . Consider now z ∈ X arbitrary, and let zn = xo + cn z ∈ D(A) for large enough n (since xo ∈ int(D(A)) and cn → 0). Choose r > 0 such that z̄ = xo + rz ∈ D(A) with ̄ X ⋆ < ∞. Using the monotonicity of A, we have that ‖A(z)‖ ̄ zn − z⟩̄ = (cn − r)⟨A(zn ) − A(z), ̄ z⟩, 0 ≤ ⟨A(zn ) − A(z), and since cn → 0 choosing n large enough we may guarantee that cn − r < 0 so that ̄ z⟩ which by the properties of z,̄ allows us by the use of the Banach– ⟨A(zn ), z⟩ ≤ ⟨A(z), Steinhaus uniform bound principle (Theorem 1.1.7) to conclude that there exists c > 0 independent of n such that ‖A(zn )‖X ⋆ < c for all n sufficiently large. We now apply the monotonicity property once more to the points xn and zn . This gives 0 ≤ ⟨A(xn ) − A(zn ), xn − zn ⟩, which by the definition of zn , upon rearrangement yields the bound cn ⟨A(xn ), z⟩ ≤ ⟨A(xn ), xn − xo ⟩ − ⟨A(zn ), (xn − xo ) − cn z⟩

≤ bn an + c (an + cn ‖z‖) ≤ bn cn2 + c( cn2 + cn ‖z‖).

Dividing by bn cn2 , we obtain that 1 1 1 1 ⟨A(xn ), z⟩ ≤ 1 + c( + ‖z‖) ≤ 1 + c( + ‖z‖) ≤ c󸀠 , bn cn bn bn cn bn for some appropriate constant c󸀠 > 0, uniformly in n, (we used the fact bn cn ≥ 1 for all n ∈ ℕ and that b−1 n → 0). Applying the Banach–Steinhaus principle once more we 1 have that b c ‖A(xn )‖X ⋆ < c󸀠󸀠 . for an appropriate constant c󸀠󸀠 > 0, uniformly in n. But n n

1 ‖A(xn )‖X ⋆ bn cn

= c1 , so we conclude that n tion with the fact that cn → 0.

1 cn

< c󸀠󸀠 uniformly in n, which is in contradic-

Remark 9.2.7. Note that A may be unbounded at 𝜕D(A). For example, let {xn : n ∈ ℕ} ⊂ ℝd , d ≥ 2, be a sequence such that |xn | = 1 for every n ∈ ℕ and consider the mapping A : B(0, 1) ⊂ ℝd → ℝd defined by x,

A(x) = {

(n + 1)xn ,

for x ∈ B(0, 1), x ≠ xn , for x = xn ,

which is a monotone operator with D(A) = B(0, 1), but is unbounded on 𝜕D(A).

9.2 Monotone operators | 413

9.2.3 Hemicontinuity and demicontinuity Definition 9.2.8. Let X be a Banach space. An operator A : D(A) ⊂ X → X ⋆ is called (i) Demicontinuous at x ∈ D(A), if for every {xn : n ∈ ℕ} ⊂ D(A) with xn → x, we have ⋆ A(xn ) ⇀ A(x) in X ⋆ . (ii) Hemicontinuous at x ∈ int(D(A)), if for every z ∈ X with x + tz ∈ D(A) for t ∈ [0, t0 ), ⋆ t0 > 0, we have A(x + tz) ⇀ A(x) in X ⋆ as t → 0+ . It is clear that in general, demicontinuity implies hemicontinuity. In the following result, we will see that for monotone operators, the converse is also true (see e. g. [86]). Proposition 9.2.9. Let X be a reflexive Banach space. Any monotone and hemicontinuous operator A : D(A) ⊂ X → X ⋆ is demicontinuous on int(D(A)). Proof. Suppose that A is hemicontinuous on int(D(A)). Let x ∈ int(D(A)) and {xn : n ∈ ℕ} ⊂ int(D(A)) be such that xn → x in X. From Theorem 9.2.6, we know that A is locally bounded on int(D(A)). So we may assume that the sequence {A(xn ) : n ∈ ℕ} is bounded in X ⋆ and by reflexivity there exists a subsequence {A(xnk ) : k ∈ ℕ} of {A(xn ) : n ∈ ℕ} and a x⋆ ∈ X ⋆ such that A(xnk ) ⇀ x⋆ in X ⋆ . We have ⟨A(xnk ) − A(z), xnk − z⟩ ≥ 0,

∀ z ∈ D(A),

and passing to the limit we obtain ⟨x⋆ − A(z), x − z⟩ ≥ 0,

∀ z ∈ D(A).

(9.3)

Since int(D(A)) is an open set, for any z ∈ X there exists to > 0, depending on z, such that xt = x + t z ∈ int(D(A)) for all t with 0 < t ≤ to . Set z = xt in (9.3) to get ⟨x⋆ − A(x + t z), z) ≤ 0. Letting t → 0+ we obtain by the hemicontinuity of A that ⟨x⋆ − A(x), z⟩ ≤ 0,

∀ z ∈ X,

so A(x) = x⋆ , i. e., A(xnk ) ⇀ A(x), and using the Urysohn property (see Remark 1.1.51) we conclude that the whole sequence A(xn ) ⇀ A(x); hence A is demicontinuous. The properties of monotonicity and hemicontinuity have the following interesting implication (often called Minty’s trick). Lemma 9.2.10 (Minty). If A : D(A) ⊂ X → X ⋆ , with D(A) = X (X not necessarily reflexive), is monotone and hemicontinuous and (xo , x⋆o ) ∈ X × X ⋆ such that ⟨x⋆o − A(x), xo − x⟩ ≥ 0, then x⋆o = A(xo ).

∀ x ∈ X,

(9.4)

414 | 9 Monotone-type operators Proof. For any t > 0, let x = xo + tz where z ∈ X. Then it follows from (9.4) that ⟨x⋆o − A(xo + tz), z⟩ ≤ 0. Taking the limit as t → 0+ , hemicontinuity gives ⟨x⋆o − A(xo ), z⟩ ≤ 0 for all z ∈ X, and replacing z by −z we obtain x⋆o = A(xo ). 9.2.4 Surjectivity of monotone operators and the Minty–Browder theory In this section, we consider the question of surjectivity of monotone operators A : D(A) ⊂ X → X ⋆ in the case where X is a separable reflexive Banach space. Even though we will consider more general results, concerning the surjectivity of a more general class of operators (see Section 9.3.4), we include this discussion here as a good point to revisit the Faedo–Galerkin method which is an important approximation method (already encountered in Chapter 7, see Example 7.5.4) in a more general and abstract framework. Our approach follows [97] (see also [85]). Definition 9.2.11. An operator A : D(A) ⊂ X → X ⋆ is called coercive if ‖x‖ → ∞.

⟨A(x),x⟩ ‖x‖

→ ∞ as

The linear operators which are generated by bilinear coercive forms (see Section 7.3) are coercive operators. An example of a nonlinear coercive operator is the p-Laplace operator (see Example 9.2.3). The following important theorem holds for coercive and monotone operators, and guarantees the solvability of the operator equation A(x) = x⋆ for any x⋆ ∈ X ⋆ . Theorem 9.2.12 (Minty–Browder). Let X be a reflexive and separable Banach space, and A : D(A) ⊂ X → X ⋆ be a monotone, hemicontinuous and coercive operator, with D(A) = X. Then R(A) = X ⋆ . Moreover, for any x⋆ ∈ X ⋆ , the solution set S(x⋆ ) := {x ∈ X : A(x) = x⋆ } is closed, bounded and convex. If, furthermore, A is strictly monotone, then for any x⋆ ∈ X ⋆ , the solution is unique. Proof. The proof is broken up into 5 steps: 1. Since X is separable, there exists an increasing sequence1 {Xn : n ∈ ℕ} of finite dimensional subspaces of X with ⋃∞ n=1 Xn = X. For each n, fixed, let jn : Xn → X be the injection mapping of Xn into X, let jn⋆ : X ⋆ → Xn⋆ be its adjoint map, and define An : Xn → Xn⋆ by An = jn∗ Ajn . Since A is coercive, and ⟨An (x), x⟩ = ⟨A(x), x⟩ for every x ∈ Xn , we conclude that An is also coercive. On the other hand, by the hemicontinuity of A we conclude that 1 Since X is separable there exists a set Φn := {zi : i = 1, . . . , n} whose span is dense in X. Then the sequence {Xn : n ∈ ℕ} can be constructed by setting Xn := span{z1 , . . . , zn } which is an n-dimensional subspace of X. In fact, Xn can be considered as isomorphic to ℝn .

9.2 Monotone operators | 415

An is continuous. By a straightforward application of Brouwer’s fixed-point theorem (see Proposition 3.2.13), we have that R(An ) = Xn⋆ . Then, for any given x⋆ ∈ X ⋆ and any n ∈ ℕ, there exists xn ∈ Xn such that An (xn ) = jn∗ x⋆ .

(9.5)

We have that ⟨A(xn ), xn ⟩ ⟨An (xn ), xn ⟩ ⟨jn∗ x⋆ , xn ⟩ ⟨x⋆ , xn ⟩ 󵄩󵄩 ⋆ 󵄩󵄩 = = = ≤ 󵄩󵄩x 󵄩󵄩X ⋆ , ‖xn ‖ ‖xn ‖ ‖xn ‖ ‖xn ‖ and since A is coercive, it follows that ‖xn ‖ is bounded, i. e., there exists c1 > 0 such that ‖xn ‖ ≤ c1 for every n ∈ ℕ. 2. We will show that the sequence {A(xn ) : n ∈ ℕ} is also bounded. Indeed, since A is monotone, by Theorem 9.2.6 it is locally bounded, therefore, 󵄩 󵄩 ∃ r, c > 0 such that if ‖x‖ ≤ r then 󵄩󵄩󵄩A(x)󵄩󵄩󵄩 ≤ c.

(9.6)

By the monotonicity of A, we have ⟨A(xn ) − A(x), xn − x⟩ ≥ 0.

(9.7)

By (9.5), we have ⟨A(xn ), xn ⟩ = ⟨x⋆ , xn ⟩, which in turn implies that 󵄨󵄨 󵄨 󵄩 ⋆󵄩 󵄩 ⋆󵄩 󵄨󵄨⟨A(xn ), xn ⟩󵄨󵄨󵄨 ≤ 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ ‖xn ‖ ≤ c1 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ .

(9.8)

We have that,2 using (9.7), 󵄩󵄩 󵄩 󵄩󵄩A(xn )󵄩󵄩󵄩X ⋆ = ≤

sup

x∈X, ‖x‖=r

sup

x∈X, ‖x‖=r

1 ⟨A(xn ), x⟩ r 1 (⟨A(x), x⟩ + ⟨A(xn ), xn ⟩ − ⟨A(x), xn ⟩) r

1 󵄩 󵄩 ≤ (c r + c1 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ + c c1 ) < ∞, r

and using (9.6) we conclude that {A(xn ) : n ∈ ℕ} is bounded. 3. By the reflexivity of X, we may assume that there exists a subsequence of {xn : n ∈ ℕ}, let us say {xnk : k ∈ ℕ} such that xnk ⇀ xo in X. The proof will be complete if we show that A(xo ) = x⋆ . We assert that A(xnk ) ⇀ x⋆ . Since the sequence {A(xnk ) : k ∈ ℕ} is bounded, it suffices to show that ⟨A(xnk ), x⟩ → ⟨x⋆ , x⟩ for each x ∈ ⋃∞ n=0 Xn , since the union is dense 2 Express ⟨A(xn ), x⟩ = ⟨A(xn ), xn ⟩ + ⟨A(x), x − xn ⟩ − ⟨A(xn ) − A(x), xn − x⟩ ≤ ⟨A(xn ), xn ⟩ + ⟨A(x), x − xn ⟩, where we used (9.7).

416 | 9 Monotone-type operators in X. Let x be any element of ⋃∞ n=1 Xn . Then x ∈ Xm , for some m, and since {Xn : n ∈ ℕ} is increasing, x ∈ Xn for every n ≥ m. Hence, for nk ≥ m, ⟨A(xnk ), x⟩ = ⟨Ank (xnk ), x⟩ = ⟨jn∗k x⋆ , x⟩ = ⟨x⋆ , x⟩, i. e., A(xnk ) ⇀ x⋆ . Finally, we have ⟨A(xnk ), xnk ⟩ = ⟨Ank (xnk ), xnk ⟩ = ⟨jn∗k x⋆ , xnk ⟩ = ⟨x⋆ , xnk ⟩ → ⟨x⋆ , xo ⟩.

(9.9)

By the monotonicity of A, we have ⟨A(x) − A(xnk ), x − xnk ⟩ ≥ 0,

∀ x ∈ X,

and for k → ∞, it follows from (9.9) that ⟨A(x) − x⋆ , x − xo ⟩ ≥ 0,

∀ x ∈ X.

We now set x = xo + t z, for arbitrary z ∈ X, in the above to obtain ⟨A(xo + t z) − x⋆ , z⟩ ≥ 0,

∀ z ∈ X, ∀ t > 0.

Passing to the limit as t → 0+ and using the hemicontinuity of A we conclude that A(xo ) = x⋆ . 4. Consider any x⋆ ∈ X ⋆ . We have just proved that S(x⋆ ) ≠ 0. Since A is coercive, ⋆ S(x ) is bounded. We now prove that it is also convex. We let x1 , x2 ∈ S(x⋆ ), i. e., A(xi ) = x⋆ , i = 1, 2, set xt = tx1 + (1 − t)x2 , t ∈ [0, 1], and consider any z ∈ X. Then ⟨x⋆ − A(z), xt − z⟩ = ⟨x⋆ − A(z), t(x1 − z)⟩ + ⟨x⋆ − A(z), (1 − t)(x2 − z)⟩

= t⟨A(x1 ) − A(z), x1 − z⟩ + (1 − t)⟨A(x2 ) − A(z), x2 − z⟩ ≥ 0,

∀z ∈ X,

by the monotonicity of A, so by Lemma 9.2.10, we have A(xt ) = x⋆ , therefore, xt ∈ S(x⋆ ); hence, S(x⋆ ) is convex. We now show that S(x⋆ ) is closed.3 Fix any x⋆ ∈ X ⋆ and let {xn : n ∈ ℕ} ⊂ S(x⋆ ) (i. e., A(xn ) = x⋆ ) with xn → x. Then, for each z ∈ X, (using first the fact that xn → x and then the fact that xn ∈ S(x⋆ )) we have that ⟨x⋆ − A(z), x − z⟩ = lim ⟨x⋆ − A(z), xn − z⟩ = lim ⟨A(xn ) − A(z), xn − z⟩ ≥ 0, n→∞

n→∞

where for the last inequality we have used the monotonicity of the operator A. Therefore, for each z ∈ X it holds that ⟨x⋆ − A(z), x − z⟩ ≥ 0 which in turn, by Lemma 9.2.10, implies that A(x) = x⋆ . Therefore, x ∈ S(x⋆ ) and S(x⋆ ) is closed. 3 Since S(x⋆ ) is convex, we do not have to worry about discriminating between strong and weak closedness.

9.3 Maximal monotone operators |

417

5. Under the extra assumption that A is strictly monotone, let x1 ≠ x2 be two solutions of A(x) = x⋆ . Then, A(x1 ) = x⋆ = A(x2 ) and 0 < ⟨A(x1 ) − A(x2 ), x1 − x2 ⟩ = 0, which is a contradiction; hence, we have uniqueness. Example 9.2.13. The surjectivity results provided by the Lions–Stampacchia and Lax–Milgram theorems in Chapter 7 can be considered as special cases of the Minty– Browder surjectivity result. For instance, if a : X × X → ℝ is a bilinear coercive form then the linear operator A : X → X ⋆ defined by ⟨Ax, z⟩ = a(x, z) for every z ∈ X, is a coercive operator, satisfying the conditions required by Theorem 9.2.12, so that the operator equation Ax = z⋆ admits a unique solution for every z⋆ ∈ X ⋆ . Important partial differential equations such as the Poisson equation can be solved using such abstract techniques, extending the results obtained in Chapters 6 and 7. ◁ Example 9.2.14. Let 𝒟 ⊂ ℝd be a bounded domain of sufficiently smooth boundary ⋆ and consider the p-Laplace Poisson equation −Δp u = f for a given f ∈ Lp (𝒟), with

+ p1⋆ = 1, p > 1. Then the nonlinear operator A := −Δp : W01,p (𝒟) → W0−1,p (𝒟) is continuous, coercive, bounded and monotone (see Section 6.8 and Example 9.2.3); hence, by Theorem 9.2.12, R(A) = X ⋆ , so the equation −Δp u = f admits a unique solution in ⋆

1 p

W01,p (𝒟) for every f ∈ W −1,p (𝒟). The solvability of this equation was treated using the direct method of the calculus of variations in Section 6.8. ◁ ⋆

9.3 Maximal monotone operators 9.3.1 Maximal monotone operators definitions and examples As usual, X will be a Banach space, X ⋆ its dual and by ⟨⋅, ⋅⟩ we will denote the duality pairing between them. Definition 9.3.1. A monotone operator A : D(A) ⊂ X → 2X is called maximal monotone if it has the following property: ⋆

Gr(A) ⊂ Gr(A󸀠 ) with A󸀠 monotone implies that A = A󸀠 . Gr(A) is called a maximal monotone graph, or we say that A generates a maximal monotone graph. In other words, an operator is maximal monotone if it has no proper monotone extension, i. e., any monotone extension of A coincides with itself; for any monotone operator A󸀠 , Gr(A) ⊂ Gr(A󸀠 ) implies A󸀠 = A (recall that in our notation A ⊂ B includes the possibility that A = B). An alternative way to put it is that A is maximal monotone if its graph is not properly contained in the graph of any other monotone operator,

418 | 9 Monotone-type operators or equivalently, the graph of A is maximal with respect to inclusion among graphs of monotone operators. As an illustrative example of that consider the set valued map ϕ : ℝ → 2ℝ defined by ϕ(x) = 0 1{x0} , which is maximal monotone if and only if A = [0, 1] (see [90]). The following characterization of maximal monotonicity is simple, but often very helpful when trying to determine whether an element of (x, x⋆ ) ∈ X × X ⋆ belongs to the graph of a maximal monotone operator A or not. Proposition 9.3.2. A monotone operator A : D(A) ⊂ X → 2X is maximal monotone if and only if it satisfies the property ⋆

⟨x⋆ − z⋆ , x − z⟩ ≥ 0,

∀ (x, x⋆ ) ∈ Gr(A) implies (z, z⋆ ) ∈ Gr(A).

(9.10)

Proof. Suppose that A is maximal monotone but (9.10) does not hold, so that there exists (z, z⋆ ) ∈ ̸ Gr(A) with the property ⟨x⋆ − z⋆ , x − z⟩ ≥ 0, ∀ (x, x⋆ ) ∈ Gr(A). Then Gr(A) ∪ (z, z⋆ ) is a monotone graph with the property that Gr(A) ⊂ Gr(A) ∪ (z, z⋆ ), a fact that contradicts the maximal monotonicity of A. For the converse, suppose that (9.10) holds but A is not maximal monotone. Then there must be a monotone graph G󸀠 which is a proper extension of Gr(A). If (z, z⋆ ) ∈ G󸀠 \ Gr(A), then (z, z⋆ ) ∈ ̸ Gr(A) (i. e., z ∈ ̸ D(A) and z⋆ ∈ ̸ A(z)) but by monotonicity of G󸀠 , ⟨x⋆ − z⋆ , x − z⟩ ≥ 0, ∀ (x, x⋆ ) ∈ Gr(A). This contradicts the hypothesis that (9.10) holds. We will show shortly that many operators we have encountered so far enjoy the maximal monotonicity property such as, for instance, the subdifferential of convex functions, linear operators A : X → X ⋆ such that ⟨Ax, x⟩ ≥ 0 for every x ∈ X, or the p-Laplace operator. Example 9.3.3. It is obvious that if A : D(A) ⊂ X → 2X is a maximal monotone operator then for each λ > 0, λ A is also maximal monotone. ◁ ⋆

Example 9.3.4. Let X be a reflexive Banach space. An operator A defines a maximal monotone graph on X × X ⋆ if and only A−1 defines a maximal monotone graph on X ⋆ × X. The result follows from the fact that the inverse operator A−1 is the operator that has the inverse graph of A, i. e., x ∈ A−1 (x⋆ ) if and only if x⋆ ∈ A(x). ◁

9.3.2 Properties of maximal monotone operators The next proposition links maximal monotonicity of the operator A with properties of the set A(x).

9.3 Maximal monotone operators |

419

Proposition 9.3.5. Let A : D(A) ⊂ X → 2X be a maximal monotone operator. Then, (i) For each x ∈ D(A), A(x) is a convex and weak⋆ closed subset of X ⋆ (if X is reflexive it is weakly closed). (ii) If x ∈ int(D(A)) then A(x) is weak⋆ compact (if X is reflexive it is weakly compact). (iii) The graph of A is closed in X×Xw⋆⋆ and in Xw ×X ⋆ . This property is sometimes referred to as demiclosedness. (iv) Let X be reflexive, and consider a sequence {(xn , x⋆n ) : n ∈ ℕ} ⊂ Gr(A) such that xn ⇀ x, x⋆n ⇀ x⋆ and lim supn ⟨x⋆n − x⋆ , xn − x⟩ ≤ 0. Then (x, x⋆ ) ∈ Gr(A) and ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. (v) Let X be reflexive and consider a sequence {(xn , x⋆n ) : n ∈ ℕ} ⊂ Gr(A) such that xn ⇀ x, x⋆n ⇀ x⋆ and lim supn,m ⟨x⋆n − x⋆m , xn − xm ⟩ ≤ 0. Then (x, x⋆ ) ∈ Gr(A) and ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. ⋆

Proof. (i) Let x⋆1 , x⋆2 ∈ A(x) and set x⋆t = (1 − t)x⋆1 + tx⋆2 , t ∈ [0, 1]. Then, for any (z, z⋆ ) ∈ Gr(A) we have ⟨x⋆t − z⋆ , x − z⟩ = (1 − t)⟨x⋆1 − z⋆ , x − z⟩ + t⟨x⋆2 − z⋆ , x − z⟩ ≥ 0. By maximal monotonicity of A (see Proposition 9.3.2), we get that x⋆t ∈ A(x); hence, A(x) is a convex set. We now prove that A(x) is weak⋆ closed. To this end, consider a net {x⋆α : α ∈ ℐ } ⊆ A(x) that converges weak⋆ (resp., weakly if X is reflexive) to x⋆ in X ⋆ . We then have ⟨x⋆α − z⋆ , x − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

and passing to the limit we obtain that ⟨x⋆ − z⋆ , x − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

and so invoking, Proposition 9.3.2, we deduce that x⋆ ∈ A(x); hence the weak⋆ (resp., weak) closedness of A(x). (ii) Since A is maximal monotone, it is also monotone, hence, by Theorem 9.2.6, we have that it is locally bounded at each interior point of D(A). Then, using the weak⋆ compactness result (Alaoglu, Theorem 1.1.35(iii)) or the Eberlein–Šmulian theorem (Theorem 1.1.58) in case X is reflexive, we obtain the stated result. ⋆ (iii) Consider a net {(xα , x⋆α ) : α ∈ ℐ } ⊂ Gr(A) such that xα → x in X and x⋆α ⇀ x⋆ in X ⋆ . For any (z, z⋆ ) ∈ Gr(A), it holds that ⟨x⋆α − z⋆ , xα − z⟩ ≥ 0 and passing to the limit, ⟨x⋆ − z⋆ , x − z⟩ ≥ 0 so that, by application of Proposition 9.3.2, we deduce that (x, x⋆ ) ∈ Gr(A), hence, the X × Xw⋆⋆ closedness of the graph of A. For the other property, consider a net (xα , x⋆α ) ∈ Gr(A) such that xα ⇀ x in X and x⋆α → x⋆ in X ⋆ and proceed accordingly. (iv) Since (xn , x⋆n ) ∈ Gr(A), we have ⟨z⋆ − x⋆n , z − xn ⟩ ≥ 0,

∀(z, z⋆ ) ∈ Gr(A), ∀ n ∈ ℕ.

420 | 9 Monotone-type operators Taking the limit superior and using the property that lim supn ⟨x⋆n , xn ⟩ ≤ ⟨x⋆ , x⟩ we obtain ⟨z⋆ − x⋆ , z − x⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

hence, by Proposition 9.3.2, we deduce that (x⋆ , x) ∈ Gr(A). Now, by the monotonicity of A we have ⟨x⋆ − x⋆n , x − xn ⟩ ≥ 0,

∀ n ∈ ℕ.

Taking the limit inferior and using the facts that xn ⇀ x and x⋆n ⇀ x⋆ , we deduce that ⟨x⋆ , x⟩ ≤ lim infn ⟨x⋆n , xn ⟩. This combined with the relevant inequality for the limsup implies that ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. (v) Using the monotonicity of A, we have that (xn , x⋆n ), (xm , x⋆m ) ∈ Gr(A) implies ⋆ ⟨xn − x⋆m , xn − xm ⟩ ≥ 0 for every n, m ∈ ℕ, so that combined with the hypothesis we obtain that lim⟨x⋆n − x⋆m , xn − xm ⟩ = 0.

(9.11)

n,m

To conclude the proof, we take a subsequence {(xnk , x⋆nk ) : k ∈ ℕ} such that ⟨x⋆nk , xnk ⟩ → lim supn ⟨x⋆n , xn ⟩ =: μ and work along this subsequence using maximal monotonicity, repeating the steps of (iv). In particular, let {nk : k ∈ ℕ} be a subsequence of {n} such that ⟨x⋆nk , xnk ⟩ → μ. Then, from (9.11) we have that 0 = lim [ lim ⟨x⋆nk − x⋆nℓ , xnk − xnℓ ⟩] = 2μ − 2⟨x⋆ , x⟩. k→∞ ℓ→∞

Hence, μ = ⟨x⋆ , x⟩ = limn ⟨x⋆n , xn ⟩. Since A is monotone, this implies that ⟨z⋆ − x⋆ , z − x⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A).

By the maximal monotonicity of A, (x, x⋆ ) ∈ Gr(A). Remark 9.3.6. If D(A) = X and X is reflexive then, from Proposition 9.3.5, A(x) is a weakly compact and convex subset of X ⋆ . We know that in general weak convergence in X and X ⋆ does not imply continuity of the duality pairing between the two spaces, i. e., if xn ⇀ x in X and x⋆n ⇀ x⋆ then it is not necessarily true that ⟨x⋆n , xn ⟩ → ⟨x⋆ , x⟩. However, if x⋆n ∈ A(xn ) where A is a maximal monotone operator, and lim supn ⟨x⋆n , xn − x⟩ ≤ 0, then the continuity property of the duality pairing holds. Theorem 9.3.7. Let X be a reflexive Banach space and A : D(A) ⊂ X → 2X be a maximal monotone operator with D(A) = X. Then A is upper semicontinuous from X into Xw⋆ (in the sense of Definition 1.7.2). ⋆

9.3 Maximal monotone operators |

421

Proof. Consider any x ∈ X. Assume that for a given open weak neighborhood V of A(x) there exists a sequence {xn : n ∈ ℕ} ⊂ X with xn → x and x⋆n ∈ A(xn ) such that x⋆n ∉ V for all n. Since, by Theorem 9.2.6, A is locally bounded at the point x, the sequence {x⋆n : n ∈ ℕ} is bounded in X ⋆ , so there exists a subsequence {x⋆nk : k ∈ ℕ} converging weakly to an element x⋆ . We have ⟨x⋆nk − z⋆ , xnk − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

so passing to the limit, we obtain ⟨x⋆ − z⋆ , x − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A).

Since A is maximal monotone, x⋆ ∈ A(x). But, on the other hand, since V is weakly open, X ⋆ \V is weakly closed and this implies that x⋆ ∉ V, which is a contradiction.

9.3.3 Criteria for maximal monotonicity The next result provides conditions under which a monotone operator is maximal monotone. Theorem 9.3.8. Let X be a reflexive Banach space and A : D(A) ⊂ X → 2X be a monotone operator with D(A) = X, such that for each x ∈ X, A(x) is a nonempty, weakly closed and convex subset of X ⋆ . Suppose that for every x, z ∈ X the mapping t 󳨃→ A(x + tz) is upper semicontinuous4 from [0, 1] into Xw⋆ . Then A is maximal monotone. ⋆

Proof. We will use Proposition 9.3.2. Suppose that ⟨x⋆ − z⋆ , x − z⟩ ≥ 0,

∀ (x, x⋆ ) ∈ Gr(A).

We must show that z⋆ ∈ A(z). Suppose that z⋆ ∉ A(z). Since, by assumption, A(z) is a weakly closed and convex subset of X ⋆ , from the separation theorem we can find zo ∈ X such that ⟨z⋆ , zo ⟩ > ⟨z⋆o , zo ⟩,

∀z⋆o ∈ A(z).

Let xt = z + t zo , t ∈ [0, 1] and set U = {z⋆o ∈ X ⋆ : ⟨z⋆ , zo ⟩ > ⟨z⋆o , zo ⟩}. 4 in the sense of Definition 1.7.2

422 | 9 Monotone-type operators Clearly, U is a weak neighborhood of A(z). Since xt → z as t → 0+ , and A is upper semicontinuous for small enough t we have that A(xt ) ⊆ U. Let x⋆t ∈ A(xt ). Then for small t, we obtain that 0 ≤ ⟨x⋆t − z⋆ , xt − z⟩ = ⟨x⋆t − z⋆ , zo ⟩ < 0, which is a contradiction. So, z⋆ ∈ A(z) and thus, A is maximal monotone. Note that in passing to the limit we have used the local boundeness theorem for monotone operators (Theorem 9.2.6). Theorem 9.3.8 has the following interesting implication. Corollary 9.3.9. If X is a reflexive Banach space and A : D(A) ⊂ X → X ⋆ is monotone, hemicontinuous and D(A) = X, then A is maximal monotone. Example 9.3.10. The maximal monotonicity of the p-Laplacian, p > 1, can be proved using Corollary 9.3.9. Recall that the p-Laplacian can be defined in variational form by 󵄨 󵄨p−2 ⟨A(u), v⟩ = ∫󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 ∇u(x) ⋅ ∇v(x)dx, 𝒟

from which follows that A is continuous (details can be seen in the proof of Proposition 9.5.1). Since A is monotone (see Example 9.2.3), by Corollary 9.3.9, A is maximal monotone. ◁ For linear operators defined on the whole of X, monotonicity and maximal monotonicity are equivalent. Proposition 9.3.11. Let X be a reflexive Banach space. A linear operator A : D(A) ⊂ X → X ⋆ , with D(A) = X, is maximal monotone if and only if it is monotone. Proof. This follows from a direct application of Proposition 9.3.2. To this end, we take any fixed (z, z⋆ ) ∈ X ×X ⋆ such that ⟨A(x)−z⋆ , x−z⟩ ≥ 0 for every x ∈ X and we will prove that z⋆ = A(z). For arbitrary zo ∈ X, and t > 0, set x = z + tzo in the above inequality. This yields ⟨A(z+tzo )−z⋆ , (z+tzo )−z⟩ ≥ 0 or equivalently ⟨A(z)−z⋆ , zo ⟩+t⟨A(zo ), zo ⟩ ≥ 0, where we have used the linearity of A. Letting t → 0, we obtain that ⟨A(z) − z⋆ , zo ⟩ ≥ 0 for every zo ∈ X. Repeating the steps with zo replaced by −zo allows us to conclude that z⋆ = A(z). Alternatively, we could use directly Corollary 9.3.9.

9.3.4 Surjectivity results Maximal monotone operators enjoy some very interesting surjectivity results (see e. g. [18, 85, 86]) which make them very useful in applications.

423

9.3 Maximal monotone operators |

Theorem 9.3.12. Let X be a reflexive Banach space, A : D(A) ⊂ X → 2X a maximal monotone operator and B : D(B) ⊂ X → X ⋆ be a monotone, hemicontinuous, bounded and coercive operator with D(B) = X. Then R(A + B) = X ⋆ . ⋆

Proof. Since monotonicity is a property which is invariant under translations, we may assume5 that (0, 0) ∈ Gr(A). We must show that given any x⋆ ∈ X ⋆ the inclusion x⋆ ∈ A(x) + B(x) has a solution x ∈ D(A) ⊂ X. Since A is maximal monotone, this problem is equivalent to: given x⋆ ∈ X ⋆ , find x ∈ D(A) such that ⟨x⋆ − B(x) − z⋆ , x − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A).

(9.12)

First, we establish an a priori bound for the solutions of this problem. Let x ∈ D(A) be a solution of (9.12). Since (0, 0) ∈ Gr(A) we obtain ⟨x⋆ − B(x), x⟩ ≥ 0, which leads to ⟨B(x), x⟩ ≤ ‖x⋆ ‖X ⋆ ‖x‖. Since B is coercive, we conclude that there exists c > 0 such that ‖x‖ ≤ c. Let 𝒳F be the family of all finite dimensional subspaces XF of X, ordered by inclusion. For each XF ∈ 𝒳F , let jF : XF → X be the inclusion map and jF∗ : X ⋆ → XF⋆ its adjoint projection map. Let AF = jF⋆ A jF and BF = jF⋆ B jF . Then, clearly AF is monotone and BF is continuous since B is hemicontinuous and dim(XF ) < ∞. Also, let KF = XF ∩ B(0, c). Then, if x⋆F = jF⋆ x⋆ , from the Debrunner–Flor Theorem 3.5.1 there exists xF ∈ KF such that ⟨x⋆ − B(xF ) − z⋆ , xF − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A), z ∈ XF .

(9.13)

Note that ⟨BF (xF ), xF − z⟩ = ⟨B(xF ), xF − z⟩, ⟨jF⋆ z⋆ , xF − z⟩ = ⟨z⋆ , xF − z⟩ and ⟨jF⋆ x⋆ , xF − z⟩ = ⟨x⋆ , xF − z⟩. Since B is bounded, there exists c1 > 0 such that ‖B(xF )‖ ≤ c1 for all xF ∈ XF and XF ∈ 𝒳F . For any XF ∈ 𝒳F , we consider the set U(XF ) = ⋃ { (xF󸀠 , B(xF󸀠 )) ∈ X × X ⋆ : xF󸀠 solves (9.13) on XF󸀠 ∈ 𝒳F with XF󸀠 ⊃ XF }. w

It is easy to see, that the family {U(XF ) : XF ∈ 𝒳F } has the finite intersection property ̄ c) × B(0, ̄ c1 ) (see Proposition 1.8.1).6 Since for all XF ∈ 𝒳F it holds that U(XF ) ⊂ B(0, ⋆ and the latter is weakly compact in the reflexive space X × X , we deduce that there w w exists (x,̄ x̄⋆ ) ∈ ⋂XF ∈𝒳F U(XF ) where by A we denote the weak closure of a set A. From Proposition 1.1.61, we know that for any fixed XF ∈ 𝒳F , we can find a sequence {(x̄n , B(x̄n )) : n ∈ ℕ} ⊂ U(XF ) such that (x̄n , B(x̄n )) ⇀ (x,̄ x̄⋆ ) in X × X ⋆ , i. e., x̄n ⇀ x̄ and B(x̄n ) ⇀ x̄⋆ . We will show that x̄ is the required solution. 5 Otherwise for some (x0 , x⋆0 ) ∈ Gr(A), we consider the operator z 󳨃→ A1 (z) = A(z + x0 ) − x⋆0 and z 󳨃→ B1 (z) = B(z + x0 ) − x⋆0 for all z ∈ D(A) − x0 . 6 Consider any XF1 , XF2 ∈ 𝒳F , then for any XF3 ∈ 𝒳F such that XF1 ∪ XF2 ⊂ XF3 it holds that U(XF3 ) ⊂ U(XF1 ) ∩ U(XF2 ).

424 | 9 Monotone-type operators We first show that there exists a (z0 , z⋆0 ) ∈ Gr(A) such that ⟨x⋆ − x̄⋆ − z⋆0 , x − z0 ⟩ ≤ 0.

(9.14)

Suppose not, then ⟨x⋆ − x̄⋆ − z⋆ , x̄ − z⟩ > 0,

∀ (z, z⋆ ) ∈ Gr(A).

This implies that x⋆ − x̄⋆ ∈ A(x)̄ since A is maximal monotone. Then if z⋆ = x⋆ − x̄⋆ and z = x̄ we have ⟨x⋆ − x̄⋆ − z⋆ , x̄ − z⟩ = 0 which is a contradiction. From (9.13), we obtain ⟨x⋆ − B(x̄n ) − z⋆ , x̄n − z⟩ ≥ 0,

(9.15)

∀ (z, z⋆ ) ∈ Gr(A), z ∈ XF .

We now consider XF0 ∈ 𝒳F such that z0 ∈ XF0 , where the fixed element z0 satisfies (9.14) (we note that ⋃XF ∈𝒳F XF = X), and restrict the above for every z ∈ XF0 , so that taking the limit superior we obtain lim sup⟨B(x̄n ), x̄n ⟩ ≤ ⟨x̄⋆ , z⟩ + ⟨x⋆ − z⋆ , x̄ − z⟩, n

∀ (z, z⋆ ) ∈ Gr(A).

Setting z = z0 and z⋆ = z⋆0 , we obtain lim sup⟨B(x̄n ), x̄n ⟩ ≤ ⟨x̄⋆ , z0 ⟩ + ⟨x⋆ − z⋆0 , x̄ − z0 ⟩. n

We claim that ̄ ⟨x̄⋆ , z0 ⟩ + ⟨x⋆ − z⋆0 , x̄ − z0 ⟩ ≤ ⟨x̄⋆ , x⟩. Suppose not, then ⟨x⋆ − x̄⋆ − z⋆0 , x̄ − z0 ⟩ > 0, which contradicts (9.14). So, lim supn ⟨B(x̄n ), x̄n ⟩ ≤ ⟨x̄⋆ , x⟩̄ and since B is maximal monotone, by Proposī It then follows by (9.15) that tion 9.3.5(iv), (x,̄ x̄⋆ ) ∈ Gr(B) and ⟨B(x̄n ), x̄n ⟩ → ⟨x̄⋆ , x⟩. ⟨x⋆ − B(x)̄ − z⋆ , x̄ − z⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

so that x̄ has the required property. This completes the proof. For the remaining part of this section, we assume that J : X → X ⋆ is the duality map corresponding to a uniformly convex renorming of both X and X ⋆ (see Theorem 2.6.16). From Theorem 9.3.12, we obtain an important characterization of maximal monotone operators. Theorem 9.3.13. Let X be a reflexive Banach space, and A : D(A) ⊂ X → 2X be a monotone operator. Then A is maximal monotone if and only if R(A + λJ) = X ⋆ for all λ > 0. ⋆

9.3 Maximal monotone operators |

425

Proof. Let A be maximal monotone. The operator J is continuous, (strictly) monotone, coercive and bounded (see Theorem 2.6.17). Then we may apply Theorem 9.3.12 with B = λ J, for any λ > 0 and infer that R(A + λ J) = X ⋆ . Conversely, assume that for any λ ≥ 0, R(A + λ J) = X ⋆ and suppose that A is not maximal monotone. Then, by Proposition 9.3.2, there exists (zo , z⋆o ) ∈ (X × X ⋆ ) \ Gr(A) such that ⟨z⋆ − z⋆o , z − zo ⟩ ≥ 0,

(9.16)

∀ (z, z⋆ ) ∈ Gr(A).

Since R(A + λJ) = X ⋆ for all λ > 0, there exists (xo , x⋆o ) ∈ Gr(A) such that (9.17)

λJ(xo ) + x⋆o = λJ(zo ) + z⋆o . We set (z, z⋆ ) = (xo , x⋆o ) in (9.16), so that (9.17)

0 ≤ ⟨x⋆o − z⋆o , xo − zo ⟩ = λ⟨J(zo ) − J(xo ), xo − zo ⟩ ≤ 0, where the last inequality follows by the monotonicity of J. Hence, 0 = ⟨J(zo )−J(xo ), zo − xo ⟩, and since J (recall Theorem 2.6.17) is strictly monotone xo = zo . Combining this with (9.17), we see that λJ(zo ) − λJ(xo ) = x⋆o − z⋆o = 0, so x⋆o = z⋆o . But since x⋆o ∈ A(xo ) = A(zo ) and (by assumption) z⋆o ∈ ̸ A(zo ) we reach a contradiction. Example 9.3.14. Let 𝒟 ⊂ ℝd be open and let A : ℝ → 2ℝ be a maximal monotone 2 ̂ : L2 (𝒟) → 2L (𝒟) be the realization of A on the operator with (0, 0) ∈ Gr(A). Let A ̂ Hilbert space L2 (𝒟) defined by A(u) = {v ∈ L2 (𝒟) : v(x) ∈ A(u(x)) a. e.}. We claim that ̂ A is maximal monotone. Since the monotonicity of  is obvious, it suffices to show that for any λ > 0, R(λI + ̂ = L2 (𝒟). Let v ∈ L2 (𝒟), so there exists u(x), unique such that v(x) ∈ λu(x) + Au(x), A) a. e. It then follows (see Example 9.2.2) that u(x) = (λI + A)−1 v(x), a. e. Observe that u is measurable. Also since (0, 0) ∈ Gr(A), and (λI + A)−1 is nonexpansive, we have that |u(x)| = |(λI + A)−1 v(x)| ≤ |v(x)| a. e. So v ∈ L2 (𝒟) and we have established the ̂ If 𝒟 is bounded, then we can drop the hypothesis that (0, 0) ∈ Gr(A) maximality of A. ∞ since L (𝒟) ⊂ L2 (𝒟). ◁ We are now ready to prove the main surjectivity result on maximal monotone operators. Theorem 9.3.15. Let X be a reflexive Banach space and A : D(A) ⊂ X → 2X a maximal monotone and coercive operator. Then R(A) = X ⋆ . ⋆

Proof. Since monotonicity is a property which is invariant under translations, it is sufficient to prove that 0 ∈ R(A). Let {ϵn : n ∈ ℕ} be a sequence of positive numbers, such that ϵn → 0 as n → ∞. By Theorem 9.3.13, there exists (xn , x⋆n ) ∈ Gr(A) such that x⋆n + ϵn J(xn ) = 0.

(9.18)

426 | 9 Monotone-type operators Since A is coercive there exists c > 0 such that ‖xn ‖ ≤ c. By the reflexivity of X, there exists xo ∈ X such that xn ⇀ xo in X (up to subsequences). On the other hand, by (9.18), we have 󵄩 󵄩 󵄩󵄩 ⋆ 󵄩󵄩 󵄩󵄩xn 󵄩󵄩X ⋆ = ϵn 󵄩󵄩󵄩J(xn )󵄩󵄩󵄩X ⋆ = ϵn ‖xn ‖ ≤ ϵn c → 0,

as n → ∞,

i. e., x⋆n → 0 in X ⋆ . By the monotonicity of A, we have ⟨z⋆ − x⋆n , z − xn ⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A),

and taking the limit as n → ∞ this implies that ⟨z⋆ , z − xo ⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(A).

Hence, since A is maximal monotone we conclude that 0 ∈ A(xo ). The proof is complete. Theorem 9.3.16 (Browder). Let X be a reflexive Banach space and A : D(A) ⊂ X → 2X be a maximal monotone operator. Then R(A) = X ⋆ , if and only if A−1 is locally bounded. ⋆

Proof. If A is maximal monotone so is A−1 (see Example 9.3.4); hence, by Theorem 9.2.6 A−1 is locally bounded on X ⋆ . Conversely, assume that A−1 is locally bounded on X ⋆ . If suffices to show that R(A) is both open and closed in X ⋆ . We first show that R(A) is closed in X ⋆ . Let {x⋆n : n ∈ ℕ} ⊂ X ⋆ be a sequence such that x⋆n ∈ A(xn ) and x⋆n → x⋆ in X ⋆ . Since A−1 is locally bounded, {xn : n ∈ ℕ} is a bounded sequence. By the reflexivity of X, there is a subsequence (denote the same for simplicity) such that xn ⇀ x. Hence, (x, x⋆ ) ∈ Gr(A) (see Proposition 9.3.5(iii)). We show next that R(A) is open in X ⋆ . Let x⋆ ∈ R(A), i. e., x⋆ ∈ A(x) for some x ∈ D(A). Since the maximal monotonicity remains invariant under translations, we may without loss of generality assume that x = 0. Let r > 0 such that A−1 is bounded on BX ⋆ (x⋆ , r). We claim that r if z⋆ ∈ BX ⋆ (x⋆ , ), then z⋆ ∈ R(A), 2

(9.19)

from which it follows directly that R(A) is open. To show this we work as follows: By Theorem 9.3.13, there exists for any λ > 0 a solution xλ ∈ D(A) of the equation λJ(xλ ) + x⋆λ = z⋆ ,

x⋆λ ∈ A(xλ ).

Since (0, x⋆ ) ∈ Gr(A), by the monotonicity of A, we have ⟨z⋆ − λJ(xλ ) − x⋆ , xλ − 0⟩ ≥ 0,

(9.20)

9.3 Maximal monotone operators |

427

which implies that ‖z⋆ − x⋆ ‖X ⋆ ‖xλ ‖ − λ‖xλ ‖2 ≥ 0; hence, r 󵄩 󵄩 λ‖xλ ‖ ≤ 󵄩󵄩󵄩z⋆ − x⋆ 󵄩󵄩󵄩X ⋆ < , 2

∀ λ > 0.

From (9.20), by rearranging and taking the norm, we have r 󵄩󵄩 ⋆ ⋆󵄩 󵄩󵄩z − xλ 󵄩󵄩󵄩X ⋆ = λ‖xλ ‖ < , 2

(9.21)

and thus from (9.20) and (9.21) we have 󵄩󵄩 ⋆ 󵄩 ⋆ 󵄩 ⋆ ⋆󵄩 ⋆󵄩 ⋆󵄩 󵄩󵄩xλ − x 󵄩󵄩󵄩X ⋆ ≤ 󵄩󵄩󵄩xλ − z 󵄩󵄩󵄩X ⋆ + 󵄩󵄩󵄩z − x 󵄩󵄩󵄩X ⋆ < r,

∀ λ > 0.

Since A−1 is bounded on BX ⋆ (x⋆ , r), the set of solutions {xλ ∈ A−1 (x⋆λ ) : λ > 0} remains bounded and 󵄩󵄩 ⋆ ⋆󵄩 󵄩󵄩z − xλ 󵄩󵄩󵄩 = λ‖xλ ‖ → 0,

as λ → 0.

Hence, since R(A) is closed in X ⋆ we have that z⋆ ∈ R(A) which proves claim (9.19). The proof is complete. Remark 9.3.17. A simple consequence of Theorem 9.3.16 is that if A is maximal monotone and D(A) is bounded, then A is surjective. 9.3.5 Maximal monotonicity of the subdifferential and the duality map The subdifferential and the duality map are important examples of maximal monotone operators. 9.3.5.1 Maximal monotonicity of the subdifferential The maximal monotonicity of the subdifferential was shown in [99]. The proof we present here is an alternative proof, due to [102] (see also [6]). Theorem 9.3.18. Let X be a Banach space and φ : X → ℝ ∪ {+∞} be a lower semi⋆ continuous proper convex function. Then the subdifferential 𝜕φ : X → 2X is a maximal monotone operator. Proof. Consider any (x, x⋆ ) ∈ X × X ⋆ , fixed, such that ⟨z⋆ − x⋆ , z − x⟩ ≥ 0,

∀ (z, z⋆ ) ∈ Gr(𝜕φ).

(9.22)

If we show that (x, x⋆ ) ∈ Gr(𝜕φ) then, by Proposition 9.3.2, 𝜕φ is a maximal monotone operator. Define φx : X → ℝ by z 󳨃→ φx (z) = φ(z + x) and consider the perturbed function ψx : X → ℝ, defined by z 󳨃→ ψx (z) := φx (z) + 21 ‖z‖2 . Since the Legendre–Fenchel

428 | 9 Monotone-type operators conjugate of φx , φ⋆x is a proper function, there exists x⋆o ∈ X ⋆ for which φ⋆x (x⋆o ) is finite. We claim that for every z ∈ X, ⟨x⋆ , z⟩ − ψx (z) < ∞ so that ψ⋆x (x⋆ ) < ∞. Indeed, 1 1 ⟨x⋆ , z⟩ − ψx (z) = ⟨x⋆ , z⟩ − φ(z + x) − ‖z‖2 = ⟨x⋆ , z⟩ − φx (z) − ‖z‖2 . 2 2

(9.23)

By the definition of φ⋆x , it holds that φ⋆x (x⋆o ) ≥ ⟨x⋆o , z⟩ − φx (z), so that by (9.23) 1 ⟨x⋆ , z⟩ − ψx (x) ≤ ⟨x⋆ , z⟩ − ⟨x⋆o , z⟩ + φ⋆x (x⋆o ) − ‖z‖2 2 1 = φ⋆x (x⋆o ) + ⟨x⋆ − x⋆o , z⟩ − ‖z‖2 . 2

(9.24)

On the other hand, ⟨x⋆ − x⋆o , z⟩ − 21 ‖z‖2 ≤ 21 ‖x⋆ − x⋆o ‖2 , as can be easily seen by noting that the function ϕ : X → ℝ defined by ϕ(z) = 21 ‖z‖2 has Legendre–Fenchel transform ϕ⋆ (z⋆ ) = 21 ‖z⋆ ‖2 and we then use the definition of the Legendre–Fenchel transform to note that 21 ‖z⋆ ‖2 ≥ ⟨z⋆ , z⟩ − 21 ‖z‖2 for every z⋆ ∈ X ⋆ and every z ∈ X, applying that for the choice z⋆ = x⋆ − x⋆o . Substituting that in (9.24) yields the estimate 1󵄩 󵄩2 ⟨x⋆ , z⟩ − ψx (x) ≤ φ⋆x (x⋆o ) + 󵄩󵄩󵄩x⋆ − x⋆o 󵄩󵄩󵄩 < ∞. 2 Having guaranteed that ψ⋆x (x⋆ ) < ∞ and since by definition ψ⋆x (x⋆ ) = supx∈X (⟨x⋆ , x⟩ − ψx (x)) ≥ ⟨x⋆ , x⟩ − ψx (x) for any x ∈ X, whereas given any ϵn = n12 , n ∈ ℕ we may find xn ∈ X such that ψ⋆x (x⋆ ) − ϵn ≤ (⟨x⋆ , xn ⟩ − ψx (xn )). Combining the above ψx (x) − ψx (xn ) +

1 1 − ⟨x⋆ , x⟩ ≥ −ψ⋆x (x⋆ ) − ψx (xn ) + 2 ≥ −⟨x⋆ , xn ⟩, 2 n n

∀ x ∈ X.

(9.25)

Clearly, upon rearrangement (recall Definition 4.5.1), this means that x⋆ belongs to the approximate subdifferential of ψx , in fact that x⋆ ∈ 𝜕ϵn ψx (xn ). We may thus apply the Brøndsted– Rockafellar approximation Theorem 4.5.3 to guarantee the existence of a sequence {(zn , z⋆n ) : n ∈ ℕ} ⊂ X × X ⋆ such that z⋆n ∈ 𝜕ψx (zn ),

‖zn − xn ‖
0, consider the inclusion 0 ∈ J(xλ − x) + λA(xλ ) which, by Theorem 9.3.13, has a solution xλ ∈ D(A), i. e., there exists x⋆λ ∈ A(xλ ) such that J(xλ − x) + λx⋆λ = 0. We claim that this solution is unique. Indeed, suppose that xi,λ ∈ D(A), i = 1, 2, satisfy J(xi,λ − x) + λ x⋆i,λ = 0,

x⋆i,λ ∈ A(xi,λ ),

i = 1, 2.

Then, by the monotonicity of A, 0 ≤ λ⟨x⋆1,λ − x⋆2,λ , x1,λ − x2,λ ⟩ = ⟨J(x2,λ − x) − J(x1,λ − x), (x1,λ − x) − (x2,λ − x)⟩ ≤ 0, with the last inequality arising by the monotonicity of J. Since J is strictly monotone (see Theorem 2.6.17), we conclude that x1,λ = x2,λ . A similar argument implies that the corresponding x⋆λ ∈ A(xλ ) for which J(xλ − x) + λx⋆λ = 0 is also unique. Therefore, given λ > 0, to each x ∈ X we may associate a unique element xλ ∈ X and a unique element x⋆λ ∈ A(xλ ) such that J(xλ − x) + λx⋆λ = 0. The above observations lead to the following definition. Definition 9.3.21 (Resolvent operator and Yosida approximation). Let X be a reflexive Banach space, X ⋆ its dual, ⟨⋅, ⋅⟩ the duality pairing among them (assuming further a renorming such that both X and its dual X ⋆ are locally uniformly convex), J : X → ⋆ X ⋆ the (single valued) duality map, and A : D(A) ⊂ X → 2X a maximal monotone (possibly multivalued) operator. For each x ∈ X and λ > 0 consider the solution, xλ ∈ X, of the operator inclusion 0 ∈ J(xλ − x) + λA(xλ ).

(9.28)

(i) The family of (single valued) operators Rλ : X → D(A) defined for each λ > 0 by Rλ (x) := xλ , is called the family of resolvent operators of A. 7 Recall that a locally uniformly convex Banach space is also strictly convex (see Section 2.6.1.3); hence by Theorem 2.6.17 the duality map J is single valued.

9.3 Maximal monotone operators |

431

(ii) The family of (single valued) operators Aλ : X → X ⋆ defined by Aλ (x) :=

1 J(x − Rλ (x)), λ

λ > 0,

is called the family of Yosida approximations of A. From the definition, it is clear that Aλ (x) ∈ A(xλ ) = A(Rλ (x)). (iii) For any x ∈ X, we define the element of minimal norm A0 (x) ∈ X ⋆ , as the element with the property, 󵄩󵄩 0 󵄩󵄩 󵄩 ⋆󵄩 󵄩󵄩A (x)󵄩󵄩X ⋆ = m(A(x)) := ⋆inf 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ . x ∈A(x) By the properties of X and X ⋆ , the element A0 (x) ∈ X ⋆ is well-defined and unique. Example 9.3.22 (Resolvent operators in Hilbert space). In the special case where X = H is a Hilbert space, since the duality mapping J coincides with the identity I, the resolvent operators can be identified to the family of operators Rλ : H → H, defined by Rλ := (I + λA)−1 . ◁ Example 9.3.23 (Yosida approximations of the subdifferential). Let X and X ⋆ be as in Definition 9.3.21, consider φ : X → ℝ ∪ {+∞} a proper lower semicontinuous convex ⋆ function and let A := 𝜕φ : X → 2X . Then for any λ > 0, the resolvent operator Rλ for the subdifferential is related to a regularization problem for φ, in the sense that for any x ∈ X, it holds that Rλ (x) = arg minz∈X φx,λ (z), where φx,λ (z) := 2λ1 ‖x − z‖2 + φ(z), and the function x 󳨃→ φλ (x) := infz∈X φx,λ (z) can be considered as the generalization of the notion of the Moreau–Yosida regularization (we have studied in Hilbert space in Section 4.7.2) for Banach spaces. In the special case where φ = IC , with C ⊂ X a closed convex set, recalling that 𝜕IC = NC (see Example 4.1.6), we may consider Rλ as a generalization of the projection operator.8 ◁ The resolvent family has several useful properties collected in the following propositions (see e. g., [19, 85]) Proposition 9.3.24 (Properties of resolvent family). Let X be a reflexive Banach space (assuming further a renorming such that both X and its dual X ⋆ are locally uniformly ⋆ convex) and A : D(A) ⊂ X → 2X a maximal monotone operator. The operators Rλ : X → D(A) are well-defined for every λ > 0, single valued, bounded and satisfy the property lim Rλ (x) = x,

λ→0

∀ x ∈ conv(D(A)).

(9.29)

Proof. We have already established the fact that Aλ is single valued. 8 If X = H is a Hilbert space then, for any x ∈ X, we can see that Rλ (x) solves the variational inequality ⟨x − Rλ (x), z − Rλ (x)⟩ ≤ 0 for every z ∈ C, which characterizes the projection PC (see Proposition 2.5.2).

432 | 9 Monotone-type operators We now show that Aλ is bounded for any λ > 0. We note that Aλ (x) ∈ A(Rλ (x)). For any (z, z⋆ ) ∈ Gr(A), by the monotonicity of A we have 1 ⟨J(x − Rλ (x)), Rλ (x) − z⟩ λ 1 1 = − ⟨J(x − Rλ (x)), x − Rλ (x)⟩ + ⟨J(x − Rλ (x)), x − z⟩. λ λ

⟨z⋆ , Rλ (x) − z⟩ ≤ ⟨Aλ (x), Rλ (x) − z⟩ =

Hence, 󵄩󵄩 󵄩2 ⋆ 󵄩󵄩x − Rλ (x)󵄩󵄩󵄩 ≤ −λ⟨z , Rλ (x) − z⟩ + ⟨J(x − Rλ (x))), x − z⟩,

(9.30)

from which using the reverse triangle inequality we have ‖Rλ (x)‖2 ≤ c1 ‖Rλ (x)‖ + c2 , for suitable c1 , c2 > 0 depending on x, z, z⋆ , λ, which implies that Rλ (x) maps bounded sets into bounded sets, so it is bounded. To show (9.29), consider {λn : n ∈ ℕ} ⊂ ℝ such that λn → 0, and J(Rλn (x) − x) ⇀ z⋆o ⋆ in X . From (9.30), we have that 󵄩 󵄩2 lim sup󵄩󵄩󵄩Rλn (x) − x󵄩󵄩󵄩 ≤ ⟨z⋆o , z − x⟩, n→∞

∀ z ∈ D(A),

and this holds for every z ∈ convD(A). In particular, this inequality remains valid for z = x, which implies the desired result. The resolvent family and its properties can be used to show important properties for maximal monotone operators. The following proposition illustrates this point. Proposition 9.3.25. Let X be a reflexive Banach space and let A : D(A) ⊂ X → 2X be a maximal monotone operator. Then both D(A) and R(A) are convex. ⋆

Proof. By Proposition 9.3.24, for every x ∈ convD(A) we have Rλ (x) → x as λ → 0+ . Since Rλ (x) ∈ D(A), we have x ∈ D(A). Hence, D(A) = convD(A) which proves the convexity of D(A). Since R(A) = D(A−1 ) and A−1 is also maximal monotone, we conclude that R(A) is also convex. The following theorem (see e. g., [19, 85]) summarizes the properties of the Yosida approximation. Theorem 9.3.26 (Properties of Yosida approximation). Let X be a reflexive Banach space (assuming further a renorming such that both X and its dual X ⋆ are locally uni⋆ formly convex), A : D(A) ⊂ X → 2X a maximal monotone operator and λ > 0. Then: (i) Aλ : X → X ⋆ is single valued with D(Aλ ) = X, monotone, bounded and demicontinuous hence, maximal monotone.9 9 See Proposition 9.3.9.

9.3 Maximal monotone operators | 433

(ii) It holds that 󵄩 ⋆󵄩 󵄩 󵄩󵄩 󵄩󵄩Aλ (x)󵄩󵄩󵄩X ⋆ ≤ m(A(x)) := ⋆inf 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ , x ∈A(x)

∀ x ∈ D(A),

(9.31)

If A0 (x) ∈ X ⋆ is the element of minimal norm, (i. e., ‖A0 (x)‖X ⋆ = m(A(x))), then Aλ (x) → A0 (x),

in X ⋆ ,

(9.32)

as λ → 0.

(iii) Consider arbitrary sequences {λn : n ∈ ℕ} ⊂ ℝ, and {xn : n ∈ ℕ} ⊂ X with the properties λn → 0, and

xn ⇀ x

in X,

Aλn (xn ) ⇀ x⋆

in X ⋆ ,

lim sup⟨Aλn (xn ) − Aλm (xm ), xn − xm ⟩ ≤ 0.

(9.33)

n,m

Then (x, x⋆ ) ∈ Gr(A),

and

lim⟨Aλn (xn ) − Aλm (xm ), xn − xm ⟩ = 0. n,m

(9.34)

Proof. (i) We consider each of the claimed properties separately. (a) We have already seen that Aλ is single valued and that D(Aλ ) = X in the introduction of the section. (b) For the monotonicity of Aλ observe that ⟨Aλ (x1 ) − Aλ (x2 ), x1 − x2 ⟩ = ⟨Aλ (x1 ) − Aλ (x2 ), Rλ (x1 ) − Rλ (x2 )⟩

+ ⟨Aλ (x1 ) − Aλ (x2 ), (x1 − Rλ (x1 )) − (x2 − Rλ (x2 ))⟩.

(9.35)

We also notice that Aλ (xi ) ∈ A(Rλ (xi )), for any xi ∈ X, i = 1, 2, and since A is monotone, this implies that ⟨Aλ (x1 ) − Aλ (x2 ), Rλ (x1 ) − Rλ (x2 )⟩ ≥ 0.

(9.36)

The second term on the right-hand side of (9.35), by the definition of Aλ is identified as ⟨Aλ (x1 ) − Aλ (x2 ), (x1 − Rλ (x1 )) − (x2 − Rλ (x2 ))⟩ 1 = ⟨J(x1 − Rλ (x1 )) − J(x2 − Rλ (x2 )), (x1 − Rλ (x1 )) − (x2 − Rλ (x2 ))⟩ ≥ 0, λ

(9.37)

by the monotonicity of the duality mapping J. Therefore, combining (9.35) with (9.36) and (9.37) yields, ⟨Aλ (x1 ) − Aλ (x2 ), x1 − x2 ⟩ ≥ 0, hence, Aλ is monotone. (c) Concerning boundedness, by Proposition 9.3.24, Rλ is bounded so that by definition Aλ is also bounded.

434 | 9 Monotone-type operators (d) Now we prove that Aλ is demicontinuous. To this end, let {xn : n ∈ ℕ} be an arbitrary sequence in D(Aλ ) = X with xn → x, and we must show that Aλ (xn ) ⇀ Aλ (x) (since X is reflexive). Set zn = Rλ (xn ) and z⋆n = Aλ (xn ). Then λ z⋆n + J(zn − xn ) = 0. So we have that ⟨J(zn − xn ) − J(zm − xm ), xm − xn ⟩

= ⟨J(zn − xn ) − J(zm − xm ), (zn − xn ) − (zm − xm )⟩ + λ⟨z⋆n − z⋆m , zn − zm ⟩.

Since ⟨J(zn − xn ) − J(zm − xm ), xm − xn ⟩ → 0 as n, m → ∞, and since both summands in the right-hand side of the above inequality are nonnegative (by the monotonicity of J and A and since Aλ (x) ∈ A(Rλ (x))), we have lim ⟨z⋆n − z⋆m , zn − zm ⟩ = 0,

n,m→∞

and lim ⟨J(zn − xn ) − J(zm − xm ), (zn − xn ) − (zm − xm )⟩ = 0.

n,m→∞

Since Rλ , Aλ and J are bounded, we may assume by reflexivity that (up to subsequences) zn ⇀ zo in X, z⋆n ⇀ z⋆o in X ⋆ and J(zn − xn ) ⇀ x⋆o in X ⋆ , for some zo ∈ X, z⋆o , x⋆o ∈ X ⋆ . Applying Proposition 9.3.5(v), we have that (zo , z⋆o ) ∈ Gr(A), J(zo − x) = x⋆o and λz⋆o + J(zo − x) = 0. Hence, zo ∈ Rλ (x) and z⋆o = Aλ (x). By the Urysohn property (see Remark 1.1.51), this holds for the whole sequence. We therefore conclude that if xn → x then Aλ (xn ) ⇀ Aλ (x), i. e., Aλ is demicontinuous. (ii) Consider any (x, x⋆ ) ∈ Gr(A), and recall that Aλ (x) ∈ A(Rλ (x)). Apply the monotonicity property of A for the pairs (x, x⋆ ) ∈ Gr(A) and (xλ , Aλ (x)) ∈ Gr(A), recalling that Rλ (x) = xλ . This yields 0 ≤ ⟨x⋆ − Aλ (x), x − Rλ (x)⟩ = ⟨x⋆ − λ−1 J(x − xλ ), x − xλ ⟩

= ⟨x⋆ , x − xλ ⟩ − λ−1 ⟨J(x − xλ ), x − xλ ⟩ = ⟨x⋆ , x − xλ ⟩ − λ−1 ‖xλ − x‖2 󵄩 󵄩 ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖x − xλ ‖ − λ−1 ‖xλ − x‖2 .

Note that for the first equality we have used the definition of Aλ (x). We therefore obtain 󵄩 󵄩 0 ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ‖x − xλ ‖ − λ−1 ‖xλ − x‖2 , which upon dividing by ‖x − xλ ‖ yields 󵄩 󵄩 λ−1 ‖xλ − x‖ ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ , and using once more the definition of Aλ , according to which ‖Aλ (x)‖X ⋆ = λ−1 ‖x − xλ ‖ we obtain the inequality 󵄩󵄩 󵄩 󵄩 ⋆󵄩 󵄩󵄩Aλ (x)󵄩󵄩󵄩X ⋆ ≤ 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ ,

∀ (x, x⋆ ) ∈ Gr(A).

(9.38)

9.3 Maximal monotone operators |

435

Since (9.38) holds for all x⋆ ∈ A(x), (9.31) follows by taking the infimum in (9.38) over all x⋆ ∈ A(x). The element A0 (x) ∈ X ⋆ is well-defined and unique (since X ⋆ is local uniformly convex it is also strictly convex by (2.18) and then we use Theorem 2.6.5). Since ‖Aλ (x)‖X ⋆ ≤ m(A(x)) for every λ > 0, if we take a sequence λn → 0, by the reflexivity of X ⋆ , there exists z⋆o ∈ X ⋆ such that Aλn (x) ⇀ z⋆o in X ⋆ . By Proposition 9.3.24, Rλn (x) → x and since Aλn (x) ∈ A(Rλn (x)) by the closedness properties of maximal monotone operators (see Proposition 9.3.5(iii)) we have that z⋆o ∈ A(x). By the definition of m(A(x)) since z⋆o ∈ A(x), we have that m(A(x)) ≤ ‖z⋆o ‖X ⋆ . On the other hand, since Aλn (x) ⇀ z⋆o in X ⋆ , by the weak lower semicontinuity of the norm,10 we have that ‖z⋆o ‖X ⋆ ≤ lim infn ‖Aλn (x)‖, and since ‖Aλn (x)‖X ⋆ ≤ m(A(x)) for every n, lim infn ‖Aλn (x)‖X ⋆ ≤ m(A(x)) so that ‖z⋆o ‖X ⋆ ≤ m(A(x)). This leads us to conclude that 󵄩󵄩 ⋆ 󵄩󵄩 󵄩 0 󵄩 󵄩󵄩zo 󵄩󵄩X ⋆ = m(A(x)) = 󵄩󵄩󵄩A (x)󵄩󵄩󵄩X ⋆ . Therefore, the limit z⋆o is an element of minimal norm, and by the uniqueness of A0 (x) we conclude that z⋆o = A0 (x). Furthermore, a similar argument shows that lim supn ‖Aλn (x)‖X ⋆ ≤ ‖z⋆o ‖X ⋆ while the weak lower semicontinuity of the norm implies ‖z⋆o ‖X ⋆ ≤ lim infn ‖Aλn (x)‖X ⋆ so that limn ‖Aλn (x)‖X ⋆ = ‖z⋆o ‖X ⋆ . Since Aλ (x) ⇀ A0 (x) and ‖Aλ (x)‖X ⋆ → ‖A0 (x)‖X ⋆ , the local uniform convexity of X ⋆ leads us to the conclusion Aλ (x) → A0 (x) (see Example 2.6.13). (iii) Recall that Aλ (x) ∈ A(Rλ (x)) = A(xλ ) for any λ > 0. Consider two sequences {λn : n ∈ ℕ} ⊂ ℝ, {xn : n ∈ ℕ} ⊂ X, satisfying property (9.33). Then ⟨Aλn (xn ) − Aλm (xm ), xn − xm ⟩

= ⟨Aλn (xn ) − Aλm (xm ), Rλn (xn ) − Rλm (xm )⟩

+ ⟨Aλn (xn ) − Aλm (xm ), (xn − Rλn (xn )) − (xm − Rλm (xm ))⟩

(9.39)

≥ ⟨Aλn (xn ) − Aλm (xm ), (xn − Rλn (xn )) − (xm − Rλm (xm ))⟩,

since ⟨Aλn (xn ) − Aλm (xm ), Rλn (xn ) − Rλm (xm )⟩ ≥ 0 (by the monotonicity of A and since Aλn (xn ) ∈ A(Rλn (xn )), Aλm (xm ) ∈ A(Rλm (xm ))). We now use the definition of Aλ (x) and substituting Aλn (xn ) and Aλm (xm ) for their equals in (9.39), we obtain the inequality ⟨Aλn (xn ) − Aλm (xm ), xn − xm ⟩

−1 ≥ ⟨λn−1 J(xn − Rλn (xn )) − λm J(xm − Rλm (xm )), (xn − Rλn (xn )) − (xm − Rλm (xm ))⟩,

(9.40)

and as λn → 0, by the proper choice of n, m the right hand side can be made positive. Then, by the fact that the sequences chosen satisfy property (9.33), inequality (9.40) yields lim ⟨Aλn (xn ) − Aλm (xm ), xn − xm ⟩ = 0,

n,m→∞

10 See Proposition 1.1.53(ii).

436 | 9 Monotone-type operators and lim ⟨Aλn (xn ) − Aλm (xm ), Rλn (xn ) − Rλm (xm )⟩ = 0.

n,m→∞

Applying Proposition 9.3.5(v), we conclude that (x, x⋆ ) ∈ Gr(A). Hence, (9.34) holds. In the special case where X = H a Hilbert space, the resolvent operator and the Yosida approximation enjoys supplementary properties. Proposition 9.3.27. Let X = H be a Hilbert space and A : D(A) ⊂ H → 2H a maximal monotone operator. Then: (i) the resolvent Rλ = (I + λA)−1 is nonexpansive on H and, (ii) the Yosida approximation Aλ is Lipschitz with Lipschitz constant λ1 . Proof. (i) The fact that the resolvent can be expressed in the form Rλ = (I + λA)−1 has already been discussed in Example 9.3.22. Consider (xi , x⋆i ) ∈ Gr(A), i = 1, 2. Exploiting the inner product structure of H, for any λ > 0, 󵄩󵄩 ⋆ ⋆ 󵄩2 2 ⋆ ⋆ 2󵄩 ⋆ ⋆ 󵄩2 󵄩󵄩x1 − x2 + λ(x1 − x2 )󵄩󵄩󵄩 = ‖x1 − x2 ‖ + λ⟨x1 − x2 , x1 − x2 ⟩ + λ 󵄩󵄩󵄩x1 − x2 󵄩󵄩󵄩 ≥ ‖x1 − x2 ‖2 ,

(9.41)

since the monotonicity of A, yields ⟨x⋆1 − x⋆2 , x1 − x2 ⟩ ≥ 0. This implies 󵄩 󵄩 ‖x1 − x2 ‖ ≤ 󵄩󵄩󵄩x1 − x2 + λ(x⋆1 − x⋆2 )󵄩󵄩󵄩, which is the nonexpansive property for Rλ . Note that the converse of (i) is also true since taking the limit of (9.41) as λ → 0 yields the monotonicity of A (see also Example 9.2.2). (ii) Setting xi,λ = (I + λA)−1 xi , i = 1, 2, so that x1 − x2 ∈ x1,λ − x2,λ + λ(A(x1,λ ) − A(x2,λ )), and taking the inner product in H by A(x1,λ ) − A(x2,λ ) yields the required result. 9.3.6.2 Applications The Yosida approximation finds interesting applications. One such application is in the study of solvability of operator equations and surjectivity theorems. Lemma 9.3.28. Let X be a reflexive Banach space (assuming a renorming so that X and ⋆ ⋆ X ⋆ are both locally uniformly convex). Let A : D(A) ⊂ X → 2X and B : D(B) ⊂ X → 2X be two maximal monotone operators satisfying D(A) ∩ D(B) ≠ 0. For any given z⋆ ∈ X ⋆ , λ > 0, the operator inclusion z⋆ ∈ A(xλ ) + Bλ (xλ ) + J(xλ ), where Bλ the Yosida approximation of B, has a unique solution xλ ∈ D(A). Proof. The operator Bλ + J is monotone, demicontinuous, coercive and bounded with D(Bλ + J) = X. By Theorem 9.3.12, we have R(A + Bλ + J) = X ⋆ . The uniqueness of the solution xλ follows from the monotonicity of A and Bλ and the strict monotonicity of J. So we have z⋆ = x⋆λ + Bλ (xλ ) + J(xλ ), x⋆λ ∈ A(xλ ).

9.3 Maximal monotone operators |

437

Proposition 9.3.29. Let X be a reflexive Banach space (assuming a renorming such that ⋆ ⋆ X and X ⋆ are locally uniformly convex), A : D(A) ⊂ X → 2X , B : D(B) ⊂ X → 2X maximal monotone operators with D(A) ∩ D(B) ≠ 0, and for any z⋆ ∈ X ⋆ , λ > 0, define xλ as the solution of the inclusion z⋆ ∈ A(xλ ) + Bλ (xλ ) + J(xλ ), where Bλ the Yosida approximation of B. Then R(A + B + J) = X ⋆ if and only if there exist λ0 > 0 and c > 0 (independent of λ) such that ‖Bλ (xλ )‖X ⋆ ≤ c, for every λ ∈ (0, λ0 ]. Proof. We will denote by Bλ and RBλ the Yosida approximation and the resolvent, respectively, of the operator B. Let R(A + B + J) = X ⋆ and consider any z⋆ ∈ R(A + B + J). Then there exist (x, x⋆1 ) ∈ Gr(A) and (x, x⋆2 ) ∈ Gr(B) such that z⋆ = x⋆1 + x⋆2 + J(x). Let xλ ∈ D(A) be the unique solution (see Lemma 9.3.28) of z⋆ = x⋆λ + Bλ (xλ ) + J(xλ ),

x⋆λ ∈ A(xλ ).

(9.42)

Using the monotonicity of J and A, we have 0 ≤ ⟨J(xλ ) − J(x), xλ − x⟩ = −⟨x⋆1 − x⋆λ , x − xλ ⟩ + ⟨x⋆2 − Bλ (xλ ), xλ − x⟩ ≤ ⟨x⋆2 − Bλ (xλ ), xλ − x⟩.

(9.43)

Recall that Bλ (xλ ) = λ1 J(xλ − RBλ (xλ )), where by RBλ we denote the resolvent of B, so that λJ−1 (Bλ (xλ )) = xλ − RBλ (xλ ), which implies that xλ = RBλ (xλ ) + λJ−1 (Bλ (xλ )). Putting this in (9.43) yields 0 ≤ ⟨x⋆2 − Bλ (xλ ), RBλ (xλ ) − x⟩ + ⟨x⋆2 − Bλ (xλ ), λJ−1 (Bλ (xλ ))⟩.

(9.44)

Since Bλ (xλ ) ∈ B(RBλ (xλ )) and because B is monotone, the first term on the right-hand side of (9.44) is nonpositive, so 0 ≤ ⟨x⋆2 − Bλ (xλ ), λJ−1 (Bλ (xλ ))⟩, which implies that λ‖Bλ (xλ )‖2 ≤ ⟨x⋆2 , λJ−1 (Bλ (xλ ))⟩, from which it follows that ‖Bλ (xλ )‖X ⋆ ≤ ‖x⋆2 ‖X ⋆ , λ > 0. For the converse, let z⋆ ∈ X ⋆ , arbitrary. Assume ‖Bλ (xλ )‖X ⋆ ≤ c for some c > 0 and all λ ∈ (0, λ0 ]. Let (zo , z⋆o ) ∈ Gr(A), using Lemma 9.3.28 consider (9.42) for z = zo , and take the duality pairing with xλ − zo . Using the monotonicity of A, we have ‖xλ ‖2 = ⟨z⋆o , xλ − zo ⟩ + ⟨J(xλ ), zo ⟩ − ⟨x⋆λ , xλ − zo ⟩ − ⟨Bλ (xλ ), xλ − zo ⟩

≤ ⟨z⋆o , xλ − zo ⟩ + ⟨J(xλ ), zo ⟩ − ⟨z⋆o , xλ − zo ⟩ − ⟨Bλ (xλ ), xλ − zo ⟩.

We now use the boundedness hypothesis for {Bλ (xλ )}λ∈(0,λ0 ] , and we have ‖xλ ‖2 ≤ c1 ‖xλ ‖ + c2 for some c1 , c2 ≥ 0 and every λ ∈ (0, λ0 ], so that {xλ : λ ∈ (0, λ0 ]} is bounded. Then, from (9.42) it follows that {x⋆λ : λ ∈ (0, λ0 ]} is bounded, so we can find a sequence λn → 0+ such that xλn ⇀ x in X and x⋆λn ⇀ x⋆1 , Bλn (xλn ) ⇀ x⋆2 , J(xλn ) ⇀ x⋆3 in

438 | 9 Monotone-type operators X ⋆ . Consider (9.42) for λ = λn and λ = λm , substract and take the duality pairing with xλn − xλm , so that 0 = ⟨x⋆λn + J(xλn ) − x⋆λm − J(xλm ), xλn − xλm ⟩ + ⟨Bλn (xλn ) − Bλm (xλm ), xλn − xλm ⟩.

(9.45)

Since A + J is monotone, we have ⟨Bλn (xλn ) − Bλm (xλm ), xλn − xλm ⟩ ≤ 0, and so lim sup⟨Bλn (xλn ) − Bλm (xλm ), xλn − xλm ⟩ ≤ 0. n,m→∞

From Proposition 9.3.26(iii), we have that (x, x⋆2 ) ∈ Gr(B) and lim ⟨Bλn (xλn ) − Bλm (xλm ), xλn − xλm ⟩ = 0.

n,m→∞

Therefore, from (9.45) it follows that lim ⟨x⋆λn + J(xλn ) − x⋆λm − J(xλm ), xλn − xλm ⟩ = 0,

n,m→∞

and because A is monotone, lim sup⟨J(xλn ) − J(xλm ), xλn − xλm ⟩ ≤ 0. n,m→∞

Since J is maximal monotone with D(J) = X, we can apply again Proposition 9.3.5(v) to obtain that x⋆3 = J(x). Also, limn,m→∞ ⟨J(xλn )−J(xλm ), xλn −xλm ⟩ = 0 and so limn,m→∞ ⟨x⋆λn − x⋆λm , xλn − xλm ⟩ = 0. Hence, another application of Proposition 9.3.5(v), yields that x⋆1 ∈ A(x). Then, z⋆ − x⋆λn − J(xλn ) ⇀ z⋆ − x⋆1 − J(x) = x⋆2 in X ⋆ , so finally z⋆ ∈ R(A + B + J). We close our treatment of the Yosida approximation for maximal monotone operators by an important example, the Yosida approximation for the subdifferential. This extends the analysis for the Moreau–Yosida approximation and the proximity operator in Hilbert spaces (see Section 4.7) to the more general case of reflexive Banach spaces. The following theorem (see e. g., [85]) is a generalization of Proposition 4.7.6 to reflexive Banach spaces. Theorem 9.3.30. Let X be a reflexive Banach space, φ : X → ℝ ∪ {+∞} be a proper convex lower semicontinuous function and A = 𝜕φ and consider its Yosida approximation defined by φλ (x) := inf { z∈X

1 ‖x − z‖2 + φ(z)}, 2λ

λ > 0.

For every λ > 0, the function φλ is convex, finite and Gâteaux differentiable on X and Aλ = Dφλ . If X = H is a Hilbert space, then φλ is Fréchet differentiable. In addition,

9.3 Maximal monotone operators | 439

(i) φλ (x) = 2λ1 ‖x − Rλ (x)‖2 + φ(Rλ (x)), ∀ λ > 0 and x ∈ X. (ii) φ(Rλ (x)) ≤ φλ (x) ≤ φ(x), ∀ λ > 0 and x ∈ X. (iii) limλ→0+ φλ (x) = φ(x), ∀ x ∈ X. Proof. (i) It is a simple consequence of Proposition 4.4.1 and Theorem 4.3.2. For every x ∈ X, fixed, define ϕx : X → ℝ ∪ {+∞}, by ϕx (z) = 2λ1 ‖x − z‖2 + φ(z) and note that 𝜕ϕx (z) = λ1 J(z − x) + 𝜕φ(z). This implies (see Proposition 4.6.1) that every solution zo of 0 ∈ λ1 J(z − x) + 𝜕φ(z) is a minimizer of φx , and since zo = Rλ (x) we obtain (i). (ii) This is immediate from (i) and the definition of φλ . (iii) We need to consider two cases. If x ∈ dom φ, then by Proposition 9.3.24, limλ→0+ Rλ (x) = x. Using (ii) and the lower semicontinuity of φ, we obtain that φλ (x) → φ(x) as λ → 0. Now, suppose that x ∈ ̸ dom φ. We must show that φλ (x) → ∞ as λ → 0. If this is not the case, then there exist λn > 0 and c > 0, such that φλn (x) = φ(Rλn (x)) +

1 󵄩󵄩 󵄩2 󵄩x − Rλn (x)󵄩󵄩󵄩 ≤ c, 2λn 󵄩

∀ n ≥ 1.

It then follows that φ(Rλn (x)) ≤ c

󵄩 󵄩 and 󵄩󵄩󵄩x − Rλn (x)󵄩󵄩󵄩 → 0

as n → ∞.

From these facts and the lower semicontinuity of φ, it follows that φ(x) ≤ c which is a contradiction. Therefore, lim φλ (x) = φ(x),

λ→0+

∀ x ∈ X.

The Gâteaux differentiability of φλ can be shown in various ways, one may simply note that φλ = φ◻ψ for ψ defined by x 󳨃→ ψ(x) = 2λ1 ‖x‖2 and then use the results on the subdifferential of the inf-convolution (see Proposition 5.3.5). By Proposition 4.4.1 and the properties of the duality map the result follows, since the subdifferential is single valued.

9.3.7 Sum of maximal monotone operators The following remarkable theorem is due to Rockafellar (see e. g., [19, 85]). Theorem 9.3.31. Let X be a reflexive Banach space (assuming a renorming such that ⋆ ⋆ X and X ⋆ are locally uniformly convex) and A : D(A) ⊂ X → 2X , B : D(B) ⊂ X → 2X maximal monotone operators. Suppose that int(D(A))∩D(B) ≠ 0. Then A+B is a maximal monotone operator.

440 | 9 Monotone-type operators Proof. Without loss of generality,11 we may assume that 0 ∈ int(D(A)) ∩ D(B), 0 ∈ A(0), 0 ∈ B(0). For x⋆ ∈ X ⋆ and λ > 0, we consider the equation (9.46)

x⋆ ∈ J(xλ ) + A(xλ ) + Bλ (xλ ),

where Bλ is the Yosida approximation of B. Since Bλ is demicontinuous and bounded it follows from Lemma 9.3.28 that (9.46) has a solution xλ ∈ D(A) for any λ > 0. Since A and Bλ are monotone, upon taking the duality pairing of (9.46) by xλ we see that 󵄩 󵄩 ‖xλ ‖ ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩X ⋆ ,

∀ λ > 0.

Since 0 ∈ int(D(A)) it follows from Theorem 9.2.6 that A is locally bounded at 0. Hence, there exist ρ > 0 and c > 0 such that 󵄩󵄩 ⋆ 󵄩󵄩 󵄩󵄩x 󵄩󵄩X ⋆ ≤ c,

∀ x⋆ ∈ ⋃{A(x) : ‖x‖ ≤ ρ}.

ρ

⋆ ⋆ Now, for λ > 0, define zλ = 2 J−1 (x⋆λ )‖x⋆λ ‖−1 X ⋆ where xλ = x − Bλ (xλ ) − J(xλ ) ∈ A(xλ ), so ρ ⋆ that ‖zλ ‖ = 2 < ρ, hence zλ ∈ D(A). Let (zλ , zλ ) ∈ Gr(A). Then, ‖z⋆λ ‖X ⋆ ≤ c. Using the monotonicity of A, we obtain

0 ≤ ⟨x⋆λ − z⋆λ , xλ − zλ ⟩ = ⟨x⋆λ , xλ ⟩ + ⟨z⋆λ , zλ ⟩ − ⟨z⋆λ , xλ ⟩ − ⟨x⋆λ , zλ ⟩,

∀ z⋆λ ∈ A(xλ ),

which, using also the bounds obtained above, implies that ρ 󵄩󵄩 ⋆ 󵄩󵄩 ρ 󵄩 ⋆󵄩 󵄩 ⋆ 󵄩2 ⋆ 󵄩x 󵄩 ⋆ = ⟨xλ , zλ ⟩ ≤ c + c󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ + 󵄩󵄩󵄩x 󵄩󵄩󵄩X ⋆ . 2 󵄩 λ 󵄩X 2 We have thus shown that {xλ : λ > 0}, {Bλ (xλ ) : λ > 0} and {x⋆λ : λ > 0} are bounded sets of X and X ⋆ , respectively. Since X is reflexive, we may assume that xλ ⇀ x0 ,

J(xλ ) ⇀ x⋆0 ,

Bλ (xλ ) ⇀ x⋆1 ,

x⋆λ ⇀ x⋆2 ,

as λ → 0.

Using (9.46) applied for any λ, μ > 0, subtracting and taking the duality pairing with xλ − xμ , we have for some x⋆λ ∈ A(xλ ), x⋆μ ∈ A(xμ ) that 0 = ⟨J(xλ ) + x⋆λ − J(xμ ) − x⋆μ , xλ − xμ ⟩ + ⟨Bλ (xλ ) − Bμ (xμ ), xλ − xμ ⟩.

(9.47)

Therefore, taking into account the monotonicity of J and A, we have lim sup⟨Bλ (xλ ) − Bμ (xμ ), xλ − xμ ⟩ ≤ 0. λ,μ→0

It then follows by Theorem 9.3.26(iii) that (x0 , x⋆1 ) ∈ Gr(B) and lim ⟨Bλ (xλ ) − Bμ (xμ ), xλ − xμ ⟩ = 0.

λ,μ→0

11 Indeed, if x0 ∈ int(D(A)) ∩ D(B) then consider A1 (x) = A(x + x0 ) − A(x0 ) and B1 (x) = B(x + x0 ) − B(x0 ).

9.3 Maximal monotone operators | 441

By (9.47), we have that lim ⟨J(xλ ) + x⋆λ − J(xμ ) − x⋆μ , xλ − xμ ⟩ = 0.

λ,μ→0

Since J + A is maximal monotone, and xλ ⇀ x0 , J(xλ ) + x⋆λ ⇀ x⋆0 + x⋆2 , by Proposition 9.3.5(v), applied to this operator we have (x0 , x⋆0 + x⋆2 ) ∈ Gr(A + J). Passing to the limit as λ → 0 in (9.46), we obtain x⋆0 + x⋆2 + x⋆1 = x⋆ , where x⋆2 ∈ A(x0 ), x⋆1 ∈ B(x0 ), i. e., x⋆ ∈ R(J + A + B). It then follows that R(J + A + B) = X ⋆ so that by Theorem 9.3.13 the operator A + B is maximal monotone. A maximal monotonicity criterion for sums of operators in Hilbert spaces is given in the next theorem. Theorem 9.3.32. Let H be a Hilbert space, and A : D(A) ⊂ H → 2H and B : D(B) ⊂ H → 2H maximal monotone operators. Assume furthermore that for every (z, z⋆ ) ∈ Gr(A) and λ > 0 it holds that ⟨z⋆ , Bλ (z)⟩ ≥ 0. Then A + B is maximal monotone. Proof. Let x⋆ ∈ H and let xλ ∈ D(A) be the unique solution of the regularized problem xλ + x⋆λ + Bλ (xλ ) = x⋆ with (xλ , x⋆λ ) ∈ Gr(A). According to Proposition 9.3.29, it suffices to prove that {Bλ (xλ ) : λ > 0} is bounded in H. Take the inner product of the above equation with Bλ (xλ ) and use the hypothesis to get 󵄩 󵄩2 󵄩 󵄩 󵄩 󵄩 ⟨Bλ (xλ ), xλ ⟩ + 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩 ≤ 󵄩󵄩󵄩x⋆ 󵄩󵄩󵄩 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩. Let (z, z⋆ ) ∈ Gr(A). By monotonicity we have 0 ≤ ⟨x⋆λ − z⋆ , xλ − z⟩ = ⟨x⋆ − xλ − Bλ (xλ ) − z⋆ , xλ − z⟩, which upon rearrangement yields ‖xλ ‖2 + ⟨Bλ (xλ ), xλ ⟩ ≤ ⟨x⋆ − z⋆ , xλ − z⟩ + ⟨xλ + Bλ (xλ ), z⟩ 󵄩 󵄩 ≤ c1 + c2 ‖xλ ‖ + c3 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩, where c1 , c2 , c3 > 0 are independent of λ > 0. Let x󸀠 ∈ D(B). By the monotonicity of Bλ we have 0 ≤ ⟨Bλ (xλ ) − Bλ (x󸀠 ), xλ − x󸀠 ⟩, so ⟨Bλ (x󸀠 ), xλ − x󸀠 ⟩ + ⟨Bλ (xλ ), x󸀠 ⟩ ≤ ⟨Bλ (xλ ), xλ ⟩, from which it follows that 󵄩 󵄩 −c4 − c5 ‖xλ ‖ − c6 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩 ≤ ⟨Bλ (xλ ), xλ ⟩,

442 | 9 Monotone-type operators where c4 , c5 , c6 > 0 are independent of λ > 0, and we used the boundedness of Bλ (x󸀠 ) (see Theorem 9.3.26(ii), applied on Bλ (x󸀠 )). Therefore, ‖xλ ‖2 ≤ −⟨Bλ (xλ ), xλ ⟩ + c1 + c2 ‖xλ ‖ + c3 ‖Bλ (xλ )‖ ≤ c7 + c8 ‖xλ ‖ + c9 ‖Bλ (xλ )‖,

where c7 , c8 , c9 > 0 are independent of λ > 0. Hence, we finally have that 󵄩󵄩 󵄩2 󵄩 ⋆󵄩 󵄩 󵄩 󵄩󵄩Bλ (xλ )󵄩󵄩󵄩 ≤ −⟨Bλ (xλ ), xλ ⟩ + 󵄩󵄩󵄩x 󵄩󵄩󵄩 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩 󵄩 󵄩 ≤ c4 + c5 ‖xλ ‖ + c10 󵄩󵄩󵄩Bλ (xλ )󵄩󵄩󵄩, with c10 > 0 independent of λ > 0. Since ‖xλ ‖ is bounded we deduce that the set {Bλ (xλ ) : λ > 0} is bounded and so A + B is maximal monotone.

9.4 Pseudomonotone operators 9.4.1 Pseudomonotone operators, definitions and examples We will start with the presentation of the theory of pseudomonotone operators in the single valued case. Definition 9.4.1. Let X be a reflexive Banach space. The operator A : X → X ⋆ is called pseudomonotone if the following holds: If {xn : n ∈ ℕ} is a sequence such that xn ⇀ x in X and lim supn ⟨A(xn ), xn −x⟩ ≤ 0, then it holds that lim infn ⟨A(xn ), xn −z⟩ ≥ ⟨A(x), x−z⟩ for all z ∈ X. The following proposition gives typical examples for pseudomonotone operators. Proposition 9.4.2. Let X be a reflexive Banach space and A, B : X → X ⋆ operators. Then the following hold: (i) If A is monotone and hemicontinuous, then A is pseudomonotone. (ii) If A is completely continuous, then A is pseudomonotone. (iii) If A and B are pseudomonotone, then A + B is pseudomonotone. (iv) If A is pseudomonotone and locally bounded, then A is demicontinuous. Proof. (i) Let {xn : n ∈ ℕ} be a sequence in X such that xn ⇀ x and lim supn ⟨A(xn ), xn − x⟩ ≤ 0. Since A is monotone, it holds that ⟨A(xn ) − A(x), xn − x⟩ ≥ 0, therefore, 0 = lim inf⟨A(x), xn − x⟩ ≤ lim inf⟨A(xn ), xn − x⟩, n→∞

n→∞

so limn ⟨A(xn ), xn −x⟩ = 0. For arbitrary z̄ ∈ X, we set z = x+ t(z̄ −x) = x+ t(z̄ −xn +xn −x), t > 0. By the monotonicity of A, ⟨A(xn ) − A(z), xn − z⟩ = ⟨A(xn ) − A(z), xn − (x + t(z̄ − xn + xn − x))⟩ ≥ 0,

9.4 Pseudomonotone operators | 443

so, upon rearranging, ̄ t⟨A(xn ), xn − z⟩̄ ≥ −(1 − t)⟨A(xn ), xn − x⟩ + (1 − t)⟨A(z), xn − x⟩ + t⟨A(z), xn − z⟩. Since xn ⇀ x and ⟨A(xn ), xn − x⟩ → 0, taking the limit inferior ̄ lim inf⟨A(xn ), xn − z⟩̄ ≥ ⟨A(z), x − z⟩, n→∞

∀ z̄ ∈ X,

∀ t > 0.

We now pass to the limit as t → 0+ . Since A is hemicontinuous, we conclude that A(z) ⇀ A(x) as t → 0+ . Therefore, for all z̄ ∈ X we have ̄ lim inf⟨A(xn ), xn − z⟩̄ ≥ ⟨A(x), x − z⟩, n→∞

i..e. A is pseudomonotone. (ii) Let {xn : n ∈ ℕ} be a sequence in X such that xn ⇀ x. Since A is completely continuous, A(xn ) → A(x) in X ⋆ . Therefore, for every z ∈ X, we have ⟨A(x), x − z⟩ = lim ⟨A(xn ), xn − z⟩, n→∞

i. e., A is pseudomonotone. (iii) Let {xn : n ∈ ℕ} be a sequence in X such that xn ⇀ x and lim sup⟨A(xn ) + B(xn ), xn − x⟩ ≤ 0. n

(9.48)

We claim that lim sup⟨A(xn ), xn − x⟩ ≤ 0, n→∞

and

lim sup⟨B(xn ), xn − x⟩ ≤ 0. n→∞

(9.49)

Suppose that lim supn ⟨A(xn ), xn − x⟩ = α > 0. From (9.48), we have that lim supn ⟨B(xn ), xn − x⟩ ≤ −α. Since B is pseudomonotone, for all z ∈ X we have ⟨B(x), x − z⟩ ≤ lim inf⟨B(xn ), xn − z⟩. n→∞

For z = x, we have that 0 ≤ −α < 0 which is a contradiction. Hence, (9.49) holds and since A and B are pseudomonotone we have ⟨A(x), x − z⟩ ≤ lim inf⟨A(xn ), xn − z⟩, n→∞

⟨B(x), x − z⟩ ≤ lim inf⟨B(xn ), xn − z⟩. n→∞

Adding the above, we get that for all z ∈ X, ⟨A(x) + B(x), x − z⟩ ≤ lim inf⟨A(xn ) + B(xn ), xn − z⟩, n→∞

i. e., A + B is pseudomonotone.

444 | 9 Monotone-type operators (iv) Let {xn : n ∈ ℕ} be a sequence in X such that xn → x. Since A is locally bounded, the sequence {A(xn ) : n ∈ ℕ} is bounded. The space X is reflexive, so there exists a subsequence {A(xnk ) : k ∈ ℕ} with A(xnk ) ⇀ x⋆ as k → ∞. Therefore, limk ⟨A(xnk ), xnk − x⟩ = 0. So, by the pseudomonotonicity of A we have ⟨A(x), x − z⟩ ≤ lim inf⟨A(xnk ), xnk − z⟩ = ⟨x⋆ , x − z⟩, k→∞

∀z ∈ X.

It then follows that A(x) = x⋆ . By the standard argument (based on the Urysohn property), we have that A(xn ) ⇀ A(x), i. e., A is demicontinuous. Now we define the concept of pseudomonotonicity for multivalued operators. This is needed in order to study a broader class of nonlinear problems, in particular nonlinear elliptic and parabolic boundary value problems. Definition 9.4.3. Let X be a reflexive Banach space. The operator A : X → 2X is called pseudomonotone if it satisfies the following conditions: (i) The set A(x) is nonempty, bounded, closed and convex for all x ∈ X. (ii) A is upper semicontinuous from every finite dimensional subspace XF ⊂ X to the weak topology of X ⋆ (in the sense of Definition 1.7.2). (iii) If {xn : n ∈ ℕ} ⊂ X is a sequence such that xn ⇀ x in X, and if for x⋆n ∈ A(xn ) it holds that lim supn ⟨x⋆n , xn − x⟩ ≤ 0, then for every xo ∈ X there exists x⋆ = x⋆ [xo ] ∈ A(x) (with the notation x⋆ [xo ] used to emphasize dependence on xo ) with the property ⋆

lim inf⟨x⋆n , xn − xo ⟩ ≥ ⟨x⋆ , x − xo ⟩. n→∞

9.4.2 Surjectivity results for pseudomonotone operators Pseudomonotone operators also enjoy some useful surjectivity results (see e. g., [85, 86]) which make them very useful in applications. We introduce the concept of coercivity for multivalued maps. Definition 9.4.4. Let X be a Banach space. The multivalued operator T : X → 2X is called coercive if there exists a real valued function c : ℝ+ → ℝ+ with limr→∞ c(r) → +∞ such that ⟨x⋆ , x⟩ ≥ c(‖x‖)‖x‖ for all (x, x⋆ ) ∈ Gr(T). ⋆

We also need the following proposition, proved by Browder and Hess in 1972 (see [86]) whose proof is based on the theory of the Brouwer degree. Proposition 9.4.5 (Browder and Hess). Let X be a finite dimensional Banach space, T : ⋆ X → 2X a mapping such that for each x ∈ X, T(x) is a nonempty bounded closed convex ⋆ subset of X ⋆ . Suppose that T is coercive and upper semicontinuous from X to 2X . Then R(T) = X ⋆ . Theorem 9.4.6. Let X be a reflexive Banach space and A : X → 2X a pseudomonotone and coercive operator. Then A is surjective, i. e., R(A) = X ⋆ . ⋆

9.4 Pseudomonotone operators |

445

Proof. The proof follows along the lines of Theorem 9.3.12. First of all, note that it suffices to show that 0 ∈ R(A). We start with a finite dimensional approximation. Let 𝒳F be the family of all finite dimensional subspaces XF of X ordered by inclusion. For any XF ∈ 𝒳F let jF : XF → X ⋆ the inclusion mapping of XF into X and jF⋆ : X ⋆ → XF⋆ its dual map, which is the ⋆ projection of X ⋆ onto XF⋆ and define the operator AF : XF → 2XF by AF := jF⋆ AjF . Since A is pseudomonotone, by property (i) of Definition 9.4.3 for every x ∈ XF , A(x) is a nonempty, convex and closed (hence weakly compact) subset of X ⋆ . Furthermore, jF⋆ is continuous from the weak topology on X ⋆ to XF⋆ .12 Again by the pseudomonotonicity ⋆ of A, by property (ii) of Definition 9.4.3, A is upper semicontinuous from XF to 2X , X ⋆ endowed with the weak topology, therefore, AF is upper semicontinuous from XF ⋆ to 2XF . Finally, AF inherits the coercivity property from A since for any pair (x, x⋆F ) ∈ Gr(AF ), there exists some x⋆ ∈ A(x) such that x⋆F = jF⋆ x⋆ and ⟨x⋆F , x⟩X ⋆ ,X = ⟨jF⋆ x⋆ , x⟩X ⋆ ,X = ⟨x⋆ , x⟩ ≥ c(‖x‖)‖x‖. F

F

F

F

By Proposition 9.4.5, it follows that R(AF ) = XF⋆ for all XF ∈ 𝒳F . Hence, for any XF ∈ 𝒳F , there exists a xF ∈ XF such that 0 ∈ AF (xF ), which in turn implies that 0 = jF⋆ x⋆F for some x⋆F ∈ A(xF ) with x⋆F ∈ XF⋆ . The set {xF : XF ∈ 𝒳F } is uniformly bounded by the coercivity of A. Let c > 0 be this uniform bound. For any XF ∈ 𝒳F , define the set U(XF ) := ⋃{xF󸀠 : 0 ∈ A(xF󸀠 ) with XF󸀠 ∈ 𝒳F and XF󸀠 ⊃ XF }. By the uniform boundedness of the sets {xF : XF ∈ 𝒳F }, it is seen that U(XF ) is contained in the closed ball B(0, c) of X, which is weakly compact. It can be shown w that the family {U(XF ) : XF ∈ 𝒳F } has the finite intersection property, therefore, w ⋂XF ∈𝒳F U(XF ) ≠ 0 (see Proposition 1.8.1). w

Let xo ∈ ⋂XF ∈𝒳F U(XF ) . We will show that 0 ∈ A(xo ). By Proposition 1.1.61, there exists a sequence {xFk : k ∈ ℕ} ⊂ VF such that xFk ⇀ xo in X. For each k, 0 = jF⋆k x⋆Fk , so that ⟨x⋆Fk , xFk − xo ⟩ = 0, therefore, taking the limit superior yields lim supk ⟨x⋆Fk , xFk − xo ⟩ = 0. Take any x ∈ X. By the pseudomonotonicity of A, there exists x⋆o = x⋆o (xo ) ∈ A(xo ) such that 0 = lim inf⟨x⋆Fk , xFk − x⟩ ≥ ⟨x⋆o , xo − x⟩ k

(9.50)

Assume now that 0 ∈ ̸ A(xo ). Then, we may apply the separation theorem to the sets ̄ which is {0} and A(xo ) and find x̄ ∈ X such that 0 < infx⋆ ∈A(xo ) ⟨x⋆ , xo − x⟩̄ ≤ ⟨x⋆o , xo − x⟩, in contradiction with (9.50). 12 Since XF⋆ is finite dimensional, the weak and the strong topology coincide.

446 | 9 Monotone-type operators If we revisit the proof of the main surjectivity Theorem 9.3.12 for maximal monotone operators, under the light of the theory of pseudomonotone operators it is clear that in the proof of this theorem, we have essentially used the fact that B enjoys the pseudomonotonicity property (see Proposition 9.4.2(i)). Based on the above remark, it is clear that Theorem 9.4.6 can be generalized as follows. Theorem 9.4.7. Let X be a reflexive Banach space, A : X → 2X be a maximal monotone operator and B : X → X ⋆ be a pseudomonotone, bounded and coercive operator with D(B) = X. Then R(A + B) = X ⋆ . ⋆

A straightforward application of the above theorem to a maximal monotone operator that maps every x ∈ X to 0, leads to the following useful corollary. Corollary 9.4.8. Let X be a reflexive Banach space and A : X → X ⋆ be a pseudomonotone, bounded and coercive operator with D(A) = X. Then R(A) = X ⋆ .

9.5 Applications of monotone-type operators 9.5.1 Quasilinear elliptic equations The surjectivity theorem for maximal monotone operators finds important applications in the theory of partial differential equations. As a first example, we consider the solvability of a quasilinear elliptic equation which is a perturbation of the one treated in Section 6.8 (see e. g. [111]). Let 𝒟 ⊆ ℝd be a bounded domain with smooth boundary 𝜕𝒟. Consider the Banach ⋆ space X = W01,p (𝒟), its dual X ⋆ = W −1,p (𝒟) and denote by ⟨⋅, ⋅⟩ the duality pairing among them. We consider the boundary value problem, which consists of finding u ∈ X = W01,p (𝒟) such that −div(|∇u|p−2 ∇u) + λu = f u=0

on 𝒟 on 𝜕𝒟.

(9.51)

for λ ≥ 0 and f ∈ Lp (𝒟) given. This can be seen as a perturbation of the p-Laplace Poisson equation treated in Section 6.8 (see also Example 9.2.14). We will bring (9.51) into an appropriate abstract operator form which will allow for the use of surjectivity results for maximal monotone operators. To this end, we recall ⋆ the weak formulation of problem (9.51) which is: Given f ∈ Lp (𝒟) find u ∈ X such that ⋆

∫ |∇u|p−2 ∇u ⋅ ∇vdx + λ ∫ uvdx = ∫ fvdx, 𝒟

𝒟

𝒟

∀ v ∈ X.

(9.52)

447

9.5 Applications of monotone-type operators |

As in Section 6.8, we define the operator A : X → X ⋆ by ⟨A(u), v⟩ := ∫ |∇u|p−2 ∇u ⋅ ∇vdx + λ ∫ uvdx, 𝒟

∀ u, v ∈ X,

(9.53)

𝒟

and the functional b ∈ X ⋆ by ⟨b, v⟩ := ∫ fvdx,

∀ v ∈ X.

(9.54)

𝒟

Proposition 9.5.1. For p ≥ solution.

2d , d+2

p > 1 and f ∈ Lp (𝒟) problem (9.51) has a unique weak ⋆

Proof. We will show that the operator A maps X into X ⋆ and is continuous, and the functional b belongs to X ⋆ , therefore the weak formulation (9.52) is equivalent to the operator equation A(u) = b in X ⋆ . Then, we apply the surjectivity theorem for maximal monotone operators (Theorem 9.3.15). The proof is broken up into 4 steps. 1. We show that A : X → X ⋆ is bounded and that b ∈ X ⋆ . To show the above, note that for u, v ∈ X, using Hölder’s inequality, we have 󵄨󵄨 󵄨 p−1 󵄨󵄨⟨A(u), v⟩󵄨󵄨󵄨 ≤ ∫ |∇u| |∇v|dx + λ ∫ |uv|dx 𝒟

𝒟 ⋆

1/p⋆

≤ (∫ |∇u|(p−1)p dx) 𝒟

=

1/p

(∫ |∇v|p dx) 𝒟

‖∇u‖p−1 ‖∇v‖Lp (𝒟) Lp (𝒟)

1/2

1/2

+ λ(∫ |u|2 dx) (∫ |v|2 dx) 𝒟

𝒟

+ λ‖u‖L2 (𝒟) ‖v‖L2 (𝒟) .

2d For p ≥ d+2 , we have the embedding X = W01,p (𝒟) 󳨅→ L2 (𝒟) (see Theorem 1.5.11) so that for all v ∈ X it holds that ‖v‖L2 (𝒟) ≤ c1 ‖v‖X = c1 ‖∇v‖Lp (𝒟) for an appropriate constant c1 > 0. It then follows that

󵄨󵄨 󵄨 p−1 󵄨󵄨⟨A(u), v⟩󵄨󵄨󵄨 ≤ ‖∇u‖Lp (𝒟) ‖∇v‖Lp (𝒟) + λ‖u‖L2 (𝒟) ‖v‖L2 (𝒟) ≤ c(‖∇u‖p−1 + λ‖∇u‖Lp (𝒟) )‖∇v‖Lp (𝒟) , Lp (𝒟)

for a suitable constant c > 0. Therefore, 󵄩󵄩 󵄩 󵄩󵄩A(u)󵄩󵄩󵄩X ⋆ =

󵄨 󵄨 sup 󵄨󵄨󵄨⟨Au, v⟩󵄨󵄨󵄨 ≤ c(‖∇u‖p−1 + λ‖∇u‖Lp (𝒟) ), Lp (𝒟)

v∈X, ‖v‖≤1

which implies that A(u) ∈ X ⋆ and the operator A; X → X ⋆ is bounded. Note that if λ = 0 2d is not necessary. the restriction p ≥ d+2 Now, by Hölder’s inequality and the Sobolev embedding X := W01,p (𝒟) 󳨅→ Lp (𝒟), 󵄨󵄨 󵄨 󵄨󵄨⟨b, v⟩󵄨󵄨󵄨 ≤ ‖f ‖Lp⋆ (𝒟) ‖v‖Lp (𝒟) ≤ c2 ‖f ‖Lp⋆ (𝒟) ‖v‖X , therefore, ‖b‖X ⋆ ≤ c2 ‖f ‖Lp (𝒟) so that b ∈ X ⋆ .

448 | 9 Monotone-type operators 2. The operator A is continuous and coercive. To show continuity, let {un : n ∈ ℕ} ⊂ X be a sequence such that un → u in X. ⋆ Then ∇un → ∇u in Lp (𝒟), so that if ϕ(s) = |s|p−2 s then ϕ(∇un ) → ϕ(∇u) in Lp (𝒟). We have for any v ∈ X, and a suitable constant c > 0, ⟨A(un ) − A(u), v⟩ = ∫(ϕ(∇un ) − ϕ(∇u)) ⋅ ∇vdx + λ ∫(un − u)vdx 𝒟

𝒟

󵄩 󵄩 ≤ 󵄩󵄩󵄩ϕ(∇un ) − ϕ(∇u)󵄩󵄩󵄩Lp⋆ (𝒟) ‖∇v‖Lp (𝒟) + λ‖un − u‖L2 (𝒟) ‖v‖L2 (𝒟) 󵄩 󵄩 ≤ 󵄩󵄩󵄩ϕ(∇un ) − ϕ(∇u)󵄩󵄩󵄩Lp⋆ (𝒟) ‖v‖X + c‖un − u‖X ‖v‖X , (where we used the Sobolev embeddings once more) which implies that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩󵄩A(un ) − A(u)󵄩󵄩󵄩X ⋆ ≤ 󵄩󵄩󵄩ϕ(∇un ) − ϕ(∇u)󵄩󵄩󵄩Lp⋆ (𝒟) + c‖un − u‖X . Therefore, A(un ) → A(u), i. e., A is continuous. To show coercivity, note that ⟨A(u), u⟩ = ‖u‖pX + λ‖u‖2L2 (𝒟) ≥ ‖u‖pX , which implies that ⟨A(u),u⟩ ≥ ‖u‖p−1 X → ∞ as ‖u‖X → ∞, if p > 1. ‖u‖X 3. The operator A is strictly monotone. Indeed, (simplifying notation ‖ ⋅ ‖X to ‖ ⋅ ‖) we have for every u1 , u2 ∈ X, ⟨A(u1 ) − A(u2 ), u1 − u2 ⟩ = ⟨A(u1 ), u1 ⟩ + ⟨A(u2 ), u2 ⟩ − ⟨A(u1 ), u2 ⟩ − ⟨A(u2 ), u1 ⟩ ≥ ‖u1 ‖p + ‖u2 ‖p − ‖u1 ‖p−1 ‖u2 ‖ − ‖u2 ‖p−1 ‖u2 ‖ + λ ∫(u1 − u2 )2 dx 𝒟

≥ λ ∫(u1 − u2 )2 dx, 𝒟

where we also used the monotonicity of the p-Laplace operator (see Example 9.2.3) and the fact that λ ≥ 0, and the strict monotonicity of A for λ > 0 follows immediately.13 4. Combining the above steps we may see that the operator A is maximal monotone. Unique solvability follows by a straightforward application of the surjectivity theorem for maximal monotone operators (Theorem 9.3.15). 9.5.2 Semilinear elliptic inclusions Another interesting application of monotone type operators is in the study of elliptic inclusions (see e. g. [70]). 13 Strict monotonicity for the operator A may also hold in the case where λ = 0 under suitable conditions on p, using more elaborate elementary estimates, such as for instance (a|a|p−2 −b|b|p−2 )⋅(a−b) ≥ c(|a|, |b|)|a − b|2 for every a, b ∈ ℝd , see e. g. [43].

9.5 Applications of monotone-type operators |

449

Let 𝒟 ⊂ ℝd be a bounded domain with C 2 boundary14 𝜕𝒟 and let β : ℝ → 2ℝ be a maximal monotone map. Given f ∈ L2 (𝒟), we consider the following nonlinear Dirichlet problem: −Δu + β(u) ∋ f u = 0,

in 𝒟, on 𝜕𝒟.

(9.55)

Let H = L2 (𝒟), and A : D(A) ⊂ H → H, with D(A) = W01,2 (𝒟) ∩ W 2,2 (𝒟). Also let B be the realization of β in H, i. e., B(u) = {v ∈ H : v(x) ∈ β(u(x)) a. e.}. We know that B is maximal monotone (see Example 9.3.3). Without loss of generality, we may assume that β is the subdifferential of a proper convex lower semicontinuous function φ : ℝ → ℝ ∪ {∞}, i. e., β = 𝜕φ. We will need the resolvent of the maximal monotone operator (graph) β and βλ its Yosida approxiβ mation defined as follows: For any s ∈ ℝ, we have that Rλ (s) = y where y is the unique solution of the equation y + λβ(y) = s, whereas, βλ (s) = λ1 (s − y). Furthermore, let φλ (s) := inf { 󸀠 s ∈ℝ

1 󵄩󵄩 󸀠 󵄩2 󸀠 󵄩s − s 󵄩󵄩󵄩 + φ(s )}. 2λ 󵄩

and recall that by Theorem 9.3.30 it holds that βλ = 𝜕φλ . We need to introduce the following lemma. Lemma 9.5.2. For every u ∈ D(A) and λ > 0, it holds that ⟨Au, Bλ (u)⟩ ≥ 0. Proof. We have that 󵄨 󵄨2 d β (u(x))dx. ⟨Au, Bλ (u)⟩ = − ∫ Δu(x)βλ (u(x))dx = ∫󵄨󵄨󵄨∇u(x)󵄨󵄨󵄨 ds λ 𝒟

𝒟

Since βλ (u(x)) = 𝜕φλ (u(x)) and because φ is a lower semicontinuous and convex funcd d2 tion on ℝ, we have ds βλ (u(x)) = ds 2 ϕλ (u(x)) ≥ 0 a. e. Therefore, ⟨Au, Bλ (u)⟩ ≥ 0 for every u ∈ D(A) and λ > 0. We can now show existence for the inclusion. Proposition 9.5.3. If f ∈ L2 (𝒟) and β; ℝ → 2ℝ is maximal monotone, then (9.55) has a unique solution u ∈ W01,2 (𝒟) ∩ W 2,2 (𝒟). Proof. Let H = L2 (𝒟) and B : H → 2H be the realization of β on H, which is maximal monotone (see Example 9.3.3). Also define A : D(A) ⊂ H → H by Au = −Δu with D(A) = W01,2 (𝒟) ∩ W 2,2 (𝒟). From Lemma 9.5.2, we know that ⟨Au, Bλ (u)⟩ ≥ 0 for every u ∈ D(A) and every λ > 0. So by Theorem 9.3.32, A + B is maximal monotone. Let 14 This hypothesis is needed for reasons related to the regularity of the Poisson equation and the definition of the Laplace operator.

450 | 9 Monotone-type operators λ1 > 0 be the first eigenvalue of A. Recalling the variational characterization of λ1 (Rayleigh quotient), we have that A − λ1 I is monotone. Hence, A + B − λ1 I is monotone. But R(A + B − λ1 I + (1 + λ1 )I) = R(A + B + I) = H, because A + B is maximal monotone. Therefore, A+B−λ1 I is maximal monotone, too, and so R(A+B) = R(A+B−λ1 I +λ1 I) = H. The uniqueness of the solution is clear. 9.5.3 Variational inequalities with monotone-type operators The theory of monotone-type operators is very important in the study of variational inequalities involving nonlinear operators (see e. g. [3]). ⋆ Let X be a reflexive Banach space, A : X → 2X a maximal monotone operator and C ⊂ D(A) ⊂ X a convex closed set. Given x⋆ ∈ X ⋆ we consider the variational inequality of finding xo ∈ C such that there exists x⋆o ∈ A(xo ) with the property ⟨x⋆o − x⋆ , z − xo ⟩ ≥ 0,

∀ z ∈ C.

(9.56)

Proposition 9.5.4. Suppose that either int(C) ≠ 0 or int(D(A)) ∩ C ≠ 0. Then the variational inequality (9.56) is equivalent to the inclusion xo ∈ C : x⋆ ∈ A(xo ) + 𝜕IC (xo )

(9.57)

Proof. Assume that xo solves the inclusion (9.57). Then there exists x⋆o ∈ A(xo ) such that x⋆ − x⋆o ∈ 𝜕IC (xo ), and using the definition of the subdifferential, we see that xo is indeed a solution of the variational inequality (9.56). Note that for this direction we do not require the conditions int(C) ≠ 0 or int(D(A)) ∩ C ≠ 0. For the converse, assume that xo and x⋆o ∈ A(xo ) solve the variational inequality (9.56). By monotonicity, we have that for every z ∈ C and z⋆ ∈ A(z) it holds that 0 ≤ ⟨z⋆ − x⋆o , z − xo ⟩ = ⟨z⋆ − x⋆ , z − xo ⟩ + ⟨x⋆ − x⋆o , z − xo ⟩, which using (9.56) implies ⟨z⋆ − x⋆ , z − xo ⟩ ≥ 0.

(9.58)

Consider now any z̄⋆ ∈ 𝜕IC (z). By the definition of the subdifferential, since xo ∈ C, this yields ⟨z̄⋆ , z − xo ⟩ ≥ 0,

(9.59)

so that adding (9.58) and (9.59) we conclude that ⟨z⋆ + z̄⋆ − x⋆ , z − xo ⟩ ≥ 0,

∀ z ∈ C, z⋆ ∈ A(z), z̄⋆ ∈ IC (z).

(9.60)

Since either int(C) ≠ 0 or int(D(A)) ∩ C ≠ 0, we have by Theorem 9.3.31 that B := A + 𝜕IC is a maximal monotone operator so that (9.60) implies that x⋆ ∈ B(xo ) = A(xo ) + IC (xo ).

9.5 Applications of monotone-type operators | 451

Using the restatement of the variational inequality (9.56) in terms of the inclusion (9.57), we may provide an existence result for (9.56). Proposition 9.5.5. Let X be a reflexive Banach space. Assume that A : D(A) ⊂ X → ⋆ 2X is maximal monotone and coercive. Suppose furthermore that either int(C) ≠ 0 or int(D(A)) ∩ C ≠ 0. Then the variational inequality (9.56) admits a solution. Proof. By Proposition 9.5.4, it suffices to consider the inclusion (9.57). As above, the operator B := A + 𝜕IC is maximal monotone and as B inherits the coercivity property from A, the surjectivity Theorem 9.3.15 yields existence. Pseudomonotone operators find important applications in variational inequalities. Let X be a reflexive Banach space, C ⊂ X closed and convex and A : X → X ⋆ a nonlinear operator. For any x⋆ ∈ X, we consider the variational inequality find xo ∈ C : ⟨A(xo ) − x⋆ , x − xo ⟩ ≥ 0,

(9.61)

∀ x ∈ C.

We make the following assumptions on the operator A. Assumption 9.5.6. The operator A : X → X ⋆ satisfies the following conditions: (i) A is pseudomonotone. (ii) There exist constants c1 , c2 > 0 such that ⟨A(x), x⟩ ≥ c1 ‖x‖2 − c2 , for every x ∈ X. (iii) There exist constants c3 , c4 > 0 such that ‖A(x)‖X ⋆ ≤ c3 + c4 ‖x‖, for every x ∈ X. Proposition 9.5.7. Let X be a reflexive Banach space, C ⊂ X closed and convex, and A : X → X ⋆ satisfy Assumption 9.5.6. Then the variational inequality (9.61) admits a solution for any x⋆ ∈ X ⋆ . Proof. We first observe that (9.61) is equivalent to the operator equation A(xo ) + 𝜕IC (xo ) ∋ x⋆ ; hence, it is sufficient to show that the multivalued operator A + B : X → 2X where B = 𝜕IC , is surjective. The operator A is pseudomonotone, bounded and coercive (by Assumption 9.5.6) while B is a maximal monotone operator. Hence, using Theorem 9.4.7 (with the roles of A and B interchanged) we obtain the required solvability result. ⋆

9.5.4 Gradient flows in Hilbert spaces Another important application of the theory of monotone-type operators is in the study of gradient flows in Hilbert spaces and the theory of nonlinear semigroups (see e. g. [18, 63]). Consider X = H a Hilbert space and let φ : H → ℝ∪{+∞} be a convex proper lower semicontinuous function. Consider the gradient algorithm for the minimization of φ,

452 | 9 Monotone-type operators which in the limit as the step size h → 0 may formally be expressed as a differential equation x󸀠 (t) + 𝜕φ(x(t)) ∋ 0,

t > 0,

x(0) = x0 ,

(9.62)

with the function x(⋅) : I := [0, T] → H considered as the continuous limit of the successive approximations {xn : n ∈ ℕ} provided by the gradient scheme xn+1 − xn ∈ h𝜕φ(xn+1 ), in the sense that xn = x(nh) in the limit as h → 0. The solvability of the differential equation (9.62) relies heavily on the fact that the subdifferential is a maximal monotone operator. We will therefore consider more general version of equation (9.62), x󸀠 (t) + A(x(t)) ∋ 0,

t > 0,

x(0) = x0 ,

(9.63)

where A : D(A) ⊂ H → 2H is in general a maximal monotone operator, and xo ∈ H. Recall that A admits a Yosida approximation {Aλ : λ > 0} consisting of operators Aλ : H → H which are Lipschitz continuous. As a result of the Lipschitz continuity of Aλ , the approximate equation x󸀠λ (t) + Aλ (xλ (t)) = 0,

t > 0,

x(0) = x0 ,

(9.64)

is well posed and admits a unique solution xλ ∈ C 1 (I; H). A reasonable question that arises is the following: Since the Yosida approximation {Aλ : λ > 0} approximates the operator A in the limit as λ → 0+ can we claim that the limit of {xλ : λ > 0} as λ → 0+ , if it exists, is a solution to the original problem (9.63)? The following theorem shows that the answer to this question is affirmative. Theorem 9.5.8. For each x0 ∈ D(A), there exists a unique function x ∈ C(I; H), with x󸀠 ∈ L∞ (I; H) which is a solution of (9.63). Proof. As mentioned above, since Aλ is Lipschitz continuous, by Theorem 3.1.5, the approximate problem (9.64) admits a unique solution xλ ∈ C 1 (I; H). To pass to the limit as λ → 0+ and show that this limit is a solution to the equation (9.63), we need to derive some estimates concerning uniform (in λ) boundedness of xλ and x󸀠λ . The proof proceeds in 4 steps: 1. We claim that for any λ > 0, it holds that ‖x󸀠λ (t)‖ ≤ ‖A0 (x0 )‖. To obtain this bound, consider the solution of problems x󸀠i,λ (t) + Aλ (xi,λ (t)) = 0,

xi,λ (0) = xi ,

t > 0, i = 1, 2.

9.5 Applications of monotone-type operators |

453

Subtracting, taking the inner product with x1,λ (t) − x2,λ (t) and using the integration by d parts formula and the definition of the subdifferential we get dt (‖x1,λ − x2,λ ‖2 )(t) ≤ 0, which leads to the estimate ‖x1,λ (t) − x2,λ (t)‖ ≤ ‖x1 − x2 ‖ for every t > 0. Since the solutions to the approximate system (9.64) are unique choosing any h > 0 and setting x2 = x1,λ (h), we have that x2,λ (t) = x1,λ (t + h), so that the previous estimate becomes ‖x1,λ (t + h) − x1,λ (t)‖ ≤ ‖x1,λ (h) − x1 ‖ and dividing by h and passing to the limit as h → 0 we have that 󵄩󵄩 󸀠 󵄩 󵄩 󸀠 󵄩 󵄩 󵄩 󵄩 0 󵄩 󵄩󵄩x1,λ (t)󵄩󵄩󵄩 ≤ 󵄩󵄩󵄩x1,λ (0)󵄩󵄩󵄩 = 󵄩󵄩󵄩Aλ (x1 )󵄩󵄩󵄩 ≤ 󵄩󵄩󵄩A (x1 )󵄩󵄩󵄩,

(9.65)

where for the last estimate we used the properties of the Yosida approximation in terms of the element of minimal norm of A (see Theorem 9.3.26(ii)). Since x1 ∈ H was arbitrary, the claim follows. 2. We now consider the solution of the approximate problem for the same initial condition x0 but two different values λ1 , λ2 of the approximation parameter. We will show that 1 󵄩󵄩 󵄩 (λ + λ2 )1/2 t 1/2 A0 (x0 ), 󵄩󵄩xλ1 (t) − xλ2 (t)󵄩󵄩󵄩 ≤ √2 1

(9.66)

for all λ1 , λ2 > 0 and t ≥ 0 To prove the claim in step 2, we need to estimate xλ1 and xλ2 . We subtract the approximate evolution equations that these functions satisfy and take inner product with the difference xλ1 (t) − xλ2 (t) for any t. Similarly as above, this yields 1 d 󵄩󵄩 󵄩2 󵄩x (t) − xλ2 (t)󵄩󵄩󵄩 + ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), xλ1 (t) − xλ2 (t)⟩ = 0. 2 dt 󵄩 λ1

(9.67)

Here, we may not use the monotonicity properties of Aλ1 directly. We will then have to rearrange this duality pairing, using the resolvent operator Rλ1 . The definition of the Yosida approximation rearranged as xλ = Rλ (xλ ) + λAλ (xλ ) for every λ > 0 allows us to express the difference xλ1 − xλ2 as xλ1 − xλ2 = Rλ1 (xλ1 ) − Rλ2 (xλ2 ) + λ1 Aλ1 (xλ1 ) − λ2 Aλ2 (xλ2 ). Therefore, ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), xλ1 (t) − xλ2 (t)⟩

= ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), Rλ1 (xλ1 (t)) − Rλ2 (xλ2 (t))⟩

(9.68)

+ ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), λ1 Aλ1 (xλ1 (t)) − λ2 Aλ2 (xλ2 (t))⟩.

Let us consider the first term of the right-hand side. By Theorem 9.3.26(i), we have that for every λi > 0, x ∈ H, it holds that Aλi (x) ∈ A(Rλi (x)), i = 1, 2. Applying that for λi , xλi (t), i = 1, 2, and recalling the monotonicity of the subdifferential operator A (see Proposition 4.2.1) we obtain immediately that ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), Rλ1 (xλ1 (t)) − Rλ2 (xλ2 (t))⟩ ≥ 0.

454 | 9 Monotone-type operators Using that in (9.68) implies ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), xλ1 (t) − xλ2 (t)⟩

≥ ⟨Aλ1 (xλ1 (t)) − Aλ2 (xλ2 (t)), λ1 Aλ1 (xλ1 (t)) − λ2 Aλ2 (xλ2 (t))⟩ 󵄩2 󵄩 󵄩2 󵄩 = λ1 󵄩󵄩󵄩Aλ1 (xλ1 (t))󵄩󵄩󵄩 + λ2 󵄩󵄩󵄩Aλ2 (xλ2 (t))󵄩󵄩󵄩 − (λ1 + λ2 )⟨Aλ1 (xλ1 (t)), Aλ2 (xλ2 (t))⟩ 󵄨 󵄨 󵄩2 󵄩 󵄩2 󵄩 ≥ λ1 󵄩󵄩󵄩Aλ1 (xλ1 (t))󵄩󵄩󵄩 + λ2 󵄩󵄩󵄩Aλ2 (xλ2 (t))󵄩󵄩󵄩 − (λ1 + λ2 )󵄨󵄨󵄨⟨Aλ1 (xλ1 (t)), Aλ2 (xλ2 (t))⟩󵄨󵄨󵄨 󵄩 󵄩󵄩 󵄩 󵄩2 󵄩 󵄩2 󵄩 ≥ λ1 󵄩󵄩󵄩Aλ1 (xλ1 (t))󵄩󵄩󵄩 + λ2 󵄩󵄩󵄩Aλ2 (xλ2 (t))󵄩󵄩󵄩 − (λ1 + λ2 )󵄩󵄩󵄩Aλ1 (xλ1 (t))󵄩󵄩󵄩 󵄩󵄩󵄩Aλ2 (xλ2 (t))󵄩󵄩󵄩 λ 󵄩 λ + λ2 󵄩󵄩 0 󵄩2 λ 󵄩 󵄩2 󵄩2 ≥ − 2 󵄩󵄩󵄩Aλ1 (xλ1 (t))󵄩󵄩󵄩 − 1 󵄩󵄩󵄩Aλ2 (xλ2 (t))󵄩󵄩󵄩 ≥ − 1 󵄩A (x0 )󵄩󵄩󵄩 4 4 4 󵄩

where for the penultimate estimate we have used the algebraic inequality (λ1 + λ2 )a b ≤ λ1 (a2 +

1 2 1 b ) + λ2 (b2 + a2 ) 4 4

and the final estimate comes from (9.65) of step 1. Substituting this estimate in (9.67) yields the differential inequality 1 d 󵄩󵄩 󵄩2 λ + λ2 󵄩󵄩 0 󵄩2 󵄩󵄩xλ1 (t) − xλ2 (t)󵄩󵄩󵄩 ≤ 1 󵄩A (x0 )󵄩󵄩󵄩 2 dt 4 󵄩 and since the right-hand side is independent of t this can be easily integrated to yield (9.66). 3. Let us consider the family xλ obtained as solutions of the approximate problem and try to go to the limit as λ → 0+ . By the a priori bounds (9.65) in step 1, we see that for any T > 0, there exists a function z ∈ L2 ([0, T]; H) such that x󸀠λ ⇀ z,

in L2 ([0, T]; H).

Furthermore, the limit satisfies the estimate ‖z(t)‖ ≤ A0 (x0 ). The a priori bound (9.66) in step 2 is in fact an equicontinuity result, which by the Ascoli–Arzela Theorem 1.8.3 guarantees the existence of a function x such that xλ → x,

uniformly in C([0, T]; H).

A simple argument shows that z = x󸀠λ . 4. We now show that x(t) ∈ D(A) for every t ∈ ℝ+ and that x󸀠 (t) and x(t) satisfy the equation a. e. in t ≥ 0. It is immediate to see that Rλ (xλ ) → x uniformly in C([0, T]; H). Indeed, by the definition of the resolvent and the Yosida approximation we have that 󵄩󵄩 󵄩 󵄩 󵄩 󵄩 󸀠 󵄩 0 󵄩󵄩Rλ (xλ (t)) − xλ (t)󵄩󵄩󵄩 = λ󵄩󵄩󵄩Aλ (xλ (t))󵄩󵄩󵄩 = λ󵄩󵄩󵄩 xλ (t)󵄩󵄩󵄩 ≤ λ A (x0 ). Therefore, 󵄩 󵄩 󵄩 󵄩󵄩 󵄩 󵄩 󵄩󵄩Rλ (xλ (t)) − x(t)󵄩󵄩󵄩 ≤ 󵄩󵄩󵄩Rλ (xλ (t)) − xλ (t)󵄩󵄩󵄩 + 󵄩󵄩󵄩xλ (t) − x(t)󵄩󵄩󵄩 → 0,

as λ → 0+ ,

with both terms of the right-hand side converging uniformly and the result follows.

9.5 Applications of monotone-type operators | 455

The solution of the approximate problem satisfies for all t ≥ 0 the equality −x󸀠λ (t) = Aλ (xλ (t)) ∈ A(Rλ (xλ (t))). This means that (−x󸀠λ (t), Rλ (xλ (t))) ∈ Gr(A) and since A is a maximal monotone operator its graph is weakly closed, so going to the limit as λ → 0+ and recalling that Rλ (xλ ) → x we obtain that (−x󸀠 (t), x(t)) ∈ Gr(A) a. e. t ≥ 0, therefore, −x󸀠 (t) ∈ A(x(t)). Remark 9.5.9. In fact, the solution can be extended on [0, ∞]; see, e. g., [27]. The solution to the evolution problem (9.63) is related to a family of operators {S(t) : t ≥ 0}, with S(t) : H → H for every t ≥ 0, defined for any fixed t by S(t)x := x(t) for every x ∈ H, where x(⋅) is the solution of the evolution problem (9.63) for the initial condition x(0) = xo ∈ H. The uniqueness of the solution guarantees that for each t > 0 the operator S(t) : H → H is well-defined and single valued. Furthermore, again by the uniqueness of solution one may see that this family of operators satisfies the semigroup property, i. e., for any t1 , t2 > 0 it holds that S(t2 )S(t1 ) = S(t2 + t1 ). To see this, we need a more complicated notation that we denote by x(t; x0 ) the solution of problem (9.63) at time t with initial condition x(0) = x0 . Then S(t)x0 := x(t; x0 ) for any t > 0 and x0 ∈ H. Clearly, S(t2 + t1 )x0 = x(t2 + t1 ; x0 ) = x(t2 ; x(t1 ; x0 )) = S(t2 )S(t1 )x0 , and since this holds for arbitrary t1 , t2 > 0 and arbitrary x0 ∈ H, the semigroup property is obtained. Finally, S(0)x = x for any x ∈ H. We are thus led to the following definition. Definition 9.5.10 (Nonlinear semigroup). A strongly continuous nonlinear operator semigroup on H is a family of nonlinear operators {S(t) : t > 0}, S(t) : H → H, for every t > 0, satisfying: (i) S(0) = I. (ii) S(t2 )S(t1 )x = S(t2 + t1 )x, for all t1 , t2 ∈ ℝ+ , x ∈ H. (iii) For every x ∈ H, the mapping t 󳨃→ S(t)x is continuous from ℝ+ into H. We see that the evolution problem (9.63) generates such a nonlinear semigroup on H, in terms of the solution map, S(t)z = x(t; z) for any t > 0, z ∈ H. The theory of nonlinear semigroups generated by maximal monotone operators has been extended by Crandall and Pazy and widely studied also in the context of reflexive Banach spaces (see, e. g., [101] and references therein). 9.5.5 The Cauchy problem in evolution triples An important class of PDEs are evolution type problems, which may be considered as extensions of the gradient flows considered in the previous Section 9.5.4. The notion of the evolution triple and the properties of monotone-type operators play a crucial role in this theory (see e. g. [18, 70]).

456 | 9 Monotone-type operators We begin by recalling the definition of an evolution triple (or Gelfand triple) as a triple of spaces X 󳨅→ H 󳨅→ X ⋆ with X a separable and reflexive Banach space with dual X ⋆ , and H a separable Hilbert space identified with its dual, with the embedding X 󳨅→ H dense and continuous. In this setting, we may also consider the Sobolev–Bochner ⋆ space W 1,p (I; X) := W 1,p,p (I; X, X ⋆ ), consisting of functions x(⋅) : I = [0, T] → X such ⋆ that x(⋅) ∈ Lp (I; X) and x󸀠 (⋅) ∈ Lp (I; X ⋆ ). For more information on Sobolev spaces on evolution triples and their embeddings, we refer to Section 1.6. In this setting, we will consider the following nonlinear (and nonautonomous) Cauchy problem: x󸀠 (t) + A(t, x(t)) = f (t),

a. e. on [0, T],

x(0) = 0,

(9.69)

where T > 0 is fixed, A : [0, T] × X → X ⋆ is a given family of nonlinear operators and f : I = [0, T] → X ⋆ is a given function. The Cauchy problem consists in, for a given f , finding x ∈ W 1,p (I; X) satisfying (9.69). Note that the initial condition makes sense since W 1,p (I; X) 󳨅→ C([0, T]; H) continuously (see Proposition 1.6.13). Here for simplicity, we consider the special case where x(0) = 0. The problem for more general initial condition x(0) = xo ∈ H can be treated using a translation (see, e. g., [18]). We will study problem (9.69) under two possible assumptions on the family of nonlinear operators {A(t, ⋅) : t ≥ 0}, (a) monotonicity and (b) pseudomonotonicity. 9.5.5.1 Monotone operators We require the following assumptions for the nonlinear operators A(t, ⋅). Assumption 9.5.11. Suppose that the family of nonlinear operators {A(t, ⋅) : t ∈ [0, T]} satisfies the following hypotheses: (i) The mapping t 󳨃→ A(t, x) is measurable. (ii) The mapping x 󳨃→ A(t, x) is demicontinuous and monotone. p⋆ ⋆ (iii) ‖A(t, x)‖X ⋆ ≤ a(t) + c ‖x‖p−1 X a. e. on [0, T] with a ∈ L ([0, T]), c > 0, p, p ∈ (1, ∞) 1 1 and p + p⋆ = 1. (iv) There exists c1 > 0 such that ⟨A(t, x), x⟩X ⋆ ,X ≥ c1 ‖x‖pX , a. e. on [0, T], for all x ∈ X.

We now prove the following existence and uniqueness result concerning problem (9.69). Theorem 9.5.12. Suppose that Assumption 9.5.11 holds. Then, for every x0 ∈ H and f ∈ ⋆ Lp (I; X ⋆ ) the problem (9.69) has a unique solution. Proof. The strategy of the proof is to express (9.69) as an abstract operator equation and use the surjectivity theorems for monotone-type operators to show solvability. The ⋆ function space setting is XT := Lp (I; X), so that XT⋆ := Lp (I; X ⋆ ). We define the linear operator L : D(L) ⊂ XT → XT⋆ , by Lx = x󸀠 for every x ∈ D(L) = {x ∈ W 1,p (I; X) : x(0) = 0},

9.5 Applications of monotone-type operators | 457

̂ : XT → X ⋆ by A(x)(⋅) ̂ where W 1,p (I; X) := W 1,p,p (I; X, X ⋆ ), as well as the operator A = T A(⋅, x(⋅)), and express (9.69) as the operator equation ⋆

(9.70)

̂ (L + A)(x) = f,

̂ = X ⋆ , then the claim follows. We intend to use the for any f ∈ Lp (I; X ⋆ ). If R(L + A) T surjectivity result of Theorem 9.3.12 to conclude the proof. The proof follows in 4 steps: 1. First, we prove the uniqueness. Let x1 , x2 ∈ W 1,p (I; X) be two solutions of (9.69). Then x1 (0) = x2 (0) and x󸀠i (t) + A(t, xi (t)) = f (t) a. e. on [0, T], i = 1, 2, where upon subtracting and taking the duality pairing of the resulting equation with x1 (t) − x2 (t), integrating over [0, t] and then using the integration by parts formula (see Theorem 1.6.13) we obtain by the monotonicity of A ⋆

t

󵄩󵄩 󵄩2 󸀠 󸀠 󵄩󵄩x1 (t) − x2 (t)󵄩󵄩󵄩H = 2 ∫⟨x1 (s) − x2 (s), x1 (s) − x2 (s)⟩X ⋆ ,X ds 0

t

= −2 ∫⟨A(s, x1 (s)) − A(s, x2 (s)), x1 (s) − x2 (s)⟩X ⋆ ,X ds ≤ 0; 0

hence, x1 = x2 and the solution of (9.69) is unique. 2. We claim that the linear operator L : XT → XT⋆ is a maximal monotone operator. Monotonicity is immediate by using the integration by parts formula (see Theorem 1.6.13), linearity and the boundary conditions, to show that for any x1 , x2 ∈ D(L) we have that 1󵄩 󵄩2 1 󵄩 󵄩2 ⟨Lx1 − Lx2 , x1 − x2 ⟩XT⋆ ,XT = 󵄩󵄩󵄩x1 (T) − x2 (T)󵄩󵄩󵄩H − 󵄩󵄩󵄩x1 (0) − x2 (0)󵄩󵄩󵄩H ≥ 0, 2 2 since x1 (0) = x2 (0). To show maximality, it suffices to prove that for any (x, x⋆ ) ∈ XT × XT⋆ such that ⟨x⋆ − Lz, x − z⟩X ⋆ ,X ≥ 0, T

T

∀ z ∈ D(L),

(9.71)

it holds that x ∈ D(L) and x⋆ = Lx. To this end, assume that (9.71) holds and choose z = ϕz̄ ∈ XT , where ϕ ∈ Cc∞ ((0, T); ℝ) and z̄ ∈ X. Clearly, z󸀠 = ϕ󸀠 z̄ and z ∈ D(L), while z(T) = 0, so using the integration by parts formula it is easy to see that ⟨Lz, z⟩X ⋆ ,XT = 0, hence applying T (9.71) to the chosen z, we obtain T

0 ≤ ⟨x , x⟩X ⋆ ,X − ∫(⟨x⋆ (t), ϕ(t)z⟩̄ X ⋆ ,X + ⟨ϕ󸀠 (t)z,̄ x(t)⟩X ⋆ ,X )dt, ⋆

T

T

0

∀ z̄ ∈ X.

458 | 9 Monotone-type operators Since z̄ ∈ X was arbitrary, this inequality holds also for λz̄ for all λ ∈ ℝ. Because ⟨x⋆ , x⟩XT⋆ ,XT is independent of z̄ (sending λ → ±∞) we obtain T

∫(⟨x⋆ (t), ϕ(t)z⟩̄ X ⋆ ,X + ⟨ϕ󸀠 (t)z,̄ x(t)⟩X ⋆ ,X )dt = 0 0

∀ ϕ ∈ Cc∞ ((0, T); ℝ),

z̄ ∈ X,

and from the definition of the generalized derivative, we obtain that x󸀠 = x⋆ and x󸀠 ∈ ⋆ Lp (I; X ⋆ ), i. e., x ∈ W 1,p (I; X). It remains to show that x ∈ D(L). By the integration by parts formula, from (9.71) we obtain 1 󵄩 󵄩2 󵄩 󵄩2 0 ≤ ⟨x󸀠 − z󸀠 , x − z⟩X ⋆ ,X = (󵄩󵄩󵄩x(T) − z(T)󵄩󵄩󵄩H − 󵄩󵄩󵄩x(0) − z(0)󵄩󵄩󵄩H ). T T 2

(9.72)

Now, if z ∈ D(L) then z(0) = 0 so (9.72) takes the form 󵄩󵄩 󵄩2 󵄩 󵄩2 󵄩󵄩x(T) − z(T)󵄩󵄩󵄩H − 󵄩󵄩󵄩x(0)󵄩󵄩󵄩H ≥ 0.

(9.73)

Since X 󳨅→ H is dense, we choose a sequence {z̄n : n ∈ ℕ} ⊂ X such that z̄n → T1 x(T) in H. Set zn (t) = t z̄n , so that zn ∈ D(L) for every n. Setting z = zn in (9.73) and letting n → ∞, we obtain that x(0) = 0, i. e., x ∈ D(L). We therefore conclude that L : XT → XT⋆ is a maximal monotone operator. ̂ is a bounded, monotone, demicontinuous and coercive opera3. We claim that A tor. We first show that it is bounded. By Assumptions 9.5.11(i) and (ii), for each x ∈ X and z ∈ X the real function t 󳨃→ ⟨A(t, x(t)), z⟩X ⋆ ,X is measurable on [0, T]. Hence, by the Pettis theorem (see Theorem 1.6.2), for each x ∈ X, the function t → A(t, x(t)) is measurable from [0, T] to X ⋆ . By the growth condition (Assumption 9.5.11(iii)) by p ̂ integration over [0, T], we obtain ‖A(x)‖ XT⋆ ≤ c1 ‖a‖Lp⋆ ([0,T]) + c2 ‖x‖XT for all x ∈ XT , and appropriate constants c1 , c2 > 0. We next show monotonicity. Indeed, by Assumption 9.5.11(ii), for all x, z ∈ XT we have T

̂ − A(z), ̂ ⟨A(x) x − z⟩X ⋆ ,X = ∫⟨A(t, x(t)) − A(t, z(t)), x(t) − z(t)⟩X ⋆ ,X dt ≥ 0. T

T

0

Coercivity follows by Assumption 9.5.11(iv), since for each x ∈ XT we have T

̂ ⟨A(x), x⟩X ⋆ ,X = ∫⟨A(t, x(t), x(t)⟩X ⋆ ,X dt ≥ c1 ‖x‖pX , T

T

which in turn implies, since p > 1, that the coercivity.

T

0

1 ‖x‖XT

̂ ⟨A(x), x⟩X ⋆ ,XT → ∞ as ‖x‖XT → ∞; hence, T

459

9.5 Applications of monotone-type operators |

Finally, we consider demicontinuity. Let xn → x in XT as n → ∞. By passing to a subsequence, if necessary, we may assume that xn (t) → x(t) in X, for any t a. e. on [0, T] as n → ∞. Then, by Assumption 9.5.11(ii), given z ∈ XT ⟨A(t, xn (t)), z(t)⟩X ⋆ ,X → ⟨A(t, x(t)), z(t)⟩X ⋆ ,X ,

a. e. on [0, T].

By Assumption 9.5.11(iii), we apply the dominated convergence theorem and we obtain that T

T

̂ n ), z⟩ ⋆ = ∫⟨A(t, xn (t), z(t)⟩ ⋆ dt → ∫⟨A(t, x(t), z(t)⟩ ⋆ dt⟨A(x), ̂ ⟨A(x z⟩X ⋆ ,X X ,X X ,X X ,X T

T

0

0

T

T

̂ n ) ⇀ A(x) ̂ in X ⋆ hence, A ̂ is as n → ∞. Since z ∈ XT is arbitrary, we conclude that A(x T demicontinuous. ̂ : XT → X ⋆ is bounded, 4. Since L : XT → XT⋆ is maximal monotone (by step 2) and A T ̂ = X⋆, monotone, demicontinuous and coercive (by step 3), by Theorem 9.3.12, R(L + A) T so that (9.70) admits a solution; hence, problem (9.69) has a solution. This is unique by step 1. 9.5.5.2 Pseudomonotone operators We now replace the monotonicity assumption on the operator family {A(t, ⋅) : t ≥ 0} in problem (9.69) by pseudomonotonicity. As before, let (X, H, X ⋆ ) be an evolution triple but we now make the extra assumpc tion that the embedding X 󳨅→ H is compact. We also need an extension of the concept of pseudomonotone operators as follows. Definition 9.5.13. Let Y be a reflexive Banach space and L : D(L) ⊂ Y → Y ⋆ be a linear maximal monotone operator. An operator K : Y → Y ⋆ is called L-pseudomonotone, if for any {yn : n ∈ ℕ} ⊂ D(L) such that (i) yn ⇀ y in Y, (ii) L(yn ) ⇀ L(y) in Y ⋆ , (iii) K(yn ) ⇀ y⋆ in Y ⋆ and (iv) lim supn ⟨K(yn ), yn − y⟩ ≤ 0, it holds that K(y) = y⋆

and

lim⟨K(yn ), yn ⟩ = ⟨K(y), y⟩. n

Assumption 9.5.14. Suppose that A satisfies the following hypotheses: (i) The mapping t 󳨃→ A(t, x) is measurable. (ii) The mapping x 󳨃→ A(t, x) is pseudomonotone. p⋆ (iii) ‖A(t, x)‖X ⋆ ≤ a(t) + c ‖x‖p−1 X a. e. on [0, T] for all x ∈ X, with a ∈ L ([0, T]), c > 0, 2 ≤ p < ∞ and p1 + p1⋆ = 1.

(iv) There exists c1 > 0 and θ ∈ L1 ([0, T]), such that ⟨A(t, x), x⟩X ≥ c1 ‖x‖pX − θ(t), a. e. on [0, T], for all x ∈ X.

The following surjectivity result is due to Lions (see Theorem 1.1. Chapter 3, p. 316 [80]).

460 | 9 Monotone-type operators Theorem 9.5.15. Let X be a reflexive Banach space, L : D(L) ⊂ X → X ⋆ , where D(L) is a dense subspace of X, be a linear maximal monotone operator and B : X → X ⋆ be a demicontinuous, bounded, coercive and L-pseudomonotone operator, then R(L + B) = X ⋆ . Proof. We sketch the proof which uses an interesting regularization argument. The proof proceeds in 3 steps. 1. For any x⋆ ∈ X ⋆ , we approximate Lx + B(x) = x⋆ by ϵL⋆ J−1 (Lxϵ ) + Lxϵ + B(xϵ ) = x⋆ ,

ϵ > 0,

(9.74)

where J : X → X ⋆ is the duality map. We equip D(L) with the graph norm ‖x‖X0 := ‖x‖X + ‖Lx‖X ⋆ , and thus turn it into a reflexive Banach space, denoted by X0 , with dual X0⋆ . Now, for every ϵ > 0 fixed define the operator Lϵ : X0 → X0⋆ , by Lϵ (x) = ϵL⋆ J−1 (Lx)+Lx, which is a bounded hemicontinuous monotone operator. Consider also the restriction of B on X0 , denoted by B̂ : X0 → X0⋆ , which is clearly pseudomonotone. Then, by Proposition 9.4.2 the operator Lϵ + B̂ : X0 → X0⋆ is pseudomonotone, and by Theorem 9.4.6 equation (9.74) admits a solution xϵ ∈ X0 . Furthermore, observe that J−1 (Lxϵ ) ∈ D(L⋆ ). 2. We see that ‖xϵ ‖X and ‖Lxϵ ‖X ⋆ are bounded uniformly in ϵ. For instance to verify the second claim, take the duality pairing of (9.74) with J−1 (Lxϵ ) ∈ D(L⋆ ), where by using the monotonicity of L⋆ (which follows by the monotonicity of L) and the definition of the duality map we obtain ‖Lxϵ ‖2X ⋆ + ⟨B(xϵ ), J−1 (Lxϵ )⟩X ⋆ ,X ≤ ⟨x⋆ , J−1 (Lxϵ )⟩X ⋆ ,X , which since ‖xϵ ‖X < c1 by some c1 > 0 independent of ϵ > 0 and B is bounded leads to the uniform boundedness of ‖Lxϵ ‖X ⋆ . 3. By reflexivity and uniform boundedness, there exists a sequence ϵn → 0, such that upon denoting for simplicity xϵn = xn we have xn ⇀ x in X, Lxn ⇀ x⋆ and B(xn ) ⇀ z in X ⋆ . By the properties of L, we have that x ∈ D(L) and x⋆ = Lx. Moving along this sequence and taking the duality pairing of (9.74) with xn − x, and using the fact that ‖Lxn ‖X ⋆ < c2 for some c2 > 0 independent of n ∈ ℕ (since the whole sequence ‖Lxϵ ‖X ⋆ < ϵ), we conclude that ⟨B(xn ), xn − x⟩X ⋆ ,X ≤ ⟨x⋆ , xn − x⟩X ⋆ ,X + c2 ϵn , whereupon taking the limit superior we get that lim sup⟨B(xn ), xn − x⟩ ≤ 0. n→∞

By L-pseudomonotonicity of B, we have B(x) = z and ⟨B(xn ), xn ⟩ → ⟨B(x), x⟩. Taking the duality pairing of (9.74) (along the chosen sequence) with xn − z0 for every z0 ∈ D(L), and using the fact that Lxn is uniformly bounded in X ⋆ , we get that ⟨x⋆ − Lx − B(x), x − zo ⟩ = 0,

∀ zo ∈ D(L),

therefore, by density of D(L) we have Lx + B(x) = x⋆ .

9.5 Applications of monotone-type operators |

461

Based upon this abstract result we may prove our existence result. Theorem 9.5.16. Suppose that A satisfies the condition of Assumption 9.5.14. Then, for ⋆ every x0 ∈ H and f ∈ Lp ([0, T]; X ⋆ ) problem (9.69) has a solution x ∈ W 1,p ([0, T]; X). Proof. The strategy of the proof, as well as the notation, is exactly the same as in Thê is bounded and coercive. We will show orem 9.5.12. By this theorem, we know that A ̂ is L-pseudomonotone. that A ̂ n ) ⇀ x⋆ in X ⋆ and Let {xn : n ∈ ℕ} ⊂ D(L) such that xn ⇀ x in XT , x󸀠n ⇀ x󸀠 , A(x T T

̂ n , xn − x⟩ ⋆ = lim sup ∫⟨A(t, xn (t)), xn (t) − x(t)⟩ ⋆ dt ≤ 0. lim sup⟨A(x X ,X X ,X n

T

n

T

0

For any t ∈ [0, T], let ξn (t) = ⟨A(t, xn (t)), xn (t) − x(t)⟩. Since W 1,p ([0, T]; X) is compactly embedded in Lp (I; H) (see Theorem 1.6.13, see also [80] p. 58), we may assume that xn (t) → x(t) a. e. in H. Let N ⊂ [0, T] be the Lebesgue null set outside of which conditions (iii) and (iv) of Assumption 9.5.14 hold. Then, for each t ∈ [0, T] \ N, 󵄩 󵄩p 󵄩 󵄩p−1 󵄩 󵄩 ξn (t) ≥ c1 󵄩󵄩󵄩xn (t)󵄩󵄩󵄩X − θ(t)(a(t) + c󵄩󵄩󵄩xn (t)󵄩󵄩󵄩X )󵄩󵄩󵄩x(t)󵄩󵄩󵄩X =: θn (t).

(9.75)

Let K = {t ∈ [0, T] : lim infn ξn (t) < 0}, which is clearly a measurable set. Suppose that the Lebesgue measure |K| > 0. By (9.75), {xn (t) : n ∈ ℕ} ⊂ X is bounded for t ∈ K ∩ ([0, T] \ N) ≠ 0. So, we may assume that xn (t) ⇀ x(t) in X. Fix t and choose a suitable subsequence {ξnk (t) : k ∈ ℕ}, such that lim infn ξn (t) = limk ξnk (t). Since A is pseudomonotone, ⟨A(t, xnk (t)), xnk (t) − x(t)⟩ → 0, which is a contradiction since t ∈ K. So μ(K) = 0, i. e., lim infn ξn (t) ≥ 0, a. e. in [0, T]. By Fatou’s lemma, T

T

T

0 ≤ ∫ lim inf ξn (t)dt ≤ lim inf ∫ ξn (t)dt ≤ lim sup ∫ ξn (t)dt ≤ 0; 0

n

n

0

n

0

T

hence, ∫0 ξn (t)dt → 0. Write |ξn (t)| = ξn+ (t) + ξn− (t) = ξn (t) + 2ξn− (t). Note that ξn− (t) → 0, a. e. on [0, T]. It is clear from (9.75) that {θn : n ∈ ℕ} ⊂ L1 ([0, T]) is uniformly integrable. Since T T 0 ≤ ξn− (t) ≤ θn− (t), we conclude that ∫0 ξn− (t)dt → 0. Therefore, ∫0 |ξn (t)|dt → 0. We may assume that ξn (t) → 0 a. e. on [0, T]. Since A is pseudomonotone, we have A(t, xn (t)) ⇀ A(t, x(t)) in X ⋆ , and ⟨A(t, xn (t)), xn (t)⟩ → ⟨A(t, x(t)), x(t)⟩,

a. e. on [0, T].

̂ n ) ⇀ A(x) ̂ ̂ n ), xn ⟩ → ⟨A(x), ̂ By dominated convergence, we get A(x in X ⋆ and ⟨A(x x⟩, ̂ is L-pseudomonotone. By Theorem 9.5.15, R(L+ A) ̂ = X, i. e., problem (9.69) therefore, A has a solution. The results of this section can be applied for instance to the heat equation or the generalized heat equation involving the p-Laplace operator.

Bibliography [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

[11] [12] [13] [14] [15] [16] [17]

[18] [19] [20] [21] [22] [23]

Robert A Adams and John JF Fournier. Sobolev spaces, volume 140. Academic press, 2003. Ravi P Agarwal, Maria Meehan, and Donal O’Regan. Fixed point theory and applications, volume 141. Cambridge university press, 2001. Yakov Alber and Irina Ryazantseva. Nonlinear ill-posed problems of monotone type. Springer, 2006. Fernando Albiac and Nigel John Kalton. Topics in Banach space theory, volume 233. Springer, 2016. Charalampos D Aliprantis and Kim C Border. Infinite dimensional analysis: a hitchhiker’s guide. Springer, 2006. M Marques Alves and Benar Fux Svaiter. A new proof for maximal monotonicity of subdifferential operators. Journal of Convex Analysis, 15(2):345, 2008. Antonio Ambrosetti and David Arcoya. An introduction to nonlinear functional analysis and elliptic problems, volume 82. Birkhäuser, Springer Science & Business Media, 2011. Antonio Ambrosetti and Paul H Rabinowitz. Dual variational methods in critical point theory and applications. Journal of Functional Analysis, 14(4):349–381, 1973. Aomar Anane. Simplicité et isolation de la premiere valeur propre du p-laplacien avec poids. Comptes Rendus de L’Académie des Sciences Series 1, Mathematics, 305(16):725–728, 1987. Aomar Anane and Jean-Pierre Gossez. Strongly nonlinear elliptic problems near resonance: a variational approach. Communications in Partial Differential Equations, 15(8):1141–1159, 1990. David Arcoya and Luigi Orsina. Landesman-Lazer conditions and quasilinear elliptic equations. Nonlinear Analysis: Theory, Methods & Applications, 28(10):1623–1632, 1997. Francisco J Aragón Artacho, Jonathan M Borwein, and Matthew K Tam. Douglas–Rachford feasibility methods for matrix completion problems. ANZIAM Journal, 55(4):299–326, 2014. Hedy Attouch, Giuseppe Buttazzo, and Gérard Michaille. Variational analysis in Sobolev and BV spaces: applications to PDEs and optimization, volume 17. Siam, 2014. Jean P Aubin and Ivar Ekeland. Applied nonlinear analysis. Wiley New York, 1984. Jean-Pierre Aubin and Hélène Frankowska. Set-valued analysis. Springer Science & Business Media, 2009. Marino Badiale and Enrico Serra. Semilinear elliptic equations for beginners: existence results via the variational approach. Springer Science & Business Media, 2010. John M Ball and Victor J Mizel. One-dimensional variational problems whose minimizers do not satisfy the Euler–Lagrange equation. In Analysis and thermomechanics, pages 285–348. Springer, 1987. Viorel Barbu. Nonlinear differential equations of monotone types in Banach spaces. Springer Science & Business Media, 2010. Viorel Barbu and Teodor Precupanu. Convexity and optimization in Banach spaces. Springer Science & Business Media, 2012. Heinz H Bauschke and Patrick L Combettes. Convex analysis and monotone operator theory in Hilbert spaces. Springer Science & Business Media, 2011. Lisa Beck. Elliptic regularity theory. Lecture notes of the Unione Matematica Italiana, volume 19, 2016. Marino Belloni and Bernd Kawohl. A direct uniqueness proof for equations involving the p-Laplace operator. Manuscripta Mathematica, 109(2):229–231, 2002. Alain Bensoussan and J-L Lions. Applications of variational inequalities in stochastic control, volume 12. Elsevier, 2011.

https://doi.org/10.1515/9783110647389-010

464 | Bibliography

[24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34]

[35] [36] [37] [38]

[39] [40] [41] [42] [43] [44]

[45] [46] [47]

[48]

Jonathan M Borwein and D Russell Luke. Duality and convex programming. In Handbook of mathematical methods in imaging, pages 229–270. Springer, 2011. Jonathan M Borwein and Jon D Vanderwerff. Convex functions: constructions, characterizations and counterexamples, volume 109. Cambridge University Press Cambridge, 2010. Jonathan M Borwein and Qiji J Zhu. Techniques of variational analysis. Springer, 2005. Haim Brezis. Operateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert, volume 50. North Holland, 1973. Haim Brezis. Functional analysis, Sobolev spaces and partial differential equations. Springer, 2011. Haïm Brezis and Louis Nirenberg. Remarks on finding critical points. Communications on Pure and Applied Mathematics, 44(8-9):939–963, 1991. Arne Brøndsted and Tyrell R Rockafellar. On the subdifferentiability of convex functions. Proceedings of the American Mathematical Society, 16(4):605–611, 1965. Felix E Browder. Nonexpansive nonlinear operators in a Banach space. Proceedings of the National Academy of Sciences of the United States of America, 54(4):1041–1044, 1965. Anca Capatina. Variational inequalities and frictional contact problems. Springer, 2014. James Caristi. Fixed point theorems for mappings satisfying inwardness conditions. Transactions of the American Mathematical Society, 215:241–251, 1976. Siegfried Carl, Vy K Le, and Dumitru Motreanu. Nonsmooth variational problems and their inequalities: comparison principles and applications. Springer Science & Business Media, 2007. Nigel L Carothers. A short course on Banach space theory. Cambridge University Press, 2004. Thierry Cazenave. An introduction to semilinear elliptic equations. Editora do Instituto de Matemática, Universidade Federal do Rio de Janeiro, Rio de Janeiro, volume 164, 2006. Antonin Chambolle and Pierre-Louis Lions. Image recovery via total variation minimization and related problems. Numerische Mathematik, 76(2):167–188, 1997. Antonin Chambolle and Thomas Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40(1):120–145, 2011. Antonin Chambolle and Thomas Pock. An introduction to continuous optimization for imaging. Acta Numerica, 25:161–319, 2016. Antonin Chambolle and Thomas Pock. On the ergodic convergence rates of a first-order primal–dual algorithm. Mathematical Programming, 159(1-2):253–287, 2016. Kung-Ching Chang. Methods in nonlinear analysis. Springer Science & Business Media, 2006. Michel Chipot. Elements of nonlinear analysis. Birkhäuser, 2000. Michel Chipot. Elliptic equations: an introductory course. Springer Science & Business Media, 2009. Doina Cioranescu and Patrizia Donato. An introduction to homogenization, volume 4. Oxford lecture series in mathematics and its applications, volume 17. The Clarendon Press Oxford University Press, New York, 1999. Ioana Cioranescu. Geometry of Banach spaces, duality mappings and nonlinear problems, volume 62. Springer Science & Business Media, 1990. Patrick L Combettes and Valérie R Wajs. Signal recovery by proximal forward-backward splitting. Multiscale Modeling & Simulation, 4(4):1168–1200, 2005. Patrick L Combettes and Isao Yamada. Compositions and convex combinations of averaged nonexpansive operators. Journal of Mathematical Analysis and Applications, 425(1):55–70, 2015. John B Conway. A course in functional analysis, volume 96. Springer Science & Business Media, 2013.

Bibliography | 465

[49] [50] [51] [52] [53]

[54] [55] [56]

[57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73]

David G Costa. An invitation to variational methods in differential equations. Springer Science & Business Media, 2010. Bernard Dacorogna. Direct methods in the calculus of variations, volume 78. Springer Verlag, 2008. Gianni Dal Maso. An introduction to Γ-convergence, volume 8. Springer Science & Business Media, 2012. Ennio De Giorgi. Teoremi di semicontinuita nel calcolo delle variazioni. Istituto Nazionale di Alta Matematica, Roma, 1969, 1968. Pablo De Nápoli and María Cristina Mariani. Mountain pass solutions to equations of p-Laplacian type. Nonlinear Analysis: Theory, Methods & Applications, 54(7):1205–1219, 2003. Gerard Debreu. Theory of value: an axiomatic analysis of economic equilibrium, volume 17. Yale University Press, 1959. Klaus Deimling. Nonlinear functional analysis. Dover, 2010. Emmanuele Di Benedetto and Neil S Trudinger. Harnack inequalities for quasi-minima of variational integrals. Annales de L’Institut Henri Poincaré Analyse Non Linéaire, 1(4):295–308, 1984. Joseph Diestel. Geometry of Banach spaces-selected topics. Lecture notes in mathematics, volume 485. Springer, 1975. Joseph Diestel and John Jerry Uhl. Vector measures, volume 15. American Mathematical Soc., 1977. James Dugundji. Topology. Allyn and Bacon, Inc, 1970. Ivar Ekeland. Sur les problèmes variationnels. Comptes Rendus Mathématiques de L’Académie Des Sciences, 275:1057–1059, 1972. Ivar Ekeland. On the variational principle. Journal of Mathematical Analysis and Applications, 47(2):324–353, 1974. Ivar Ekeland and Roger Temam. Convex analysis and variational problems, volume 28. Siam, 1999. Lawrence C Evans. Partial differential equations. AMS, Providence, 1998. Alessio Figalli. Free boundary regularity in obstacle problems. arXiv preprint arXiv:1807.01193, 2018. Dario C. Figueiredo. Lectures on the Ekeland variational principle with applications and detours. Springer Berlin etc, 1989. Irene Fonseca and Giovanni Leoni. Modern methods in the calculus of variations: Lp spaces. Springer New York, 2007. Enrico Giusti. Direct methods in the calculus of variations. World Scientific, 2003. Antoine Henrot. Extremum problems for eigenvalues of elliptic operators. Springer Science & Business Media, 2006. Theophil Henry Hildebrandt and Lawrence M Graves. Implicit functions and their differentials in general analysis. Transactions of the American Mathematical Society, 29(1):127–153, 1927. Shouchuan Hu and Nikolaos Socrates Papageorgiou. Handbook of multivalued analysis, volume 1. Kluwer Dordrecht, 1997. Kazufumi Ito and Karl Kunisch. Lagrange multiplier approach to variational problems and applications, volume 15. Siam, 2008. Youssef Jabri. The mountain pass theorem: variants, generalizations and some applications, volume 95. Cambridge Univ Pr, 2003. David Kinderlehrer and Guido Stampacchia. An introduction to variational inequalities. Academic Press, New York, 1981.

466 | Bibliography

[74] [75] [76]

[77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87]

[88] [89] [90] [91] [92] [93]

[94] [95] [96] [97]

William Art Kirk. A fixed point theorem for mappings which do not increase distances. The American Mathematical Monthly, 72(9):1004–1006, 1965. Gottfried Köthe. Topological vector spaces. Springer-Verlag, 1969. Nikolai Vladimirovich Krylov and Mikhail V Safonov. A certain property of solutions of parabolic equations with measurable coefficients. Izvestiâ Akademii Nauk SSSR Seriâ Matematičeskaâ, 44(1):161–175, 1980. Giovanni Leoni. A first course in Sobolev spaces. American Mathematical Soc., 2009. Elliott H Lieb, Michael Loss, et al.Analysis, volume 14. American Mathematical Society Providence, Rhode Island, 2001. Peter Lindqvist. On the equation div(|∇u|p−2 ∇u) + λ|u|p−2 u = 0. Proceedings of the American Mathematical Society, 109(1):157–164, 1990. Jacques-Louis Lions. Quelques méthodes de résolution des problemes aux limites non linéaires, volume 31. Dunod Paris, 1969. Anthony R Lovaglia. Locally uniformly convex Banach spaces. Transactions of the American Mathematical Society, 78(1):225–238, 1955. Robert E Megginson. Banach space theory, volume 183. Springer Science & Business Media, 2012. Ernest Michael. Continuous selections. II. Annals of Mathematics, 361–382, 1956. Jean-Jacques Moreau. Proximité et dualité dans un espace hilbertien. Bulletin de la Société Mathématique de France, 93:273–299, 1965. Nikolaos S Papageorgiou and Sophia Th Kyritsi-Yiallourou. Handbook of applied analysis, volume 19. Springer Science & Business Media, 2009. Dan Pascali and Silviu Sburlan. Nonlinear mappings of monotone type. Editure Academiei Romania, 1978. Electra V Petracou and Athanasios N Yannacopoulos. Decision theory under risk and applications in social sciences: II. Game theory. In Mathematical modeling with multidisciplinary applications, pages 421–447. Wiley, 2013. Juan Peypouquet. Convex optimization in normed spaces: theory, methods and examples. Springer, 2015. Robert R Phelps. Convex functions, monotone operators and differentiability, volume 1364. Springer, 1993. Robert R Phelps. Lectures on maximal monotone operators. Extracta Mathematicae, 12(3):193–230, 1997. Lee C Potter and KS Arun. A dual approach to linear inverse problems with convex constraints. SIAM Journal on Control and Optimization, 31(4):1080–1092, 1993. Radu Precup. Methods in nonlinear integral equations. Springer Science & Business Media, 2013. Patrizia Pucci and Vicentiu Radulescu. The impact of the mountain pass theory in nonlinear analysis: a mathematical survey. Bollettino dell’Unione Matematica Italiana, 3(3):543–582, 2010. Paul H Rabinowitz. Minimax methods in critical point theory with applications to differential equations, volume 65. Amer Mathematical Society, 1986. Vicentiu D Radulescu. Qualitative analysis of nonlinear elliptic partial differential equations. Hindawi Publishing Corporation, 2008. Simeon Reich. Weak convergence theorems for nonexpansive mappings in Banach spaces. Journal of Mathematical Analysis and Applications, 67(2):274–276, 1979. Michael Renardy and Robert C. Rogers. An introduction to partial differential equations, volume 13. Springer Verlag, 2004.

Bibliography | 467

[98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111]

Sergey Repin. A posteriori error estimation for variational problems with uniformly convex functionals. Mathematics of Computation, 69(230):481–500, 2000. R Tyrell Rockafellar et al.On the maximal monotonicity of subdifferential mappings. Pacific Journal of Mathematics, 33(1):209–216, 1970. Jose-Francisco Rodrigues. Obstacle problems in mathematical physics, volume 134. Elsevier, 1987. Tomáš Roubíček. Nonlinear partial differential equations with applications, volume 153. Springer Science & Business Media, 2013. Stephen Simons. A new proof of the maximal monotonicity of subdifferentials. Journal of Convex Analysis, 16(1):165–168, 2009. Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171–176, 1958. Guido Stampacchia. Le problème de dirichlet pour les équations elliptiques du second ordre à coefficients discontinus. In Annales de l’institut Fourier, volume 15, pages 189–257, 1965. Michael Struwe. Variational methods: applications to nonlinear partial differential equations and Hamiltonian systems. Springer Verlag, 2008. Raymond Trémolières, J-L Lions, and R Glowinski. Numerical analysis of variational inequalities, volume 8. Elsevier, 2011. S Troyanski. On locally uniformly convex and differentiable norms in certain non-separable Banach spaces. Studia Mathematica, 37(2):173–180, 1971. Jack Warga. Optimal control of differential and functional equations. Academic press, 2014. James Yeh. Real analysis: theory of measure and integration. World Scientific, 2006. Eberhard Zeidler. Nonlinear functional analysis vol. 1: fixed-point theorems. Springer-Verlag Berlin and Heidelberg GmbH & Co. K, 1986. Eberhard Zeidler. Nonlinear functional analysis and its applications, part II/B: nonlinear monotone operators, volume 2. Springer Verlag, 1989.

Index Γ-regularization (Def. 5.2.13) 201 Adjoint operator 6 Algorithm – Chambolle–Pock PDHG (Alg. 5.6.1) 237 – Chambolle–Pock PDHG convergence (Prop. 5.6.2) 238 – Douglas–Rachford (Alg. 4.8.11) 188 – Douglas–Rachford convergence (Prop. 4.8.12) 188 – Forward-backward (Alg. 4.8.6) 185 – Forward-backward convergence (Prop. 4.8.7) 186 – Primal-Dual 235 – Projected gradient (Ex. 4.8.9) 186 – Proximal point (Alg. 4.8.1) 179 – Proximal point convergence (Prop. 4.8.2) 179 Augmented Lagrangian Method 232 Averaged operator 134 Banach space – Locally uniformly convex 97 – Smooth 165 – Strictly convex 92 – Uniformly convex 95 Bilinear form 342 – Coercive (Def. 7.3.1) 342 – Continuous (Def. 7.3.1) 342 Bilinear forms 359 Blinear forms – Elliptic operators 360 C 2 domain 366 Caccioppoli inequalities (Eq. (6.57)) 281 Calculus of variations – Euler Lagrange equation (Th. 6.5.2) 277 – Existence (Th. 6.4.2) 274 – Regularity 281 Coercive operator (Def. 9.2.11) 414 Contraction map (Def. 3.1.1) 105 Convergence – a. e. 36 – various modes in Lebesgue spaces 36 – Weak 18 – Weak ⋆ 13 Convex feasibility problems (Ex. 4.8.13) 189 Convex series (Def. 1.2.6) 23

Convexity – Locally uniformly convex spaces 97 – Optimization 87 – Strictly convex spaces 92 – Uniformly convex spaces 95 De Giorgi class 281 Demicontinuous operator (Def. 9.2.8) 413 Derivative – Directional, vector valued (Def. 2.1.15) 69 – Directional(Def. 2.1.1) 61 – Frechet, vector valued (Def. 2.1.15) 69 – Frechet (Def. 2.1.6) 65 – Gâteaux, vector valued (Def. 2.1.15) 69 – Gâteaux (Def. 2.1.2) 61 Difference operator 289 Dirichlet functional 66, 252 Douglas–Rachford method (Alg. 4.8.11) 188 Dual problem – Fenchel duality (Prop. 5.4.1) 213 Duality gap 191, 226 Duality map – Definition (Def. 1.1.16) 6 – Maximal monotonicity (Ex. 9.3.20) 429 Ekeland variational principle 141 Elliptic problems – Quasilinear 309, 399, 446 – Semilinear 294, 376, 385 – Weak solution, Definition(Def. 7.6.4) 362 – Weak solutions, Existence (Prop. 7.6.5) 362 Evolution triple (Def. 1.6.11) 51 Extreme points of convex set 92 Faedo–Galerkin method 414 Fejér sequence 135, 179 Fenchel Duality – Dual problem (Prop. 5.4.1) 213 – Linear programming (Ex. 5.4.5) 219 Fenchel duality – Minimization convex function linear constraints (Ex. 5.4.6) 221 Fenchel Duality – Primal problem (Prop. 5.4.1) 213 – Proposition 5.4.1 213 – Strong (Prop. 5.4.1) 213 – Weak (Prop. 5.4.1) 213

470 | Index

Fenchel–Legendre biconjugate – Definition (Def. 5.2.11) 200 – Fenchel–Moreau–Rockafellar (Th. 5.2.15) 202 Fenchel–Legendre conjugate – Definition (Def. 5.2.1) 196 – Epigraph (Ex. 5.2.8) 198 – Indicator convex set (Ex. 5.2.7) 198 – Indicator unit ball (Ex. 5.2.6) 198 – Moreau–Yosida (Ex. 5.2.9) 198 – Norm (Ex. 5.2.4) 197 – Properties (Prop. 5.2.10) 199 – Subdifferential (Prop. 5.2.18) 204 – Young–Fenchel inequality (Prop. 5.2.2) 196 Finite intersection property 57 Forward-Backward method, optimization (Alg. 4.8.6) 185 Fredholm alternative 30 Functional – C 1 (Def. 2.1.9) 66 – Coercive (Def. 2.2.6) 72 – Convex (Def. 2.3.2) 76 – Dirichlet 252 – Frechet differentiable (Def. 2.1.6) 65 – Gâteaux differentiable (Def. 2.1.2) 61 – Hemicontinuous (Def. 2.1.12) 68 – Integral – Semicontinuity 268 – Semicontinuity, Lebesgue spaces 269 – Semicontinuity, Sobolev spaces 273 – Local maximum (Def. 2.2.4) 72 – Local minimum (Def. 2.2.4) 72 – Lower semicontinuous (Def. 2.2.1) 71 – Maximum (Def. 2.2.4) 72 – Minimum (Def. 2.2.4) 72 – Minimum first order condition (Th. 2.2.13) 75 – Rayleigh–Fisher, FR 261 – Sequentially lower semicontinuous (Def. 2.2.1) 71 – Strongly convex (Def. 2.3.3) 76 – Uniformly convex (Def. 2.3.3) 76 – Weakly lower semicontinuous (Def. 2.2.1) 71 – Weakly sequentially lower semicontinuous (Def. 2.2.1) 71 Gagliardo–Nirenberg–Sobolev inequality (eq. (1.10)) 44 Galerkin method 355 Gronwall inequality 145

Half space (Def. 1.2.8) 24 Hammerstein equations 129 Harnack inequality 283 Hemicontinuity – Functionals (Def. 2.1.12) 68 – Operators (Def. 9.2.8) 413 Hölder continuous function 58 Hyperplane (Def. 1.2.8) 24 Indicator function of convex set (Def. 2.3.9) 79 Inequalities – Caccioppoli (Eq. (6.57)) 281 – Gagliardo–Nirenberg–Sobolev (eq. (1.10)) 44 – Gronwall 145 – Harnack 283 – Hölder (Th. 1.4.2) 34 – Minkowski (Th. 1.4.2) 34 – Polya–Szego 267 – Polyak–Lojasiewicz (Ex. 2.3.26) 86 – Stampacchia (Eq. (6.113) and (6.114)) 316 – Stampacchia (Eqs. (6.14)–(6.15)) 258 – Young–Fenchel (Prop. 5.2.2) 196 Inf-convolution – Definition (Def. 5.3.1) 207 – Properties (Prop. 5.3.4) 207 – Subdifferential (Prop. 5.3.5) 210 Integral equations 108 – Hammerstein 129 – Volterra 125 Lagrangian Duality – Augmented Lagrangian Method (Ex. 5.5.7) 232 Lagrangian duality – Dual problem (Prop. 5.5.2) 225 Lagrangian Duality – Lagrange multipliers (Ex. 5.5.5) 228 Lagrangian duality – Primal problem (Prop. 5.5.2) 225 – Prop. 5.5.2 225 – Slater condition (Prop. 5.5.2) 226 – Strong (Prop. 5.5.2) 225 – Weak (Prop. 5.5.2) 225 Lagrangian function (Lem. 5.5.1) 225 Laplacian – Courant–Fisher minimax (Th. 6.2.14) 266 – Eigenvalue problems (Prop. 6.2.12) 262 – Existence (Prop. 6.2.3) 253 – Free boundary value problem (Prop. 7.2.1) 338 – Lp estimates (Th. 6.2.10) 258

Index |

471

– Regularity (Prop. 6.2.9) 257 – Strong maximum principle (Prop. 6.2.7) 256 – Variational theory 252 – Weak maximum principle (Prop. 6.2.5) 255 Lasso problems (Ex. 4.8.10) 187 Lebesgue points 59 Lebesgue spaces – Density 36 – Dual spaces 35 Lebesgue spaces Lp (𝒟), 1≤p≤∞. 33 Lebesgues spaces – weak compactness 38 Leray–Schauder alternative 127 Linear programming (Ex. 5.4.5) 219 Lipschitz domain 43 Locally uniformly convex spaces 97

– of support function subdifferential (Ex. 5.3.7) 212 – Properties (Prop. 4.7.6) 175 – Support function (Ex. 5.3.6) 211 Mountain pass theorem 380 Multivalued map 52 – Closed 52 – Closed valued 52 – Compact valued 52 – Domain 52 – Graph 52 – Image 52 – Lower inverse 52 – Lower semicontinuous 52 – Upper inverse 52 – Upper semicontinuous 52

Maximal monotone operator – Criteria (Th. 9.3.8) 421 – Definition (Def. 9.3.1) 417 – Duality map (Ex. 9.3.20) 429 – Element of minimal norm, Definition (Def. 9.3.21) 430 – Element of minimal norm A0 (x), (Th. 9.3.26) 432 – Properties (Prop. 9.3.5) 419 – Resolvent, Definition (Def. 9.3.21) 430 – Resolvent, Properties (Prop. 9.3.24) 431 – Subdifferential (Th. 9.3.18) 427 – Sum (Th. 9.3.31) 439 – Surjectivity (Th. 9.3.12) 423 – Yosida approximation, Definition (Def. 9.3.21) 430 – Yosida approximation, Properties (Th. 9.3.26) 432 Mean value of a function 58 Measure – Compactness 38 – Lebesgue 33 – Radon 34 Minimax equality 191 Minkowski functional (Def. 2.3.10) 79 Monotone operator – Definition (Def. 9.2.1) 410 – Local boundedness (Th. 9.2.6) 411 – Strictly, Definition (Def. 9.2.1) 410 Moreau decomposition (Ex. 5.2.22) 206 Moreau–Yosida approximation – Definition (Def. 4.7.3) 174

Norm – Lp , 1≤p≤∞ 34 1,p – Sobolev space W0 (𝒟) 45 – Sobolev space W 1,p (𝒟) 40 – Total variation 35 Normal cone (Def. 4.1.5) 149 Normed spaces – Bidual 6 – Canonical embedding 7 – Reflexive space 8 – Weak convergence 18 – Weak topology 14 – Weak ⋆ topology 10 – Weak ⋆ topology 10 Normed vector spaces – Dual space 5 One-sided derivative (Def. 4.2.6) 154 Operator – Averaged 134 – Coercive (Def. 9.2.11) 414 – Demicontinuous (Def. 9.2.8) 413 – Difference 289 – Domain 1 – Graph 1 – Hemicontinuous (Def. 9.2.8) 413 – Kernel 1 – Linear – Adjoint 6 – Bounded 1 – Compact 28 – Completely continuous 28

472 | Index

– Fredholm 30 – Maximal monotone – Criteria (Th. 9.3.8) 421, 422 – Definition (Def. 9.3.1) 417 – Properties (Prop. 9.3.5) 419 – Monotone – Definition (Def. 9.2.1) 410 – Surjectivity (Th. 9.2.12) 414 – p-Laplacian, Defnition (Eq. (6.100)) 307 – Proximity proxφ – Definition (Def. 4.7.1) 171 – Properties (Prop. 4.7.2) 171 – Range 1 – Resolvent, Definition (Def. 9.3.21) 430 – Soft-thresholding (shrinkage) (Def. 4.7.5) 174 – Strictly Monotone – Definition (Def. 9.2.1) 410 – Translation 289 – Yosida approximation (Def. 9.3.21) 430 Oscillation of a function 58, 321 p-Laplacian 410 p-Laplacian, Defnition (Eq. (6.100)) 307 Palais–Smale condition 144 Partition of unity 58 Penalization method 351 Primal problem – Fenchel duality (Prop. 5.4.1) 213 Projected gradient method (Ex. 4.8.9) 186 Projection in Hilbert space 89 Proximal method, optimization 178 Proximal point method, optimization (Alg. 4.8.1) 179 Proximity operator proxφ – Definition (Def. 4.7.1) 171 – Properties (Prop. 4.7.2) 171 Q-minimizer (Def. 6.6.6) 284 Quasi-concave function 193 Quasi-convex function 193 Quasilinear elliptic problems 309, 399, 446 Rayleigh–Fisher functional, FR 261 Resolvent operator – Definition (Def. 9.3.21) 430 – Properties 432 – Properties (Prop. 9.3.24) 431 Saddle point (Def. 5.1.1) 192

Saddle value 191 Semicontinuity of integral functionals 268 – Lebesgue spaces 269 – Sobolev spaces 273 Semilinear elliptic problems 294, 376, 385 Set – 𝔸(F ) (Def. 5.2.13) 201 – Algebraic interior (core) (Def. 1.2.5) 23 – Convex (Def. 1.2.1) 23 – CS closed (Def. 1.2.6) 23 Slater condition 226 Smooth Banach space 165 Sobolev spaces – Embeddings 43 1,p – W0 (𝒟) 41 k,p

– W0 (𝒟) 41 ⋆

1,p

⋆

k,p

– W −1,p (𝒟) := (W0 (𝒟))⋆ 41

– W −k,p (𝒟) := (W0 (𝒟))⋆ 41 – W 1,p (𝒟) 39 – W k,p (𝒟) 40 Soft-thresholding (shrinkage) operator (Def. 4.7.5) 174 Spaces – Cc∞ (𝒟) 36 – Cc (𝒟) 35 – Hölder C 0,α 58 – Lebesgue, Lp (𝒟), 1≤p≤∞ 33 – Lebesgue–Bochner Lp (I; X ) 49 – Reflexive 8 1,p – Sobolev W0 (𝒟) 41 k,p

– Sobolev W0 (𝒟) 41 ⋆

1,p

⋆

k,p

– Sobolev W −1,p (𝒟) := (W0 (𝒟))⋆ 41

– Sobolev W −k,p (𝒟) := (W0 (𝒟))⋆ 41 – Sobolev W 1,p (𝒟) 39 – Sobolev W k,p (𝒟) 40 Stampacchia inequalities (Eqs. (6.14)–(6.15)) 258 Stampacchia inequality (Eq. (6.113) and (6.114)) 316 Strictly convex spaces 92 Strictly monotone operator (Def. 9.2.1) 410 Sub solution – Variational inequality 373 Subdifferential – approximation ϵ-subdifferential (Def. 4.5.1) 166 – Calculus (Prop. 4.3.1) 157

Index |

– Composition of convex function with continuous linear operator (Prop. 4.3.4) 160 – Convex function (Prop. 4.2.1) 151 – Definition (Def. 4.1.1) 147 – Differentiable function (Prop. 4.1.2) 147 – Indicator function (Ex. 4.1.6) 150 – Inf-convolution (Prop. 5.3.5) 210 – Maximal monotonicity (Th. 9.3.18) 427 – One-sided derivative (Prop. 4.2.7) 154 – Optimization (Prop. 4.6.1) 168 – Power of norm (Prop. 4.4.1) 163 – Sum of convex functions (Th. 4.3.2) 158 – Support function (Ex. 5.2.23) 207 Super solution – Variational inequality 373 Support function – Definition (Def. 2.3.12) 80 – Moreau–Yosida approximation (Ex. 5.3.6) 211 – Subdifferential of Moreau–Yosida approximation (Ex. 5.3.7) 212 Surjectivity – Maximal monotone operators (Th. 9.3.12) 423 – Monotone operators (Th. 9.2.12) 414 – Pseudomonotone operators (Th. 9.4.6) 444 Theorem – Alaoglu (Th. 1.1.36) 12 – Arzelá–Ascoli (Th. 1.8.3) 57 – Baire (Th. 2.7.1) 103 – Banach fixed point (Th. 3.1.2) 105 – Banach–Steinhaus (Th. 1.1.7) 3 – Brouwer fixed point (Th. 3.2.4) 117 – Browder fixed point (Th. 3.4.4) 131 – Browder maximal monotone operators surjectivity (Th. 9.3.16) 426 – Calculus of variations, Euler–Lagrange (Th. 6.5.2) 277 – Calculus of variations, Existence (Th. 6.4.2) 275 – Calculus of variations, Semicontinuity I (Th. 6.3.2) 269 – Calculus of variations, Semicontinuity II (Th. 6.3.10) 274 – Caristi fixed point (Th. 3.6.2) 143 – Cauchy, Lipschitz, Picard (Th. 3.1.5) 107 – Closed graph (Th. 1.1.11) 4 – Courant–Fisher minimax (Th. 6.2.14) 266 – Debrunner-Flor lemma (Th. 3.5.1) 138

473

– Duality map properties strictly convex spaces (Th. 2.6.17) 98 – Duals of Lebesgue spaces (Th. 1.4.5) 35 – Eberlein–Smulian (Th. 1.1.56) 19 – Edelstein fixed point (Th. 3.1.4) 107 – Ekeland variational principle (Th. 3.6.1) 141 – Fatou lemma (Th. 1.4.9) 37 – Fenchel duality (Prop. 5.4.1) 213 – Fenchel–Moreau–Rockafellar (Th. 5.2.15) 202 – Fredholm alternative (Th. 1.3.9) 30 – Fredholm operators (Th. 1.3.8) 30 – Hahn–Banach strict separation (Th. 1.2.9) 24 – Hahn–Banach (Th. 1.1.4) 3 – Implicit function 112 – Inverse function 109 – Kakutani fixed point for multivalued maps (Th. 3.5.2) 139 – Kakutani (Prop. 3.2.15) 123 – Knaster–Kuratowski–Mazurkiewicz Lemma 119 – Krasnoselskii–Mann (Th. 3.4.9) 134 – Lagrangian duality (Prop. 5.5.2) 225 – Lax–Milgram (Th. 7.3.5) 345 – Lax–Milgram–Stampacchia (Th. 7.3.4) 343 – Lebesgue–Bochner embedding (Th. 1.6.6) 50 – Lebesgue’s dominated convergence (Th. 1.4.9) 37 – Leray–Schauder alternative (Th. 3.3.5) 127 – Lions–Stampacchia (Th. 7.3.9) 347 – Maximal monotone operator surjectivity (Th. 9.3.12) 423 – Maximal monotone operators properties (Prop. 9.3.5) 419 – Maximal monotonicity of subdifferential (Th. 9.3.18) 427 – Mazur (Prop. 1.2.18) 27 – Mazur’s lemma (Prop. 1.2.17) 26 – Mean value (Prop. 2.1.5) 64 – Michael selection (Th. 1.7.7) 55 – Minimax (Th. 5.1.5) 193 – Minty 345 – Minty–Browder, surjectivity (Th. 9.2.12) 414 – Monotone convergence (Th. 1.4.9) 37 – Monotone operators, Local boundedness (Th. 9.2.6) 411 – Moreau–Rockafellar (Th. 4.3.2) 158 – Mountain pass Abrosetti–Rabinowitz 380 – Open mapping (Th. 1.1.9) 4 – Pettis measurability (Th. 1.6.2) 48

474 | Index

– Projection Hilbert space (Th. 2.5.2) 90 – Radon measures as a dual space (Th. 1.4.4) 35 – Resolvent operator properties (Prop. 9.3.24) 431 – Riemann–Lebesgue lemma (Lem. 6.9.1) 314 – Riesz representation, Hilbert spaces, (Th. 1.1.14) 5 – Rockafellar, sum of maximal monotone operators (Th. 9.3.31) 439 – Schaeffer fixed point (Th. 3.3.6) 128 – Schauder fixed point (Th. 3.3.2) 124 – Sobolev embeddings (Theorem 1.5.11) 43 – Strictly convex space properties (Th. 2.6.5) 93 – Troyanski (Th. 2.6.16) 97 – Vitali (Prop. 1.4.12) 38 – Weierstrass minimum (2.2.5) 72 – Yosida approximation properties (Th. 9.3.26) 432 Topology – Strong 9 – Weak 14 – Weak ⋆ 10

Trace operators (Def. 1.5.16) 47 Trace space H1/2 (𝜕𝒟) (Prop. 1.5.17) 47 Translation operator 289 Uniformly convex spaces 95 Urysohn property 18 Variational inequalities – First kind 341 – Internal approximation 356 – Penalization method 351 – Regularity 374 – Second kind 348 – Sub solutions 373 – Super solutions 373 Volterra integral equation 125 Yosida approximation – Banach space (Th. 9.3.30) 438 – Definition (Def. 9.3.21) 430 – Properties, Hilbert space (Prop. 9.3.27) 436 – Properties (Th. 9.3.26) 432