Analysis by its history 9780387770314, 9780387770369, 0387770313

Citation preview

Undergraduate Texts in Mathematics Readings in Mathematics Editors

s. Axler

K.A. Ribet

Graduate Texts in Mathematics Readings in Mathematics Ebbinghaus/Hermes/HirzebruchlKoecher/Mainzer/NeukirchIPrestellRemmert: Numbers Fulton/Harris: Representation Theory: A First Course Murty: Problems in Analytic Number Theory Remmert: Theory of Complex Functions Walter: Ordinary Differential Equations

Undergraduate Texts in Mathematics Readings in Mathematics Anglin: Mathematics: A Concise History and Philosophy Anglin/Lambek: The Heritage of Thales Bressoud: Second Year Calculus HairerlWanner: Analysis by Its History Hammerlin/Hoffmann: Numerical Mathematics Isaac: The Pleasures of Probability Knoebel/Laubenbacher/Lodder/Pengelley: Mathematical Masterpieces: Further Chronicles by the Explorers Laubenbacher/PengeIley: Mathematical Expeditions: Chronicles by the Explorers Samuel: Projective Geometry Stillwell: Numbers and Geometry Toth: Glimpses of Algebra and Geometry, Second Edition

E. Hairer

G. Wanner

Analysis by Its History

~ Springer

Editors E. Hairer G. Wanner Department of Mathematics University of Geneva Geneva, Switzerland

Editorial Board S. Axler Mathematics Department Sau Francisco State University San Francisco, CA 94132 USA [email protected]

K.A. Ribet Mathematics Department University of California at Berkeley Berkeley, CA 94720-3840 USA [email protected]

ISBN 978-0-387-77031-4 ISBN 978-0-387-77036-9 (eBook) DOI 10.1 007/978-0-387 -77036-9 Library of Congress Control Number: 2008925883

© 2008 Springer Science+Business Media New York AII rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC), except for brief excerpts in connection with reviews or scholarly anaJysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

987654321 springer.com

Preface ... that departed from the traditional dry-as-dust mathematics textbook. (M. Kline, from the Preface to the paperback edition of Kline 1972) Also for this reason, I have taken the trouble to make a great number of drawings. (Brieskom & Knorrer, Plane algebraic curves, p. ii) ... I should like to bring up again for emphasis ... points, in which my exposition differs especially from the customary presentation in the textbooks: 1. Illustration of abstract considerations by means of figures. 2. Emphasis upon its relation to neighboring fields, such as calculus of differences and interpolation ... 3. Emphasis upon historical growth. It seems to me extremely important that precisely the prospective teacher should take account of all of these. (F. Klein 1908, Eng\. ed. p. 236)

Traditionally, a rigorous first course in Analysis progresses (more or less) in the following order: limits, sets, continuous derivatives integration. mappings functions

'*

'*

'*

On the other hand, the historical development of these subjects occurred in reverse order: Archimedes Cantor 1875 Cauchy 1821 Newton 1665 .;::: Kepler 1615 Dedekind .;::: Weierstrass .;::: Leibniz 1675 Fermat 1638 In this book, with the four chapters Chapter I. Chapter II. Chapter III. Chapter IV.

Introduction to Analysis of the Infinite Differential and Integral Calculus Foundations of Classical Analysis Calculus in Several Variables,

we attempt to restore the historical order, and begin in Chapter I with Cardano, Descartes, Newton, and Euler's famous Introductio. Chapter II then presents 17th and 18th century integral and differential calculus "on period instruments" (as a musician would say). The creation of mathematical rigor in the 19th century by Cauchy, Weierstrass, and Peano for one and several variables is the subject of Chapters III and IV. This book is the outgrowth of a long period of teaching by the two authors. In 1968, the second author lectured on analysis for the first time, at the University of Innsbruck, where the first author was a first-year student. Since then, we have given these lectures at several universities, in German or in French, influenced by many books and many fashions. The present text was finally written up in French for our students in Geneva, revised and corrected each year, then translated into English, revised again, and corrected with the invaluable help of our colleague John Steinig. He has corrected so many errors that we can hardly imagine what we would have done without him.

VI

Preface

Numbering: each chapter is divided into sections. Formulas, theorems, figures, and exercises are numbered consecutively in each section, and we also indicate the section number, but not the chapter number. Thus, for example, the 7th equation to be labeled in Section 11.6 is numbered "(6.7)". References to this formula in other chapters are given as "(11.6.7)". References to the bibliography: whenever we write, say, "Euler (1737)" or "(Euler 1737)", we refer to a text of Euler's published in 1737, detailed references to which are in the bibliography at the end of the book. We occasionally give more precise indications, as for instance "(Euler 1737, p. 25)". This is intended to help the reader who wishes to look up the original sources and to appreciate the often elegant and enthusiastic texts of the pioneers. When there is no corresponding entry in the bibliography, we either omit the parentheses or write, for example, "(in 1580)". Quotations: we have included many quotations from the literature. Those appearing in the text are usually translated into English; the non-English originals can be consulted in the Appendix. They are intended to give the flavor of mathematics as an international science with a long history, sometimes to amuse, and also to compensate those readers without easy access to a library with old books. When the source of a quotation is not included in the bibliography, its title is indicated directly, as for example the book by Brieskorn and Knorrer from which we have quoted above. Acknowledgments: the text was processed in plain Tpc on our Sun workstations at the University of Geneva using macros from Springer-Verlag New York. We are grateful for the help of J.M. Naef, "Mr. Sun" of the "Services Informatiques" of our university. The figures are either copies from old books (photographed by J.M. Meylan from the Geneva University Library and by A. Perruchoud) or have been computed with our Fortran codes and included as Postscript files. The final printing was done on the 1200dpi laser printer of the Psychology Department in Geneva. We also thank the staff of the mathematics department library and many colleagues, in particular R. Bulirsch, P. Deuflhard, Ch. Lubich, R. Marz, A. Ostermann, J.-Cl. Pont, and J.M. Sanz-Serna for valuable comments and hints. Last but surely not least we want to thank Dr. Ina Lindemann and her equipe from Springer-Verlag New York for all her help, competent remarks, and the agreeable collaboration. March 1995

E. Hairer and G. Wanner.

Preface to the 2nd, 3rd, and 4th Corrected Printings. These new printings allowed us to correct several misprints and to improve the text in many places. In particular, we give a more geometric exposition of Tartaglia's solution of the cubic equation, improve the treatment of envelopes, and give a more complete proof of the transformation formula of multiple integrals. We are grateful to many students and colleagues who have helped us to discover errors and possible improvements, in particular R.B. Burckel, H. Fischer, J.-L. Gaudin, and H.-M. Maire. We would like to address special thanks to Y. Kanie, the translator of the Japanese edition. March 1997, April 2000, Sept 2007

E. Hairer and G. Wanner.

Contents Chapter I Introduction to Analysis of the Infinite 1.1 Cartesian Coordinates and Polynomial Functions. . . . . . . . . . . . . . . . . . . . . . . .

2

Algebra.............................................................. 2 "Algebra Nova" . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 6 Descartes's Geometry .................................................. 8 Polynomial Functions .................................................. 10 Exercises ............................................................. 14

1.2 Exponentials and the Binomial Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17 Binomial Theorem .............................. . . . . . . . . . . . . . . . . . . . . . .. Exponential Funcion ................................................... Exercises ............................................................. 1.3 Logarithms and Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Computation of Logarithms ............................................. Computation of Areas .................................................. Area of the Hyperbola and Natural Logarithms ............................ Exercises .............................................................

18 25 28 29 30 33 34 39

1.4 Trigonometric Functions . ............................................. , 40 Basic Relations and Consequences ....................................... Series Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Inverse Trigonometric Functions ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Computation of Pi ..................................................... Exercises .............................................................

1.5 Complex Numbers and Functions . ..................................... , Euler's Formula and Its Consequences .................................... A New View on Trigonometric Functions ................................. Euler's Product for the Sine Function ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Exercises .............................................................

1.6 Continued Fractions . ................................................. , Origins ............................................................... Convergents .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Irrationality ........................................................... Exercises .............................................................

43 46 49 52 55 57 58 61 62 66 68 68 71 76 78

Chapter II Differential and Integral Calculus II.I The Derivative. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 81 The Derivative ......................................................... Differentiation Rules ................................................... Parametric Representation and Implicit Equations .......................... Exercises .............................................................

81 84 88 89 11.2 Higher Derivatives and Taylor Series . .................................. , 91 The Second Derivative ................................................. , 91 De Conversione Functionum in Series .................................... 94 Exercises ............................................................. 97 II.3 Envelopes and Curvature ............................................. , 98 Envelope of a Family of Straight Lines ................................... , 98 The Caustic of a Circle ................................................. 99 Envelope of Ballistic Curves ............................................ 101 Curvature ............................................................. 10 1 Exercises ............................................................. 105

II.4 Integral Calculus . ..................................................... 107 Primitives ............................................................. 107

viii

Contents Applications .......................................................... Integration Techniques ................................................. Taylor's Formula with Remainder ........................................ Exercises .............................................................

11.5

lUi

11.7

11.8

11.9

11.10

109 112 116 117 Functions with Elementary Integral . .................................... 118 Integration of Rational Functions ........................................ 118 Useful Substitutions .................................................... 123 Exercises ............................................................. 125 Approximate Computation ofIntegrais . ................................. 126 Series Expansions ...................................................... 126 Numerical Methods .................................................... 128 Asymptotic Expansions ................................................. 131 Exercises ............................................................. 132 Ordinary Differential Equations .. ...................................... 134 Some Types ofIntegrable Equations ...................................... 139 Second Order Differential Equations ..................................... 140 Exercises ............................................................. 143 Linear Differential Equations . ......................................... 144 Homogeneous Equation with Constant Coefficients ......................... 145 Inhomogeneous Linear Equations ........................................ 148 Cauchy's Equation ..................................................... 152 Exercises ............................................................. 152 Numerical Solution of Differential Equations .... ........................ 154 Euler's Method ........................................................ 154 Taylor Series Method ................................................... 156 Second Order Equations ................................................ 158 Exercises ............................................................. 159 The Euler-Maclaurin Summation Formula ........... ................... 160 Euler's Derivation of the Formula ........................................ 160 De Usu Legitimo Formulae Summatoriae Maclaurinianae ................... 163 Stirling's Formula ...................................................... 165 The Harmonic Series and Euler's Constant ................................ 167 Exercises ............................................................. 169

Chapter ill Foundations of Classical Analysis 111.1 Infinite Sequences and Real Numbers . .................................. 172

Convergence of a Sequence ............................................. 172 Construction of Real Numbers ........................................... 177 Monotone Sequences and Least Upper Bound ............................. 182 Accumulation Points ................................................... 184 Exercises ............................................................. 185 111.2 Infinite Series . ........................................................ 188 Criteria for Convergence ................................................ 189 Absolute Convergence .................................................. 192 Double Series ......................................................... 195 The Cauchy Product of Two Series ....................................... 197 Exchange of Infinite Series and Limits .................................... 199 Exercises ............................................................. 200 III.3 Real Functions and Continuity ......................................... 202 Continuous Functions .................................................. 204 The Intermediate Value Theorem ......................................... 206 The Maximum Theorem ................................................ 206 Monotone and Inverse Functions ......................................... 208 Limit of a Function .................................................... 209 Exercises ............................................................. 210

Contents

ix

111.4 Uniform Convergence and Uniform Continuity . ......................... 213 The Limit of a Sequence of Functions .................................... 213 Weierstrass's Criterion for Uniform Convergence .......................... 216 Uniform Continuity .................................................... 217 Exercises ............................................................. 220 111.5 The Riemann Integral ................................................. 221 Definitions and Criteria of Integrability ................................... 221 Integrable Functions .................................................... 226 Inequalities and the Mean Value Theorem ................................. 228 Integration of Infinite Series ............................................. 230 Exercises ............................................................. 232 111.6 Differentiable Functions ............................................... 235 The Fundamental Theorem of Differential Calculus ........................ 239 The Rules of de L'Hospital .............................................. 242 Derivatives ofInfinite Series ............................................. 245 Exercises ............................................................. 246 111.7 Power Series and Taylor Series .. ....................................... 248 Determination of the Radius of Convergence .............................. 249 Continuity ............................................................ 250 Differentiation and Integration ........................................... 251 Taylor Series .......................................................... 252 Exercises ............................................................. 255 111.8 Improper Integrals . ................................................... 257 Bounded Functions on Infinite Intervals ................................... 257 Unbounded Functions on a Finite Interval ................................. 260 Euler's Gamma Function ................................................ 261 Exercises ............................................................. 262 111.9 Two Theorems on Continuous Functions . ............................... 263 Continuous, but Nowhere Differentiable Functions ......................... 263 Weierstrass's Approximation Theorem .................................... 265 Exercises ............................................................. 269

Chapter IV Calculus in Several Variables IV.I Topology of n-Dimensional Space . ...................................... 273 Distances and Norms ................................................... 273 Convergence of Vector Sequences ........................................ 275 Neighborhoods, Open and Closed Sets .................................... 278 Compact Sets ......................................................... 283 Exercises ............................................................. 285 IV.2 Continuous Functions ................................................. 287 Continuous Functions and Compactness .................................. 289 Uniform Continuity and Uniform Convergence ............................ 290 Linear Mappings ....................................................... 293 Hausdorff's Characterization of Continuous Functions ...................... 294 Integrals with Parameters ............................................... 297 Exercises ............................................................. 298 IV.3 Differentiable Functions of Several Variables ............................ 300 Differentiability ....................................................... 302 Counter-Examples ..................................................... 304 A Geometrical Interpretation of the Gradient .............................. 305 The Mean Value Theorem ............................................... 308 The Implicit Function Theorem .......................................... 309 Differentiation of Integrals with Respect to Parameters ...................... 311 Exercises ............................................................. 313 IV.4 Higher Derivatives and Taylor Series . ................................... 316 Taylor Series for Two Variables .......................................... 319

x

Contents

Taylor Series for n Variables ............................................ 320 Maximum and Minimum Problems ....................................... 323 Conditional Minimum (Lagrange Multiplier) .............................. 325 Exercises ............................................................. 328 IV.S Multiple Integrals . .................................................... 330 Double Integrals over a Rectangle ........................................ 330 Null Sets and Discontinuous Functions ................................... 334 Arbitrary Bounded Domains ............................................ 336 The Transformation Formula for Double Integrals .......................... 338 Integrals with Unbounded Domain ....................................... 345 Exercises ............................................................. 347

Appendix: Original Quotations .............................................. 351 References ................................................................. 358 Symbol Index ............................................................... 369 Index ...................................................................... 371

I Introduction to Analysis of the Infinite

.. , our students of mathematics would profit much more from a study of Euler's Introductio in Analysin Infinitorum, rather than of the available modern textbooks. (Andre Weil 1979, quoted by J.D. Blanton 1988, p. xii) ... since the teacher was judicious enough to allow his unusual pupil (Jacobi) to occupy himself with Euler's Introductio, while the other pupils made great efforts. . . . (Dirichlet 1852, speech in commemoration of Jacobi, in Jacobi's Werke, vol. I, p.4)

This chapter explains the origin of elementary functions and the impact of Descartes's "Geometrie" on their calculation. The interpolation polynomial leads to Newton's binomial theorem and to the infinite series for exponential, logarithmic, and trigonometric functions. The chapter ends with a discussion of complex numbers, infinite products, and continued fractions. The presentation follows the historical development of this subject, with the mathematical rigor of the period. The justification of dubious conclusions will be an additional motivation for the rigorous treatment of convergence in Chapter III. Large parts of this chapter - as well as its title - were inspired by Euler's Introductio in Analysin Infinitorum (1748).

E. Hairer et al. (eds.), Analysis by Its History © Springer Science+Business Media New York 2008

2

I. Introduction to Analysis of the Infinite

1.1 Cartesian Coordinates and Polynomial Functions As long as Algebra and Geometry were separated, their progress was slow and their use limited; but once these sciences were united, they lent each other mutual support and advanced rapidly together towards perfection. We owe to Descartes the application of Algebra to Geometry; this has become the key to the greatest discoveries in all fields of mathematics. (Lagrange 1795, Oeuvres, vol. 7, p. 271)

Greek civilization produced the first great flowering of mathematical talent. Starting with Euclid's era ('" 300 B.C.), Alexandria became the world center of science. The city was devastated three times (in 47 B.C. by the Romans, in 392 by the Christians, and finally in 640 by the Moslems), and this led to the decline of this civilization. Following the improvement of Arabic writing (necessary for the Koran), Arab writers eagerly translated the surviving fragments of Greek works (Euclid, Aristotle, Plato, Archimedes, Apollonius, Ptolemy), as well as Indian arithmeticians, and started new research in mathematics. Finally, during the Crusades (1100-1300), the Europeans discovered this civilization; Gerard of Cremona (1114--1187), Robert of Chester (XIIth century), Leonardo da Pisa ("Fibonacci", around 1200) and Regiomontanus (1436-1476) were the main translators and the first scientists of Western Europe. At that time, mathematics were clearly separated: on one side algebra, on the other geometry.

Algebra Diophantus can be considered the inventor of Algebra; . . . (Lagrange 1795, Oeuvres, vol. 7, p. 219)

Algebra is a heritage from Greek and Oriental antiquity. The famous book Al-jabr w'al muqabala by Mohammed ben Musa Al-Khowarizmi1 (A.D. 830) starts by dealing with the solution of quadratic equations. The oldest known manuscript dates from 1342 and begins as follows: 2

1 2

The words "algebra" and "algorithm" originate from AI-jabr and AI-Khowarizml, respectively. This picture as well as Figs. 1.1 and 1.2 are reproduced with permission of the Bodleian Library, University of Oxford, Ms. Huntington 214, folios 1R, 4R and 4Y. English translation: F. Rosen (1831).

I.l Cartesian Coordinates and Polynomial Functions

3

AI-Khowarizmi's Examples. Consider the quadratic equation

x2

(1.1)

+ lOx = 39.

Such an equation hides the unknown solution x which is called by the arabs dshidr (root), a word that originally stood for the side of a square of a given surface ("A root is any quantity which is to be multiplied by itself", F. Rosen 1831, p. 6).

5.1

5.1

25

Manu cript of 1342

Modern Drawing

FIGURE 1.1. Solution of x 2 + lOx

= 39

Solution. AI-Khowarizmi sketches a square of side x to represent x 2 and two rectangles of sides 5 and x for the term lOx (see Fig. 1.1). Equation (1.1) shows that the shaded region of Fig. 1.1 is 39; consequently, the area of the whole square is 39 + 25 = 64 = 8 . 8, thus 5 + x = 8 and x = 3.

A

Manuscript of 1342

Modern Drawing

FIGURE 1.2. Solution of x 2 + 21 = lOx

With a second example (from AI-Khowfuizmi), (\.2)

x2

+ 21 = lOx

(or, if you prefer the Latin of Robert of Chester's translation: "Substancia vero et 21 dragmata 10 rebus equiparantur"), we demonstrate that different signs require different figures. To obtain its solution we sketch a square for x 2 and we attach a rectangle of width x and of unknown length for the 21 (Fig. 1.2). Because of (1.2), the total figure has length 10. It is split in the middle and the small rectangle (A) contained between x 2 and the bisecting line is placed on top (B). This gives a figure of height 5. The gray area is 21 and the complete square (gray and black) is

4

1. Introduction to Analysis of the Infinite

5·5 = 25. Consequently, the small black square must be 25 - 21 = 4 = 2·2 and we obtain x = 3. Using a similar drawing (you can have a try), AI-Khowarizmi also finds the second solution x = 7. Mohammed ben Musa AI-Khowarizmi describes his solution as follows (Rosen 1831, p.ll): ... for instance, "a square :ind twenty-one in numbers are equal to ten roots of the same square." That is to say, what must be the amount of a square, which, when twenty-one dirhems are added to it, becomes equal to the equivalent of ten roots of that square? Solution: Halve the number of the roots; the moiety is five. Multiply this by itself; the product is twenty-five. Subtract from this the twenty-one which are connected with the square; the remainder is four. Extract its root; it is two. Subtract this from the moiety of the roots, which is five; the remainder is three. This is the root of the square which you required, and the square is nine. Or you may add the root to the moiety of the roots; the sum is seven; this is the root of the square which you sought for, and the square itself is forty-nine.

As an application, AI-Khowarizmi solves the following puzzle: "I have divided 10 into two parts, and mUltiplying one of these by the other, the result was 21". Putting for one of the two parts x and the other 10 - x, and multiplying them, we obtain

0.3)

x· (10 - x) = 21

which is equivalent to (1.2). Hence, the solution is given by the two roots of Eq. (1.2), i.e., 3 and 7 or vice versa.

The Solution for Equations of Degree 3. Tartalea presented his solution in bad italian verse ... (Lagrange 1795, Oeuvres, vol. 7, p. 22) ... I have discovered the general rule, but for the moment I want to keep it secret for several reasons. (Tartaglia 1530, see M. Cantor 1891, vol. II, p. 485)

For example, let us try to solve (1.4)

x3

+ 6x = 20,

or, in "bad" italian verse, "Quando che'l cubo con Ie cose appresso, Se agguaglia numero discreto ... " (see M. Cantor 1891, vol. II, p.488). Nicolo Tartaglia (1499-1557) and Scipione dal Ferro (1465-1526) found independently the method for solving the problem, but they kept it secret in order to win competitions. Under pressure, and lured by false promises, Tartaglia divulged it to Gerolamo Cardano (1501-1576), veiled in verses and without derivation ("suppressa demonstratione"). Cardano reconstructed the derivation with great difficulty ("quod difficillimum fuit") and published it in his "Ars Magna" 1545 (see also di Pasquale 1957, and Struik 1969, p. 63-67).

a qualche

Derivation. We represent x 3 by a cube with edges of length x (what else?, gray in Fig. 1.3a); the term 6x is attached in the form of 3 square prisms of volume x 2 v and three of volume xv 2 (white in Fig. 1.3a). We obtain a body of volume 20 (by (1.4» which is the difference of a cube u 3 and a cube v 3 (see Fig. 1.3a), i.e.,

1. 1 Cartesian Coordinates and Polynomial Functions

FIGURE l.3a. Cubic equation (1.4)

5

FIGURE 1.3b. Justification of 0 .6)

razlbmatio.Exanplum.cubus &6 pof~ cionn,zquantur .zo, ducic:o z,tcniam par. um"ad cubum,6t8 .duc 10 wmidium nu lIItriin (f,St loo,iung~ 100 & 8,6t 108,aco puadican qUE ~a RZ 108,& ~am gtmtnina ~is, ahm addt'si 0, dimidium numm, a b a1uro minues tantun~~m , habebis B~no­ lllium At 108 p: 10,& Apotomen Rt J08 m: JO,horum accipe ~.. cub" & minu~ illam

rub' p:6rrb9f~lfs 20 10

Z

8 -10 1"8

~ 1e>(lP:IO ~108m:l/)

~ ~:CU.~

r08 p:'O

m:R.!v:cu.r~u08 m:r o _______

~------

FIGURE I.3c. Extract from Cardano, Ars Magna 1545, ed. Basilea 15703

where

u = x + v.

(1.5)

Arranging the six new prisms as in Fig. 1.3b, we see that their volume is equal to 6x (what is required) if (1.6)

3uvx

=

6x

or

uv = 2.

We now know the sum (= 20) and the product (= -8) of u 3 and -v 3 and can thus reconstruct these two numbers, as in AI-Khowarizmi's puzzle (l.3), as

_ v 3 = 10 - V108. Taking then cube roots and using x Fig.1.3c)

u - v we obtain (see the facsimile in

(1.7)

3

Reproduced with permission of Bib!. Pub!. Univ. Geneve.

6

1. Introduction to Analysis of the Infinite

Some years later a method of solving equations of degree 4 was found (Ludovico Ferrari, see Stroik 1969, p. 69f, and Exercises 1.1 and 1.2); the equation of degree 5 remained a mystery for centuries, until Abel's proof about the impossibility of solutions by radicals in 1826.

"Algebra Nova" The Numerical Logistic is the one displayed and treated by numbers; the Specific is displayed by kinds or forms of things: as by the letters of the Alphabet. (Viete 1600, Algebra nova, French edition 1630) ALGEBRA is a general Method of Computation by certain Signs and Symbols which have been contrived for this Purpose, and found convenient. (Maclaurin 1748, A Treatise of Algebra, p. 1)

The ancient texts dealt only with particular examples and their authors carried out "arithmetical" calculations using only numbers. Fran~ois Viete (= Franciscus Vieta 1540-1603, 1591 In artem analyticam isagoge, 1600 Algebra nova) had the fundamental idea of writing letters A, B, C, X, ... for the known and unknown quantities of a problem (often geometric) and to use these letters for algebraic calculations (see the facsimile in Fig. 1.4a). Since no problem of the Greek era appeared to resist the method ,----------; put letters ,-_ _ _---, calculations Algebraic Geometrical 1 1 Problem Problem

I Solution I

Viete wrote in capital letters "NVLLVM NON PROBLEMA SOLVERE" (i.e., "GIVING SOLVTION TO ANY PROBLEM"). The perfection of this idea led to Descartes's "Geometry".

E~''''pl,.

'lu'it Eaille ujouAer A+0, luee • + 2. OJ la (OIlUllC {era A B O. obfcrutDC cc qui a eM

.bt.

+ +s

B A

+ :LO.

+

D.

FIGURE 1.4a. Facsimile of the French edition (1630) of Viete (1600)4

4 Reproduced with permission of Bib!. Pub!. Univ. Geneve.

1.1 Cartesian Coordinates and Polynomial Functions

7

Sxquabuur Aquad. -+ B A, zquetur Z plano. A -+ B efto E. Igitur E quad. Z plano -+ B quad. I

2. ill

haque. ,/ lpLaoi .. n'IUJd. -

Confcfrarium. B tit A, de qua primum quzrebatur.

Iraque fi A cubus- B plano; in A, zquetur Z Colido lo. B pl....plu•. plu. T tI crczZifohhbJi,jj=_,,:-;;:~'::;:.b= ....:::lob4l:;=.::==.=pI;::::._='::;:I... =.PI::..:::::.... Eft. de qua quzritur.

c.

f! ZColidi -+ "zr.u_di

FIGURE lAb. Extracts ofViete (l591a)5(Opera p.129 and 150); Solution of A2 + 2BA = Z and A 3 - 3BA = 2Z

Example. (Trisection of an angle). The famous classical problem "Datum angulum in tres partes ::equales secare" becomes, with the help of

sin(3a) =3sinacos2 a-sin 3 a

(1.8)

(see (4.14) below) and of some simple calculations, the algebraic equation

-4X 3 +3X = B (see Viete 1593, Opera, p. 290). Its solution is obtained from (1.14) below. (1.9)

Formula for the Equation of Degree 2. In Viete's notation, the complicated text by AI-Khowfuizmi' (see p. 4) becomes the "formula"

x 2 +ax+b=0

(1.10)

Xl,x2=-a/2±Ja2/4-b.

===?

Formula for the Equation of Degree 3. (1.11)

y+ a/3 = x

+ ay2 + by + c = 0 ===> x 3 + px+q = O. u + v (this corresponds to (1.5) with "-v" replaced by "v"), so that

y3

We set x = Eq. (Ll1) becomes (1.12)

Putting uv = -p/3 (this corresponds to (1.6)), we obtain (1.13) By AI-Khowfuizmi' 's puzzle (1.3) and formulas (1.10), we get (see the facsimile in Fig. lAb), (1.14) 5

x =

V

-q/2 + J q2/4 + p3/27 +

V

-q/2 - J q2/4 + p3/27.

Reproduced with permission of BibL PubL Univ. Geneve. Here, the unknown variable is A Only with Descartes came into use the choice of x, y, z for unknowns.

8

1. Introduction to Analysis of the Infinite

Descartes's Geometry Here I beg you to observe in passing that the scruples that prevented ancient writers from using arithmetical terms in geometry, and which can only be a consequence of their inability to perceive clearly the relation between these two subjects, introduced much obscurity and confusion into their explanations. (Descartes 1637)

Geometry, the gigantic heritage of Greek antiquity, was brought to Europe thanks to the Arabic translations. For example, Euclid's Elements (around 300 B.e.) consist of 13 "Books" containing "Definitions", "Postulates", in all 465 "Propositions", that are rigorously proved. The Conics by Apollonius (200 B.C.) are of equal importance. Nevertheless, different unsolved problems eluded the efforts of these scientists: trisection of the angle, quadrature of the circle, and the problem mentioned by Pappus (in the year 350), which inspired Descartes's research. Problem by Pappus. ("The question, then, the solution of which was begun by Euclid and carried farther by Apollonius, but was completed by no one, is this"): Let three straight lines a, b, c and three angles 0:, {3, 'Y be given. For a point C, arbitrarily chosen, let B, D, F be points on a, b, c such that CB, CD, CF form with a, b, c the angles 0:, {3, 'Y, respectively (see Figs. 1.5a and 1.5b). We wish to find the locus of points C for which CB· CD = (CF)2.

(1.15)

Descartes solved this problem using Viete's "new" and prestigious algebra; the point C is determined by the distances AB and Be. These two "unknown values" are denoted by the letters "x" and "y" ("Que Ie segment de la ligne AB, qui est entre les points A & B, soit nomme x. & que BC soit nomme y".) For the moment, consider only two of these straight lines (Fig. l.5c) ("& pour me demesler de la cofusion de toutes ces lignes ... "). We draw the parallel to EF passing through e. All angles being given, we see that there are constants Kl and K2 such that u = K 1 · CF, v = K 2 • y. As AE = x (1.16)

+u +v = CF

K3 , we get

= d + ex + ky,

d, C, k constants.

Similarly, (1.17)

CD = mx+ny,

m, n constants.

("And thus you see that, ... the length of any such line ... can always be expressed by three terms, one of which consists of the unknown quantity y multiplied or divided by some known quantity; another consisting of the unknown quantity x multiplied or divided by some other known quantity; and the third consisting of a known quantity. An exception must be made in the case where the given lines are

1.1 Cartesian Coordinates and Polynomial Functions

9

FIGURE 1.5a. Problem by Pappus, sketch by Descartes6

F

FIGURE 1.5h. Problem by Pappus

FIGURE 1.5c. Equation of a straight line

parallel ... " Descartes 1637, p. 312, trans!. D.E.Smith and M.L.Latham 1925). Thus the condition (1.15) becomes

y . (mx

+ ny)

=

(d

+ ex + ky)2,

which is an equation of the form (1.18)

Ax2

+ Bxy + Cy2 + Dx + Ey + F

= O.

For each arbitrary y, (1.18) becomes a quadratic equation that is solved by algebra (see (1.10». Coordinate transformations show that (1.18) always represents a conic.

6

Fig. I.5a is reproduced with permission of Bib\. Pub\. Univ. Geneve.

10

I. Introduction to Analysis of the Infinite

Polynomial Functions Algebra not only helps geometry, but geometry also helps algebra, because the cartesian coordinates show algebra in a new light. In fact, if instead of (1.1) and (1.4) we consider (1.19)

Y=

x2

+ lOx -

Y = x3

39,

+ 6x -

20

and if we attribute arbitrary values to x, then for each x we can compute a value for y and can study the curves obtained in this way (Fig. 1.6). The roots of (1.1) or (1.4) appear as the points of intersection of these curves with the x-axis (horizontal axis). For example, we discover that the solution of (1.4) is simply x = 2 (a bit nicer than Eq. (1.7)).

FIGURE 1.6. Polynomials x 2 + lOx - 39 and x 3 + 6x - 20

(1.1) Definition. A polynomial is an expression of the form

where ao, al,···, an are arbitrary constants. degree n.

If an =I 0, the polynomial is of

Interpolation Problem. Given n + 1 points Xi, Yi (see Fig. 1.7), we look for a polynomial of degree n passing through all these points. We are mainly interested in the situation where the Xi are equidistant, and in particular where Xo = 0,

Xl

= 1,

X2

= 2,

X3

= 3, ... .

The solution of this problem, which was very useful in the computation of logarithms and maritime navigation, emerged in the early 17th century from the work of Briggs and Sir Thomas Harriot (see Goldstine 1977, p.23f). Newton (1676) attacked the problem in the spirit of Viete's "algebra nova" (see Fig. 1.8): write letters for the unknown coefficients of our polynomial, e.g., (1.20)

y=A+Bx+Cx 2 +Dx 3 .

l. I Cartesian Coordinates and Polynomial Functions

II

2

FIGURE 1.7. Interpolation polynomial AblcliJi

A+p A+9 ..1+,

.A+~

i'

Ordin3tZ

A+Z+ 'P' + + tp.= .. .4+ 9"1"cq'+ ql+tq.=~ ...1 + h, + CJ" + J,I + at =,. ..I + h. + C~· + III I + ". = J' ..1+ ht+ ct· + Jtl + tit.

=.

.

A+t Divifor. Di!". Ord . Quod per divifioncm prodcuntes. [I-q) .. -II b +o 0 (3.1)

1

[~(x . y) = C(x) ~ C(y)]

Reproduced with permission of Bibl. Publ. Univ. Geneve.

30

I. Introduction to Analysis of the Infinite

If we set first y

= z / x and then x = y =

1 in (3.1), we obtain

£(z/x) = £(z) - £(x),

(3.2) (3.3)

o.

£(1) =

Applying (3.1) twice to x . y. z

= (x· y) . z gives

£(x . y. z) = £(x)

(3.4)

+ £(y) + £(z),

and similarly for products with four or more terms. Next, applying (3.4) to ~ . ~. ~ = x, we obtain £(~) = ~£(x), or in general (3.5)

m

£(x n ) = - £(x), n Tn

where

x~

= :cfxID.

Bases. Let a fixed logarithmic function £(x) be given and suppose that there exists a number a for which £( a)

=

1. Then, (3.5) becomes

'"

m

£(a n ) =-, n

(3.6)

i.e., the logarithmic function is the inverse function for the exponential function aX. We call this the logarithm to the base a and write (3.7)

y = 10ga x

if

x = aY •

Logarithms to the base 10 (Briggs' logarithms) are the most suitable for numerical computations, since a shift of the decimal point just adds an integer to the logarithm. The best base for theoretical work, as we soon shall see, is Euler's number e (natural or Naperian or hyperbolic logarithms). These logarithms are usually denoted by In x or log x.

Euler's "Golden Rule". If the logarithms for one base are known, the logarithms for all other bases are obtained by a simple division. To see this, take the logarithm to the base b of x = aY and use (3.7) and (3.5). This yields (3.8)

10gb X = Y . 10gb a

Computation of Logarithms By computing the square root of the base a, then the square root of the square root, and so on, and by multiplying all these values, we obtain, with the help of (3.6) and (3.1), the logarithms of many numbers. This is illustrated for a = 10 in Fig. 3.2.

1.3 Logarithms and Areas 1.00 Logarithms

Numbers

I. 0.875 0.75 0.625 0.5 0.375 0.25 0.125

10.0000 7.4989 5.6234 4.2170 3.1623 2.3714 1.7783 1.3335 1.0000

. .-".

.75

;

..~

..... ..., .

.•. ...

31

..

..•..' ..

.SO t---

.25

o.

.00

10

FIGURE 3.2. Successive roots of lO and their products

There remains a problem: we would prefer to know the logarithms of such numbers as 2, 3, 4, .. . and not of 4.2170 or 2.3714.

Briggs' Method. Compute the root of 10, then the root of the root, and continue doing so 54 times (see facsimile in Fig. 3.3). This gives, with c = 1/2 54 , (3.9a)

= 1.00000000000000012781914932003235 = 1 + a.

lOc

Then, compute in the same way the successive roots of 2: (3.9b)

2c

The value x

= 1.00000000000000003847739796558310 = 1 + b.

= loglO 2 we are searching for satisfies 2 = lOX. Hence,

1 + b (3;2b) 2c = (10C)X (3;2a) (1

+ a)X (Theo~m 2.2) 1 + ax

and we obtain (

3.10

)

10

glO

(2)

= ~~= x

a

3847739796558310

12781914932003235

~ 0.3010299956638812.

This gives us one value. The amount of work necessary for the whole table is hardly imaginable.

Interpolation. Interpolation was an important tool for speeding up the computation of logarithms in ancient times. Say, for example, that four values of loglO have been computed. We compute the difference scheme log( 44) = 1.6434526765 10g(45)

= 1.6532125138

0.0097598373 -0.0002145194 0.0095453179

log( 46) = 1.6627578317 0.0093400262 log( 47) = 1.6720978579

0.0000092277. -0.0002052917

32

I. Introduction to Analysis of the Infinite D

N-mcwuilt., M,J; M'It'D_ _ d' ".,,11111;' I 0

',000

; .61~.77660,16837.9j319,?889M.07'.~S~O::"'------"""------.7781'79+10.03891.,sOI ••9na....3 0,1$ IH ~$,SI+31•• 6 B'r'fOa'f6,6nfl9J3C8 O,UI 111+1.1198+.689ir,817¢.6191S,IlJ 0,0611 107+6,o1h8.3'1 31,7497J.1311'7,6, ;8 0.03 12,

10;66,319111,+3769,79971,90617,3'3'

O.p'ltJ·,~$:----------------

tOI8 •• , 1711 ,71818,.8.t'4Jn713,8I'" .oogo.310OO'SO~,"i"

ooooo,ooooo,ooooo,88817.... ~OOIII.S313J:It9Dj' lOOo,ooooo.ooooo.JO~:f,~3194>S60'S,9U L 0,00000,00000,ooooo.......08.9'09I.S0062,6.6.6,514Y" /000,00000.00000,°1 J I 1.16 '97,:&801'.94'U J rooO',oooco.ooooO,01.H6,38'l911.6+oo6>479 .00000,ooooo,ooooo,I'101,'30S4.61 1 I $.6s u

I-

.

0-""""., ... '"., .

· ...,O-"H~.r ...... ----------,..~

FIGURE 4.9. Autographic table of tan Q by Regiomontanus (see Kaunzner 1980)3

A very precise computation of sin 1° was made by AI-Kashl (Samarkand in 1429) by solving numerically the equation (see Eq. (1.9»

- 4x 3

(4.20)

+ 3x =

sin 3°

with the help of an iterative method and giving the solution in base 60 ("We extracted it by inspired strength from the Eternal Presence .. .", see A. Aaboe 1954) sin 10

= 0; 1, 2, 49, 43 , 11 , 14, 44, 16, 19, 16 ....

Here is the true value in base 60 calculated by a modern computer, sin 10 = 0; 1, 2,49, 43, 11 , 14, 44, 16, 26, 18, 28, 49, 20, 26,50, 41 , . . . 3

Reproduced with permission of Niirnberger Stadtbibliothek, Cent V, 63, f. 30r .

1.4 Trigonometric Functions 49 Once again, we see the enormous progress of the series method (4.l7), which gives sin 10 = sin( 7r / 180) = sin(0.0174532925 ... ) with only three terms as sin 10 ::::: 0 .0174532925199 - 0.0000008860962 + 0.000000000013496 ::::: 0.0174524064373.

Inverse Trigonometric Functions Trigonometric functions define sin x, cos x , tan x , for a given arc x . Inverse trigonometric functions define the arc x as a function of sin x , cos x, or t a n x. (4.3) Definition. Consider a right-angled triangle with hypotenuse 1. If x de notes the length of the leg opposite the angle, arcsin x is the length of the arc (see Fig. 4. JOa). The values arccos x and arcta n x are defined analogously (Figs. 4.10b and 4.JOc).

FIGURE 4.10. Definition of arcsin x , arccos x , and arctan x

Because of the periodicity of the trigonometric functions, the inverse trigonometric functions are muItivalued. The so-called principal branches satisfy the following inequalities:

y = arcsin x

¢o}

x = sin y

for -1:::; x:::; l, -7r/ 2 :::; y :::; 7r/ 2,

y = a rccos x

¢o}

x = cos y

for - 1 :::; x :::; 1, 0 :::; y :::; 7r ,

y = a rctan x

¢o}

x = t any

for -

00

< x < 00 ,

-7r/ 2

3. Fig. 5.3 represents the map Z f-+ W = z3 for varying values of z and its inverse function W f-+ Z = w1 / 3 = ijW. The animal that thereby undergoes painful deformations is known as "Arnold's cat". The inverse map produces three cats out of one.

w

FIGURE 5.3. The function w =

Z3

=z

and its inverse z = w l / 3

Exponential Function and Logarithm. The exponential function can be extended to complex arguments as follows:

(5.10)

for

c

= a + ib.

This definition retains the fundamental property e C +w = e C • e W , which is obtained from Eq. (5.7). The nature of the logarithms of negative numbers gave rise to long and heated disputes between Leibniz and Joh. Bernoulli. Euler (1751) gave a marvelous survey of these discussions, which were kept as secret as possible since such disputes

1.5 Complex Numbers and Functions

61

would have damaged the prestige of pure mathematics as an exact and rigorous science. The true nature of logarithms of negative and complex numbers was then revealed by Euler ("Denouement des difficultes precedentes") with the help, once again, ofEq. (5.4). Many of the contradictions of the earlier disputes were resolved by the fact that the logarithm of a complex number does not represent one number, but an infinity of values. We write c in polar coordinate form k

= 0, ±1 , ±2, ... ,

which is a product. In order to retain properties (3.1) and (3.7) for the logarithm with complex arguments, we define (5.11)

In(c) = In(r)

+ i(

:::::

'1 ' 7 '

2

FIGURE 6.1. Errors for convergents Ak / Bk (logarithmic scale)

1.6 Continued Fractions

73

The approximations for v2 and v'3 were known in antiquity (Archimedes used 265/ 153 < v'3 < 1351/ 780 without further comment). The two convergents 22/ 7 (Archimedes) and 355/ 113 (Tsu Chung-chih around 480 in China, Adrianus Metius 1571-1635 in Europe) for 7r are of a better than average quality. Explanation: the first denominator qk+l to be neglected is large (15, respectively, 292). Two other very precise approximations for 7r, which are the 11th and 26th convergents respectively, have been calculated 1766 in Japan by Y. Arima as 5419351 / 1725033 and 428224593349304/ 136308121570117 (see Hayashi 1902). On the other hand, for the golden mean (all qk = 1) we have slow convergence. Here, (6.11) becomes the recursion formula for the Fibonacci numbers (Leonardo da Pisa 1170-1250, also called Fibonacci). Some convergents of the continued fraction (6.6) for tan x, (6.13) x 3x 15x - x 3 105x - lOx 3 945x - 105x 3 + x 5 "1 ' 3 - x 2 ' 15 - 6x 2 ' 105 - 45x 2 + X4 ' 945 - 420X2 + 15x4 , . . . are displayed in Fig. 6.2 and nicely approach the function tan x , even beyond the singularities x = 7r / 2, 37r / 2, . ... k=2 5 1-

t! 5 1-

1

1

-:J

IU

k=5

k=4

k=3 5 1-

5

5

[7IlL ~. ~ 5 10 5 1-

FIGURE 6.2. Convergents of the continued fraction for tan x

Infinite Series from Continued Fractions. The difference of two successive convergents satisfies (6.14)

A k+1 _ Ak = Ak+1 Bk - AkBk+1 = (_I)k . PIP2 . . ... Pk+l . Bk BkBk+l B kB k+l Bk+l

The last identity is seen as follows: using (6.11) we have

Ak+l B k - AkBk+l = (qk+1 A k + Pk+ lAk - l)Bk - Ak(qk+lBk = - Pk+1 (A kB k- l - Ak- l B k) = . . .

+ Pk+lBk- d

= P2 ····· pk+l (- I)k(A l B a - AaBd

and (AlBa - AaBd = PI because of (6.12). Writing the convergent A k / Bk as

74

I. Introduction to Analysis of the Infinite

Ak = (Ak _ A k - l ) Bk Bk Bk-l

+ (Ak-l

Bk-l

_ A k- 2 ) Bk-2

+ ... + (Al Bl

_ AO) + A o , Bo

Bo

we see from (6.14) that (6.15)

Ak Bk

= qo + ~ _ Bl

PlP2 BlB2

+ PlP2P3

_ ...

B2B3

+ (_I)k-l . PlP2····· Pk Bk-lBk

and we have (6.3) Theorem. The convergents 0/(6.7) are the truncated sums o/the series (6.16)

For regular continued fractions (all Pk = 1) we have (6.16')

1

qo

+ Bl

1 - BlB2

1

+ B2B3

1 - B3 B 4

+ ...

Since 1/ (Bk-l B k) is the smallest possible distance between two different rational numbers with denominators Bk-l and Bk, the interval between Ak-d Bk-l and A k / Bk cannot contain a rational number whose denominator is not larger than Bk • Continued Fractions from Infinite Series. Let (6.17)

11111

---+---+--+ ... Cl C2 C3 C4 C5

be a given series with integer Ci; we want to find integers Pi, qi such that the series (6.17) coincides term by term with (6.16) (with qo = 0). Solution. We put Pl = 1 and ql = Bl = Cl. Then, we divide two successive terms of (6.16) (so that the products of Pi simplify), which gives (6.18)

This resembles, apart from the factors Ck-l and Ck, the Eq. (6.11). We therefore subtract from (6.18) Eq. (6.11), once multiplied by Ck-l, once by Ck, and obtain ck-lqkBk-l = (Ck - Ck-l)Pk B k-2 (Ck-l -

ck)Bk

=

In the first formula we replace k by k This eliminates the Bk'S and gives (6.19)

-ckqkBk-l·

+ 1 and then divide the two expressions.

1.6 Continued Fractions

75

The Pi, qi are, of course, not uniquely defined. Since we want them to be integers, a natural choice that satisfies (6.19) is

(6.20)

I Pk+1 =

c%,

for k ~ 1. Thus, we have the following formula of Euler (1748, §369): (6.21)

-1 - -1+1 - -1 - + ... = Cl

C2

C3

C4

1

--...".------------

ci

Cl+~---~---------

C2 -

Cl

c~ + ----~---c~ C3 - C2 + -"----C4 -

C3

+ ...

When applied to two well-known series (see Sects. 1.3 and 1.4), this formula gives 1 1 1 1 In 2 = 1 - - + - - - + ... = ----,--------2 3 4 1 1 + --4:-----1+----(6.22) 1 + _9_ __

1+~ 1 + ...

7r

4" (6.23)

1 1 1 1 = 1-"3 + 5" -7+'" = -1---=-1-----+ 9 2+ 25 2+---:-::--

2+~

2+ ... The second continued fraction is the one found by Lord Brouncker, obtained here from Leibniz's series. Similarily, we prove (Euler 1748, §370)

(6.24)

11 1 -1 - -+ - - - ... = ------------Cl CIC2 CIC2 C3 Cl Cl + - - - - - - - - - - - C2 C2 - 1 + - - - - - - - C3

c3- 1 + - - - c4- 1 + ... whence, for example,

(6.25)

I-!= 1- _1_+_1_ _ ... = _1_ _ _ _ _ __ e 1·2 1·2·3 1 1+-----1+ 2 3 2+

4 3+-4+ ...

76

I. Introduction to Analysis of the Infinite

Irrationality I have good reason to doubt that the present article will be read, or even understood, by those who should profit most by it, namely those who spend time and efforts in trying to square the circle. There will always be enough such persons ... who understand very little of geometry ... (Lambert 1770a)

One of the great unsolved problems of classical analysis was the quadrature of the circle (i.e., the construction of 1f) by ruler and compass. Lambert was one of the first to believe that this construction, which challenged mathematicians for 2000 years, was impossible. A first hint toward this result would be the fact that 1f is irrational. We are therefore interested in a theorem that states that an infinite continued fraction represents an irrational number.

First difficulty. It can happen that a continued fraction represents no number at all. To see this, we start from the series

234

5

6

7

---+---+---+ ... 1 2 345 6

(6.26)

Since its terms approach ±1, it clearly does not converge. To obtain a corresponding continued fraction, we put Ck = k / (k + 1) (see (6.17)) and obtain from (6.19), after simplification, Pk+1 = k 3 (k + 2). qk+1 . qk

With Pk+1 = + 2) and qk = 1 we have integer coefficients and see that the convergents of the continued fraction

k3 (k

(6.27)

2 1+

3 1+

32 135

1+-1+ ...

do not tend to a real number. Second difficulty. There are infinite continued fractions that represent a rational number. For example, we have 2 = 1 + 2/2 and obtain, by inserting 2 repeatedly, (6.28)

2 2 = 1 + --2=----1+---:2~-

1+-1+ .. , which is rational. (6.4) Theorem.lfthe Pj and qj are integers and iffrom a certain index) onward

(6.29)

then the continuedfraction (6.7) tends to a number a that is irrational.

~

)0

1.6 Continued Fractions

77

Proof Without loss of generality we may assume that 0 < Pj :::; qj is satisfied for all j. Otherwise, we consider the continued fraction starting with Pja / qja' Convergence of this continued fraction and its irrationality are equivalent to convergence and irrationality of the original one. The assumption that 0 < Pj :::; qj guarantees that the convergents of the continued fraction tend to a real number. This is a consequence of the "Leibniz criterion" and will be discussed in Sect. 111.2. Following an idea of Legendre (1794, Elements de Geometrie, Note IV), we now write the continued fraction (6.7) without qo as

(6.30)

(3=P_2_ _ __

with

_3_ _ q2+ P q3 + ... Since ql 2: PI and (3 > 0 we have a < 1. Suppose now that a = B / A is rational with 0 < B < A. A simple reformulation of (6.30) yields

(3

= PI

- ql a

= API -

Bql

B'

a

so that (3 is expressed as a rational number with denominator smaller than that of a. If we repeat the same reasoning with (3 = P2/(q2 + ')') and so on, we find smaller and smaller denominators that are all integers. This is not possible an 0 infinite number of times. Negative Pj' The conclusion of Theorem 6.4 is also valid, if (6.29) is replaced by (6.29') This is seen by repeated application of the identity (valid for Pj

(qj-I-l)+

1

< 0)

1.1

1 + -,-P..::..J.:.......,---,-_ _ qj -Ipjl + (3

which, under the assumption (6.29'), transforms the continued fraction into another one satisfying (6.29). (6.5) Theorem (Lambert 1768, 1770a, Legendre 1794). For each rational x (x 0) the value tan x is irrational.

min is rational and insert this into (6.6): m min = ----'---:2'/---;;2:-----

Proof Suppose that x =

(6.31)

m tan -

n

m n 1 - ----",....---;;:---

3-

m /n m2 /n 2 5--2

2

----;c-~

7- ...

m2 n - -----:"...._---m2 3n - -----,:--m2 5n---7n- ...

"I-

78

1. Introduction to Analysis of the Infinite

On the right we have a continued fraction with integer coefficients. Since the factors 1, 3, 5, 7, 9, ... approach infinity, condition (6.29') is, for all m and n, satisfied beyond a certain index i o. 0 The same result is true for the arctan function; indeed, for y rational, x = arctan y must be irrational, otherwise y = tan x would be irrational by Theorem 6.5. In particular, Jr = 4 arctan 1 must be irrational. The proof of the analogous result for the hyperbolic tangent tanh x = (e - e-X)/(e X + e- X) = (e 2x - 1)/(e 2X + 1) is even easier, since all minus signs in (6.31) become plus signs. Inverting the last formula, we have eX = (1 + tanh(x /2)) / (1 - tanh(x /2)), and still obtain the irrationality of eX and In x for rational x f- 0 and x f- 1, respectively.

Exercises 6.1 Show that with the use of matrix notation, the numerators and denominators Ak and Bk of the convergents (6.8) can be expressed in the following form:

( Ak Ak-I) Bk Bk-I

=

(qO

1) (ql 1)

1 0

PI 0

1) ...

(q2 P2 0

1)

(qk-I Pk-I 0

1).

(qk Pk 0

6.2 Compute numerically the regular continued fractions for the numbers

h, J3, V5, v6, v7, and discover a significant difference between the square and the cube roots. 6.3 Show that and a+----a+ ----,--1 a+-a+ ...

a+------b + _1_,---_ __ a+----,---b+ _1__ a+ ...

are solutions of a second-degree equation. Compute their values. 6.4 The length of an astronomical year is (Euler 1748, §382)

365 days 5 hours 48'55/1. Compute the development of 5 hours 48'55/1 (measured in days) into a regular continued fraction and compute the corresponding convergents. Don't forget to give your valuable advice to Pope Gregory XIII for the reform of his calendar.

6.5 Give a detailed proof ofEq. (6.24).

1.6 Continued Fractions

79

6.6 Prove formula (6.6) . Hint (Legendre 1794). Define

0 !'(xo) = 0 and f"(xo) < 0

=}

Xo is a local minimum,

=}

Xo is a local maximum.

These facts "sequentibus exemplis illustrabimus": Example 1. We choose

y = x3 (2.2)

x2

-

y' = 3x 2

-

3x,

-

2x - 3,

y

y" = 6x - 2. The function can be seen to increase where y' > 0, i.e., for x < (1 - /10)/3 and for x > (1 + /10)/3. It is convex downward for x> 1/3 and convex upward for x < 1/3. The point x = 1/3 is an inflection point. The point x = (1 - /10)/3 is a local (but not global) maximum, the point x = (1 + /10)/3 is a local minimum.

3

Example 2. We consider the function (see Euler 1755, Pars Posterior, §265) (2.3)

,

1- x 2

Y = (1

+ x2)2 '

" -6x + 2x 3 y = (1 + x 2 )3 ,

which, together with its first and second derivative, is plotted in Fig. 2.2. The function y(x) possesses a (global) minimum for x = -I, a (global) maximum for x = I, and inflection points at x = 0 and x = ±.J3. It is convex downward on the intervals - .J3 < x < 0 and J3 < x < 00 and convex upward elsewhere.

II.2 Higher Derivatives and Taylor Series

Max

In

~

Inft

.-

·· ··

..

- ]

93

.

:

...

.:

•

..

/I •

.. y.~

....

FIGURE 2.2. Maxima, minima, inflection points of Euler's example

Fermat's Principle. A

FIGURE 2.3. Drawing by Joh. Bernoulli 16911922

B FIGURE 2.4. Fermat's principle

Fermat wishes to explain the law of Snellius for the refraction of light between two media in which the velocities are V1 and V2, respectively. Let two points A, B (see Fig. 2.4) be given. Find angles 0:1 and 0:2 such that light travels from A to B in minimal time or with minimal resistance. This means, find x such that (2.4) Fermat himself found the problem too difficult for an analytical treatment ("I admit that this problem is not one of the easiest"). The computations were then proudly performed by Leibniz (1684) "in tribus lineis". The derivative of T as a function of x is 2

Reproduced with permission of Univ. Bib!. Base!.

94

II. Differential and Integral Calculus

T' = ~ VI

2x + -2(£ - x) ~. 2v'a 2 + X2 2Jb 2 + (£ - x)2 V2

Observing that sin al = xl v' a2 + x 2 and sin a2 see that this derivative vanishes whenever

= (£ -

x)1 Jb 2 + (£ - X)2, we

(2.5) (law of Snellius). The computation of Til, 1 T il = VI

a2

(a 2 + x 2)3/2

b2

1

+V2 (b 2 + (£ -

x)2)3/2

>0

'

shows that our result is really a minimum.

De Conversione Functionum in Series Taylor's Approach. We have here, in fact, a passage to the limit of unexampled audacity. (F. Klein 1908, Eng!. ed., p. 233)

We consider (Taylor 1715) for a function f (x) the points xo, Xl = Xo + L1x, x2 = Xo + 2L1x, ... and the function values Yo = f(xo), YI = f(xd, Y2 = f(X2), ....

f(x)

C /

Xo

,........- ~)

Ll.t ~ O.4 Xl

Xz

FIGURE 2.5. Creation of the Taylor polynomial

Then we compute the interpolation polynomial passing through these points (see Fig. 2.5 and Theorem 1.1.2; for the latter we define x = Xo + tL1x , t = X';;;O) (2.6)

( ) _

P x - Yo

+

x - Xo L1yo 1 L1x

+

(x - xo)(x - Xl) L12yO 1.2 L1x2 '

or with more such terms for higher degrees. If we let L1x ---) 0, Xl ---) xo, X2 ---) Xo (or, as we said: if we take L1x infinitely small), the quotient L1yo I L1x in the second term tends to f'(xo). Further, the product (x - xo)(x - xd, which appears in the third term, will tend to (x - xO)2. It was then postulated by Taylor that the second differences (divided by L1x 2) will tend to the second derivative (see Exercises 2.5 and 111.6.4); in general, (2.7)

11.2 Higher Derivatives and Taylor Series

95

If we consider in the interpolation polynomial (2.6) more and more terms, and, at the same time, take the limit as Llx ~ 0, we obtain the famous formula (2.8)

'( ) (X_XO)21/( ) (X_XO)3 111 ( ) f () x = f (Xo) + (x - Xo ) f Xo + I f Xo + 1 f .TO 2.

3.

+... .

All the series of the first chapter are special cases of this "series universalissima". For example, the function f(x) = In(l + x) has the derivatives

f(O)

= 0,

1'(0) = I,

and we obtain

In(l

+ x)

= x

f(k)(O)

x2

=

x3

(_l)k-l(k - I)! X4

-:2 +"3 - 4 ± ....

Remarks. Formula (2.8) was believed to be generally true for more than a century. Cauchy then found an example of a function for which the series (2.8) converges, but not to f(x) (see Sect. III.7). There are also examples of functions for which the series (2.8) does not converge at all for x f=. Xo (see Exercise III.7.6). A more satisfactory proof of (2.8) (due to Joh. Bernoulli) uses integral calculus and will be given in Sect. IIA. Maclaurin's Approach (Maclaurin 1742, p. 223-224, art. 255). For the function Y = f (x) and a given point Xo we look for a series (or polynomial)

p(x)

(2.9)

=

Po

+ (x - xo)qO + (x - xo)2ro + (x - xO)3 s0 + ... ,

for which i

(2.10)

= 0, I, 2, . .. ,

i.e., both functions have the same derivatives up to a certain order at x = Xo. Setting x = Xo in (2.9) yields Po = p(xo) = f(xo) by (2.10). We then differentiate (2.9), again set x = Xo, and obtain qo = p' (xo) = I' (xo). Further differentiations give 2!ro = fl/(xo), 3!so = f"'(xo), and so on. Therefore, the series (2.9) is identical to that of (2.8). Partial sums of the series (2.8) are called Taylor polynomials.

Example. For the function given in (2.2) we choose the point Xo = 1 and have f(xo) = -3, I'(xo) = -2, f"(xo) = 4, and f"'(xo) = 6. Thus, the Taylor polynomials of degree 1, 2, and 3 become

Pl(X)

=

I

I

_-'-_~I/-IJ

-3 - 2(x - 1) = -2x - I,

P2(X) = Pl(X) P3(X) = P2(X)

+ ~(x + ~(x -

1)2 = 2X2 - 6x

1)3

= x3 -

I

+ I,

x 2 - 3x.

/

-4

/

I

I

I

I

IP2

13

96

II. Differential and Integral Calculus

Newton's Method for Roots of Equations. The Taylor polynomials are an extremely useful tool for the approximate computation of roots. We consider the example treated by Newton (1671),

x3

(2.11)

-

2x - 5 = O.

Trying out a few values ofthe function f(x) = x 3 - 2x - 5, we find f(O) = -5, f(l) = -6, f(2) = -1, f(3) = 16. Hence, there is a root close to Xo = 2. The idea is now to replace the curve f(x) by its tangent line at the point xo, which is PI (x) = -1 + lO(x - 2). The root of pdx) = 0, which is x = 2.1, is then an improved approximation to the root of (2.11). We now choose Xo = 2.1 and repeat the calculation. This gives PI (x) = 0.061 + 11.23(x - 2.1) and x = 2.0945681 as new approximation of the root of (2.11). A further step yields x = 2.0945515, where all digits shown are correct (see in Fig. 2.6 a facsimile of the calculation done by Newton).

1'-21-'1=0 2+1""'1

1-0,00004814 +

+1 ' +8+ 121+61'+1' ooIo2J 1-4-21' - f 1-1 Summa - 1 + JCfI + 61' +1' + 0,001'" o,03f+ 0,39' +.' + 1,2 + 6,0 +61' ++ 0,06 1, ... 10, + 101 - I 1-1 SUmma + 0,061 + IJ,23f'" 6,3" + fl "'::j:7,W + 0,000183708 0,068004"+6,3' +11,239 - 0,06064& + 11,2; siio,061 + 0,061 --.!!!!!.. +0,0001417°8+11,16196,+6,3'

,--:r;;

0,1+9

-.0014+ r

\ + 2,10000000 -o,oo!148n + 2,09411 147 =1

=.

I

=

r

FIGURE2.6. Newton's calculation for x 3

-

2x - 5 = 03

Use of the second degree polynomial (E.Halley 1694). We choose for the above example the point Xo = 2.1 and use two terms of the Taylor polynomial. This gives 0.061 + 11.23(x - 2.1) + 6.3(x - 2.1)2 = 0, a quadratic equation in z = x - 2.1, which has two roots. We choose the one that is smaller in absolute value (i.e., for which x is closer to 2.1) and obtain z 3

=x

- 2.1

=

-11.23 + \",11.23 2 - 4 . 0.061 ·6.3 12.6 '

Reproduced with permission of Bib!. Pub!. Univ. Geneve.

11.2 Higher Derivatives and Taylor Series

97

hence, x = 2.0945515. Again, all digits shown are correct, obtained this time with only one iteration.

Exercises 2.1 (Euler 1755, §26l). Study the functions

y = x4 - 8x 3 + 22x2 - 24x + 12, Find maxima, minima, convex downward regions, inflection points. 2.2 (Euler 1755, §272). The sequence of numbers

Vi = 1, V2 = 1.4142, V3 = 1.4422, V4 = 1.4142, Vs = 1.3797, ... suggests that the function y = {IX = xlix possesses a maximum value close to x = 3. Where exactly? In which relation is this value with the minimum value of y = XX? 2.3 (Joh. Bernoulli 1691/92). Find x such that the rectangle formed by the abscissa and the ordinate for a point on the circle y = v'x - x 2 has maximal area. Verify the maximality by computing the second derivative.

o

x

2.4 (Euler 1755, §272). Find x such that x sin x possesses a (local) maximum (you will find an equation that is best solved by Newton's or Halley's method; Euler gives the result x = 116°14'21//20111 35////47//'//; the correct value of the last digits is 32////38//'//). 2.5 Compute for the function y = x 3 the second difference

i1 2y

=

(x

+ 2i1x)3 - 2(x + i1X)3 + x 3.

Show that this difference, divided by i1x 2 , tends, for i1x second derivative.

-t

0, to 6x, the

2.6 Let f(x) = sin(x 2 ). Compute f'(x), f"(x), f"'(x), f"//(x), ... to obtain the series of Taylor 2

3

4

f(x)=f(O)+!,(O)x+f"(O)~! +f"'(O)~! +f"//(O):! + .. . Is there a much better way of obtaining this result? 2.7 Show that Newton's method, applied to x 2 - 2 = 0, is identical to (1.2.13), the Babylonian computation of v'2. However, formula (1.2.14) is different from Halley 's method. Why? 2.8 (Leibniz 1710). For a function y(x) = u(x) . v(x) show, by extending (1.4), that

y// = u//v + 2u'v' Find a general rule for

+ uv" , y(n).

ylll

= u lll V

+ 3u// v' + 3u'v// + uv

lll .

98

II. Differential and Integral Calculus

II.3 Envelopes and Curvature My Brother, Professor at Baste, has taken this opportunity to investigate several curves that Nature sets before our eyes every day . .. (Joh. Bernoulli 1692) I am quite convinced that there is hardly a geometer in the world who can be compared to you. (de L'HospitaI1695, letter to Joh. Bernoulli)

Envelope of a Family of Straight Lines Inspired by a drawing of A. Diirer (1525, p. 38, see Fig. 3.1, right), we consider a point (a,O) moving on the x-axis and the point (0, 13 - a) moving on the y-axis in opposite direction. If we connect these points by a straight line (3.1)

a - 13 13x y = - - (x - a) = 13 + x - a - a

a

we obtain an infinity of lines which are displayed in Fig. 3.1, and which create an interesting curve, called the envelope, which is tangent to each of these lines. The problem is to compute this curve. This kind of problems was extensively discussed between Leibniz (see Leibniz 1694a), Joh. Bernoulli and de L' Hospital.

FIGURE 3.1. Family of straight lines forming a parabola and a sketch by DUrer (1525)'

Idea. We fix the variable x to an arbitrary value, say, x = 4, for which the family (3.1) becomes y = 17 - a - 52/ a. We then observe that this value first increases for increasing a (see Fig. 3.1; for a = 3,4,5, 6 we have y = -3.33,0, 1.6, 2.33 respectively). During this time the point (4, y) approaches the envelope. The envelope is finally reached precisely when this function attains its maximum value, , Reproduced with permission of Verlag Dr. Alfons Uhl, Nordlingen.

II.3 Envelopes and Curvature

99

whence where the derivative y' = -1 + 52 / a2 = 0, i.e., for a = V52. This value is y = 17 - 2V52 = 2.58. The same idea works for any value of x: we have to compute the derivative of (3.1) with respect to a by considering x as a constant ("differentiare secundum a" ). This is called the partial derivative with respect to a. At points of the envelope this derivative must vanish. Today we denote this as (see Sect. IV.3 below, see also Jacobi 1827, Oeuvres, vol. 3, p. 65)

oy oa

(3 .2)

=0

.

For Eq.(3.l) this becomes oy / oa = - 1 + 13x/ a2 and condition (3 .2) gives a = V13 x. We obtain the envelope by inserting this into (3 .1), (3.3)

y

= x - 2V13x + 13

or

(y - x - 13)2 = 52x.

(3.4)

This is the equation of a conic, which, in our case, turns out to be a parabola.

The Caustic of a Circle Problem. Let x 2 + y2 = 1 be a circle (Fig. 3.2) and suppose that parallel vertical rays are reflected by this circle. This yields a new family of straight lines which apparently produce an interesting envelope. Find the equation of this envelope.

FIGURE 3.2. The caustic of the circle (loh . Bernoulli 1692)

Joh. Bernoulli (1692) gives a solution "per vulgarem Geometriam Cartesianam"; on the other hand, in his "Lectiones" (Joh. Bernoulli 1691192b, Lectio

100 II. Differential and Integral Calculus

y

x

o

1 2

1 2 cos 0'

, ,,

,,

,

///'2a - rc/2

- 1

FIGURE 3.3. The reflected ray

XXVII, "Caustica circularis radiorum parallelorum", Opera, vol. 3, p.467), he uses the "modern" differential calculus. Solution. For representing the family of reflected rays, we choose as parameter the angle 0' between the ray and the radius vector (see Fig. 3.3). After some elementary geometry and from the fact that the reflected ray has slope tan(20' - 1f / 2) = - cos 2a/ sin 20', we find the equation

(3.5)

x( a

a)

1 cos 20' 1 sin cos Y=-2cosa-xsin2a=-2cosa+2 cosa-sina '

As required by (3.2), the "X)dx. The obvious substitution u = eAX gives du = Ae AX dx and dx = du / (A u), and the resulting integral is that of a rational function. Example.

Here we have used the formula of Exercise 5.1 below.

Integrals of the Form JR(sin x, cos x, tan x )dx. We know from antiquity (Pythagoras 570-501 B.c., see also R.C. Buck 1980, Sherlock Holmes in Babylon, Am. Math. Monthly vol. 87,Nr. 5, p. 335-345) that the triples (3,4,5), (5,12,13), (7,24,25), ... , satisfy a 2 +b2 = c2 and are of the form (u, (u 2 -1) /2, (u 2 + 1) /2). This suggests the substitution (Euler 1768, Caput V, §261)

124 II. Differential and Integral Calculus

.

(5.21)

SlnX

1- u 2

2u l+u

=

--2'

COSX

= -1--2 '

tanx=

+u

2u l-u

--2.

One verifies that sin x = u (1 + cos x), so that the point (cos x, sin x) lies at the intersection of the line TJ = u(1 + ~) with the unit circle (see the figure). Consequently, we have u = tan(x/2), x = 2 arctan u, and

2 dx = 1 +u 2 duo

I

All this inserted into R(sin x, cos x, tan x )dx provides an integral of a rational function.

Example.

J

dx

2 + sinx -

J

2du

(1 + u 2)(2 +

1!:2) -

J

du u 2 + u + 1·

The last integral is known from Eq. (4.18), thus,

J

d~

= 22 + sm x.J3

arctan(2-(u+~)) .J3 2

.J3 arctan(2-(tan~+~)). .J3 2 2

= 2-

I

Integrals of the Form R( v' ax 2 + 2bx + c, x )dx. The idea (Euler 1768, § 88) is to define a new variable z by the relation ax 2 + 2bx + c = a(x - z)2. This yields the substitution

az 2 - c x = 2(b + az) , (5.22)

dx =

a(az 2 + 2bz + c) 2(b + az)2 dz,

J ax 2 + 2bx + c = ±Va (z _ x) = ±Va . az22(;+az 2bz ~ c, z= x±

J ax 2 + 2bx + c/ Va,

and we again get an integral of a rational function. For a < 0 this leads to complex arithmetic, which can be avoided by the transformation of Exercise 5.3. Sometimes it is more convenient to transform the expression y"a-x"2-+-2:=":b"-x-+-c by a suitable linear substitution z = ax + f3 into one of the forms

Vz2=1,

~.

Then, the substitutions (5.23)

z = sinhu,

z = coshu,

z = sinu

can be applied to eliminate the square root in the integral.

11.5 Functions with Elementary Integral 125

Example. Consider again the integral (4.27). Putting x

Jvi

x2

+ 1 dx =

J

cosh 2 U du

u

= "2 +

J(~ + cos~

=

sinhucoshu 2

For the inverse function of x

1 1 ( n x

="2

= sinh u, we get

2u ) du

+y

= ~ + Sin~ 2u

~) x~ + 1

+

XVX2

2

+1

.

= sinh u see Exercise 1.4.3.

Exercises 5.1 (Joh. Bernoulli, see quotation at the beginning of this section). Prove that

J

x2

dx -

a2

=~ 2a

In x - a x +a

5.2 Show that

+ C.

is an elementary function.

5.3 (Euler 1768, Caput II, §88). Suppose that ax 2 + 2bx + c has distinct real roots a, /3. Show that the substitution Z2 = a( x - /3) / (x - a) transforms the integral

J

viax 2 + 2bx + c, x) dx (R is a rational function oftwo arguments) into J R(z) dz, where R is ratioR(

nal. 5.4 Mr. C.L. Ever simplifies Eq. (5.16) with the help of (1.4.32) to

J~+ x4

1

= v'2 In x 2

8

x2

+ v'2x + 1 + v'2 arctan xv'2 - v'2x + 1 4 1 - x2

and obtains, e.g.,

l

v'2 dx v'2 v'2 -4-- = In 5 + -4 arctan( -2) = -0.1069250677, o x +1 8

a negative value for the integral of a positive function. Where did he make a mistake and what is the correct value?

Jvx +

5.5 Compute

dx

2 1 twice; once with the substitution (5.22) and once with the substitution (5.23). This leads to the formula arsinh x = In( x + Vx 2 + 1) (see Exercise 1.4.3). 5.6 Prove that R(sin 2 x, cos 2 x, tan x) dx

J

can be integrated with the substitution . 2 SIn

x

=

2 U --2'

l+u

2

cos x

=

1 l+u

--2'

tan x

= u.

126 II. Differential and Integral Calculus

II.6 Approximate Computation of Integrals ... because after all these attempts, analysts have finally concluded that one must abandon all hope of expressing elliptical arcs with the use of algebraic formulas, logarithms and circular arcs. (Lambert 1772, Rectification elliptischer Bogen ... , Opera vol. I, p. 312) Although the problem of numerical quadrature is about two hundred years old and has been considered by many geometers: Newton, Cotes, Gauss, Jacobi, Hermite, Tchebychef, Christoffel, Heine, Radeau [sic], A. Markov, T. Stitjes [sic], C. Posse, C. Andreev, N. Sonin and others, it can nevertheless not be considered sufficiently exhausted. (Steklov 1918)

J

eXxdx , One easily convinces oneself by our method that the integral which has greatly occupied geometers, is impossible in finite form ... (Liouville 1835, p. 113)

In spite of the extraordinary results of the previous sections, many integrals resisted the ingenuity of the Bemoullis, of Euler, of Lagrange, and of many others. Amongst these integrals, we note

j e-x2 dx, j

dx J4x 3 - g2X - g3'

j

viI -

j eXxdx,

j dx

k2 cos2 Xdx,

j

lnx'

dx J(1 - x2)(1 - k2x 2)'

The last three are so-called "elliptic integrals". Legendre, Abel, Jacobi, and Weierstrass devote a great deal of their work to the study of these integrals. The above integrals cannot be expressed in finite terms of elementary functions (Liouville 1835, see quotation), and we are confronted with new functions that have to be computed with new methods. We consider three approaches: (1) series expansions; (2) approximation by polynomials (numerical integration); and (3) asymptotic expansions.

Series Expansions The idea is to develop the function into a series (either in terms of powers of x, or in terms of other expressions) and to integrate term by term. A justification of this procedure will be given in Sect. 111.5 below. Historical Examples. The computations of Mercator (see Eq. (1.3.13))

In(l+x)=j_l_ dx =j(I-X+x2 _ ... )dx=x_x2 +x3 _ ... l+x 2 3 are the oldest example. The computation of the length of an arc of the circle y v'1 - x 2 (see Eq. (4.9) and Theorem 1.2.2) arcsinx

= =

r Jl + y'(t)2 dt = ior V11 + ~ dt = r (1- t 2)-1/2 dt 1- t io

io

l

x (

o

1

2

1.3 2·

4 ) dt=x+--+--+ 1x 1.3 x + ... ...

1+-2 t +-4t

is precisely Newton's approach to Eq. (1.4.25).

3

5

2 3

2·4 5

=

II.6 Approximate Computation of Integrals 127

Perimeter of the Ellipse. We wish to compute the perimeter of the ellipse with semiaxes 1 and b: 2

2+!L=1 X b2

or

x=cost,

y=bsint.

Since dx = - sin t dt and dy = b cos t dt, the perimeter is {27r

p= Jo

r/2 JdX2+dy2=4Jo Jsin 2 t+b 2 cos 2 tdt

(6.1)

1

7r/2

= 4

o

Jl - (1 - b2 ) cos 2 tdt. "-v--'

a

This is an "elliptic integral" (whence the name), which is not elementary. We compute it as follows: suppose that 1 > b > 0, thus 0 < a < 1. The idea is to use (Theorem 1.2.2), Newton's series for

vr=x

v'1=X = 1 - ~ - ~ x 2 - ~ x 3 -

(6.2)

2

which gives (6.3)

P = 4

1

7r/2

o

2·4

a

2·4·6

... ,

1·1

(1 - - cos2 t - a 2 cos 4 t - ... ) dt. 2 2·4

With the techniques of Sect. 11.4 (see Eq. (4.28», we find that

1

7r/2

o

cos

2n

d 7r 1·3·5· ... ·(2n-l) t t - -. --------'---:-----:--'- 2 2·4·6· .... (2n) ,

and (6.3) becomes (cf. Euler 1750, Opera, vol. XX, p. 49) (6.4) P

1

1

= 27r ( 1 - a2" . "2 - a

21.1

1·3

31.1.31.3.5

)

2.4· 2.4 - a 2.4.6· 2.4.6 - ....

The convergence of this formula is illustrated in Fig. 6.1. For a = 0 (i.e., b = 1) we have a circle, and P = 27r. For a = 1 (i.e., b = 0) the series converges very slowly to the correct value, 4. Fresnel's Integrals. The Fresnel Integrals (Fresnel 1818), (6.5) have interesting properties (Exercise 6.4) and produce, in the (x, y) plane, a beautiful spiral (Fig. 6.2). They are not elementary. However, the functions sin( u 2 ) and cos( u 2 ) have a simple infinite series (the series of sin z and cos z where z = u 2 ; see (1.4.16) and (1.4.17», of which we evaluate the integral term by term, as follows:

128 II. Differential and Integral Calculus

21t

b 1.0

-1

FIGURE 6. 1. Convergence of the series (6.4) (perimeter of the ellipse)

sin( u 2 ) du =

lt

cos( u 2 ) du =

It( 1 -

lt l o

t

o

0

(u 2

-

o

u6 3.

FIGURE 6.2. Fresnel's Integrals

u lO 5.

1" + -, - ... ) du u4 -2' ·

t3

= -

3

e

- --I

7 . 3.

tll

+-I 11 . 5.

5 9 + .. . )du = t - -5 t-. 2.I + -9 t-. 4.I -

t l3

--I

13 . 6.

.. .

+ ....

The convergence of these series is illustrated in Fig. 6.3. The results are excellent for small values of t. For increasing values of Itl, more and more terms need to be taken into account.

FIGURE 6.3. Fresnel's Integrals by power series; the numbers 5, 9, 13 and 7, 11, 15 indicate the last power of t taken into account

Numerical Methods

J:

Suppose we want to compute the integral f(x)dx, where the integration interval is given. The idea is the following: we fix N, subdivide the interval [a , b] into N subintervals of length h = (b - a) / N, Xo = a,

Xl

= a

+ h,

Xi

= a + ih,

XN

= b,

and replace the function f (x) locally by polynomials that can easily be integrated.

II.6 Approximate Computation of Integrals 129

Trapezoidal Rule. On the interval [Xi, XHd , the function f(x) is replaced by a straight line passing through (Xi, f(Xi)) and (Xi+l ' f(XHd). The integral between Xi and Xi+l is then approximated by the trapezoidal area h . (J(Xi) + f(XHl))/2 and we obtain

I

a

(6.6)

b

f(x) dx ~

N- l h

L

2 (J(Xi) + f(XHd)

i= O

f(xo)

= h ( -2-

+ f(xd + f(X2) + . .. +f(XN - d + -f(XN)) 2- .

Example. The upper pictures of Fig. 6.4 show the functions cos x 2 and sin x 2 together with the trapezoidal approximations (step size h = 0.5, N = 10). The points of the lower pictures represent approximations to Fresnel's Integrals obtained with h = 1/2 and h = 1/8; the corresponding values are connected by straight lines.

I

r sint

10

2 dt

2

3

4

5

FIGURE 6.4. Fresnel's Integrals by the Trapezoidal Rule

Simpson's Method (named after Simpson 1743). The idea is to choose three successive values of f (Xi) (Yi = f (Xi) and to compute the parabola of interpolation through these points (see Theorem I.l.2 and Eq. (2.6»:

_ p (x ) - Yo

( _ ) Llyo Xo h

+ x

With the substitution x = Xo becomes

(6.7)

+

(x - xo)(x - xd Ll 2 yo 2 h2 .

+ th, the area between the x-axis and this parabola

130 II. Differential and Integral Calculus

We find Simpson's Rule (N even) (6.S)

l

b

a

f(x) dx ~

h

:3 (!(xo) +4f(xt} + 2f(X2) +4f(X3) + 2f(X4) + ... + f(XN)).

Newton-Cotes Methods. Taking higher degree interpolation polynomials, we find, in the same way,

and so on. The first one, due to Newton (1671), is called the 3/S-rule. In 1711, Cotes computed these formulas for all degrees up to 10 (see Goldstine 1977, p.77).

I:

Numerical Examples. We compute approximations of o ~ = In(lO) with the above methods for N = 12,24,48, .... The results are presented in Table 6.1. We observe a genuine improvement only in every second column (for an explanation, see Exercise 6.5).

TABLE 6.1. Computation of

I/O d: with different quadrature formulas

N

Trapezoid

Simpson

Newton

Cotes

12 24 48 96 192 384 768 1536 3072 6144

2.34 2.31 2.305 2.303 2.3027 2.3026 2.3025 2.302587 2.3025858 2.3025852

2.307 2.303 2.3026 2.302587 2.3025852 2.3025851 2.302585093 2.3025850930 2.302585092996 2.3025850929941

2.31 2.303 2.3026 2.30259 2.3025854 2.3025851 2.302585094 2.3025850930 2.302585092999 2.3025850929943

2.305 2.3027 2.30259 2.3025852 2.302585095 2.3025850930 2.3025850929947 2.30258509299405 2.3025850929940458 2.302585092994045686

An interesting phenomenon can be observed when applying the trapezoidal rule to the elliptic integral P = VI - a cos 2 t dt (here with b = 0.2, a = 0.96, see Table 6.2). It converges much better than expected. The reason is that the function f(t) is periodic and the "superconvergence" is explained by the EulerMaclaurin formula of Sect. 11.10.

I;7r

II.6 Approximate Computation ofIntegrals 131

TABLE 6.2. Computation of an elliptic integral with the trapezoidal rule N Trapezoid

12 24 48 96 192 384

4.1 4.201 4.2020080 4.20200890792 4.20200890793780018891 4.202008907937800 1889398329176947477824

Asymptotic Expansions

J;

This method was used by Laplace (1812) for e- t2 dt (see Oeuvres, tome VII, p. 104 and Exercise 6.7) and by Cauchy in 1842 for Fresnel's integrals (see Kline 1972, p. 1100). Whereas series expansions and numerical methods are useful for small and moderate values of x, the method of asymptotic expansions is especially adapted for large x. We illustrate this technique on the example of Fresnel's integrals. For the limiting case x ---; 00 the exact value of the integral is known to be (Exercise IY.S.14)

{'Xl cos edt = {'XJ sin edt = ~

(6.9)

10

10

The idea is now to split the integral according to

l

(6.10)

x

1~- -

cost 2 dt = -

22

0

E.

2V2

J; = Jooo - Jxoo, i.e.,

1

00

cost 2 dt.

x

To the integral on the right, we artificially add the factors 2t and 1/ (2t) and apply integration by parts with u(t) = l/t, v(t) = sine. This yields -

1x

00

cos t 2 dt

= - -1 2

1x - . 00

1 2t cos t 2 dt t

= -1 -1. sm x 2 - -1 2x

2

1x

00

1 sm . t 2 dt. ""2 t

We find an integral that appears by no means easier than the first one. However, for x large, the integral on the right, which contains the additional factor 1/t2 , is much smaller than the original one. Therefore, (2X)-1 sinx 2 will be a good approximation for - Jxoo cos t 2 dt. If the precision is not yet good enough, we repeat the same procedure (here with u(t) = 1/t 3 and v(t) = - cost 2 ), (6.11)

1 -2

1

00

x

. 2 dt -1 smt

t2

1 -1 = --3 2.2 x

cos x 2

31

1. + -2.2

x

00

-14 cost2 dt. t

132 II. Differential and Integral Calculus

Continuing like this, we find from (6.10) that

l

1 ~ 1 1. 2 1 1 2 1.3 1 . 2 2 cost dt = - - + --smx - - - - cosx - - - - - smx 2 2 2x 2 . 2 x3 2 . 2 . 2 x5 1.3.5 1 2 1.3.5.7 1 . 2 (6.12) + 2 . 2 . 2 . 2 7x cos x + 2 . 2 . 2 . 2 . 2 -x 9 sm x x

o

An analogous formula is valid for

l

x

o

.

2

sm t dt

(6.13)

= -1 ~ - - -1 -1 cos x 2 2

+

2x

2

1·3·5 ~sinx2 2 . 2 . 2 . 2 x7

1 1 . 2 - - - sm x 2 . 2 x3

+ -1-. 3- -15 2.2.2 x

cos x

2

1·3·5·71 cosx2 2 . 2 . 2 . 2 . 2 x9

The extraordinary precision of these approximations for large x is illustrated in Fig. 6.5. The numbers 1,3,5 indicate the last power of 1/x taken into account.

O' - - --i.

FIGURE 6.5. Asymptotic expansions (6.12) and (6.13) with 1,2,3,10,20, and 30 terms

(6.1) Remark. The error of the truncated series (6.12) can easily be estimated. For example, if we truncate after the term (2x) -1 sin x 2, the above derivation shows that the error is given by the value of the integralin (6.1l)(taken over x S t < (0). Using Icos t 21 1 this yields the estimate (2x 3)- 1, which, for x > 2, is less than 0.0625.

s

(6.2) Remark. The infinite series (6.12) and (6.13) do not converge for a fixed x. The reason is that the general term contains the factor 1 . 3 . 5 . 7 . 9 .... in the numerator, which dominates all other factors. Such series were called asymptotic expansions by Poincare.

Exercises 6.1 (Joh. Bernoulli 1697). Derive the "series mirabili"

1 1

o

xlIII + - - - + -55 &c. 22 33 44

x dx = 1 - -

11.6 Approximate Computation ofIntegrals 133

Hint. Use the series for the exponential function in XX = xn (In x)n dx by integration by parts.

I

eX

In X

and compute

I

6.2 The integral x 2 dx / VI - X4 was encountered by Jac. Bernoulli in his computation of the elastic line and by Leibniz in his study of the Isochrona Paracentrica. Verify the formula (Leibniz 1694b)

J

X2 dx 1 3 1 7 ---=-x +--x +

v'1 -

x4

3

7.2.1

1.3 11 1.3.5 15& x + x c. 11 . 4 . 1 . 2 15 . 8 . 1 . 2 . 3

6.3 As in (6.7), derive the formulas of Newton and Cotes by integrating the interpolation polynomials of degree 3 and 4 on the intervals [xo, X3] and [xo, X4], respectively. 6.4 For the curve defined by (6.5) (see Fig. 6.2) prove that a) the length ofthe arc between the origin and (x(t), y(t)) is equal to t; and b) the radius of curvature at the point (x(t), y(t)) is equal to 1/(2t). 6.5 Prove that Simpson's method is exact for all polynomials of degree 3. 6.6 Compute

In(1 + x) dx Jo 1 + x 2 with the help of Simpson's method. Study the decrease of the error with increasing N. Result. The correct value is (71"/8) In 2 = 0.2721982613.

f1

6.7 Using

Iooo e- t2 dt =

Vir/2 (see (IY.S.4I) below), derive an asymptotic ex-

Jrr I;

pansion for the error function p(x) = e- t2 dt that is valid for large values of x (Laplace 1812, Livre premier, No. 44).

e- x2 (1

1 r;;; - - --3 y71" X 2· x 6.8 Compute numerically the integral Result.

p(x)

rOO

Jo

= 1-

1

v'X cosx

2

dx =

.3 5 + -12 2 .x

1 ·3 . 5 7 -3 2

.x

) + ....

71"v'2V2 + v'2 4. r(3/4) ::::: 1.674813394.

Choose two numbers A::::: 1/10 and B ::::: 10 and compute the integral a) on the interval (0, A] by a series; b) on the interval [A, B] by Simpson's method; and c) on the interval [B, (0) by an asymptotic expansion.

134 II. Differential and Integral Calculus

II. 7 Ordinary Differential Equations Ergo & horum integralia aequantur.

(Jac. Bernoulli 1690)

In Sects. 11.4 and 11.5, we treated the problem of finding a primitive of a given function f(x), i.e., we were looking for a function y(x) satisfying y'(x) = f(x). Here, we consider the more difficult problem where the function f may also depend on the unknown function y( x). An ordinary differential equation is a relation of the form

y' = f(x , y).

(7.1)

We are searching for a function y(x) such that y'(x) = f(x , y(x)) for all x in a certain interval. Let us begin with some historical examples (for more details, see Wanner 1988). The Isochrone of Leibniz. Galilei discovered that a body, falling from the origin along the y-axis, increases its velocity according to v = J - 2gy, where 9 is the acceleration due to gravity. During his dispute with the Cartesians about mechanics, Leibniz (in the Sept. 1687 issue of the journal Nouvelles de la Republique des lettres) poses the following problem: find a curve y(x) (see Fig. 7.1) such that, when the body is sliding along this curve, its vertical velocity dy / dt is everywhere equal to a given constant - b.

L_

I

-1

x

dx

jI

lIT" I I

FIGURE 7.1. Leibniz's isochrone

One month later, "Vir Celeberrimus Christianus Hugenius" (Huygens) gives the solution, "sed suppressa demonstratione & explicatione". The "demonstratio", then published in Leibniz (1689), is unsatisfactory, since the solution is guessed and then shown to possess the desired property. A general method for finding the solution with the help of the "modern" differential calculus was then published by lac. Bernoulli (1690). This started the era of spectacular discoveries made by lac. and loh. Bernoulli, later by Euler and Daniel Bernoulli, and made Basel for several decades the world center of mathematical research. Let us write Galilei's formula as

II.7 Ordinary Differential Equations 135

(7.2)

2 ( ds)2 = dx + dy2 = - 2 dt dt2 gy

(s = arc length) ,

divide by (dy / dt)2 = +b2 (which is the required condition), and obtain (7.3)

( dX)2 dy

_ - 2gy b2

+1-

-1

dy dx

or

J -1 - 2gy/ b2 '

a differential equation as in (7 .1). In order to understand Bernoulli's idea, we write (7.3) as

dx= -

(7.4)

V- l - -2gybd y 2

'

which expresses the fact (see Fig. 7.1) that the two striped rectangles always have the same area. So Jacob writes "Ergo & horum Integralia aequantur" (this is the first appearence in mathematics of the word "integral"), meaning that the areas 51 and 52 also have to be equal. After integrating, we find the solution

x

= ~ (-1 3g

_29y) 3/2 b2

'

and the "Solutio sit linea paraboloeides quadrato cubica ... " (Leibniz).

The Tractrix. The distinguished Parisian physician Claude Perrault, equally famous for his work in mechanics and in architecture, well known for his edition of Vitruvius, and in his lifetime an important member of the Royal French Academy of Science, proposed this problem to me and to many others before me, readily admitting that he had not been able to solve it ... (Leibniz 1693)

While Leibniz was in Paris (1672-1676) taking mathematicallessons from Huygens, the famous anatomist and architect Claude Perrault formulated the following problem: for which curve is the tangent at each point P of constant length a between P and the x-axis (Fig. 7 .2)? To illustrate this question, he took out of his fob a "horologio portabili suae thecae argenteae" and pulls it across __ the table. He mentioned that no mathematician from Paris or Toulouse (Fermat) was able to find the formula. Leibniz published his solution in 1693 (see Leibniz 1693), asserting that he had known it for quite some time, as

dy dx

y

i.e.,

-

Ja2 - y2

y

dy

= dx ,

one finds ("ergo & horum .. .") the solution by quadrature (Figs. 7.2 or 7.3). Leibniz asserts that it was "a well-known fact" that this area is expressible with the logarithm, which, using the substitution J a 2 - y2 = v , a2 _y2 = v 2 , -y dy = v dv,

136 II. Differential and Integral Calculus

-2

-I FIGURE 7.2. The tractrix

(7.5)

x

=

l

y

a

Ja2 - y2

Y

FIGURE7.3. Sketch by Leibniz (1693)1

dy

= -Ja 2 -

a - Ja 2 - y2 y2 - a log ---!...----''-Y

turns out to be true (see also Exercise 7.1). We mention that Leibniz's interest in this theory also went the other way around: use Perrault's watch as a mechanical integration machine for the computation of integral (7.5) (and hence of logarithms) and design other mechanical devices for similar integrals. The Catenary. But to better judge the quality of your algorithm I wait impatiently to see the results you have obtained concerning the shape of the hanging rope or chain, which Mr. Bernouilly proposed that you investigate, for which I am very grateful to him, because this curve possesses remarkable properties. I considered it long ago in my youth, when I was only 15 years old, and I proved to Father Mersenne that it was not a parabola ... (Letter of Huygens to Leibniz, Oct. 9, 1690) The efforts of my brother were without success, I myself was more fortunate, since I found the way ... It is true that this required meditation which robbed me of sleep for an entire night . . . (Joh. Bernoulli, see Briefwechsel, vol. I, p. 98)

Galilei (1638) asserted that a chain hanging from two nails forms "ad unguem" a parabola. Some 20 years later, a 16 year old Dutch boy (Christiaan Huygens) discovered that this result must be wrong. Finally, the solution of the problem of the shape of a hanging flexible line ("Linea Catenaria vel Funicularis") by Leibniz (169 I b) and Joh. Bernoulli (169 I) was an enormous success for the "new" calculus. Here are Johann's ideas (Opera vol. III, p. 491-493). We let B be the lowest point and A an arbitrary point on the curve (Fig. 7.4). We then draw the tangents AE and BE and imagine the mass of the chain of length s between A and B concentrated in the point E hanging on two threads without mass ("duorum filiorum nullius gravitatis"). Since the mass in E is proportional to s, the parallelogram of forces in E shows that the slope in A is proportional to the arc length, i.e., I

Reproduced with permission of Bib!. Pub!. Univ. Geneve.

11.7 Ordinary Differential Equations 137

y

/

'r----------~r-----T-~~~.&

x )If "

FIGURE 7.4. The catenary

Of

FIGURE 7.5. Catenary (Leibniz 1691)2

c· y' = s.

(7.6)

From here, Johann's computations are very complicated, using second differentials (see Opera vol. III, p. 426). They become easy, however, if we replace, in the spirit of Riccati (see (7.21) below), the derivative y' by a new variable p and have after differentiation

c· dp = ds =

(7.7)

VI + p2 dx,

a differential equation between the variables p and x. Integration gives

c

(7.8)

J

P

dp ~ =

VI

+ p2

J

X -c-XO) = sinh ( ---

i.e.,

dx,

and

y

X - Xo

arsinh(p) = - - , c

X -cXo) = K + c . cosh ( - .

The Brachistochrone. Given two points A and B in a vertical plane, determine the path AM B along which a moving particle M, starting at A and descending solely under the influence of its weight, reaches B in the shortest time. (Joh. Bernoulli 1696) This problem seems to be one of most curious and beautiful that has ever been proposed, and I would very much like to apply my efforts to it, but for this it would be necessary that you reduce it to pure mathematics, since physics bothers me . . . (de L'Hospital, letter to Joh. Bernoulli, June 15, 1696) 2

Reproduced with permission of Bibl. Publ. Univ. Geneve.

138 II. Differential and Integral Calculus

Galilei proves in 1638 that a body sliding from A to C (Fig. 7.7) takes less time on the detour ADC than on the shortest path (due to its larger initial velocity). He continues and proves that ADEC, ADEFC, ADEFGC are always quicker and finally concludes that the circle is the quickest of all paths. Hearing that his brother Jacob makes the same mistake, Johann (1696) seizes this as the occasion for organizing a public contest to find the brachistochrone line (f3{!OXU~ = short, X{!6vo~ = time). The solutions handed in on time, including Jacob's, were unfortunately all correct; nevertheless, Johann's is the most elegant one: he makes an analogy to "Fermat's Priciple" (see Eq. (2.5»: ,-------------~ A

A

x

dx ds ex y

C

C FIGURE 7.7. The wrong brachistochrone as seen by Galilei

FIGURE 7.6. The brachistochrone

He thinks of many layers where the "speed of light" is given by v = V2gy (see (7.2) and Fig. 7.6). The quickest path is the one satisfying everywhere the law of refraction (Fermat's principle),

_v_ =K. sino:

Hence, we have, because of sin 0: = dx / ds, (7.9)

or

dx

=

J

y

c-y

. dy.

Still in accordance with "ergo & horum integralia requantur", the substitution (7.10)

. 2

C

C

2

2

Y = c . sm u = - - - cos 2u

leads to the formula (7.11)

. 2u x - xo = c u - -c sm 2

"ex qua concludo Curvam Brachystochronam esse Cycloidem vulgarem".

11.7 Ordinary Differential Equations 139

Some Types of Integrable Equations We now discuss some of the simplest types of differential equations, which can be solved by the computation of integrals. Equation with Separable Variables.

y' = f(x)g(y).

(7.12)

All of the preceding examples, namely, (7.3), (7.5), (7.7), and (7.9), are of this type. They are solved by writing y' = dy / dx, by "separation of variables" and integration ("ergo & ... "), i.e., (7.13)

dy g(y) = f(x) dx

and

j

g~~) =

j f(x)dx+C.

If G (y) and F (x) are primitives of 1/9 (y) and f (x), respectively, the solution is expressed by G(y) = F(x) + C. Linear Homogeneous Equation.

y' = f(x)y.

(7.14)

This is a special case of (7.12). Its solution is given by (7.15)

Iny= j f(x)dx+C,

or

y=C.exp(j f(x)dx).

Linear Inhomogeneous Equation.

y' = f(x)y

(7.16)

+ g(x).

Joh. Bernoulli proposes to write the solution as a product of two functions y( x) = u(x) . v(x) (like Tartaglia's idea, Eq. (1.1.5». We then obtain

du dv dx . v + dx . u = f(x) . U· v + g(x). We can now equalize the two terms separately and find (7. 17a) (7.17b)

du dx dv dx

- = f(x)· u g(x) u(x)

to obtain u, to obtain v.

Equation (7. 17a) is a homogeneous linear equation for u and its solution is given by (7.15). The function v(x) is then obtained by integration of (7.17b). Consequently, the solution of (7.16) is (7.18)

y(x) = C· u(x)

+ u(x) foX ~~~~ dt,

140 II. Differential and Integral Calculus

This relation expresses the fact that the solution of (7.16) is a sum of the general solution of the homogeneous equation with a particular solution of the inhomogeneous equation.

Bernoulli's Differential Equation. In truth, there is nothing more ingenious than the solution that you give for your brother's equation; and this solution is so simple that one is surprised at how difficult the problem appeared to be: this is indeed what one calls an (P. Varignon, letter to Joh. Bernoulli "6 Aoust 1697") elegant solution.

In 1695, Jac. Bernoulli struggles for months on the solution of

y' = f(x) . y + g(x) . yn.

(7.19)

This is a good occasion for Jacob to organize an official contest. Unfortunately, Johann has straightaway two elegant ideas (see Joh. Bernoulli 1697b). The first idea is treated in Exercise 7.2. The second one is the same as explained above, namely to write the solution as y(x) = u(x) . v(x). For the differential equation (7.19) this again yields (7.17a) for u and

dv dx = g(x)un-1(x)v n ,

(7.20)

a differential equation that can be solved by separation of variables. This leads to the solution

y(x)

=

r

(

u(x) C + (1 - n) io g(t)un-1(t) dt

)l/(l-n)

,

where u(x) is as in (7.18).

Second-Order Differential Equations To free the above formula from the second differences, ... , we denote the subnormal BF by p. (Riccati 1712)

A second-order differential equation is of the form

y"

=

f(x, y, y').

The analytic solution of such an equation is very seldom possible. There are a few exceptions.

Equations Independent of y. It is natural to put p = y', so that the differential equation y" = f(x, y') becomes the first-order equationp' = f(x,p). We remark that the differential equation (7.7) of the catenary is actually of this type.

Equations Independent of x. (7.21)

y" = f(y, y').

The idea (Riccati 1712) is to consider y as an independent variable and to search for a function p(y) such that y' = p(y). The chain rule gives

II.7 Ordinary Differential Equations 141

" dp dp dy I Y = dx = dy . dx = P . p, and Eq. (7.21) becomes the first-order equation

pl. P = f(y , p) .

(7.22)

When the function p(y) has been found from (7 .22), it remains to integrate yl = p(y), which is an equation of type (7.12).

Example. The movement of a pendulum (see the sketch by Leonardo da Vinci) is described by the equation (7.23)

y"

+ siny =

0

(y denotes the deviation from equilibrium).

Since Eq. (7.23) does not depend on t (we write t instead of x , because this variable denotes the time in this example), we can use the above transformation to obtain ©Bibl. Nacional, Codex Madrid I 147r

p . dp

=-

sin y . dy

and

p2

2" =

cos y

+ C.

If we denote the amplitude of the oscillations by A (for which p = yl = 0) we have C = - cos A and get (7.24)

p

dy

,------------:-

= dt = V2cosy - 2cosA ,

which is a differential equation for y. Separation of the variables finally yields the solution expressed in implicit form with an elliptic integral

Jro ~==d=Tf=;:-"y,

y(O) = 1

= eA. The result is a well-known

in order to obtain an approximation of y(l) formula of Chap. I.

9.2 (Inverse Error Function). Define a function y( x) by the relation

x

=

~ faY e- t2 dt.

Differentiate this formula and show that y(x) satisfies the differential equation

y(O) = O. Compute the first four terms of the Taylor series for y (x) (developed at the point x = 0). 9.3 (Van der Pol's Equation). Compute y (i ) and v(i) for i = 1,2,3 for the solutions of the differential equation

=v, v' = c:(1-y2)v - y, y'

and compute numerically the solution using the third-order Taylor series method for c: = 0.3, the initial values y(O) = 2.00092238555422, v(O) = 0, and for 0 :s: x :s: 6.31844320345412. The correct solution is periodic for this interval and the given initial values.

160 II. Differential and Integral Calculus

II.IO The Euler-Maclaurin Summation Formula The King calls me "my Professor", and I am the happiest man in the world! (Euler is proud to serve Frederick II in Berlin) I have here a geometer who is a big cyclops ... who has only one eye left, and a new curve, which he is presently computing, could render him totally blind. (Frederick II; see Spiess 1929, p. 165-166.)

This formula was developed independently by Euler (1736) and Maclaurin (1742) as a powerful tool for the computation of sums such as the harmonic sum 1 + ~ + ~ + ... + ~, the sum oflogarithms In 2 + In 3 + In 4 + ... + In n = In n!, the sum of powers 1k + 2k + 3 k + ... + nk, or the sum of reciprocal powers 1 + ~ + f,; + ... + ~, with the help of differential calculus.

Problem. For a given function f(x), find a formula for n

S = f(l)

(10.1)

+ f(2) + f(3) + ... + f(n) = L

f(i)

i=1

("investigatio summae serierum ex termino generali").

Euler's Derivation of the Formula The first idea (see Euler 1755, pars posterior, § 105, Maclaurin 1742, Book II, Chap. N, p. 663f) is to consider also the sum with shifted arguments (10.2)

8

= f(O) + f(l) + f(2) + .,. + f(n - 1).

We compute the difference S -8 using Taylor's series (Eq. (2.8) with X-Xo = -1)

and find

f(n) - f(O)

=

t!'(i) i=1

~2. t!"(i) + ~3. t!"'(i) - 4.~ t!""(i) + ... i=1

i=1

In order to tum this formula for L !' (i) into a formula for by its primitive (again denoted by f): (10.3)

t

i=1

f(i) =

in 0

f(x) dx + ;!

t

i=1

!,(i) -

~!

t

i=1

!,,(i) +

i=1

L f (i), we replace f

~!

t

!"'(i) - ....

i=1

The second idea is to remove the sums L f', L f", L f"', on the right by using the same formula, with f successively replaced by f', 1", 1'" etc. This will lead to a formula of the type

IUO The Euler-Maclaurin Summation Formula 161

t

(10.4)

f(i) =

i=l

l

n

f(x) dx - ex(J(n) - f(O))

+ (3(J'(n) -

1'(0))

0

- 'Y(J"(n) - 1"(0)) + 8(J"'(n) - 1"'(0)) - ... For the computation of the coefficients ex, (3, 'Y, ... we successively replace (10.4) by 1',1", ... to obtain

'L,f(i)

-if 'L, 1'(i)

f

in

= Jon f(x) dx -ex(f(n) - f(O)) +(3(f'(n) - 1'(0)) - .. . = -if(f(n) - f(O)) +~(f'(n) - 1'(0)) - .. .

if 'L, 1" (i)

= +if(f'(n) - 1'(0)) - .. .

The sum of all this, by (10.3), has to be Jon f(x) dx. Therefore, we obtain (10.5)

1

ex+ 2! = 0,

from which we can compute ex have

L f(i) = ior n

(10.6)

i=l

f(x) dx

=

-~, (3

=

l2' 'Y = 0, 8 = - 7~O' . ..

1

1

+ "2 (J(n)- f(O)) + 12 (J'(n) -

and we

1'(0))

0

__ 1 (J"'(n)- 1"'(0)) 720

+ _1_ (J(5)(n)- f(5)(0)) + .... 30240

(10.1) Example. This formula, applied to a sum of nearly a million terms,

-1 + -1 + -1 + ... + 11

12

+

1 6 1 -6 1 = In(lO ) - In(10) + - 10 -13 1000000 2 20 1 1 -4 1 -6 1200 - 120 10 + 252 10 + ... ~ 11.463758469,

gives an excellent approximation of the exact result by a couple of terms only. The formula is, however, of no use for the computation of the first terms 1+ ~ +... + l~'

BernouHi Numbers. It is customary to replace the coefficients ex, (3, 'Y, ... by Bdi! (Bo = 1, ex = Bd1!, (3 = B2/2!, .. .), so that (10.5) becomes (10.5')

2Bl

+ Bo = 0,

3B2 + 3B 1 + Bo = 0,

... ,

The Bernoulli numbers, as far as Euler calculated them, are

162 II. Differential and Integral Calculus

1 B 1 =-"2'

Bo=l, B

1 B 2 ="6'

1 B 4 =-30'

1 B 6 =42'

1 B8=-30'

5 B - _ 691 B14 = -67 , B __ 3617 B _ 43867 122730' 16510' 18- 798' 66 , - _ 174611 B _ 854513 B __ 236364091 330' 22 138' 24 2730' - 8553103 B __ 23749461029 B _ 8615841276005 6' 28 870 ' 30 14322 ' -

10-

B 20

B 26

and B3

=

B5

= ... = O. In this notation, Eq. (10.6) becomes n

Lf(i)

(10.6')

i=l

r

= io

f(x)dx+

1

"2 (J(n) - f(O))

0

+ L B2k (f(2k-1)(n) - f(2k-1)(0)) k2:1 (2k)! . Example. For f(x) = x q the series ofEq. (10.6') is finite and gives the well-known formula of Jac. Bernoulli (1.1.28), (1.1.29).

Generating Function. In order to get more insight into the Bernoulli numbers, we apply one of Euler's great ideas: consider the function V(u) whose Taylor coefficients are the numbers under consideration, i.e., define

(10.7)

V(u) = 1 + au + /3u 2 + "(u 3 + 5u 4 + ... B1 B2 2 B3 3 B4 4 = 1 + -, u + -, u + -3' u + -4' u + .... 1. 2. . .

Now the formulas (10.5) alias (10.5') say simply that

that is,

(10.8)

V(u)

u

= --. eU -1

Thus, the infinitely many algebraic equations become one analytic formula. The fact that

u

V(u)

+ "2 =

u

u

eU - 1 +"2 =

u

e u / 2 + e- u / 2

"2 . eu / 2 _ e- u / 2 is an even function shows that B3 = B5 = B7 = ... = o. (10.9)

II.1D The Euler-Maclaurin Summation Formula 163

De Usu Legitimo Formulae Summatoriae Maclaurinianae We now insert f(x) = cos(27rx), for which f(i) = 1 for all i, into Eq. (10.6'). This gives 1 + 1 + ... + 1 to the left, and 0 + 0 + 0 + ... to the right, because cos( 27rx) together with all its derivatives is periodic with period 1. We see that the formula as it stands is wrong! Another problem is that for most functions f the infinite series in (10.6') usually does not converge. It is therefore necessary to truncate the formula after a finite number of terms and to obtain an expression for the remainder. This was done in beautiful Latin (see above) by Jacobi (1834) by rearranging Euler's proof using the error term (4.32) of Bernoulli-Cauchy throughout. It was later discovered (Wirtinger 1902) that the proof can be done simply by repeated integration by parts in a similar manner to the proof ofEq. (4.32). The main ingredient of the proof is the so-called Bernoulli polynomials.

Bernoulli Polynomials. The polynomials

Bl(X) B 2(x) B3(X) B4(X)

+ Bl Box2 + 2B1 x + B2 Box3 + 3B1 x 2 + 3B2 x + B3 Box4 + 4B1 x 3 + 6B2 x 2 + 4B3X + B4

= Box = = =

or, in general, (10.10) satisfy

(k 2 2).

(10.11)

Indeed, the first formula of (10.11) is a property of the binomial coefficients (see Theorem 1.2.1); the second formula follows from the definition and from (10.5').

(10.2) Theorem. We have

L n

i=l

f(i)

=

r

io

1

f(x) dx + "2 (i(n) - f(O))

0

where (10.12)

Rk =

(_~!k-l

Ion Bk(X) f Ck )(x) dx.

Here, Bk(X) is equal to Bk(X) for 0 ::::; x ::::; 1 and extended periodically with period 1 (see Fig. 10.1).

164 II. Differential and Integral Calculus

2

3

4

FIGURE lO.1. Bernoulli polynomials

=

Proof We start by proving the statement for n grating by parts we have

{1 io f(x)dx

=

{1 io B~(x)f(x)dx

=

1. U sing B~ (x)

=

1 and inte-

1 (1 B1(x)f(x)lo - io B1(X)f'(x)dx.

The first term is ~(f(1) + f(O)). In the second term we insert from (10.11) B1 (x) = ~ B2 (x) and integrate once again. This gives

or, continuing like this, (10.13)

~ (1(1) + f(O)) = 10

1

(-j; B

k·

f(x) dx + ~

j

(1(j-1)(1) - f(j-1)(0))

+ Rk ,

with (10.14) We next apply Eq. (10.14) to the shifted functions f(x

+ i-I), observe that

and obtain the statement of Theorem 10.2 by summing these formulas from i to i = n.

=

1 0

II.10 The Euler-Maclaurin Summation Formula 165

Estimating the Remainder. The estimates (for 0

:s: x :s: 1)

which are easy to check, and the fact that I Ion g(x) dxl

:s: Ion Ig(x) Idx, show that

(10.15) These are the desired rigorous estimates of the remainder of Euler-Maclaurin's summation formula. Further maximal and minimal values of the Bernoulli polynomials have been computed by Lehmer (1940); see Exercise 10.3. (10.3) Remark. If we apply the formula of Theorem 10.2 to the function f(t)

=

hg(a + th) with h = (b - a)/n and if we pass the term (J(n) - f(O)) /2 to the left side, we obtain (with Xi = a + ih)

"2 g(xo) + h L g(Xi) + "2 g(x n ) = h

n-1

h

i=l

Ib g(x) dx a

(10.16)

where we recognize on the left the trapezoidal rule. Equation (10.16) shows that the dominating term of the error is (h 2 /12)(g'(b) - g'(a)). However, if 9 is periodiS then all terms in the Euler-Maclaurin series disappear and the error is equal to Rk for an arbitrary k; this explains the surprisingly good results of Table 6.2 (Sect. 11.6).

Stirling's Formula

= lnx in the Euler-Maclaurin formula. Since

We put f(x) n

Lf(i)

= In2 + In3 + In4 + In5 + ... + Inn = In (n!),

i=2

we will obtain an approximate expression for the factorials n!

= 1 . 2 ..... n.

(10.4) Theorem (Stirling 1730). We have (10.17)

n!

=

y!27fn nn ( 1 1 1 1 - ) e n ' exp 12n - 360n3 + 1260n5 - 1680n7 + Rg ,

166 II. Differential and Integral Calculus

where

IRgl :::; 0.0006605/n8. This gives,for n

~

00,

the approximation

(10.18)

Remark. This famous formula is especially useful in combinatorial analysis, statistics, and probability theory. Equation (10.17) is truncated after the 4th term simply because one additional term would not fit into the same line. The numerical values of (10.18) and (10.17) (with one, two and three terms) for n = 10 and n = 100 are compared to n! in Table 10.1.

TABLE 10.1. Factorial function and approximations by Stirling's formula

n

n

= 10:

=

Stirling 0 = 0.3$869561874103592162317593283· 10 7 Stirling 1 = 0.3628~1005142693352994116531675· 107 Stirling 2 = 0.3628799!V141301292538591223941· 107 Stirling 3 = 0.362880000q21301281279077612862. 107 n! = 0.362880000000000000000000000000.10 7

100: Stirling 0 = 0.93f2484762526934324776475612718. 10 158 Stirling 1 = 0.93326215/7031762340989619195146. 10 158 Stirling 2 = 0.93326215443$67463946383356624. 10 158 Stirling 3 = 0.933262154439441$2371338864918. 10 158 n! = 0.933262154439441526816992388563· 10 158

Proof We have seen above (Example 10.1) that the Euler-Maclaurin formula is inefficient if the higher derivatives of f (x) become large on the considered interval. We therefore apply the formula with f (x) = In x for the sum from i = n + 1 to i = m. Since

J

Inxdx

= xInx -

x,

we obtain from Theorem 10.2 that

.L m

f (i) = In m! - In n! = mIn m - m -

(n In n - n)

1

+ '2 (In m -

In n)

t=n+l

(10.19)

+ ~ (~ _ ~) __ 1 (~_~) + R5 12

m

n

360

m3

n3

'

where IR51 :::; 0.00123/n4 for all m > n. This estimate is obtained from (10.12) and (10.15) and the fact that IB5(X)1 :::; 0.02446 for 0 :::; x :::; 1. In (10.19), the terms In n!, n In n, n, and (1/2) In n diverge individually for n ~ 00. We therefore take them together and set

II.1D The Euler-Maclaurin Summation Formula 167

(10.20)

"in

= In n! + n

- (n

+ ~) In n,

and (10.19) becomes

(10.21)

"in = "im

+ ~ (~-~) 12

n

m

_1 360

(~-~) 3 3 n

m

-

R5 .

For n and m sufficiently large "in and "im become arbitrarily close. Therefore, it appears that the values "im converge, for m --+ 00, to a value that we denote by "i (the precise proof will be given in Theorem 111.1.8 of Cauchy). We then take the limit m --+ 00 in Eq. (10.21) and obtain

In n! where get

+n

- (n

+ -1) In n = "i + - 1 - -1-3 + R~ 5 , 2

12n

360n

IR51 :s; 0.00123/ n4. Taking the exponential function of this expression we with

1 1 ~ ) Dn=e'Y·exp ( - - - - + R 5 . 12n 360n3

This proves (10.18) and also (10.17), as soon as we have seen that the limit of Dn (i.e., D = e'Y) is actually equal to vfiii. To this end, we compute, from (10.22),

n! . n! . (2n)2n . e- 2n J2ri n2n . e- 2n . n· (2n)!

2 . 4 . 6 . 8 ..... 2n

1·3·5·7· .... (2n - 1) .

J2

vn'

which tends to D too. This formula reminds us of Wallis's product ofEq. (1.5.27). Indeed, its square,

. 2(2n+1) ( D n .Dn )2 = 2·2·4·4·6·6 ... (2n)(2n) D2n 1 . 3 . 3 . 5 . 5 . 7 ... (2n - 1) (2n + 1) n ' ,

,

--+~/2

'---v----'

--+4

tends to 21f, so that D = vfiii. The stated estimate for and IBg(x)1 :s; 0.04756.

Rg follows from (10.12) 0

The Harmonic Series and Euler's Constant We try to compute

111

1

1+"2+"3+4+···+; by putting f(x) = l/x in Theorem 10.2. Since f(j) (x) = (-I)j j!x- j - 1 , we get, instead of (10.19),

168 II. Differential and Integral Calculus

FIGURE 10.2. Euler's autograph (letter to Joh. Bernoulli 1740, see Fellmann 1983, p. 96)1

(10.23)

f

~=

i=n+l

Z

1 ~dx+ ~(~ _~) m

n

2 m

X

n

_

~(_12 -~) 12 m

1(11) 1(11) m 4 - n4 - 252 m 6 - n 6

+ 120

n2

1(1 m8

+ 240

-

1) n8

~

+ R9,

where, because of IB9(X)1 ::; 0.04756, we have IR91 ::; 0.00529/n9. The diverging terms to collect will now be, instead of (10.20), "in

=

1

L -:- - In n, n

i=l

Z

which is investigated precisely as above and seen to converge. This time, the constant obtained,

1 1 1 1 + - + - + ... + - - In n - t "i = 0.57721566490153286 .. . , 2 3 n is a new constant in mathematics and is called "Euler's constant" (see Fig. 10.2 for an autograph of Euler containing his constant and its use for the computation of the sum of Example 10.1). Letting, as before, m - t 00 in (10.23), we obtain

(10.24)

1

Li n

(10.25)

i=l

1

= "i + In n + 2n

1 - 12n2

1

+ 120n4

1 - 252n 6

1

~

+ 240n8 + R 9,

where IR91 ::; 0.00529/n9. To find the constant "i, we put, for example, n = 10 (as did Euler) in Eq. (10.25) and obtain the value of (10.24). This constant was computed with great precision by D. Knuth (1962). It is still not known whether it is rational or irrational. I

Reproduced with permission of Birkhaeuser Verlag, Basel.

11.10 The Euler-Maclaurin Summation Formula 169

Exercises 10.1 The spiral of Theodorus is composed of rectangular triangles of sides 1, y'n, and In + 1. It performs a complete rotation after 17 triangles (this seems to be the reason why Theodorus did not consider roots beyond Jl7). No longer prevented by such scruples, we now want to know how many rotations a billion such triangles perform. This requires the calculation of (see Fig. 10.3)

1+

1 27r

1

1000000000

L

arctan r:

i = 18

v~

with an error smaller than 1. This exercise is not only a further occasion to admire the power of the Euler-Maclaurin formula, but also leaves us with an interesting integral to evaluate.

FIGURE 10.3. The spiral of Theodorus of Cyrene, 470-390 B.C.

10.2 (Formula for the Taylor series of tanx). If we let cotx = l/tanx and cothx = 1/ tanh x, Eq. (10.9) can be seen to represent the Taylor series of (x/2) coth(x/2). This allows us to obtain the series expansion of.7.: . coth x , and, by letting x ~ ix, that of x . cot x. Finally, use the formula

2· cot2x = cotx - tan x and obtain the coefficients of the expansion of tan x. Compare it with Eq. (1.4.18). 10.3 Verify numerically the estimates (Lehmer 1940)

IB3(X)1 ::; 0.04812, IBg(x)1 ::; 0.04756, for 0 ::; x ::; 1.

IB5(X)1 ::; 0.02446, IBll(X)1 ::; 0.13250,

IB7(X)1 ::; 0.02607, IB13(X)1 ::; 0.52357

III Foundations of Classical Analysis

. . . I am not sure that I shall still do geometry ten years from now. I also think that the mine is already almost too deep, and must sooner or later be abandoned. Today, Physics and Chemistry offer more brilliant discoveries and which are easier to exploit . . . (Lagrange, Sept. 21, 1781, Letter to d’Alembert, Oeuvres, vol. 13, p. 368)

Euler’s death in 1783 was followed by a period of stagnation in mathematics. He had indeed solved everything: an unsurpassed treatment of infinite and differential calculus (Euler 1748, 1755), solvable integrals solved, solvable differential equations solved (Euler 1768, 1769), the secrets of liquids (Euler 1755b), of mechanics (Euler 1736b, Lagrange 1788), of variational calculus (Euler 1744), of algebra (Euler 1770), unveiled. It seemed that no other task remained than to study about 30,000 pages of Euler’s work. The “Th´eorie des fonctions analytiques” by Lagrange (1797), “freed from all considerations of infinitely small quantities, vanishing quantities, limits and fluxions”, the thesis of Gauss (1799) on the “Fundamental Theorem of Algebra” and the study of the convergence of the hypergeometric series (Gauss 1812) mark the beginning of a new era. Bolzano points out that Gauss’s first proof is lacking in rigor; he then gives in 1817 a “purely analytic proof of the theorem, that between two values which produce opposite signs, there exists at least one root of the equation” (Theorem III.3.5 below). In 1821, Cauchy establishes new requirements of rigor in his famous “Cours d’Analyse”. The questions are the following: – What is a derivative really? Answer: a limit. – What is an integral really? Answer: a limit. – What is an infinite series a1 + a2 + a3 + . . . really? Answer: a limit.

171

This leads to – What is a limit? Answer: a number. And, finally, the last question: – What is a number? Weierstrass and his collaborators (Heine, Cantor), as well as M´eray, answer that question around 1870–1872. They also fill many gaps in Cauchy’s proofs by clarifying the notions of uniform convergence (see picture below), uniform continuity, the term by term integration of infinite series, and the term by term differentiation of infinite series. Sections III.5, III.6, and III.7, on, respectively, the integral calculus, the differential calculus, and infinite power series, will be the heart of this chapter. The preparatory Sections III.1 through III.4 will enable us to build our theories on a solid foundation. Section III.8 completes the integral calculus and Section III.9 presents two results of Weierstrass on continuous functions that were both spectacular discoveries of the epoch.

Weierstrass explains uniform convergence to Cauchy who meditates over Abel’s counterexample (Drawing by K. Wanner)

172 III. Foundations of Classical Analysis

III.1 Infinite Sequences and Real Numbers If, for every positive integer n, we have given a number sn , then we speak of an (infinite) sequence and we write (1.1)

{sn } = {s1 , s2 , s3 , s4 , s5 , . . .}.

The number sn is called the nth term or the general term of the sequence. A first example is (1.2)

{1, 2, 3, 4, 5, 6, . . .},

which is an arithmetic progression. This means that the difference of two successive terms is constant. The sequence (1.3)

{q 0 , q 1 , q 2 , q 3 , q 4 , q 5 , . . .}

is a geometric progression (the quotient of two successive terms is constant).

Convergence of a Sequence One says that a quantity is the limit of another quantity, if the second approaches the first closer than any given quantity, however small . . . (D’Alembert 1765, Encyclop´edie, tome neuvieme, a` Neufchastel.) When a variable quantity converges towards a fixed limit, it is often useful to indicate this limit by a specific notation, which we shall do by setting the abbreviation lim in front of the variable in question . . .

(Cauchy 1821, Cours d’Analyse)

If the terms sn of a sequence (1.1) approach arbitrarily closely a number s for n large enough, we call this number the limit of (1.1). This concept is very important and calls for more precision: – “arbitrarily closely” means “closer than any positive number ε”, i.e., |sn − s| < ε. Here, | · | is the absolute value and forces sn to be close to s in the positive and the negative direction. – “for n large enough” means that there must be an N such that the above estimate is true for all n ≥ N . With the symbols ∀ (“for all”) and ∃ (“there exists”), we can thus express the above situation in the following compact form. (1.1) Definition (D’Alembert 1765, Cauchy 1821). We say that a sequence (1.1) converges if there exists a number s such that (1.4) We then write

∀ε > 0 ∃N ≥ 1 ∀n ≥ N

|sn − s| < ε.

III.1 Infinite Sequences and Real Numbers 173 1.5

s2

1.4 s6

1.3

s10 1.2

s3

1.1

s7 s5

1.0 s 1

s11 s13

s9 s8

.9

s14

s12

s15

s18 s19

ε

s17

ε

s

s16

N

s4

FIGURE 1.1. Convergence of the sequence (1.6)

(1.5)

s = lim sn n→∞

or

sn → s.

If (1.4) is not true for any s, the sequence (1.1) is said to diverge. (1.2) Examples. Consider the sequence o n1 2 3 4 5 , , , , , ... , 2 3 4 5 6

where

sn =

n . n+1

This sequence converges to 1, because n 1 |sn − 1| = − 1 = 1/ε − 1. Therefore, for a given ε > 0, we can take for N an integer that is larger than 1/ε − 1 and condition (1.4) is verified. As the next example, we choose the sequence 1 1 1 s3 = 1 + − , s2 = 1 + , 2 2 3 n X 1 1 1 1 s4 = 1 + − − , ... , sn = (−1)[(i−1)/2] 2 3 4 i i=1

s1 = 1, (1.6)

(here [i/2] denotes the largest integer k not exceeding i/2; i.e., [i/2] = k if i = 2k or i = 2k + 1). This sequence is somewhat less trivial and is illustrated in Fig. 1.1. It seems to converge to a number close to 1.13 (which we guess, after our experience of Chap. I, to be π/4 + ln 2/2). We observe that for a given ε (here ε = 0.058), there is a last sn (here s16 ) violating |sn − s| < ε. Hence, for N = 17, (1.4) is satisfied. The fact that several earlier terms (s3 , s5 , . . .) also satisfy this estimate does not contradict (1.4).

174 III. Foundations of Classical Analysis

(1.3) Theorem. If a sequence {sn } converges, then it is bounded, i.e., (1.7)

∃B ∀n ≥ 1

|sn | ≤ B.

Proof. We put ε = 1. By the definition of convergence, we know the existence of an integer N such that |sn − s| < 1 for all n ≥ N . Using the triangle inequality (see Exercise 1.1), we obtain |sn | = |sn −s+s| ≤ |sn −s|+|s| < 1+|s| for n ≥ N and the statement is proved with B = max {|s1 |, |s2 |, . . . , |sN −1 |, |s| + 1}. ⊔ ⊓ For the boundedness of a sequence it is not necessary that it converge. For example, the sequence  (1.8) {sn } = 1, 0, 1, 0, 1, 0, 1, 0, . . .

is bounded (with B = 1) but does not converge. The sequence (1.2) is neither bounded nor does it converge. The general arithmetic progression  (1.9) {sn } = d, 2d, 3d, 4d, 5d, . . .

is also unbounded (for d 6= 0). For d > 0 this sequence satisfies (1.10)

∀M > 0 ∃N ≥ 1 ∀n ≥ N

sn > M.

To see this, take an integer N satisfying N > M/d. If (1.10) is verified, we say that the sequence {sn } tends to infinity and we write or

lim sn = ∞

n→∞

sn → ∞.

In a similar way, one can define limn→∞ sn = −∞. We next investigate the convergence of sequence (1.3). (1.4) Lemma. For the geometric progression (1.3), we have   0 for |q| < 1, lim q n = 1 for q = 1, n→∞  ∞ for q > 1. The sequence (1.3) diverges for q ≤ −1.

Proof. Let us start with the case q > 1. We write q = 1 + r (with r > 0) and apply Theorem I.2.1 to obtain q n = (1 + r)n = 1 + nr +

n(n − 1) 2 r + . . . ≥ 1 + nr. 2

Therefore, the terms q n tend to infinity (for a given M choose N ≥ M/r in (1.10)). The statement is trivial for q = 1. For |q| < 1 we consider the sequence sn = (1/|q|)n , which tends to infinity by the above considerations. For a given ε > 0 we put M = 1/ε and apply (1.10)

III.1 Infinite Sequences and Real Numbers 175

to the sequence {sn }. This proves the existence of an integer N such that for all n ≥ N we have sn > M or equivalently |q n | < ε. This proves that q n → 0. For q = −1 the sequence oscillates between −1 and 1 and for q < −1 it is unbounded and oscillating. ⊔ ⊓ The following theorem simplifies the computation of limits. (1.5) Theorem. Consider two convergent sequences sn → s and vn → v. Then, the sum, the product, and the quotient of the two sequences, taken term by term, converge as well, and we have (1.11) (1.12) (1.13)

lim (sn + vn ) = s + v

n→∞

lim (sn · vn ) = s · v s  s n if lim = n→∞ vn v n→∞

vn 6= 0 and v 6= 0.

Proof. We begin with the proof of (1.11). We estimate |(sn + vn ) − (s + v)| = |sn − s + vn − v| ≤ |sn − s| + |vn − v| < 2ε = ε′ | {z } | {z } 0 such that 2ε = ε′ . By hypothesis, the two sequences {sn } and {vn } converge to s and v. This means that there exist N1 and N2 such that |sn − s| < ε for n ≥ N1 and |vn − v| < ε for n ≥ N2 . If we choose N = max(N1 , N2 ), we see that (1.4) is satisfied for the sequence {sn + vn }. Once we are accustomed to this argument, repeating these explanations will not be necessary. For the proof of (1.12) we have to estimate sn vn −sv. Let us add and subtract “mixed products” −svn + svn such that |sn vn − sv| = |sn vn − svn + svn − sv| ≤ |vn | · |sn − s| + |s| · |vn − v| < (B + |s|) ε = ε′ . Here, we have used Theorem 1.3 for the sequence {vn }. It is sufficient to prove (1.13) for the special case where sn = 1 for all n, and hence s = 1. The general result will then follow from (1.12) because sn /vn is the product of (1/vn ) and sn . We first observe that the values of |vn | cannot become arbitrarily small. Indeed, if we put ε = |v|/2 in the definition of convergence, we obtain |vn − v| < |v|/2 (and hence also |vn | > |v|/2) for sufficiently large n. With this estimate, we now obtain 1 − vn

|v − vn | 2|vn − v| 1 2ε ≤ ≤ = ε′ . = v |vn | · |v| |v|2 |v|2

⊔ ⊓

176 III. Foundations of Classical Analysis

(1.6) Theorem. Assume that a sequence {sn } converges to s and that sn ≤ B for all sufficiently large n. Then, the limit also satisfies s ≤ B.

Proof. We shall show that s > B leads to a contradiction. For this we put ε = s − B > 0 and use (1.4). This implies that for sufficiently large n we have s − sn ≤ |sn − s| < ε = s − B, so that sn > B, which is in contradiction to our assumption.

⊔ ⊓

Remark. The analogous result for strict inequalities (sn < B for all n implies s < B) is wrong. This is seen by the counterexample sn = n/(n + 1) < 1 with limn→∞ sn = 1. Cauchy Sequences. Let us now tackle an important problem. The definition of convergence (1.4) forces us to estimate |sn − s|; the limit s has to be known. But what can we do if the limit s is unknown, or, as in Example (1.6), is not known to arbitrary precision? It is then impossible to estimate with rigor |s − sn | < ε for any ε > 0. To bypass this obstacle, Cauchy had the idea of replacing |sn − s| < ε in (1.4) by |sn − sn+k | < ε for all the successors sn+k of sn . (1.7) Definition. A sequence {sn } is a Cauchy sequence if (1.14)

∀ε > 0 ∃N ≥ 1 ∀n ≥ N ∀k ≥ 1

s2

s2

1.4

1.4 s6

1.2

|sn − sn+k | < ε.

s6 s10

s3

s7 s5

1.0 s 1

s9 s8

s4

s14

s18 s19

s11 s15 s13 s17 s16 s 12

N

ε

1.2

s10 s3

s7 s5

ε 1.0 s 1

s9 s8

s4

s14

s11

s18 s19

s15 s13 s17 s16 s

ε ε

12

N

FIGURE 1.2. Sequence (1.6) as a Cauchy sequence

Example. Fig. 1.2 illustrates condition (1.14) for the sequence (1.6). We see that, e.g., for ε = 0.11 condition (1.14) is satisfied for n ≥ 17. Similarly, it is also seen that (1.14) is true for any ε > 0, because 1/(n + 2) + 1/(n + 3) tends to zero. (1.8) Theorem (Cauchy 1821). A sequence {sn } of real numbers is convergent (with a real number as limit) if and only if it is a Cauchy sequence.

III.1 Infinite Sequences and Real Numbers 177

It is an immediate consequence of |sn − sn+k | ≤ |sn − s| + |s − sn+k | < 2ε that convergent sequences must be Cauchy sequences. A rigorous proof of the converse implication, beyond Cauchy’s intuition, is only possible after having understood the concept of irrational and real numbers. In contrast to the results obtained until now (Theorems 1.3, 1.5, and 1.6), Theorem 1.8 is not true in the setting of rational numbers. Consider, for example, the sequence  (1.15) 1 , 1.4 , 1.41 , 1.414 , 1.4142 , 1.41421 , . . . .

It is indeed a Cauchy sequence (we have |sn − sn+k | < 10−n+1 ), but its limit is not rational.

√ 2

Construction of Real Numbers The more I meditate on the principles of the theory of functions — and I do this unremittingly — the stronger becomes my conviction that the foundations upon which these must be built are the truths of Algebra . . . (Weierstrass 1875, Werke, vol. 2, p. 235) Please forget everything you have learned in school; for you haven’t learned it. . . . My daughters have been studying (chemistry) for several semesters already, think they have learned differential and integral calculus in school, and even today don’t know why x · y = y · x is true. (Landau 1930, Engl. transl. 1945) √ 3 is thus only a symbol for a number which has yet to be found, but is not its definition. This definition is, however, satisfactorily given by my method as, say (1.7, 1.73, 1.732, . . . .) (G. Cantor 1889) . . . the definition of irrational numbers, on which geometric representations have often had a confusing influence. . . . I take in my definition a purely formal point of view, calling some given symbols numbers, so that the existence of these numbers is beyond doubt. (Heine 1872) At that point, my sense of dissatisfaction was so strong that I firmly resolved to start thinking until I should find a purely arithmetic and absolutely rigorous foundation of the principles of infinitesimal analysis. . . . I achieved this goal on November 24th, 1858, . . . but I could not really decide upon a proper publication, because, firstly, the subject is not easy to present, and, secondly, the material is not very fruitful. (Dedekind 1872) Demeaning Analysis to a mere game with symbols . . . (Du Bois-Reymond, Allgemeine Funktionentheorie, T¨ubingen 1882)

For many decades nobody knew how irrational numbers should be put into a rigorous mathematical setting, how to grasp correctly what should be the “ultimate term” of a Cauchy sequence such as (1.15). This “Gordian knot” was finally resolved independently by Cantor (1872), Heine (1872), M´eray (1872) (and similarly by Dedekind 1872) by the following audacious idea: the whole Cauchy sequence is declared “to be” the real number in question (see quotations). This means that we associate to a Cauchy sequence of rational numbers sn (henceforth called a rational Cauchy sequence) a real number.

178 III. Foundations of Classical Analysis

This seems to resolve Theorem 1.8 in an elegant manner. But there remains much to do: we shall have to identify different rational Cauchy sequences that represent the same real number, define algebraic and order relations for these new objects, and finally we shall find the proof of Theorem 1.8 more complicated than we might have thought, because the terms sn in (1.14) may now themselves be real numbers, i.e., rational Cauchy sequences. All these details have been worked out in full detail by Landau (1930) in a famous book, where he admits himself that many parts are “eine langweilige M¨uhe”. Equivalence Relation. Suppose that √ 2 is associated to {1.4 ; 1.41 ; 1.414 ; . . .} √ 3 is associated to {1.7 ; 1.73 ; 1.732 ; . . .}, √ √ then 2 · 3 should be associated to the sequence of the products {2.38 ; 2.4393 ; 2.449048, . . .}. √ On the other hand, 6 is also associated to {2.4 ; 2.44 ; 2.449 ; . . .}. So we have to identify the two sequences. Two rational Cauchy sequences {sn } and {vn } are called equivalent, if limn→∞ (sn − vn ) = 0, i.e., if (1.16)

∀ε > 0 ∃N ≥ 1 ∀n > N

|sn − vn | < ε.

We then write {sn } ∼ {vn }. It is not difficult to check that (1.16) defines an equivalence relation on the set of all rational Cauchy sequences. This means that we have {sn } ∼ {sn } {sn } ∼ {vn }

(reflexive) =⇒ {vn } ∼ {sn }

{sn } ∼ {vn }, {vn } ∼ {wn }

=⇒

(symmetric) {sn } ∼ {wn }

(transitive).

Therefore, it is possible to partition the set of rational Cauchy sequences into equivalence classes, n o {sn } = {vn } {vn } is a rational Cauchy sequence and {vn } ∼ {sn } . Elements of equivalence classes are called representatives.

(1.9) Definition. Real numbers are equivalence classes of rational Cauchy sequences, i.e., n o R = {sn } {sn } is a rational Cauchy sequence .

The set Q of rational numbers can be interpreted as a subset of R in the following way: if r is an element of Q (abbreviated: r ∈ Q), then the constant

III.1 Infinite Sequences and Real Numbers 179

sequence {r, r, r, . . .} is a rational Cauchy sequence. Hence, we identify the rational number r with the real number {r, r, . . .}. Addition and Multiplication. In order to be able to work with R, we have to define the usual operations. Let s = {sn } and v = {vn } be two real numbers. We then define their sum (difference), product (quotient) by (1.17)

s + v := {sn + vn },

s · v := {sn · vn }.

We have to take some care with this definition. First of all, we have to ensure that the sequences {sn + vn } and {sn ·vn } are rational Cauchy sequences (this follows from |(sn + vn ) − (sn+k + vn+k )| ≤ |sn − sn+k | + |vn − vn+k | for the sum and is obtained as in the proof of Theorem 1.5 for the product). Then, we have to prove that (1.17) is well-defined. If we choose different representatives of the equivalence classes s and v, say {s′n } and {vn′ }, then the result s + v has to be the same. For this we have to prove that sn − s′n → 0 and vn − vn′ → 0 imply (sn + vn ) − (s′n + vn′ ) → 0 and (sn · vn ) − (s′n · vn′ ) → 0. But this is obtained exactly as in the proof of Theorem 1.5. In a next step, we have to verify the known rules of computation with real numbers (commutativity, associativity, distributivity). Here begins Landau’s “langweilige M¨uhe”. We omit these details and refer the reader either to Landau’s marvelous book or to any introductory algebra text. Order. Let s = {sn } and v = {vn } be two real numbers. We then define (1.18)

s < v :⇐⇒

∃ ε′ > 0 ∃ M ≥ 1 ∀ m ≥ M

s ≤ v :⇐⇒ s < v or s = v

sm ≤ vm − ε′ ,

(here the number ε′ has to be rational in order to avoid an ambiguous definition). The rather complicated definition of s < v means that for sufficiently large m the elements sm and vm have to be well separated. It also implies that the relation is well defined. Obviously, it is not sufficient to require sm < vm (the sequences {1, 1/2, 1/3, 1/4, . . .} and {0, 0, 0, . . .} both represent the real number 0 and serve as a counterexample). The relation s ≤ v of (1.18) defines an order relation. This means that s≤s

(reflexive)

s ≤ v, v ≤ w s ≤ v, v ≤ s

=⇒ s ≤ w (transitive) =⇒ s = v (antisymmetric).

We just indicate the proof of antisymmetry. Suppose that s ≤ v and v ≤ s, but s 6= v. Then, there exist positive rational numbers ε′1 and ε′2 such that sm ≤ vm − ε′1 for m ≥ M1 and vm ≤ sm − ε′2 for m ≥ M2 . Hence, for m ≥ max(M1 , M2 ), we have ε′2 ≤ sm − vm ≤ −ε′1 , which is a contradiction. (1.10) Lemma. The order ≤ of (1.18) is total, i.e., for any two real numbers s and v with s 6= v we have either s < v or v < s.

180 III. Foundations of Classical Analysis

Remark. s 6= v is the negation of s = v, which is expressed by Eq. (1.16). In order to formulate the negation of a statement like (1.16), we recall a little bit of logic. Let S(x) be a statement depending on x ∈ A (A is some set) and ¬S(x) its negation. Then, we have 2 ∀ x ∈ A S(x)

∃ x ∈ A S(x)

is the negation of

∃ x ∈ A ¬S(x),

is the negation of

∀ x ∈ A ¬S(x).

In order to obtain the negation of a long statement we have to reverse all quantifiers ( ∀ ↔ ∃ ) and replace the final statement by its negation. Hence, s 6= v is obtained from (1.16) as (1.19)

∃ε > 0 ∀N ≥ 1 ∃n ≥ N

|sn − vn | ≥ ε.

Proof of Lemma 1.10. Let s = {sn } and v = {vn } be two distinct real numbers, such that (1.19) holds. We then put ε′ = ε/3. Since {sn } and {vn } are Cauchy sequences, there exists N1 such that |sn − sn+k | < ε′ for n ≥ N1 and k ≥ 1 and there exists N2 such that |vn − vn+k | < ε′ for n ≥ N2 and k ≥ 1. We then put N = max(N1 , N2 ) and deduce from (1.19) the existence of an integer n ≥ N such that |sn − vn | ≥ ε. There are two possibilities, (1.20)

or

sn − vn ≥ ε

vn − sn ≥ ε.

≥ε

≥ε

ε′

ε′ vn

≥ ε′

ε′

ε′ sn

sn

≥ ε′

vn

FIGURE 1.3. Illustration of the two cases in (1.20)

For k ≥ 1 the numbers sn+k and vn+k stay in the disks of radius ε′ = ε/3 (see Fig. 1.3). Therefore, (1.18) is satisfied with M = N and we have s > v in the first case, whereas v > s in the second case. ⊔ ⊓

Absolute Value. Once we have shown that the order is total (Lemma 1.10), it is possible to define the absolute value of a number s as being s (for s ≥ 0) and −s (for s < 0). An easy consequence of this definition is that (1.21)

|s| = {|sn |}

for

s = {sn }.

The triangle inequality |s + v| ≤ |s| + |v| and all its consequences are valid for real numbers. Remark. In the Definitions and Theorems 1.1 through 1.7, we have not been very precise about the concept of “number”. To be logically correct, they should have 2

The statement “all (∀) polar bears are white” is wrong if there exists (∃) at least one colored (nonwhite) polar bear; and vice versa.

III.1 Infinite Sequences and Real Numbers 181

been stated only for rational numbers. After having now introduced with much pain the concept of real numbers, we can extend these definitions to real numbers and check that the statements of the theorems remain valid also in the more general context. Proof of Theorem 1.8. . . . until now these propositions were considered axioms. (M´eray 1869, see Dugac 1978, p. 82)

Let {si } be a Cauchy sequence of real numbers, such that each si itself is an equivalence class of rational Cauchy sequences, i.e., si = {sin }n≥1 . The idea is to choose for each i a number becoming smaller and smaller (for example 1/2i) and to apply the definition of a rational Cauchy sequence in order to obtain ∃ Ni ≥ 1 ∀ n ≥ Ni ∀ k ≥ 1 |sin − si,n+k |
0, there is an αn satisfying αn > ξ − ε. Since αn is not an upper bound of X, ξ − ε cannot be one either. ⊓ ⊔ (1.13) Theorem. Consider a sequence {sn } that is monotonically increasing (sn ≤ sn+1 ) and majorized (sn ≤ B for all n). Then, it converges to a real limit. Proof. By hypothesis, the set X = {s1 , s2 , s3 , . . .} is nonempty and majorized (see Fig. 1.5). Therefore, ξ = sup X exists by Theorem 1.12. By the definition of sup X, the value ξ − ε is, for a given ε > 0, not an upper bound of X. Consequently, there exists an N such that sN > ξ − ε. Since X is majorized by ξ, we have ξ − ε < sN ≤ sN +1 ≤ sN +2 ≤ sN +3 ≤ . . . ≤ ξ, so that ξ − ε < sn ≤ ξ (and thus |sn − ξ| < ε) for all n ≥ N . This proves the convergence of {sn } to ξ. ⊔ ⊓

(1.14) Corollary. Consider two sequences {sn } and {vn }. Suppose that {sn } is monotonically increasing (sn ≤ sn+1 ) and that sn ≤ vn for all (sufficiently large) n. Then, we have {vn } converges {sn } diverges

=⇒ =⇒

{sn } converges, {vn } diverges.

Proof. If {vn } converges, then it is bounded by Theorem 1.3. Hence, {sn } is also bounded and its convergence follows from Theorem 1.13. The second line is the logical reversion of the first one. ⊔ ⊓ Remark. In an analogous way, we define the lower bound of a set, we define minorized and monotonically decreasing sequences, and we use the notation

184 III. Foundations of Classical Analysis

(1.23)

ξ = inf X

for the greatest lower bound or infimum of X (i.e., x ≥ ξ for all x ∈ X and ∀ ε > 0 ∃ x ∈ X with x < ξ + ε). There are theorems analogous to Theorems 1.12 and 1.13.

Accumulation Points I find it really surprising that Mr. Weierstrass and Mr. Kronecker can attract so many students — between 15 and 20 — to lectures that are so difficult and at such a high level. (letter of Mittag-Leffler 1875, see Dugac 1978, p. 68)

The sequence (1.24)

1 3,

⌣

2 3,

⌣

1 4,

⌣

3 4,

⌣

1 5,

⌣

4 5,

⌣

1 6,

⌣

5 6,

⌣

1 7,

⌣

6 7,

⌣

... →0 →1

does not converge, but if every other term is removed, it converges either to 0 or to 1. A sequence with missing terms is a “subsequence”. More precisely, (1.15) Definition. A sequence {s′n } is called subsequence of {sn } if there exists an increasing mapping σ : N → N with s′n = sσ(n) (increasing means that σ(n) < σ(m) if n < m). (1.16) Definition. A point s is called an accumulation point of a sequence {sn }, if there exists a subsequence converging to s. Examples. The points 0 and 1 are accumulation points of the sequence (1.24). An interesting example is the sequence n1 1 2 1 2 3 1 2 3 4 1 2 3 4 5 1 2 3 4 5 6 1 o (1.25) , , , , , , , , , , , , , , , , , , , , , ,... , 2 3 3 4 4 4 5 5 5 5 6 6 6 6 6 7 7 7 7 7 7 8

which admits all numbers between 0 and 1 (0 and 1 included) as accumulation points. To see that, for example, ln 2 is an accumulation point of (1.25), consider the sequence n 6 69 693 6931 69314 693147 o , , , , , ,... . 10 100 1000 10000 100000 1000000

It is certainly included somewhere in (1.25) and converges to ln 2. The unbounded sequences {1, 2, 3, 4, 5, . . .}, {−1, −2, −3, −4, −5, . . .} and {1, −1, 2, −2, 3, −3, 4, −4, . . .}, don’t have accumulation points. (1.17) Theorem of Bolzano-Weierstrass (Weierstrass’s lecture of 1874). A bounded sequence {sn } has at least one accumulation point.

Proof. Weierstrass’s original proof used bisection, as in the proof of Theorem 1.12. Having this theorem at our disposal, we consider the set

III.1 Infinite Sequences and Real Numbers 185

X

ξ − 1/n

ξ + 1/n ξ = sup X

FIGURE 1.6. Proof of the theorem of Bolzano-Weierstrass

(1.26)

 X = x sn > x for infinitely many n ,

and simply put ξ = sup X, which will turn out to be an accumulation point (see Fig. 1.6). This number exists because X is nonempty and majorized (the sequence {sn } is bounded). By definition of the supremum, only a finite number of sn can satisfy sn ≥ ξ + ε and there is an infinity of terms sn that are larger than ξ − ε (ε is an arbitrary positive number). Hence, an infinity of terms sn lie in the interval [ξ − ε, ξ + ε]. We now choose arbitrarily an element of the sequence that lies in [ξ −1, ξ +1] and we denote it by s′1 = sσ(1) . Then, we choose an element in [ξ − 1/2, ξ + 1/2] whose index is larger than σ(1) (this is surely possible since there must be infinitely many) and we denote it by s′2 = sσ(2) . At the nth step, we choose for s′n = sσ(n) an element of the sequence that lies in [ξ − 1/n, ξ + 1/n] and whose index is larger than σ(n − 1). The subsequence obtained in this way converges to ξ, because |s′n − ξ| ≤ 1/n. ⊔ ⊓ Remark. This proof did not exhibit an arbitrary accumulation point but precisely the largest accumulation point. We call it the “limit superior” of the sequence and we denote it by  (1.27) ξ = lim sup sn = sup x ∈ R sn > x for infinitely many n n→∞

(see also Exercise 1.12). The smallest accumulation point is denoted by  (1.28) ξ = lim inf sn = inf x ∈ R sn < x for infinitely many n . n→∞

 Example. For the sequence 23 , − 12 , 43 , − 31 , 54 , − 14 , 65 , − 51 , 76 , − 16 , . . . , we have lim supn→∞ sn = 1, lim inf n→∞ sn = 0, sup{sn } = 3/2, inf{sn } = −1/2.

Exercises 1.1 (Triangle inequality). Show, by discussing all possible combinations of signs, that for any two real numbers u and v we have (1.29)

|u + v| ≤ |u| + |v|.

186 III. Foundations of Classical Analysis

Then, show that for any three real numbers u, v, and w we have (1.29′)

|u − w| ≤ |u − v| + |v − w|.

1.2 Show that the sequence {sn } with sn =

2n − 1 n+3

converges to s = 2. For a given ε > 0, say for ε = 10−5 , find a number N such that |sn − s| < ε for n ≥ N . 1.3 Show that the sequences sn =

1 1 1 1 1 + + + + ...+ 1 · 5 3 · 7 5 · 9 7 · 11 (2n − 1)(2n + 3)

sn =

1 1 1 1 + + + ...+ 1·2·3 2·3·4 3·4·5 n(n + 1)(n + 2)

are Cauchy sequences and find their limits. Hint. Decompose the rational functions into partial fractions. 1.4 Construct sequences sn and vn with lim sn = ∞ and lim vn = 0 to n→∞ n→∞ illustrate each of the following possibilities. a) lim (sn · vn ) = ∞; n→∞

b) lim (sn · vn ) = c, where c is an arbitrary constant; and n→∞

c) sn · vn is bounded but not convergent. 1.5 Consider the three sequences sn =

q √ √ √ √ n + 1000− n, vn = n + n − n, un =

r n+

√ n − n. 1000

Show that sn > vn > un for n < 106 and compute lim sn , lim vn , n→∞ n→∞ lim un , if they exist. Arrange these limits in increasing order. n→∞

1.6 Show with the help of the estimates of Exercise I.2.5 that  1 n vn = 1 + n

is a Cauchy sequence. Find, for ε = 10−5 , an integer N such that |vn − vn+k | < ε for n ≥ N and k ≥ 1. 1.7 For two rational Cauchy sequences {an } and {bn }, we denote by {an · bn } the sequence formed by the products term by term. Show a) the sequence {an · bn } is again a Cauchy sequence; and b) if {an } ∼ {sn } and {bn } ∼ {vn } as defined in (1.16), then {an · bn } ∼ {sn · vn }. This shows that the product of two real numbers defined in (1.17) is independent of the choice of the representatives.

III.1 Infinite Sequences and Real Numbers 187

1.8 Show the following: if s is the only accumulation point of a bounded sequence {sn }, then the sequence is convergent and limn→∞ sn = s. Show by a counterexample that this property is not true for unbounded sequences. 1.9 (Cauchy 1821, p. 59; also called “Ces`aro summation”). Let limn→∞ an = a and n 1X bn = ak . n k=1

Show that limn→∞ bn = a. √ 1.10 Let α be an irrational number (for example, α = 2 ). Consider the sequence {sn } defined by sn = (nα) mod 1, i.e., sn ∈ (0, 1) is nα with the integer part removed. Compute s1 , s2 , s3 , s4 , . . . and sketch these values. Show that every point in [0, 1] is an accumulation point of this sequence. Hint. For ε > 0 and n ≥ 1/ε at least two points among s1 , s2 , . . . , sn+1 (call them sk and sk+ℓ ) are closer than ε. Then, the points sk , sk+ℓ , sk+2ℓ , . . . form a grid with mesh size < ε. Remark. At the beginning of the computer era, this procedure was the standard method for creating pseudo random numbers. 1.11 Let {sn } and {vn } be two bounded sequences. Show that lim sup (sn + vn ) ≤ lim sup sn + lim sup vn n→∞

n→∞

n→∞

lim inf (sn + vn ) ≥ lim inf sn + lim inf vn . n→∞

n→∞

n→∞

Show with the help of examples that the inequality can be strict. 1.12 Prove that for a sequence {sn } we have  lim sup sn = lim vn , where vn = sup sn , sn+1 , sn+2 , . . . . n→∞

n→∞

1.13 Compute all accumulation points of the sequence

 {sn } = p11 , p21 , p22 , p31 , p32 , p33 , p41 , p42 , . . . ,

pkℓ =

k X 1 . i2 i=ℓ

Show that (see Eq. (I.5.23)) lim sup sn = π /6 and that lim inf sn = 0 (see Fig. 1.7). 2

0

π2/6 − 1

1

π2/6

FIGURE 1.7. Sequence with a countable number of accumulation points

188 III. Foundations of Classical Analysis

III.2 Infinite Series I shall devote all my efforts to bring light into the immense obscurity that today reigns in Analysis. It so lacks any plan or system, that one is really astonished that so many people devote themselves to it — and, still worse, it is absolutely devoid of any rigour. (Abel 1826, Oeuvres, vol. 2, p. 263) Cauchy is mad, and there is no way of being on good terms with him, although at present he is the only man who knows how mathematics should be treated. What he does is excellent, but very confused . . . (Abel 1826, Oeuvres, vol. 2, p. 259)

Since Newton and Leibniz, infinite series (2.1)

a0 + a1 + a2 + a3 + . . .

have been the universal tool for all calculations (see Chap. I). We will make precise here what (2.1) really represents. The idea is to consider the sequence {sn } of partial sums (2.2)

s0 = a 0 ,

s1 = a0 + a1 ,

...,

sn =

n X

ai ,

i=0

and to apply the definitions and results of the preceding section. A classical reference for infinite series is the book of Knopp (1922). (2.1) Definition. We say that the infinite series (2.1) converges, if the sequence {sn } of (2.2) converges. We write ∞ X i=0

ai = lim sn n→∞

or

X i≥0

ai = lim sn . n→∞

FIGURE 2.1. “Geometric” view of the geometric series

(2.2) Example. Consider the geometric series whose nth partial sum is given by sn = 1 + q + q 2 + . . . + q n (see Fig. 2.1). Multiplying this expression by 1 − q, most terms cancel, and we get (for q 6= 1) sn = 1 + q + q 2 + . . . + q n =

1 − q n+1 . 1−q

III.2 Infinite Series 189

From Lemma 1.4, together with Theorem 1.5, we thus have  1    1−q 1 + q + q2 + q3 + q4 + q5 + . . . = diverges → ∞    diverges

if |q| < 1, if q ≥ 1, if q ≤ −1.

Criteria for Convergence Usually it is not possible to find a simple expression for sn and it is difficult to compute explicitly the limit of {sn }. In this case, it is natural to apply Cauchy’s criterion of Theorem 1.8 to the sequence of partial sums. Since sn+k − sn = an+1 + an+2 + . . . + an+k , we get (2.3) Lemma. The infinite series (2.1) converges to a real number if and only if ⊔ ∀ ε > 0 ∃ N ≥ 0 ∀ n ≥ N ∀ k ≥ 1 an+1 + an+2 + . . . + an+k < ε. ⊓ Putting k = 1 in this criterion, we see that

(2.3)

lim ai = 0

i→∞

is a necessary condition for the convergence of (2.1). However, (2.3) is not sufficient for the convergence of (2.1). This can be seen with the counterexample 1 1 1 1 1 1 1 1 1 1 + + + + + + + + + + . . . → ∞. 2 2 3 3 3 4 4 4 4 5 In what follows, we shall discuss some sufficient conditions for the convergence of (2.1). 1+

Leibniz’s Criterion. Consider an infinite series where the terms have alternating signs X (2.4) a 0 − a1 + a2 − a3 + a4 − . . . = (−1)i ai . i≥0

(2.4) Theorem (Leibniz 1682). Suppose that the terms ai of the alternating series (2.4) satisfy for all i ai > 0,

ai+1 ≤ ai ,

lim ai = 0;

i→∞

then, the series (2.4) converges to a real value s and we have the estimate (2.5)

|s − sn | ≤ an+1 ,

i.e., the error of the nth partial sum is not larger than the first neglected term.

190 III. Foundations of Classical Analysis

FIGURE 2.2. Proof of Leibniz’s criterion

Proof. Denote by sn the nth partial sum of (2.4). It then follows from the monotonicity assumption that s2k+1 = s2k−1 + a2k − a2k+1 ≥ s2k−1 and that s2k+2 = s2k − a2k+1 + a2k+2 ≤ s2k . From the positivity of a2k+1 , we have s2k+1 < s2k so that, by combining these inequalities, s1 ≤ s3 ≤ s5 ≤ s7 ≤ . . . ≤ s 6 ≤ s4 ≤ s2 ≤ s0 (see Fig. 2.2). Consequently, sn+k lies for all k between sn and sn+1 , and we have (2.6)

|sn+k − sn | ≤ |sn+1 − sn | = an+1 .

This implies the convergence of {sn } by Theorem 1.8, since an+1 tends to 0 for n → ∞. Finally, the estimate (2.5) is obtained by considering the limit k → ∞ in (2.6) (use Theorem 1.6). ⊔ ⊓ Examples. The convergence of (see (I.4.29) and (I.3.13a)) 1−

1 1 1 + − + ... 3 5 7

and

1−

1 1 1 + − + ... 2 3 4

is thus established. However, we have not yet rigorously proved that the first sum represents π/4 and the second one ln 2 (see Example 7.11 below). If a continued fraction (I.6.7) is converted into an infinite series, we obtain (see Eq. (I.6.16)) q0 +

p1 p1 p2 p1 p2 p3 p1 p2 p3 p4 − + − + ... . B1 B1 B2 B2 B3 B3 B4

Assuming that the integers pi and qi are positive, this is an alternating series (from the second term onward). Furthermore, we have Bk = qk Bk−1 + pk Bk−2 > pk Bk−2 , implying that the terms of the series are monotonically decreasing. Under the additional assumption that 0 < pi ≤ qi for all i ≥ 1 (see Theorem I.6.4), we have 2 Bk Bk−1 = qk Bk−1 + pk Bk−1 Bk−2 > 2pk Bk−1 Bk−2 and consequently also Bk Bk−1 > 2k−1 pk pk−1 ·. . .·p1 . This proves that the terms of the series tend to zero and, by Theorem 2.4, that the series under consideration converges.

III.2 Infinite Series 191

Majorizing or Minorizing a Series. For infinite series with non-negative terms the following criterion is extremely useful. (2.5) Theorem. Suppose that 0 ≤ ai ≤ bi for all (sufficiently large) i. Then P∞ P∞ =⇒ ai converges, i=0 bi converges Pi=0 P ∞ ∞ =⇒ i=0 ai diverges i=0 bi diverges.

Pn Pn Proof. Putting sn = i=0 ai and vn = i=0 bi , this result is an immediate consequence of Corollary 1.14. ⊔ ⊓

AsPa first application, we give an easy proof of the divergence of the harmonic series i≥1 1i (N. Oresme, around 1350; see Struik 1969, p. 320). We minorize this series as follows: P 1 1 1 bi = 1 + 12 + 31 + 14 + 15 + 16 + 17 + 18 + 91 + 10 + . . . + 16 + 17 + ... P 1 1 1 ai = 1 + 12 + 14 + 14 + 18 + 18 + 18 + 18 + 16 + 16 + . . . + 16 + 1 + ... . | {z } | {z } | {z } 32 1/2

1/2

1/2

P

P Since ai diverges, it follows from 0 < ai ≤ bi that the harmonic series bi diverges too. As a further example, we consider the series (I.2.18) for ex (e.g., for x = 10),

103 104 105 102 + + + + ... . 2! 3! 4! 5! We omit the first 10 terms (this does not influence the convergence), and compare the resulting series with the geometric series (Example 2.2 with q = 10/11 < 1)  1010 1011 1012 1010  10 10 · 10 10 · 10 · 10 + + + ... = 1+ + + + ... 10! 11! 12! 10! 11 11 · 12 11 · 12 · 13  10 102 103 1010  1+ + 2 + 3 + ... . ≤ 10! 11 11 11 The convergence of the geometric series implies the convergence of (2.7). Similarly, one can prove that the series (I.2.18) converges for all x. This comparison with the geometric series will be used on several occasions (see Criteria 2.10 and 2.11, Lemma 7.1, and Theorems 7.5 and 7.7). (2.7)

1 + 10 +

(2.6) Lemma. The series 1 1 1 1 1 + α + α + α + α + ... 1α 2 3 4 5 converges for all α > 1. It diverges for α ≤ 1.

(2.8)

Proof. The divergence of the series for α = 1 (harmonic series) has been established above. For α < 1 the individual terms become still larger, so that the series diverges by Theorem 2.5.

192 III. Foundations of Classical Analysis

We shall next prove the convergence of (2.8) for α = (k +1)/k, where k ≥ 1 is an integer. The idea is to consider the series X 1 1 1 1 1 (−1)i+1 √ 1− √ + √ − √ + √ − ... = , k k k k k 2 3 4 5 i i≥1

which converges by Leibniz’s criterion. The sum of two successive terms can be minorized as follows: √ √ k 2i − k 2i − 1 1 1 1 − √ (2.9) = √ ≥ Ck · (k+1)/k , √ √ k k k k i 2i − 1 2i 2i − 1 · 2i where Ck = 1/(k · 2(k+1)/k ) is a constant independent of i. The last inequality in (2.9) is obtained from the identity ak − bk = (a − b)(ak−1 + ak−2 b + ak−3 b2 + √ √ k k−1 ) with a = 2i and b = k 2i − 1 as follows: ... + b √ √ 1 1 k 2i − k 2i − 1 = ≥ . (k−1)/k (k−1)/k (2i) + . . . + (2i − 1) k · (2i)(k−1)/k

Thus, by Theorem 2.5, the series (2.8) converges for α = (k + 1)/k. Finally, for an arbitrary α > 1 there exists an integer k with α > (k + 1)/k. Theorem 2.5 applied once more then shows convergence for all α > 1. ⊔ ⊓

Absolute Convergence Example. The series 1 1 1 1 1 + − + − + ... 2 3 4 5 6 is convergent by Leibniz’s criterion (actually to ln 2). If we rearrange the series as follows: 1 1 1 1 1 1 1 1 1 1 1 − + − − + ... , 1− − + − − + − 2 4 3 6 8 5 10 12 7 14 16 | {z } | {z } | {z } | {z }

(2.10)

1−

1/2

1/6

1/10

1/14

we obtain  1 1 1 1 1 1 1 1 1 1 − + − + − ... = 1 − + − + − ... , 2 4 6 8 10 2 2 3 4 5 which is now half as much as originally. This shows that the value of an infinite sum can depend on the order of summation. P P∞ ′ (2.7) A seriesP ∞ i=0 ai is a rearrangement of i=0 ai , if every term PDefinition. ∞ ∞ of i=0 ai appears in i=0 a′i exactly once and conversely (this means that there exists a bijective mapping σ : N0 → N0 such that a′i = aσ(i) ; here N0 = {0, 1, 2, 3, 4, . . .}).

III.2 Infinite Series 193

Explanation. An elegant explanation for the above phenomenon was given by Riemann (1854, Werke, p. 235, “. . . ein Umstand, welcher von den Mathematikern des vorigen Jahrhunderts u¨ bersehen wurde . . .”). In fact, Riemann observed much more: for any given real number A it is possible to rearrange the terms of (2.10) in such a way that the resulting series converges to A. The reason is that the sum of the positive terms of (2.10) and the sum of the negative terms, 1+

1 1 1 1 + + + + ... 3 5 7 9

and

−

1 1 1 1 1 − − − − − ... , 2 4 6 8 10

are both divergent (or equivalently: the series (2.10) with each term replaced by its absolute value diverges). The idea is to take first the positive terms 1 + 1/3 + . . . until the sum exceeds A (this certainly happens because the series with positive terms diverges). Then, we take the negative terms until we are below A (this certainly happens because −1/2 − 1/4 − . . . diverges). Then, we go on adding positive terms until A is again exceeded, and so on. In this way, we obtain a rearranged series that converges to A (cf. examples in Fig. 2.3). 1.3 1.2 1.1 1.0 .9 .8 1.5 1.4 1.3 1.2 1.1 1.0

1.2 = 1 +

1 3

−

1 2

+

1 5

+

20

1.5 = 1 +

1 7

+

1 9

−

1 4

+

1 11

+ ...

40

1 3

+

1 5

−

20

1 2

+

1 7

+

60

1 9

+

1 11

+ ...

40

60

FIGURE 2.3. Rearrangements of the series (2.10)

(2.8) Definition. The series (2.1) is absolutely convergent if |a0 | + |a1 | + |a2 | + |a3 | + . . . converges. P (2.9) Theorem (Dirichlet 1837b). If the series ∞ i=0 ai is absolutely convergent, then all its rearrangements converge to the same limit. Proof. By Cauchy’s criterion, absolute convergence means that ∀ε > 0 ∃N ≥ 0 ∀n ≥ N ∀k ≥ 1

|an+1 | + |an+2 | + . . . + |an+k | < ε.

194 III. Foundations of Classical Analysis

For a given ε > 0 and the corresponding N ≥ 0 we choose an integer M in such a way that all terms a0 , a1 , . . . , aN appear in the M th partial sum s′M = PM ′ ′ i=0 ai of the rearrangement. Therefore, in the difference sm − sm , all the terms a0 , a1 , . . . , aN disappear (for m ≥ M ) and we have |sm − s′m | ≤ |aN +1 | + |aN +2 | + . . . + |aN +k | < ε, where k is a sufficiently large integer. This shows that sm − s′m → 0 and that the rearrangement converges to the same limit as the original series. ⊔ ⊓ We next present two criteria for the absolute convergence of an infinite series. (2.10) The Ratio Test (Cauchy 1821). If the terms an of the series (2.1) satisfy (2.11)

lim sup n→∞

|an+1 | < 1, |an |

then the series is absolutely convergent. If lim inf n→∞ |an+1 |/|an | > 1, then it diverges. Proof. Choose a number q that satisfies lim supn→∞ |an+1 |/|an | < q < 1. Then, only a finite number of quotients |an+1 |/|an | are larger than q and we have ∃N ≥ 0 ∀n ≥ N

|an+1 | ≤ q. |an |

This, in turn, implies |aN +1 | ≤ q|aN |, |aN +2 | ≤ q 2 |aN |, |aN +3 | ≤ q 3P |aN |, etc. Since the geometric series converges (we have 0 < q < 1), the series i≥0 |ai | also converges. If lim inf n→∞ |an+1 |/|an | > 1, then the sequence {|an |} is monotonically increasing for n ≥ N and the necessary condition (2.3) is not satisfied. ⊔ ⊓ Examples. The general term of the series for ex is an = xn /n!. Here, we have |an+1 |/|an | = |x|/(n + 1) → 0 so that the series (I.2.18) converges absolutely for all real x. Similarly, the series for sin x and cos x converge absolutely for all x. For the series (2.8) this criterion cannot be applied because |an+1 |/|an | = (n/(n + 1))α → 1. (2.11) The Root Test (Cauchy 1821). If p (2.12) lim sup n |an | < 1, n→∞

then the series (2.1) is absolutely convergent. If lim supn→∞ diverges.

p n |an | > 1, then it

Proof. As in the proof of p the ratio test, we choose a number q < 1 that is strictly larger than lim supn→∞ n |an |. Hence,

III.2 Infinite Series 195

∃N ≥ 0 ∀n ≥ N

p n |an | ≤ q.

This implies |an | ≤ q n for n ≥ NP , and a comparison with p the geometric series ∞ yields the absolute convergence of i=0 ai . If lim supn→∞ n |an | > 1, then the condition (2.3) is not satisfied and the series cannot converge. ⊔ ⊓

Double Series Consider a two-dimensional array of real numbers a00 + a01 + a02 + a03 + . . . = s0 + + + + + a10 + a11 + a12 + a13 + . . . = s1 + + + + + a20 + a21 + a22 + a23 + . . . = s2 + + + + + (2.13) a30 + a31 + a32 + a33 + . . . = s3 + + + + + : : : : : = = = = = v0 + v1 + v2 + v3 + . . . = ??? and suppose we want to sum up all of them. There are many natural ways of doing this. One can eitherPadd up the elements of the ith row, denote the result by si , ∞ and then compute i=0 si ; or one can add up the elements of the jth column, P∞ denote the result by vj , and then compute j=0 vj . It is also possible to write all elements in a linear arrangement. For example, we can start with a00 , then add the elements aij for which i + j = 1, then those with i + j = 2, and so on. This gives (2.14)

a00 + (a10 + a01 ) + (a20 + a11 + a02 ) + (a30 + . . .) + . . . .

Here, we denote the pairs (0, 0), (1, 0), (0, 1), (2, 0), . . . by σ(0), σ(1), σ(2), σ(3), . . . , so that σ is a map σ : N0 → N0 × N0 , where N0 × N0 = {(i, j) | i ∈ N0 , j ∈ N0 } is the so-called Cartesian product of N0 with N0 . So, we define in general, P∞ (2.12) Definition. A series k=0 bk is called a linear arrangement of the double series (2.13) if there exists a bijective mapping σ : N0 → N0 × N0 such that bk = aσ(k) . The question now is: do the different possibilities of summation lead to the same value? Do we have ∞ X ∞ ∞ X ∞  X  X (2.15) s 0 + s1 + . . . = aij = aij = v0 + v1 + . . . , i=0

j=0

j=0

i=0

and do linear arrangements converge to the same value? The counterexample of Fig. 2.4a shows that this is not true without some additional assumptions.

196 III. Foundations of Classical Analysis M

1 − 1 + 0 + 0 +... = 0 + + + + + 0 + 1 − 1 + 0 +... = 0 + + + + + 0 + 0 + 1 − 1 +... = 0 + + + + + 0 + 0 + 0 + 1 − . . .= 0 + + + + + .. .. .. .. .. . . . . . = = = = = 1 + 0 + 0 + 0 + . . . = 1 6= 0

a00 a01

b1

a10 b2

b0

bN+1 b3

bN+3

bN

M bN+4

FIGURE 2.4a. Counterexample

bN−1

bN+2 bN+k

FIGURE 2.4b. Double series

(2.13) Theorem (Cauchy 1821, “Note VII”). Suppose for the double series (2.13) that m m X X (2.16) ∃B ≥ 0 ∀m ≥ 0 |aij | ≤ B. i=0 j=0

Then, all the series in (2.15) are convergent and the identities of (2.15) are satisfied. Furthermore, every linear arrangement of the double series converges to the same value.

Proof. Let bP 0 +b1 +b2 +. . . be a linear arrangement of the double series (2.13). The sequence { ni=0 |bi |} is monotonically P increasing and bounded (by assumption P∞ ∞ (2.16)) so that i=0 |bi |, and hence also i=0 bi , converge. P∞ P Analogously, we can establish the convergence of si = j=0 aij and vj = ∞ i=0 aij . Inspired by the proof of Theorem 2.9, we apply Cauchy’s criterion to the P∞ series i=0 |bi | and have ∀ ε > 0 ∃ N ≥ 0 ∀ n ≥ N ∀ k ≥ 1 |bn+1 | + |bn+2 | + . . . + |bn+k | < ε.

For a given ε > 0 and the corresponding N ≥ 0 we choose an integer M in such a way that all elements b0 , b1 , . . . , bN are present in the box 0 ≤ i ≤ M , 0 ≤ j ≤ M (see Fig. 2.4b). With this choice, b0 , b1 , . . . , bN appear in the sum Pm Pn Pl i=0 bi (for l ≥ N ) as well as in i=0 j=0 aij (for m ≥ M and n ≥ M ). Hence, we have for l ≥ N , m ≥ M , n ≥ M , m n l X X X (2.17) aij − bi ≤ |bN +1 | + . . . + |bN +k | < ε, i=0 j=0

i=0

P∞ with a sufficiently large k. We set s = i=0 bi and take the limits Pml → Pn∞ and n → ∞ in (2.17). Then, we exchange the finite summations i=0 j=0 ↔

III.2 Infinite Series 197

Pn

j=0

1.6,

Pm

i=0

and take the limits l → ∞ and m → ∞. This yields, by Theorem m X si − s ≤ ε

and

i=0

Hence

P∞

i=0 si

and

P∞

j=0

n X vj − s ≤ ε. j=0

vj both converge to the same limit s.

⊔ ⊓

The Cauchy Product of Two Series If we want to compute the product of two infinite series we have to add all elements of the two-dimensional array

(2.18)

a0 b 0 a1 b 0

a0 b 1 a1 b 1

a0 b 2 a1 b 2

a0 b 3 a1 b 3

... ...

a2 b 0 a3 b 0 :

a2 b 1 a3 b 1 :

a2 b 2 a3 b 2 :

a2 b 3 a3 b 3 :

... ... .

P∞

i=0

ai and

P∞

j=0 bj ,

If we arrange the elements as indicated in Eq. (2.14), we obtain the so-called Cauchy product of the two series. P P∞ (2.14) Definition. The Cauchy product of the series ∞ i=0 ai and j=0 bj is defined by  ∞ X n X an−j · bj = a0 b0 + (a0 b1 + a1 b0 ) + (a0 b2 + a1 b1 + a2 b0 ) + . . . . n=0

j=0

The question is whether the Cauchy product is aPconvergent P series and whether it really represents the product of the two series i≥0 ai and j≥0 bj .

(2.15) Counterexample (Cauchy 1821). The series

1 1 1 1 1 − √ + √ − √ + √ − ... 2 3 4 5 converges by Leibniz’s criterion. We consider the Cauchy product of this series with itself. Since X X n n 1 2n + 2 = √ √ ≥ a · b n−j j n+2 n+1−j · j+1 j=0 j=0

(the inequality is a consequence of (n+1−x)(x+1) ≤ (1+n/2)2 for 0 ≤ x ≤ n), the necessary condition (2.3) for the convergence of the Cauchy product is not satisfied (see Fig. 2.5). This example illustrates the fact that the Cauchy product of two convergent series need not converge.

198 III. Foundations of Classical Analysis

1

0

50

100

−1

FIGURE 2.5. Divergence of the Cauchy product of Counterexample 2.15

P P∞ (2.16) Theorem (Cauchy 1821). If the two series ∞ i=0 ai and j=0 bj are absolutely convergent, then its Cauchy product converges and we have X  X  X  ∞ ∞ ∞ X n (2.19) ai · bj = an−j · bj . i=0

n=0

j=0

j=0

P P∞ Proof. By hypothesis, we have ∞ i=0 |ai | ≤ B1 and j=0 |bj | ≤ B2 . Therefore, we have for the two-dimensional array (2.18) that for all m ≥ 0 m m X X i=0 j=0

|ai ||bj | ≤ B1 B2 ,

P and Theorem 2.13 can be applied. The sum of the ith row gives si = ai · ∞ j=0 bj P∞ P∞ P∞ and i=0 si = ( i=0 ai )( j=0 bj ). By Theorem 2.13, the Cauchy product, which is a linear arrangement of (2.18), also converges to this value. ⊔ ⊓ Examples. For |q| < 1 consider the two series 1 + q + q2 + q3 + . . . = Their Cauchy product is

1 1−q

and

1 − q + q2 − q3 + . . . =

1 + q2 + q4 + q6 + . . . =

1 . 1+q

1 , 1 − q2

which, indeed, is the product of (1 − q)−1 and (1 + q)−1 . The Cauchy product of the absolutely convergent series x3 y3 x2 y2 + + ... and ey = 1 + y + + + ... 2! 3! 2! 3! gives the series for ex+y (use the binomial identity of Theorem I.2.1). ex = 1 + x +

III.2 Infinite Series 199

Remark. The statement of Theorem 2.16 remains true if only one of the two series is absolutely convergent and the second is convergent (F. Mertens 1875, see Exercise 2.3). P P Under the assumption that the series i ai , j bj and also their Cauchy product (Definition 2.14) converge, the identity (2.19) holds (Abel 1826, see Exercise 7.9).

Exchange of Infinite Series and Limits At several places in Chap. I, we were confronted with the problem of exchanging an infinite series with a limit (for example, for the derivation of the series for ex in Sect. and of those for sin x and cos x in Sect. I.4). We considered PI.2 ∞ series dn = snj depending on an integer parameter n, and used the fact j=0 P that limn→∞ dn = ∞ j=0 limn→∞ snj . Already in Sect. I.2 (after Eq. (I.2.17)), it was observed that this is not always true and that some caution is necessary. The following theorem states sufficient conditions for the validity of such an exchange. (2.17) Theorem. Suppose that the elements of the sequence {s0j , s1j , s2j , . . .} all have the same signPand that |sn+1,j | ≥ |snj | for all n and j. If there exists a bound B such that nj=0 |snj | ≤ B for all n ≥ 0, then (2.20)

lim

n→∞

∞ X

snj =

j=0

∞ X j=0

lim snj .

n→∞

Proof. The idea is to reformulate the hypotheses in such a way that Theorem 2.13 is directly applicable. At the beginning of this section, we saw that every series can be converted to an infinite sequence by considering the partial sums (2.2). Conversely, if the partial P sums s0 , s1 , s2 , . . . are given, we can uniquely define elements ai such that ni=0 ai = sn . We just have to set a0 = s0 and ai = si − si−1 for i ≥ 1. Applying this idea to the sequence {s0j , s1j , s2j , . . .}, we define a0j := s0j ,

aij := sij − si−1,j ,

so that

n X

aij = snj .

i=0

Replacing snj by this expression, (2.20) becomes (2.21)

lim

n→∞

n ∞ X X j=0 i=0

aij =

∞ X j=0

lim

n→∞

n X

aij .

i=0

Exchanging the summations in the expression on the left side of (2.21) (this is permitted by Theorem 1.5), we see that (2.21) is equivalent to (2.15). Therefore, we only have to verify condition (2.16). The assumptions on {s0j , s1j , . . .} imply that the elements a0j , a1j , . . . all have the same sign. Hence, we have

200 III. Foundations of Classical Analysis n X i=0

and

|aij | = |snj |

n n X X i=0 j=0

|aij | =

n X j=0

|snj | ≤ B.

By Theorem 2.13, this implies (2.21) and thus also (2.20).

⊔ ⊓

(2.18) Example. We will give here a rigorous proof of Theorem I.2.3. From the binomial theorem, we have 

(2.22)

1+

y 2 (1 − n1 ) y 3 (1 − n1 )(1 − n2 ) y n + + ... , =1+y+ n 1·2 1·2·3

which is a series depending on the parameter n. We set sn0 = 1,

sn1 = y,

y 2 (1 − n1 ) , 1·2

sn2 =

sn3 =

y 3 (1 − n1 )(1 − n2 ) , 1·2·3

... .

For a fixed y the elements of the sequence {s0j , s1j , . . .} all have the same sign, and {|s0j |, |s1j |, . . .} is monotonically increasing. Furthermore, we have n X j=0

because, by the ratio test, rem 2.17 yields

|snj | ≤

P∞

j=0

n X |y|j ≤B j! j=0

|y|j /j! is a convergent series. Hence, Theo-

 y3 y4 y n y2 + + + ... . lim 1 + =1+y+ n→∞ n 2! 3! 4!

Exercises 2.1 Compute the Cauchy product of the two series f (x) = x −

x5 x3 + − ... 3! 5!

and

g(y) = 1 −

y2 y4 + − ... 2! 4!

and find the series for f (x)g(y) + g(x)f (y). Justify the computations. Does the result seem familiar? 2.2 Show that the Cauchy product of the two divergent series    2 + 2 + 22 + 23 + 24 + . . . −1 + 1 + 1 + 1 + 1 + 1 + . . . converges absolutely.

III.2 Infinite Series 201

2.3 (Mertens P 1875). Suppose that the seriesP ∞ i=0 ai is convergent and ∞ that j=0 bj is absolutely convergent. Prove that the Cauchy product of Definition 2.14 is convergent and that (2.19) holds. P Hint. Put cn = nj=0 an−j bj and apply the triangle inequality (but only to the first sums) in the identity 2n X i=0

ci −

n X i=0

ai

n  X j=0

n a0b0 a0b1 a0b2 a0b3 a0b4 a0b5 a0b6 a1b0 a1b1 a1b2 a1b3 a1b4 a1b5 a2b0 a2b1 a2b2 a2b3 a2b4 n a3b0 a3b1 a3b2 a3b3 a4b0 a4b1 a4b2 a5b0 a5b1 a6b0

2n−j 2n  n−1 X 2n−j X X X bj = bj ai + bj ai . j=0

i=n+1

j=n+1

i=0

2.4 Determine the constants a1 , a2 , a3 , a4 , . . . so that the Cauchy product of the two series      1 − a 1 + a2 − a3 + . . . 1 − a 1 + a2 − a3 + . . . = 1 − 1 + 1 − 1 + . . . becomes the divergent series 1 − 1 + 1 − . . . . Show that the series 1 − a1 + a2 − a3 + . . . converges (Fig. 2.6). Can it converge absolutely? Hint. The use of the generating function for the numbers 1, −a1 , a2 , −a3 , . . . reduces this exercise to a known formula of Chap. I and to Wallis’s product. 1

X

(−1)n an

n

0 1

50

X n

0

(−1)n

n X i=0

ai · an−i

50

FIGURE 2.6. Divergence of the Cauchy product of Exercise 2.4

2.5 Justify Eq. (I.5.26) by taking the logarithm and applying the ideas of Example 2.18.

202 III. Foundations of Classical Analysis

III.3 Real Functions and Continuity We call here Function of a variable magnitude, a quantity that is composed in any possible manner of this variable magnitude & of constants. (Joh. Bernoulli 1718, Opera, vol. 2, p. 241) Consequently, if f ( xa + c) denotes an arbitrary function . . . (Euler 1734, Opera, vol. XXII, p. 59) If now to any x there corresponds a unique, finite y, . . . then y is called a function of x for this interval.. . . This definition does not require a common rule for the different parts of the curve; one can imagine the curve as being composed of the most heterogeneous components or as being drawn without following any law. (Dirichlet 1837)

Real functions y = f (x) of a real variable x were, since Descartes, the universal tool for the study of geometric curves and, since Galilei and Newton, for mechanical and astronomical calculations. The word “functio” was proposed by Leibniz and Joh. Bernoulli, the symbol y = f (x) was introduced by Euler (1734) (see quotations). In the Leibniz-Bernoulli-Euler era, real functions were mainly thought of as being composed of elementary√functions (“expressio analytica quomodocunque . . . . Sic a + 3z, az − 4z 2, az + b a2 − z 2 , cz etc. sunt functiones ipsius z”, Euler 1748), perhaps with different formulas for different domains (“curvas discontinuas seu mixtas et irregulares appellamus”). The 19th century, mainly under the influence of Fourier’s heat equation and Dirichlet’s study of Fourier series, brought a wider notion: “any sketched curve” or “any values y defined in dependence of the values x” (see the quotation above). (3.1) Definition (Dirichlet 1837). A function f : A → B consists of two sets, the domain A and the range B, and of a rule that assigns to each x ∈ A a unique element y ∈ B. This correspondence is denoted by y = f (x)

or

x 7→ f (x).

We say that y is the image of x and that x is an inverse image of y. Throughout this section, the range will be R (or an interval) and the domain will be an interval or a union of intervals of the form (a, b) = {x ∈ R | a < x < b}

(a, b] = {x ∈ R | a < x ≤ b}

or [a, b] = {x ∈ R | a ≤ x ≤ b} or or [a, ∞) = {x ∈ R | a ≤ x < ∞}

or . . . .

The interval (a, b) is called open, while [a, b] is closed. As in the following examples, we usually use braces for functions that are defined by different expressions on different parts of A. Examples. 1. The function f : [0, 1] → R,  x 0 ≤ x ≤ 1/2 (3.1) f (x) = 1−x 1/2 ≤ x ≤ 1,

is plotted on the right. We observe that some y ∈ R have no inverse image, and that some have more than one.

.5

.0

.5

1.0

III.3 Real Functions and Continuity 203

2. Our second function can be defined either by a single expression, as a limit, or with braces by separating three cases:

(3.2)

f (x) = lim arctan(nx) n→∞  x>0   π/2 = 0 x=0   −π/2 x < 0.

1

−2

3. The following function, which is difficult to plot, is due to Dirichlet (see Werke, vol. 2, p. 132, 1829, “On aurait un exemple d’une fonction . . .”):  0 x irrational (3.3) f (x) = 1 x rational. 4. This function is of a similar nature to Dirichlet’s, but the peaks become lower for increasing denominators of x:  0 x irrational (3.4) f (x) = 1/q x = p/q simpl. fraction.

−1

0 0 −1

1

2

n = 1, 2, 4, 8, 16, . .

1

0

1

.5

0

5. When x tends to zero, 1/x tends to ∞, therefore 
sin 1/x x 6= 0 (3.5) f (x) = 0 x=0 will produce an infinity of oscillations in the neighborhood of the origin (Cauchy 1821).

1

.5

6. Here the oscillations close to the origin are less violent, due to the factor x, but there are still infinitely many (Weierstrass 1874): 
x · sin 1/x x 6= 0 (3.6) f (x) = 0 x = 0. 7. Our last example was proposed, according to Weierstrass (1872), by Riemann (see Sect. III.9 below) and is defined via an infinite convergent sum: (3.7)

f (x) =

∞ X sin(n2 x) . n2 n=1

.5

1

0

1

2

3

204 III. Foundations of Classical Analysis

Continuous Functions . . . f (x) will be called a continuous function, if . . . the numerical values of the difference f (x + α) − f (x) decrease indefinitely with those of α . . . (Cauchy 1821, Cours d’Analyse, p. 43) Here we call a quantity y a continuous function of x, if after choosing a quantity ε the existence of δ can be proved, such that for any value between x0 − δ . . . x0 + δ the corresponding value of y lies between y0 − ε . . . y0 + ε. (Weierstrass 1874)

Cauchy (1821) introduced the concept of continuous functions by requiring that indefinite small changes of x should produce indefinite small changes of y (see quotation). Bolzano (1817) and Weierstrass (1874) were more precise (second quotation): the difference f (x) − f (x0 ) must be arbitrarily small, if the difference x − x0 is sufficiently small. (3.2) Definition. Let A be a subset of R and x0 ∈ A. The function f : A → R is continuous at x0 if for every ε > 0 there exists a δ > 0 such that for all x ∈ A satisfying |x − x0 | < δ we have |f (x) − f (x0 )| < ε, or in symbols: ∀ ε > 0 ∃ δ > 0 ∀ x ∈ A : |x − x0 | < δ

|f (x) − f (x0 )| < ε.

The function f (x) is called continuous, if it is continuous at all x0 ∈ A. See Fig. 3.1a for a continuous function and Figs. 3.1b–3.1f for functions with discontinuities.

FIGURE 3.1. Continuous and discontinuous functions

III.3 Real Functions and Continuity 205

Discussion of Examples (3.1) to (3.7). The function (3.1) is continuous everywhere, even at x0 = 1/2; the function (3.2) is discontinuous at 0; (3.3) is discontinuous everywhere; (3.4) is continuous for irrational x0 and discontinuous for rational x0 (Exercise 3.1); (3.5) is discontinuous at x0 = 0; (3.6) is continuous everywhere, even at x = 0; (3.7), which appears to exhibit violent variations, is nevertheless everywhere continuous (as we shall see later in Theorem 4.2). (3.3) Theorem. A function f : A → R is continuous at x0 ∈ A if and only if for every sequence {xn }n≥1 with xn ∈ A we have (3.8)

lim f (xn ) = f (x0 )

n→∞

if

lim xn = x0 .

n→∞

Proof. For a given ε > 0, choose δ > 0 as in Definition 3.2. Since xn → x0 , there exists N such that |xn − x0 | < δ for n ≥ N . By continuity at x0 , we then have |f (xn ) − f (x0 )| < ε for n ≥ N and (3.8) holds. Suppose now that (3.8) holds, but that f (x) is discontinuous at x0 . The negation of continuity at x0 is ∃ ε > 0 ∀ δ > 0 ∃ x ∈ A : |x − x0 | < δ

|f (x) − f (x0 )| ≥ ε.

The idea is to take δ = 1/n and to attach an index n to x (which depends on δ). This gives us a sequence {xn } with elements in A such that |xn − x0 | < 1/n (hence xn → x0 ) and at the same time |f (xn ) − f (x0 )| ≥ ε. This contradicts (3.8). ⊔ ⊓ (3.4) Theorem. Let f : A → R and g : A → R be continuous at x0 ∈ A and let λ be a real number. Then, the functions f + g,

λ · f,

f · g,

f /g

( if g(x0 ) 6= 0)

are also continuous at x0 . Proof. We take a sequence {xn } with elements in A and converging to x0 . The continuity of f and g implies that f (xn ) → f (x0 ) and g(xn ) → g(x0 ) for n → ∞. Theorem 1.5 then shows that f (xn ) + g(xn ) → f (x0 ) + g(x0 ), so that f + g is seen to be continuous at x0 (Theorem 3.3). The continuity of the other functions can be deduced in the same way.

⊔ ⊓

Example. It is obvious that the constant function f (x) = a is continuous. The function f (x) = x is continuous too (choose δ = ε in Definition 3.2). As a consequence of Theorem 3.4, all polynomials P (x) = a0 + a1 x + . . . + an xn are continuous, and rational functions R(x) = P (x)/Q(x) are continuous at all points x0 , where Q(x0 ) 6= 0.

206 III. Foundations of Classical Analysis

The Intermediate Value Theorem This theorem has been known for a long time . . . (Lagrange 1807, Oeuvres vol. 8, p. 19, see also p. 133)

This theorem appears geometrically evident and was used by Euler and Gauss without scruples (see quotation). Only Bolzano found that a “rein analytischer Beweis” was necessary to establish more rigor in Analysis and Algebra. (3.5) Theorem (Bolzano 1817). Let f : [a, b] → R be a continuous function. If f (a) < c and f (b) > c, then there exists ξ ∈ (a, b) such that f (ξ) = c.

Proof. We shall prove the statement for c = 0. The general result then follows from this special case by considering f (x) − c instead of f (x). The set X = {x ∈ [a, b] ; f (x) < 0} is nonempty (a ∈ X) and it is majorized by b. Hence, the supremum ξ = sup X exists by Theorem 1.12. We shall show that f (ξ) = 0 (Fig. 3.2). Assume that f (ξ) = K > 0. We put ε = K/2 > 0 and deduce from the continuity of f (x) at ξ the existence of some δ > 0 such that |f (x) − K| < K/2

for

|x − ξ| < δ.

This implies that f (x) > K/2 > 0 for ξ − δ < x ≤ ξ, which contradicts the fact that ξ is the supremum of X. We exclude the case f (ξ) = K < 0 in a similar way. ⊔ ⊓

f(b)

X c

ξ = sup X

f(a) a

b FIGURE 3.2. Proof of Bolzano’s Theorem

The Maximum Theorem With his theorem, which states that a continuous function of a real variable actually attains its least upper and greatest lower bounds, i.e., necessarily possesses a maximum and a minimum, Weierstrass created a tool which today is indispensable to all mathematicians for more refined analytical or arithmetical investigations. (Hilbert 1897, Gesammelte Abh. , vol. 3, p. 333)

The following theorem is called “Hauptlehrsatz” (“Principal Theorem”) in Weierstrass’ lectures of 1861 and was published by Cantor (1870).

III.3 Real Functions and Continuity 207

(3.6) Theorem. If f : [a, b] → R is a continuous function, then it is bounded on [a, b] and admits a maximum and a minimum, i.e., there exist u ∈ [a, b] and U ∈ [a, b] such that (3.9)

f (u) ≤ f (x) ≤ f (U )

for all

x ∈ [a, b].

Discussion of the Assumptions. The function f : (0, 1] → R defined by f (x) = 1/x is not bounded on A = (0, 1]. Therefore, the assumption that the domain A be closed is important. The function f : [0, ∞) → R, given by f (x) = x2 , shows that the boundedness of the domain of f (x) is important. The function f : [0, 1] → R defined by f (1/2) = 0 and f (x) = (x − 1/2)−2

for

x 6= 1/2

is discontinuous at x = 1/2 and unbounded. Hence, it is important to assume that the function be continuous everywhere. 1.0 Our last example exhibits a function f : [0.1] → R which is bounded, but does not admit a maximum: .5  .0 −3x + sin(1/x) if x 6= 0 .0 .1 .2 f (x) = 0 if x = 0 . −.5 The supremum of the set {f (x) | x ∈ [0, 1]} is equal to 1, but there is no U ∈ [0, 1] with f (U ) = 1.

−1.0

Proof of Theorem 3.6. We first prove that f (x) is bounded on [a, b]. We suppose the contrary: (3.10)

∀ n ≥ 1 ∃ xn ∈ [a, b]

|f (xn )| > n.

The sequence x1 , x2 , x3 , . . . admits a convergent subsequence by the BolzanoWeierstrass Theorem (Theorem 1.17). In order to avoid writing this subsequence with new symbols, we denote it again by x1 , x2 , x3 , . . . and we simply say: “after extracting a subsequence, we suppose that” limn→∞ xn = ξ. Since f is continuous at ξ, it follows from Theorem 3.3 that limn→∞ f (xn ) = f (ξ). This contradicts (3.10) and proves the boundedness of f (x). In order to prove the existence of U ∈ [a, b] such that (3.9) holds, we consider the set Y = {y ; y = f (x), a ≤ x ≤ b}. This set is nonempty and bounded (as we have just seen). Therefore, the supremum M = sup Y exists. By Definition 1.11 of the supremum, the value M − ε (for an arbitrary ε > 0) is no longer an upper bound of Y . Taking ε = 1/n, we thus find a sequence of elements xn ∈ [a, b] satisfying (3.11)

M − 1/n < f (xn ) ≤ M.

208 III. Foundations of Classical Analysis

Applying the Bolzano-Weierstrass Theorem, after extracting a subsequence, we suppose that {xn } converges and we denote the limit by U = limn→∞ xn . Because of the continuity of f (x) at U , it follows from (3.11) that f (U ) = M . The existence of a minimum is proved similarly. ⊔ ⊓

Monotone and Inverse Functions (3.7) Definition. Let A and B be subsets of R. The function f : A → B is

• • • • • • • • •

injective if f (x1 ) 6= f (x2 ) for x1 6= x2 , surjective if ∀ y ∈ B ∃ x ∈ A f (x) = y , bijective if it is injective and surjective, increasing if f (x1 ) < f (x2 ) for x1 < x2 , decreasing if f (x1 ) > f (x2 ) for x1 < x2 , nondecreasing if f (x1 ) ≤ f (x2 ) for x1 < x2 , nonincreasing if f (x1 ) ≥ f (x2 ) for x1 < x2 , monotone if it is nonincreasing or nondecreasing, and strictly monotone if it is increasing or decreasing.

Strictly monotone functions are injective. It is interesting that for real continuous functions, defined on an interval, the converse statement is true, too. (3.8) Lemma. If f : [a, b] → R is continuous and injective, then f is strictly monotone. Proof. For any three points u < v < w we have (3.12)

f (v) is between f (u) and f (w) .

Indeed, suppose f (v) is outside this interval and, say, closer to f (u). Then there is a ξ between v and w with f (u) = f (ξ) (Theorem 3.5). This is in contradiction to the injectivity of f . Therefore, for a < c < d < b the only possibilities are f (a) < f (c) < f (d) < f (b)

or

f (a) > f (c) > f (d) > f (b);

all other configurations of the inequalities contradict (3.12).

⊔ ⊓

Surjectivity of a function f : A → B implies that every y ∈ B has at least one inverse image. Injectivity then implies uniqueness of this inverse image. Therefore, a bijective function has an inverse function f −1 : B → A, defined by (3.13)

f −1 (y) = x

⇐⇒

f (x) = y.

(3.9) Theorem. Let f : [a, b] → [c, d] be continuous and bijective. Then, the inverse function f −1 : [c, d] → [a, b] is also continuous.

III.3 Real Functions and Continuity 209

Proof. Let {yn } with yn ∈ [c, d] be a sequence satisfying limn→∞ yn = y0 . By Theorem 3.3, we have to show that limn→∞ f −1 (yn ) = f −1 (y0 ). We therefore consider the sequence {xn } = {f −1 (yn )}. Let {x′n } be a convergent subsequence (which exists by the theorem of Bolzano-Weierstrass), and denote its limit by x0 . The continuity of f (x) at x0 implies that f (x0 ) = lim f (x′n ) = lim yn′ = y0 , n→∞

n→∞

and consequently x0 = f −1 (y0 ). Therefore, each convergent subsequence of {xn } = {f −1 (yn )} converges to f −1 (y0 ). This point is the only accumulation point of the sequence {f −1 (yn )} and we have f −1 (yn ) → f −1 (y0 ) (see also Exercise 1.8). ⊔ ⊓ Example. Each of the real functions x2 , x3 , . . . is strictly monotone on [0, ∞) and √ √ has there an inverse function: x, 3 x, . . . . By Theorem 3.9, these functions are continuous.

Limit of a Function The concept of the limit of a function was probably first defined with sufficient rigour by Weierstrass. (Pringsheim 1899, Enzyclop¨adie der Math. Wiss., Band II.1, p. 13)

Assume that f (x) is not continuous at x0 or not even defined there; in such a situation it is interesting to know whether there exists, at least, the limit of f (x) for x approaching x0 . Obviously, x0 has to be close to the domain of f . We say that x0 is an accumulation point of a set A if (3.14)

∀δ > 0 ∃x ∈ A

0 < |x − x0 | < δ.

For a bounded interval, the accumulation points consist of the interval and of the two endpoints. (3.10) Definition. Consider a function f : A → R and let x0 be an accumulation point of A. We say that the limit of f (x) at x0 exists and is equal to y0 , i.e., (3.15)

lim f (x) = y0

x→x0

if (3.16)

∀ ε > 0 ∃ δ > 0 ∀ x ∈ A : 0 < |x − x0 | < δ

|f (x) − y0 | < ε.

This definition can be modified to cover the situations x0 = ±∞ and/or y0 = ±∞ (see, for example, Eq. (1.10)). The assumption that x0 is an accumulation point implies that the set of x ∈ A satisfying 0 < |x − x0 | < δ is never empty. With Definition 3.10, the continuity of f (x) at x0 can be expressed as follows (see Definition 3.2):

210 III. Foundations of Classical Analysis

(3.17)

lim f (x)

x→x0

exists

and

lim f (x) = f (x0 ).

x→x0

Examples. The function of Fig. 3.1b has a limit limx→x0 f (x) that is different from f (x0 ). For the function (3.4), the limit limx→x0 f (x) exists for all x0 (see Exercise 3.1; remember that the point x0 is explicitly excluded in Definition 3.10) and limx→x0 f (x) = 0. A still weaker property is the existence of one-sided limits. (3.11) Definition. We say that the left-sided (respectively right-sided) limit of f (x) at x0 exists if (3.16) holds under the restriction x < x0 (respectively x0 < x). These limits are denoted by (3.18)

lim f (x) = y0

x→x0 −

respectively

lim f (x) = y0 .

x→x0 +

The functions of Figs. 3.1b, 3.1c, and 3.1d possess left- and right-sided limits (often = f (x0 )); these limits do not exist for the functions of Figs. 3.1e and 3.1f. The following theorem is an analog to Cauchy’s criterion in Theorem 1.8. (3.12) Theorem (Dedekind 1872). The limit limx→x0 f (x) exists if and only if (3.19) 0 < |x − x0 | < δ ∀ ε > 0 ∃ δ > 0 ∀ x, x b∈A : |f (x) − f (b x)| < ε. 0 < |b x − x0 | < δ Proof. The “only if” part follows from

|f (x) − f (b x)| ≤ |f (x) − y0 | + |y0 − f (b x)| < 2ε. For the “if” part we choose a sequence {xi } with xi ∈ A which converges to x0 . Because of (3.19) the sequence {yi } with yi = f (xi ) is a Cauchy sequence and, by Theorem 1.8, converges to, say, y0 . For an x satisfying 0 < |x − x0 | < δ we now have, again from (3.19), |f (x) − y0 | ≤ |f (x) − f (xi )| + |f (xi ) − y0 | < 2ε, for i sufficiently large.

⊔ ⊓

Analoguous results hold for the situation where x0 = ±∞ or for one-sided limits.

Exercises 3.1 Show that the function (3.4) is continuous at all irrational x0 and, of course, discontinuous at rational x0 .

III.3 Real Functions and Continuity 211

√ Hint. If you have difficulties, set x0 = 2 − 1 and ε = 1/10 and determine for which values of x you have f (x) ≥ ε. This gives you a δ for which the statement in Definition 3.2 is satisfied. 3.2 (Pringsheim 1899, p. 7). Show that Dirichlet’s function (3.3) can be written as m f (x) = lim lim cos(n!πx) . n→∞ m→∞

3.3 Compute the limits

√ √ x2 + 3x + 2 4+x− 4−x , lim . lim x→−1 x→0 x2 − 1 2x √ √ √ √ Remember that ( a − b)( a + b) = a − b.

3.4 Show: if f : [a, b] −→ [c, d] is continuous at x0 , and g : [c, d] −→ [u, v] is continuous at y0 = f (x0 ), then the composite function (g ◦ f )(x) = g(f (x)) is continuous at x0 . 3.5 Here is a list of functions f : A → R, 1) f (x) = x · sin(1/x) − 2x 2) f (x) = x/(x2 + 1) 3) the same

√ 4) f (x) = (1/ sin x) − 1 5) the same √ 6) f (x) = x · sin(x2 )

7) the same 8) f (x) = arctan((x − 0.5)/(x2 − 0.1x − 0.7))

9) f (x) = sin(x2 ) 10) the same √ 11) f (x) = 3 x 12) the same

13) f (x) = cos x + 0.1 sin(40x) 14) f (x) = x − [x] √ √ 15) f (x) = x · sin(1/x) − 2 x p 16) f (x) = 3 − 1/ x(1 − x)

17) f (x) = sin(5/x) − x

A = [0, 0.2] A = [−4, +4] A = (−∞, +∞) A = (0, π) A = [0, π] A = [0, 7] A = [0, ∞) A = [−1.5, 1.5] A = [−5, 5] A = (−∞, ∞)

A = [−1, 1] A = (−∞, ∞)

A = [−1.6, 1.6] A = [0, 3] A = [0, 0.1] A = (0, 1) A = [0, 0.4]

where [x] denotes the largest integer not exceeding x. Whenever the above definitions for f (x) do not make sense (for example when a certain denominator is zero), set f (x) = 0. Decide which of these functions are graphed in Fig. 3.3.

212 III. Foundations of Classical Analysis 0

3

1

0

2

0

0

1 0

0 0 −1

0

1

2

3

4

5

6

−2

−1

−3 1

1

0 −5 −4 −3 −2 −1 0 1 2 3 4 5

1 0

2

1

−1

0

−1

−1

7

0

0

1

1

−1 0

1

0

0

1

2

3

1 0

1 0 −4 −3 −2 −1 0

−1

1

2

3

4

0

0

−2 −3

0

0

1

2

3

FIGURE 3.3. Plot of 12 functions for Exercise 3.5

3.6 Which of the functions of Exercise 3.5 are continuous on A? What are the points of discontinuity? 3.7 Which of the functions of Exercise 3.5 possess a maximum value on A; which possess a minimum value on A?

III.4 Uniform Convergence and Uniform Continuity 213

III.4 Uniform Convergence and Uniform Continuity The following theorem can be found in the work of Mr. Cauchy: “If the various terms of the series u0 + u1 + u2 + . . . are continuous functions, . . . then the sum s of the series is also a continuous function of x.” But it seems to me that this theorem admits exceptions. For example the series sin x − 12 sin 2x + 13 sin 3x . . . is discontinuous at each value (2m + 1)π of x, . . . (Abel 1826, Oeuvres, vol. 1, p. 224-225)

The Cauchy-Bolzano era (first half of 19th century) left analysis with two important gaps: first the concept of uniform convergence, which clarifies the limit of continuous functions and the integral of limits; second the concept of uniform continuity, which ensures the integrability of continuous functions. Both gaps were filled by Weierstrass and his school (second half of 19th century).

The Limit of a Sequence of Functions We consider a sequence of functions f1 , f2 , f3 , . . . : A → R. For a chosen x ∈ A the values f1 (x), f2 (x), f3 (x), . . . are a sequence of numbers. If the limit (4.1)

lim fn (x) = f (x)

n→∞

exists for all x ∈ A, we say that {fn (x)} converges pointwise on A to f (x). Cauchy announced in his Cours (1821, p. 131; Oeuvres II.3, p. 120) that if (4.1) converges for all x in A and if all fn (x) are continuous, then f (x) is also continuous. Here are four counterexamples to this assertion; the first one is due to Abel (1826, see the quotation above). Examples. a) (Abel 1826, see the upper left picture of Fig. 4.1) (4.2a)

fn (x) = sin x −

sin nx sin 2x sin 3x sin 4x + − + ...± . 2 3 4 n

Fig. 4.1 shows f1 (x), f2 (x), f3 (x) and f100 (x). Apparently, {fn (x)} converges to the line y = x/2 for −π < x < π (this can be proved using the theory of Fourier series), but fn (π) = 0 and for π < x < 3π the limit is y = x/2 − π. Thus, the limit function is discontinuous. b) (upper right picture of Fig. 4.1)  0 x 1.

214 III. Foundations of Classical Analysis

2

y = sin x −

1

n =1

2

3

sin 2x sin 3x + − ... 2 3

1

y = xn

n = 100 n =1

0

1

2

3

4

5

6

7

2

−1 −2

3

n = 100

4

0

1

n

1

1

x −1 xn + 1

y=

n =1 2

4 0

1

n =1 2

4

2 n = 100

n = 100 −1

0

1

y = (1 − x2 )n

−1

FIGURE 4.1. Sequences of continuous functions with a discontinuous limit

d) (lower right picture of Fig. 4.1) (4.2d) fn (x) = (1 − x2 )n on A = [−1, 1],

lim fn (x) =

n→∞



0 1

x 6= 0 x = 0.

Another example, which we have already encountered, is fn (x) = arctan(nx) (see (3.2)).

1

0

ε = .140

1

0

ε = .070

1

ε = .035

0

FIGURE 4.2. Sequence of uniformly convergent functions

Explanation (Seidel 1848). We look at the upper right picture of Fig. 4.1. The closer x is chosen to the point x = 1, the slower is the convergence and the larger we must take n in order to obtain the prescribed precision ε. This allows the discontinuity to be created. We must therefore require that, for a given ε > 0, the difference fn (x) − f (x) be smaller than ε for all x ∈ A, if, of course, n ≥ N (see Fig. 4.2).

III.4 Uniform Convergence and Uniform Continuity 215

(4.1) Definition (Weierstrass 1841). The sequence fn : A → R converges uniformly on A to f : A → R if (4.3)

∀ε > 0 ∃N ≥ 1 ∀n ≥ N ∀x ∈ A

|fn (x) − f (x)| < ε.

In this definition, it is important that N depends only on ε and not on x ∈ A. This is why “∀ x ∈ A” stands after “∃ N ≥ 1” in (4.3). As in Sect. III.1 (Definition 1.7), we can replace f (x) in (4.3) by all successors of fn (x). We then get Cauchy’s criterion for uniform convergence: (4.4) ∀ ε > 0 ∃ N ≥ 1 ∀ n ≥ N ∀ k ≥ 1 ∀ x ∈ A |fn (x)−fn+k (x)| < ε. (4.2) Theorem (Weierstrass’s lectures of 1861). If fn : A → R are continuous functions and if fn (x) converges uniformly on A to f (x), then f : A → R is continuous. f(x) 0

∀ x0 ∈ A ∀ x ∈ A : |x − x0 | < δ

|f (x) − f (x0 )| < ε.

Remark. The uniform continuity of a given function can often be shown using Lagrange’s Mean Value Theorem (see Theorem III.6.11 below), (4.11)

f (x) − f (x0 ) = f ′ (ξ)(x − x0 ).

If A is an interval and f differentiable in A with (4.12)

M = sup |f ′ (ξ)| < ∞, ξ∈A

then, for a given ε, we satisfy the condition of Definition 4.4 by simply putting δ = ε/M (see also Exercise 4.3 below). However, differentiability is by no means necessary, and we have the following astonishing theorem.

III.4 Uniform Convergence and Uniform Continuity 219

(4.5) Theorem (Heine 1872). Let A be a closed interval [a, b] and let the function f : A → R be continuous on A; then f is uniformly continuous on A. First Proof (after Heine 1872, p. 188). We assume the negation of the condition in Definition 4.4 and choose δ = 1/n for n = 1, 2, . . . . This yields (4.13a) (4.13b)

∃ ε > 0 ∀ 1/n > 0 ∃ x0n ∈ A ∃ xn ∈ A : |xn − x0n | < 1/n such that |f (xn ) − f (x0n )| ≥ ε.

After extracting a convergent subsequence from {xn } (which we again denote by {xn }; see Theorem 1.17), we have limn→∞ xn = x, and since |xn − x0n | < 1/n we also have limn→∞ x0n = x. Since f is continuous, we have (see Theorem 3.3) lim f (xn ) = f (x) = lim f (x0n ),

n→∞

n→∞

in contradiction with (4.13b).

⊔ ⊓

Second Proof (L¨uroth 1873). Let an ε > 0 be chosen. For each x ∈ [a, b] let δ(x) > 0 be the length of the largest open interval I of center x such that |f (y) − f (z)| < ε for y, z ∈ I. More precisely, (4.14)

δ(x) = sup{δ > 0 | ∀ y, z ∈ [x − δ/2, x + δ/2] |f (y) − f (z)| < ε}

(where, of course, the values x, y, and z have to lie in A). By continuity of f (x) at x, the set {δ > 0 | . . .} in (4.14) is nonempty, so that δ(x) > 0 for all x ∈ A. If δ(x) = ∞ for some x ∈ A, the estimate |f (y) − f (z)| < ε holds without any restriction and any δ > 0 will satisfy the condition in Definition 4.4.

FIGURE 4.6. L¨uroth’s proof of Theorem 4.3

If δ(x) < ∞ for all x ∈ A, we move x to x ± η. The new interval I ′ cannot be longer than δ(x)+2|η|, otherwise I would be entirely in I ′ and could be extended. Neither can it be smaller than δ(x)−2|η|. Thus, this δ(x) is a continuous function. Weierstrass’s Maximum Theorem (Theorem 3.6), applied here in its “minimum” version, ensures that there is a value x0 such that δ(x0 ) ≤ δ(x) for all x ∈ A. This value δ(x0 ) is positive by definition and can be used to satisfy the condition in Definition 4.4. ⊔ ⊓ Remark. If you are unsatisfied with both proofs above, you can read a third one, published by Darboux (1875, p. 73-74), which is based on repeated subdivision of intervals.

220 III. Foundations of Classical Analysis

Exercises 4.1 Show that the functions fn (x) = (n + 1)xn (1 − x),

x ∈ A = [0, 1]

converge to zero for all x ∈ A, but possess a maximum at x = n/(n + 1) of asymptotic height 1/e. Therefore, we do not have uniform convergence despite the fact that the limiting function is continuous. 4.2 (Pringsheim 1899, p. 34). Show that the series f (x) =

x2  1 n 1 + x2 1 + x2 n=1 ∞ X

a) converges absolutely for all x ∈ R and b) does not converge uniformly on [−1, 1]. c) Compute f (x). Is it continuous? 4.3 The function f : [0, 1] → R defined by √
x · sin x1 + 2 if 0 < x ≤ 1, f (x) = 0 if x = 0 is continuous on [0, 1], and should therefore be uniformly continuous. Find explicitly for a given ε > 0, say ε = 0.01, a δ > 0 for which we have ∀ x1 , x2 ∈ [0, 1] : |x1 − x2 | < δ

|f (x1 ) − f (x2 )| < ε.

Hint. Use the Mean Value Theorem away from the origin and a direct estimate for values close to 0. 4.4 Which of the functions of Fig. 3.3 (see Exercise 3.5) are uniformly continuous on A ?

III.5 The Riemann Integral 221

III.5 The Riemann Integral Rb

Our first question is therefore: what meaning should we give to a f (x) dx ? (Riemann 1854, Werke, p. 239) By one of those insights of which only the greatest minds are capable, the famous geometer [Riemann] generalises the concept of the definite integral, . . . (Darboux 1875)

The discussion of the integral in Sects. II.5 and II.6 was based on the formula Z b f (x) dx = F (b) − F (a), (5.1) a

where F (x) is a primitive of f (x). We have implicitly assumed that such a primitive always exists and is unique (up to an additive constant). Here, we will give a Rb precise definition of a f (x) dx independent of differential calculus. This allows Rb us to interpret a f (x) dx for a larger class of functions, including discontinuous functions or functions for which a primitive is not known. A rigorous proof of (5.1) for continuous f will then be given in Sect. III.6 below. Cauchy (1823) described, as rigorously as was then possible, the integral of a continuous function as the limit of a sum. Riemann (1854), merely as a sideremark in his habilitation thesis on trigonometric series, defined the integral for more general functions. In this section, we shall describe Riemann’s theory and its extensions by Du Bois-Reymond and Darboux. Still more general theories, not treated here, are due to Lebesgue (in 1902) and Kurzweil in 1957. General Assumptions. Throughout this section, we shall consider functions f : [a, b] → R, where [a, b] = {x | a ≤ x ≤ b} is a bounded interval and f (x) is a bounded function, i.e., (5.2)

∃ M ≥ 0 ∀ x ∈ [a, b] |f (x)| ≤ M.

Otherwise, the definition of Darboux sums (below) would not be possible. Situations that violate one of these assumptions will be discussed in Sect. III.8.

Definitions and Criteria of Integrability We want to define the integral as the area between the function and the horizontal axis. The idea is to divide the interval [a, b] into small subintervals and to approximate the area by a sum of small rectangles. A division into subintervals is denoted by (5.3)

D = {x0 , x1 , x2 , . . . , xn }

(where a = x0 < x1 < . . . < xn = b) and the length of a subinterval is δi = xi − xi−1 . We then define the lower and upper Darboux sums (see Fig. 5.1) by

222 III. Foundations of Classical Analysis

(5.4)

s(D) =

n X

f i δi ,

S(D) =

i=1

n X

Fi δi ,

i=1

where (5.5)

fi =

inf

xi−1 ≤x≤xi

f (x),

Fi =

sup

f (x).

xi−1 ≤x≤xi

Obviously, we have s(D) ≤ S(D) and any reasonable definition of the integral Rb a f (x) dx must give a value between s(D) and S(D). A division D ′ of [a, b] is called a refinement of D, if it contains the points of D, i.e., if D′ ⊃ D. Fi

(Fi −fi )δi

fi

x0 δi

xnx0 δi s(D)

S(D)

xnx0 δi S(D) − s(D)

xn

FIGURE 5.1. Darboux sums

s(D)

s(D′ )

S(D′ )

S(D)

FIGURE 5.2. Refinement of a division

(5.1) Lemma. If D′ is a refinement of D, then s(D) ≤ s(D′ ) ≤ S(D′ ) ≤ S(D). Proof. Adding a single point to the division D increases the lower Darboux sum (or does not change it) and decreases the upper sum (or does not change it, Fig. 5.2). Repeated addition of points yields the statement. ⊔ ⊓

III.5 The Riemann Integral 223

(5.2) Lemma. Let D1 and D2 be two arbitrary divisions, then s(D1 ) ≤ S(D2 ). Proof. We take D′ = D1 ∪ D2 , the division containing all points of the two divisions (points appearing twice are counted only once). Since D′ is a refinement of D1 and of D2 , the statement follows from Lemma 5.1. ⊔ ⊓ Lemma 5.2 implies that, for a given function f : [a, b] → R, the set of lower Darboux sums is majorized by every upper Darboux sum (and vice versa): s(D) (5.6)

S(D)

| || | |||

. {z } ||| | ||| || ? Therefore (Theorem 1.12), it makes sense to consider the supremum of the lower sums and the infimum of the upper sums. Following Darboux (1875), we introduce the notation Z b (5.7) f (x) dx = inf S(D) the upper integral, D

a

(5.8)

Z

|

b

f (x) dx = sup s(D)

the lower integral.

D

a

(5.3) Definition. A function f : [a, b] → R, satisfying (5.2), is called integrable (in the sense of Riemann), if the lower and upper integrals (5.7) and (5.8) are equal. In that case, we remove the bars in (5.7) and (5.8) and we obtain the “Riemann integral”. (5.4) Theorem. A function f : [a, b] → R is integrable if and only if e ∀ε > 0 ∃D

(5.9)

e − s(D) e < ε. S(D)

Proof. By definition, the function f (x) is integrable if and only if the two sets in (5.6) are arbitrarily close. This means that, for a given ε > 0, there exist two e = D1 ∪ D2 and divisions D1 and D2 such that S(D2 ) − s(D1 ) < ε. Taking D applying Lemma 5.1 yields the statement. ⊔ ⊓

(5.5) Example. Consider the function f (x) = x on an interval [a, b]. For the equidistant division Dn = {xi = a + ih | i = 0, 1, . . . , n, h = (b − a)/n}, we obtain from (I.1.28) that s(Dn ) =

n X i=1

S(Dn ) =

n X i=1

xi−1 · (xi − xi−1 ) = xi · (xi − xi−1 ) =

a2 (b − a)2 b2 − − 2 2 2n

a2 (b − a)2 b2 − + , 2 2 2n

224 III. Foundations of Classical Analysis

so that S(Dn ) − s(Dn ) = (b − a)2 /n. For sufficiently large n this expression is smaller than any ε > 0. Therefore, the function is integrable and the integral equals b2 /2 − a2 /2. (5.6) Example. Dirichlet’s function f : [0, 1] → R, defined by (see (3.3))  1 x rational f (x) = 0 x irrational, is not integrable in the sense of Riemann, because in every subinterval there are rational and irrational numbers so that fi = 0 and Fi = 1 for all i. Consequently, s(D) = 0, S(D) = 1 for all divisions. (5.7) Example. The function f : [0, 1] → R (see (3.4))  0 x irrational or x = 0 f (x) = 1/q x = p/q reduced fraction is discontinuous at all positive rational x. However, for a fixed ε > 0, only a finite number (say k) of x-values are such that f (x) > ε. We now choose a division D with maxi δi < ε/k, such that the x-values for which f (x) > ε lie in the interior of the subintervals. Because of f (x) ≤ 1, this implies S(D) ≤ ε + k · max δi < 2ε. i

Since s(D) = 0, we see that our function is integrable and that

R1 0

f (x) dx = 0.

The Theorem of Du Bois-Reymond and Darboux. I feel, however, that the manner in which the criterion of integrability was formulated leaves something to be desired. (Du Bois-Reymond 1875, p. 259)

(5.8) Theorem (Du Bois-Reymond 1875, Darboux 1875). A function f (x), satisfying (5.2), is integrable if and only if ∀ ε > 0 ∃ δ > 0 ∀ D ∈ Dδ

S(D) − s(D) < ε.

Here, Dδ denotes the set of all divisions satisfying maxi δi ≤ δ. Proof. The “if” part is a simple consequence of Theorem 5.4. The difficulty of the “only if” part resides in the fact that the division D, about which we know nothing e of Theorem 5.4. but maxi δi ≤ δ, can be quite different from the D e be a division satisfying (5.9), i.e., the shaded Let ε > 0 be fixed and let D e = S(D) e − s(D) e in Fig. 5.3a is smaller than ε. The important point is that area ∆ e D = {e x0 , x e1 , . . . , x en˜ } consists of a finite number of points. Now take an arbitrary

III.5 The Riemann Integral 225

a)

b)

e 0 )

are again integrable. Proof. We shall use throughout the proof the fact that Fi − fi represents the least upper bound for the variations of f (x) on [xi−1 , xi ], i.e., (5.13)

sup x,y∈[xi−1 ,xi ]

|f (x) − f (y)| = Fi − fi .

Indeed, suppose that ε > 0 is a given number. By the definition of Fi and fi , there exist ξ, η ∈ [xi−1 , xi ] such that f (ξ) > Fi − ε, f (η) < fi + ε and therefore f (ξ) − f (η) > Fi − fi − 2ε. Consequently, Fi − fi is not only an upper bound for |f (x) − f (y)|, but also the least upper bound. a) Let h(x) = f (x) + g(x), and denote by Fi , Gi , Hi , respectively, fi , gi , hi , the supremum, respectively, infimum of f , g, h, on [xi−1 , xi ] (see (5.5)). We then have for x, y ∈ [xi−1 , xi ], using the triangle inequality and (5.13), (5.14)

|h(x) − h(y)| ≤ |f (x) − f (y)| + |g(x) − g(y)| ≤ (Fi − fi ) + (Gi − gi ).

III.5 The Riemann Integral 227

Thus, Eq. (5.13), applied to the function h, shows that (Hi − hi ) ≤ (Fi − fi ) + (Gi − gi ), and the differences of the upper and lower Darboux sums satisfy X X X (5.15) (Hi − hi )δi ≤ (Fi − fi )δi + (Gi − gi )δi .

For a given ε > 0 we choose a division D (Theorem 5.4) such that each term in the sum on the right side of (5.15) is smaller than ε (in fact, we have two different divisions for f and g, P but by taking their union we may suppose that they are the same). Consequently, i (Hi − hi )δi < 2ε and the function h(x) = f (x) + g(x) is integrable by Theorem 5.4. b) The proofs of the remaining assertions are very similar. For example, for h(x) = λ · f (x) we use |h(x) − h(y)| = |λ| · |f (x) − f (y)| instead of (5.14), conclude that (Hi −hi ) ≤ |λ|·(Fi −fi ), and deduce integrability as above. For the product h(x) = f (x) · g(x) we use |h(x) − h(y)| ≤ |f (x)| · |g(x) − g(y)| + |g(y)| · |f (x) − f (y)| ≤ M · |g(x) − g(y)| + N · |f (x) − f (y)|

(both functions f (x) and g(x) are bounded by assumption (5.2)). Finally, for the last assertion it suffices to prove that 1/g(x) is integrable (because f (x)/g(x) = f (x) · (1/g(x)). We set h(x) = 1/g(x) and replace (5.14) by |g(y) − g(x)| |g(x) − g(y)| |h(x) − h(y)| = ≤ . |g(y)| · |g(x)| C2 ⊔ ⊓ Since the constant function and f (x) = x are integrable (Example 5.5), the above theorem implies that polynomials and rational functions (away from singularities) are integrable. The following theorem was asserted by Cauchy (1823), but was proved rigorously only some 50 years later with the notion of uniform continuity. (5.10) Theorem. If f : [a, b] → R is continuous, then it is integrable.

Proof. The essential point is that f is uniformly continuous (Theorem 4.5). This means that for a given ε > 0 there exists a δ > 0 such that |x − y| < δ

=⇒

|f (x) − f (y)| < ε.

We take a division D satisfying maxi δi < δ. For x, y ∈ [xi−1 , xi ] we thus have |f (x) − f (y)| < ε and,P by (5.13), Fi − fi ≤ ε. This implies that S(D) − s(D) = P n n i=1 (Fi − fi )δi ≤ ε i=1 δi = ε(b − a) and the integrability of f (x) follows from Theorem 5.4. ⊔ ⊓

228 III. Foundations of Classical Analysis

(5.11) Theorem. If f : [a, b] → R is nondecreasing (or nonincreasing), then it is integrable. Proof. The smallest value of a nondecreasing function is at the left end point and the largest at the right end point of the interval [xi−1 , xi ]. Hence, fi = f (xi−1 ), Fi = f (xi ) so that fi+1 = Fi for i = 1, . . . , n − 1. The idea is now to consider equidistant divisions where the length of all subintervals is equal to δ. We then have X
(Fi −fi )δ = F1 δ−f1 δ+F2 δ−f2 δ+F3 δ−f3 δ+. . . = f (xn )−f (x0 ) ·δ < ε, if δ is sufficiently small. This proves the integrability of f (x).

⊔ ⊓

(5.12) Remark. If we change an integrable function at a finite number of points, the function remains integrable and the value of the integral does not change. This is seen by an argument similar to that of Example 5.7 above. (5.13) Remark. Let a < b < c and assume that f : [a, c] → R is a function whose restrictions to [a, b] and to [b, c] are integrable. Then f is integrable on [a, c] and we have Z c Z b Z c (5.16) f (x) dx = f (x) dx + f (x) dx. a

a

b

This holds because adding the Darboux sums for the restrictions to [a, b] and [b, c] yields a Darboux sum for [a, c]. For a > b or a = b we define Z b Z a Z a (5.17) f (x) dx = − f (x) dx and f (x) dx = 0, a

b

a

so that Eq. (5.16) is true for any triple (a, b, c).

Inequalities and the Mean Value Theorem The following inequalities are often useful for estimating integrals. We have already used them in Sect. II.10 to obtain the estimates (II.10.15). (5.14) Theorem. If f (x) and g(x) are integrable on [a, b] (with a < b) and if f (x) ≤ g(x) for all x ∈ [a, b], then Z b Z b f (x) dx ≤ g(x) dx. a

a

P P Proof. The Riemann sums satisfy ni=1 f (ξi )δi ≤ ni=1 g(ξi )δi , because δi > 0. For maxi δi → 0 we obtain the above inequality (see (5.12) and Theorem 1.6). ⊔ ⊓

III.5 The Riemann Integral 229

(5.15) Corollary. For integrable functions we have Z b Z b f (x) dx ≤ |f (x)| dx. a

a

Proof. We apply Theorem 5.14 to −|f (x)| ≤ f (x) ≤ |f (x)|.

⊔ ⊓

By applying Corollary 5.15 to a product of two integrable functions f (x) · g(x) and using |f (x) · g(x)| ≤ M · |g(x)|, where M = supx∈[a,b] |f (x)|, we obtain the following useful estimate: Z b Z b (5.18) f (x) · g(x) dx ≤ sup |f (x)| · |g(x)| dx. x∈[a,b]

a

a

The next inequality is similar to (5.18), but treats the two functions f and g symmetrically.

(5.16) The Cauchy-Schwarz Inequality (Cauchy 1821 in Rn , Bunyakovski 1859 for integrals, Schwarz 1885, §15, for double integrals). For integrable functions f (x) and g(x) we have s Z b sZ b Z b 2 (5.19) f (x)g(x) dx ≤ f (x) dx · g 2 (x) dx· a

a

a

Proof. By Theorem 5.9, we know that f · g, f , and g 2 are integrable. Using Theorem 5.14 and the linearity of the integral, we have Z b 2 0≤ f (x) − γg(x) dx 2

a

=

Z

a

b

f 2 (x) dx − 2γ

Z

a

b

f (x)g(x) dx + γ 2

Z

b

g 2 (x) dx.

a

Rb Rb Rb We put A = a f 2 (x) dx, B = a f (x)g(x) dx, C = a g 2 (x) dx, and we see that A − 2γB + γ 2 C ≥ 0 for all real γ. For C = 0 this implies that B = 0. For C 6= 0 the discriminant of the quadratic equation cannot be positive (see (I.1.12)). Therefore, we must have B 2 ≤ AC, which is (5.19). ⊔ ⊓ (5.17) The Mean Value Theorem (Cauchy 1821). If f : [a, b] → R is a continuous function, then there exists ξ ∈ [a, b] such that Z b (5.20) f (x) dx = f (ξ) · (b − a). a

Proof. Let m and M be the minimum and the maximum of f (x) on [a, b] (see Theorem 3.6), so that m ≤ f (x) ≤ M for all x ∈ [a, b]. Applying Theorem 5.14 and dividing by (b − a) yields

230 III. Foundations of Classical Analysis

1 · m≤ b−a

Z

a

b

f (x) dx ≤ M.

Rb The value a f (x) dx/(b − a) lies between m = f (u) and M = f (U ). Therefore, by Bolzano’s Theorem 3.5, we deduce the existence of ξ ∈ [a, b] such that this value equals f (ξ). This proves Eq. (5.20). ⊔ ⊓ (5.18) Theorem (Cauchy 1821). Let f : [a, b] → R be continuous and let g : [a, b] → R be an integrable function that is everywhere positive (or everywhere negative). Then, there exists ξ ∈ [a, b] such that Z

(5.21)

b

f (x)g(x) dx = f (ξ) a

Z

b

g(x) dx. a

Proof. Suppose that g(x) ≥ 0 for all x (otherwise replace g by −g). In this situation, we have m · g(x) ≤ f (x)g(x) ≤ M · g(x)

for

x ∈ [a, b],

where m and M are the minimum and maximum of f (x). The rest of the proof is the same as for the Mean Value Theorem. ⊔ ⊓

Integration of Infinite Series Until very recently it was believed, that the integral of a convergent series . . . is equal to the sum of the integrals of the individual terms, and Mr. Weierstrass was the first to observe . . . (Heine 1870, Ueber trig. Reihen, J. f. Math., vol. 70, p. 353)

On several occasions we found it useful to integrate an infinite series term by term (e.g., in the derivation of Mercator’s series (I.3.13) and in the examples of Sect. II.6). This means that we exchanged integration with a limit of functions. We will discuss here under what conditions this is permitted. First Example. Let r1 , r2 , r3 , r4 , . . . be a sequence containing all rational numbers between 0 and 1, for example 1 1 2 1 2 3 1 2 3 4 1 , , , , , , , , , , , ... . 2 3 3 4 4 4 5 5 5 5 6 We then define (5.22)

fn (x) =



1 0

if x ∈ {r1 , r2 , r3 , . . . , rn } else.

By Remark 5.12, each function fn : [0, 1] → R is integrable with integral zero. However, the limit function f (x), which is Dirichlet’s function of Example 5.6, is not integrable. (The Lebesgue integral will get rid of this difficulty.)

III.5 The Riemann Integral 231

Second Example. The graphs of the functions  2 0 ≤ x ≤ 1/n  n x 2 (5.23) fn (x) = 2n − n x 1/n ≤ x ≤ 2/n   0 2/n ≤ x ≤ 2 are triangles with decreasing bases and increasing altitudes with the property that Z 2 fn (x) dx = 1 for all n.

4

3

2

n=4

n=3

n=2

0

However, the limit function is f (x) = 0 for all x ∈ [0, 1]. Here, f (x) is integrable, but Z 2 Z 2 fn (x) dx 6= lim fn (x) dx. lim n→∞

n=1

1

0 0

1

2

0 n→∞

0

(5.19) Theorem. Consider a sequence fn (x) of integrable functions and suppose that it converges uniformly on [a, b] to a function f (x). Then f : [a, b] → R is integrable and Z b Z b lim fn (x) dx = f (x) dx. n→∞

a

a

Proof. Uniform convergence means that, for a given ε > 0, there exists an integer N such that for all n ≥ N and for all x ∈ [a, b] we have |fn (x) − f (x)| < ε. Consequently, we have for all x, y ∈ [a, b] that |f (x) − f (y)| ≤ |fN (x) − fN (y)| + 2ε. Applying (5.13), we see that (Fi − fi ) ≤ (FN i − fN i ) + 2ε, where, as in (5.5), we have used the notation FN i = supxi−1 ≤x≤xi fN (x) and fN i = inf xi−1 ≤x≤xi fN (x). The function fN (x) is integrable, so that for a suitable division of [a, b] the difference of the upper and the lower Darboux P sums, i.e., (F than ε (Theorem 5.4). This implies that N i − fN i )δi , is smaller i
P (F − f )δ < ε 1 + 2(b − a) and f (x) is seen to be integrable. i i i i Once the integrability of the limit function f (x) is established, Corollary 5.15 implies that for n ≥ N Z b Z b Z b fn (x) − f (x) dx ≤ ε(b − a). fn (x) dx − f (x) dx ≤ a

a

a

This implies the conclusion of the theorem.

⊔ ⊓

232 III. Foundations of Classical Analysis

(5.20) Corollary. P Consider a sequence fn (x) of integrable functions and suppose that the series ∞ n=0 fn (x) converges uniformly on [a, b]. Then, we have ∞ Z X

n=0

b

fn (x) dx =

a

Z

∞ bX

fn (x) dx.

a n=0

FIGURE 5.4. Riemann’s example of an integrable function

Riemann’s Example.

Since these functions have never been considered yet, it will be useful to start from a particular example. (Riemann 1854, Werke, p. 228)

Riemann (1854), in order to demonstrate the power of his theory of integration, proposed the following example of a function that is discontinuous in every interval (see Fig. 5.4): (5.24)

∞ X B(nx) f (x) = , n2 n=1

where

B(x) =



x − hxi 0

if x 6= k/2 if x = k/2

and hxi denotes the nearest integer to x. This function is discontinuous at x = 1/2, 1/4, 3/4, 1/6, 3/6, 5/6, . . . , nevertheless, the series (5.24) converges uniformly by Theorem 4.3 and the functions fn (x) are integrable by Remark 5.13. Hence, f is integrable.

Exercises 5.1 For the function f (x) =



1

if x = 0, 21 , 13 , 14 , 15 , . . .

x

otherwise

III.5 The Riemann Integral 233

and a given ε > 0, say ε = 0.01, construct explicitly a division for which S(D) − s(D) < ε. This will make clear that f is integrable in the sense of Riemann. 5.2 Consider the function f (x) = x2 on the interval [0, 1]. Compute the lower and upper Darboux sums for the equidistant division xi = i/n, i = 0, 1, . . . , n. Conclude from the results obtained that f is integrable. 5.3 Show that the numerical approximations obtained from the trapezoidal rule (see Sect. II.6), Z b  f (ξ ) f (ξN )  0 + f (ξ1 )+ f (ξ2 )+ f (ξ3 )+ . . .+ f (ξN −1 )+ f (x) dx ≈ h 2 2 a (h = (b − a)/N and ξi = a + ih), as well as for Simpson’s rule (N even), Z b  h f (ξ0 ) + 4f (ξ1 ) + 2f (ξ2 ) + 4f (ξ3 ) + . . . + f (ξN ) , f (x) dx ≈ 3 a

are Riemann sums for a certain division D. Therefore, convergence of these methods is ensured for N → ∞ for all Riemann integrable functions. 5.4 (Dini 1878, Chap. 13). Show that Z π
ln 1 − 2α cos x + α2 dx = 0 for α2 < 1, Z0 π
ln 1 − 2α cos x + α2 dx = π ln α2 for α2 > 1, 0

by computing Riemann sums for an equidistant division xi = iπ/n, with ξi the left end point xi−1 . The Riemann sums will become the logarithm of a product with which we are familiar (see Sect. I.5). 5.5 Let f : [a, b] → R satisfy i) f is continuous, ii) ∀x ∈ [a, b] we have f (x) ≥ 0, and iii) ∃x0 ∈ (a, b) with f (x0 ) > 0. Then, show that Z b (5.25) f (x) dx > 0. a

Show with the help of counterexamples that each of the three hypotheses i), ii), and iii) is necessary for proving (5.25). 5.6 Compute the integrals Z π/2 π 1 · 3 · 5 · . . . · (2n − 1) , sin2n x dx = · 2 2 · 4 · 6 · . . . · 2n 0 Z π/2 2 · 4 · 6 · . . . · 2n sin2n+1 x dx = . 3 · 5 · 7 · . . . · (2n + 1) 0 Then, use 0 < sin x < 1 for 0 < x < π/2 and Theorem 5.14 to establish Z π/2 Z π/2 Z π/2 2n 2n+1 sin x dx > sin x dx > sin2n+2 x dx. 0

0

0

234 III. Foundations of Classical Analysis

The above values inserted into these inequalities lead to a proof of Wallis’s product (I.5.27) with a rigorous error estimate. 5.7 Show that Z Z 1 4 Z 1 1 1 4 x (1 − x)4 x (1 − x)4 dx ≤ dx ≤ x4 (1 − x)4 dx. 2 2 0 1 + x 0 0

The actual computation of these integrals leads to an amusing result (old souvenirs from Sect. I.6). R1 Hint. To calculate 0 x4 (1 − x)4 dx see Exercise II.4.3.

5.8 Show that the series

1 = 1 − x2 + x4 − x6 + x8 − . . . 1 + x2 converges uniformly on A = [−b, b] for each b with 0 < b < 1. Hence, this series can be integrated term by term on A = [0, b] (or on A = [−b, 0]) and leads to the well-known series for arctan b. a)

b)

n=4

1

1 n=3

n=2 n=1

n=1

n=2 0

1

0

1

FIGURE 5.5. Exchange of lim and integral

5.9 For the following sequences of functions fn : [0, 1] → R (Fig. 5.5), a)

fn (x) =

nx , (1 + n2 x2 )2

b)

fn (x) =

n2 x , (1 + n2 x2 )2

compute limn→∞ fn (x) (distinguish the cases x = 0 and x 6= 0). Find the maximal point of fn (x) and decide whether convergence is uniform. Finally, check whether the following equality holds: Z 1 Z 1 lim fn (x) dx = lim fn (x) dx. n→∞

0

0 n→∞

III.6 Differentiable Functions 235

III.6 Differentiable Functions . . . rigor, which I wanted to be absolute in my Cours d’analyse, . . . (Cauchy 1829, Lec¸ons) The total variation f (x + h) − f (x) . . . can in general be decomposed into two terms . . . (Weierstrass 1861)

The derivative of a function was introduced and discussed in Sect. II.1. Now that we have the notion of limit at our disposal, it is possible to give a precise definition. (6.1) Definition (Cauchy 1821). Let I be an interval and let x0 ∈ I. The function f : I → R is differentiable at x0 if the limit f ′ (x0 ) = lim

(6.1)

x→x0

f (x) − f (x0 ) x − x0

exists. The value of this limit is the derivative of f at x0 and is denoted by f ′ (x0 ). If the function f is differentiable at all points of I and if f ′ : I → R is continuous, then f is called continuously differentiable. Sometimes it is advantageous to write x = x0 + h, so that f ′ (x0 ) = lim

(6.2)

h→0

f (x0 + h) − f (x0 ) . h

One can also, for a given x0 , consider the function r : I → R defined by r(x0 ) = 0 and (6.3)

r(x) =

f (x) − f (x0 ) − f ′ (x0 ) x − x0

for x 6= x0 .

Then, Eq. (6.1) is equivalent to limx→x0 r(x) = 0 and we have the following criterion. (6.2) Weierstrass’s Formulation (Weierstrass 1861, see the above quotation). A function f (x) is differentiable at x0 if and only if there exists a number f ′ (x0 ) and a function r(x), continuous at x0 and satisfying r(x0 ) = 0, such that (6.4)

f (x) = f (x0 ) + f ′ (x0 )(x − x0 ) + r(x)(x − x0 ).

⊔ ⊓

Equation (6.4) has the advantage of containing no limit (this is replaced by the continuity of r(x)) and of exhibiting the equation of tangent line y = f (x0 ) + f ′ (x0 )(x − x0 ) to f (x) at x = x0 . Moreover, it will be the basis for the differentiability theory of functions of several variables. Still simpler formulas and proofs are obtained, if the two terms in Eq. (6.4) are collected by setting (6.5)

ϕ(x) = f ′ (x0 ) + r(x).

236 III. Foundations of Classical Analysis

(6.3) Carath´eodory’s Formulation (Carath´eodory 1950, p. 121). A function f (x) is differentiable at x0 if and only if there exists a function ϕ(x), continuous at x0 , such that (6.6)

f (x) = f (x0 ) + ϕ(x)(x − x0 ).

The value ϕ(x0 ) is the derivative f ′ (x0 ) of f at x0 . We see immediately from (6.6) that if f is differentiable at x0 , then it is also continuous at x0 . Furthermore, since from (6.5) and (6.3) (or directly from (6.6)) (6.7)

ϕ(x) =

f (x) − f (x0 ) x − x0

for x 6= x0

is uniquely determined for x 6= x0 , the derivative f ′ (x0 ) is uniquely determined if it exists. Remarks and Examples. 1. Obviously, the functions f (x) = 1 and f (x) = x are differentiable. The differentiability of f (x) = x2 follows, for example, from (6.6) with the identity x2 − x20 = (x + x0 )(x − x0 ) (see also Sect. II.1). 2. We emphasize that differentiability at x0 is a local property. Changing the function outside (x0 −ε, x0 +ε) for some ε > 0 changes neither its differentiability at x0 nor the derivative f ′ (x0 ). 3. If I = [a, b] is a closed interval and x0 = a, then (6.1) should be replaced by the right-sided limit. 4. Consider the function f (x) = |x| (absolute value). At x0 > 0, it is differentiable with f ′ (x0 ) = 1; at x0 < 0 it is also differentiable, but with derivative f ′ (x0 ) = −1. This function is not differentiable at x0 = 0, because f (x)/x = |x|/x does not have a limit for x → 0. 5. The function  0 if x is irrational or integer f (x) = 2 1/q if x = p/q (reduced fraction) is discontinuous at every non-integer rational x0 . It is, nevertheless, differentiable at x0 = 0, since the function ϕ(x) of Eq. (6.6) becomes ϕ(x) = f (x)/x. Since |f (x)| ≤ |x|2 , we have limx→0 ϕ(x) = 0 and f ′ (x0 ) = 0.

0

(6.4) Theorem. If f : (a, b) → R is differentiable at x0 ∈ (a, b) and f ′ (x0 ) > 0, then there exists δ > 0 such that f (x) > f (x0 ) f (x) < f (x0 )

for all x satisfying x0 < x < x0 + δ, for all x satisfying x0 − δ < x < x0 .

If the function possesses a maximum (or minimum) at x0 , then f ′ (x0 ) = 0. Proof. f ′ (x0 ) > 0 means that ϕ(x0 ) > 0 (see (6.6)). By continuity, ϕ(x) > 0 in a neighborhood of x0 . Now the stated inequalities follow from (6.7).

III.6 Differentiable Functions 237

If the function possesses a maximum at x0 , then we have f (x) ≤ f (x0 ) on both sides of x0 . This is only possible if f ′ (x0 ) = 0. ⊔ ⊓ (6.5) Remark. The statement of Theorem 6.4 does not imply that a function, satisfying f ′ (x0 ) > 0, is monotonically increasing in a neighborhood of x0 . As a counterexample, consider the function f (x) (see Fig. 6.1), given by f (0) = 0 and
f (x) = x + x2 sin 1/x2 for x 6= 0.

It is differentiable everywhere and satisfies f ′ (0) = 1 (because f (x) = x+r(x)·x with |r(x)| ≤ |x|). For x 6= 0 the derivative 1 1 2 f ′ (x) = 1 + 2x sin 2 − cos 2 x x x

oscillates strongly near the origin. Hence, even though f (x) is contained between two parabolas, there are points with negative derivatives arbitrarily close to the origin. By Theorem 6.4, there exist points ξ1 < ξ2 , arbitrarily close to 0, for which f (ξ1 ) > f (ξ2 ). We shall show later (Corollary 6.12) that, if f ′ (x) > 0 for all x ∈ (a, b), the function is monotonically increasing. Thus, this counterexample is only possible because f is not continuously differentiable.

.5

50

f

.0

.5

.0

f ′

.5

FIGURE 6.1. Graph of the function y = x + x2 sin(1/x2 ) and its derivative

(6.6) Theorem. If f and g are differentiable at x0 , then so are f + g,

f · g,

f /g ( if g(x0 ) 6= 0).

The formulas of Sect. II.1 for their derivatives are correct. Proof. We shall present two different proofs for the product f · g. For f + g and f /g the proofs are similar.

238 III. Foundations of Classical Analysis

The first proof is based on the identity f (x)g(x) − f (x0 )g(x0 ) g(x) − g(x0 ) f (x) − f (x0 ) = f (x) + g(x0 ) , x − x0 x − x0 x − x0

which is obtained by adding and subtracting the term f (x)g(x0 ). Using the continuity of f at x0 (Eq. (6.4)), the differentiability of f and g, and Theorem 1.5, we see that for x → x0 the expression on the right has the finite limit f (x0 )g ′ (x0 ) + g(x0 )f ′ (x0 ). Hence, the product f · g is differentiable at x0 . Our second proof is based on Carath´eodory’s formulation 6.3. By hypothesis, we have (6.8)

f (x) = f (x0 ) + ϕ(x)(x − x0 ), g(x) = g(x0 ) + ψ(x)(x − x0 ),

ϕ(x0 ) = f ′ (x0 ), ψ(x0 ) = g ′ (x0 ).

We multiply both equations of (6.8), and obtain   f (x)g(x) = f (x0 )g(x0 ) + f (x0 )ψ(x) + g(x0 )ϕ(x) + ϕ · ψ · (x − x0 ) (x − x0 ).

The function in tall brackets is evidently continuous at x0 and its value for x = x0 is f (x0 )g ′ (x0 ) + g(x0 )f ′ (x0 ). ⊔ ⊓

(6.7) Theorem (Chain rule for composite functions). If y = f (x) is differentiable at x0 and if z = g(y)
is differentiable at y0 = f (x0 ), then the composite function (g ◦ f )(x) = g f (x) is differentiable at x0 , and we have (6.9)

(g ◦ f )′ (x0 ) = g ′ (y0 ) · f ′ (x0 ).

Many of our students will appreciate the pithy elegance of this proof. (Kuhn 1991)

Proof. We use Eq. (6.6) to write the hypothesis in the form f (x) − f (x0 ) = ϕ(x)(x − x0 ), g(y) − g(y0 ) = ψ(y)(y − y0 ),

ϕ(x0 ) = f ′ (x0 ), ψ(y0 ) = g ′ (y0 ).

Inserting y − y0 = f (x) − f (x0 ) from the first equation into the second, we obtain


g f (x) − g f (x0 ) = ψ f (x) ϕ(x)(x − x0 ).
The function ψ f (x) ϕ(x) is again continuous at x0 , and its value for x = x0 is
⊔ ⊓ g ′ f (x0 ) · f ′ (x0 ).

(6.8) Theorem (Inverse functions). Let f : I → J be bijective, continuous, and differentiable at x0 ∈ I, and suppose that f ′ (x0 ) 6= 0. Then, the inverse function f −1 : J → I is differentiable at y0 = f (x0 ), and we have (6.10)

(f −1 )′ (y0 ) =

1 . f ′ (x0 )

III.6 Differentiable Functions 239

Proof. In Carath´eodory’s formulation (6.6), we have by hypothesis ϕ(x0 ) = f ′ (x0 ),

f (x) − f (x0 ) = ϕ(x)(x − x0 ),

we replace x and x0 by f −1 (y) and f −1 (y0 ), and f (x) and f (x0 ) by y and y0 , and get

y − y0 = ϕ f −1 (y) f −1 (y) − f −1 (y0 ) .

From the proof of Theorem 3.9
it follows that f −1 (y) is continuous
at y0 . Because by hypothesis ϕ f −1 (y0 ) 6= 0, we therefore have ϕ f −1 (y) 6= 0 in a neighborhood of y0 and we may divide this formula to obtain f −1 (y) − f −1 (y0 ) =

1 ϕ


(y − y0 ).

f −1 (y)


This concludes the proof, since the function 1/ϕ f −1 (y) is continuous at y0 .

⊔ ⊓

The Fundamental Theorem of Differential Calculus Formula (II.4.6) is the central result of all the computations of Sect. II.4. We shall give here a rigorous proof of this result. In particular, we shall show that every continuous function f (x) has a primitive, which is unique up to an additive constant. (6.9) Theorem (Existence of a primitive). Let f : [a, b] → R be a continuous function. The function Z x (6.11) F (x) = f (t) dt a

(which exists by Theorem 5.10) is differentiable on (a, b) and satisfies F ′ (x) = f (x). Hence, it is a primitive of f (x). Proof. By Eq. (5.16), we have (6.12)

F (x) − F (x0 ) =

Z

x

f (t) dt. x0

Applying the Mean Value Theorem 5.17, we get F (x) − F (x0 ) = f (ξ)(x − x0 ), where ξ = ξ(x, x0 ) lies between x and x0 . For x → x0 the value ξ(x, x0 ) necessarily tends to x0 , so that by continuity of f at x0 , we have limx→x0 f (ξ) = f (x0 ). This proves (see (6.6)) the differentiability of F (x), with F ′ (x0 ) = f (x0 ). ⊔ ⊓

240 III. Foundations of Classical Analysis

Uniqueness of Primitives. This was supplied by the mean value theorem; and it was Cauchy’s great service to have recognized its fundamental importance. . . . because of this, we adjudge Cauchy as the founder of exact infinitesimal calculus. (F. Klein 1908, Engl. ed. p. 213) See the beautiful proof of this theorem due to Mr. O. Bonnet, in the Trait´e de Calcul diff´erentiel et int´egral of Mr. Serret, vol. I, p. 17. (Darboux 1875, p. 111)

Our next aim is to prove the uniqueness (up to an additive constant) of the primitive. The following concatenation of theorems, which accomplishes this task, has been one of the cornerstones of the foundations of Analysis since Serret’s book (1868; Serret attributes these ideas to O. Bonnet; see the quotations). (6.10) Theorem (Rolle 1690). Let f : [a, b] → R be continuous on [a, b], differentiable on (a, b), and such that f (a) = f (b). Then, there exists a ξ ∈ (a, b) such that f ′ (ξ) = 0.

(6.13)

Proof. From Theorem 3.6, we know there exist u, U ∈ [a, b] such that f (u) ≤ f (x) ≤ f (U ) for all x ∈ [a, b]. We now distinguish two situations. If f (u) = f (U ), then f (x) is constant and its derivative is zero everywhere. If f (u) < f (U ), then at least one of the two values (say f (U )) is different from f (a) = f (b). We then have a < U < b, and by Theorem 6.4, f ′ (U ) = 0. ⊔ ⊓ (6.11) Theorem (Lagrange 1797). Let f : [a, b] → R be continuous on [a, b] and differentiable on (a, b). Then, there exists a number ξ ∈ (a, b) such that f (b) − f (a) = f ′ (ξ)(b − a).

(6.14)

f(b) f(b)

f(a)

a

b

a f(a)

b

FIGURE 6.2. Proof of Rolle’s and Lagrange’s Theorems

Proof. The idea is to subtract from f (x) the straight line connecting the points
(a, f (a)) and (b, f (b)), of slope f (b) − f (a) /(b − a), and to apply Rolle’s Theorem (Fig. 6.2). We define

III.6 Differentiable Functions 241

(6.15)

 f (b) − f (a)  . h(x) = f (x) − f (a) + (x − a) b−a

Because of h(a) = h(b) = 0 and

h′ (x) = f ′ (x) −

f (b) − f (a) , b−a

Eq. (6.14) follows from h′ (ξ) = 0 (Theorem 6.10).

⊔ ⊓

(6.12) Corollary. Let f, g : [a, b] → R be continuous on [a, b] and differentiable on (a, b). We then have a) b) c) d)

if f ′ (ξ) = 0 for all ξ ∈ (a, b), then f (x) = C (constant); if f ′ (ξ) = g ′ (ξ) for all ξ ∈ (a, b), then f (x) = g(x) + C; if f ′ (ξ) > 0 for all ξ ∈ (a, b), then f (x) is monotonically increasing, i.e., f (x1 ) < f (x2 ) for a ≤ x1 < x2 ≤ b; and if |f ′ (ξ)| ≤ M for all ξ ∈ (a, b), then |f (x1 ) − f (x2 )| ≤ M |x1 − x2 | for x1 , x2 ∈ [a, b].

Proof. Applying Eq. (6.14) to the interval [a, x] yields statement (a) with C = f (a). Statement (b) follows from (a). The remaining two statements are obtained from Theorem 6.11 applied to the interval [x1 , x2 ]. ⊔ ⊓ (6.13) The Fundamental Theorem of Differential Calculus. Let f (x) be a continuous function on [a, b]. Then, there exists a primitive F (x) of f (x), unique up to an additive constant, and we have Z b (6.16) f (x) dx = F (b) − F (a). a

Proof. The existence of F (x) is clear from Theorem 6.9. Uniqueness (up to a constant) is a consequence ofR Corollary 6.12b. If F (x) is an arbitrary primitive of x f (x), then we have F (x) = a f (t) dt + C. Setting x = a yields C = F (a), and Eq. (6.16) is obtained on setting x = b. ⊔ ⊓

Fig. 6.3 shows the impressive genealogical tree of the theorems that are needed for a rigorous proof of the fundamental theorem. If Leibniz had known about this diagram, he might not have had the courage to state and use this theorem. The “Fundamental Theorem of Differential Calculus” allows us to formulate theorems of Differential Calculus (Sect. III.6) as theorems of Integral Calculus (Sect. III.5) and vice versa. This fact was exploited in Sect. II.4 on several occasions. “Integration by Substitution” (Eq. (II.4.14)) and “Integration by Parts” (Eq. (II.4.20)) now have a sound theoretical basis. One has only to require that the functions involved be continuous, so that the integrals exist.

242 III. Foundations of Classical Analysis Fundamental Theorem of Calculus Thm. 6.11 Lagrange

Thm. 5.10 R exist

Thm. 6.10 Rolle

R cRem. R b5.13R c a

=

a

+

Thm. 6.4 f ′ > 0 ⇒ ...

b

RThm. 5.14 R f≤

g

Thm. 1.6 lim ↔≤

Thm. 5.17 Mean Val.

Thm. 3.5 intermed. val.

Thm. 1.12 ∃ sup

Thm. 1.5 lim ↔ +

Thm. 4.5 Unif. Cont.

Thm. 3.6 Max, Min

Thm. 1.17 Bolz. Weier.

Thm. 3.3 cont. func.

Thm. 1.8 Cauchy sequ. conv.

Def. of real numbers, Def. of lim, Logic FIGURE 6.3. Genealogical tree of the Fundamental Theorem

The Rules of de L’Hospital . . . entirely above the vain glory, which most scientists so avidly seek . . . (Fontenelle’s opinion concerning Guillaume-Franc¸ois-Antoine de L’Hospital, Marquis de Sainte-Mesme et du Montellier, Comte d’Antremonts, Seigneur d’Ouques, 1661–1704) Besides, I acknowledge that I owe very much to the bright minds of the Bernoulli brothers, especially to the young one presently Professor in Groningen. I have made free use of their discoveries . . . (de L’Hospital 1696)

We start with the following generalization of Lagrange’s Theorem 6.11. (6.14) Theorem (Cauchy 1821). Let f : [a, b] → R and g : [a, b] → R be continuous on [a, b] and differentiable on (a, b). If g ′ (x) 6= 0 for a < x < b, then g(b) 6= g(a) and there exists ξ ∈ (a, b) such that (6.17)

f ′ (ξ) f (b) − f (a) = ′ . g(b) − g(a) g (ξ)

III.6 Differentiable Functions 243

Proof. We first observe that g(b) 6= g(a) by Rolle’s theorem, since g ′ (ξ) 6= 0 for all ξ ∈ (a, b). We then note that for g(x) = x this result reduces to Theorem 6.11. Inspired by the proof of this theorem, we replace (6.15) by  
f (b) − f (a) (6.18) h(x) = f (x) − f (a) + g(x) − g(a) . g(b) − g(a)

The conditions of Rolle’s Theorem 6.10 are satisfied, and consequently there exists ξ ∈ (a, b) with h′ (ξ) = 0. This is equivalent to (6.17). ⊔ ⊓

Problem. Suppose we want to compute the limit of a quotient f (x)/g(x). If both functions, f (x) and g(x), tend to 0 or to ∞ when x → b, then we are confronted with undetermined expressions of the form 0 ∞ or . 0 ∞ The following theorems and examples show how such situations can be handled. (6.15) Theorem (Joh. Bernoulli 1691/92, de L’Hospital 1696). Let f : (a, b) → R and g : (a, b) → R be differentiable on (a, b) and suppose that g ′ (x) 6= 0 for a < x < b. If (6.19)

lim f (x) = 0

x→b−

and

lim g(x) = 0

x→b−

and if lim f ′ (x)/g ′ (x) = λ exists, then x→b−

(6.20)

f (x) f ′ (x) = lim ′ . x→b− g(x) x→b− g (x) lim

Proof. The existence of the limit of f ′ (x)/g ′ (x) for x → b− means that for a given ε > 0 there exists a δ > 0 such that f ′ (ξ) − λ < ε for b − δ < ξ < b. (6.21) ′ g (ξ) For u, v ∈ (b − δ, b) it then follows from Theorem 6.14 that f (u) − f (v) f ′ (ξ) (6.22) − λ = ′ − λ < ε. g(u) − g(v) g (ξ)

In this formula, we let v → b−, use (6.19), and so obtain |f (u)/g(u) − λ| ≤ ε for b − δ < u < b. This proves (6.20). ⊔ ⊓ (6.16) Remark. With slight modifications of the above proof, one sees that – the theorem remains true for b = +∞; – the theorem remains true for λ = +∞ or λ = −∞; and – the theorem remains true for the limit x → a+.

244 III. Foundations of Classical Analysis

(6.17) Theorem. Under the assumptions of Theorem 6.15, where (6.19) is replaced by (6.23)

and

lim f (x) = ∞

x→b−

lim g(x) = ∞,

x→b−

we also have (6.20). g(u) g(v) − g(u) =1− , which gives g(v) g(v) f (v) − f (u)  g(u)  g(u) −λ 1− < ε 1 − . g(v) g(v) g(v)

Proof. We multiply (6.22) by (6.24)

We wish to isolate |f (v)/g(v)−λ| in the expression on the left. Using the modified triangle inequality |A| − |B| ≤ |A − B| (or |A| ≤ |A − B| + |B|), we obtain f (v) g(u) f (u) − λg(u) − λ < ε 1 − + . g(v) g(v) g(v) Now we keep u fixed and let v → b−. Because of (6.23), the expression on the right side approaches ε. Therefore, |f (v)/g(v) − λ| < 2ε for v sufficiently close to b. This proves the statement. ⊔ ⊓

Examples. The quotient of the functions f (x) = sin x and g(x) = x gives, for x → 0, the undetermined expression 0/0. Applying Theorem 6.15, we compute (6.25)

lim

x→0

sin x cos x = lim = 1. x→0 x 1

Obviously, these equalities have to be read from right to left. Since limx→0 cos x = 1 exists, limx→0 sin x/x also exists and equals 1. Next, we consider f (x) = eαx (α > 0) and g(x) = xn , which both tend to ∞ for x → ∞. Repeated application of Theorem 6.17 (and Remark 6.16) yields (6.26)

eαx αeαx α2 eαx αn eαx = ∞. = lim = lim = . . . = lim x→∞ xn x→∞ n xn−1 x→∞ n(n − 1)xn−2 n! lim

This shows that the exponential function eαx increases faster (for x → ∞) than any polynomial. For a > 0 we obtain from Theorem 6.17 and Remark 6.16 (6.27)

lim

x→∞

ln x 1/x 1 = lim = lim = 0. a a−1 x→∞ x→∞ x ax axa

Hence, any polynomial increases faster than a logarithm. Undetermined expressions of the form 0·∞

or

00

or

can be treated as explained in the following examples:

∞0

III.6 Differentiable Functions 245

ln x 1/x = lim = lim (−x) = 0, 1/x x→0+ −1/x2 x→0   (6.29) lim xx = lim exp(x ln x) = exp lim x ln x = exp(0) = 1, x→0+ x→0+ x→0+  √ ln x (6.30) lim x x = lim x1/x = exp lim = exp(0) = 1. x→∞ x→∞ x→∞ x

(6.28) lim (x · ln x) = lim x→0+

x→0+

In the last two examples, we have exploited the continuity of the exponential function.

Derivatives of Infinite Series Where is it proved that one obtains the derivative of an infinite series by taking the derivative of each term? (Abel, Janv. 16, 1826, Oeuvres, vol. 2, p. 258)

The term-by-term differentiation of infinite series is justified by the following theorem. (6.18) Theorem. Let fn : (a, b) → R be a sequence of continuously differentiable functions. If i) lim fn (x) = f (x) on (a, b), and n→∞

lim f ′ (x) n→∞ n

ii)

= p(x) , where the convergence is uniform on (a, b),

then f (x) is continuously differentiable on (a, b), and for all x ∈ (a, b) we have lim f ′ (x) n→∞ n

(6.31)

= f ′ (x).

Proof. As we can guess, the essential “ingredient” of this proof (in addition to the Fundamental Theorem of Differential Calculus) is Theorem 5.19 on the exchange of limits and integrals. We fix x0 ∈ (a, b). Because {fn′ (x)} converges uniformly on (a, b), we obtain Z x Z x
p(t) dt = lim fn′ (t) dt = lim fn (x) − fn (x0 ) = f (x) − f (x0 ). x0

n→∞

x0

n→∞

By Theorem 6.9, this shows that p(x) = f ′ (x) and that (6.31) holds. The continuity of f ′ (x) follows from Theorem 4.2. ⊔ ⊓

(6.19) Counterexamples. The functions (see Fig. 6.4) (6.32)

fn (x) =

x 1 + n2 x2

and

fn (x) =

1 sin(nx) n

show that hypothesis (i) (even with uniform convergence) is not sufficient to prove (6.31).

246 III. Foundations of Classical Analysis

2

n=1

1

n=2 −1

0

1

−2

0

n=1 n=2 2

−1 −2 FIGURE 6.4. Uniform convergence with lim fn′ 6= (lim fn )′

Exercises 6.1 Let a positive integer n be given and define fn : R → R by  n if x 6= 0, x sin(1/x3 ) fn (x) = 0 if x = 0 . How often is fn differentiable and which derivatives of fn are continuous? 6.2 Show by two different methods (using (6.1) as well as Carath´eodory’s formulation (6.6)) that if g(x) is differentiable at x0 with g(x0 ) 6= 0, then 1/g(x) is also differentiable at x0 . 6.3 Show that the following function is increasing on [0, 1]: 
x · 2 − cos(ln x) − sin(ln x) 0 ̺, and we have uniform convergence on [−η, η] if 0 < η < ̺. Proof. Let x be a value with |x| < ̺. Then, there is an x e with |x| < |e x| < ̺ such that (7.1) converges for x e (put ε = (̺ − |x|)/2 in Definition 1.11). Thus, from Lemma 7.1, we have convergence for x. The uniform convergence on [−η, η] is seen in the same way. ⊔ ⊓ This theorem says nothing about the convergence at x = −̺ and x = ̺. In fact, anything can happen at these points, as we shall see in the following example.

III.7 Power Series and Taylor Series 249

(7.4) Example. The series (7.3)

∞ X xn x x2 x3 x4 = α + α + α + α + ... α n 1 2 3 4 n=1

is the geometric series (apart from the first term; Example 2.2) for α = 0, reduces to −ln(1−x) for α = 1 (see Eq. (I.3.14)), and is, for α = 2, “Euler’s Dilogarithm” (Euler 1768, Inst. Calc. Int., Sectio Prima, Caput IV, Exemplum 2). Independently of α, the radius of convergence of (7.3) is ̺ = 1 (see Example 7.6 below). For α = 0 the series diverges at both ends of the convergence interval. For α = 1 we have divergence for x = +1 (harmonic series), but convergence for x = −1 (by Leibniz’s criterion). For α = 2 the series converges for x = +1 and also for x = −1 (see Lemma 2.6).

Determination of the Radius of Convergence The following theorems give useful formulas for the computation of the radius of convergence. (7.5) Theorem (Cauchy 1821). If lim |cn /cn+1 | exists (or is ∞), then, we have n→∞ c n (7.4) ̺ = lim . n→∞ cn+1

P Proof. We apply the Ratio Test 2.10 to the series n an with an = cn xn . Since a c c c . n+1 n+1 xn+1 n+1 n = |x| lim = |x| lim lim = lim , n→∞ n→∞ n→∞ n→∞ cn+1 an cn xn cn

the series (7.1) converges if |x| < lim |cn /cn+1 |. For |x| > lim |cn /cn+1 | it diverges. This implies Eq. (7.4). ⊔ ⊓ (7.6) Examples. For the series (7.3), where cn = 1/nα , we have |cn /cn+1 | = (1+ 1/n)α → 1 for n → ∞. Therefore, the radius of convergence is ̺ = 1. Similarly, for the binomial series for (1 + x)a (Theorem I.2.2) we have |cn /cn+1 | = (n + 1)/|a − n| → 1 and ̺ = 1. The series expansions for ex (see Theorem I.2.3) for sin x and cos x (see Eqs. (I.4.16) and (I.4.17)) have been proved to converge for all real x (Sect. III.2). Hence, their radius of convergence is ̺ = ∞. An example for a series with ̺ = 0 is 1 + x + 2! x2 + 3! x3 + 4! x4 + . . . . Here, we have cn = n! and |cn /cn+1 | = 1/(n + 1) → 0. The formula of Theorem 7.5 is not directly applicable to the series (7.5)

arctan x = x −

x5 x7 x3 + − + ... , 3 5 7

250 III. Foundations of Classical Analysis

because |cn /cn+1 | is alternatively 0 and ∞. If we divide by x and x2 P replace n by the new variable z, then the series (7.5) divided by x becomes n cn z with cn = (−1)n /(2n + 1). For this series we have ̺ = 1 by Theorem 7.5. Hence, the series (7.5) converges for |x2 | < 1 (i.e., |x| < 1) and we have ̺ = 1. While Eq. (7.4) requires the existence of the limit, the next result is valid without restriction (see also Exercise 7.1 below). (7.7) Theorem (Hadamard 1892). The radius of convergence of the series (7.1) is given by 1 p (7.6) ̺= . lim sup n |cn | n→∞

P Proof. We apply the Root Test 2.11 to the series n an with an = cn xn . Since p p lim sup n |an | = |x| · lim sup n |cn |, n→∞

n→∞

we see that the series p (7.1) converges if |x| < 1/ lim sup |x| > 1/ lim sup n |cn |.

p n |cn |. It diverges if ⊔ ⊓

Continuity Let D be the domain of convergence  (7.7) D = x | series (7.1) converges

so that the series (7.1) defines a function f : D → R given by ∞ X (7.8) f (x) = cn xn for x ∈ D. n=0

It is clear from the uniform convergence on [−η, η] for 0 < η < ̺ (see Theorems 7.3 and 4.2) that f (x) is a continuous function in the open interval (−̺, ̺). The following famous theorem of Abel handles the question of continuity at the end points of the convergence interval. (7.8) Theorem (Abel 1826). Suppose that the series (7.8) converges for x0 = ̺ (or for x0 = −̺). Then, the function f (x) is continuous at x0 = ̺ (or at x0 = −̺).

Proof. For simplicity we assume that ̺ = 1 and x0 = +1. Otherwise, we stretch and/or reverse the convergence interval by replacing x0 by ±x0 /̺. Since, by hypothesis, we have convergence for x0 = 1, it follows from Lemma 2.3 that for n ≥ N and k ≥ 1,

(7.9)

|cn+1 + cn+2 + . . . + cn+k | < ε.

Now, let x be chosen arbitrarily in [0, 1]. Then, for fn (x) = (7.10)

n+1

fn+k (x) − fn (x) = cn+1 x

n+2

+ cn+2 x

Pn

we have

i i=0 ci x

n+k

+ . . . + cn+k x

.

III.7 Power Series and Taylor Series 251

If all ci are ≥ 0, it is clear from Eq. (7.9) that |fn+k (x) − fn (x)| < ε. Otherwise, we split up (7.10) somewhat more carefully (written here for k = 4): cn+1 xn+4 + cn+2 xn+4 + cn+3 xn+4 +cn+4 xn+4 +cn+1 (xn+3 −xn+4 )+cn+2 (xn+3 −xn+4 )+cn+3 (xn+3 −xn+4 )

+cn+1 (xn+2 −xn+3 )+cn+2 (xn+2 −xn+3 ) +cn+1 (xn+1 −xn+2 )

(this process is called Abel’s partial summation, see Exercise 7.2). In each row, we can now factor out a common (positive) factor xn+k , xn+k−1 −xn+k , . . . and obtain, by (7.9) and the triangle inequality, |fn+k (x) − fn (x)| < ε · (xn+k + xn+k−1 −xn+k + . . . + xn+1 −xn+2 ) ≤ ε uniformly on [0, 1]. Therefore, the continuity of f (x) at x0 = 1 follows from Theorem 4.2. ⊔ ⊓

Differentiation and Integration √ Since n n → 1 for n → ∞ (see Eq. (6.30)), it follows from Theorem 7.7 that the (term by term) differentiated and integrated power series have the same radius of convergence as the original series. We then have the following result. P∞ n (7.9) Theorem. The function f (x) = n=0 cn x is differentiable for |x| < ̺ (where ̺ is the radius of convergence and ̺ > 0), and we have (7.11)

f ′ (x) =

∞ X

ncn xn−1 .

n=1

It has a primitive on (−̺, ̺), which is given by Z x ∞ X xn+1 . f (t) dt = cn (7.12) n+1 0 n=0 Proof. For 0 ≤ η < ̺ the convergence of these series is uniform on [−η, η] (and, of course, also on (−η, η)). It then follows from Theorem 6.18 that f (x) is differentiable on (−η, η) and that its derivative is given by (7.11). Similarly, Eq. (7.12) follows from Corollary 5.20. ⊔ ⊓ (7.10) Remark. If the series (7.1) converges, say, at x = ̺, then the differentiated series (7.11) need not converge there. This is the case, for example, with the series (7.3) for α = 2. However, the convergence of (7.1) at x = ̺ implies the convergence of (7.12) at x = ̺ (see Exercise 7.3). With the use of Theorem 7.8, we thus see that identity (7.12) holds for all x ∈ D.

252 III. Foundations of Classical Analysis

(7.11) Example. The geometric series (Example 2.2) has radius of convergence ̺ = 1. Integrating it term by term, we obtain from Theorem 7.9 and the definition of ln (Sect. I.3) that for x ∈ (−1, 1) Z x dt x2 x3 x4 x5 =x− + − + − ... . (7.13) ln(1 + x) = 2 3 4 5 0 1+t Moreover, the series in (7.13) converges for x = 1 and, by Theorem 7.8, we obtain ln 2 = 1 − 1/2 + 1/3 − 1/4 + . . . , this time rigorously.

Taylor Series . . . and to estimate the value of the remainder of the series. This problem, one of the most important in the theory of series, has not yet been solved . . . (Lagrange 1797, p. 42-43, Oeuvres, vol. 9, p. 71)

In 1797 (second ed. 1813), Lagrange wrote an entire treatise basing analysis on the Taylor series expansion of a function (see Eq. (II.2.8)) (7.14)

f (x) =

∞ X (x − a)i i=0

i!

f (i) (a),

which allowed him, as he thought, to banish infinitely small quantities, limits, and fluxions (“d´egag´es de toute consid´eration d’infiniment petits, d’´evanouissans, de limites ou de fluxions”). This dream, however, only lasted some 25 years. Regarding x − a as a new variable, this series is of the form (7.1) and the previous results on the convergence of the series can be applied. The first problem is that there are infinitely differentiable functions for which the series (7.14) does not converge for any x 6= a (see Exercise 7.6 below). But even convergence of the series in (7.14) does not necessarily imply the identity in (7.14), as we shall see in the subsequent counterexample. (7.12) Counterexample. . . . Taylor’s formula, which can no longer be admitted in general . . . (Cauchy 1823, R´esum´e, p. 1)

Cauchy (1823) considered the function  −1/x2 e (7.15) f (x) = 0

if x 6= 0 if x = 0,

which is continuous everywhere. This function is so terribly flat at the origin (see Fig. 7.1), that f (i) (0) = 0 for all i. In fact, by the rules of differentiation, we obtain (for x 6= 0)  6 2 2 2 4 f ′ (x) = 3 · e−1/x , f ′′ (x) = − 4 + 6 · e−1/x x x x 2

and we see that f (i) (x) is a polynomial in 1/x multiplied by e−1/x . Since for 2 all n the functions x−n e−1/x tend to zero as x → 0 (see the examples after

III.7 Power Series and Taylor Series 253

f ′′/2

−3

−2

0

−1

f

f ′

1

0

1

−1

2

3

f ′′′/6 2

FIGURE 7.1. Graph of e−1/x and its derivatives

Theorem 6.17), we have f (i) (x) → 0 for x → 0. The fact that also f (i) (x)/x → 0 for x → 0 implies that f (i+1) (0) = limh→0 f (i) (h)/h = 0. Thus, the Taylor series for the function f (x) of (7.15) is 0 + 0 + 0 + . . . and obviously converges for all x. But, formula (7.14) is wrong for x 6= 0. In order to establish Eq. (7.14) for particular functions, we have to consider partial sums of Taylor series and to estimate their error. A useful formula in this context has already been derived at the end of Sect. II.4. It is summarized in the following theorem. (7.13) Theorem. Let f (x) be k + 1 times continuously differentiable on [a, x] (or on [x, a] if x < a). Then, we have f (x) =

k X (x − a)i i=0

i!

f

(i)

(a) +

Z

a

x

(x − t)k (k+1) f (t) dt. k!

⊔ ⊓

The Binomial Series. . . . but the one which gives me most pleasure is a paper . . . on the simple series m(m − 1) 2 x + ... 1 + mx + 2 I dare say that this is the first rigorous proof of the binomial formula . . . (Abel, letter to Holmboe 1826, Oeuvres, vol. 2, p. 261)

A rigorous proof of the binomial identity (7.16)

(1 + x)a = 1 + ax +

a(a − 1) 2 a(a − 1)(a − 2) 3 x + x + ... 2! 3!

for |x| < 1 and arbitrary a was first considered by Abel in 1826. A proof based on Taylor series can be found in Weierstrass’s lecture of 1861 (see Weierstrass 1861).

254 III. Foundations of Classical Analysis

If we put f (x) = (1 + x)a and compute its derivatives f ′ (x) = a(1 + x)a−1 , f (x) = a(a − 1)(1 + x)a−2 , . . . , we observe that the series of (7.16) is simply the Taylor series of f (x) = (1 +x)a . Its radius of convergence has been computed as ̺ = 1 in Example 7.6. In order to prove identity (7.16) for |x| < 1, we have to show that the remainder (see Theorem 7.13) Z x (x − t)k (7.17) Rk (x) = a(a − 1) · . . . · (a − k)(1 + t)a−k−1 dt k! 0 ′′

converges to zero for k → ∞. Using Theorem 5.17 and putting ξ = θk x with 0 < θk < 1, we obtain

(x − θk x)k a(a − 1) · · · (a − k)(1 + θk x)a−k−1 · x k! k  (a − 1)(a − 2) · · · (a − k) k 1 − θk ·x · = · (1 + θk x)a−1 · ax. k! 1 + θk x

Rk (x) =

The factor ax is a constant; (1 + θk x)a−1 lies between (1 + x)a−1 and 1 and is bounded; 0 < 1 − θk < 1 + θk x for all x satisfying |x| < 1 implies that the factor
k (1 − θk )/(1 + θk x) is bounded by 1. Since the remaining factor

(a − 1)(a − 2) · · · (a − k) k ·x k! is, for |x| < 1, the general term of a convergent series, it tends to zero by (2.3). Consequently, we have Rk (x) → 0 for k → ∞ and the identity (7.16) is established for |x| < 1. Whenever the series (7.16) converges for x = +1 or x = −1, it represents a continuous function and thus equals (1 + x)a at these points also (Theorem 7.8). Estimate of the Remainder without Integral Calculus. The attempts of Lagrange (1797) to evaluate the remainder in Taylor’s formula were crowned by the following elegant formulas (“ce th´eor`eme nouveau et remarquable par sa simplicit´e et sa g´en´eralit´e . . .”): f (x) = f (a) + (x−a)f ′ (ξ) (x−a)2 ′′ f (ξ) (7.18) 2! (x−a)2 ′′ (x−a)3 ′′′ f (a) + f (ξ), f (x) = f (a) + (x−a)f ′ (a) + 2! 3! etc., where ξ is an unknown value between a and x. f (x) = f (a) + (x−a)f ′ (a) +

(7.14) Theorem (Lagrange 1797). Let f (x) be continuous on [a, x] and k + 1 times differentiable on (a, x). Then, there exists ξ ∈ (a, x) such that f (x) =

k X (x − a)i i=0

i!

f (i) (a) +

(x − a)k+1 (k+1) f (ξ). (k + 1)!

III.7 Power Series and Taylor Series 255

Proof. We follow an elegant idea of Cauchy (1823), denote the remainder by (7.19)

Rk (x) = f (x) −

k X (x − a)i i=0

i!

f (i) (a),

and compare it to the function Sk (x) = (x − a)k+1 /(k + 1)!. We have Rk (a) = 0,

Rk′ (a) = 0,

...

(k)

, Rk (a) = 0,

(i)

and similarly, Sk (a) = 0 for i = 0, 1, . . . , k. Applying Theorem 6.14 repeatedly, we get Rk (x) Rk (x) − Rk (a) R′ (ξ1 ) R′ (ξ1 ) − Rk′ (a) = = ′k = k′ Sk (x) Sk (x) − Sk (a) Sk (ξ1 ) Sk (ξ1 ) − Sk′ (a) (k+1)

(7.20)

=

Rk (ξk+1 ) R′′ (ξ2 ) − Rk′′ (a) Rk′′ (ξ2 ) = k′′ = . . . = (k+1) , ′′ Sk (ξ2 ) Sk (ξ2 ) − Sk′′ (a) Sk (ξk+1 ) (k+1)

where ξ1 lies between x and a, ξ2 between ξ1 and a, and so on. Since Sk (k+1) 1 and Rk (x) = f (k+1) (x), we obtain from (7.20) that

(x) =

Rk (x) = Sk (x) · f (k+1) (ξ)

with ξ = ξk+1 . This completes the proof of the theorem.

⊔ ⊓

Remark. The relation between the remainders of Theorems 7.13 and 7.14 is given by Theorem 5.18. For the original proof of Lagrange see Exercise 7.8 below.

Exercises 7.1 Determine the radius of convergence of the series f (x) = 1 + 2x + x2 + 2x3 + x4 + 2x5 + . . . and show that Theorem 7.5 is not applicable, but that Theorem 7.7 is. 7.2 (Partial summation, Abel 1826). Let {an } and {bn } be two sequences. Prove that N N X X an b n = An (bn − bn+1 ) + AN bN +1 − A−1 b0 , n=0

n=0

where A−1 = α is an arbitrary constant and An = α + a0 + a1 + . . . + an . Hint. Use the identity an bn = (An − An−1 )bn = An (bn − bn+1 ) − An−1 bn + An bn+1 .

7.3 Consider the series ∞ X

n=1

cn

and

∞ X cn . n n=1

256 III. Foundations of Classical Analysis

Prove that the convergence of the first series implies that of the second. The proof will encounter a difficulty similar to that in the proof of Theorem 7.8, which can be settled by a similar idea (partial summation, see preceding exercise). 7.4 Investigate the convergence of the series of Newton-Gregory arcsin(x) = x +

1 x3 1 · 3 x5 1 · 3 · 5 x7 + + + ... 2 3 2·4 5 2·4·6 7

for x = 1 and x = −1. Hint. Wallis’s product will be useful for understanding the asymptotic behavior of the coefficients. 7.5 Let D′ be the domain of convergence for the series in Eq. (7.11). Prove that the identity in Eq. (7.11) holds for all x ∈ D′ . 7.6 An infinitely differentiable function whose Taylor series does not converge (see Lerch 1888, 1 Pringsheim 1893); show that the series cos 2x cos 4x cos 8x cos 16x + + + +... 1! 2! 3! 4! and all its derivatives converge uniformly in R. −1 Show that its Taylor series at the origin is f (x) =

0 0

1

f (0) + f ′ (0)x + . . . e4 − 1 2 e16 − 1 4 e64 − 1 6 x + x − x + ... 2! 4! 6! and diverges for all x 6= 0. Nevertheless, for the computation of, say, f (0.01) (correct value f (0.01) = 1.71572953) the first two terms of this series are useful. Why? 7.7 Investigate the convergence of the series (7.16) for x = 1 and x = −1. 7.8 Find formulas (7.18) in the footprints of Lagrange by using, as we would say today, a “homotopy” argument. Hint. Put = (e − 1) −

z 2 x2 ′′ x3 f (x − zx) + R(z), 2! 3! where z is a variable between 0 and 1 and where x is considered as a fixed constant. Setting z = 0, we find R(0) = 0, and with z = 1, we see that (x3 /3!)R(1) is the error term we are looking for. Now, differentiate (7.21) with respect to z and find R′ (z) = 3z 2 f ′′′ (x − zx) . Finally, integrate from 0 to 1 and apply Theorem 5.18. P P 7.9 (Abel 1826). Prove that if the series i ai , j bj and their Cauchy product converge, identity (2.19) holds. Hint. Apply Abel’s Theorem 7.8 to the function f (x) · g(x), where f (x) = P P i j i ai x and g(x) = j bj x . (7.21) f (x) = f (x − zx) + zxf ′ (x − zx) +

III.8 Improper Integrals 257

III.8 Improper Integrals Rb The theory of the Riemann integral a f (x) dx in Sect. III.5 is based on the assumptions that [a, b] is a finite interval and the function f (x) is bounded on this interval. We shall show how these restrictions can be circumvented. If at least one of the two assumptions is violated, we speak of an improper integral.

Bounded Functions on Infinite Intervals (8.1) Definition. Let f : [a, ∞) → R be integrable on every interval [a, b] with b > a. If the limit Z ∞ Z b f (x) dx := lim f (x) dx b→∞

a

a

exists, then we say that f (x) is integrable on [a, ∞) and that convergent integral.

R∞ a

f (x) dx is a

Only wimps do the general case. True teachers tackle examples. (Parlett, see Math. Intelligencer, vol. 14, No. 1, p. 35)

(8.2) Examples. Consider first the exponential function on the interval [0, ∞). By Definition 8.1, we have Z ∞ Z b b   = lim (1 − e−b ) = 1. e−x dx = lim e−x dx = lim −e−x 0

b→∞

b→∞

0

0

b→∞

Once we are accustomed to this definition, we simply write Z ∞ ∞ e−x dx = −e−x = 1. (8.1) 0

0

Next, consider the function x−α on [1, ∞):  Z ∞ Z ∞ diverges dx x1−α ∞ −α (8.2) = x dx = = α x 1−α 1 (α − 1)−1 1 1

if α < 1 if α > 1 .

For α = 1 a primitive is ln x and the improper integral diverges. But how can we check the integrability on [a, ∞) if no primitive is known explicitly? (8.3) Lemma. Let f : [a, ∞) → R be integrable Ron every interval [a, b]. ∞ a) If |f (x)| ≤ g(x) for all x ≥ a and if a g(x) dx is convergent, then R∞ f (x) dx is also convergent. a R∞ b) If 0 ≤ g(x) ≤ f (x) for all x ≥ a and if a g(x) dx is divergent, then R∞ a f (x) dx also diverges.

258 III. Foundations of Classical Analysis

Proof. Part (a) follows from Cauchy’s criterion (Theorem 3.12), and from TheoR bb R bb R bb rem 5.14, because | b f (x) dx| ≤ b |f (x)| dx ≤ b g(x) dx < ε for sufficiently large b < bb. Part (b) is obvious. ⊔ ⊓

(8.4) Example. For α > 0 we consider the function (1 + xα )−1 on the interval [0, ∞). We split the integral according to Z 1 Z ∞ Z ∞ dx dx dx = + . (8.3) α α 1+x 1 + xα 0 0 1+x 1 The first integral is “proper”. For the second integral we use the estimates 1 1 1 ≤ ≤ α α α 2x 1+x x

for

x ≥ 1.

It thus follows from Lemma 8.3 and Eq. (8.2) that the integral (8.3) converges for α > 1 and diverges for α ≤ 1. 1

sin x x + 0

+ π

–

2π

3π

–

4π

+

FIGURE 8.1. Graph of sin x/x

(8.5) Example. Let us investigate the existence of Z ∞ sin x dx. (8.4) x 0 The function f (x) = sin x/x is continuous at x = 0 with f (0) = 1 and so poses no difficultyRat this point. Using the estimate | sin x| ≤ 1 would be pointless, since ∞ the integral 1 x−1 dx diverges. But the graph of f (x) (see Fig. 8.1) shows that the integral can be written as an alternating series of the form a0 − a1 + a2 − a3 + . . . , where Z π Z 2π Z 3π sin x sin x sin x a0 = dx, a1 = − dx, a2 = dx, . . . . x x x 0 π 2π This series converges by Leibniz’s criterion (Theorem 2.4). The condition ai+1 ≤ ai can be verified with help of the substitution x 7→ x − π and ai → 0 follows from the simple estimate 0 < ai ≤ 1/i.

III.8 Improper Integrals 259

(8.6) Theorem (Maclaurin 1742). Let f (x) ≥ 0 be nonincreasing on [1, ∞). Then, we have Z ∞ ∞ X f (n) converges ⇐⇒ f (x) dx converges . 1

n=1

f g

1

g h

g h

0

1

2

g

h 3

g

h 4

h 5

6

FIGURE 8.2. Majorization and minorization of f (x)

Proof. Let g(x) = f ([x]) and h(x) = f ([x] + 1) be the step functions drawn in Fig. 8.2 (here [x] denotes the largest integer not exceeding x). These functions are integrable on finite intervals (Theorem 5.11), and, since f (x) is monotonic, we have h(x) ≤ f (x) ≤ g(x) for all x. Consequently, Z N N N −1 X X f (n) ≤ f (x) dx ≤ f (n) n=2

1

n=1

and the statement follows from Theorem 1.13 since f (x) ≥ 0.

⊔ ⊓

As integrals are often easier to calculate than sums, this theorem is very useful for discussing the convergence of series. For example, the computation of Eq. (8.2) gives an elegant new proof for Lemma 2.6. P If we try to study 1/n and Pwhatαhappens “between” the divergent series the convergent series 1/n (for some α > 1), we are led to the investigation of (8.5)

∞ X

1 n(ln n)β n=2

(for large n and any α > 1 and β > 0 we have n < n(ln n)β < nα by Eq. (6.27)). With the transformation u = ln x, we have Z ∞ Z ∞ dx du = , β β x · (ln x) 2 ln 2 u

and Theorem 8.6, together with Eq. (8.2), proves that the series (8.5) converges for β > 1, but diverges for β ≤ 1.

260 III. Foundations of Classical Analysis

Integrals from −∞ to +∞. It would be injudicious to define Z ∞ Z r (8.6) f (x) dx = lim f (x) dx r→∞

−∞

−r

(if the limit exists). This would produce nonsense, for example, by applying the transformation formula (II.4.14) with z = x + 1 (dz = dx). With the above definition, we would have Z +∞ Z +∞ z dz = 0 and (x + 1) dx = ∞. −∞

−∞

(8.7) Definition. Let f : R → R be integrable on every bounded interval [a, b]. Then, we say that Z ∞ Z 0 Z ∞ f (x) dx := f (x) dx + f (x) dx −∞

0

−∞

exists if both improper integrals to the right exist. The two integrals Z ∞

−∞

dx 1 + x2

and

Z

∞

2

e−x dx

−∞

converge in the sense of Definition 8.7. The first one tends to π (a primitive is arctan x). The convergence of the second integral is seen from Lemma 8.3 by 2 using e−x ≤ e−x for x ≥ 1.

Unbounded Functions on a Finite Interval (8.8) Definition (Gauss 1812, §36). If f : (a, b] → R is integrable on every interval of the form [a + ε, b], then we define Z b Z b f (x) dx := lim f (x) dx, a

ε→0+

a+ε

if the limit exists.

This definition includes situations where |f (x)| → ∞ for x → a. A similar definition is possible when |f (x)| → ∞ for x → b. In order to check the integrability of such a function, Lemma 8.3 can be adapted without any difficulty. (8.9) Examples. For the function x−α considered on the interval (0, 1] we have 1  Z 1 Z 1 diverges if α > 1 dx dx x1−α (8.7) = lim = lim = (1 − α)−1 α α ε→0+ ε→0+ x x 1 − α if α < 1. 0 ε ε

The case α = 1 also leads to a divergent integral. Hence, the hyperbola y = 1/x (α = 1) is the limiting case with infinite area on the left (0 < x ≤ 1) and on the right (x ≥ 1). If α decreases, the left area becomes finite, if α increases, the right area becomes finite.

III.8 Improper Integrals 261

The integral

Z

0

1

sin x dx = xα

Z

0

1

sin x 1 · α−1 dx x x

converges if and only if α − 1 < 1, i.e., α < 2. This is due to the fact that f (x) = sin x/x is continuous at zero with f (0) = 1.

Euler’s Gamma Function Throughout his life, Euler was interested in “interpolating” the factorials 0! = 1, 1! = 1, 2! = 2, 3! = 6, 4! = 24, . . . at noninteger values. He wrote for this 1 · 2 · 3 · 4 · . . . · x (“De Differentiatione Functionum Inexplicabilium”, see 1755, Caput XVI of Inst. Calc. Diff., Opera, vol. X). He finally found the definition (totally “explicabilium”) used today in 1781: integration by parts applied to the following integral (with u(x) = xn , v ′ (x) = e−x ) yields Z ∞ Z ∞ ∞ xn e−x dx = −xn e−x + n xn−1 e−x dx. (8.8) 0

0

n −x

The term x e

(8.9)

0

vanishes for x = 0 (n > 0) and for x → ∞, so we find that Z ∞ xn e−x dx = n! 0

Here, we have no problem replacing n by a noninteger real number: (8.10) Definition. For α > 0 we define Z ∞ (8.10) Γ (α) := xα−1 e−x dx. 0

We have to show that the integral of Eq. (8.10) is convergent. There are two difficulties: the integrated function is unbounded for x → 0 (if α < 1) and the integration interval is infinite. We therefore split the integral into Z 1 Z ∞ (8.11) xα−1 e−x dx + xα−1 e−x dx. 0

1

It follows from the estimate xα−1 e−x ≤ xα−1 , from Lemma 8.3, and from Eq. (8.7) that the first integral in (8.11) converges for α > 0. For the second integral in (8.11) we use the estimate xα−1 e−x = xα−1 e−x/2 · e−x/2 ≤ M e−x/2 (see the examples after Theorem 6.17) and again Lemma 8.3. Equation (8.9) and the computation in Eq. (8.8) show that (8.12)

Γ (n + 1) = n!,

Γ (α + 1) = αΓ (α)

for α > 0.

With the help of the second relation in (8.12), one can extend the definition of Γ (α) to negative α (α 6= −1, −2, −3, . . . ) by setting

262 III. Foundations of Classical Analysis

(8.13)

Γ (α − 1) =

Γ (α) α−1

(see Fig. 8.3). We shall see in Sect. IV.5 that Γ (1/2) = 6

√ π.

Γ(x)

3!

5 4 3 2

2!

1 −4

−3

−2

−1

0

0

0!

1!

1

2

3

4

−1 −2 −3 −4

FIGURE 8.3. Gamma function

Exercises 8.1 Show that the Fresnel integrals (see Fig. II.6.2) Z ∞ Z ∞ 2. . sin x dx , cos x2 dx 0

0

converge (you can also use a change of coordinates and find an integral similar to (8.4); compare with Fig. 8.1). 8.2 Show that for the sequence n X √ 1 √ an = 2 n − k k=1

limn→∞ R a√n exists and√1 ≤ limn→∞ an ≤ 2 (it might be helpful to remember that (1/ x) dx = 2 x ).

8.3 Show, by using an appropriate change of coordinates, that Z ∞ 2 1 1 . e−x dx = Γ 2 2 0

III.9 Two Theorems on Continuous Functions 263

III.9 Two Theorems on Continuous Functions This section is devoted to two results of Weierstrass. The first proves the existence of continuous functions that are nowhere differentiable. The second shows that a continuous function f : [a, b] → R can be approximated arbitrarily closely by polynomials.

Continuous, but Nowhere Differentiable Functions Until very recently it was generally believed, that a . . . continuous function . . . always has a first derivative whose value can be indefinite or infinite only at some isolated points. Even in the work of Gauss, Cauchy, Dirichlet, mathematicians who were accustomed to criticize everything in their field most severely, there can not be found, as far as I know, any expression of a different opinion. (Weierstrass 1872) A hundred years ago such a function would have been considered an outrage on common sense. (Poincar´e 1899, L’oeuvre math. de Weierstrass, Acta Math., vol. 22, p. 5)

n =3

n =1

.05

n =2

n =4

3.1

π

3.2

−.05

FIGURE 9.1. Riemann’s function (9.1) near x = π

Before the era of Riemann and Weierstrass, it was generally believed that every continuous function was also differentiable, with the possible exception of some singular points (see quotations). In 1806, A.-M. Amp`ere (a name that you have surely heard) even published a “proof” of this fact (J. Ecole Polyt., vol. 6, p. 148). The first shock was Riemann’s example (5.24), which, when integrated, produces a function which is not differentiable on an everywhere dense set of points. This opened the way to the search for functions that were nowhere differentiable. About 1861 (see Weierstrass 1872), Riemann thought that the function (see Eq. (3.7)) (9.1)

f (x) =

∞ X 1 1 sin(n2 x) = sin x + sin(4x) + sin(9x) + . . . , 2 n 4 9 n=1

264 III. Foundations of Classical Analysis

which is continuous since the series converges uniformly (see Theorems 4.3 and 4.2), is nowhere differentiable. Weierstrass declared himself unable to prove this assertion and, indeed, Gerver (1970) found that (9.1) is differentiable at selected points, for example at x = π (see Fig. 9.1). (9.1) Theorem (Weierstrass 1872). There exist continuous functions that are nowhere differentiable. Proof. Weierstrass showed, after two pages of calculation, that (9.2)

f (x) =

∞ X

bn cos(an x),

n=1

which is uniformly convergent for b < 1, is nowhere differentiable for ab > 1+3π/2. Many later researchers, intrigued by this phenomenon, found new examples, in particular Dini (1878, Chap. 10), von Koch (1906, see Fig. IV.5.6 below), Hilbert (1891, see Fig. IV.2.3 below), and Takagi (1903). Takagi’s function was reinvented by Tall (1982) and named the “blancmange function”. This function is defined as follows: we consider the function  x 0 ≤ x ≤ 1/2 (9.3) K(x) = 1−x 1/2 ≤ x ≤ 1

and extend it periodically (i.e., K(x + 1) = K(x) for all x) in order to get a continuous zigzag function. Then, we define (see Fig. 9.2) (9.4) f (x) =

∞ X 1 1 1 1 K(2n x) = K(x)+ K(2x)+ K(4x)+ K(8x)+. . . . n 2 2 4 8 n=0

Since |K(x)| ≤ 1/2 and 1+1/2+1/4+1/8+. . . converges, the series (9.4) is seen to converge uniformly (Theorem 4.3) and represents a continuous function f (x) (Theorem 4.2). In order to see that it is nowhere differentiable, we use an elegant argumentation of de Rham (1957). Let a point x0 be given. The idea is to choose αn = i/2n and βn = (i + 1)/2n , where i is the integer with αn ≤ x0 < βn , and to consider the quotient (9.5)

rn =

f (βn ) − f (αn ) . βn − αn

Since at the values P αn 1and βnj the sum in (9.4) is finite, rn is the slope of the truncated series n−1 j=0 2j K(2 x) on the interval (αn , βn ) (see Fig. 9.2 where, for x0 = 1/3, these slopes can be seen to be 0, 1, 0, 1 . . .). With increasing n, we always have rn+1 = rn ± 1, and the sequence {rn } cannot converge. On the other hand, {rn } is a mean of the slopes rn = λn

f (βn ) − f (x0 ) f (x0 ) − f (αn ) + (1 − λn ) βn − x0 x0 − αn

III.9 Two Theorems on Continuous Functions 265

2/3

0

1

1/3 FIGURE 9.2. The “blancmange” function

where λn = (βn − x0 )/(βn − αn ) ∈ (0, 1] (if αn = x0 we have λn = 1 and the second term is not present). Differentiability at x0 would therefore imply that |rn − f ′ (x0 )| < λn ε + (1 − λn )ε = ε for sufficiently large n, which is a contradiction.

⊔ ⊓

Weierstrass’s Approximation Theorem This is the fundamental proposition established by Weierstrass. (Borel 1905, p. 50)

We have just digested the first Weierstrass surprise, which is the existence of continuous functions without a derivative; now comes the second: we can make them differentiable as often as we want, even polynomials, if only we allow an arbitrarily small error ε. (9.2) Theorem (Weierstrass 1885). Let f : [a, b] → R be a continuous function. For every ε > 0 there exists a polynomial p(x) such that (9.6)

|p(x) − f (x)| < ε

for all

x ∈ [a, b].

In other terms, f (x) − ε ≤ p(x) ≤ f (x) + ε, i.e., the polynomial p(x) is bounded between f (x) − ε and f (x) + ε on the entire interval [a, b].

266 III. Foundations of Classical Analysis

n = 20

n = 30

ϕn (x)

ϕn (x) 2

2

>1−ε 1

1

n=1

< ε/2

−1

0

1

FIGURE 9.3a. Dirac sequence (9.7)

< ε/2 0

−1

δ

1

FIGURE 9.3b. Mass concentration

The list of mathematicians, compiled from Borel (1905, p. 50) and Meinardus (1964, p. 7), who provided proofs for this theorem, shows how much they were fascinated by this result: Weierstrass (1885), Picard (1890, p. 259), Lerch 1892, Volterra 1897, Lebesgue 1898, Mittag-Leffler 1900, Landau (1908), D. Jackson 1911, S. Bernstein 1912, P. Montel 1918, Marchand 1927, W. Gontscharov 1934. This theorem, which is related to approximation by trigonometric polynomials, has also been generalized in various ways (see Meinardus 1964, §2). The following proof is based on the idea of, as we say today, “Dirac sequences”. Dirac Sequences. We set, with Landau (1908, see Fig. 9.3a),  µn (1 − x2 )n if −1 ≤ x ≤ 1 (9.7) ϕn (x) = 0 otherwise, where the factor (9.8) is chosen such that (9.9)

µn =

1 · 3 · 5 · 7 · . . . · (2n + 1) 2 · 2 · 4 · 6 · . . . · 2n Z

+∞

ϕn (x) dx = 1

−∞

(see Exercise II.4.3). These functions concentrate, for increasing n, more and more of their “mass” at the origin:

III.9 Two Theorems on Continuous Functions 267

(9.3) Lemma. Let ϕn (x) be given by (9.7). For every ε > 0 and for every δ > 0 with 0 < δ < 1 there exists an integer N such that for all n ≥ N (see Fig. 9.3b) Z δ 1−ε< (9.10) ϕn (x) dx ≤ 1, Z

(9.11)

−δ

−δ

ϕn (x) dx +

Z

1

ϕn (x) dx < ε.

δ

−1

Proof. We start with the proof of (9.11). Since 1 − x2 ≥ 1 − x for 0 ≤ x ≤ 1, we R1 R1 have 0 (1−x2 )n dx ≥ 0 (1−x)n dx = 1/(n+1), and therefore µn ≤ 21 (n+1). Hence, we have for δ ≤ |x| ≤ 1 0 ≤ ϕn (x) ≤ ϕn (δ) ≤ 21 (n + 1) · (1 − δ 2 )n .

Now q := 1 − δ 2 < 1 and (1 − δ 2 )n = q n decreases exponentially, so that (n + 1) · (1 − δ 2 )n → 0 (see (6.26)). This implies that for n sufficiently large 0 ≤ ϕn (x) ≤ ε/2 for δ ≤ |x| ≤ 1, and Eq. (9.11) is a consequence of Theorem 5.14. The estimate (9.10) is obtained by subtracting (9.11) from (9.9). ⊔ ⊓ A Proof of Weierstrass’s Approximation Theorem. We may assume that 0 < a < b < 1 (the general case is reduced to this one by a transformation of the form x 7→ α + βx with suitably chosen constants α and β ). We then extend f (x) to a continuous function on [0, 1], e.g., by putting f (x) = f (a) for 0 ≤ x < a and f (x) = f (b) for b < x ≤ 1. Then, we set for ξ ∈ [a, b] (9.12)

pn (ξ) :=

Z

1 0

f (x)ϕn (x − ξ) dx = µn

Z

1

0

f (x) 1 − (x − ξ)2


n

dx.

If we expand the factor (1 − (x − ξ)2 )n by the binomial theorem, we obtain a polynomial in ξ of degree 2n, whose coefficients are functions of x. On inserting it into (9.12), we see that pn (ξ) is a polynomial of degree 2n. Motivation. For a fixed ξ ∈ [a, b] the function ϕn (x − ξ) will have its peak shifted to the point ξ (Fig. 9.4). Hence, the product f (x) · ϕn (x − ξ) multiplies (more or less) the peak by the value f (ξ). We therefore expect, because of (9.9), that the integral (9.12) will be close to f (ξ). Estimation of the Error. For the error between pn (ξ) and f (ξ) we shall use the triangle inequality as follows: Z ξ+δ Z 1 |pn (ξ) − f (ξ)| ≤ f (x)ϕn (x − ξ) dx − f (x)ϕn (x − ξ) dx (9.13)

Z +

0 ξ+δ

ξ−δ

ξ−δ Z ξ+δ

f (x)ϕn (x − ξ) dx −

Z + f (ξ)

ξ+δ

ξ−δ

ξ−δ

f (ξ)ϕn (x − ξ) dx

ϕn (x − ξ) dx − f (ξ) .

268 III. Foundations of Classical Analysis

ϕn(x−ξ) f 1

ε

δ 0 0

a

ξ−δ

ξ

ξ+δ

b

1

FIGURE 9.4. Landau’s proof

We fix some ε > 0. Since f is continuous on [0, 1], it is uniformly continuous there (Theorem 4.5). Hence, there exists a δ > 0 independent of ξ such that (9.14)

|f (x) − f (ξ)| < ε

if

|x − ξ| < δ.

This δ is, if necessary, further reduced to satisfy δ ≤ a and δ ≤ 1 − b. Hence, we always have [ξ − δ, ξ + δ] ⊂ [0, 1]. Furthermore, the function f (x) is bounded, i.e., satisfies |f (x)| ≤ M for x ∈ [0, 1] (Theorem 3.6). The three terms to the right of Eq. (9.13) can now be estimated as follows: for the first one we use boundedness of f (x) and Eq. (9.11) and we see that it is bounded by M ε; similarly, the use of Eq. (9.10) shows that the third term is bounded by M ε; finally, it follows from (9.14) and (9.9) that the second term is bounded by ε. We thus have |pn (ξ) − f (ξ)| ≤ (2M + 1)ε for sufficiently large n. Since this estimate holds uniformly on [a, b], the theorem is proved. ⊔ ⊓

III.9 Two Theorems on Continuous Functions 269

1.0

.8

.6

n = 100

.4 n = 10 n = 1000 .2

.0

.25

.50

.75

1.00

FIGURE 9.5. Convergence of polynomials (9.12) to f (x) of (9.15)

(9.4) Example. Consider the function f : [1/8, 7/8] → R defined by  −3.2x + 0.8 if 1/8 ≤ x ≤ 1/4 ,   p   1/64 − (x − 3/8)2 if 1/4 ≤ x ≤ 1/2 , p (9.15) f (x) = 2  7 · 1/64 − (x − 5/8) if 1/2 ≤ x ≤ 3/4 ,    7.6x − 5.7 if 3/4 ≤ x ≤ 7/8 .

As in the above proof, we extend it to a continuous function on [0, 1]. The polynomials pn (ξ) of Eq. (9.12) are plotted in Fig. 9.5 for n = 10, 100, and 1000. We can observe uniform convergence on [1/8, 7/8] but not on [0, 1]. This is due to the fact that for ξ = 0 or ξ = 1 half of the peak of ϕn (x) is cut off in (9.12). The hypothesis 0 < a < b < 1 in the above proof can therefore not be omitted. The graphs in Fig. 9.5 were actually computed by numerically evaluating the integral in (9.12) for 400 values of ξ by a method similar to those described in Sect. II.6. It would be a waste of effort to calculate the 2000 coefficients of the polynomial.

Exercises 9.1 Show, with the help of Wallis’p product, that the factors µn in (9.8) behave, for n → ∞, asymptotically as n/π, and that the estimation in the proof of Lemma 9.3 is a little crude.

270 III. Foundations of Classical Analysis

9.2 Show that (9.16)

ϕn (x) =

r

n −nx2 e , π

n = 1, 2, 3, . . .

is a Dirac sequence, R ∞ i.e.,2 satisfies√(9.9), (9.10), and (9.11) (we shall see in Sect. IV.5 that −∞ e−x dx = π ). This was actually the sequence on which Weierstrass based his proof. 9.3 Find the constants cn such that (  πx
n −1 ≤ x ≤ 1, cn cos (9.17) ϕn (x) = 2 0 otherwise is a Dirac sequence (see Exercise 5.6). This sequence, with the help of trigonometric formulas like (I.4.4′ ), leads to approximations on [−π, π] by trigonometric polynomials. 9.4 Let  n if |x| ≤ 1/(2n), ϕn (x) = 0 otherwise. Show that for every continuous function f (x) Z b ϕn (x − ξ)f (x) dx = f (ξ) for all a < ξ < b. lim n→∞

a

9.5 Expand (1 − (x − ξ)2 )3 in powers of ξ and show that √ Z 1 3 4 + cos(x4 + x) − sin(3x)  1 − (x − ξ)2 dx x 72 ln(x + 1) + x 0 is a polynomial in ξ.

IV Calculus in Several Variables

r

Drawing by K. Wanner The influence of physics in stimulating the creation of such mathematical entities as quaternions, Grassmann's hypernumbers, and vectors should be noted. These creations became part of mathematics. (M. Kline 1972, p.791)

Functions of several variables have their origin in geometry (e.g., curves depending on parameters (Leibniz 1694a» and in physics. A famous problem throughout the 18th century was the calculation of the movement of a vibrating string (d'Alembert 1748, Fig.O.I). The position of a string u(x,t ) is actually a function of x, the space coordinate, and of t, the time. An important breakthrough for the systematic study of several variables, which occured around the middle of the 19th century, was the idea of denoting pairs (then n-tuples)

by a single letter and of considering them as new mathematical objects. They were E. Hairer et al. (eds.), Analysis by Its History © Springer Science+Business Media New York 2008

272 IV. Calculus in Several Variables called "extensive Grosse" by Grassmann (1844, 1862), "complexes" by Peano (1888), and "vectors" by Hamilton (1853).

FIGURE 0.1. Movement of a vibrating string (harpsichord) The first section, lVI, will introduce norms in n-dimensional spaces, which enable us to extend the definitions and theorems on convergence and continuity quite easily (Section IV2). However, differential calculus (Sections IV3 and IV.4) as well as integral calculus (Section IVS) in several variables will lead to new difficulties (interchange of partial derivatives, of integrations, and of integrations with derivatives).

IV.1 Topology of n-Dimensional Space 273

IV.1 Topology of n-Dimensional Space It may appear remarkable that this idea, which is so simple and consists basically in considering a multiple expression of different magnitudes (such as the "extensive magnitudes" in the sequel) as a new independent magnitude, should in fact develop into a new science; ... (Grassmann 1862, Ausdehnungslehre, p. 5) ... it is very useful to consider "complex" numbers, or numbers formed with several units, . . . (Peano 1888a, Math. Ann., vol. 32, p. 450)

We denote pairs of real numbers by (Xl,X2), n-tuples by call them vectors. The set of all pairs is

(Xl,X2, ...

,xn ), and

(1.1)

and the set of all n-tuples is denoted by

(1.2)

~n

= ~ X ~ X ... x ~ = {(Xl, X2, ... , Xn)

; Xk E

~, k

= 1, ... , n}.

Vectors can be added (componentwise) and multiplied by a real number. With these operations, we call ~n an n-dimensional real vector space.

Distances and Norms The two-dimensional space ~2 can be imagined as a plane, the components Xl and X2 being the cartesian coordinates. The distance between two points X = (Xl, X2) and Y = (Yl, Y2) is, by Pythagoras's Theorem, given by (Fig. 1.1) (1.3)

This distance only depends on the difference y-x and is also denoted by where IIzI12 = zi + z§ if z = (Zl, Z2).

J

Xl

Yl

FIGURE 1.1. Distance in ~2

Y2-X2

Ily-xI12,

C

FIGURE 1.2. Distance in ~3

In three-dimensional space, the distance between X = (Xl, X2, X3) and Y = (Yl, Y2, Y3) is obtained by applying Pythagoras's Theorem twice (first to the triangle DEF and then to ABC, see Fig. 1.2). In this way, we get d(x, y) = IIY - xlb where IIzI12 = J zi + z§ + z~.

274 IV. Calculus in Several Variables

For n-dimensional space ]Rn we define, by analogy, (1.4)

and call it the Euclidean norm of Z = (ZI' Z2, ... , Zn). The distance between x E]Rn and Y E ]Rn is then given by d(x, y) = Ily - x112' (1.1) Theorem. The Euclidean norm (1.4) has the following properties:

(Nl) (N2) (N3)

IIxll 2 0 and Ilxll = 0 {::} x = 0, IIAXII = IAI . Ilxll for A E IR, Ilx + yll :s; I xII + Ilyll (triangle inequality).

Proof Property (Nl) is trivial. Since AX = (AXI, ... , AX n ), we have IIAxll~ = (Axd 2 + ... + (AXn)2 = IAI2 . Ilxll~, which proves (N2). For the proof of (N3) we compute n

n

Ilx + YII~ = ~)Xk + Yk)2 = L k=1

n

xk

+ 2L

k=1

k=1

n

XkYk

+ LYk k=1

:s; Ilxll~ + 211 xl1211YI12 + IIYII~ = (11x112 + IIYI12)2.

D

Remark. In the above proof, we have used the estimate (1.5)

which is known as the Cauchy-Schwarz inequality. It is obtained from L~=I (Xk-

1'Yk)2 20 in exactly the same way as (III.5.l9). With the notation n

(1.6)

(x, y) := L XkYk, k=1

for the scalar product of the two vectors x and y, inequality (1.5) can be written more briefly as (l.5')

In the sequel, we rarely need the explicit formula of Eq. (1.4). We shall usually just use the properties (Nl) through (N3). (1.2) Definition. A mapping I ·11 : ]Rn - t ]R, which satisfies (Nl), (N2), and (N3), is called a norm on ]Rn. The space ]Rn, together with a norm, is called a normed vector space.

IV.1 Topology of n-Dimensional Space 275

Examples (Jordan 1882, Cours d'Analyse, vol. I, p. 18, Peano 1890b, footnote on p. 186, Fn!chet 1906). Besides the Euclidean norm (1.4), we have

(1.7) k=l

(1.8)

maximum norm,

(1.9)

The verification of properties (Nl) and (N2) for all these norms and the verification of (N3) for (1.7) and (1.8) are easy. We will see later ("HOlder's inequality", see (4.42)) that the triangle inequality (N3) also holds for (1.9) for any p ~ 1. (1.3) Theorem. For any x E lRn , we have (LlO)

Proof We only prove the second inequality (the proof of the others is very easy and therefore omitted). Taking the square Ilxlli in Eq. (1.7) and multiplying out, we obtain the sum of squares L x~ (which is Ilxll~) and the mixed products IXk I· lxIi, which all are non-negative. This implies that Ilxlli ~ IIxll~. 0

Each of these norms can be minorized or majorized (up to a positive factor) by each of the others. This shows that the norms Ilxlh, Ilx112' and Ilxll oo are equivalent in the sense of the following definition. (1.4) Definition. Two norms I . lip and positive constants C 1 and C2 such that

I . Ilq are called equivalent if there exist for all

(Lll)

x E

]R.n.

Convergence of Vector Sequences Our next aim is to extend the definitions and results of Sect. 111.1 to infinite sequences of vectors. We consider {xih2':l' where each Xi is itself a vector, i.e., (Ll2)

i

= 1,2,3, ....

276 IV. Calculus in Several Variables

(1.5) Definition. We say that the sequence the vector a = (aI, a2 , ... , an) E ]Rn if

{Xd i;:: l' given by (1.12), converges to

As in the one-dimensional case, we then write lim Xi

'--->00

= a.

J lj

FIGURE 1.3. Convergent sequence in ]R2

This is exactly the same definition as in (III. 1.4 ), except that "absolute values" are replaced by "norms". (1.6) Remark. In order to be precise, one has to specify the norm used in Definition 1.5, e.g., the Euclidean norm. But if II . lip is equivalent to II . Ilq, then we have

(1.13)

convergence in

II · lip

{=::=?

convergence in

II · Ilq·

Indeed, Ilxi - allp < E and 0.11) imply that Ilxi - all q < C2 E . Since E > 0 is arbitrary in Definition 1.5, we can replace it by E' = C2 E and we see that convergence in I . lip implies convergence in I . Ilq· Theorem 1.3 shows that I . III, I . 112, and I . 11 00 are equivalent, and later (Theorem 2.4) we shall see that all norms in ]Rn are equivalent. Therefore, we may take any norm in Definition 1.5 and the convergence of {Xi} is independent of the chosen norm. (1.7) Theorem. For a vector sequence (1.12) we have

lim Xi

'--->00

=a

lim

i~oo

X k i = ak

for

k = 1, 2, ... , n ,

i.e., convergence in ]Rn means componentwise convergence.

IV.1 Topology of n-Dimensional Space 277

Proof. For the maximum norm (1.8) we have (1.14) Ilxi - all oo

l of first components. It is also a bounded sequence, and by Theorem 111.1.17, we can extract a convergent subsequence, say, (1.16)

We then consider the second components. The main idea, however, consists in considering them only for the subsequence corresponding to (1.16) and not for the whole sequence. This sequence is bounded, and we can again apply Theorem III.I.I7 to find a convergent subsequence, say, (1.17)

Now, the sequence Xl, Xg, XS8, XS76, ... converges in the first and in the second component. For n = 2 the proof is complete. Otherwise, we consider the third components corresponding to (1.17), and so on. After the nth extraction of a subsequence, there are still infinitely many terms left and we have a sequence that 0 converges in all components.

278 IV. Calculus in Several Variables

Neighborhoods, Open and Closed Sets By "set" we mean the entity M formed by gathering together certain definite and distinguishable objects m of our intuition or of our thought. These objects are called the "elements" of M. (G. Cantor 1895, Werke, p. 282) No one shall expel us from the paradise that Cantor has created for us. (Hilbert, Math. Ann., vol. 95, p. 170)

A new mathematical era began when Dedekind (about 1871) and Cantor (about 1875) considered sets of points as new mathematical objects. For sets A , B in IR n we shall use the symbols

(1.18) (1.19) (l .20) (l.2l) ( 1.22)

Ac B if all elements of A also belong to B , A n B = {x E IR n ; x E A and x E B} , Au B = {x E IR n ; x E A or x E B}, A \ B = {x E IR n ; x E A but x t/. B}, CA={xElRn; x t/.A} .

The role of open intervals is played by

Be(a) = {x

( 1.23)

E IR n ;

Ilx - all < E},

which we call a disc (or ball) of radius E and center a (see Fig. 1.4).

FIGURE 1.4. Discs ofradius

E

= 1, 1/ 2,1 / 4 for

Ilxll p ,

p = 1, 1.5, 2, 3, 100

(1.10) Definition (Hausdorff 1914, Chap. VII, §1; see also p. 456). Let a E IR n be given. A set V c IR n is called a neighborhood of a, if there exists an E > 0 such that Be(a) c V .

The discs Be(a) depend on the norm (i1·111 , 11·112, or 11 ·11 00, .. .); the definition of a "neighborhood", however, is independent of the norm used, provided that the norms are equivalent. Each Be (a) corresponding to one norm will always contain a Bel (a) for any other norm (Fig. 1.5). (1.11) Definition (Weierstrass, Hausdorff 1914, p. 215). A set U c IR n is open (originally: "ein Gebiet") ifU is a neighborhood of each of its points, i.e.,

U open

'V x

EU

:JE > 0 Be(x) C U.

IV. I Topology of n-Dimensional Space 279

FIGURE 1.5. Neighborhoods

(1.12) Definition (G. Cantor 1884, p. 470; see Ges. Abhandlungen, p. 226). A set F c lR n is closed if each convergent sequence {Xdi >l with Xi E F has its limit point in F, i.e.,

a = lim

F closed

Xi

and

Xi

E F

imply

a E F.

"->00

Examples in R The so-called "open interval" (a , b) = {x E lR; a < X < b} is an open set. Indeed, for every X E (a , b) the number E = min( X - a, b - x) is strictly positive and we have Bc(x) C (a , b). On the other hand, the sequence {a + Iii } (for i 2: 1) is convergent, its elements lie in (a , b) for sufficiently large i, but its limit is not in (a , b). Therefore, the set (a , b) is not closed. The set [a , bj = {x E lR ; a ~ x ~ b} is closed (see Theorem 111.1.6). However, neither a nor b have a neighborhood that is entirely in [a , bj. Hence, [a, bj is not open. The interval A = [a , b) is neither open nor closed, because a has no neighborhood lying in [a, b) and the limit of the convergent sequence {b - Iii } is not in [a, b). Finally, the set lR = (-00, +00) is both open and closed, and so is the empty set 0. (1.13) Lemma. a)

The set A = {x E lRn ;

b)

The set A

=

{x E lR n ;

Ilxll < I} Ilxll ~ I}

is open.

is closed.

Proof a) For a E A we take E = 1 - Iiall, which is positive. With this choice, we have B€(a) C A (see Fig. 1.6), since, with the use of the triangle inequality, we have for x E B€ (a) that

Il xll = Il x - a+ all Hence, A is open.

~

Il x - all + Iiall < E + Iiall = 1.

280 IV. Calculus in Several Variables

p

= 1.

p=2.

p= 1.5

FIGURE 1.6. Open sets

p= 1.5

p= I.

p=3.

p = 100.

{x E ]R2; Ilxll p < I}

p=3.

FIGURE 1.7. Closed sets

{x E ]R2; Ilxll p

p :::;

= 100.

I}

b) Consider a sequence {Xdi2:1 satisfying Xi E A (for all i) and converging A (i.e., Iiall > 1, to a. We have to show that a E A. Suppose the contrary, a see Fig. 1.7), and take € = Iiall - 1. For this € there exists an N 2: 1 such that Ilxi - all < dor i 2: N. Using the triangle inequality (or better yet Exercise 1.1), we deduce

rt

Ih II = Ilxi - a + all 2: Iiall - Ilxi - all> Iiall - € = 1 for sufficiently large i. This contradicts the fact that Xi E A for all i. Hence, A = {x E ]Rn ; Ilxll ::; I} is closed. 0 Further Examples. The set A = {x E ]R2 ; Xl , X2 E Q, Ilxll ::; I} is neither open nor closed. Indeed, each disc contains irrational points and a limit of rational points can be irrational.

FIG RE 1.8.

antor et

The famous Cantor set (1883, see Werke, p.207, Example 11; Fig. 1.8) is given by

A = [0,1] \ {(1/3, 2/3) U (1/9,2/9) U (7/9,8/9) U ... } (1.24)

{x = Lai3 - i; ai E {O,2}}. 00

=

i= ]

It is not open (e.g., x

1.16 below).

= 1/:1 has no neighborhood in A), but is closed (see Remark

IV. 1 Topology of n-Dimensional Space 281

"Sierpinski's triangle" (Fig. 1.9) and Sierpinski 's carpet (Fig. 1.10) (Sierpinski 1915, 1916) are bidimensional generalizations of Cantor's set. The drawings in Figs. 1.9 and 1.10 are not only charming because of their aesthetic appeal, but remind us as well that sets can be rather complicated objects.

1:11 EI

1:11 EI

1:11

E

~

~

1::1

1::1 tI

I:

a

~ 1::1

a

a

~ 1::1

a tI

I:

~ Irt

~ 1::1 l1:li EI

FIGURE 1.9. Sierpinski 's triangle

111:1

111:1

FIGURE 1.10. Sierpinski's carpet

(1.14) Theorem. We have i)

F closed

=:}

CF open,

ii)

U open

=:}

Cu closed.

Proof i) Suppose that CF is not open. Then there exists an a E CF (i.e., a (j. F) such that for all c > 0 we have B,, (a) CF. Taking c = Iii, we can choose a sequence { xih ~ 1 satisfying Xi E F and Ilxi - all < Iii. Since F is closed, we have a E F, a contradiction. ii) Suppose that Cu is not closed. This means that there exists a sequence Xi E Cu (i.e., Xi (j. U) converging to an a (j. CU , (i.e., a E U). Since U is open, we have B,, (a) c U for an c > O. Thus, Xi (j. B,, (a) for all i, a contradiction. 0

ct

(1.15) Theorem (Hausdorff 1914, p. 216). For a finite number of sets, we have i)

U I , U2 , . . . , m U open

ii)

F I , F2 , ... , Fm closed

=:} =:}

UI

n U2 n . . . nUm is open,

FI U F2 U . .. U Fm is closed.

For an arbitrary family of sets (with index set A), we have

>. ::} U). EA U). = {x F). closed for all >. ::} n). E A F). =

iii) U). openforall

E]Rn ; ::3>. E A,

iv) closed.

{ x E ]Rn ;

V>.

X

E U).} is open,

E A,

X

E F).} is

282 IV. Calculus in Several Variables

FIGURE 1.11 . Open sets with closed intersection

FIGURE 1.12. Closed sets with open union

Proof We begin with the proof of (i). Let x E UI n ... n Um so that x E Uk for all k = 1, ... , m. Since Uk is open, there exists an Ek > 0 such that Be:k (x) C Uk. With E = min(EI'" ., Em), we have found a positive E such that Be: (x) C UI n ... nUm . The proof of (iii) is even easier and hence omitted. The equivalences (i) ? (ii) and (iii) ? (iv) are obtained from the "de Morgan rules" (1.25)

C(UI n U2) = (CUI) C(UI U U2)

=

U

(CU2)

(CUI) n (CU2) ,

o

together with Theorem 1.14.

(1.16) Remark. With this theorem, we see that the Cantor set of Eq. (l.24) is closed. Indeed, its complement

CA = (-00,0) U (1,00) U (1/3,2/3) U (1/9,2/9) U (7/9,8/9) U ... is an infinite union of open intervals and thus open by Theorem 1.15. (1.17) Remark. The statements (i) and (ii) of Theorem 1.15 are not true in general

for an infinite number of sets. Consider, for example, the family of open sets (1.26)

Ui =

whose intersection U2 n U3 (Fig. 1.11).

{x E

]R2 ;

IIxll < 1 + I/i} ,

n U4 n ... = {x E

]R2 ;

Ilxll
l with elements in K there exists a subsequence that converges to some element a E K. (1.19) Theorem. For KeIRn we have

I K compact

K bounded and closed.

I

Proof. Let K be bounded (i.e., Ilxll ~ B for all x E K) and closed. We then take a sequence {Xi h> 1 with elements in K. This sequence is bounded and has, by Theorem 1.9, a convergent subsequence. The limit of this subsequence lies in K, because K is closed. Hence, K is compact. On the other hand, let K be a compact set. This implies that K is closed, because every subsequence of a convergent sequence converges to the same limit. In order to see that K is bounded, we assume the contrary, i.e., the existence of a sequence {xd satisfying Xi E K for all i and Ilxill -+ 00. Obviously, it is impossible to extract a convergent subsequence, so that K cannot be compact in 0 this case. (1.20) Remark. Compact sets are, by Definition 1.18, precisely the sets in which the Bolzano-Weierstrass theorem can be applied. Since this theorem is the basis for all deep results on uniform convergence, uniform continuity, maximum and minimum, Frechet was not exaggerating (see quotation). (1.21) Theorem (Heine 1872, Borel 1895). Let K be compact and let

be a family of open sets U,x with (1.28)

(open covering).

Then, there exists a finite number of indices >'1, A2, ... , Am such that

{U,xhEA

284 IV. Calculus in Several Variables

Counterexamples. Before proceeding to the proof of this theorem, we show that none of the assumptions may be omitted. In the example

K =

{x; Il xll < I} ,

Ui = {X ; Ilxll < 1 - Iii} ,

i = 1,2, . .. ,

it is not possible to find a finite covering of K. This is due to the fact that K is not closed. In the situation

K =

jRn ,

Ui =

{x; Ilxll < i}, i =

1,2, ... ,

the set K is not bounded. Again, it is not possible to find a finite covering of K. Hence, the boundedness of K is essential. In our last example, we consider the compact set K = {x; Ilxll :::; I}, but we consider nonopen sets Ui given by

Ui

= { (rcos'P , rsin'P) ; 0:::; r:::; I,

2i~1

:::; ;: :::; ;i } .

None of the Ui is superfluous in the covering {Udi ~ l (Fig. 1.13).

K

I FIGURE 1.13. Non open covering of K

FIGURE 1.14. Heine's proof

Proof Following Heine (1872), we enclose the compact set K in an n-dimensional cube I (a square for n = 2; see Fig. 1.14). Suppose that we need an infinite number of U).. to cover K. The idea is to split I into 2n small cubes by halving its sides (here, h, h , 13 , 14). One of the sets K n I j (j = 1, ... , 2n) requires an infinite number of U).. in order to be covered. We assume that this is K nIe and denote it by K 1 . Again we split Ie into 2n small cubes, and so on. We thus obtain a sequence of sets each of which requires an infinite number of U).. in order to be covered.

IV. I Topology of n-Dimensional Space 285

In each K i , we choose a Xi E K i . The sequence {Xi} is a Cauchy sequence, because the diameter of the Ki tends to zero. Therefore (Theorem 1.8), it converges and we denote its limit by a. Since K is compact (hence closed), we have a E K. By (1.28), there exists a A with a E U)... Since this U).. is open, there exists an c > 0 with Bc(a) c U)... Using again the factthat the diameter of the Ki tends to zero, we conclude that for sufficiently large m we have Km C Bc(a) C U)... Hence, K m is covered by one single U)... This contradicts the assumption that K cannot be covered by a finite number of U)... 0

Exercises 1.1 Let

I . I be a norm on IRn. Prove that

Illxll - Ilylll :::; Ilx - YII· Hint. Apply the triangle inequality to Ilxll = Ilx - y + YII. 1.2 Show that Show that these estimates are "optimal", i.e., if

then c :::; 1 and C ;::::

Vn .

1.3 Mr. C.L. Ever might have the idea of defining the "norm"

Show that this "norm" does not satisfy the triangle inequality. Study also the set B = {x E 1R2 ; Ilxlll/2:::; I} and show that it is not convex. 1.4 For each set A in IR n define the interior AD of A by

A=

{x

I A is neighborhood of x}

and the closure A of A by

A = {x

I A meets every neighborhood of x}.

Show that A is a closed set (in fact the smallest closed set containing A) and o

that A is an open set (the largest open set contained in A). 1.5 Show that for two sets A and B in IRn o

AUB=AUB, Find two sets A and B in IR for which

~

0

0

AnB=AnB.

286 IV. Calculus in Several Variables o ,..-""--..

0

0

AUB¥=AUB.

1.6 (Sierpinski's Triangle 1915). Let a, b, c be three points in]R2 forming an equilateral triangle. Consider the set

= {Aa + J.tb + ve

00 Ai ; A = "'"""' --;L..J 2'

00

J.ti

00

Vi

, J.t = L..J "'"""' --;, v = L..J "'"""' --:}, i=l i=l 2' i=l 2' where Ai, J.ti, vi are 0 or 1 such that Ai + J.ti + Vi = 1 for all i. Determine the T

shape ofT. Is it open? Closed? Compact? 1. 7 Show that

Ilxll = ~(IXll + IX21) + ~max{IXll, IX21}

is a norm on ]R2. Determine for this norm the shape of the "unit disc"

B1(O) =

{x

E]R2 ; IIxlI::; I}.

1.8 Show that the map N : ]R2 ----+ ]R defined by

N(Xl,X2) = Jaxi is a norm on]R2 if and only if a

+ 2bxIX2 + ex§

> 0 and ae - b2 > O.

1.9 Deduce the Bolzano-Weierstrass theorem from the Heine-Borel theorem. Hint. Suppose that {xn} is a sequence with IIxnll ::; M, with no accumulation point. Then, for each a with II a II ::; M there is an c > 0 such that Be (a ) contains only a finite number of terms of the sequence { X n }. 1.10 Prove that ]Rn and 0 are the only subsets of]Rn that are open and closed.

IY.2 Continuous Functions 287

IV.2 Continuous Functions ... according to the judgment of all mathematicians, the difficulty that readers of this work experience is caused by the more philosophical than mathematical form of the text .... Now, to remove this difficulty was an essential task for me, if I wanted the book to be read and understood not only by myself, but also by others. (Grassmann 1862, "Professor am Gymnasium zu Stettin")

Let A be a subset of ]R.n . A function

f :A

(2.1 )

-+]R.m

maps the vector x = (Xl ' "'' Xn) E A to the vector Y = (YI,"" Ym) E ]R.m. Each component of Y is a function of n independent variables. We thus write

= h (Xl , . . . ,Xn)

YI

Y = f( x )

(2.2)

or

Ym =

f m (Xl, . .. , Xn).

~

XI

FIGURE2.1a. The function y

= x~ + x~

FIGURE 2. lb. YI

O:S;x :S; 3

= cos lOx , Y2 = sin lax,

Examples. a) One function (m = 1) of two variables (n = 2) can be interpreted as a surface in IR3. For example, the function Y = xi + x~ represents a paraboloid (Fig.2.la). b) Two functions (m = 2) of one variable (n = 1) represent a curve in ]R.3. For example, the spiral of Fig. 2.1 b is given by YI = cos lOx, Y2 = sin lOx. If we project the curve onto the (Yl, Y2)-plane, we obtain a "parametric representation" of a curve in]R.2 (in our example a circle). (2.1) Definition. A function

f :A

-+ ]R.m, A

'liE> O 3 0 'lix EA:

c

]R.n

Ilx-xoll O. In order to explain its behavior close to the origin, we use polar coordinates Xl = r cos 'P, X2 = r sin 'P so that (for r > 0)

y=

r2 cos 'P sin 'P r2

1 . = - sm2'P.

2

IY.2 Continuous Functions 289

Hence, the function is constant on lines going through the origin, with the constant depending on the angle cpo In each neighborhood of (0, 0), the function (2.3) assumes all values between +1/2 and -1/2. Therefore, it cannot be continuous at (0,0). The interest of this example is that the partial functions Xl f--4 f (Xl, 0) and X2 f--4 f(O, X2) are continuous also at the origin. Therefore, there is no analog of Theorem 2.2 for the independent variables x, as Cauchy (1821, p.37) actually thought. He was corrected, with the above counterexample, by Peano (1884, "Annotazione N. 99").

Continuous Functions and Compactness We continue extending the results of Sect. III.3 to functions of several variables. Many of these extensions are straightforward. For example, the analog of Theorem III.3.6 is as follows:

(2.3) Theorem. Let K c JRn be a compact set and let f : K ----; JR be continuous on K. Then, f is bounded on K and admits a maximum and a minimum, i.e., there exists u E K and U E K such that

f(u)

~

f(x)

~

f(U)

for ali

x E K.

o

This theorem leads to the following result, which we already announced in Remark 1.6.

(2.4) Theorem. All norms in JRn are equivalent. This means that if N : JRn ----; JR is a mapping satisfying the conditions (Nl) through (N3) of Theorem 1.1, i.e., (Nl) (N2) (N3)

°{:}

N(x) :::=: 0 and N(x) = x = 0, N(>..x) = 1>"1 N(x) for >.. E JR, N(x + y) ~ N(x) + N(y) (triangle inequality),

then there exist numbers C l (2.4)

>

°

and C 2

>

°

such that forali

x

E JR n .

Proof We first show that N(x) is continuous. We write x = Xlel + X2e2 + ... + xne n , where el = (1,0, ... ,0), e2 = (0,1,0, ... ,0), and so on. It then follows from (N3), (N2), and the Cauchy-Schwarz inequality (1.5) that (2.5)

+ ... + xnen) ~ N(xled + ... + N(xnen) ~ IXll· N(ed + ... + Ixnl· N(en) ~ Ilx112· C2,

N(x) = N(xlel

J

with C2 = N(el)2 + ... + N(e n )2. This proves the second inequality of (2.4). We now see the continuity of N(x) as follows:

290 IV, Calculus in Several Variables

N(x) - N(xo) = N(x - Xo + xo) - N(xo) :s N(x - xo) + N(xo) - N(xo)

:s C211x - xo112'

:s C211xo - x112' so that N(xo)1 :s C211x - xo112.

and similarly N(xo) - N(x) = ... (2.6)

IN(x) -

We then consider the function N (x) on the compact set

K

= {x E IRn; IIxl12 = I}.

By Theorem 2.3, it admits a minimum at some u E K, i.e., (2.7)

N(z) 2 N(u)

for all

z

E K.

Putting C 1 = N(u), which is positive by (Nl), we have for an arbitrary x E IR n (x i- 0) that x/llxll2 E K, and hence also

C1 :s N(II~IJ = 11:112 N(x). o

This proves the first inequality of (204).

Uniform Continuity and Uniform Convergence Exactly as in Sect. IlIA, we call a function f : A --) IR m , A C IR n uniformly continuous if it is continuous on A and if the b in Definition 2.1 can be chosen independently of Xo E A. We have the following extension of Theorem III.4.5.

(2.5) Theorem (Heine 1872). Let f : K --) IRm be continuous on K and let KeIRn be a compact set. Then, f is uniformly continuous on K.

Proof The two proofs of Theorem 111.4.5 can easily be adapted to the case of several variables. Let us give, for our pleasure and as an exercise, a third proof using Theorem 1.21 of Heine-Borel. We know by hypothesis that (2.8)

'VxoEK 'VE>O :Jb>O 'VxEK:

Ilx-xoll 1 be positive numbers. Then

satisfy x P

+ yp

=

1, and it follows from Example 4.9 that

~+~

( ~P+7JP ) 1/ P =

ax + by :::: axo + byo =

~+~

1/ .

(aq+b q) P

We thus obtain where P and q are related by (4.40). By induction on n, this inequality can be generalized to n

(n

LXiYi:::: Lxf

(4.42)

i=l

)l/P(

i=l

n

LY[

)l/q

i=l

for positive numbers Xi and Yi. This is the so-called Holder inequality. For p = q = 2, it reduces to the Cauchy-Schwarz inequality (1.5). With (4.42), we can prove the triangle inequality for the norm Ilxllp of Eq. (1.9). Indeed, for two vectors x, Y E ]Rn, we have

Ilx + YII~ =

n

n

n

i=l

i=l

i=l

L IXi + Yil P:::: L IXil·lxi + YiI P- 1 + L IYil· IXi + YiI P- 1.

We apply (4.42) to the two sums on the right side ofthis inequality and obtain n

n

Ll xil·lxi+YiI P- 1 :::: (LlxiIP)

i=l

This yields Ilx + YII~ inequality Ilx + Yllp ::::

i=l

l/p

n

(Ll xi+Yilq(P-1»)

l/q

i=l

= Ilxllp . Ilx + YII~-l.

:::: (1lxllp + IIYllp) . Ilx + YII~-l, Ilxllp + IIYllp·

and hence the triangle

328 IV. Calculus in Several Variables

Exercises 4.1 (H.A. Schwarz 1873). Show that for

X2 arctan 11.. f(x, y) = { 0 x

-

y2 arctan :£ y

if xy

i' 0

ifxy

= 0,

8 2 f r--I- 8y8x 8 2f . · I d" the second parha envahves at th" e ongm are d'cc lllerent: 8x8y 4.2 Show that Taylor's formula (4.14) only holds if all partial derivatives involved are continuous. This is in contrast to the case of one variable (see, e.g., Theorem III.6.11). The following counterexample by Peano (1884, "Annotazione N.109"), xy {

f(x, y) =

/x2

+ y2

otherwise,

Xo

=

=

Yo

-a, h

=

k

= a + b, shows that Eq. (4.14), written with the

first -order error term

f(xo

8f

8f

+ h, Yo + k) = f(xo, Yo) + 8x (~, 7])h + 8y (C 7])k,

where~ = xo+Bhand7] = yo+Bk are intermediate points, might be wrong. This corrected an error in Serret's book.

4.3 Analyze for Example 4.4 the intersections of the graph of f(x, y) with that of its Taylor approximation of order 2 in the neighborhood of (xo, Yo) and explain the star-shaped curves (see Fig. 4.1). Why do you think the authors chose the point (0.9,1.2) for their figure and not, as in Fig. 3.1, the point (0.8,1.0)? Hint. Use the error formula in (4.14). 4.4 Let f

: ffi.2

---+

ffi. be a differentiable function that satisfies

gradf(x) = g(x)· xT, where 9 : ffi.2 ---+ R Show that

r}. 4.5 Show that U Laplace

f

is constant on the circle {x E ffi.2;

= (x 2 + y2 + z2)-1/2

I xII =

satisfies the differential equation of

4.6 Find the stationary points of the function

f(x, y) = (x 2 + y2)2 - 8xy and study the level curves f(x, y) = Const in their neighborhood. (Any similarity of these curves with curves already seen is intentional).

IV.4 Higher Derivatives and Taylor Series 329

4.7 Find the maximum value of ~xyz subject to (x + y + z)/3 = 1. What conclusion can be drawn from this result? (We have already seen in Example 4.9 that the computation of a conditional maximum is an excellent tool for obtaining interesting inequalities.) 4.8 Find the maxima or minima of x 2

+ y2 + Z2 subject to the conditions and

z = x+y.

Remark. If there are two conditions to satisfy, you will have to introduce two Lagrange multipliers. 4.9 Let

A=

(v'20+ 1

~)

be the matrix of the example in Sect. IY.2. Find the maximum of the function f(x) = IIAxll~ subject to Ilxll~ - 1 = O. The result is the value of IIAI12' defined in Eq. (2.14). 4.10 Show that the function

f : ]R2

-+ ]R

given by

has the origin as a stationary point, but not as a local minimum. Nevertheless, on all straight lines through the origin, the function has a local minimum. With this counterexample, Peano (1884, "Annotazioni N. l33-l36") corrected another error in Serret's book. Such irreverent criticism of the work of the greatest French mathematicians by a 25-year-old Italian "nobody" did not delight everybody (see, e.g., Peano's Opere, p. 40-46).

330 IV. Calculus in Several Variables

IV.5 Multiple Integrals We know that the evaluation or even only the reduction of multiple integrals generally presents very considerable difficulties ... (Dirichlet 1839, Werke, vol. I, p. 377)

The Riemann integral for a function of one variable (Sect. III.5) represents the area between the function and the x-axis. We shall extend this concept to functions f : A -+ ]R (where A c ]R2 ) of two variables in such a way that the integral represents the volume between the surface z = f(x, y) and the (x, y)-plane. Many definitions and results of Sect. IlLS can be extended straightforwardly. However, additional technical difficulties occur, because domains in]R2 are often more complicated than those in]R (see Fig. 5.1). The extension to functions of more than two variables is then more or less straightforward.

I ca

nonconvex

rectangle

nonconnected

FIGURES.I. Possible domains in]R2

Double Integrals over a Rectangle We begin by considering functions f : I -+ ]R, whose domain I = [a , b] x [e, d] = {(x , y) I a :::; x :::; b, e:::; y :::; d} is a closed and bounded rectangle in]R2, and we assume that the function is bounded, i.e., that (5.1)

3 M 2: 0 Y (x, y) E I

If(x, y)1 :::; M.

We consider divisions (5.2)

Dx = {XO,XI,'" ,xn} Dy = {Yo, YI,· .. , Ym}

of [a ,b]' of [e, d],

where a = Xo < Xl < ... < Xn = band e = Yo < YI < .. . 0 315 > 0 \I(Dx,Dy) E '0 0

S(Dx x Dy) - s(Dx x Dy) < E.

Proof For an E > 0 let (Dx , Dy) be given by (S.8). This induces a grid whose length (in the interior of [a , bJ x [c, dJ) is L = (n - l)(d - c) + (iii - l)(b - a) (see Fig. S.3, left picture). We then take an arbitrary division (Dx , Dy) E '00 , set .1 = S(Dx x Dy) - s(Dx x Dy), and put D~ = Dx U Dx , D~ = Dy U Dy, .1' = S(D~ x D~) - s(D~ x D~). We then get, exactly as in Eq. (III.S . lO) (see Fig. S.3, right picture), .1 :::; .1' + L . 15 . 2M. The conclusion is now the same as in the proof of Theorem III.S.8.

I

I

1

1

1

I

I

~

IJ

1 I I

I I

FIGURE ~3 . Di;:ision Dx x intersect Dx x Dy (right)

Dy (left), division D~

D

J

I

L. ,

r

t. . ~

-1

l

i II

II

x D~, elements Iij of D~ x D~ that

Let 6 , ... , ~n be such that Xi-i :::; ~i :::; Xi and TJi, ... , TJm be such that Yj-i :::; TJj :::; Yj· It then follows from Theorem S.2 that

providedthatmaxi(xi-xi_ d < 15 and maxj(Yj-Yj- d < 15. This is true because the sum and the integral in (S.9) both lie between s(Dx x Dy) and S(Dx x Dy).

Iterated Integrals. The inner sum in Eq. (S.9), namely "L,';i f(~i' TJj )(Yj Yj-d, is a Riemann sum for the function f(~i ' y). Assuming this function to be integrable (in the sense of Definition III.S.3) for all i, we obtain from (S.9) that

IY.S Multiple Integrals 333

I:

Here, we are again confronted with a Riemann sum, this time for the function

x

f(x , y) dy. The estimate (5.10) expresses the fact that the Riemann sums converge to III f( x, y) d( x, y) if maxi (xi - Xi- I) -+ 0. Hence, we have (Exer1--+

cise5 .1)

lb (l

(5.11)

d

f( x, y) dY) d x =

Jl

f (X, y) d( x , y)

and have proved the following result. (5.3) Theorem (Stolz 1886, p. 93). Let f : I -+ lR be integrable and assume that for each x E [a, b] the function y 1--+ f( x, y) is integrable on [c, d]. Then, the fun ction x

1--+

led f( x, y) dy is integrable on [a , b] and identity (5.11) holds.

D

Consequently, the computation of a double integral is reduced to the computation of two simple (iterated) integrals and the techniques developed in Sects. 11.4, 11.5, and III.5 can be applied. By symmetry, we also have (5 .12)

ld(l

b

f( x, y) dX) dy =

Jl

f (x, y) d( x , y) ,

provided that f : I -+ lR is integrable and that the function x 1--+ f( x, y) is integrable on [a , b] for each y E [c, d]. The two identities (5.11) and (5.12) together show that the iterated integrals are independent of the order of integration (under the stated assumptions) . Counterexamples. We shall show that the existence of one of the integrals in (5.11) does not necessarily imply the existence of the other.

'(................................................ -:::' () >((.:: ())) (.()J . . . . . . . . ,

:' .............. , .. " .............. , ...................;

............. .... .. .. ......... .. ..............

,

::1·::1.:·.:. :::·)-::V: ::V():: ::k():: ::H::-)·) FIGURES.4a. Nonintegrable function

1) Let (5.13)

FIGURE S.4b. Integrable function

f : [0, 1] x [0, 1] -+ lR be defined by (Fig. 5.4a) f( x, y) =

{~

I'f

( X, y ) =

else.

(~ 2k- l , ~ 2 e-l) W It . h

' n mtegersn , k , "-,

334 IV. Calculus in Several Variables

For a fixed x E [0, 1] there are only a finite number of points with f( x, y) =I=- O. Hence, Jo1 f(x , y)dy = 0 and the iterated integral to the left of (5.11) exists. However, every rectangle [Xi - 1, Xi] X [Yj - 1, yj] contains points with f( x, y) = 1 and points with f( x, y) = O. Consequently, s(Dxx Dy) = OandS(Dx xDy) = 1 for all divisions and the integral to the right of (5.11) does not exist. 2) The function (Fig. 5.4b)

f( x, y) =

(5.14)

I { 01

if

(x = 0 or x = 1) and y E Q

if

(y

= 0 or y = 1) and x E Q

else is integrable, because the points with f (x, y) =I=- 0 form a set that can be neglected (see below). But, for x = 0 or x = 1, the function y ~ f( x, y) is the Dirichlet function of Example 111.5.6, which is not integrable.

Null Sets and Discontinuous Functions Continuous functions f : I ---+ ]R are uniformly continuous (l is compact, Theorem 2.5) and hence integrable. The proof of this fact is the same as for Theorem 111.5.1 O. In the sequel, we shall prove the integrability of functions whose set of discontinuities is not too large. (5.4) Definition. A set X c I C ]R2 is said to be a null set iffor every E > 0 there existfinitely many rectangles h = [ak , bk ] x [Ck ' dk], (k = 1, . .. , n) such that n

n

and

(5.15)

Typical null sets are the boundaries of "regular" sets, e.g., triangles, disks, polygons (see the example of Fig. 5.5 1 ). This is a consequence of the following result.

2 0

J=

110 ,

Ef.1

= 0.880

J

=

210 '

Ef.1

= 0.475

FIGURE 5.5. A null set

I

A null set only in the strict mathematical sense, of course!

2 J

= 4~ '

Ef.1

= 0.263

IV5 Multiple Integrals 335

(5.5) Lemma. Let cp : [0, 1] (S.16)

--+ ]R2

represent a curve in the plane and suppose that

IIcp(s) - cp(t) 1100 :::; M

'Is - tl

forall s,tE [0,1).

Then, the image set cp([O, 1]) is a null set. Proof. We divide [0,1) into n equidistant intervals J1, J2,"" I n of length lin. For s, t E Jk we have IIcp(s) - cp(t) 1100 :::; Min, i.e., cp(Jk) is contained in a square h of side:::; 2M Therefore, the entire curve is contained in a union of n squares h, ... , In, whose area is bounded by

In.

if n is sufficiently large. This proves (S.lS).

o

Condition (S.16) is sufficient, but not necessary, for a curve to be a null set. For example, von Koch's curve (von Koch 1906) of Fig. S.6 is a null set (see Exercise S.S) that has infinite length (hence, (S.16) cannot be satisfied). The curve of Peano-Hilbert (Fig. 2.3) is not a null set, of course. However, Sierpinski's triangle and carpet (Fig. 1.9 and Fig. 1.10) are other interesting examples of null sets.

FIGURE 5.6. A null set, the curve of von Koch

336 IV. Calculus in Several Variables

(5.6) Theorem. Let f

:I

-+ ~

be a boundedfunction (satisfying (5.1)) and define

f

X = {(x, y) E I;

If X

is not continuous at (x, y) }.

is a null set, then the function f(x, y) is integrable.

Proof. Let E: > 0 be given and let U~=1 h be a finite covering of X satisfying (5.15). We enlarge the h slightly and consider open rectangles J 1 , ... , I n such that Jk :J h for all k and L~=1 J.l(Jk) < 2E:. The set H := I \ U~=1 Jk is then closed (Theorems 1.15 and 1.14) and therefore compact (Theorem 1.19). Restricted to H, the function f(x, y) is uniformly continuous (Theorem 2.5), which means that there exists a 6 > 0 such that If(x, y) - f(~, 1])1 < E: whenever Ix - ~I < 6 and Iy - 1]1 < 6. We now start from a grid Dx x Dy containing all the vertices of the rectangles J 1, ... , I n and refine it until the distances Xi - Xi-1 and Yj - Yj-1 are smaller than 6. We then split the difference S (D x x D y) - s(D x x D y) according to

The sum on the left is :S E: J.l (I) because of the uniform continuity of f (x, y) on H; the sum on the right is :S 4ME: because the union of the rectangles Iij (which do not lie in H) is contained in U~=l Jk with an area smaller than 2E:. Both estimates together show that S(Dx x Dy) - s(Dx x Dy) can be made arbitrarily small. 0

Arbitrary Bounded Domains Dirichlet was particularly proud for his method of the discontinuous factor for multiple integrals. He used to say that it's a very simple idea, and added with a smile, but one must have it. CH. Minkowski, Jahrber. DMV, 14 (1905), p. 161)

Let A C ~2 be a bounded domain contained in a rectangle I (i.e., A c I) and let f : A -+ ~ be a bounded function. We want to find the volume under the surface z = f(x, y), with (x, y) restricted to A. The idea (Dirichlet 1839) is to consider the function F : I -+ ~ defined by

F(x,y) = {f(X,Y) if

(5.17)

o

(x,y)EA

else.

If F is integrable in the sense of Definition 5.1, then we define (5.18)

Ii

f(x, y) d(x, y) =

A common situation is where boundary of A, i.e., (5.19)

8A '= .

f :A

{(X ) E ~21 ,y

11

-+ ~

F(x, y) d(x, y). is continuous on A and where the

each neighborhood of (x, y) } contains elements of A and of CA

'

IY.S Multiple Integrals 337

is a null set. In this case, the discontinuities of F all lie in CIA and Theorem S.6 implies the integrability of F . Iterated Integrals. The set A can often be described in one of the following ways:

= {(x,y) I a:::; x:::; b, 'PI (x) :::; y:::; 'P2(X)}, A = {(x,y) Ie:::; y:::; d, 1/h(y) :::; x:::; 1/!2(y)},

(S.20)

A

(S.21)

where 'Pi (x) and 1/!j(Y) are known functions (see Fig. S.7). In this case, the formulas (S.11), (S.12), together with (S.18), yield

ff f(x,y)d(.T ,y) =

(S.22)

}}A

ff f(x , y)d(x,y) }}A

(S.23)

=

la(1'P2(X) 'P,(x) b

f(x,Y)d Y) dx,

jd( }

N

6

-

2Y

A

...8.

B

3

I

•

N

-2

e

I

-1

- 1

-2 FIGURES.8b. Polar coordinates

FIGURE S.8a. Area of the parallelogram

Polar Coordinates. One of the most important applications of Theorem S.7 is when

g(r, cp) = (x, y) ,

(S.26)

x = r coscp,

y = r sincp

(polar coordinates, see Sect. 1.S) and when A = {(x , y) I x 2 + y2 :::; R2}. With B = [0 , R] x [0, 27fJ, the assumption (i) of Theorem S.7 is satisfied. The function 9 of (S.26) is not injective on B (we have g(r, 0) = g( r, 27f) for all r, and g(O , cp) = (0, 0) for all cp). However, if we remove from B the null set N = ({O} x [0, 27fD u ([0, R] X {27f}) (see Fig. S.8b), the function 9 becomes injective on B \ N. Since

detg I (r,cp) = det (cos . cp SIllCP

- rsincp) = r, r cos cp

it follows from Theorem S. 7 and Eq. (S .11) that (S.27)

rr

JJx2+ y 2 "S R2

f(x , y)d(x , y) =

r r f(rcoscp , rsincp)rdrdcp. Jo Jo 27r

R

Proof of Theorem 5.7. Main Ideas. We cover B by a division of closed squares Jj3 with side length (j (see Fig. 5.9, left picture), set B = {)3 I Jj3 n B oj 0}, and let (U j3, vj3) be the left bottom vertex of Jj3 . We assume that (j is sufficiently small, so that all Jj3 ()3 E B) still lie in U. The image set g(Jj3 ) of Jj3 is approximately a parallelogram with sides (Fig. S.9, right picture; Fig. S.lO) (S.28)

340 IV. Calculus in Several Variables

2

-

g

/I I

2

x

o

2

FIGURE 5.9. Transformation of the lattice

(see Example 3.2). Now, from elementary geometry we know that the area of this parallelogram is equal to the determinant 2 (5.29)

area parall. = I det(a b)1 =

Idet (~~ ~~) I =

la 1 b2 - a2 bll ,

and, inspired by Eq. (5.9), we have

Jj

A

f(x , y) d(x, y)

~

L f(g(u {3, V(3)) . (area of g(J{3)) (3E B

~

L

Jl

f(g(U{3, V(3)) i det g'(u{3, V(3 )i M(J{3 )

(3E B

~

i

f(g(u , v)) detg'(u ,

v)i d(u , v ).

This motivates the validity of Eq. (5 .25).

Rigorous Estimates. The integrands in Eq. (5.25) are continuous on A and B, respectively. Since A and B are compact, these functions are bounded. Moreover, DA and DB are null sets, so that by (5.18) the two integrals in Eq. (5.25) exist. In the following we extend the domain of f to ~2 by putting f( x, y) = 0 outside of A. In order to grasp the precise meaning of the left integral of (5 .25), we introduce, in addition to the above division of B, a division of A into squares Ie"" set A = {a I 10: n A i= 0}, and choose (xo:,Yo: ) E 10: n A (these are the fish-eyes 2 The two expressions on the left and on the right of (5.29) are i) invariant under transformations of the type b f-> b + Aa (Cavalieri 's principle), and ii) equal for rectangles parallel to the axis == diagonal matrices; see Fig. S.8a. For more details see Strang (1976, p. 164).

IY.S Multiple Integrals 341

in Fig. 5.10). Equation (5.25) will be proved by showing that the difference of the Riemann sums of the two integrals (see Theorem 5.2 and Eq. (5.9))

(l!EA

[3E B

is smaller than E for any given E > O. It turns out that the side length of the squares must be much smaller than 0 (the side length of J(3 ). We take it::; E . O.

J (l!

FIGURES .lO. Squares Ie> for which a belongs to P{3

Partition of A. The left sum of (5.30) contains much more terms than the right one. In order to compare corresponding terms in this difference, we partition the set A as (disjoint union),

in such a way that (5.31a)

(X(l!, y(l! ) E g(J[3 ) if a E P[3 ,

(5.31b)

a E P [3 if J(l! C 9 (J(3 ) and J [3 C B \ N

(see Fig. 5.10). For a given a E A we can, since (X(l!, Y(l!) E A = g(B) C U [3E B g( J (3), always find a (3 which satisfies (5.31a). In order to be able to satisfy (5.31 b), we have to show that there is at most one (3 E 13 with J [3 C B \ N such that J (l! C g(J[3 ). Suppose that J (l! C g(J[3 ) n g(J{3' ) for some (3 i= (3'. Since 9 is injective on B \ N, we have g(J[3 ) n g(J{3' ) C g(J[3 n J[3I ), so that also J (l! C g(J[3 n J{3' ). But J[3 n J[31 is either empty, or a point, or a segment of a line, so that g( J [3 n J (3' ) is a null set by Lemma 5.5. Hence J(l! C g( J [3 n J(3' ) is impossible for (3 i= (3'.

342 IV. Calculus in Several Variables

Once the sets P{3 determined, we use I:aEA = I:{3EB I:aEP{3 , write the expression ofEq. (S.30) as I:{3EB D{3 with

L

D{3 =

(S.32)

J(xa, Ya) f.L(la) - J(g(U{3, V(3)) 1detg'(u{3, V(3)1 f.L(J{3),

aEP{3

and estimate these terms. For the moment, we consider only so-called "interior" Jis, i.e., we suppose that J{3 c B \ N. We write D{3 as

D{3 =

(S.33a)

L

(!(Xa, Ya) - J(g(U{3, V(3))) f.L(la)

aEP{3

+ J(g(U{3,V{3)) (

(S.33b)

L

f.L(Ia)

-I detg'(u{3,V{3)1 f.L(J{3))

aEP{3

and estimate these two expressions separately.

Estimation of (5.33a). Since g(u, v) is continuously differentiable, g'(u, v) is bounded on the compact set B (Theorem 2.3), i.e.,

11g'(u,v)11 :; Ml

(S.34)

for

(u, v) E B.

Hence, the Mean Value Theorem 3.7 implies that

(indeed, (xa, Ya) lies in g( J(3) and the points of J{3 have from (U{3, V(3) a distance of at most 8 . y'2 ). It then follows from the uniform continuity of J on A (f is continuous on the compact set A) that If(x a , Ya) - J(g(U{3, V(3)) 1 < E: for sufficiently small 8 (remember that 9 (J(3) c A since J(3 is interior). Therefore,

IL

(S.3S)

(!(Xa,Ya) - f(g(u{3,V{3))) f.L(Ia) I :;

aEP{3

E:

L

f.L(Ia).

aEP{3

Estimation of (5.33b). We now must concentrate more seriously on the question how precisely the set g(J{3) is approached by the parallelogram spanned by the vectors a and b in (S.28). We denote this set by

R{3

=

{g(U{3,V{3)

+ ~~(U{3'V{3)S+ ~~(U{3'V{3)t I S E [0,8],

t

E

[0,8]}.

We compute the distance of two corresponding points g(u{3 + s, v{3 + t) in g(J{3) andg( u{3, V(3) +¥U (u{3, v(3) S+~( u{3, v(3) t in R{3 in the following way: Equation (III. 6. 16) written for F(r) = g(u{3 + rs, v{3 + rt) means that

g(U{3+s,V{3+t)-g(U{3,V{3) = Subtracting 8gj8u(u{3, v(3) . s

1

1g'(U{3+rs,V{3+rt). (;) dr.

+ 8gj8v(u{3, V(3) . t

from both sides, we obtain

IY.5 Multiple Integrals 343

Ilg(U(3+s,V(3+t) - (g(U(3,V(3) =

+ ~~(U(3'V(3)S+

~~(U(3'V(3)t)11

1111 (g'(U(3 + TS, V(3 + Tt) - g'(U(3, V(3)) . (~) dT11 ~ V2e8

for 0 ~ s, t ~ 8. The last estimate follows from the uniform continuity of g' on the compact set B (recall that J(3 is interior). Next we enclose R(3 between two sets

where (see Fig. 5.10)

Rt

= {set of points with distance ~ 2V2e8 from the closest point of R(3 }

R~ = {set of points in R(3 with distance::::: 2V2e8 from the border}.

Since the distance 2V2e8 chosen in these definitions is twice V2e8, which, on one side, is the maximal distance between corresponding points of g( J(3) and R(3, and on the other side the maximal diameter of the squares fa, the sets R~ and Rt also enclose, because of (5.31a) and (5.31b), the union of fa for a E P(3 (see Fig. 5.10 again)

U fa cRt·

R~ c

aEP{1

Since Rt \ R~ is a "ring" of length ~ 4M1 8 (see (5.34)) and of "thickness" ~ 4V2e8, the above inclusions lead to the estimate

IL

JL(Ia) - JL(R(3) I ~ JL(Rt \

R~) ~

(4M1 8) (4V2e8).

aEP{1

Consequently, we have (5.36)

If(g(U(3,V(3)) (

L

JL(Ia) -i det g'(U(3,v(3)iJL(J(3))1 ~ Ce8 2

=

CeJL(J(3)

aEP{1

with C = M . 4Ml . 4V2.

Finale. If J(3 rt. B \ N (so that J(3 intersects the null set 8B UN), we estimate D(3 of Eq. (5.32) by ID(31 ~ M2JL(J(3), where M2 is a constant depending on bounds of f and g'. If 8 is sufficiently small, it follows from (5.15) that the sum ofthese ID(31 is ~ M 2 e. For the remaining J(3 we use (5.35) and (5.36), together with (5.33), and obtain

ID(31 ~ e

L

JL(Ia)

+ CeJL(J(3).

aEP{1

All in all, the difference (5.30) of the Riemann sums, i.e., small (~Const· e).

Lj3E13 Dj3,

is arbitrarily 0

344 IV. Calculus in Several Variables

(5.8) Example. Let A can be computed as

I X 2 + y2 :s R2} be the disc of radius R. Its area

= {(x, y)

rr

r r

27r R R2 JJA1.d(x,y) = Jo Jo rdrd