Lecture Notes in Numerical Analysis with Mathematica 160805943X, 9781608059430

Citation preview

LECTURE NOTES in NUMERICAL ANALYSIS with MATHEMATICA ˇ & Tadeusz STYS ˇ Krystyna STYS

Bentham Science Publishers Ltd. Executive Suite Y - 2 PO Box 7917, Saif Zone Sharjah, U.A.E. [email protected] All rights reserved-© 2014 Bentham Science Publishers Ltd. Please read this license agreement carefully before using this eBook. Your use of this eBook/chapter constitutes your agreement to the terms and conditions set forth in this License Agreement. This work is protected under copyright by Bentham Science Publishers Ltd. to grant the user of this eBook/chapter, a non-exclusive, nontransferable license to download and use this eBook/chapter under the following terms and conditions: 1. This eBook/chapter may be downloaded and used by one user on one computer. The user may make one back-up copy of this publication to avoid losing it. The user may not give copies of this publication to others, or make it available for others to copy or download. For a multiuser license contact [email protected] 2. All rights reserved: All content in this publication is copyrighted and Bentham Science Publishers Ltd. own the copyright. You may not copy, reproduce, modify, remove, delete, augment, add to, publish, transmit, sell, resell, create derivative works from, or in any way exploit any of this publication’s content, in any form by any means, in whole or in part, without the prior written permission from Bentham Science Publishers Ltd.. 3. The user may print one or more copies/pages of this eBook/chapter for their personal use. The user may not print pages from this eBook/chapter or the entire printed eBook/chapter for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained from the publisher for such requirements. Requests must be sent to the permissions department at E-mail: [email protected] 4. The unauthorized use or distribution of copyrighted or other proprietary content is illegal and could subject the purchaser to substantial money damages. The purchaser will be liable for any damage resulting from misuse of this publication or any violation of this License Agreement, including any infringement of copyrights or proprietary rights. 5. The following DRM (Digital Rights Management) policy is applicable on this eBook for the non-library / personal / single-user. Library / institutional / multi-users will get a DRM free copy and they may implement their own institutional DRM policy. • 25 ‘Copy’ commands can be executed every 7 days. The text selected for copying cannot extend to more than one single page. • 25 pages can be printed every 7 days. • eBook files are not transferable to multiple computer/devices. If you wish to use the eBook on another device, you must send a request to [email protected] along with the original order number that you received when the order was placed. Warranty Disclaimer: The publisher does not guarantee that the information in this publication is error-free, or warrants that it will meet the users’ requirements or that the operation of the publication will be uninterrupted or error-free. This publication is provided "as is" without warranty of any kind, either express or implied or statutory, including, without limitation, implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the results and performance of this publication is assumed by the user. In no event will the publisher be liable for any damages, including, without limitation, incidental and consequential damages and damages for lost data or profits arising out of the use or inability to use the publication. The entire liability of the publisher shall be limited to the amount actually paid by the user for the eBook or eBook license agreement. Limitation of Liability: Under no circumstances shall Bentham Science Publishers Ltd., its staff, editors and authors, be liable for any special or consequential damages that result from the use of, or the inability to use, the materials in this site. eBook Product Disclaimer: No responsibility is assumed by Bentham Science Publishers Ltd., its staff or members of the editorial board for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products instruction, advertisements or ideas contained in the publication purchased or read by the user(s). Any dispute will be governed exclusively by the laws of the U.A.E. and will be settled exclusively by the competent Court at the city of Dubai, U.A.E. You (the user) acknowledge that you have read this Agreement, and agree to be bound by its terms and conditions. Permission for Use of Material and Reproduction Permission Information for Users Outside the USA: Bentham Science Publishers Ltd. grants authorization for individuals to photocopy copyright material for private research use, on the sole basis that requests for such use are referred directly to the requestor's local Reproduction Rights Organization (RRO). The copyright fee is US $25.00 per copy per article exclusive of any charge or fee levied. In order to contact your local RRO, please contact the International Federation of Reproduction Rights Organisations (IFRRO), Rue Joseph II, 9-13 I000 Brussels, Belgium; Tel: +32 2 234 62 60; Fax: +32 2 234 62 69; E-mail: [email protected]; url: www.ifrro.org This authorization does not extend to any other kind of copying by any means, in any form, and for any purpose other than private research use. Permission Information for Users in the USA: Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by Bentham Science Publishers Ltd. for libraries and other users registered with the Copyright Clearance Center (CCC) Transactional Reporting Services, provided that the appropriate fee of US $25.00 per copy per chapter is paid directly to Copyright Clearance Center, 222 Rosewood Drive, Danvers MA 01923, USA. Refer also to www.copyright.com

Contents 0.1 0.2

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . i Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

1 Floating Point Computer Arythmetic 1.1 Computer Representation of Numbers 1.2 Conditional Number . . . . . . . . . . P 1.3 Round-off Error of the Sum xi . . . Q 1.4 Round-off Error of the Product xi . . 1.5 Horner Scheme . . . . . . . . . . . . . 1.6 Exercises . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

2 Natural and Generalized Interpolating Polynomials 2.1 Lagrange Interpolation . . . . . . . . . . . . . . . . . . 2.2 Certain Chebyshev’s Systems . . . . . . . . . . . . . . 2.3 Relationship Between a Function and its Interpolating Polynomial . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Optimal Interpolating Polynomial . . . . . . . . . . . 2.5 Hermite Interpolation. . . . . . . . . . . . . . . . . . 2.6 Relationship Between a Function and its Hermite Interpolating Polynomial. . . . . . . . . . . . . . . . . . . . 2.7 Trigonometric Polynomials . . . . . . . . . . . . . . . 2.8 Relationship between Trigonometric Polynomials and Fourier’s Transformation . . . . . . . . . . . . . . . . . 2.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 3 Polynomial Splines 3.1 Space Sm (∆, k). . . . . . . . . 3.2 Properties of Splines. . . . . . . 3.3 Examples of Polynomial Splines 3.4 Lagrange Interpolating Splines

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

1 . 1 . 6 . 9 . 13 . 15 . 22 27 . 28 . 30 . 38 . 44 . 49 . 51 . 54 . 58 . 59

. . . .

63 63 67 71 86

3.5 3.6 3.7

Polynomial Splines of Two Variables on Rectangular Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Space Π1 of Piecewise Linear Polynomial Splines on Triangular Networks . . . . . . . . . . . . . . . . . . . . . . 97 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4 Uniform Approximation 4.1 Taylor Polynomials and Taylor Theorem . . . . . 4.2 Bernstein Polynomials and Weierstrass Theorem 4.3 Best Approximation Polynomials . . . . . . . . . 4.4 Chebyshev Series . . . . . . . . . . . . . . . . . . 4.5 Exercises . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

103 103 111 116 127 131

5 Introduction to the Least Squares Analysis 5.1 Introduction . . . . . . . . . . . . . . . . . . . 5.2 Line of Regression. . . . . . . . . . . . . . . . 5.3 Functions Determined by Experimental Data 5.4 Least Squares Method in Space L2 (a, b) . . . 5.5 Exercises . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

133 133 137 139 145 155

6 Selected Methods for Numerical Integration 6.1 Introduction . . . . . . . . . . . . . . . . . . . 6.2 Newton-Cotes Methods . . . . . . . . . . . . 6.3 Trapezian Rule. . . . . . . . . . . . . . . . . . 6.4 Simpson Rule. . . . . . . . . . . . . . . . . . . 6.5 Romberg Method and Recursive Formulas . . 6.6 Gauss Methods . . . . . . . . . . . . . . . . . 6.7 Gauss-Legendre Knots . . . . . . . . . . . . . 6.8 Gauss Formulas . . . . . . . . . . . . . . . . . 6.9 Error of Gauss Methods . . . . . . . . . . . . 6.10 Exercises . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

157 158 158 160 166 170 182 183 187 189 194

7 Solving Nonlinear Equations by Iterative Methods 7.1 Fixed Point Iterations . . . . . . . . . . . . . . . . . 7.2 Newton’s Method . . . . . . . . . . . . . . . . . . . . 7.3 Secant Method . . . . . . . . . . . . . . . . . . . . . 7.4 Bisection Method . . . . . . . . . . . . . . . . . . . . 7.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

199 201 207 214 223 226

References

1 230

Index

233

i

0.1

FOREWORD

The lecture notes have been developed during over thirty years of teaching experience by the authors in the area of Numerical Analysis. They have taught the course on Numerical Methods designed for Science and Engineering students, at the University of Warsaw, University of Jos, Nigeria and the University of Botswana. The content of the notes covers the following topics: Computer Numbers and Roundoff Errors Estimates, Interpolation and Approximation of Functions, Polynomial Splines and Applications, Numerical Integration and Solution of Non-linear Equations. The authors have presented the subjects in exact and comprehensive way with emphasis put on formulation of fundamental theorems with proofs supported by well selected examples. They used Mathematica, System for doing Mathematics, in solving problems specific to the subjects. In the notes, the reader will find interesting algorithms and their implementation in the Mathematica System. The lecture notes are well written and recommended as reading material for a basic course in Numerical Methods for science and engineering students. O. A. Daman University of Botswana, Botswana

iii

0.2

PREFACE

This text is intended for science and engineering students. It covers most of the topics taught in a first course on numerical analysis and requires some basic knowledge in calculus, linear algebra and computing. The text has been used as recommended handbook for courses taught on numerical analysis at undergraduate level. Each chapter ends with a number of questions. It is taken for granted if the reader has access to computer facilities for solving some of these questions using Mathematica. There is extensive literature published on numerical analysis including books on undergraduate and postgraduate levels. As the text for a first course in numerical analysis, this handbook contains classical content of the subject with emphases put on error analysis, optimal algorithms and their implementation in computer arithmetic. There is also a desire that the reader will find interesting theorems with proofs and verity of examples with programs in Mathematica which help reading the text. The first chapter is designed for floating point computer arithmetic and round-off error analysis of simple algorithms. It also includes the notion of well conditioned problems and concept of complexity and stability of algorithms. Within chapter 2, interpolation of functions is discussed. The problem of interpolation first is stated in its simplest form for polynomials, and then is extended to generalized polynomials. Different Chebyshev’s systems for generalized interpolating polynomials are considered. In chapter 3, polynomial splines are considered for uniform approximation of an one variable function. Fundamental theorems on uniform approximation (Taylor’s theorem, Weierstrass theorem, Equi-Oscillation Chebeshev’s

iv

theorem) are stated and proved in chapter 4. Chapter 5 is an introduction to the least squares method and contains approximation of functions in the norm of L2 (a, b) space. Also, it contains approximation of discrete data and an analysis of experimental data. In the chapter 6, two techniques of numerical integration are developed, the Newton-Cotes methods and Gauss methods. In both methods an error analysis is carried out. For solution of non-linear algebraic equations, the most popular methods, such as Fixed Point Iterations, Newton Method, Secant Method and Bisection Method, are described in chapter 7. ACKNOWLEDGEMETNTS Declared None CONFLICT OF INTEREST The authors confirm that this chapter contents have no conflict of interest ˇ Krystyna STYS University of Warsaw, Poland ˇ Tadeusz STYS University of Warsaw, Poland

v

The List of Mathematica Functions and Modulae Horner function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Optimal interpolating polynomial . . . . . . . . . . . . . . . 47 Hermit interpolating polynomial . . . . . . . . . . . . . . . . 54 Linear spline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Cubic spline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Quadratic spline

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Quadratic Lagrange spline. . . . . . . . . . . . . . . . . . . . . . . 91 Chebyshev first polynomial . . . . . . . . . . . . . . . . . . . . . 130 Chebyshev second polynomial . . . . . . . . . . . . . . . . . . . . 130 Line of regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Gram-Schmidt orthogonalization . . . . . . . . . . . . . . . . 152 Romberg table of integral. . . . . . . . . . . . . . . . . . . . . . 176 Fixed point iterations . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Newton iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Secant iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Bisection iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 1-26 1

CHAPTER 1

Floating Point Computer Arythmetic Abstract In this chapter, propagation of round-off errors in floating point arithmetic operations of computers numbers is presented. The notions of the conditional number, stability and complexity of algorithms are introduced and illustrated by examples. The Horners scheme for evaluation of Polynomials is given to elucidate the optimal and well-conditioned algorithms when they are implemented in a computer system like Mathematica. The chapter ends with a set of questions. Keywords: Computer numbers, Round-off errors.

1.1

Computer Representation of Numbers

Normally, computers operate in floating point arithmetic, and decimal numbers are presented in the following form: x = ∓m10c , Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

2 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

where m = 0.α1 α2 . . . αr ; α1 6= 0 ; 0 ≤ αi ≤ 9 ; i = 1, 2, . . . , r is called mantissa of x, and the integer c is called exponent of x. The most significant digit α1 is always different from zero. Therefore, mantissa m satisfies the following inequality: 0.1 ≤ m < 1.

Clearly, a number can have exact floating point computer representation if its mantissa consists of a finite number of digits. For example 1/4 has exact floating point computer representation and its mantissa m = 0.25, while the power c = 0. However 1 = 0.333 . . . 3 cannot be accepted as a floating point number by a computer since its mantissa m = 0.333..., has infinite number of digits. Nevertheless, any real number, even with infinite number of digits, can be also represented in a computer floating point arithmetic with certain round-off error  ≤ 21 10−r . For example 2 x = = 0.66666666666... 3 has the 4-digit floating point representation f l(x) = 0.6667

and its round-off error  = 0.0000333... In order to round off in Mathematica at fourth decimal place, we execute the following instruction (cf. [28]) 2 N[Round[ 104]/104] = 0.6667. 3 1

Below, we shall consider round-off up operation of a floating point number (cf. [5, 15]) 1 Note

x = ±m 10c

that in computers round off down rule is used

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 3

at the r-th decimal place. In the round-off up operation x is replaced by fl(x), so that f l(x) = ±m 10c , where the new mantissa m = f l(m) = 2

 r −r  [m ∗ 10 + 0.5] ∗ 10 

if c = c,

[m ∗ 10r + 0.5] ∗ 10−r+1 if c = c + 1.

Clearly, the exponent c = c + 1 if α1 = α2 = · · · = αr = 9 and αr+1 ≥ 5, otherwise c does not change its value. For example, if x = 0.99999 ∗ 10−2 and r = 4, then the mantissa m = f l(0.99999) = 0.1000, and the exponent c = c + 1 = −2 + 1 = −1.

Thus, the 4 digit floating point representation of x is f l(0.99999 ∗ 10−2 ) = 0.1000 ∗ 10−1 . Also, for x = 0.234584 ∗ 102 and r = 3, we have m = [0.234584 ∗ 103 + 0.5] ∗ 10−3 = 0.235, and c = c = 2. Hence, the 3 digit floating point representation of x is f l(0.234584 ∗ 102 ) = 0.235 ∗ 102 . The Absolute and Relative Round-off Errors The absolute round-off error of a real number x = ∓m10c is x = fl(x) − x 2 Here

[y] is the greatest integer not greater than y.

4 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

This error satisfies the inequality | f l(x) − x |≤  ∗ 10c , where  = 12 10−r . For example, let x = 0.57367864 ∗ 102 and r = 3. Then, we have | f l(0.57367864 ∗ 102 ) − 0.57367864 ∗ 102 |= 1 −3 10 ∗102 = 0.05. 2 The relative round-off error of a floating point number x = ∓m 10c 6= 0 is | 0.574∗102 −0.57367864∗102 |= 0.032136
0. Below, we shall estimate propagation of these errors in the four arithmetic operations. For addition and subtraction |(x ± y) − (x ± y)| = |∆x ± ∆y| ≤ 2β. For multiplication |xy − xy| = |(x + ∆x)(y + ∆y) − xy|. Hence, for small ∆x and ∆y the term ∆x∆y ≈ 0, and then the error estimate is: |xy − xy| ≤ |x| |∆y| + |y| |∆x| ≤ (|x| + |y|)β.

For division, when y 6= 0, we have

x + ∆x x x x − | | − | =| y y y + ∆y y xy + y∆x − xy − x∆y | =| y(y + ∆y) |x| |∆y| + |y| |∆x| ≤ . |y|2(1 − ∆y ) y

∆y ≈ 0, we arrive at the following error y estimate in division |x| |∆y| + |y| |∆x| x x |x| + |y| ≤ β. | − |≤ 2 y y y y2 Hence, for small

1.2

Conditional Number

Conditional number is naturally affiliated with a problem under consideration. This number is used to assess perturbation of the final results caused by round-off errors or errors

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 7

in measurement of input data. For example, the problem of evaluating of a differentiable function f(x) for a given x leads to the conditional number determined below. Namely, if x∗ is an approximate value to x then the relative errors f(x) − f(x∗ ) f(x)

x − x∗ x

and

determine the conditional number Cond(f(x)) =

f (x)−f (x∗ ) f (x) lim x−x∗ x∗ →x x

0

=x

f (x) , f(x)

x 6= 0,

f(x) 6= 0,

at a point x. Thus, the conditional number is expressed in terms of f(x) 0 and f (x) and it is independent of an algorithm used to evaluate f(x). Example 1.1 Let f(x) = xα , for a real α. We get 0

f (x) αxα−1 Cond(f(x)) = x = x α = α. f(x) x Hence, the conditional number of the power function equals to the constant α. Let us note the following relations which hold for two differentiable functions: Cond(f(x)g(x)) = Cond(f(x)) + Cond(g(x)), Cond(

f(x) ) = Cond(f(x)) − Cond(g(x)). g(x)

Indeed, we have 0

0

f (x)g(x) + f(x)g (x) Cond(f(x)g(x)) = x f(x)g(x) = Cond(f(x)) + Cond(g(x)),

8 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

and 0

(x) Cond( fg(x) )

0

f (x)g(x) − f(x)g (x) g(x) = x g 2 (x) f(x) = Cond(f(x)) − Cond(g(x)).

A problem is said to be well-conditioned if its conditional number is relatively small, otherwise it is referred to as illconditioned problem. In order to solve a problem, we should choose an algorithm which is stable, that is, resistant against round-off errors accumulation. Here, we consider stability with respect to round-of errors accumulation, in the sense of the following definition: Definition 1.1 An algorithm is said to be stable with respect to round-off errors accumulation, if it produces a solution biased by error proportional to a computer precision. Normally, stable algorithms are known for well-conditioned problems, that is, for problems with relatively small conditional numbers. Example 1.2 Let us evaluate f(x) = x1024, at x = 0.999, in three digits floating point arithmetic, by the following two algorithms: Algorithm I . • Let x := 1. • For i:=1 to 1024, compute x := 0.999 ∗ x, in three digit floating point arithmetic. It is clear that the conditional number of the product is equal to 1024. The result obtained by the algorithm is equal to 0.5.

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 9

Algorithm II . • Let x := 0.999, i := 0; • Repeat

x := x ∗ x, i := i + 1,

until i = 10; The conditional number of the product is 20. The result of the second algorithm is equal to 0.3530 Comparing both the results with the exact value f(0.999) = 0.3590, we note that the result from the first algorithm is less accurate than the one obtained by the second algorithm. This is due to relatively small conditional number involved in the second algorithm. A part of stability, an algorithm is also characterized by the number of operations needed to transform input data into expected results. This number of operations is called computational complexity of an algorithm. In the above example, the computational complexity of the first algorithm is equal to 1024 operations, and the computational complexity of the second algorithm is equal to 10 operations.

1.3

Round-off Error of the Sum

P

xi .

Let xi = ∓mi10ci , i = 1, 2, . . . , n; be r-digit floating point numbers and let i be round-off errors associated with adding xi to the previously evaluated sum x1 +x2 +· · ·+xi−1 which do not exceed the computer precision δ = 21 101−r , i.e.

10 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

| i |, ≤ δ , i = 1, 2, . . . , n. Then, we have f l(x1 + x2 + · · · + xn ) = = (· · · (x1 + x2)(1 + 2 ) + x3)(1 + 3) + · · · + xn )(1 + n ) =

n X

xi

i=1

n Y

(1 + j ), when 1 = 0,

j=i

and |f l(

n X i=1

xi ) −

n X i=1

xi | = | ≤

n X

i=1 n X i=1

xi [

n Y

(1 + j ) − 1]|

j=i

| xi|[(1 + δ)n−i+1 − 1]

≤ [(1 + δ)n − 1]

n X i=1

|xi |.

Since, for small δ, we have (1 + δ)n − 1) ≈ nδ, the absolute and relative round-off errors of the sum x1 + x2 + . . . + xn satisfy the following inequalities: |f l(

n X i=1

|f l(

xi ) −

Pn

n X i=1

xi |

Pn

xi ) − Pn | i=1 xi |

i=1

i=1

≤ nδ xi |

n X i−1

|xi |,

Pn

|xi | ≤ nδ Pi=1 . n | i=1 xi |

For example, let us compute the sum

98.7 + 2.48 + 0.875 in 3-digit floating point arithmetic f l(98.7 + 2.48 + 0.875) = fl(fl(98.7 + 2.48) + 0.875 = fl(101 + 0.875) = 102.

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 11

The exact result 98.7 + 2.48 + 0.875 = 102.055. and the absolute round-off error is equal to 0.055 The relative round off error f l(98.7 + 2.48 + 0.875) − (98.7 + 2.48 + 0.875) 0.055 = 102.055 102.055 = 0.0005363. Example 1.3 Consider the following function: g(x, y, z) = x + y + z,

x, y, z ∈ (0, ∞).

(i). Evaluate g(0.95, 0.475, 0.625) in 3-digit floating point arithmetic. Determine absolute and relative round-off errors. (ii). Give an estimate of absolute and relative round-off errors for evaluation of the function g(x, y, z) assuming that x, y, z ∈ (0, ∞) are 3-digit floating point numbers. Solution (i). We compute f l(0.95 + 0.475 + 0.625) = fl(fl(0.95 + 0.475) + 0.625) = fl(0.143 ∗ 10 + 0.625) = 0.206 ∗ 10. The exact value of the sum 0.95 + 0.475 + 0.625 = 2.05. Thus, the absolute error | f l(0.95 + 0.475 + 0.625) − (0.95 + 0.475 + 0.625) | = 2.06 − 2.05 = 0.01,

12 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

and the relative round-off error f l(0.95 + 0.475 + 0.625) − (0.95 + 0.475 + 0.625) 0.95 + 0.475 + 0.625 0.01 = = 0.004878. 2.05 Using Mathematica, when round off down rule is used, the result is N[0.95 + 0.475 + 0.625, 3] = 2.05. Solution (ii). The computer precision 1 δ = 101−3 = 0.005. 2 Let 2 and 3 be round-off errors which may appear when the two additions are executing i.e. | 2 |≤ δ and | 3 |≤ δ. Then, we have f l(x + y + z) = [(x + y)(1 + 2) + z](1 + 3) = (x + y)(1 + 2)(1 + 3) + z(1 + 3) = x + y + z + (x + y)(2 + 3 + 23 ) + z3. Since 23 is small compared with 2 or 3 then the term 23 ≈ 0 may be ignored. Therefore, the absolute round-off error estimate is | f l((x + y + z) − (x + y + z)) | ≤ 2δ(x + y) + δz ≤ 2δ(x + y + z) ≤ 0.01 (x + y + z), for all 3-digit floating point numbers x, y, z ∈ (0, ∞). Hence, the estimate of relative round-off error of the sum is 0.01 (x + y + z) f l(x + y + z) − (x + y + z) |≤ = 0.01, | x+y+z x+y+z for all 3-digit floating point numbers x, y, z ∈ (0, ∞), x + y + z 6= 0.

Floating Point Computer Arythmetic

1.4

Numerical Analysis with Mathematica 13

Round-off Error of the Product P

Q

xi.

As in the sum xi , we assume that xi = ∓mi 10ci , i = 1, 2, . . . , n; are r-digit floating point numbers and the relative round-off errors ηi in the multiplications do not exceed the computer precision δ = 12 101−r , i.e. | ηi |≤ δ; i = 1, 2, . . . , n. We then arrive at f l(x1x2 · · · xn ) = x1x2 · · · xn (1 + η2 ) · · · (1 + ηn ) =

n Y

xi

i=1

n Y

(1 + ηk ) when η1 = 0.

k=1

Hence, we obtain the following estimate of the absolute round-off error of the product | f l(

n Y

i=1

xi ) −

n Y

i=1

xi | = | [

n Y

i=1

(1 + ηi ) − 1] n

≤ ((1 + δ) − 1) |

n Y

i=1 n Y

i=1

xi |

xi |≤ nδ |

n Y

i=1

xi | .

Therefore, the relative round-off error satisfies the following inequality: |f l(

Qn

Qn

xi ) − Qn | i=1 xi |

i=1

i=1

xi |

= |Πni=1 (1 + ηi ) − 1| ≤ (1 + δ)n − 1) ≤ nδ.

For example, we may evaluate the product 57.6 ∗ 4.68 ∗ 0.385 in 3-digit floating point arithmetic as follows: f l(57.6 ∗ 4.68 ∗ 0.385) = fl(fl(57.6 ∗ 4.68) ∗ 0.385) = fl(270 ∗ 0.385) = 104. The exact result is 57.6 ∗ 4.68 ∗ 0.385 = 103.78368.

14 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

The absolute round-off error | f l(57.6 ∗ 4.68 ∗ 0.385) − (57.6 ∗ 4.68 ∗ 0.385) | = 104 − 103.78368 = 0.21632

f l(57.6 ∗ 4.68 ∗ 0.385) − (57.6 ∗ 4.68 ∗ 0.385) | 57.6 ∗ 4.68 ∗ 0.385 0.21632 = = 0.00208. 103.78368 Example 1.4 Consider the following function: |

g(x, y, z) = x y z,

for x, y, z ∈ (0, ∞).

(i). Evaluate g(3.75, 5.86, 8.65) in 3-digit floating point arithmetic. Determine absolute and relative round-off errors. (ii). Give an estimate of absolute and relative round-off errors assuming that x, y, z ∈ (0, ∞) are computer numbers and the computer precision δ = 0.005. Solution (i). We compute f l(3.75 ∗ 5.86 ∗ 8.65) = fl((3.75 ∗ 5.86) ∗ 8.65)) The exact result

= fl(22 ∗ 8.65) = 190.

3.75 ∗ 5.86 ∗ 8.65 = 190.08375.

The absolute round-off error | f l(3.75 ∗ 5.86 ∗ 8.65) − 3.75 ∗ 5.86 ∗ 8.65) | =| 190 − 190.08375 |= 0.08375.

and the relative round-off error f l(3.75 ∗ 5.86 ∗ 8.65) − 3.75 ∗ 5.86 ∗ 8.65 | | 3.75 ∗ 5.86 ∗ 8.65 0.08375 = 0.000440595. = 190.08375 In Mathematica, we obtain the same result when the following statement is executed

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 15

N[3.75*5.86*8.65,3]; Solution (ii). In order to estimate the absolute and relative round-off errors for arbitrary x, y, z ∈ (−∞, ∞), we find |f l(x ∗ y ∗ z) − x ∗ y ∗ z| = | x ∗ y ∗ (1 + η2 ) ∗ z ∗ (1 + η3 ) − x∗y∗z | = |x ∗ y ∗ z ∗ (η2 + η3 + η2 ∗ η3 |. Since η2 ∗ η3 ≈ 0, the absolute round-off error estimate is | f l(x ∗ y ∗ z) − x ∗ y ∗ z |≤ 2δ|x ∗ y ∗ z| = 0.01 ∗ |x ∗ y ∗ z| and relative round-off error estimate is |

f l(x ∗ y ∗ z) − x ∗ y ∗ z |≤ 2δ = 0.01. x∗y ∗z

for all x, y, z ∈ (−∞, ∞) and xyz 6= 0.

1.5

Horner Scheme

In order to compute the value of the polynomial Pn (x) = an xn + an−1 xn−1 + . . . + a1x + a0 at a point x, we can use the Horner’s scheme, Pn (x) = (...((anx + an−1 )x + an−2 )x + . . . + a1 )x + a0 which leads to the following algorithm: Let p = an , then evaluate

for i = n − 1, n − 2, . . . , 0, p := px + ai .

16 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

As a result of execution of this algorithm, we obtain the value of Pn (x) for n multiplications and n additions. Later on, we shall show that the total number of 2n arithmetic operations to determine Pn (x) cannot be less. Let us compare the algorithm based on Horner scheme with the following one: Compute x 2 , x 3 , . . . , xn , and Pn (x) = an xn + an−1 xn−1 + ... + a1x + a0. Clearly, to compute x2 , x3, . . . , xn , we have to execute n − 1 multiplications, and then, to compute Pn (x), we need additionally n multiplications, and n additions. Thus, the total number of arithmetic operations involved in the second algorithm is 3n-1. The above two ways of evaluation of Pn (x) suggest that any numerical problem can be solved by a number of algorithms with different number of arithmetic operations involved in each, that is, with different computational complexity. Obviously, we apply optimal algorithm with minimum number of arithmetic operations, when such algorithm is known and produces satisfactory solution. However, in complex problems, optimal algorithm may be unknown or even may not exist, or the optimal algorithm is known but produces unsatisfactory solution because of its instability. In the case when optimal and stable algorithm is not available, we may use any stable algorithm with relatively small computational complexity. There are two important properties which are expected from any good algorithm 1. algorithm with the smallest computational complexity, that is, optimal algorithm with minimum number of arithmetic operations, or close to an optimal algorithm, 2. Stable algorithm, that is, algorithm which is resistance against any perturbation of input data or partial results

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 17

caused be round-off errors. As we have stated earlier, an algorithm is stable if in its implementation round-of errors accumulation is proportional to a computer precision. We shall show that Horner scheme is an optimal and a stable algorithm. In order to prove that Horner scheme is the optimal algorithm, we apply the principle of mathematical induction. Namely, let the thesis T (n) be to evaluate Pn (x) = an xn + an−1 xn−1 + · · · + a1x + a0, we have to execute 2n multiplications and additions. The thesis T (1) is true, since to find P1 (x) = a1 x + a0 , we must execute one multiplication and one addition. Now, assume that the thesis T (n) is true i.e. to compute Pn (x) = an xn + an−1 xn−1 + · · · + a1x + a0 = (...((anx + an−1 )x + an−2 )x + . . . + a1 )x + a0 , we must execute 2n multiplications and additions. If it is so, then T (n) implies T (n + 1). Indeed, the polynomial Pn+1 (x)

= an+1 xn+1 + an xn + · · · + a1 x + a0 = (...((an+1 x + an )x + an−1 )x + an−2 )x + . . . + a1 )x + a0

|

{z

Pn (x)

}

By assumption, we may find Pn (x) using 2n multiplications and 2n additions. Then, to evaluate Pn+1 (x), we additionally need one multiplication and one addition. Thus, the

18 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

minimum number of operations required in Horner scheme is 2n + 2 = 2(n + 1). Now, we shall investigate the round-off error in Horner scheme. Let x and ai, i = 0, 1, . . . , n be computer floating point numbers and let i and ηi be round-off errors associated with addition and multiplication, respectively. Then, we have f l(Pn (x)) = (· · · (an x(1 + ηn ) + an−1 )(1 + n−1 )x(1 + ηn−1 ) +an−2 )(1 + n−2 )x(1 + ηn−2 +an−3 )(1 + n−3 )x(1 + ηn−3 +···············+ +a1(1 + 1 )x(1 + η1 ) +a0(1 + 0 ). Hence, for η0 = 0 and n = 0, we have f l(Pn (x)) = an xn (1 + ηn )(1 + ηn−1 ) · · · (1 + η1 )(1 + n−1 ) · · · (1 + 0 ) +an−1 xn−1 (1 + ηn−1 ) · · · (1 + η1 )(1 + n−1 ) · · · (1 + 0 ) +··················+ +a1 x(1 + η1 )(1 + 1)(1 + 0) +a0 (1 + 0). and f l(Pn (x)) =

n X i=0

a i xi

i Y

(1 + ηj )(1 + j ).

j=0

Because | ηi |≤ δ, η0 = 0 and | i |≤ δ, therefore

i = 0, 1, · · · , n,

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 19

|f l(Pn (x)) − Pn (x)| = |

n X

a i xi [

i=0

i Y

(1 + ηj )(1 + j ) − 1]|

j=0

≤ [(1 + δ)2n − 1]

n X i=0

|ai xi |

Let us note that (1 + δ)2n − 1 ≈ 2nδ for a small δ, so that the absolute error is proportional to the computer precision, that is |f l(Pn (x)) − Pn (x)| ≈ 2nδ

n X i=0

|aixi |.

The relative error estimate is: P

n i |f l(Pn (x)) − Pn (x))| 2n i=0 | aix | ≤ ((1 + δ) − 1) P P | ni=0 ai xi | | ni=0 ai xi| Pn i i=0 | ai x | ≤ 2nδ Pn . | i=0 aixi |

Example 1.5 Consider the following polynomial: P4 (x) = 126.7x4 + 34.82x3 + 8.458x2 + 0.2347x − 9554 (i). Evaluate P4 (2.875) in 4-digit floating point arithmetic using Horner scheme. Determine absolute and relative roundoff errors. (ii). Give an estimate of the absolute and relative round-off errors for evaluation of Pn (x) by Horner scheme assuming that x is an 4-digit floating point number. Solution (i). The Horner’s scheme of the polynomial P4 (x) is P4 (x) = (((126.7∗x+34.82)∗x+8.458)∗x+0.2347)∗x−9554

20 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Rounding off after each operation in 4-digit floating point arithmetic, we obtain f l(P4(2.875))

= f l((((126.7 ∗ 2.875 + 34.82) ∗ 2.875 + 8.458) ∗2.875 + 0.2347) ∗ 2.875 − 9554) = f l(((399.1 ∗ 2.875 + 8.458) ∗ 2.875 + 0.2347) ∗2.875 − 9554) = f l((1155 ∗ 2.875 + 0.2347) ∗ 2.875 − 9554) = f l(3321 ∗ 2.875 − 9554) = f l(9548 − 9554) = −6.

and the exact value P4 (2.875) = 0.25. So that, the absolute round-off error | f l(P4 (2.875) − P4 (2.875) |= | − 6 − 0.25| = 6.25 and the relative round-off error |

f l(P4 (2.875)) − P4 (2.875) 6.25 |= ≈ 25. P4 (2.875) 0.25

To evaluate a polynomial Pn (x) = an xn + an−1 xn−1 + ... + a0 at a point x by Horner’s scheme in Mathematica, we input data x and the list of coefficients {an , an−1 , ..., a0}. Then, we execute the following commands: horner[x_,a_]:=Fold[(#1*x+#2)&, First[a],Rest[a]]; horner[x,a] For example, to evaluate P4 (x) , we input data x=2.875; a={126.7,34.82,8.458,0.2347,-9554}; Then, we define the function horner[x_,a_]:=Fold[(#1*x+#2)&, First[a],Rest[a]]; Calling the function

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 21

horner[x_,a_] we obtain P4 (2.875) = 0.474172 Following the details of the analysis of round-off error in the Horner’s scheme, we may obtain the upper bounds of the round-off errors. Namely, we have f l(P4(x))

= ((((126.7x(1 + η4 ) + 34.82)(1 + 3 )x(1 + η3 ) +8.458)(1 + 2 )x(1 + η2 ) + 0.2347)(1 + 1 )x(1 + η1 ) −9554(1 + 0 ).

Hence, for η0 = 0 f l(P4 (x)) = 126.7x4 (1 + η4 )(1 + η3 )(1 + η2 )(1 + η1 ) ∗(1 + 3 )(1 + 2 )(1 + 1)(1 + 0) +34.82x3 (1 + η3 )(1 + η2 )(1 + η1 )(1 + 3 )(1 + 2 ) ∗(1 + 1 )(1 + 0 ) +8.458x2 (1 + η2 )(1 + η1 )(1 + 2 )(1 + 1)(1 + 0) +0.2347x(1 + η1)(1 + 1)(1 + 0) −9554(1 + 0) =

4 X i=0

ai x

i

i Y

(1 + ηj )(1 + j ),

j=0

where a0 = −9554, a1 = 0.2347, a2 = 8.458, a3 = 34.82, a4 = 126.7. Since | ηi , |, η0 = 0 and 1 | i |< δ = 10−3 = 0.0005, i = 0, 1, 3, 4, 2

22 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

we get the following estimation of the absolute round-off error 8

|f l(P4 (x)) − P4 (x)| ≤ ((1 + δ) − 1) ≤ 8δ

4 X i=0

4 X i=0

|ai xi |

| ai xi | = 0.004

4 X i=0

|ai xi |.

and of the relative round-off error P

4 i f l(P4 (x)) − P4 (x) i=0 | ai x | | |≤ 0.004 P P4 (x) | 4i=0 ai xi |

for P4 (x) 6= 0.

1.6

Exercises

Question 1.1 Calculate f l(104 + 1.414213562) in 5-digit floating point arithmetic. Determine the absolute and relative round-off errors. Question 1.2 Calculate in the 4-digit floating point arithmetic the following expressions: (a) 43.65 + 126.8 + 0.9754, (b) (c) (d)

54.09 ∗ 0.8756 ∗ 2.645, 4.876 − 23.65, 0.6543 45.56 ∗ 4.875 + 102.6 ∗ 0.795 − 1251 ∗ 346.4.

Determine the absolute and relative round-off errors. Question 1.3 Let the numbers x and y be rounded-off in r-digit floating point arithmetic, i.e. x = fl(x) , y = fl(y), ∆x = |x − f l(x)| ≤  and ∆y = |y − fl(y)| ≤ , where  is the round-off error. Suppose that in propagation of relative round-off errors x ≈ x and y ≈ y. Show the following estimates of the relative round-off errors:

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 23

1. (a) |∆(x + y)| |∆x| |∆y|   ≤ max{ , } ≤ max{ , }. |x + y| |x| |y| |x| |y|

provided that sign x = sign y. (b)

|∆(xy)| |∆x| |∆y|   ≈ + ≤ + ≤ 2δ. |xy| |x| |y| |x| |y| Question 1.4 Consider the following function: g(x, y, z) = xy + z for x, y, z ∈ (−∞, ∞). (a). Evaluate g(8.75, 4.86, 9.65) in 3-digit floating point arithmetic. (b). Give an estimate of the absolute and relative roundoff errors for evaluation of g(x, y, z) assuming that x, y, z ∈ (−∞, ∞) are computer numbers and the computer precision δ = 0.005. Question 1.5 Give an estimate of the absolute and relative round-off errors of the following products (i)

n Y 1

i=1

xi

,

(ii)

n X

a i bi ,

i=1

where xi , ai and bi , i = 1, 2, . . . , n are computer numbers with r-decimal digit mantissa. Question 1.6 Assume that the function G(x, y, z) = x2 ∗ y 2 ∗ z 2 ,

x, y, z ∈ (−∞, ∞),

has been evaluated in the floating point arithmetic with com1 puter precision δ = 101−r . 2 1. (a) Give the estimates of the absolute and relative roundoff errors of f l(G(x, y, z)).

24 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

(b) Evaluate G(1.25, 2.12, 4.50) in the 3-digit floating point arithmetic. Determine the computer precision of the three digit arithmetic and estimate the absolute and relative round-off errors of the value fl(G(1.25, 2.12, 4.50)). Question 1.7 Consider the following polynomial: P4 (x) = 2x4 + 2x3 + 3x2 + 4x + 5 for

0 ≤ x ≤ 1.

(a). Evaluate P4 (0.475) in 4-digit floating point arithmetic using Horner scheme. Determine absolute and relative roundoff errors. (b). Assume that x ∈ [0, 1] is a 4-digit floating point number. Give an estimate of the absolute and relative round-off errors to evaluate P4 (x) by Horner scheme in 4-digit floating point arithmetic. Question 1.8 Give an estimate of the absolute and relative round-off errors for evaluation of the function s(x) =

n X ixi−1 i=1

3i

by Horner scheme in 3-digit floating point arithmetic. What is the error bound for n = 5, x = 0.75. Question 1.9 Consider the function f(x) =

1 + sin2 x 1 + cos2 x

− ∞ < x < ∞.

1. (a) Compute the conditional number of the function f(x) at a point x ∈ (−∞, ∞). and show that the 3 conditional number does not exceed |x|. 2 (b) Find all functions for which the conditional number is constant.

Floating Point Computer Arythmetic

Numerical Analysis with Mathematica 25

EXERCISES IN MATHEMATICA In order to convert a rational number x to a real number x, we execute the command: N[x] or N[x, r], where r is the number of digits. For example x = 1/7; r = 20; N[x, r] 0.14285714285714285714 In order to round off a real number x to the integer closest to x, we execute the command: Round[x]. For example Round[12.5123] 13 In order to round a number x to r decimal digits after dot, we execute the command: N[Round[10^r x]/10^r]; For example x = 23.453879; r = 1; N[Round[10^r x]/10^r] 23.5 The command Floor[x] gives the greatest integer less than or equal to x. For example Floor[-21.546] -22 The command Ceiling[x] gives the smallest integer greater than or equal to x. For example

26 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Ceiling[-21.546] -21 Question 1. Let x = 432.7568. (a) Round off x up to two digits after dot. (b) Find the greatest integer less or equal to x. (c) Find the smallest integer greater or equal to x.

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 27-62

27

CHAPTER 2

Natural and Generalized Interpolating Polynomials Abstract

In this chapter, Lagranges and Hermits interpolation by polynomials, by trigonometric polynomials, by Chebyshevs polynomials and by generalized polynomials spanned on Chebyshevs systems of coordinates are presented. Lagranges and Newtons formulas to find the interpolating polynomials are derived and clarified. Mathematica modules are designed to determine interpolating polynomials. Fundamental theorems on interpolation with the errors bounds are stated and proved. The application of the theorems has been elucidated by examples. The Chapter ends with a set of questions. Keywords: Lagrange polynomials, Newton polynomials, Chebyshev polynomials. Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

28 Numerical Analysis with Mathematica

2.1

Krystyna STYŠ & Tadeusz STYŠ

Lagrange Interpolation

In this section, we shall describe Lagrange interpolation of a function using natural and generalized polynomials. In order to state the problem of Lagrange interpolation, we consider a partition of the bounded interval [a, b] by the following points: a = x0 < x1 < x2 < . . . < xn = b. Let us assume that to each point xi a number yi , 0, 1, ..., n, is assigned, where

i =

y 0 , y 1 , . . . , yn are measured or calculated values of a function y = f(x). In fact, f(x) can only be given at the interpolating points xi , i = 0, 1, ..., n, so that yi = f(xi ),

i = 0, 1, . . . , n.

Now, let us state the problem of Lagrange interpolation (cf. [2,3,4,5,8, 10,12,13,14,15,18,23]) Find a polynomial pn (x) = a0 + a1x + · · · + an xn of degree at most n such that pn (xi ) = yi ,

i = 0, 1, . . . , n.

(2.1)

The following theorem holds: Theorem 2.1 There exists a unique interpolating polynomial pn (x) to f(x) which satisfies the Lagrange conditions (2.1)

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 29

Proof. From the conditions of interpolation (2.1), we obtain the following linear system of algebraic equations: a0 + a1 x0 + a2 x20 + · · · + an xn0 = y0 a0 + a1 x1 + a2 x21 + · · · + an xn1 = y1 ·································

(2.2)

a0 + a1 xn + a2x2n + · · · + an xnn = yn where a0 , a1, · · · , an are unknown coefficients of the polynomial pn (x). The matrix of this system of equations is non-singular since its determinant 1

1 1 ··· 1

x0 x1 ··· xn

x20 x21 ··· x2n

··· ··· ··· ···

xn0 xn1 ··· xnn

n Y = (xi − xj ) i>j

is different from zero if xi 6= xj for i 6= j. Therefore, the system of algebraic equations (2.2) has one solution. This solution determines the unique interpolating polynomial pn (x). We may find polynomial pn (x) solving the system of linear equations (2.2). However, this system of linear equations can be ill-conditioned, so that, the round-off errors may strongly affect the final results. Therefore, we should look for more economic and stable methods to find the interpolating polynomial pn (x). We shall deal with such methods later. Now, let us state the problem of Lagrange interpolation for generalized polynomials. Let φ0 (x), φ1(x), . . . , φn (x) be a sequence of linearly independent functions given on the interval [a, b]. 1 Vandermonde’s

determinant

30 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

A linear combination Pn (x) = a0φ0 (x) + a1 φ1(x) + · · · + an φn (x) is called generalized polynomial. We shall assume that any generalized polynomial Pn (x) cannot have more than n different roots in the interval [a, b]. Then, the sequence of functions φ0 (x), φ1(x), . . . , φn (x), which satisfies this assumption is called Chebyshev’s system. Similarly, we state the problem of Lagrange interpolation for generalized polynomials Find a generalized polynomial Pn (x) = a0φ0 (x) + a1 φ1(x) + · · · + an φn (x)

which satisfies the following conditions: Pn (xi ) = yi ,

i = 0, 1, . . . , n.

Below, we present examples of Chebyshev’s systems.

2.2

Certain Chebyshev’s Systems

Let us list the most common used basis of interpolating polynomials. (a). Base of natural polynomials. Let φi (x) = xi ,

i = 0, 1, . . . , n.

Then, the natural form of a polynomial is: Pn (x) = a0 + a1 x + a2 x2 + . . . + an xn . where −∞ < x < ∞, i = 0, 1, · · · , n. (b). Lagrange base of polynomials. Let us consider

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 31

polynomials li (x), i = 0, 1 . . . , n, of degree n which satisfy the following conditions of interpolation: li (xj ) =

(

1 if i = j, 0 if i = 6 j.

(2.3)

Clearly, the above conditions satisfy the following Lagrange polynomials: li (x) =

(x − x0)(x − x1) · · · (x − xi−1 )(x − xi+1 ) · · · (x − xn ) , (xi − x0)(xi − x1) · · · (xi − xi−1 )(xi − xi+1 ) · · · (xi − xn )

where −∞ < x < ∞, i = 0, 1, . . . , n. Thus, the Lagrange interpolating polynomial takes the form Pn (x) = a0l0 (x) + a1 l1(x) + . . . + an ln (x). Indeed, by conditions (2.3), we have Pn (xi ) = ai = yi , and

i = 0, 1, · · · , n,

Pn (x) = y0 l0(x) + y1l1(x) + · · · + yn ln (x).

(2.4)

Example 2.1 Find the Lagrange interpolating polynomial which attains values 4,2,-1,10 at points -2,1,2,4. Write down the natural form of this polynomial. Solution. From the Lagrange formula (2.4), we have (x − 1)(x − 2)(x − 4) (−2 − 1)(−2 − 2)(−2 − 4) (x + 2)(x − 2)(x − 4) (x + 2)(x − 1)(x − 4) +2 −1 (1 + 2)(1 − 2)(1 − 4) (2 + 2)(2 − 1)(2 − 4) (x + 2)(x − 1)(x − 2) +10 (4 + 2)(4 − 1)(4 − 2) 41 3 83 2 127 55 = x − x − x+ . 72 72 36 9 In Mathematica, we find the polynomial P3 (x) by the following command: P3 (x) = 4

32 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Expand[InterpolatingPolynomial[{{-2,4},{1,2}, {2,-1},{4,10}},x]]

(c). Newton’s base of interpolating polynomials. Let us consider the following polynomials:

φ0 (x) = 1 φ1 (x) = x − x0 φ2 (x) = (x − x0 )(x − x1 ) φ3 (x) = (x − x0 )(x − x1 )(x − x2 ) ····················· φn (x) = (x − x0 )(x − x1 )(x − x2 ) · · · (x − xn−1 ) Then, Newton interpolating polynomial takes the following form:

Pn (x) = a0 + a1(x − x0) + a2 (x − x0 )(x − x1 ) + · · · + an (x − x0 )(x − x1 ) · · · (x − xn−1 ), where the coefficients a0, a1, . . . , an, are determined by the following conditions of interpolation (2.1):

a0

= y0

a0 + a1 (x1 − x0 )

= y1

a0 + a1 (x2 − x0 ) + a2 (x2 − x0 )(x2 − x1 )

= y2

······················································

···

a0 + a1 (xn − x0 ) + · · · + an (xn − x0 ) · · · (xn − xn−1 )

= yn

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 33

The solution of the this system of linear equations is expressed in terms of the following finite differences: a0 = [x0] = y0 a1 = [x0x1 ] =

[x1] − [x0] x1 − x0

a2 = [x0x1 x2] =

[x1x2 ] − [x0x1 ] x2 − x0

························ [x1x2 · · · xn ] − [x0x1 · · · xn−1 ] an = [x0x1x2 · · · xn ] = xn − x0 (2.5) Thus, Newton’s interpolating polynomial is: Pn (x) = [x0] + [x0x1 ](x − x0) + [x0x1 x2](x − x0 )(x − x1 ) + · · · +[x0x1x2 · · · xn ](x − x0)(x − x1) · · · (x − xn−1 ). (2.6) In order to compute the coefficients a0, a1, · · · , an from the formulas (2.5), it is convenient to use the following table: x0 x1 x2 x3

| | | | | | | |

y0 [x0x1 ] y1

[x0x1 x2 ] [x1x2]

y2

[x0 x1 x2 x3 ] [x1x2x3 ]

[x2x3] y3

[x0 x1 x2 x3 x4 ] [x1x2 x3x4 ]

[x2x3x4 ] [x3x4]

.. . . | .. xn−1 | yn−1 | [xn−1 xn ] xn | yn Example 2.2 Find the Newton’s interpolating polynomial

34 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

which attains values 1,3,2 and 5 at the points 0,2,3,5. Reduce this polynomial to its natural form. By the Newton’s formula (2.6), we have P3 (x) = 1 + [02](x − 0) + [023](x − 0)(x − 2) + [0235](x − 0)(x − 2)(x − 3). We can get the coefficients [02], [023] and [0235] from the following table: 0 | | 2 | | 3 | | 5 |

1 1 3 2 5

−1 3 2

− 32 5 6

3 10

Hence, the Newton’s interpolating polynomial is 2 3 P3 (x) = 1 + x + (− x(x − 2) + x(x − 2)(x − 3) 3 10 13 2 3 3 62 = 1+ x− x + x . 15 6 10 To find the Newton’s interpolating polynomial P3 (x) in Mathematica, we execute the following command: InterpolatingPolynomial[{{0,1},{2,3},{3,2}, {5,5}},x] We obtain the natural form of the polynomial P3 (x) by the command Expand[InterpolatingPolynomial[{{0,1},{2,3},{3,2}, {5,5}},x]]

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 35

(d). Base of piecewise linear splines. Let us consider the following piecewise linear splines

ψi(x) =

 0         x − xi−1

xi − xi−1      xi+1 − x   

xi+1 − xi

for i = 1, 2, ..., n − 1,

ψ0(x) = and ψn (x) =

 0   

if x ≤ xi−1 orx ≥ xi+1 , if xi−1 ≤ x ≤ xi , if xi ≤ x ≤ xi+1 ,

  0   

if x > x1 ,

x1 − x     x1 − x0

x − xn−1   

.

if x0 ≤ x1. if x < xn−1 , if xn−1 ≤ x ≤ xn .

xn − xn−1

6 ψ0 @ @ @

ψ1

@ @ @ @x

ψi

@ @

@

Since ψi (x) = we get



x








@

@ @

ψn @ @x

xi i+1 Fig. 2.1. Linear spline ψi (x)

1

(2.7)

i−1

(








x
n−1




-

1 if i = j, 0 if i 6= j,

ai = yi , i = 0, 1, . . . , n. Hence, the linear piecewise interpolating polynomial (see Fig. 2.1) takes the following form: P1 (x) = y0ψ0 (x) + y1 ψ1(x) + · · · + yn (x)ψn (x).

(2.8)

36 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 2.3 Find a piecewise linear interpolating spline which attains values 1,3,0, and 4 at points 0,1,2 and 3. Following the formula (2.8), we find P1 (x) = 1ψ0 (x) + 3ψ1 (x) + 0ψ2 (x) + 4ψ3 (x) =

=

 1−x x−0   +3    1−0 1−0      2 − x   3 +            

2−1 x−2 4 3−2 0

if 0 ≤ x ≤ 1, if 1 ≤ x ≤ 2, if 2 ≤ x ≤ 3, otherwise,

 1 + 2x if 0 ≤ x ≤ 1,     6 − 3x if 1 ≤ x ≤ 2, =  4x − 8 if 2 ≤ x ≤ 3,   

0

otherwise.

We present the piecewise linear spline on the following figure (see Fig. 2.2.) 4 7→ 6 3 7→

!b !! b ! b ! ! b ! b ! ! b 1 7→ ! b

0

b b

1 Fig. 2.2. P1 (x).

b#

2

#

# #

# #

# #

# #

-

3

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 37

(e) Chebyshev’s polynomials. Let us consider Chebyshev’s polynomials T0(x) = 1, T1(x) = x, T2(x) = 2x2 − 1, ···············, Tn (x) = 2x Tn−1 (x) − Tn−2 (x), −∞ < x < ∞, n = 2, 3, . . . . or Tn (x) = cos(n arc cos x), −∞ < x < ∞, n = 0, 1, . . . ; The interpolating polynomial spaned by Chebyshev’s polynomials is Pn (x) = a0 T0(x) + a1T1(x) + · · · + an Tn (x), −∞ < x < ∞. Example 2.4 Find an interpolating polynomial spaned by Chebyshev’s polynomials which attains values 1,0,1 at points 0,1 and 2. Reduce this polynomial to its natural form. Thus, we have P2 (x) = a0 T0(x) + a1T1(x) + a2T2 (x), where the coefficients a0, a1, a2 are determined by the equations a0 T0(0) + a1 T1(0) + a2 T2(0) = 1, a0 T0(1) + a1 T1(1) + a2 T2(1) = 0, a0 T0(2) + a1 T1(2) + a2 T2(2) = 1, which take the following form: a0 − a2 =1 a0 + a1 + a2

=0

a0 + 2a1 + 7a2 = 1.

38 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence, we find 3 a0 = , 2

a1 = −2,

1 a2 = , 2

and 3 1 − 2x + (2x2 − 1) = 1 − 2x + x2. 2 2 To find the Chebyshev’s interpolating polynomial P2 (x) we execute the following program: P2 (x) =

m=Table[ChebyshevT[n,x],{x,0,2},{n,0,2}]; b={1,0,1}; a={a1,a2,a3}; a=LinearSolve[m,b]; a[[1]]-a[[2]]x+a[[3]](2x^2-1)

2.3

Relationship Between a Function and its Interpolating Polynomial

In the previous section, we have shown that there is only one interpolating polynomial pn (x) spaned by the points x0 < x1 < · · · < xn , which satisfies the conditions: yi = f(xi ), i = 0, 1, . . . n. As we have seen, this unique polynomial pn (x) can be given by different formulas, (Lagrange or Newton’s formulas). Obviously, it is interesting to know what is a relationship between the function y = f(x) and its unique interpolating polynomial pn (x). This relationship is given in the following theorem: Theorem 2.2 If y = f(x) is a function (n + 1) times continuously differentiable in the interval [a,b], then there exists a point ξx ∈ (a, b) such that f(x) = pn (x) +

f (n+1) (ξx ) (x − x0)(x − x1) · · · (x − xn ) (n + 1)!

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 39

for all x ∈ [a, b]. Proof. Let us consider the following auxiliary function g(t) = pn (t) − f(t) −

wn+1 (t) [pn (x) − f(x)], wn+1 (x)

(2.9)

for x ∈ [a, b], x 6= xi , i = 0, 1, . . . , n, where wn+1 (x) = (x − x0)(x − x1) · · · (x − xn ).

(2.10)

Since f(x) is (n + 1) times continuously differentiable in the interval [a, b], g(t) is also (n + 1) times continuously differentiable in the interval [a, b]. From the formula (2.9), it follows that g(t) possesses (n + 2) different zeros in the interval [a, b]. Then, we have g(x) = 0 and g(xi ) = 0,

i = 0, 1, . . . , n.

By the Rolle’s theorem, the derivative g 0 (t) has (n + 1) different zeros in the open interval (a, b). Therefore, its second derivative g 00 (t) has n different zeros in the interval (a, b). In this way, we can conclude that the derivative g (n+1) (t) of order (n + 1) possesses at least one zero in the interval (a, b). Therefore, there exists a point ξx ∈ (a, b) such that g (n+1) (ξx ) = 0.

(2.11)

On the other hand, by (2.11), we have g (n+1) (ξx ) = −f (n+1) (ξx ) −

(n + 1)! [pn (x) − f(x)] = 0. wn+1 (x)

Hence f(x) = pn (x) +

f (n+1) (ξx ) (x − x0)(x − x1) · · · (x − xn ). (n + 1)!

Error of interpolation. From the theorem, it follows that the error of interpolation f(x) − pn (x)

40 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

satisfies the equality f(x)−pn (x) =

f (n+1) (ξx ) (x−x0)(x−x1) · · · (x−xn ). (2.12) (n + 1)!

for certain ξx ∈ (a, b). Hence M (n+1) | f(x) − pn (x) |≤ |(x − x0)(x − x1) · · · (x − xn )|, (n + 1)! where

M (n+1) = max | f (n+1) (x) | . a≤x≤b

Now, let us note that | (x − x0)(x − x1) · · · (x − xn ) |≤

n!hn+1 , 4

for n ≥ 1, and h = max (xi+1 − xi ). 0≤i≤n

Indeed, we have

| (x − xi )(x − xi+1) |≤

h2 , 4

x ∈ [xi, xi+1 ],

and Hence

| x − xi+k |≤ (k + 1)h,

k = 0, 1, . . . , n − i.

n!hn+1 . 4 (2.13) Combining the above two inequalities, we obtain the following error estimate: | (x−x0)(x−x1 ) · · · (x−xi )(x−xi+1 ) · · · (x−xn ) |≤

| f(x) − pn (x) |≤

M (n+1) n+1 h , 4(n + 1)

when a ≤ x ≤ b. (2.14)

Example 2.5 Estimate the error of interpolation of a function twice continuously differentiable in the interval [a, b] by piecewise linear spline.

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 41

In this case, the interpolating generalized polynomial is Pn (x) = f(x0 )ψ(x) + f(x1 )ψ1(x) + · · · + f(xn )ψn (x). Let xi < x < xi+1 . Then, by formula (2.12), we have f(x)−Pn (x) =

f 00 (ξx ) (x−xi )(x−xi+1), 2

xi < x < xi+1.

Hence | f(x) − Pn (x) |≤

M (2) | (x − xi)(x − xi+1 ). 2

Since | (x − xi )(x − xi+1 ) |≤

h2 , 4

xi < x < xi+1 ,

the error of interpolation satisfies the following inequality | f(x) − Pn (x) |≤

M (2) 2 h, 8

a ≤ x ≤ b.

Let us note that the constant 81 is possible, that is, if we consider any estimate of the form | f(x) − Pn (x) |≤ CM (2)h2, then the constant C must be greater than or equal to 18 . Indeed, for the function f(x) = 12 x(1 − x), 0 ≤ x ≤ 1 and h = 1, we have Pn (x) = 0 and | f(x) − Pn (x) |=

1 8

for

1 x= . 2

Example 2.6 Function f(x) = cosh( x2 ) is tabulated below.

x y

-2 1.54308

-1 1 2 1.12763 1.12763 1.54308

42 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

(a) Using data in the table, find Lagrange interpolating polynomial P (x) and reduce it to the natural form. (b) Construct a divided difference table based on the above data and determine Newton’s interpolating polynomial. (c) Interpolate the value f(0.8) and show that the error of interpolation satisfies the following inequality | f(x) − P (x) |< 0.0161 for each x ∈ [−2, 2]. Solution of (a). From the Lagrange formula (x + 1)(x − 1)(x − 2) (−2 + 1)(−2 − 1)(−2 − 2) (x + 2)(x − 1)(x − 2) +1.12763 (−1 + 2)(−1 − 1)(−1 − 2) (x + 2)(x + 1)(x − 2) +1.12763 (1 + 2)(1 + 1)(1 − 2) (x + 2)(x + 1)(x − 1) . +1.54308 (2 + 2)(2 + 1)(2 − 1)

P3 (x) = 1.54308

Hence, the natural form of the interpolating polynomial is P2 (x) = 0.1384834x2 + 0.9891464. Solution of (b). By the Newton’s formula P2 (x)

= 1.54308 + [−2, −1](x + 2) + [−2, −1, 1](x + 2)(x + 1) +[−2, −1, 1, 2](x + 2)(x + 1)(x − 1).

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 43

We can get the coefficients [-2,-1], [-2,-1,1] and [-2,-1,1,2] from the following table −2 | 1.54308 |

−0.41545

−1 | 1.12763 | 1

0

| 1.12763 |

2

0.13848 0 0.13848 0.41545

| 1.54308

Hence, the Newton’s interpolating polynomial is P2 (x) = 1.54308 − 0.41545(x + 2) + 0.13848(x + 2)(x + 1). In Mathematica, we can obtain the natural form of the interpolating polynomial P2 (x) by the following program: data={{-2,Cosh[-2/2]},{-1,Cosh[-1/2]}, {1,Cosh[1/2]},{2,Cosh[2/2]}}; N[Expand[InterpolatingPolynomial[data,x]],4] Solution of (c). The interpolating value of f(0.8) ≈ P (0.8) = 1.077759. In order to estimate the error of interpolation, we apply the formula

E(x) = f(x)−Pn (x) =

f (n+1) (ξx ) (x−x0)(x−x1) . . . (x−xn). (n + 1)!

44 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence, for n = 3, h = 2, by (2.14), we have E(x) = cosh( x2 ) − (1.54308 − 0.41545(x + 2) +0.13848(x + 2)(x + 1)) cosh(ξx /2) (x + 2)(x + 1)(x − 1)(x − 2) 16 ∗ 4! cosh(ξx /2) ≤ 24 ≤ 0.0964. 16 ∗ 4 ∗ (3 + 1)

=

However, using the inequality |(x + 2)(x + 1)(x − 1)(x − 2)| ≤ 4

for all x ∈ [−2, 2],

we obtain more optimistic error estimate |E(x)| ≤ 0.0161 for all x ∈ [−2, 2].

2.4

Optimal Interpolating Polynomial

As we know, the error of interpolation E(x) = f(x) − Pn (x) =

f (n+1) (ξx ) wn+1 (x), (n + 1)!

where wn+1 (x) = (x − x0 )(x − x1 )...(x − xn ). Clearly, the error E(x) depends on distribution of the interpolating points x0 , x1, . . . , xn , in the interval a, b]. Thus, E(x) attains its minimum if max |wn+1 (x)| =

a≤x≤b

min

max |wn+1 (x)|,

x0 ,x1 ,...xn a≤x≤b

(2.15)

From formula (2.15), it follows that the maximal error of interpolation attains its smallest value if wn+1 (x) is the best polynomial among all polynomials with the coefficient

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 45

an+1 = 1 at xn+1 , approximating the zero-function on the interval [a, b]. Then, points x0 , x1, . . . , xn which satisfy the formula (2.15) are called Chebyshev’s interpolating knots and the interpolating polynomial Pn (x) spaned by the Chebyshev’s knots is called optimal interpolating polynomial. In 1857, E.W. Chebyshev found that 1 wn+1 (x) = T˜n+1 (x) = n cos((n + 1) arc cos x) 2 is the best polynomial approximating the function identically equal to zero on the interval [−1, 1]. The Chebyshev polynomials T˜n+1 (x), n = 0, 1, · · · ; satisfy the following recursive formula: T˜0(x) = 2, T˜1(x) = x, T˜2(x) = x2 − 21 , ············ T˜n+1 (x) = xT˜n (x) − 14 T˜n−1 (x). Thus, the optimal interpolating polynomial for a function f(x) on the interval [−1, 1] is spaned by the roots of the Chebyshev’s polynomial Tn+1 (x). Solving the equation cos((n + 1) arc cos x) = 0, we find the roots xk = cos

(2k + 1)π , 2(n + 1)

k = 0, 1, . . . , n.

Since |T˜n+1 (x)| = |(x − x0)(x − x1 ) · · · (x − xn )| ≤

1 , 2n

we arrive at the following error estimate: |f(x) − Pn (x)| ≤

M (n+1) , 2n (n + 1)!

−1 ≤ x ≤ 1.

(2.16)

46 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

In order to determine an optimal interpolating polynomial on the interval [a, b] 6= [−1, 1], we may use the following linear mapping: z=

1 (2x − a − b), b−a

to transform the interval [a, b] on the interval [−1.1]. Example 2.7 Find an optimal interpolating polynomial of degree at most two for the following functions f(x) = cos π2 x, −1 ≤ x ≤ 1, √ 0 ≤ x ≤ 2. (ii) f(x) = 1 + x, (i)

Estimate the error of interpolation. Solution (i). For n = 2, the Chebyshev’s interpolating knots are √ √ 3 3 x0 = − , x1 = 0, x2 = . 2 2 Therefore

√ π 3 y0 = f(x0 ) = cos (− ) = 0.208897, 4 y1 = f(x1 ) = cos 0 = 1, √ π 3 = 0.208897. y2 = f(x2 ) = cos 2

Using the Lagrange formula, we find P2 (x) = +

√ 3 ) 2√ √ √ .2089 3 3 + ( 2 +√ 23 ) 2 x(x + 3 ) .2089 √3 √32 √3 . ( 2 + 2 )2

x(x −

(x + (0 +

√ 3 )(x − √2 3 )(0 − 2

√

3 ) √2 3 ) 2

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 47

Hence, we get the natural form of the optimal interpolating polynomial P2 (x) = −1.0548 x2 + 1.

From (2.16), we can get the following error estimate:

π M (3) |f(x) − P2 (x)| = |cos x + 1.0548x2 − 1| ≤ . 2 3! 4 Since 000

f (x) =

π3 π sin x, 8 2

we get π π3 π3 max |sin x| = . 8 2 8 Thus, the error of interpolation satisfies the inequality M (3) =

π π3 |cos x + 1.0548 x2 − 1| ≤ = 0.161. 2 8 3! 4 In Mathematica, we can find the optimal interpolating polynomial P2 (x) by the following program: data={{-Sqrt[3]/2,Cos[-Pi*Sqrt[3]/4]},{0,1}, {Sqrt[3]/2,Cos[Pi*Sqrt[3]/4]}}; N[Expand[InterpolatingPolynomial[data,x]],4] Solution (ii). At first, let us transform the interval [0, 2] on the interval [−1, 1] using the following mapping: z = x − 1. Now, we shall interpolate the function √ −1 ≤ z ≤ 1. g(z) = f(z + 1) = 2 + z, Also, in this case, the Chebyshev’s interpolating knots are: √ √ 3 3 z0 = − , z1 = 0, z2 = . 2 2

48 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Therefore y0 = g(z0) =

q

√

y2 = g(z2) =

q

√ 3 2

2 − 23 = 1.06488, √ y1 = g(z1) = 2 = 1.4142414, 2+

= 1.931852.

Using the Lagrange formula, we find s √ √ √ (z + 3 z(z − 23 ) √ √ √ + P2 (z) = 2− 2 2 3( 3 + 3) (0 + s √ 2 2 √ 32 3 z(z + 2 ) √ √ √ . + 2+ 2 ( 3 + 3) 3 2

2

√ 3 )(z √2 3 )(0 2

−

−

√ 3 ) √2 3 ) 2

2

Hence, after simplification q √ 1 √ [6 + ( 3(4 + 3) 3 2 q q q √ √ √ 4 − 3( 3))z + 2( 3(4 + 3) − 3(4 − 3))z 2]. −

P2 (z) =

Coming back to the original variable x, we obtain the following optimal interpolating polynomial P2 (x) = 1.00453 + 0.456757x − 0.0470738x2 .

The error of interpolation satisfies the following inequality: |f(x) − P2 (x)| ≤ Since we have

M (3) . 3! 22

3 5 000 |f (x)| = (1 + x)− 2 , 8

3 000 M (3) = max|f (x)| = , 8 and the error estimate 3 |f(x) − P2 (x)| ≤ = 0.0156. 8 3! 4

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 49

In Mathematica, we can find the optimal interpolating polynomial P2 (x) by the following program: Clear[x,z]; z0=-Sqrt[3]/2; z1=0; z2=Sqrt[3]/2; {{z0,Sqrt[2+z0]},{z1,Sqrt[2+z1]},{z2,Sqrt[2+z2]}}; P2=InterpolatingPolynomial[data,z]; z=x-1; P2=Expand[P2,4];

2.5

Hermite Interpolation.

Within this section, we shall consider the Hermite’s interpolation of a given data by a polynomial. This kind of interpolation requires as input data y0, y1 , . . . , yn and the 0 00 (s −1) derivatives yi , yi , . . . , yi i ; i = 0, 1, . . . , si − 1; at the interpolating knots x0 , x1, . . . , xn . Let us state Hermite interpolation problem as follows: Find a polynomial Pm (x) = a0 + a1 x + · · · + am xm , which satisfies the following interpolating conditions: Pm (xi ) = a0 + a1xi + a2x2i + · · · + am xm i = yi , Pm (xi ) = a1 + 2a2xi + 3a3 x2i + · · · + mamxm−1 = yi , i 0

0

Pm (xi ) = 2a2 + 3 ∗ 2a3 xi + · · · + m(m − 1)am xm−2 = yi , i 00

00

············································· Pm(si −1)(xi ) = 2 ∗ 3 ∗ · · · ∗ (si − 1)xasi + · · · (si −1)

i +2 +m(m − 1)(m − 2) · · · (m − si + 1)xm−s a m = yi i

,

50 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

for i = 0, 1, . . . , n; and s0 + s1 + · · · + sn = m + 1. Under these conditions, there exists a unique Hermite interpolating polynomial Pm (x). This can be shown in a similar way as it has been proved for the Lagrange interpolating polynomial. Let us note that the Hermite interpolation problem reduces to the Lagrange interpolation problem, if s0 = s1 = s2 . . . = sn = 1, . Below, we shall determine a Hermite interpolating polynomial in the case when s0 = s1 = · · · = sn = 2 and m = 2n + 1. We shall first find a polynomial P2n+1 (x) = a0 + a1x + · · · + a2n+1 x2n+1 , which satisfies the interpolating conditions 0 P2n+1 (xi ) = yi , P2n+1 (xi) = y 0(xi ), i = 0, 1, . . . , n.

This polynomial, we shall find in the following form: H2n+1 (x) = Ln (x) + wn+1 (x)Hn (x),

(2.17)

where Ln (x) is the Lagrange interpolating polynomial (2.4) and wn+1 (x) is the polynomial given by formula (2.10). The polynomial H2n+1 (x) already satisfies the Lagrange interpolating conditions H2n+1 (xi) = yi , i = 0, 1, . . . , n, for any polynomial Hn (x). Therefore, to determine H2n+1 (x), we look for a polynomial Hn (x) such that 0 H2n+1 (xi) = yi0 , i = 0, 1, . . . , n.

By differentiation of the formula (2.17), we have 0 0 (x)Hn (x) + wn+1 (x)Hn0 (x). H2n+1 (x) = L0n (x) + wn+1

Hence 0 yi0 = L0n (xi ) + wn+1 (xi )Hn (xi ),

Natural and Generalized Interpolating Polynomials

Hn (xi ) =

Numerical Analysis with Mathematica 51

yi0 − L0n (xi) . 0 wn+1 (xi)

and, we get Hn (x) =

n X yi0 − L0n (xi) i=0

wn0 (xi )

li (x),

where li (x), i = 0, 1, ..., n, are Lagrange polynomials. Finally, combining the above equalities, we obtain the following Hermite interpolating polynomial: H2n+1 (x) = Ln (x) + wn+1 (x)

n X yi0 − L0n (xi) i=0

2.6

0 wn+1 (xi)

li (x). (2.18)

Relationship Between a Function and its Hermite Interpolating Polynomial.

Let Hm (x) be Hermite interpolating polynomial to a given function f(x), i.e. (si −1) Hm (xi ) = f (si −1) (xi ), i = 0, 1, . . . , n, s0 +s1 +· · ·+sn = m+1.

Then, the following theorem holds: Theorem 2.3 If the function f(x) is (m + 1) times continuously differentiable on a closed interval [a, b], then there exists a point ξx ∈ (a, b) such that f(x) = Hm (x) +

f (m+1) (ξx ) Ωm+1 (x), x ∈ [a, b], (m + 1)!

where Ωm+1 (x) = (x − x0)s0 (x − x1 )s1 · · · (x − xn )sn . Proof. Let us consider the auxiliary function g(t) = f(t) − Hm (t) − KΩm+1 (t),

52 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

of the variable t ∈ [a, b] for fixed x 6= xi , i = 0, 1, . . . , n, where f(x) − Hm (x) . K= Ωm+1 (x) The function g(t) has zeros at the points x0, x1, . . . , xn ; of multiplicity s0, s1 , . . . , sn , respectively. Also, g(x) = 0. Therefore, g(t) has the total number of zeros in the interval [a, b] equal s0 + s1 + · · · + sn + 1 = m + 2.

By the Rolle’s theorem, the first derivative g 0 (t) possesses m + 1 zeros in the interval (a, b), the second derivative g 00(t) has m zeros in the interval (a, b), and finally, the derivative g (m+1) (t) has one zero in the interval (a, b). Therefore, there exists a point ξx ∈ (a, b) such that g (m+1) (ξx ) = 0.

On the other hand g (m+1) (ξx ) = f (m+1) (ξx ) − K(m + 1)! and

f(x) − Hm (x) (m + 1)! = 0. Ωm+1 (x) Hence, we obtain the formula f (m+1) (ξx ) −

f(x) = Hm (x) +

f (m+1) (ξx ) Ωm+1 (x), x ∈ [a, b]. (m + 1)!

Clearly, from the theorem, the error of Hermite interpolation f(x) − Hm (x) =

f (m+1) (ξx ) Ωm+1 (x), x ∈ [a, b]. (m + 1)!

satisfies the inequality |f(x) − Hm (x)| ≤

M (m+1) |Ωm+1 (x)|, x ∈ [a, b], (2.19) (m + 1)!

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 53

where M (m+1) = max |f (m+1) (x)|. a≤x≤b

In the case when s0 = s1 = · · · = sn = 2, by (2.13), we arrive at the following error estimate |f(x) − H2n+1 (x)| ≤

M (2n+2) n!n! 2n+2 h , 16(2n + 2)!

x ∈ [a, b], (2.20)

Example 2.8 Find Hermite interpolating polynomial H5 (x) for the function f(x) = ex which satisfies the following conditions H5 (xi) = exi , i = 0, 1, 2, 0

H5 (x) = exi ,

i = 0, 1, 2,

where x0 = 0, x1 = 1, x2 = 2. Estimate the error of interpolation when x ∈ [0, 2]. Solution. In order to find Hermite interpolating polynomial H5 (x) for the function f(x) = ex, we shall determine the Lagrange interpolating polynomial P2 (x) spaned by the points 0, 1 and 2. Thus, we have (x − 1)(x − 2) 1 (0 − 1)(0 − 2) x(x − 2) + 2.7183 (1 − 0)(1 − 2) x(x − 1) + 7.3891 = 1.47625x2 + 0.24205x + 1, (2 − 0)(2 − 1)

L2 (x) =

and L02 (x) = 2.9525x + 0.24205. Next, we have w3 (x) = x(x−1)(x−2) = x3−3x2 +2x and w30 (x) = 3x2−6x+2.

54 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence, by formula (2.18), we obtain the following Hermite polynomial y 0 − L0 (0) y 0 − L0 (1) H5 (x) = L2 (x) + w3 (x)[ 0 0 2 l0 (x) + 1 0 2 l1(x) w3 (0) w3(1) 0 0 y − L (2) + 2 0 2 l2(x)] w3(2) = 1.4765x2 + 0.24205x + 1 1 − 0.242 (x − 1)(x − 2) 2 (0 − 1)(0 − 2) 2.7183 − 3.19455 x(x − 2) + −1 (1 − 0)(1 − 2) 7.3891 − 6.14705 x(x − 1) + ] 2 (2 − 0)(2 − 1) = 0.0237209x5 + 0.00240976x4 + 0.2057x3 + (x3 − 3x2 + 2x)[

+ 0.48647x2 + x + 1. By (2.19), we get the following error estimate: e2 4 eξx max x2(x − 1)2 (x − 2)2 ≤ ≈ 0.0015. 6! 0≤x≤2 6! 27 for all x ∈ [0, 2]. We find the Hermite’s interpolating polynomial H5 (x) (cf. [2,3], [4],[5],[6], [12], [13]) by the following program:

|ex − H5 (x)| ≤

data={{{0,{1,1}},{1,{Exp[1],Exp[1]}}, {2,{Exp[2],Exp[2]}}}; H5=InterpolatingPolynomial[data,x]; N[Expand[H5],6]

2.7

Trigonometric Polynomials

Let f(x) be a continuous and periodic function with the period 2L > 0, so that f(x + 2L) = f(x)

for all real x.

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 55

Below, we shall consider the trigonometric polynomial in the following form: n X 1 kπx kπx T Rn (x) = a0 + ak cos + bk sin , 2 L L k=1

where

1ZL kπx ak = f(x)cos dx, L −L L Z kπx 1 L f(x)sin dx. bk = L −L L Now, let us consider the Riemann’s sums of the above integrals n kπxj 1 X f(xj )cos ∆x = αk , L j=−n L n 1 X kπxj bk ≈ f(xj )sin ∆x = βk , L j=−n L

ak ≈

(2.21)

where

2L 2L , xj = j 2n + 1 2n + 1 for j = −n, −n + 1, −n + 2, . . . , 0, 1, . . . , n. The following theorem holds: ∆x =

Theorem 2.4 The interpolating trigonometric polynomial T Rn (x) for the function f(x) at the point xj = ∆xj , j = −n, −n + 1, −n + 2, . . . , 0, 1, . . . , n which satisfies the conditions T Rn (xj ) = f(xj ),

j = −n, −n + 1, −n + 2, . . . , 0, 1, . . . , n,

is given by the formula n X 1 kπx kπx T Rn (x) = α0 + αk cos + βk sin , 2 L L k=1

56 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

where the coefficients αk and βk are determined by formula (2.21). Proof. We note that n 1 X T Rn (xj ) = f(xs )∆x 2L s=−n n n X 1 X kπxs kπxj + f(xs )cos cos ∆x L s=−n L L k=1 n n X 1 X kπxs kπxj + f(xs )sin sin ∆x L s=−n L L k=1 n n 1X 1 X kπxs kπxj = f(xs )∆x[ + cos cos L −n 2 k=1 L L kπxs kπxj +sin sin ] L L n n kπ 1 X 1 X cos(xs − xj ) ]. = f(xs )∆x[ + L s=−n 2 k=1 L Hence, by the following trigonometric identity n 1 X + cosµk = 2 k=1

we obtain

(

1 2

+ n if µ = 0, 0 if µ = 6 0,

1 1 f(xj )∆x( + n) = f(xj ), L 2 for j = −n, −n + 1, −n + 2, . . . , 0, 1, . . . , n. Thus, T Rn (x) is the interpolating trigonometric polynomial to the periodic function f(x) in the interval [−L, L]. The conditions for convergence of the sequence T Rn (x), n = 1, 2, . . . ; to the function f(x) are presented in the following theorem: T Rn (xj ) =

Theorem 2.5 If f(x) is a continuous function in the interval [−L, L] except at a finite number of points x0, x1 , . . . , xm at which f(x) has both left and right sides limits i.e., lim f(x) = f(xk +) and

x→xk +

lim f(x) = f(xk −).

x→xk −

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 57

then the sequence T Rn (x), n = 1, 2, . . . ; is convergent and 1 lim T Rn (x) = [f(x+) + f(x−)] 2 for all x ∈ [−L, L]. n→∞

Example 2.9 Find an interpolating trigonometric polynomial T R2(x) for the following function f(x) = x − r

for r − 1 ≤ x < r + 1, r = 0, ±2, ±4, . . . ;

Solution. Let us note that f(x) is a periodic function with the period 2L = 2. Therefore, we may consider f(x) only for x ∈ [−1, 1] setting r = 0. Then, we have n = 2, 2L = 2, ∆x = 0.4, xj = 0.4j, j = −2, −1, 0, 1, 2.

Thus

2 X 1 T R2(x) = α0 + αj cos jπx + βj sin jπx, 2 j=−2

where

2 X

α0 = α1 = α2 =

j=−2 2 X j=−2 2 X

j=−2

and β1 = β2 =

2 X

j=−2 2 X

j=−2

0.16 j = 0, 0.16 j cos 0.4πj = 0, 0.16 j cos 2 ∗ 0.4π j = 0,

0.16 j sin 0.4 π j = 0.6805206, 0.16 j sin 0.8 j π = −.42058489.

Hence, the interpolating trigonometric polynomial is T R2(x) = 0.6805206 sin πx − 0.42058489 sin 2πx.

58 Numerical Analysis with Mathematica

2.8

Krystyna STYŠ & Tadeusz STYŠ

Relationship between Trigonometric Polynomials and Fourier’s Transformation

Let us assume that f(x) is a continuous function for all real x and there exists the improper integral Z

∞

−∞

|f(x)|dx < ∞.

The trigonometric polynomial T Rn (x) for the function f(x) is: n a0 X kπx kπx T Rn (x) = + ak cos + bk sin 2 L L k=1 n a0 X ei = + ak 2 k=1

+

n X

bk

ei

kπx L

kπx L

− e−i 2i

k=1

+ e−i 2

kπx L

=

kπx L

n X

Ak ei

kπx L

,

k=−n

where −L ≤ x ≤ L, and the coefficients Ak =

1 2L

Z

L

f(t)e−i

−L

Hence n X

1 T Rn (x) = ( 2L k=−n

Z

L

−L

kπx L

f(t)e−i

dt.

kπt L

dt)ei

kπx L

Let us put sk =

kπ , L

∆k = sk+1 − sk =

π . L

Then, we have T Rn (x) =

1 2L

Z

L

(

n X

−L k=−n

f(t)e−isk t eisk x )dt.

Now, we note that T Rn (x) → f(x) and

n X

k=−n

ei(x−t)sk ∆sk →

Z

∞

−∞

ei(x−t)s ds,

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 59

when n → ∞ and L → ∞. Hence 1 f(x) = 2L

Z

∞

−∞

(

Z

∞

f(t)e−ist dt)eisx ds.

−∞

(2.22)

Hence, we arrive at the Fourier transform of f(x) Z

1 F (s) = √ 2L

∞

−∞

f(t)e−ist dt,

and, by (2.22), we get the inverse Fourier transform 1 f(x) = √ 2L

2.9

Z

∞ −∞

F (s)eistds.

Exercises

Question 2.1 The function f(x) = low . x f(x)

-2 2

-1 2.2336

√ 6 + x is tabulated be-

1 2 . 2.6446 2.8284

1. (a) Find the Newton’s interpolating polynomial and reduce it to the natural form. (b) Calculate the approximate value of f(1.5) and give an estimate of the error of interpolation for every x ∈ [−2, 2]. Question 2.2 The function f(x) = ln(6 + x) is tabulated below . x f(x)

-2 -1 1.3863 1.6094

1 2 . 1.9459 2.0794

1. (a) Find the Lagrange’s interpolating polynomial and reduce it to the natural form.

60 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

(b) Calculate the approximate value of f(1.5) and give an estimate of the error of interpolation for every x ∈ [−2, 2]. Question 2.3 Consider the following function 1 f(x) = , 0 ≤ x ≤ 1, 2+x 1. (a) Find the Hermite interpolating polynomial H3 (x) that satisfies the following interpolating conditions H3 (i) = f(i),

0

0

H3 (i) = f (i), i = 0, 1.

(b) Estimate the error of interpolation. Question 2.4 Find an optimal interpolating polynomial of degree at most two for the following functions (i)

f(x) = sin π2 x,

(ii)

f(x) = ln(1 + x),

Estimate the error of interpolation.

−1 ≤ x ≤ 1, 0 ≤ x ≤ 2.

Question 2.5 Assume that f(x) is a given function at the knots a ≤ x0 < x1 < · · · < xk ≤ b. Show that the divided difference [x0x1 ...xk] of order k can be written in the following form [x0x1 ...xk] = +

f(x0 ) (x0 − x1)(x0 − x2 ) · · · (x0 − xk ) f(x1 ) (x1 − x0)(x1 − x2 ) · · · (x1 − xk )

... ........................................................ +

f(xk ) (xk − x0 )(xk − x1) · · · (xk − xk−1 )

Natural and Generalized Interpolating Polynomials

Numerical Analysis with Mathematica 61

Hint: Use the principle of mathematical induction. Question 2.6 Let f(x) be a function (n+1) times continuously differentiable in interval [a, b]. 1. (a) Use the following formula f (x) = Ln (x) + (x − x0 )(x − x1 ) · · · (x − xn )[xx0 x1 x2 · · · xn ] large to show that there exits a point ξ ∈ (a, b) for which [xx0x1 · · · xn ] = (b) Find

f (n+1) (ξ) (n + 1)!

[xx0 x1 x2 · · · xn ] for f (x) = xn+1 .

Question 2.7 Let f(x) = xn . Show that

[x0, x1...xk ] =

              

X

xα0 0 xα1 1 ...xαk k ,

if n > k,

α0 +α1 +···+αk =n−k

1

if n = k,

0

if n < k.

Hint: Use the principle of mathematical induction. Question 2.8 Find an interpolating trigonometric polynomial T R3(x) to the following function f(x) = x − r, for r − 1 ≤ x < r + 1, r = 0, ±2, ±4, . . . ;

62 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 63-102

63

CHAPTER 3

Polynomial Splines Abstract In the chapter, the space Sm (∆, m − 1) of piecewise polynomial splines of degree m and differentiable up to the order m − 1 is introduced. In particular, the space S1 (∆, 0) of piecewise linear splines and the space S3 (∆, 2) of cubic splines are determined . Theorems on interpolation by the splines are stated and proved. The space S11(∆, 0, 0) of belinear splines and the space S33(∆, 2, 2) of be-cubic splines in two variables defined on rectangular grids are presented. On triangular grids the spaces Π1(∆) and Π3(∆) are considered. Mathematica modules have been designed for solving problems associated with application of splines. The chapter ends with a set of questions. Keywords: Polynomial splines, linear splines, cubic splines.

3.1

Space Sm(∆, k).

Polynomial splines of the class Sm (∆, k) are successfully applied in the theory of approximation of functions as well as in solving of problems which arise in the fields of differential equations and engineering. Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

64 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

In order to introduce the definition of polynomial splines of degree m, let us first define normal partition ∆ of the interval [a, b]. A partition ∆ :

a = x0 < x1 < · · · < xN = b,

is normal if there exits constant σ such that max0≤i≤N −1 (xi+1 − xi ) = σN , min0≤i≤N −1 (xi+1 − xi ) and σN ≤ σ for all natural N. Definition 3.1 A function s(x) is said to be a polynomial spline of degree m if the following conditions are satisfied: • s(x) is a polynomial of degree at most m on each subinterval [xi, xi+1 ], i = 0, 1, . . . , N − 1. • s(x) and its first m−1 derivatives are continuous functions on the interval [a, b]. The class of all polynomial splines of degree m spanned over the partition ∆ shall be denoted by the symbol Sm (∆, m−1). The Basis. Now, we shall determine a basis of the space Sm (∆, m − 1). Let us consider the following auxiliary function: (x − t)m + =

  (x − t)m 

0

if x ≤ t, if x > t.

Polynomial Splines

Numerical Analysis with Mathematica 65

r

r

r

-

xi−m−1 xi xi+m+1 t Fig. 3.1. Auxiliary function (xi − t)m + The finite difference of order m + 1 of the auxiliary function is ∆

m+1

(xi −

t)m +

= (−1)

m−1

m+1 X

(−1)

ν=0

ν

!

m+1 (xi+ν − t)m +. ν

Example 3.1 We compute the differences m = 1, ∆2 (xi − t)+ = (xi − t)+ − 2(xi+1 − t)+ + (xi+2 − t)+ m = 2, ∆3 (xi − t)2+ = −(xi − t)2+ + 3(xi+1 − t)2+ − 3(xi+2 − t)2+ − (xi+3 − t)2+ m = 3, ∆4 (xi − t)3+ = (xi − t)3+ − 3(xi+1 − t)3+ + 6(xi+2 − t)3+ − 3(xi+3 − t)3+ + (xi+4 − t)3+ Assuming that ∆ is a uniform partition of interval [a, b], so that b−a xi = a + ih, i = 0, 1, ..., N; h= . N we find the difference ∆m+1 (xi − t)m + = 0

for t ≥ xi .

66 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Indeed, the function m (xi+ν −t)m + = (xi+ν −t) , for t ≥ xi+ν , ν = 0, 1, ..., m+1

is the polynomial of degree m, therefore its difference of order m + 1 is equal to zero. Also, we note that ∆m+1 (xi − t)m + = 0

t ≤ xi ,

for

Because (xi+ν − t)m+1 = 0, +

t ≤ xi

for ν = 0, 1, .., m + 1. Hence, we have

∆m+1 (xi − t)m + =

(3.1)

 ! m+1  X m + 1    (xi+ν − t)m (−1)ν+m+1  + , xi+q−1 < t < xi+q ,   ν  ν=0

q = 1, 2, ..., m + 1,

        0,

It is clear, by (3.1), that the functions Km (xi , t) = ∆m+1 (xi − t)m +,

t ≤ xi or t > xi+m+1

i = 0, 1, ..., N,

(3.2)

are polynomials of degree m on each subinterval [xi, xi+1 ], and they are m − 1 times continuously differentiable in the interval [a, b], i.e., Km (xi , t) ∈ C m−1 [a, b]. Therefore, each Km (xi , t) is a polynomial spline of the class Sm (∆, m − 1). Obviously, Km (xi , t) can be considered on the whole real line with infinite number of knots xi, i = 0, ±1, ±2, ...; However, only N +m of them are not identically equal to zero on the interval [a, b]. These non-zero and linearly independent splines are: Km (x−m , t), Km (x−m+1 , t), Km (x−m+2 , t), · · · , Km (xN −1 , t).

Polynomial Splines

Numerical Analysis with Mathematica 67

Example 3.2 Let m = 1, then, by (3.2), (see Fig. 3.1), we obtain  0,     

K1 (i, t) =

    

6K1 (i, t)

t ≤ i, or t ≥ i + 2,

−i + t,

i ≤ t ≤ i + 1,

i − t + 2, i + 1 ≤ t ≤ i + 2. @ @

q

@ @

q

@q

-t

i i+1 i+2 Fig 3.2. Linear spline K1 (i, t)

0

For the uniform partion of the interval [a, b], we shall consider normalized splines 1 Km (xi−m+1 , x), hm

i = 1, 2, ..., N + 3,

a ≤ x ≤ b,

as a basis of the space Sm (∆, m − 1) of the dimension dim(Sm (∆, m − 1)) = N + m.

3.2

Properties of Splines.

Minimum property. Below, we shall consider the space Sm (∆, m − 1), when m = 2q + 1 is an odd positive integer. Splines from this space minimize the following functional F (g) =

Z

a

b

[g (q+1)(x)]2dx,

g ∈ C (q+1)[a, b].

Precisely, let us consider the following variational problem (cf. [1,2,7,9, 22,23,25,26]): Variational Problem. Find a function s ∈ C (q+1)[a, b] at

68 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

which the functional F (g) attains its minimum under the interpolation conditions: g(xi ) = f(xi ),

i = 0, 1, ..., N,

and either g (j)(a) = g (j)(b) = 0,

j = q + 1, q + 2, ..., 2q,

(3.3)

or g (j)(a) = g (j)(b) = 0,

j = 1, 2, ..., q,

for a given function f(x), x ∈ [a, b]. 1 The following theorem holds: Theorem 3.1 There exists a unique spline s ∈ Sm (∆, m − 1) which satisfies the interpolation conditions (3.3). Proof. Every spline s ∈ Sm (∆, m − 1) can be written as the following linear combination: s(x) = a−m Km (x−m , x) + a−m+1 K−m+1 (xm+1 , x) + · · · + aN −1Km (xN −1, x). By the interpolation conditions, N + m coefficients a−m , a−m+1 , ..., aN −1 have to satisfy the following system of N + m linear equations: s(xi ) = f(xi ), i = 0, 1, ..., N s(j)(a) = s(j)(b) = 0,

j = q + 1, q + 2, ..., 2q, m = 2q + 1.

Clearly, there exists a unique interpolating spline s ∈ Sm (∆, m − 1) if the corresponding system of homogeneous equations possesses only trivial solution. To show this, we assume that 1 The conditions put on the derivatives can be replaced by periodicity conditions.

Polynomial Splines

Numerical Analysis with Mathematica 69

f(xi ) = 0, for i = 0, 1, ..., N. Then, we have F (s) =

Z

b

a

=−

[s(q+1)(x)]2dx

Z

b

a

s(q)(x)s(q+2)(x)dx

................................................ = (−1) (−1)q −

Z

q

Z

s0 (x)s(2q+1)(x)dx

a

N −1 X

i=0 xi+1

xi

b

{s(xi+1 )s(2k+1)(xi+1 ) − s(xi )s(2k+1)(xi )

s(x)s(2q+2)(x)dx}.

Hence, F (s) = 0, since s(xi ) = 0, for i = 0, 1, ..., N and s(2q+2)(x) = 0, for xi ≤ x ≤ xi+1 . Thus, s(q+1) (x) = 0 in the interval [a, b]. Therefore, s(x) is a polynomial of degree at most q which has at least N + 1, (q ≤ N) roots in the interval [a, b]. Then, s(x) = 0, for x ∈ [a, b] and a−m = a−m+1 = ... = aN −1 = 0. This means that s(x) is a unique interpolating spline in Sm (∆, m − 1). The following theorem holds: Theorem 3.2 There exists a unique solution s ∈ Sm (∆, m− 1) of the variational problem. Proof. Let s ∈ Sm (∆, m − 1) be the unique interpolating spline which satisfies the conditions (3.3). Then, we note that Z

a

b

[g (q+1)]2dx =

Z

+2

b

a

Z

[s(q+1)]2 dx +

a

b

Z

b

a

[g (q+1) − s(q+1) (x)]2dx

s(q+1) (x)(g (q+1)(x) − s(q+1)(x))dx.

70 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Integrating by parts under the conditions (3.3), we find Z

b

a

=

s(q+1) (x)[g (q+1)(x) − s(q+1) (x)]dx

N −1 Z xi+1 X xi

i=0

s(q+1) (x)[g (q+1)(x) − s(q+1) (x)]dx =

..................................................................... = (−1)

q

N −1 Z xi+1 X xi

i=0

= (−1)q+1

N −1 Z xi+1 X xi

i=0

0

0

s(2q+1)(x)[g (x) − s (x)]dx s(2q+2)(x)[g(x) − s(x)]dx.

Recalling the equality s(2k+2)(x) = 0 for xi ≤ x ≤ xi+1,

i = 0, 1, ..., N,

we find Z

b a

s(q+1) (x)(g (q+1)(x) − s(q+1)(x))dx = 0.

and F (g) = −

Z

b

a b

Z

a

[g (q+1)(x)]2 =

Z

b

a

[s(q+1)(x)]2dx

[g (q+1)(x) − s(q+1)(x)]2dx

Hence, the functional F (g) attains minimum at g(x) = s(x), x ∈ [a, b]. In order to prove that there is a unique spline which minimizes the functional F (g), let us assume for contrary that there are at least two such splines s1 (x) and s2 (x). Then, the difference s(x) = s1 (x) − s2 (x) is the spline which satisfies homogeneous interpolation conditions. Therefore, s(x) ≡ 0, x ∈ [a, b].

Polynomial Splines

Numerical Analysis with Mathematica 71

Example 3.3 Let us consider the space S3(∆, 2) of cubic splines. Then, the interpolation conditions (3.3) take the following form: g(xi ) = f(xi ), i = 0, 1, ..., N,

00

00

g (a) = g (b) = 0. (3.4)

By theorems (4.2) and (4.3), there exists a unique cubic spline s ∈ S3 (∆, 2) at which the functional F (g) =

Z

b

a

00

[g (x)]2dx,

g ∈ C (2)[a, b],

attains minimum under the interpolation conditions (3.4), i.e. F (s) ≤

3.3

Z

b

a

00

[g (x)]2dx, for all g ∈ C (2)[a, b].

Examples of Polynomial Splines

Space S1 (∆, 0) of Piecewise Linear Splines. Elements of the space S1 (∆, 0) (see Fig. 3.2) are piecewise linear splines of the following form: s(x) = a0ψ0(x) + a1ψ1 (x) + · · · + aN ψN (x),

where the basis splines  x − xi−1   if xi−1 ≤ x < xi      xi − xi−1 xi+1 − x ψi (x) = if xi ≤ x < xi+1    xi+1 − xi     0 ifx < xi−1 or x ≥ xi+1

(3.5)

For the uniform distribution of the points xi = ih, i = 0, ±1, ±2 . . . , the piecewise linear splines ψi(x) are given by the following formulae:  x − xi−1 if      1 xi+1 − x if ψi (x) = h    

0

xi−1 ≤ x < xi xi ≤ x < xi+1

ifx < xi−1 or x ≥ xi+1

(3.6)

72 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Let us note that there are N + 1 piecewise linear splines not identically equal to zero on the interval [a, b]. Thus, the space of piecewise linear splines S1 (∆, 0) = span{ψ0 , ψ1, . . . , ψN }

has dimension N + 1. Now, we observe that every piecewise linear spline s(x) can be written as the following linear combination: s(x) = a0ψ(x) + a1 ψ1(x) + · · · + aN ψN (x).

In particular, we find the Lagrange’s interpolating piecewise linear spline s(x) = f(x0 )ψ0(x)+f(x1 )ψ1(x)+f(x2)ψ2 (x)+· · ·+f(xN )ψN (x),

to a given function f(x) which satisfies the following conditions of interpolation: s(xi) = f(xi ),

i = 0, 1, ..., N.

The following theorem holds: Theorem 3.3 If f is a function twice continuously differentiable on the interval [a, b], then the error of interpolation f(x) − s(x) satisfies the inequality: h2 (2) M , x ∈ [a, b]. 8 Proof. Let xi ≤ x ≤ xi+1 . Then, we have | f(x) − s(x) |≤

f 00(ξx ) f(x) − s(x) = (x − xi )(x − xi+1 ). 2!

Since we get where

| (x − xi )(x − xi+1 ) |≤ | f(x) − s(x) |≤

h2 , 4

h2 (2) M , 8

M (2) = max | f 00 (x) | . a≤x≤b

Polynomial Splines

Numerical Analysis with Mathematica 73

Example 3.4 Approximate the function √ f(x) = 1 + x, 0 ≤ x ≤ 2 by a piecewise linear spline with accuracy  = 0, 01. Solution. We shall start by determining h. The error of interpolation of a function f(x) by a piecewise linear spline s(x) satisfies the following inequality: | f(x) − s(x) |≤

h2 (2) M , 8

a ≤ x ≤ b.

Since 1 1 = , M (2) = max | f 00(x) |= max q a≤x≤b a≤x≤b 4 4 (1 + x)3

we may choose h such that

h2 (2) h2 M = ≤  = 0.01. 8 32 Hence, we find h = 0.5. For h = 0.5, N = 4, x0 = 0, x1 = 0.5, x2 = 1; x3 = 1.5, and x4 = 2, the piecewise linear spline is given by the following formula √ s(x) = (2x( 1.5 − 1) + 1)θ0 (x) √ √ √ √ +(2x( 2 − 1.5) + 2 1.5 − 2)θ1(x) √ √ √ √ +(2x( 2.5 − 2) + 3 2 − 2 2.5)θ2(x) √ √ √ √ +(2x( 3 − 2.5) + 4 2.5 − 3 3)θ3 =

=

 0.4495x + 1          0.3789x + 1.0353

   0.3339x + 1.0804      

if 0 ≤ x ≤ 0.5, if 0.5 ≤ x ≤ 1, if 1 ≤ x ≤ 1.5,

0.3018x + 1.1284 if 1.5 ≤ x ≤ 2.

74 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Now, let us solve this example using the following Mathematica module Program 3.1 Mathematica module that finds linear spline for a given data table. linearSpline[f_,a_,b_,n_,xstep_]:=Module[{h,sol}, h=(b-a)/n; onex[x_]:=Module[{xr,r}, xr=Table[a+r*h,{r,0,n+1}]; k=Floor[(x-a)/h+1]; N[f[xr[[k]]]+(f[xr[[k+1]]] -f[xr[[k]]])*(x-xr[[k]])/h] ]; Print[" Linear spline approximating f(x) "]; Print[" -----------------------"]; linear=Table[{N[t],onex[t],N[f[t]]}, {t,a,b,xstep}]; TableForm[PrependTo[linear,{" x ", " linear ",f[x]}]] ] By executing the following instructions f[x_]:=Sqrt[1+x]; linearSpline[f,0,2,4,0.2]; we obtain the following table: linearSpline[f,0,2,4,0.2] Linear spline approximating f(x) ----------------------------------Out[3]/TableForm= x linear Sqrt[1+x] 0 0.2

1. 1.0899

1. 1.09545

Polynomial Splines

0.4 0.6 0.8 1. 1.2 1.4 1.6 1.8 2.

Numerical Analysis with Mathematica 75

1.1798 1.26264 1.33843 1.41421 1.48098 1.54775 1.61132 1.67169 1.73205

1.18322 1.26491 1.34154 1.41421 1.48324 1.54919 1.61245 1.67322 1.73205

Space S3 (∆, 2) of Cubic Splines. As we know, a function f can be approximated by a piecewise linear spline with the accuracy O(h2 ). More accurate approximation of a smooth function can be found in the space S3 (∆, 2) of cubic splines. To determine a base of the space S3 (∆, 2), we start with the auxiliary function

K3 (xi, t) = ∆4(xi − t)3+ , xi = ih, i = 0. ± 1 ± 2, . . . , where (xi −

t)3+

=

(

(xi − t)3 if x ≤ t, 0 if x > t.

We can consider the following cubic splines as the basis of the space S3 (∆, 2) when the points xi , i = 0, ±, 1, ±2, ..., are uniformly distributed.

Bi (x) =

1 K3 (xi−2 , x), i = −1, 0, 1, 2, . . . , N + 1. h3

76 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

The explicit form of the cubic splines Bi (x), i = 0, ±1, ±2, ..., is given below:

Bi (x) =

                               

0 if x ≤ xi−2 (x − xi−2 )3 if xi−2 ≤ x ≤ xi−1 , h3 + 3h2 (x − xi−1 ) + 3h(x − xi−1 )2 − 3(x − xi−1 )3 if xi−1 ≤ x ≤ xi ,

1 h3   h3 + 3h2 (xi+1 − x) + 3h(xi+1 − x)2 − 3(xi+1 − x)3        if xi ≤ x ≤ xi+1 ,       (xi+2 − x)3      if xi+1 ≤ x ≤ xi+2 ,         0   

if x ≥ xi+2 ,

(3.7) for i = 0, ±1, ±2, . . . . x ∈ (−∞, ∞). Now, let us note that the splines Bi (x) = 0 for x ≤ xi−2 or x ≥ xi+2 , so that the only non-zero cubic splines in the interval [a, b] are Bi (x) for i = −1, 0, 1, . . . , N +1, N +2, . Therefore, any cubic spline s(x) can be represented on the interval [a, b] as a linear combination of cubic splines Bi (x), i = −1, 0, 1, ..., N + 1 i.e., s(x) = a−1 B−1 (x)+a0B0 (x)+· · ·+aN BN (x)+aN +1BN +1(x) for x ∈ [a, b]. x xi−2 xi−1 xi xi+1 xi+2 Bi (x) 0 1 4 1 0 3 3 Bi0(x) 0 0 − 0 h h 6 12 6 Bi00(x) 0 − 0 h2 h2 h2

Polynomial Splines

Numerical Analysis with Mathematica 77

r 4 6

r

xi−2

1r

r1

q

q

r -

xi+1

xi+2

xi−1

xi

Fig. 3.3 Bi (x) It can be proved that the cubic splines B−1 (x), B0(x), . . . , BN +1 (x) create the complete basis of the space S3 (∆, 2), (see Fig. 3.3)) so that S3 (∆, 2) = span{B−1 , B0, . . . , BN +1}. In order to find an interpolating cubic spline s(x) in the space S3 (∆, 2), we can use the following conditions: 0 s (x0) = f 0 (x0), s(xi ) = f(xi ), i = 0, 1, . . . , N, 0

s (xN ) = f 0 (xN ), where f(x) is a given function on the interval [a, b]. There exists only one interpolating cubic spline s(x) which satisfies the above conditions. From these conditions, we get the following linear system of equations 0 0 0 s0 (x0 ) = a−1 B−1 (x0 ) + a0 B00 (x0 ) + · · · + aN +1 BN +1 (x0 ) = f (x0 ) s(x0 ) = a−1 B−1 (x0 ) + a0 B0 (x0 ) + · · · + aN +1 BN +1 (x0 ) = f (x0 ) s(x1 ) = a−1 B−1 (x1 ) + a0 B0 (x1 ) + · · · + aN +1 BN +1 (x1 ) = f (x1 ) ···························································· ···························································· s(xN ) = a−1 B−1 (xN ) + a0 B0 (xN ) + · · · + aN +1 BN +1 (xN ) = f (xN ) s0 (xN ) = a−1 B−1 (xN ) + a0 B0 (xN ) + · · · + aN +1 BN +1 (xN ) = f 0 (xN ) (3.8)

where a−1, a0 , a1, . . . , aN +1 are unknown coefficients of the cubic spline s(x).

78 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

The system of equations (3.8) has a the matrix  3 3   − 0 0 ···    h h     1 4 1 0 ···    0 1 4 1 ··· M=  · · · · · · · · · · · · ···     0 0 0 ···   0      0

0

0

0

unique solution, since 0

0 0 ··· 1 3 ··· − h

0

0

0 0 ··· 4

0 0 ··· 1 3 h

0

is non-singular. The following theorem holds (cf. [1, 2, 9]):

                        

Theorem 3.4 If s(x) is interpolating cubic spline to a function f(x) four times continuously differentiable on the interval [a, b], then the error of interpolation satisfies the following inequality: || f (r) − s(r) ||∞ ≤ r h4−r ||f (4)||∞ ,

r = 0, 1, 2, 3;

where √ 5 1 1 1 , 1 = (9+ 3) 2 = (3σ+1), 3 = (σ 2+1), 384 216 12 2 h = max (xi − xi−1), || f ||∞ = inf sup | f(x) |,

0 =

1≤i≤N

µ(Ω)=0 a≤x≤b

µ(Ω) is the measure of the set Ω, and σ is the constant which defines the normal partition of the interval [a, b], and σ = 1 for the uniform partition of the interval [a, b] Now, let us determine the coefficients a−1 , a0, a1, ..., aN +1 of the cubic spline s(x). Using the table, we can reduce the system of equations (3.8). Namely, from the first equation, we find h 0 a−1 = a1 − f (x0 ), 3 and from the last equation, we find h 0 aN +1 = aN −1 + f (xN ). 3

Polynomial Splines

Numerical Analysis with Mathematica 79

Then, the other N + 1, equations, we write in the following form: 1 h 2a0 + a1 = [f(x0) + f 0 (x0 )] 2 3 ai−1 + 4ai + ai+1 = f(xi ),

i = 1, 2, ..., N − 1,

h 1 aN −1 + 2aN = [f(xN ) − f 0 (xN )] 2 3 In order to solve this system of equations with tri-diagonal matrix   2     1     0

1 4 1 M=  ··· ···     0 0     0 0

0 1 4 ··· 0 0

0 0 1 ··· 0 0

··· ··· ··· ··· ··· ···

0 0 0 ··· 1 0

0 0 0 ··· 4 0

0 0 0 ··· 1 2

                  

(N +1)(N +1)

we apply Gauss elimination method for the vector F = {F0, F1, ..., FN } in right sides:

Fi =

 h 0 1    (f(x ) + f (x0 ), 0   2 3   

f(xi ),

i = 0, i = 1, 2, ..., N − 1,

     1 h 0    (f(xN ) − f (xN ), i = N

2

3

80 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Then, we apply the following algorithm: 1 α0 = , 2 f or i = 1, 2, ..., N − 1, 1 , 4 − αi−1 1 , αN = 2 − αN −1 aN = βN ,

αi =

1 β0 = F0 , 2 Fi − βi−1 , 4 − αi−1 FN − βN −1 βN = , 2 − αN −1

βi =

f or i = N − 1, N − 2, ..., 1, 0, ai = βi − αi ai+1, h 0 a−1 = a1 − f0 , 3

h 0 aN +1 = aN −1 + fN . 3

Below, we give the Mathematica module based on the above algorithm which produces the table of the cubic spline for a give f(x) in the interval [a, b]. How to invoke the module, we shall explain in the example. Program 3.2 Mathematica module that finds a cubic spline in the form of a table for given data table. cubicSpline[f_,a_,b_,n_,tstep_]:=Module[{h,sol,sp3, onet,cub}, h=(b-a)/n; sol=sp3[h]; sp3[h_]:=Module[{xi,f1a,f1b,fx,al,be,sa,sb,s}, xi=Table[a+i*h,{i,0,n}]; fx=N[Map[f,xi]]; df[x_]:=D[f[x],x]; f1a=N[df[x]/.x->a];

Polynomial Splines

Numerical Analysis with Mathematica 81

f1b=N[df[x]/.x->b]; fx[[1]]=(fx[[1]]+h*f1a/3)/2; fx[[n+1]]=(fx[[n+1]]-h*f1b/3)/2; al[1]=1/2; al[i_]:=al[i]=If[ia]; f1b=N[df[x]/.x->b];

Polynomial Splines

Numerical Analysis with Mathematica 85

fx[[1]]=(fx[[1]]+h*f1a/3)/2; fx[[n+1]]=(fx[[n+1]]-h*f1b/3)/2; al[1]=1/2; al[i_]:=al[i]=If[i v0 > 0, h = max max{lT1 , lT2 , lT3 }. T ∈∆

3.7

T ∈∆

Exercises

Question 3.1 Approximate the function f(x) = ln(1 + x), 0 ≤ x ≤ 2, by a piecewise linear spline with accuracy  = 0.01. Find an approximate value f(1.3). Question 3.2 Approximate the function √ f(x) = 1 + x, 0 ≤ x ≤ 2, and its first, second and third derivatives by a cubic spline with accuracy  = 0.01. Question 3.3 Give a basis of S2(∆, 1) space. Use this basis to determine a quadratic spline that approximates a function f(x).

Polynomial Splines

Numerical Analysis with Mathematica 101

Question 3.4 Write a module in Mathematica that tabulates a quadratic spline s ∈ S2(∆, 1) approximating the function f(x). Question 3.5 Give a basis of the space S5(∆, 4) of quintic splines. Use this basis to determine a quintic spline approximating a function f(x). Question 3.6 Write a module in Mathematica that tabulates a quintic spline s ∈ S5 (∆, 4) approximating the function f(x). Question 3.7 Let s ∈ S3 (∆, 2) be a cubic spline. Following the proof of theorem 4.3, show that s minimizes the functional Z b 00 F (g) = [g (x)]2dx, g ∈ C 2 [a, b], a

under the interpolation conditions:

0

0

g(xi ) = f(xi ), i = 0, 1, ..., N, g (a) = s (b) = 0.

102 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 103-132

103

CHAPTER 4

Uniform Approximation Abstract In the subject of uniform approximations the fundamental theorems with their proofs are presented. Like Taylor and Weierstrass Theorems, Bernstein’s and Chebyshev’s polynomials, Chebyshev’s series and the best uniform approximation of a function are considered . The theorems are clarified by examples. The Mathematica modules are designed for uniform approximation of one variable functions. The chapter ends with a set of questions. Keywords: Uniform approximation, Bernstein’s polynomials, Chebyshe’s polynomials.

4.1

Taylor Polynomials and Taylor Theorem

Let f be a function n-times continuously differentiable in the interval [a, b]. Then, the polynomial f 00 (x0) f 0 (x0) (x − x0) + (x − x0)2 1! 2! f 000(x0 ) f (n) (x0) + (x − x0 )3 + · · · + (x − x0)n , 3! n!

T Ln (x) = f(x0 ) +

Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

104 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

is called Taylor polynomial of degree n of the function f about the point x0.1 and the numbers f (k) (x0 ) , n!

k = 0, 1, . . . , n,

are called Taylor coefficients of f. The relationship between a function and its Taylor polynomial is given in the following Taylor’s theorem: Theorem 4.1 If f is a function (n + 1) times continuously differentiable in the closed interval [a, b], then there exists a point ξx ∈ (a, b) such that f 0 (x0) f 00 (x0) (x − x0 ) + (x − x0 )2 1! 2! f (n) f 000(x0) (x − x0)3 + · + + (x − x0)n + Rn+1 (ξx ), + 3! n!

f(x = f(x0 ) +

for all x0 , x ∈ [a, b], where the remainder R(n+1) (ξx ) can be written in the following forms: (a) The Lagrange’s form: Rn+1 (ξx ) =

f (n+1) (ξx ) (x − x0 )(n+1) . (n + 1)!

where ξx is between x and x0 . (b) The Cauchy’s form: Rn+1 (ξx ) =

f (n+1) (x0) (x − x0)(x − ξx )n . n!

(c) The Integral form: Rn+1 (ξx ) = 1 Taylor

f.

Z

x

x0

(x − t)n (n+1) f (t)dt. n!

polynomial about x0 = 0 of f is referred as Maclaurin’s polynomial of

Uniform Approximation

Numerical Analysis with Mathematica 105

Proof. We shall prove Taylor theorem with the remainder Rn+1 (ξx ) given in the Lagrange’s form (a). Let us consider the following auxiliary function: g(t) = f(x) − f(t) − −

x−t 0 (x − t)2 00 f (t) − f (t) · · · 1! 2!

(x − t)n (n) f (t), n!

(4.1)

for x, t ∈ [a, b]. Obviously, the derivative g 0 (t) exists and x − t 00 x − t 00 f (t) + f (t) · · · 1! 1! (x − t)n (n+1) (x − t)n (n+1) − f (t) = − f (t). n! n!

g 0 (t) = −f 0 (t) + f 0 (t) −

Now, let us consider another auxiliary function G(t) = g(t) −

g(x0 ) (x − t)k+1 , k+1 (x − x0)

where t is between x0 and x. This function satisfies Rolle’s theorem for any integer k, since we have G(x0 ) = G(x) = 0, and G0 (t) exists in the open interval (a, b). By the Rolle’s theorem, there exists a point ξx such that G0 (ξx ) = 0. On the other hand G0 (t) = g 0 (t) +

g(x0 ) (k + 1)(x − t)k . (x − x0)k+1

Thus, for k = n and t = ξx , we have −

(x − ξx )n (n+1) f (ξx ) + g(x0)(n+ 1)(x − x0)n+1 (x − ξx)n = 0. n!

106 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence

f (n+1) (ξ) (x − x0 )n+1 . (n + 1)! and by (4.1), we obtain the Taylor formula g(x0 ) =

f(x) = f(x0) +

f 0 (x0 ) f 00(x0 ) (x − x0 ) + (x − x0 )2 + · · · 1! 2!

f (n) (x − x0)n + Rn+1 (ξx )(x − x0)n+1 , n! where the remainder +

Rn+1 (ξx ) =

f (n+1) (ξx ) (x − x0)n+1 . (n + 1)!

Example 4.1 Find Taylor polynomial and determine the remainder in the Lagrange’s form for the function f(x) = ex , −∞ < x < ∞, when x0 = 0. Solution. In order to determine the Taylor polynomial f 00(x0 ) f 0 (x0) (x − x0 ) + (x − x0 )2 + · · · T Ln (x) = f(x0) + 1! 2! (n) f + (x − x0)n , n! we shall find the Taylor coefficients of ex. Clearly f (n) (x) = ex

for all n = 0, 1, 2, . . . .

Thus, Taylor polynomial x x2 x3 xn + + + ··· + , 1! 2! 3! n! where the Lagrange’s remainder T Ln (x) = 1 +

Rn+1 (ξx ) =

f (n+1) (ξx ) n+1 x . (n + 1)!

One can obtain the Taylor’s polynomial by executing the following instruction in Mathematica

Uniform Approximation

Numerical Analysis with Mathematica 107

Series[Exp[x],{x,0,n}] for given n = 0, 1, , , , Example 4.2 Find Taylor polynomial for the function f(x) = ln(1 + x), 0 ≤ x ≤ 1, about x0 = 0. How many terms of the Taylor polynomial are required to approximate the function ln(1 + x), 0 ≤ x ≤ 1, by its Taylor polynomial with accuracy  = 0.0001. Solution. In order to obtain Taylor polynomial of f(x) = ln(1 + x), we calculate f(x) = ln(1 + x), 1 , 1+x 1! f 00(x) = − , (1 + x)2 2! f 000(x) = , (1 + x)3 3! , f (4)(x) = − (1 + x)4 f 0 (x) =

f (0) = 0, f 0 (0) = 1, f 00 (0) = −1, f 000(0) = 2, f 4 (0) = 4!.

In general f (n) (x) = (−1)n−1

(n − 1)! and f n (0) = (−1)n−1 (n − 1)!, (1 + x)n

for n = 0, 1, 2, . . . . Hence, Taylor polynomial of ln(1 + x) at x0 = 0 is T Ln (x) = x −

x2 x3 x4 xn + − + · · · + (−1)n−1 , 2 3 4 n

where the Lagrange’s remainder Rn+1 (ξx ) = (−1)n

xn+1 . (n + 1)(1 + ξx )n+1

108 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

The error of approximation ln(1 + x) − T Ln (x) = (−1)n

xn+1 (n + 1)(1 + ξx )n+1

satisfies the inequality | ln(1 + x) − T Ln (x) |≤

1 , n+1

0 ≤ x ≤ 1.

The required number of terms of Taylor polynomial T Ln (x) to get the accuracy  = 0.0001 is determined by the following inequality: 1 ≤ 0.0001 n+1

or

n ≥ 9999.

We note that the Taylor’s series of the function ln(1 + x) is slowly convergent. For example, to compute ln 2 with 1 the accuracy , we need to add about [ ] terms. We can  compute this sum by the instructions N[Sum[(-1)^(n+1)/n,{n,1,9999}]] Then, we obtain ln 2 ≈ 0.693197, while direct computation from Mathematica gives Log[2]=0.693147. Example 4.3 Consider the following functions: π 1. f(x) = sin x, 0 ≤ x ≤ , 2 π 2. f(x) = cos x, 0 ≤ x ≤ . 2 (a) Find Taylor polynomial for the above functions at x0 = 0. (b) For what value of n will Taylor polynomial approximate the above functions correctly upto three decπ imal places in the interval [0, ]. 2

Uniform Approximation

Numerical Analysis with Mathematica 109

Solution (a). In order to find Taylor polynomial T Ln (x) = f(x0) +

f 0 (x0) f 00(x0 ) (x − x0 ) + (x − x0 )2 + · · · 1! 2!

f (n) (x0 ) + (x − x) )n , n! we shall determine Taylor coefficients f (k) (0) , k! We have

k = 0.1, . . . , n.

f(x) = sin x,

f(0) = 0,

f 0 (x) = cos x,

f 0 (0) = 1,

f 00 (x) = −sin x,

f 00 (0) = 0,

000

In general

and f

(n)

000

f (x) = −cos x, f (0) = 0.

 sin x     cos x f (n) (x) =  −sin x   

−cos x

(0) = (sin 0)

(n)

=

(

n = 4k, k = 0, 1, . . .; n = 4k + 1, k = 0.1, . . .; n = 4k + 2, k = 0, 1, . . .; n = 4k + 3, k = 0, 1, . . .;

(−1)k n = 2k + 1, k = 0, 1, . . .; 0 n = 2k, k = 0, 1, . . .;

Thus, Taylor polynomial for sin x is T L2n+1(x) = x −

x3 x5 x7 x2n+1 + − + · · · + (−1)n+1 , 3! 5! 7! (2n + 1)!

where the Lagrange’s remainder R2n+2 (ξx ) =

(sin ξx )(2n+2) 2n+2 x . (2n + 2)!

One can obtain the Taylor’s polynomial T L9 (x) using the instruction

110 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Normal[Series[Sin[x],{x,0,9}]] In order to get correct three decimal places, we should consider accuracy  = 0.0005, and to choose a smallest n for which the remainder Rn+1 (ξx ) satisfies the following inequality (sinξx )(2n+2) 2n+2 | x | ≤ . (2n + 2)! Obviously, the above inequality holds if 1 π ( )2n+2 ≤ 0.0005. (2n + 2)! 2 Hence n = 4, so that the Taylor polynomial T L9(x) = x −

x3 x5 x7 x9 + − + 3! 5! 7! 9!

approximates sin x in the interval [0, π2 ] with accuracy upto three decimal places. Solution of (b), (ii) For the function cos x, we find

f (n) (x) = (cos x)(n)

 cos x for n = 4k, k = 0, 1, . . . ;     −sin x for n = 4k + 1, k = 0, 1, . . . ; =  −cos x for n = 4k + 2, k = 0, 1, . . . ;   

sin x

and cos

(n)

0=

(

for n = 4k + 3, k = 0, 1, . . . ;

(−1)k for n = 2k, k = 0, 1, . . . ; 0 for n = 2k + 1, k = 0, 1, . . . ;

Thus, Taylor polynomial T L2n (x) = 1 −

x2 n x2 x4 + + · · · + (−1)n , 2! 4! 2n!

where the remainder R2n+1 (ξx ) = (−1)n+1

(cos ξx )(2n+1) 2n+1 x . (2n + 1)!

Uniform Approximation

Numerical Analysis with Mathematica 111

In order to get accuracy of three decimal places, we choose n = 4 and the polynomial x2 x4 x6 x8 + − + . 2! 4! 6! 8! Then, for n = 4, we have the following remainder estimate T L8(x) = 1 −

(cos ξx )(2n+1) | (2n + 1)! 1 π ≤ ( )(2n+1) ≤ 0.0005. (2n + 1)! 2

| R2n+1 (ξx ) | = | (−1)n+1

Also, we can obtain the Taylor’s polynomial T L8 (x) by the Mathematica instruction Normal[Series[Cos[x],{x,0,8}]]

4.2

Bernstein Polynomials and Weierstrass Theorem

In the previous section, it has been shown that an (n + 1) times continuously differentiable function f can be approximated by its Taylor polynomial in a neighbourhood of a point x0 , provided that the remainder Rn+1 (ξx ) tends to zero when n → ∞. A continuous function f (not necessarily differentiable) on the interval [a, b] can also be approximated by a polynomial pn (x) = a0 + a1 x + a2 x2 + · · · + an xn , with an arbitrary accuracy . This has been established by Weierstrass in the following theorem: Theorem 4.2 (Weierstrass Theorem.) If f is a continuous function on a finite and closed interval [a, b], then for every  > 0 there exists a polynomial pn (x) such that | f(x) − pn (x) |<  for all x ∈ [a, b]

112 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Proof. There are a few proofs of Weierstrass approximation theorem. Some of them have constructive character and they lead to an explicit form of the polynomial pn (x). Such proof due to Bernstein (1912) is presented below. Let us first consider the case when the interval [a, b] = [0, 1]. Then, we shall show that the sequence Bn (x) of Bernstein’s polynomials Bn (x)=

n X

k f( )xk (1 − x)n−k , for n = 0, 1, . . . ; n k=0

is uniformly convergent to the function f(x) on the interval [0, 1]. The following identities hold: (i)

n X

k=0 n X

Ckn xk (1 − x)n−k = 1,

k n k Ck x (1 − x)n−k = x, k=0 n n X 1 1 k2 n k Ck x (1 − x)n−k = (1 − )x2 + x, (iii) 2 n n n k=0 (ii)

(4.2)

n! is Newton’s coefficient. k!(n − k)! We can easily get the inequality (i) setting u = x and v = 1 − x in Newton’s binomial where Ckn =

n

(u + v) =

n X

Ckn uk v n−k .

k=0

Also, we can obtained identities (ii) and (iii) by differentiating Newton’s binomial once and twice, and then substituting u = x and v = 1 − x. The identities (i), (ii) and (iii) yield the following equality: (iv)

n X

k=0

Ckn (

k x(1 − x) − x)2xk (1 − x)n−k = . n n

for all x ∈ [0, 1]. Now, let us multiply both hand sides of (i) by f(x) and then

Uniform Approximation

Numerical Analysis with Mathematica 113

subtract Bn (x). Then, we have f(x) − Bn (x) =

n X

k [f(x) − f( )]Ckn xk (1 − x)n−k . n k=0

By assumption, f(x) is a continuous function on the closed interval [0, 1]. Therefore, f(x) is also a function uniformly continuous and a bounded on the interval [0, 1], i.e., for every  > 0, there exists δ > 0 such that | f(x1 ) − f(x2 ) |
0 on the ball K, i.e., G(a0 , a1, . . . , an ) ≥ µ > 0 for (a0, a1, ..., an) ∈ K. This minimum µ must be positive, since

G(a0 , a1, . . . , an ) = 0 if and only if a0 = a1 = · · · = an = 0, and the polynomial Pn (x) is not identically equal to zero. The function H(a0 , a1, · · · , an ) is non-negative and has nonnegative minimum in the whole space Rn+1 . Let m = inf H(a0, a1 , . . . , an ) ≥ 0. n+1 R

Now, let us divide the space Rn+1 into two sets as follows: S1 = {(a0, a1, . . . , an ) :

n X

a2i ≤ r2 },

S2 = {(a0, a1, . . . , an ) :

n X

a2i > r2 },

where r=

i=0

i=0

1 (1 + m + max |f(x)|) > 0. a≤x≤b µ

118 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Then, we get the inequality n X

a2i = λ2 > r2

i=0

for any (a0, a1, . . . , an) ∈ S2 , and H(a0, a1, . . . , an ) = max |f(x) − a≤x≤b

≥ max | a≤x≤b

n X

n X i=0

a i xi |

ai xi | − max |f(x)|

i=0 n X

a≤x≤b

ai i x | − max |f(x)| a≤x≤b a≤x≤b i=0 λ ≥ λµ − max |f(x)| ≥ rµ − maxa≤x≤b |f(x)| = λ max |

a≤x≤b

≥ 1+m for all (a0 , a1, . . . , an ) ∈ S2 . Hence, the function H(a0 , a1, . . . , an ) attains its minimum equal to m on the set S1 , (not on S2), i.e., there exists a point (a∗0 , a∗1, . . . , a∗n ) ∈ S1 such that H(a∗0, a∗1, . . . , a∗n ) = minS1 H(a0 , a1, . . . , an ) = inf Rn+1 H(a0, a1, . . . , an ) = m. Hence

Pn (x) = a∗0 + a∗1x + a∗2x2 + · · · + a∗n xn

is the best approximation polynomial to the function f(x) on the interval [a, b]. Proof of Uniqueness. Let us assume that there are two different polynomials Pn (x) and Qn (x) which approximate the function f on the interval [a, b] with the minimum error E(f, Pn ). Then, we have max |f(x) − Pn (x)| = max |f(x) − Qn (x)| = m.

a≤x≤b

a≤x≤b

Uniform Approximation

Numerical Analysis with Mathematica 119

Let us consider the ball B(f) = {g ∈ C0[a, b] : max |f(x) − g(x)| ≤ m} a≤x≤b

2

The polynomials Pn (x), Qn (x) and R(q) n (x) = qPn (x) + (1 − q)Qn (x),

0 ≤ q ≤ 1,

are members of the ball B(f), i.e., Pn , Qn , R(q) n ∈ B(f) Also, we have m ≤ =

max |f(x) − R(q) n (x)|

a≤x≤b

max |f(x) − qPn (x) − (1 − q)Qn (x)|

a≤x≤b

≤ q max |f(x) − Pn (x)| a≤x≤b

+ (1 − q) max |f(x) − Qn (x)| = m. a≤x≤b

Hence max |f(x) − R(q) n (x)| = m,

a≤x≤b

(4.4)

and therefore R(q) n ∈ B(f). But, for Pn (x) = 6 Q(x) and 0 < q < 1, polynomial R(q) n (x) lies within the ball B(f), i.e., max |f(x) − R(q) n (x)| < m

a≤x≤b

This inequality contradicts to (4.4). Thus, Pn (x) = Qn (x) for all x ∈ [a, b]. Chebyshev equi-oscillation theorem. In general, most algorithms to find the best approximation polynomial Pn (x) for a given continuous function f(x) on a closed interval [a, b] are based on the following Chebyshev equi-oscillation theorem: 2 C [a, b] 0

is the set of all continuous functions on the closed interval [a, b]

120 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Theorem 4.3 Let f be a continuous function on a finite and closed interval [a, b] and let Pn be a polynomial with the oscillation L = max |f(x) − Pn (x)|. a≤x≤b

Then, Pn (x) is the best approximating polynomial to f(x) on the interval [a, b] if and only if there exist at least n + 2 points a ≤ x0 < x1 < · · · < xn+1 ≤ b such that

|f(xi ) − Pn (xi)| = L

for i = 0, 1, . . . , n + 1

(4.5)

and f(xi ) − Pn (xi ) = −[f(xi+1) − Pn (xi+1 )] for i = 0, 1, . . . , n. (4.6) The points xi , i = 0, 1, . . . , n + 1 which satisfy the above conditions (4.5) and (4.6) are called Chebyshev knots. Proof. We shall first prove that (4.5) and (4.6) are necessary conditions for Pn (x) to be the best approximating polynomial to the function f(x). Necessity. Let us assume that Pn (x) is the best approximating polynomial to the function f(x) on the interval [a, b] and, on the contrary, that the conditions (4.5) and (4.6) are satisfied for certain number of Chebyshev knots s < n + 2. The number s must be greater than one i.e., s ≥ 2 . Indeed, if s = 1 then the error function E(x) = f(x) − Pn (x), x ∈ [a, b] attains its maximum at x0 i.e., E(x0) = f(x0) − Pn (x0 ) = L or E(x) attains its minimum at x0 i.e., E(x0 ) = f(x0) − Pn (x0) = −L.

Uniform Approximation

Numerical Analysis with Mathematica 121

Therefore −L ≤ E(x) ≤ L for all x ∈ [a, b]. Let E(x0) = L. As a continuous function on the closed interval [a, b], E(x) attaints its minimum at a point x1 ∈ [a, b], and for s = 1 min E(x) = E(x1) > −L.

a≤x≤b

Then, the polynomial 1 P n (x) = Pn (x) + [E(x1) + L] 2 approximates the function f(x) better than the polynomial Pn (x) does. Indeed, we have 1 E(x) = f(x) − P n (x) = E(x) − [E(x1) + L]. 2 and −L < E(x) < L

for all x ∈ [a, b].

Now, let E(x0) = −L. Then E(x) attains its maximum at x1, and for s = 1. max E(x) = E(x1 ) < L.

a≤x≤b

In this case, the polynomial 1 P n (x) = Pn (x) + [L − E(x1)] 2 approximates the function f(x) better than the polynomial Pn (x) does. Indeed, we have 1 E(x) = f(x) − P n (x) = E(x) + [L − E(x1 )], 2 and 1 1 1 1 1 −L < − L− E(x1) ≤ E(x)+ [L−E(x1)] ≤ E(x1)+ L < L. 2 2 2 2 2

122 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence −L < E(x) < L for all x ∈ [a, b].

We may illustrate the above relations, on the following figures (see Fig 4.1, Fig 4.2):

L

6

L

6

-

x0

-

x1

x0

-L

x1

-L Fig. 4.1

Fig. 4.2

Thus, if Pn (x) is the best approximating polynomial to f(x), then the number of knots s ≥ 2. Thus, if Pn (x) is the best approximating polynomial to f(x), then the number of knots s ≥ 2. Now, let assume that s ≤ n + 1. In this case, we shall also find a polynomial P n (x) which is better Chebyshev approximation to f(x) than the best polynomial Pn (x). Namely, let us divide the interval [a, b] into subintervals I1 , I2, . . . , IN each of the same length δ. Since E(x) is a uniformly continuous function on the interval [a, b], we get 0

00

|E(x ) − E(x )| ≤ max |E(x)| a≤x≤b

Uniform Approximation 0

Numerical Analysis with Mathematica 123

00

0

00

for all x , x ∈ [a, b] such that |x −x | < δ. Let J1, J2 , . . . , Js , be the subintervals (chosen from I 0s intervals) on which max |E(x)| = L, i = 1, 2, . . . , s. x∈Ji

Let us choose δ so small to be a non-empty ”gap” between two following subintervals, Ji and Ji+1 , i = 1, 2, . . . , s − 1 and such that on each subinterval Ji , i = 1, 2 . . . , s, E(x) has a constant sign (see Fig 4.3). Such choice of δ is possible, since E(x) is uniformly continuous function on interval [a, b]. L − 6− − − − − − − − − − − − − − − − − − − − − − − − x0 s .....J1........

s

q

y1 q

.....J2........ s s q x1

y2 q

x2 s .....J3........

s

q

y3

q -

-L − − − − − − − − − − − − − − − − − − − − − − − − − Fig 4.3. Error function E(x).

Now, let us choose y1 , y2, . . . , ys−1 from the ”gaps”, so that yi lies between Ji and Ji+1 , for i = 1, 2, . . . , s − 1. Under the assumption s ≤ n + 1, we shall show that the polynomial P n (x) = Pn (x) + Qs−1 (x), where Qs−1 (x) = (x − y1)(x − y2) . . . (x − ys−1 ), is better Chebyshev approximation to the function f(x) than the polynomial Pn (x), if  > 0 is sufficiently small. Indeed, we have E(x) = f(x) − P n (x) = E(x) − Qs−1 (x). The polynomial Qs−1 (x) > 0 for x ∈ J1 and Qs−1 (x) changes its sign only at yi , i = 1, 2, . . . , s − 1. If E(x) > 0 for x ∈ J1 then Qs−1 (x) and E(x) have the same signs on Ji , i =

124 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

1, 2, . . . , s. Clearly, for sufficiently small  > 0 the following inequality holds: |Qs−1 (x)| < |E(x)| for x ∈ [a, b] and max |E(x)| < max |E(x)| for i = 1, 2, . . . , s. x∈Ji

x∈Ji

Hence −L < E(x) < L

for x ∈ Ji , i = 1, 2, . . . , s.

(4.7)

To complete the proof, we must show that inequality (4.7) is satisfied for all x ∈ [a, b]. Let Ω = [a, b] − J1 ∪ J2 ∪ · · · ∪ Js . Then max |E(x)| ≤ max |E(x)| +  max |Qs (x)| Ω

Ω

Ω

and max |E(x)| < max |E(x)|. Ω

a≤x≤a

Now, let us choose  > 0 so small to be |Qs (x)| < max |E(x)| − max |E(x)| a≤x≤b

Ω

for x ∈ [a, b].

Then, by the above inequalities max |E(x)| < max |E(x)| = L. Ω

a≤x≤b

Hence, by (4.7) −L < E(x) < L for all x ∈ [a, b]. This means that the polynomial P n (x) approximates the function f(x) better than Pn (x) does, provided that s ≤ n + 1. Thus, s ≥ n + 2, if Pn (x) is the best approximating polynomial to the function f(x) on the interval [a, b]. Sufficiency. Let us assume that polynomial Pn (x) satisfies

Uniform Approximation

Numerical Analysis with Mathematica 125

conditions (4.5) and (4.6) i.e., there exist at least n + 2 points a ≤ x0 < x1 < . . . < xn+1 ≤ b such that |f(xi ) − Pn (xi )| = L,

i = 0, 1, 2, . . . , n + 1,

and f(xi ) − Pn (xi ) = −[f(xi+1) − Pn (xi+1 )],

i = 0, 1, . . . , n.

Also, for contrary, we assume that Pn (x) is not the best Chebyshev approximating to f(x) on the interval [a, b], and P n (x) is the one. Now, let us consider the following polynomial: Ψn (x) = P n (x) − Pn (x). Then, we have Ψn (x) = [f(x) − Pn (x)] − [f(x) − Pn (x)].

(4.8)

From the assumptions (4.5) and (4.6), it follows that f(xi ) − Pn (xi) = ± L,

i = 0, 1, . . . , n + 1.

P n (x) is the best Chebyshev approximation of f(x), therefore |f(x) − P n (x)| = |E(x)| = L < L,

a ≤ x ≤ b.

From the above, by (4.8), we conclude that polynomial Ψn (x) of degree at most n has a root in each subinterval (xi , xi+1 ) , i = 0, 1, . . . , n. Thus, Ψn (x) possesses at least n+1 roots. This is possible when Ψn (x) = 0 for all x ∈ [a, b]. Therefore, Pn (x) = P n (x) for all x ∈ [a, b], and Pn (x) must be the best Chebyshev approximation to f(x) on interval [a, b]. An application of the Chebyshev theorem. A special interest is paid to approximate the function f(x) = 0 for all x ∈ [a, b] by a polynomial Pn (x) with the leading coefficient at xn equal to one. We have used this approximation of a

126 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

”zero” function to determine an optimal interpolating polynomial to a function f(x). So, we have used the following thesis: Among all polynomials of degree n with the leading coefficient at xn equal to one the polynomial T n (x) =

1 2n−1

cos (n arc cos x), x ∈ [−1, 1]

is the best approximation of the ”zero” function f(x) = 0 in the interval [−1, 1]. Now, we can prove this thesis using the equi-oscillation Chebyshev theorem. Namely, let us note that the polynomial T n (x) attains its maximum modulus on the interval 1 at the following n + 1 points: [−1, 1] equal to 2n−1 xk = cos

kπ , n

k = 0, 1, . . . , n.

Indeed, the derivative

if

dT n (x) n sin (n arccos x) √ = = 0, dx 1 − x2

x ∈ (−1, 1)

n arc cos xk = kπ. Then xk = cos

kπ , n

k = 1, 2, . . . , n − 1.

Also, T n (x) attains its maximum modulus of the interval [−1, 1]. Hence, we have T n (xk ) =

1 2n−1

at the end

kπ (−1)k cos n (arccos(cos )) = 2n−1 n 2n−1 1

for k = 0, 1, . . . , n. Therefore, the function T n (x) satisfies the assumptions (4.5)

Uniform Approximation

Numerical Analysis with Mathematica 127

and (4.6) of the equi-oscillation theorem, and T n (x) is the best Chebyshev approximation of the ”zero” function on interval [−1, 1]. Example 4.4 Find the best approximation of the polynomial f(x) = xn , −1 ≤ x ≤ 1,

by a polynomial of degree at most n − 1 with the leading coefficient equal to one at xn−1 .

This kind of approximation maybe interesting when we need to minimize the number of terms in the following polynomial: f(x) = a0 + a1x + a2x2 + · · · + an xn with minimum loss of accuracy. In order to determine the best approximating polynomial Pn−1 (x) to xn , we consider the following error function E(x) = xn − Pn−1 (x),

−1 ≤ x ≤ 1.

This error function attains its minimum oscillation if xn − Pn−1 (x) = T n (x), −1 ≤ x ≤ 1. Hence Pn−1 (x) = xn − T n (x),

4.4

−1 ≤ x ≤ 1.

Chebyshev Series

We may use the Chebyshev polynomials to approximate a continuous function by a partial sum of the following Chebyshev series: ∞ a0 X a0 + ak Tk (x) = T0 (x) + a1 T1(x) + · · · + an Tn (x) + · · · 2 2 k=1

128 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

for x ∈ [−1, 1], where Tk (x) = cos (k arccos x) and the coefficients 2 ak = π

Z

1

−1

f(x)Tk (x) √ , 1 − x2

k = 0, 1, . . . ,

In fact, the above Chebyshev series reduces to Fourier’s series of cosines of the function f(cos Θ), where x = cos Θ, Θ ∈ [0, π]. Indeed, for x = cos Θ, we have dx = −sin Θ dΘ and ak

Z

2 1 f(x)Tk (x) √ = dx = π −1 1 − x2 Z 2 π f(cos Θ) cos kΘ dΘ. = π 0

On the other hand, the Fourier’s series of cosines of f(cos Θ) takes the following form: ∞ A0 X + Ak cos kΘ + Bk sin kΘ, 2 k=1

where Z

1 π f(cos Θ)cos kΘ dΘ, k = 0, 1, . . . ; π −π Z 1 π Bk = f(cos Θ)sin Θ dΘ, k = 1, 2, . . . ; π −π

Ak =

Since f(cos Θ) cos Θ is an even function and f(cos Θ) sin Θ is an odd function, we get Ak =

1 π

Z

π

−π

f(cos Θ) cos Θ dΘ =

and

Z

2 π

Z

0

π

f(cos Θ) cos kΘ dΘ = ak

1 π f(cos Θ) sin n Θ dΘ = 0. π −π So, the Chebyshev series of f(x) is the Fourier’s series of f(cos Θ), where x = cos Θ. Therefore, the Chebyshev series converges to f(x) when Bk =

Uniform Approximation

Numerical Analysis with Mathematica 129

Fourier’s series of cosines converges to f(cos Θ). Then, for continuous functions f(x) = a0T0(x)+a1T1(x)+· · ·+an Tn (x)+· · · ;

x ∈ [−1, 1].

To obtain Chebyshev’s series for a continuous function f(x) in the interval [a, b], we can proceed as follows: 1. (a) i. We apply the linear mapping b−a (t + 1) + a, t ∈ [−1, 1]. 2 to transform interval [−1, 1] on interval [a, b]. ii. Then, we find Chebyshev’s series for the function b−a (t + 1) + a), F (t) = f( 2 given in the interval [−1, 1], iii. The Chebyshev series of F (t) is: x=

F (t) =

∞ a0 X + ak Tk (t), 2 k=1

t ∈ [−1, 1],

where the coefficients Z 2 π ak = F (cosΘ)cos kΘ dΘ, k = 0, 1, . . . , π 0 iv. We get the Chebyshev’s series in terms of x , by the substitution x−a − 1, a ≤ x ≤ b, t=2 b−a so that the series is: ∞ a0 X x−a f(x) = + ak Tk (2 − 1), a ≤ x ≤ b. 2 b−a k=1 Example 4.5 Determine the partial sum s4 (f), of the Chebyshev’s series for the function 2

f(x) = e−x ,

−1 ≤ x ≤ 1.

130 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

We shall find the partial sum of the series 2

e−x =

∞ a0 X + ak Tk (x), 2 k=1

−1 ≤ x ≤ 1,

using the following Mathematica program: czebyszew1 f[x_]:=Exp[-x^2]; ak[f_,n_]:=With[{coeff=Table[ NIntegrate[f[x]*ChebyshevT[i,x]/Sqrt[1-x^2], {x,-1,1}],{i,0,n}]}, N[ReplacePart[coeff,coeff[[1]]/2,1]*2/Pi]]; czebyszew1[n_]:=ak[f,n] Table[ChebyshevT[i,x], {i,0,n}]//Expand//Chop By the instruction czebyszew1[4], we obtain s4 (f) = 0.996581 − 0.935316x2 + 0.309633x4 . Example 4.6 Determine the partial sum s5 (f) of the Chebyshev’s series for the function √ f(x) = 1 + x2, 0 ≤ x ≤ 2. Using the linear mapping x = t + 1, we transform the interval [0, 2] onto interval [−1, 1], so that, we consider the Chebyshev’s series for the function F (t) =

q

1 + (1 + t)2,

−1 ≤ t ≤ 1.

By the program czebyszew2[5], f[x_]:=Sqrt[1+(1+x)^2]; ak[f_,n_]:=With[{coeff=Table[ NIntegrate[f[Cos[x]]*Cos[i*x], {x,0,Pi}],{i,0,n}]}, N[ReplacePart[coeff,coeff[[1]]/2,1]*2/Pi]]; czebyszew2[n_]:=ak[f,n].Table[ChebyshevT[i,x], {i,0,n}]//Expand//Chop

Uniform Approximation

Numerical Analysis with Mathematica 131

we find the partial sum s5(F ) = 1.41399 + 0.707595t + 0.180681t2 − 0.0923949t3 + 0.0235995t4 + 0.00277143t5 . Coming back to the variable x, we have F (t) = F (x − 1) ≈ s5 (f) = 1.41399 + 0.707595(x − 1) + 0.180681(x − 1)2 − 0.0923949(x − 1)3 + 0.0235995(x − 1)4 + 0.00277143(x − 1)5 . Hence, after simplification s5(f) = 1.0003 − 0.0114927x + 0.571748x2 − 0.159078x3 + 0.00974232x4 + 0.00277143x5 .

4.5

Exercises

Question 4.1 Find the range of x for which 1. (a) (b)

sin x ≈ x 1 sin x ≈ x − x3 6

with accuracy  = 0.0001. Question 4.2 Find the range of x for which 1. (a) 1 cos x ≈ 1 − x2 2 (b)

1 1 cos x ≈ 1 − x2 + x4 2 24 with accuracy  = 0.0001.

132 Numerical Analysis with Mathematica

Question 4.3 Let f(x) =

√ 1 + x2 ,

Krystyna STYŠ & Tadeusz STYŠ

0 ≤ x ≤ 1.

Find a linear function y = ax + b which is the best approximate of f(x) Question 4.4 Consider the following approximation of sin x: sin x ≈ p5 (x) where

1 1 5 p5 (x) = x − x3 + x. 6 120 Express p5 (x) in terms of Chebyshev polynomials. Estimate the error sin x − p5 (x). Question 4.5 Use the Mathematica programs czebyszew1 and czebyszew2 to determine the partial sum s6(f) for the function f(x) = ArcT anx in the intervals (i) [−1, 1] and (ii) [0, 4].

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 133-156

133

CHAPTER 5

Introduction to the Least Squares Analysis Abstract This chapter is designed for the least squares method . It begins with the least squares method to determine a polynomial of degree n well fitted to m discrete points (xi , yi ), i = 1, 2, ..., m, when n ≤ m. In the simplest form, the line of regression through m points is determined. The algorithm to find well fitted function to given discrete data is also presented. The Mathematica modules are designed and example illustrating the method are provided . The chapter ends with a set of questions. Keywords: The least squares method, Regressions.

5.1

Introduction

In order to approximate the following discrete data: x0 x1 x2 x3 · · · · · · · · · xm y0 y1 y2 y3 · · · · · · · · · ym we can use interpolating polynomial for a relatively small m. For large number of points, an interpolating polynomial Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

134 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

is usually unstable and costly in terms of arithmetic operations. For large m, the least squares method is used to approximate discrete data points. By least squares method, we determine a polynomial Pn (x) = a∗0 + a∗1x + a∗2x2 + · · · + a∗n xn , for which the function D(a0 , a1, . . . , am ) = [y0 − Pn (x0)]2 + [y1 − Pn (x1)]2 + [y2 − Pn (x2)]2 + · · · + [ym − Pn (xm )]2 attains its minimum at a∗0, a∗1 , . . . , a∗n in the whole real space Rn+1 . Let us note that D(a0 , a1, . . . , an ) is a non-negative quadratic function in the whole real space Rn+1 . Therefore, this function attains its minimum if and only if ∂D(a0, a1 , a2, . . . , an ) = 0, ∂ak

k = 0, 1, . . . , n.

On the other hand ∂D(a0, a1 , a2, . . . , an ) = ∂ak = −2 = −2 = 2an

m X i=0

m X

[yi − a0 − a1 xi − a2 x2i − · · · − an xni ]xki xki yi + 2a0

i=0 m X

m X

+2a1

i=0

m X

xk+1 + 2a2 i

i=0

m X i=0

xk+2 + ··· i

xk+n = 0. i

i=0

Hence, we obtain the following system of linear equations: a0

m X i=0

xki +a1

m X i=0

xk+1 +a2 i

m X i=0

xk+2 +· · ·+an i

m X i=0

xk+n = i

m X

xki yi

i=0

(5.1)

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 135

for k = 0, 1, . . . , n. Let S0 = m + 1,

Sk =

m X

xki ,

i=0

V0 =

m X

yi ,

i=0

Vk =

m X

xki yi ,

i=0

for k = 1, 2, . . . , n. Let us write the system of linear equations (5.1) in the following form: S0a0 + S1 a1 + S2 a2 + · · · + Sn an = V0 S1a0 + S2 a2 + S3 a3 + · · · + Sn an = V1 ···································· ···································· Sn a0 + Sn+1 a1 + Sn+2 a2 + · · · + S2n a2n = Vn

(5.2)

The system of equations (5.2) has a unique solution a∗0, a∗1, a∗2 , . . . , a∗n at which the function D(a0 , a1, a2, . . . , an ) attains its minimum equal to D(a∗0 , a∗1, a∗2, . . . , a∗n ). If m = n, then D(a∗0 , a∗1, a∗2, . . . , a∗n ) = 0, and Pn (x) = a∗0 + a∗1x + a∗2x2 + · · · + a∗n xn ,

is the interpolating polynomial that satisfies the conditions Pn (xk ) = yk ,

k = 0, 1, 2, . . . , m.

In general, the number of points m > n, and then D(a∗0 , a∗1, . . . , a∗n ) > 0, so that Pn (x) is not an interpolating polynomial. In such a case, we estimate the root mean square error using the following formula: E=

s

D(a∗0 , a∗1, a∗2, . . . , a∗n ) . m+1

136 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 5.1 Approximate the following discrete data by a least squares polynomial of degree two. 0 1 2 3 4 5 1 −1 0 2 −1 0 Determine the error of approximation. Solution. By the formulas (5.1), when m = 5 and n = 2 , we find S0 = m + 1 = 6, S1 = 15, S2 = 55, S3 = 225, S4 = 979,

V0 = 1,

V1 = 1, V2 = 1.

Hence, we obtain the following system of linear equations: 6a0 + 15a1 + 55a2 = 1, 15a0 + 55a1 + 225a2 = 1, 55a0 + 225a1 + 979a2 = 1. Solving the above system of equations, we find the coefficients a∗0 = 0.3214286,

a∗1 = 0.003571429,

a∗2 = −0.01785714.

and the polynomial P2 (x) = 0.3214286 + 0.003571429x − 0.01785714x2 that approximates discrete data with the root mean square error s

D(a∗0 , a∗1, a∗2) m+1 1 = √ {(1 − P2 (0))2 + (−1 − P2 (1))2 6 +(0 − P2 (2))2 + (2 − P2 (3))2 + (−1 − P2 (4))2

E =

1

+(0 − P2 (5))2 } 2 = 1.4469.

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 137

To obtain the quadratic polynomial P2 (x), we execute the following commands (cf. [28]): data={{0,1},{1,-1},{2,0},{3,2},{4,-1},{5,0}}; Fit[data,{1,x,x^2},x]

5.2

Line of Regression.

Special attention is paid to approximate discrete data by the linear function P1 (x) = a∗0 + a∗1x. The graph of the linear function P1 (x) which is the best fitted line to discrete data in the sense of least squares method is called the line of regression. Following the idea of least squares method, we can find coefficients a0 and a1 by solving the following system of equations: S0 a0 + S1 a1 = V0 (5.3) S1 a0 + S2 a1 = V1 where S0 = m + 1, S1 =

m X

xi ,

S2 =

i=0

V0 =

m X

yi ,

i=0

The function

D(a0 , a1 ) =

V1 =

m X

x2i ,

i=0

m X

xi yi .

i=0

m X i=0

(yi − a0 − a1xi )2

attains its minimum at the solution a∗0, a∗1 of the system of equations (5.3).

138 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 5.2 Find the line of regression for the following data: −1 0 1 2 3 4 1 2 −1 0 1 0 Estimate the of approximation. Solution. We have m = 5,

n = 1, S0 = 6,

S1 = 9,

S2 = 31,

V0 = 3,

V1 = 1.

The coefficients a∗0 and a∗1 of the line of regression P1 (x) = a∗0 + a∗1 x are determined by the following system of two equations: 6a0 + 9a1 = 3 9a0 + 31a1 = 1.

(5.4)

Solving this system of equations, we find the coefficients 1 4 a∗0 = , a∗1 = − 5 5 and the equation of the line of regression P1 (x) =

4 1 − x. 5 5

In the example. the root mean square error is E=

s

D(0.8, −0.2) = 5+1

s

4.64 = 0.879 6

To find the line of regression through data points, we execute the following commands: data={-1,1},{0,2},{1,-1},{2,0},{3,1},{4,0}}; Fit[data,{1,x},x];

Introduction to the Least Squares Analysis

5.3

Numerical Analysis with Mathematica 139

Functions Determined by Experimental Data

A relationship between two physical phenomena can be found by the least squares method using experimental data. Namely, let us assume that y depends on x in the following way: y = F (x; a0, a1, a2, . . . , an ), where F is a given function of the variable x and (n + 1) parameters a0, a1, a2, . . . , an . Within this (n + 1) parameter family of functions, we shall find one which is the best fitted to the experimental data (cf. [2,3,16,17]): x0 x1 x2 x3 · · · · · · · · · xm y0 y1 y2 y3 · · · · · · · · · ym The parameters a0 , a1, a2, . . . , an should satisfy the following conditions: y0 = F (x0, a0, a1, . . . , an ), y1 = F (x1, a0, a1, . . . , an )

(5.5)

........................ ym = F (xm , a0, a1, . . . , an ) The system of nonlinear equations (5.5) may not have a solution when m > n. However, we can determine such coefficients a∗0, a∗1, . . . , a∗n for which the function D(a0 , a1, . . . , an ) =

m X i=0

[yi − F (xi , a0, a1, . . . , an )]2,

attains its minimum in the whole real space Rn+1 . If F (x, a0, a1, . . . , an ) is continuously differentiable with respect to parameters a0, a1 , . . . , an , then the function D(a0 , a1, . . . , an )

140 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

attains its minimum at a∗0, a∗1, . . . , a∗n, for which ∂D(a∗0 , a∗1, . . . , a∗n ) = 0, ∂ak

k = 0, 1, . . . , n.

Thus, we have to solve the following nonlinear algebraic system of equations: m X i=0

[yi − F (xi, a0, a1, . . . , an )]

∂F (xi, a0 , a1, . . . , an ) = 0, ∂ak (5.6)

k = 0, 1, . . . , n. Example 5.3 Let F (x, a0, a1) = a0 e−a1 x . From an experiment the following data have been determined: x 0 1 2 y = F 3 1 0.5 Find in the family F (a0, a1) the best fitted function to the given data in the table. In this example m = 2, n = 1 and F (x, a0, a1) = a0 e−a1 x. Then, the system of equations (5.6) takes the following form: 2 X i=0

2 X

[yi − a0 e−a1 xi ]e−a1 xi = 0 [−yi + a0 e

−a1 xi

]a0 xi e

−a1 xi

(5.7) =0

i=0

Hence, we get (3 − a0) + (1 − a0 e−a1 ) e−a1 + (0.5 − a0 e−2a1 ) e−2a1 = 0 (a0 e−a1 − 1) a0 e−a1 + 2 a0 (a0 e−2a1 − .5) e−2a1 = 0

(5.8)

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 141

Let z = e−a1 and a = a0, then, we may rewrite the system of equations (5.8) as follows: a = 3 + z + (0.5 − a)z 2 − az 4 z=

1 − 2az 3 a−1

(5.9)

Applying the iterative method a(0) = 0,

z (0) = 0,

a(s+1) = 3 + z (s) + (0.5 − a(s) )[z (s)]2 − a(s)[z (s) ]4, z (s+1) =

1 − 2 a(s+1) [z (s)]3 , a(s+1) − 1

s = 0, 1, . . . ;

we arrive at the solution a = 2.985652, Hence a∗0 = 2.985652,

z = 0.36152142.

a∗1 = −ln z = 1.017434.

Then, the best fitted function to data is F (x, a∗0, a∗1) = 2.985652 e−1.017434 x Now, we shall find a function within the family F (x, a0, a1, . . . , an ) by the least squares method. (0) (0) Let a0 , a1 , . . . , a(0) n be a solution of (n + 1) first equations m > n. If F (x, a0, a1, . . . , an ) is a continuously differentiable function with respect to a0, a1, . . . , an , then (0)

(0)

F (xk , a0, a1 , . . . , an ) ≈ F (xk , a0 , a1 , . . . a(0) n ) +

(0) (0) m X ∂F (xk , a0 , a1 , . . . , a(0) n )

∂ai

i=0

for k = 0, 1, . . . , n. Denoting by (0)

(0)

∂F (xk, a0 , a1 , . . . , a(0) n ) bki = , ∂ai

αi

142 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

and

(0)

(0)

lk = yk − F (xk , a0 , a1 , . . . , a(0) n ), we can rewrite the system of equations (5.5) in the following form: b00 α0 + b01α1 + · · · + b0n αn = l0 b10 α0 + b11α1 + · · · + b1n αn = l1

(5.10)

························ bm0 α0 + bm1 α1 + · · · + bmn αn = lm

In general, the system of equations (5.10) does not have a solution when m > n. However, it is possible to find such α∗0 , α∗1 , . . . , α∗n for which the function D(α0 , α1, . . . , αn ) =

m X

k=0

attains its minimum at pk =

µk , µ

pk [lk −

α∗0 , α∗1 , . . . , α∗n ,

µ=

s

m X

bki αi ]2

i=0

where

µ20 + µ21 + · · · + µ2m , m+1

and µk is an average error related with evaluation of yk . The function D(α0 , α1 , . . . , αn ) has minimum at α∗0 , α∗1 , . . . , α∗n if ∂D(α∗0 , α∗1 , . . . , α∗n ) = 0, k = 0, 1, . . . , n. ∂αk We can find α∗0 , α∗1 , . . . , α∗n solving the following system of linear equations: m X i=0

(pi

m X

m X

bki bkj )αi =

k=0

pi li bkj

i=0

j = 0, 1, . . . , n. Now, we set a∗i = a0i + α∗i ,

i = 0, 1, . . . , n,

(5.11)

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 143

as approximate values of the parameters a∗0 , a∗1, . . . , a∗n . Then, the best fitted function to the data table is y = F (x, a∗0, a∗1, . . . , a∗n ). Example 5.4 Let us apply the above method to find F (x, a0, a1) = a0 e−a1 x for the data as in example 1. x 0 1 2 y = F 3 1 0.5 Obviously, we have n = 1, F (a0, a1) = a0 e−a1 x

m = 2, and

y0 = a0e−a1 x0 ,

a0 = 3

y1 = a0e−a1 x1 ,

a0e−a1 = 1

(5.12)

y2 = a0e−a1 x2 , a0 e−2a1 = 0.5 We can set the initial values (0)

a0 = 3,

(0)

a1 = ln 3 = 1.098612.

as the solution of two first equations in (5.12). Then, we have (0)

b00 = e−a1

x0

= 1,

(0)

(0)

(0)

(0)

b01 = −x0a0 e−a1 l0 = y0 − a0 e−a1

(0)

b10 = e−a1

x1

x0

= 0,

x0

= 0,

1 = , b11 = −x1a0e−a1 x1 = −1, 3 l1 = y1 − a0e−a1 x1 = 0,

1 b20 = e−a1 x2 = , 9

2 b21 = −x2a0e−a1 x2 = − , 3 1 l2 = y2 − a0e−a1 x2 = . 6

144 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Thus, the system of equations (5.10) takes the following form: α0 = 0 1 α 3 0

− α1 = 0

1 α 9 0

− 23 α1 =

So that, the function

(5.13) 1 6

1 1 2 1 D(α0 , α1) = α20 + ( α0 − α1 )2 + ( − α0 + α1 )2 3 6 9 3 attains its minimum if ∂D 182 22 1 α0 − α1 − = 0, = ∂a0 81 27 27 26 2 ∂D 22 = − α0 + α1 + = 0. ∂a1 27 9 9 Hence 182α0 − 198α1 = 3, −22α0 + 78α1 = −6.

and

α∗0 = −0.09695122, α∗1 = −0.1042683. Finally, we determine the coefficients (0)

a∗0 = a0 + α∗0 = 2.9030488,

(0)

a∗1 = a1 + α∗1 = 0.9943437,

and the function F (x, a∗0, a∗1) = 2.9030488e−0.9943437x . Example 5.5 The function f(x) = e−x (2 + sin πx) is tabulated below. x y=F

1 0.735759

1.2 0.425351

1.4 0.258666

1.6 0.211778

1.8 0.233438

2 0.270671

Use Mathematica Fit function to find the best fitted exponential function in the two parameters family F (x, a0, a1) = ea0 +a1 x .

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 145

data=N[Table[{x,Exp[-x]*(2+Sin[Pi x])},{x,1,2,0.2}]]; Exp[Fit[Log[data],{1,x},x]]; we obtain the best fitted exponential function g(x) = e−0.551954−1.54912 x = 0.575824e−1.54912x .

5.4

Least Squares Method in Space L2 (a, b)

The spaces L2 (a, b) and Lρ2 (a, b). Let us consider the space of all functions integrable with squares on interval [a, b], so that Z b

L2 (a, b) = {f :

a

f 2 (x)dx < +∞}.

The inner product of two functions f, g ∈ L2 (a, b) is defined as follows: Z b

(f, g) =

a

f(x)g(x)dx,

and this product generates the norm ||f|| =

q

(f, f) =

s Z

b

a

f 2 (x)dx,

f ∈ L2 (a, b).

The space L2 (a, b) is naturally associated with the following space (cf. [2, 18, 19]) Lρ2 (a, b) = {f :

Z

a

b

ρ(x)f 2 (x)dx < +∞}.

where weight 1 ρ(x) ≥ 0. The inner product in Lρ2 (a, b) space (f, g)ρ = and the norm

Z

a

b

ρ(x)f(x)g(x)dx,

||f||ρ = 1 It

q

(f, f)ρ ,

f, g ∈ Lρ2 (a, b),

f ∈ Lρ2 (a, b).

is assumed that the weight ρ(x) 6= 0 almost everywhere

146 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Consequently, two functions f and g are called orthogonal either in L2 (a, b) space or in Lρ2 (a, b) if (f, g) = 0

or

(f, g)ρ = 0.

A system of functions φ0(x), φ1(x), ..., φn(x), ..., is orthogonal if (φm , φn ) = 0

m 6= n,

for

and it is called orthonormal if (φm , φn ) =

(

1 m = n, 0 m= 6 n,

Example 5.6 The system of trigonometric functions πx πx 2πx 2πx πx mπx , cos , sin , cos , ..., sin m , cos , ..., l l l l l l is orthogonal in interval [−l, l].

1, sin

Indeed, we note that Z

l

−l

sin

mπx dx = 0, l

Z

l

−l

cos

mπx dx = 0 l

for m = 1, 2, ..., and by the trigonometric identities mπx nπx 1 π(m − n) π(m + n) cos = [sin x + sin x], l l 2 l l mπx nπx 1 π(m − n) π(m + n) sin sin = [cos x − cos x] l l 2 l l mπx nπx 1 π(m − n) π(m + n) cos cos = [cos x + cos x], l l 2 l l we find sin

Z

mπx nπx sin sin dx = l l −l l

(

l m = n, 0 m= 6 n,

Introduction to the Least Squares Analysis

Z

Numerical Analysis with Mathematica 147

(

mπx nπx l m = n, sin cos dx = 0 m 6= n, l l −l ( Z l mπx nπx l m = n, cos cos dx = 0 m 6= n. l l −l Hence, the system of trigonometric functions is orthogonal in L2 (−l, l) space. Clearly, the system of functions 1 1 πx 1 πx 1 2πx 1 πx , sin , cos , sin , ..., sin m , ... 2l l l l l l l l l is orthonormal in space L2 (−l, l). Orthogonal functions play an important role in the theory and applications of the least squares method. Therefore, we shall present some of known systems of orthogonal functions. Orthogonal polynomials. Below, we shall list orthogonal polynomials in space Lρ2 (a, b). l

1. (a) Chebyshev Polynomials T n (x) =

1 2n−1

cos(n arccos x),

n = 0, 1, ...,

1 [T ]0(x) = 2, [T ]1(x) = x, [T ]2(x) = x2 − , ..., 2 are orthogonal in space Lρ2 (−1, 1) with the weight

, so that Z

1

−1

1 ρ(x) = √ 1 − x2

π , m = n, [T ]m (x)[T ]n(x) 22m−1 √ dx =  1 − x2  0, m= 6 n.   

Indeed, let x = cos t. Then, we have Z

1

−1

Tm (x)Tn(x) 1 √ dx = m+n−2 2 1 − x2

Z

π 0

cos mt cos nt dt

148 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

  4π,     

=

m = n = 0,

π

 22m−1     0,

, m = n 6= 0, m 6= n.

(b) Hermite Polynomials take the following forms:

Hn (x) =

n −x2 n x2 d e (−1) e , dxn

or after differentiation [n/2]

Hn (x) =

X

k=0

(−1)k n! (2x)n−2k k!(n − 2k)!

n = 0, 1, ...,

Hence H0 (x) = 1,

H1 (x) = 2x,

H2 (x) = 4x2 − 2,

H3 (x) = 8x3 − 12x, ...,

These polynomials are orthogonal in space Lρ2 (−∞, ∞) 2 with the weight ρ(x) = e−x , so that Z

∞

−∞

e

−x2

Hm (x)Hn (x)dx =

 m √  2 m! π, 

0,

m = n, m 6= n.

Let us note that Z

∞

−∞

e

−x2

Hm (x)Hn (x)dx = (−1)

n

2

dn e−x Hm (x) dx, dxn −∞

Z

∞

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 149

Integrating by parts, we obtain 2

dn e−x (−1) Hm (x) dx dxn −∞ 2 dn−1 e−x ∞ n = (−1) Hm (x) | dxn−1 −∞ 2 Z ∞ dn−1 e−x 0 n dx −(−1) Hm (x) dxn−1 −∞ 2 Z ∞ dn−1 e−x 0 n+1 = (−1) Hm (x) dx dxn−1 −∞ .................................................. n

Z

∞

= (−1)2n

Z

∞

2

−∞

(n) Hm (x)e−x dx.

(n) Because Hm (x) = 0 for n > m, therefore

Z

∞

−∞

2

e−x Hm (x)Hn (x)dx = 0,

m 6= n.

Using the equality (n) Hm (x) = 2n n! for m = n,

we compute Z

∞ −∞

e

−x2

n

Hm (2)dx = 2 n!

Z

∞ −∞

√ 2 e−x dx = 2n n! π.

(c) Laguerre Polynomials are given by the following formulas: ex dn e−x xn , LEn (x) = n! dxn or after differentiation LEn (x) =

n X

k=0

!

n (−x)k , k k!

n = 0, 1, ...,

Hence 1 LE0 (x) = 1, LE1 (x) = 1−x, LE2 (x) = 1−2x+ x2; ..., 2

150 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Laguerre polynomials are orthogonal in space Lρ2 (0, ∞) with the weight ρ(x) = e−x , so that Z

0

∞

−x

e LEm (x)LEn (x)dx =

(

1, m = n, 0, m 6= n.

(d) Legendre Polynomials are given by the following formula: 1 dn (x2 − 1)n LGn (x) = n , n = 0, 1, ..., 2 n! dxn 3 1 LG0 (x) = 1, LG1 (x) = x, LG2 (x) = x2− , ..., 2 2 These polynomials are orthogonal in space L2 (−1.1) and  2  Z 1  , m = n, 2n + 1 LGm (x)LGn (x)dx =  −1  0, m 6= n. Gram - Schmidt orthogonalization. Let φ0 (x), φ1(x), ..., φn(x), ..., be a system of linearly independent functions with φk ∈ L2 (a, b) ∪ Lρ2 (a, b), for k = 0, 1, ... Such a system of functions can be transformed into the orthogonal one ψ0 (x), ψ1 (x), ..., ψn (x), ..., using the following Gram-Schmidt algorithm: ψ 0 = φ0 , for m = 1, 2, ..., evaluate ψ m = φm −

m−1 X k=0

(φm , ψk ) ψk (ψk , ψk )

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 151

Indeed, we note that (ψ0, ψ0) = (φ0, φ0 ) = ||φ0||2, (ψ1, ψ0) = (φ1, ψ0 ) −

(φ1 , ψ0) (ψ0 , ψ0) = 0, (ψ0, ψ0)

and by mathematical induction (ψm , ψn ) = (φm , ψn ) −

m−1 X k=0

(φm , ψk ) (ψk , ψn ). (ψk , ψk )

Since (ψk , ψn ) = 0 for k 6= n, and therefore (ψm , ψn ) = (φm , ψn ) −

k, n < m,

(φm , ψn ) (ψn , ψn ) = 0. (ψn , ψn )

An implementation of Gram-Schmidt algorithm in a computer may lead to a non-orthogonal system because of roundoff errors accumulation. Nevertheless, one can minimized the affect of round-off errors by applying the algorithm to the newly obtained system of functions, repeatedly. Example 5.7 The polynomials: φn (x) = xn , n = 0, 1, ..., are linearly independent for all real x. Applying Gram-Schmidt algorithm to these polynomials in the space L2 (−1, 1), we arrive at orthogonal polynomials which are different from the Legendre polynomials listed above by a constant factor, so that s

3 1 ψ0(x) = √ , ψ1(x) = x, 2 2 s s 3 5 2 1 5 7 3 3 ψ2(x) = (x − ), ψ3 (x) = (x − x), ..., 2 2 3 2 2 5 We can use Mathematica module GramSchmidt (cf. [28]) to transform a system of linearly independent functions into a

152 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

system of orthogonal functions. For example, to transform polynomials 1, x, x2 , x3, x4 into Legendre polynomials, we execute the following commands: Needs["LinearAlgebra‘Master‘"]; GramSchmidt[{1,x,x^2,x^3,x^4},InnerProduct-> (Integrate[#1 #2,{x,-1,1}]&)]//Simplify The best least squares polynomial. Let us consider a function f ∈ Lρ2 (a, b) and the generalized polynomial Pn (x) = a0 φ0(x) + a1φ1 (x) + a2 φ2(x) + · · · + an φn (x).

In order to approximate f by a polynomial of degree not greater than n, we shall minimize the function 2

D(a0 , a1, ..., an) = ||f − Pn || =

Z

b

a

ρ(x)[f(x) − Pn (x)]2dx.

The function D attains its minimum at a point a∗0 , a∗1 , , ..., a∗n if the partial derivatives ∂D(a∗0 , a∗1, ..., a∗n) = 0, k = 0, 1, ..., n. ∂ak On the other hand Z b ∂D(a∗0, a∗1, ..., a∗n) = −2 ρ(x)[f(x) − Pn (x)]φk (x) dx. ∂ak a Hence, the coefficients of the best approximating polynomial must satisfy the following system of linear algebraic equations: (φ0 , φ0)a0 + (φ1, φ0 )a1 + · · · + (φn , φ0 )an

= (f, φ0)

(φ0 , φ1)a0 + (φ1, φ1 )a1 + · · · + (φn , φ1 )an

= (f, φ1)

................................................................

........

................................................................

........

(φ0 , φn )a0 + (φ1 , φn )a1 + · · · + (φn , φn )2nan = (f, φn ) (5.14)

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 153

where (φk , φm ) =

Z

b

a

ρ(x)x

k+m

dx,

(f, φk ) =

Z

b

a

ρ(x)xk f(x) dx.

The above system of equations has a unique solution, since its Gram matrix of linearly independent functions is nonsingular. Thus, the solution a∗0, a∗1, ..., a∗n determines the best least squares polynomial approximating function f on interval [a, b]. In order to estimate the error of approximation the following formula can be used: E(f, Pn ) =

1 q D(a∗0 , a∗1, ..., a∗n). b−a

In particular, let us consider the polynomial Pn (x) = a0 + a1x + a2x2 + · · · + an xn . The coefficients of this polynomial must satisfy the system of equations ∂D(a∗0 , a∗1, ..., a∗n) = −2 ∂ak

Z

b

a

ρ(x)[f(x) − Pn (x)]xk dx = 0.

This system of equations has the following explicit form: s0 a 0 + s1 a 1 + · · · + sn a n

= m0

s1 a0 + s2 a1 + · · · + sn+1 an

= m1

........................................

......

........................................

......

(5.15)

sn a0 + sn+1 a1 + · · · + s2n an = mn where sk =

Z

b

a

k

ρ(x)x dx,

mk =

Z

a

b

ρ(x)xk f(x) dx,

k = 0, 1, ...,

154 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 5.8 Apply the least squares method either in L2 (−1, 1) space or in Lρ2 (−1, 1) space to approximate the function: √ f(x) = 1 − x2, −1 ≤ x ≤ 1, by a polynomial P2 (x) = a0φ0 (x) + a1φ1 (x) + a2φ2 (x), where 1. (a) φk (x) = xk , k = 0, 1, 2; and ρ(x) = 1, are natural polynomials 1 (b) φk (x) = Tk (x), k = 0, 1, 2; and ρ(x) = √ , 1 − x2 are Chebyshev’s polynomials. Solution (a). For φk (x) = xk , k = 0, 1, 2; the system of linear equations (5.15) takes the following form: π 2a0 + 23 a2 = 2 2 a1 =0 3 π 2 a + 25 a2 = 3 0 8 Solving the above system of equations, we find the coefficients 15π 21π , a∗1 = 0, a∗2 = − a∗0 = 64 64 and the polynomial 3π (7 − 5x2), −1 ≤ x ≤ 1. 64 Solution (b). Now, we apply the least squares method in space 1 Lρ2 (−1, 1), ρ(x) = √ , 1 − x2 where φk (x) = Tk (x), k = 0, 1, 2; Because Chebyshev’s polynomials are orthogonal in Lρ2 (−1, 1), therefore (5.14) is a diagonal system of linear equations, so that the coefficients P2 (x) =

a∗0 =

1 , π

a∗1 = 0,

a∗2 = −

2π . 3

Introduction to the Least Squares Analysis

Numerical Analysis with Mathematica 155

and the polynomial 2 (2 − x2), −1 ≤ x ≤ 1. 3π In the table, we compare the error of approximation for the best squares polynomials and the interpolating polynomial 2 P √2 (x) = 1 − x , x0 = −1, x1 = 0 x2 = 1, when f(x) = 1 − x2 . P2 (x) =

Table 6.1

LS-Method

φk (x)

in L2 (−1, 1)

xk

in Lρ2 (−1, 1)

Tk (x)

Interpolation

xk

5.5

P2 (x)

D(a∗0 , a∗1, a∗2 ) E(P2 , f)

3π (7 − 5x2 ) 64 2 (2 − x2) 3π 1 − x2

0.00325

0.0285

0.42485

0.3259

0.04381

0.10426

Exercises

Question 5.1 Find the line of regression for the following data: −2 −1 0 1 2 3 1 0 2 3 2 4 Estimate the error of approximation. Question 5.2 Approximate the following data: 0 0.25 0.5 0.75 1 0 0.707 1 0.707 0 by a polynomial of second degree. Estimate the error of approximation. Question 5.3 Consider the following two parameter family of functions: F (x, a0, a1) = a0 + a1 ln(1 + x),

x ≥ 0.

156 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Find in the family the best fitted function to data: 0 1 2 3 2 2.693 3.099 3.386 Question 5.4 Apply the least squares method either in L2 (−2, 2) space or in Lρ2 (−2, 2) space to approximate the function √ f(x) = 4 − x2, −2 ≤ x ≤ 2, by a polynomial P2 (x) = a0 + a1x + a2x2 , where 1. (a) natural polynomials φk (x) = xk k = 0, 1, 2; in L2 (−2, 2) space, (b) Chebyshev polynomials φk (x) = Tk ( 12 x), k = 0, 1, 2; 1 in Lρ2 (−2, 2) space, ρ = √ . 1 − x2 Question 5.5 Write a Mathematica module to approximate a given function f(x) in the interval [a, b] by a cubic polynomial

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 157-198

157

CHAPTER 6

Selected Methods for Numerical Integration Abstract

Two classes of methods for numerical integration are presented : The class of Newton-Cotes methods and the class of Gauss methods. In the class of Newton-Cotes methods, the trapezoidal rule, Simpson’s rule and other higher order rules like Romberg’s method have been derived . The methods are clarified by examples. Mathematica modules are designed to derive Newton Cotes methods of any accuracy and to apply them for evaluation of integrals. Gauss methods are derived in general form with n Gauss-Legendres knots and in particular with n=1,2,3,4 knots. Example illustrating the methods are presented. The exact analysis of both methods of numerical integration has been carrying out. A set of questions is enclosed in the chapter. Keywords: Newtons- Cotes formulas, Gauss formulas.

Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

158 Numerical Analysis with Mathematica

6.1 .

Krystyna STYŠ & Tadeusz STYŠ

Introduction

Let us consider a function f(x) in an interval [a, b]. If F (x) is an antiderivative to f(x), i.e. F 0(x) = f(x), x ∈ [a, b], then by the fundamental theorem of calculus I(f) =

Z

a

b

f(x)dx = F (b) − F (a).

In general, the antiderivative F (x) may not be known, al2 though it may exist. For instance, the function f(x) = e−x possesses an antiderivative. However, any antiderivative to this function cannot be expressed by elementary functions. Nevertheless, we may evaluate the integral Z

b

2

e−x dx

a

numerically. There are many techniques of numerical integration based on different principles. Below, we shall present two classes of methods, Newton-Cotes methods and Gauss methods (cf. [3, 11,20,27]).

6.2

Newton-Cotes Methods

Let Pn (x) be an interpolating polynomial to a sufficiently regular function f(x) spaned by interpolating knots a ≤ x0 < x1 < · · · < xn ≤ b. Then, by the theorem 2.2, we have f(x) = Pn (x) + Rn (f),

a ≤ x ≤ b,

where Rn (f) is the error of interpolation. Replacing the integrand f(x) by its interpolating polynomial Pn (x), we obtain the formula Z

a

b

f(x)dx =

Z

a

b

Pn (x)dx + E(f),

Selected Methods for Numerical Integration

where E(f) =

Numerical Analysis with Mathematica 159

Z

b

a

Rn (f)dx

(6.1)

is the truncation error. Canceling the truncation error E(f) in the above formula, we get the following approximation of the integral: Z

b

f(x)dx ≈

a

Z

b

a

Pn (x)dx.

Let us write the interpolating polynomial Pn (x) in the Lagrange form Pn (x) = l0 (x)f(x0) + l1(x)f(x1 ) + · · · + ln (x)f(xn ), where li (x) =

(x − x0)(x − x1) · · · (x − xi−1 )(x − xi+1 ) · · · (x − xn ) (xi − x0)(xi − x1) · · · (xi − xi−1 )(xi − xi+1 ) · · · (xi − xn )

for i = 0, 1, . . . , n; Then, we have Z

b

Pn (x)dx =

a

n X

ci f(xi ),

i=0

where the coefficients ci =

Z

b

a

li(x)dx,

i = 0, 1, · · · , n;

(6.2)

are independent of f. In this way, we arrived at the following Newton-Cotes formulas: Z n b

a

f(x)dx ≈

X

ci f(xi ),

i=0

where the truncation error E(f) is given by (6.1). Before, we shall derive a few Newton-Cotes algorithms, let us explain the idea of simple and composed numerical methods. All numerical methods of integration can be classified in two

160 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

groups, either simple methods or composed methods. This classification is based on the additive property of Riemann integral. Namely, we have I(f) =

Z

b

a

f(x)dx =

n−1 X Z xi+1 i=0

xi

f(x)dx =

n−1 X

Ii(f).

i=0

Then, a numerical method is called simple method, if it produces approximate values of the partial results Ii (f) =

Z

xi+1

xi

f(x)dx,

i = 0, 1, . . . , n − 1,

and a numerical method is called composed method, if it produces an approximate value to the integral I(f) as the sum of all approximate values to individuals Ii (f), i = 0, 1, . . . , n − 1.

6.3

Trapezian Rule.

Let us consider the linear interpolating polynomial P1 (x) to the function f(x) in the subinterval [xi , xi+1]. Then, we have x − xi+1 x − xi P1 (x) = f(xi ) + f(xi+1 ). xi − xi+1 xi+1 − xi and f(x) = P1 (x) +

f 00 (ξi (x)) (x − xi )(x − xi+1) 2!

for certain ξi (x) ∈ (xi , xi+1 ), and for f(x) twice continuously differentiable in interval[a, b]. Hence, we get Ii (f) = +

Z

xi+1

xi

Z

xi+1

xi

f(x)dx =

Z

xi+1

xi

P1 (x)dx

f 00(ξi (x)) (x − xi )(x − xi+1 )dx. 2!

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 161

where Z

xi+1

xi

P1 (x)dx = + = =

Z

xi+1 f(xi ) (x − xi+1 )dx xi − xi+1 xi Z f(xi+1 ) xi+1 (x − xi )dx xi+1 − xi xi xi+1 − xi [f(xi ) + f(xi+1 )] 2 hi [f(xi) + f(xi+1 )], 2

for hi = xi+1 − xi , i = 0, 1, . . . , n − 1. We illustrate the method on following figure (Fig. 6.1) 6

a

I3(f)

b

q

q

q

q

q

q

q

q

x0

x1

x2

x3

x4

x5

x6

x7

Fig. 6.1 I(f)

In order to estimate the error of the simple trapezian rule

Ei (f) =

Z

xi+1

xi

f 00 (xi(x)) (x − xi)(x − xi+1 )dx, 2

we note that the quadratic function (x − xi )(x − xi+1 ) does not change its sign in the interval [xi , xi+1 ). Therefore, by

-

162 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

the mean value theorem Z xi+1 00 f (ξi (x)) (x − xi)(x − xi+1 )dx 2 xi Z f 00(ηi ) xi+1 = (x − xi)(x − xi+1 )dx 2 xi f 00(ηi ) =− (xi+1 − xi )3. 12 for certain ηi ∈ (xi , xi+1 ). Hence, the error of the simple trapezian rule is: f 00 (ηi ) 3 f 00(ηi ) (xi+1 − xi )3 = − h. 12 12 i for i = 0, 1, . . . , n − 1. We shall consider the composed trapezian rule for a uniform partition of the interval [a, b], i.e. for xi+1 − xi = h, i = 0, 1, . . . , n − 1. Applying the simple trapezian rule to each of the subinterval [xi, xi+1 ], we obtain Ei (f) = −

Z

b

a

f(x)dx = =

n−1 X Z xi+1 i=0 xi n−1 X

h 2

i=0

f(x)dx

[f(xi) + f(xi+1 )] −

X h3 n−1 f 00(ηi ). 12 i=0

By the intermediate value theorem, there exists η ∈ (a, b) such that n−1 X i=0

f 00 (ηi ) = nf 00 (η) =

(b − a) 00 f (η), h

so that Z

h [f(x0) + · · · + 2f(xn−1 ) + f(xn )] + ET (f, h). 2 a (6.3) In this way, we arrived at the composed trapezian rule b

f(x)dx =

Th (f) =

h [f(x0) + 2f(x1 ) + 2f(x2 ) + · · · + 2f(xn−1 ) + f(xn )] 2

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 163

where the truncation error h2 ET (f, h) = − (b − a)f 00(η), 12 for certain η ∈ (a, b), so that I(f) = Th (f) + ET (f, h). The truncation error of the trapezian rule satisfies the following inequality: | ET (f, h) |≤

M (2) (b − a)h2, 12

where M (2) = sup | f 00 (x) | . a≤x≤b

Example 6.1 Evaluate the integral Z

2

ln(1 + x)dx

0

by trapezian rule with the accuracy  = 0.05. Solution. In order to get accuracy  = 0.05, we shall estimate the step-size h, so that, we choose the greatest b−a for which the following inequality holds: h= n ET (f, h) ≤

h2 (b − a)M (2) ≤ . 12

Because f(x) = ln(1 + x), f 0 (x) = we have

1 1 , f 00 (x) = − , 1+x (1 + x)2

1 = 1. 0≤x≤2 (1 + x)2

M (2) = max

164 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

So, the inequality ET (f, h) ≤

h2 2 < 0.05 12

holds for h = 0.5 and n = 4. The approximate value of the integral is: T (f) = 0.25[f(x0 ) + 2f(x1 ) + 2f(x2 ) + 2f(x3 ) + f(x4 )] = 0.25[ln(1) + 2ln(1.5) + 2ln(2) + 2ln(2.5) + ln(3)] = 1.282105. Further error analysis. We shall give more specific analysis of the errors of trapezian method. Let us note that (x − yi)2 00 f (yi ) 2 (x − yi )4 (4) (x − yi )3 000 f (yi ) + f (ξi ), + 6 24

f(x) = f(yi ) + (x − yi )f 0 (yi ) +

for certain ξi ∈ (xi, xi+1 ), and for yi = 12 (xi + xi+1 ), h = xi+1 − xi , i = 0, 1, . . . , n − 1; Hence 1 1 1 1 4 (4) f(xi ) = f(yi )− hf 0 (yi )+ h2 f 00 (yi)− h3 f 000(yi )+ h f (ξi ) 2 8 48 384 and 1 0 1 1 1 4 (4) f(xi+1 ) = f(yi )+ hf (xi )+ h2 f 00(yi )+ h3 f 000(yi )+ h f (ηi ). 2 8 48 384 Therefore 1 h2 1 (4) [f(xi) + f(xi+1 )] = f(yi ) + f 00(yi ) + f (ηi ). 2 8 384

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 165

Clearly, we have Z

xi+1

xi

f(x)dx = f(yi )

Z

xi+1

xi

dx + f 0 (yi)

Z

xi+1 xi

(x − yi )dx

f 00(yi ) xi+1 (x − yi )2dx 2 xi Z f 000(yi) xi+1 + (x − yi )3 dx 6 xi Z f (4)(ξi ) xi+1 + (x − yi )4 dx 24 xi h3 h5 (4) = hf(yi ) + f 00 (yi ) + f (ξi ) 24 1920 Substituting to the above formula +

Z

h2 h4 (4) 1 f (ηi ), f(yi ) = [f(xi) + f(xi+1 )] − f 00(yi ) − 2 8 384 we obtain the simple trapezian rule Z xi+1 h h3 h5 (4) f(x)dx = [f(xi ) + f(xi+1 )] − f 00(yi ) − f (ηi ). 2 12 480 xi where the truncation error h3 h5 (4) Ei (f, h) = − f 00(yi ) − f (ηi ). 12 480 Hence, we obtain the composed rule Z b h [f(x0) + 2f(x1 ) + · · · + 2f(xn−1 ) + f(xn )] f(x)dx = 2 a + ET (f, h), (6.4) which has the truncation error ET (f, h) =

n−1 X

Ei (f, h)

i=0

=− −

h3 00 [f (y0) + f 00(y1 ) + · · · + f 00(yn−1 ] 12

h5 (4) [f (η0) + f (4) (η1 ) + · · · + f (4)(ηn−1 )]. 480

166 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

By the intermediate value theorem f 00 (y0) + f 00 (y1 ) + · · · + f 00(yn−1 ) = nf 00 (ξ) =

b − a 00 f (ξ) h

and f (4) (η0)+f (4) (η1)+· · ·+f (4) (ηn−1 ) = nf (4) (η) =

b − a (4) f (η). h

Finally, we determine the truncation error of the trapezian rule as ET (f, h) = −

6.4

h2 h4 (b − a)f 00(ξ) − (b − a)f (4)(η). 12 480

Simpson Rule.

In order to derive Simpson rule, we consider Hermite interpolating polynomial H3 (x) for the function f(x) spaned by the three knots: xi−1 , xi and xi+1 which satisfies the following conditions: H3 (xj ) = f(xj ),

for j = i − 1, i, i + 1,

0

for j = i.

and H3 (xj ) = f 0 (xj ), It is easy to check that H3 (x) = L2 (x) + (x − xi−1 )(x − xi )(x − xi+1 )H1 (x), where Lagrange interpolating polynomial (x − xi)(x − xi+1 ) f(xi−1 ) (xi−1 − xi)(xi−1 − xi+1 ) (x − xi−1)(x − xi+1 ) + f(xi ) (xi − xi−1)(xi − xi+1 ) (x − xi−1 )(x − xi ) + f(xi+1 ), (xi+1 − xi−1 )(xi+1 − xi )

L2 (x) =

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 167

and the constant polynomial H1 (x) = −

1 1 0 f (x ) + [f(xi+1 ) − f(xi−1 )]. i h2 2h3

Let f(x) be a function four times continuously differentiable in the interval [a, b]. By theorem 2.3, we have f (4) (ξi ) (x − xi−1 )(x − xi )2(x − xi+1 ), 4!

f(x) = H3 (x) +

for certain ξi ∈ (xi−1 , xi+1 ). Hence, we get Z

xi+1

xi−1

f(x)dx=

+

Z

1 4!

xi+1

Z

xi−1

xi+1

xi−1

H3 (x)dx

f (4) (ξi (x))(x − xi−1 )(x − xi)2 (x − xi+1)dx.

Let us note that Z

xi+1

xi−1

(x − xi−1 )(x − xi)(x − xi+1 )dx = 0.

So that Z

xi+1

xi−1

H3 (x)dx =

Z

xi+1

xi−1

L2 (x)dx =

h [f(xi−1) + 4f(xi ) + f(xi+1 )]. 3

By the mean value theorem for integrals, we get 1 4! =

Z

xi+1

xi−1

f (4) (ξi (x))(x − xi−1 )(x − xi )2(x − xi+1 )dx =

f (4)(ηi ) 4!

Z

xi+1

xi−1

(x − xi−1 )(x − xi)2 (x − xi+1)dx = −

f (4)(ηi ) 5 h. 90

In this way, we have arrived at the simple Simpson rule Z

xi+1

xi−1

h [f(xi−1 ) + 4f(xi ) + f(xi+1 )]] 3 f (4) (ηi ) 5 − h, 90

f(x)dx =

(6.5)

168 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

where the error of the method is: f (4) (ηi ) 5 ES (f, h) = − h. 90 for certain ηi ∈ (xi−1 , xi+1 ). We illustrate the method on followung figure (Fig. 6.2) 6

a

I3(f)

b

q

q

q

q

q

q

q

q

x0

x1

x2

x3

x4

x5

x6

x7

Fig. 6.2. I(f)

In order to derive the composed Simpson rule, we consider the uniform portion of the interval [a, b] by the following points: b−a xi = a + ih, i = 0, 1, . . . , 2n, h = . 2n Applying simple Simpson rule to each term of the following sum: Z

b

a

f(x)dx =

Z

x2

x0

f(x)dx +

Z

x4

x2

f(x)dx + · · · +

Z

x2n

x2n−2

f(x)dx,

we obtain composed Simpson rule Z

a

where Sh (f) =

b

f(x)dx = Sh (f) + ES (f, h),

h [f(x0 ) + 4f(x1 ) + 2f(x2 ) 3

+4f(x3 ) + · · · + 2f(x2n−2 ) + 4f(x2n−1 ) + f(x2n )],

-

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 169

and the error h5 (4) ES (f, h) = − [f (η1 ) + f (4)(η2 ) + · · · + f (4) (ηn )]. 90 Hence, by the intermediate value theorem, there exists η ∈ (a, b) such that f (4) (η1) + f (4) (η2 ) + · · · + f (4) (ηn ) = nf (4) (η) =

b − a (4) f (η). 2h

Therefore, the error of the composed Simpson rule is: ES (f, h) = −

h5 (4) h4 nf (η) = − (b − a)f (4)(η), 90 180

for certain η ∈ (a, b). This error satisfies the following inequality: | ES (f, h) |≤

h4 (b − a)M (4). 180

(6.6)

where M (4) = max | f (4) (x) | . a≤x≤b

Example 6.2 Evaluate the integral I(f) =

Z

2 0

ln(1 + x)dx

by Simpson rule using step-size h = 0.5. Estimate the truncation error ES (f, h). Solution. We note that 2n =

b−a = 4 and h

h [f(x0) + 4f(x1 ) + 2f(x2 ) + 4f(x3 ) + f(x4)] 3 0.5 = [ln(1) + 4ln(1.5) + 2ln(2) + 4ln(2.5) + ln(3)] 3 = 1.295322

Sh (f) =

170 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

The exact value of I(f) = 1.295837, so that the error ES (f, h) = I(f) − S(f) = 1.295837 − 1.295322 = 0.000515. Also, we note that f (4)(x) = −

6 6 (4) and M = max = 6. 0≤x≤2 1 + x)4 (1 + x)4

Hence | ES (f, h) ≤

6.5

h4 0.0625 (b − a)M (4) = 2 ∗ 6 = 0.00417. 180 180

Romberg Method and Recursive Formulas

Below, we shall present methods of numerical integratione based on the trapezian rule and Richartson extrapolation Romberg Method Let us assume that the error ET (f, h) is proportional to h2 , i.e. ET (f, h) = C h2 , for certain constant C. Then, we have ET (f, 2h) = C 4h2 = 4 ET (f, h), I(f) = T (h, f) + ET (f, h). So that I(f) = T (2h, f) + ET (f, 2h) = T (2h, f) + 4ET (f, h). Hence, we find the truncation error 1 E(f, h) = [T (h, f) − T (2h, f)]. 3 In this way, we obtain the Romberg method 1 Rh (T, f) = [4 T (h, f) − T (2h, f)], 3

(6.7)

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 171

for which Z

b

a

f(x)dx = Rh (T, f) + ER (T, h, f),

where the truncation error h4 (b − a)f (4)(η), 180 Indeed, the Romberg method has the same truncation error ES (f, h) as the Simpson rule, since ER (T, h, f) = ES (f, h) = −

1 Rh (T, f) = [4Th (f) − T2h (f)] 3 h = [f(x0 ) + 4f(x1 ) + 2f(x2 ) 3 +4f(x3 ) + · · · + 4f(x2n−1 ) + f(x2n )] = Sh (f). Example 6.3 Consider the following integral: Z

1.2

0

ln(1 + x)dx.

(a). Evaluate this integral by Th (f) rule with accuracy  = 0.01. (b). Improve the result, that obtained in (a), using the Romberg formula (??). Estimate the error of the final result. Solution. We have f(x) = ln(1 + x) and | f 00 (x) |=| −

1 |≤ 1, (1 + x)2

for x ∈ [0, 1, 2]. To determine h, we consider the inequality h2 h2 (b − a)f 00(ξ) |≤ ≤  = 0.01, 12 10 for h ≤ 0.31622. So, we can choose h = 0.3. Then | ET (f, h) |≈| −

n=

1 (b − a) = 4 h

172 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

for b − a = 1.2 , h = 0.3 and xi = 0.3i, i = 0, 1, 2, 3, 4. By (6.4), we get h [f(x0 ) + 2f(x1 ) + 2f(x2 ) + 2f(x3 ) + f(x4 )] 2 = 0.15[ln(1) + 2ln(1.3) + 2ln(1.6) + 2ln(1.9) + ln(1.2)

Th (f) =

= 0.5305351. For 2h = 0.6, we have n = 2 and 2h [f(x0) + 2f(x1 ) + f(x2 )] 2 = 0.3[ln(1) + 2ln(1.6) + ln(2.2)] = 0.5185395.

T2h (f) =

Using the Romberg formula, we compute Z

0

1.2

1 ln(1 + x)dx ≈ [4Th (f) − T2h (f)] = 3

1 (4 ∗ 0.5305351 − 0.5185395) = 0.5345337. 3

Hence, the error of the final result is: | ER (f, h) |=| −

h4 0.34 (b − a)f (4)(η) |≤ 1.2 ∗ 6 = 0.000324. 180 180 Z

1.2

Let us note that ln(1 + x) dx = 0.534606. 0 Recursive Formulas. Now, we consider a uniform partition of the interval [a, b] into 2n subintervals of the length b−a h = n . The trapezian rule for such a partition is: 2 RT (n, 0) = h

n −1 2X

i=1

h f(a + ih) + [f(a) + f(b)]. 2

(6.8)

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 173

From formula (6.8), we have RT (0, 0) RT (1, 0) RT (2, 0)

b−a [f(a) + f(b)], 2 h = hf(a + h) + [f(a) + f(b)], 2 = h[f(a + h) + f(a + 2h) + f(a + 3h)] =

+ .................

h [f(a) + f(b)], 2 .......................................................

RT (n − 1, 0) = h

2n−1 X−1 i=1

h f(a + ih) + [f(a) + f(b)] 2

We can compute RT (n, 0) using known values f(a), f(b) and f(a+2h), f(a+4h), ..., f(a+(2n−1 −2)h), while RT (n−1, 0) has been evaluated. Thus, to compute RT (n, 0), we need to evaluate f(x) at the points a + h, a + 3h, ..., a + (2n − 1)h. These computations can be done by the following recursive formulas: b−a [f(a) + f(b)], h = b − a, 2 1 b−a RT (1, 0) = RT (0, 0) + hf(a + h), h = , 2 2 1 b−a RT (2, 0) = RT (1, 0) + h[f(a + h) + f(a + 3h)], h = , 2 4 4 X 1 b−a RT (3, 0) = RT (2, 0) + h f(a + (2k − 1)h), h = , 2 8 k=1 ........................................................................................... RT (0, 0) =

n−1

2X 1 RT (n, 0) = RT (n − 1, 0) + h f(a + (2k − 1)h), 2 k=1 b−a in general for h = n . 2

174 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Assuming that the error of the trapezian method has the following series expansion ET (f, h) = a2h2 + a4 h4 + a6 h6 + · · · , we can accept that ET (f, h) ≈ a2 h2 . Then, the Romberg metod gives the following 0(h4 ) accurate result: 1 Rh (T, f) = RT (n, 1) = RT (n, 0)+ [RT (n, 0)−RT (n−1, 0)], 4−1

We shall obtain much more accurate result using the recursive formula 1 [RT (n, m−1)−RT (n−1, m−1)], RT (n, m) = RT (n, m−1)+ m 4 −1 for m, n ≥ 1. Using this recursive formula, we can build the following table of the Romberg’s results: RT (0, 0) RT (1, 0) RT (1, 1) RT (2, 0) RT (2, 1) RT (2, 2) RT (3, 0) RT (3, 1) RT (3, 2) RT (3, 3) .......

.........

.........

.......

.......

RT (n, 0) RT (n, 1) RT (n, 2) RT (n, 3) ....... RT (n, n) Error estimate Let us assume that the error of the trapezian b−a rule is given by the following power series in terms of h = n 2 ET (f, h) =

Z

b

a

f(x)dx − RT (n, 0) = a2 h2 + a4h4 + a6h6 + · · · , (6.9)

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 175

Then, the error of the Romberg’s method is (n,1) ERT (f, h)

Z

=

b

a

Z

=

b

a

f(x) dx − RT (n, 1) f(x) dx − [RT (n, 0)

1 (RT (n, 0) − RT (n − 1, 0))] 3 1 h = [4ET (f, ) − ET (f, h)]. 3 2 +

Hence, by the assumption (6.9), we obtain power series of the error (n,1)

ERT (f, h) =

Z

b

a

f(x)dx − RT (n, 1)

5 1 = − a4h4 − a6 h6 − · · · = b4 h4 + b6h6 + · · ·. 4 16 (6.10) Hence, we obtain the following estimate of the error |

Z

a

b

1 f(x)dx − RT (n, 1)| ≈ a4h4 , 4

h=

b−a . 2n

In order to estimate the error Z

a

b

f(x)dx − RT (n, 2),

we repeat the above analysis. Then, we have (n,2) ERT (f, h)

= =

Z

b

a

Z

a

b

f(x) dx − RT (n, 2) f(x) dx − [RT (n, 1)

1 (RT (n, 1) − RT (n − 1, 1))] 15 1 h (n,1) (n,1) = [16ERT (f, ) − ERT (f, h)]. 15 2 +

176 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

So that, the error (n,2)

ERT (f, h) =

Z

b

a

= −

f(x)dx − RT (n, 2)

1 1 b6 h6 − b8h8 − · · · = c6 h6 + c8 h8 + · · ·, 20 16 (6.11)

b−a for h = n . 2 Using the following Mathematica module we can obtain the elements RT (i, j), i = 0, 1, ..., n, j = 0, 1, .., i, of Romberg’s table for a given function f(x) in the interval [a, b]. Program 6.1 Mathematica module that gives the Romberg’s table. rombergTable[f_,a_,b_,p_,q_]:=Module[{r,s,h}, h=(b-a)/2; s[1]=f[a+h]; s[i_]:=s[i]=Module[{h=(b-a)/2^i}, Apply[Plus,Table[f[a+k h],{k,1,2^i-1,2}]]]; r[0,0]=(b-a)*(f[a]+f[b])/2; r[1,0]=(r[0,0]+(b-a)*f[(a+b)/2])/2; r[i_,0]:=r[i,0]=r[i-1,0]/2+(b-a)/2^i*s[i]; r[i_,j_]:=r[i,j]= r[i,j-1]+(r[i,j-1]-r[i-1,j-1])/(4^j-1); r[p,q] ] Example 6.4 Using th module rombergTable, build the Romberg’s table RT (0, 0) RT (1, 0) RT (1, 1) RT (2, 0) RT (2, 1) RT (2, 2) RT (3, 0) RT (3, 1) RT (3, 2) RT (3, 3)

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 177

for the function f(x) = ln(1 + x), 0 ≤ x ≤ 2. Give an estimate of the error Z

0

2

ln(1 + x) − RT (2, 2).

To obtain the Romberg’s table, we define the function f[x_]:=Log[1+x] and execute the following Mathematica instruction: Table[N[rombergTable[f,0,2,p,q]],{p,0,3},{q,0,p}] //TableForm From Mathematica, we obtain the following table: 1.09861 1.24245 1.2904 1.2821

1.29532 1.29565

1.29237 1.2958

1.29583 1.29583

To obtain the error estimate , we apply formula (??), in 1 the case when n = 2, m = 2, a = 0, b = 2, h = . Then, 2 we have |

Z

2

0

ln(1 + x)dx − RT (2, 2)| ≈ c6h6 = 0.015635c6 .

Simpson’s Extrapolating Method Assuming that the error ES (f, h) of Simpson rule is proportional to h4 , i.e. there is a constant C such that ES (f, h) ≈ Ch4 . we obtain ES (f, 2h) ≈ 16Ch4 ≈ 16ES (f, h). Because I(f) = Sh (f) + ES (f, h)

178 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

and I(f) = S2h (f) + ES (f, 2h) ≈ S2h (f) + 16ES (f, h), therefore

Sh (f) − S2h (f) 15 In this way, we obtain the following numerical method: ES (f, h) ≈

I(f) = Sh (f) +

Sh (f) − S2h (f) + ER(S, f, h). 15

Hence Z

a

b

f(x)dx = Rh (S, f) + ER (S, f, h),

(6.12)

where

16Sh (f) − S2h (f) . 15 In order to show that ER (S, f, h) = O(h6 ), we consider the integral Z x J (x) = f(t)dt, x ∈ [xi, xi+1 ]. Rh (S, f) =

xi

By Taylor formula

f 00(xi ) f 0 (xi ) 2 (x − xi ) + + (x − xi )3 J (x) = f(xi )(x − xi ) + 2! 3! 000 (4) f (xi) f (xi ) f (5)(xi ) + (x − xi )4 + (x − xi )5 + (x − xi)6 4! 5! 6! (6) f (ξi ) + (x − xi )7. 7! (6.13) Hence, we have J (xi+1) +

f 000(xi ) 4 h 4!

f 0 (xi ) 2 f 00 (xi ) 3 h + h 2! 3! xi f (4)(xi ) 5 f (5)(xi ) 6 f (6)(ξi ) 7 + h + h + h 5! 6! 7! (6.14) =

Z

xi+1

f(t)dt = f(xi )h +

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 179

for certain ξi ∈ (xi , xi+1 ). Also, we have f 0 (xi) 2 f 00 (xi) 3 h − h 2! 3! xi−1 f 000(xi ) 4 f (4) (xi) 5 f (5) (xi ) 6 f (6) (ξi ) 7 h − h + h − h + 4! 5! 6! 7! (6.15) for certain ξi ∈ (xi−1 , xi ). Combining (6.14) and (6.15), we obtain the following formula: Z xi+1 h3 h5 h7 (6) f(x)dx = 2hf(xi )+ f 00 (xi )+ f (4) (xi )+ f (ζi ) 3 60 2520 xi−1 (6.16) for certain ζi ∈ (xi−1 , xi+1 ). Now, let us note that f 00 (xi) 2 h± f(xi±1 ) = f(xi ) ± f 0 (xi )h + 2! f 000(xi ) 3 f (4)(xi ) 4 f (5)(xi ) 5 f (6)(ηi ) 6 ± h ± h ± h + h, 3! 4! 5! 6! (6.17) for certain ηi ∈ (xi−1 , xi+1 ). Applying (6.17), we find J (xi−1) = −

Z

xi

f(t)dt = −f(xi )h +

f(xi−1 ) − 2f(xi ) + f(xi+1 ) = f 00 (xi ) + ei(f, h), (6.18) 2 h where the truncation error f (4) (xi ) 2 f (6) (ηi ) 4 h + h, ei(f, h) = 12 360 for certain ηi ∈ (xi−1 , xi+1 ). By (6.16) and (6.18), we have arrived at the simple Simpson rule Z xi+1 h f(x)dx = [f(xi−1) + 4f(xi ) + f(xi+1 )] 3 xi−1 h3 h5 − ei (f, h) + f (4) (xi ) + O(h7 ). 3 60

180 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Now, we can write the error of this rule in the following form: h5 (4) h5 f (xi ) + f (4) (xi ) + O(h7 ) 36 60 (4) f (xi ) 5 h + O(h7 ). =− 90

ES (f, h, xi ) = −

Then, the truncation error of the composed Simpson rule is: ES (f, h) = −

X h5 n−1 f (4)(x2i+1 ) + O(h6 ). 90 i=0

In order to determine the truncation error ER (S, f, h) of the method (6.12), we note that n−2

2 32h5 X f (4) (x4i+2) + O(h6 ) ES (f, 2h) = − 90 i=0

and ER (S, f, h) =

16ES (f, h) − ES (f, 2h) 15 n−2

2 X 32h5 X 16h5 n−1 f (4)(x2i+1 ) + = − f (4) (x4i+2) 1350 i=0 1350 i=0 + O(h6 )

= −

16h5 {[f (4)(x1 ) − 2f (4) (x2) + f (4) (x3)] 1350

+ [f (4)(x5 ) − 2f (4)(x6 ) + f (4)(x7 )]+ + [f (4)(x2n−3 ) − 2f (4) (x2n−2 ) + f (4)(x2n−1 )]} + O(h6 ). By formula (6.18) f (4) (x4i+1)−2f (4) (x4i+2)+f (4) (x4i+3) = f (6)(x4i+2 )h2+O(h4 ).

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 181

Hence, we have 16h7 (6) {f (x2)+f (6) (x6)+· · ·+f (6) (x2n−2)}+O(h6 ). 1350 By the intermediate value theorem, there exists η such that ER (S, f, h) = −

n (6) b − a (6) f (η) = f (η). 2 4h Therefore, the truncation error of the method is: f (6) (x2)+f (6)(x6 )+· · ·+f (6)(x2n−2 ) =

4h6 (b − a) + O(h6 ) = O(h6 ). 1350 Example 6.5 Evaluate the integral ER (S, f, h) = −

Z

0

2

ln(1 + x)dx

by the modified method (6.12) using h = 0.5. Estimate the truncation error ER (S, f, h). Solution. We shall apply formula (6.12). Then, for h = 0.5, b − a = 2, we have 2n = 4 and h [f(x0 ) + 4f(x1 ) + 2f(x2 ) + 4f(x3 ) + f(x4 )] 3 = 1.295322,

Sh (f) =

2h [f(x0 ) + 4f(x2 ) + f(x4 )] = 1.2904. 3 Hence, by formula (6.12) S2h (f) =

16Sh (f) − S2h (f) = 1.29565. 15 The truncation error ER (S, f, h) = O(h6 ) ≈ 0.015625. In fact, the result Rh (S, F ) = 1.29565 is much more accurate, since the error constant in the O(h6 ) asymptotic estimate of the error ER (S, f, h) is small. Indeed, we have Rh (S, f) =

I(f) =

Z

0

2

ln(1 + x)dx = 3ln(3) − 2 = 1.295837

182 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Hence, the absolute error I(f) − Rh (S, f) = 1.295837 − 1.29565 = 0.000187.

6.6

Gauss Methods

Introduction. In general, numerical methods of integration have the following form: Z

b

a

f(x)dx = C0n f(x0 ) + C1n f(x1 ) + · · · + Cnn f(xn ) + E(f),

(6.19) where the coefficients are independent of f and the error of the method E(f) = E(f, x0, x1, . . . , xn ) depends on f and of a distribution of the knots x0 , x1, . . . , xn , in the interval [a, b]. For example, the trapezian rule has the coefficients  1   h if i = 0, n, n 2 Ci =   h if i = 1, 2, . . . , n − 1, C0n , C1n , . . . , Cnn;

and the error of the method

h2 (b − a)f 00(η) 12 for certain η ∈ (a, b), where h = xi+1 − xi , i = 0, 1, . . . , n− 1. Let us observe that Newton-Cotes methods, based on an interpolating polynomial Pn (x) spaned by knots x0 , x1, . . . , xn , are exact (E(f) = 0), when E(f, h) = −

f(x) = a0 + a1x + a2x2 + · · · + am xm

is a polynomial of degree m ≤ n. Indeed, in this case f (n+1) (ξ(x)) = 0, if f(x) is a polynomial of degree m ≤ n. Then, the truncation error of NewtonCotes methods is equal to zero, that is 1 E(f) = n!

Z

a

b

f (n+1) (ξ(x))(x − x0 )x − x1 ) · · · (x − xn ) = 0.

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 183

Gauss methods are built on the principle to be the truncation error equal to zero (E(f) = 0) for any polynomial f of degree m ≤ 2n + 1.

6.7

Gauss-Legendre Knots

The points x0, x1, . . . , xn are called Gauss knots if the formula (6.19) of numerical integration produces exact result for any polynomial f(x) = a0 + a1x + a2 x2 + · · · + am xm of degree m ≤ 2n + 1. We can find Gauss knots applying the following theorem (cf. [2,3,11]): Theorem 6.1 For every polynomial fm (x) of degree m ≤ 2n + 1 the truncation error of a numerical method E(f) = 0

(6.20)

if and only if Z

b

a

ωn+1 (x)gm (x)dx = 0

(6.21)

for each polynomial gm (x) of degree m ≤ n, where ωn+1 (x) = (x − x0)(x − x1) · · · (x − xn ). Proof. At first, let us assume that E(f) = 0 for any polynomial fm (x) of degree m ≤ 2n + 1. In particular, fm (x) = ωn+1 (x)gk (x) is a polynomial of degree m ≤ 2n + 1 provided that gk (x) is an polynomial of degree k ≤ n. By the assumption, E(fm ) = 0. Evidently ωn+1 (xi ) = 0, for i = 0, 1, . . . , n.

184 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Then, we have Z

b

Z

fm (x)dx =

a

b

ωn+1 (x)gm (x)dx a C0n ωn+1 (x0)gk (x0) +

=

C1n ωn+1 (x1 )gk (x1 ) + · · ·

+Cnn ωn+1 (xn )gk (xn ) = 0. Now, let us assume that Z

b

a

ωn+1 (x)gk (x)dx = 0

(6.22)

for each polynomial gk (x) of degree k ≤ n Then, the truncation error E(fm ) = 0 for every polynomial fm (x) of degree m ≤ 2n + 1. Indeed, we can write the polynomial fm (x) in the following form: fm (x) = ωn+1 (x)gk (x) + rs (x), where gk (x) and rs (x) are polynomials of degree k, s ≤ n. Then fm (xi ) = rs (xi), i = 0, 1, . . . , n. By assumption (6.22) Z

b

fm (x)dx =

a

Z

b

a

ωn+1 (x)gk (x)dx+

Z

b

a

rs (x)dx =

Z

b

a

rs (x)dx. (6.23)

Since, rs (x) is apolynomial of degree s ≤ n, we get Z

b

rs (x)dx = C0n rs (x0) + C1n rs (x1 ) + · · · + Cnnrs (xn ). (6.24)

a

From (6.23) and (6.24) Z

b

a

fm (x)dx = C0nfm (x0) + C1n fm (x1) + · · · + Cnn fm (xn ).

and the truncation error E(fm ) = 0, when fm (x) is a polynomial of degree m ≤ 2n + 1. This ends the proof. Applying the above theorem, we can determine Gauss knots

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 185

x0, x1 , . . . , xn for each integer n. Namely, let us introduce the following sequence of polynomials: φ1 (x) = φ2 (x) =

Rx a

Rx a

ωn+1 (t)dt φ1 (t)dt

R

Also, we have

φ2 (x) = ax φ1 (t)dt ·················· · · · · · · · · · · ·R· · · · · · · φn+1 (x) = ax φn (t)dt Z

b

a

(6.25)

ωn+1 (x)gk (x)dx = 0

for every polynomial gk (x) of degree k ≤ n, (see (6.21)). On the other hand, integrating by parts, we obtain Z

b

a

ωn+1 (x)gk (x)dx = [φ1(x)gk (x) − φ2 (x)gk (x) + · · · +(−1)n+1 φn+1 (x)gk (x)]x=b x=a = 0

for every polynomial gk (x) of degree k ≤ n. Therefore φi (a) = 0

and φi (b) = 0,

i = 0, 1, . . . , n + 1.

This means that the polynomial φn+1 (x) possesses the roots x = a and x = b of multiplicity n + 1, i.e. φn+1 (x) = K (x − a)n+1 (x − bn+1 , where K is a constant. Following the definition (see (6.25)) of φi (x), i = 0, 1, . . . , n+ 1, we get ωn+1 (x) = K

dn+1 (x − a)n+1 (x − b)n+1 . d xn+1

186 Numerical Analysis with Mathematica

Hence, for K =

Krystyna STYŠ & Tadeusz STYŠ

(n + 1)! , we arrive at Legendre polynomial (2n + 2)!

ωn+1 (x) =

(n + 1)! dn+1 (x − a)n+1 (x − b)n+1 . (2n + 2)! d xn+1

The Legendre polynomial ωn+1 (x) has all roots real and distinct in the interval [a, b]. This can be proved by applying Rolle theorem to the polynomials φn+1 (x), φn (x), . . . , φ1(x) and ωn+1 (x). Also, let us note that the roots of Lagrange polynomial ωn+1 (x) are symmetrically distributed about the a+b mid-point d= of the interval [a, b]. Since, for x = y +d, 2 (n + 1)! (x − a)n+1 (x − b)n+1 (2n + 2)! (n + 1)! = (y + d)n+1 (y − d)n+1 (2n + 2)! (n + 1)! 2 = (y − d2 )n+1 , (2n + 2)!

φn+1 (x) =

and ωn+1 (y + d) =

(n + 1)! dn+1 2 (y − d2 )n+1 . 2n + 2)! dxn+1

So, Gauss-Legendre methods are associated with the knots a ≤ x0 < x1 , · · · < xn ≤ b, which are roots of Legendre polynomial ωn+1 (x). Example 6.6 Find Gauss knots in the interval [−1, 1], when n = 1 or n = 2. Solution. For n = 1 ω2 (x) =

2! d2 2 1 (x − 1)2 = x2 − = 0 2 4! dx 3

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 187

1 1 Hence, Gauss knots x0 = − √ and x1 = √ 3 3 For n = 2, we determine ω3 (x) =

3! d3 2 3 (x − 1)3 = x(x2 − ) = 0. 3 6! dx 5

So that, the Gauss knots are: x0 = − x2 =

6.8

s

s

3 , x1 = 0, and 5

3 . 5

Gauss Formulas

In order to determine the coefficients C0n , C1n, . . . , Cnn of a Gauss formula, let us consider the following polynomials for k=0,1,. . . ,n: lkn (x) = (x−x0)(x−x1) · · · (x−xk−1 )(x−xk+1 ) · · · (x−xn ), of degree n with the roots x0 , x1, . . . , xk−1 , xk+1, . . . xn which are roots of Legendre polynomial ωn (x). 2 Because lkn (x) is a polynomial of degree 2n, Gauss formula spaned on n + 1 knots produces exact value of the integral Z b

a

Z

b

a

2 2 lkn (x)dx, i.e. the truncation error E(lkn ) = 0 and

2 2 2 2 2 lkn (x)dx = C0n lkn (x0)+C1n lkn (x1 )+· · ·+Cnn lkn (xn ) = Ckn lkn (xk ).

Hence, we obtain the following formula for Gauss coefficients: Rb 2 l (x)dx n , k = 0, 1, . . . , n. Ck = a 2kn lkn (xk ) We can express the coefficients Ckn , k = 0, 1, . . . , n; in the other terms. Namely, let us observe that Z

a

b

0 (x−xk )lkn (x)lkn (x)dx =

n X i=0

0 Cin (xi −xk )lkn (xi)lkn (xi) = 0,

188 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

0 since (x − xk )lkn (x)lkn (x) is a polynomial of degree 2n. Then, we have 2 Ckn lkn (xk ) =

Z

a

b

2 lkn (x)dx

= (x −

2 xk )lkn (x) |ba

= (b −

2 xk )lkn (b)

−2

Z

b a

0

(x − xk )lkn (x)lkn (x)dx.

2 − (a − xk )lkn (a).

(6.26)

Also, we note that 2 2 (b − xk )lkn (b) − (a − xk )lkn (a) =

By the Leibniz formula

2 2 ωn+1 (b) ωn+1 (a) − . (6.27) b − xk a − xk

(n + 1)! dn+1 (x − a)n+1 (x − b)n+1 (2n + 2)! dxn+1 X (n + 1)! n+1 (n + 1)! di (x − a)n+1 dn−i+1 (x − b)n−i+1 = . (2n + 2)! i=0 i!(n − i + 1)! dxi dxn−i+1 (6.28) Hence, we find ωn+1 (x)=

[(n + 1)!]2 (a − b)n+1 ωn+1 (a) = (2n + 2)! ωn+1 (b) =

[(n + 1)!] (b − a)n+1 . (2n + 2)!

and 2 2 (a) ωn+1 (b) ωn+1 − = b − xk a − xk

[(n + 1)!]4 1 1 (b − xk )2n+2 ( − ) 2 [(2n + 2)!] b − xk a − xk [(n + 1)!]4 (b − a)2n+3 = . [(2n + 2)!] (a − xk )(b − xk ) (6.29) =

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 189

Combining (6.26) and (6.29), we obtain the following formula for Gauss coefficients: [(n + 1)!]4(b − a)2n+3 Ckn = , k = 0, 1, . . . , n. 0 [(2n + 2)!]2(xk − a)(b − xk )(ωn+1 (xk ))2 (6.30) One can check that Gauss coefficients are positive and satisfy the following conditions of symmetry: n Ckn = Cn−k ,

6.9

k = 0, 1, . . . , n.

(6.31)

Error of Gauss Methods

Let H2n+1 (x) be Hermite interpolating polynomial satisfying the following conditions: 0 H2n+1 (xi ) = f(xi ), H2n+1 (xi ) = f 0 (xi ),

i = 0, 1, . . . , n.

If f(x) is (2n + 2) times continuously differentiable function in the interval [a, b] then by the theorem 2.3 f(x) = H2n+1 (x) +

f (2n+2) (η) Ω2n+2 (x), (2n + 1)!

(6.32)

for certain η ∈ (a, b), where

Ω2n+2 (x) = (x − x0 )2(x − x1)2 · · · (x − xn )2.

and x0 , x1, . . . , xn are roots of the Legendre polynomial ωn+1 (x), (see (6.27)). Integrating both hand sides of the formula (6.32), we obtain Z

a

b

f(x)dx = +

Z

b

H2n+1 (x)dx +

a

Z

a

b

f

(2n+2)

1 (2n + 2)!

(6.33)

(η(x))Ω2n+2 (x)dx.

On other hand, by Gauss formula Z

b

a

f(x)dx = C0n f(x0 ) + C1n f(x1 ) + · · · + Cnn f(xn ) + E(f),

(6.34)

190 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

where Gauss coefficients Cin , i = 0, 1, . . . , n are given by the formula (6.30) and the error of Gauss method 1 E(f) = (2n + 1)!

Z

b

a

f (2n+2) (η(x))Ω2n+2 (x)dx.

Since Ω2n+2 (x) ≥ 0, x ∈ [a, b], by mean value theorem, we get Z

b

a

f

(2n+1)

(η(x))Ω2n+2(x)dx = f

(2n+2)

(ξ)

Z

b a

Ω2n+2 (x)dx.

for certain ξ ∈ (a, b). Hence, the truncation error is f (2n+2) (ξ) E(f) = (2n + 2)!

Z

b

Ω2n+2 (x)dx.

a

(6.35)

We can express the error E(f) in the other form using the formula of integration by parts. Namely, we have Z

a

b

Ω2n+2 (x)dx = − =

Z

Z

b

a

b

a

0 φ1(x)ωn+1 (x)dx

00 φ2(x)ωn+1 (x)dx

····················· = (−1)

n+1

Z

b

a

(n+1)

φn+1 (x)ωn+1 (x)dx

= (−1)n+1 (n + 1)!

Z

b

a

φn+1 (x)dx. (6.36)

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 191

and Z

Z

(n + 1)! b φn+1 (x)dx = (x − a)n+1 (x − b)n+1 dx 2n + 2)! a a Z (n + 1)!(n + 1) b (x − a)n+2 (x − b)n+1 =− dx 2n + 2)! n+2 a 2 Z b (x − a)2n+2 n+1 [(n + 1)!] dx = (−1) (2n + 2)! a (n + 2)(n + 3) · · · (2n + 2) [(n + 1)!]3 (b − a)2n+3 = (−1)n+1 . [(2n + 2)!]2 (2n + 3) (6.37) From (6.36) and (6.37), we obtain the following formula for the truncation error: [(n + 1)!]4(b − a)2n+3 (2n+2) E(f) = f (ξ), (6.38) [(2n + 2)!]3(2n + 3) b

for certain ξ ∈ (a, b). Example 6.7 Let n = 0 and [a, b] = [−1, 1]. Then, there is only one Gauss knot x0 = 0, and then Gauss formula is: Z

where the error

1

−1

f(x)dx = 2f(0) + E(f),

1 E(f) = f 00 (ξ) 3 for certain ξ ∈ (−1, 1). Example 6.8 Let n = 1 and [a, b] = [−1, 1]. Then, there 1 1 are two Gauss knots x0 = − √ , x1 = √ and then Gauss 3 3 formula is: Z 1 1 1 f(x)dx = f(− √ ) + f( √ ) + E(f), −1 3 3 where the error 1 (4) E(f) = f (ξ) 135

192 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

for certain ξ ∈ (−1, 1). Example 6.9 Let n = 2 andq[a, b] = [−1, 1]. Then, q there 3 are three Gauss knots x0 = − 5 , x1 = 0 and x2 = 35 and then Gauss formula is: Z

s

s

5 3 8 5 3 f(x)dx = f(− ) + f(0) + f( ), 9 5 9 9 5 −1 1

where the error E(f) =

1 f (6)(ξ) 15750

for certain ξ ∈ (−1, 1). In order to apply Gauss method to an interval [a, b], at first, we should transform this interval to the interval [−1, 1] using the following linear transform: Z

b a

b−a f(x)dx = 2

Z

1

−1

F (y)dy,

where y=

1 2x − a − b , F (y) = f( ((b − a)y + a + b)), b−a 2

Example 6.10 Evaluate the integral Z

0

2

ln(1 + x)dx

(6.39)

by Gauss formula based on four knotes. Estimate the error of the method. Solution. At first, let us transform the integral using the linear mapping y = x − 1. Then Z 2 Z 1 ln(1 + x)dx = ln(2 + y)dy. 0

−1

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 193

In order to get Gauss knotes, when n = 3, we shall find the roots of Legendre polynomial ω4 (y) =

4! d4 (y 2 − 1)4 6 3 = y4 − y2 + =0 4 8! dy 7 35

Hence, the roots are:

x0 = −

v u u6 t

x1 = −

v u u6 t

7

7

x2 =

v u u6 t

x3 =

v u u6 t

7

7

s

96 = −0.8611363..., 245

−

s

96 = −0.3399811..., 245

−

s

96 = 0.3399811..., 245

s

96 = 0.8611363..., 245

+

+

Following the formula (6.30) and (6.38), we find Gauss coefficients C03 = C33 =

[4!]4 29 = 0.3478473..., 0 [8!]2 (x0 + 1)(1 − x0 )ω42 (x0)

C13 = C23 =

[4!]4 29 = 0.6521451..., 0 [8!]2 (x1 + 1)(1 − x1 )ω42 (x1)

Then, the truncation error is: E(f) =

1 [4!]4 29 (8) f (ξ) = f (8)(ξ). 2 [8!] 9 3472875

194 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Now, we have Z

0

Z

2

ln(1 + x)dx =

1

−1

ln(2 + y)dy

= 0.3478473 ln(2 − 0.8611363) + 0.6521451 ln(2 − 0.3399811) + 0.6521451 ln(2 + 0.3399811) + 0.348473 ln(2 + 0.8611363) + E(f) = 1.295837 + E(f),

where the truncation error 1 E(f) = f (8) (ξ) 3472875 = 0.0000002879

7! ≤ 0.0000002879 ∗ 5040 (2 + ξ)8

= 0.00145. In fact, the result is much more accurate, since the above error estimation is pessimistic one.

6.10

Exercises

Question 6.1 Consider the integral I= Evaluate the integral I by 1. the trapezian rule 2. Simpson rule

Z

0

2

2

e−x dx.

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 195

for the two steps h = 0.25 and h = 0.5. In the both cases, express the truncation errors ET (f, h) and ES (f, h) by the results Th (f), T2h(f), Sh (f) and S2h (f). Question 6.2 Let Sh (f) and S h (f) be two Simpson approx2 imate values of the integral I(f) =

Z

b

a

f(x)dx

h h evaluated for h and with the errors ES (f, h) and ES (f, ), 2 2 respectively. h 1. Express the errors ES (f, h) and ES (f, ) in terms of 2 the results Sh (f) and S2h (f). 2. Find a modified Simpson rule using the values Sh (f) and S h . 2

3. Evaluate the integral Z

0

2

√

1 + xdx

by the modified Simpson rule with the accuracy  = 0.001. Question 6.3 Derive simple and composed the rectangular rules RCh (f, xi), i = 0, 1, . . . , n; and RCh (f) based on the piecewise constant interpolating polynomial pn (x) = f(x0)θ0 (x) + f(x1 )θ1 (x) + · · · + f(xn−1 )θn−1 (x), where θi (x) is the characteristic function to the interval [xi, xi+1 ], i.e. θi(x) =

(

1 for xi ≤ x ≤ xi+1 , 0 for x < xi or x > xi+1 ,

for i = 0, 1, . . . , n − 1. Determine the truncation error ERC (f, h) of the rectangular

196 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

rule RCh (f). Question 6.4 Consider the simple Mid-Point method: Z xi+1 xi + xi+1 i f(x)dx = h f( ) + EM (f, h), 2 xi for i = 0, 1, ..., n − 1, nh = b − a, and the composed Mid-Point method: Z

a

b

f(x)dx = h

n−1 X i=0

f(

xi + xi+1 ) + EM (f, h). 2

i (f, h) and EM (f, h). 1. (a) Determine the errors EM (b) Find a modification of Mid-Point method based on Richardson extrapolation to get a new method of accuracy O(h4 ). (c) Use Mid-Point method and the modified Mid-Point method to evaluate the integral: Z 2 x dx, 0 1+x with accuracy  = 0.001.

Question 6.5 Build the Romberg’s table RT (0, 0) RT (1, 0) RT (1, 1) RT (2, 0) RT (2, 1) RT (2, 2) RT (3, 0) RT (3, 1) RT (3, 2) RT (3, 3) 1 , 0 ≤ x ≤ 4. 1+x Give an estimate of the error Z 4 1 dx − RT (2, 2). 0 1+x for the function f(x) =

Selected Methods for Numerical Integration

Numerical Analysis with Mathematica 197

Question 6.6 Evaluate the integral Z

0

3

e−2x dx

by Gauss method spaned by four Gauss-Legendre knots. Estimate the truncation error. Question 6.7 Find a Gauss method spaned by five Gauss-Legendre knots. Use the method to evaluate the integral Z 3 √ x. 1

Estimate the truncation error.

Question 6.8 Use Gauss-Legendre integrating formula based on three points to evaluate the integral: 1. (a)

Z

1

(b)

Z

π 2

0

xe−xdx.

− π2

cos3 (x)dx

Estimate the truncation error.

198 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Send Orders for Reprints to [email protected] Lecture Notes in Numerical Analysis with Mathematica, 2014, 199-229

199

CHAPTER 7

Solving Nonlinear Equations by Iterative Methods Abstract In this chapter the equation F (x) = 0, a ≤ x ≤ b, is solved by the Fix Point Iterations, Newton’s Method, Secant Method and Bisection Method. The theorems on convergence and errors estimates of the methods have been stated and proved. Also, the rates of convergence of the iterative methods are determined . The methods are illustrated by a number of selected examples. The chapter ends with a set of questions. Keywords: Fix point iterations, Newton’s method, Secant method, Bisection method. Introduction In this chapter, we shall solve the equation (cf. [3,22,24,25]) F (x) = 0,

a ≤ x ≤ b,

by the following iterative methods: • Fixed point iterations, • Newton’s method, Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

(7.1)

200 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

• Secant method,

• Bisection method.

The rate of convergence of these iterative methods is determined by the order which is defined as follows: Definition 7.1 Let {xn }, n = 0, 1, . . . ; be a sequence convergent to a root α of the equation F (x) = 0, x ∈ [a, b]. An iterative method is said to be of order p if there exists a constant C = 6 0 such that lim

n→∞

| α − xn+1 | = C. | α − x n |p

To illustrate the definition, let us consider the following equation: x = f(x), a ≤ x ≤ b,

where f is a given differentiable function and its derivative 0 f (x) 6= 0 in the interval (a, b). Suppose that the sequence xn+1 = f(xn ),

n = 0, 1, . . . ;

(7.2)

is convergent to the root α ∈ [a, b], for any choice of the starting point x0 ∈ [a, b]. Obviously, we have α − xn+1 = f(α) − f(xn ) = f 0 (ξn )(α − xn ) for certain ξn between α and xn . Hence α − xn+1 = f 0 (ξn ) α − xn and | α − xn+1 | = f 0 (α). lim n→∞ | α − x | n

Therefore, iteration method (7.2) has of order p = 1, since f 0 (x) 6= 0 in the interval (a, b). Let us note that the order of the method is greater than one, if f 0 (α) = 0.

Solving Nonlinear Equations by Iterative Methods

7.1

Numerical Analysis with Mathematica 201

Fixed Point Iterations

In order to describe fixed point iterations, we write equation (7.1) in the following form: x = f(x),

a ≤ x ≤ b,

(7.3)

Then, f(x) is called iteration function. There are many ways to write equation (7.1) in the iterative form (7.3). We present this in the following example: Example 7.1 Consider the equation x2 − x − 4 = 0,

−∞ < x < ∞.

(7.4)

Write this equation in the iterative form (7.3). Obviously, equation (7.4) can be written in one of the following form: x = x2 − 4, f(x) = x2 − 4, −∞ < x < ∞, √ √ x = x + 4, f(x) = x + 4, x ≥ −4, 4 4 , f(x) = , x 6= 1, x= x−1 x−1 4 4 x = 1+ , f(x) = 1 + , x 6= 0. x x We can approximate a root of equation (7.3) by the sequence of successive iterations xn+1 = f(xn ),

n = 0, 1, . . . ,

where x0 is a starting value. For example, the equation 4x − e−x = 0,

−∞ < x < ∞,

has the iterative form 1 x = e−x , 4

−∞ < x < ∞.

202 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Then, the sequence 1 xn+1 = e−xn , n = 0, 1, ..., 4 converges to the root α = 0.2039... with an arbitrary starting value x0 . Indeed, choosing x0 = 0 as starting value, we calculate terms of the sequence x1 = 0, 25 x2 = 0.1947 x3 = 0.2058 x4 = 0.2035 x5 = 0.2040 x6 = 0.2039 x7 = 0.2039 ········· We can obtain the fixed point sequence by the following instructions: f[x_]:=Exp[-x]/4; N[NestList[f,0,7],4] or f[x_]:=Exp[-x]/4; N[FixedPointList[f,0,7],4] Let us observe that the above sequence converges to the root α = 0.2039... of the equation x = 41 e−x , so that 1 α = e−α . 4 In general, if there exists an α ∈ [a, b] such that α = f(α),

then α is called the fixed point of the function f. The fixed point iteration sequence xn+1 = f(xn ), n = 0, 1, . . . , converges to α under assumptions specified in the following fixed point theorem:

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 203

Theorem 7.1 Assume that the iteration function f satisfies the following conditions: 1. f is a continuous on the closed interval [a, b] and continuously differentiable in the open interval (a, b), 2. f maps the interval [a, b] into [a, b], i.e., f : [a, b] → [a, b]. 3. there exists a constant K < 1 such that | f 0 (x) |≤ K for all x ∈ (a, b), Then the iteration function f has a unique fixed point α ∈ [a, b], and α is the limit of the fixed point sequence xn+1 = f(xn ),

n = 0, 1, . . . ;

The limit α is independent of the starting value x0, so that lim xn = α

n→∞

for any x0 ∈ [a, b]. Proof. We shall present the proof of this theorem according to the following plan: • First, we shall show that there exists a fixed point of f in the interval [a, b], • Secondly, we shall show that there is no more than one fixed point in the interval [a, b], • Thirdly, we shall show that the fixed point sequence xn+1 = f(xn ),

n = 0, 1, . . . ;

converges to the unique fixed point α. Clearly, if f(a) = a or f(b) = b, then f has a fixed point in [a, b]. Otherwise, let f(a) = 6 a and f(b) 6= b. By assumption 2, we note that f(a) ∈ [a, b] and f(b) ∈ [a, b]. Therefore f(a) > a

and

f(b) < b.

204 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Then, the function h(x) = f(x) − x,

a ≤ x ≤ b,

is positive at x = a and negative at x = b, i.e., h(a) > 0

and

h(b) < 0.

Since h is a continuous function and changes its sign in [a, b], we conclude, by the intermediate-value theorem, that h(x) must vanish somewhere in (a, b). Therefore, f has at least one fixed point in the interval [a, b]. In order to prove that there is one fixed point of f in [a, b], let us assume that there are two different fixed points α and β such that α = f(α),

β = f(β),

α= 6 β.

(7.5)

Then, by the mean value theorem f(α) − f(β) = f 0 (ξ)(α − β) for certain ξ ∈ (a, b). By (7.5), we get α − β = f 0 (ξ)(α − β). Hence, the equality f 0 (ξ) = 1. contradicts assumption 3. Therefore, f has a unique fixed point in the interval [a, b]. In order to prove that the fixed point iteration sequence xn+1 = f(xn ), n = 0, 1, . . . ; is convergent to α, let us consider the error of n-th iteration en = α − xn . Then, we have en+1 = α − xn+1 = f(α) − f(xn ) = f 0 (ξ)(α − xn ) = f 0 (ξ)en

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 205

for certain ξ ∈ (a, b). Hence, by assumption 3, | en+1 |≤ K | en |,

n = 0, 1, . . . ;

As repeating application of this inequality, we obtain | en+1 |≤ K | en |≤ K 2 | en−1 |≤ · · · ≤ K n+1 | e0 | . So that | α − xn |≤ K n | α − x0 | . Since 0 ≤ K < 1, we get

(7.6)

lim K n = 0

n→∞

Hence, by (7.6) lim xn = α.

n→∞

This ends the proof. In order to terminate the fixed point iteration sequence {xn }, n = 0, 1, . . . , we can use the following condition: | xm+1 − xm |≤ , where m is the least integer for which the above condition hold, and  is a given accuracy of the root α. Let as note that, the fixed point iterative method has order of conver0 gence p = 1, if f (x) = 6 0 for x ∈ (a, b), (see (7.2). Example 7.2 Consider the following equation: 1 , x ∈ [0, 1] (7.7) 4 + ax (a). Show that the iteration function satisfies the assumptions of the fixed point theorem for each 0 ≤ a < 16. (b). Compute the root of equation (7.7) for a = 2 with accuracy  = 0.000001. x=

Solution (a) The iteration function f(x) =

1 4 + ax

206 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

is continuous for all x ∈ [0, 1] and a ∈ [0, 16), since then 4 + ax ≥ 4. Thus, assumption 1 is satisfied. For assumption 2, we have 1 1 0≤ ≤ , when 0 ≤ x ≤ 1, and 0 ≤ a < 16. 4 + ax 4 Hence f : [0, 1] → [0, 1] and assumption 2 holds, too. Finally, assumption 3 is also satisfied, since we have a a | f 0 (x) |= ≤ (Abs[#1-#2] 0 and F 00(x) > 0, x ∈ [a, b]. In this case, F (x) is an increasing and convex function. 1

is a given accuracy to calculate α

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 209

6

a

α

s r





x2





r

  

 

  

x1



b r

-

x0

Fig. 7.1. Case II : F 0(x) < 0 and F 00(x) > 0, x ∈ [a, b]. In this, case F (x) is a decreasing and convex function. 6

C C C

a x0

r

C C C

C C

A C Cr

x1

A

A

AA r

x2

b s

α

-

Fig. 7.2. Case III : F 0(x) > 0 and F 00(x) < 0, x ∈ [a, b]. In this case, F (x) is an increasing and concave function.

210 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

6

x1 x2

x0 r

r 

a





r

 



α

-

s

b

Fig. 7.3. Case IV : F 0(x) < 0 and F 00(x) < 0, x ∈ [a, b]. In this case, F (x) is a decreasing and concave function.

6

r

a

α

r

rx2 @ @

x1r B @

x r0 b

B

@ B

B B

B

Fig. 7.4. Obviously, we can choose the starting value x0 in such a way that F (x0)F 0(x0) > 0. Now, let us note that in cases I and IV when, either both F 0(x) and F 00(x) are positive or both

-

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 211

F 0(x) and F 00(x) are negative, the first approximation x1 of root α is between α and x0. Indeed, we have x1 = x0 −

F (x0) < x0 . F 0(x0)

Since 0 = F (α) = F (x0) + F 0(x0 )(α − x0) +

F 00(ξ) (α − x0)2 (7.9) 2

for certain ξ between x0 and α, we get α − x1 = α − x0 +

F (x0) F 00(ξ) = − (α − x0)2 < 0. F 0(x0 ) 2F 0(x0 )

Therefore, we have the inequality α < x1 < x0 . In a similar way, we can show that successive Newton approximations satisfy the inequalities α < x2 < x1 , α < x3 < x2 , ............ α < xn+1 < xn , ............ From these inequalities, we note that b ≥ x0 > x1 > x2 > · · · > xn > xn+1 > · · · > α. This means that the sequence {xn }, n = 0, 1, . . . ; is decreasing and lower bounded. Therefore, xn converges to a limit g ≥ α, i.e., xn → g when n → ∞. In fact, g = α, since F (g) = lim F (xn ) = lim (xn −xn+1 )F 0(xn ) = (g−g)F 0(g) = 0 n→∞

n→∞

and α is a unique root of equation (7.8). Now, let us choose x0 such that F (x0)F 0(x0) < 0 (for instance x0 = a). In the cases II or III, when either F 0(x) < 0

212 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

and F 00(x) > 0 or F 0(x) > 0 and F 00(x) < 0, the first approximation x1 lies between a and x0. Indeed, we have x1 = x0 −

F (x0) > x0 . F 0(x0)

and F (x0) F 00(ξ) α − x1 = α − x0 + 0 =− 0 > 0, F (x0) 2F (x0) for certain ξ ∈ (a, b). Hence a ≥ x0 < x1 < α.

Likewise, we can show, that the next approximations lie also between x0 and α and consequently, we note that a ≤ x0 < x1 < · · · < xn < xn+1 < · · · < α. Thus, in cases II and III, the sequence {xn }, n = 0, 1, . . . ; is increasing and upper bounded. Therefore, xn → α when n → ∞. Let us note that the iteration function of Newton’s method is F (x) f(x) = x − 0 , F (x) so that, the following equations F (x) = 0

and

x = f(x),

a ≤ x ≤ b,

are equivalent. Clearly the first derivative 0

f (α) =

F (α)F 00(α) = 0. [F 0(α)]2

This indicates the order of Newton’s method p ≥ 2. Indeed, by formula (7.9)), we have α − xn+1 = α − xn +

F (xn) F 00(ξ) = − (α − xn )2 . F 0(xn ) 2F 0(xn )

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 213

Then

α − xn+1 F 00(ξn ) = (α − xn )2 2F 0(xn ) for certain ξn between α and xn . Hence

(7.10)

00

|α − xn+1 | |F 00(ξn )| F (α)| lim = lim = 6= 0. n→∞ |α − x |2 n→∞ 2|F 0 (x )| 2|F 0 (α)| n n Therefore, the order of Newton’s method p = 2 since F 00(α) 6= 0 . Now, we can state the following theorem: Theorem 7.2 If the conditions 1, 2 and 3 are satisfied then Newton’s method is convergent with order p = 2 to a unique root α for any choice of the starting point x0 ∈ [a, b]. Example 7.3 Write Newton’s algorithm to evaluate square √ root c for any c > 0. Let F√(x) = x2 − c for x > 0. Then, we can find the square root c solving the equation x2 − c = 0,

x > 0, c > 0.

Obviously, F (a) = −c < 0, for a = 0 and F (b) = b2 −c > 0, for b > c, that is F (a)F (b) < 0. Also, we have F 0(x) = 2x > 0, x > 0, and F 00(x) = 2 > 0. Thus, the function F (x) satisfies conditions 1, 2 and 3 when x > 0. Therefore, by the theorem, Newton sequence 1 c x2n − c = (xn + ), n = 0, 1, . . . ; 2xn 2 xn √ converges quadratically to the root c = α, when n → ∞ for any starting value x0 > 0. xn+1 = xn −

214 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

√ Applying the above algorithm to find 3 with the accuracy of four digits after decimal point, we obtain x0 = 1 1 3 x1 = (x0 + ) = 2 2 x0 1 3 x2 = (x1 + ) = 1.75 2 x1 1 3 x3 = (x2 + ) = 1.7321 2 x2 1 3 x4 = (x3 + ) = 1.73205 2 x3 1 3 x5 = (x4 + ) = 1.73205 2 x4 We can repeat the above calculation in Mathematica by executing the following recursive formula sol[1]=1.; sol[n_]:=sol[n]=(sol[n-1]+3/sol[n-1])/2; Table[sol[n],{n,1,5}];

7.3

Secant Method

The secant method is a simple modification of Newton’s method. Namely, let us replace first derivative F 0(xn ) which appears in Newton’s method by the divided difference F 0(xn ) ≈

F (xn) − F (xn−1 ) . xn − xn−1

Then, we obtain the secant method xn+1 = xn − and its equivalent form xn+1 =

F (xn )(xn − xn−1 ) F (xn ) − F (xn−1)

xn−1 F (xn ) − xn F (xn−1) , F (xn) − F (xn−1 )

(7.11)

(7.12)

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 215

for n = 1, 2, . . . ; In order to start with the secant method, we have to choose two initial values x0 and x1. Starting with these initial values, we can produce the sequence {xn }, n = 1, 2, . . . ; using formula (7.12). Let us note that to evaluate xn+1 , we can use the already known value F (xn−1 ) and the new value F (xn ) must be calculated. Thus, at each iteration F (x) is evaluated one time. But, using Newton’s method both values F (xn) and F 0(xn ) must be evaluated at each iteration. If F 0(x) is expensive in terms of arithmetic operations, then Newton’s method may occur to be slower than the secant method when a computer is used, in spite of its higher order of convergence.

6

, % , % , , % , % , % , % , % x2 ,,x3 % x1 x0 r% s ,r % , α a b , % , % , % ,

Fig. 7.5

Convergence of the secant method. We assume that conditions 1, 2 and 3 are satisfied. Then, there is one root α of equation F (x) = 0 in the interval [a, b], and there are four possible cases I, II, III and IV (see figures 7.1, 7.2, 7.3, 7.4.) First, let us consider cases I and IV, when either F 0(x) > 0 and F 00(x) > 0 or F 0(x) < 0 and F 00(x) < 0 in the interval

-

216 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

[a, b]. So, we can choose x0 < x1 such that F (x0) < 0 and F (x1) > 0 in case I or F (x0) > 0 and F (x1) < 0 in case IV, (for instance x0 = a and x1 = b). Now, we shall show that the sequence {xn }, n = 2, 3, . . . ; produced by the secant method (see (7.11) and (Fig. 7.5), converges to the root α, i.e., xn → α, when n → ∞. In cases I and IV, the following inequalities hold: x0 < x2 < x3 < x5 < · · · < α < · · · < x10 < x7 < x4 < x1 (7.13) Thus, every third term x3k+1, k = 0, 1, . . . ; of the sequence {xn }, n = 0, 1, . . . ; lies on the right hand side of the root α. We shall prove that x0 < x2 < x3 < α < x4 < x1 .

(7.14)

We note that x2 = x1 − and

F (x1)(x1 − x0) F (x1) = x1 − 0 < x1 F (x1) − F (x0) F (ξ0 )

F (x1)(x1 − x0) F (x1) − F (x0) F (x0)(x1 − x0) F (x0) = x0 − = x0 − 0 > x0 , F (x1) − F (x0) F (ξ0 )

x2 = x0 + (x1 − x0 ) −

for certain ξ0 ∈ (x0, x1 ). Hence x0 < x2 < x1 . By the assumption, F (x) is a convex (concave) function in the interval [a, b]. Therefore F (x) < F (x0) + or F (x) > F (x0) +

F (x1) − F (x0) (x − x0), when F 00(x) > 0, x1 − x0 F (x1) − F (x0) (x − x0), when F 00(x) < 0 x1 − x0

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 217

for x0 < x < x1. Let x = x2. Then F (x2) < F (x0) + or

F (x1) − F (x0) (x2 − x0 ) = 0, x1 − x0

F (x1) − F (x0) (x2 − x0 ) = 0. x1 − x0 This means that x2 < α in cases I and IV, so that F (x2) > F (x0) +

x0 < x2 < α.

(7.15)

Now, we shall show that x2 < x3 < α. Indeed, we have x3 = x2 − and

F (x2)(x2 − x1) F (x2) = x2 − 0 > x2 F (x2) − F (x1) F (ξ1 )

x3 = x1 + (x2 − x1) −

F (x1) F (x2)(x2 − x1) = x1 − 0 < x1 F (x2) − F (x1) F (ξ1 )

for certain ξ1 ∈ (x2, x1 ). Hence x2 < x3 < x1 . Again, in case I F (x) < F (x1) + and in case IV F (x) > F (x1) +

F (x2) − F (x1) (x − x1 ) x2 − x1 F (x2) − F (x1) (x − x1 ) x2 − x1

for x2 < x < x1. Substituting x = x3 to these inequalities, we obtain F (x3) < F (x1) +

F (x2) − F (x1) (x3 − x1 ) = 0. x2 − x1

(7.16)

218 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

or F (x3) > F (x1) +

F (x2) − F (x1) (x3 − x1 ) = 0. x2 − x1

Hence, x3 < α. By (7.15) and (7.16), we have x0 < x2 < x3 < α < x1 .

To complete the proof of inequality (7.14), we shall show that α < x4 < x1 . Let us note that F (x3)(x3 − x2) F (x3) x4 = x3 − = x3 − 0 > x3 F (x3) − F (x2) F (ξ1 )

and

F (x) > F (x2) +

F (x3) − F (x2) (x − x2 ) x3 − x2

for x < x2 or x > x3 . Hence, for x = x4 > x3 , we have F (x4) > F (x2) +

F (x3) − F (x2) (x4 − x2 ) = 0. x3 − x2

Since F (x4) > 0, α < x4 . Clearly, x4 < x1. This completes the proof of inequality (7.14). Replacing x0 by x3 and x1 by x4 , we can repeat the above estimates to get the following inequality: x3 < x5 < x6 < α < x7 < x4 .

(7.17)

Next, we may replace x3 by x6 and x4 by x7, to obtain x6 < x8 < x9 < α < x10 < x7

(7.18)

Finally, by the repetition of the replacement x3k−3 by x3k and x3k−2 by x3k+1 for k = 1, 2, . . . ; , we can arrive at the inequalities x3k < x3k+2 < x3k+3 < α < x3k+4 < x3k+1 , k = 1, 2, . . . ; (7.19)

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 219

Hence, we obtain the inequality (7.13). Then, in the case I and IV, the sequence {xn }, n = 2, 3, . . . ; produced by the secant method is splited into two monotonic and bounded subsequences {xn }, when n mod 3 6= 0, and {xn }, when n mod 3 = 0, n = 0, 1, . . . ; Therefore, both subsequences are convergent. To complete the analysis of convergence of the second method, we should show that these subsequences are convergent to the same limit α. Namely, let lim xn = g. n→∞, n6=0mod3

From the formula F (xn)(xn − xn−1 ) F (xn) xn+1 = xn − = xn − 0 , ξn ∈ (xn−1 , xn ), F (xn) − F (xn−1 F (ξn )

it follows that

lim xn+1 = lim xn − lim

F (xn) . F 0(ξn )

Hence

F (g) and F (g) = 0. F 0(g) Since the equation F (x) = 0 has unique root α ∈ [a, b], we get g = α. In the same way, we can show that g=g−

lim

n→∞, n mod 3=0

xn = α

The proof of convergence of the sequence {xn }, n = 2, 3, . . . ; produced by the secant method, in cases II and III, is dealt within questions. Order of convergence of the secant method. Let us note that the equation of the secant line through the points (xn−1 , F (xn−1 ))

and

(xn , F (xn))

is the interpolating polynomial P1 (x) =

x − xn x − xn−1 F (xn−1 ) + F (xn ). xn−1 − xn xn − xn−1

220 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Therefore F 00(ξn ) (x − xn−1 )(x − xn ), F (x) = P1 (x) + 2!

ξn ∈ (xn−1 , xn ).

Hence 0 = F (α) = P1 (α) +

F 00(ξn ) (α − xn−1 )(α − xn ). 2!

Thus, we have F 00(ξn ) (α − xn−1 )(α − xn ). 2!

P1 (α) = −

(7.20)

On the other hand α − xn α − xn−1 F (xn−1 ) + F (xn ) xn−1 − xn xn − xn−1 F (xn ) − F (xn−1) xn F (xn−1 ) − xn−1 F (xn) +α . = xn − xn−1 xn − xn−1

P1 (α) =

By the mean value theorem F (xn ) − F (xn−1) = F 0(ηn ), xn − xn−1 for certain ηn ∈ (xn−1 , xn ). From formula (7.12), we get xn F (xn−1 ) − xn−1 F (xn) = −xn+1 F 0(ηn ). xn − xn−1 Hence P1 (α) = −xn+1 F 0(ηn )+αF 0 (ηn ) = (α−xn+1 )F 0(ηn ). (7.21) Combining (7.20) and (7.21), we obtain the following relation between the errors en+1 = α − xn+1 , en−1 = α − xn−1 and en = α − xn : en+1 =

F 00(ξn ) en−1 en 2F 0(ηn )

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 221

for n = 0, 1, . . . ; Now, we note the following: | en+1 | | F 00(ξn ) | | F 00(ξn ) | | en | β 1−p = | e | | e |= [ ] , n n−1 | e n |p 2 | F 0(ηn ) | 2 | F 0(ηn ) | | en−1 |p for β = 1 − p and βp = −1. Hence, we have (1 − p)p = −1 and p2 − p − 1 = 0.

√

The positive root of the above equation is : p = 1+2 5 . √ Choosing p = 1+2 5 and β = 1+2√5 , we obtain the following equation: yn+1 = an yn , where | F 00(ξn ) | | en+1 | | en | an = , yn+1 = , yn = . 0 p 2 | F (ηn ) | | en | | en−1 |p Then, yn → γ, when n → ∞, where γ is the fixed point of the equation 1 x = ax− p , a = lim an . Let us note that 1 | F 00(α) | 1p γ = ap = [ ] . 2 | F 0(α) | This shows that the error en satisfies the following: | en+1 | | F 00(α) | p1 lim = [ ] = constant 6= 0, n→∞ | en |p 2 | F 0(α) | √ 1+ 5 . The secant method has order for p = 2 √ 1+ 5 p= ≈ 1.618. 2 In order to stop the iterations, we can use the same condition as in Newton’s method. Namely, we stop the secant iterations at the least m such that | xm+1 − xm |≤ .

222 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 7.4 Solve the equation e−x − 2x = 0,

0 ≤ x ≤ 1,

by the secant method with accuracy  = 0.01. Solution. The function F (x) = e−x − 2x, x ∈ [0, 1] satisfies assumptions 1, 2 and 3. Indeed, F (x) is infinitely times differentiable on the real line,. F (0)F (1) = 1 ∗ (e−1 − 2 < 0 and F 0(x) = −e−x − 2 < 0, F 00(x) = e−x > 0 for all x ∈ [0, 1]. In this example, the algorithm of the secant method is xn−1 (exp(−xn ) − xn ) − xn (exp(−xn−1 ) − xn−1 ) xn+1 = exp(−xn ) − 2xn − exp(−xn−1 ) − 2xn−1 for n = 1, 2, . . . ; where x0 and x1 are starting values. Let the staring values x0 = 0 and x1 = 1. Then, we have x2

=

0 ∗ exp(−1) − 1 = 0.3799218 exp(−1) − 3

x3

=

exp(−0.3799218) − 0.3799218 ∗ exp(−1) = 0.349667 exp(−0.3799218) − exp(−1) − 2(1 − 0.3799218)

x4

= =

0.3799218 ∗ exp(−0.349667) − 0.349667 ∗ exp(−0.3799218) exp(−0.349667) − exp(−0.3799218) + 2(0.3799218 − 0.349667) 0.3517

We reach the accurate at m = 3, since then | x4 − x3 |= 0.3517 − 0.3497 = 0.002.

Then, we consider α ≈ 0.3517 as the approximate value of the root. Also, we can find the approximate value of the root α by the following Mathematica program f[x_]:=Exp[-x]-2*x; sol[1]=0; sol[2]=1; sol[n_]:=sol[n]= N[(sol[n-2]*f[sol[n-1]]-sol[n-1]*f[sol[n-2]])/ (f[sol[n-1]]-f[sol[n-2]])]; Table[sol[n],{n,3,6}]

Solving Nonlinear Equations by Iterative Methods

7.4

Numerical Analysis with Mathematica 223

Bisection Method

Let F (x) be a continuous function in the interval [a, b] such that F (a)F (b) < 0. This assumption guarantees the existence of at least one root in the interval [a, b]. The idea of bisection method is based on the partition of the interval [a, b] into two equal subintervals. Then, we choose the subinterval, where the root is allocated. The chosen subinterval is divided into two equal subintervals again, and the new subinterval with the root α is to be divided. We continue this partition of the interval [a, b] as long as the chosen subinterval has the length smaller than a given accuracy  and it contains the root. Algorithm of the bisection method Set a0 = a, repeat

b0 = b, n = 0 a n + bn 2 if F (an )F (dn ) < 0

dn =

then set an+1 = an , bn+1 = dn else set an+1 = dn , bn+1 = bn , n := n + 1, until

b−a bn+1 − an+1 = n+1 ≤ . 2 2 Then, the approximate root is α≈

bn+1 + an+1 . 2

The bisection method has order p = 1.

224 Numerical Analysis with Mathematica

Krystyna STYŠ & Tadeusz STYŠ

Example 7.5 Find root of the equation e−x − 2x = 0,

0 ≤ x ≤ 1,

by the bisection method with accuracy  = 0.02. Solution. We note that F (x) = e−x − 2x is a continuous function and F (0)F (1) = 1 ∗ (e−1 − 2) < 0. Following the algorithm, we compute a0 = 0, b0 = 1,  = 0.02 a 0 + b0 0+1 = = 0.5, 2 2 F (a0)F (d0 ) = −0.39, a1 = 0, b1 = 0.5

for n = 0,

d0 =

0 + 0.5 a 1 + b1 = = 0.25 2 2 F (a1)F (d1 ) = 0.078, a2 = 0.25, b2 = 0.5 for n = 1,

d1 =

a 2 + b2 0.25 + 0.5 = = 0.375 2 2 F (a2)F (d2 ) = −0.017 a3 = 0.25, b3 = 0.375 for n = 2,

d2 =

0.25 + 0.375 a 3 + b3 = = 0.3125 2 2 F (a3)F (d3 ) = 0.011, a4 = 0.3125, b4 = 0.375 for n = 3,

d3 =

a 4 + b4 0.3125 + 0.375 = = 0.34375 2 2 F (a4)F (d4 ) = 0.0005, a5 = 0.34375, b5 = 0.375 for n = 4,

d4 =

We reach the accuracy at n = 5, since b5 − a 5 0.375 − 0.34375 = = 0.015625, 2 2 Then, we get the approximate value of the root α≈

0.34375 + 0.375 = 0.359375 . 2

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 225

We can also find the approximate value of the root α by bisection method using the following in Mathematica program f[x_]:=Exp[-x]-2*x; a=0.; b=1.; n=5; Do[{d=(a+b)/2,Print[d], If[f[a]*f[b] 0 for x ∈ [a, b]. Show that the secant method is convergent for any choice of the starting values x0 and x1 . Question 7.8 . Solve the equation e−2x − 4x = 0,

with accuracy  = 0.005, by

0 ≤ x ≤ 1,

Solving Nonlinear Equations by Iterative Methods

Numerical Analysis with Mathematica 229

1. (a) fixed point iterations (b) Newton’s method (c) secant method (d) bisection method using minimum number of iterations.

Lecture Notes in Numerical Analysis with Mathematica, 2014, 230-231

230

References [1 ] Ahlberg, J.H., Nilson, E.N., & Walsh, J.L., (1967), The Theory of Splines and Their Applications, Academic Press. [2 ] Aikinson Kendell, (1997), An Introdution to Numerical Analysis, John Wiley and Sons Publishershing. [3 ] Berezin, I.S. & Zidkov, N.P. (1962), Numerical Methods, v. I & II, Phys. Mat. Literature. [4 ] Cheney, W. & Kincaid, D., (1994), Numerical Mathematics & Computing, Brooks Cole Publishing Company. [5 ] Conte, S. D., & de Boor, C., (1983), Elementary Numerical Analysis, Algorithmic Approach, McGraw-Hill. [6 ] Douglas Faires and Richard L. Burden, (2004), Numerical Methods, Brooks cole Publishing. [7 ] Epperson James, (2001), An Introduction to Numerical Methods and Analysis, John Wiley and Sons Publishing. [8 ] Gerald Curtis, E.and Wheatley P.O. (2003), Applied Numerical Analysis, Pearson Publishers. [9 ] Hall, C.A. (1968), On error bounds for spline interpolation, J. Approx. Theory, 1. [10 ] Haauseholder A.S. (2006), Principles of Numerical Analysis, Dver Publishers. [11 ] Henrici, P. (1962), Discrete Variable Methods in Ordinary Differential Equations, John Wiley & Sons. [12 ] Hohmann A. and Deuflhard P. (2010), Numerical Analysis in Modern Scietific Computing: An Introduction, Springer Publishers. [13 ] Linz, P.(1994), Theoretical Numerical Analysis: An Introduction to Advanced Techniques, Dver Publications. [14 ] Lukaszewicz, J. & Warmus, M., (1956), Numerical and Graphical Methods, Biblioteka Matematyczna, [15 ] Moris, J.L., (1983), Computational Methods in Numerical Analysis, John Wiley & Sons. Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

Referecnes

Numerical Analysis with Mathematica 231

[16 ] Neumair A. (2012), Introduction to Numerical Analysis, Cambride University Press. [17 ] Ortega J. M. (1990), Numerical Analysis, A Second Course, SIAM 3, In Applied Mathematics. [18 ] Phillips, G.M., & Taylor, P.J., (1973), Theory and Applications of Numerical Methods, Academic Press. [19 ] Powell, M.J.D., (1981), Approximation Theory and Methods, Cambridge University Press. [20 ] Phillips G.M. and Taylor P.J. (1996), Numerical Analysis,Academic Press. [21 ] Prenter, P.M. (1975), Splines and Variational Methods, WileyInterscience Publications. [22 ] Ralston, A., (1965), A First Course in Numerical Analysis, McGraw-Hill. [23 ] Sauer T. (2005), Numerical Analysis, Pearson Publishers. [24 ] Scott, L. (2011), Numerical Analysis, Princeton University Press. [25 ] Stoer, J. & Bulrisch R. (1980), Introduction to Numerical Analysis, Springer-Verlag [26 ] Watson, G.A., (1980), Approximation Theory and Numerical Analysis, John Wiley & Sons. [27 ] Walter Gautschi, (2012), Numerical Analysis, Birkhause Publications. [28 ] Wolfram, S., (1992), Mathematica a System for Doing Mathematics by Computers, Addison-Wesley Publishing Company. [29 ] M. Zlamal, (1969), On some Finite Element Procedures for Solving Second order Boundary Value Problems, Numer. Math. 14(1969), pp. 42-48.

Lecture Notes in Numerical Analysis with Mathematica, 2014, 233-235

233

Index absolute round-off error, 2

convergence of Newton’s method, 160 Bernstein polynomials, 83 convergence of the secant method, best approximation polyno166 mials, 87 convolution, 184 best fitted function to exper- convolution theorem, 185 imental data, 106 cubic splines, 55 least squares polynomial, 115 cubic splines on triangular netbicubic splines, 71 works, 73 bilinear splines, 70 error of composed Simpson rule, binary digits, 183 129 bisection method, 153, 171 error of Gauss methods, 145 Buniakowsky inequality, 86 error of interpolation, 30, 53 Chebyshev equi-oscillation the-error of simple trapezian rule, 124 orem, 89 error of trapezian rule, 125 Chebyshev’s knots, 33 Chebyszhev’s polynomials, 28, Fast Fourier Transform, 177 34, 112 finite difference, 25 Chebyszhev’s series, 95 fixed point iterations, 153 Chebyshev’s theorem, 87 fixed point theorem, 155 Chebyshev’s system, 22 floating point numbers, 1 composed methods, 123 Fourier series of cosines, 96 composed Simpson rule, 129 Fourier transformation, 43 composed trapezian rule, 125 functional, 50 computational complexity, 7, 12 Gauss coefficients, 145 computer precision, 4 Gauss formulas of integration, conditional number, 5 143 Krystyna STYŠ & Tadeusz STYŠ All rights reserved - © 2014 Bentham Science Publishers

234 Numerical Analysis with Mathematica

Gauss knots, 140 Gauss methods of integration, 139 generalized polynomial, 22 Gram Schmidt orthogonalization, 114 Hadamard product, 186 Hermite interpolating polynomial, 37 Hermite interpolation, 37 Hermite polynomials, 112 Horner scheme, 11, 12

Krystyna STYŠ & Tadeusz STYŠ

minimum property of splines, 50 module cubic spline, 59 module linear spline, 54 module quadratic spline, 65, 68 module rt, 135

Newton binomial, 84 Newton formula, 25 Newton’s interpolating polynomial, 24 Newton’s method, 153, 159 Newton-Cotes formulas, 122 ill conditioned, 6 Newton-Cotes methods, 121 inner product, 110 nonlinear equations, 153 interpolating cubic splines, 58 norm, 110 inverse discrete Fourier transnumerical integration, 121 form, 178 iteration function, 154 optimal interpolating polynoiteration sequence, 155 mial, 33 order of the secant method, L2 (a, b) space, 110 ρ 169 L2 (a, b) space, 110 Lagrange interpolating poly- orthogonal functions, 110 orthogonal polynomials, 111 nomial, 23 orthonormal functions, 111 Lagrange interpolation, 21 Lagrange polynomials, 23 percent round error, 3 Laguerre polynomials, 113 least square approximation, periodic function, 41 piecewise interpolating poly101 nomial, 27 Legendre polynomials, 114, 143 piecewise linear splines, 26, line of regression, 104 52 Lipschitz condition, 85 piecewise splines on triangumantissa, 1 lar networks, 72 minimax problem, 87 polynomial splines, 47

Index

Numerical Analysis with Mathematica 235

polynomial splines of two vari- Vandermonde determinant, 22 ables, 69 Weierstrass theorem, 83 well conditioned, 6 relative round-off error, 3 remainder in Cauchy form, 78 remainder in integral form, 78 remainder in Lagrange form, 77 Riemann’s sum, 41 Romberg method, 132 root mean square error, 103 round off error in a product, 9 round off error in a sum, 7 round-off error, 1 secant method, 153, 165 simple methods, 123 Simpson rule, 127, 128 space Π3 (∆), 73 space S3 (∆, k), 55 space Sm (∆, k), 47 space S11 (∆, 0, 0), 70 space S33 (∆, 2, 2), 71 stable algorithm, 12 Taylor polynomial, 77 Taylor theorem, 77 trapezian rule, 123 triangular networks, 72 trigonometric interpolating polynomial, 41 trigonometric polynomial, 41, 177 truncation error, 122 uniform approximation, 77