A ABS algorithm for solving singular nonlinear system with space transformation

Citation preview

J Appl Math Comput DOI 10.1007/s12190-008-0176-7

JAMC

A ABS algorithm for solving singular nonlinear system with space transformation Ge Rendong · Xia Zunquan · Wang Jinzhi

Received: 3 September 2007 / Revised: 25 January 2008 © Korean Society for Computational and Applied Mathematics 2008

Abstract A modified ABS algorithm for solving a class of singular nonlinear systems, F (x) = 0, where x, F ∈ R n , is presented. This method is constructed by combining the discreted Brown algorithm with the space transformation method. The second order information of F (x) at a point is not required calculating, which is different from the tensor method and the Hoy’s method. The Q-quadratic convergence of this algorithm and some numerical examples are given as well. Keywords System of nonlinear equations · Singular system of linear equations · ABS algorithms · Jacobian matrix Mathematics Subject Classification (2000) 65H10 · 65F10 1 Introduction In this paper, we consider the following nonlinear system F (x) = 0

(1)

where x ∈ Rn , F (x) = (f1 (x), f2 (x), . . . , fn (x))T . We assume that there exists a solution x ∗ of (1), F (x ∗ ) = 0, F ′′ is Lipschitzian around x ∗ and furthermore that  rank(F ′ (x ∗ )) = n − r, 1 ≤ r ≪ n, (2) ∇fi (x ∗ ) = 0 i = 1, 2, . . . , n, G. Rendong () · W. Jinzhi School of Science, Dalian Nationalities University, Dalian 116600, China e-mail: [email protected] X. Zunquan Department of Applied Mathematics, Dalian University of Technology, Dalian 116024, China

G. Rendong et al.

Let u1 , u2 , . . . , ur and v1 , v2 , . . . , vr are arbitrary norm orthogonal vector sets such that Null(F ′ (x ∗ )) = span{u1 , u2 , . . . , ur },

(3)

Null(F ′ (x ∗ )T ) = span{v1 , v2 , . . . , vr }. And let U = (u1 , u2 , . . . , ur ),

V = (v1 , v2 , . . . , vr ).

There are several publications dealt with solving nonlinear systems of F ′ with full rank by the ABS algorithm, see for instance, Abaffy, Broyden and Spedicato [1], Abaffy, Galantai and Spedicato [2], Abaffy and Spedicato [3], Galantai [5, 6], Huang [11], Spedicato, Chen and Deng [15, 16], Spedicato and Huang [17]. For the survey on general progress of the ABS algorithms it also can be referred to Spedicato [13, 14]. A discrete ABS algorithm for solving nonlinear systems of F ′ with full rank was introduced in Huang [11]. For the case in which F ′ is of rank n − 1 some algorithms were designed in the last several years, see for instance, Hoy [8, 9], Hoy and Schwetlick [10] and the tensor model introduced by Schnabel and Frank [4, 12]. An algorithm proposed due to Hoy and Schwetlick is quoted here for constructing a modified ABS algorithm for solving problem (1) with rank defects, reading that S0 S1 S2 S3

Choose x0 ≈ x ∗ , q ≈ v, p ≈ u, set k = 0. Set Bk = F ′ (xk ) + pq T . Determine dk from Bk dk = F (xk ), vk from Bk vk = p, uk from Bk uk = q. Set xk+1 = xk − dk −

1 − q T vk − uTk F ′′ (xk )vk dk uTk F ′′ (xk )vk vk

.

Under the usual conditions, the above algorithm converges Q-quadratically to x ∗ . The tensor model introduced by Schnabel and Frank [4, 12] is a quadratic model of F (x) formed by adding a second order term to the linear model, given by 1 MT (xc + d) = F (xc ) + Jc d + Tc dd, 2 where Tc ∈ R N ×N ×N is used to give the second order information about F (xc ) around xc . Tc = arg min{ Tc F |Tc sk sk = zk , k = 1, 2, . . . , p}, where sk , zk are defined by Frank and Schnabel [4]. Solution of the tensor model is to find d ∈ R N such that d is a solution of mind∈R N MT (xc + d) 2 . Under the assumptions that F ′ (x ∗ ) is singular with only one zero singular value and uT F ′′ (x ∗ )vv = 0, the sequence of iterations generated by the tensor method based on an ideal tensor model converges locally and two-step Q-superlinearly to the solution with Q-order 3/2, and the sequence of iterates generated by the tensor method based on a practical tensor model converges locally and three-step Q-superlinearly to the solution with Q-order 3/2. For the system with rank one defect, Ge and Xia [7] constructed a modified ABS algorithm for solving problem (1) under the same conditions by combining the discreted ABS algorithm with the idea of Hoy and Schwetlick [10]. In this paper,

A ABS algorithm for solving singular nonlinear system with space

a modified ABS method is proposed for solving systems of nonlinear equations with rank (F ′ (x ∗ )) = n − r, 1 < r ≪ n and the second order information of F (x) is not required, see Algorithm 2.1 in Sect. 2. The new algorithm converges Q-quadratically to x ∗ . We now recall the scaled block ABS method, due to Abaffy and Spedicato [3], for solving nonlinear systems of F with full rank. Algorithm 1.1 The scaled equationwise nonlinear ABS cycle Step 1 Let y1 = xi , H1 = I , set k = 1. Step 2 The vector uk ∈ R n is defined tion of yj uk =

k 

by

a convex combina-

(4)

τk,j yj .

j =1

Step 3 Define the search vector pk ∈ R n,mk is defined by pk = HkT zk ,

(5)

where zk ∈ R n is arbitrary, save that  0. det [pkT (A(uk ))T vk ] =

(6)

Step 4 Define the step size αk by αk =

vkT f (yk ) pkT (A(uk ))T vk

.

(7)

Step 5 Update the minor iterate yk by yk+1 = yk − αk pk .

(8)

If k = n, set xi+1 = yn+1 , the cycle is completed. Step 6 Update Hk by Hk+1 = Hk − Hk (A(uk ))T vk wkT Hk ,

(9)

where wk ∈ R n satisfies wkT Hk (A(uk ))T vk = 1.

(10)

Step 7 Set k = k + 1 and go to Step 2. End For convergence of the above algorithm, the condition of full rank of F ′ is required. A discrete scheme of Algorithm 1.1, due to Huang [11], also requires that the Jacobian of F ′ is full rank. In his scheme only the first order approximation to derivatives via the difference quotient form is considered. In this paper the rank of

G. Rendong et al.

Jacobian of F ′ is assumed to be rank n − r and a modification by using the space rotating transformation technique is considered, see Step 4 of Algorithm 2.1 in the next section. In Sect. 2, a new algorithm will be presented. Convergence analysis including Q-quadratic rate of convergence will be given in Sect. 3. A short discussion on choices of parameters are given in Sect. 4. 2 A new algorithm In this section, let P , Q be the n × r matrix with columns full rank such that P T N , QT V is nonsingular, then it is can be obtained easily that F ′ (x ∗ ) + QP T also is nonsingular and the solution of the matrix equations (F ′ (x ∗ ) + QP T )X = Q is U (P T U )−1 . Moreover, let U (P T U )−1 = (w1 , w2 , . . . , wr ). We assume that the set of vectors consisting of the rows of F ′ (x ∗ ), indexed by ik , k = 1, . . . , n − r, is one of the largest linear independent sets of F ′ (x ∗ ), where i1 < i2 < · · · < in−r . The other rows of F ′ (x ∗ ) are indexed by j1 < j2 < · · · < jr . We can construct an auxiliary function as follows T (x) = (fi1 (x), fi2 (x), . . . , fin−r (x), fj1 (x), fj2 (x), . . . , fjr (x))T ,

where fjs (x) = fjs (G∗s x + bs∗ ), s = 1, 2, . . . , r, (G∗s )T =

ws (∇fjs (x ∗ ) + ws )T ,

ws 2

bs∗ = (I − G∗s )x ∗ .

(11)

By the definition of (11), we know that the matrix G∗s has the following idempotency: ∗ G∗2 s = Gs .

It is easy to see that T (x ∗ ) = 0 and T ′ (x ∗ ) is nonsingular. Based on the above discussion, a modified ABS algorithm for solving the problem (1) with condition (2) is given. Let aj ∈ R n , i = 1, . . . , j , Aj = (a1 , . . . , aj ) and G(a1 , . . . , aj ) = ATj Aj . Clearly, G(a1 , a2 , . . . , aj ) is positive semi-definite and {a1 , a2 , . . . , aj } are linearly independent iff det G(a1 , a2 , . . . , aj ) = 0. Let G∗j = G(∇fi1 (x ∗ ), ∇fi2 (x ∗ ), . . . , ∇fij (x ∗ )) denote the matrix consisting of j column vectors taken arbitrarily (k) (k) (k) from F ′ (x ∗ ) and G(k) j = G(ai1 , ai2 , . . . , aij ), where

aj(k)T

=



(k)

(k)

(k)

fj (yj + hk e1 ) − fj (yj ) hk

(k)

,...,

(k)

fj (yj + hk en ) − fj (yj ) hk



,

(k)

yj (j = 1, 2, . . . , n) is computed by Algorithm 2.1 and hk = O F (y1 ) . Let r ∗ = min1≤j ≤n {det G∗j = 0, 1 ≤ i1 < i2 < · · · < ij ≤ n} and ε ∗ < r ∗ /2. One has that (k)

(k)

if yj closes enough to x ∗ , then det(Gj ) ≤ ε ∗ iff det(G∗j ) = 0. A modified ABS algorithm for solving a class of singular nonlinear systems is defined by the following steps.

A ABS algorithm for solving singular nonlinear system with space

Algorithm 2.1 ∗ = 0. Given ε > 0, x0 close enough to x ∗ and set k = 0, jmin Step 0 If F (xk ) < ε, then stop. (k) (k) ∗ = 0. Step 1 Set H1 = I, y1 = xk , j = 1, G(k) =  and jmin (k) (k) Step 2 For 1 ≤ j ≤ n. Compute aj and det (G(k) , aj ) (k)

If det(G(k) , aj ) < ε ∗ and j < n − jmin + 1, then {jmin ← jmin + 1, exchange the positions between fj (x) and fn−jmin +1 (x) in F (x), aj (x) and an−jmin +1 (x) in A(k) (x). go to the beginning of Step 2} (k)

otherwise G(k) = (G(k) , aj ), and choose (k)

(k)T

zj ∈ R n such that aj (k)

Compute

(k) (k)

Hj zj = 0 (k)T

and wj ∈ R n such that wj

(k)

(k) (k)

(k)

(k)

pj = Hj zj ,

(k)

(k) (k)

Hj aj = 0.

(k)

(k)T (k) aj , (k) (k) (k) (k) (k)T (k) (k)T (k) (k) Hj +1 = Hj − Hj aj wj Hj /wj Hj aj ,

yj +1 = yj − fj (yj )pj /pj

j = j + 1, if j < n − jmin + 1, then go to Step 2.

∗ jmin

= 0 or j = n + 1, then go to Step 5. If (k) (k) (k) Step 3 If j = n − jmin + 1, then let Ak = (a1 , a2 , . . . , an )T , r = jmin , compute Uk such that (k)

(k)

where Uk = (w1 , w2 , . . . , wr(k) ),

(Ak + QP T )Uk = Q,

where matrices Q, P ∈ R n×r are generated by a random function. Step 4 Set s = j − n + r, let (k)

G(k)T s

=

(k)

(k)

ws (aj + ws )T

ws(k) 2

(12)

,

compute bs(k) = (1 − G(k) s )xk

and

set

(k) (k) fj (x) = fj (G(k) s x + bs ) → fj (x),

(13)

go to Step 2. (k) Step 5 Set xk+1 = yn+1 , hk+1 = O( F (xk+1 ) ), k = k + 1 and go to Step 0. End (k)

(k)

Note that if det(G(k) , aj ) < ε ∗ in Step 2, then aj bination of columns of

G(k) ,

and hence

(k) aj

is regarded as a linear com(k)

(k)

has no effect for forming hj and yj .

G. Rendong et al.

It seems that the identification of dependency mentioned above could be replaced by computing Hj(k) aj(k) in a certain sense. It needs to be further investigated. (k)T

of Step 4 of Algorithm 2.1 needn’t be formed because it is only In addition, Gs (k) (k) (k) used to the calculation of Gs yj + bs .

3 Convergence analysis In this section, the local convergence is investigated and Q-quadratic convergence is demonstrated. In the following, x denotes the Euclidean norm for x ∈ R n and A

the Frobenius norm for A ∈ R n,n . Note that (2) is assumed and there exist two positive constants r0 > 0 and K0 > 0, such that for any x, y ∈ B(x ∗ , r0 ), j = 1, 2, . . . , n

F (x) − F (y) ≤ K0 x − y ,

F ′ (x) − F ′ (y) ≤ K0 x − y ,

(14)

To begin with, the following basic assumption is needed. Assumption A There exist a constant γ > 0 and assume that ai1 , ai2 , . . . , ain−r is linear independent chosen in algorithm such that Dk ≤ γ , k = 0, 1, 2, . . . , where (k)

(k)T (k) (k) (k)T (k) ai1 , . . . , pn−r /pn−r ain−r ,

Dk = (p1 /p1 (k)

(k)T

(k)

(k)

pn−r+1 a j1 , . . . , p n(k) / pn(k)T  a jr ) p n−r+1 /

(k)T (k) (k) and p n−r+1 , aj1 is defined by fj (x) in Algorithm 2.1.

Assumption B

rank(F ′ (x ∗ )) = n − r, 1 ≤ r ≤ n, ∇fi (x ∗ ) = 0, i = 1, 2, . . . , n. Assumption C The rows of J (x ∗ ) = F ′ (x ∗ ), indexed by i1 < i2 < · · · < in−r , form a largest linearly independent set, and the subscripts of the rest are denoted by j1 , j2 , . . . , jr . By the beginning of the algorithm, if we make P approximate sufficiently U and Q approximate sufficiently V it is easy to see that P T U and QT V is nonsingular and the conclusion of Lemmas 1 and 2 are hold form Assumptions B and C. Therefore, when k is large enough, the nonlinear equations and its approximation Ak to Jacobian matrix J (x ∗ ) = F ′ (x ∗ ) are rearranged as follows: F (x) = (fi1 (x), fi2 (x), . . . , fin−r (x), fj1 (x), fj2 (x), . . . , fjr (x))T , (k) (yn−r ), Ak = (ai(k) (y1(k) ), ai(k) (y2(k) ), . . . , ai(k) n−r 1 2

))T , (yt(k) (yt(k) ), aj(k) (yt(k) ), . . . , aj(k) aj(k) r 1 2 r 1 2 (k)

(k)

(k)

(k)

(k)

(k)

where yt1 , yt2 , . . . , ytr are some elements among yi1 , yi2 , . . . , yin−r .

A ABS algorithm for solving singular nonlinear system with space

Lemma 1 If P T U and QT V are nonsingular, then there exist r1 > 0 and δ1 > 0 such that (1) B(x) = F ′ (x) + QP T is nonsingular if x − x ∗ < r1 , (2) B = A + QP T is nonsingular if A − F ′ (x ∗ ) < δ1 . Lemma 2 Let T (x) = (fi1 (x), fi2 (x), . . . , fin−r (x), fj1 (x), fj2 (x), . . . , fjr (x))T . Then T (x ∗ ) = 0 and T ′ (x ∗ ) is of full rank. Lemmas 1 and 2 can be obtained throughout calculation from (2), (14). Lemma 3 (a) There exists an r2 > 0 such that if x − x ∗ < r2 , then ∇fj (x) ≤ 2 ∇fj (x ∗ ) , j = 1, 2, . . . , n and B(x)−1 ≤ 2 B(x ∗ )−1 ; (b) There exists a δ2 > 0 such that if F ′ (x) − F ′ (x ∗ ) < δ2 , then Bk−1 (x) ≤ 2 B(x ∗ )−1 and B −1 (x) − B(x ∗ )−1 ≤ 2 B(x ∗ )−1 2 F ′ (x) − F ′ (x ∗ ) . Lemma 4 If r2 (r2 < r0 ) is small enough, then there exist δ > 0 and L > 0 such that if x, y ∈ B(x ∗ , r2 ), one has that (1) T ′ (x) − T ′ (x ∗ ) ≤ L x − x ∗ ,

T (y) − T (x) − T ′ (x ∗ )(y − x) ≤ L y − x max{ x − x ∗ , y − x ∗ }; (2) x − x ∗ ≤ δ T ′ (x ∗ )−1

T (x) . Proof From (14), it is easy to see that the conclusion (1) of this lemma holds. By virtue of T (·) in Lemma 2, one has T ′ (x ∗ )−1 T (z) = T ′ (x ∗ )−1 (T (z) − T (x ∗ ))  1 ′ ∗ −1 = T (x ) T ′ (x ∗ + t (z − x ∗ ))(z − x ∗ )dt 0

= (z − x ∗ ) + T ′ (x ∗ )−1



0

1

(T ′ (x ∗ + t (z − x ∗ )) − T ′ (x ∗ ))(z − x ∗ )dt.

There exists L such that

z − x ∗ − L/2 T ′ (x ∗ )−1 z − x ∗ 2 ≤ T ′ (x ∗ )−1 T (z) , according to the conclusion (1) of this lemma. Let δ −1 = 1 − (L/2)r0 T ′ (x ∗ )−1

(δ −1 < 1). The value on the right hand side of the formula above, i.e., δ −1 , is greater than zero when r2 is small enough. It leads to the second conclusion.  Lemma 5 If r2 is small enough, then

T (x) ≤ K2 x − x ∗ ,

x ∈ B(x ∗ , r2 ), where K2 = 0.5Lr2 + T ′ (x ∗ ) .

G. Rendong et al.

Lemma 6 Assume that Assumption A is valid, then there exists an r3 < r2 < 1 such (k) that for any y1 ∈ B(x ∗ , r3 ) one has that (k)

and

(k)

yj

(k) − x ∗ ≤ (1 + γ K2 )j −1 y1

yj ∈ B(x ∗ , r2 )

− x ∗ ,

1 ≤ j ≤ n − r + 1,

where r3 = (1 + γ K2 )−n−r+1 r2 < r2 . From Step 5 of the algorithm given in Sect. 2 it follows that there exists a K3 > 0, such that |hk+1 | ≤ K3 F (xk+1 ) . Lemma 7 Under the Basic Assumption A, there exist an r4 satisfying 0 < r4 < r3 , and M1 > 0 such that if xk = y1(k) ∈ B(x ∗ , r4 ), then we have that (k)

(k)

(1) aj − ∇fj (x ∗ ) ≤ M1 xk − x ∗ , 1 ≤ j ≤ n; where aj Step 3 of Algorithm 2.1 √ (2) Bk−1 − B(x ∗ )−1 ≤ 2 B(x ∗ )−1 nM1 xk − x ∗ .

is defined by Ak in

Lemma 8 Under the assumption of Lemma 7, and if P T U is nonsingular, and if r3 is small enough, then there exist K3 > 0 and K4 > 0 such that if xk − x ∗ < r3 , one has Ak + QP T is nonsingular and (1) Uk − U (P T U )−1 ≤ K3 xk − x ∗ , (k) (2) G∗s − Gs ≤ K4 xk − x ∗ . Proof The conclusion (1) can be obtained by Step 3 of Algorithm 2.1 and Lemma 7. By Lemma 7, we have

G(k) s

− G∗s

(k) (k) (k) ws (ajs + ws )T ws∗ (∇fjs (x ∗ ) + ws∗ )T − = (k)

ws∗ 2

ws 2 =

1

(k)

ws∗ 2 ws 2



( ws∗ 2 ws(k) − ws(k) 2 ws∗ )(∇fjs (x ∗ ) + ws∗ )T (k)

+ ( ws∗ 2 ws(k) (ws(k) − ws∗ + ajs − ∇fjs (x ∗ ))T

≤

1 (k)

ws∗ 2 ws 2

(( ∇fjs (x ∗ ) + ws∗ )

ws∗ 2 ws(k) − ws(k) 2 ws∗

− ∇fjs (x ∗ ) )) + ws∗ 2 ws(k) ( ws(k) − ws∗ + aj(k) s ≤

1 (k)

ws∗ 2 ws 2

(( ∇fjs (x ∗ ) + ws∗ )(2 ws∗ 2 + ws∗

ws(k) )

+ (M1 + K3 ) ws∗ 2 ws(k) ) xk − x ∗

≤ K4 xk − x ∗ .



A ABS algorithm for solving singular nonlinear system with space

Lemma 9 Under the basic assumptions given at the beginning of this section, it follows from Lemmas 7 and 8 that there exist r4 (r4 < r3 ), L3 , L4 , such that for any 1 ≤ s ≤ r and x, y ∈ B(x ∗ , r4 ), one has that (k) (1) ∇ fjs (x ∗ ) − ∇ fjs (x ∗ ) ≤ L3 x (k) − x ∗ , (k) (k) (2) |fjs (x) − fjs (y) − ∇ fjs (x ∗ )(x − y)| ≤ L4 x − y {max{ y − x ∗ , x − x ∗ } +

xk − x ∗ }.

Proof Firstly, one has (k) ∗ (k) (k) ∗ ∗ ∗

G(k) s x + bs − x = Gs x + bs − Gs x − bs

∗ (k) ∗ ≤ xk − x ∗ + G(k) s

x − x + Gs

xk − x

≤ (1 + G∗s + K4 r0 )max{ xk − x ∗ , x − x ∗ }.

(15)

Evidently, if r4 is small enough and x, xk ∈ B(x ∗ , r4 ), then we have G∗s x + bs∗ , (k) (k) Gs x + bs ∈ B(x ∗ , r2 ). Consequently, from (15) we have (k)

∇ fjs (x ∗ ) − ∇ fjs (x ∗ )

T (k) ∗ (k) ∗ T ∗ = (G(k) s ) ∇fjs (Gs x + bs ) − (Gs ) ∇fjs (x )

∗ ∗ (k) ∗ ≤ G(k) s − Gs ( ∇fjs (x ) + K0 Gs

xk − x ) (k) ∗ + K0 G(k) s

Gs

xk − x

≤ L3 xk − x ∗ . Since (k)

(k)

(k)

|fjs (x) − fjs (y) − ∇ fjs (x ∗ )(x − y)|

≤ K0 G(k) 2 x − y max{ y − x ∗ , x − x ∗ } ≤ L2 x − y max{ y − x ∗ , x − x ∗ },

we have (k) (k) |fjs (x) − fjs (y) − ∇ fjs (x ∗ )(x − y)|

≤ L2 x − y max{ y − x ∗ + x − x ∗ } + L3 xk − x ∗

x − y

≤ L4 x − y {max{ y − x ∗ , x − x ∗ } + xk − x ∗ },

where L4 = max{L2 , L3 }. The demonstration is completed.



In order to simplify the following discussion, we give the notations below f m (x) = fim (x), f m (x) = fjs (x), a (k) m

(k) = a js ,

(k) a (k) m = am , m = 1, 2, . . . , n − r,

(k) (k) f m (x) = fjs (x),

m = n − r + 1, . . . , n, s = m − n + r.

(16)

G. Rendong et al.

Lemma 10 Assume that Assumption A is valid, then there exists an r5 (< r4 < 1) and (k) K5 such that for any y1 ∈ B(x ∗ , r5 ) one has that (k)

(k)

(k)

(1) yj ∈ B(x ∗ , r4 ) and yj − x ∗ ≤ K5 y1 − x ∗ , 1 ≤ j ≤ n + 1, (2) If M1 in Lemma 7 is large enough, then for any 1 ≤ s ≤ r we have (k)

 ajs − ∇ fjs (x ∗ ) ≤ M1 xk − x ∗ .

Proof The conclusion (1) is easily to obtained from Lemmas 6 and 9. Let M1 ≥ √ nL4 (K5 + K0 K3 ) and r5 < (K5 + K0 K3 )−1 r4 ≤ r4 . It is easy to proved (k)

yj + hk ei ∈ B(x ∗ , r4 ). Hence, one has from Lemmas 8, 9 and the basic assumption, (k)

 ajs − ∇ fjs (x ∗ ) 2 =

n  i=1

(k) (k) (k) (k) |(fjs (yj + hk ei ) − fjs (yj ))/ hk − ∇fjs (x ∗ )T ei |2 (k)

≤ nL24 ( yj − x ∗ + xk − x ∗ + |hk |)2

≤ nK02 [K5 + K0 K3 ]2 (xk − x ∗ ) 2 ≤ M12 xk − x ∗ 2 .

 (0)

(0)

(0)

Lemma 11 If r4 is small enough, x0 ∈ B(x ∗ , r4 ) and G(a1 , a2 , . . . , aj ) < r ∗ /2, then ∇f1 (x ∗ ), ∇f2 (x ∗ ), . . . , ∇fj (x ∗ ) are linearly dependent. Theorem 1 (Convergence theorem) Suppose Assumptions A, B, and C are valid. Then there exists a constant τ > 0 such that for any x0 ∈ B(x ∗ , τ ) the sequence {xk } generated by the algorithm converges Q-quadratically to x ∗ . (k)

Proof First, consider the case where r5 < r4 and y1 ∈ B(x ∗ , r5 ), one has from Step 4 of this algorithm that (k)T

(yn+1 − yj ) = −f j (yj ),

(k)T

(yn+1 − yj ) = −f j (yj ),

aj aj

(k)

(k)

(k)

(k)

(k)

(k)

1 ≤ j ≤ n − r,

(k)

(17)

n − r + 1 ≤ j ≤ n.

According to (17), we have for 1 ≤ j ≤ n − r that (k) (k) (k) f j (yn+1 ) = f j (yj(k) ) + ∇ T f j (yj(k) + θj(k) (yn+1 − yj(k) ))(yn+1 − yj(k) ) (k)

(k)T

= f j (yj ) + a j

(k)

(k)

(yn+1 − yj )

(k)

(k)

(k)

(k)

(k)

(k)

(k)

+ [∇f j (yj + θj (yn+1 − yj )) − a j ]T [yn+1 − yj ] (k)

(k)

(k)

(k)

(k)

(k)

(k)

= [∇f j (yj + θj (yn+1 − yj )] − a j ]T (yn+1 − yj );

(18)

A ABS algorithm for solving singular nonlinear system with space

we have for n − r + 1 ≤ j ≤ n that (k)

(k)

(k)

(k)

(k)

(k)

(k)

(k)

(k)

(k)

f j (yn+1 ) = f j (yj ) + ∇ T f j (yj + θj (yn+1 − yj ))(yn+1 − yj ) (k)

(k)T

= f j (yj ) + a j

(k)

(k)

(yn+1 − yj )

(k)

(k)

(k)

(k)

(k)

+ [∇f j (yj + θj (yn+1 − yj )) − a j ]T [yn+1 − yj ] (k)

(k)

(k)

(k)

(k)

(k)

(k)

= [∇f j (yj + θj (yn+1 − yj )] − a j ]T (yn+1 − yj ) (k)

(k)

(k)

(19)

+ f j (yj ) − f j (yj ); (k)

furthermore according to Lemma 10, we have that if y1 ∈ B(x ∗ , r5 ) ⊆ B(x ∗ , r3 ), (k)

yn+1 − x ∗ ≤ K5 y1(k) − x ∗ . (k)

(k)

(k)

(k)

(k)

By Lemmas 7 and 10, it follows from uj = yj + θj (yn+1 − yj ) that (k)

(k)

(k)

(k)

a j − ∇f j (uj ) ≤ a j − ∇f j (x ∗ ) + ∇f j (uj ) − ∇f j (x ∗ )

≤ (M1 + LK5 + K0 K5 ) · xk − x ∗ . For j such that n − r + 1 ≤ j ≤ n, we have from the idempotent property of G∗s that (k)

(k)

(k)

f j (yj ) − f j (yj ) (k) (k) (k) = fjs (yj ) − fjs (yj ) (k)

(k)

(k) = fjs (G∗s yj + bs∗ ) − fjs (G(k) s yj + bs ) (k)

(k)

(k)

∗ (k) ∗ (k) = ∇fjTs (vj )G(k) s (Gs yj − Gs yj + bs − bs ) (k)

(k)

∗ (k) (k) ∗ ∗ ∗ = ∇fjTs (vj )(G(k) s (Gs − Gs )(yj − xk ) + (Gs − Gs )(I − Gs )(x − xk )), (k) (k) where vj(k) = θ (G∗s yj(k) + bs∗ ) + (1 − θ )(G(k) s yj + bs ). By the definition of r4 in (k)

Lemma 9, it is easy to known vj ∈ B(x ∗ , r2 ). Consequently, there exists a positive number L6 such that for any n − r + 1 ≤ j ≤ n, we have from Lemma 9 and the equivalence property of matrix norm (k)

|f j (yj(k) ) − f j (yj(k) )| ≤ 2 ∇fjs (x ∗ ) (α1 (G∗s + K4 r3 )K4 (K5 + 1) √ + α2 ( n + (G∗s )K4 )) xk − x ∗ 2

≤ L6 xk − x ∗ 2 , where α1 and α2 are some positive constants.

(20)

G. Rendong et al.

Also from (18), (19) and (20), we have (k)

(k) (k) (k) (k) (k) (k) |f j (yn+1 )| ≤ a (k) j − ∇f j (uj )

yn+1 − yj + |f j (yj ) − f j (yj )|

(The second term of right hand above exists only when j > n − r) ≤ (2K5 (M1 + LK5 + K0 K5 ) + L6 ) xk − x ∗ 2 .

(21)

There exists a constant N > 0 such that

T (xk+1 ) ≤ N xk − x ∗ 2

(22)

in terms of (22), it follows from Lemma 4 that

xk+1 − x ∗ ≤ δN T ′ (x ∗ )−1

xk − x ∗ 2 .

(23)

If τ is small enough, τ < min{r5 , (δ T ′ (x ∗ )−1 N )−1 }, and x0 − x ∗ < τ , then by induction, it can be proved from (23) that for all k > 0 and x0 ∈ B0 (x ∗ , τ ) one has that y1k+1 ∈ B(x ∗ , r5 ), xk+1 − x ∗ ≤ δτ N T ′ (x ∗ )−1

xk − x ∗

(24)

where B0 (· , ·) denotes an open ball. Taking the limit on the second line of (24), one has lim xk = x ∗

k→∞

and hence it follows from (20) and (21) that lim Uk = U (P T U )−1 .

k→∞

Summarizing the statement given above, it follows from (23) that {xk } generated by the algorithm converges Q-quadratically to x ∗ . 

4 On the choices of parameters and the computation of Gj 4.1 On the choices of parameters The following theorem is similar to [3, Theorem 13.2] and [10, Theorems 4.1 and 4.2]. Theorem 2 Suppose that (2) and (14) are valid. For any fixed β ∗ ∈ [0, π/2), if we (k) (k) (k) take zj satisfying Angle(pj , a j ) ≤ β ∗ , j = 1, 2, . . . , n, k = 0, 1, . . . . Then there exists a positive scalar r6 > 0 such that for any x0 ∈ B(x ∗ , r6 ), Basic Assump(k) tion A holds and a j is uniformly bounded.

A ABS algorithm for solving singular nonlinear system with space

Proof Define (k)

(k)

(k)

(k)

(k) T A(k) = [(a 1 (y1 ), a 2 (y2 ), . . . , a (k) n (yn ))] ,   f j (x + hk e1 ) − f j (x) f j (x + hk en ) − f j (x) (k)T ,..., aj (x) = , hk hk

(25)

J (x ∗ ) = [∇f 1 (x ∗ ), ∇f 2 (x ∗ ), . . . , ∇f n (x ∗ )]T . (k)

Moreover, let Aj is the first j columns of A(k) and Jj (x ∗ ) is the first j columns of J (x ∗ ) (1 ≤ j ≤ n). Let r = n (J (x ∗ ))−1 /[(1 − α) cos β ∗ ]. Following the line of (k)

proofs in [3, Theorem 13.3] or [11, Theorem 4.1] and estimating Aj − Jj (x ∗ )

(k)

and (Aj )−1 , we have the following inequality holds (0)

√ (0)T (0) a j ≤ 1/ nr,

pj /pj

1 ≤ j ≤ n.

The demonstration is completed by induction for all k = 0, 1, 2, . . . . (k)

(k)



(k)

Theorem 3 If parameters zj , wj and a j are chosen as zj(k) = wj(k) = a (k) j ,

1 ≤ j ≤ n,

then there exist parameters γ and r7 > 0 such that for any x0 ∈ B(x ∗ , r7 ), we have that (k)T

(a) a j

(k)

(k) (k)

Hij a j = 0, 1 ≤ j ≤ n, k = 0, 1, 2, . . . , (k)T (k) aj ≤ γ ,

(b) pij /pj

1 ≤ j ≤ n, k = 0, 1, 2, . . . .

(k)

Proof Take Aj and Jj (x ∗ ) as defined in Theorem 2. Following the line of proof in [11, Theorem 4.2], the theorem follows from property (1) of Lemmas 7 and 10 .  4.2 On the computation of Gj = |G(a1 , a2 , . . . , aj )| There are different methods for calculating values of determinants det Gj , for instance, Choleski factorization method might be available for this purpose.

References 1. Abaffy, J., Broyden, C.G., Spedicato, E.: A class of direct methods for linear equations. Numer. Math. 45, 361–367 (1984) 2. Abaffy, J., Galantai, A., Spedicato, E.: The local convergence of ABS methods for nonlinear algebraic equations. Numer. Math. 51, 429–439 (1987) 3. Abaffy, J., Spedicato, E.: ABS Projection Algorithms: Mathematical Techniques for Linear and Nonlinear Equations. Ellis Horwood, Chichester (1989)

G. Rendong et al. 4. Dan, F., Frank, P.D., Schnabel, R.B.: Local convergence analysis of tensor methods for nonlinear equations. Math. Program. 62, 427–459 (1993) 5. Galantai, A.: The global convergence of the ABS methods for a class of nonlinear problems. OMS 4, 283–295 (1995) 6. Galantai, A.: Purterbation theory for full rank factorizations. QDMSIA 40(99) (1999). Bergamo 7. Ge, R., Xia, Z.: An ABS algorithm for solving sigular nonlinear systems with rank one defect. Korean J. Comput. & Appl. Math. 9(1), 167–184 (2002) 8. Hoy, A.: A relation between Newton and Gauss–Newton steps for singular nonlinear equations. Computing 40, 19–27 (1988) 9. Hoy, A.: An efficiently implementable Gauss–Newton-like method for solving singular nonlinear equations. Computing 41, 107–122 (1989) 10. Hoy, A., Schwetlick, H.: Some superlinearly convergent methods for solving singular nonlinear equations. In: Lectures in Applied Mathematics, vol. 26, pp. 285–299. Am. Math. Soc., Providence (1990) 11. Huang, Z.: Modified ABS methods for nonlinear systems without evaluating Jacobian matrices. J. Numer. Math. Chin. Univ. 13(1), 60–71 (1991) 12. Schnabel, R.B., Dan, F., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21(5), 815–843 (1984) 13. Spedicato, E.: ABS algorithms from Luoyang to Beijing. Optim. Methods Softw. 8, 87–97 (1997) 14. Spedicato, E.: Twenty years of ABS algorithms: a bibliography. QDMSIA 2001/1, Bergamo (2001) 15. Spedicato, E., Chen, Z., Deng, N.: A class of difference ABS-type algorithms for a nonlinear system of equations. Numer. Linear Algebra Appl. 13, 313–329 (1994) 16. Spedicato, E., Chen, Z., Deng, N.: A new nonlinear ABS-type algorithm and its convergence property. QDMSIA 33(96) (1996). Bergamo 17. Spedicato, E., Huang, Z.: Numerial experience with Newton-like methods for nonlinear algebraic equations. QDMSIA 5(96) (1995). Bergamo