Advanced Linear Algebra for Engineers with MATLAB (Instructor Solution Manual, Solutions) [1 ed.] 9781439801290, 9781420095234, 1420095234

Citation preview

SOLUTIONS MANUAL FOR Advanced Linear Algebra for Engineers with MATLAB Solutions ®

by Sohail A. Dianat & Eli Saber

CONTENTS

Chapter 1. Matrices, Matrix Algebra, and Solutions to Systems of Linear Equations

1

Chapter 2. Determinants and Solutions to Systems of Linear Equations

38

Chapter 3. Linear Vector Spaces

69

Chapter 4. Eigenvalues and Eigenvectors

98

Chapter 5. Matrix Polynomials and Functions of Square Matrices

143

Chapter 6. Introduction to Optimization

180

1

CHAPTER 1 Problem # 1 For each given matrix, state whether it is square, upper triangular, lower triangular or diagonal. 0 0 0⎤ ⎡4 ⎢6 3 0 0 ⎥⎥ ⎢ b) B = ⎢− 1 − 2 5 0⎥ ⎢ ⎥ 4 7 8 ⎥ ⎢3 ⎢⎣ 9 11 − 1 − 2⎥⎦

⎡1 7 6⎤ a) A = ⎢⎢0 8 9⎥⎥ ⎢⎣0 0 6⎥⎦

2 1⎤ ⎡ 3 ⎢ c) C = ⎢− 2 − 1 0 ⎥⎥ ⎢⎣ 2 8 10⎥⎦

⎡1 − 4 7 8 9 ⎤ e) ⎢⎢0 2 − 1 3 5⎥⎥ ⎢⎣0 0 3 2 1⎥⎦

⎡1 0 ⎤ d) D = ⎢ ⎥ ⎣0 2 ⎦ Solution:

Matrix Square Upper triangular Lower triangular Diagonal a

X

X

b

X

c

X

d

X

e

X X

Problem # 2 ⎡1 + j 2 2 + j 3⎤ ⎡1 2⎤ T H If A1 = ⎢ and A2 = ⎢ , compute A1 , ( A1 + A2H ) H and A2 . ⎥ ⎥ ⎣3 + j 4 4 + j 5⎦ ⎣3 4⎦ Solution: ⎡1 3 ⎤ ⎡ 2 − j 2 5 − j 4⎤ A1 = ⎢ ( A1 + A2H ) H = ⎢ ⎥ ⎥ ⎣ 2 4⎦ ⎣ 5 − j3 8 − j5⎦ T

H

⎡2 + j 2 5 + j 3⎤ =⎢ ⎥, ⎣5 + j 4 8 + j 5⎦

1

and A2

H

⎡1 − j 2 3 − j 4⎤ =⎢ ⎥ ⎣2 − j 3 4 − j 5⎦

Problem # 3 1 6⎤ ⎡3 4 5⎤ ⎡ 2 ⎢ ⎥ ⎢ Given that A = ⎢3 − 6 7 ⎥ and B = ⎢− 2 − 1 4⎥⎥ , compute the following: ⎢⎣2 1 4⎥⎦ ⎢⎣ 5 3 9⎥⎦

a)

A+ B

b) A − B c) AB d)

B− A

e) BA f) Trace( A) g) Trace(B )

(

h) Trace A 2 B

)

Solution: 1 6⎤ ⎡5 5 11⎤ ⎡3 4 5⎤ ⎡ 2 ⎢ ⎥ ⎢ a) A + B = ⎢3 − 6 7 ⎥ + ⎢− 2 − 1 4⎥⎥ = ⎢⎢1 − 7 11⎥⎥ ⎢⎣2 1 4⎥⎦ ⎢⎣ 5 3 9 ⎥⎦ ⎢⎣7 4 13⎥⎦ 1 6⎤ ⎡ 1 3 − 1⎤ ⎡3 4 5⎤ ⎡ 2 ⎢ ⎥ ⎢ ⎥ ⎢ b) A − B = ⎢3 − 6 7 ⎥ − ⎢− 2 − 1 4⎥ = ⎢ 5 − 5 3 ⎥⎥ ⎢⎣2 1 4⎥⎦ ⎢⎣ 5 3 9 ⎥⎦ ⎢⎣− 3 − 2 − 5⎥⎦ 1 6⎤ ⎡ 23 14 79⎤ ⎡3 4 5⎤ ⎡ 2 ⎢ ⎥ ⎢ c) AB = ⎢3 − 6 7 ⎥ ⎢− 2 − 1 4⎥⎥ = ⎢⎢53 30 57 ⎥⎥ ⎢⎣2 1 4⎥⎦ ⎢⎣ 5 3 9⎥⎦ ⎢⎣22 13 52⎥⎦

2

d)

1 6⎤ ⎡3 4 5⎤ ⎡ − 1 − 3 1 ⎤ ⎡2 ⎢ B − A = ⎢− 2 − 1 4⎥⎥ − ⎢⎢3 − 6 7 ⎥⎥ = ⎢⎢− 5 5 − 3⎥⎥ ⎢⎣ 5 3 9 ⎥⎦ ⎢⎣2 1 4⎥⎦ ⎢⎣ 3 2 5 ⎥⎦

1 6⎤ ⎡3 4 5⎤ ⎡ 21 8 41⎤ ⎡2 ⎢ e) BA = ⎢− 2 − 1 4⎥⎥ ⎢⎢3 − 6 7 ⎥⎥ = ⎢⎢− 1 2 − 1⎥⎥ ⎢⎣ 5 3 9 ⎥⎦ ⎢⎣2 1 4⎥⎦ ⎢⎣ 42 11 82 ⎥⎦

f) Trace( A) = 1 g) Trace(B ) = 10

(

)

h) Trace A 2 B = 767 Problem # 4

1 10⎤ ⎡− 3 1 ⎡8 0 − 3 4 ⎤ ⎥ ⎢ ⎢5 5 8 − 7⎥ 3 4 15 12⎥⎥ If A = ⎢ and B = ⎢ , compute ⎢ 0 −4 −8 9 ⎥ ⎢3 − 2 − 1 0 ⎥ ⎥ ⎥ ⎢ ⎢ 5 4⎦ 1 1 2⎦ ⎣2 ⎣0 7 a) 2 A

e) AB

b) A + B

f) BA

d) (2 A) − (3B )

c) 2 A − 3B

g) AT B T

T

h) (BA)

Solution:

⎡8 0 − 3 4 ⎤ ⎡16 0 − 6 8 ⎤ ⎢5 5 8 − 7 ⎥⎥ ⎢⎢10 10 16 − 14⎥⎥ ⎢ a) 2A = 2 = ⎢3 − 2 − 1 0 ⎥ ⎢ 6 − 4 − 2 0 ⎥ ⎥ ⎥ ⎢ ⎢ 5 4 ⎦ ⎣ 0 14 10 8 ⎦ ⎣0 7 1 10⎤ ⎡5 1 − 2 14⎤ ⎡8 0 − 3 4 ⎤ ⎡− 3 1 ⎢ ⎥ ⎢5 5 8 − 7⎥ ⎢ 3 4 15 12⎥⎥ ⎢⎢8 9 23 5 ⎥⎥ b) A + B = ⎢ = + ⎢3 − 2 − 1 0 ⎥ ⎢ 0 − 4 − 8 9 ⎥ ⎢ 3 − 6 − 9 9 ⎥ ⎥ ⎥ ⎢ ⎥ ⎢ ⎢ 5 4⎦ ⎣2 1 1 2 ⎦ ⎣2 8 6 5⎦ ⎣0 7

3

T

T

1 10⎤ ⎡ 25 − 3 − 9 − 22⎤ ⎡8 0 − 3 4 ⎤ ⎡− 3 1 ⎢ ⎥ ⎢5 5 8 − 7⎥ ⎢ 3 4 15 12⎥⎥ ⎢⎢ 1 − 2 − 29 − 50 ⎥⎥ c) 2 A − 3B = 2⎢ = −3 ⎢3 − 2 − 1 0 ⎥ ⎢ 0 − 4 − 8 9 ⎥ ⎢ 6 8 22 − 27⎥ ⎥ ⎥ ⎢ ⎥ ⎢ ⎢ 5 4⎦ ⎣2 1 1 2 ⎦ ⎣− 6 11 7 5 ⎦ ⎣0 7 5 3 0⎤ ⎡− 3 3 0 2⎤ ⎡ 25 − 6⎤ 1 6 ⎡8 ⎥ ⎢ ⎢ ⎥ ⎢0 5 − 2 7⎥ ⎢ 1 4 − 4 1⎥ ⎢ 3 −2 8 11 ⎥⎥ T T ⎢ = d) 2 A − 3B = 2 −3 ⎢− 3 8 − 1 5⎥ ⎢ 1 15 − 8 1 ⎥ ⎢ − 9 − 29 22 7⎥ ⎥ ⎥ ⎢ ⎥ ⎢ ⎢ ⎣ 4 − 7 0 2⎦ ⎣ 10 12 9 2⎦ ⎣− 22 − 50 − 27 5 ⎦ 1 10⎤ ⎡− 16 24 36 57 ⎤ ⎡8 0 − 3 4 ⎤ ⎡− 3 1 ⎢5 5 8 − 7 ⎥⎥ ⎢⎢ 3 4 15 12⎥⎥ ⎢⎢− 14 − 14 9 175⎥⎥ e) AB = ⎢ = ⎢3 − 2 − 1 0 ⎥ ⎢ 0 − 4 − 8 9 ⎥ ⎢− 15 − 1 − 19 − 3 ⎥ ⎥ ⎥ ⎢ ⎥⎢ ⎢ 5 4 ⎦⎣ 2 1 1 2 ⎦ ⎣ 29 12 69 133⎦ ⎣0 7

f)

1 10⎤ ⎡8 0 − 3 4 ⎤ ⎡ − 16 ⎡− 3 1 ⎢3 4 15 12⎥⎥ ⎢⎢5 5 8 − 7⎥⎥ ⎢⎢ 89 ⎢ = BA = ⎢ 0 − 4 − 8 9 ⎥ ⎢3 − 2 − 1 0 ⎥ ⎢− 44 ⎥ ⎢ ⎥⎢ ⎢ 1 1 2 ⎦ ⎣0 7 5 4 ⎦ ⎣ 24 ⎣2

5 3 0⎤ ⎡− 3 3 0 2⎤ ⎡− 16 ⎡8 ⎢0 5 − 2 7⎥⎥ ⎢⎢ 1 4 − 4 1 ⎥⎥ ⎢⎢ 73 T T ⎢ = g) A B = ⎢− 3 8 − 1 5⎥ ⎢ 1 15 − 8 1 ⎥ ⎢ 66 ⎥ ⎢ ⎥⎢ ⎢ ⎣ 4 − 7 0 2⎦ ⎣ 10 12 9 2⎦ ⎣ 21 ⎡− 16 ⎢ 73 h) ( BA) T = AT B T = ⎢ ⎢ 66 ⎢ ⎣ 21

89 − 44 24⎤ 74 59 10 ⎥⎥ 68 21 6⎥ ⎥ 32 64 5⎦

Problem # 5

Prove that: a) ( AT )T = A and ( A H ) H = A b) ( AB)T = B T AT and ( AB) H = B H A H Solution:

4

73 66 21⎤ 74 68 32⎥⎥ 59 21 64⎥ ⎥ 10 6 5 ⎦ 89 − 44 24⎤ 74 59 10 ⎥⎥ 68 21 6⎥ ⎥ 32 64 5⎦

a) ( A H ) ij = ( A) ji

→ [( A H ) H ]ij = ( A) ij = ( A) ij Therefore, ( A H ) H = A

b) N

N

k =1

k =1

( AB) ij = ∑ aik bkj → [( AB) H ]ij = ∑ a jk bki

(1)

Also N

N

k =1

k =1

( B H A H ) ij = ∑ ( B H ) ik ( A H ) kj = ∑ bki a jk

(2)

Compare Equation (1) with Equation (2), then ( AB) H = B H A H Problem # 6

Which of the following matrices are in row echelon form? ⎡1 − 1 0 8 9 ⎤ a) ⎢ ⎥ ⎣0 1 3 0 0 ⎦

⎡1 0 0 ⎤ b) ⎢⎢0 0 0⎥⎥ ⎢⎣0 1 0⎥⎦

⎡1 2 3 ⎤ c) ⎢⎢0 0 1⎥⎥ ⎢⎣0 0 0⎥⎦

⎡1 − 8 − 7 0 0 ⎤ d) ⎢⎢0 0 1 2 3⎥⎥ ⎢⎣0 0 0 1 1⎥⎦

⎡0 0 ⎤ e) ⎢ ⎥ ⎣1 0 ⎦

⎡1 0 9 ⎤ f) ⎢ ⎥ ⎣0 0 1 ⎦

Solution:

⎡1 − 1 0 8 9 ⎤ a) ⎢ ⎥ is in row echelon form. ⎣0 1 3 0 0 ⎦ ⎡1 0 0 ⎤ b) ⎢⎢0 0 0⎥⎥ is not in row echelon form since row two with entries all zero is above row three ⎢⎣0 1 0⎥⎦

with nonzero entry.

5

⎡1 2 3 ⎤ c) ⎢⎢0 0 1⎥⎥ is in row echelon form. ⎢⎣0 0 0⎥⎦ ⎡1 − 8 − 7 0 0 ⎤ d) ⎢⎢0 0 1 2 3⎥⎥ is in row echelon form. ⎢⎣0 0 0 1 1⎥⎦

⎡0 0 ⎤ e) ⎢ ⎥ is not in row echelon form. ⎣1 0 ⎦ f)

⎡1 0 9 ⎤ ⎢0 0 1⎥ is in row echelon form. ⎣ ⎦

Problem # 7

For the following matrices, show that Trace( AB) = Trace( BA) 3⎤ ⎡2 ⎢ B = ⎢− 3 − 4⎥⎥ ⎢⎣ 2 − 6⎥⎦

⎡ − 6 2 − 5⎤ A=⎢ ⎥ ⎣ 2 3 − 1⎦ Solution 3⎤ ⎡2 ⎡− 28 4⎤ ⎡− 6 2 − 5⎤ ⎢ − 3 − 4⎥⎥ = ⎢ AB = ⎢ ⎥ ⎥ ⎢ ⎣ 2 3 − 1⎦ ⎢ 2 − 6 ⎥ ⎣ − 7 0 ⎦ ⎣ ⎦

→ Trace( AB) = −28

3⎤ ⎡ − 6 13 − 13⎤ ⎡2 ⎡ − 6 2 − 5⎤ ⎢ ⎢ ⎥ = ⎢ 10 − 18 19 ⎥⎥ → Trace( BA) = −6 − 18 − 4 = −28 BA = ⎢− 3 − 4⎥ ⎢ ⎥ 2 3 − 1⎦ ⎢⎣− 24 − 14 − 4 ⎥⎦ ⎢⎣ 2 − 6⎥⎦ ⎣

Therefore, Trace( AB) = Trace( BA) Problem # 8

a) Find AB and BA :

6

⎡1 0 − 1⎤ B = ⎢⎢0 1 2 ⎥⎥ ⎢⎣1 2 − 1⎥⎦

⎡ 1 2 3⎤ A = ⎢⎢4 5 6⎥⎥ , ⎢⎣7 8 9⎥⎦

b) Is AB = BA ? Solution:

a) ⎡1 2 3⎤ ⎡1 0 − 1⎤ ⎡ 4 8 0⎤ AB = ⎢⎢4 5 6⎥⎥ ⎢⎢0 1 2 ⎥⎥ = ⎢⎢10 17 0⎥⎥ ⎢⎣7 8 9⎥⎦ ⎢⎣1 2 − 1⎥⎦ ⎢⎣16 26 0⎥⎦

b) ⎡1 0 − 1⎤ ⎡1 2 3⎤ ⎡− 6 − 6 − 6⎤ BA = ⎢⎢0 1 2 ⎥⎥ ⎢⎢4 5 6⎥⎥ = ⎢⎢ 18 21 24 ⎥⎥ ⎢⎣1 2 − 1⎥⎦ ⎢⎣7 8 9⎥⎦ ⎢⎣ 2 4 6 ⎥⎦

Therefore, AB ≠ BA

Problem # 9

Given matrices A and B , compute: a) AB b) BA c) AB − B T AT

⎡1 2 3 ⎤ A = ⎢⎢0 1 2⎥⎥ , ⎢⎣0 0 1⎥⎦

⎡1 0 0 ⎤ B = ⎢⎢2 1 0⎥⎥ ⎢⎣3 2 1⎥⎦

Solution:

a)

7

⎡1 2 3⎤ ⎡1 0 0⎤ ⎡14 8 3⎤ AB = ⎢⎢0 1 2⎥⎥ ⎢⎢2 1 0⎥⎥ = ⎢⎢ 8 5 2⎥⎥ ⎢⎣0 0 1⎥⎦ ⎢⎣3 2 1⎥⎦ ⎢⎣ 3 2 1 ⎥⎦

b) ⎡ 1 0 0 ⎤ ⎡1 2 3 ⎤ ⎡ 1 2 3 ⎤ BA = ⎢⎢2 1 0⎥⎥ ⎢⎢0 1 2⎥⎥ = ⎢⎢2 5 8 ⎥⎥ ⎢⎣3 2 1⎥⎦ ⎢⎣0 0 1 ⎥⎦ ⎢⎣3 8 14⎥⎦

c) Since AT = B and B T = A , then ⎡0 0 0 ⎤ AB − B A = AB − AB = 0 = ⎢⎢0 0 0⎥⎥ ⎢⎣0 0 0⎥⎦ T

T

Problem # 10

2 ⎡ 3 ⎢9 0 Given that A = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4

8 − 12⎤ ⎡− 4 0 ⎢ 3 ⎥ 3 0 3 ⎥ , B=⎢ ⎢1 7 5 8 ⎥ ⎢ ⎥ 2 1 ⎦ ⎣ 2 −1

that: a) A(B + C ) = AB + AC b) A(BC ) = ( AB )C Solution:

a)

8

0 0 ⎤ ⎡6 ⎢0 ⎥ 7 − 2⎥ and C = ⎢ ⎢0 6 − 5⎥ ⎢ ⎥ 0 9 ⎦ ⎣3

1 − 10 5 −5 0 2 7 6

0⎤ 4⎥⎥ , show 1⎥ ⎥ 1⎦

⎡− 4 0 ⎢3 3 B+C = ⎢ ⎢1 7 ⎢ ⎣ 2 −1

0 0 ⎤ ⎡6 1 − 10 7 − 2⎥⎥ ⎢⎢0 5 − 5 + 6 − 5 ⎥ ⎢0 0 2 ⎥ ⎢ 0 9 ⎦ ⎣3 7 6

2 ⎡3 ⎢9 0 A(B + C ) = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4 2 ⎡3 ⎢9 0 AB = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4 2 ⎡3 ⎢9 0 AC = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4

8 − 12⎤ ⎡2 0 3 ⎥⎥ ⎢⎢3 5 8 ⎥ ⎢1 ⎥⎢ 2 1 ⎦ ⎣5

1 − 10 5 −5 0 2 7 6

1 − 10 0 ⎤ 8 2 2 ⎥⎥ 7 8 − 4⎥ ⎥ 6 6 10 ⎦

1 − 10 0 ⎤ ⎡− 40 3 − 34 − 148⎤ ⎥ ⎢ 8 2 2 ⎥ ⎢ 33 27 − 72 30 ⎥⎥ = 7 8 72 ⎥ − 4⎥ ⎢ 67 133 80 ⎥ ⎢ ⎥ 6 6 10 ⎦ ⎣ − 9 − 14 34 −6 ⎦

8 − 12⎤ ⎡− 4 0 0 3 ⎥⎥ ⎢⎢ 3 3 5 8 ⎥⎢ 1 7 ⎥⎢ 2 1 ⎦⎣ 2 −1 8 − 12⎤ ⎡6 0 3 ⎥⎥ ⎢⎢0 5 8 ⎥ ⎢0 ⎥⎢ 2 1 ⎦ ⎣3

0⎤ ⎡2 4⎥⎥ ⎢⎢3 = 1 ⎥ ⎢1 ⎥ ⎢ 1 ⎦ ⎣5

0 0 ⎤ ⎡− 22 74 62 − 152⎤ 7 − 2⎥⎥ ⎢⎢ − 30 − 3 0 27 ⎥⎥ = 6 − 5⎥ ⎢ 31 45 72 35 ⎥ ⎥ ⎢ ⎥ 0 9 ⎦ ⎣ 0 1 − 16 7 ⎦ 0⎤ ⎡− 18 − 71 − 96 4 ⎤ 4⎥⎥ ⎢⎢ 63 30 − 72 3 ⎥⎥ = 1 ⎥ ⎢ 36 88 8 37 ⎥ ⎥ ⎢ ⎥ 1 ⎦ ⎣ − 9 − 15 50 − 13⎦

3 − 34 − 148⎤ ⎡− 22 74 62 − 152⎤ ⎡− 18 − 71 − 96 4 ⎤ ⎡− 40 ⎢ − 30 − 3 0 ⎥ ⎢ ⎥ ⎢ 27 − 72 30 ⎥⎥ 27 ⎥ ⎢ 63 30 − 72 3 ⎥ ⎢ 33 ⎢ AB + AC = + = ⎢ 31 45 72 72 ⎥ 35 ⎥ ⎢ 36 88 8 37 ⎥ ⎢ 67 133 80 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 1 − 16 7 ⎦ ⎣ − 9 − 15 50 − 13⎦ ⎣ − 9 − 14 34 −6 ⎦ ⎣ 0 Therefore, A(B + C ) = AB + AC

b) ⎡− 4 0 ⎢3 3 BC = ⎢ ⎢1 7 ⎢ ⎣ 2 −1

0 0 ⎤ ⎡6 1 − 10 7 − 2⎥⎥ ⎢⎢0 5 − 5 6 − 5 ⎥ ⎢0 0 2 ⎥⎢ 0 9 ⎦ ⎣3 7 6

0⎤ ⎡− 24 − 4 40 0⎤ ⎥ ⎢ 4⎥ ⎢ 12 4 − 43 17 ⎥⎥ = 1⎥ ⎢ − 9 1 − 63 29⎥ ⎥ ⎢ ⎥ 1 ⎦ ⎣ 39 60 39 5⎦

9

2 ⎡3 ⎢9 0 AB = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4

8 − 12⎤ ⎡− 4 0 0 3 ⎥⎥ ⎢⎢ 3 3 5 8 ⎥⎢ 1 7 ⎥⎢ 2 1 ⎦⎣ 2 −1

2 ⎡3 ⎢9 0 A(BC ) = ⎢ ⎢2 6 ⎢ ⎣− 2 − 4

0 0 ⎤ ⎡− 22 74 62 − 152⎤ 7 − 2⎥⎥ ⎢⎢ − 30 − 3 0 27 ⎥⎥ = 6 − 5⎥ ⎢ 31 45 72 35 ⎥ ⎥ ⎢ ⎥ 0 9 ⎦ ⎣ 0 1 − 16 7 ⎦

8 − 12⎤ ⎡− 24 − 4 40 0 ⎤ ⎡− 588 − 716 − 938 206⎤ ⎥ ⎢ 0 3 ⎥ ⎢ 12 4 − 43 17 ⎥⎥ ⎢⎢ − 99 144 477 15 ⎥⎥ = 5 8 ⎥⎢ − 9 1 − 63 29⎥ ⎢ 291 501 − 181 287 ⎥ ⎥⎢ ⎥ ⎢ ⎥ 2 1 ⎦ ⎣ 39 60 39 5 ⎦ ⎣ 21 54 5 −5⎦

⎡− 22 74 62 − 152⎤ ⎡6 ⎢ − 30 − 3 0 27 ⎥⎥ ⎢⎢0 ( AB )C = ⎢ ⎢ 31 45 72 35 ⎥ ⎢0 ⎢ ⎥⎢ 1 − 16 7 ⎦ ⎣3 ⎣ 0

1 − 10 5 −5 0 2 7 6

0⎤ ⎡− 588 − 716 − 938 206⎤ 4⎥⎥ ⎢⎢ − 99 144 477 15 ⎥⎥ = 1 ⎥ ⎢ 291 501 − 181 287⎥ ⎥ ⎢ ⎥ 1 ⎦ ⎣ 21 54 5 −5⎦

Therefore, A(BC ) = ( AB )C

Problem # 11 ⎡ 1 ⎢ Let A = ⎢ 2 1 ⎢− ⎣ 2

1⎤ − ⎥ 2 . Compute A 2 and A 3 . What can be said about A n ? 1 ⎥ ⎥ 2 ⎦

Solution ⎡ 1 ⎢ A2 = A × A = ⎢ 2 1 ⎢− ⎣ 2 ⎡ 1 ⎢ A3 = A × A 2 = ⎢ 2 1 ⎢− ⎣ 2

1⎤ − ⎥ 2 1 ⎥ ⎥ 2 ⎦

⎡ 1 ⎢ 2 ⎢ 1 ⎢− ⎣ 2

1⎤ − ⎥ 2 1 ⎥ ⎥ 2 ⎦

⎡ 1 ⎢ 2 ⎢ 1 ⎢− ⎣ 2

1⎤ ⎡ 1 − ⎥ ⎢ 2 = 2 1 ⎥ ⎢ 1 ⎥ ⎢− 2 ⎦ ⎣ 2 1⎤ ⎡ 1 − ⎥ ⎢ 2 = 2 1 ⎥ ⎢ 1 ⎥ ⎢− 2 ⎦ ⎣ 2

1⎤ − ⎥ 2 1 ⎥ ⎥ 2 ⎦ 1⎤ − ⎥ 2 1 ⎥ ⎥ 2 ⎦

10

⎡ 1 ⎢ → An = ⎢ 2 1 ⎢− ⎣ 2

1⎤ − ⎥ 2 1 ⎥ ⎥ 2 ⎦

Problem # 12

Show that the inverse of the DFT matrix A is A −1 =

1 H A , where: N

⎡WN0 WN0 WN0 ⎢ 0 1 WN2 ⎢WN WN A=⎢ M M M ⎢ 0 N −2 2 ( N −2 ) WN ⎢WN WN ⎢W 0 W N −1 W 2 ( N −1) N N ⎣ N

L L L L L

⎤ ⎥ W ⎥ ⎥ ( N −2 )( N −1) ⎥ WN ⎥ ( N −1)( N −1) ⎥ WN ⎦ WN0

N −1 N

Solution:

Let B = A H , then bkl = alk∗ = [W N( l −1)( k −1) ]∗ = W N− ( l −1)( k −1)

Now N

N

N

N

m =1

m =1

m =1

m =1

( AB) kl = ∑ a km bml = ∑ W N( k −1)( m −1)W N−( m −1)(l −1) = ∑ W N( k −1)( m −1) −( m −1)(l −1) = ∑ W N( m −1)( k −l )

N −1

= ∑W m =0

( k −l ) m N

⎧N 1 − W N( k −l ) N 1 − e − j 2π ( k −l ) ⎪ = = =⎨ 2π − j ( k −l ) 1 − W N( k −l ) ⎪0 1− e N ⎩

k =l k ≠l

Therefore, AB = NI → A −1 =

1 1 B = AH N N

Problem # 13

For each of the following systems of equations, use the technique of Gaussian elimination to solve the system.

a)

3x1 − 6 x 2 = 9 4 x1 − 2 x 2 = 18

− 2 x1 + 4 x 2 − 2 x3 = 4

3x1 + 6 x2 + 3x3 = 6 b) − x1 − x 2 + 2 x3 = 3 4 x1 + 6 x2 = 0

11

c)

− 2 x1 + 2 x 2 + x3 = 4 6 x1 + 4 x 2 + 4 x3 = 10 − 3x1 + 8 x 2 + 5 x3 = 17

3⎤ 3 ⎤ ⎡3 − 6 9 ⎤ ⎡1 − 2 ⎡1 − 2 3⎤ ⎡1 − 2 →⎢ →⎢ →⎢ a) ⎢ ⎥ ⎥ ⎥ ⎥ ⎣4 − 2 18⎦ ⎣1 − 0.5 4.5⎦ ⎣0 1 1⎦ ⎣0 − 1.5 − 1.5⎦

Therefore:

x2 = 1 x1 = 3 + 2 x1 = 5

2 1 2⎤ 2 1 2⎤ ⎡ 3 6 3 6⎤ ⎡1 ⎡1 ⎡1 2 1 2 ⎤ ⎢ ⎢ ⎢ ⎥ ⎥ ⎥ b) ⎢− 1 − 1 2 3⎥ → ⎢− 1 − 1 2 3⎥ → ⎢− 1 − 1 2 3⎥ → ⎢⎢0 1 3 5⎥⎥ ⎢⎣ 4 6 0 0⎥⎦ ⎢⎣ 1 1.5 0 0⎥⎦ ⎢⎣ 0 0.5 2 3⎥⎦ ⎢⎣0 0.5 2 3⎥⎦

x1 = 2 − 2 x2 − x3 = 2 − 4 − 1 = −3 ⎡1 2 1 2⎤ ⎡1 2 1 2⎤ ⎢ ⎢ ⎥ ⎥ → ⎢0 1 3 5⎥ → ⎢0 1 3 5⎥ Then, x2 = 5 − 3 x3 = 2 ⎢⎣0 1 4 6⎥⎦ ⎢⎣0 0 1 1 ⎥⎦ x3 = 1 − 2 x1 + 4 x 2 − 2 x3 = 4 − 2 x1 + 2 x 2 + x3 = 4 6 x1 + 4 x 2 + 4 x3 = 10 − 3x1 + 8 x 2 + 5 x3 = 17 ⎡− 2 ⎢− 2 c) ⎢ ⎢6 ⎢ ⎣− 3

1 1 −2 ⎤ − 2⎤ ⎡1 − 2 ⎡1 − 2 4 −2 4⎤ ⎢1 − 1 − 0.5 − 2 ⎥ ⎢1 − 1 − 0.5 − 2⎥ ⎢ ⎢ 4 ⎥⎥ 2 1 5 ⎥ 5⎥ 2 2 2 2 ⎥ → ⎢1 ⎥→ → ⎢1 4 4 10⎥ 3 ⎥ 3⎥ 3 3 3 3 ⎢ ⎢ ⎥ 17 ⎥ 22 ⎥ 8 5 10 7 ⎢ ⎢ 8 5 17⎦ ⎢⎣1 − 3 − 3 − 3 ⎥⎦ ⎢⎣0 3 3 ⎥⎦ 3

− 2⎤ 1 1 − 2⎤ ⎡1 − 2 ⎡1 − 2 1 − 2⎤ ⎡1 − 2 ⎢0 1 − 1.5 0 ⎥ ⎢1 − 1 − 0.5 − 2⎥ ⎢ ⎢ ⎢ 0 1 − 1.5 0 ⎥⎥ 11 ⎥ 11 ⎥ 7 5 7 ⎢ ⎥→ ⎢0 ⎥ → ⎢0 1 7 11 → ⎢0 1 ⎥ 3⎥ 5⎥ 10 3 6 ⎢ ⎢ 10 5 ⎢ ⎥ 22 22 7 7 ⎥ ⎢ ⎢ ⎥ 0 0 0 0 0 1 0 1 ⎢ ⎥⎦ ⎣ ⎢⎣ ⎢⎣ 10 ⎥⎦ 10 ⎥⎦ 10 10

1 − 2⎤ ⎡1 − 2 − 2⎤ 1 ⎡1 − 2 x1 = −2 + 2 x2 − x3 = −2 + 3 − 1 = 0 ⎢0 1 − 1.5 0 ⎥ ⎢0 1 − 1.5 0 ⎥ ⎢ ⎥ ⎥ Therefore: x2 = 0 + 1.5 x3 = 1.5 22 11 → ⎢ ⎢0 0 ⎥ ⎢0 0 1⎥ 1 10 5⎥ x3 = 1 ⎢ ⎥ ⎢ 0⎦ 0 0 0 ⎥⎦ ⎢⎣0 0 ⎣0 0

12

Problem # 14

Write the transformation matrices for each of the following transformations. a) R2 → R2 − 4R1 b) R3 → −16 R3 c) C1 → C1 − 2C 3 d) C 2 → 3C 2 Solution:

Assume 3 × 3 matrices, then

a) R2 → R2 − 4R1

b) R3 → −16 R3

c) C1 → C1 − 2C 3

d) C 2 → 3C 2

⎡ 1 0 0⎤ E = ⎢⎢− 4 1 0⎥⎥ , and B = EA ⎢⎣ 0 0 0⎥⎦

→

→

0 ⎤ ⎡1 0 ⎢ E = ⎢0 1 0 ⎥⎥ , ⎢⎣0 0 − 16⎥⎦

and B = EA

→

⎡ 1 0 0⎤ E = ⎢⎢ 0 1 0⎥⎥ , ⎢⎣− 2 0 1⎥⎦

→

⎡1 0 0 ⎤ E = ⎢⎢0 3 0⎥⎥ , ⎢⎣0 0 1⎥⎦

and B = AE

and B = AE

Problem # 15

Show that if a matrix is skew-symmetric ( AT = − A ), then its diagonal entries must all be zero. Solution:

Since AT = − A , then a ji = −aij .

13

Now let i = j , therefore aii = − aii → aii = 0 Problem # 16

Let A be an n × n matrix and let B = A + AT and C = A − AT . Show that: a) B is symmetric and C is skew-symmetric. b) Every n × n matrix can be represented as a sum of a symmetric and skew-symmetric matrix. Solution:

a) B T = ( A + AT ) T = AT + ( AT ) T = AT + A = A + AT = B → Matrix B is symmetric.

C T = ( A − AT ) T = AT − ( AT ) T = AT − A = −( A − AT ) = −C → Matrix C is skewsymmetric. Let A be an arbitrarily matrix, then we can decompose matrix A into sum of two

b)

matrices as:

A=

where

A + AT A − AT + = B+C 2 2

B=

A + AT A − AT and C = . Cleary B is symmetric, C is skew-symmetric, and 2 2

A= B+C.

Problem # 17

For each of the following pairs of matrices, find an elementary matrix E such that EA = B . a)

⎡12 32⎤ ⎡3 8 ⎤ = A=⎢ , B ⎢ 5 16 ⎥ ⎥ ⎣ ⎦ ⎣5 16⎦

⎡ 7 6 3⎤ ⎡ − 8 7 4⎤ ⎥ ⎢ b) A = ⎢ 2 0 9 ⎥ , B = ⎢⎢ 2 0 9 ⎥⎥ ⎢⎣− 8 7 4⎥⎦ ⎢⎣ 7 6 3⎥⎦

14

4 ⎡1 ⎢0 6 c) A = ⎢ ⎢− 4 5 ⎢ ⎣ 3 −3

9⎤ 4 5 9⎤ ⎡1 ⎥ ⎢ 1⎥ 6 0 20 1 ⎥⎥ , B=⎢ ⎢− 4 5 2 − 3⎥ 2 − 3⎥ ⎥ ⎢ ⎥ 9 0⎦ 0⎦ ⎣ 3 −3 9 5 2

Solution:

⎡12 32⎤ ⎡4 0⎤ ⎡3 8 ⎤ a) B = ⎢ ⎥=⎢ ⎥⎢ ⎥ = EA ⎣ 5 16 ⎦ ⎣0 1⎦ ⎣5 16⎦

→

⎡4 0⎤ E=⎢ ⎥ ⎣0 1 ⎦

⎡ − 8 7 4 ⎤ ⎡0 0 1 ⎤ ⎡ 7 6 3 ⎤ b) B = ⎢⎢ 2 0 9 ⎥⎥ = ⎢⎢0 1 0⎥⎥ ⎢⎢ 2 0 9 ⎥⎥ = EA ⎢⎣ 7 6 3⎥⎦ ⎢⎣1 0 0⎥⎦ ⎢⎣− 8 7 4⎥⎦

4 5 9 ⎤ ⎡1 ⎡1 ⎢6 0 20 1 ⎥⎥ ⎢⎢0 = c) B = ⎢ ⎢− 4 5 2 − 3⎥ ⎢0 ⎢ ⎥ ⎢ 0 ⎦ ⎣0 ⎣ 3 −3 9

⎡0 0 1 ⎤ → E = ⎢⎢0 1 0⎥⎥ ⎢⎣1 0 0⎥⎦

0 0 0⎤ ⎡ 1 4 ⎥ ⎢ 1 0 2⎥ ⎢ 0 6 0 1 0⎥ ⎢− 4 5 ⎥⎢ 0 0 1⎦ ⎣ 3 − 3

5 9⎤ ⎡1 ⎥ ⎢0 2 1⎥ = EA → E = ⎢ ⎢0 2 − 3⎥ ⎥ ⎢ 9 0⎦ ⎣0

0 1 0 0

0 0 1 0

Problem # 18

For each of the following pairs of matrices, find an elementary matrix E such that AE = B . ⎡5 − 8 ⎤ ⎡5 2 ⎤ , B=⎢ a) A = ⎢ ⎥ ⎥ ⎣3 − 12⎦ ⎣3 − 6 ⎦

b)

4 0⎤ 2 0⎤ ⎡2 ⎡4 ⎥ ⎢ ⎢ A = ⎢ − 1 − 3 2 ⎥ , B = ⎢− 3 − 1 2 ⎥⎥ ⎢⎣− 5 6 10⎥⎦ ⎢⎣ 6 − 5 10⎥⎦

7 3⎤ 7 3⎤ ⎡3 8 ⎡10 8 ⎢2 − 1 0 ⎥ ⎢ 6⎥ 2 −1 0 6 ⎥⎥ ⎢ ⎢ , B= c) A = ⎢3 2 − 4 − 5⎥ ⎢− 1 2 − 4 − 5⎥ ⎢ ⎥ ⎢ ⎥ 5⎦ 5⎦ ⎣7 − 9 6 ⎣13 − 9 6 Solution:

⎡5 − 8 ⎤ ⎡5 2 ⎤ ⎡1 − 2⎤ a) B = ⎢ ⎥=⎢ ⎥⎢ ⎥ = AE ⎣3 − 12⎦ ⎣3 − 6⎦ ⎣0 1 ⎦

→

15

⎡1 − 2⎤ E=⎢ ⎥ ⎣0 1 ⎦

0⎤ 2⎥⎥ 0⎥ ⎥ 1⎦

4 0 ⎤ ⎡0 1 0 ⎤ 2 0⎤ ⎡ 2 ⎡4 ⎢ ⎥ ⎢ b) B = ⎢− 3 − 1 2 ⎥ = ⎢ − 1 − 3 2 ⎥⎥ ⎢⎢1 0 0⎥⎥ = AE ⎢⎣ 6 − 5 10⎥⎦ ⎢⎣− 5 6 10⎥⎦ ⎢⎣0 0 1⎥⎦

7 3 ⎤ ⎡3 8 7 3 ⎤ ⎡1 ⎡10 8 ⎢ 2 −1 0 ⎥ ⎢ 6 ⎥ ⎢2 − 1 0 6 ⎥⎥ ⎢⎢0 ⎢ = c) B = ⎢− 1 2 − 4 − 5⎥ ⎢3 2 − 4 − 5⎥ ⎢1 ⎢ ⎥ ⎢ ⎥⎢ 5 ⎦ ⎣7 − 9 6 5 ⎦ ⎣0 ⎣13 − 9 6

⎡0 1 0⎤ → E = ⎢⎢1 0 0⎥⎥ ⎢⎣0 0 1⎥⎦

0 0 0⎤ ⎡1 ⎥ ⎢0 1 0 0⎥ = AE → E = ⎢ ⎢1 0 1 0⎥ ⎥ ⎢ 0 0 1⎦ ⎣0

0 0 0⎤ 1 0 0⎥⎥ 0 1 0⎥ ⎥ 0 0 1⎦

Problem # 19

Is the transpose of an elementary matrix the same type of elementary matrix? Is the product of two elementary matrices an elementary matrix? Solution:

If E is a type I elementary matrix then it is symmetric and its transpose is a type I. If it is type II , it is symmetric and its transpose is type II elementary matrix . If E is a type III elementary matrix, then it is not symmetric however its transpose is type III.

For example if

⎡0 1 0 ⎤ ⎡0 1 0 ⎤ ⎥ ⎢ T E1 = ⎢1 0 0⎥ , then E1 = ⎢⎢1 0 0⎥⎥ = E1 . The product of two elementary matrices is another ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦

elementary matrix. Problem # 20

Let B = AT A . Show that bij = ai a j . T

Solution

Since B = AT A , we have: bij = ∑ ( AT ) ik ( A) kj = ∑ a ki a kj = aiT a j k

k

16

Problem # 21 5⎤ ⎡5 7 − 1 − 2 ⎤ ⎡3 4 ⎢ ⎥ Let A = ⎢2 − 1 − 7 ⎥ and B = ⎢⎢0 8 3 − 6⎥⎥ . ⎢⎣4 2 − 1 0 ⎥⎦ ⎢⎣4 6 − 8 ⎥⎦

a) Calculate Ab1 and Ab2 . b) Calculate AB and verify that the column vectors are as obtained in (a). Solution: 5 ⎤ ⎡5⎤ ⎡ 35 ⎤ 5 ⎤ ⎡7 ⎤ ⎡ 63 ⎤ ⎡3 4 ⎡3 4 ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ a) Ab1 = ⎢2 − 1 − 7 ⎥ ⎢0⎥ = ⎢ − 18⎥ , and Ab1 = ⎢2 − 1 − 7 ⎥⎥ ⎢⎢8 ⎥⎥ = ⎢⎢− 8⎥⎥ ⎢⎣4 6 − 8 ⎥⎦ ⎢⎣4⎥⎦ ⎢⎣− 12⎥⎦ ⎢⎣4 6 − 8 ⎥⎦ ⎢⎣2⎥⎦ ⎢⎣ 60 ⎥⎦

b) ⎡ 35 63 4 − 30⎤ 2 ⎥⎥ AB = ⎢⎢ − 18 − 8 2 ⎢⎣− 12 60 22 − 44⎥⎦ ↑ ↑ Ab1 Ab2 Problem # 22

Let ⎡1 0⎤ ⎡0 0 ⎤ ⎡4 3 ⎤ ⎡ 3 − 1⎤ ⎡0 1 ⎤ I =⎢ , E=⎢ , O=⎢ , A=⎢ , B=⎢ ⎥ ⎥ ⎥ ⎥ ⎥ ⎣0 1 ⎦ ⎣0 0 ⎦ ⎣ 2 − 6⎦ ⎣− 2 8 ⎦ ⎣1 0 ⎦

⎡C and C = ⎢ 11 ⎣C 21

⎡1 C12 ⎤ ⎢1 =⎢ C 22 ⎥⎦ ⎢1 ⎢⎣2

Perform each of the following block multiplications: ⎡O I ⎤ ⎡C11 a) ⎢ ⎥⎢ ⎣ I O ⎦ ⎣C 21

C12 ⎤ C 22 ⎥⎦

17

5⎤ 4 − 1 3 ⎥⎥ 1 0 − 4⎥ 2 5 6 ⎥⎦ 3

3

⎡ A O ⎤ ⎡C11 b) ⎢ ⎥⎢ ⎣O A⎦ ⎣C 21

C12 ⎤ C 22 ⎥⎦

⎡ B O ⎤ ⎡C11 c) ⎢ ⎥⎢ ⎣O I ⎦ ⎣C 21

C12 ⎤ C 22 ⎥⎦

⎡ E O ⎤ ⎡C11 d) ⎢ ⎥⎢ ⎣O E ⎦ ⎣C 21

C12 ⎤ C 22 ⎥⎦

Solution:

⎡O I ⎤ ⎡C11 a) ⎢ ⎥⎢ ⎣ I O ⎦ ⎣C 21

⎡ A O ⎤ ⎡C11 b) ⎢ ⎥⎢ ⎣O A⎦ ⎣C 21

⎡ B O ⎤ ⎡C11 c) ⎢ ⎥⎢ ⎣O I ⎦ ⎣C 21

⎡ E O ⎤ ⎡C11 d) ⎢ ⎥⎢ ⎣O E ⎦ ⎣C 21

C12 ⎤ ⎡C 21 = C 22 ⎥⎦ ⎢⎣C11

⎡1 C 22 ⎤ ⎢⎢2 = C12 ⎥⎦ ⎢1 ⎢ ⎣1

− 4⎤ 2 5 6 ⎥⎥ 3 3 5⎥ ⎥ 4 −1 3 ⎦

1

0

C12 ⎤ ⎡ AC11 = C 22 ⎥⎦ ⎢⎣ AC 21

24 9 29 ⎤ ⎡ 7 ⎢ AC12 ⎤ ⎢ − 4 − 18 12 − 8 ⎥⎥ = AC 22 ⎥⎦ ⎢ 10 10 15 2 ⎥ ⎢ ⎥ ⎣− 10 10 − 30 − 44⎦

C12 ⎤ ⎡ BC11 = C 22 ⎥⎦ ⎢⎣ C 21

⎡2 5 10 12 ⎤ BC12 ⎤ ⎢⎢6 26 − 14 14 ⎥⎥ = C 22 ⎥⎦ ⎢1 1 − 4⎥ 0 ⎢ ⎥ 5 6⎦ ⎣2 2

C12 ⎤ ⎡ EC11 = C 22 ⎥⎦ ⎢⎣ EC 21

⎡1 EC12 ⎤ ⎢⎢1 = EC 22 ⎥⎦ ⎢2 ⎢ ⎣1

4 −1 3 3 2 5 1

0

3⎤ 5 ⎥⎥ 6⎥ ⎥ − 4⎦

Problem # 23

Write each of the following systems of equations as a matrix equation: ⎧5 x + 6 x 2 = 1 a) ⎨ 1 ⎩ 8 x1 − 3 x 2 = 9

18

x1 + x3 = 4 ⎧ ⎪ b) ⎨ 3 x 2 + 8 x3 = 10 ⎪x − 5x − 7 x = 0 2 3 ⎩ 1

⎧ x1 + x 2 + x3 + x 4 ⎪ x3 + 7 x 4 ⎪ c) ⎨ ⎪ x 2 + 5 x3 − 2 x 4 ⎪⎩ x1 − 2 x 2 − 5 x3

=6 =9 =4 =7

Solution: ⎧5 x + 6 x 2 = 1 a) ⎨ 1 ⎩ 8 x1 − 3 x 2 = 9

⎡5 6 ⎤ ⎡ x1 ⎤ ⎡1⎤ → ⎢ ⎥⎢ ⎥ = ⎢ ⎥ ⎣8 − 3⎦ ⎣ x 2 ⎦ ⎣9⎦

x1 + x3 = 4 ⎧ ⎪ b) ⎨ 3 x 2 + 8 x3 = 10 ⎪x − 5x − 7 x = 0 2 3 ⎩ 1 ⎧ x1 + x 2 + x3 + x 4 ⎪ x3 + 7 x 4 ⎪ c) ⎨ ⎪ x 2 + 5 x3 − 2 x 4 ⎪⎩ x1 − 2 x 2 − 5 x3

→

2 ⎤ ⎡ x1 ⎤ ⎡ 4 ⎤ ⎡1 0 ⎢0 3 8 ⎥⎥ ⎢⎢ x 2 ⎥⎥ = ⎢⎢10⎥⎥ ⎢ ⎢⎣1 − 5 − 7 ⎥⎦ ⎢⎣ x3 ⎥⎦ ⎢⎣ 0 ⎥⎦

→

1 1 ⎤ ⎡ x1 ⎤ ⎡6⎤ ⎡1 1 ⎢0 0 1 7 ⎥⎥ ⎢⎢ x 2 ⎥⎥ ⎢⎢9 ⎥⎥ ⎢ = ⎢0 1 5 − 2⎥ ⎢ x3 ⎥ ⎢4 ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎣1 − 2 − 5 0 ⎦ ⎣ x 4 ⎦ ⎣ 7 ⎦

=6 =9 =4 =7

Problem # 24

Use back-substitution to solve the following systems of equations: ⎧ x + 5 x 2 = 10 a) ⎨ 1 2 x2 = 4 ⎩ ⎧ x1 + 3 x 2 − 6 x3 = 12 ⎪ b) ⎨ x 2 − 9 x3 = 8 ⎪ x3 = 5 ⎩

19

⎧5 x1 + 4 x 2 − 8 x3 + 7 x 4 − 5 x5 ⎪ x 2 − 10 x3 + 2 x 4 − x5 ⎪⎪ c) ⎨ x3 + 16 x 4 + 6 x5 ⎪ 3 x 4 − 4 x5 ⎪ ⎪⎩ x5

=1 =9 =6 =0 =9

Solution: ⎧ x + 5 x 2 = 10 a) ⎨ 1 2 x2 = 4 ⎩

→ x2 =

⎧ x1 + 3 x 2 − 6 x3 = 12 ⎪ b) ⎨ x 2 − 9 x3 = 8 ⎪ x3 = 5 ⎩

4 = 2 and x1 = 10 − 5 x 2 = 10 − 10 = 0 2

⎧ x3 = 5 ⎪ → ⎨ x 2 = 8 + 9 x3 = 8 + 45 = 53 ⎪ x = 12 + 6 x − 3 x = 12 + 30 − 159 = −117 3 2 ⎩ 1

⎧5 x1 + 4 x 2 − 8 x3 + 7 x 4 − 5 x5 ⎪ x 2 − 10 x3 + 2 x 4 − x5 ⎪⎪ x3 + 16 x 4 + 6 x5 c) ⎨ ⎪ 3 x 4 − 4 x5 ⎪ ⎪⎩ x5

⎧ x5 ⎪ =1 ⎪ x4 =9 ⎪ ⎪ = 6 → ⎨ x3 ⎪x =0 ⎪ 2 ⎪ =9 ⎪ x1 ⎩

=9 4 x5 + 0 36 = = 12 3 3 = 6 − 16 x 4 − 6 x5 = 6 − 16 × 12 − 6 × 9 = −240

=

= 9 + 10 x3 − 2 x 4 + x5 = 9 − 2400 − 24 + 9 = −2406 =

1 − 4 x 2 + 8 x3 − 7 x 4 + 5 x5 7666 = 5 5

Problem # 25

Find the inner product for the following vectors: a) x = [1 8 9 0] and y = [3 9 10 − 5] T

T

b) x = [8 − 9 − 2 0 0 − 1 3 4] and y = [6 6 − 5 0 2 2 − 1 0] T

T

Solution:

a) < x, y >= x T y = 1 × 3 + 8 × 9 + 9 × 10 − 0 × 5 = 3 + 72 + 90 = 165 b) < x, y >= x T y = 8 × 6 − 9 × 6 + 2 × 5 − 1 × 2 − 3 × 1 = 48 − 54 + 10 − 2 − 3 = −1

20

Problem # 26

Compute the outer product for each of the given combinations of vectors: a) x = [3 4 5 6] and y = [− 1 0 5] T

T

b) x = [4 3 − 7 0 6] and y = [2 1 0 − 9 3] T

T

Solution: ⎡− 34 0 170⎤ a) > x, y x, y | 2

y=

2

x Is the solution unique? If not, find another vector. Solution:

Using Cauchy inequality we have:

y=

|< x, s >| 2 2

x

≤

x

2

x

s 2

2

= s

2

→

⎡2⎤ x = s = ⎢⎢− 3⎥⎥ and y max = s ⎢⎣ 1 ⎥⎦

2

= 4 + 9 + 1 = 14

⎡2⎤ Solution is not unique. The general solution is x = αs = α ⎢⎢− 3⎥⎥ , where α is an arbitrary scale ⎢⎣ 1 ⎥⎦

factor. Problem # 8

Suppose that V is a real vector space with an inner product. Prove that: x+ y

2

= x + y 2

2

if and only if x and y are orthogonal. Solution:

x+ y

2

=< x + y, x + y >=< x, x > +2 < x, y > + < y, y >= x + y + 2 < x, y > 2

If x is orthogonal to y then, < x, y >= 0 and x + y Also if x + y

2

2

= x + y 2

2

2

= x + y , then 2

2

x + y + 2 < x, y >= x + y , therefore, < x, y >= 0 and x is orthogonal to y . 2

2

2

2

74

Problem # 9

Show that the following two vectors are orthogonal: x(n) = 3 exp( j

2π n) N

and y (n) = 4 exp( j

6π n) , n = 0, 1, L, N − 1 N

Solution: N −1

N −1

n =0

n =0

< x, y >= ∑ x( n) y ( n) =∑12 exp( j

2π 6π n) exp(− j n) = N N

4π 1 − exp(− j 4π ) 1−1 = 12 =0 n) = 12 4π 4π N 1 − exp(− j ) 1 − exp(− j ) N N

N −1

= 12∑ exp(− j n =0

Therefore the two vectors x and y are orthogonal. Problem # 10

Show that the L1 , L2 and L∞ norms on R n (or C n ) satisfy the following inequalities: x

2

n x

2

≤ x

∞

≤ x

2

≤ x1≤ n x

x1 n

≤ x

∞

2

≤ x1

Solution:

1.

x

= max( xi ) = max( xi ) ≤ 2

∞

i

i

x1 + x2 + L + xn 2

2

2

= x

(1)

2

Assume that k = arg max ( xi ) , then i

xk + xk + L + xk 2

x

= max( xi ) = max( xi ) = 2

∞

i

i

2

n

75

2

x1 + x2 + L + xn 2

≥

2

n

2

=

x

2

n

(2)

Using inequalities (1) and (2), we have: x

2

n

≤ x

∞

≤ x

2

2. x 1 = x1 + x2 + L + xn = ( x1 + x2 + L + xn ) 2 (3) =

n −1

n

x1 + x2 + L + xn + 2∑ ∑ xi x j ≥ 2

2

2

i =1 j =i +1

x1 + x2 + L + xn 2

2

n

n

i =1

i =1

2

x 1 = x1 + x2 + L + xn = ( x1 + x2 + L + xn ) 2 = (∑ xi ) 2 ≤ n∑ xi

Therefore we have: x

2

≤ x1≤ n x

= x

2

2

= n x

(4)

2

2

3. x

x

∞

= max( xi ) ≤ x1 + x2 + L + xn = x 1

∞

= max( xi ) =

(5)

i

i

xk + xk + L + xk n

≥

x1 + x2 + L + xn n

=

x1

(6)

n

Equations (5) and (6) indicate that:

x1 n

≤ x

∞

≤ x1

Problem # 11

If V is a real inner product space, show that: 1 2 2 < x, y >= [ x + y − x − y ] 4 Solution:

x+ y

2

= < x + y, x + y > = < x, x > + < x, y > + < y, x > + < y, y >= x + 2 < x, y > + y 2

76

2

(1)

Similarly, x− y

2

= < x − y, x − y > = < x, x > − < x, y > − < y, x > + < y, y >= x

2

− 2 < x, y > + y (2) 2

Subtracting equation (2) from (1) yields: [ x+ y − x− y 2

2

= 4 < x, y >

Therefore:

1 2 2 < x, y >= [ x + y − x − y ] 4 Problem # 12

If V is a complex inner product space, show that: 1 2 2 a) Real < x, y >= [ x + y − x − y ] 4 1 2 2 b) Im < x, y >= − [ jx + y − jx − y ] 4 Solution:

a) x+ y

2

= < x + y , x + y > = < x, x > + < x, y > + < y , x > + < y , y > = x + < x, y > + < x , y > + y 2

2

Therefore x+ y

2

= x + y + 2real(< x, y >) 2

2

(1)

x− y

2

= x + y

2

(2)

Similarly 2

− 2real(< x, y >)

Subtracting Equation (2) from Equation (1) results in:

77

x+ y − x− y 2

= 4Real < x, y > , Hence

2

1 2 2 Real < x, y >= [ x + y − x − y ] 4 b) jx + y

2

= < jx + y, jx + y > = < jx, jx > + < jx, y > + < y, jx > + < y, y >

= x + j < x , y > − j < x, y > + y 2

2

Therefore, jx + y = x + y − 2Im(< x, y >)

(3)

jx − y = x + y + 2Im(< x, y >)

(4)

2

2

2

Similarly, 2

2

2

Subtracting Equation (4) from Equation (3) results in: jx + y − jx − y = −4Im < x, y > . Therefore, 2

2

1 2 2 Im < x, y >= − [ jx + y − jx − y ] 4 Problem # 13

Show that if f (t ) is bandlimited to B Hz, then:

αn =

∞

∫ f (t )

2 B sin c(2 B(t −

−∞

n 1 n ))dt = f( ) 2B 2B 2B

where sinc(t ) is the bandlimited sinc function defined as: sinc(t ) =

sin(πt ) πt

Solution: Since f (t ) is bandlimited to B Hz. then,

78

1 2π

f (t ) =

2πB

∫π F (ω )e

j ωt

dω

−2 B

Substituting for f (t ) we have:

αn =

∞

∫

f (t ) 2 B sin c(2 B(t −

−∞

=

1 2π

2πB

∞

∫π

−2 B

F (ω )[ 2 B ∫ sin c(2 B(t − −∞

Let τ = 2 B(t −

αn = =

1 2π

1 2π

2πB

∫

− 2πB

1 2B

∞

−∞

2πB

∫

F (ω )e

n jω ( ) 2B

− 2πB

∫ sin c(τ )e

1 2π

1 2B

∫ ∫π F (ω )e

jωt

2 B sin c(2 B(t −

−∞− 2 B

n ))dtdω 2B

n ))e jωt dt ]dω 2B

F (ω )[ 2 B ∫ sin c(2 B(t −

2πB

∫π

n 1 ))e jωt dt ]dω = 2B 2π

∞

[ ∫ sin c(τ )e

j

ω 2B

τ

2πB

∫

− 2πB

τ +n

∞

F (ω )[ ∫

−∞

) jω ( 1 sin c(τ )e 2 B dτ ]dω 2B

dτ ]dω

−∞

j

−∞

αn =

∞ 2πB

n ) then, 2B

∞

Since

n 1 ))dt = 2B 2π

ω 2B

τ

⎧1 − 2πB < ω < 2πB dτ = u (ω + 2πB) − u (ω − 2πB) = ⎨ , then we have: otherwise ⎩0

F (ω )e

jω (

n ) 2B

[u (ω + 2πB ) − u (ω − 2πB )]dω =

−2 B

1 1 2 B 2π

2πB

∫π

F (ω )e

jω (

n ) 2B

−2 B

dω =

1 n f( ) 2B 2B

Problem # 14 (many errors)

Let S = {φ n (t )}n =0 be a set of functions defined over the time interval [0 T ] as: M −1

ϕ n (t ) =

2 cos(2πf c t + 2πnΔft ) T

This is a complete orthonormal set. Show that it is an orthonormal set, where f c = the carrier frequency for some integer k , and Δf is the frequency separation.

79

k represent T

This is a set of signals used to transmit digital data using M - ary FSK. Find the minimum frequency separation Δf that makes the set S an orthonormal set. Solution: T

T

0

0

< ϕ n (t ), ϕ m (t ) >= ∫ ϕ n (t )ϕ m (t )dt = ∫

2 2 cos(2πf c t + 2πnΔft ) cos(2πf c t + 2πmΔft )dt T T

T

T

=

1 1 cos(4πf c t + 2πnΔft + 2πmΔft )dt + ∫ cos(2πnΔft − 2πmΔft )dt ∫ T 0 T 0

=

sin(4πf c t + 2πnΔft + 2πmΔft ) T sin( 2πnΔft − 2πmΔft ) T |0 + |0 T (4πf c + 2πnΔf + 2πmΔf ) T (2πnΔf − 2πmΔf )

=

sin(4πf cT + 2πnΔfT + 2πmΔfT ) sin(2πnΔfT − 2πmΔfT ) + T (4πf c + 2πnΔf + 2πmΔf ) T (2πnΔf − 2πmΔf )

=

sin(4πk + 2πnΔfT + 2πmΔfT ) sin(2πnΔfT − 2πmΔfT ) + T (2πnΔf − 2πmΔf ) (4πk + 2πnTΔf + 2πmTΔf )

=

sin(2πnΔfT + 2πmΔfT ) sin(2πnΔfT − 2πmΔfT ) + 2(2πk + πnΔfT + πmΔfT ) 2(πnΔfT − πmΔfT )

Therefore, < ϕ n (t ),ϕ m (t ) > =

1 sin( 2πnΔfT + 2πmΔfT ) sin( 2πnΔfT − 2πmΔfT ) [ + ] 2π 2k + nΔfT + mΔfT nΔfT − mΔfT

If we choose 2ΔfT = l , where l is an integer, then we have: ⎧1 < ϕ n (t ), ϕ m (t ) > = ⎨ ⎩0

n=m n≠m

Problem # 15 ∞

For the inner product < x(t ), y (t ) > = ∫ x(t ) y (t )dt , prove the Schwarz inequality if: 0

a) x(t ) = 2e − t u (t ) and y (t ) = −3e −2t u (t )

80

b) x(t ) = te − t u (t ) and y (t ) = 2e − t u (t ) c) x(t ) = e − (1− j ) t u (t ) and y (t ) = 2e − (1+ j 2 ) t u (t ) Solution ∞

∞

0

0

∞

a) < x(t ), y (t ) > = ∫ x(t ) y (t )dt = − ∫ 2e 3e dt = −6∫ e −3t dt = 2e −3t |∞0 = −2 ∞

∞

−t

−2t

0

∞

1 1 b) < x(t ), y (t ) > = ∫ x(t ) y (t )dt = ∫ 2te e dt = 2 ∫ te −2t dt = −te −2t − e −2t |∞0 = 2 2 0 0 0 −t

−t

c) ∞

∞

0

0

< x(t ), y (t ) > = ∫ x(t ) y (t )dt = 2 ∫ e

=

−(1− j ) t

e

−(1− j 2 ) t

∞

dt = 2∫ e

−( 2 − j 3) t

0

2e − ( 2− j 3) t ∞ dt = |0 − (2 − j 3)

2 = 0.3077 + j 0.4615 2 − j3

Problem # 16

Show that in a normed vector space the following inequality holds: x − y ≤ x− y Solution: x = ( x − y) + y ≤ x − y + y

⇒

y = ( y − x) + x ≤ y − x + x = x − y + x

x − y ≤ x− y

⇒

x − y ≥− x− y

Combining Equations (1) and (2) we have: − x − y ≤ x − y ≤ x − y , or

x − y ≤ x− y

Problem # 17

Solve the following set of linear equations using QR factorization:

81

(1) (2)

x1 + 2 x 2 + 3x3 = 14 4 x1 + 5 x 2 + 6 x3 = 32 7 x1 − 3 x 2 − 2 x3 = −5 Solution:

Ax = b

3 ⎤ ⎡ x1 ⎤ ⎡ 14 ⎤ ⎡1 2 ⎢4 5 6 ⎥⎥ ⎢⎢ x 2 ⎥⎥ = ⎢⎢ 32 ⎥⎥ ⎢ ⎢⎣7 − 3 − 2⎥⎦ ⎢⎣ x3 ⎥⎦ ⎢⎣− 5⎥⎦

⇒

QR factorization of matrix A : 0.322 0.9387 ⎤ ⎡ 0.1231 ⎡8.124 0.1213 1.6002 ⎤ ⎢ ⎥ A = QR , where Q = ⎢0.4924 0.8014 − 0.3395⎥ and R = ⎢⎢ 0 6.1632 6.7827 ⎥⎥ ⎢⎣0.8616 − 0.5040 0.0599 ⎥⎦ ⎢⎣ 0 0 0.6591⎥⎦ Ax = b

⇒

QRx = b ⇒

Q T QRx = Q T b ⇒

0.322 0.9387 ⎤ ⎡ 0.1231 ⎢ T Q b = ⎢0.4924 0.8014 − 0.3395⎥⎥ ⎢⎣0.8616 − 0.5040 0.0599 ⎥⎦

T

Rx = Q T b

⎡ 14 ⎤ ⎡13.1708 ⎤ ⎢ 32 ⎥ = ⎢32.6745⎥ ⎢ ⎥ ⎢ ⎥ ⎢⎣− 5⎥⎦ ⎢⎣ 1.9772 ⎥⎦

Therefore: ⎡8.124 0.1213 1.6002 ⎤ ⎡ x1 ⎤ ⎡13.1708 ⎤ ⎢ 0 6.1632 6.7827 ⎥⎥ ⎢⎢ x 2 ⎥⎥ = ⎢⎢32.6745⎥⎥ ⎢ ⎢⎣ 0 0 0.6591⎥⎦ ⎢⎣ x3 ⎥⎦ ⎢⎣ 1.9772 ⎥⎦

Using back substitution method, we have: x3 =

1.9772 =3 0.6591

x2 =

32.6745 − 6.7827 x3 32.6745 − 6.7827 × 3 12.3264 = = =2 6.1632 6.1632 6.1632

x1 =

13.1708 − 0.1213 x 2 − 1.6002 x3 13.1708 − −0.1213 × 2 − 1.6002 × 3 8.124 = = =1 8.124 8.124 8.124

82

Problem # 18

Find the QR factorization of the 4 × 3 matrix A . ⎡ 1 −2 ⎢− 1 1 A=⎢ ⎢6 3 ⎢ 2 ⎣1

3⎤ 0⎥⎥ 4⎥ ⎥ 4⎦

To get Q matrix perform Gram-Schmidt orthogonalization processes on the columns of matrix A . ⎡ 1 ⎤ ⎡ 0.1601 ⎤ ⎢ ⎥ ⎢ ⎥ q1 1 ⎢− 1⎥ ⎢− 0.1601⎥ = = u1 = q1 39 ⎢ 6 ⎥ ⎢ 0.9608 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ 1 ⎦ ⎣ 0.1601 ⎦ ⎡− 2⎤ ⎡ 0.1601 ⎤ ⎡− 2.4359⎤ ⎢1 ⎥ ⎢− 0.1601⎥ ⎢ 1.4359 ⎥ ⎢ ⎥ ⎥=⎢ ⎥ − 2.722 × ⎢ x 2 = q 2 − < q 2 , u1 > u1 = ⎢3⎥ ⎢ 0.9608 ⎥ ⎢ 0.3846 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣2⎦ ⎣ 0.1601 ⎦ ⎣ 1.5641 ⎦

u2 =

x2 x2

⎡− 0.7485⎤ ⎢ 0.4412 ⎥ ⎥ =⎢ ⎢ 0.1182 ⎥ ⎢ ⎥ ⎣ 0.4806 ⎦

⎡3⎤ ⎡ 0.1601 ⎤ ⎡− 0.7485⎤ ⎡ 2.3172 ⎤ ⎢0 ⎥ ⎢− 0.1601⎥ ⎢ ⎥ ⎢ ⎥ ⎥ − 0.1497 × ⎢ 0.4412 ⎥ = ⎢ 0.7288 ⎥ x3 = q3 − < q3 , u1 > u1 − < q3 , u 2 > u 2 = ⎢ ⎥ − 4.964 × ⎢ ⎢4⎥ ⎢ 0.9608 ⎥ ⎢ 0.1182 ⎥ ⎢− 0.7869⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣4⎦ ⎣ 0.1601 ⎦ ⎣ 0.4806 ⎦ ⎣ 3.1332 ⎦ ⎡ 0.5733 ⎤ ⎡ 0.1601 − 0.7485 0.5733 ⎤ ⎢ 0.1803 ⎥ ⎢ 0.1803 ⎥⎥ x ⎥ . Therefore, Q = ⎢− 0.1601 0.4412 u3 = 3 = ⎢ ⎢- 0.1947⎥ ⎢ 0.9608 0.1182 − 0.1947 ⎥ x3 ⎢ ⎢ ⎥ ⎥ 0.4806 0.7752 ⎦ ⎣ 0.7752 ⎦ ⎣ 0.1601

83

2.7222 4.9640⎤ 3.2542 0.1497⎥⎥ 0 4.0419⎥⎦

⎡6.2450 and the matrix R is R = ⎢⎢ 0 ⎢⎣ 0

⎡ 1 −2 ⎢− 1 1 A=⎢ ⎢6 3 ⎢ 2 ⎣1

3⎤ ⎡ 0.1601 − 0.7485 0.5733 ⎤ ⎡6.2450 ⎥ ⎢− 0.1601 0.4412 0⎥ 0.1803 ⎥⎥ ⎢ ⎢ 0 = QR = ⎢ 0.9608 4⎥ 0.1182 − 0.1947⎥ ⎢ ⎥ ⎢ ⎥ ⎢⎣ 0 4⎦ 0 . 1601 0 . 4806 0 . 7752 ⎣ ⎦

2.7222 4.9640⎤ 3.2542 0.1497⎥⎥ 0 4.0419⎥⎦

Problem # 19 (MATLAB)

The following three functions are defined over the interval of (0 ∞) . Use the Gram-Schmidt orthogonalization process to obtain an orthonormal set over the interval of (0 ∞) . a)

f 1 ( x) = e − x u ( x)

b)

f 2 ( x) = 2e − x u ( x) − 3e −2 x u ( x)

c)

f 3 ( x) = xe − x u ( x)

Solution: ∞

f1 ( x)

2

= ∫ e − 2 x dx = 0.5

u1 ( x) =

→

0

∞

∞

0

0

f1 ( x) = 2e − x u ( x ) f1 ( x)

< f 2 ( x), u1 ( x) >= ∫ f 2 ( x)u1 ( x)dx = ∫ 2e − x (2e − x − 3e − 2 x )dx = 0

g 2 ( x) = f 2 ( x)− < f 2 ( x), u1 ( x) > u1 ( x) = 2e − x − 3e −2 x ∞

g 2 ( x)

2

= ∫ (2e − x − 3e − 2 x ) 2 dx = 0.25

u 2 ( x) =

→

0

∞

∞

0

0

∞

∞

< f 3 ( x), u1 ( x) >= ∫ f 3 ( x)u1 ( x)dx = ∫ 2 xe − 2 x dx = < f 3 ( x), u 2 ( x) >= ∫ f 3 ( x)u 2 ( x)dx = ∫ 2 xe (2e 0

−x

−x

2 4

− 3e

0

−2 x

∞

)dx = ∫ 4 xe 0

84

g 2 ( x) = 2(2e − x − 3e − 2 x )u ( x) g 2 ( x)

−2 x

∞

dx − ∫ 6 xe −3 x dx = 1 − 0

2 1 = 3 3

2 2 2e − x − (2e − x − 3e − 2 x ) 4 3 1 + 2e − 2 x ) 2 dx = 72

g 3 ( x) = f 3 ( x)− < f 3 ( x), u1 ( x) > u1 ( x)− < f 3 ( x), u 2 ( x) > u 2 ( x) = xe − x − g 3 ( x) = xe

u 3 ( x) =

−x

11 − e − x + 2e − 2 x 6

∞

→

g 3 ( x)

2

= ∫ ( xe − x − 0

g 3 ( x) = 2( 6 xe − x − 11e − x + 12e − 2 x )u ( x) → g 3 ( x)

11 − x e 6

⎧u1 ( x) = 2e − x u ( x) ⎪⎪ −x −2 x ⎨u 2 ( x) = 2(2e − 3e )u ( x) ⎪ −x −x −2 x ⎪⎩u 3 ( x) = 2( 6 xe − 11e + 12e )u ( x)

We can also use MATLAB to find the three functions u1 ( x), u1 ( x), and u3 ( x) numerically. The following MATLAB code is used to generate these three functions. function [V]=gramschmidt(A) [m,n]=size(A); V=[A(:,1)/norm(A(:,1))]; for j=2:1:n v=A(:,j); for i=1:size(V,2) a=v'*V(:,i); v=v-a*V(:,i); end if (norm(v))^4>=eps V=[V v/norm(v)]; else end end

N=8000; x=linspace(0,10,N); f1=exp(-x); f2=2*exp(-x)-3*exp(-2*x); f3=x.*exp(-x); dx=x(2)-x(1); a=sqrt(1/dx); [V]=gramschmidt([f1' f2' f3']); u1=a*V(:,1); u2=a*V(:,2); u3=a*V(:,3); plot(x,u1,x,u2,x,u3)

The plots of the three functions obtained using MATLAB are shown in Figure 3.1. They match the closed form solution nicely.

85

1.5

u1 ( x) 1

u3 ( x) 0.5

0

-0.5

u 2 ( x) -1

-1.5

-2 0

1

2

3

4

5

6

7

8

9

10

Figure 3.1: Plots of the three orthonormal functions obtained numerically Problem # 20

Show that for any two real sequences {a i }i =1 and {bi }i =1 n

n

∑ ai bi

n

2

i =1

n

≤ ∑ ai

2

i =1

n

∑b

2

i

i =1

with equality if and only if bi = kai for i = 1, 2, L , n Solution:

Let a = [a1

a 2 L a n ] and b = [b1 T

b2 L bn ] , then by Cauchy inequality we have: T

|< a, b >| 2 ≤ a n

But < a, b >= ∑ ai bi , a i =1

2

n

= ∑ ai ,and b i =1

2

2

n

2

= ∑ bi i =1

86

b 2

2

Therefore: n

∑ ai bi

2

i =1

n

≤ ∑ ai i =1

2

n

∑b i =1

i

2

.

Equality if and only if b = ka or bi = kai for i = 1, 2, L , n Problem # 21 (MATLAB)

Consider the set of vectors S = {x1 ⎡1⎤ ⎢− 1⎥ x1 = ⎢ ⎥ , ⎢2⎥ ⎢ ⎥ ⎣4⎦

x2

x3

x 4 }∈ R 4

⎡ − 2⎤ ⎢5 ⎥ x2 = ⎢ ⎥ , ⎢6 ⎥ ⎢ ⎥ ⎣9 ⎦

⎡1⎤ ⎢2⎥ x3 = ⎢ ⎥ , ⎢0⎥ ⎢ ⎥ ⎣− 3⎦

⎡4⎤ ⎢ − 2⎥ x4 = ⎢ ⎥ ⎢ − 4⎥ ⎢ ⎥ ⎣− 7 ⎦

Use the Gram-Schmidt orthogonalization process to obtain an orthonormal set of vectors. Solution: MATLAB Code

function [V]=gramschmidt(A) [m,n]=size(A); V=[A(:,1)/norm(A(:,1))]; for j=2:1:n v=A(:,j); for i=1:size(V,2) a=v'*V(:,i); v=v-a*V(:,i); end if (norm(v))^4>=eps V=[V v/norm(v)]; else end end Answers: U = [u1

u2

u3

x1=[1 -1 2 4]'; x2=[-2 5 6 9]'; x3=[1 2 0 -3]'; x4=[4 -2 -4 -7]'; A=[x1 x2 x3 x4]; [U]=gramschmidt(A);

u 4 ] , where

87

⎡ 0.2132 ⎤ ⎢- 0.2132⎥ ⎥, u1 = ⎢ ⎢ 0.4264 ⎥ ⎢ ⎥ ⎣ 0.8528 ⎦

⎡- 0.4631⎤ ⎢ 0.8228 ⎥ ⎥, u2 = ⎢ ⎢ 0.2724 ⎥ ⎢ ⎥ ⎣ 0.1853 ⎦

⎡ 0.7728 ⎤ ⎢ 0.3671 ⎥ ⎥, u3 = ⎢ ⎢ 0.4154 ⎥ ⎢ ⎥ ⎣- 0.3091⎦

⎡ 0.3780 ⎤ ⎢ 0.3780 ⎥ ⎥ u4 = ⎢ ⎢- 0.7559⎥ ⎢ ⎥ ⎣ 0.3780 ⎦

Problem # 22 (MATLAB)

a) Write a MATLAB code to compute L p norm of a continuous function f (x) defined over the interval of a ≤ x ≤ b . Assume that the function f is given by a look-up table with vector x and y containing N samples of the function uniformly spaced between a and b b) Test your code with a = 0 , b = 1 , N = 2000 , and f ( x) = 2 x 2 − x + 1 c) Use the code to find L p norm of the following functions for p = 1, 2, and ∞

f ( x) = 1 − x ln( x) − (1 − x) ln(1 − x) g ( x) = 1 + x + x cos(2πx) + 4 sin(πx) 0 < x = T); em(I)=0; figure(2) imshow(em) title('Gradient based edge map') h=[0 1 0; 1 -4 1;0 1 0]; alpha1=0.14; alpha2=0.05; Y=filter2(h,fg); T=ones(7,7)/49; fm=filter2(T,fg); e=fg-fm; var=filter2(T,e.^2); sd=sqrt(var); ZZ=ones(size(Y)); I=find(abs(Y)=alpha2); ZZ(I)=0; figure(3) imshow(ZZ) title('Laplacian based edge map') h=[-1 0 1;-2 0 2;-1 0 1]; GRx=filter2(h,f(:,:,1)); GRy=filter2(h',f(:,:,1)); GGx=filter2(h,f(:,:,2)); GGy=filter2(h',f(:,:,2)); GBx=filter2(h,f(:,:,3)); GBy=filter2(h',f(:,:,3)); A=GRx.^2+GGx.^2+GBx.^2; B=GRx.*GRy+GGx.*GGy+GBx.*GBy; C=GRy.^2+GGy.^2+GBy.^2; edge=0.5*(A+C)+0.5*sqrt((A-C).^2+4*B.^2); edge=255*edge/max(max(edge)); figure(3) imshow(edge) title('Gradient based color edge map') figure(4) imshow(f) title('RGB image')

133

Problem # 28 (MATLAB)

Consider the following two matrices: ⎡1 2⎤ ⎡1 3⎤ ⎢ ⎥ A=⎢ ⎥ and B = ⎢0 3⎥ 4 5 ⎣ ⎦ ⎣⎢2 4⎥⎦

a) Find eigenvalues and eigenvectors of A , BB T and B T B . b) Compute the SVD of matrix B . c) Compute the modal matrix of A . Solution: MATLAB Code:

A=[1 3; 4 5]; B=[1 2;0 3;2 4]; [V1,D1]=eig(A); [V2,D2]=eig(B*B'); [V3,D3]=eig(B'*B); [U,D,V]=svd(B);

Answers:

a) Eigenvalues and eigenvectors of matrix A :

λ1 = −1, λ2 = 7

⎡- 0.4472⎤ ⎡- 0.8321⎤ x1 = ⎢ , x2 = ⎢ ⎥ ⎥ ⎣- 0.8944⎦ ⎣ 0.5547 ⎦

Eigenvalues and eigenvectors of matrix BB T :

λ1 = 0,

λ2 = 1.3795,

and λ3 = 32.6205

⎡ 0.8944 ⎤ ⎡ - 0.4472 ⎤ ⎡ 0.3889⎤ ⎢ ⎢ ⎥ ⎥ x1 = ⎢ 0.0000 ⎥ , x2 = ⎢- 0.8695 ⎥ , and x3 = ⎢⎢ 0.4939 ⎥⎥ ⎢⎣- 0.4472⎥⎦ ⎢⎣ 0.4417 ⎥⎦ ⎢⎣ 0.7777 ⎥⎦

Eigenvalues and eigenvectors of matrix B T B :

134

λ1 = 1.3795,

and λ2 = 32.6205

⎡ - 0.9403⎤ ⎡0.3404⎤ x1 = ⎢ , and x2 = ⎢ ⎥ ⎥ ⎣ 0.3404 ⎦ ⎣0.9403⎦ b) SVD of matrix B . B =

rank ( B )

∑σ i =1

T 2 i i i

uv

σ 12 = 5.7114,

and σ 22 = 1.1745

rank ( B) = 2 ⎡- 0.3889⎤ ⎡ 0.2209 ⎤ ⎡ 0.9403 ⎤ ⎡- 0.3404⎤ ⎢ ⎥ , and v2 = ⎢ u1 = ⎢- 0.4939⎥ , u 2 = ⎢⎢ - 0.8695 ⎥⎥ , v1 = ⎢ ⎥ ⎥ ⎣- 0.3404 ⎦ ⎣- 0.9403⎦ ⎢⎣- 0.7777 ⎥⎦ ⎢⎣ 0.4417 ⎥⎦

c) Modal matrix of A : M = [x1

⎡- 0.8321 - 0.4472⎤ x2 ] = ⎢ ⎥ ⎣ 0.5547 - 0.8944 ⎦

Problem # 29 (MATLAB)

Consider the following two matrices: ⎡1 ⎢2 ⎢ A = ⎢− 4 ⎢ ⎢2 ⎢⎣ 1

2 3 3 3 2

3 −1 2 ⎤ 0 3 4 ⎥⎥ ⎡1 2⎤ 6 2 8 ⎥ and B = ⎢ ⎥ ⎥ ⎣3 4⎦ 5 7 8⎥ 0 − 1 − 1⎥⎦

a) Find eigenvalues and eigenvectors of A . b) Find eigenvalues and eigenvectors of B . c) Find eigenvalues and eigenvectors of A ⊗ B . d) Can you establish a relationship between the eigenvalues of A ⊗ B and the eigenvalues of A and B ?

135

e) Compute the SVD of matrix A . f) Compute the SVD of matrix B . g) Compute the SVD of matrix A ⊗ B . h) Is there a relationship between the SVDs of A , B , and A ⊗ B ? Solution: MATLAB Code: A=[1 2 3 -1 2; 2 3 0 3 4;-4 3 6 2 8; 2 3 5 7 8; 1 2 0 -1 -1]; B=[1 2; 3 4]; C=kron(A,B); DA=eig(A); DB=eig(B); DC=eig(C); [UA,SA,VA]=svd(A); [UB,SB,VB]=svd(B); [UC,SC,VC]=svd(C);

Results:

a) Eigenvalues of A are:

{λ Ak }5k =1 = {11.0510

- 1.7135

1.6579 + 3.4466i

1.6579 - 3.4466i 3.3467 }

b) Eigenvalues of B are:

{λBk }2k =1 = {- 0.3723

5.3723 }

c) Eigenvalues of C = A ⊗ B are:

{λCk }10k =1 = ⎧⎨

59.3691

⎩

8.9066 + 18.5162i 8.9066 - 18.5162i 17.9795 - 9.2054 0.6379 - 0.6172 + 1.2831i - 0.6172 - 1.2831i - 1.2459

{

- 4.1141

⎫ ⎬ ⎭

}

d) Eigenvalues of C = A ⊗ B are related to eigenvalues of A and B by: λCk = λBi λ Aj That is:

{λCk }10k =1 = {λB1λ A1 λB1λ A2 λB1λ A3 λB1λ A4 λB1λ A5 λB2 λ A1 λB2 λ A2 λB2 λ A3 λB2 λ A4 λB2 λ A5 }

136

e) Singular values of A are:

{σ }

2 5 Ak k =1

= {16.8139

6.3169

3.8577

2.3644

0.9569}

f) Singular values of B are:

{σ }

2 2 Bk k =1

= {5.4650

0.3660}

g) Singular values of C = A ⊗ B are:

{σ }

2 10 Ck k =1

= {91.8879 34.5220 21.0821 12.9216 6.1533 5.2293 2.3118 1.4118 0.8653 0.3502 }

{

2 = σ Bi2 σ Aj2 h) Singular values of C = A ⊗ B are related to Singular values of A and B by: σ Ck

}

That is:

{σ }

2 10 Ck k =1

{

= σ B21σ A21 σ B21σ A2 2 σ B21σ A2 3 σ B21σ A2 4 σ B21σ A2 5 σ B2 2σ A21 σ B2 2σ A2 2 σ B2 2σ A2 3 σ B2 2σ A2 4 σ B2 2σ A2 5

Problem # 30 (MATLAB)

A 2-d FIR filter h(n, m) is said to be separable if it can be written as product of two 1-d filters, that is: h(n, m) = h1 (n)h2 (m)

a) Find the necessary and sufficient condition for FIR filter h(n, m) to be separable. b) If h(n, m) is not separable, it can be approximated by a separable filter. Write a MATLAB script to approximate a non-separable filter by a separable filter c) Test your code using the following two filters: ⎡ 1 −1 2 ⎤ 1⎢ hI (n, m) = ⎢− 1 5 − 1⎥⎥ 7 ⎢⎣ 2 − 1 1 ⎥⎦

⎡1 0 − 1 ⎤ and hII (n, m) = ⎢⎢2 0 − 2⎥⎥ ⎢⎣1 0 − 1 ⎥⎦

Solution:

137

}

a) The necessary and sufficient condition for FIR filter h(n, m) to be separable is that matrix h(n, m) has rank one. randk(h) = 1 b) Matrix h(n, m) can be written in terms of its SVD as h=

rank ( h )

∑σ i =1

T 2 i i i

uv

If we ignore all singular values except the largest one, we will have: h ≈ σ 12 u1v1T = (σ i u1 )(σ i v1T ) = h1h2 where h1 = σ i u1 and h2 = σ i v1T are the two 1-d filters. The following MTALB function file can be used to approximate a nonseparable filter with a separable filter. c) % h = input 2-d filter % hr = 1-d row filter % hc = 1-d column filter % hs=hc*hr; % index=1 if filter h is separable % index=0 filter is not separable function [hr,hc,hs,index]=nons2sep(h) [U,S,V]=svd(h); a=sqrt(S(1,1)); hr=a*V(:,1)'; hc=a*U(:,1); b=hr(1); hr=hr/b; hc=hc*b; hs=hc*hr; if rank(h)==1 index=1; else index=0; end

h=(1/7)*[1 -1 2;-1 5 -1;2 -1 1]; [hr,hc,hs,index]=nons2sep(h);

h=[1 0 -1;2 0 -2;1 0 -1]; [hr,hc,hs,index]=nons2sep(h);

138

⎡ 1 −1 2 ⎤ 1⎢ hI (n, m) = ⎢− 1 5 − 1⎥⎥ index = 0 → Nonseparable. 7 ⎢⎣ 2 − 1 1 ⎥⎦ hr = [1

⎡ 0.0865 ⎤ - 2.7321 1], and hc = ⎢⎢- 0.2364⎥⎥ ⎢⎣ 0.0865 ⎥⎦

⎡1 0 − 1 ⎤ hII (n, m) = ⎢⎢2 0 − 2⎥⎥ ⎢⎣1 0 − 1 ⎥⎦ hr = [1

index = 1 → Separable.

⎡1 ⎤ 0 - 1] , and hc = ⎢⎢2⎥⎥ and hII = hr hc ⎢⎣1 ⎥⎦

Problem # 31 (MATLAB)

Let r (n), n = 0, ± 1, ± 2, L, ± m be the autocorrelation function of a zero mean real stationary process x(n) . The power spectral density of this process is defined as: S (ω ) =

m

∑ r ( n)e

− jnω

n=− m

a) If m = 7 and r (n) = 1 −

|n| , show that S (ω ) ≥ 0 and the (m + 1) × (m + 1) Toeplitz m

autocorrelation matrix R defined as R(i, j ) = r (i − j ) is positive definite. b) Show that in general if S (ω ) ≥ 0 .then the (m + 1) × (m + 1) Toeplitz autocorrelation matrix R defined as R (i, j ) = r (i − j ) is positive semidefinite. Solution:

a) S (ω ) =

m

∑ r ( n )e

n=− m

− jnω

2m

2m

2m

l =0

l =0

l =0

= ∑ r (l − m)e − j ( l − m )ω = e jmω ∑ r (l − m)e − jlω = e jmω ∑ ~ r (l )e − jlω

139

where ~ r (n) = r (n − m), n = 0, 1, 2, L, 2m , The above can be computed using FFT as: 2π

S(

2π

2π

− jl jm k 2 m k jm k ~ 2π k) = e N ∑~ r (l )e N = e N R (k ) N l =0

~ r (l ) . where R (k ) = N − point DFT(FFT) of 2m + 1 point sequence ~ The MATLAB code implementation is shown below.

m=7; n=-m:1:m; r=1-(abs(n)/m); R=toeplitz(r(8:end)); D=eig(R); N=4*1024; k=0:1:N-1; S=(exp(j*2*pi*m*k/N)).*(fft(r,N)); S=real(fftshift(S)); w=linspace(-pi,pi,N); plot(w,S) grid on xlabel('\omega') ylabel('S(\omega)')

140

7

6

5

S(ω)

4

3

2

1

0 -4

-3

-2

-1

0

1

2

3

4

ω

Figure 4.4 Plot of the power spectral density S (ω )

The eigenvalues of the (m + 1) × (m + 1) Toeplitz autocorrelation matrix R defined as R (i, j ) = r (i − j ) are:

{λk }8k =1 = { 0.0743

0.0831

0.1033

0.1374

0.2314

0.4011

1.8767

5.0927}

Since all eigenvalues are positive, the Toeplitz matrix R is positive definite. b) Let x be an arbitrary complex vector in C m×1 , then m

m

m

m

m

m

1 l = 0 k =0 2π

x ∗ Rx = ∑∑ R(k , l ) xk∗ xl = ∑∑ r (k − l ) xk∗ xl = ∑∑ l =0 k =0

=

=

1 2π 1 2π

π

l = 0 k =0

m

m

j ( k −l ) ω ∗ xk xl ) S (ω )dω = ∫ (∑∑ e

−π l =0 k =0

m

π

∑ ∫| x e k =1 −π

k

jkω 2

| S (ω )dω

141

1 2π

π

m

π

∫π S (ω )e

j ( k −l )ω

−

jkω 2 ∫ (∑| xk e | )S (ω )dω

−π k =0

dωxk∗ xl

Since S (ω ) ≥ 0 , we have: x ∗ Rx =

1 2π

m

π

∑ ∫| x e k =1 −π

k

matrix R is a positive semi definite matrix.

142

jkω 2

| S (ω )dω ≥ 0 for any vector x , therefore

CHAPTER 5 Problem # 1 ⎡2 − 1⎤ Consider the following square matrix A = ⎢ ⎥. ⎣0 3 ⎦ a) Compute A k and e At using the Cayley-Hamilton Theorem. b) Find the square roots of the matrix, i.e. find all matrices B such that B 2 = A using the Cayley-Hamilton Theorem method. Solution: 1 ⎤ ⎡λ − 2 λI − A = ⎢ = (λ − 2)(λ − 3) = 0 ⇒ λ1 = 2, λ 2 = 3 λ − 3⎥⎦ ⎣ 0 f (λ i ) = α 0 + α 1 λ i

i = 1, 2

Therefore: a)

f ( A) = A k

λ1k = α 0 + α1λ1 λk2 = α 0 + α1λ2

→

λ1k = α 0 + α 1λ1 λk2 = α 0 + α 1λ 2

→

⎧α 0 + 2α 1 = 2 k ⎨ k ⎩α 0 + 3α 1 = 3

⎧α 0 = 3 × 2 k − 2 × 3 k → ⎨ k k ⎩ α1 = 3 − 2

− α 1 ⎤ ⎡2 k ⎡1 0⎤ ⎡2 − 1⎤ ⎡α 0 + 2α 1 + = = α Ak = α 0 I + α1 A = α 0 ⎢ 1⎢ ⎥ ⎥ ⎢ 0 α 0 + 3α 1 ⎥⎦ ⎢⎣ 0 ⎣0 1 ⎦ ⎣0 3 ⎦ ⎣

f ( A) = e At e λ1t = α 0 + α 1λ1 e λ2 t = α 0 + α 1 λ 2

→

⎧α 0 + 2α 1 = e 2t ⎨ 3t ⎩α 0 + 3α 1 = e

⎧α = 3e 2t − 2e 3t → ⎨ 0 3t 2t ⎩ α1 = e − e

143

2 k − 3k ⎤ ⎥ 3k ⎦

− α 1 ⎤ ⎡e 2 t ⎡1 0⎤ ⎡2 − 1⎤ ⎡α 0 + 2α 1 + = = α e At = α 0 I + α 1 A = α 0 ⎢ 1⎢ ⎥ ⎥ ⎢ α 0 + 3α 1 ⎥⎦ ⎢⎣ 0 0 ⎣0 1 ⎦ ⎣0 3 ⎦ ⎣

e 2 t − e 3t ⎤ ⎥ e 3t ⎦

b) B=

A

± λ1 = α 0 + α1λ1 ± λ2 = α 0 + α1λ2

→

⎧α 0 + 2α 1 = ± 2 ⎨ ⎩α 0 + 3α 1 = ± 3

⎧α = ±3 × 2 m 2 × 3 → ⎨ 0 ⎩ α1 = ± 3 m 2

− α 1 ⎤ ⎡± 2 ⎡1 0 ⎤ ⎡2 − 1⎤ ⎡α 0 + 2α 1 + α1 ⎢ =⎢ B = α 0 I + α1 A = α 0 ⎢ =⎢ ⎥ ⎥ 0 α 0 + 3α 1 ⎥⎦ ⎣ 0 ⎣0 1 ⎦ ⎣0 3 ⎦ ⎣ There are 4 solutions listed below: ⎡ 2 B1 = ⎢ ⎣0

2 − 3 ⎤ ⎡1.4142 − 0.3178⎤ ⎥=⎢ 1.7321 ⎥⎦ 3 ⎦ ⎣ 0

⎡ 2 B2 = ⎢ ⎣0

2 + 3 ⎤ ⎡1.4142 3.1463 ⎤ ⎥=⎢ − 1.7321⎥⎦ − 3 ⎦ ⎣ 0

⎡− 2 B3 = ⎢ ⎣ 0

− 2 + 3 ⎤ ⎡− 1.4142 0.3178 ⎤ ⎥=⎢ − 1.7321⎥⎦ − 3 ⎦ ⎣ 0

⎡− 2 B4 = ⎢ ⎣ 0

− 2 − 3 ⎤ ⎡− 1.4142 − 3.1463⎤ ⎥=⎢ 1.7321 ⎥⎦ 3 ⎦ ⎣ 0

Problem # 2 ⎡− 1 2 − 1⎤ Let A = ⎢⎢ 0 1 3 ⎥⎥ . ⎢⎣− 1 2 5 ⎥⎦

a) Find P(λ ) , the characteristic polynomial of A . b) Find eigenvalues of A . c) Show that P( A) = 0 . 144

± 2 m 3⎤ ⎥ ± 3 ⎦

Solution: 1 ⎤ ⎡λ + 1 − 2 ⎢ a) P(λ ) = λI − A = 0 λ − 1 − 3 ⎥⎥ = λ3 − 5λ 2 − 8λ + 6 = 0 ⎢ − 2 λ − 5⎦⎥ ⎣⎢ − 1

b) λ3 − 5λ2 − 8λ + 6 = 0

⇒ λ1 = 6.1433, λ 2 = −1.7133,

λ3 = 0.5701

c) 2 ⎤ ⎡ 2 − 2 2 ⎤ ⎡− 1 2 − 1⎤ ⎡6 0 0⎤ ⎡ −4 6 ⎢ P ( A) = A − 5 A − 8 A + 6 I = ⎢ − 15 37 114⎥⎥ − 5⎢⎢ − 3 7 18 ⎥⎥ − 8⎢⎢ 0 1 3 ⎥⎥ + ⎢⎢0 6 0⎥⎥ ⎢⎣− 28 66 194⎥⎦ ⎢⎣− 4 10 32⎥⎦ ⎢⎣− 1 2 5 ⎥⎦ ⎢⎣0 0 6⎥⎦ 3

2

2 ⎤ ⎡− 10 10 8 ⎤ ⎡6 0 0 ⎤ − 10 ⎤ ⎡8 − 16 ⎡ −4 6 ⎢ ⎥ ⎥ ⎢ ⎢ = ⎢ − 15 37 114⎥ + ⎢ 15 − 35 − 90 ⎥ + ⎢0 − 8 − 24⎥⎥ + ⎢⎢0 6 0⎥⎥ ⎢⎣− 28 66 194⎥⎦ ⎢⎣ 20 − 50 − 160⎥⎦ ⎢⎣8 − 16 − 40⎥⎦ ⎢⎣0 0 6⎥⎦ 6 + 10 − 16 2 − 10 + 8 ⎡− 4 − 10 + 8 + 6 ⎤ ⎡0 0 0 ⎤ ⎢ 37 − 35 − 8 + 6 114 − 90 − 24 ⎥⎥ = ⎢⎢0 0 0⎥⎥ = ⎢ − 15 + 15 ⎢⎣ − 28 + 20 + 8 66 − 50 − 16 194 − 160 − 40 + 6⎥⎦ ⎢⎣0 0 0⎥⎦

Problem # 3 ⎡1 1 0⎤ Consider the following square matrix A = ⎢⎢0 0 1⎥⎥ . ⎣⎢0 0 1⎥⎦

a) Compute A10 and A103 using the Cayley-Hamilton Theorem. b) Compute e At using the Cayley-Hamilton Theorem. Solution:

Eigenvalues:

145

0 ⎤ ⎡λ − 1 − 1 ⎢ λI − A = ⎢ 0 λ − 1 ⎥⎥ = (λ − 1)(λ − 1)λ = 0 ⇒ λ1 = 0, λ 2 = 1, ⎢⎣ 0 0 λ − 1⎥⎦ f (λi ) = α 0 + α 1 λi + α 2 λi2

λ3 = 1

i = 1, 2, 3

Therefore: f ( A) = A k

a)

λ 1k = α 0 + α 1 λ 1 + α 2 λ 12

λ1k = α 0 + α 1λ1 + α 2 λ12

λ k2 = α 0 + α 1 λ 2 + α 2 λ 22

→

dλ d (α 0 + α 1 λ 23 + α 2 λ ) = dλ 3 dλ3 2 3

k 3

λ = α 0 + α 1λ 2 + α 2 λ k 2

kλk2 −1 = α 1 + 2α 2 λ 2

2 2

α 0 = δ (k ) ⎧ ⎪ → ⎨α 0 + α 1 + α 2 = 1k = 1 ⎪ α + 2α = k 1 2 ⎩

⎧α 0 = δ (k ) ⎪ → ⎨α1 = 2 − k ⎪α = k − 1 ⎩ 2

⎡1 0 0⎤ ⎡1 1 0 ⎤ ⎡ 1 1 0 ⎤ ⎡1 1 0 ⎤ ⎢ ⎥ ⎢ ⎥ A = α 0 I + α 1 A + α 2 A = α 0 ⎢0 1 0⎥ + α 1 ⎢0 0 1⎥ + α 2 ⎢⎢0 0 1⎥⎥ ⎢⎢0 0 1⎥⎥ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ k

2

α2 ⎡1 0 0⎤ ⎡1 1 0⎤ ⎡1 1 1⎤ ⎡α 0 + α 1 + α 2 α 1 + α 2 ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 α0 α1 + α 2 ⎥⎥ A = α 0 ⎢0 1 0⎥ + α 1 ⎢0 0 1⎥ + α 2 ⎢0 0 1⎥ = ⎢ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣ 0 0 α 0 + α 1 + α 2 ⎥⎦ k

1 k −1 ⎤ ⎡1 1 102⎤ ⎡1 + δ (k ) ⎡1 1 9 ⎤ ⎢ ⎢ ⎥ ⎥ 10 103 δ (k ) A =⎢ 0 1 ⎥ , Hence A = ⎢0 0 1⎥ and A = ⎢⎢0 0 1 ⎥⎥ ⎢⎣0 0 1 ⎥⎦ ⎢⎣ 0 ⎢⎣0 0 1⎥⎦ 0 1 + δ (k )⎥⎦ k

f ( A) = e At b)

146

e λ1t = α 0 + α1λ1 + α 2 λ12

e λ1t = α 0 + α 1λ1 + α 2 λ12 e

λ2 t

= α 0 + α 1λ 2 + α 2 λ

→

2 2

d (α 0 + α 1λ 2 + α 2 λ ) de = dλ 2 dλ 2 λ2 t

2 2

e

λ2 t

te

λ2t

= α 0 + α1λ2 + α λ

2 2 2

= α1 + 2α 2 λ2

α0 = 1 ⎧ ⎪ → ⎨α 0 + α1 + α 2 = et ⎪ α + 2α = te t 2 ⎩ 1

α0 = 1 ⎧ ⎪ → ⎨α1 = −tet + 2et − 2 ⎪ α = tet − e t + 1 ⎩ 2

e

At

e

At

⎡0 0 0 ⎤ ⎡1 1 0 ⎤ ⎡1 1 0⎤ ⎡1 1 0⎤ ⎢ ⎥ ⎢ ⎥ = α 0 I + α 1 A + α 2 A = α 0 ⎢0 1 0⎥ + α 1 ⎢0 0 1⎥ + α 2 ⎢⎢0 0 1⎥⎥ ⎢⎢0 0 1⎥⎥ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ 2

⎡1 0 0⎤ ⎡1 1 0⎤ ⎡1 1 1⎤ ⎡α 0 + α 1 + α 2 ⎢ ⎥ ⎢ ⎥ = α 0 ⎢0 1 0⎥ + α 1 ⎢0 0 1⎥ + α 2 ⎢⎢0 0 1⎥⎥ = ⎢⎢ 0 ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣0 0 1⎥⎦ ⎢⎣ 0

⎡e t ⎢ e At = ⎢ 0 ⎢0 ⎣

α1 + α 2 α0 0

α2 ⎤ α 1 + α 2 ⎥⎥ α 0 + α 1 + α 2 ⎥⎦

et − 1 tet − et + 1⎤ ⎥ 1 et − 1 ⎥ ⎥ 0 et ⎦

Problem # 4

Assume that the following matrix is nonsingular: ⎡ a 1 0⎤ C = ⎢⎢ 0 a 0⎥⎥ ⎢⎣ 0 0 b ⎥⎦

Compute the natural logarithm of the matrix C; that is, identify the matrix B = ln(C ) that satisfies the equation C = e B . Is the assumption of non-singularity of C really needed for the solution of this problem? Solution:

Matrix C is nonsingular → | C | ≠ 0 → | C | = a 2 b ≠ 0 → a ≠ 0, and b ≠ 0 147

λ−a

−1

0 0

λ−a

| λI − C | =

0

0 0 = (λ − a )(λ − a )(λ − b) = 0 λ −b

λ1 = λ2 = a, λ3 = b

⎡1 0 0 ⎤ ⎡ a 1 0⎤ ⎡ a 1 0⎤ ⎡ a 1 0⎤ ⎢ ⎥ ⎢ ⎥ ln C = ξ 0 I + ξ1C + ξ 2 C = ξ 0 ⎢0 1 0⎥ + ξ1 ⎢ 0 a 0⎥ + ξ 2 ⎢⎢ 0 a 0⎥⎥ ⎢⎢ 0 a 0⎥⎥ ⎣⎢0 0 1⎥⎦ ⎣⎢ 0 0 b⎦⎥ ⎣⎢ 0 0 b⎦⎥ ⎢⎣ 0 0 b⎦⎥ ⎡ a 2 2a 0 ⎤ ⎡1 0 0 ⎤ ⎡ a 1 0⎤ ⎢ ⎥ ln C = ξ 0 ⎢⎢0 1 0⎥⎥ + ξ1 ⎢⎢ 0 a 0⎥⎥ + ξ 2 ⎢ 0 a 2 0 ⎥ ⎢ 0 0 b2 ⎥ ⎢⎣0 0 1⎥⎦ ⎢⎣ 0 0 b⎥⎦ ⎣ ⎦ 2

⎡ξ 0 + aξ1 + a 2ξ 2 ⎢ ln C = ⎢ 0 ⎢ 0 ⎣

ln(λ1 ) = ξ 0 + ξλ1 + ξ 2 λ12 ln(λ1 ) = 0 + ξ1 + 2ξ 2 λ1 dλ1

ln(λ3 ) = ξ 0 + ξλ3 + ξ 2 λ

0

→ →

2 3

⎧ξ 0 + ξ1 a + ξ 2 a 2 = ln(a ) ⎪⎪ 1 ξ1 + 2ξ 2 a = ⎨ a ⎪ 2 ⎪⎩ξ 0 + ξ1b + ξ 2 b = ln(b) 1 ⎡ ⎤ 0 ⎥ ⎢ln a a ln C = ⎢ 0 ln a 0 ⎥ ⎢ ⎥ 0 ln b ⎥ ⎢ 0 ⎢⎣ ⎥⎦

ξ 0 + 2aξ1 ξ 0 + aξ1 + a 2ξ 2

⎤ ⎥ ⎥ ξ 0 + bξ1 + b 2ξ 2 ⎥⎦ 0 0

ln (a) = ξ 0 + ξ1 a + ξ 2 a 2 1

λ1

= 0 + ξ1 + 2ξ 2 λ1

→

→

ln (b) = ξ 0 + ξ1b + ξ 2 b 2

→

⎡1 a a 2 ⎤ ⎡ξ 0 ⎤ ⎡ln a ⎤ ⎢ ⎥⎢ ⎥ ⎢ 1 ⎥ ⎢0 1 2 a ⎥ ⎢ ξ 1 ⎥ = ⎢ a ⎥ ⎢1 b b 2 ⎥ ⎢⎣ξ 2 ⎥⎦ ⎢ ln b ⎥ ⎣ ⎦ ⎣ ⎦

1 = ξ 1 + 2ξ 2 a a

Clearly a , and b must be nonzero

Problem # 5

Given the 3× 3 matrix:

148

⎡0 1 0 ⎤ A = ⎢⎢0 0 1 ⎥⎥ ⎢⎣2 − 1 4⎥⎦

Express the matrix polynomial f ( A) = A 4 + 2 A 3 + A 2 − A + 3I as a linear combination of the matrices A 2 , A and I only. Solution: 0 ⎤ ⎡ λ −1 ⎢ λI − A = ⎢ 0 λ − 1 ⎥⎥ = λ (λ 2 − 4λ + 1) + 1(−2) = λ3 − 4λ 2 + λ − 2 ⎢⎣− 2 1 λ − 4⎥⎦

P( A) = A 3 − 4 A 2 + A − 2 I = 0 → A 3 = 4 A 2 − A + 2 I f ( A) = A 4 + 2 A 3 + A 2 − A + 3I = AA 3 + 2 A 3 + A 2 − A + 3I = A(4 A 2 − A + 2 I ) + 2(4 A 2 − A + 2 I ) + A 2 − A + 3I f ( A) = 4 A 3 − A 2 + 2 A + 8 A 2 − 2 A + 4 I + A 2 − A + 3I = 4 A 3 + 8 A 2 − A + 7 I = 4(4 A 2 − A + 2 I ) + 8 A 2 − A + 7 I f ( A) = 24 A 2 − 5 A + 15 I

Problem # 6

Let the 2 × 2 matrix A be defined as:

⎡0 1 ⎤ A=⎢ ⎥ ⎣1 0⎦

Compute A −20 and e At . Solution: ⎡ λ − 1⎤ 2 λI − A = ⎢ ⎥ = λ − 1 = 0 ⇒ λ1 = 1, λ 2 = −1 1 λ − ⎣ ⎦ f (λ i ) = α 0 + α 1 λ i

i = 1, 2

Therefore:

149

a) f ( A) = A k

λ1k = α 0 + α1λ1 λk2 = α 0 + α1λ2

→

λ1k = α 0 + α1λ1 λk2 = α 0 + α1λ2

→

⎡1 0⎤ ⎡0 1⎤ ⎡α 0 + α1 ⎢ A = α 0 I + α1 A = α 0 ⎢ ⎥ ⎥=⎢ ⎣0 1 ⎦ ⎣1 0⎦ ⎣α 1 k

⎡0.5 + 0.5(−1) −20 A − 20 = ⎢ − 20 ⎣0.5 − 0.5(−1)

⎧α 0 + α 1 = 1k = 1 ⎨ k ⎩α 0 − α 1 = (−1)

⎧α = 0.5 + 0.5(−1) k → ⎨ 0 k ⎩α 1 = 0.5 − 0.5(−1)

α 1 ⎤ ⎡0.5 + 0.5(−1) k = α 0 ⎥⎦ ⎢⎣0.5 − 0.5(−1) k

0.5 − 0.5(−1) k ⎤ ⎥ 0.5 + 0.5(−1) k ⎦

0.5 − 0.5(−1) −20 ⎤ ⎡1 0⎤ ⎥=⎢ ⎥ 0.5 + 0.5(−1) − 20 ⎦ ⎣0 1⎦

b) f ( A) = e At e λ1t = α 0 + α 1λ1 e λ2 t = α 0 + α 1 λ 2

→

⎧ α 0 + α1 = et ⎨ −t ⎩α 0 − α 1 = e

⎡1 0⎤ ⎡0 1⎤ ⎡α 0 + α1 ⎢ e At = α 0 I + α 1 A = α 0 ⎢ ⎥ ⎥=⎢ ⎣0 1 ⎦ ⎣1 0⎦ ⎣α 1

⎧α = 0.5e t + 0.5e − t → ⎨ 0 t −t ⎩α 1 = 0.5e − 0.5e

α 1 ⎤ ⎡0.5e t + 0.5e −t = α 0 ⎥⎦ ⎢⎣0.5e t − 0.5e −t

Problem # 7

Let the 3× 3 matrix A be defined as: ⎡0 1 0 ⎤ A = ⎢⎢1 0 1⎥⎥ ⎢⎣0 1 0⎥⎦

Compute A 2004 and e At . Solution:

150

0.5e t − 0.5e − t ⎤ ⎥ 0.5e t + 0.5e −t ⎦

⎡ λ −1 0 ⎤ λI − A = ⎢⎢− 1 λ − 1⎥⎥ = λ (λ2 − 1) + (−λ − 1) + 1(−λ ) = λ3 − 2λ = λ (λ2 − 2) = 0 ⎢⎣ 0 − 1 λ ⎥⎦

λ1 = 2, λ 2 = 0, λ3 = − 2

⎡0 ⎡1 0 0⎤ ⎥ ⎢ A = α 0 I + α 1 A + α 2 A = α 0 0 1 0 + α 1 ⎢1 ⎢ ⎥ ⎢ ⎢⎣0 ⎢⎣0 0 1⎥⎦ ⎡1 0 ⎡0 1 0 ⎤ ⎡1 0 0⎤ ⎥ ⎢ ⎥ ⎢ k A = α 0 ⎢0 1 0⎥ + α 1 ⎢1 0 1⎥ + α 2 ⎢⎢0 2 ⎢⎣1 0 ⎢⎣0 1 0⎥⎦ ⎢⎣0 0 1⎥⎦ k

2

1 0⎤ ⎡0 ⎥ 0 1 + α 2 ⎢1 ⎢ ⎥ ⎢⎣0 1 0⎥⎦ 1⎤ ⎡α 0 + α 2 0⎥⎥ = ⎢⎢ α 1 1⎥⎦ ⎢⎣ α 2

1 0 ⎤ ⎡0 1 0 ⎤ 0 1⎥ ⎢⎢1 0 1⎥⎥ ⎥ 1 0⎥⎦ ⎢⎣0 1 0⎥⎦

α1 α 0 + 2α 2 α1

α2 ⎤ α 1 ⎥⎥ α 0 + α 2 ⎥⎦

where,

λ = α 0 + α 1λ1 + α λ k 1

2 2 1

λk2 = α 0 + α 1λ2 + α 2 λ22

→

λ3k = α 0 + α 1λ3 + α 2 λ32

α1 ⎡α 0 + α 2 ⎢ A = ⎢ α1 α 0 + 2α 2 ⎢⎣ α 2 α1 k

A 2004

e

At

⎡21001 ⎢ =⎢ 0 ⎢21001 ⎣

0 21002 0

k ⎧ 2 ⎪ 2 = α 0 + 2α 1 + 2α 2 ⎪ ⎨ δ (k ) = α 0 + α1 0 + α 2 0 → k ⎪ k ⎪(−1) 2 2 = α 0 − 2α 1 + 2α 2 ⎩

k k k ⎡ δ (k ) ⎤ 1 + (−1) k 1 − (−1) k 1 + (−1) k 2 2 2 + 2 [ ] 2 [ ] 2 [ ] ⎥ ⎢ 4 4 2 2 ⎥ α2 ⎤ ⎢ 2 k k k k k 1 − (−1) 1 + (−1) 1 − (−1) k ⎢ ⎥ ⎥ 2 2 2 ] 2 [ ] 2 [ ] ⎥ α1 ⎥ = ⎢ 2 [ 2 2 2 2 2 ⎥ α 0 + α 2 ⎥⎦ ⎢ k k k k k 1 + ( − 1 ) 1 − ( − 1 ) ( ) 1 + (−1) k ⎥ δ k ⎢ 22[ 2 2 ] 2 [ ] +2 [ ] ⎢ ⎥ 4 2 4 2 2 ⎣ ⎦

21001 ⎤ ⎥ 0 ⎥ 21001 ⎥⎦

⎡α 0 + α 2 = α 0 I + α 1 A + α 2 A = ⎢⎢ α 1 ⎢⎣ α 2 2

⎧ ⎪ α 0 = δ (k ) ⎪ k 1 − (−1) k ⎪ 2 α 2 [ ] = ⎨ 1 2 2 ⎪ k 1 + (−1) k δ (k ) ⎪ 2 α = 2 [ ]− ⎪⎩ 2 4 2

α1 α 0 + 2α 2 α1

α2 ⎤ α 1 ⎥⎥ α 0 + α 2 ⎥⎦

where,

151

e

λ1t

= α 0 + α 1λ1 + α λ

2 2 1

e λ2t = α 0 + α 1λ 2 + α 2 λ22 e λ3t = α 0 + α 1λ3 + α 2 λ32

⎧ e = α 0 + 2α 1 + 2α 2 ⎪ ⎨ 1 = α 0 + α1 0 + α 2 0 → ⎪e − 2t = α − 2α + 2α 0 1 2 ⎩ 2t

→

⎧ ⎪ α0 = 1 ⎪ e 2t − e − 2t ⎪ α = ⎨ 1 2 2 ⎪ 2t e e − 2t − 2 + ⎪ α = ⎪⎩ 2 4

Therefore:

α1 ⎡α 0 + α 2 ⎢ e = ⎢ α1 α 0 + 2α 2 ⎢⎣ α 2 α1 At

⎡ e 2t + e − 2t + 2 ⎢ 4 α2 ⎤ ⎢ 2t ⎢ − e e − 2t α1 ⎥⎥ = ⎢ 2 2 α 0 + α 2 ⎥⎦ ⎢ 2t − 2t −2 ⎢e + e ⎢ 4 ⎣

⎡ e 2t + e − 2t + 2 ⎢ 4 ⎢ 2t ⎢ e − e − 2t e At = ⎢ ⎢ 2t 2 −2 2t −2 ⎢e + e ⎢ 4 ⎣

− e− 2 2 e 2t + e − 2 2t e − e− 2 2

e

⎡ ⎢0.5 cosh( 2t ) + 0.5 ⎢ 2 =⎢ sinh( 2t ) ⎢ 2 ⎢ ⎢0.5 cosh( 2t ) − 0.5 ⎢⎣

2t

2t

2t

2t

− e− 2 2 2t e + e− 2 2t e − e− 2 2

e

2t

2t

2t

2t

+ e − 2t − 2 ⎤ ⎥ 4 ⎥ e 2t − e − 2t ⎥ ⎥ 2 2 ⎥ e 2t + e − 2t + 2 ⎥ ⎥ 4 ⎦ e

2t

+ e − 2t − 2 ⎤ ⎥ 4 ⎥ e 2t − e − 2t ⎥ ⎥ 2 2 ⎥ e 2t + e − 2t + 2 ⎥ ⎥ 4 ⎦

e

2t

⎤ 2 sinh( 2t ) 0.5 cosh( 2t ) − 0.5⎥ 2 ⎥ 2 cosh( 2t ) sinh( 2t ) ⎥ ⎥ 2 ⎥ 2 sinh( 2t ) 0.5 cosh( 2t ) + 0.5⎥ 2 ⎥⎦

Problem # 8

Show that functions of matrix A commute. That is: f ( A) g ( A) = g ( A) f ( A) for any two functions f (.) and g (.) and any square matrix A .

152

Solution:

Let A be an n × n matrix, then by Cayley-Hamilton theorem we have: n −1

n −1

i =0

i =0

f ( A) = ∑α i Ai and g ( A) = ∑ β i Ai Therefore: n −1 n −1

n −1 n −1

i = 0 k =0

i =0 k =0

n −1 n −1

n −1 n −1

i =0 k =0

i =0 k =0

f ( A) g ( A) = ∑∑α i Ai β k A k = ∑∑ α i β k Ai + k and g ( A) f ( A) = ∑∑ β k A k α i Ai = ∑∑ α i β k Ai + k Comparing the above two equation, we have: f ( A) g ( A) = g ( A) f ( A) Problem # 9

⎡3 1⎤ ⎡2 1 ⎤ Let B = ⎢ , and C = ⎢ ⎥ ⎥ , and A = BC ⎣1 3⎦ ⎣1 2 ⎦ a) Show that BC = CB b) Find ln(B) , ln(C ) ,and ln(A) c) Is ln( A) = ln( B) + ln(C ) ? Can this be generalized? Solution:

⎡3 1⎤ ⎡2 1⎤ ⎡7 5⎤ ⎡2 1⎤ ⎡3 1⎤ ⎡7 5⎤ and CB = ⎢ a) BC = ⎢ =⎢ ⎥ ⎥ ⎥ ⎢ ⎥ ⎥⎢ ⎥=⎢ ⎣1 3⎦ ⎣1 2⎦ ⎣5 7 ⎦ ⎣1 2⎦ ⎣1 3⎦ ⎣5 7 ⎦ b)

⎡λ − 3 − 1 ⎤ = (λ − 3) 2 − 1 = 0 ⇒ λ1 = 2, λ 2 = 4 ⎥ ⎣ − 1 λ − 3⎦

λI − B = ⎢

153

f (λ i ) = α 0 + α 1 λ i

i = 1, 2

Therefore: ln(λ1 ) = α 0 + α 1λ1

→

ln(λ 2 ) = α 0 + α 1λ 2

ln(2) = α 0 + 2α 1 ln(4) = α 0 + 4α 1

⎧α + 2α 1 = ln(2) → ⎨ 0 ⎩α 0 + 4α 1 = ln(4)

⎧ α0 = 0 → ⎨ ⎩α 1 = 0.5 ln(2)

α 1 ⎤ ⎡1.5 ln(2) 0.5 ln(2)⎤ ⎡1 0⎤ ⎡3 1⎤ ⎡α 0 + 3α 1 ln( B ) = α 0 I + α 1 B = α 0 ⎢ + α1 ⎢ =⎢ = ⎥ ⎥ α 0 + 3α 1 ⎥⎦ ⎢⎣0.5 ln(2) 1.5 ln(2) ⎥⎦ ⎣0 1 ⎦ ⎣1 3⎦ ⎣ α 1

⎡λ − 2 − 1 ⎤ λI − C = ⎢ = (λ − 2) 2 − 1 = 0 ⇒ λ1 = 1, λ 2 = 3 ⎥ ⎣ − 1 λ − 2⎦

f (λ i ) = α 0 + α 1 λ i

i = 1, 2

Therefore: ln(λ1 ) = α 0 + α 1λ1

→

ln(λ 2 ) = α 0 + α 1λ 2

ln(1) = α 0 + α 1 ln(3) = α 0 + 3α 1

⎧ α 0 + α1 = 0 → ⎨ ⎩α 0 + 3α 1 = ln(3)

⎧α = −0.5 ln(3) → ⎨ 0 ⎩ α 1 = 0.5 ln(3)

α 1 ⎤ ⎡0.5 ln(3) 0.5 ln(3)⎤ ⎡1 0 ⎤ ⎡2 1 ⎤ ⎡α 0 + 2α 1 ln(C ) = α 0 I + α 1C = α 0 ⎢ + α1 ⎢ =⎢ = ⎥ ⎥ α 0 + 2α 1 ⎥⎦ ⎢⎣0.5 ln(3) 0.5 ln(3)⎥⎦ ⎣0 1 ⎦ ⎣1 2 ⎦ ⎣ α 1

⎡λ − 7 − 5 ⎤ = (λ − 7) 2 − 25 = 0 ⇒ λ1 = 2, λ 2 = 12 ⎥ ⎣ − 5 λ − 7⎦

λI − A = ⎢

f (λ i ) = α 0 + α 1 λ i

i = 1, 2

Therefore: ln(λ1 ) = α 0 + α 1λ1 ln(λ 2 ) = α 0 + α 1λ 2

→

ln(2) = α 0 + 2α 1

⎧ α + 2α 1 = ln(2) ⎧ α = 1.2 ln(2) − 0.2 ln(12) →⎨ 0 →⎨ 0 ln(12) = α 0 + 12α 1 ⎩α 1 = −0.1 ln(2) + 0.1 ln(12) ⎩α 0 + 12α 1 = ln(12)

5α 1 ⎤ ⎡1 0⎤ ⎡7 5⎤ ⎡α 0 + 7α 1 ln( A) = α 0 I + α 1 A = α 0 ⎢ + α1 ⎢ =⎢ ⎥ ⎥ α 0 + 7α 1 ⎥⎦ ⎣0 1 ⎦ ⎣5 7 ⎦ ⎣ 5α 1

154

⎡ 0.5 ln(2) + 0.5 ln(12) − 0.5 ln(2) + 0.5 ln(12)⎤ ln(A) = ⎢ ⎥ ⎣− 0.5 ln(2) + 0.5 ln(12) 0.5 ln(2) + 0.5 ln(12) ⎦ ⎡1.5 ln(2) + 0.5 ln(3) 0.5 ln(2) + 0.5 ln(3)⎤ =⎢ ⎥ ⎣0.5 ln(2) + 0.5 ln(3) 1.5 ln(2) + 0.5 ln(3) ⎦ c) In this case ln( A) = ln( B) + ln(C ) . This cannot be generalized, that is in general if A = BC ln( A) ≠ ln( B ) + ln(C ) . Under certain conditions if A = BC = CB , then ln( A) = ln( B) + ln(C ) .

Problem # 10

Find the continuous state transition matrix corresponding to: ⎡σ A=⎢ 0 ⎣− ω 0

ω0 ⎤ σ 0 ⎥⎦

Solution:

⎡λ − σ 0

λI − A = ⎢

⎣ ω0

− ω0 ⎤ = (λ − σ 0 ) 2 + ω 02 = 0 ⇒ λ1 = σ 0 + jω 0 , λ 2 = σ 0 − jω 0 ⎥ λ −σ0 ⎦

e At = α 0 I + α 1 A

e

λ1t

e λ2 t

= α 0 + α 1λ1

(σ 0 + jω0 ) t

⎧α + (σ 0 + jω 0 )α 1 = e → ⎨ 0 (σ 0 − j ω 0 ) t = α 0 + α 1λ 2 ⎩α 0 + (σ 0 − jω 0 )α 1 = e

σ 0 sin ω 0 t ⎧ σ 0t ) ⎪⎪α 0 = e (cos ω 0 t − ω0 → ⎨ e σ 0t sin ω 0 t ⎪ α1 = ⎪⎩ ω0

Therefore:

ω 0 ⎤ ⎡α 0 + α1σ 0 α1ω 0 ⎤ ⎡σ ⎡1 0⎤ e At = α 0 I + α 1 A = α 0 ⎢ + α1 ⎢ 0 =⎢ ⎥ ⎥ α 0 + α 1σ 0 ⎥⎦ ⎣0 1 ⎦ ⎣− ω 0 σ 0 ⎦ ⎣ − α 1ω 0 ⎡ eσ 0t cos ω 0 t eσ 0t sin ω 0 t ⎤ =⎢ σt ⎥ σ 0t 0 ⎣− e sin ω 0 t e cos ω 0 t ⎦

155

Problem # 11

Show that the discrete state transition matrix φ (k ) = A k satisfies the following properties: a) φ (0) = I b) φ (− k ) = A − k = ( A k ) −1 = φ −1 (k ) c) φ (k1 + k 2 ) = A k1 + k 2 = A k1 A k 2 = φ (k1 )φ (k 2 ) (d) φ (k 3 − k 2 )φ (k 2 − k1 ) = φ (k 3 − k1 ) Solution:

a) φ (0) = A0 = I b) φ (−k )φ (k ) = A − k A k = A0 + I

→ φ (− k ) = A − k = ( A k ) −1 = φ −1 (k )

c) φ (k1 + k 2 ) = A k1 + k2 = Ak1 Ak2 = φ (k1 )φ (k 2 ) d) φ (k 3 − k 2 )φ (k 2 − k1 ) = A k3 −k2 A k2 −k1 = A k3 −k2 + k2 −k1 = A k3 −k1 = φ (k 3 − k1 ) Problem # 12

Show that the continuous state transition matrix φ (t ) = e At satisfies the following properties: a) φ (0) = I b) φ (−t ) = e − At = (e At ) −1 = φ −1 (t ) c)

φ (t1 + t 2 ) = φ (t1 )φ (t 2 )

d) φ (t 3 − t 2 )φ (t 2 − t1 ) = φ (t 3 − t1 ) Solution:

a) φ (0) = e A0 = I b) φ (−t )φ (t ) = e − At e At = e − At + At = e A0 = φ (0) = I

→ φ (−t ) = e − At = (e At ) −1 = φ −1 (t )

c) φ (t1 + t 2 ) = e ( t1 +t2 ) A = e At1 e At2 = φ (t1 )φ (t 2 )

156

d) φ (t 3 − t 2 )φ (t 2 − t1 ) = e A(t3 −t2 ) e A( t2 −t1 ) = e A(t3 −t2 +t2 −t1 ) = e A( t3 −t1 ) = φ (t 3 − t1 ) Problem # 13

Find the state space representation of the following dynamical systems: a)

dy 3 (t ) dy 2 (t ) dy (t ) + 4 +4 + 3 y (t ) = u (t ) 3 2 dt dt dt

b) y (k + 2) + 0.98 y (k + 1) + 0.8 y (k ) = 4u (k ) Solution:

a) x1 (t ) = y (t ),

x2 (t ) =

dy (t ) , and dt

x3 (t ) =

d 2 y (t ) dt 2

Then we have: dy (t ) = x2 (t ) dt d 2 y (t ) x& 2 (t ) = = x3 (t ) dt 2 d 3 y (t ) dy (t ) dy 2 (t ) x&3 (t ) = = − 3 y ( t ) − 4 − 4 + u (t ) = −3 x1 (t ) − 4 x2 (t ) − 4 x3 (t ) + u (t ) dt dt 3 dt 2 x&1 (t ) =

Therefore: 1 0 ⎤ ⎡ x1 (t ) ⎤ ⎡0⎤ ⎡ x&1 (t ) ⎤ ⎡ 0 ⎢ x& (t )⎥ = ⎢ 0 0 1 ⎥⎥ ⎢⎢ x2 (t )⎥⎥ + ⎢⎢0⎥⎥u (t ) ⎢ 2 ⎥ ⎢ ⎢⎣ x&3 (t ) ⎥⎦ ⎢⎣− 3 − 4 − 4⎥⎦ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣1⎥⎦ ⎡ x1 (t ) ⎤ y (t ) = [1 0 0] ⎢⎢ x2 (t )⎥⎥ ⎢⎣ x3 (t ) ⎥⎦ b) Define the state variables to be: x1 (k ) = y (k ),

and

x2 (k ) = y (k + 1)

Then,

157

x1 (k + 1) = y (k + 1) = x2 (k ) x2 ( k ) = y (k + 2) = −0.8 y ( k ) − 0.98 y ( k + 1) + 4u (k ) = −0.8 x1 (k ) − 0.98 x2 (k ) + 4u (k ) Therefore: 1 ⎤ ⎡ x1 (k ) ⎤ ⎡0⎤ ⎡ x1 (k + 1) ⎤ ⎡ 0 ⎢ x (k + 1)⎥ = ⎢− 0.8 − 0.98⎥ ⎢ x (k )⎥ + ⎢4⎥u (k ) ⎦⎣ 2 ⎦ ⎣ ⎦ ⎣ 2 ⎦ ⎣ ⎡ x (k ) ⎤ y (k ) = [1 0] ⎢ 1 ⎥ ⎣ x2 (k )⎦ Problem # 14

Find the state space representation of the following dynamical systems in controllable canonical form: a)

dy 3 (t ) dy 2 (t ) dy (t ) + 3 +4 + 2 y (t ) = 7u (t ) 3 2 dt dt dt

b) 3

dy 3 (t ) dy 2 (t ) dy (t ) du (t ) + + + 2 y (t ) = 6 + 9u (t ) 3 2 dt dt dt dt

Solution:

a)

a1 = 3 , a 2 = 4 , a3 = 2 , b0 = 0 , b1 = 0 , b2 = 0 ,and b3 = 7

⎡ x&1 ⎤ ⎡ 0 ⎢ x& ⎥ ⎢ 0 ⎢ 2 ⎥ ⎢ ⎢ M ⎥=⎢ M ⎥ ⎢ ⎢ ⎢ x& n−1 ⎥ ⎢ 0 ⎢⎣ x& n ⎥⎦ ⎢⎣− a n

y = [bn − an b0

1 0

0 1

M 0

M 0

− a n−1

− an−2

bn−1 − an−1b0

0 ⎤ ⎡ x1 ⎤ ⎡0⎤ L 0 ⎥⎥ ⎢⎢ x2 ⎥⎥ ⎢0⎥⎥ ⎢ M M ⎥⎢ M ⎥ + ⎢M ⎥ u ⎥ ⎢ ⎥ ⎥⎢ L 0 ⎥ ⎢ xn−1 ⎥ ⎢0⎥ L − a1 ⎥⎦ ⎢⎣ xn ⎥⎦ ⎢⎣1⎥⎦ L

⎡ x1 ⎤ ⎢x ⎥ L b1 − a1b0 ] ⎢ 2 ⎥ + b0 u ⎢M⎥ ⎢ ⎥ ⎣ xn ⎦

Therefore:

158

1 0 ⎤ ⎡ x1 (t ) ⎤ ⎡0⎤ ⎡ x&1 (t ) ⎤ ⎡ 0 ⎢ x& (t )⎥ = ⎢ 0 0 1 ⎥⎥ ⎢⎢ x2 (t )⎥⎥ + ⎢⎢0⎥⎥u (t ) ⎢ 2 ⎥ ⎢ ⎣⎢ x&3 (t ) ⎦⎥ ⎣⎢− 2 − 4 − 3⎦⎥ ⎣⎢ x3 (t ) ⎦⎥ ⎣⎢1⎦⎥ ⎡ x1 (t ) ⎤ y (t ) = [7 0 0] ⎢⎢ x2 (t )⎥⎥ ⎢⎣ x3 (t ) ⎥⎦ b) 3

dy 3 (t ) dy 2 (t ) dy (t ) du (t ) + + + 2 y (t ) = 6 + 9u (t ) 3 2 dt dt dt dt

Divide both sides of the above differential equation by three, then we have: du (t ) dy 3 (t ) 1 dy 2 (t ) 1 dy (t ) 2 + + + y (t ) = 2 + 3u (t ) and 3 2 dt 3 dt 3 dt 3 dt

1 1 2 a1 = , a2 = , a3 = , b0 = 0 , b1 = 0 , b2 = 2 ,and b3 = 3 3 3 3 ⎡ ⎡ x&1 (t ) ⎤ ⎢ 0 ⎢ x& (t )⎥ = ⎢ 0 ⎢ 2 ⎥ ⎢ 2 ⎢⎣ x&3 (t ) ⎥⎦ ⎢− ⎣ 3

⎤ 1 0 ⎥ ⎡ x1 (t ) ⎤ ⎡0⎤ 0 1 ⎥ ⎢⎢ x2 (t )⎥⎥ + ⎢⎢0⎥⎥u (t ) 1 1⎥ − − ⎥ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣1⎥⎦ 3 3⎦ ⎡ x1 (t ) ⎤ y (t ) = [3 2 0] ⎢⎢ x2 (t )⎥⎥ ⎢⎣ x3 (t ) ⎥⎦

Problem # 15

Find the state space representation of the following dynamical systems in observable canonical form: a)

dy 3 (t ) dy 2 (t ) dy (t ) du 3 (t ) du 2 (t ) du (t ) + 3 + 4 + 2 y ( t ) = 3 + 8 + + 4u (t ) dt dt dt 3 dt 2 dt 3 dt 2

b)

dy 3 (t ) dy 2 (t ) dy (t ) du (t ) + + + 2 y (t ) = 6 + 27u (t ) 3 2 dt dt dt dt

Solution:

159

a) a1 = 3 , a2 = 4 , a3 = 2 , b0 = 3 , b1 = 8 , b2 = 1 ,and b3 = 4 The general form of canonical observable realization is: ⎡ x&1 ⎤ ⎡0 ⎢ x& ⎥ ⎢1 ⎢ 2 ⎥ ⎢ ⎢ M ⎥ = ⎢M ⎥ ⎢ ⎢ ⎢ x&n −1 ⎥ ⎢0 ⎢⎣ x&n ⎥⎦ ⎢⎣0

0 L 0 − an ⎤ ⎡ x1 ⎤ ⎡ bn − anb0 ⎤ ⎡ x1 ⎤ ⎥ ⎢ ⎢x ⎥ ⎥ ⎢ ⎥ 0 L 0 − an −1 ⎥ ⎢ x2 ⎥ ⎢bn −1 − an −1b0 ⎥ ⎢ 2 ⎥ ⎥ ⎢ ⎥ ⎢ ⎥ y = [0 0 L 0 1] ⎢ M ⎥ + b0 u M M M M M + M ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ 0 1 0 − a2 ⎥ ⎢ xn −1 ⎥ ⎢ b2 − a2b0 ⎥ ⎢ xn−1 ⎥ ⎢⎣ xn ⎥⎦ 0 0 1 − a1 ⎥⎦ ⎢⎣ xn ⎥⎦ ⎢⎣ b1 − a1b0 ⎥⎦

Therefore: ⎡ x&1 (t ) ⎤ ⎡0 0 − a3 ⎤ ⎡ x1 (t ) ⎤ ⎡b3 − a3b0 ⎤ ⎢ x& (t )⎥ = ⎢1 0 − a ⎥ ⎢ x (t )⎥ + ⎢b − a b ⎥ u (t ) 2⎥ ⎢ 2 2 0⎥ ⎢ 2 ⎥ ⎢ ⎥ ⎢ 2 ⎢⎣ x&3 (t ) ⎥⎦ ⎢⎣0 1 − a1 ⎥⎦ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣ b1 − a1b0 ⎥⎦ ⎡ x1 (t ) ⎤ y (t ) = [0 0 1] ⎢⎢ x2 (t )⎥⎥ + b0 u (t ) ⎢⎣ x3 (t ) ⎥⎦ Substituting for a' s and b' s , we have: ⎡ x&1 (t ) ⎤ ⎡0 0 − 2⎤ ⎡ x1 (t ) ⎤ ⎡ − 2 ⎤ ⎢ x& (t )⎥ = ⎢1 0 − 4⎥ ⎢ x (t )⎥ + ⎢− 11⎥u (t ) ⎢ 2 ⎥ ⎢ ⎥⎢ 2 ⎥ ⎢ ⎥ ⎢⎣ x& 3 (t ) ⎥⎦ ⎢⎣0 1 − 3⎥⎦ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣ − 1 ⎥⎦ ⎡ x1 (t ) ⎤ y (t ) = [0 0 1] ⎢⎢ x 2 (t )⎥⎥ + 3u (t ) ⎢⎣ x3 (t ) ⎥⎦ b)

dy 3 (t ) dy 2 (t ) dy (t ) du (t ) + + + 2 y (t ) = 6 + 27u (t ) 3 2 dt dt dt dt

a1 = 1 , a2 = 1 , a3 = 2 , b0 = 0 , b1 = 0 , b2 = 6 ,and b3 = 27

160

⎡ x&1 (t ) ⎤ ⎡0 0 − a3 ⎤ ⎡ x1 (t ) ⎤ ⎡b3 − a3b0 ⎤ ⎢ x& (t )⎥ = ⎢1 0 − a ⎥ ⎢ x (t )⎥ + ⎢b − a b ⎥ u (t ) 2⎥ ⎢ 2 2 0⎥ ⎢ 2 ⎥ ⎢ ⎥ ⎢ 2 ⎢⎣ x&3 (t ) ⎥⎦ ⎢⎣0 1 − a1 ⎥⎦ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣ b1 − a1b0 ⎥⎦ ⎡ x1 (t ) ⎤ y (t ) = [0 0 1] ⎢⎢ x2 (t )⎥⎥ + b0 u (t ) ⎢⎣ x3 (t ) ⎥⎦ Substituting for a' s and b' s , we have: ⎡ x&1 (t ) ⎤ ⎡0 0 − 2⎤ ⎡ x1 (t ) ⎤ ⎡27⎤ ⎢ x& (t )⎥ = ⎢1 0 − 1 ⎥ ⎢ x (t )⎥ + ⎢ 6 ⎥ u (t ) ⎢ 2 ⎥ ⎢ ⎥⎢ 2 ⎥ ⎢ ⎥ ⎢⎣ x& 3 (t ) ⎥⎦ ⎢⎣0 1 − 1 ⎥⎦ ⎢⎣ x3 (t ) ⎥⎦ ⎢⎣ 0 ⎥⎦ ⎡ x1 (t ) ⎤ y (t ) = [0 0 1] ⎢⎢ x2 (t )⎥⎥ ⎢⎣ x3 (t ) ⎥⎦ Problem # 16

Consider the discrete-time system: 1 x1 (k + 1) = x1 (k ) − x 2 (k ) 3 x 2 (k + 1) = 3 x1 (k ) − x 2 (k ) x3 (k + 1) = 4 x1 (k ) −

4 1 x 2 (k ) − x3 (k ) 3 3

y (k ) = x3 (k ) a)

Find the state transition matrix A k .

b)

Find y (k ) if x(0) = [1 1 1] .

c)

1 Find the initial condition x(0) if y (k ) = 4(− ) k for k ≥ 0 . 3

T

Solution:

a) State transition matrix A k

161

1 ⎡ ⎢1 − 3 A = ⎢3 − 1 ⎢ 4 ⎢4 − 3 ⎢⎣

⎤ 0 ⎥ 1 0 ⎥ . The eigenvalues of matrix A are: λ1 = − , and λ2 = λ3 = 0 . Then, 3 1⎥ − ⎥ 3 ⎥⎦

1 ⎤ ⎡ ⎡1 0 0⎤ ⎢1 − 3 0 ⎥ A k = α 0 I + α 1 A + α 2 A 2 = α 0 ⎢⎢0 1 0⎥⎥ + α 1 ⎢3 − 1 0 ⎥ ⎢ 4 1⎥ ⎢4 − − ⎥ ⎣⎢0 0 1⎥⎦ 3 3 ⎥⎦ ⎢⎣ 1 1 ⎡ ⎤⎡ ⎤ ⎢1 − 3 0 ⎥ ⎢1 − 3 0 ⎥ + α 2 ⎢3 − 1 0 ⎥ ⎢3 − 1 0 ⎥ ⎢ 4 1⎥⎢ 4 1⎥ ⎢4 − − ⎥ ⎢4 − − ⎥ 3 3 ⎦⎥ ⎣⎢ 3 3 ⎦⎥ ⎣⎢ 1 ⎡ − 1 ⎡1 0 0⎤ ⎢ 3 A k = α 0 ⎢⎢0 1 0⎥⎥ + α 1 ⎢3 − 1 ⎢ 4 ⎢⎣0 0 1⎥⎦ ⎢4 − 3 ⎣⎢ ⎡ ⎢ α 0 + α1 A k = ⎢ 3α 1 ⎢ ⎢4α 1 − 4 α 2 ⎢⎣ 3

⎤ ⎡ 0 ⎥ ⎢ 0 ⎥ 0 +α2 ⎢ 0 ⎢ 4 1⎥ − ⎥ ⎢− 3 ⎦⎥ ⎣ 3

0 0 4 9

⎤ 0⎥ 0⎥ 1⎥ ⎥ 9⎦

1 ⎤ − α1 0 ⎥ 3 ⎥ α 0 − α1 0 ⎥ 4 4 1 1 ⎥ − α1 + α 2 α 0 − α1 + α 2 3 9 3 9 ⎥⎦

where,

1 1 ⎧ 1 k ⎪(− 3 ) = α 0 − 3 α1 + 9 α 2 ⎧λ1k = α 0 + α1λ1 + α 2 λ12 ⎪⎪ ⎪ λ k2 = α 0 + α 1 λ 2 + α 2 λ 22 → ⎨λk2 = α 0 + α1λ2 + α 2 λ22 → ⎨(0) k = α 0 + 0 + 0 ⎪ ⎪ k −1 k −1 d λ 3k d (α 0 + α 1 λ 23 + α 2 λ 32 ) ⎩kλ2 = α1 + 2α 2 λ2 ⎪ k ( 0) = α 1 + 0 = ⎪⎩ dλ 3 dλ3

λ 1k = α 0 + α 1 λ 1 + α 2 λ 12

162

→

1 1 1 k ⎧ ⎪α 0 − 3 α1 + 9 α 2 = (− 3 ) ⎪ ⎨α 0 = δ (k ) ⎪α = kδ (k − 1) ⎪ 1 ⎩

→

⎧ ⎪α 0 = δ (k ) ⎪ ⎨α1 = kδ (k − 1) ⎪ 1 ⎪α 2 = 9(− ) k + 3kδ (k − 1) − 9δ (k ) 3 ⎩

Therefore: ⎡ ⎢ α 0 + α1 A k = ⎢ 3α 1 ⎢ ⎢4α 1 − 4 α 2 3 ⎣⎢

1 ⎤ 0 − α1 ⎥ 3 ⎥ 0 α 0 − α1 ⎥ 4 4 1 1 − α1 + α 2 α 0 − α1 + α 2 ⎥ 3 9 3 9 ⎦⎥

1 ⎡ ⎤ 0 ⎥ − kδ (k − 1) ⎢ kδ (k − 1) + δ ( k ) 3 3kδ (k − 1) 0 ⎥ =⎢ δ (k ) − kδ (k − 1) ⎢ ⎥ ⎢− 12(− 1 ) k + 12δ ( k ) 4(− 1 ) k − 4δ (k ) ( − 1 ) k ⎥ 3 3 3 ⎦⎥ ⎣⎢

b) 1 ⎡ ⎤ 0 ⎥ ⎡1⎤ − kδ (k − 1) ⎢ kδ (k − 1) + δ (k ) 3 3kδ (k − 1) 0 ⎥ ⎢⎢1⎥⎥ x(k ) = A k x(0) = ⎢ δ (k ) − kδ (k − 1) ⎢ ⎥ ⎢− 12(− 1 ) k + 12δ (k ) 4(− 1 ) k − 4δ (k ) (− 1 ) k ⎥ ⎢⎣1⎥⎦ 3 3 3 ⎦⎥ ⎣⎢ 2 ⎡ ⎤ ⎢δ (k ) + 3 kδ (k − 1)⎥ x(k ) = ⎢ δ (k ) + 2kδ (k − 1) ⎥ ⎢ ⎥ ⎢ − 7(− 1 ) k + 8δ (k ) ⎥ 3 ⎣⎢ ⎦⎥

1 y (k ) = x3 (k ) = −7(− ) k u (k ) + 8δ (k ) 3 c)

1 1 1 1 y (k ) = x3 (k ) = (−12( ) k + 12δ (k )) x1 (0) + (4(− ) k − 4δ (k )) x2 (0) + (− ) k x3 (0) = 4(− ) k 3 3 3 3 Comparing the right hand side with the left hand side, we have:

163

12 x1 (0) − 4 x2 (0) = 0 ⎧ ⎨ ⎩− 12 x1 (0) + 4 x2 (0) + x3 (0) = 4

⎧ x2 (0) = 3x1 (0) ⎨ ⎩ x 3 ( 0) = 4

→

where α is an arbitrary number. Problem # 17

Consider the discrete-time system: ⎡0 ⎢0 x(k + 1) = ⎢ ⎢0 ⎢ ⎣0

1 0 0 0

0 1 0 0

0⎤ ⎡1⎤ ⎢0 ⎥ ⎥ 0⎥ x(k ) + ⎢ ⎥u (k ) ⎢0 ⎥ 1⎥ ⎢ ⎥ ⎥ 0⎦ ⎣0 ⎦

y (k ) = [1 0 0 0]x(k ) a) Find the state transition matrix A k . b) Find y (k ) if x(0) = [1 1 1 1] and u (k ) = 0 . T

c) Find y (k ) if x(0) = [1 1 1 1] and u (k ) = 1 for k ≥ 0 . T

Solution:

a) The eigenvalues of matrix A are λ1 = λ2 = λ3 = λ4 = 0 .

164

→

⎡α ⎤ x(0) = ⎢⎢3α ⎥⎥ ⎢⎣ 4 ⎥⎦

⎡1 ⎢0 A k = α 0 I + α 1 A + α 2 A 2 + α 3 A3 = α 0 ⎢ ⎢0 ⎢ ⎣0 ⎡0 ⎢0 + α3 ⎢ ⎢0 ⎢ ⎣0

0 1 0 0

0 0 1 0

0⎤ ⎡0 ⎥ ⎢0 0⎥ + α1 ⎢ ⎢0 0⎥ ⎥ ⎢ 1⎦ ⎣0

1 0 0⎤ ⎡0 ⎥ ⎢0 0 1 0⎥ +α2 ⎢ ⎢0 0 0 1⎥ ⎥ ⎢ 0 0 0⎦ ⎣0

0 1 0⎤ 0 0 1⎥⎥ 0 0 0⎥ ⎥ 0 0 0⎦

0 0 1⎤ 0 0 0⎥⎥ 0 0 0⎥ ⎥ 0 0 0⎦

⎡α 0 α 1 α 2 α 3 ⎤ ⎢0 α α α ⎥ 0 1 2⎥ =⎢ ⎢0 0 α 0 α1 ⎥ ⎢ ⎥ 0 0 α0 ⎦ ⎣0

where

λ1k = α 0 + α1λ1 + α 2 λ12 + α 3 λ13 dλk2 d (α 0 + α 1λ2 + α 2 λ22 + α 3λ32 ) = dλ2 dλ2 d 2 λ3k dλ32 d 3λk4 dλ33

⎧λ1k = α 0 + α1λ1 + α 2 λ12 + α 3λ13 ⎪ k −1 2 ⎪kλ1 = α 1 + 2α 2 λ1 + 3α 3λ1 2 2 3 → → ⎨ d (α 0 + α 1λ3 + α 2 λ3 + α 3λ3 ) k −2 = − = + k ( k 1 ) λ 2 α 6 α λ ⎪ 1 2 3 1 dλ32 ⎪k (k − 1)(k − 2)λk −3 = 6α 1 3 ⎩ d 2 (α 0 + α 1λ4 + α 2 λ24 + α 3λ34 ) = dλ34

⎧α 0 = δ (k ) ⎪α = kδ (k − 1) ⎪ 1 ⎪ 1 ⎨α 2 = k (k − 1)δ (k − 2) 2 ⎪ ⎪ 1 ⎪α 3 = k (k − 1)(k − 2)δ (k − 3) 6 ⎩ Therefore:

165

⎡ ⎡α 0 α 1 α 2 α 3 ⎤ ⎢δ (k ) kδ (k − 1) ⎢0 α α α ⎥ ⎢ 0 1 2⎥ δ (k ) =⎢ 0 Ak = ⎢ ⎢0 0 α 0 α1 ⎥ ⎢ 0 0 ⎥ ⎢ 0 0 α 0 ⎦ ⎢⎢ ⎣0 0 ⎣ 0

1 k (k − 1)δ (k − 2) 2 kδ (k − 1)

δ (k ) 0

1 ⎤ k (k − 1)(k − 2)δ (k − 3)⎥ 6 ⎥ 1 k (k − 1)δ (k − 2) ⎥ 2 ⎥ kδ (k − 1) ⎥ ⎥ δ (k ) ⎦

b) ⎡ ⎢δ (k ) kδ (k − 1) ⎢ δ (k ) x(k ) = A k x(0) = ⎢ 0 ⎢ 0 ⎢ 0 ⎢ 0 0 ⎣

1 k (k − 1)δ (k − 2) 2 kδ (k − 1)

δ (k ) 0

1 ⎤ k (k − 1)(k − 2)δ (k − 3)⎥ ⎡1⎤ 6 ⎥ ⎢1⎥ 1 k (k − 1)δ (k − 2) ⎥ ⎢ ⎥ 2 ⎥ ⎢1⎥ kδ (k − 1) ⎥ ⎢1⎥ ⎥⎣⎦ δ (k ) ⎦

1 1 ⎤ ⎡ ⎢δ (k ) + kδ (k − 1) + 2 k (k − 1)δ (k − 2) + 6 k (k − 1)(k − 2)δ ( k − 3)⎥ ⎥ ⎢ 1 ⎥ δ (k ) + kδ (k − 1) + k (k − 1)δ (k − 2) x(k ) = ⎢ 2 ⎥ ⎢ δ (k ) + kδ (k − 1) ⎥ ⎢ ⎥ ⎢ δ ( k ) ⎦ ⎣

1 1 y (k ) = [1 0 0 0]x(k ) = δ (k ) + kδ (k − 1) + k (k − 1)δ (k − 2) + k (k − 1)(k − 2)δ (k − 3) 2 6 c) ⎡ k −1 k −i −1 ⎤ )11 ⎥ ⎢∑ ( A i =0 ⎡ k −1 ⎤ ⎡1⎤ ⎢ k −1 k −i −1 ⎥ ⎢∑ δ ( k − i − 1)⎥ ⎡u (k − 1)⎤ ⎢ ⎥ ) 21 ⎢ ⎥ ∑(A k −1 k −1 ⎥ ⎢ 0 ⎥ ⎢ i =0 ⎥ ⎢ i =0 k −i −1 k −i −1 ⎢0⎥ ⎥ 0 A Bu (i ) = ∑ A = =⎢ ⎥=⎢ ∑ ⎢ ⎥ ⎢0⎥ ⎢ k −1 k −i −1 ⎥ ⎢ 0 i =0 i =0 ⎥ ) 31 ⎥ 0 ⎢ ⎢ ⎥ ⎢∑ ( A ⎥ ⎣ 0 ⎥⎦ ⎥ ⎢ ⎣0⎦ ⎢ ki =−01 0 ⎦ ⎢ ⎥ ⎣ k −i −1 ) 41 ⎥ ⎢∑ ( A ⎣ i =0 ⎦

⎧1 k ≥ 0 where u(k) = ⎨ is the discrete unit step function. 0 k < 0 ⎩

166

k −1

x(k ) = A k x(0) + ∑ A k −i −1 Bu (i) i =0

1 1 ⎤ ⎡ ⎢δ (k ) + kδ ( k − 1) + 2 k ( k − 1)δ (k − 2) + 6 k (k − 1)(k − 2)δ ( k − 3) ⎥ ⎡u (k − 1)⎤ ⎥ ⎢ 0 ⎥ ⎢ 1 ⎥ ⎥+⎢ δ (k ) + kδ (k − 1) + k (k − 1)δ (k − 2) =⎢ 2 ⎥ ⎢ 0 ⎥ ⎢ δ (k ) + kδ (k − 1) ⎥ ⎢ 0 ⎥ ⎢ ⎦ ⎥ ⎣ ⎢ δ (k ) ⎦ ⎣ 1 1 ⎤ ⎡ ⎢u (k − 1) + δ (k ) + kδ (k − 1) + 2 k (k − 1)δ (k − 2) + 6 k (k − 1)(k − 2)δ ( k − 3)⎥ ⎥ ⎢ 1 ⎥ δ (k ) + kδ (k − 1) + k (k − 1)δ (k − 2) =⎢ 2 ⎥ ⎢ δ (k ) + kδ (k − 1) ⎥ ⎢ ⎥ ⎢ δ (k ) ⎦ ⎣

y (k ) = [1 0 0 0]x(k ) = x1 (k )

1 1 y (k ) = u (k − 1) + δ (k ) + kδ (k − 1) + k (k − 1)δ (k − 2) + k (k − 1)(k − 2)δ (k − 3) 2 6 Problem # 18

Is the following system state controllable? Is it state observable? 0⎤ 1 0 ⎤ ⎡ x1 (k ) ⎤ ⎡ 1 ⎡ x1 (k + 1) ⎤ ⎡ 0 ⎥ ⎡ u1 (k ) ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ x (k + 1)⎥ = ⎢ 1 x k + 0 1 ( ) 0 1 2 2 ⎥ ⎢u (k )⎥ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎢ ⎣ 2 ⎦ ⎢ ⎥ ⎢ ⎥ ⎣⎢ x3 (k + 1) ⎦⎥ ⎣⎢− 1 − 2 − 3⎦ ⎣ x3 (k ) ⎦ ⎣− 1 − 2⎥⎦ ⎡ x1 (k ) ⎤ ⎡ y1 (k ) ⎤ ⎡1 − 1 2⎤ ⎢ ⎥ ⎢ y ( k ) ⎥ = ⎢0 1 1 ⎥ ⎢ x 2 ( k ) ⎥ ⎦ ⎢ x (k ) ⎥ ⎣ 2 ⎦ ⎣ ⎣ 3 ⎦

Solution:

[

P= B

AB

0 0 1 0 − 2⎤ ⎡1 A 2 B = ⎢⎢ 0 1 0 −2 2 5 ⎥⎥ ⎢⎣− 1 − 2 2 4 − 6 − 9 ⎥⎦

]

rank( P) = 3 = n → The system is completely state controllable.

167

⎡1 ⎢0 ⎡ C ⎤ ⎢ ⎢− 3 Q = ⎢⎢ CA ⎥⎥ = ⎢ 0 ⎢⎣CA 2 ⎥⎦ ⎢ ⎢4 ⎢ ⎢⎣ 0

−1 2 ⎤ 1 1 ⎥⎥ − 3 − 7⎥ ⎥ − 2 − 2⎥ 11 18 ⎥ ⎥ 4 4 ⎥⎦

rank (Q) = 3 = n → The system is completely state observable. Problem # 19 (MATLAB)

Consider the dynamic system x(k + 1) = Ax(k ) y (k ) = Cx(k ) where all the computations are performed modulo 2 arithmetic. Let ⎡1 ⎢1 A=⎢ ⎢0 ⎢ ⎣0

1 1 0 1

0 1 0 1

0⎤ 1⎥⎥ , and C = [1 0 0 0] 1⎥ ⎥ 1⎦

a)

Find y (k ) for 0 ≤ k ≤ 15 . Assume initial condition to be x(0) = [1 0 0 0] .

b)

Find A16 .

c)

Use the result of part (b) to show that y (k ) is periodic. What is the period of y (k ) ?

T

Solution:

a) MATLAB Code: A=[1 1 0 0; 1 1 1 1; 0 0 0 1;0 1 1 1]; C=[1 0 0 0]; x0=[1 0 0 0]'; x=x0; for k=1:15 x=mod(A^k,2)*x; y(k)=mod(C*x,2); end D=mod(A^16,2);

168

Answers:

{y (k )}15k =0 = [1

1 0 0 1 1 0 1 0 1 1 0 0 1 1 1]

b) ⎡1 ⎢1 D = A16 = ⎢ ⎢0 ⎢ ⎣0

1 1 0 1

0 1 0 1

0⎤ 1⎥⎥ =A 1⎥ ⎥ 1⎦

c) Since A16 = A , the sequence y (k ) is periodic. The period of y (k ) is P = 15 . Problem # 20 (MATLAB)

Consider the nonlinear system described by: x&1 = − x1 ( x12 + x 22 − 1) + x 2 x& 2 = − x1 − x 2 ( x12 + x 22 − 1)

a) Find the equilibrium points of the system. b) Using MATLAB, plot the trajectory of the motion of the system in the x1 − x2 plane for different initial conditions. Is this system stable? Solution:

a) Equilibrium points ⎧− x1 ( x12 + x22 − 1) + x2 = 0 ⎪ ⎨ ⎪− x − x ( x 2 + x 2 − 1) = 0 2 ⎩ 1 2 1 From first equation we have: x2 = x1 ( x12 + x22 − 1) . Substituting into the second equation yields: − x1 − x2 ( x12 + x22 − 1) = − x1 − x1 ( x12 + x22 − 1) 2 = − x1[1 + ( x12 + x22 − 1) 2 ] = 0 .

169

⎡0 ⎤ Therefore x1 = 0 and x2 = 0 . Hence there is one equilibrium point and that is xeq = ⎢ ⎥ . ⎣0 ⎦ ⎡0.5⎤ ⎡1.5⎤ b) We plot the trajectory for two initial conditions: x0 = ⎢ ⎥ , and x0 = ⎢ ⎥ ⎣0.5⎦ ⎣1.5⎦ MATLAB Codes: function xdot=myfunc1(t,x) xdot=[-x(1)*(x(1)^2+x(2)^2-1)+x(2);-x(1)-x(2)*(x(1)^2+x(2)^2-1)];

tspan=[0 20]; x0=[0.5;0.5]; [t,x]=ode45('myfunc1',tspan,x0); plot(x(:,1),x(:,2)) hold on x0=[1.5;1.5]; [t,x]=ode45('myfunc3',tspan,x0); plot(x(:,1),x(:,2),'r') hold off xlabel('x_1(t)') ylabel('x_2(t)') legend('a=b=0.5','a=b=1.5') 1.5 a=b=0.5 a=b=1.5 1

x2(t)

0.5

0

-0.5

-1

-1.5 -1.5

-1

-0.5

0 x1(t)

170

0.5

1

1.5

Figure 5.1 Trajectory of the motion of the system for two different initial conditions

As it is seen from the plot shown in Figure 5.1, the system has stable limit cycle (oscillation). Independent of the initial condition, the trajectory will approach the limit cycle oscillation. The ⎡0.5⎤ plots of the two states of the system for the x0 = ⎢ ⎥ initial conditions are shown in Figure 5.2. ⎣0.5⎦ It shows the oscillatory behavior of the system.

tspan=[0 20]; x0=[0.5;0.5]; [t,x]=ode45('myfunc3',tspan,x0); plot(t,x(:,1)) hold on plot(t,x(:,2),'r') hold off xlabel('t') ylabel('x(t)') legend('x1(t)','x2(t)')

1.5 x1(t) x2(t) 1

x(t)

0.5

0

-0.5

-1

-1.5

0

2

4

6

8

10 t

12

14

16

18

20

Figure 5.2 Plots of the states of the system as a function of time

171

Problem # 21 (MATLAB)

Consider the nonlinear system described by: x&1 = x1 ( x12 + x22 − 1) − x2 x& 2 = x1 + x2 ( x12 + x 22 − 1)

a) Find the equilibrium points of the system. b) Using MATLAB, plot the trajectory of the motion of the system in the x1 − x2 plane for different initial conditions. Is this system stable? Solution:

a) ⎧ x1 ( x12 + x22 − 1) − x2 = 0 ⎪ ⎨ ⎪ x + x ( x 2 + x 2 − 1) = 0 2 ⎩ 1 2 1 From first equation we have: x2 = x1 ( x12 + x22 − 1) . Substituting into the second equation yields: x1 + x2 ( x12 + x22 − 1) = x1 + x1 ( x12 + x22 − 1) 2 = x1[1 + ( x12 + x22 − 1) 2 ] = 0 . Therefore x1 = 0 and ⎡0 ⎤ x2 = 0 . Hence there is one equilibrium point and that is xeq = ⎢ ⎥ . ⎣0 ⎦ ⎡0.5⎤ ⎡0.3⎤ b) The plots of the trajectory for two different initial conditions x0 = ⎢ ⎥ and x0 = ⎢ ⎥ are ⎣0.5⎦ ⎣0.3⎦ shown in Figure 5.3. MATLAB Codes: function xdot=myfunc2(t,x) xdot=[x(1)*(x(1)^2+x(2)^2-1)-x(2);x(1)+x(2)*(x(1)^2+x(2)^2-1)];

172

tspan=[0 8]; x0=[0.5;0.5]; [t,x]=ode45('myfunc4',tspan,x0); plot(x(:,1),x(:,2)) hold on x0=[0.3 0.3]; [t,x]=ode45('myfunc4',tspan,x0); plot(x(:,1),x(:,2),'r') hold off xlabel('x_1(t)') ylabel('x_2(t)') legend('a=b=0.5','a=b=0.3')

0.6 a=b=0.5 a=b=0.3

0.5

0.4

x 2(t)

0.3

0.2

0.1

0

-0.1 -0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

x 1(t)

Figure 5.3 Trajectory of the motion of the system for two different initial conditions

173

⎡0 ⎤ The plot of the trajectories (Figure 5.3) shows that the equilibrium point xeq = ⎢ ⎥ is unstable. ⎣0 ⎦ ⎡0 ⎤ Any trajectory with initial condition different from xeq = ⎢ ⎥ will diverge toward infinity. ⎣0 ⎦ ⎡0 ⎤ Therefore, the equilibrium point xeq = ⎢ ⎥ is globally unstable. ⎣0 ⎦

Problem # 22 (MATLAB)

A continuous time system can be converted to a discrete time system using the following transformation. ~ ~ x(k + 1) = A x(k ) + B u (k )

x& (t ) = Ax(t ) + Bu (t ) ⇒

~ ~ y (k ) = Cx(k ) + Du (k )

y (t ) = Cx(t ) + Du (t ) T

~ ~ ~ ~ where A = e AT , B = ( ∫ e Aτ dτ ) B , C = C , D = D , and T = sampling period. 0

a) Write a general MATLAB function file that accepts A , B , C , D and T as inputs and ~ ~ ~ ~ produces A , B , C , and D as outputs. b) Use the code developed in part (a) to convert the following continuous time systems into discrete systems. Assume sampling period of T = 1 .

⎡0 1 ⎤ ⎡0 ⎤ x& (t ) = ⎢ x(t ) + ⎢ ⎥u (t ) ⎥ ⎣0 0 ⎦ ⎣1⎦ I)

y (t ) = [1 0]x(t )

174

1 0⎤ ⎡0 ⎡1⎤ ⎢ ⎥ x& (t ) = ⎢ 0 0 1 ⎥ x(t ) + ⎢⎢ 2 ⎥⎥u (t ) ⎢⎣− 1 − 2 − 3⎥⎦ ⎢⎣− 1⎥⎦

II)

y (t ) = [1 0 − 1]x(t ) + 2u (t )

Solution:

a) MATLAB Code

function [AT,BT,CT,DT]=con2dis(A,B,C,D,T); [m,m]=size(A); if det(A)==0 N=2000; t=linspace(0,T,N); f=zeros(m,m,N); for i=1:N f(:,:,i)=expm(A*t(i)); end for i=1:m for j=1:m G(i,j)=trapz(t,squeeze(f(i,j,:))); end end BT=G*B; else BT=inv(A)*(expm(A*T)-eye(size(A)))*B; end AT=expm(A*T); CT=C; DT=D; T

~ Note that there is closed form solution for B = ( ∫ e Aτ dτ ) B if A is a nonsingular matrix. The 0

T

~ closed form solution is: B = ( ∫ e Aτ dτ ) B = A −1 (e AT − I ) B . If matrix A is singular there is no 0

~ ~ closed form solution for B and numerical integration is used to compute B .

175

c) ⎡0 1 ⎤ ⎡0 ⎤ x& (t ) = ⎢ x(t ) + ⎢ ⎥u (t ) ⎥ ⎣0 0 ⎦ ⎣1⎦ y (t ) = [1 0]x(t ) A=[0 1; 0 0]; B=[0 1]'; C=[1 0]; D=0; T=1; [AT,BT,CT,DT]=con2dis(A,B,C,D,T);

~ ⎡1 1⎤ ~ ⎡0.5⎤ ~ ~ Results: A = ⎢ , B = ⎢ ⎥ , C = [1 0] , and D = 0 ⎥ ⎣0 1⎦ ⎣1⎦ 1 0⎤ ⎡0 ⎡1⎤ ⎢ ⎥ x& (t ) = ⎢ 0 0 1 ⎥ x(t ) + ⎢⎢ 2 ⎥⎥u (t ) ⎢⎣− 1 − 2 − 3⎥⎦ ⎢⎣− 1⎥⎦ y (t ) = [1 0 − 1]x(t ) + 2u (t ) A=[0 1 0; 0 0 1;-1 -2 -3]; B=[1 2 -1]'; C=[1 0 -1]; D=2; T=1; [AT,BT,CT,DT]=con2dis(A,B,C,D,T); ⎡ 0.9166 0.8092 ~ ⎢ Results: A = ⎢- 0.1966 0.5234 ⎢⎣- 0.2194 - 0.6355

0.1966 ⎤ ⎡ 1.7864 ⎤ ~ ~ ~ ⎢ ⎥ 0.2194 ⎥ , B = ⎢ 1.3384 ⎥⎥ , C = [1 0 − 1] , and D = 2 ⎢⎣- 1.3692⎥⎦ - 0.1349⎥⎦

176

Problem # 23 (MATLAB)

a) Find the transformation matrix P that transforms matrix A into a Jordan Canonical form. 0 3⎤ ⎡− 2 1 ⎢ 0 −3 1 1 ⎥⎥ ⎢ A= ⎢0 0 −2 1 ⎥ ⎢ ⎥ 0 − 1 − 4⎦ ⎣0 b) Repeat part (a) for the matrix B given by:

2 ⎡0 0 ⎢0 0 2 1 B= ⎢ 2 ⎢0 − 2 2 ⎢ ⎣2 2 − 4

0 ⎤ ⎡3 2⎥⎥ ⎢⎢0 0 ⎥ ⎢0 ⎥⎢ 2 ⎦ ⎣0

0 ⎤⎡ 2 −1 1 3 0 0 ⎥⎥ ⎢⎢ 1 0 −1 0 − 2 0 ⎥⎢ 1 0 0 ⎥⎢ 0 0 − 3⎦ ⎣− 1 1 0

1

0

1⎤ 0⎥⎥ 0⎥ ⎥ 0⎦

Solution:

a) The characteristic polynomial of matrix A is

λ+2 P (λ ) = λ I − A =

0 0 0

−1 0 −3 −1 λ + 3 −1 = (λ + 2)(λ + 3) 3 = 0 . Therefore, the eigenvalues of 0 λ + 2 −1 λ+4 0 1

matrix A are λ1 = −2 , and λ2 = λ3 = λ4 = −3

Ax1 = λ1 x1

0 3 ⎤ ⎡a ⎤ ⎡− 2 1 ⎡a ⎤ ⎧− 2a + b + 3d = −2a ⎢ 0 −3 1 ⎥ ⎢ ⎥ ⎢ ⎥ ⎪− 3b + c + d = −2b 1 ⎥ ⎢b ⎥ b⎥ ⎪ ⎢ ⎢ = −2 → → ⎨ → ⎢0 ⎢c ⎥ 0 − 2 1 ⎥⎢c ⎥ ⎪− 2c + d = −2c ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎪⎩− c − 4d = −2d 0 − 1 − 4⎦ ⎣d ⎦ ⎣0 ⎣d ⎦

⎡1 ⎤ ⎢0 ⎥ → x1 = ⎢ ⎥ ⎢0 ⎥ ⎢ ⎥ ⎣0 ⎦

177

⎡ a ⎤ ⎡1⎤ ⎢ b ⎥ ⎢0 ⎥ ⎢ ⎥=⎢ ⎥ ⎢ c ⎥ ⎢0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ d ⎦ ⎣0 ⎦

0 3 ⎤⎡a ⎤ ⎧− 2a + b + 3d = −3a ⎡− 2 1 ⎡a ⎤ ⎪− 3b + c + d = −3b ⎢ 0 −3 1 ⎢b ⎥ 1 ⎥⎥ ⎢⎢ b ⎥⎥ ⎪ Ax2 = λ2 x2 → ⎢ = −3 ⎢ ⎥ → ⎨ → ⎢0 ⎢c ⎥ 2 3 c d c − + = − 0 − 2 1 ⎥⎢c ⎥ ⎪ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎪⎩− c − 4d = −3d 0 0 1 4 d d − − ⎣ ⎦⎣ ⎦ ⎣ ⎦ ⎡− 1⎤ ⎡3⎤ ⎡− b + 3c ⎤ ⎢1⎥ ⎢0⎥ ⎢ b ⎥ ⎢ ⎥ ⎥ ⎢ =b + c ⎢ ⎥ . Therefore there are two independent eigenvetors x2 = ⎢0⎥ ⎢1⎥ ⎢ c ⎥ ⎢ ⎥ ⎢ ⎥ ⎥ ⎢ ⎣0⎦ ⎣− 1⎦ ⎣ −c ⎦ ⎡3⎤ ⎡− 1⎤ ⎢0⎥ ⎢1⎥ ⎢ ⎥ corresponding to the second eigenvalue, they are: x2 = and x3 = ⎢ ⎥ ⎢1⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎣− 1⎦ ⎣0⎦ Define Ax4 = λ2 x4 + x2 + x3 , then we have:

0 3 ⎤⎡a ⎤ ⎡− 2 1 ⎡a ⎤ ⎡ 2 ⎤ ⎧− 2a + b + 3d = −3a + 2 ⎢ 0 −3 1 ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎪− 3b + c + d = −3b + 1 b 1 ⎥ ⎢b ⎥ 1 ⎪ ⎢ = −3 ⎢ ⎥ + ⎢ ⎥ → ⎨ → ⎢0 ⎢c ⎥ ⎢ 1 ⎥ 0 − 2 1 ⎥⎢c ⎥ c d c 2 3 1 − + = − + ⎪ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎪⎩− c − 4d = −3d − 1 d d 0 0 1 4 1 − − − ⎣ ⎦⎣ ⎦ ⎣ ⎦ ⎣ ⎦

⎡ a ⎤ ⎡2⎤ ⎢ b ⎥ ⎢0 ⎥ ⎢ ⎥=⎢ ⎥ ⎢ c ⎥ ⎢1 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ d ⎦ ⎣0 ⎦

Therefore: ⎡ 2⎤ ⎢0 ⎥ x4 = ⎢ ⎥ and the transformation T is T = [x1 ⎢1 ⎥ ⎢ ⎥ ⎣0 ⎦

178

x2

x3

⎡1 − 1 3 ⎢0 1 0 x4 ] = ⎢ ⎢0 0 1 ⎢ ⎣0 0 − 1

2⎤ 0⎥⎥ and 1⎥ ⎥ 0⎦

⎡1 − 1 3 ⎢0 1 0 −1 T AT = ⎢ ⎢0 0 1 ⎢ ⎣0 0 − 1 ⎡1 ⎢0 =⎢ ⎢0 ⎢ ⎣0

2⎤ 0⎥⎥ 1⎥ ⎥ 0⎦

1 −2 1 ⎤ 1 0 0 ⎥⎥ 0 0 − 1⎥ ⎥ 0 1 1⎦

−1

−1

0 3 ⎤ ⎡1 − 1 3 ⎡− 2 1 ⎢ 0 −3 1 1 ⎥⎥ ⎢⎢0 1 0 ⎢ ⎢0 0 − 2 1 ⎥ ⎢0 0 1 ⎢ ⎥⎢ 0 − 1 − 4 ⎦ ⎣0 0 − 1 ⎣0

2⎤ 0⎥⎥ 1⎥ ⎥ 0⎦

0 3 ⎤ ⎡1 − 1 3 ⎡− 2 1 ⎢ 0 −3 1 1 ⎥⎥ ⎢⎢0 1 0 ⎢ ⎢0 0 − 2 1 ⎥ ⎢0 0 1 ⎢ ⎥⎢ 0 − 1 − 4 ⎦ ⎣0 0 − 1 ⎣0

0 0⎤ 2⎤ ⎡ − 2 0 ⎢ ⎥ 1 ⎥⎥ 0⎥ ⎢ 0 − 3 0 = 0 −3 1 ⎥ 1⎥ ⎢ 0 ⎥ ⎥ ⎢ 0 0 − 3⎦ 0⎦ ⎣ 0

b) 2 ⎡0 0 ⎢0 0 2 1 B= ⎢ 2 ⎢0 − 2 2 ⎢ ⎣2 2 − 4 ⎡3 ⎢0 Since matrix ⎢ ⎢0 ⎢ ⎣0

0 ⎤ ⎡3 2⎥⎥ ⎢⎢0 0 ⎥ ⎢0 ⎥⎢ 2 ⎦ ⎣0

1 0 3 0 0 −2 0

0

0 ⎤⎡ 2 −1 1 3 0 0 ⎥⎥ ⎢⎢ 1 0 − 1 0 − 2 0 ⎥⎢ 1 0 0 ⎥⎢ 0 0 − 3⎦ ⎣− 1 1 0

1

0

1⎤ 0⎥⎥ 0⎥ ⎥ 0⎦

⎤ ⎥ ⎥ is in the Jordan canonical form, then B = T −1 AT and matrix ⎥ ⎥ − 3⎦ 0 0 0

T is given by: ⎡ 2 −1 1 ⎢ 1 0 −1 T =⎢ ⎢1 0 0 ⎢ ⎣− 1 1 0

2 1⎤ ⎡0 0 ⎢ ⎥ 2 0⎥ 1 0 0 and its inverse is T −1 = ⎢ 0⎥ 2 ⎢0 − 2 2 ⎢ ⎥ 0⎦ ⎣2 2 − 4

179

0 ⎤ ⎡0 0 1 ⎥ ⎢ 2 ⎥ ⎢0 0 1 = 0 ⎥ ⎢0 − 1 1 ⎥ ⎢ 2⎦ ⎣1 1 − 2

0⎤ 1⎥⎥ 0⎥ ⎥ 1⎦

CHAPTER 6 Problem # 1 Obtain the expression for the gradient vector ∇f and Hessian matrix ∇ 2 f for the following functions of n variables:

1 T x Ax + b T x 2

a) f ( x) =

b) f ( x) =

where A is symmetric

1 T ( x Ax) 2 + (b T x) 2 where A is symmetric 2

Solution:

a) ∇f =

∂f ( x) ∂ 2 f ( x) = Ax + b and ∇ 2 f = =A ∂x ∂x 2

b) ∇f =

∂f ( x) = 2 Ax( x T Ax) + 2b(bT x) , and ∂x

∇2 f =

∂ 2 f ( x) = 2( x T Ax) A + 4( Ax) T ( Ax) + 2bb T ∂x 2

Problem # 2

Obtain an expression for the gradient vector ∇f and Hessian matrix ∇ 2 f for the function of two variables given below: f ( x) = 100( x 2 − x12 ) 2 + (1 − x1 ) 2

Verify that x ∗ = [1 1] satisfies ∇f = 0 and ∇ 2 f positive definite. Show that ∇ 2 f is singular T

if and only if x satisfies the condition: x 2 − x12 = 0.005

Hence, show that ∇ 2 f is positive definite for all x such that x2 − x12 < 0.005 . Solution:

180

⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎡− 400( x − x 2 ) x − 2(1 − x )⎤ ⎧ x1 = 1 2 1 1 1 → ∇f = ⎢ 1 ⎥ = ⎢ = 0 ⎨ ⎥ 200( x2 − x12 ) ⎢ ∂f ⎥ ⎣ ⎩ x2 = 1 ⎦ ⎢⎣ ∂x2 ⎥⎦ ⎡ ∂2 f ⎢ ∂x 2 ∇2 f = ⎢ 2 1 ⎢ ∂ f ⎢ ∂x ∂x ⎣ 2 1

∂2 f ⎤ 2 ∂x1∂x2 ⎥⎥ ⎡− 400 x2 + 1200 x1 + 2 − 400 x1 ⎤ = ⎥ ∂ 2 f ⎥ ⎢⎣ 200 ⎦ − 400 x1 ∂x12 ⎥⎦

⎡− 400 x2 + 1200 x12 + 2 − 400 x1 ⎤ ⎡ 802 − 400⎤ ∇2 f = ⎢ =⎢ ⎥ ⎥>0 − 400 x1 200 ⎦ x =x =1 ⎣− 400 200 ⎦ ⎣ 1 2

∇ 2 f is singular if and only if det(∇ 2 f ) = 0 , that is 200(−400 x2 + 1200 x12 + 2) − 160000 x12 = 0 ⇒ − 400 x2 + 1200 x12 + 2 − 800 x12 = 0 ⇒ − 200 x2 + 200 x12 + 1 = 0

⇒

x2 − x12 = 0.005

Matrix ∇ 2 f is positive definite if

⎧⎪− 400 x2 + 1200 x12 + 2 > 0 ⎧⎪ x2 − 3 x12 − 0.005 < 0 ⇒ ⎨ ⎨ ⎪⎩200(−400 x2 + 1200 x12 + 2) − 160000 x12 > 0 ⎪⎩ x2 − x12 − 0.005 < 0

⇒ x2 − x12 − 0.005 < 0

Therefore ∇ 2 f is positive definite if and only if x2 − x12 < 0.005 Problem # 3

Show that the function f ( x) = ( x 2 − x12 ) 2 + x15 has only one stationary point, which is neither a local maximum nor a local minimum. Solution:

⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎡− 4 x ( x − x 2 ) + 5 x 4 ⎤ ⎡0⎤ ⎡ x ⎤ ⎡0 ⎤ 1 2 1 1 = ⎢ ⎥ → ⎢ 1⎥ = ⎢ ⎥ ∇f = ⎢ 1 ⎥ = ⎢ ⎥ 2 2( x2 − x1 ) ⎢ ∂f ⎥ ⎣ ⎣ x 2 ⎦ ⎣0 ⎦ ⎦ ⎣0 ⎦ ⎢⎣ ∂x2 ⎥⎦

181

⎡ ∂2 f ⎢ ∂x 2 ∇2 f = ⎢ 2 1 ⎢ ∂ f ⎢ ∂x ∂x ⎣ 2 1

∂2 f ⎤ 2 3 ⎥ ∂x1∂x2 ⎥ ⎡− 4 x2 + 12 x1 + 20 x1 = ⎢ ∂2 f ⎥ ⎣ − 4 x1 2 ⎥ ∂x1 ⎦

− 4 x1 ⎤ ⎡0 0⎤ ⎥=⎢ ⎥ 2 ⎦ ⎣0 2 ⎦

⎡0 0 ⎤ Matrix ∇ 2 f = ⎢ ⎥ is positive semidefinte therefore the test is inconclusive. However since ⎣0 2 ⎦ the stationary point is on the curve x2 = x12 , the function f (x) is f ( x) = x15 . This function has a stationary point x1 = 0 which is neither a local maximum nor a local minimum. Therefore ⎡ x1 ⎤ ⎡0⎤ ⎢ x ⎥ = ⎢0⎥ is neither a local maximum nor a local minimum. ⎣ 2⎦ ⎣ ⎦

Problem # 4

Find the stationary points of the function: f ( x) = 2 x13 − 3 x12 − 6 x1 x 2 ( x1 − x 2 − 1) Which of these points are local minima, which are local maxima, and which are neither? Solution: ⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎡6 x 2 − 6 x − 12 x x + 6 x 2 + 6 x ⎤ ⎡0⎤ 1 1 2 2 2 ∇f = ⎢ 1 ⎥ = ⎢ 1 ⎥=⎢ ⎥ 2 f ∂ − 6 x1 + 12 x1 x 2 + 6 x1 ⎥ ⎣ ⎢ ⎦ ⎣0⎦ ⎢⎣ ∂x 2 ⎥⎦

There are four solutions: ⎡ x ⎤ ⎡0 ⎤ ⎡x ⎤ ⎡ 0 ⎤ a) ⎢ 1 ⎥ = ⎢ ⎥ , b) ⎢ 1 ⎥ = ⎢ ⎥ , c) ⎣ x 2 ⎦ ⎣0 ⎦ ⎣ x2 ⎦ ⎣− 1⎦

⎡ x1 ⎤ ⎡1⎤ ⎡ x1 ⎤ ⎡− 1⎤ ⎢ x ⎥ = ⎢0⎥ , and d) ⎢ x ⎥ = ⎢− 1⎥ ⎣ 2⎦ ⎣ ⎦ ⎣ 2⎦ ⎣ ⎦

The Hessian matrix is:

182

⎡ ∂2 f ⎢ ∂x 2 ∇2 f = ⎢ 2 1 ⎢ ∂ f ⎢ ∂x ∂x ⎣ 2 1

∂2 f ⎤ ⎥ ∂x1∂x2 ⎥ ⎡ 12 x1 − 12 x2 − 6 − 12 x1 + 12 x2 + 6⎤ = ⎥ ∂ 2 f ⎥ ⎢⎣− 12 x1 + 12 x2 + 6 12 x1 ⎦ ∂x12 ⎥⎦

⎡ x ⎤ ⎡0 ⎤ a) ⎢ 1 ⎥ = ⎢ ⎥ ⎣ x 2 ⎦ ⎣0 ⎦

⎡− 6 6⎤ → The Hessian matrix ∇ 2 f = ⎢ ⎥ ⎣ 6 0⎦

is an indefinite matrix therefore,

⎡ x1 ⎤ ⎡0⎤ ⎢ x ⎥ = ⎢0⎥ is neither local minimum nor local maximum. ⎣ 2⎦ ⎣ ⎦ ⎡x ⎤ ⎡ 0 ⎤ b) ⎢ 1 ⎥ = ⎢ ⎥ ⎣ x2 ⎦ ⎣− 1⎦

⎡ 6 − 6⎤ → The Hessian matrix ∇ 2 f = ⎢ ⎥ ⎣− 6 0 ⎦

is indefinite

therefore,

⎡ x1 ⎤ ⎡ 0 ⎤ ⎢ x ⎥ = ⎢− 1⎥ is neither local minimum nor local maximum. ⎣ 2⎦ ⎣ ⎦ ⎡ x ⎤ ⎡1⎤ ⎡ 6 − 6⎤ c) ⎢ 1 ⎥ = ⎢ ⎥ → The Hessian matrix ∇ 2 f = ⎢ ⎥ is a positive definite matrix therefore, ⎣− 6 12 ⎦ ⎣ x 2 ⎦ ⎣0 ⎦ ⎡ x1 ⎤ ⎡1⎤ ⎢ x ⎥ = ⎢0⎥ is a local minimum. ⎣ 2⎦ ⎣ ⎦ ⎡ x ⎤ ⎡− 1⎤ d) ⎢ 1 ⎥ = ⎢ ⎥ ⎣ x2 ⎦ ⎣− 1⎦

⎡− 6 6 ⎤ → The Hessian matrix ∇ 2 f = ⎢ ⎥ ⎣ 6 − 12⎦

is a negative definite matrix

⎡ x ⎤ ⎡− 1⎤ therefore, ⎢ 1 ⎥ = ⎢ ⎥ is a local maximum. ⎣ x2 ⎦ ⎣− 1⎦

Problem # 5

Investigate the stationary points of the function: f ( x) = x12 x 22 − 4 x12 x 2 + 4 x12 + 2 x1 x 22 + x 22 − 8 x1 x 2 + 8 x1 − 4 x 2 + 6 Solution:

183

⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎡ 2 x x 2 − 8 x x + 8 x + 2 x 2 − 8 x + 8 ⎤ ⎡0⎤ 1 2 1 2 2 f ∇ = ⎢ 1 ⎥ = ⎢ 21 2 ⎥=⎢ ⎥ → 2 f ∂ ⎥ ⎣2 x1 x2 − 4 x1 + 4 x1 x2 + 2 x2 − 8 x1 − 4⎦ ⎣0⎦ ⎢ ⎢⎣ ∂x2 ⎥⎦ ⎡ x1 x22 − 4 x1 x2 + 4 x1 + x22 − 4 x2 + 4 ⎤ ⎡0⎤ ⎡( x1 + 1)( x2 − 2) 2 ⎤ ⎡0⎤ → = ⎥ ⎢ ⎥ ⎥=⎢ ⎥ ⎢ 2 ⎢ 2 2 ⎣ x1 x2 − 2 x1 + 2 x1 x2 + x2 − 4 x1 − 4⎦ ⎣0⎦ ⎣( x1 + 1) ( x2 − 2)⎦ ⎣0⎦

There are infinite numbers of solutions: ⎡ x ⎤ ⎡α ⎤ ⎡ x ⎤ ⎡− 1⎤ a) ⎢ 1 ⎥ = ⎢ ⎥ , b) and ⎢ 1 ⎥ = ⎢ ⎥ where α and β are two arbitrary real numbers. ⎣ x2 ⎦ ⎣ 2 ⎦ ⎣ x2 ⎦ ⎣ β ⎦

The Hessian matrix is:

⎡ ∂2 f ⎢ ∂ 2 x 2 ∇ f = ⎢ 21 ⎢ ∂ f ⎢ ∂x ∂x ⎣ 2 1

∂2 f ⎤ 2 2( x1 + 1)( x2 − 2)⎤ ∂x1∂x2 ⎥⎥ ⎡ ( x2 − 2) =⎢ ⎥ 2 ∂ f ⎥ ⎣2( x1 + 1)( x2 − 2) ( x1 + 1) 2 ⎦ ∂x12 ⎥⎦

0 ⎤ ⎡ x ⎤ ⎡α ⎤ ⎡0 is a positive semidefinite matrix a) ⎢ 1 ⎥ = ⎢ ⎥ → The Hessian matrix ∇ 2 f = ⎢ 2⎥ ⎣0 (α + 1) ⎦ ⎣ x2 ⎦ ⎣ 2 ⎦ therefore the test is inconclusive.

However since the function f can be written

⎡ x ⎤ ⎡α ⎤ as f ( x) = ( x1 + 1) 2 ( x2 − 2) 2 + 2 . Then, the points ⎢ 1 ⎥ = ⎢ ⎥ are all local minima of the ⎣ x2 ⎦ ⎣ 2 ⎦

function. ⎡( β − 2) 2 ⎡ x ⎤ ⎡− 1⎤ b) ⎢ 1 ⎥ = ⎢ ⎥ → ∇ 2 f = ⎢ ⎣ x2 ⎦ ⎣ β ⎦ ⎣ 0

0⎤ ⎥ is a positive semidefinite matrix therefore the test is 0⎦

inconclusive. However since the function f can be written as f ( x) = ( x1 + 1) 2 ( x2 − 2) 2 + 2 . ⎡ x ⎤ ⎡− 1⎤ Then, the points ⎢ 1 ⎥ = ⎢ ⎥ are all local minima of the function. ⎣ x2 ⎦ ⎣ β ⎦

Problem # 6

184

List all the stationary points of the function: f ( x) = − x12 − 4 x22 − 16 x32 subject to the constraint c( x) = 0 , where c(x) is given by: (i) c( x) = x1 − 1 (ii) c( x) = x1 x 2 − 1 (iii) c( x) = x1 x2 x3 − 1 Solution:

(i) H ( x1 , x2 , x3 , λ ) = − x12 − 4 x22 − 16 x32 + λ ( x1 − 1) ∂H ( x1 , x2 , x3 , λ ) = −2 x1 + λ = 0 ∂x1 ∂H ( x1 , x2 , x3 , λ ) = −8 x2 = 0 ∂x2 ∂H ( x1 , x2 , x3 , λ ) = −32 x3 = 0 ∂x31

⇒

∂H ( x1 , x2 , x3 , λ ) = x1 − 1 ∂λ

⎡ x1∗ ⎤ ⎡1 ⎤ ⎢ ∗⎥ ⎢ ⎥ ⎢ x 2 ⎥ = ⎢0⎥ ⎢ x3∗ ⎥ ⎢0⎥ ⎢ ∗⎥ ⎢ ⎥ ⎢⎣λ ⎥⎦ ⎣2⎦

(ii) H ( x1 , x2 , x3 , λ ) = − x12 − 4 x22 − 16 x32 + λ ( x1 x2 − 1) ∂H ( x1 , x2 , x3 , λ ) = −2 x1 + λx2 = 0 ∂x1 ∂H ( x1 , x2 , x3 , λ ) = −8 x2 + λx1 = 0 ∂x2 ∂H ( x1 , x2 , x3 , λ ) = −32 x3 = 0 ∂x31 ∂H ( x1 , x2 , x3 , λ ) = x1 x2 − 1 = 0 ∂λ

⇒

⎡ x1∗ ⎤ ⎡ ⎢ ∗⎥ ⎢ ⎢ x2 ⎥ = ⎢ ⎢ x3∗ ⎥ ⎢ ⎢ ∗⎥ ⎢ ⎢⎣λ ⎥⎦ ⎢⎣

185

2⎤ ⎡ x1∗ ⎤ ⎡ − 2 ⎤ ⎥ 1 ⎢ ∗⎥ ⎢ 1 ⎥ x2 ⎢− ⎥ ⎥ 2 ⎥ and ⎢ ∗ ⎥ = ⎢ 2⎥ ⎢ x3 ⎥ 0 ⎥ ⎢ ∗⎥ ⎢ 0 ⎥ ⎢⎣λ ⎥⎦ ⎢⎣ 4 ⎥⎦ 4 ⎥⎦

(iii) H ( x1 , x2 , x3 , λ ) = − x12 − 4 x22 − 16 x32 + λ ( x1 x2 x3 − 1) ∂H ( x1 , x2 , x3 , λ ) = −2 x1 + λx2 x3 = 0 ∂x1

⎡ x1∗ ⎤ ⎡ 2 ⎤ ⎢ ∗⎥ ⎢ ⎥ 1 x ⇒ ⎢ 2∗ ⎥ = ⎢ ⎥ , ⎢ x3 ⎥ ⎢0.5⎥ ∂H ( x1 , x2 , x3 , λ ) = −32 x3 + λx1 x2 = 0 ⎢ ∗⎥ ⎢ ⎥ ∂x31 ⎣⎢λ ⎦⎥ ⎣ 8 ⎦ ∂H ( x1 , x2 , x3 , λ ) = x1 x2 x3 − 1 = 0 ∂λ

∂H ( x1 , x2 , x3 , λ ) = −8 x2 + λx1 x3 = 0 ∂x2

⎡ x1∗ ⎤ ⎡ 2 ⎤ ⎢ ∗⎥ ⎢ ⎥ ⎢ x2 ⎥ = ⎢ − 1 ⎥ , ⎢ x3∗ ⎥ ⎢− 0.5⎥ ⎢ ∗⎥ ⎢ ⎥ ⎣⎢λ ⎦⎥ ⎣ 8 ⎦

⎡ x1∗ ⎤ ⎡ − 2 ⎤ ⎢ ∗⎥ ⎢ ⎥ x2 ⎥ ⎢ 1 ⎥ ⎢ and ∗ = ⎢ x3 ⎥ ⎢− 0.5⎥ ⎢ ∗⎥ ⎢ ⎥ ⎣⎢λ ⎦⎥ ⎣ 8 ⎦

Problem # 7

Solve the optimization problem below: minimize f ( x, y ) = x 2 + y 2 + 3 xy + 6 x + 19 y subject to x + 3 y = 5 Solution:

H ( x, y, λ ) = x 2 + y 2 + 3 xy + 6 x + 19 y + λ ( x + 3 y − 5) ∂H ( x, y, λ ) = 2x + 3 y + 6 + λ = 0 ∂x ∂H ( x, y, λ ) = 2 y + 3 x + 19 + 3λ = 0 ∂y ∂H ( x, y, λ ) = x + 3y − 5 = 0 ∂λ

⇒

⎡ x ∗ ⎤ ⎡− 16⎤ ⎢ ∗⎥ ⎢ ⎥ ⎢y ⎥ = ⎢ 7 ⎥ ⎢ λ∗ ⎥ ⎢⎣ 5 ⎥⎦ ⎣ ⎦

Problem # 8

186

⎡ x1∗ ⎤ ⎡− 2⎤ ⎢ ∗⎥ ⎢ ⎥ ⎢ x2 ⎥ = ⎢ − 1 ⎥ , ⎢ x3∗ ⎥ ⎢0.5⎥ ⎢ ∗⎥ ⎢ ⎥ ⎣⎢λ ⎦⎥ ⎣ 8 ⎦

Let a, b, and c be positive constants. Find the least value of a sum of three positive numbers x, y, and z subject to the constraint: a b c + + =1 x y z by the method of Lagrange multipliers. Solution:

a b c H ( x, y, z , λ ) = x + y + z + λ ( + + − 1) x y z ∂H ( x, y, z , λ ) = 1− λ ∂x ∂H ( x, y, z , λ ) = 1− λ ∂y ∂H ( x, y, z , λ ) = 1− λ ∂z

a = 0 ⇒ x 2 = aλ ⇒ x = aλ 2 x b = 0 ⇒ y 2 = bλ ⇒ y = bλ 2 y c = 0 ⇒ z 2 = cλ ⇒ z = cλ z2

Substituting the above results into the constraint equation results in ∂H ( x, y, z , λ ) a b c = + + −1 = 0 ⇒ ∂λ x y z

a b c + + −1 = 0 ⇒ aλ bλ cλ

λ = ( a + b + c )2

Therefore: x ∗ = aλ = a ( a + b + c ) y ∗ = bλ = b ( a + b + c ) z ∗ = cλ = c ( a + b + c ) The minimum value of the sum is: x ∗ + y ∗ + z ∗ = ( a + b + c ) 2 Problem # 9

Find the point on the ellipse defined by the intersection of the surfaces x + y = 1 and x 2 + 2 y 2 + z 2 = 1 that is nearest to the origin. Solution:

187

H ( x, y, z , λ ) = x 2 + y 2 + z 2 + λ1 ( x + y − 1) + λ2 ( x 2 + 2 y 2 + z 2 − 1) ∂H ( x, y, z, λ1 , λ2 ) ∂x ∂H ( x, y, z, λ1 , λ2 ) ∂y ∂H ( x, y, z, λ1 , λ2 ) ∂z ∂H ( x, y, z, λ1 , λ2 ) ∂λ1

= 2 x + λ1 + 2 xλ2 = 0 = 2 y + λ1 + 4 yλ2 = 0 = 2 z + 2 zλ2 = 0 = x + y −1 = 0

∂H ( x, y, z, λ1 , λ2 ) = x 2 + 2 y 2 + z 2 −1 = 0 ∂λ2

There are two solutions are: ⇒

⎡x⎤ ⎡1⎤ ⎢y⎥ ⎢0⎥ ⎢ ⎥ ⎢ ⎥ ⎢ z ⎥ =⎢ 0 ⎥, ⎢ ⎥ ⎢ ⎥ ⎢ λ1 ⎥ ⎢ 0 ⎥ ⎢⎣λ2 ⎥⎦ ⎢⎣− 1⎥⎦

and

⎡ 1 ⎤ ⎢ ⎥ ⎡x⎤ ⎢ 3 ⎥ ⎢y⎥ ⎢ 2 ⎥ ⎢ ⎥ ⎢ 3 ⎥ ⎢ z⎥=⎢ 0 ⎥ ⎢ ⎥ ⎢ 4⎥ ⎢ λ1 ⎥ ⎢− ⎥ ⎢⎣λ2 ⎥⎦ ⎢ 9 ⎥ 1 ⎢− ⎥ ⎣ 3⎦

The second solution is closest to the origin. Hence: 1 2 x ∗ = , y ∗ = and z ∗ = 0 3 3 Problem # 10

The equation x1 + 4 x 2 = 5 has infinite number of solutions. Find the solution that will give the minimum norm. Solution:

The problem is to minimize x = x12 + x22 subject to the constraint x1 + 4 x 2 = 5 . 2

H ( x1 , x2 , λ ) = x12 + x22 + λ ( x1 + 4 x2 − 5)

188

∂H ( x1 , x2 , λ ) = 2 x1 + λ = 0 ∂x1 ∂H ( x1 , x2 , λ ) = 2 x2 + 4λ = 0 ∂x2

x1∗ =

5 17

x2∗ =

20 17

⇒

∂H ( x1 , x2 , λ ) = x1 + 4 x2 − 5 = 0 ∂λ

The minimum norm is x = x12 + x22 =

5 17 17

Problem # 11

Maximize

1 n 3 ∑ xi subject to: 3 i =1 n

∑ xi = 0 , and i =1

n

∑x i =1

2 i

=n

Solution: n n 1 n 3 2 x + λ x + λ ∑ i 1∑ i 2 ∑ xi 3 i =1 i =1 i =1

H=

∂H = xk2 + λ1 + 2λ2 xk = 0 ∂xk

→

xk2 + 2λ2 xk + λ1 = 0

→ xk = −λ2 ± λ22 − λ1

⎧− λ + λ2 − λ ⎧ a 2 1 ⎪⎪ 2 ⎪ = ⎨or . Assume that n1 variables are equal to a and n − n1 are or Therefore, xk = ⎨ ⎪ − λ − λ2 − λ ⎪ b 2 1 ⎩ ⎪⎩ 2 equal to b . Applying the constraints we have: n

∑x i =1

i

n

∑x i =1

2 i

= an1 + (n − n1 )b = 0

= n1a 2 + (n − n1 )b 2 = n

Solving the above equations for a and b results in

189

b=

n1 n − n1 , and a = − n − n1 n1

If a =

or b = −

n1 n − n1 , and a = . n − n1 n1

n − n1 n1 and b = − , then the maximum is: n1 n − n1

n 1 n 1 n − n1 2 1 J = max ∑ xi3 = n1 ( ) − (n − n1 )( 1 ) 2 n1 n − n1 3 i =1 3 3 3

3

3

3

n(n − 2n1 ) (n − n1 ) 2 − n12 1 (n − n1 ) 2 1 (n1 ) 2 = − = = 3 3 n − n1 3 n1 (n − n1 ) 3 n1 (n − n1 ) n1 Plotting J as a function of n1 for a fixed value of n shows that J has a maximum if n1 = 1 , therefore there are two solutions. The first solution is:

⎧ 1 ⎪− ⎪ n −1 xk = ⎨ ⎪ n −1 ⎪ ⎩ maximum value of

k =i where i is an arbitrary nonnegative integer less than n . The

k ≠i

n(n − 2) 1 n 3 xi is J = ∑ 3 i =1 3 n −1

And the second solution is:

⎧ ⎪ n −1 ⎪ xk = ⎨ ⎪ 1 ⎪− ⎩ n −1 maximum value of

k =i where i is an arbitrary nonnegative integer less than n . The

k ≠i 1 n 3 n(n − 2) xi is J = ∑ 3 i =1 3 n −1

Problem # 12 n

Maximize − ∑ xi ln( xi ) subject to: i =1

190

n

∑x i =1

i

= 1.

Solution: n

n

i =1

i =1

H = −∑ xi ln( xi ) + λ (∑ xi − 1) ∂H 1 = − ln( xk ) − xk + λ = − ln( xk ) − 1 + λ = 0 ∂xk xk n ∂H = ∑ xi − 1 = 0 → ∂λ i =1

Therefore, xk =

n

∑ xi = 1 → i =1

n

∑ eλ

−1

→

xk = e λ −1 for k = 1, 2, L, n

= 1 → ne λ −1 = 1 → e λ −1 =

i =1

1 n

n 1 for k = 1, 2, L, n and max(−∑ xi ln( xi )) = ln(n) n i =1

Problem # 13

Consider the optimization problem below: Maximize 2 x1 x 2 + x 2 x 3 + x1 + x 3 + 4 Subject to:

x1 + x2 + x3 = 3

Determine the solution ( x1 , x 2 , x3 , λ ) . Compute the Hessian matrix and determine if the solution is a minimum, maximum, or neither. Solution:

H = 2 x1 x 2 + x2 x3 + x1 + x3 + 4 + λ ( x1 + x2 + x3 − 3) ∂H ∂H ∂H = 2 x2 + 1 + λ = 0 , = 2 x1 + x3 + λ = 0 , = x2 + 1 + λ = 0 , and ∂x1 ∂x2 ∂x3 ∂H = x1 + x2 + x3 − 3 = 0 . In matrix form ∂λ

191

⎡0 ⎢2 ⎢ ⎢0 ⎢ ⎣1

2 0 1 1

0 1 0 1

1⎤ ⎡ x1 ⎤ ⎡− 1⎤ ⎡ x1 ⎤ ⎡0 ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ x ⎥ ⎢2 1⎥ ⎢ x2 ⎥ ⎢ 0 ⎥ → ⎢ 2⎥ = ⎢ = ⎢ x3 ⎥ ⎢0 1⎥ ⎢ x3 ⎥ ⎢− 1⎥ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0⎦ ⎣ λ ⎦ ⎣ 3 ⎦ ⎣ λ ⎦ ⎣1

2 0 1 1

0 1 0 1

1⎤ 1⎥⎥ 1⎥ ⎥ 0⎦

−1

⎡− 1⎤ ⎡− 2⎤ ⎢0⎥ ⎢ 0 ⎥ ⎢ ⎥=⎢ ⎥ ⎢− 1⎥ ⎢ 5 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ 3 ⎦ ⎣ − 1⎦

⎡0 2 0⎤ 2 ∂2H ⎢ ⎥ . The eigenvalues of the Hessian matrix ∂ H are : = 2 0 1 ⎥ ∂x 2 ∂x 2 ⎢ ⎢⎣0 1 0⎥⎦

λ1 = 0 , λ2 = 2.2361 , and λ3 = −2.2361 . Therefore the Hessian matrix is indefinite and the ⎡ − 2⎤ stationary point x ∗ = ⎢⎢ 0 ⎥⎥ is neither a maximum nor a minimum. ⎢⎣ 5 ⎥⎦

Problem # 14

Let: ⎡4 4 6 ⎤ R = ⎢⎢4 10 12 ⎥⎥ ⎢⎣6 12 20⎥⎦

a) Show that the eigenvalues of R are λ1 = λ 2 = 2, and λ3 = 30 . b) Maximize x T Rx , subject to: x12 + x22 + x32 = 1 . Solution:

a) The characteristic equation of matrix R is

λ −4 λI − R = − 4 −6

−4

λ − 10 − 12

−6 − 12 = λ3 − 34λ2 + 124λ − 120 = (λ − 2)(λ − 2)(λ − 30) λ − 20

Therefore the eigenvalues of R are:

λ1 = 2, λ2 = 2, and λ3 = 30

192

b)

H = x T Rx + λ ( x12 + x22 + x32 − 1) = x T Rx + λ ( x T x − 1) ∂H = 2 Rx + 2λx = 0 ∂x

→

Rx = −λx , and max( x T Rx) = x T (−λx) = −λx T x = −λ

Therefore, the optimal solution x is the normalized eigenvector corresponding to the largest eigenvalue of matrix R . The eigenvalues and eigenvectors of matrix R are: ⎡ 0.3586 ⎤ λ1 = 2, x1 = ⎢⎢ 0.7171 ⎥⎥ λ2 = 2, ⎢⎣ - 0.5976⎥⎦

⎡ 0.8944 ⎤ x2 = ⎢⎢- 0.4472⎥⎥ , and λ3 = 30, ⎢⎣ 0 ⎥⎦

⎡0.2673⎤ x3 = ⎢⎢0.5345⎥⎥ ⎢⎣0.8018⎥⎦

⎡0.2673⎤ Thus, x = ⎢⎢0.5345⎥⎥ and max( x T Rx) = λ3 = 30 . ⎢⎣0.8018⎥⎦

Problem # 15 (MATLAB)

Consider a linear time invariant system with input x(n) and output y (n) given by the second order Markov model z (n) = a1 z (n − 1) + a 2 z (n − 2) + bx(n) y ( n) = z ( n) + v ( n)

Where v(n) is zero mean white Gaussian noise with variance σ v2 . Let the system parameters be a1 = 0.7 , a 2 = −0.21 and b = 1 .

a) Use MATLAB to generate 20 samples of y (n) if x(n) = u (n) and SNR = ∞ dB(noise free) b) Use LS technique; estimate the parameters of the system. Compute MSE and Compare with the true values. c) Repeat parts (a) and (b) if SNR = 80 dB and SNR = 40 dB and Solution:

193

MATLAB Code: SNR = ∞

⎡ aˆ1 ⎤ ⎡ 0.7 ⎤ θˆ1 = ⎢⎢aˆ 2 ⎥⎥ = ⎢⎢− 0.21⎥⎥ ⎢⎣ bˆ ⎥⎦ ⎢⎣ 1 ⎥⎦ SNR = 80 dB

⎡ aˆ1 ⎤ ⎡ 0.7009 ⎤ θˆ1 = ⎢⎢aˆ 2 ⎥⎥ = ⎢⎢− 0.2104⎥⎥ ⎢⎣ bˆ ⎥⎦ ⎢⎣ 0.9990 ⎥⎦ SNR = 40 dB

⎡ aˆ1 ⎤ ⎡ 0.5997 ⎤ θˆ1 = ⎢⎢aˆ 2 ⎥⎥ = ⎢⎢− 0.1974⎥⎥ ⎢⎣ bˆ ⎥⎦ ⎢⎣ 1.1741 ⎥⎦

N=20; a1=0.7; a2=-0.21; b=1; a=[1 -a1 -a2]; x=ones(N,1); z=filter(b,a,x); E=mean(z.^2); SNR=500; s=sqrt(E*10^(-SNR/10)); y=z+s*randn(size(x)); p=3; A=[y(p-1:end-1) y(p-2:end-2) x(p:end)]; B=y(p:end); tet1=inv(A'*A)*A'*B; SNR=80; s=sqrt(E*10^(-SNR/10)); y=z+s*randn(size(x)); p=3; A=[y(p-1:end-1) y(p-2:end-2) x(p:end)]; B=y(p:end); tet2=inv(A'*A)*A'*B; SNR=40; s=sqrt(E*10^(-SNR/10)); y=z+s*randn(size(x)); p=3; A=[y(p-1:end-1) y(p-2:end-2) x(p:end)]; B=y(p:end); tet3=inv(A'*A)*A'*B;

194

Problem # 16 (MATLAB)

Design a linear phase low-pass FIR filter of size M = 31 to have a passband of ω p =

π

stopband of 0.67π . Solution: function [h]=lslpf(wp,ws,M,a) % % Design of a linear phase FIR filter using LS % wp=passband frequency % ws=stopband frequency % M= Number of filter taps (odd) % a=tradeoff parameter between passband and stopband % h=Desigend filter taps. % N=(M-1)/2; k=1:1:N+1; l=k; [y,x]=meshgrid(k,l); AP=0.5*(sin((x-y+eps)*wp))./((x-y+eps))+0.5*(sin((x+y-2+eps)*wp))./((x+y-2+eps)); AP=AP-((sin((x-1+eps)*wp))./((x-1+eps)))-((sin((y-1+eps)*wp))./((y-1+eps)))+wp; AP=AP/pi; AS=-0.5*(sin((x-y+eps)*ws))./((x-y+eps))-0.5*(sin((x+y-2+eps)*ws))./((x+y-2+eps)); AS=AS+0.5*(sin((x-y+eps)*pi))./((x-y+eps))+0.5*(sin((x+y-2+eps)*pi))./((x+y-2+eps)); AS=AS/pi; R=a*AP+(1-a)*AS; c=ones(N+1,1); z=inv(R)*c/(c'*inv(R)*c); h=[0.5*z(end:-1:2);z(1)]; h=[h;h(end-1:-1:1)]; W=linspace(0,pi,4000); [H]=freqz(h,1,W); plot(W,20*log10(abs(H))) grid on xlabel('\omega') ylabel('|H(j\omega)|') wp=0.5*pi; ws=0.67*pi; M=31; a=0.75; [h]=lslpf(wp,ws,M,a);

195

2

and a

Results:

Frequency response plot is shown in Figure 6.1 20

0

-20

|H(jω)|

-40

-60

-80

-100

-120

0

0.5

1

1.5

2

2.5

3

3.5

ω

Figure 6.1 Frequency response of the designed FIR low pass filter

196

Impulse response of the designed filter is shown in Table 6.1. n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

h(n) 0.000348929034407 0.002041620256007 -0.003699194546301 -0.001605169562357 0.009164322687317 -0.004218401546690 -0.013938530163267 0.017816703296919 0.011121566859506 -0.038879337531751 0.009066680362718 0.062482550639260 -0.066110218910854 -0.081398966517970 0.303531435657815 0.588552019970479 0.303531435657815 -0.081398966517970 -0.066110218910854 0.062482550639260 0.009066680362718 -0.038879337531751 0.011121566859506 0.017816703296919 -0.013938530163267 -0.004218401546690 0.009164322687317 -0.001605169562357 -0.003699194546301 0.002041620256007 0.000348929034407

Table 6.1 Impulse response of the low pass filter

197

Problem # 17 (MATLAB)

Design a linear phase low-pass FIR filter of size M = 41 to have a passband of ω p = stopband of 0.4π . wp=pi/3; ws=0.4*pi; M=41; a=0.75; [h]=lslpf(wp,ws,M,a);

Results:

Frequency response: of the designed filter is shown in Figure 6.2

20

0

-20

|H(jω)|

-40

-60

-80

-100

-120

0

0.5

1

1.5

2

2.5

3

3.5

ω

Figure 6.2 Frequency response of the designed FIR low pass filter

198

π 3

and a

Impulse Response: Impulse response of the filter is shown in Table 6.2. h(n) n 0 -0.005819936189642 1 -0.001377508671024 2 0.006819259377689 3 0.007788830444081 4 -0.002277995286545 5 -0.012334477640978 6 -0.007861761362152 7 0.009408197192394 8 0.018999354147589 9 0.004878060834274 10 -0.020679750764100 11 -0.025381601602936 12 0.004636527834546 13 0.039478498476106 14 0.031177218902214 15 -0.027625379300053 16 -0.076679207755462 17 -0.034824507559385 18 0.115403588391439 19 0.291437705004432 20 0.369669771055028 21 0.291437705004432 22 0.115403588391439 23 -0.034824507559385 24 -0.076679207755462 25 -0.027625379300053 26 0.031177218902214 27 0.039478498476106 28 0.004636527834546 29 -0.025381601602936 30 -0.020679750764100 31 0.004878060834274 32 0.018999354147589 33 0.009408197192394 34 -0.007861761362152 35 -0.012334477640978 36 -0.002277995286545 37 0.007788830444081 38 0.006819259377689 39 -0.001377508671024 40 -0.005819936189642 Table 6.2 Impulse response of the low pass filter

199

Problem # 18 (MATLAB)

Consider the linear model y = 2 x + 1 . Let us assume that we have the following noisy measured samples of x and y: ⎡ 0.6598 ⎤ ⎢ 1.1699 ⎥ ⎢ ⎥ xn = ⎢ 1.2316 ⎥ ⎢ ⎥ ⎢ 1.7602 ⎥ ⎢⎣ 2.3432 ⎥⎦

⎡ 1.3620 ⎤ ⎢ 2.3908 ⎥ ⎢ ⎥ y n = ⎢ 3.5360 ⎥ ⎢ ⎥ ⎢5.0183 ⎥ ⎢⎣ 5.4835 ⎥⎦

Using the above measured noisy data, do the following: a) Fit a straight line using LS algorithm b) Fit a line using TLS approach. c) Plot the resulting fits and compare them with the true model. Solution: xn=[0.6598 1.1699 1.2316 1.7602 2.3432]; yn=[1.3620 2.3908 3.5360 5.0183 5.4835]; p=polyfit(xn,yn,1); X=[xn' ones(size(xn'))]; [z,EA,eb]=tls(X,yn'); x=linspace(min(xn),max(xn),100); yls=polyval(p,x); ytls=polyval(z,x); plot(x,2*x+1) hold on plot(x,yls ,'--') plot(x,ytls,':') legend('true model','LS','TLS') plot(xn,yn,'O') hold off grid on

% Given x (nx1), % this function computes v (nx1) % with v(1)=1 and b (1x1) such that % P=I-bvv' is orthogonal % (PP'=I, or P'=inv(P)) % and Px=||x||e1 (e1=[1 0 ... 0]') % v is called Householder Vector. % function [v,b]=house(x) n=length(x); s=x(2:n)'*x(2:n); v=[1;x(2:n)]; if s==0 b=0; else mu=sqrt(s+x(1)^2); if x(1)n) , b=mx1, the following MATLAB code % computes a vector xtls (nx1) such that (A+EA)xtls=(b+eb) and % F-norm of ||[E,e]|| is minimum. % xtls is called total least square (TLS) solution. % function [xtls,EA,eb]=tls(A,b) [m,n]=size(A); C=[A b]; [U,S,V] = svd(C); sigma=diag(S); h=diff(sigma); I=find(h==0); g=length(I); if g==0 p=0; else p=n+1-I(1); end j=n; m=n-p+1; [x,b]=house(V(j:m,j)); V(j:m,j:n)=(eye(m-j+1)-b*x*x')*V(j:m,j:n); V(j+1:m,j)=x(2:m-j+1); for i=1:n xtls(i)=-V(i,n+1)/V(n+1,n+1); end d=V(1:n+1,n+1); EE=-C*d*d'; EA=EE(:,1:n); eb=EE(:,n+1);

The plots of the true model, the LS fit and the TLS fit are shown in Figure 6.3

201

7 true model LS TLS

6

y

5

4

3

2

1

0.8

1

1.2

1.4

1.6 x

1.8

2

2.2

2.4

2.6

Figure 6.3 The plots of the true model, the LS fit and the TLS fit Problem # 19 (MATLAB)

Consider the set of linear system of equations given by: Ax = b

where A is an n × n symmetric positive definite matrix , b and x are n × 1 vectors. We would like to solve the above equation by using the following recursive algorithm x(k + 1) = x(k ) − μAT ( Ax(k ) − b) a) Show that the above recursive equation minimizes the cost function J = Ax − b

2

b) Find a range for the step size parameter μ that guarantees convergence of the algorithm to the true solution x ∗ = A −1b independent of the initial condition x(0) . c) Implement the above algorithm in MATLAB for the following two cases

202

I)

⎡1 2⎤ A=⎢ ⎥, ⎣ 2 5⎦

⎡5⎤ b=⎢ ⎥ ⎣12⎦

⎡ 2 − 1 1⎤ II) A = ⎢⎢− 1 3 1⎥⎥ , ⎢⎣ 1 1 4⎥⎦

⎡ 2⎤ b = ⎢⎢3⎥⎥ ⎢⎣6⎥⎦

Solution:

a) Let lim x(k ) = x LS then, k →∞

x LS = x LS − μAT ( Ax LS − b) → AT ( Ax LS − b) = 0 → x LS = ( AT A) −1 AT b b) Let e(k ) = x(k ) − x LS , then e(k + 1) = x(k + 1) − x LS = x(k ) − μAT ( Ax(k ) − b) − x Ls = x(k ) − x Ls − μAT Ax(k ) + μAT b = e(k ) − μAT Ax(k ) + μAT AxLS = e(k ) − μAT A( x(k ) − x LS ) = e(k ) − μAT Ae(k ) = ( I − μAT A)e(k ) lim e(k ) = 0 if − 1 < λi ( I − μAT A) < 1 k →∞

for all values of i or

− 1 < 1 − μλi ( AT A) < 1 0