Linear Algebra 3

Industrial AI Lab.

Prof. Seungchul Lee

System of Linear Equations

• Well-determined linear systems

• Under-determined linear systems

• Over-determined linear systems

2

Well-Determined Linear Systems

• System of linear equations

• Geometric interpretation

3

Well-Determined Linear Systems

• System of linear equations

• Generalization

4

Under-Determined Linear Systems

• System of linear equations

• Geometric interpretation

5

Under-Determined Linear Systems

• System of linear equations

• Generalization

6

Over-Determined Linear Systems

• System of linear equations

• Geometric interpretation

7

Over-Determined Linear Systems

• System of linear equations

• Generalization

8

Summary of Linear Systems

• Square: Well-determined

• Fat: Under-determined

• Skinny: Over-determined

9

Least-Norm Solution

• For Under-Determined Linear System

• Find the solution of 𝐴𝑋 = 𝐵 that minimize 𝑋 or 𝑋 2

• i.e., optimization problem

10

Least-Norm Solution

• Optimization problem

• Geometric interpretation

• Select one solution among many solutions

• Often control problem

11

Least-Square Solution

• For Over-Determined Linear System

• Find 𝑋 that minimizes 𝐸 or 𝐸 2 (error)

• i.e. optimization problem

12

Least-Square Solution

• i.e. optimization problem

• Geometric interpretation

• Often estimation problem

13

Vector Projection onto Y

• The vector projection of a vector 𝑋 on (or onto) a nonzero vector 𝑌 is the orthogonal projection of 𝑋onto a straight line parallel to 𝑌

14

Vector Projection onto Y

• Another way of computing 𝜔 and 𝑊

15

Orthogonal Projection onto a Subspace

• Projection of 𝐵 onto a subspace 𝑈 of span of 𝐴1 and 𝐴2• Orthogonality

16