![Page 1: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/1.jpg)
ENGG2013 Unit 5 Linear Combination
& Linear IndependenceJan, 2011.
![Page 2: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/2.jpg)
Last time
• How to multiply a matrix and a vector• Different ways to write down a system of linear equations
– Vector equation
– Matrix-vector product
– Augmented matrix
kshum ENGG2013 2
Column vectors
![Page 3: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/3.jpg)
Review: matrix notation
• In ENGG2013, we use capital bold letter for matrix.• The first subscript is the row index, the second
subscript is the column index.• The number in the i-th row and the j-th column is
called the (i,j)-entry.– cij is the (i,j)-entry in C.
kshum ENGG2013 3
m n
![Page 4: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/4.jpg)
Matrix-vector multiplication
kshum ENGG2013 4
![Page 5: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/5.jpg)
Today• When is A x = b solvable?
– Given A, under what condition does a solution exist for all b?
• For example, the nutrition problem: find a combination of food A, B, C and D in order to satisfy the nutrition requirement exactly.
kshum ENGG2013 5
Food A Food B Food C Food D Requirement
Protein 9 8 3 3 5
Carbohydrate 15 11 1 4 5
Vitamin A 0.02 0.003 0.01 0.006 0.01
Vitamin C 0.01 0.01 0.005 0.05 0.01
Different people havedifferent requirements
Can we solve A x = b for fixed A and various b?
![Page 6: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/6.jpg)
Today
• Basic concepts in linear algebra– Linear combination– Linear independence– Span
kshum ENGG2013 6
![Page 7: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/7.jpg)
Three cases: 0, 1,
kshum ENGG2013 7
A x = b
No solution Unique solution Infinitely manysolutions
How to determine?m equationsn variables
![Page 8: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/8.jpg)
GEOMETRY FOR LINEAR SYSTEM TWO EQUATIONS
kshum ENGG2013 8
![Page 9: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/9.jpg)
Scaling
kshum ENGG2013 9
y
x1
1
y
x
c
c
c is any real number
![Page 10: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/10.jpg)
Representing a straight line by vector
kshum ENGG2013 10
y
x
c
c
y
x
y=x
Any point on the line y=x canbe written as
![Page 11: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/11.jpg)
Adding one more vector
kshum ENGG2013 11
y
x
y=xy
x
y=x+1
![Page 12: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/12.jpg)
We can add another vector and get the same result
kshum ENGG2013 12
y
x
y=x+1
y
x
y=x+1
=
![Page 13: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/13.jpg)
The whole plane
kshum ENGG2013 13
y
x
Scanner
![Page 14: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/14.jpg)
Question 1
kshum ENGG2013 14
Can you find c and d such that
1
2
3
4
5
6
7
2 43 5
?
![Page 15: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/15.jpg)
Question 2
kshum ENGG2013 15
Can you find c and d such that
1
2
3
4
5
6
7
2 43 5
![Page 16: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/16.jpg)
Question 3
kshum ENGG2013 16
Can you find c, d, and e such that
1
2
3
4
5
6
7
2 43 5
![Page 17: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/17.jpg)
GEOMETRY FOR LINEAR SYSTEM THREE EQUATIONS
kshum ENGG2013 17
![Page 18: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/18.jpg)
From line to plane to space
kshum ENGG2013 18
x
y
z
Scalar multiplesof
x
z
Any point in the 3-D spacecan be written as
x
yz
Any point in the x-y planecan be written as
![Page 19: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/19.jpg)
Question 4
kshum ENGG2013 19
x
y
z
The three red arrowsall lie in the x-y plane
Can you find a, b, and c, such that
?
![Page 20: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/20.jpg)
Question 5
kshum ENGG2013 20
x
y
z
The three red arrowsall lie in the shaded plane.
Can you scale up (or down) the three red arrowssuch that the resulting vector sum is equal to theblue vector?
![Page 21: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/21.jpg)
Question 6
kshum ENGG2013 21
x
y
z
The three red arrowsall lie in a straight line.
Can you find x, y and z such that
?
![Page 22: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/22.jpg)
Question 7
kshum ENGG2013 22
x
y
z
The three red arrowsand the blue arroware all on the same line.
Can you find x, y and z such that
?
![Page 23: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/23.jpg)
ALGEBRA FOR LINEAR EQUATIONS
kshum ENGG2013 23
![Page 24: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/24.jpg)
Review on notation
• A vector is a list of numbers.• The set of all vectors with two components is
called .• is a short-hand notation for saying
that – v is a vector with two components– The two components in v are real numbers.
kshum ENGG2013 24
![Page 25: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/25.jpg)
• The set of all vectors with three components is called .
• is a short-hand notation for saying that – v is a vector with three components– The three components in v are real numbers.
kshum ENGG2013 25
![Page 26: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/26.jpg)
• The set of all vectors with n components is called .
• We use a zero in boldface, 0, to represent the all-zero vector
kshum ENGG2013 26
![Page 27: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/27.jpg)
Definition: Linear Combination
• Given vectors v1, v2, …, vi in , and i real number c1, c2, …, ci, the vector w obtained by w = c1 v1+ c2 v2+ …+ ci vi
is called a linear combination of v1, v2, …, vi .
• Examples of linear combination of v1 and v2:
kshum ENGG2013 27
![Page 28: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/28.jpg)
Picture
• Linear combinations of two vectors u and v.
kshum ENGG2013 28
u
v
2u+2v
u–2v
–v
0
3u
2u+0.5v
![Page 29: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/29.jpg)
Definition: Span
• Given r vectors v1, v2, …, vr, the set of all linear combinations of v1, v2, …, vr called the span of v1, v2, …, vr,
• We use the notation span(v1, v2, …, vr) for the span of span of v1, v2, …, vr.
• We also say that span(v1, v2, …, vr) is spanned by, or generated by v1, v2, …, vr .
• span(v1, v2, …, vr) is the collections of all vectors which can be written as c1v1 + c2v2 + … + c2vr for some scalars c1, c2, …, cr.
kshum ENGG2013 29
![Page 30: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/30.jpg)
Span of u and v• Linear combinations of this two vectors u and v form the whole plane
kshum ENGG2013 30
u
v
2u+2v
u–2v
–v
0
3u
![Page 31: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/31.jpg)
Span of a single vector u
kshum ENGG2013 31
x
y
z
consists of thepoints on a straight linewhich passes through the origin.
u
![Page 32: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/32.jpg)
Span of two vectors in 3D
kshum ENGG2013 32
x
y
z
u
v
is a planethrough the origin.
![Page 33: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/33.jpg)
Example
kshum ENGG2013 33
1
2
3
4
5
6
7
2 43 5
is a linear combination of
and , because
We therefore say that
![Page 34: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/34.jpg)
Mathematical language
kshum ENGG2013 34
President Obama is not a Chinese.
Ordinary language Mathematical language
Let C be the set of all Chinese people.
President Obama
![Page 35: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/35.jpg)
Example
kshum ENGG2013 35
x
y
z
![Page 36: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/36.jpg)
A fundamental fact• Let
– A be an mn matrix– b be an m1 vector
• Let the columns of A be v1, v2,…, vn. • The followings are logically equivalent:
kshum ENGG2013 36
We can find a vector x such that1
2
3
“Logically equivalent” meansif one of them is true, then all of them is trueif one of them is false, then all of them is false.
![Page 37: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/37.jpg)
Theorem 1
• With notation as in previous slide, if the span of be v1, v2,…, vn contains all vectors inthen the linear system Ax = b has at least one solution.
• In other words, if every vector in can be written as a linear combination of v1, v2,…, vn, then Ax = b is solvable for any choice of b.
kshum ENGG2013 37
“Solvable” means there is onesolution or more than one solutions.
![Page 38: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/38.jpg)
Example
kshum ENGG2013 38
1
2
3
4
5
6
7
2 43 5
Since and span
the whole plane, the linearsystem
is solvable for any b1 and b2.
![Page 39: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/39.jpg)
Example
kshum ENGG2013 39
x
y
z
The three red arrowsall lie in the x-y plane
x
y
z
(Infinitely many solutions)
![Page 40: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/40.jpg)
Example
kshum ENGG2013 40
1
2
3
4
5
6
7
2 43 5
because is not a
linear combination of
![Page 41: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/41.jpg)
Example
kshum ENGG2013 41
1
2
3
4
5
6
7
2 43 5
has infinitely many solutions.
![Page 42: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/42.jpg)
Infinitely many solutions
kshum ENGG2013 42
x
y
z
x
y
Notice that is a scalar
multiple of
is a linear combination of and
There is one common feature inthe examples with infinitely many solutions
The common feature is that one of the vectoris a linear combination of the others.
![Page 43: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/43.jpg)
Definition: Linear dependence
• Vectors v1, v2, …, vr are said to be linear dependent if we can find r real number c1, c2, …, cr, not all of them equal to zero, such that 0 = c1 v1+ c2 v2+ …+ cr vr
• Otherwise, are v1, v2, …, vr are said to be linear independent.
• In other words, v1, v2, …, vr are be linear independent if, the only choice of c1, c2, …, cr, such that 0 = c1 v1+ c2 v2+ …+ cr vr
is c1 = c2 = …= cr=0. kshum ENGG2013 43
![Page 44: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/44.jpg)
Example of linear independent vectors
kshum ENGG2013 44
![Page 45: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/45.jpg)
Example of linear dependent vectors
kshum ENGG2013 45
![Page 46: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/46.jpg)
Example of linear independent vectors
kshum ENGG2013 46
![Page 47: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/47.jpg)
Example
• and are linear dependent, because
• , and are linear dependent because
kshum ENGG2013 47
![Page 48: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/48.jpg)
Picture
kshum ENGG2013 48
x
y
z
The three vectors lie onthe same plane, namely, the x-y plane.
![Page 49: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/49.jpg)
Theorem 2
• Let – A be an mn matrix– b be an m1 vector
• Let the columns of A be v1, v2,…, vn.
• Theorem: If v1, v2,…, vn, are linear independent, then Ax = b has at most one solution.
kshum ENGG2013 49
![Page 50: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/50.jpg)
Proof (by contradiction)• Suppose that and are two different solutions to Ax=b,
i.e.,
• Therefore
• Move every term to the left
• But v1, v2,…, vn are linear independent by assumption. So, the onlychoice is
• This contradicts the fact that vector x and vector x’ are different.
kshum ENGG2013 50
![Page 51: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/51.jpg)
Example
• and are linearly independent.
• has a unique solution for
any choice of b1 and b2.
kshum ENGG2013 51
In fact, x must equal b1,and y must equal b2/3in this example.
![Page 52: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/52.jpg)
Example
• is solvable
by Theorem 1, because the bluevector lies on the planespanned by the two red vectors.
• The solution is unique
because and are linearly independent.
kshum ENGG2013 52
x
z
y
![Page 53: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/53.jpg)
Summary
kshum ENGG2013 53
At most one solution At least one solution
Ax=bm equationsn variables
Every vector inis a linear combinationof the columns in A.
Columns of A are linearly independent
Uni
que
solu
tion
The columns of Acontain a lot of informationabout the nature of thesolutions.
![Page 54: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/54.jpg)
A kind of mirror symmetry
kshum ENGG2013 54
If the columns of A are linear independent,
then I am pretty sure that there is one or no
solution to Ax=b,no matter what b is.
If any vector in can be written as a
linear combination of the column vectors in A,
then Ax=b must have one or more than one
solutions.
![Page 55: ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011](https://reader036.vdocument.in/reader036/viewer/2022081417/56649d7e5503460f94a608e6/html5/thumbnails/55.jpg)
Basis
• A set of vector in which are simultaneously – linearly independent, and – spanning the whole space
is of particular importance, and is called a set of basis vectors.
(We will talk about basis in more detail later.)
kshum ENGG2013 55