mathematical methods in engineering and science
DESCRIPTION
Mathematical Methods in Engineering and ScienceTRANSCRIPT
![Page 1: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1.jpg)
Mathematical Methods in Engineering and Science 1,
Mathematical Methods in Engineering and
Science[http://home.iitk.ac.in/˜dasgupta/MathCourse]
Bhaskar [email protected]
An Applied Mathematics course forgraduate and senior undergraduate students
and also forrising researchers.
![Page 2: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/2.jpg)
Mathematical Methods in Engineering and Science 2,
Textbook: Dasgupta B., App. Math. Meth.(Pearson Education 2006, 2007).
http://home.iitk.ac.in/˜ dasgupta/MathBook
![Page 3: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/3.jpg)
Mathematical Methods in Engineering and Science 3,
Contents I
Preliminary Background
Matrices and Linear Transformations
Operational Fundamentals of Linear Algebra
Systems of Linear Equations
Gauss Elimination Family of Methods
Special Systems and Special Methods
Numerical Aspects in Linear Systems
![Page 4: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/4.jpg)
Mathematical Methods in Engineering and Science 4,
Contents II
Eigenvalues and Eigenvectors
Diagonalization and Similarity Transformations
Jacobi and Givens Rotation Methods
Householder Transformation and Tridiagonal Matrices
QR Decomposition Method
Eigenvalue Problem of General Matrices
Singular Value Decomposition
Vector Spaces: Fundamental Concepts*
![Page 5: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/5.jpg)
Mathematical Methods in Engineering and Science 5,
Contents III
Topics in Multivariate Calculus
Vector Analysis: Curves and Surfaces
Scalar and Vector Fields
Polynomial Equations
Solution of Nonlinear Equations and Systems
Optimization: Introduction
Multivariate Optimization
Methods of Nonlinear Optimization*
![Page 6: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/6.jpg)
Mathematical Methods in Engineering and Science 6,
Contents IV
Constrained Optimization
Linear and Quadratic Programming Problems*
Interpolation and Approximation
Basic Methods of Numerical Integration
Advanced Topics in Numerical Integration*
Numerical Solution of Ordinary Differential Equations
ODE Solutions: Advanced Issues
Existence and Uniqueness Theory
![Page 7: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/7.jpg)
Mathematical Methods in Engineering and Science 7,
Contents V
First Order Ordinary Differential Equations
Second Order Linear Homogeneous ODE’s
Second Order Linear Non-Homogeneous ODE’s
Higher Order Linear ODE’s
Laplace Transforms
ODE Systems
Stability of Dynamic Systems
Series Solutions and Special Functions
![Page 8: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/8.jpg)
Mathematical Methods in Engineering and Science 8,
Contents VI
Sturm-Liouville Theory
Fourier Series and Integrals
Fourier Transforms
Minimax Approximation*
Partial Differential Equations
Analytic Functions
Integrals in the Complex Plane
Singularities of Complex Functions
![Page 9: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/9.jpg)
Mathematical Methods in Engineering and Science 9,
Contents VIIVariational Calculus*
Epilogue
Selected References
![Page 10: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/10.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 10,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Outline
Preliminary BackgroundTheme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
![Page 11: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/11.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 11,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Theme of the Course
To develop a firm mathematical background necessary for graduatestudies and research
a fast-paced recapitulation of UG mathematics
extension with supplementary advanced ideas for a matureand forward orientation
exposure and highlighting of interconnections
To pre-empt needs of the future challenges
trade-off between sufficient and reasonable
target mid-spectrum majority of students
Notable beneficiaries (at two ends)
would-be researchers in analytical/computational areas
students who are till now somewhat afraid of mathematics
![Page 12: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/12.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 12,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Course Contents
Applied linear algebra
Multivariate calculus and vector calculus
Numerical methods
Differential equations + +
Complex analysis
![Page 13: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/13.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 13,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Sources for More Detailed Study
If you have the time, need and interest, then you may consult
individual books on individual topics;
another “umbrella” volume, like Kreyszig, McQuarrie, O’Neilor Wylie and Barrett;
a good book of numerical analysis or scientific computing, likeActon, Heath, Hildebrand, Krishnamurthy and Sen, Press etal, Stoer and Bulirsch;
friends, in joint-study groups.
![Page 14: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/14.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 14,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Logistic Strategy
Study in the given sequence, to the extent possible.
Do not read mathematics.
![Page 15: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/15.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 15,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Logistic Strategy
Study in the given sequence, to the extent possible.
Do not read mathematics.
Use lots of pen and paper.Read “mathematics books” and do mathematics.
![Page 16: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/16.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 16,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Logistic Strategy
Study in the given sequence, to the extent possible.
Do not read mathematics.
Use lots of pen and paper.Read “mathematics books” and do mathematics.
Exercises are must. Use as many methods as you can think of, certainly including
the one which is recommended. Consult the Appendix after you work out the solution. Follow
the comments, interpretations and suggested extensions. Think. Get excited. Discuss. Bore everybody in your known
circles. Not enough time to attempt all? Want a selection ?
![Page 17: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/17.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 17,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Logistic Strategy
Study in the given sequence, to the extent possible.
Do not read mathematics.
Use lots of pen and paper.Read “mathematics books” and do mathematics.
Exercises are must. Use as many methods as you can think of, certainly including
the one which is recommended. Consult the Appendix after you work out the solution. Follow
the comments, interpretations and suggested extensions. Think. Get excited. Discuss. Bore everybody in your known
circles. Not enough time to attempt all? Want a selection ?
Program implementation is needed in algorithmic exercises. Master a programming environment. Use mathematical/numerical library/software.
Take a MATLAB tutorial session?
![Page 18: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/18.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 18,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Logistic Strategy
Tutorial Plan
Chapter Selection Tutorial Chapter Selection Tutorial2 2,3 3 26 1,2,4,6 43 2,4,5,6 4,5 27 1,2,3,4 3,44 1,2,4,5,7 4,5 28 2,5,6 65 1,4,5 4 29 1,2,5,6 66 1,2,4,7 4 30 1,2,3,4,5 47 1,2,3,4 2 31 1,2 1(d)8 1,2,3,4,6 4 32 1,3,5,7 79 1,2,4 4 33 1,2,3,7,8 810 2,3,4 4 34 1,3,5,6 511 2,4,5 5 35 1,3,4 312 1,3 3 36 1,2,4 413 1,2 1 37 1 1(c)14 2,4,5,6,7 4 38 1,2,3,4,5 515 6,7 7 39 2,3,4,5 416 2,3,4,8 8 40 1,2,4,5 417 1,2,3,6 6 41 1,3,6,8 818 1,2,3,6,7 3 42 1,3,6 619 1,3,4,6 6 43 2,3,4 320 1,2,3 2 44 1,2,4,7,9,10 7,1021 1,2,5,7,8 7 45 1,2,3,4,7,9 4,922 1,2,3,4,5,6 3,4 46 1,2,5,7 723 1,2,3 3 47 1,2,3,5,8,9,10 9,1024 1,2,3,4,5,6 1 48 1,2,4,5 525 1,2,3,4,5 5
![Page 19: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/19.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 19,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Expected Background
moderate background of undergraduate mathematics
firm understanding of school mathematics and undergraduatecalculus
Take the preliminary test. [p 3, App. Math. Meth.]
Grade yourself sincerely. [p 4, App. Math. Meth.]
![Page 20: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/20.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 20,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Expected Background
moderate background of undergraduate mathematics
firm understanding of school mathematics and undergraduatecalculus
Take the preliminary test. [p 3, App. Math. Meth.]
Grade yourself sincerely. [p 4, App. Math. Meth.]
Prerequisite Problem Sets* [p 4–8, App. Math. Meth.]
![Page 21: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/21.jpg)
Mathematical Methods in Engineering and Science Preliminary Background 21,
Theme of the CourseCourse ContentsSources for More Detailed StudyLogistic StrategyExpected Background
Points to note
Put in effort, keep pace.
Stress concept as well as problem-solving.
Follow methods diligently.
Ensure background skills.
Necessary Exercises: Prerequisite problem sets ??
![Page 22: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/22.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 22,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Outline
Matrices and Linear TransformationsMatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
![Page 23: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/23.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 23,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrices
Question: What is a “matrix”?
![Page 24: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/24.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 24,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrices
Question: What is a “matrix”?Answers:
a rectangular array of numbers/elements ?
![Page 25: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/25.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 25,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrices
Question: What is a “matrix”?Answers:
a rectangular array of numbers/elements ?
a mapping f : M × N → F , where M = 1, 2, 3, · · · ,m,N = 1, 2, 3, · · · , n and F is the set of real numbers orcomplex numbers ?
![Page 26: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/26.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 26,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrices
Question: What is a “matrix”?Answers:
a rectangular array of numbers/elements ?
a mapping f : M × N → F , where M = 1, 2, 3, · · · ,m,N = 1, 2, 3, · · · , n and F is the set of real numbers orcomplex numbers ?
Question: What does a matrix do?
![Page 27: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/27.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 27,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrices
Question: What is a “matrix”?Answers:
a rectangular array of numbers/elements ?
a mapping f : M × N → F , where M = 1, 2, 3, · · · ,m,N = 1, 2, 3, · · · , n and F is the set of real numbers orcomplex numbers ?
Question: What does a matrix do?Explore: With an m × n matrix A,
y1 = a11x1 + a12x2 + · · ·+ a1nxn
y2 = a21x1 + a22x2 + · · ·+ a2nxn
......
......
...ym = am1x1 + am2x2 + · · ·+ amnxn
or Ax = y
![Page 28: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/28.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 28,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
MatricesConsider these definitions:
y = f (x) y = f (x) = f (x1, x2, · · · , xn) yk = fk(x) = fk(x1, x2, · · · , xn), k = 1, 2, · · · ,m y = f(x)
![Page 29: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/29.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 29,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
MatricesConsider these definitions:
y = f (x) y = f (x) = f (x1, x2, · · · , xn) yk = fk(x) = fk(x1, x2, · · · , xn), k = 1, 2, · · · ,m y = f(x) y = Ax
![Page 30: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/30.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 30,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
MatricesConsider these definitions:
y = f (x) y = f (x) = f (x1, x2, · · · , xn) yk = fk(x) = fk(x1, x2, · · · , xn), k = 1, 2, · · · ,m y = f(x) y = Ax
Further Answer:
A matrix is the definition of a linear vector function of avector variable.
![Page 31: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/31.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 31,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
MatricesConsider these definitions:
y = f (x) y = f (x) = f (x1, x2, · · · , xn) yk = fk(x) = fk(x1, x2, · · · , xn), k = 1, 2, · · · ,m y = f(x) y = Ax
Further Answer:
A matrix is the definition of a linear vector function of avector variable.
Caution: Matrices do not define vector functions whose components are
of the form
yk = ak0 + ak1x1 + ak2x2 + · · ·+ aknxn.
![Page 32: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/32.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 32,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
MatricesConsider these definitions:
y = f (x) y = f (x) = f (x1, x2, · · · , xn) yk = fk(x) = fk(x1, x2, · · · , xn), k = 1, 2, · · · ,m y = f(x) y = Ax
Further Answer:
A matrix is the definition of a linear vector function of avector variable.
Anything deeper?
Caution: Matrices do not define vector functions whose components are
of the form
yk = ak0 + ak1x1 + ak2x2 + · · ·+ aknxn.
![Page 33: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/33.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 33,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Geometry and Algebra
Let vector x = [x1 x2 x3]T denote a point (x1, x2, x3) in
3-dimensional space in frame of reference OX1X2X3.Example: With m = 2 and n = 3,
y1 = a11x1 + a12x2 + a13x3
y2 = a21x1 + a22x2 + a23x3
.
Plot y1 and y2 in the OY1Y2 plane.
![Page 34: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/34.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 34,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Geometry and Algebra
Let vector x = [x1 x2 x3]T denote a point (x1, x2, x3) in
3-dimensional space in frame of reference OX1X2X3.Example: With m = 2 and n = 3,
y1 = a11x1 + a12x2 + a13x3
y2 = a21x1 + a22x2 + a23x3
.
Plot y1 and y2 in the OY1Y2 plane.
3
2
A R2: R
Co−domain Domain
1x y
3
X
YY
X
X 1
2O O
Figure: Linear transformation: schematic illustration
What is matrix A doing?
![Page 35: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/35.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 35,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Geometry and Algebra
Operating on point x in R3, matrix A transforms it to y in R2.
Point y is the image of point x under the mapping defined bymatrix A.
![Page 36: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/36.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 36,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Geometry and Algebra
Operating on point x in R3, matrix A transforms it to y in R2.
Point y is the image of point x under the mapping defined bymatrix A.
Note domain R3, co-domain R2 with reference to the figure andverify that A : R3 → R2 fulfils the requirements of a mapping, bydefinition.
![Page 37: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/37.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 37,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Geometry and Algebra
Operating on point x in R3, matrix A transforms it to y in R2.
Point y is the image of point x under the mapping defined bymatrix A.
Note domain R3, co-domain R2 with reference to the figure andverify that A : R3 → R2 fulfils the requirements of a mapping, bydefinition.
A matrix gives a definition of a linear transformationfrom one vector space to another.
![Page 38: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/38.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 38,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Linear Transformations
Operate A on a large number of points xi ∈ R3.Obtain corresponding images yi ∈ R2.
The linear transformation represented by A implies the totality ofthese correspondences.
![Page 39: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/39.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 39,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Linear Transformations
Operate A on a large number of points xi ∈ R3.Obtain corresponding images yi ∈ R2.
The linear transformation represented by A implies the totality ofthese correspondences.
We decide to use a different frame of reference OX ′1X
′2X
′3 for R3.
[And, possibly OY ′1Y
′2 for R2 at the same time.]
Coordinates change, i.e. xi changes to x′i (and possibly yi to y′i ).Now, we need a different matrix, say A′, to get back thecorrespondence as y′ = A′x′.
![Page 40: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/40.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 40,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Linear Transformations
Operate A on a large number of points xi ∈ R3.Obtain corresponding images yi ∈ R2.
The linear transformation represented by A implies the totality ofthese correspondences.
We decide to use a different frame of reference OX ′1X
′2X
′3 for R3.
[And, possibly OY ′1Y
′2 for R2 at the same time.]
Coordinates change, i.e. xi changes to x′i (and possibly yi to y′i ).Now, we need a different matrix, say A′, to get back thecorrespondence as y′ = A′x′.
A matrix: just one description.
Question: How to get the new matrix A′?
![Page 41: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/41.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 41,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Matrix Terminology
· · · · · · Matrix product
Transpose
Conjugate transpose
Symmetric and skew-symmetric matrices
Hermitian and skew-Hermitian matrices
Determinant of a square matrix
Inverse of a square matrix
Adjoint of a square matrix
· · · · · ·
![Page 42: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/42.jpg)
Mathematical Methods in Engineering and Science Matrices and Linear Transformations 42,
MatricesGeometry and AlgebraLinear TransformationsMatrix Terminology
Points to note
A matrix defines a linear transformation from one vector spaceto another.
Matrix representation of a linear transformation depends onthe selected bases (or frames of reference) of the source andtarget spaces.
Important: Revise matrix algebra basics as necessary tools.
Necessary Exercises: 2,3
![Page 43: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/43.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 43,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Outline
Operational Fundamentals of Linear AlgebraRange and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
![Page 44: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/44.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 44,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
Consider A ∈ Rm×n as a mapping
A : Rn → Rm, Ax = y, x ∈ Rn, y ∈ Rm.
![Page 45: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/45.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 45,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
Consider A ∈ Rm×n as a mapping
A : Rn → Rm, Ax = y, x ∈ Rn, y ∈ Rm.
Observations
1. Every x ∈ Rn has an image y ∈ Rm, but every y ∈ Rm neednot have a pre-image in Rn.
Range (or range space) as subset/subspace ofco-domain: containing images of all x ∈ Rn.
![Page 46: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/46.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 46,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
Consider A ∈ Rm×n as a mapping
A : Rn → Rm, Ax = y, x ∈ Rn, y ∈ Rm.
Observations
1. Every x ∈ Rn has an image y ∈ Rm, but every y ∈ Rm neednot have a pre-image in Rn.
Range (or range space) as subset/subspace ofco-domain: containing images of all x ∈ Rn.
2. Image of x ∈ Rn in Rm is unique, but pre-image of y ∈ Rm
need not be.It may be non-existent, unique or infinitely many.
Null space as subset/subspace of domain:containing pre-images of only 0 ∈ Rm.
![Page 47: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/47.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 47,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
RmR
n
Null ( ) A
ORange ( ) A
Domain Co−domain
0
A
Figure: Range and null space: schematic representation
![Page 48: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/48.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 48,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
RmR
n
Null ( ) A
ORange ( ) A
Domain Co−domain
0
A
Figure: Range and null space: schematic representation
Question: What is the dimension of a vector space?
![Page 49: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/49.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 49,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
RmR
n
Null ( ) A
ORange ( ) A
Domain Co−domain
0
A
Figure: Range and null space: schematic representation
Question: What is the dimension of a vector space?Linear dependence and independence: Vectors x1, x2, · · · , xr
in a vector space are called linearly independent if
k1x1 + k2x2 + · · ·+ krxr = 0 ⇒ k1 = k2 = · · · = kr = 0.
![Page 50: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/50.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 50,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Range and Null Space: Rank and Nullity
RmR
n
Null ( ) A
ORange ( ) A
Domain Co−domain
0
A
Figure: Range and null space: schematic representation
Question: What is the dimension of a vector space?Linear dependence and independence: Vectors x1, x2, · · · , xr
in a vector space are called linearly independent if
k1x1 + k2x2 + · · ·+ krxr = 0 ⇒ k1 = k2 = · · · = kr = 0.
Range(A) = y : y = Ax, x ∈ RnNull(A) = x : x ∈ Rn, Ax = 0
Rank(A) = dimRange(A)
Nullity(A) = dimNull(A)
![Page 51: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/51.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 51,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Basis
Take a set of vectors v1, v2, · · · , vr in a vector space.Question: Given a vector v in the vector space, can we describe itas
v = k1v1 + k2v2 + · · ·+ krvr = Vk,
where V = [v1 v2 · · · vr ] and k = [k1 k2 · · · kr ]T?
![Page 52: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/52.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 52,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Basis
Take a set of vectors v1, v2, · · · , vr in a vector space.Question: Given a vector v in the vector space, can we describe itas
v = k1v1 + k2v2 + · · ·+ krvr = Vk,
where V = [v1 v2 · · · vr ] and k = [k1 k2 · · · kr ]T?
Answer: Not necessarily.
![Page 53: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/53.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 53,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Basis
Take a set of vectors v1, v2, · · · , vr in a vector space.Question: Given a vector v in the vector space, can we describe itas
v = k1v1 + k2v2 + · · ·+ krvr = Vk,
where V = [v1 v2 · · · vr ] and k = [k1 k2 · · · kr ]T?
Answer: Not necessarily.
Span, denoted as < v1, v2, · · · , vr >: the subspacedescribed/generated by a set of vectors.
Basis:
A basis of a vector space is composed of an orderedminimal set of vectors spanning the entire space.
The basis for an n-dimensional space will have exactly nmembers, all linearly independent.
![Page 54: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/54.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 54,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
BasisOrthogonal basis: v1, v2, · · · , vn with
vTj vk = 0 ∀ j 6= k.
Orthonormal basis:
vTj vk = δjk =
0 if j 6= k1 if j = k
![Page 55: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/55.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 55,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
BasisOrthogonal basis: v1, v2, · · · , vn with
vTj vk = 0 ∀ j 6= k.
Orthonormal basis:
vTj vk = δjk =
0 if j 6= k1 if j = k
Members of an orthonormal basis form an orthogonal matrix.Properties of an orthogonal matrix:
V−1 = VT or VVT = I, and
detV = +1 or − 1,
![Page 56: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/56.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 56,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
BasisOrthogonal basis: v1, v2, · · · , vn with
vTj vk = 0 ∀ j 6= k.
Orthonormal basis:
vTj vk = δjk =
0 if j 6= k1 if j = k
Members of an orthonormal basis form an orthogonal matrix.Properties of an orthogonal matrix:
V−1 = VT or VVT = I, and
detV = +1 or − 1,
Natural basis:
e1 =
100...0
, e2 =
010...0
, · · · , en =
000...1
.
![Page 57: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/57.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 57,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Suppose x represents a vector (point) in Rn in some basis.Question: If we change over to a new basis c1, c2, · · · , cn, howdoes the representation of a vector change?
![Page 58: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/58.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 58,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Suppose x represents a vector (point) in Rn in some basis.Question: If we change over to a new basis c1, c2, · · · , cn, howdoes the representation of a vector change?
x = x1c1 + x2c2 + · · ·+ xncn
= [c1 c2 · · · cn]
x1
x2...xn
.
![Page 59: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/59.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 59,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Suppose x represents a vector (point) in Rn in some basis.Question: If we change over to a new basis c1, c2, · · · , cn, howdoes the representation of a vector change?
x = x1c1 + x2c2 + · · ·+ xncn
= [c1 c2 · · · cn]
x1
x2...xn
.
With C = [c1 c2 · · · cn],
new to old coordinates: Cx = x andold to new coordinates: x = C−1x.
Note: Matrix C is invertible. How?
![Page 60: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/60.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 60,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Suppose x represents a vector (point) in Rn in some basis.Question: If we change over to a new basis c1, c2, · · · , cn, howdoes the representation of a vector change?
x = x1c1 + x2c2 + · · ·+ xncn
= [c1 c2 · · · cn]
x1
x2...xn
.
With C = [c1 c2 · · · cn],
new to old coordinates: Cx = x andold to new coordinates: x = C−1x.
Note: Matrix C is invertible. How?Special case with C orthogonal:
orthogonal coordinate transformation.
![Page 61: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/61.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 61,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Question: And, how does basis change affect the representation ofa linear transformation?
![Page 62: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/62.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 62,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Question: And, how does basis change affect the representation ofa linear transformation?
Consider the mapping A : Rn → Rm, Ax = y.
Change the basis of the domain through P ∈ Rn×n and that of theco-domain through Q ∈ Rm×m.
![Page 63: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/63.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 63,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Question: And, how does basis change affect the representation ofa linear transformation?
Consider the mapping A : Rn → Rm, Ax = y.
Change the basis of the domain through P ∈ Rn×n and that of theco-domain through Q ∈ Rm×m.
New and old vector representations are related as
Px = x and Qy = y.
Then, Ax = y ⇒ Ax = y, with
A = Q−1AP
![Page 64: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/64.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 64,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Change of Basis
Question: And, how does basis change affect the representation ofa linear transformation?
Consider the mapping A : Rn → Rm, Ax = y.
Change the basis of the domain through P ∈ Rn×n and that of theco-domain through Q ∈ Rm×m.
New and old vector representations are related as
Px = x and Qy = y.
Then, Ax = y ⇒ Ax = y, with
A = Q−1AP
Special case: m = n and P = Q gives a similarity transformation
A = P−1AP
![Page 65: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/65.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 65,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Elementary Transformations
Observation: Certain reorganizations of equations in a systemhave no effect on the solution(s).
![Page 66: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/66.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 66,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Elementary Transformations
Observation: Certain reorganizations of equations in a systemhave no effect on the solution(s).
Elementary Row Transformations:
1. interchange of two rows,
2. scaling of a row, and
3. addition of a scalar multiple of a row to another.
![Page 67: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/67.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 67,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Elementary Transformations
Observation: Certain reorganizations of equations in a systemhave no effect on the solution(s).
Elementary Row Transformations:
1. interchange of two rows,
2. scaling of a row, and
3. addition of a scalar multiple of a row to another.
Elementary Column Transformations: Similar operations withcolumns, equivalent to a corresponding shuffling of the variables(unknowns).
![Page 68: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/68.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 68,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Elementary Transformations
Equivalence of matrices: An elementary transformation definesan equivalence relation between two matrices.
Reduction to normal form:
AN =
[
Ir 00 0
]
Rank invariance: Elementary transformations do not alter therank of a matrix.
Elementary transformation as matrix multiplication:
an elementary row transformation on a matrix isequivalent to a pre-multiplication with an elementarymatrix, obtained through the same row transformation onthe identity matrix (of appropriate size).
Similarly, an elementary column transformation is equivalent topost-multiplication with the corresponding elementary matrix.
![Page 69: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/69.jpg)
Mathematical Methods in Engineering and Science Operational Fundamentals of Linear Algebra 69,
Range and Null Space: Rank and NullityBasisChange of BasisElementary Transformations
Points to note
Concepts of range and null space of a linear transformation.
Effects of change of basis on representations of vectors andlinear transformations.
Elementary transformations as tools to modify (simplify)systems of (simultaneous) linear equations.
Necessary Exercises: 2,4,5,6
![Page 70: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/70.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 70,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Outline
Systems of Linear EquationsNature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
![Page 71: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/71.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 71,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Nature of Solutions
Ax = b
![Page 72: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/72.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 72,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Nature of Solutions
Ax = bCoefficient matrix: A, augmented matrix: [A | b].
![Page 73: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/73.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 73,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Nature of Solutions
Ax = bCoefficient matrix: A, augmented matrix: [A | b].Existence of solutions or consistency:
Ax = b has a solution
⇔ b ∈ Range(A)
⇔ Rank(A) = Rank([A | b])
![Page 74: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/74.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 74,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Nature of Solutions
Ax = bCoefficient matrix: A, augmented matrix: [A | b].Existence of solutions or consistency:
Ax = b has a solution
⇔ b ∈ Range(A)
⇔ Rank(A) = Rank([A | b])
Uniqueness of solutions:
Rank(A) = Rank([A | b]) = n
⇔ Solution of Ax = b is unique.
⇔ Ax = 0 has only the trivial (zero) solution.
![Page 75: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/75.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 75,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Nature of Solutions
Ax = bCoefficient matrix: A, augmented matrix: [A | b].Existence of solutions or consistency:
Ax = b has a solution
⇔ b ∈ Range(A)
⇔ Rank(A) = Rank([A | b])
Uniqueness of solutions:
Rank(A) = Rank([A | b]) = n
⇔ Solution of Ax = b is unique.
⇔ Ax = 0 has only the trivial (zero) solution.
Infinite solutions: For Rank(A) = Rank([A|b]) = k < n, solution
x = x + xN , with Ax = b and xN ∈ Null(A)
![Page 76: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/76.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 76,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Basic Idea of Solution Methodology
To diagnose the non-existence of a solution,
To determine the unique solution, or
To describe infinite solutions;
decouple the equations using elementary transformations.
![Page 77: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/77.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 77,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Basic Idea of Solution Methodology
To diagnose the non-existence of a solution,
To determine the unique solution, or
To describe infinite solutions;
decouple the equations using elementary transformations.
For solving Ax = b, apply suitable elementary row transformationson both sides, leading to
RqRq−1 · · ·R2R1Ax = RqRq−1 · · ·R2R1b,
or, [RA]x = Rb;
such that matrix [RA] is greatly simplified.In the best case, with complete reduction, RA = In, andcomponents of x can be read off from Rb.
![Page 78: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/78.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 78,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Basic Idea of Solution Methodology
To diagnose the non-existence of a solution,
To determine the unique solution, or
To describe infinite solutions;
decouple the equations using elementary transformations.
For solving Ax = b, apply suitable elementary row transformationson both sides, leading to
RqRq−1 · · ·R2R1Ax = RqRq−1 · · ·R2R1b,
or, [RA]x = Rb;
such that matrix [RA] is greatly simplified.In the best case, with complete reduction, RA = In, andcomponents of x can be read off from Rb.
For inverting matrix A, treat AA−1 = In similarly.
![Page 79: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/79.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 79,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Homogeneous Systems
To solve Ax = 0 or to describe Null(A),apply a series of elementary row transformations on A to reduce it
to the∼A,
the row-reduced echelon form or RREF.
![Page 80: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/80.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 80,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Homogeneous Systems
To solve Ax = 0 or to describe Null(A),apply a series of elementary row transformations on A to reduce it
to the∼A,
the row-reduced echelon form or RREF.
Features of RREF:
1. The first non-zero entry in any row is a ‘1’, the leading ‘1’.
2. In the same column as the leading ‘1’, other entries are zero.
3. Non-zero entries in a lower row appear later.
![Page 81: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/81.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 81,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Homogeneous Systems
To solve Ax = 0 or to describe Null(A),apply a series of elementary row transformations on A to reduce it
to the∼A,
the row-reduced echelon form or RREF.
Features of RREF:
1. The first non-zero entry in any row is a ‘1’, the leading ‘1’.
2. In the same column as the leading ‘1’, other entries are zero.
3. Non-zero entries in a lower row appear later.
Variables corresponding to columns having leading ‘1’sare expressed in terms of the remaining variables.
![Page 82: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/82.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 82,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Homogeneous Systems
To solve Ax = 0 or to describe Null(A),apply a series of elementary row transformations on A to reduce it
to the∼A,
the row-reduced echelon form or RREF.
Features of RREF:
1. The first non-zero entry in any row is a ‘1’, the leading ‘1’.
2. In the same column as the leading ‘1’, other entries are zero.
3. Non-zero entries in a lower row appear later.
Variables corresponding to columns having leading ‘1’sare expressed in terms of the remaining variables.
Solution of Ax = 0: x =[
z1 z2 · · · zn−k
]
u1
u2
· · ·un−k
Basis of Null(A): z1, z2, · · · , zn−k
![Page 83: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/83.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 83,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Pivoting
Attempt:To get ‘1’ at diagonal (or leading) position, with ‘0’ elsewhere.Key step: division by the diagonal (or leading) entry.
![Page 84: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/84.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 84,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Pivoting
Attempt:To get ‘1’ at diagonal (or leading) position, with ‘0’ elsewhere.Key step: division by the diagonal (or leading) entry.
Consider
A =
Ik . . . . .. δ . . . .. . . . BIG .. big . . . .. . . . . .. . . . . .
.
Cannot divide by zero. Should not divide by δ.
![Page 85: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/85.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 85,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Pivoting
Attempt:To get ‘1’ at diagonal (or leading) position, with ‘0’ elsewhere.Key step: division by the diagonal (or leading) entry.
Consider
A =
Ik . . . . .. δ . . . .. . . . BIG .. big . . . .. . . . . .. . . . . .
.
Cannot divide by zero. Should not divide by δ.
partial pivoting: row interchange to get ‘big’ in place of δ
complete pivoting: row and column interchanges to get‘BIG’ in place of δ
![Page 86: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/86.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 86,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Pivoting
Attempt:To get ‘1’ at diagonal (or leading) position, with ‘0’ elsewhere.Key step: division by the diagonal (or leading) entry.
Consider
A =
Ik . . . . .. δ . . . .. . . . BIG .. big . . . .. . . . . .. . . . . .
.
Cannot divide by zero. Should not divide by δ.
partial pivoting: row interchange to get ‘big’ in place of δ
complete pivoting: row and column interchanges to get‘BIG’ in place of δ
Complete pivoting does not give a huge advantage over partial pivoting,
but requires maintaining of variable permutation for later unscrambling.
![Page 87: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/87.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 87,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Partitioning and Block Operations
Equation Ax = y can be written as
[
A11 A12 A13
A21 A22 A23
]
x1
x2
x3
=
[
y1
y2
]
,
with x1, x2 etc being themselves vectors (or matrices).
For a valid partitioning, block sizes should be consistent.
Elementary transformations can be applied over blocks.
Block operations can be computationally economical at times.
Conceptually, different blocks of contributions/equations canbe assembled for mathematical modelling of complicatedcoupled systems.
![Page 88: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/88.jpg)
Mathematical Methods in Engineering and Science Systems of Linear Equations 88,
Nature of SolutionsBasic Idea of Solution MethodologyHomogeneous SystemsPivotingPartitioning and Block Operations
Points to note
Solution(s) of Ax = b may be non-existent, unique orinfinitely many.
Complete solution can be described by composing a particularsolution with the null space of A.
Null space basis can be obtained conveniently from therow-reduced echelon form of A.
For a strategy of solution, pivoting is an important step.
Necessary Exercises: 1,2,4,5,7
![Page 89: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/89.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 89,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Outline
Gauss Elimination Family of MethodsGauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
![Page 90: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/90.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 90,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gauss-Jordan Elimination
Task: Solve Ax = b1, Ax = b2 and Ax = b3; find A−1 andevaluate A−1B, where A ∈ Rn×n and B ∈ Rn×p.
![Page 91: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/91.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 91,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gauss-Jordan Elimination
Task: Solve Ax = b1, Ax = b2 and Ax = b3; find A−1 andevaluate A−1B, where A ∈ Rn×n and B ∈ Rn×p.
Assemble C = [A b1 b2 b3 In B] ∈ Rn×(2n+3+p)
and follow the algorithm .
![Page 92: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/92.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 92,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gauss-Jordan Elimination
Task: Solve Ax = b1, Ax = b2 and Ax = b3; find A−1 andevaluate A−1B, where A ∈ Rn×n and B ∈ Rn×p.
Assemble C = [A b1 b2 b3 In B] ∈ Rn×(2n+3+p)
and follow the algorithm .
Collect solutions from the result
C −→∼C = [In A−1b1 A−1b2 A−1b3 A−1 A−1B].
![Page 93: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/93.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 93,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gauss-Jordan Elimination
Task: Solve Ax = b1, Ax = b2 and Ax = b3; find A−1 andevaluate A−1B, where A ∈ Rn×n and B ∈ Rn×p.
Assemble C = [A b1 b2 b3 In B] ∈ Rn×(2n+3+p)
and follow the algorithm .
Collect solutions from the result
C −→∼C = [In A−1b1 A−1b2 A−1b3 A−1 A−1B].
Remarks:
Premature termination: matrix A singular — decision?
If you use complete pivoting, unscramble permutation.
Identity matrix in both C and∼C? Store A−1 ‘in place’.
For evaluating A−1b, do not develop A−1.
Gauss-Jordan elimination an overkill? Want somethingcheaper ?
![Page 94: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/94.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 94,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gauss-Jordan Elimination
Gauss-Jordan Algorithm
∆ = 1
For k = 1, 2, 3, · · · , (n − 1)
1. Pivot : identify l such that |clk | = max |cjk | for k ≤ j ≤ n.If clk = 0, then ∆ = 0 and exit.Else, interchange row k and row l .
2. ∆←− ckk∆,Divide row k by ckk .
3. Subtract cjk times row k from row j , ∀j 6= k .
∆←− cnn∆If cnn = 0, then exit.Else, divide row n by cnn.
In case of non-singular A, default termination .
This outline is for partial pivoting.
![Page 95: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/95.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 95,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionGaussian elimination:
Ax = b
−→∼Ax =
∼b
or,
a′11 a′12 · · · a′1na′22 · · · a′2n
. . ....
a′nn
x1
x2...xn
=
b′1
b′2...
b′n
![Page 96: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/96.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 96,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionGaussian elimination:
Ax = b
−→∼Ax =
∼b
or,
a′11 a′12 · · · a′1na′22 · · · a′2n
. . ....
a′nn
x1
x2...xn
=
b′1
b′2...
b′n
Back-substitutions:
xn = b′n/a
′nn,
xi =1
a′ii
b′i −
n∑
j=i+1
a′ijxj
for i = n − 1, n − 2, · · · , 2, 1
![Page 97: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/97.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 97,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionGaussian elimination:
Ax = b
−→∼Ax =
∼b
or,
a′11 a′12 · · · a′1na′22 · · · a′2n
. . ....
a′nn
x1
x2...xn
=
b′1
b′2...
b′n
Back-substitutions:
xn = b′n/a
′nn,
xi =1
a′ii
b′i −
n∑
j=i+1
a′ijxj
for i = n − 1, n − 2, · · · , 2, 1
Remarks Computational cost half compared to G-J elimination. Like G-J elimination, prior knowledge of RHS needed.
![Page 98: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/98.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 98,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionAnatomy of the Gaussian elimination:The process of Gaussian elimination (with no pivoting) leads to
U = RqRq−1 · · ·R2R1A = RA.
![Page 99: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/99.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 99,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionAnatomy of the Gaussian elimination:The process of Gaussian elimination (with no pivoting) leads to
U = RqRq−1 · · ·R2R1A = RA.
The steps given by
for k = 1, 2, 3, · · · , (n − 1)
j-th row ←− j-th row − ajk
akk× k-th row for
j = k + 1, k + 2, · · · , n
involve elementary matrices
Rk |k=1 =
1 0 0 · · · 0− a21
a111 0 · · · 0
− a31a11
0 1 · · · 0...
......
. . ....
− an1a11
0 0 · · · 1
etc .
![Page 100: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/100.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 100,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Gaussian Elimination with Back-SubstitutionAnatomy of the Gaussian elimination:The process of Gaussian elimination (with no pivoting) leads to
U = RqRq−1 · · ·R2R1A = RA.
The steps given by
for k = 1, 2, 3, · · · , (n − 1)
j-th row ←− j-th row − ajk
akk× k-th row for
j = k + 1, k + 2, · · · , n
involve elementary matrices
Rk |k=1 =
1 0 0 · · · 0− a21
a111 0 · · · 0
− a31a11
0 1 · · · 0...
......
. . ....
− an1a11
0 0 · · · 1
etc .
With L = R−1, A = LU.
![Page 101: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/101.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 101,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
A square matrix with non-zero leading minors is LU-decomposable.
![Page 102: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/102.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 102,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
A square matrix with non-zero leading minors is LU-decomposable.
No reference to a right-hand-side (RHS) vector!
![Page 103: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/103.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 103,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
A square matrix with non-zero leading minors is LU-decomposable.
No reference to a right-hand-side (RHS) vector!
To solve Ax = b, denote y = Ux and split as
Ax = b ⇒ LUx = b
⇒ Ly = b and Ux = y.
![Page 104: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/104.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 104,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
A square matrix with non-zero leading minors is LU-decomposable.
No reference to a right-hand-side (RHS) vector!
To solve Ax = b, denote y = Ux and split as
Ax = b ⇒ LUx = b
⇒ Ly = b and Ux = y.
Forward substitutions:
yi =1
lii
bi −i−1∑
j=1
lijyj
for i = 1, 2, 3, · · · , n;
Back-substitutions:
xi =1
uii
yi −n∑
j=i+1
uijxj
for i = n, n− 1, n − 2, · · · , 1.
![Page 105: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/105.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 105,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: How to LU-decompose a given matrix?
![Page 106: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/106.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 106,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: How to LU-decompose a given matrix?
L =
l11 0 0 · · · 0l21 l22 0 · · · 0l31 l32 l33 · · · 0...
......
. . ....
ln1 ln2 ln3 · · · lnn
and U =
u11 u12 u13 · · · u1n
0 u22 u23 · · · u2n
0 0 u33 · · · u3n
......
.... . .
...0 0 0 · · · unn
![Page 107: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/107.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 107,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: How to LU-decompose a given matrix?
L =
l11 0 0 · · · 0l21 l22 0 · · · 0l31 l32 l33 · · · 0...
......
. . ....
ln1 ln2 ln3 · · · lnn
and U =
u11 u12 u13 · · · u1n
0 u22 u23 · · · u2n
0 0 u33 · · · u3n
......
.... . .
...0 0 0 · · · unn
Elements of the product give
i∑
k=1
likukj = aij for i ≤ j ,
and
j∑
k=1
likukj = aij for i > j .
n2 equations in n2 + n unknowns: choice of n unknowns
![Page 108: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/108.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 108,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Doolittle’s algorithm
Choose lii = 1
For j = 1, 2, 3, · · · , n1. uij = aij −
∑i−1k=1 likukj for 1 ≤ i ≤ j
2. lij = 1ujj
(aij −∑j−1
k=1 likukj) for i > j
![Page 109: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/109.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 109,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Doolittle’s algorithm
Choose lii = 1
For j = 1, 2, 3, · · · , n1. uij = aij −
∑i−1k=1 likukj for 1 ≤ i ≤ j
2. lij = 1ujj
(aij −∑j−1
k=1 likukj) for i > j
Evaluation proceeds in column order of the matrix (for storage)
A∗ =
u11 u12 u13 · · · u1n
l21 u22 u23 · · · u2n
l31 l32 u33 · · · u3n...
......
. . ....
ln1 ln2 ln3 · · · unn
![Page 110: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/110.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 110,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: What about matrices which are not LU-decomposable?Question: What about pivoting?
![Page 111: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/111.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 111,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: What about matrices which are not LU-decomposable?Question: What about pivoting?
Consider the non-singular matrix
0 1 23 1 22 1 3
=
1 0 0l21 =? 1 0
l31 l32 1
u11 = 0 u12 u13
0 u22 u23
0 0 u33
.
![Page 112: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/112.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 112,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: What about matrices which are not LU-decomposable?Question: What about pivoting?
Consider the non-singular matrix
0 1 23 1 22 1 3
=
1 0 0l21 =? 1 0
l31 l32 1
u11 = 0 u12 u13
0 u22 u23
0 0 u33
.
LU-decompose a permutation of its rows
0 1 23 1 22 1 3
=
0 1 01 0 00 0 1
3 1 20 1 22 1 3
=
0 1 01 0 00 0 1
1 0 00 1 023
13 1
3 1 20 1 20 0 1
.
![Page 113: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/113.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 113,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
LU Decomposition
Question: What about matrices which are not LU-decomposable?Question: What about pivoting?
Consider the non-singular matrix
0 1 23 1 22 1 3
=
1 0 0l21 =? 1 0
l31 l32 1
u11 = 0 u12 u13
0 u22 u23
0 0 u33
.
LU-decompose a permutation of its rows
0 1 23 1 22 1 3
=
0 1 01 0 00 0 1
3 1 20 1 22 1 3
=
0 1 01 0 00 0 1
1 0 00 1 023
13 1
3 1 20 1 20 0 1
.
In this PLU decomposition, permutation P is recorded in a vector.
![Page 114: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/114.jpg)
Mathematical Methods in Engineering and Science Gauss Elimination Family of Methods 114,
Gauss-Jordan EliminationGaussian Elimination with Back-SubstitutionLU Decomposition
Points to note
For invertible coefficient matrices, use
Gauss-Jordan elimination for large number of RHS vectorsavailable all together and also for matrix inversion,
Gaussian elimination with back-substitution for small numberof RHS vectors available together,
LU decomposition method to develop and maintain factors tobe used as and when RHS vectors are available.
Pivoting is almost necessary (without further special structure).
Necessary Exercises: 1,4,5
![Page 115: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/115.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 115,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Outline
Special Systems and Special MethodsQuadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
![Page 116: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/116.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 116,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Quadratic Forms, Symmetry and Positive Definiteness
Quadratic form
q(x) = xTAx =n∑
i=1
n∑
j=1
aijxixj
![Page 117: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/117.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 117,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Quadratic Forms, Symmetry and Positive Definiteness
Quadratic form
q(x) = xTAx =n∑
i=1
n∑
j=1
aijxixj
defined with respect to a symmetric matrix.
![Page 118: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/118.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 118,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Quadratic Forms, Symmetry and Positive Definiteness
Quadratic form
q(x) = xTAx =n∑
i=1
n∑
j=1
aijxixj
defined with respect to a symmetric matrix.
Quadratic form q(x), equivalently matrix A, is called positivedefinite (p.d.) when
xTAx > 0 ∀ x 6= 0
![Page 119: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/119.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 119,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Quadratic Forms, Symmetry and Positive Definiteness
Quadratic form
q(x) = xTAx =n∑
i=1
n∑
j=1
aijxixj
defined with respect to a symmetric matrix.
Quadratic form q(x), equivalently matrix A, is called positivedefinite (p.d.) when
xTAx > 0 ∀ x 6= 0
and positive semi-definite (p.s.d.) when
xTAx ≥ 0 ∀ x 6= 0.
![Page 120: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/120.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 120,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Quadratic Forms, Symmetry and Positive Definiteness
Quadratic form
q(x) = xTAx =n∑
i=1
n∑
j=1
aijxixj
defined with respect to a symmetric matrix.
Quadratic form q(x), equivalently matrix A, is called positivedefinite (p.d.) when
xTAx > 0 ∀ x 6= 0
and positive semi-definite (p.s.d.) when
xTAx ≥ 0 ∀ x 6= 0.
Sylvester’s criteria:
a11 ≥ 0,
∣
∣
∣
∣
a11 a12
a21 a22
∣
∣
∣
∣
≥ 0, · · · , detA ≥ 0;
i.e. all leading minors non-negative, for p.s.d.
![Page 121: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/121.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 121,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Cholesky Decomposition
If A ∈ Rn×n is symmetric and positive definite, then there exists anon-singular lower triangular matrix L ∈ Rn×n such that
A = LLT .
![Page 122: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/122.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 122,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Cholesky Decomposition
If A ∈ Rn×n is symmetric and positive definite, then there exists anon-singular lower triangular matrix L ∈ Rn×n such that
A = LLT .
Algorithm For i = 1, 2, 3, · · · , n Lii =
√
aii −∑i−1
k=1 L2ik
Lji = 1Lii
(
aji −∑i−1
k=1 LjkLik
)
for i < j ≤ n
![Page 123: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/123.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 123,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Cholesky Decomposition
If A ∈ Rn×n is symmetric and positive definite, then there exists anon-singular lower triangular matrix L ∈ Rn×n such that
A = LLT .
Algorithm For i = 1, 2, 3, · · · , n Lii =
√
aii −∑i−1
k=1 L2ik
Lji = 1Lii
(
aji −∑i−1
k=1 LjkLik
)
for i < j ≤ n
For solving Ax = b,
Forward substitutions: Ly = b
Back-substitutions: LTx = y
![Page 124: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/124.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 124,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Cholesky Decomposition
If A ∈ Rn×n is symmetric and positive definite, then there exists anon-singular lower triangular matrix L ∈ Rn×n such that
A = LLT .
Algorithm For i = 1, 2, 3, · · · , n Lii =
√
aii −∑i−1
k=1 L2ik
Lji = 1Lii
(
aji −∑i−1
k=1 LjkLik
)
for i < j ≤ n
For solving Ax = b,
Forward substitutions: Ly = b
Back-substitutions: LTx = y
Remarks
Test of positive definiteness.
Stable algorithm: no pivoting necessary!
Economy of space and time.
![Page 125: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/125.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 125,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Sparse Systems*
What is a sparse matrix?
Bandedness and bandwidth
Efficient storage and processing
Updates Sherman-Morrison formula
(A + uvT )−1 = A−1 − (A−1u)(vTA−1)
1 + vTA−1u
Woodbury formula
Conjugate gradient method efficiently implemented matrix-vector products
![Page 126: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/126.jpg)
Mathematical Methods in Engineering and Science Special Systems and Special Methods 126,
Quadratic Forms, Symmetry and Positive DefinitenessCholesky DecompositionSparse Systems*
Points to note
Concepts and criteria of positive definiteness and positivesemi-definiteness
Cholesky decomposition method in symmetric positive definitesystems
Nature of sparsity and its exploitation
Necessary Exercises: 1,2,4,7
![Page 127: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/127.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 127,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Outline
Numerical Aspects in Linear SystemsNorms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
![Page 128: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/128.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 128,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition NumbersNorm of a vector: a measure of size
Euclidean norm or 2-norm
‖x‖ = ‖x‖2 =[
x21 + x2
2 + · · ·+ x2n
]12 =
√
xT x
![Page 129: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/129.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 129,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition NumbersNorm of a vector: a measure of size
Euclidean norm or 2-norm
‖x‖ = ‖x‖2 =[
x21 + x2
2 + · · ·+ x2n
]12 =
√
xT x
The p-norm
‖x‖p = [|x1|p + |x2|p + · · · + |xn|p]1p
![Page 130: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/130.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 130,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition NumbersNorm of a vector: a measure of size
Euclidean norm or 2-norm
‖x‖ = ‖x‖2 =[
x21 + x2
2 + · · ·+ x2n
]12 =
√
xT x
The p-norm
‖x‖p = [|x1|p + |x2|p + · · · + |xn|p]1p
The 1-norm: ‖x‖1 = |x1|+ |x2|+ · · · + |xn| The ∞-norm:
‖x‖∞ = limp→∞
[|x1|p + |x2|p + · · ·+ |xn|p]1p = max
j|xj |
![Page 131: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/131.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 131,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition NumbersNorm of a vector: a measure of size
Euclidean norm or 2-norm
‖x‖ = ‖x‖2 =[
x21 + x2
2 + · · ·+ x2n
]12 =
√
xT x
The p-norm
‖x‖p = [|x1|p + |x2|p + · · · + |xn|p]1p
The 1-norm: ‖x‖1 = |x1|+ |x2|+ · · · + |xn| The ∞-norm:
‖x‖∞ = limp→∞
[|x1|p + |x2|p + · · ·+ |xn|p]1p = max
j|xj |
Weighted norm
‖x‖w =√
xTWx
where weight matrix W is symmetric and positive definite.
![Page 132: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/132.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 132,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition Numbers
Norm of a matrix: magnitude or scale of the transformation
![Page 133: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/133.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 133,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition Numbers
Norm of a matrix: magnitude or scale of the transformation
Matrix norm (induced by a vector norm) is given by the largestmagnification it can produce on a vector
‖A‖ = maxx
‖Ax‖‖x‖ = max
‖x‖=1‖Ax‖
![Page 134: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/134.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 134,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition Numbers
Norm of a matrix: magnitude or scale of the transformation
Matrix norm (induced by a vector norm) is given by the largestmagnification it can produce on a vector
‖A‖ = maxx
‖Ax‖‖x‖ = max
‖x‖=1‖Ax‖
Direct consequence: ‖Ax‖ ≤ ‖A‖ ‖x‖
![Page 135: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/135.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 135,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition Numbers
Norm of a matrix: magnitude or scale of the transformation
Matrix norm (induced by a vector norm) is given by the largestmagnification it can produce on a vector
‖A‖ = maxx
‖Ax‖‖x‖ = max
‖x‖=1‖Ax‖
Direct consequence: ‖Ax‖ ≤ ‖A‖ ‖x‖
Index of closeness to singularity: Condition number
κ(A) = ‖A‖ ‖A−1‖, 1 ≤ κ(A) ≤ ∞
![Page 136: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/136.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 136,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Norms and Condition Numbers
Norm of a matrix: magnitude or scale of the transformation
Matrix norm (induced by a vector norm) is given by the largestmagnification it can produce on a vector
‖A‖ = maxx
‖Ax‖‖x‖ = max
‖x‖=1‖Ax‖
Direct consequence: ‖Ax‖ ≤ ‖A‖ ‖x‖
Index of closeness to singularity: Condition number
κ(A) = ‖A‖ ‖A−1‖, 1 ≤ κ(A) ≤ ∞
** Isotropic, well-conditioned, ill-conditioned and singular matrices
![Page 137: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/137.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 137,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and Sensitivity
0.9999x1 − 1.0001x2 = 1x1 − x2 = 1 + ǫ
Solution: x1 = 10001ǫ+12 , x2 = 9999ǫ−1
2
![Page 138: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/138.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 138,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and Sensitivity
0.9999x1 − 1.0001x2 = 1x1 − x2 = 1 + ǫ
Solution: x1 = 10001ǫ+12 , x2 = 9999ǫ−1
2
sensitive to small changes in the RHS insensitive to error in a guess See illustration
![Page 139: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/139.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 139,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and Sensitivity
0.9999x1 − 1.0001x2 = 1x1 − x2 = 1 + ǫ
Solution: x1 = 10001ǫ+12 , x2 = 9999ǫ−1
2
sensitive to small changes in the RHS insensitive to error in a guess See illustration
For the system Ax = b, solution is x = A−1b and
δx = A−1δb − A−1δA x
![Page 140: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/140.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 140,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and Sensitivity
0.9999x1 − 1.0001x2 = 1x1 − x2 = 1 + ǫ
Solution: x1 = 10001ǫ+12 , x2 = 9999ǫ−1
2
sensitive to small changes in the RHS insensitive to error in a guess See illustration
For the system Ax = b, solution is x = A−1b and
δx = A−1δb − A−1δA x
If the matrix A is exactly known, then
‖δx‖‖x‖ ≤ ‖A‖ ‖A
−1‖‖δb‖‖b‖ = κ(A)‖δb‖‖b‖
![Page 141: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/141.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 141,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and Sensitivity
0.9999x1 − 1.0001x2 = 1x1 − x2 = 1 + ǫ
Solution: x1 = 10001ǫ+12 , x2 = 9999ǫ−1
2
sensitive to small changes in the RHS insensitive to error in a guess See illustration
For the system Ax = b, solution is x = A−1b and
δx = A−1δb − A−1δA x
If the matrix A is exactly known, then
‖δx‖‖x‖ ≤ ‖A‖ ‖A
−1‖‖δb‖‖b‖ = κ(A)‖δb‖‖b‖
If the RHS is known exactly, then
‖δx‖‖x‖ ≤ ‖A‖ ‖A
−1‖‖δA‖‖A‖ = κ(A)‖δA‖‖A‖
![Page 142: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/142.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 142,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Ill-conditioning and SensitivityX
Xo 1
2
X(a)
(2)
(1)
(a) Reference system
X
Xo 1
2
X(a)
X(b)
(2)
(1)
(2b)
(b) Parallel shift
o
(2)
X1
X2
(1)
X(c)
(c) Guess validation
X
Xo 1
2
(1)
(d) Singularity
(2) (2d)
Figure: Ill-conditioning: a geometric perspective
![Page 143: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/143.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 143,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = n < m.
ATAx = ATb ⇒ x = (ATA)−1ATb
![Page 144: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/144.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 144,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = n < m.
ATAx = ATb ⇒ x = (ATA)−1ATb
Square of error norm
U(x) =1
2‖Ax − b‖2 =
1
2(Ax − b)T (Ax − b)
=1
2xTATAx − xTATb +
1
2bTb
Least square error solution:
∂U
∂x= ATAx −ATb = 0
![Page 145: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/145.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 145,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = n < m.
ATAx = ATb ⇒ x = (ATA)−1ATb
Square of error norm
U(x) =1
2‖Ax − b‖2 =
1
2(Ax − b)T (Ax − b)
=1
2xTATAx − xTATb +
1
2bTb
Least square error solution:
∂U
∂x= ATAx −ATb = 0
Pseudoinverse or Moore-Penrose inverse or left-inverse
A# = (ATA)−1AT
![Page 146: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/146.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 146,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = m < n.
![Page 147: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/147.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 147,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = m < n.Look for λ ∈ Rm that satisfies ATλ = x and
AATλ = b
Solutionx = ATλ = AT (AAT )−1b
![Page 148: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/148.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 148,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = m < n.Look for λ ∈ Rm that satisfies ATλ = x and
AATλ = b
Solutionx = ATλ = AT (AAT )−1b
Consider the problem
minimize U(x) = 12x
Tx subject to Ax = b.
![Page 149: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/149.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 149,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Rectangular Systems
Consider Ax = b with A ∈ Rm×n and Rank(A) = m < n.Look for λ ∈ Rm that satisfies ATλ = x and
AATλ = b
Solutionx = ATλ = AT (AAT )−1b
Consider the problem
minimize U(x) = 12x
Tx subject to Ax = b.
Extremum of the Lagrangian L(x,λ) = 12x
Tx − λT (Ax − b) isgiven by
∂L∂x
= 0,∂L∂λ
= 0 ⇒ x = ATλ, Ax = b.
Solution x = AT (AAT )−1b gives foot of the perpendicular on thesolution ‘plane’ and the pseudoinverse
A# = AT (AAT )−1
here is a right-inverse!
![Page 150: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/150.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 150,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
![Page 151: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/151.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 151,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
Ax = b may have conflict: form ATAx = ATb.
![Page 152: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/152.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 152,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
Ax = b may have conflict: form ATAx = ATb.
ATA may be ill-conditioned: rig the system as
(ATA + ν2In)x = ATb
Coefficient matrix: symmetric and positive definite!
![Page 153: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/153.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 153,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
Ax = b may have conflict: form ATAx = ATb.
ATA may be ill-conditioned: rig the system as
(ATA + ν2In)x = ATb
Coefficient matrix: symmetric and positive definite!The idea: Immunize the system, paying a small price.
![Page 154: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/154.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 154,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
Ax = b may have conflict: form ATAx = ATb.
ATA may be ill-conditioned: rig the system as
(ATA + ν2In)x = ATb
Coefficient matrix: symmetric and positive definite!The idea: Immunize the system, paying a small price.
Issues:
The choice of ν?
![Page 155: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/155.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 155,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Singularity-Robust Solutions
Ill-posed problems: Tikhonov regularization
recipe for any linear system (m > n, m = n or m < n), withany condition!
Ax = b may have conflict: form ATAx = ATb.
ATA may be ill-conditioned: rig the system as
(ATA + ν2In)x = ATb
Coefficient matrix: symmetric and positive definite!The idea: Immunize the system, paying a small price.
Issues:
The choice of ν?
When m < n, computational advantage by
(AAT + ν2Im)λ = b, x = ATλ
![Page 156: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/156.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 156,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Iterative Methods
Jacobi’s iteration method:
x(k+1)i =
1
aii
bi −n∑
j=1, j 6=i
aijx(k)j
for i = 1, 2, 3, · · · , n.
![Page 157: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/157.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 157,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Iterative Methods
Jacobi’s iteration method:
x(k+1)i =
1
aii
bi −n∑
j=1, j 6=i
aijx(k)j
for i = 1, 2, 3, · · · , n.
Gauss-Seidel method:
x(k+1)i =
1
aii
bi −i−1∑
j=1
aijx(k+1)j −
n∑
j=i+1
aijx(k)j
for i = 1, 2, 3, · · · , n.
![Page 158: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/158.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 158,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Iterative Methods
Jacobi’s iteration method:
x(k+1)i =
1
aii
bi −n∑
j=1, j 6=i
aijx(k)j
for i = 1, 2, 3, · · · , n.
Gauss-Seidel method:
x(k+1)i =
1
aii
bi −i−1∑
j=1
aijx(k+1)j −
n∑
j=i+1
aijx(k)j
for i = 1, 2, 3, · · · , n.
The category of relaxation methods:
diagonal dominance and availability of good initialapproximations
![Page 159: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/159.jpg)
Mathematical Methods in Engineering and Science Numerical Aspects in Linear Systems 159,
Norms and Condition NumbersIll-conditioning and SensitivityRectangular SystemsSingularity-Robust SolutionsIterative Methods
Points to note
Solutions are unreliable when the coefficient matrix isill-conditioned.
Finding pseudoinverse of a full-rank matrix is ‘easy’.
Tikhonov regularization provides singularity-robust solutions.
Iterative methods may have an edge in certain situations!
Necessary Exercises: 1,2,3,4
![Page 160: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/160.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 160,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Outline
Eigenvalues and EigenvectorsEigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
![Page 161: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/161.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 161,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
![Page 162: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/162.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 162,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
Av = λv
Eigenvector (v) and eigenvalue (λ): eigenpair (λ, v)
![Page 163: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/163.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 163,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
Av = λv
Eigenvector (v) and eigenvalue (λ): eigenpair (λ, v)
algebraic eigenvalue problem
![Page 164: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/164.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 164,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
Av = λv
Eigenvector (v) and eigenvalue (λ): eigenpair (λ, v)
algebraic eigenvalue problem
(λI − A)v = 0
For non-trivial (non-zero) solution v,
det(λI − A) = 0
Characteristic equation: characteristic polynomial: n roots
n eigenvalues — for each, find eigenvector(s)
![Page 165: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/165.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 165,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
Av = λv
Eigenvector (v) and eigenvalue (λ): eigenpair (λ, v)
algebraic eigenvalue problem
(λI − A)v = 0
For non-trivial (non-zero) solution v,
det(λI − A) = 0
Characteristic equation: characteristic polynomial: n roots
n eigenvalues — for each, find eigenvector(s)
Multiplicity of an eigenvalue: algebraic and geometric
![Page 166: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/166.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 166,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Eigenvalue Problem
In mapping A : Rn → Rn, special vectors of matrix A ∈ Rn×n
mapped to scalar multiples, i.e. undergo pure scaling
Av = λv
Eigenvector (v) and eigenvalue (λ): eigenpair (λ, v)
algebraic eigenvalue problem
(λI − A)v = 0
For non-trivial (non-zero) solution v,
det(λI − A) = 0
Characteristic equation: characteristic polynomial: n roots
n eigenvalues — for each, find eigenvector(s)
Multiplicity of an eigenvalue: algebraic and geometric
Multiplicity mismatch: diagonalizable and defective matrices
![Page 167: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/167.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 167,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Generalized Eigenvalue Problem
1-dof mass-spring system: mx + kx = 0
Natural frequency of vibration: ωn =√
km
![Page 168: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/168.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 168,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Generalized Eigenvalue Problem
1-dof mass-spring system: mx + kx = 0
Natural frequency of vibration: ωn =√
km
Free vibration of n-dof system:
Mx + Kx = 0,
Natural frequencies and corresponding modes?
![Page 169: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/169.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 169,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Generalized Eigenvalue Problem
1-dof mass-spring system: mx + kx = 0
Natural frequency of vibration: ωn =√
km
Free vibration of n-dof system:
Mx + Kx = 0,
Natural frequencies and corresponding modes?Assuming a vibration mode x = Φ sin(ωt + α),
(−ω2MΦ + KΦ) sin(ωt + α) = 0 ⇒ KΦ = ω2MΦ
![Page 170: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/170.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 170,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Generalized Eigenvalue Problem
1-dof mass-spring system: mx + kx = 0
Natural frequency of vibration: ωn =√
km
Free vibration of n-dof system:
Mx + Kx = 0,
Natural frequencies and corresponding modes?Assuming a vibration mode x = Φ sin(ωt + α),
(−ω2MΦ + KΦ) sin(ωt + α) = 0 ⇒ KΦ = ω2MΦ
Reduce as(
M−1K)
Φ = ω2Φ? Why is it not a good idea?
![Page 171: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/171.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 171,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Generalized Eigenvalue Problem
1-dof mass-spring system: mx + kx = 0
Natural frequency of vibration: ωn =√
km
Free vibration of n-dof system:
Mx + Kx = 0,
Natural frequencies and corresponding modes?Assuming a vibration mode x = Φ sin(ωt + α),
(−ω2MΦ + KΦ) sin(ωt + α) = 0 ⇒ KΦ = ω2MΦ
Reduce as(
M−1K)
Φ = ω2Φ? Why is it not a good idea?
K symmetric, M symmetric and positive definite!!
With M = LLT ,∼Φ = LTΦ and
∼K = L−1KL−T ,
∼K
∼Φ = ω2
∼Φ
![Page 172: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/172.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 172,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Eigenvalues of transpose
Eigenvalues of AT are the same as those of A.
Caution: Eigenvectors of A and AT need not be same.
![Page 173: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/173.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 173,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Eigenvalues of transpose
Eigenvalues of AT are the same as those of A.
Caution: Eigenvectors of A and AT need not be same.
Diagonal and block diagonal matricesEigenvalues of a diagonal matrix are its diagonal entries.Corresponding eigenvectors: natural basis members (e1, e2 etc).
![Page 174: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/174.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 174,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Eigenvalues of transpose
Eigenvalues of AT are the same as those of A.
Caution: Eigenvectors of A and AT need not be same.
Diagonal and block diagonal matricesEigenvalues of a diagonal matrix are its diagonal entries.Corresponding eigenvectors: natural basis members (e1, e2 etc).
Eigenvalues of a block diagonal matrix: those of diagonal blocks.Eigenvectors: coordinate extensions of individual eigenvectors.With (λ2, v2) as eigenpair of block A2,
A∼v2 =
A1 0 00 A2 00 0 A3
0v2
0
=
0A2v2
0
= λ2
0v2
0
![Page 175: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/175.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 175,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Triangular and block triangular matricesEigenvalues of a triangular matrix are its diagonal entries.
![Page 176: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/176.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 176,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Triangular and block triangular matricesEigenvalues of a triangular matrix are its diagonal entries.
Eigenvalues of a block triangular matrix are the collection ofeigenvalues of its diagonal blocks.
![Page 177: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/177.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 177,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Triangular and block triangular matricesEigenvalues of a triangular matrix are its diagonal entries.
Eigenvalues of a block triangular matrix are the collection ofeigenvalues of its diagonal blocks.
Take
H =
[
A B0 C
]
, A ∈ R r×r and C ∈ R s×s
![Page 178: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/178.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 178,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Triangular and block triangular matricesEigenvalues of a triangular matrix are its diagonal entries.
Eigenvalues of a block triangular matrix are the collection ofeigenvalues of its diagonal blocks.
Take
H =
[
A B0 C
]
, A ∈ R r×r and C ∈ R s×s
If Av = λv, then
H
[
v0
]
=
[
A B0 C
] [
v0
]
=
[
Av0
]
=
[
λv0
]
= λ
[
v0
]
![Page 179: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/179.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 179,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Triangular and block triangular matricesEigenvalues of a triangular matrix are its diagonal entries.
Eigenvalues of a block triangular matrix are the collection ofeigenvalues of its diagonal blocks.
Take
H =
[
A B0 C
]
, A ∈ R r×r and C ∈ R s×s
If Av = λv, then
H
[
v0
]
=
[
A B0 C
] [
v0
]
=
[
Av0
]
=
[
λv0
]
= λ
[
v0
]
If µ is an eigenvalue of C, then it is also an eigenvalue of CT and
CTw = µw ⇒ HT
[
0w
]
=
[
AT 0BT CT
] [
0w
]
= µ
[
0w
]
![Page 180: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/180.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 180,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Shift theoremEigenvectors of A + µI are the same as those of A.Eigenvalues: shifted by µ.
![Page 181: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/181.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 181,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
Shift theoremEigenvectors of A + µI are the same as those of A.Eigenvalues: shifted by µ.
DeflationFor a symmetric matrix A, with mutually orthogonal eigenvectors,having (λj , vj ) as an eigenpair,
B = A − λj
vjvTj
vTj vj
has the same eigenstructure as A, except that the eigenvaluecorresponding to vj is zero.
![Page 182: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/182.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 182,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
EigenspaceIf v1, v2, · · · , vk are eigenvectors of A corresponding to the sameeigenvalue λ, then
eigenspace: < v1, v2, · · · , vk >
![Page 183: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/183.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 183,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
EigenspaceIf v1, v2, · · · , vk are eigenvectors of A corresponding to the sameeigenvalue λ, then
eigenspace: < v1, v2, · · · , vk >
Similarity transformationB = S−1AS: same transformation expressed in new basis.
![Page 184: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/184.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 184,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
EigenspaceIf v1, v2, · · · , vk are eigenvectors of A corresponding to the sameeigenvalue λ, then
eigenspace: < v1, v2, · · · , vk >
Similarity transformationB = S−1AS: same transformation expressed in new basis.
det(λI − A) = det S−1 det(λI − A) det S = det(λI − B)
Same characteristic polynomial!
![Page 185: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/185.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 185,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Some Basic Theoretical Results
EigenspaceIf v1, v2, · · · , vk are eigenvectors of A corresponding to the sameeigenvalue λ, then
eigenspace: < v1, v2, · · · , vk >
Similarity transformationB = S−1AS: same transformation expressed in new basis.
det(λI − A) = det S−1 det(λI − A) det S = det(λI − B)
Same characteristic polynomial!
Eigenvalues are the property of a linear transformation,not of the basis.
An eigenvector v of A transforms to S−1v, as the correspondingeigenvector of B.
![Page 186: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/186.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 186,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
![Page 187: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/187.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 187,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
For vector x = α1v1 + α2v2 + · · ·+ αnvn,
Apx = λp1
[
α1v1 +
(
λ2
λ1
)p
α2v2 +
(
λ3
λ1
)p
α3v3 + · · ·+(
λn
λ1
)p
αnvn
]
![Page 188: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/188.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 188,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
For vector x = α1v1 + α2v2 + · · ·+ αnvn,
Apx = λp1
[
α1v1 +
(
λ2
λ1
)p
α2v2 +
(
λ3
λ1
)p
α3v3 + · · ·+(
λn
λ1
)p
αnvn
]
As p →∞, Apx → λp1α1v1, and
![Page 189: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/189.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 189,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
For vector x = α1v1 + α2v2 + · · ·+ αnvn,
Apx = λp1
[
α1v1 +
(
λ2
λ1
)p
α2v2 +
(
λ3
λ1
)p
α3v3 + · · ·+(
λn
λ1
)p
αnvn
]
As p →∞, Apx → λp1α1v1, and
λ1 = limp→∞
(Apx)r(Ap−1x)r
, r = 1, 2, 3, · · · , n.
![Page 190: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/190.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 190,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
For vector x = α1v1 + α2v2 + · · ·+ αnvn,
Apx = λp1
[
α1v1 +
(
λ2
λ1
)p
α2v2 +
(
λ3
λ1
)p
α3v3 + · · ·+(
λn
λ1
)p
αnvn
]
As p →∞, Apx → λp1α1v1, and
λ1 = limp→∞
(Apx)r(Ap−1x)r
, r = 1, 2, 3, · · · , n.
At convergence, n ratios will be the same.
![Page 191: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/191.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 191,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Power Method
Consider matrix A with
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn−1| > |λn|
and a full set of n eigenvectors v1, v2, · · · , vn.
For vector x = α1v1 + α2v2 + · · ·+ αnvn,
Apx = λp1
[
α1v1 +
(
λ2
λ1
)p
α2v2 +
(
λ3
λ1
)p
α3v3 + · · ·+(
λn
λ1
)p
αnvn
]
As p →∞, Apx → λp1α1v1, and
λ1 = limp→∞
(Apx)r(Ap−1x)r
, r = 1, 2, 3, · · · , n.
At convergence, n ratios will be the same.
Question: How to find the least magnitude eigenvalue?
![Page 192: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/192.jpg)
Mathematical Methods in Engineering and Science Eigenvalues and Eigenvectors 192,
Eigenvalue ProblemGeneralized Eigenvalue ProblemSome Basic Theoretical ResultsPower Method
Points to note
Meaning and context of the algebraic eigenvalue problem
Fundamental deductions and vital relationships
Power method as an inexpensive procedure to determineextremal magnitude eigenvalues
Necessary Exercises: 1,2,3,4,6
![Page 193: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/193.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 193,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Outline
Diagonalization and Similarity TransformationsDiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
![Page 194: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/194.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 194,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Consider A ∈ Rn×n, having n eigenvectors v1, v2, · · · , vn;with corresponding eigenvalues λ1, λ2, · · · , λn.
![Page 195: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/195.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 195,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Consider A ∈ Rn×n, having n eigenvectors v1, v2, · · · , vn;with corresponding eigenvalues λ1, λ2, · · · , λn.
AS = A[v1 v2 · · · vn] = [λ1v1 λ2v2 · · · λnvn]
= [v1 v2 · · · vn]
λ1 0 · · · 00 λ2 · · · 0...
.... . .
...0 0 · · · λn
= SΛ
⇒ A = SΛS−1 and S−1AS = Λ
![Page 196: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/196.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 196,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Consider A ∈ Rn×n, having n eigenvectors v1, v2, · · · , vn;with corresponding eigenvalues λ1, λ2, · · · , λn.
AS = A[v1 v2 · · · vn] = [λ1v1 λ2v2 · · · λnvn]
= [v1 v2 · · · vn]
λ1 0 · · · 00 λ2 · · · 0...
.... . .
...0 0 · · · λn
= SΛ
⇒ A = SΛS−1 and S−1AS = Λ
Diagonalization: The process of changing the basis of a lineartransformation so that its new matrix representation is diagonal,i.e. so that it is decoupled among its coordinates.
![Page 197: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/197.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 197,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Diagonalizability:
A matrix having a complete set of n linearly independenteigenvectors is diagonalizable.
![Page 198: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/198.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 198,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Diagonalizability:
A matrix having a complete set of n linearly independenteigenvectors is diagonalizable.
Existence of a complete set of eigenvectors:
A diagonalizable matrix possesses a complete set of nlinearly independent eigenvectors.
![Page 199: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/199.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 199,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Diagonalizability
Diagonalizability:
A matrix having a complete set of n linearly independenteigenvectors is diagonalizable.
Existence of a complete set of eigenvectors:
A diagonalizable matrix possesses a complete set of nlinearly independent eigenvectors.
All distinct eigenvalues implies diagonalizability.
But, diagonalizability does not imply distinct eigenvalues!
However, a lack of diagonalizability certainly implies amultiplicity mismatch.
![Page 200: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/200.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 200,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Jordan canonical form (JCF)
Diagonal (canonical) form
Triangular (canonical) form
![Page 201: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/201.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 201,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Jordan canonical form (JCF)
Diagonal (canonical) form
Triangular (canonical) form
Other convenient forms
Tridiagonal formHessenberg form
![Page 202: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/202.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 202,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical FormsJordan canonical form (JCF): composed of Jordan blocks
J =
J1
J2
. . .
Jk
, Jr =
λ 1λ 1
λ. . .. . . 1
λ
![Page 203: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/203.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 203,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical FormsJordan canonical form (JCF): composed of Jordan blocks
J =
J1
J2
. . .
Jk
, Jr =
λ 1λ 1
λ. . .. . . 1
λ
The key equation AS = SJ in extended form gives
A[· · · Sr · · · ] = [· · · Sr · · · ]
. . .
Jr
. . .
,
where Jordan block Jr is associated with the subspace of
Sr = [v w2 w3 · · · ]
![Page 204: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/204.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 204,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical FormsEquating blocks as ASr = SrJr gives
[Av Aw2 Aw3 · · · ] = [v w2 w3 · · · ]
λ 1λ 1
λ. . .. . .
![Page 205: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/205.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 205,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical FormsEquating blocks as ASr = SrJr gives
[Av Aw2 Aw3 · · · ] = [v w2 w3 · · · ]
λ 1λ 1
λ. . .. . .
Columnwise equality leads to
Av = λv, Aw2 = v + λw2, Aw3 = w2 + λw3, · · ·
![Page 206: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/206.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 206,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical FormsEquating blocks as ASr = SrJr gives
[Av Aw2 Aw3 · · · ] = [v w2 w3 · · · ]
λ 1λ 1
λ. . .. . .
Columnwise equality leads to
Av = λv, Aw2 = v + λw2, Aw3 = w2 + λw3, · · ·
Generalized eigenvectors w2, w3 etc:
(A − λI)v = 0,
(A − λI)w2 = v and (A − λI)2w2 = 0,
(A − λI)w3 = w2 and (A − λI)3w3 = 0, · · ·
![Page 207: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/207.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 207,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Diagonal form
Special case of Jordan form, with each Jordan block of 1× 1size
Matrix is diagonalizable
Similarity transformation matrix S is composed of n linearlyindependent eigenvectors as columns
None of the eigenvectors admits any generalized eigenvector
Equal geometric and algebraic multiplicities for everyeigenvalue
![Page 208: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/208.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 208,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Triangular formTriangularization: Change of basis of a linear tranformation so asto get its matrix in the triangular form
![Page 209: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/209.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 209,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Triangular formTriangularization: Change of basis of a linear tranformation so asto get its matrix in the triangular form
For real eigenvalues, always possible to accomplish withorthogonal similarity transformation
Always possible to accomplish with unitary similaritytransformation, with complex arithmetic
Determination of eigenvalues
![Page 210: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/210.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 210,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Triangular formTriangularization: Change of basis of a linear tranformation so asto get its matrix in the triangular form
For real eigenvalues, always possible to accomplish withorthogonal similarity transformation
Always possible to accomplish with unitary similaritytransformation, with complex arithmetic
Determination of eigenvalues
Note: The case of complex eigenvalues: 2× 2 real diagonal block
[
α −ββ α
]
∼[
α+ iβ 00 α− iβ
]
![Page 211: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/211.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 211,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Forms that can be obtained with pre-determined number ofarithmetic operations (without iteration):
Tridiagonal form: non-zero entries only in the (leading) diagonal,sub-diagonal and super-diagonal
useful for symmetric matrices
![Page 212: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/212.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 212,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Forms that can be obtained with pre-determined number ofarithmetic operations (without iteration):
Tridiagonal form: non-zero entries only in the (leading) diagonal,sub-diagonal and super-diagonal
useful for symmetric matrices
Hessenberg form: A slight generalization of a triangular matrix
Hu =
∗ ∗ ∗ · · · ∗ ∗∗ ∗ ∗ · · · ∗ ∗∗ ∗ · · · ∗ ∗
. . .. . .
......
. . .. . .
...∗ ∗
![Page 213: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/213.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 213,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Canonical Forms
Forms that can be obtained with pre-determined number ofarithmetic operations (without iteration):
Tridiagonal form: non-zero entries only in the (leading) diagonal,sub-diagonal and super-diagonal
useful for symmetric matrices
Hessenberg form: A slight generalization of a triangular matrix
Hu =
∗ ∗ ∗ · · · ∗ ∗∗ ∗ ∗ · · · ∗ ∗∗ ∗ · · · ∗ ∗
. . .. . .
......
. . .. . .
...∗ ∗
Note: Tridiagonal and Hessenberg forms do not fall in thecategory of canonical forms.
![Page 214: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/214.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 214,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Similar results for complex Hermitian matrices.
![Page 215: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/215.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 215,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Eigenvalues must be real.
Similar results for complex Hermitian matrices.
![Page 216: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/216.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 216,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Eigenvalues must be real.A complete set of eigenvectors exists.
Similar results for complex Hermitian matrices.
![Page 217: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/217.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 217,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Eigenvalues must be real.A complete set of eigenvectors exists.Eigenvectors corresponding to distinct eigenvalues are
necessarily orthogonal.
Similar results for complex Hermitian matrices.
![Page 218: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/218.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 218,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Eigenvalues must be real.A complete set of eigenvectors exists.Eigenvectors corresponding to distinct eigenvalues are
necessarily orthogonal.Corresponding to repeated eigenvalues, orthogonal eigenvectors
are available.
Similar results for complex Hermitian matrices.
![Page 219: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/219.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 219,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
A real symmetric matrix has all real eigenvalues and
is diagonalizable through an orthogonal similarity
transformation.
Eigenvalues must be real.A complete set of eigenvectors exists.Eigenvectors corresponding to distinct eigenvalues are
necessarily orthogonal.Corresponding to repeated eigenvalues, orthogonal eigenvectors
are available.
In all cases of a symmetric matrix, we can form anorthogonal matrix V, such that VTAV = Λ is a realdiagonal matrix.
Further, A = VΛVT .
Similar results for complex Hermitian matrices.
![Page 220: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/220.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 220,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvalues of a real symmetric matrix must be real.
![Page 221: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/221.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 221,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvalues of a real symmetric matrix must be real.
Take A ∈ Rn×n such that A = AT , with eigenvalue λ = h + ik.
k = 0 and λ = h
![Page 222: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/222.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 222,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvalues of a real symmetric matrix must be real.
Take A ∈ Rn×n such that A = AT , with eigenvalue λ = h + ik.
Since λI − A is singular, so is
B = (λI − A) (λI − A) = (hI − A + ikI)(hI −A − ikI)
= (hI − A)2 + k2I
k = 0 and λ = h
![Page 223: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/223.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 223,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvalues of a real symmetric matrix must be real.
Take A ∈ Rn×n such that A = AT , with eigenvalue λ = h + ik.
Since λI − A is singular, so is
B = (λI − A) (λI − A) = (hI − A + ikI)(hI −A − ikI)
= (hI − A)2 + k2I
For some x 6= 0, Bx = 0, and
xTBx = 0⇒ xT (hI − A)T (hI − A)x + k2xT x = 0
k = 0 and λ = h
![Page 224: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/224.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 224,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvalues of a real symmetric matrix must be real.
Take A ∈ Rn×n such that A = AT , with eigenvalue λ = h + ik.
Since λI − A is singular, so is
B = (λI − A) (λI − A) = (hI − A + ikI)(hI −A − ikI)
= (hI − A)2 + k2I
For some x 6= 0, Bx = 0, and
xTBx = 0⇒ xT (hI − A)T (hI − A)x + k2xT x = 0
Thus, ‖(hI − A)x‖2 + ‖kx‖2 = 0
k = 0 and λ = h
![Page 225: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/225.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 225,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: A symmetric matrix possesses a complete set ofeigenvectors.
![Page 226: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/226.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 226,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: A symmetric matrix possesses a complete set ofeigenvectors.
Consider a repeated real eigenvalue λ of A and examine its Jordanblock(s).
All Jordan blocks will be of 1× 1 size.
![Page 227: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/227.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 227,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: A symmetric matrix possesses a complete set ofeigenvectors.
Consider a repeated real eigenvalue λ of A and examine its Jordanblock(s).
Suppose Av = λv.The first generalized eigenvector w satisfies (A − λI)w = v, giving
vT (A − λI)w = vT v ⇒ vTATw − λvTw = vT v
⇒ (Av)Tw − λvTw = ‖v‖2
⇒ ‖v‖2 = 0
which is absurd.
All Jordan blocks will be of 1× 1 size.
![Page 228: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/228.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 228,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: A symmetric matrix possesses a complete set ofeigenvectors.
Consider a repeated real eigenvalue λ of A and examine its Jordanblock(s).
Suppose Av = λv.The first generalized eigenvector w satisfies (A − λI)w = v, giving
vT (A − λI)w = vT v ⇒ vTATw − λvTw = vT v
⇒ (Av)Tw − λvTw = ‖v‖2
⇒ ‖v‖2 = 0
which is absurd.
An eigenvector will not admit a generalized eigenvector.
All Jordan blocks will be of 1× 1 size.
![Page 229: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/229.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 229,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
![Page 230: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/230.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 230,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
Take two eigenpairs (λ1, v1) and (λ2, v2), with λ1 6= λ2.
vT1 v2 = 0
![Page 231: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/231.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 231,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
Take two eigenpairs (λ1, v1) and (λ2, v2), with λ1 6= λ2.
vT1 Av2 = vT
1 (λ2v2) = λ2vT1 v2
vT1 Av2 = vT
1 ATv2 = (Av1)Tv2 = (λ1v1)
T v2 = λ1vT1 v2
vT1 v2 = 0
![Page 232: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/232.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 232,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
Take two eigenpairs (λ1, v1) and (λ2, v2), with λ1 6= λ2.
vT1 Av2 = vT
1 (λ2v2) = λ2vT1 v2
vT1 Av2 = vT
1 ATv2 = (Av1)Tv2 = (λ1v1)
T v2 = λ1vT1 v2
From the two expressions, (λ1 − λ2)vT1 v2 = 0
vT1 v2 = 0
![Page 233: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/233.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 233,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
Take two eigenpairs (λ1, v1) and (λ2, v2), with λ1 6= λ2.
vT1 Av2 = vT
1 (λ2v2) = λ2vT1 v2
vT1 Av2 = vT
1 ATv2 = (Av1)Tv2 = (λ1v1)
T v2 = λ1vT1 v2
From the two expressions, (λ1 − λ2)vT1 v2 = 0
vT1 v2 = 0
Proposition: Corresponding to a repeated eigenvalue of asymmetric matrix, an appropriate number of orthogonaleigenvectors can be selected.
![Page 234: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/234.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 234,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Proposition: Eigenvectors of a symmetric matrix corresponding todistinct eigenvalues are necessarily orthogonal.
Take two eigenpairs (λ1, v1) and (λ2, v2), with λ1 6= λ2.
vT1 Av2 = vT
1 (λ2v2) = λ2vT1 v2
vT1 Av2 = vT
1 ATv2 = (Av1)Tv2 = (λ1v1)
T v2 = λ1vT1 v2
From the two expressions, (λ1 − λ2)vT1 v2 = 0
vT1 v2 = 0
Proposition: Corresponding to a repeated eigenvalue of asymmetric matrix, an appropriate number of orthogonaleigenvectors can be selected.
If λ1 = λ2, then the entire subspace < v1, v2 > is an eigenspace.Select any two mutually orthogonal eigenvectors for the basis.
![Page 235: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/235.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 235,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Facilities with the ‘omnipresent’ symmetric matrices:
Expression
A = VΛVT
= [v1 v2 · · · vn]
λ1
λ2
. . .
λn
vT1
vT2...
vTn
= λ1v1vT1 + λ2v2v
T2 + · · ·+ λnvnv
Tn =
n∑
i=1
λivivTi
![Page 236: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/236.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 236,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Facilities with the ‘omnipresent’ symmetric matrices:
Expression
A = VΛVT
= [v1 v2 · · · vn]
λ1
λ2
. . .
λn
vT1
vT2...
vTn
= λ1v1vT1 + λ2v2v
T2 + · · ·+ λnvnv
Tn =
n∑
i=1
λivivTi
Reconstruction from a sum of rank-one components Efficient storage with only large eigenvalues and corresponding
eigenvectors
![Page 237: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/237.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 237,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Symmetric Matrices
Facilities with the ‘omnipresent’ symmetric matrices:
Expression
A = VΛVT
= [v1 v2 · · · vn]
λ1
λ2
. . .
λn
vT1
vT2...
vTn
= λ1v1vT1 + λ2v2v
T2 + · · ·+ λnvnv
Tn =
n∑
i=1
λivivTi
Reconstruction from a sum of rank-one components Efficient storage with only large eigenvalues and corresponding
eigenvectors Deflation technique Stable and effective methods: easier to solve the eigenvalue
problem
![Page 238: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/238.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 238,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Similarity Transformations
Hessenberg
Symmetric Tridiagonal Triangular
Symmetric Tridiagonal
Diagonal
General
Figure: Eigenvalue problem: forms and steps
![Page 239: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/239.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 239,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Similarity Transformations
Hessenberg
Symmetric Tridiagonal Triangular
Symmetric Tridiagonal
Diagonal
General
Figure: Eigenvalue problem: forms and steps
How to find suitable similarity transformations?
![Page 240: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/240.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 240,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Similarity Transformations
Hessenberg
Symmetric Tridiagonal Triangular
Symmetric Tridiagonal
Diagonal
General
Figure: Eigenvalue problem: forms and steps
How to find suitable similarity transformations?
1. rotation
2. reflection
3. matrix decomposition or factorization
4. elementary transformation
![Page 241: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/241.jpg)
Mathematical Methods in Engineering and Science Diagonalization and Similarity Transformations 241,
DiagonalizabilityCanonical FormsSymmetric MatricesSimilarity Transformations
Points to note
Generally possible reduction: Jordan canonical form
Condition of diagonalizability and the diagonal form
Possible with orthogonal similarity transformations: triangularform
Useful non-canonical forms: tridiagonal and Hessenberg
Orthogonal diagonalization of symmetric matrices
Caution: Each step in this context to be effected throughsimilarity transformations
Necessary Exercises: 1,2,4
![Page 242: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/242.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 242,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Outline
Jacobi and Givens Rotation Methods(for symmetric matrices)
Plane RotationsJacobi Rotation MethodGivens Rotation Method
![Page 243: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/243.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 243,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane Rotations
Y
X
X
Y /
/
x
y
x
y
/
/
N
P (x, y)
O
φ
φL
K
M
Figure: Rotation of axes and change of basis
![Page 244: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/244.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 244,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane Rotations
Y
X
X
Y /
/
x
y
x
y
/
/
N
P (x, y)
O
φ
φL
K
M
Figure: Rotation of axes and change of basis
x = OL + LM = OL + KN = x ′ cosφ+ y ′ sinφ
y = PN −MN = PN − LK = y ′ cosφ− x ′ sinφ
![Page 245: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/245.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 245,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane Rotations
Orthogonal change of basis:
r =
[
xy
]
=
[
cosφ sinφ− sinφ cosφ
] [
x ′
y ′
]
= ℜr′
![Page 246: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/246.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 246,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane Rotations
Orthogonal change of basis:
r =
[
xy
]
=
[
cosφ sinφ− sinφ cosφ
] [
x ′
y ′
]
= ℜr′
Mapping of position vectors with
ℜ−1 = ℜT =
[
cosφ − sinφsinφ cosφ
]
![Page 247: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/247.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 247,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane Rotations
Orthogonal change of basis:
r =
[
xy
]
=
[
cosφ sinφ− sinφ cosφ
] [
x ′
y ′
]
= ℜr′
Mapping of position vectors with
ℜ−1 = ℜT =
[
cosφ − sinφsinφ cosφ
]
In three-dimensional (ambient) space,
ℜxy =
cosφ sinφ 0− sinφ cosφ 0
0 0 1
, ℜxz =
cosφ 0 sinφ0 1 0
− sinφ 0 cosφ
etc.
![Page 248: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/248.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 248,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane RotationsGeneralizing to n-dimensional Euclidean space (Rn),
Ppq =
1 0 01 0 0
. . ....
...1 0 0
0 0 · · · 0 c 0 · · · 0 s · · · 00 1 0...
. . ....
0 1 00 0 · · · 0 −s 0 · · · 0 c · · · 0
......
. . .
0 0 1
![Page 249: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/249.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 249,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Plane RotationsGeneralizing to n-dimensional Euclidean space (Rn),
Ppq =
1 0 01 0 0
. . ....
...1 0 0
0 0 · · · 0 c 0 · · · 0 s · · · 00 1 0...
. . ....
0 1 00 0 · · · 0 −s 0 · · · 0 c · · · 0
......
. . .
0 0 1
Matrix A is transformed as
A′ = P−1pq APpq = PT
pqAPpq,
only the p-th and q-th rows and columns being affected.
![Page 250: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/250.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 250,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation Method
a′pr = a′rp = carp − sarq for p 6= r 6= q,
a′qr = a′rq = carq + sarp for p 6= r 6= q,
a′pp = c2app + s2aqq − 2scapq ,
a′qq = s2app + c2aqq + 2scapq , and
a′pq = a′qp = (c2 − s2)apq + sc(app − aqq)
![Page 251: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/251.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 251,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation Method
a′pr = a′rp = carp − sarq for p 6= r 6= q,
a′qr = a′rq = carq + sarp for p 6= r 6= q,
a′pp = c2app + s2aqq − 2scapq ,
a′qq = s2app + c2aqq + 2scapq , and
a′pq = a′qp = (c2 − s2)apq + sc(app − aqq)
In a Jacobi rotation,
a′pq = 0⇒ c2 − s2
2sc=
aqq − app
2apq= k (say).
Left side is cot 2φ: solve this equation for φ.
![Page 252: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/252.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 252,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation Method
a′pr = a′rp = carp − sarq for p 6= r 6= q,
a′qr = a′rq = carq + sarp for p 6= r 6= q,
a′pp = c2app + s2aqq − 2scapq ,
a′qq = s2app + c2aqq + 2scapq , and
a′pq = a′qp = (c2 − s2)apq + sc(app − aqq)
In a Jacobi rotation,
a′pq = 0⇒ c2 − s2
2sc=
aqq − app
2apq= k (say).
Left side is cot 2φ: solve this equation for φ.
Jacobi rotation transformations P12, P13, · · · , P1n; P23, · · · , P2n;· · · ; Pn−1,n complete a full sweep.
![Page 253: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/253.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 253,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation Method
a′pr = a′rp = carp − sarq for p 6= r 6= q,
a′qr = a′rq = carq + sarp for p 6= r 6= q,
a′pp = c2app + s2aqq − 2scapq ,
a′qq = s2app + c2aqq + 2scapq , and
a′pq = a′qp = (c2 − s2)apq + sc(app − aqq)
In a Jacobi rotation,
a′pq = 0⇒ c2 − s2
2sc=
aqq − app
2apq= k (say).
Left side is cot 2φ: solve this equation for φ.
Jacobi rotation transformations P12, P13, · · · , P1n; P23, · · · , P2n;· · · ; Pn−1,n complete a full sweep.
Note: The resulting matrix is far from diagonal!
![Page 254: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/254.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 254,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation MethodSum of squares of off-diagonal terms before the transformation
S =∑
r 6=s
|ars |2 = 2
∑
r 6=p
a2rp +
∑
p 6=r 6=q
a2rq
= 2
∑
p 6=r 6=q
(a2rp + a2
rq) + a2pq
![Page 255: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/255.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 255,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation MethodSum of squares of off-diagonal terms before the transformation
S =∑
r 6=s
|ars |2 = 2
∑
r 6=p
a2rp +
∑
p 6=r 6=q
a2rq
= 2
∑
p 6=r 6=q
(a2rp + a2
rq) + a2pq
and that afterwards
S ′ = 2
∑
p 6=r 6=q
(a′2rp + a′2rq) + a′2pq
= 2∑
p 6=r 6=q
(a2rp + a2
rq)
![Page 256: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/256.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 256,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Jacobi Rotation MethodSum of squares of off-diagonal terms before the transformation
S =∑
r 6=s
|ars |2 = 2
∑
r 6=p
a2rp +
∑
p 6=r 6=q
a2rq
= 2
∑
p 6=r 6=q
(a2rp + a2
rq) + a2pq
and that afterwards
S ′ = 2
∑
p 6=r 6=q
(a′2rp + a′2rq) + a′2pq
= 2∑
p 6=r 6=q
(a2rp + a2
rq)
differ by
∆S = S ′ − S = −2a2pq ≤ 0; and S → 0.
![Page 257: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/257.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 257,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
While applying the rotation Ppq, demand a′rq = 0: tanφ = − arq
arp
![Page 258: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/258.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 258,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
While applying the rotation Ppq, demand a′rq = 0: tanφ = − arq
arp
r = p − 1: Givens rotation
Once ap−1,q is annihilated, it is never updated again!
![Page 259: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/259.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 259,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
While applying the rotation Ppq, demand a′rq = 0: tanφ = − arq
arp
r = p − 1: Givens rotation
Once ap−1,q is annihilated, it is never updated again!
Sweep P23, P24, · · · , P2n; P34, · · · , P3n; · · · ; Pn−1,n toannihilate a13, a14, · · · , a1n; a24, · · · , a2n; · · · ; an−2,n.
Symmetric tridiagonal matrix
![Page 260: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/260.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 260,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
While applying the rotation Ppq, demand a′rq = 0: tanφ = − arq
arp
r = p − 1: Givens rotation
Once ap−1,q is annihilated, it is never updated again!
Sweep P23, P24, · · · , P2n; P34, · · · , P3n; · · · ; Pn−1,n toannihilate a13, a14, · · · , a1n; a24, · · · , a2n; · · · ; an−2,n.
Symmetric tridiagonal matrix
How do eigenvectors transform through Jacobi/Givens rotationsteps?
∼A = · · ·P(2)T P(1)T AP(1)P(2) · · ·
Product matrix P(1)P(2) · · · gives the basis.
![Page 261: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/261.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 261,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
While applying the rotation Ppq, demand a′rq = 0: tanφ = − arq
arp
r = p − 1: Givens rotation
Once ap−1,q is annihilated, it is never updated again!
Sweep P23, P24, · · · , P2n; P34, · · · , P3n; · · · ; Pn−1,n toannihilate a13, a14, · · · , a1n; a24, · · · , a2n; · · · ; an−2,n.
Symmetric tridiagonal matrix
How do eigenvectors transform through Jacobi/Givens rotationsteps?
∼A = · · ·P(2)T P(1)T AP(1)P(2) · · ·
Product matrix P(1)P(2) · · · gives the basis.
To record it, initialize V by identity and keep multiplying newrotation matrices on the right side.
![Page 262: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/262.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 262,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
Contrast between Jacobi and Givens rotation methods
What happens to intermediate zeros?
What do we get after a complete sweep?
How many sweeps are to be applied?
What is the intended final form of the matrix?
How is size of the matrix relevant in the choice of the method?
![Page 263: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/263.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 263,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Givens Rotation Method
Contrast between Jacobi and Givens rotation methods
What happens to intermediate zeros?
What do we get after a complete sweep?
How many sweeps are to be applied?
What is the intended final form of the matrix?
How is size of the matrix relevant in the choice of the method?
Fast forward ...
Householder method accomplishes ‘tridiagonalization’ moreefficiently than Givens rotation method.
But, with a half-processed matrix, there come situations inwhich Givens rotation method turns out to be more efficient!
![Page 264: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/264.jpg)
Mathematical Methods in Engineering and Science Jacobi and Givens Rotation Methods 264,
Plane RotationsJacobi Rotation MethodGivens Rotation Method
Points to note
Rotation transformation on symmetric matrices
Plane rotations provide orthogonal change of basis that canbe used for diagonalization of matrices.
For small matrices (say 4 ≤ n ≤ 8), Jacobi rotation sweepsare competitive enough for diagonalization upto a reasonabletolerance.
For large matrices, one sweep of Givens rotations can beapplied to get a symmetric tridiagonal matrix, for efficientfurther processing.
Necessary Exercises: 2,3,4
![Page 265: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/265.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 265,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Outline
Householder Transformation and Tridiagonal MatricesHouseholder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
![Page 266: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/266.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 266,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Reflection Transformationu
w
v
u − v
OPlane of Reflection
Figure: Vectors in Householder reflection
Consider u, v ∈ Rk , ‖u‖ = ‖v‖ and w = u−v‖u−v‖ .
![Page 267: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/267.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 267,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Reflection Transformationu
w
v
u − v
OPlane of Reflection
Figure: Vectors in Householder reflection
Consider u, v ∈ Rk , ‖u‖ = ‖v‖ and w = u−v‖u−v‖ .
Householder reflection matrix
Hk = Ik − 2wwT
is symmetric and orthogonal.
![Page 268: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/268.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 268,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Reflection Transformationu
w
v
u − v
OPlane of Reflection
Figure: Vectors in Householder reflection
Consider u, v ∈ Rk , ‖u‖ = ‖v‖ and w = u−v‖u−v‖ .
Householder reflection matrix
Hk = Ik − 2wwT
is symmetric and orthogonal.
For any vector x orthogonal to w,
Hkx = (Ik −2wwT )x = x and Hkw = (Ik−2wwT )w = −w.
![Page 269: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/269.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 269,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Reflection Transformationu
w
v
u − v
OPlane of Reflection
Figure: Vectors in Householder reflection
Consider u, v ∈ Rk , ‖u‖ = ‖v‖ and w = u−v‖u−v‖ .
Householder reflection matrix
Hk = Ik − 2wwT
is symmetric and orthogonal.
For any vector x orthogonal to w,
Hkx = (Ik −2wwT )x = x and Hkw = (Ik−2wwT )w = −w.
Hence, Hky = Hk(yw + y⊥) = −yw + y⊥, Hku = v and Hkv = u.
![Page 270: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/270.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 270,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Method
Consider n × n symmetric matrix A.Let u = [a21 a31 · · · an1]
T ∈ Rn−1 and v = ‖u‖e1 ∈ Rn−1.
![Page 271: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/271.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 271,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Method
Consider n × n symmetric matrix A.Let u = [a21 a31 · · · an1]
T ∈ Rn−1 and v = ‖u‖e1 ∈ Rn−1.
Construct P1 =
[
1 00 Hn−1
]
and operate as
A(1) = P1AP1 =
[
1 00 Hn−1
] [
a11 uT
u A1
] [
1 00 Hn−1
]
=
[
a11 vT
v Hn−1A1Hn−1
]
.
![Page 272: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/272.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 272,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder Method
Consider n × n symmetric matrix A.Let u = [a21 a31 · · · an1]
T ∈ Rn−1 and v = ‖u‖e1 ∈ Rn−1.
Construct P1 =
[
1 00 Hn−1
]
and operate as
A(1) = P1AP1 =
[
1 00 Hn−1
] [
a11 uT
u A1
] [
1 00 Hn−1
]
=
[
a11 vT
v Hn−1A1Hn−1
]
.
Reorganizing and re-naming,
A(1) =
d1 e2 0e2 d2 uT
2
0 u2 A2
.
![Page 273: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/273.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 273,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder MethodNext, with v2 = ‖u2‖e1, we form
P2 =
[
I2 00 Hn−2
]
and operate as A(2) = P2A(1)P2.
![Page 274: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/274.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 274,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder MethodNext, with v2 = ‖u2‖e1, we form
P2 =
[
I2 00 Hn−2
]
and operate as A(2) = P2A(1)P2.
After j steps,
A(j) =
d1 e2
e2 d2. . .
. . .. . . ej+1
ej+1 dj+1 uTj+1
uj+1 Aj+1
![Page 275: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/275.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 275,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Householder MethodNext, with v2 = ‖u2‖e1, we form
P2 =
[
I2 00 Hn−2
]
and operate as A(2) = P2A(1)P2.
After j steps,
A(j) =
d1 e2
e2 d2. . .
. . .. . . ej+1
ej+1 dj+1 uTj+1
uj+1 Aj+1
By n − 2 steps, with P = P1P2P3 · · ·Pn−2,
A(n−2) = PTAP
is symmetric tridiagonal.
![Page 276: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/276.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 276,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
T =
d1 e2
e2 d2. . .
. . .. . . en−1
en−1 dn−1 en
en dn
![Page 277: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/277.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 277,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
T =
d1 e2
e2 d2. . .
. . .. . . en−1
en−1 dn−1 en
en dn
Characteristic polynomial
p(λ) =
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
λ− d1 −e2
−e2 λ− d2. . .
. . .. . . −en−1
−en−1 λ− dn−1 −en
−en λ− dn
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
∣
.
![Page 278: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/278.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 278,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Characteristic polynomial of the leading k × k sub-matrix: pk(λ)
p0(λ) = 1,
p1(λ) = λ− d1,
p2(λ) = (λ− d2)(λ− d1)− e22 ,
· · · · · · · · · ,pk+1(λ) = (λ− dk+1)pk(λ)− e2
k+1pk−1(λ).
![Page 279: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/279.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 279,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Characteristic polynomial of the leading k × k sub-matrix: pk(λ)
p0(λ) = 1,
p1(λ) = λ− d1,
p2(λ) = (λ− d2)(λ− d1)− e22 ,
· · · · · · · · · ,pk+1(λ) = (λ− dk+1)pk(λ)− e2
k+1pk−1(λ).
P(λ) = p0(λ), p1(λ), · · · , pn(λ) a Sturmian sequence if ej 6= 0 ∀j
![Page 280: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/280.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 280,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Characteristic polynomial of the leading k × k sub-matrix: pk(λ)
p0(λ) = 1,
p1(λ) = λ− d1,
p2(λ) = (λ− d2)(λ− d1)− e22 ,
· · · · · · · · · ,pk+1(λ) = (λ− dk+1)pk(λ)− e2
k+1pk−1(λ).
P(λ) = p0(λ), p1(λ), · · · , pn(λ) a Sturmian sequence if ej 6= 0 ∀j
Question: What if ej = 0 for some j?!
![Page 281: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/281.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 281,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Characteristic polynomial of the leading k × k sub-matrix: pk(λ)
p0(λ) = 1,
p1(λ) = λ− d1,
p2(λ) = (λ− d2)(λ− d1)− e22 ,
· · · · · · · · · ,pk+1(λ) = (λ− dk+1)pk(λ)− e2
k+1pk−1(λ).
P(λ) = p0(λ), p1(λ), · · · , pn(λ) a Sturmian sequence if ej 6= 0 ∀j
Question: What if ej = 0 for some j?!Answer: That is good news. Split the matrix.
![Page 282: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/282.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 282,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Sturmian sequence property of P(λ) with ej 6= 0:
Interlacing property: Roots of pk+1(λ) interlace theroots of pk(λ). That is, if the roots of pk+1(λ) areλ1 > λ2 > · · · > λk+1 and those of pk(λ) areµ1 > µ2 > · · · > µk ; then
λ1 > µ1 > λ2 > µ2 > · · · · · · > λk > µk > λk+1.
This property leads to a convenient procedure .
![Page 283: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/283.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 283,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Sturmian sequence property of P(λ) with ej 6= 0:
Interlacing property: Roots of pk+1(λ) interlace theroots of pk(λ). That is, if the roots of pk+1(λ) areλ1 > λ2 > · · · > λk+1 and those of pk(λ) areµ1 > µ2 > · · · > µk ; then
λ1 > µ1 > λ2 > µ2 > · · · · · · > λk > µk > λk+1.
This property leads to a convenient procedure .Proof
p1(λ) has a single root, d1.
p2(d1) = −e22 < 0,
![Page 284: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/284.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 284,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Sturmian sequence property of P(λ) with ej 6= 0:
Interlacing property: Roots of pk+1(λ) interlace theroots of pk(λ). That is, if the roots of pk+1(λ) areλ1 > λ2 > · · · > λk+1 and those of pk(λ) areµ1 > µ2 > · · · > µk ; then
λ1 > µ1 > λ2 > µ2 > · · · · · · > λk > µk > λk+1.
This property leads to a convenient procedure .Proof
p1(λ) has a single root, d1.
p2(d1) = −e22 < 0,
Since p2(±∞) =∞ > 0, roots t1 and t2 of p2(λ) are separated as∞ > t1 > d1 > t2 > −∞.
![Page 285: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/285.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 285,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Sturmian sequence property of P(λ) with ej 6= 0:
Interlacing property: Roots of pk+1(λ) interlace theroots of pk(λ). That is, if the roots of pk+1(λ) areλ1 > λ2 > · · · > λk+1 and those of pk(λ) areµ1 > µ2 > · · · > µk ; then
λ1 > µ1 > λ2 > µ2 > · · · · · · > λk > µk > λk+1.
This property leads to a convenient procedure .Proof
p1(λ) has a single root, d1.
p2(d1) = −e22 < 0,
Since p2(±∞) =∞ > 0, roots t1 and t2 of p2(λ) are separated as∞ > t1 > d1 > t2 > −∞.
The statement is true for k = 1.
![Page 286: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/286.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 286,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Next, we assume that the statement is true for k = i .Roots of pi(λ): α1 > α2 > · · · > αi
Roots of pi+1(λ): β1 > β2 > · · · > βi > βi+1
Roots of pi+2(λ): γ1 > γ2 > · · · > γi > γi+1 > γi+2
![Page 287: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/287.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 287,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Next, we assume that the statement is true for k = i .Roots of pi(λ): α1 > α2 > · · · > αi
Roots of pi+1(λ): β1 > β2 > · · · > βi > βi+1
Roots of pi+2(λ): γ1 > γ2 > · · · > γi > γi+1 > γi+2
Assumption: β1 > α1 > β2 > α2 > · · · · · · > βi > αi > βi+1
p ( )λi
pi+1
λ
Ο Οα α α
β ββ
β β
α α
i α i−1 1
α jj+1 j−1
i+1
2
iβ
2 1
j+1 j
ve ve
(a) Roots of and ( )
Sign of ip(b) p
i+2
γ γ i+2 1
8
8
Figure: Interlacing of roots of characteristic polynomials
To show: γ1 > β1 > γ2 > β2 > · · · · · · > γi+1 > βi+1 > γi+2
![Page 288: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/288.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 288,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Since β1 > α1, pi(β1) is of the same sign as pi(∞), i.e. positive.
![Page 289: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/289.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 289,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Since β1 > α1, pi(β1) is of the same sign as pi(∞), i.e. positive.
Therefore, pi+2(β1) = −e2i+2pi(β1) is negative.
But, pi+2(∞) is clearly positive.
![Page 290: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/290.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 290,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Since β1 > α1, pi(β1) is of the same sign as pi(∞), i.e. positive.
Therefore, pi+2(β1) = −e2i+2pi(β1) is negative.
But, pi+2(∞) is clearly positive.
Hence, γ1 ∈ (β1,∞).
![Page 291: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/291.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 291,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Since β1 > α1, pi(β1) is of the same sign as pi(∞), i.e. positive.
Therefore, pi+2(β1) = −e2i+2pi(β1) is negative.
But, pi+2(∞) is clearly positive.
Hence, γ1 ∈ (β1,∞).Similarly, γi+2 ∈ (−∞, βi+1).
Question: Where are the rest of the i roots of pi+2(λ)?
![Page 292: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/292.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 292,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Since β1 > α1, pi(β1) is of the same sign as pi(∞), i.e. positive.
Therefore, pi+2(β1) = −e2i+2pi(β1) is negative.
But, pi+2(∞) is clearly positive.
Hence, γ1 ∈ (β1,∞).Similarly, γi+2 ∈ (−∞, βi+1).
Question: Where are the rest of the i roots of pi+2(λ)?
pi+2(βj) = (βj − di+2)pi+1(βj )− e2i+2pi(βj ) = −e2
i+2pi (βj )
pi+2(βj+1) = −e2i+2pi (βj+1)
That is, pi and pi+2 are of opposite signs at each β.Refer figure.
Over [βi+1, β1], pi+2(λ) changes sign over each sub-interval[βj+1, βj ], along with pi (λ), to maintain opposite signs at each β.
Conclusion: pi+2(λ) has exactly one root in (βj+1, βj ).
![Page 293: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/293.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 293,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Examine sequence P(w) = p0(w), p1(w), p2(w), · · · , pn(w).If pk(w) and pk+1(w) have opposite signs then pk+1(λ) has oneroot more than pk(λ) in the interval (w ,∞).
![Page 294: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/294.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 294,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Examine sequence P(w) = p0(w), p1(w), p2(w), · · · , pn(w).If pk(w) and pk+1(w) have opposite signs then pk+1(λ) has oneroot more than pk(λ) in the interval (w ,∞).
Number of roots of pn(λ) above w = number of signchanges in the sequence P(w).
Consequence: Number of roots of pn(λ) in (a, b) = differencebetween numbers of sign changes in P(a) and P(b).
![Page 295: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/295.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 295,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Examine sequence P(w) = p0(w), p1(w), p2(w), · · · , pn(w).If pk(w) and pk+1(w) have opposite signs then pk+1(λ) has oneroot more than pk(λ) in the interval (w ,∞).
Number of roots of pn(λ) above w = number of signchanges in the sequence P(w).
Consequence: Number of roots of pn(λ) in (a, b) = differencebetween numbers of sign changes in P(a) and P(b).
Bisection method: Examine the sequence at a+b2 .
Separate roots, bracket each of them and then squeezethe interval!
![Page 296: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/296.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 296,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Examine sequence P(w) = p0(w), p1(w), p2(w), · · · , pn(w).If pk(w) and pk+1(w) have opposite signs then pk+1(λ) has oneroot more than pk(λ) in the interval (w ,∞).
Number of roots of pn(λ) above w = number of signchanges in the sequence P(w).
Consequence: Number of roots of pn(λ) in (a, b) = differencebetween numbers of sign changes in P(a) and P(b).
Bisection method: Examine the sequence at a+b2 .
Separate roots, bracket each of them and then squeezethe interval!
Any way to start with an interval to include all eigenvalues?
![Page 297: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/297.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 297,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Examine sequence P(w) = p0(w), p1(w), p2(w), · · · , pn(w).If pk(w) and pk+1(w) have opposite signs then pk+1(λ) has oneroot more than pk(λ) in the interval (w ,∞).
Number of roots of pn(λ) above w = number of signchanges in the sequence P(w).
Consequence: Number of roots of pn(λ) in (a, b) = differencebetween numbers of sign changes in P(a) and P(b).
Bisection method: Examine the sequence at a+b2 .
Separate roots, bracket each of them and then squeezethe interval!
Any way to start with an interval to include all eigenvalues?
|λi | ≤ λbnd = max1≤j≤n
|ej |+ |dj |+ |ej+1|
![Page 298: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/298.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 298,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Eigenvalues of Symmetric Tridiagonal Matrices
Algorithm
Identify the interval [a, b] of interest.
For a degenerate case (some ej = 0), split the given matrix.
For each of the non-degenerate matrices, by repeated use of bisection and study of the sequence P(λ),
bracket individual eigenvalues within small sub-intervals, and by further use of the bisection method (or a substitute) within
each such sub-interval, determine the individual eigenvalues tothe desired accuracy.
Note: The algorithm is based on Sturmian sequence property .
![Page 299: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/299.jpg)
Mathematical Methods in Engineering and Science Householder Transformation and Tridiagonal Matrices 299,
Householder Reflection TransformationHouseholder MethodEigenvalues of Symmetric Tridiagonal Matrices
Points to note
A Householder matrix is symmetric and orthogonal. It effectsa reflection transformation.
A sequence of Householder transformations can be used toconvert a symmetric matrix into a symmetric tridiagonal form.
Eigenvalues of the leading square sub-matrices of a symmetrictridiagonal matrix exhibit a useful interlacing structure.
This property can be used to separate and bracket eigenvalues.
Method of bisection is useful in the separation as well assubsequent determination of the eigenvalues.
Necessary Exercises: 2,4,5
![Page 300: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/300.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 300,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
Outline
QR Decomposition MethodQR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
![Page 301: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/301.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 301,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Decomposition (or factorization) A = QR into two factors,orthogonal Q and upper-triangular R:
(a) It always exists.(b) Performing this decomposition is pretty straightforward.(c) It has a number of properties useful in the solution of the
eigenvalue problem.
![Page 302: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/302.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 302,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Decomposition (or factorization) A = QR into two factors,orthogonal Q and upper-triangular R:
(a) It always exists.(b) Performing this decomposition is pretty straightforward.(c) It has a number of properties useful in the solution of the
eigenvalue problem.
[a1 · · · an] = [q1 · · · qn]
r11 · · · r1n. . .
...rnn
![Page 303: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/303.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 303,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Decomposition (or factorization) A = QR into two factors,orthogonal Q and upper-triangular R:
(a) It always exists.(b) Performing this decomposition is pretty straightforward.(c) It has a number of properties useful in the solution of the
eigenvalue problem.
[a1 · · · an] = [q1 · · · qn]
r11 · · · r1n. . .
...rnn
A simple method based on Gram-Schmidt orthogonalization:Considering columnwise equality aj =
∑ji=1 rijqi ,
for j = 1, 2, 3, · · · , n;
rij = qTi aj ∀i < j , a′j = aj −
j−1∑
i=1
rijqi , rjj = ‖a′j‖;
qj =
a′j/rjj , if rjj 6= 0;
any vector satisfying qTi qj = δij for 1 ≤ i ≤ j , if rjj = 0.
![Page 304: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/304.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 304,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Practical method: one-sided Householder transformations,starting with
u0 = a1, v0 = ‖u0‖e1 ∈ Rn and w0 =u0 − v0
‖u0 − v0‖
and P0 = Hn = In − 2w0wT0 .
![Page 305: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/305.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 305,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Practical method: one-sided Householder transformations,starting with
u0 = a1, v0 = ‖u0‖e1 ∈ Rn and w0 =u0 − v0
‖u0 − v0‖
and P0 = Hn = In − 2w0wT0 .
Pn−2Pn−3 · · ·P2P1P0A = Pn−2Pn−3 · · ·P2P1
[
‖a1‖ ∗∗0 A0
]
= Pn−2Pn−3 · · ·P2
r11 ∗ ∗∗r22 ∗∗
A1
= · · · · · · = R
With
Q = (Pn−2Pn−3 · · ·P2P1P0)T = P0P1P2 · · ·Pn−3Pn−2,
we have QTA = R ⇒ A = QR.
![Page 306: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/306.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 306,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Alternative method useful for tridiagonal and Hessenbergmatrices: One-sided plane rotations
rotations P12, P23 etc to annihilate a21, a32 etc in thatsequence
Givens rotation matrices!
![Page 307: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/307.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 307,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Decomposition
Alternative method useful for tridiagonal and Hessenbergmatrices: One-sided plane rotations
rotations P12, P23 etc to annihilate a21, a32 etc in thatsequence
Givens rotation matrices!
Application in solution of a linear system: Q and R factors ofa matrix A come handy in the solution of Ax = b
QRx = b ⇒ Rx = QTb
needs only a sequence of back-substitutions.
![Page 308: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/308.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 308,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Iterations
Multiplying Q and R factors in reverse,
A′ = RQ = QTAQ,
an orthogonal similarity transformation.
1. If A is symmetric, then so is A′.
2. If A is in upper Hessenberg form, then so is A′.
3. If A is symmetric tridiagonal, then so is A′.
![Page 309: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/309.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 309,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Iterations
Multiplying Q and R factors in reverse,
A′ = RQ = QTAQ,
an orthogonal similarity transformation.
1. If A is symmetric, then so is A′.
2. If A is in upper Hessenberg form, then so is A′.
3. If A is symmetric tridiagonal, then so is A′.
Complexity of QR iteration: O(n) for a symmetric tridiagonalmatrix, O(n2) operation for an upper Hessenberg matrix andO(n3) for the general case.
![Page 310: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/310.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 310,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Iterations
Multiplying Q and R factors in reverse,
A′ = RQ = QTAQ,
an orthogonal similarity transformation.
1. If A is symmetric, then so is A′.
2. If A is in upper Hessenberg form, then so is A′.
3. If A is symmetric tridiagonal, then so is A′.
Complexity of QR iteration: O(n) for a symmetric tridiagonalmatrix, O(n2) operation for an upper Hessenberg matrix andO(n3) for the general case.
Algorithm: Set A1 = A and for k = 1, 2, 3, · · · , decompose Ak = QkRk ,
reassemble Ak+1 = RkQk .
As k →∞, Ak approaches the quasi-upper-triangular form.
![Page 311: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/311.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 311,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Iterations
Quasi-upper-triangular form:
λ1 ∗ · · · ∗ ⋆⋆ · · · ∗ ∗λ2 · · · ∗ ⋆⋆ · · · ∗ ∗
. . . ∗ ⋆⋆ · · · ∗ ∗λr ⋆⋆ · · · ∗ ∗
Bk · · · ∗ ∗. . .
......
[
α −ωω β
]
,
with |λ1| > |λ2| > · · · . Diagonal blocks Bk correspond to eigenspaces of equal/close
(magnitude) eigenvalues. 2× 2 diagonal blocks often correspond to pairs of complex
eigenvalues (for non-symmetric matrices). For symmetric matrices, the quasi-upper-triangular form
reduces to quasi-diagonal form.
![Page 312: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/312.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 312,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
Conceptual Basis of QR Method*
QR decomposition algorithm operates on the basis of the relativemagnitudes of eigenvalues and segregates subspaces.
![Page 313: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/313.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 313,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
Conceptual Basis of QR Method*
QR decomposition algorithm operates on the basis of the relativemagnitudes of eigenvalues and segregates subspaces.
With k →∞,
AkRangee1 = Rangeq1 → Rangev1
and (a1)k → QTk Aq1 = λ1QT
k q1 = λ1e1.
![Page 314: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/314.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 314,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
Conceptual Basis of QR Method*
QR decomposition algorithm operates on the basis of the relativemagnitudes of eigenvalues and segregates subspaces.
With k →∞,
AkRangee1 = Rangeq1 → Rangev1
and (a1)k → QTk Aq1 = λ1QT
k q1 = λ1e1.
Further,
AkRangee1, e2 = Rangeq1,q2 → Rangev1, v2.
and (a2)k → QTk Aq2 =
(λ1 − λ2)α1
λ2
0
.
And, so on ...
![Page 315: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/315.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 315,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Algorithm with Shift*
For λi < λj , entry aij decays through iterations as(
λi
λj
)k
.
![Page 316: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/316.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 316,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Algorithm with Shift*
For λi < λj , entry aij decays through iterations as(
λi
λj
)k
.
With shift,
Ak = Ak − µk I;
Ak = QkRk , Ak+1 = RkQk ;
Ak+1 = Ak+1 + µk I.
![Page 317: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/317.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 317,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Algorithm with Shift*
For λi < λj , entry aij decays through iterations as(
λi
λj
)k
.
With shift,
Ak = Ak − µk I;
Ak = QkRk , Ak+1 = RkQk ;
Ak+1 = Ak+1 + µk I.
Resulting transformation is
Ak+1 = RkQk + µk I = QTk AkQk + µk I
= QTk (Ak − µk I)Qk + µk I = QT
k AkQk .
![Page 318: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/318.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 318,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Algorithm with Shift*
For λi < λj , entry aij decays through iterations as(
λi
λj
)k
.
With shift,
Ak = Ak − µk I;
Ak = QkRk , Ak+1 = RkQk ;
Ak+1 = Ak+1 + µk I.
Resulting transformation is
Ak+1 = RkQk + µk I = QTk AkQk + µk I
= QTk (Ak − µk I)Qk + µk I = QT
k AkQk .
For the iteration,
convergence ratio = λi−µk
λj−µk.
![Page 319: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/319.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 319,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
QR Algorithm with Shift*
For λi < λj , entry aij decays through iterations as(
λi
λj
)k
.
With shift,
Ak = Ak − µk I;
Ak = QkRk , Ak+1 = RkQk ;
Ak+1 = Ak+1 + µk I.
Resulting transformation is
Ak+1 = RkQk + µk I = QTk AkQk + µk I
= QTk (Ak − µk I)Qk + µk I = QT
k AkQk .
For the iteration,
convergence ratio = λi−µk
λj−µk.
Question: How to find a suitable value for µk?
![Page 320: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/320.jpg)
Mathematical Methods in Engineering and Science QR Decomposition Method 320,
QR DecompositionQR IterationsConceptual Basis of QR Method*QR Algorithm with Shift*
Points to note
QR decomposition can be effected on any square matrix.
Practical methods of QR decomposition use Householdertransformations or Givens rotations.
A QR iteration effects a similarity transformation on a matrix,preserving symmetry, Hessenberg structure and also asymmetric tridiagonal form.
A sequence of QR iterations converge to an almostupper-triangular form.
Operations on symmetric tridiagonal and Hessenberg formsare computationally efficient.
QR iterations tend to order subspaces according to therelative magnitudes of eigenvalues.
Eigenvalue shifting is useful as an expediting strategy.
Necessary Exercises: 1,3
![Page 321: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/321.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 321,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Outline
Eigenvalue Problem of General MatricesIntroductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
![Page 322: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/322.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 322,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Introductory Remarks
A general (non-symmetric) matrix may not be diagonalizable.We attempt to triangularize it.
With real arithmetic, 2× 2 diagonal blocks are inevitable —signifying complex pair of eigenvalues.
Higher computational complexity, slow convergence and lackof numerical stability.
![Page 323: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/323.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 323,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Introductory Remarks
A general (non-symmetric) matrix may not be diagonalizable.We attempt to triangularize it.
With real arithmetic, 2× 2 diagonal blocks are inevitable —signifying complex pair of eigenvalues.
Higher computational complexity, slow convergence and lackof numerical stability.
A non-symmetric matrix is usually unbalanced and is prone tohigher round-off errors.
Balancing as a pre-processing step: multiplication of a row anddivision of the corresponding column with the same number,ensuring similarity.
![Page 324: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/324.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 324,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Introductory Remarks
A general (non-symmetric) matrix may not be diagonalizable.We attempt to triangularize it.
With real arithmetic, 2× 2 diagonal blocks are inevitable —signifying complex pair of eigenvalues.
Higher computational complexity, slow convergence and lackof numerical stability.
A non-symmetric matrix is usually unbalanced and is prone tohigher round-off errors.
Balancing as a pre-processing step: multiplication of a row anddivision of the corresponding column with the same number,ensuring similarity.
Note: A balanced matrix may get unbalanced again throughsimilarity transformations that are not orthogonal!
![Page 325: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/325.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 325,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Reduction to Hessenberg Form*
Methods to find appropriate similarity transformations
1. a full sweep of Givens rotations,
2. a sequence of n− 2 steps of Householder transformations, and
3. a cycle of coordinated Gaussian elimination.
![Page 326: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/326.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 326,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Reduction to Hessenberg Form*
Methods to find appropriate similarity transformations
1. a full sweep of Givens rotations,
2. a sequence of n− 2 steps of Householder transformations, and
3. a cycle of coordinated Gaussian elimination.
Method based on Gaussian elimination or elementarytransformations:
The pre-multiplying matrix corresponding to theelementary row transformation and the post-multiplyingmatrix corresponding to the matching columntransformation must be inverses of each other.
![Page 327: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/327.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 327,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Reduction to Hessenberg Form*
Methods to find appropriate similarity transformations
1. a full sweep of Givens rotations,
2. a sequence of n− 2 steps of Householder transformations, and
3. a cycle of coordinated Gaussian elimination.
Method based on Gaussian elimination or elementarytransformations:
The pre-multiplying matrix corresponding to theelementary row transformation and the post-multiplyingmatrix corresponding to the matching columntransformation must be inverses of each other.
Two kinds of steps
Pivoting
Elimination
![Page 328: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/328.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 328,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Reduction to Hessenberg Form*
Pivoting step: A = PrsAPrs = P−1rs APrs .
Permutation Prs : interchange of r -th and s-th columns.
P−1rs = Prs : interchange of r -th and s-th rows.
Pivot locations: a21, a32, · · · , an−1,n−2.
![Page 329: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/329.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 329,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Reduction to Hessenberg Form*
Pivoting step: A = PrsAPrs = P−1rs APrs .
Permutation Prs : interchange of r -th and s-th columns.
P−1rs = Prs : interchange of r -th and s-th rows.
Pivot locations: a21, a32, · · · , an−1,n−2.
Elimination step: A = G−1r AGr with elimination matrix
Gr =
Ir 0 00 1 00 k In−r−1
and G−1r =
Ir 0 00 1 00 −k In−r−1
.
G−1r : Row (r + 1 + i)← Row (r + 1 + i)− ki× Row (r + 1)
for i = 1, 2, 3, · · · , n − r − 1
Gr : Column (r + 1)← Column (r + 1)+∑n−r−1
i=1 [ki× Column (r + 1 + i) ]
![Page 330: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/330.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 330,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
QR Algorithm on Hessenberg Matrices*
QR iterations: O(n2) operations for upper Hessenberg form.
Whenever a sub-diagonal zero appears, the matrix is splitinto two smaller upper Hessenberg blocks, and they areprocessed separately, thereby reducing the cost drastically.
![Page 331: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/331.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 331,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
QR Algorithm on Hessenberg Matrices*
QR iterations: O(n2) operations for upper Hessenberg form.
Whenever a sub-diagonal zero appears, the matrix is splitinto two smaller upper Hessenberg blocks, and they areprocessed separately, thereby reducing the cost drastically.
Particular cases:
an,n−1 → 0: Accept ann = λn as an eigenvalue, continue withthe leading (n − 1)× (n − 1) sub-matrix.
an−1,n−2 → 0: Separately find the eigenvalues λn−1 and λn
from
[
an−1,n−1 an−1,n
an,n−1 an,n
]
, continue with the leading
(n − 2)× (n − 2) sub-matrix.
![Page 332: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/332.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 332,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
QR Algorithm on Hessenberg Matrices*
QR iterations: O(n2) operations for upper Hessenberg form.
Whenever a sub-diagonal zero appears, the matrix is splitinto two smaller upper Hessenberg blocks, and they areprocessed separately, thereby reducing the cost drastically.
Particular cases:
an,n−1 → 0: Accept ann = λn as an eigenvalue, continue withthe leading (n − 1)× (n − 1) sub-matrix.
an−1,n−2 → 0: Separately find the eigenvalues λn−1 and λn
from
[
an−1,n−1 an−1,n
an,n−1 an,n
]
, continue with the leading
(n − 2)× (n − 2) sub-matrix.
Shift strategy: Double QR steps.
![Page 333: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/333.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 333,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Assumption: Matrix A has a complete set of eigenvectors.
(λi )0: a good estimate of an eigenvalue λi of A.
![Page 334: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/334.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 334,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Assumption: Matrix A has a complete set of eigenvectors.
(λi )0: a good estimate of an eigenvalue λi of A.
Purpose: To find λi precisely and also to find vi .
![Page 335: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/335.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 335,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Assumption: Matrix A has a complete set of eigenvectors.
(λi )0: a good estimate of an eigenvalue λi of A.
Purpose: To find λi precisely and also to find vi .
Step: Select a random vector y0 (with ‖y0‖ = 1) and solve
[A − (λi )0I]y = y0.
![Page 336: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/336.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 336,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Assumption: Matrix A has a complete set of eigenvectors.
(λi )0: a good estimate of an eigenvalue λi of A.
Purpose: To find λi precisely and also to find vi .
Step: Select a random vector y0 (with ‖y0‖ = 1) and solve
[A − (λi )0I]y = y0.
Result: y is a good estimate of vi and
(λi )1 = (λi )0 +1
yT0 y
is an improvement in the estimate of the eigenvalue.
![Page 337: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/337.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 337,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Assumption: Matrix A has a complete set of eigenvectors.
(λi )0: a good estimate of an eigenvalue λi of A.
Purpose: To find λi precisely and also to find vi .
Step: Select a random vector y0 (with ‖y0‖ = 1) and solve
[A − (λi )0I]y = y0.
Result: y is a good estimate of vi and
(λi )1 = (λi )0 +1
yT0 y
is an improvement in the estimate of the eigenvalue.
How to establish the result and work out an algorithm ?
![Page 338: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/338.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 338,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
With y0 =∑n
j=1 αjvj and y =∑n
j=1 βjvj , [A− (λi)0I]y = y0 gives
n∑
j=1
βj [A − (λi )0I]vj =
n∑
j=1
αjvj
⇒ βj [λj − (λi)0] = αj ⇒ βj =αj
λj − (λi )0.
βi is typically large and eigenvector vi dominates y.
![Page 339: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/339.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 339,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
With y0 =∑n
j=1 αjvj and y =∑n
j=1 βjvj , [A− (λi)0I]y = y0 gives
n∑
j=1
βj [A − (λi )0I]vj =
n∑
j=1
αjvj
⇒ βj [λj − (λi)0] = αj ⇒ βj =αj
λj − (λi )0.
βi is typically large and eigenvector vi dominates y.
Avi = λivi gives [A − (λi )0I]vi = [λi − (λi )0]vi . Hence,
[λi − (λi )0]y ≈ [A − (λi )0I]y = y0.
Inner product with y0 gives
[λi − (λi )0]yT0 y ≈ 1 ⇒ λi ≈ (λi )0 +
1
yT0 y
.
![Page 340: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/340.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 340,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Algorithm:
Start with estimate (λi )0, guess y0 (normalized).For k = 0, 1, 2, · · ·
Solve [A − (λi)k I]y = yk .
Normalize yk+1 = y‖y‖ .
Improve (λi )k+1 = (λi )k + 1yTk
y.
If ‖yk+1 − yk‖ < ǫ, terminate.
![Page 341: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/341.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 341,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Inverse Iteration
Algorithm:
Start with estimate (λi )0, guess y0 (normalized).For k = 0, 1, 2, · · ·
Solve [A − (λi)k I]y = yk .
Normalize yk+1 = y‖y‖ .
Improve (λi )k+1 = (λi )k + 1yTk
y.
If ‖yk+1 − yk‖ < ǫ, terminate.
Important issues
Update eigenvalue once in a while, not at every iteration.
Use some acceptable small number as artificial pivot.
The method may not converge for defective matrix or for onehaving complex eigenvalues.
Repeated eigenvalues may inhibit the process.
![Page 342: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/342.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 342,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Recommendation
Table: Eigenvalue problem: summary of methods
Type Size Reduction Algorithm Post-processingGeneral Small Definition: Polynomial Solution of
(up to 4) Characteristic root finding linear systemspolynomial (eigenvalues) (eigenvectors)
Symmetric Intermediate Jacobi sweeps Selective(say, 4–12) Jacobi rotations
Tridiagonalization Sturm sequence Inverse iteration(Givens rotation property: (eigenvalueor Householder Bracketing and improvementmethod) bisection and eigenvectors)
(rough eigenvalues)Large Tridiagonalization QR decomposition
(usually iterationsHouseholder method)Balancing, and then
Non- Intermediate Reduction to QR decomposition Inverse iterationsymmetric Large Hessenberg form iterations (eigenvectors)
(Above methods or (eigenvalues)Gaussian elimination)
General Very large Power method,(selective shift and deflationrequirement)
![Page 343: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/343.jpg)
Mathematical Methods in Engineering and Science Eigenvalue Problem of General Matrices 343,
Introductory RemarksReduction to Hessenberg Form*QR Algorithm on Hessenberg Matrices*Inverse IterationRecommendation
Points to note
Eigenvalue problem of a non-symmetric matrix is difficult!
Balancing and reduction to Hessenberg form are desirablepre-processing steps.
QR decomposition algorithm is typically used for reduction toan upper-triangular form.
Use inverse iteration to polish eigenvalue and findeigenvectors.
In algebraic eigenvalue problems, different methods orcombinations are suitable for different cases; regarding matrixsize, symmetry and the requirements.
Necessary Exercises: 1,2
![Page 344: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/344.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 344,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Outline
Singular Value DecompositionSVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
![Page 345: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/345.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 345,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionEigenvalue problem: A = UΛV−1 where U = V
Do not ask for similarity. Focus on the form of the decomposition.
Similar result for complex matrices
![Page 346: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/346.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 346,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionEigenvalue problem: A = UΛV−1 where U = V
Do not ask for similarity. Focus on the form of the decomposition.
Guaranteed decomposition with orthogonal U, V, andnon-negative diagonal entries in Λ — by allowing U 6= V.
A = UΣVT such that UTAV = Σ
Similar result for complex matrices
![Page 347: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/347.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 347,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionEigenvalue problem: A = UΛV−1 where U = V
Do not ask for similarity. Focus on the form of the decomposition.
Guaranteed decomposition with orthogonal U, V, andnon-negative diagonal entries in Λ — by allowing U 6= V.
A = UΣVT such that UTAV = Σ
SVD Theorem For any real matrix A ∈ Rm×n, thereexist orthogonal matrices U ∈ Rm×m and V ∈ Rn×n suchthat
UTAV = Σ ∈ Rm×n
is a diagonal matrix, with diagonal entries σ1, σ2, · · · ≥ 0,obtained by appending the square diagonal matrixdiag (σ1, σ2, · · · , σp) with (m − p) zero rows or (n − p)zero columns, where p = min(m, n).
Singular values: σ1, σ2, · · · , σp.Similar result for complex matrices
![Page 348: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/348.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 348,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionQuestion: How to construct U, V and Σ?
This provides a proof as well!
![Page 349: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/349.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 349,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionQuestion: How to construct U, V and Σ?
For A ∈ Rm×n,
ATA = (VΣTUT )(UΣVT ) = VΣTΣVT = VΛVT ,
where Λ = ΣTΣ is an n × n diagonal matrix.
Σ =
σ1 |σ2 |
. . . | 0σp |
−− −− −− −− −+− −−0 | ×
This provides a proof as well!
![Page 350: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/350.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 350,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and ConstructionQuestion: How to construct U, V and Σ?
For A ∈ Rm×n,
ATA = (VΣTUT )(UΣVT ) = VΣTΣVT = VΛVT ,
where Λ = ΣTΣ is an n × n diagonal matrix.
Σ =
σ1 |σ2 |
. . . | 0σp |
−− −− −− −− −+− −−0 | ×
Determine V and Λ. Work out Σ and we have
A = UΣVT ⇒ AV = UΣ
This provides a proof as well!
![Page 351: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/351.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 351,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and Construction
From AV = UΣ, determine columns of U.
1. Column Avk = σkuk , with σk 6= 0: determine column uk .
Columns developed are bound to be mutuallyorthonormal!
Verify uTi uj =
(
1σi
Avi
)T (
1σj
Avj
)
= δij .
2. Column Avk = σkuk , with σk = 0: uk is left indeterminate(free).
3. In the case of m < n, identically zero columns Avk = 0 fork > m: no corresponding columns of U to determine.
4. In the case of m > n, there will be (m − n) columns of U leftindeterminate.
Extend columns of U to an orthonormal basis.
![Page 352: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/352.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 352,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
SVD Theorem and Construction
From AV = UΣ, determine columns of U.
1. Column Avk = σkuk , with σk 6= 0: determine column uk .
Columns developed are bound to be mutuallyorthonormal!
Verify uTi uj =
(
1σi
Avi
)T (
1σj
Avj
)
= δij .
2. Column Avk = σkuk , with σk = 0: uk is left indeterminate(free).
3. In the case of m < n, identically zero columns Avk = 0 fork > m: no corresponding columns of U to determine.
4. In the case of m > n, there will be (m − n) columns of U leftindeterminate.
Extend columns of U to an orthonormal basis.
All three factors in the decomposition are constructed, as desired.
![Page 353: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/353.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 353,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
For a given matrix, the SVD is unique up to
(a) the same permutations of columns of U, columns of V anddiagonal elements of Σ;
(b) the same orthonormal linear combinations among columns ofU and columns of V, corresponding to equal singular values;and
(c) arbitrary orthonormal linear combinations among columns ofU or columns of V, corresponding to zero or non-existentsingular values.
![Page 354: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/354.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 354,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
For a given matrix, the SVD is unique up to
(a) the same permutations of columns of U, columns of V anddiagonal elements of Σ;
(b) the same orthonormal linear combinations among columns ofU and columns of V, corresponding to equal singular values;and
(c) arbitrary orthonormal linear combinations among columns ofU or columns of V, corresponding to zero or non-existentsingular values.
Ordering of the singular values:
σ1 ≥ σ2 ≥ · · · ≥ σr > 0, and σr+1 = σr+2 = · · · = σp = 0.
![Page 355: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/355.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 355,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
For a given matrix, the SVD is unique up to
(a) the same permutations of columns of U, columns of V anddiagonal elements of Σ;
(b) the same orthonormal linear combinations among columns ofU and columns of V, corresponding to equal singular values;and
(c) arbitrary orthonormal linear combinations among columns ofU or columns of V, corresponding to zero or non-existentsingular values.
Ordering of the singular values:
σ1 ≥ σ2 ≥ · · · ≥ σr > 0, and σr+1 = σr+2 = · · · = σp = 0.
Rank(A) = Rank(Σ) = r
Rank of a matrix is the same as the number of itsnon-zero singular values.
![Page 356: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/356.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 356,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
Ax = UΣVT x = UΣy = [u1 · · · ur ur+1 · · · um]
σ1y1...
σryr
0
= σ1y1u1 + σ2y2u2 + · · ·+ σryrur
has non-zero components along only the first r columns of U.
U gives an orthonormal basis for the co-domain such that
Range(A) = < u1,u2, · · · ,ur > .
![Page 357: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/357.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 357,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
Ax = UΣVT x = UΣy = [u1 · · · ur ur+1 · · · um]
σ1y1...
σryr
0
= σ1y1u1 + σ2y2u2 + · · ·+ σryrur
has non-zero components along only the first r columns of U.
U gives an orthonormal basis for the co-domain such that
Range(A) = < u1,u2, · · · ,ur > .
With VTx = y, vTk x = yk , and
x = y1v1 + y2v2 + · · ·+ yrvr + yr+1vr+1 + · · · ynvn.
V gives an orthonormal basis for the domain such that
Null(A) = < vr+1, vr+2, · · · , vn > .
![Page 358: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/358.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 358,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
In basis V, v = c1v1 + c2v2 + · · · + cnvn = Vc and the norm isgiven by
‖A‖2 = maxv
‖Av‖2‖v‖2 = max
v
vTATAv
vT v
= maxc
cTVTATAVc
cTVTVc= max
c
cT ΣTΣc
cTc= max
c
∑
k σ2kc2
k∑
k c2k
.
![Page 359: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/359.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 359,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
In basis V, v = c1v1 + c2v2 + · · · + cnvn = Vc and the norm isgiven by
‖A‖2 = maxv
‖Av‖2‖v‖2 = max
v
vTATAv
vT v
= maxc
cTVTATAVc
cTVTVc= max
c
cT ΣTΣc
cTc= max
c
∑
k σ2kc2
k∑
k c2k
.
‖A‖ =
√
maxc
P
k σ2kc2k
P
k c2k
= σmax
![Page 360: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/360.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 360,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
In basis V, v = c1v1 + c2v2 + · · · + cnvn = Vc and the norm isgiven by
‖A‖2 = maxv
‖Av‖2‖v‖2 = max
v
vTATAv
vT v
= maxc
cTVTATAVc
cTVTVc= max
c
cT ΣTΣc
cTc= max
c
∑
k σ2kc2
k∑
k c2k
.
‖A‖ =
√
maxc
P
k σ2kc2k
P
k c2k
= σmax
For a non-singular square matrix,
A−1 = (UΣVT )−1 = VΣ−1UT = V diag
(
1
σ1,
1
σ2, · · · , 1
σn
)
UT .
![Page 361: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/361.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 361,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
In basis V, v = c1v1 + c2v2 + · · · + cnvn = Vc and the norm isgiven by
‖A‖2 = maxv
‖Av‖2‖v‖2 = max
v
vTATAv
vT v
= maxc
cTVTATAVc
cTVTVc= max
c
cT ΣTΣc
cTc= max
c
∑
k σ2kc2
k∑
k c2k
.
‖A‖ =
√
maxc
P
k σ2kc2k
P
k c2k
= σmax
For a non-singular square matrix,
A−1 = (UΣVT )−1 = VΣ−1UT = V diag
(
1
σ1,
1
σ2, · · · , 1
σn
)
UT .
Then, ‖A−1‖ = 1σmin
and the condition number is
κ(A) = ‖A‖ ‖A−1‖ =σmax
σmin
.
![Page 362: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/362.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 362,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
Revision of definition of norm and condition number:
The norm of a matrix is the same as its largest singularvalue, while its condition number is given by the ratio ofthe largest singular value to the least.
![Page 363: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/363.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 363,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
Revision of definition of norm and condition number:
The norm of a matrix is the same as its largest singularvalue, while its condition number is given by the ratio ofthe largest singular value to the least.
Arranging singular values in decreasing order, with Rank(A) = r ,
U = [Ur U] and V = [Vr V],
A = UΣVT = [Ur U]
[
Σr 00 0
] [
VTr
VT
]
,
or,
A = UrΣrVTr =
r∑
k=1
σkukvTk .
![Page 364: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/364.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 364,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Properties of SVD
Revision of definition of norm and condition number:
The norm of a matrix is the same as its largest singularvalue, while its condition number is given by the ratio ofthe largest singular value to the least.
Arranging singular values in decreasing order, with Rank(A) = r ,
U = [Ur U] and V = [Vr V],
A = UΣVT = [Ur U]
[
Σr 00 0
] [
VTr
VT
]
,
or,
A = UrΣrVTr =
r∑
k=1
σkukvTk .
Efficient storage and reconstruction!
![Page 365: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/365.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 365,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Generalized inverse: G is called a generalized inverse or g-inverseof A if, for b ∈ Range(A), Gb is a solution of Ax = b.
![Page 366: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/366.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 366,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Generalized inverse: G is called a generalized inverse or g-inverseof A if, for b ∈ Range(A), Gb is a solution of Ax = b.
The Moore-Penrose inverse or the pseudoinverse:
A# = (UΣVT )# = (VT )#Σ#U# = VΣ#UT
![Page 367: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/367.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 367,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Generalized inverse: G is called a generalized inverse or g-inverseof A if, for b ∈ Range(A), Gb is a solution of Ax = b.
The Moore-Penrose inverse or the pseudoinverse:
A# = (UΣVT )# = (VT )#Σ#U# = VΣ#UT
With Σ =
[
Σr 00 0
]
, Σ# =
[
Σ−1r 00 0
]
.
![Page 368: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/368.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 368,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Generalized inverse: G is called a generalized inverse or g-inverseof A if, for b ∈ Range(A), Gb is a solution of Ax = b.
The Moore-Penrose inverse or the pseudoinverse:
A# = (UΣVT )# = (VT )#Σ#U# = VΣ#UT
With Σ =
[
Σr 00 0
]
, Σ# =
[
Σ−1r 00 0
]
.
Or, Σ# =
ρ1 |ρ2 |
. . . | 0ρp |
−− −− −− −− −+− −−0 | ×
,
where ρk =
1σk, for σk 6= 0 or for |σk | > ǫ;
0, for σk = 0 or for |σk | ≤ ǫ.
![Page 369: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/369.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 369,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Inverse-like facets and beyond
(A#)# = A.
If A is invertible, then A# = A−1. A#b gives the correct unique solution.
If Ax = b is an under-determined consistent system, thenA#b selects the solution x∗ with the minimum norm.
If the system is inconsistent, then A#b minimizes the leastsquare error ‖Ax − b‖.
If the minimizer of ‖Ax − b‖ is not unique, then it picks upthat minimizer which has the minimum norm ‖x‖ among suchminimizers.
![Page 370: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/370.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 370,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Pseudoinverse and Solution of Linear Systems
Inverse-like facets and beyond
(A#)# = A.
If A is invertible, then A# = A−1. A#b gives the correct unique solution.
If Ax = b is an under-determined consistent system, thenA#b selects the solution x∗ with the minimum norm.
If the system is inconsistent, then A#b minimizes the leastsquare error ‖Ax − b‖.
If the minimizer of ‖Ax − b‖ is not unique, then it picks upthat minimizer which has the minimum norm ‖x‖ among suchminimizers.
Contrast with Tikhonov regularization:
Pseudoinverse solution for precision and diagnosis.Tikhonov’s solution for continuity of solution overvariable A and computational efficiency.
![Page 371: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/371.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 371,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Pseudoinverse solution of Ax = b:
x∗ = VΣ#UTb =r∑
k=1
ρkvkuTk b =
r∑
k=1
(uTk b/σk)vk
![Page 372: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/372.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 372,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Pseudoinverse solution of Ax = b:
x∗ = VΣ#UTb =r∑
k=1
ρkvkuTk b =
r∑
k=1
(uTk b/σk)vk
Minimize
E (x) =1
2(Ax − b)T (Ax − b) =
1
2xTATAx − xTATb +
1
2bTb
![Page 373: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/373.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 373,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Pseudoinverse solution of Ax = b:
x∗ = VΣ#UTb =r∑
k=1
ρkvkuTk b =
r∑
k=1
(uTk b/σk)vk
Minimize
E (x) =1
2(Ax − b)T (Ax − b) =
1
2xTATAx − xTATb +
1
2bTb
Condition of vanishing gradient:
∂E
∂x= 0 ⇒ ATAx = ATb
⇒ V(ΣTΣ)VT x = VΣTUTb
⇒ (ΣT Σ)VTx = ΣTUTb
⇒ σ2kv
Tk x = σku
Tk b
⇒ vTk x = uT
k b/σk for k = 1, 2, 3, · · · , r .
![Page 374: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/374.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 374,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
With V = [vr+1 vr+2 · · · vn], then
x =r∑
k=1
(uTk b/σk)vk + Vy = x∗ + Vy.
![Page 375: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/375.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 375,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
With V = [vr+1 vr+2 · · · vn], then
x =r∑
k=1
(uTk b/σk)vk + Vy = x∗ + Vy.
How to minimize ‖x‖2 subject to E (x) minimum?
![Page 376: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/376.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 376,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
With V = [vr+1 vr+2 · · · vn], then
x =r∑
k=1
(uTk b/σk)vk + Vy = x∗ + Vy.
How to minimize ‖x‖2 subject to E (x) minimum?
Minimize E1(y) = ‖x∗ + Vy‖2.
![Page 377: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/377.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 377,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
With V = [vr+1 vr+2 · · · vn], then
x =r∑
k=1
(uTk b/σk)vk + Vy = x∗ + Vy.
How to minimize ‖x‖2 subject to E (x) minimum?
Minimize E1(y) = ‖x∗ + Vy‖2.
Since x∗ and Vy are mutually orthogonal,
E1(y) = ‖x∗ + Vy‖2 = ‖x∗‖2 + ‖Vy‖2
is minimum when Vy = 0, i.e. y = 0.
![Page 378: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/378.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 378,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Anatomy of the optimization through SVDUsing basis V for domain and U for co-domain, the variables aretransformed as
VT x = y and UTb = c.
![Page 379: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/379.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 379,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Anatomy of the optimization through SVDUsing basis V for domain and U for co-domain, the variables aretransformed as
VT x = y and UTb = c.
Then,
Ax = b ⇒ UΣVT x = b ⇒ ΣVTx = UTb ⇒ Σy = c.
A completely decoupled system!
![Page 380: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/380.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 380,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Anatomy of the optimization through SVDUsing basis V for domain and U for co-domain, the variables aretransformed as
VT x = y and UTb = c.
Then,
Ax = b ⇒ UΣVT x = b ⇒ ΣVTx = UTb ⇒ Σy = c.
A completely decoupled system!Usable components: yk = ck/σk for k = 1, 2, 3, · · · , r .For k > r ,
completely redundant information (ck = 0)
purely unresolvable conflict (ck 6= 0)
![Page 381: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/381.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 381,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Optimality of Pseudoinverse Solution
Anatomy of the optimization through SVDUsing basis V for domain and U for co-domain, the variables aretransformed as
VT x = y and UTb = c.
Then,
Ax = b ⇒ UΣVT x = b ⇒ ΣVTx = UTb ⇒ Σy = c.
A completely decoupled system!Usable components: yk = ck/σk for k = 1, 2, 3, · · · , r .For k > r ,
completely redundant information (ck = 0)
purely unresolvable conflict (ck 6= 0)
SVD extracts this pure redundancy/inconsistency.Setting ρk = 0 for k > r rejects it wholesale!At the same time, ‖y‖ is minimized, and hence ‖x‖ too.
![Page 382: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/382.jpg)
Mathematical Methods in Engineering and Science Singular Value Decomposition 382,
SVD Theorem and ConstructionProperties of SVDPseudoinverse and Solution of Linear SystemsOptimality of Pseudoinverse SolutionSVD Algorithm
Points to note
SVD provides a complete orthogonal decomposition of thedomain and co-domain of a linear transformation, separatingout functionally distinct subspaces.
If offers a complete diagnosis of the pathologies of systems oflinear equations.
Pseudoinverse solution of linear systems satisfy meaningfuloptimality requirements in several contexts.
With the existence of SVD guaranteed, many importantresults can be established in a straightforward manner.
Necessary Exercises: 2,4,5,6,7
![Page 383: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/383.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 383,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Outline
Vector Spaces: Fundamental Concepts*GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
![Page 384: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/384.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 384,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Group
A set G and a binary operation, say ‘+’, fulfilling
Closure: a + b ∈ G ∀a, b ∈ G
Associativity: a + (b + c) = (a + b) + c , ∀a, b, c ∈ G
Existence of identity: ∃0 ∈ G such that ∀a ∈ G , a + 0 = a = 0 + a
Existence of inverse: ∀a ∈ G , ∃(−a) ∈ G such thata + (−a) = 0 = (−a) + a
Examples: (Z ,+), (R ,+), (Q − 0, ·), 2× 5 real matrices,Rotations etc.
![Page 385: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/385.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 385,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Group
A set G and a binary operation, say ‘+’, fulfilling
Closure: a + b ∈ G ∀a, b ∈ G
Associativity: a + (b + c) = (a + b) + c , ∀a, b, c ∈ G
Existence of identity: ∃0 ∈ G such that ∀a ∈ G , a + 0 = a = 0 + a
Existence of inverse: ∀a ∈ G , ∃(−a) ∈ G such thata + (−a) = 0 = (−a) + a
Examples: (Z ,+), (R ,+), (Q − 0, ·), 2× 5 real matrices,Rotations etc.
Commutative group
Examples:(Z ,+), (R ,+), (Q − 0, ·), (F ,+).
![Page 386: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/386.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 386,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Group
A set G and a binary operation, say ‘+’, fulfilling
Closure: a + b ∈ G ∀a, b ∈ G
Associativity: a + (b + c) = (a + b) + c , ∀a, b, c ∈ G
Existence of identity: ∃0 ∈ G such that ∀a ∈ G , a + 0 = a = 0 + a
Existence of inverse: ∀a ∈ G , ∃(−a) ∈ G such thata + (−a) = 0 = (−a) + a
Examples: (Z ,+), (R ,+), (Q − 0, ·), 2× 5 real matrices,Rotations etc.
Commutative group
Examples:(Z ,+), (R ,+), (Q − 0, ·), (F ,+).
Subgroup
![Page 387: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/387.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 387,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Field
A set F and two binary operations, say ‘+’ and ‘·’, satisfying
Group property for addition: (F ,+) is a commutative group.(Denote the identity element of this group as ‘0’.)
Group property for multiplication: (F − 0, ·) is a commutativegroup. (Denote the identity element of this group as‘1’.)
Distributivity: a · (b + c) = a · b + a · c , ∀a, b, c ∈ F .
Concept of field: abstraction of a number system
![Page 388: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/388.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 388,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Field
A set F and two binary operations, say ‘+’ and ‘·’, satisfying
Group property for addition: (F ,+) is a commutative group.(Denote the identity element of this group as ‘0’.)
Group property for multiplication: (F − 0, ·) is a commutativegroup. (Denote the identity element of this group as‘1’.)
Distributivity: a · (b + c) = a · b + a · c , ∀a, b, c ∈ F .
Concept of field: abstraction of a number system
Examples: (Q,+, ·), (R ,+, ·), (C ,+, ·) etc.
![Page 389: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/389.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 389,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Field
A set F and two binary operations, say ‘+’ and ‘·’, satisfying
Group property for addition: (F ,+) is a commutative group.(Denote the identity element of this group as ‘0’.)
Group property for multiplication: (F − 0, ·) is a commutativegroup. (Denote the identity element of this group as‘1’.)
Distributivity: a · (b + c) = a · b + a · c , ∀a, b, c ∈ F .
Concept of field: abstraction of a number system
Examples: (Q,+, ·), (R ,+, ·), (C ,+, ·) etc.
Subfield
![Page 390: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/390.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 390,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
A vector space is defined by
a field F of ‘scalars’,
a commutative group V of ‘vectors’, and
a binary operation between F and V, that may be called‘scalar multiplication’, such that ∀α, β ∈ F , ∀a,b ∈ V; thefollowing conditions hold.
Closure: αa ∈ V.Identity: 1a = a.Associativity: (αβ)a = α(βa).Scalar distributivity: α(a + b) = αa + αb.Vector distributivity: (α+ β)a = αa + βa.
Examples: Rn, Cn, m × n real matrices etc.
![Page 391: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/391.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 391,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
A vector space is defined by
a field F of ‘scalars’,
a commutative group V of ‘vectors’, and
a binary operation between F and V, that may be called‘scalar multiplication’, such that ∀α, β ∈ F , ∀a,b ∈ V; thefollowing conditions hold.
Closure: αa ∈ V.Identity: 1a = a.Associativity: (αβ)a = α(βa).Scalar distributivity: α(a + b) = αa + αb.Vector distributivity: (α+ β)a = αa + βa.
Examples: Rn, Cn, m × n real matrices etc.
Field↔ Number systemVector space ↔ Space
![Page 392: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/392.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 392,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
![Page 393: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/393.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 393,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
![Page 394: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/394.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 394,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
![Page 395: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/395.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 395,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
![Page 396: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/396.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 396,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
Question: Are the elements of V exhausted now?
![Page 397: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/397.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 397,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
Question: Are the elements of V exhausted now?· · · · · · · · ·
![Page 398: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/398.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 398,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
Question: Are the elements of V exhausted now?· · · · · · · · ·Question: Will this process ever end?
![Page 399: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/399.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 399,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
Question: Are the elements of V exhausted now?· · · · · · · · ·Question: Will this process ever end?
Suppose it does.
![Page 400: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/400.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 400,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Suppose V is a vector space.Take a vector ξ1 6= 0 in it.
Then, vectors linearly dependent on ξ1:α1ξ1 ∈ V ∀α1 ∈ F .
Question: Are the elements of V exhausted?
If not, then take ξ2 ∈ V: linearly independent from ξ1.
Then, α1ξ1 + α2ξ2 ∈ V ∀α1, α2 ∈ F .
Question: Are the elements of V exhausted now?· · · · · · · · ·Question: Will this process ever end?
Suppose it does.
finite dimensional vector space
![Page 401: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/401.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 401,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Finite dimensional vector space
Suppose the above process ends after n choices of linearlyindependent vectors.
χ = α1ξ1 + α2ξ2 + · · ·+ αnξn
![Page 402: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/402.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 402,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Finite dimensional vector space
Suppose the above process ends after n choices of linearlyindependent vectors.
χ = α1ξ1 + α2ξ2 + · · ·+ αnξn
Then,
n: dimension of the vector space
ordered set ξ1, ξ2, · · · , ξn: a basis
α1, α2, · · · , αn ∈ F : coordinates of χ in that basis
![Page 403: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/403.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 403,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Finite dimensional vector space
Suppose the above process ends after n choices of linearlyindependent vectors.
χ = α1ξ1 + α2ξ2 + · · ·+ αnξn
Then,
n: dimension of the vector space
ordered set ξ1, ξ2, · · · , ξn: a basis
α1, α2, · · · , αn ∈ F : coordinates of χ in that basis
Rn, Rm etc: vector spaces over the field of real numbers
![Page 404: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/404.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 404,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Vector Space
Finite dimensional vector space
Suppose the above process ends after n choices of linearlyindependent vectors.
χ = α1ξ1 + α2ξ2 + · · ·+ αnξn
Then,
n: dimension of the vector space
ordered set ξ1, ξ2, · · · , ξn: a basis
α1, α2, · · · , αn ∈ F : coordinates of χ in that basis
Rn, Rm etc: vector spaces over the field of real numbers
Subspace
![Page 405: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/405.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 405,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A mapping T : V →W satisfying
T(αa + βb) = αT(a) + βT(b) ∀α, β ∈ F and ∀a,b ∈ V
where V and W are vector spaces over the field F .
![Page 406: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/406.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 406,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A mapping T : V →W satisfying
T(αa + βb) = αT(a) + βT(b) ∀α, β ∈ F and ∀a,b ∈ V
where V and W are vector spaces over the field F .
Question: How to describe the linear transformation T?
![Page 407: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/407.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 407,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A mapping T : V →W satisfying
T(αa + βb) = αT(a) + βT(b) ∀α, β ∈ F and ∀a,b ∈ V
where V and W are vector spaces over the field F .
Question: How to describe the linear transformation T?
For V, basis ξ1, ξ2, · · · , ξn For W, basis η1, η2, · · · , ηm
ξ1 ∈ V gets mapped to T(ξ1) ∈W.
T(ξ1) = a11η1 + a21η2 + · · ·+ am1ηm
Similarly, enumerate T(ξj) =∑m
i=1 aijηi .
![Page 408: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/408.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 408,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A mapping T : V →W satisfying
T(αa + βb) = αT(a) + βT(b) ∀α, β ∈ F and ∀a,b ∈ V
where V and W are vector spaces over the field F .
Question: How to describe the linear transformation T?
For V, basis ξ1, ξ2, · · · , ξn For W, basis η1, η2, · · · , ηm
ξ1 ∈ V gets mapped to T(ξ1) ∈W.
T(ξ1) = a11η1 + a21η2 + · · ·+ am1ηm
Similarly, enumerate T(ξj) =∑m
i=1 aijηi .
Matrix A = [a1 a2 · · · an] codes this description!
![Page 409: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/409.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 409,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A general element χ of V can be expressed as
χ = x1ξ1 + x2ξ2 + · · · + xnξn
Coordinates in a column: x = [x1 x2 · · · xn]T
![Page 410: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/410.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 410,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A general element χ of V can be expressed as
χ = x1ξ1 + x2ξ2 + · · · + xnξn
Coordinates in a column: x = [x1 x2 · · · xn]T
Mapping:
T(χ) = x1T(ξ1) + x2T(ξ2) + · · · + xnT(ξn),
with coordinates Ax, as we know!
![Page 411: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/411.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 411,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
A general element χ of V can be expressed as
χ = x1ξ1 + x2ξ2 + · · · + xnξn
Coordinates in a column: x = [x1 x2 · · · xn]T
Mapping:
T(χ) = x1T(ξ1) + x2T(ξ2) + · · · + xnT(ξn),
with coordinates Ax, as we know!
Summary:
basis vectors of V get mapped to vectors in W whosecoordinates are listed in columns of A, and
a vector of V, having its coordinates in x, gets mapped to avector in W whose coordinates are obtained from Ax.
![Page 412: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/412.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 412,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
Understanding:
Vector χ is an actual object in the set V and the columnx ∈ Rn is merely a list of its coordinates.
T : V →W is the linear transformation and the matrix Asimply stores coefficients needed to describe it.
By changing bases of V and W, the same vector χ and thesame linear transformation are now expressed by different xand A, respectively.
![Page 413: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/413.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 413,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
Understanding:
Vector χ is an actual object in the set V and the columnx ∈ Rn is merely a list of its coordinates.
T : V →W is the linear transformation and the matrix Asimply stores coefficients needed to describe it.
By changing bases of V and W, the same vector χ and thesame linear transformation are now expressed by different xand A, respectively.
Matrix representation emerges as the natural descriptionof a linear transformation between two vector spaces.
![Page 414: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/414.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 414,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Linear Transformation
Understanding:
Vector χ is an actual object in the set V and the columnx ∈ Rn is merely a list of its coordinates.
T : V →W is the linear transformation and the matrix Asimply stores coefficients needed to describe it.
By changing bases of V and W, the same vector χ and thesame linear transformation are now expressed by different xand A, respectively.
Matrix representation emerges as the natural descriptionof a linear transformation between two vector spaces.
Exercise: Set of all T : V →W form a vector space of their own!!Analyze and describe that vector space.
![Page 415: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/415.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 415,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider T : V →W that establishes a one-to-one correspondence.
Linear transformation T defines a one-one onto mapping,which is invertible.
dimV = dimW
Inverse linear transformation T−1 : W → V
T defines (is) an isomorphism.
Vector spaces V and W are isomorphic to each other.
Isomorphism is an equivalence relation. V and W areequivalent!
![Page 416: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/416.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 416,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider T : V →W that establishes a one-to-one correspondence.
Linear transformation T defines a one-one onto mapping,which is invertible.
dimV = dimW
Inverse linear transformation T−1 : W → V
T defines (is) an isomorphism.
Vector spaces V and W are isomorphic to each other.
Isomorphism is an equivalence relation. V and W areequivalent!
If we need to perform some operations on vectors in one vectorspace, we may as well
1. transform the vectors to another vector space through anisomorphism,
2. conduct the required operations there, and
3. map the results back to the original space through the inverse.
![Page 417: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/417.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 417,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider vector spaces V and W over the same field F and of thesame dimension n.
![Page 418: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/418.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 418,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider vector spaces V and W over the same field F and of thesame dimension n.
Question: Can we define an isomorphism between them?
![Page 419: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/419.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 419,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider vector spaces V and W over the same field F and of thesame dimension n.
Question: Can we define an isomorphism between them?
Answer: Of course. As many as we want!
![Page 420: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/420.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 420,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider vector spaces V and W over the same field F and of thesame dimension n.
Question: Can we define an isomorphism between them?
Answer: Of course. As many as we want!
The underlying field and the dimension togethercompletely specify a vector space, up to an isomorphism.
![Page 421: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/421.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 421,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Isomorphism
Consider vector spaces V and W over the same field F and of thesame dimension n.
Question: Can we define an isomorphism between them?
Answer: Of course. As many as we want!
The underlying field and the dimension togethercompletely specify a vector space, up to an isomorphism.
All n-dimensional vector spaces over the field F areisomorphic to one another.
In particular, they are all isomorphic to F n.
The representation (columns) can be considered as theobjects (vectors) themselves.
![Page 422: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/422.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 422,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner product (a,b) in a real or complex vector space: a scalarfunction p : V × V → F satisfying
Closure: ∀ a,b ∈ V, (a,b) ∈ F
Associativity: (αa,b) = α(a,b)
Distributivity: (a + b, c) = (a, c) + (b, c)
Conjugate commutativity: (b, a) = (a,b)
Positive definiteness: (a, a) ≥ 0; and (a, a) = 0 iff a = 0
Note: Property of conjugate commutativity forces (a, a) to be real.
Examples: aTb, aTWb in R , a∗b in C etc.
![Page 423: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/423.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 423,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner product (a,b) in a real or complex vector space: a scalarfunction p : V × V → F satisfying
Closure: ∀ a,b ∈ V, (a,b) ∈ F
Associativity: (αa,b) = α(a,b)
Distributivity: (a + b, c) = (a, c) + (b, c)
Conjugate commutativity: (b, a) = (a,b)
Positive definiteness: (a, a) ≥ 0; and (a, a) = 0 iff a = 0
Note: Property of conjugate commutativity forces (a, a) to be real.
Examples: aTb, aTWb in R , a∗b in C etc.
Inner product space: a vector space possessing an inner product
Euclidean space: over R
Unitary space: over C
![Page 424: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/424.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 424,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner products bring in ideas of angle and length in the geometryof vector spaces.
![Page 425: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/425.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 425,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner products bring in ideas of angle and length in the geometryof vector spaces.
Orthogonality: (a,b) = 0
Norm: ‖ · ‖ : V → R , such that ‖a‖ =√
(a, a)
![Page 426: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/426.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 426,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner products bring in ideas of angle and length in the geometryof vector spaces.
Orthogonality: (a,b) = 0
Norm: ‖ · ‖ : V → R , such that ‖a‖ =√
(a, a)
Associativity: ‖αa‖ = |α| ‖a‖Positive definiteness: ‖a‖ > 0 for a 6= 0 and ‖0‖ = 0
Triangle inequality: ‖a + b‖ ≤ ‖a‖+ ‖b‖Cauchy-Schwarz inequality: |(a,b)| ≤ ‖a‖ ‖b‖
![Page 427: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/427.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 427,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Inner Product Space
Inner products bring in ideas of angle and length in the geometryof vector spaces.
Orthogonality: (a,b) = 0
Norm: ‖ · ‖ : V → R , such that ‖a‖ =√
(a, a)
Associativity: ‖αa‖ = |α| ‖a‖Positive definiteness: ‖a‖ > 0 for a 6= 0 and ‖0‖ = 0
Triangle inequality: ‖a + b‖ ≤ ‖a‖+ ‖b‖Cauchy-Schwarz inequality: |(a,b)| ≤ ‖a‖ ‖b‖
A distance function or metric: dV : V × V → R such that
dV(a,b) = ‖a − b‖
![Page 428: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/428.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 428,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Suppose we decide to represent a continuous functionf : [a, b]→ R by the listing
vf = [f (x1) f (x2) f (x3) · · · f (xN)]T
with a = x1 < x2 < x3 < · · · < xN = b.
Note: The ‘true’ representation will require N to be infinite!
![Page 429: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/429.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 429,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Suppose we decide to represent a continuous functionf : [a, b]→ R by the listing
vf = [f (x1) f (x2) f (x3) · · · f (xN)]T
with a = x1 < x2 < x3 < · · · < xN = b.
Note: The ‘true’ representation will require N to be infinite!
Here, vf is a real column vector.Do such vectors form a vector space?
![Page 430: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/430.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 430,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Suppose we decide to represent a continuous functionf : [a, b]→ R by the listing
vf = [f (x1) f (x2) f (x3) · · · f (xN)]T
with a = x1 < x2 < x3 < · · · < xN = b.
Note: The ‘true’ representation will require N to be infinite!
Here, vf is a real column vector.Do such vectors form a vector space?
Correspondingly, does the set F of continuous functionsover [a, b] form a vector space?
infinite dimensional vector space
![Page 431: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/431.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 431,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Vector space of continuous functions
First, (F ,+) is a commutative group.
![Page 432: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/432.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 432,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Vector space of continuous functions
First, (F ,+) is a commutative group.
Next, with α, β ∈ R , ∀x ∈ [a, b],
if f (x) ∈ R , then αf (x) ∈ R
1 · f (x) = f (x)
(αβ)f (x) = α[βf (x)]
α[f1(x) + f2(x)] = αf1(x) + αf2(x)
(α+ β)f (x) = αf (x) + βf (x)
![Page 433: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/433.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 433,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Vector space of continuous functions
First, (F ,+) is a commutative group.
Next, with α, β ∈ R , ∀x ∈ [a, b],
if f (x) ∈ R , then αf (x) ∈ R
1 · f (x) = f (x)
(αβ)f (x) = α[βf (x)]
α[f1(x) + f2(x)] = αf1(x) + αf2(x)
(α+ β)f (x) = αf (x) + βf (x)
Thus, F forms a vector space over R .
Every function in this space is an (infinite dimensional) vector.
Listing of values is just an obvious basis.
![Page 434: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/434.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 434,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Linear dependence of (non-zero) functions f1 and f2
f2(x) = kf1(x) for all x in the domain
k1f1(x) + k2f2(x) = 0, ∀x with k1 and k2 not both zero.
Linear independence: k1f1(x) + k2f2(x) = 0 ∀x ⇒ k1 = k2 = 0
![Page 435: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/435.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 435,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Linear dependence of (non-zero) functions f1 and f2
f2(x) = kf1(x) for all x in the domain
k1f1(x) + k2f2(x) = 0, ∀x with k1 and k2 not both zero.
Linear independence: k1f1(x) + k2f2(x) = 0 ∀x ⇒ k1 = k2 = 0
In general,
Functions f1, f2, f3, · · · , fn ∈ F are linearly dependent if∃ k1, k2, k3, · · · , kn, not all zero, such thatk1f1(x) + k2f2(x) + k3f3(x) + · · ·+ knfn(x) = 0 ∀x ∈ [a, b].
k1f1(x) + k2f2(x) + k3f3(x) + · · ·+ knfn(x) = 0 ∀x ∈ [a, b] ⇒k1, k2, k3, · · · , kn = 0 means that functions f1, f2, f3, · · · , fn arelinearly independent.
![Page 436: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/436.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 436,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Linear dependence of (non-zero) functions f1 and f2
f2(x) = kf1(x) for all x in the domain
k1f1(x) + k2f2(x) = 0, ∀x with k1 and k2 not both zero.
Linear independence: k1f1(x) + k2f2(x) = 0 ∀x ⇒ k1 = k2 = 0
In general,
Functions f1, f2, f3, · · · , fn ∈ F are linearly dependent if∃ k1, k2, k3, · · · , kn, not all zero, such thatk1f1(x) + k2f2(x) + k3f3(x) + · · ·+ knfn(x) = 0 ∀x ∈ [a, b].
k1f1(x) + k2f2(x) + k3f3(x) + · · ·+ knfn(x) = 0 ∀x ∈ [a, b] ⇒k1, k2, k3, · · · , kn = 0 means that functions f1, f2, f3, · · · , fn arelinearly independent.
Example: functions 1, x , x2, x3, · · · are a set of linearlyindependent functions.
Incidentally, this set is a commonly used basis.
![Page 437: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/437.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 437,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Inner product: For functions f (x) and g(x) in F , the usual innerproduct between corresponding vectors:
(vf , vg ) = vTf vg = f (x1)g(x1) + f (x2)g(x2) + f (x3)g(x3) + · · ·
Weighted inner product: (vf , vg ) = vTf Wvg =
∑
i wi f (xi )g(xi )
![Page 438: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/438.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 438,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Inner product: For functions f (x) and g(x) in F , the usual innerproduct between corresponding vectors:
(vf , vg ) = vTf vg = f (x1)g(x1) + f (x2)g(x2) + f (x3)g(x3) + · · ·
Weighted inner product: (vf , vg ) = vTf Wvg =
∑
i wi f (xi )g(xi )
For the functions,
(f , g) =
∫ b
a
w(x)f (x)g(x)dx
![Page 439: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/439.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 439,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Function Space
Inner product: For functions f (x) and g(x) in F , the usual innerproduct between corresponding vectors:
(vf , vg ) = vTf vg = f (x1)g(x1) + f (x2)g(x2) + f (x3)g(x3) + · · ·
Weighted inner product: (vf , vg ) = vTf Wvg =
∑
i wi f (xi )g(xi )
For the functions,
(f , g) =
∫ b
a
w(x)f (x)g(x)dx
Orthogonality: (f , g) =∫ b
aw(x)f (x)g(x)dx = 0
Norm: ‖f ‖ =√
∫ b
aw(x)[f (x)]2dx
Orthonormal basis:(fj , fk) =
∫ b
aw(x)fj(x)fk(x)dx = δjk ∀j , k
![Page 440: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/440.jpg)
Mathematical Methods in Engineering and Science Vector Spaces: Fundamental Concepts* 440,
GroupFieldVector SpaceLinear TransformationIsomorphismInner Product SpaceFunction Space
Points to note
Matrix algebra provides a natural description for vector spacesand linear transformations.
Through isomorphisms, Rn can represent all n-dimensionalreal vector spaces.
Through the definition of an inner product, a vector spaceincorporates key geometric features of physical space.
Continuous functions over an interval constitute an infinitedimensional vector space, complete with the usual notions.
Necessary Exercises: 6,7
![Page 441: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/441.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 441,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Outline
Topics in Multivariate CalculusDerivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
![Page 442: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/442.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 442,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Derivatives in Multi-Dimensional Spaces
Gradient
∇f (x) ≡ ∂f
∂x(x) =
[
∂f
∂x1
∂f
∂x2· · · ∂f
∂xn
]T
Up to the first order, δf ≈ [∇f (x)]T δxDirectional derivative
∂f
∂d= lim
α→0
f (x + αd)− f (x)
α
![Page 443: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/443.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 443,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Derivatives in Multi-Dimensional Spaces
Gradient
∇f (x) ≡ ∂f
∂x(x) =
[
∂f
∂x1
∂f
∂x2· · · ∂f
∂xn
]T
Up to the first order, δf ≈ [∇f (x)]T δxDirectional derivative
∂f
∂d= lim
α→0
f (x + αd)− f (x)
α
Relationships:
∂f
∂ej=∂f
∂xj,
∂f
∂d= dT∇f (x) and
∂f
∂g= ‖∇f (x)‖
Among all unit vectors, taken as directions,
the rate of change of a function in a direction is the same asthe component of its gradient along that direction, and
the rate of change along the direction of the gradient is thegreatest and is equal to the magnitude of the gradient.
![Page 444: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/444.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 444,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Derivatives in Multi-Dimensional Spaces
Hessian
H(x) =∂2f
∂x2=
∂2f∂x1
2∂2f
∂x2∂x1· · · ∂2f
∂xn∂x1∂2f
∂x1∂x2
∂2f∂x2
2 · · · ∂2f∂xn∂x2
......
. . ....
∂2f∂x1∂xn
∂2f∂x2∂xn
· · · ∂2f∂xn
2
Meaning: ∇f (x + δx) −∇f (x) ≈[
∂2f∂x2 (x)
]
δx
![Page 445: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/445.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 445,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Derivatives in Multi-Dimensional Spaces
Hessian
H(x) =∂2f
∂x2=
∂2f∂x1
2∂2f
∂x2∂x1· · · ∂2f
∂xn∂x1∂2f
∂x1∂x2
∂2f∂x2
2 · · · ∂2f∂xn∂x2
......
. . ....
∂2f∂x1∂xn
∂2f∂x2∂xn
· · · ∂2f∂xn
2
Meaning: ∇f (x + δx) −∇f (x) ≈[
∂2f∂x2 (x)
]
δx
For a vector function h(x), Jacobian
J(x) =∂h
∂x(x) =
[
∂h
∂x1
∂h
∂x2· · · ∂h
∂xn
]
Underlying notion: δh ≈ [J(x)]δx
![Page 446: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/446.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 446,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Taylor’s Series
Taylor’s formula in the remainder form:
f (x + δx) = f (x) + f ′(x)δx
+1
2!f ′′(x)δx2 + · · ·+ 1
(n − 1)!f (n−1)(x)δxn−1 +
1
n!f (n)(xc)δx
n
where xc = x + tδx with 0 ≤ t ≤ 1Mean value theorem: existence of xc
![Page 447: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/447.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 447,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Taylor’s Series
Taylor’s formula in the remainder form:
f (x + δx) = f (x) + f ′(x)δx
+1
2!f ′′(x)δx2 + · · ·+ 1
(n − 1)!f (n−1)(x)δxn−1 +
1
n!f (n)(xc)δx
n
where xc = x + tδx with 0 ≤ t ≤ 1Mean value theorem: existence of xc
Taylor’s series:
f (x + δx) = f (x) + f ′(x)δx +1
2!f ′′(x)δx2 + · · ·
![Page 448: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/448.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 448,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Taylor’s Series
Taylor’s formula in the remainder form:
f (x + δx) = f (x) + f ′(x)δx
+1
2!f ′′(x)δx2 + · · ·+ 1
(n − 1)!f (n−1)(x)δxn−1 +
1
n!f (n)(xc)δx
n
where xc = x + tδx with 0 ≤ t ≤ 1Mean value theorem: existence of xc
Taylor’s series:
f (x + δx) = f (x) + f ′(x)δx +1
2!f ′′(x)δx2 + · · ·
For a multivariate function,
f (x + δx) = f (x) + [δxT∇]f (x) +1
2![δxT∇]2f (x) + · · ·
+1
(n − 1)![δxT∇]n−1f (x) +
1
n![δxT∇]nf (x + tδx)
f (x + δx) ≈ f (x) + [∇f (x)]T δx +1
2δxT
[
∂2f
∂x2(x)
]
δx
![Page 449: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/449.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 449,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
For f (x), the total differential:
df = [∇f (x)]T dx =∂f
∂x1dx1 +
∂f
∂x2dx2 + · · ·+ ∂f
∂xndxn
Ordinary derivative or total derivative:
df
dt= [∇f (x)]T
dx
dt
![Page 450: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/450.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 450,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
For f (x), the total differential:
df = [∇f (x)]T dx =∂f
∂x1dx1 +
∂f
∂x2dx2 + · · ·+ ∂f
∂xndxn
Ordinary derivative or total derivative:
df
dt= [∇f (x)]T
dx
dt
For f (t, x(t)), total derivative: dfdt
= ∂f∂t
+ [∇f (x)]T dxdt
![Page 451: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/451.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 451,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
For f (x), the total differential:
df = [∇f (x)]T dx =∂f
∂x1dx1 +
∂f
∂x2dx2 + · · ·+ ∂f
∂xndxn
Ordinary derivative or total derivative:
df
dt= [∇f (x)]T
dx
dt
For f (t, x(t)), total derivative: dfdt
= ∂f∂t
+ [∇f (x)]T dxdt
For f (v, x(v)) = f (v1, v2, · · · , vm, x1(v), x2(v), · · · , xn(v)),
∂f
∂vi
(v, x(v)) =
(
∂f
∂vi
)
x
+
[
∂f
∂x(v, x)
]T ∂x
∂vi
=
(
∂f
∂vi
)
x
+[∇x f (v, x)]T∂x
∂vi
⇒ ∇f (v, x(v)) = ∇v f (v, x) +
[
∂x
∂v(v)
]T
∇x f (v, x)
![Page 452: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/452.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 452,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
![Page 453: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/453.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 453,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
System of equations h(x) = 0 means h(z,w) = 0.
Question: Can we work out the function w = w(z)?
![Page 454: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/454.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 454,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
System of equations h(x) = 0 means h(z,w) = 0.
Question: Can we work out the function w = w(z)?
Solution of m equations in m unknowns?
![Page 455: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/455.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 455,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
System of equations h(x) = 0 means h(z,w) = 0.
Question: Can we work out the function w = w(z)?
Solution of m equations in m unknowns?
Question: If we have one valid pair (z,w), then is it possible todevelop w = w(z) in the local neighbourhood?
![Page 456: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/456.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 456,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
System of equations h(x) = 0 means h(z,w) = 0.
Question: Can we work out the function w = w(z)?
Solution of m equations in m unknowns?
Question: If we have one valid pair (z,w), then is it possible todevelop w = w(z) in the local neighbourhood?Answer: Yes, if Jacobian ∂h
∂w is non-singular.
Implicit function theorem
![Page 457: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/457.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 457,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Let x ∈ Rm+n and h(x) ∈ Rm.
Partition x ∈ Rm+n into z ∈ Rn and w ∈ Rm.
System of equations h(x) = 0 means h(z,w) = 0.
Question: Can we work out the function w = w(z)?
Solution of m equations in m unknowns?
Question: If we have one valid pair (z,w), then is it possible todevelop w = w(z) in the local neighbourhood?Answer: Yes, if Jacobian ∂h
∂w is non-singular.
Implicit function theorem
∂h
∂z+∂h
∂w
∂w
∂z= 0 ⇒ ∂w
∂z= −
[
∂h
∂w
]−1 [∂h
∂z
]
Upto first order, w1 = w +[
∂w∂z
]
(z1 − z).
![Page 458: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/458.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 458,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
For a multiple integral
I =
∫ ∫
A
∫
f (x , y , z) dx dy dz ,
change of variables x = x(u, v ,w), y = y(u, v ,w), z = z(u, v ,w)gives
I =
∫ ∫
A
∫
f (x(u, v ,w), y(u, v ,w), z(u, v ,w)) |J(u, v ,w)| du dv dw ,
where Jacobian determinant |J(u, v ,w)| =∣
∣
∣
∂(x ,y ,z)∂(u,v ,w)
∣
∣
∣.
![Page 459: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/459.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 459,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
For the differential
P1(x)dx1 + P2(x)dx2 + · · ·+ Pn(x)dxn,
we ask: does there exist a function f (x),
of which this is the differential;
or equivalently, the gradient of which is P(x)?
Perfect or exact differential: can be integrated to find f .
![Page 460: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/460.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 460,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Differentiation under the integral sign
How To differentiate φ(x) = φ(x , u(x), v(x)) =∫ v(x)u(x) f (x , t) dt?
![Page 461: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/461.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 461,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Differentiation under the integral sign
How To differentiate φ(x) = φ(x , u(x), v(x)) =∫ v(x)u(x) f (x , t) dt?
In the expression
φ′(x) =∂φ
∂x+∂φ
∂u
du
dx+∂φ
∂v
dv
dx,
we have ∂φ∂x
=∫ v
u∂f∂x
(x , t)dt.
![Page 462: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/462.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 462,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Differentiation under the integral sign
How To differentiate φ(x) = φ(x , u(x), v(x)) =∫ v(x)u(x) f (x , t) dt?
In the expression
φ′(x) =∂φ
∂x+∂φ
∂u
du
dx+∂φ
∂v
dv
dx,
we have ∂φ∂x
=∫ v
u∂f∂x
(x , t)dt.
Now, considering function F (x , t) such that f (x , t) = ∂F (x ,t)∂t
,
φ(x) =
∫ v
u
∂F
∂t(x , t)dt = F (x , v)− F (x , u) ≡ φ(x , u, v).
![Page 463: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/463.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 463,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Chain Rule and Change of Variables
Differentiation under the integral sign
How To differentiate φ(x) = φ(x , u(x), v(x)) =∫ v(x)u(x) f (x , t) dt?
In the expression
φ′(x) =∂φ
∂x+∂φ
∂u
du
dx+∂φ
∂v
dv
dx,
we have ∂φ∂x
=∫ v
u∂f∂x
(x , t)dt.
Now, considering function F (x , t) such that f (x , t) = ∂F (x ,t)∂t
,
φ(x) =
∫ v
u
∂F
∂t(x , t)dt = F (x , v)− F (x , u) ≡ φ(x , u, v).
Using ∂φ∂v
= f (x , v) and ∂φ∂u
= −f (x , u),
φ′(x) =
∫ v(x)
u(x)
∂f
∂x(x , t)dt + f (x , v)
dv
dx− f (x , u)
du
dx.
Leibnitz rule
![Page 464: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/464.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 464,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Numerical DifferentiationForward difference formula
f ′(x) =f (x + δx) − f (x)
δx+O(δx)
![Page 465: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/465.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 465,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Numerical DifferentiationForward difference formula
f ′(x) =f (x + δx) − f (x)
δx+O(δx)
Central difference formulae
f ′(x) =f (x + δx) − f (x − δx)
2δx+O(δx2)
f ′′(x) =f (x + δx) − 2f (x) + f (x − δx)
δx2+O(δx2)
![Page 466: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/466.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 466,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Numerical DifferentiationForward difference formula
f ′(x) =f (x + δx) − f (x)
δx+O(δx)
Central difference formulae
f ′(x) =f (x + δx) − f (x − δx)
2δx+O(δx2)
f ′′(x) =f (x + δx) − 2f (x) + f (x − δx)
δx2+O(δx2)
For gradient ∇f (x) and Hessian,
∂f
∂xi(x) =
1
2δ[f (x + δei )− f (x − δei )],
∂2f
∂xi2(x) =
f (x + δei )− 2f (x) + f (x − δei )
δ2, and
∂2f
∂xi∂xj
(x) =
f (x + δei + δej )− f (x + δei − δej )− f (x − δei + δej ) + f (x − δei − δej )
4δ2
![Page 467: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/467.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 467,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
An Introduction to Tensors*
Indicial notation and summation convention
Kronecker delta and Levi-Civita symbol
Rotation of reference axes
Tensors of order zero, or scalars
Contravariant and covariant tensors of order one, or vectors
Cartesian tensors
Cartesian tensors of order two
Higher order tensors
Elementary tensor operations
Symmetric tensors
Tensor fields
· · · · · · · · ·
![Page 468: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/468.jpg)
Mathematical Methods in Engineering and Science Topics in Multivariate Calculus 468,
Derivatives in Multi-Dimensional SpacesTaylor’s SeriesChain Rule and Change of VariablesNumerical DifferentiationAn Introduction to Tensors*
Points to note
Gradient, Hessian, Jacobian and the Taylor’s series
Partial and total gradients
Implicit functions
Leibnitz rule
Numerical derivatives
Necessary Exercises: 2,3,4,8
![Page 469: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/469.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 469,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Outline
Vector Analysis: Curves and SurfacesRecapitulation of Basic NotionsCurves in SpaceSurfaces*
![Page 470: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/470.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 470,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Recapitulation of Basic Notions
Dot and cross products: their implications
Scalar and vector triple products
Differentiation rules
![Page 471: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/471.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 471,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Recapitulation of Basic Notions
Dot and cross products: their implications
Scalar and vector triple products
Differentiation rules
Interface with matrix algebra:
a · x = aTx,
(a · x)b = (baT )x, and
a × x =
aT⊥x, for 2-d vectors∼ax, for 3-d vectors
where
a⊥ =
[
−ay
ax
]
and∼a =
0 −az ay
az 0 −ax
−ay ax 0
![Page 472: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/472.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 472,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Explicit equation: y = y(x) and z = z(x)
Implicit equation: F (x , y , z) = 0 = G (x , y , z)
Parametric equation:
r(t) = x(t)i + y(t)j + z(t)k ≡ [x(t) y(t) z(t)]T
![Page 473: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/473.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 473,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Explicit equation: y = y(x) and z = z(x)
Implicit equation: F (x , y , z) = 0 = G (x , y , z)
Parametric equation:
r(t) = x(t)i + y(t)j + z(t)k ≡ [x(t) y(t) z(t)]T
Tangent vector: r′(t) Speed: ‖r′‖ Unit tangent: u(t) = r′
‖r′‖ Length of the curve: l =
∫ b
a‖dr‖ =
∫ b
a
√
r′ · r′ dt
![Page 474: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/474.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 474,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Explicit equation: y = y(x) and z = z(x)
Implicit equation: F (x , y , z) = 0 = G (x , y , z)
Parametric equation:
r(t) = x(t)i + y(t)j + z(t)k ≡ [x(t) y(t) z(t)]T
Tangent vector: r′(t) Speed: ‖r′‖ Unit tangent: u(t) = r′
‖r′‖ Length of the curve: l =
∫ b
a‖dr‖ =
∫ b
a
√
r′ · r′ dt
Arc length function
s(t) =
∫ t
a
√
r′(τ) · r′(τ) dτ
with ds = ‖dr‖ =√
dx2 + dy2 + dz2 and dsdt
= ‖r′‖
![Page 475: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/475.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 475,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curve r(t) is regular if r′(t) 6= 0 ∀t. Reparametrization with respect to parameter t∗, some
strictly increasing function of t
![Page 476: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/476.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 476,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curve r(t) is regular if r′(t) 6= 0 ∀t. Reparametrization with respect to parameter t∗, some
strictly increasing function of t
Observations
Arc length s(t) is obviously a monotonically increasingfunction.
For a regular curve, dsdt6= 0.
Then, s(t) has an inverse function.
Inverse t(s) reparametrizes the curve as r(t(s)).
![Page 477: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/477.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 477,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curve r(t) is regular if r′(t) 6= 0 ∀t. Reparametrization with respect to parameter t∗, some
strictly increasing function of t
Observations
Arc length s(t) is obviously a monotonically increasingfunction.
For a regular curve, dsdt6= 0.
Then, s(t) has an inverse function.
Inverse t(s) reparametrizes the curve as r(t(s)).
For a unit speed curve r(s), ‖r′(s)‖ = 1 and the unit tangent is
u(s) = r′(s).
![Page 478: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/478.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 478,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curvature: The rate at which the direction changes with arclength.
κ(s) = ‖u′(s)‖ = ‖r′′(s)‖Unit principal normal:
p =1
κu′(s)
![Page 479: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/479.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 479,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curvature: The rate at which the direction changes with arclength.
κ(s) = ‖u′(s)‖ = ‖r′′(s)‖Unit principal normal:
p =1
κu′(s)
With general parametrization,
r′′(t) =d‖r′‖dt
u(t) + ‖r′(t)‖du
dt=
d‖r′‖dt
u(t) + κ(t)‖r′‖2p(t)
![Page 480: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/480.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 480,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Curvature: The rate at which the direction changes with arclength.
κ(s) = ‖u′(s)‖ = ‖r′′(s)‖Unit principal normal:
p =1
κu′(s)
With general parametrization,
r′′(t) =d‖r′‖dt
u(t) + ‖r′(t)‖du
dt=
d‖r′‖dt
u(t) + κ(t)‖r′‖2p(t)
Osculating plane
Centre of curvature
Radius of curvature
AC = ρ = 1/κ
/ /r
uC
A
/
x
y
z
r
O
r
p
Figure: Tangent and normal to a curve
![Page 481: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/481.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 481,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Binormal: b = u × p
Serret-Frenet frame: Right-handed triad u,p,b Osculating, rectifying and normal planes
![Page 482: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/482.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 482,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Binormal: b = u × p
Serret-Frenet frame: Right-handed triad u,p,b Osculating, rectifying and normal planes
Torsion: Twisting out of the osculating plane
rate of change of b with respect to arc length s
b′ = u′ × p + u × p′ = κ(s)p × p + u × p′ = u × p′
What is p′?
![Page 483: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/483.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 483,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
Binormal: b = u × p
Serret-Frenet frame: Right-handed triad u,p,b Osculating, rectifying and normal planes
Torsion: Twisting out of the osculating plane
rate of change of b with respect to arc length s
b′ = u′ × p + u × p′ = κ(s)p × p + u × p′ = u × p′
What is p′?
Taking p′ = σu + τb,
b′ = u × (σu + τb) = −τp.
Torsion of the curve
τ(s) = −p(s) · b′(s)
![Page 484: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/484.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 484,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
We have u′ and b′. What is p′?
![Page 485: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/485.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 485,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
We have u′ and b′. What is p′?
From p = b × u,
p′ = b′ × u + b × u′ = −τp × u + b × κp = −κu + τb.
![Page 486: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/486.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 486,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
We have u′ and b′. What is p′?
From p = b × u,
p′ = b′ × u + b × u′ = −τp × u + b × κp = −κu + τb.
Serret-Frenet formulae
u′ = κp,p′ = −κu + τb,b′ = −τp
![Page 487: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/487.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 487,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Curves in Space
We have u′ and b′. What is p′?
From p = b × u,
p′ = b′ × u + b × u′ = −τp × u + b × κp = −κu + τb.
Serret-Frenet formulae
u′ = κp,p′ = −κu + τb,b′ = −τp
Intrinsic representation of a curve is complete with κ(s) and τ(s).
The arc-length parametrization of a curve is completelydetermined by its curvature κ(s) and torsion τ(s)functions, except for a rigid body motion.
![Page 488: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/488.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 488,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Surfaces*
Parametric surface equation:
r(u, v) = x(u, v)i+y(u, v)j+z(u, v)k ≡ [x(u, v) y(u, v) z(u, v)]T
Tangent vectors ru and rv define a tangent plane T .
N = ru × rv is normal to the surface and the unit normal is
n =N
‖N‖ =ru × rv‖ru × rv‖
.
![Page 489: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/489.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 489,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Surfaces*
Parametric surface equation:
r(u, v) = x(u, v)i+y(u, v)j+z(u, v)k ≡ [x(u, v) y(u, v) z(u, v)]T
Tangent vectors ru and rv define a tangent plane T .
N = ru × rv is normal to the surface and the unit normal is
n =N
‖N‖ =ru × rv‖ru × rv‖
.
Question: How does n vary over the surface?
Information on local geometry: curvature tensor
Normal and principal curvatures
Local shape: convex, concave, saddle, cylindrical, planar
![Page 490: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/490.jpg)
Mathematical Methods in Engineering and Science Vector Analysis: Curves and Surfaces 490,
Recapitulation of Basic NotionsCurves in SpaceSurfaces*
Points to note
Parametric equation is the general and most convenientrepresentation of curves and surfaces.
Arc length is the natural parameter and the Serret-Frenetframe offers the natural frame of reference.
Curvature and torsion are the only inherent properties of acurve.
The local shape of a surface patch can be understood throughan analysis of its curvature tensor.
Necessary Exercises: 1,2,3,6
![Page 491: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/491.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 491,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Outline
Scalar and Vector FieldsDifferential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
![Page 492: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/492.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 492,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Scalar point function or scalar field φ(x , y , z): R3 → RVector point function or vector field V(x , y , z): R3 → R3
The del or nabla (∇) operator
∇ ≡ i∂
∂x+ j
∂
∂y+ k
∂
∂z
∇ is a vector, it signifies a differentiation, and it operates from the left side.
![Page 493: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/493.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 493,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Scalar point function or scalar field φ(x , y , z): R3 → RVector point function or vector field V(x , y , z): R3 → R3
The del or nabla (∇) operator
∇ ≡ i∂
∂x+ j
∂
∂y+ k
∂
∂z
∇ is a vector, it signifies a differentiation, and it operates from the left side.
Laplacian operator:
∇2 ≡ ∂2
∂x2+
∂2
∂y2+
∂2
∂z2= ∇ · ∇ ??
Laplace’s equation:
∂2φ
∂x2+∂2φ
∂y2+∂2φ
∂z2= 0
Solution of ∇2φ = 0: harmonic function
![Page 494: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/494.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 494,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Gradient
grad φ ≡ ∇φ =∂φ
∂xi +
∂φ
∂yj +
∂φ
∂zk
is orthogonal to the level surfaces.
Flow fields: −∇φ gives the velocity vector.
![Page 495: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/495.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 495,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Gradient
grad φ ≡ ∇φ =∂φ
∂xi +
∂φ
∂yj +
∂φ
∂zk
is orthogonal to the level surfaces.
Flow fields: −∇φ gives the velocity vector.
Divergence
For V(x , y , z) ≡ Vx(x , y , z)i + Vy(x , y , z)j + Vz(x , y , z)k,
div V ≡ ∇ · V =∂Vx
∂x+∂Vy
∂y+∂Vz
∂z
Divergence of ρV: flow rate of mass per unit volume out of thecontrol volume.
Similar relation between field and flux in electromagnetics.
![Page 496: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/496.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 496,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Curl
curl V ≡ ∇×V =
∣
∣
∣
∣
∣
∣
i j k∂∂x
∂∂y
∂∂z
Vx Vy Vz
∣
∣
∣
∣
∣
∣
=
(
∂Vz
∂y− ∂Vy
∂z
)
i +
(
∂Vx
∂z− ∂Vz
∂x
)
j +
(
∂Vy
∂x− ∂Vx
∂y
)
k
If V = ω × r represents the velocity field, then angular velocity
ω =1
2curl V.
Curl represents rotationality.
Connections between electric and magnetic fields!
![Page 497: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/497.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 497,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Composite operations
Operator ∇ is linear.
∇(φ+ ψ) = ∇φ+∇ψ,∇ · (V + W) = ∇ ·V +∇ ·W, and
∇× (V + W) = ∇× V +∇×W.
![Page 498: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/498.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 498,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Composite operations
Operator ∇ is linear.
∇(φ+ ψ) = ∇φ+∇ψ,∇ · (V + W) = ∇ ·V +∇ ·W, and
∇× (V + W) = ∇× V +∇×W.
Considering the products φψ, φV, V ·W, and V ×W;
∇(φψ) = ψ∇φ+ φ∇ψ∇ · (φV) = ∇φ ·V + φ∇ · V∇× (φV) = ∇φ× V + φ∇× V∇(V ·W) = (W ·∇)V +(V ·∇)W +W× (∇×V)+V× (∇×W)∇ · (V ×W) = W · (∇×V)− V · (∇×W)∇× (V×W) = (W · ∇)V−W(∇ ·V)− (V · ∇)W + V(∇ ·W)
![Page 499: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/499.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 499,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Composite operations
Operator ∇ is linear.
∇(φ+ ψ) = ∇φ+∇ψ,∇ · (V + W) = ∇ ·V +∇ ·W, and
∇× (V + W) = ∇× V +∇×W.
Considering the products φψ, φV, V ·W, and V ×W;
∇(φψ) = ψ∇φ+ φ∇ψ∇ · (φV) = ∇φ ·V + φ∇ · V∇× (φV) = ∇φ× V + φ∇× V∇(V ·W) = (W ·∇)V +(V ·∇)W +W× (∇×V)+V× (∇×W)∇ · (V ×W) = W · (∇×V)− V · (∇×W)∇× (V×W) = (W · ∇)V−W(∇ ·V)− (V · ∇)W + V(∇ ·W)
Note: the expression V · ∇ ≡ Vx∂∂x
+ Vy∂∂y
+ Vz∂∂z
is an operator!
![Page 500: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/500.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 500,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Second order differential operators
div grad φ ≡ ∇ · (∇φ)
curl grad φ ≡ ∇× (∇φ)
div curl V ≡ ∇ · (∇× V)
curl curl V ≡ ∇× (∇× V)
grad div V ≡ ∇(∇ ·V)
![Page 501: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/501.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 501,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Differential Operations on Field Functions
Second order differential operators
div grad φ ≡ ∇ · (∇φ)
curl grad φ ≡ ∇× (∇φ)
div curl V ≡ ∇ · (∇× V)
curl curl V ≡ ∇× (∇× V)
grad div V ≡ ∇(∇ ·V)
Important identities:
div grad φ ≡ ∇ · (∇φ) = ∇2φ
curl grad φ ≡ ∇× (∇φ) = 0
div curl V ≡ ∇ · (∇× V) = 0
curl curl V ≡ ∇× (∇× V)
= ∇(∇ ·V)−∇2V = grad div V −∇2V
![Page 502: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/502.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 502,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Operations on Field Functions
Line integral along curve C :
I =
∫
C
V · dr =
∫
C
(Vxdx + Vydy + Vzdz)
For a parametrized curve r(t), t ∈ [a, b],
I =
∫
C
V · dr =
∫ b
a
V · dr
dtdt.
![Page 503: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/503.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 503,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Operations on Field Functions
Line integral along curve C :
I =
∫
C
V · dr =
∫
C
(Vxdx + Vydy + Vzdz)
For a parametrized curve r(t), t ∈ [a, b],
I =
∫
C
V · dr =
∫ b
a
V · dr
dtdt.
For simple (non-intersecting) paths contained in a simplyconnected region, equivalent statements:
Vxdx + Vydy + Vzdz is an exact differential.
V = ∇φ for some φ(r).
∫
CV · dr is independent of path.
Circulation∮
V · dr = 0 around any closed path.
curl V = 0.
Field V is conservative.
![Page 504: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/504.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 504,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Operations on Field Functions
Surface integral over an orientable surface S :
J =
∫
S
∫
V · dS =
∫
S
∫
V · ndS
For r(u,w), dS = ‖ru × rw‖ du dw and
J =
∫
S
∫
V · ndS =
∫
R
∫
V · (ru × rw ) du dw .
![Page 505: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/505.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 505,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Operations on Field Functions
Surface integral over an orientable surface S :
J =
∫
S
∫
V · dS =
∫
S
∫
V · ndS
For r(u,w), dS = ‖ru × rw‖ du dw and
J =
∫
S
∫
V · ndS =
∫
R
∫
V · (ru × rw ) du dw .
Volume integrals of point functions over a region T :
M =
∫ ∫
T
∫
φdv and F =
∫ ∫
T
∫
Vdv
![Page 506: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/506.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 506,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Green’s theorem in the plane
R: closed bounded region in the xy-planeC: boundary, a piecewise smooth closed curveF1(x , y) and F2(x , y): first order continuous functions
∮
C
(F1dx + F2dy) =
∫
R
∫ (
∂F2
∂x− ∂F1
∂y
)
dx dy
![Page 507: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/507.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 507,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Green’s theorem in the plane
R: closed bounded region in the xy-planeC: boundary, a piecewise smooth closed curveF1(x , y) and F2(x , y): first order continuous functions
∮
C
(F1dx + F2dy) =
∫
R
∫ (
∂F2
∂x− ∂F1
∂y
)
dx dy
x
y
R
(b) General domain(a) Simple domain
OO
1
R
D
C
B
A
x (y)
2y (x)
2x (y)
1y (x)
ba
d
c
y
x
Figure: Regions for proof of Green’s theorem in the plane
![Page 508: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/508.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 508,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Proof:
∫
R
∫
∂F1
∂ydxdy =
∫ b
a
∫ y2(x)
y1(x)
∂F1
∂ydydx
=
∫ b
a
[F1x , y2(x) − F1x , y1(x)]dx
= −∫ a
b
F1x , y2(x)dx −∫ b
a
F1x , y1(x)dx
= −∮
C
F1(x , y)dx
![Page 509: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/509.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 509,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Proof:
∫
R
∫
∂F1
∂ydxdy =
∫ b
a
∫ y2(x)
y1(x)
∂F1
∂ydydx
=
∫ b
a
[F1x , y2(x) − F1x , y1(x)]dx
= −∫ a
b
F1x , y2(x)dx −∫ b
a
F1x , y1(x)dx
= −∮
C
F1(x , y)dx
∫
R
∫
∂F2
∂xdxdy =
∫ d
c
∫ x2(y)
x1(y)
∂F2
∂xdxdy =
∮
C
F2(x , y)dy
Difference:∮
C(F1dx + F2dy) =
∫
R
∫
(
∂F2∂x− ∂F1
∂y
)
dx dy
![Page 510: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/510.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 510,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Proof:
∫
R
∫
∂F1
∂ydxdy =
∫ b
a
∫ y2(x)
y1(x)
∂F1
∂ydydx
=
∫ b
a
[F1x , y2(x) − F1x , y1(x)]dx
= −∫ a
b
F1x , y2(x)dx −∫ b
a
F1x , y1(x)dx
= −∮
C
F1(x , y)dx
∫
R
∫
∂F2
∂xdxdy =
∫ d
c
∫ x2(y)
x1(y)
∂F2
∂xdxdy =
∮
C
F2(x , y)dy
Difference:∮
C(F1dx + F2dy) =
∫
R
∫
(
∂F2∂x− ∂F1
∂y
)
dx dy
In alternative form,∮
CF · dr =
∫
R
∫
curl F · k dx dy .
![Page 511: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/511.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 511,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Gauss’s divergence theorem
T: a closed bounded regionS: boundary, a piecewise smooth closed orientablesurfaceF(x , y , z): a first order continuous vector function
∫ ∫
T
∫
div Fdv =
∫
S
∫
F · ndS
Interpretation of the definition extended to finite domains.
![Page 512: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/512.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 512,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Gauss’s divergence theorem
T: a closed bounded regionS: boundary, a piecewise smooth closed orientablesurfaceF(x , y , z): a first order continuous vector function
∫ ∫
T
∫
div Fdv =
∫
S
∫
F · ndS
Interpretation of the definition extended to finite domains.
∫ ∫
T
∫ (
∂Fx
∂x+∂Fy
∂y+∂Fz
∂z
)
dx dy dz =
∫
S
∫
(Fxnx+Fyny+Fznz)dS
To show:∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
S
∫
FznzdS
![Page 513: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/513.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 513,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Gauss’s divergence theorem
T: a closed bounded regionS: boundary, a piecewise smooth closed orientablesurfaceF(x , y , z): a first order continuous vector function
∫ ∫
T
∫
div Fdv =
∫
S
∫
F · ndS
Interpretation of the definition extended to finite domains.
∫ ∫
T
∫ (
∂Fx
∂x+∂Fy
∂y+∂Fz
∂z
)
dx dy dz =
∫
S
∫
(Fxnx+Fyny+Fznz)dS
To show:∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
S
∫
FznzdS
First consider a region, the boundary of which is intersected atmost twice by any line parallel to a coordinate axis.
![Page 514: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/514.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 514,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Lower and upper segments of S : z = z1(x , y) and z = z2(x , y).
∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
R
∫ [∫ z2
z1
∂Fz
∂zdz
]
dx dy
=
∫
R
∫
[Fzx , y , z2(x , y) − Fzx , y , z1(x , y)]dx dy
R : projection of T on the xy -plane
![Page 515: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/515.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 515,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Lower and upper segments of S : z = z1(x , y) and z = z2(x , y).
∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
R
∫ [∫ z2
z1
∂Fz
∂zdz
]
dx dy
=
∫
R
∫
[Fzx , y , z2(x , y) − Fzx , y , z1(x , y)]dx dy
R : projection of T on the xy -plane
Projection of area element of the upper segment: nzdS = dx dyProjection of area element of the lower segment: nzdS = −dx dy
![Page 516: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/516.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 516,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Lower and upper segments of S : z = z1(x , y) and z = z2(x , y).
∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
R
∫ [∫ z2
z1
∂Fz
∂zdz
]
dx dy
=
∫
R
∫
[Fzx , y , z2(x , y) − Fzx , y , z1(x , y)]dx dy
R : projection of T on the xy -plane
Projection of area element of the upper segment: nzdS = dx dyProjection of area element of the lower segment: nzdS = −dx dy
Thus,∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
S
∫
FznzdS .
Sum of three such components leads to the result.
![Page 517: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/517.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 517,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Lower and upper segments of S : z = z1(x , y) and z = z2(x , y).
∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
R
∫ [∫ z2
z1
∂Fz
∂zdz
]
dx dy
=
∫
R
∫
[Fzx , y , z2(x , y) − Fzx , y , z1(x , y)]dx dy
R : projection of T on the xy -plane
Projection of area element of the upper segment: nzdS = dx dyProjection of area element of the lower segment: nzdS = −dx dy
Thus,∫ ∫
T
∫
∂Fz
∂zdx dy dz =
∫
S
∫
FznzdS .
Sum of three such components leads to the result.
Extension to arbitrary regions by a suitable subdivision of domain!
![Page 518: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/518.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 518,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Green’s identities (theorem)
Region T and boundary S: as required in premises ofGauss’s theoremφ(x , y , z) and ψ(x , y , z): second order continuous scalarfunctions
∫
S
∫
φ∇ψ · ndS =
∫ ∫
T
∫
(φ∇2ψ +∇φ · ∇ψ)dv
∫
S
∫
(φ∇ψ − ψ∇φ) · ndS =
∫ ∫
T
∫
(φ∇2ψ − ψ∇2φ)dv
Direct consequences of Gauss’s theorem
![Page 519: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/519.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 519,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Green’s identities (theorem)
Region T and boundary S: as required in premises ofGauss’s theoremφ(x , y , z) and ψ(x , y , z): second order continuous scalarfunctions
∫
S
∫
φ∇ψ · ndS =
∫ ∫
T
∫
(φ∇2ψ +∇φ · ∇ψ)dv
∫
S
∫
(φ∇ψ − ψ∇φ) · ndS =
∫ ∫
T
∫
(φ∇2ψ − ψ∇2φ)dv
Direct consequences of Gauss’s theorem
To establish, apply Gauss’s divergence theorem on φ∇ψ, and thenon ψ∇φ as well.
![Page 520: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/520.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 520,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Stokes’s theorem
S: a piecewise smooth surfaceC: boundary, a piecewise smooth simple closed curveF(x , y , z): first order continuous vector function
∮
C
F · dr =
∫
S
∫
curl F · ndS
n: unit normal given by the right hand clasp rule on C
![Page 521: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/521.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 521,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Stokes’s theorem
S: a piecewise smooth surfaceC: boundary, a piecewise smooth simple closed curveF(x , y , z): first order continuous vector function
∮
C
F · dr =
∫
S
∫
curl F · ndS
n: unit normal given by the right hand clasp rule on C
For F(x , y , z) = Fx(x , y , z)i,
∮
C
Fxdx =
∫
S
∫ (
∂Fx
∂zj − ∂Fx
∂yk
)
·ndS =
∫
S
∫ (
∂Fx
∂zny −
∂Fx
∂ynz
)
dS .
![Page 522: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/522.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 522,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Stokes’s theorem
S: a piecewise smooth surfaceC: boundary, a piecewise smooth simple closed curveF(x , y , z): first order continuous vector function
∮
C
F · dr =
∫
S
∫
curl F · ndS
n: unit normal given by the right hand clasp rule on C
For F(x , y , z) = Fx(x , y , z)i,
∮
C
Fxdx =
∫
S
∫ (
∂Fx
∂zj − ∂Fx
∂yk
)
·ndS =
∫
S
∫ (
∂Fx
∂zny −
∂Fx
∂ynz
)
dS .
First, consider a surface S intersected at most once by any lineparallel to a coordinate axis.
![Page 523: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/523.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 523,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Represent S as z = z(x , y) ≡ f (x , y).
Unit normal n = [nx ny nz ]T is proportional to [∂f
∂x∂f∂y− 1]T .
ny = −nz∂z
∂y
![Page 524: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/524.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 524,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Integral Theorems
Represent S as z = z(x , y) ≡ f (x , y).
Unit normal n = [nx ny nz ]T is proportional to [∂f
∂x∂f∂y− 1]T .
ny = −nz∂z
∂y
∫
S
∫ (
∂Fx
∂zny −
∂Fx
∂ynz
)
dS = −∫
S
∫ (
∂Fx
∂y+∂Fx
∂z
∂z
∂y
)
nzdS
Over projection R of S on xy -plane, φ(x , y) = Fx(x , y , z(x , y)).
LHS = −∫
R
∫
∂φ
∂ydx dy =
∮
C ′
φ(x , y)dx =
∮
C
Fxdx
Similar results for Fy(x , y , z)j and Fz(x , y , z)k.
![Page 525: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/525.jpg)
Mathematical Methods in Engineering and Science Scalar and Vector Fields 525,
Differential Operations on Field FunctionsIntegral Operations on Field FunctionsIntegral TheoremsClosure
Points to note
The ‘del’ operator ∇ Gradient, divergence and curl
Composite and second order operators
Line, surface and volume intergals
Green’s, Gauss’s and Stokes’s theorems
Applications in physics (and engineering)
Necessary Exercises: 1,2,3,6,7
![Page 526: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/526.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 526,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Outline
Polynomial EquationsBasic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
![Page 527: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/527.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 527,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Basic Principles
Fundamental theorem of algebra
p(x) = a0xn + a1x
n−1 + a2xn−2 + · · ·+ an−1x + an
has exactly n roots x1, x2, · · · , xn; with
p(x) = a0(x − x1)(x − x2)(x − x3) · · · (x − xn).
In general, roots are complex.
![Page 528: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/528.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 528,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Basic Principles
Fundamental theorem of algebra
p(x) = a0xn + a1x
n−1 + a2xn−2 + · · ·+ an−1x + an
has exactly n roots x1, x2, · · · , xn; with
p(x) = a0(x − x1)(x − x2)(x − x3) · · · (x − xn).
In general, roots are complex.Multiplicity: A root of p(x) with multiplicity k satisfies
p(x) = p′(x) = p′′(x) = · · · = p(k−1)(x) = 0.
![Page 529: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/529.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 529,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Basic Principles
Fundamental theorem of algebra
p(x) = a0xn + a1x
n−1 + a2xn−2 + · · ·+ an−1x + an
has exactly n roots x1, x2, · · · , xn; with
p(x) = a0(x − x1)(x − x2)(x − x3) · · · (x − xn).
In general, roots are complex.Multiplicity: A root of p(x) with multiplicity k satisfies
p(x) = p′(x) = p′′(x) = · · · = p(k−1)(x) = 0.
Descartes’ rule of signs
Bracketing and separation
Synthetic division and deflation
p(x) = f (x)q(x) + r(x)
![Page 530: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/530.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 530,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quadratic equation
ax2 + bx + c = 0 ⇒ x =−b ±
√b2 − 4ac
2a
![Page 531: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/531.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 531,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quadratic equation
ax2 + bx + c = 0 ⇒ x =−b ±
√b2 − 4ac
2a
Method of completing the square:
x2 +b
ax +
(
b
2a
)2
=b2
4a2− c
a⇒
(
x +b
2a
)2
=b2 − 4ac
4a2
![Page 532: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/532.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 532,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quadratic equation
ax2 + bx + c = 0 ⇒ x =−b ±
√b2 − 4ac
2a
Method of completing the square:
x2 +b
ax +
(
b
2a
)2
=b2
4a2− c
a⇒
(
x +b
2a
)2
=b2 − 4ac
4a2
Cubic equations (Cardano):
x3 + ax2 + bx + c = 0
Completing the cube?
![Page 533: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/533.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 533,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quadratic equation
ax2 + bx + c = 0 ⇒ x =−b ±
√b2 − 4ac
2a
Method of completing the square:
x2 +b
ax +
(
b
2a
)2
=b2
4a2− c
a⇒
(
x +b
2a
)2
=b2 − 4ac
4a2
Cubic equations (Cardano):
x3 + ax2 + bx + c = 0
Completing the cube?Substituting y = x + k,
y3 + (a − 3k)y2 + (b − 2ak + 3k2)y + (c − bk + ak2 − k3) = 0.
Choose the shift k = a/3.
![Page 534: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/534.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 534,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
y3 + py + q = 0
![Page 535: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/535.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 535,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
y3 + py + q = 0
Assuming y = u + v , we have y3 = u3 + v3 + 3uv(u + v).
uv = −p/3
u3 + v3 = −q
and hence (u3 − v3)2 = q2 +4p3
27.
Solution:
u3, v3 = −q
2±√
q2
4+
p3
27= A,B (say).
![Page 536: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/536.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 536,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
y3 + py + q = 0
Assuming y = u + v , we have y3 = u3 + v3 + 3uv(u + v).
uv = −p/3
u3 + v3 = −q
and hence (u3 − v3)2 = q2 +4p3
27.
Solution:
u3, v3 = −q
2±√
q2
4+
p3
27= A,B (say).
u = A1,A1ω,A1ω2, and v = B1,B1ω,B1ω
2
y1 = A1 + B1, y2 = A1ω + B1ω2 and y3 = A1ω
2 + B1ω.
At least one of the solutions is real!!
![Page 537: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/537.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 537,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quartic equations (Ferrari)
x4+ax3+bx2+cx+d = 0 ⇒(
x2 +a
2x)2
=
(
a2
4− b
)
x2−cx−d
![Page 538: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/538.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 538,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quartic equations (Ferrari)
x4+ax3+bx2+cx+d = 0 ⇒(
x2 +a
2x)2
=
(
a2
4− b
)
x2−cx−d
For a perfect square,
(
x2 +a
2x +
y
2
)2=
(
a2
4− b + y
)
x2 +(ay
2− c)
x +
(
y2
4− d
)
Under what condition, the new RHS will be a perfect square?
![Page 539: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/539.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 539,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Quartic equations (Ferrari)
x4+ax3+bx2+cx+d = 0 ⇒(
x2 +a
2x)2
=
(
a2
4− b
)
x2−cx−d
For a perfect square,
(
x2 +a
2x +
y
2
)2=
(
a2
4− b + y
)
x2 +(ay
2− c)
x +
(
y2
4− d
)
Under what condition, the new RHS will be a perfect square?
(ay
2− c)2− 4
(
a2
4− b + y
)(
y2
4− d
)
= 0
Resolvent of a quartic:
y3 − by2 + (ac − 4d)y + (4bd − a2d − c2) = 0
![Page 540: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/540.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 540,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Analytical Solution
Procedure
Frame the cubic resolvent.
Solve this cubic equation.
Pick up one solution as y .
Insert this y to form
(
x2 +a
2x +
y
2
)2= (ex + f )2.
Split it into two quadratic equations as
x2 +a
2x +
y
2= ±(ex + f ).
Solve each of the two quadratic equations to obtain a total offour solutions of the original quartic equation.
![Page 541: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/541.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 541,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Analytical solution of the general quintic equation?
![Page 542: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/542.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 542,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Analytical solution of the general quintic equation?
Galois: group theory:
A general quintic, or higher degree, equation is notsolvable by radicals.
![Page 543: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/543.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 543,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Analytical solution of the general quintic equation?
Galois: group theory:
A general quintic, or higher degree, equation is notsolvable by radicals.
General polynomial equations: iterative algorithms
Methods for nonlinear equations
Methods specific to polynomial equations
![Page 544: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/544.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 544,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Analytical solution of the general quintic equation?
Galois: group theory:
A general quintic, or higher degree, equation is notsolvable by radicals.
General polynomial equations: iterative algorithms
Methods for nonlinear equations
Methods specific to polynomial equations
Solution through the companion matrix
Roots of a polynomial are the same as the eigenvalues ofits companion matrix.
Companion matrix:
0 0 · · · 0 −an
1 0 · · · 0 −an−1...
.... . .
......
0 0 · · · 0 −a2
0 0 · · · 1 −a1
![Page 545: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/545.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 545,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Bairstow’s method
to separate out factors of small degree.
Attempt to separate real linear factors?
![Page 546: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/546.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 546,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Bairstow’s method
to separate out factors of small degree.
Attempt to separate real linear factors?
Real quadratic factors
Synthetic division with a guess factor x2 + q1x + q2:
remainder r1x + r2
r = [r1 r2]T is a vector function of q = [q1 q2]
T .
Iterate over (q1, q2) to make (r1, r2) zero.
![Page 547: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/547.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 547,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
General Polynomial Equations
Bairstow’s method
to separate out factors of small degree.
Attempt to separate real linear factors?
Real quadratic factors
Synthetic division with a guess factor x2 + q1x + q2:
remainder r1x + r2
r = [r1 r2]T is a vector function of q = [q1 q2]
T .
Iterate over (q1, q2) to make (r1, r2) zero.
Newton-Raphson (Jacobian based) iteration: see exercise.
![Page 548: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/548.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 548,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Two Simultaneous Equations
p1x2 + q1xy + r1y
2 + u1x + v1y + w1 = 0
p2x2 + q2xy + r2y
2 + u2x + v2y + w2 = 0
Rearranging,
a1x2 + b1x + c1 = 0
a2x2 + b2x + c2 = 0
Cramer’s rule:
x2
b1c2 − b2c1=
−x
a1c2 − a2c1=
1
a1b2 − a2b1
⇒ x = −b1c2 − b2c1
a1c2 − a2c1= − a1c2 − a2c1
a1b2 − a2b1
Consistency condition:
(a1b2 − a2b1)(b1c2 − b2c1)− (a1c2 − a2c1)2 = 0
A 4th degree equation in y
![Page 549: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/549.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 549,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Elimination Methods*
The method operates similarly even if the degrees of the originalequations in y are higher.
What about the degree of the eliminant equation?
![Page 550: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/550.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 550,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Elimination Methods*
The method operates similarly even if the degrees of the originalequations in y are higher.
What about the degree of the eliminant equation?
Two equations in x and y of degrees n1 and n2:x-eliminant is an equation of degree n1n2 in y
Maximum number of solutions:
Bezout number = n1n2
Note: Deficient systems may have less number of solutions.
![Page 551: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/551.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 551,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Elimination Methods*
The method operates similarly even if the degrees of the originalequations in y are higher.
What about the degree of the eliminant equation?
Two equations in x and y of degrees n1 and n2:x-eliminant is an equation of degree n1n2 in y
Maximum number of solutions:
Bezout number = n1n2
Note: Deficient systems may have less number of solutions.
Classical methods of elimination
Sylvester’s dialytic method
Bezout’s method
![Page 552: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/552.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 552,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Advanced Techniques*
Three or more independent equations in as many unknowns?
Cascaded elimination? Objections!
Exploitation of special structures through clever heuristics
(mechanisms kinematics literature)
Grobner basis representation
(algebraic geometry)
Continuation or homotopy method by Morgan
For solving the system f(x) = 0, identify anotherstructurally similar system g(x) = 0 with knownsolutions and construct the parametrized system
h(x) = tf(x) + (1− t)g(x) = 0 for t ∈ [0, 1].
Track each solution from t = 0 to t = 1.
![Page 553: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/553.jpg)
Mathematical Methods in Engineering and Science Polynomial Equations 553,
Basic PrinciplesAnalytical SolutionGeneral Polynomial EquationsTwo Simultaneous EquationsElimination Methods*Advanced Techniques*
Points to note
Roots of cubic and quartic polynomials by the methods ofCardano and Ferrari
For higher degree polynomials, Bairstow’s method: a clever implementation of
Newton-Raphson method for polynomials Eigenvalue problem of a companion matrix
Reduction of a system of polynomial equations in twounknowns by elimination
Necessary Exercises: 1,3,4,6
![Page 554: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/554.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 554,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Outline
Solution of Nonlinear Equations and SystemsMethods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
![Page 555: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/555.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 555,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Algebraic and transcendental equations in the form
f (x) = 0
Practical problem: to find one real root (zero) of f (x)
Example of f (x): x3 − 2x + 5, x3 ln x − sin x + 2, etc.
![Page 556: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/556.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 556,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Algebraic and transcendental equations in the form
f (x) = 0
Practical problem: to find one real root (zero) of f (x)
Example of f (x): x3 − 2x + 5, x3 ln x − sin x + 2, etc.
If f (x) is continuous, then
Bracketing: f (x0)f (x1) < 0⇒ there must be a root of f (x)between x0 and x1.
Bisection: Check the sign of f (x0+x12 ). Replace either x0 or x1
with x0+x12 .
![Page 557: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/557.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 557,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Fixed point iteration
Rearrange f (x) = 0 inthe form x = g(x).
Example:For f (x) = tan x − x3 − 2,possible rearrangements:g1(x) = tan−1(x3 + 2)g2(x) = (tan x − 2)1/3
g3(x) = tan x−2x2
Iteration: xk+1 = g(xk)
![Page 558: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/558.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 558,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Fixed point iteration
Rearrange f (x) = 0 inthe form x = g(x).
Example:For f (x) = tan x − x3 − 2,possible rearrangements:g1(x) = tan−1(x3 + 2)g2(x) = (tan x − 2)1/3
g3(x) = tan x−2x2
Iteration: xk+1 = g(xk)
l
xx
c
q rp
y
y = xw
vu
x
y = g(x)
O
na
eg
f
d
b
m
Figure: Fixed point iteration
If x∗ is the unique solution in interval J and|g ′(x)| ≤ h < 1 in J, then any x0 ∈ J converges to x∗.
![Page 559: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/559.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 559,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Newton-Raphson method
First order Taylor seriesf (x + δx) ≈ f (x) + f ′(x)δxFrom f (xk + δx) = 0,δx = −f (xk)/f ′(xk)Iteration:xk+1 = xk − f (xk)/f ′(xk)
Convergence criterion:|f (x)f ′′(x)| < |f ′(x)|2
![Page 560: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/560.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 560,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Newton-Raphson method
First order Taylor seriesf (x + δx) ≈ f (x) + f ′(x)δxFrom f (xk + δx) = 0,δx = −f (xk)/f ′(xk)Iteration:xk+1 = xk − f (xk)/f ′(xk)
Convergence criterion:|f (x)f ′′(x)| < |f ′(x)|2Draw tangent to f (x).Take its x-intercept.
c
d
e
x0 x
af(x)
Ob f
gx*
Figure: Newton-Raphson method
![Page 561: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/561.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 561,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Newton-Raphson method
First order Taylor seriesf (x + δx) ≈ f (x) + f ′(x)δxFrom f (xk + δx) = 0,δx = −f (xk)/f ′(xk)Iteration:xk+1 = xk − f (xk)/f ′(xk)
Convergence criterion:|f (x)f ′′(x)| < |f ′(x)|2Draw tangent to f (x).Take its x-intercept.
c
d
e
x0 x
af(x)
Ob f
gx*
Figure: Newton-Raphson method
Merit: quadratic speed of convergence: |xk+1 − x∗| = c |xk − x∗|2Demerit: If the starting point is not appropriate,
haphazard wandering, oscillations or outright divergence!
![Page 562: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/562.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 562,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Secant method and method of false position
In the Newton-Raphson formula,
f ′(x) ≈ f (xk )−f (xk−1)xk−xk−1
⇒ xk+1 = xk − xk−xk−1
f (xk)−f (xk−1)f (xk)
Draw the chord orsecant to f (x) through(xk−1, f (xk−1)) and (xk , f (xk)).Take its x-intercept.
![Page 563: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/563.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 563,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Secant method and method of false position
In the Newton-Raphson formula,
f ′(x) ≈ f (xk )−f (xk−1)xk−xk−1
⇒ xk+1 = xk − xk−xk−1
f (xk)−f (xk−1)f (xk)
Draw the chord orsecant to f (x) through(xk−1, f (xk−1)) and (xk , f (xk)).Take its x-intercept.
f(x )1
f(x )0
x0
3xx21xO x
x*
f(x)
Figure: Method of false position
Special case: Maintain a bracket over the root at every iteration.
The method of false position or regula falsi
Convergence is guaranteed!
![Page 564: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/564.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 564,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Quadratic interpolation method or Muller methodEvaluate f (x) at three pointsand model y = a + bx + cx2.Set y = 0 and solve for x .
![Page 565: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/565.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 565,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Quadratic interpolation method or Muller methodEvaluate f (x) at three pointsand model y = a + bx + cx2.Set y = 0 and solve for x .
Inverse quadratic interpolationEvaluate f (x) at three pointsand model x = a + by + cy2.Set y = 0 to get x = a.
![Page 566: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/566.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 566,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Quadratic interpolation method or Muller methodEvaluate f (x) at three pointsand model y = a + bx + cx2.Set y = 0 and solve for x .
Inverse quadratic interpolationEvaluate f (x) at three pointsand model x = a + by + cy2.Set y = 0 to get x = a.
InverseQuadraticInterpolation
x3
1 1(x ,y )
0 0(x ,y )
2 2
x3
y
x(x ,y )
O
InterpolationQuadratic
Figure: Interpolation schemes
![Page 567: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/567.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 567,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Methods for Nonlinear Equations
Quadratic interpolation method or Muller methodEvaluate f (x) at three pointsand model y = a + bx + cx2.Set y = 0 and solve for x .
Inverse quadratic interpolationEvaluate f (x) at three pointsand model x = a + by + cy2.Set y = 0 to get x = a.
InverseQuadraticInterpolation
x3
1 1(x ,y )
0 0(x ,y )
2 2
x3
y
x(x ,y )
O
InterpolationQuadratic
Figure: Interpolation schemes
Van Wijngaarden-Dekker Brent method
maintains the bracket,
uses inverse quadratic interpolation, and
accepts outcome if within bounds, else takes a bisection step.
Opportunistic manoeuvring between a fast method and a safe one!
![Page 568: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/568.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 568,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Systems of Nonlinear Equations
f1(x1, x2, · · · , xn) = 0,
f2(x1, x2, · · · , xn) = 0,
· · · · · · · · · · · ·fn(x1, x2, · · · , xn) = 0.
f(x) = 0
![Page 569: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/569.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 569,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Systems of Nonlinear Equations
f1(x1, x2, · · · , xn) = 0,
f2(x1, x2, · · · , xn) = 0,
· · · · · · · · · · · ·fn(x1, x2, · · · , xn) = 0.
f(x) = 0
Number of variables and number of equations? No bracketing! Fixed point iteration schemes x = g(x)?
![Page 570: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/570.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 570,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Systems of Nonlinear Equations
f1(x1, x2, · · · , xn) = 0,
f2(x1, x2, · · · , xn) = 0,
· · · · · · · · · · · ·fn(x1, x2, · · · , xn) = 0.
f(x) = 0
Number of variables and number of equations? No bracketing! Fixed point iteration schemes x = g(x)?
Newton’s method for systems of equations
f(x + δx) = f(x) +
[
∂f
∂x(x)
]
δx + · · · ≈ f(x) + J(x)δx
⇒ xk+1 = xk − [J(xk )]−1f(xk)
with the usual merits and demerits!
![Page 571: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/571.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 571,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Closure
Modified Newton’s method
xk+1 = xk − αk [J(xk )]−1f(xk)
Broyden’s secant method
Jacobian is not evaluated at every iteration, but getsdeveloped through updates.
![Page 572: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/572.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 572,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Closure
Modified Newton’s method
xk+1 = xk − αk [J(xk )]−1f(xk)
Broyden’s secant method
Jacobian is not evaluated at every iteration, but getsdeveloped through updates.
Optimization-based formulation
Global minimum of the function
‖f(x)‖2 = f 21 + f 2
2 + · · ·+ f 2n
Levenberg-Marquardt method
![Page 573: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/573.jpg)
Mathematical Methods in Engineering and Science Solution of Nonlinear Equations and Systems 573,
Methods for Nonlinear EquationsSystems of Nonlinear EquationsClosure
Points to note
Iteration schemes for solving f (x) = 0
Newton (or Newton-Raphson) iteration for a system ofequations
xk+1 = xk − [J(xk)]−1f(xk)
Optimization formulation of a multi-dimensional root findingproblem
Necessary Exercises: 1,2,3
![Page 574: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/574.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 574,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Outline
Optimization: IntroductionThe Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
![Page 575: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/575.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 575,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
The Methodology of Optimization
Parameters and variables
The statement of the optimization problem
Minimize f (x)subject to g(x) ≤ 0,
h(x) = 0.
Optimization methods
Sensitivity analysis
Optimization problems: unconstrained and constrained
Optimization problems: linear and nonlinear
Single-variable and multi-variable problems
![Page 576: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/576.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 576,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
For a function f (x), a point x∗ is defined as a relative (local)minimum if ∃ ǫ such that f (x) ≥ f (x∗) ∀ x ∈ [x∗ − ǫ, x∗ + ǫ].
x1
f( )x
x2 x4x3 x5xa bO x6
Figure: Schematic of optima of a univariate function
![Page 577: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/577.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 577,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
For a function f (x), a point x∗ is defined as a relative (local)minimum if ∃ ǫ such that f (x) ≥ f (x∗) ∀ x ∈ [x∗ − ǫ, x∗ + ǫ].
x1
f( )x
x2 x4x3 x5xa bO x6
Figure: Schematic of optima of a univariate function
Optimality criteria
First order necessary condition: If x∗ is a local minimum ormaximum point and if f ′(x∗) exists, then f ′(x∗) = 0.
Second order necessary condition: If x∗ is a local minimum pointand f ′′(x∗) exists, then f ′′(x∗) ≥ 0.
Second order sufficient condition: If f ′(x∗) = 0 and f ′′(x∗) > 0then x∗ is a local minimum point.
![Page 578: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/578.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 578,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
Higher order analysis: From Taylor’s series,
∆f = f (x∗ + δx) − f (x∗)
= f ′(x∗)δx +1
2!f ′′(x∗)δx2 +
1
3!f ′′′(x∗)δx3 +
1
4!f iv(x∗)δx4 + · · ·
For an extremum to occur at point x∗, the lowest orderderivative with non-zero value should be of even order.
![Page 579: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/579.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 579,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
Higher order analysis: From Taylor’s series,
∆f = f (x∗ + δx) − f (x∗)
= f ′(x∗)δx +1
2!f ′′(x∗)δx2 +
1
3!f ′′′(x∗)δx3 +
1
4!f iv(x∗)δx4 + · · ·
For an extremum to occur at point x∗, the lowest orderderivative with non-zero value should be of even order.
If f ′(x∗) = 0, then
x∗ is a stationary point, a candidate for an extremum.
Evaluate higher order derivatives till one of them is found tobe non-zero.
If its order is odd, then x∗ is an inflection point. If its order is even, then x∗ is a local minimum or maximum,
as the derivative value is positive or negative, respectively.
![Page 580: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/580.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 580,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
Iterative methods of line searchMethods based on gradient root finding
Newton’s method
xk+1 = xk −f ′(xk)
f ′′(xk)
Secant method
xk+1 = xk −xk − xk−1
f ′(xk)− f ′(xk−1)f ′(xk)
Method of cubic estimationpoint of vanishing gradient of the cubic fit withf (xk−1), f (xk), f ′(xk−1) and f ′(xk)
Method of quadratic estimation
point of vanishing gradient of the quadratic fitthrough three points
Disadvantage: treating all stationary points alike!
![Page 581: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/581.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 581,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
Bracketing:
x1 < x2 < x3 with f (x1) ≥ f (x2) ≤ f (x3)
Exhaustive search method or its variants
![Page 582: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/582.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 582,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Single-Variable Optimization
Bracketing:
x1 < x2 < x3 with f (x1) ≥ f (x2) ≤ f (x3)
Exhaustive search method or its variantsDirect optimization algorithms
Fibonacci search uses a pre-defined number N, of functionevaluations, and the Fibonacci sequence
F0 = 1, F1 = 1, F2 = 2, · · · , Fj = Fj−2 + Fj−1, · · ·
to tighten a bracket with economized number of functionevaluations.
Golden section search uses a constant ratio
τ =
√5− 1
2≈ 0.618,
the golden section ratio, of interval reduction, that isdetermined as the limiting case of N →∞ and the actualnumber of steps is decided by the accuracy desired.
![Page 583: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/583.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 583,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Unconstrained minimization problem
x∗ is called a local minimum of f (x) if ∃ δ such thatf (x) ≥ f (x∗) for all x satisfying ‖x − x∗‖ < δ.
![Page 584: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/584.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 584,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Unconstrained minimization problem
x∗ is called a local minimum of f (x) if ∃ δ such thatf (x) ≥ f (x∗) for all x satisfying ‖x − x∗‖ < δ.
Optimality criteriaFrom Taylor’s series,
f (x) − f (x∗) = [g(x∗)]T δx +1
2δxT [H(x∗)]δx + · · · .
For x∗ to be a local minimum,
necessary condition: g(x∗) = 0 and H(x∗) is positive semi-definite,
sufficient condition: g(x∗) = 0 and H(x∗) is positive definite.
Indefinite Hessian matrix characterizes a saddle point.
![Page 585: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/585.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 585,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
ConvexitySet S ⊆ Rn is a convex set if
∀ x1, x2 ∈ S and α ∈ (0, 1), αx1 + (1− α)x2 ∈ S .
Function f (x) over a convex set S : a convex function if∀ x1, x2 ∈ S and α ∈ (0, 1),
f (αx1 + (1− α)x2) ≤ αf (x1) + (1− α)f (x2).
Chord approximation is an overestimate at intermediate points!
![Page 586: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/586.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 586,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
ConvexitySet S ⊆ Rn is a convex set if
∀ x1, x2 ∈ S and α ∈ (0, 1), αx1 + (1− α)x2 ∈ S .
Function f (x) over a convex set S : a convex function if∀ x1, x2 ∈ S and α ∈ (0, 1),
f (αx1 + (1− α)x2) ≤ αf (x1) + (1− α)f (x2).
Chord approximation is an overestimate at intermediate points!
x1
x2
X1
X2
O
Figure: A convex domain
x2
xf( )
x1f( )
f( )x2
x1 xO
Figure: A convex function
![Page 587: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/587.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 587,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
First order characterization of convexity
From f (αx1 + (1− α)x2) ≤ αf (x1) + (1− α)f (x2),
f (x1)− f (x2) ≥f (x2 + α(x1 − x2))− f (x2)
α.
As α→ 0, f (x1) ≥ f (x2) + [∇f (x2)]T (x1 − x2).
Tangent approximation is an underestimate at intermediate points!
![Page 588: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/588.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 588,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
First order characterization of convexity
From f (αx1 + (1− α)x2) ≤ αf (x1) + (1− α)f (x2),
f (x1)− f (x2) ≥f (x2 + α(x1 − x2))− f (x2)
α.
As α→ 0, f (x1) ≥ f (x2) + [∇f (x2)]T (x1 − x2).
Tangent approximation is an underestimate at intermediate points!
Second order characterization: Hessian is positive semi-definite.
![Page 589: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/589.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 589,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
First order characterization of convexity
From f (αx1 + (1− α)x2) ≤ αf (x1) + (1− α)f (x2),
f (x1)− f (x2) ≥f (x2 + α(x1 − x2))− f (x2)
α.
As α→ 0, f (x1) ≥ f (x2) + [∇f (x2)]T (x1 − x2).
Tangent approximation is an underestimate at intermediate points!
Second order characterization: Hessian is positive semi-definite.
Convex programming problem: convex function over convex set
A local minimum is also a global minimum, and allminima are connected in a convex set.
Note: Convexity is a stronger condition than unimodality!
![Page 590: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/590.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 590,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Quadratic function
q(x) =1
2xTAx + bTx + c
Gradient ∇q(x) = Ax + b and Hessian = A is constant.
![Page 591: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/591.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 591,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Quadratic function
q(x) =1
2xTAx + bTx + c
Gradient ∇q(x) = Ax + b and Hessian = A is constant.
If A is positive definite, then the unique solution of Ax = −bis the only minimum point.
If A is positive semi-definite and −b ∈ Range(A), then theentire subspace of solutions of Ax = −b are global minima.
If A is positive semi-definite but −b /∈ Range(A), then thefunction is unbounded!
![Page 592: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/592.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 592,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Quadratic function
q(x) =1
2xTAx + bTx + c
Gradient ∇q(x) = Ax + b and Hessian = A is constant.
If A is positive definite, then the unique solution of Ax = −bis the only minimum point.
If A is positive semi-definite and −b ∈ Range(A), then theentire subspace of solutions of Ax = −b are global minima.
If A is positive semi-definite but −b /∈ Range(A), then thefunction is unbounded!
Note: A quadratic problem (with positive definite Hessian) acts asa benchmark for optimization algorithms.
![Page 593: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/593.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 593,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Optimization Algorithms
From the current point, move to another point, hopefully better.
![Page 594: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/594.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 594,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Optimization Algorithms
From the current point, move to another point, hopefully better.
Which way to go? How far to go? Which decision is first?
![Page 595: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/595.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 595,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Optimization Algorithms
From the current point, move to another point, hopefully better.
Which way to go? How far to go? Which decision is first?
Strategies and versions of algorithms:
Trust Region: Develop a local quadratic model
f (xk + δx) = f (xk) + [g(xk)]T δx +1
2δxTFkδx,
and minimize it in a small trust region around xk .(Define trust region with dummy boundaries.)
Line search: Identify a descent direction dk and minimize thefunction along it through the univariate function
φ(α) = f (xk + αdk). Exact or accurate line search Inexact or inaccurate line search
Armijo, Goldstein and Wolfe conditions
![Page 596: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/596.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 596,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Conceptual Background of Multivariate Optimization
Convergence of algorithms: notions of guarantee and speed
Global convergence: the ability of an algorithm to approach andconverge to an optimal solution for an arbitraryproblem, starting from an arbitrary point
Practically, a sequence (or even subsequence) ofmonotonically decreasing errors is enough.
Local convergence: the rate/speed of approach, measured by p,where
β = limk→∞
‖xk+1 − x∗‖‖xk − x∗‖p <∞
Linear, quadratic and superlinear rates ofconvergence for p = 1, 2 and intermediate.
Comparison among algorithms with linear ratesof convergence is by the convergence ratio β.
![Page 597: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/597.jpg)
Mathematical Methods in Engineering and Science Optimization: Introduction 597,
The Methodology of OptimizationSingle-Variable OptimizationConceptual Background of Multivariate Optimization
Points to note
Theory and methods of single-variable optimization
Optimality criteria in multivariate optimization
Convexity in optimization
The quadratic function
Trust region
Line search
Global and local convergence
Necessary Exercises: 1,2,5,7,8
![Page 598: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/598.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 598,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Outline
Multivariate OptimizationDirect MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
![Page 599: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/599.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 599,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct Methods
Direct search methods using only function values
Cyclic coordinate search
Rosenbrock’s method
Hooke-Jeeves pattern search
Box’s complex method
Nelder and Mead’s simplex search
Powell’s conjugate directions method
Useful for functions, for which derivative either does not exist at allpoints in the domain or is computationally costly to evaluate.
![Page 600: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/600.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 600,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct Methods
Direct search methods using only function values
Cyclic coordinate search
Rosenbrock’s method
Hooke-Jeeves pattern search
Box’s complex method
Nelder and Mead’s simplex search
Powell’s conjugate directions method
Useful for functions, for which derivative either does not exist at allpoints in the domain or is computationally costly to evaluate.
Note: When derivatives are easily available, gradient-basedalgorithms appear as mainstream methods.
![Page 601: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/601.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 601,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct Methods
Nelder and Mead’s simplex methodSimplex in n-dimensional space: polytope formed by n + 1 vertices
Nelder and Mead’s method iterates over simplices that arenon-degenerate (i.e. enclosing non-zero hypervolume).
![Page 602: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/602.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 602,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct Methods
Nelder and Mead’s simplex methodSimplex in n-dimensional space: polytope formed by n + 1 vertices
Nelder and Mead’s method iterates over simplices that arenon-degenerate (i.e. enclosing non-zero hypervolume).
First, n + 1 suitable points are selected for the starting simplex.
Among vertices of the current simplex, identify the worst point xw ,the best point xb and the second worst point xs .
Need to replace xw with a good point.
![Page 603: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/603.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 603,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct Methods
Nelder and Mead’s simplex methodSimplex in n-dimensional space: polytope formed by n + 1 vertices
Nelder and Mead’s method iterates over simplices that arenon-degenerate (i.e. enclosing non-zero hypervolume).
First, n + 1 suitable points are selected for the starting simplex.
Among vertices of the current simplex, identify the worst point xw ,the best point xb and the second worst point xs .
Need to replace xw with a good point.
Centre of gravity of the face not containing xw :
xc =1
n
n+1∑
i=1,i 6=w
xi
Reflect xw with respect to xc as xr = 2xc − xw . Consider options.
![Page 604: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/604.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 604,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct MethodsDefault xnew = xr .Revision possibilities:
xr
f( )xwf( )xbPositiveContraction Contraction
Negative
xxw = xxrxw xw xr xw xr
x
Expansion Defaultx
newnew
new
sf( )
Figure: Nelder and Mead’s simplex method
![Page 605: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/605.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 605,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Direct MethodsDefault xnew = xr .Revision possibilities:
xr
f( )xwf( )xbPositiveContraction Contraction
Negative
xxw = xxrxw xw xr xw xr
x
Expansion Defaultx
newnew
new
sf( )
Figure: Nelder and Mead’s simplex method
1. For f (xr ) < f (xb), expansion:xnew = xc + α(xc − xw ), α > 1.
2. For f (xr ) ≥ f (xw ), negative contraction:xnew = xc − β(xc − xw ), 0 < β < 1.
3. For f (xs) < f (xr ) < f (xw ), positive contraction:xnew = xc + β(xc − xw ), with 0 < β < 1.
Replace xw with xnew . Continue with new simplex.
![Page 606: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/606.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 606,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
From a point xk , a move through α units in direction dk :
f (xk + αdk) = f (xk) + α[g(xk )]Tdk +O(α2)
Descent direction dk : For α > 0, [g(xk )]Tdk < 0
Direction of steepest descent: dk = −gk [ or dk = −gk/‖gk‖]
![Page 607: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/607.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 607,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
From a point xk , a move through α units in direction dk :
f (xk + αdk) = f (xk) + α[g(xk )]Tdk +O(α2)
Descent direction dk : For α > 0, [g(xk )]Tdk < 0
Direction of steepest descent: dk = −gk [ or dk = −gk/‖gk‖]
Minimizeφ(α) = f (xk + αdk).
Exact line search:
φ′(αk) = [g(xk + αkdk)]Tdk = 0
Search direction tangential to the contour surface at (xk + αkdk).
Note: Next direction dk+1 = −g(xk+1) orthogonal to dk
![Page 608: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/608.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 608,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
Steepest descent algorithm
1. Select a starting point x0, set k = 0 and several parameters:tolerance ǫG on gradient, absolute tolerance ǫA on reductionin function value, relative tolerance ǫR on reduction infunction value and maximum number of iterations M.
2. If ‖gk‖ ≤ ǫG , STOP. Else dk = −gk/‖gk‖.3. Line search: Obtain αk by minimizing φ(α) = f (xk + αdk),α > 0. Update xk+1 = xk + αkdk .
4. If |f (xk+1)− f (xk)| ≤ ǫA + ǫR |f (xk)|,STOP. Else k ← k + 1.
5. If k > M, STOP. Else go to step 2.
![Page 609: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/609.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 609,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
Steepest descent algorithm
1. Select a starting point x0, set k = 0 and several parameters:tolerance ǫG on gradient, absolute tolerance ǫA on reductionin function value, relative tolerance ǫR on reduction infunction value and maximum number of iterations M.
2. If ‖gk‖ ≤ ǫG , STOP. Else dk = −gk/‖gk‖.3. Line search: Obtain αk by minimizing φ(α) = f (xk + αdk),α > 0. Update xk+1 = xk + αkdk .
4. If |f (xk+1)− f (xk)| ≤ ǫA + ǫR |f (xk)|,STOP. Else k ← k + 1.
5. If k > M, STOP. Else go to step 2.
Very good global convergence.
But, why so many “STOPS”?
![Page 610: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/610.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 610,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
Analysis on a quadratic function
For minimizing q(x) = 12x
TAx + bTx, the error function:
E (x) =1
2(x − x∗)TA(x − x∗)
Convergence ratio: E(xk+1)E(xk) ≤
(
κ(A)−1κ(A)+1
)2
Local convergence is poor.
![Page 611: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/611.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 611,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Steepest Descent (Cauchy) Method
Analysis on a quadratic function
For minimizing q(x) = 12x
TAx + bTx, the error function:
E (x) =1
2(x − x∗)TA(x − x∗)
Convergence ratio: E(xk+1)E(xk) ≤
(
κ(A)−1κ(A)+1
)2
Local convergence is poor.
Importance of steepest descent method
conceptual understanding
initial iterations in a completely new problem
spacer steps in other sophisticated methods
Re-scaling of the problem through change of variables?
![Page 612: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/612.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 612,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Newton’s MethodSecond order approximation of a function:
f (x) ≈ f (xk) + [g(xk)]T (x − xk) +1
2(x − xk)TH(xk)(x − xk)
Vanishing of gradient
g(x) ≈ g(xk) + H(xk)(x − xk)
gives the iteration formula
xk+1 = xk − [H(xk )]−1g(xk).
Excellent local convergence property!
‖xk+1 − x∗‖‖xk − x∗‖2 ≤ β
![Page 613: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/613.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 613,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Newton’s MethodSecond order approximation of a function:
f (x) ≈ f (xk) + [g(xk)]T (x − xk) +1
2(x − xk)TH(xk)(x − xk)
Vanishing of gradient
g(x) ≈ g(xk) + H(xk)(x − xk)
gives the iteration formula
xk+1 = xk − [H(xk )]−1g(xk).
Excellent local convergence property!
‖xk+1 − x∗‖‖xk − x∗‖2 ≤ β
Caution: Does not have global convergence.
If H(xk ) is positive definite then dk = −[H(xk)]−1g(xk)is a descent direction.
![Page 614: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/614.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 614,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Newton’s Method
Modified Newton’s method
Replace the Hessian by Fk = H(xk) + γI .
Replace full Newton’s step by a line search.
Algorithm
1. Select x0, tolerance ǫ and δ > 0. Set k = 0.
2. Evaluate gk = g(xk) and H(xk).Choose γ, find Fk = H(xk) + γI , solve Fkdk = −gk for dk .
3. Line search: obtain αk to minimize φ(α) = f (xk + αdk).Update xk+1 = xk + αkdk .
4. Check convergence: If |f (xk+1)− f (xk)| < ǫ, STOP.Else, k ← k + 1 and go to step 2.
![Page 615: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/615.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 615,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Hybrid (Levenberg-Marquardt) Method
Methods of deflected gradients
xk+1 = xk − αk [Mk ]gk
identity matrix in place of Mk : steepest descent step
Mk = F−1k : step of modified Newton’s method
Mk = [H(xk )]−1 and αk = 1: pure Newton’s step
![Page 616: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/616.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 616,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Hybrid (Levenberg-Marquardt) Method
Methods of deflected gradients
xk+1 = xk − αk [Mk ]gk
identity matrix in place of Mk : steepest descent step
Mk = F−1k : step of modified Newton’s method
Mk = [H(xk )]−1 and αk = 1: pure Newton’s step
In Mk = [H(xk ) + λk I ]−1, tune parameter λk over iterations.
Initial value of λ: large enough to favour steepest descenttrend
Improvement in an iteration: λ reduced by a factor
Increase in function value: step rejected and λ increased
Opportunism systematized!
![Page 617: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/617.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 617,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Hybrid (Levenberg-Marquardt) Method
Methods of deflected gradients
xk+1 = xk − αk [Mk ]gk
identity matrix in place of Mk : steepest descent step
Mk = F−1k : step of modified Newton’s method
Mk = [H(xk )]−1 and αk = 1: pure Newton’s step
In Mk = [H(xk ) + λk I ]−1, tune parameter λk over iterations.
Initial value of λ: large enough to favour steepest descenttrend
Improvement in an iteration: λ reduced by a factor
Increase in function value: step rejected and λ increased
Opportunism systematized!
Note: Cost of evaluating the Hessian remains a bottleneck.Useful for problems where Hessian estimates come cheap!
![Page 618: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/618.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 618,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Least Square Problems
Linear least square problem:
y(θ) = x1φ1(θ) + x2φ2(θ) + · · ·+ xnφn(θ)
For measured values y(θi ) = yi ,
ei =n∑
k=1
xkφk(θi)− yi = [Φ(θi )]T x − yi .
Error vector: e = Ax − y
Last square fit:
Minimize E = 12
∑
i e2i = 1
2eTe
Pseudoinverse solution and its variants
![Page 619: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/619.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 619,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Least Square Problems
Nonlinear least square problem
For model function in the form
y(θ) = f (θ, x) = f (θ, x1, x2, · · · , xn),
square error function
E (x) =1
2eTe =
1
2
∑
i
e2i =
1
2
∑
i
[f (θi , x) − yi ]2
![Page 620: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/620.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 620,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Least Square Problems
Nonlinear least square problem
For model function in the form
y(θ) = f (θ, x) = f (θ, x1, x2, · · · , xn),
square error function
E (x) =1
2eTe =
1
2
∑
i
e2i =
1
2
∑
i
[f (θi , x) − yi ]2
Gradient: g(x) = ∇E (x) =∑
i [f (θi , x) − yi ]∇f (θi , x) = JTe
Hessian: H(x) = ∂2
∂x2 E (x) = JTJ +∑
i ei∂2
∂x2 f (θi , x) ≈ JTJ
![Page 621: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/621.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 621,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Least Square Problems
Nonlinear least square problem
For model function in the form
y(θ) = f (θ, x) = f (θ, x1, x2, · · · , xn),
square error function
E (x) =1
2eTe =
1
2
∑
i
e2i =
1
2
∑
i
[f (θi , x) − yi ]2
Gradient: g(x) = ∇E (x) =∑
i [f (θi , x) − yi ]∇f (θi , x) = JTe
Hessian: H(x) = ∂2
∂x2 E (x) = JTJ +∑
i ei∂2
∂x2 f (θi , x) ≈ JTJ
Combining a modified form λ diag(JTJ) δx = −g(x) of steepestdescent formula with Newton’s formula,
Levenberg-Marquardt step: [JTJ + λ diag(JTJ)]δx = −g(x)
![Page 622: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/622.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 622,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Least Square Problems
Levenberg-Marquardt algorithm
1. Select x0, evaluate E (x0). Select tolerance ǫ, initial λ and itsupdate factor. Set k = 0.
2. Evaluate gk and Hk = JTJ + λ diag(JTJ).Solve Hkδx = −gk . Evaluate E (xk + δx).
3. If |E (xk + δx) − E (xk)| < ǫ, STOP.
4. If E (xk + δx) < E (xk), then decrease λ,update xk+1 = xk + δx, k ← k + 1.Else increase λ.
5. Go to step 2.
Professional procedure for nonlinear least square problems and alsofor solving systems of nonlinear equations in the form h(x) = 0.
![Page 623: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/623.jpg)
Mathematical Methods in Engineering and Science Multivariate Optimization 623,
Direct MethodsSteepest Descent (Cauchy) MethodNewton’s MethodHybrid (Levenberg-Marquardt) MethodLeast Square Problems
Points to note
Simplex method of Nelder and Mead
Steepest descent method with its global convergence
Newton’s method for fast local convergence
Levenberg-Marquardt method for equation solving and leastsquares
Necessary Exercises: 1,2,3,4,5,6
![Page 624: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/624.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 624,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Outline
Methods of Nonlinear Optimization*Conjugate Direction MethodsQuasi-Newton MethodsClosure
![Page 625: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/625.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 625,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Conjugacy of directions:
Two vectors d1 and d2 are mutually conjugate withrespect to a symmetric matrix A, if dT
1 Ad2 = 0.
Linear independence of conjugate directions:
Conjugate directions with respect to a positive definitematrix are linearly independent.
![Page 626: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/626.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 626,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Conjugacy of directions:
Two vectors d1 and d2 are mutually conjugate withrespect to a symmetric matrix A, if dT
1 Ad2 = 0.
Linear independence of conjugate directions:
Conjugate directions with respect to a positive definitematrix are linearly independent.
Expanding subspace property: In Rn, with conjugate vectorsd0,d1, · · · ,dn−1 with respect to symmetric positive definite A,for any x0 ∈ Rn, the sequence x0, x1, x2, · · · , xn generated as
xk+1 = xk + αkdk , with αk = − gTk dk
dTk Adk
,
where gk = Axk + b, has the property that
xk minimizes q(x) = 12x
TAx + bTx on the linexk−1 + αdk−1, as well as on the linear variety x0 + Bk ,where Bk is the span of d0, d1, · · · , dk−1.
![Page 627: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/627.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 627,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Question: How to find a set of n conjugate directions?
Gram-Schmidt procedure is a poor option!
![Page 628: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/628.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 628,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Question: How to find a set of n conjugate directions?
Gram-Schmidt procedure is a poor option!
Conjugate gradient method
Starting from d0 = −g0,
dk+1 = −gk+1 + βkdk
Imposing the condition of conjugacy of dk+1 with dk ,
βk =gT
k+1Adk
dTk Adk
=gT
k+1(gk+1 − gk)
αkdTk Adk
Resulting dk+1 conjugate to all the earlier directions, fora quadratic problem.
![Page 629: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/629.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 629,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Using k in place of k + 1 in the formula for dk+1,
dk = −gk + βk−1dk−1
⇒ gTk dk = −gT
k gk and αk =gT
k gk
dTk Adk
![Page 630: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/630.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 630,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Using k in place of k + 1 in the formula for dk+1,
dk = −gk + βk−1dk−1
⇒ gTk dk = −gT
k gk and αk =gT
k gk
dTk Adk
Polak-Ribiere formula:
βk =gT
k+1(gk+1 − gk)
gTk gk
No need to know A!
![Page 631: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/631.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 631,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Using k in place of k + 1 in the formula for dk+1,
dk = −gk + βk−1dk−1
⇒ gTk dk = −gT
k gk and αk =gT
k gk
dTk Adk
Polak-Ribiere formula:
βk =gT
k+1(gk+1 − gk)
gTk gk
No need to know A!Further,
gTk+1dk = 0 ⇒ gT
k+1gk = βk−1(gTk + αkd
Tk A)dk−1 = 0.
Fletcher-Reeves formula:
βk =gT
k+1gk+1
gTk gk
![Page 632: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/632.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 632,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Extension to general (non-quadratic) functions
Varying Hessian A: determine the step size by line search. After n steps, minimum not attained.
But, gTk dk = −gT
k gk implies guaranteed descent.Globally convergent, with superlinear rate of convergence.
What to do after n steps? Restart or continue?
![Page 633: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/633.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 633,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Extension to general (non-quadratic) functions
Varying Hessian A: determine the step size by line search. After n steps, minimum not attained.
But, gTk dk = −gT
k gk implies guaranteed descent.Globally convergent, with superlinear rate of convergence.
What to do after n steps? Restart or continue?
Algorithm
1. Select x0 and tolerances ǫG , ǫD . Evaluate g0 = ∇f (x0).2. Set k = 0 and dk = −gk .3. Line search: find αk ; update xk+1 = xk + αkdk .4. Evaluate gk+1 = ∇f (xk+1). If ‖gk+1‖ ≤ ǫG , STOP.
5. Find βk =gT
k+1(gk+1−gk)
gTkgk
(Polak-Ribiere)
or βk =gT
k+1gk+1
gTkgk
(Fletcher-Reeves).
Obtain dk+1 = −gk+1 + βkdk .
6. If 1−∣
∣
∣
dTkdk+1
‖dk‖ ‖dk+1‖
∣
∣
∣ < ǫD , reset g0 = gk+1and go to step 2.
Else, k ← k + 1 and go to step 3.
![Page 634: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/634.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 634,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Powell’s conjugate direction methodFor q(x) = 1
2xTAx + bT x, suppose
x1 = xA + α1d such that dTg1 = 0 andx2 = xB + α2d such that dTg2 = 0.
Then, dTA(x2 − x1) = dT (g2 − g1) = 0.
![Page 635: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/635.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 635,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Powell’s conjugate direction methodFor q(x) = 1
2xTAx + bT x, suppose
x1 = xA + α1d such that dTg1 = 0 andx2 = xB + α2d such that dTg2 = 0.
Then, dTA(x2 − x1) = dT (g2 − g1) = 0.
Parallel subspace property: In Rn, consider two parallellinear varieties S1 = v1 + Bk and S2 = v2 + Bk , withBk = d1,d2, · · · ,dk, k < n.If x1 and x2
minimize q(x) = 12x
TAx+bTx on S1 and S2, respectively,
then x2 − x1 is conjugate to d1, d2, · · · , dk .
Assumptions imply g1,g2 ⊥ Bk and hence
(g2−g1) ⊥ Bk ⇒ dTi A(x2−x1) = dT
i (g2−g1) = 0 for i = 1, 2, · · · , k.
![Page 636: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/636.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 636,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
Algoithm
1. Select x0, ǫ and a set of n linearly independent (preferablynormalized) directions d1, d2, · · · , dn; possibly di = ei .
2. Line search along dn and update x1 = x0 + αdn; set k = 1.
3. Line searches along d1, d2, · · · , dn in sequence to obtainz = xk +
∑nj=1 αjdj .
4. New conjugate direction d = z − xk . If ‖d‖ < ǫ, STOP.
5. Reassign directions dj ← dj+1 for j = 1, 2, · · · , (n − 1) anddn = d/‖d‖.(Old d1 gets discarded at this step.)
6. Line search and update xk+1 = z + αdn; set k ← k + 1 andgo to step 3.
![Page 637: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/637.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 637,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
x0-x1 and b-z1: x1-z1 is conjugate to b-z1. b-z1-x2 and c-d-z2: c-d , d-z2 and x2-z2 are mutually
conjugate.
x1
x0
x1
x3
x2
x3
z2
x2
z1
ba
d
c
Figure: Schematic of Powell’s conjugate direction method
![Page 638: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/638.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 638,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Conjugate Direction Methods
x0-x1 and b-z1: x1-z1 is conjugate to b-z1. b-z1-x2 and c-d-z2: c-d , d-z2 and x2-z2 are mutually
conjugate.
x1
x0
x1
x3
x2
x3
z2
x2
z1
ba
d
c
Figure: Schematic of Powell’s conjugate direction method
Performance of Powell’s method approaches that of theconjugate gradient method!
![Page 639: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/639.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 639,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Variable metric methods
attempt to construct the inverse Hessian Bk .
pk = xk+1 − xk and qk = gk+1 − gk ⇒ qk ≈ Hpk
With n such steps, B = PQ−1: update and construct Bk ≈ H−1.
![Page 640: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/640.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 640,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Variable metric methods
attempt to construct the inverse Hessian Bk .
pk = xk+1 − xk and qk = gk+1 − gk ⇒ qk ≈ Hpk
With n such steps, B = PQ−1: update and construct Bk ≈ H−1.Rank one correction: Bk+1 = Bk + akzkz
Tk ?
![Page 641: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/641.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 641,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Variable metric methods
attempt to construct the inverse Hessian Bk .
pk = xk+1 − xk and qk = gk+1 − gk ⇒ qk ≈ Hpk
With n such steps, B = PQ−1: update and construct Bk ≈ H−1.Rank one correction: Bk+1 = Bk + akzkz
Tk ?
Rank two correction:
Bk+1 = Bk + akzkzTk + bkwkw
Tk
Davidon-Fletcher-Powell (DFP) method
![Page 642: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/642.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 642,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Variable metric methods
attempt to construct the inverse Hessian Bk .
pk = xk+1 − xk and qk = gk+1 − gk ⇒ qk ≈ Hpk
With n such steps, B = PQ−1: update and construct Bk ≈ H−1.Rank one correction: Bk+1 = Bk + akzkz
Tk ?
Rank two correction:
Bk+1 = Bk + akzkzTk + bkwkw
Tk
Davidon-Fletcher-Powell (DFP) method
Select x0, tolerance ǫ and B0 = In. For k = 0, 1, 2, · · · , dk = −Bkgk . Line search for αk ; update pk = αkdk , xk+1 = xk + pk ,
qk = gk+1 − gk . If ‖pk‖ < ǫ or ‖qk‖ < ǫ, STOP.
Rank two correction: BDFPk+1 = Bk +
pkpTk
pTkqk− BkqkqT
kBk
qTkBkqk
.
![Page 643: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/643.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 643,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Properties of DFP iterations:
1. If Bk is symmetric and positive definite, then so is Bk+1.
2. For quadratic function with positive definite Hessian H,
pTi Hpj = 0 for 0 ≤ i < j ≤ k,
and Bk+1Hpi = pi for 0 ≤ i ≤ k.
![Page 644: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/644.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 644,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Properties of DFP iterations:
1. If Bk is symmetric and positive definite, then so is Bk+1.
2. For quadratic function with positive definite Hessian H,
pTi Hpj = 0 for 0 ≤ i < j ≤ k,
and Bk+1Hpi = pi for 0 ≤ i ≤ k.
Implications:
1. Positive definiteness of inverse Hessian estimate is never lost.
2. Successive search directions are conjugate directions.
3. With B0 = I, the algorithm is a conjugate gradient method.
4. For a quadratic problem, the inverse Hessian gets completelyconstructed after n steps.
![Page 645: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/645.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 645,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Quasi-Newton Methods
Properties of DFP iterations:
1. If Bk is symmetric and positive definite, then so is Bk+1.
2. For quadratic function with positive definite Hessian H,
pTi Hpj = 0 for 0 ≤ i < j ≤ k,
and Bk+1Hpi = pi for 0 ≤ i ≤ k.
Implications:
1. Positive definiteness of inverse Hessian estimate is never lost.
2. Successive search directions are conjugate directions.
3. With B0 = I, the algorithm is a conjugate gradient method.
4. For a quadratic problem, the inverse Hessian gets completelyconstructed after n steps.
Variants: Broyden-Fletcher-Goldfarb-Shanno (BFGS)method and the Broyden family of methods
![Page 646: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/646.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 646,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Closure
Table 23.1: Summary of performance of optimization methods
Cauchy Newton Levenberg-Marquardt DFP/BFGS FR/PR Powell(Steepest (Hybrid) (Quasi-Newton) (Conjugate (DirectionDescent) (Deflected Gradient) (Variable Metric) Gradient) Set)
For QuadraticProblems:
Convergence steps N 1 N n n n2
Indefinite Unknown
Evaluations Nf 2f Nf (n+ 1)f (n+ 1)f n2fNg 2g Ng (n+ 1)g (n+ 1)g
1H NH
Equivalent functionevaluations N(2n+ 1) 2n2 + 2n+ 1 N(2n2 + 1) 2n2 + 3n+ 1 2n2 + 3n+ 1 n2
Line searches N 0 N or 0 n n n2
Storage Vector Matrix Matrix Matrix Vector MatrixPerformance in
general problems Slow Risky Costly Flexible Good OkayPractically good for Unknown Good NL Eqn. systems Bad Large Small
start-up functions NL least squares functions problems problems
![Page 647: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/647.jpg)
Mathematical Methods in Engineering and Science Methods of Nonlinear Optimization* 647,
Conjugate Direction MethodsQuasi-Newton MethodsClosure
Points to note
Conjugate directions and the expanding subspace property
Conjugate gradient method
Powell-Smith direction set method
The quasi-Newton concept in professional optimization
Necessary Exercises: 1,2,3
![Page 648: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/648.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 648,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Outline
Constrained OptimizationConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
![Page 649: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/649.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 649,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
ConstraintsConstrained optimization problem:
Minimize f (x)subject to gi (x) ≤ 0 for i = 1, 2, · · · , l , or g(x) ≤ 0;
and hj(x) = 0 for j = 1, 2, · · · ,m, or h(x) = 0.
Conceptually, “minimize f (x), x ∈ Ω”.
![Page 650: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/650.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 650,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
ConstraintsConstrained optimization problem:
Minimize f (x)subject to gi (x) ≤ 0 for i = 1, 2, · · · , l , or g(x) ≤ 0;
and hj(x) = 0 for j = 1, 2, · · · ,m, or h(x) = 0.
Conceptually, “minimize f (x), x ∈ Ω”.
Equality constraints reduce the domain to a surface or a manifold,possessing a tangent plane at every point.
![Page 651: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/651.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 651,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
ConstraintsConstrained optimization problem:
Minimize f (x)subject to gi (x) ≤ 0 for i = 1, 2, · · · , l , or g(x) ≤ 0;
and hj(x) = 0 for j = 1, 2, · · · ,m, or h(x) = 0.
Conceptually, “minimize f (x), x ∈ Ω”.
Equality constraints reduce the domain to a surface or a manifold,possessing a tangent plane at every point.
Gradient of the vector function h(x):
∇h(x) ≡ [∇h1(x) ∇h2(x) · · · ∇hm(x)] ≡
∂hT
∂x1
∂hT
∂x2...
∂hT
∂xn
,
related to the usual Jacobian as Jh(x) = ∂h∂x = [∇h(x)]T .
![Page 652: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/652.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 652,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Constraints
Constraint qualification
∇h1(x), ∇h2(x) etc are linearly independent, i.e. ∇h(x) isfull-rank.
If a feasible point x0, with h(x0) = 0, satisfies the constraintqualification condition, we call it a regular point.
![Page 653: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/653.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 653,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Constraints
Constraint qualification
∇h1(x), ∇h2(x) etc are linearly independent, i.e. ∇h(x) isfull-rank.
If a feasible point x0, with h(x0) = 0, satisfies the constraintqualification condition, we call it a regular point.
At a regular feasible point x0, tangent plane
M = y : [∇h(x0)]Ty = 0
gives the collection of feasible directions.
![Page 654: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/654.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 654,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Constraints
Constraint qualification
∇h1(x), ∇h2(x) etc are linearly independent, i.e. ∇h(x) isfull-rank.
If a feasible point x0, with h(x0) = 0, satisfies the constraintqualification condition, we call it a regular point.
At a regular feasible point x0, tangent plane
M = y : [∇h(x0)]Ty = 0
gives the collection of feasible directions.
Equality constraints reduce the dimension of the problem.
Variable elimination?
![Page 655: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/655.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 655,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Constraints
Active inequality constraints gi (x0) = 0:
included among hj(x0)
for the tangent plane.
Cone of feasible directions:
[∇h(x0)]Td = 0 and [∇gi (x0)]
Td ≤ 0 for i ∈ I
where I is the set of indices of active inequality constraints.
![Page 656: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/656.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 656,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Constraints
Active inequality constraints gi (x0) = 0:
included among hj(x0)
for the tangent plane.
Cone of feasible directions:
[∇h(x0)]Td = 0 and [∇gi (x0)]
Td ≤ 0 for i ∈ I
where I is the set of indices of active inequality constraints.
Handling inequality constraints:
Active set strategy maintains a list of active constraints,keeps checking at every step for a change of scenario andupdates the list by inclusions and exclusions.
Slack variable strategy replaces all the inequality constraintsby equality constraints as gi (x) + xn+i = 0 with the inclusionof non-negative slack variables (xn+i ).
![Page 657: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/657.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 657,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Suppose x∗ is a regular point with
active inequality constraints: g(a)(x) ≤ 0
inactive constraints: g(i)(x) ≤ 0
Columns of ∇h(x∗) and ∇g(a)(x∗): basis for orthogonalcomplement of the tangent plane
![Page 658: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/658.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 658,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Suppose x∗ is a regular point with
active inequality constraints: g(a)(x) ≤ 0
inactive constraints: g(i)(x) ≤ 0
Columns of ∇h(x∗) and ∇g(a)(x∗): basis for orthogonalcomplement of the tangent plane
Basis of the tangent plane: D = [d1 d2 · · · dk ]
Then, [D ∇h(x∗) ∇g(a)(x∗)]: basis of Rn
![Page 659: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/659.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 659,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Suppose x∗ is a regular point with
active inequality constraints: g(a)(x) ≤ 0
inactive constraints: g(i)(x) ≤ 0
Columns of ∇h(x∗) and ∇g(a)(x∗): basis for orthogonalcomplement of the tangent plane
Basis of the tangent plane: D = [d1 d2 · · · dk ]
Then, [D ∇h(x∗) ∇g(a)(x∗)]: basis of Rn
Now, −∇f (x∗) is a vector in Rn.
−∇f (x∗) = [D ∇h(x∗) ∇g(a)(x∗)]
zλ
µ(a)
with unique z, λ and µ(a) for a given ∇f (x∗).
![Page 660: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/660.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 660,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Suppose x∗ is a regular point with
active inequality constraints: g(a)(x) ≤ 0
inactive constraints: g(i)(x) ≤ 0
Columns of ∇h(x∗) and ∇g(a)(x∗): basis for orthogonalcomplement of the tangent plane
Basis of the tangent plane: D = [d1 d2 · · · dk ]
Then, [D ∇h(x∗) ∇g(a)(x∗)]: basis of Rn
Now, −∇f (x∗) is a vector in Rn.
−∇f (x∗) = [D ∇h(x∗) ∇g(a)(x∗)]
zλ
µ(a)
with unique z, λ and µ(a) for a given ∇f (x∗).
What can you say if x∗ is a solution to the NLP problem?
![Page 661: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/661.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 661,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Components of ∇f (x∗) in the tangent plane must be zero.
z = 0 ⇒ −∇f (x∗) = [∇h(x∗)]λ + [∇g(a)(x∗)]µ(a)
For inactive constraints, insisting on µ(i) = 0,
−∇f (x∗) = [∇h(x∗)]λ + [∇g(a)(x∗) ∇g(i)(x∗)]
[
µ(a)
µ(i)
]
,
or∇f (x∗) + [∇h(x∗)]λ + [∇g(x∗)]µ = 0
where g(x) =
[
g(a)(x)
g(i)(x)
]
and µ =
[
µ(a)
µ(i)
]
.
![Page 662: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/662.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 662,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Components of ∇f (x∗) in the tangent plane must be zero.
z = 0 ⇒ −∇f (x∗) = [∇h(x∗)]λ + [∇g(a)(x∗)]µ(a)
For inactive constraints, insisting on µ(i) = 0,
−∇f (x∗) = [∇h(x∗)]λ + [∇g(a)(x∗) ∇g(i)(x∗)]
[
µ(a)
µ(i)
]
,
or∇f (x∗) + [∇h(x∗)]λ + [∇g(x∗)]µ = 0
where g(x) =
[
g(a)(x)
g(i)(x)
]
and µ =
[
µ(a)
µ(i)
]
.
Notice: g(a)(x∗) = 0 and µ(i) = 0 ⇒ µigi (x∗) = 0 ∀ i , or
µTg(x∗) = 0.
Now, components in g(x) are free to appear in any order.
![Page 663: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/663.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 663,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Finally, what about the feasible directions in the cone?
![Page 664: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/664.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 664,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Finally, what about the feasible directions in the cone?
Answer: Negative gradient −∇f (x∗) can have no component
towards decreasing g(a)i (x), i.e. µ
(a)i ≥ 0, ∀ i .
Combining it with µ(i)i = 0, µ ≥ 0.
![Page 665: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/665.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 665,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Finally, what about the feasible directions in the cone?
Answer: Negative gradient −∇f (x∗) can have no component
towards decreasing g(a)i (x), i.e. µ
(a)i ≥ 0, ∀ i .
Combining it with µ(i)i = 0, µ ≥ 0.
First order necessary conditions or Karusch-Kuhn-Tucker(KKT) conditions: If x∗ is a regular point of the constraints anda solution to the NLP problem, then there exist Lagrangemultiplier vectors, λ and µ, such that
Optimality: ∇f (x∗) + [∇h(x∗)]λ + [∇g(x∗)]µ = 0, µ ≥ 0;Feasibility: h(x∗) = 0, g(x∗) ≤ 0;Complementarity: µTg(x∗) = 0.
![Page 666: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/666.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 666,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Finally, what about the feasible directions in the cone?
Answer: Negative gradient −∇f (x∗) can have no component
towards decreasing g(a)i (x), i.e. µ
(a)i ≥ 0, ∀ i .
Combining it with µ(i)i = 0, µ ≥ 0.
First order necessary conditions or Karusch-Kuhn-Tucker(KKT) conditions: If x∗ is a regular point of the constraints anda solution to the NLP problem, then there exist Lagrangemultiplier vectors, λ and µ, such that
Optimality: ∇f (x∗) + [∇h(x∗)]λ + [∇g(x∗)]µ = 0, µ ≥ 0;Feasibility: h(x∗) = 0, g(x∗) ≤ 0;Complementarity: µTg(x∗) = 0.
Convex programming problem: Convex objective function f (x)and convex domain (convex gi (x) and linear hj(x)):
KKT conditions are sufficient as well!
![Page 667: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/667.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 667,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Lagrangian function:
L(x,λ,µ) = f (x) + λTh(x) + µTg(x)
Necessary conditions for a stationary point of the Lagrangian:
∇xL = 0, ∇λL = 0
![Page 668: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/668.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 668,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Lagrangian function:
L(x,λ,µ) = f (x) + λTh(x) + µTg(x)
Necessary conditions for a stationary point of the Lagrangian:
∇xL = 0, ∇λL = 0
Second order conditionsConsider curve z(t) in the tangent plane with z(0) = x∗.
d2
dt2f (z(t))
∣
∣
∣
∣
t=0
=d
dt[∇f (z(t))T z(t)]
∣
∣
∣
∣
t=0
= z(0)TH(x∗)z(0) + [∇f (x∗)]T z(0) ≥ 0
Similarly, from hj(z(t)) = 0,
z(0)THhj(x∗)z(0) + [∇hj(x
∗)]T z(0) = 0.
![Page 669: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/669.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 669,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Including contributions from all active constraints,
d2
dt2f (z(t))
∣
∣
∣
∣
t=0
= z(0)THL(x∗)z(0) + [∇xL(x∗,λ,µ)]T z(0) ≥ 0,
where HL(x) = ∂2L∂x2 = H(x) +
∑
j λjHhj(x) +
∑
i µiHgi(x).
![Page 670: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/670.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 670,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Including contributions from all active constraints,
d2
dt2f (z(t))
∣
∣
∣
∣
t=0
= z(0)THL(x∗)z(0) + [∇xL(x∗,λ,µ)]T z(0) ≥ 0,
where HL(x) = ∂2L∂x2 = H(x) +
∑
j λjHhj(x) +
∑
i µiHgi(x).
First order necessary condition makes the second term vanish!
Second order necessary condition:
The Hessian matrix of the Lagrangian function is positivesemi-definite on the tangent planeM.
Sufficient condition: ∇xL = 0 and HL(x) positive definite on M.
Restriction of the mapping HL(x∗) : Rn → Rn on subspace M?
![Page 671: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/671.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 671,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Take y ∈M, operate HL(x∗) on it, project the image back toM.
Restricted mapping LM :M→M
Question: Matrix representation for LM of size (n−m)× (n−m)?
![Page 672: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/672.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 672,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Optimality Criteria
Take y ∈M, operate HL(x∗) on it, project the image back toM.
Restricted mapping LM :M→M
Question: Matrix representation for LM of size (n−m)× (n−m)?
Select local orthonormal basis D ∈ Rn×(n−m) for M.
For arbitrary z ∈ Rn−m, map y = Dz ∈ Rn as HLy = HLDz.
Its component along di : dTi HLDz
Hence, projection back on M:
LMz = DTHLDz,
The (n −m)× (n −m) matrix LM = DTHLD: the restriction!
Second order necessary/sufficient condition: LM p.s.d./p.d.
![Page 673: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/673.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 673,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Sensitivity
Suppose original objective and constraint functions as
f (x,p), g(x,p) and h(x,p)
By choosing parameters (p), we arrive at x∗. Call it x∗(p).
Question: How does f (x∗(p),p) depend on p?
![Page 674: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/674.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 674,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Sensitivity
Suppose original objective and constraint functions as
f (x,p), g(x,p) and h(x,p)
By choosing parameters (p), we arrive at x∗. Call it x∗(p).
Question: How does f (x∗(p),p) depend on p?
Total gradients
∇pf (x∗(p),p) = ∇px∗(p)∇x f (x∗,p) +∇pf (x∗,p),
∇ph(x∗(p),p) = ∇px∗(p)∇xh(x∗,p) +∇ph(x∗,p) = 0,
and similarly for g(x∗(p),p).
In view of ∇xL = 0, from KKT conditions,
∇pf (x∗(p),p) = ∇pf (x∗,p) + [∇ph(x∗,p)]λ + [∇pg(x∗,p)]µ
![Page 675: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/675.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 675,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Sensitivity
Sensitivity to constraintsIn particular, in a revised problem, with h(x) = c and g(x) ≤ d,using p = c,
∇pf (x∗,p) = 0, ∇ph(x∗,p) = −I and ∇pg(x∗,p) = 0.
∇c f (x∗(p),p) = −λ
Similarly, using p = d, we get ∇d f (x∗(p),p) = −µ.
![Page 676: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/676.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 676,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Sensitivity
Sensitivity to constraintsIn particular, in a revised problem, with h(x) = c and g(x) ≤ d,using p = c,
∇pf (x∗,p) = 0, ∇ph(x∗,p) = −I and ∇pg(x∗,p) = 0.
∇c f (x∗(p),p) = −λ
Similarly, using p = d, we get ∇d f (x∗(p),p) = −µ.
Lagrange multipliers λ and µ signify costs of pulling the minimumpoint in order to satisfy the constraints!
Equality constraint: both sides infeasible, sign of λj identifiesone side or the other of the hypersurface.
Inequality constraint: one side is feasible, no cost of pullingfrom that side, so µi ≥ 0.
![Page 677: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/677.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 677,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
Dual problem:Reformulation of a problem in terms of the Lagrange multipliers.Suppose x∗ as a local minimum for the problem
Minimize f (x) subject to h(x) = 0,
with Lagrange multiplier (vector) λ∗.
∇f (x∗) + [∇h(x∗)]λ∗ = 0
![Page 678: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/678.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 678,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
Dual problem:Reformulation of a problem in terms of the Lagrange multipliers.Suppose x∗ as a local minimum for the problem
Minimize f (x) subject to h(x) = 0,
with Lagrange multiplier (vector) λ∗.
∇f (x∗) + [∇h(x∗)]λ∗ = 0
If HL(x∗) is positive definite (assumption of local duality), then x∗
is also a local minimum of
f (x) = f (x) + λ∗Th(x).
If we vary λ around λ∗, the minimizer of
L(x,λ) = f (x) + λTh(x)
varies continuously with λ.
![Page 679: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/679.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 679,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
In the neighbourhood of λ∗, define the dual function
Φ(λ) = minx
L(x,λ) = minx
[f (x) + λTh(x)].
For a pair x,λ, the dual solution is feasible if and onlyif the primal solution is optimal.
![Page 680: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/680.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 680,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
In the neighbourhood of λ∗, define the dual function
Φ(λ) = minx
L(x,λ) = minx
[f (x) + λTh(x)].
For a pair x,λ, the dual solution is feasible if and onlyif the primal solution is optimal.
Define x(λ) as the local minimizer of L(x,λ).
Φ(λ) = L(x(λ),λ) = f (x(λ)) + λTh(x(λ))
![Page 681: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/681.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 681,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
In the neighbourhood of λ∗, define the dual function
Φ(λ) = minx
L(x,λ) = minx
[f (x) + λTh(x)].
For a pair x,λ, the dual solution is feasible if and onlyif the primal solution is optimal.
Define x(λ) as the local minimizer of L(x,λ).
Φ(λ) = L(x(λ),λ) = f (x(λ)) + λTh(x(λ))
First derivative:
∇Φ(λ) = ∇λx(λ)∇xL(x(λ),λ) + h(x(λ)) = h(x(λ))
For a pair x,λ, the dual solution is optimal if and onlyif the primal solution is feasible.
![Page 682: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/682.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 682,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
Hessian of the dual function:
Hφ(λ) = ∇λx(λ)∇xh(x(λ))
Differentiating ∇xL(x(λ),λ) = 0, we have
∇λx(λ)HL(x(λ),λ) + [∇xh(x(λ))]T = 0.
Solving for ∇λx(λ) and substituting,
Hφ(λ) = −[∇xh(x(λ))]T [HL(x(λ),λ)]−1∇xh(x(λ)),
negative definite!
![Page 683: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/683.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 683,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
Hessian of the dual function:
Hφ(λ) = ∇λx(λ)∇xh(x(λ))
Differentiating ∇xL(x(λ),λ) = 0, we have
∇λx(λ)HL(x(λ),λ) + [∇xh(x(λ))]T = 0.
Solving for ∇λx(λ) and substituting,
Hφ(λ) = −[∇xh(x(λ))]T [HL(x(λ),λ)]−1∇xh(x(λ)),
negative definite!
At λ∗, x(λ∗) = x∗, ∇Φ(λ∗) = h(x∗) = 0, Hφ(λ∗) is negativedefinite and the dual function is maximized.
Φ(λ∗) = L(x∗,λ∗) = f (x∗)
![Page 684: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/684.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 684,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Duality*
Consolidation (including all constraints)
Assuming local convexity, the dual function:
Φ(λ,µ) = minx
L(x,λ,µ) = minx
[f (x) + λTh(x) + µTg(x)].
Constraints on the dual: ∇xL(x,λ,µ) = 0, optimality of theprimal.
Corresponding to inequality constraints of the primal problem,non-negative variables µ in the dual problem.
First order necessary conditons for the dual optimality:equivalent to the feasibility of the primal problem.
The dual function is concave globally!
Under suitable conditions, Φ(λ∗) = L(x∗,λ∗) = f (x∗).
The Lagrangian L(x,λ,µ) has a saddle point in the combinedspace of primal and dual variables: positive curvature along xdirections and negative curvature along λ and µ directions.
![Page 685: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/685.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 685,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Structure of Methods: An Overview*
For a problem of n variables, with m active constraints,nature and dimension of working spaces
Penalty methods (Rn): Minimize the penalized function
q(c , x) = f (x) + cP(x).
Example: P(x) = 12‖h(x)‖2 + 1
2 [max(0,g(x))]2.
Primal methods (Rn−m): Work only in feasible domain, restrictingsteps to the tangent plane.Example: Gradient projection method.
Dual methods (Rm): Transform the problem to the space ofLagrange multipliers and maximize the dual.Example: Augmented Lagrangian method.
Lagrange methods (Rm+n): Solve equations appearing in the KKTconditions directly.Example: Sequential quadratic programming.
![Page 686: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/686.jpg)
Mathematical Methods in Engineering and Science Constrained Optimization 686,
ConstraintsOptimality CriteriaSensitivityDuality*Structure of Methods: An Overview*
Points to note
Constraint qualification
KKT conditions
Second order conditions
Basic ideas for solution strategy
Necessary Exercises: 1,2,3,4,5,6
![Page 687: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/687.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 687,
Linear ProgrammingQuadratic ProgrammingOutline
Linear and Quadratic Programming Problems*Linear ProgrammingQuadratic Programming
![Page 688: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/688.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 688,
Linear ProgrammingQuadratic ProgrammingLinear Programming
Standard form of an LP problem:
Minimize f (x) = cT x,subject to Ax = b, x ≥ 0; with b ≥ 0.
![Page 689: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/689.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 689,
Linear ProgrammingQuadratic ProgrammingLinear Programming
Standard form of an LP problem:
Minimize f (x) = cT x,subject to Ax = b, x ≥ 0; with b ≥ 0.
Preprocessing to cast a problem to the standard form
Maximization: Minimize the negative function.
Variables of unrestricted sign: Use two variables.
Inequality constraints: Use slack/surplus variables.
Negative RHS: Multiply with −1.
![Page 690: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/690.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 690,
Linear ProgrammingQuadratic ProgrammingLinear Programming
Standard form of an LP problem:
Minimize f (x) = cT x,subject to Ax = b, x ≥ 0; with b ≥ 0.
Preprocessing to cast a problem to the standard form
Maximization: Minimize the negative function.
Variables of unrestricted sign: Use two variables.
Inequality constraints: Use slack/surplus variables.
Negative RHS: Multiply with −1.
Geometry of an LP problem
Infinite domain: does a minimum exist?
Finite convex polytope: existence guaranteed
Operating with vertices sufficient as a strategy
Extension with slack/surplus variables: original solution spacea subspace in the extented space, x ≥ 0 marking the domain
Essence of the non-negativity condition of variables
![Page 691: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/691.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 691,
Linear ProgrammingQuadratic ProgrammingLinear Programming
The simplex method
Suppose x ∈ RN , b ∈ RM and A ∈ RM×N full-rank, with M < N.
IMxB + A′xNB = b′
Basic and non-basic variables: xB ∈ RM and xNB ∈ RN−M
Basic feasible solution: xB = b′ ≥ 0 and xNB = 0
![Page 692: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/692.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 692,
Linear ProgrammingQuadratic ProgrammingLinear Programming
The simplex method
Suppose x ∈ RN , b ∈ RM and A ∈ RM×N full-rank, with M < N.
IMxB + A′xNB = b′
Basic and non-basic variables: xB ∈ RM and xNB ∈ RN−M
Basic feasible solution: xB = b′ ≥ 0 and xNB = 0
At every iteration,
selection of a non-basic variable to enter the basis edge of travel selected based on maximum rate of descent no qualifier: current vertex is optimal
selection of a basic variable to leave the basis based on the first constraint becoming active along the edge no constraint ahead: function is unbounded
elementary row operations: new basic feasible solution
![Page 693: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/693.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 693,
Linear ProgrammingQuadratic ProgrammingLinear Programming
The simplex method
Suppose x ∈ RN , b ∈ RM and A ∈ RM×N full-rank, with M < N.
IMxB + A′xNB = b′
Basic and non-basic variables: xB ∈ RM and xNB ∈ RN−M
Basic feasible solution: xB = b′ ≥ 0 and xNB = 0
At every iteration,
selection of a non-basic variable to enter the basis edge of travel selected based on maximum rate of descent no qualifier: current vertex is optimal
selection of a basic variable to leave the basis based on the first constraint becoming active along the edge no constraint ahead: function is unbounded
elementary row operations: new basic feasible solution
Two-phase method: Inclusion of a pre-processing phase withartificial variables to develop a basic feasible solution
![Page 694: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/694.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 694,
Linear ProgrammingQuadratic ProgrammingLinear Programming
General perspectiveLP problem:
Minimize f (x, y) = cT1 x + cT
2 y;subject to A11x + A12y = b1, A21x + A22y ≤ b2, y ≥ 0.
Lagrangian:
L(x, y,λ,µ,ν) = cT1 x + cT
2 y
+ λT (A11x + A12y − b1) + µT (A21x + A22y − b2)− νTy
![Page 695: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/695.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 695,
Linear ProgrammingQuadratic ProgrammingLinear Programming
General perspectiveLP problem:
Minimize f (x, y) = cT1 x + cT
2 y;subject to A11x + A12y = b1, A21x + A22y ≤ b2, y ≥ 0.
Lagrangian:
L(x, y,λ,µ,ν) = cT1 x + cT
2 y
+ λT (A11x + A12y − b1) + µT (A21x + A22y − b2)− νTy
Optimality conditions:
c1 + AT11λ + AT
21µ = 0 and ν = c2 + AT12λ + AT
22µ ≥ 0
Substituting back, optimal function value: f ∗ = −λTb1 −µTb2
Sensitivity to the constraints: ∂f ∗
∂b1= −λ and ∂f ∗
∂b2= −µ
![Page 696: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/696.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 696,
Linear ProgrammingQuadratic ProgrammingLinear Programming
General perspectiveLP problem:
Minimize f (x, y) = cT1 x + cT
2 y;subject to A11x + A12y = b1, A21x + A22y ≤ b2, y ≥ 0.
Lagrangian:
L(x, y,λ,µ,ν) = cT1 x + cT
2 y
+ λT (A11x + A12y − b1) + µT (A21x + A22y − b2)− νTy
Optimality conditions:
c1 + AT11λ + AT
21µ = 0 and ν = c2 + AT12λ + AT
22µ ≥ 0
Substituting back, optimal function value: f ∗ = −λTb1 −µTb2
Sensitivity to the constraints: ∂f ∗
∂b1= −λ and ∂f ∗
∂b2= −µ
Dual problem:
maximize Φ(λ,µ) = −bT1 λ− bT
2 µ;subject to AT
11λ + AT21µ = −c1, AT
12λ + AT22µ ≥ −c2, µ ≥ 0.
Notice the symmetry between the primal and dual problems.
![Page 697: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/697.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 697,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
A quadratic objective function and linear constraints define
a QP problem.
Equations from the KKT conditions: linear!
Lagrange methods are the natural choice!
![Page 698: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/698.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 698,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
A quadratic objective function and linear constraints define
a QP problem.
Equations from the KKT conditions: linear!
Lagrange methods are the natural choice!
With equality constraints only,
Minimize f (x) =1
2xTQx + cTx, subject to Ax = b.
First order necessary conditions:
[
Q AT
A 0
] [
x∗
λ
]
=
[
−cb
]
Solution of this linear system yields the complete result!
![Page 699: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/699.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 699,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
A quadratic objective function and linear constraints define
a QP problem.
Equations from the KKT conditions: linear!
Lagrange methods are the natural choice!
With equality constraints only,
Minimize f (x) =1
2xTQx + cTx, subject to Ax = b.
First order necessary conditions:
[
Q AT
A 0
] [
x∗
λ
]
=
[
−cb
]
Solution of this linear system yields the complete result!
Caution: This coefficient matrix is indefinite.
![Page 700: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/700.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 700,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
Active set method
Minimize f (x) = 12x
TQx + cT x;subject to A1x = b1,
A2x ≤ b2.
![Page 701: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/701.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 701,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
Active set method
Minimize f (x) = 12x
TQx + cT x;subject to A1x = b1,
A2x ≤ b2.
Start the iterative process from a feasible point.
Construct active set of constraints as Ax = b. From the current point xk , with x = xk + dk ,
f (x) =1
2(xk + dk)TQ(xk + dk) + cT (xk + dk)
=1
2dT
k Qdk + (c + Qxk)Tdk + f (xk).
Since gk ≡ ∇f (xk) = c + Qxk , subsidiary quadratic program:
minimize 12d
Tk Qdk + gT
k dk subject to Adk = 0.
Examining solution dk and Lagrange multipliers, decide toterminate, proceed or revise the active set.
![Page 702: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/702.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 702,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
Linear complementary problem (LCP)
Slack variable strategy with inequality constraints
Minimize 12x
TQx + cTx, subject to Ax ≤ b, x ≥ 0.
![Page 703: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/703.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 703,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
Linear complementary problem (LCP)
Slack variable strategy with inequality constraints
Minimize 12x
TQx + cTx, subject to Ax ≤ b, x ≥ 0.
KKT conditions: With x, y,µ,ν ≥ 0,
Qx + c + ATµ− ν = 0,
Ax + y = b,
xT ν = µTy = 0.
![Page 704: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/704.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 704,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
Linear complementary problem (LCP)
Slack variable strategy with inequality constraints
Minimize 12x
TQx + cTx, subject to Ax ≤ b, x ≥ 0.
KKT conditions: With x, y,µ,ν ≥ 0,
Qx + c + ATµ− ν = 0,
Ax + y = b,
xT ν = µTy = 0.
Denoting
z =
[
xµ
]
,w =
[
ν
y
]
,q =
[
cb
]
and M =
[
Q AT
−A 0
]
,
w −Mz = q, wT z = 0.
Find mutually complementary non-negative w and z.
![Page 705: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/705.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 705,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
If q ≥ 0, then w = q, z = 0 is a solution!
Lemke’s method: artificial variable z0 with e = [1 1 1 · · · 1]T :
Iw −Mz − ez0 = q
With z0 = max(−qi),
w = q + ez0 ≥ 0 and z = 0: basic feasible solution
![Page 706: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/706.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 706,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
If q ≥ 0, then w = q, z = 0 is a solution!
Lemke’s method: artificial variable z0 with e = [1 1 1 · · · 1]T :
Iw −Mz − ez0 = q
With z0 = max(−qi),
w = q + ez0 ≥ 0 and z = 0: basic feasible solution
Evolution of the basis similar to the simplex method.
Out of a pair of w and z variables, only one can be there inany basis.
At every step, one variable is driven out of the basis and itspartner called in.
The step driving out z0 flags termination.
![Page 707: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/707.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 707,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
If q ≥ 0, then w = q, z = 0 is a solution!
Lemke’s method: artificial variable z0 with e = [1 1 1 · · · 1]T :
Iw −Mz − ez0 = q
With z0 = max(−qi),
w = q + ez0 ≥ 0 and z = 0: basic feasible solution
Evolution of the basis similar to the simplex method.
Out of a pair of w and z variables, only one can be there inany basis.
At every step, one variable is driven out of the basis and itspartner called in.
The step driving out z0 flags termination.
Handling of equality constraints?
![Page 708: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/708.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 708,
Linear ProgrammingQuadratic ProgrammingQuadratic Programming
If q ≥ 0, then w = q, z = 0 is a solution!
Lemke’s method: artificial variable z0 with e = [1 1 1 · · · 1]T :
Iw −Mz − ez0 = q
With z0 = max(−qi),
w = q + ez0 ≥ 0 and z = 0: basic feasible solution
Evolution of the basis similar to the simplex method.
Out of a pair of w and z variables, only one can be there inany basis.
At every step, one variable is driven out of the basis and itspartner called in.
The step driving out z0 flags termination.
Handling of equality constraints? Very clumsy!!
![Page 709: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/709.jpg)
Mathematical Methods in Engineering and Science Linear and Quadratic Programming Problems* 709,
Linear ProgrammingQuadratic ProgrammingPoints to note
Fundamental issues and general perspective of the linearprogramming problem
The simplex method
Quadratic programming The active set method Lemke’s method via the linear complementary problem
Necessary Exercises: 1,2,3,4,5
![Page 710: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/710.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 710,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Outline
Interpolation and ApproximationPolynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
![Page 711: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/711.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 711,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Problem: To develop an analytical representation of a functionfrom information at discrete data points.Purpose
Evaluation at arbitrary points
Differentiation and/or integration
Drawing conclusion regarding the trends or nature
![Page 712: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/712.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 712,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Problem: To develop an analytical representation of a functionfrom information at discrete data points.Purpose
Evaluation at arbitrary points
Differentiation and/or integration
Drawing conclusion regarding the trends or nature
Interpolation: one of the ways of function representation
sampled data are exactly satisfied
Polynomial: a convenient class of basis functionsFor yi = f (xi ) for i = 0, 1, 2, · · · , n with x0 < x1 < x2 < · · · < xn,
p(x) = a0 + a1x + a2x2 + · · ·+ anx
n.
Find the coefficients such that p(xi) = f (xi ) for i = 0, 1, 2, · · · , n.
Values of p(x) for x ∈ [x0, xn] interpolate n + 1 valuesof f (x), an outside estimate is extrapolation.
![Page 713: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/713.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 713,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
To determine p(x), solve the linear system
1 x0 x20 · · · xn
0
1 x1 x21 · · · xn
1
1 x2 x22 · · · xn
2...
......
. . ....
1 xn x2n · · · xn
n
a0
a1
a2
· · ·an
=
f (x0)f (x1)f (x2)· · ·
f (xn)
?
Vandermonde matrix: invertible, but typically ill-conditioned!
![Page 714: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/714.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 714,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
To determine p(x), solve the linear system
1 x0 x20 · · · xn
0
1 x1 x21 · · · xn
1
1 x2 x22 · · · xn
2...
......
. . ....
1 xn x2n · · · xn
n
a0
a1
a2
· · ·an
=
f (x0)f (x1)f (x2)· · ·
f (xn)
?
Vandermonde matrix: invertible, but typically ill-conditioned!
Invertibility means existence and uniqueness of polynomial p(x).
Two polynomials p1(x) and p2(x) matching the function f (x) atx0, x1, x2, · · · , xn imply
n-th degree polynomial ∆p(x) = p1(x)− p2(x) withn + 1 roots!
∆p ≡ 0 ⇒ p1(x) = p2(x): p(x) is unique.
![Page 715: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/715.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 715,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Lagrange interpolationBasis functions:
Lk(x) =
∏nj=0,j 6=k(x − xj)
∏nj=0,j 6=k(xk − xj)
=(x − x0)(x − x1) · · · (x − xk−1)(x − xk+1) · · · (x − xn)
(xk − x0)(xk − x1) · · · (xk − xk−1)(xk − xk+1) · · · (xk − xn)
Interpolating polynomial:
p(x) = α0L0(x) + α1L1(x) + α2L2(x) + · · ·+ αnLn(x)
![Page 716: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/716.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 716,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Lagrange interpolationBasis functions:
Lk(x) =
∏nj=0,j 6=k(x − xj)
∏nj=0,j 6=k(xk − xj)
=(x − x0)(x − x1) · · · (x − xk−1)(x − xk+1) · · · (x − xn)
(xk − x0)(xk − x1) · · · (xk − xk−1)(xk − xk+1) · · · (xk − xn)
Interpolating polynomial:
p(x) = α0L0(x) + α1L1(x) + α2L2(x) + · · ·+ αnLn(x)
At the data points, Lk(xi) = δik .
Coefficient matrix identity and αi = f (xi ).
Lagrange interpolation formula:
p(x) =
n∑
k=0
f (xk)Lk(x) = L0(x)f (x0)+L1(x)f (x1)+· · ·+Ln(x)f (xn)
Existence of p(x) is a trivial consequence!
![Page 717: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/717.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 717,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Two interpolation formulae
one costly to determine, but easy to process
the other trivial to determine, costly to process
![Page 718: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/718.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 718,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Two interpolation formulae
one costly to determine, but easy to process
the other trivial to determine, costly to process
Newton interpolation for an intermediate trade-off:p(x) = c0 + c1(x − x0)+ c2(x − x0)(x − x1)+ · · ·+ cn
∏n−1i=0 (x − xi)
![Page 719: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/719.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 719,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Two interpolation formulae
one costly to determine, but easy to process
the other trivial to determine, costly to process
Newton interpolation for an intermediate trade-off:p(x) = c0 + c1(x − x0)+ c2(x − x0)(x − x1)+ · · ·+ cn
∏n−1i=0 (x − xi)
Hermite interpolation
uses derivatives as well as function values.
Data: f (xi ), f ′(xi ), · · · , f (ni−1)(xi ) at x = xi , for i = 0, 1, · · · ,m:
At (m + 1) points, a total of n + 1 =∑m
i=0 ni conditions
![Page 720: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/720.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 720,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Polynomial Interpolation
Two interpolation formulae
one costly to determine, but easy to process
the other trivial to determine, costly to process
Newton interpolation for an intermediate trade-off:p(x) = c0 + c1(x − x0)+ c2(x − x0)(x − x1)+ · · ·+ cn
∏n−1i=0 (x − xi)
Hermite interpolation
uses derivatives as well as function values.
Data: f (xi ), f ′(xi ), · · · , f (ni−1)(xi ) at x = xi , for i = 0, 1, · · · ,m:
At (m + 1) points, a total of n + 1 =∑m
i=0 ni conditions
Limitations of single-polynomial interpolation
With large number of data points, polynomial degree is high.
Computational cost and numerical imprecision
Lack of representative nature due to oscillations
![Page 721: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/721.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 721,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Piecewise linear interpolation
f (x) = f (xi−1) +f (xi )− f (xi−1)
xi − xi−1(x − xi−1) for x ∈ [xi−1, xi ]
Handy for many uses with dense data. But, not differentiable.
![Page 722: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/722.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 722,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Piecewise linear interpolation
f (x) = f (xi−1) +f (xi )− f (xi−1)
xi − xi−1(x − xi−1) for x ∈ [xi−1, xi ]
Handy for many uses with dense data. But, not differentiable.
Piecewise cubic interpolationWith function values and derivatives at (n + 1) points,
n cubic Hermite segments
Data for the j-th segment:
f (xj−1) = fj−1, f (xj ) = fj , f ′(xj−1) = f ′j−1 and f ′(xj ) = f ′j
Interpolating polynomial:
pj(x) = a0 + a1x + a2x2 + a3x
3
Coefficients a0, a1, a2, a3: linear combinations of fj−1, fj , f ′j−1, f ′j
![Page 723: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/723.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 723,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Piecewise linear interpolation
f (x) = f (xi−1) +f (xi )− f (xi−1)
xi − xi−1(x − xi−1) for x ∈ [xi−1, xi ]
Handy for many uses with dense data. But, not differentiable.
Piecewise cubic interpolationWith function values and derivatives at (n + 1) points,
n cubic Hermite segments
Data for the j-th segment:
f (xj−1) = fj−1, f (xj ) = fj , f ′(xj−1) = f ′j−1 and f ′(xj ) = f ′j
Interpolating polynomial:
pj(x) = a0 + a1x + a2x2 + a3x
3
Coefficients a0, a1, a2, a3: linear combinations of fj−1, fj , f ′j−1, f ′j
Composite function C1 continuous at knot points.
![Page 724: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/724.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 724,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
General formulation through normalization of intervals
x = xj−1 + t(xj − xj−1), t ∈ [0, 1]
With g(t) = f (x(t)), g ′(t) = (xj − xj−1)f′(x(t));
g0 = fj−1, g1 = fj , g ′0 = (xj − xj−1)f
′j−1 and g ′
1 = (xj − xj−1)f′j .
Cubic polynomial for the j-th segment:
qj(t) = α0 + α1t + α2t2 + α3t
3
![Page 725: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/725.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 725,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
General formulation through normalization of intervals
x = xj−1 + t(xj − xj−1), t ∈ [0, 1]
With g(t) = f (x(t)), g ′(t) = (xj − xj−1)f′(x(t));
g0 = fj−1, g1 = fj , g ′0 = (xj − xj−1)f
′j−1 and g ′
1 = (xj − xj−1)f′j .
Cubic polynomial for the j-th segment:
qj(t) = α0 + α1t + α2t2 + α3t
3
Modular expression:
qj(t) = [α0 α1 α2 α3]
1tt2
t3
= [g0 g1 g ′0 g ′
1] W
1tt2
t3
= GjWT
Packaging data, interpolation type and variable terms separately!
![Page 726: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/726.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 726,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
General formulation through normalization of intervals
x = xj−1 + t(xj − xj−1), t ∈ [0, 1]
With g(t) = f (x(t)), g ′(t) = (xj − xj−1)f′(x(t));
g0 = fj−1, g1 = fj , g ′0 = (xj − xj−1)f
′j−1 and g ′
1 = (xj − xj−1)f′j .
Cubic polynomial for the j-th segment:
qj(t) = α0 + α1t + α2t2 + α3t
3
Modular expression:
qj(t) = [α0 α1 α2 α3]
1tt2
t3
= [g0 g1 g ′0 g ′
1] W
1tt2
t3
= GjWT
Packaging data, interpolation type and variable terms separately!
Question: How to supply derivatives? And, why?
![Page 727: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/727.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 727,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Spline interpolation
Spline: a drafting tool to draw a smooth curve through key points.
![Page 728: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/728.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 728,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Spline interpolation
Spline: a drafting tool to draw a smooth curve through key points.
Data: fi = f (xi ), for x0 < x1 < x2 < · · · < xn.
If kj = f ′(xj), then
pj(x) can be determined in terms of fj−1, fj , kj−1, kj
and pj+1(x) in terms of fj , fj+1, kj , kj+1.
Then, p′′j (xj) = p′′
j+1(xj): a linear equation in kj−1, kj and kj+1
![Page 729: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/729.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 729,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Spline interpolation
Spline: a drafting tool to draw a smooth curve through key points.
Data: fi = f (xi ), for x0 < x1 < x2 < · · · < xn.
If kj = f ′(xj), then
pj(x) can be determined in terms of fj−1, fj , kj−1, kj
and pj+1(x) in terms of fj , fj+1, kj , kj+1.
Then, p′′j (xj) = p′′
j+1(xj): a linear equation in kj−1, kj and kj+1
From n− 1 interior knot points,
n − 1 linear equations in derivative values k0, k1, · · · , kn.
Prescribing k0 and kn, a diagonally dominant tridiagonal system!
![Page 730: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/730.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 730,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Piecewise Polynomial Interpolation
Spline interpolation
Spline: a drafting tool to draw a smooth curve through key points.
Data: fi = f (xi ), for x0 < x1 < x2 < · · · < xn.
If kj = f ′(xj), then
pj(x) can be determined in terms of fj−1, fj , kj−1, kj
and pj+1(x) in terms of fj , fj+1, kj , kj+1.
Then, p′′j (xj) = p′′
j+1(xj): a linear equation in kj−1, kj and kj+1
From n− 1 interior knot points,
n − 1 linear equations in derivative values k0, k1, · · · , kn.
Prescribing k0 and kn, a diagonally dominant tridiagonal system!
A spline is a smooth interpolation, with C2 continuity.
![Page 731: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/731.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 731,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Piecewise bilinear interpolation
Data: f (x , y) over a dense rectangular grid
x = x0, x1, x2, · · · , xm and y = y0, y1, y2, · · · , yn
Rectangular domain: (x , y) : x0 ≤ x ≤ xm, y0 ≤ y ≤ yn
![Page 732: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/732.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 732,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Piecewise bilinear interpolation
Data: f (x , y) over a dense rectangular grid
x = x0, x1, x2, · · · , xm and y = y0, y1, y2, · · · , yn
Rectangular domain: (x , y) : x0 ≤ x ≤ xm, y0 ≤ y ≤ yn
For xi−1 ≤ x ≤ xi and yj−1 ≤ y ≤ yj ,
f (x , y) = a0,0 + a1,0x + a0,1y + a1,1xy = [1 x ]
[
a0,0 a0,1
a1,0 a1,1
] [
1y
]
With data at four corner points, coefficient matrix determined from
[
1 xi−1
1 xi
] [
a0,0 a0,1
a1,0 a1,1
] [
1 1yj−1 yj
]
=
[
fi−1,j−1 fi−1,j
fi ,j−1 fi ,j
]
.
![Page 733: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/733.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 733,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Piecewise bilinear interpolation
Data: f (x , y) over a dense rectangular grid
x = x0, x1, x2, · · · , xm and y = y0, y1, y2, · · · , yn
Rectangular domain: (x , y) : x0 ≤ x ≤ xm, y0 ≤ y ≤ yn
For xi−1 ≤ x ≤ xi and yj−1 ≤ y ≤ yj ,
f (x , y) = a0,0 + a1,0x + a0,1y + a1,1xy = [1 x ]
[
a0,0 a0,1
a1,0 a1,1
] [
1y
]
With data at four corner points, coefficient matrix determined from
[
1 xi−1
1 xi
] [
a0,0 a0,1
a1,0 a1,1
] [
1 1yj−1 yj
]
=
[
fi−1,j−1 fi−1,j
fi ,j−1 fi ,j
]
.
Approximation only C0 continuous.
![Page 734: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/734.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 734,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Alternative local formula through reparametrizationWith u =
x−xi−1
xi−xi−1and v =
y−yj−1
yj−yj−1, denoting
fi−1,j−1 = g0,0, fi ,j−1 = g1,0, fi−1,j = g0,1 and fi ,j = g1,1;
bilinear interpolation:
g(u, v) = [1 u]
[
α0,0 α0,1
α1,0 α1,1
] [
1v
]
for u, v ∈ [0, 1].
![Page 735: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/735.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 735,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Alternative local formula through reparametrizationWith u =
x−xi−1
xi−xi−1and v =
y−yj−1
yj−yj−1, denoting
fi−1,j−1 = g0,0, fi ,j−1 = g1,0, fi−1,j = g0,1 and fi ,j = g1,1;
bilinear interpolation:
g(u, v) = [1 u]
[
α0,0 α0,1
α1,0 α1,1
] [
1v
]
for u, v ∈ [0, 1].
Values at four corner points fix the coefficient matrix as[
α0,0 α0,1
α1,0 α1,1
]
=
[
1 0−1 1
] [
g0,0 g0,1
g1,0 g1,1
] [
1 −10 1
]
.
Concisely, g(u, v) = UTWTGi ,jWV in which
U =
[
1u
]
, V =
[
1v
]
, W =
[
1 −10 1
]
, Gi ,j =
[
fi−1,j−1 fi−1,j
fi ,j−1 fi ,j
]
.
![Page 736: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/736.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 736,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Piecewise bicubic interpolation
Data: f , ∂f∂x
, ∂f∂y
and ∂2f∂x∂y
over grid points
With normalizing parameters u and v ,
∂g∂u
= (xi − xi−1)∂f∂x, ∂g
∂v= (yj − yj−1)
∂f∂y, and
∂2g∂u∂v
= (xi − xi−1)(yj − yj−1)∂2f
∂x∂y
![Page 737: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/737.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 737,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Interpolation of Multivariate Functions
Piecewise bicubic interpolation
Data: f , ∂f∂x
, ∂f∂y
and ∂2f∂x∂y
over grid points
With normalizing parameters u and v ,
∂g∂u
= (xi − xi−1)∂f∂x, ∂g
∂v= (yj − yj−1)
∂f∂y, and
∂2g∂u∂v
= (xi − xi−1)(yj − yj−1)∂2f
∂x∂y
In (x , y) : xi−1 ≤ x ≤ xi , yj−1 ≤ y ≤ yj or (u, v) : u, v ∈ [0, 1],
g(u, v) = UTWTGi ,jWV,
with U = [1 u u2 u3]T , V = [1 v v2 v3]T , and
Gi ,j =
g(0, 0) g(0, 1) gv (0, 0) gv (0, 1)g(1, 0) g(1, 1) gv (1, 0) gv (1, 1)gu(0, 0) gu(0, 1) guv (0, 0) guv (0, 1)gu(1, 0) gu(1, 1) guv (1, 0) guv (1, 1)
.
![Page 738: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/738.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 738,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
A Note on Approximation of Functions
A common strategy of function approximation is to
express a function as a linear combination of a set of basisfunctions (which?), and
determine coefficients based on some criteria (what?).
![Page 739: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/739.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 739,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
A Note on Approximation of Functions
A common strategy of function approximation is to
express a function as a linear combination of a set of basisfunctions (which?), and
determine coefficients based on some criteria (what?).
Criteria:
Interpolatory approximation: Exact agreement with sampled data
Least square approximation: Minimization of a sum (or integral) ofsquare errors over sampled data
Minimax approximation: Limiting the largest deviation
![Page 740: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/740.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 740,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
A Note on Approximation of Functions
A common strategy of function approximation is to
express a function as a linear combination of a set of basisfunctions (which?), and
determine coefficients based on some criteria (what?).
Criteria:
Interpolatory approximation: Exact agreement with sampled data
Least square approximation: Minimization of a sum (or integral) ofsquare errors over sampled data
Minimax approximation: Limiting the largest deviation
Basis functions:
polynomials, sinusoids, orthogonal eigenfunctions orfield-specific heuristic choice
![Page 741: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/741.jpg)
Mathematical Methods in Engineering and Science Interpolation and Approximation 741,
Polynomial InterpolationPiecewise Polynomial InterpolationInterpolation of Multivariate FunctionsA Note on Approximation of FunctionsModelling of Curves and Surfaces*
Points to note
Lagrange, Newton and Hermite interpolations
Piecewise polynomial functions and splines
Bilinear and bicubic interpolation of bivariate functions
Direct extension to vector functions: curves and surfaces!
Necessary Exercises: 1,2,4,6
![Page 742: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/742.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 742,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Outline
Basic Methods of Numerical IntegrationNewton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
![Page 743: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/743.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 743,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
J =
∫ b
a
f (x)dx
![Page 744: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/744.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 744,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
J =
∫ b
a
f (x)dx
Divide [a, b] into n sub-intervals with
a = x0 < x1 < x2 < · · · < xn−1 < xn = b,
where xi − xi−1 = h = b−an
.
J =
n∑
i=1
hf (x∗i ) = h[f (x∗
1 ) + f (x∗2 ) + · · ·+ f (x∗
n )]
Taking x∗i ∈ [xi−1, xi ] as xi−1 and xi , we get summations J1 and J2.
As n→∞ (i.e. h→ 0), if J1 and J2 approach the samelimit, then function f (x) is integrable over interval [a, b].
A rectangular rule or a one-point rule
![Page 745: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/745.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 745,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
J =
∫ b
a
f (x)dx
Divide [a, b] into n sub-intervals with
a = x0 < x1 < x2 < · · · < xn−1 < xn = b,
where xi − xi−1 = h = b−an
.
J =
n∑
i=1
hf (x∗i ) = h[f (x∗
1 ) + f (x∗2 ) + · · ·+ f (x∗
n )]
Taking x∗i ∈ [xi−1, xi ] as xi−1 and xi , we get summations J1 and J2.
As n→∞ (i.e. h→ 0), if J1 and J2 approach the samelimit, then function f (x) is integrable over interval [a, b].
A rectangular rule or a one-point rule
Question: Which point to take as x∗i ?
![Page 746: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/746.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 746,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Mid-point ruleSelecting x∗
i as xi =xi−1+xi
2 ,
∫ xi
xi−1
f (x)dx ≈ hf (xi) and
∫ b
a
f (x)dx ≈ h
n∑
i=1
f (xi ).
![Page 747: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/747.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 747,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Mid-point ruleSelecting x∗
i as xi =xi−1+xi
2 ,
∫ xi
xi−1
f (x)dx ≈ hf (xi) and
∫ b
a
f (x)dx ≈ h
n∑
i=1
f (xi ).
Error analysis: From Taylor’s series of f (x) about xi ,∫ xi
xi−1
f (x)dx =
∫ xi
xi−1
[
f (xi ) + f ′(xi )(x − xi ) + f ′′(xi )(x − xi)
2
2+ · · ·
]
dx
= hf (xi) +h3
24f ′′(xi) +
h5
1920f iv (xi ) + · · · ,
third order accurate!
![Page 748: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/748.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 748,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Mid-point ruleSelecting x∗
i as xi =xi−1+xi
2 ,
∫ xi
xi−1
f (x)dx ≈ hf (xi) and
∫ b
a
f (x)dx ≈ h
n∑
i=1
f (xi ).
Error analysis: From Taylor’s series of f (x) about xi ,∫ xi
xi−1
f (x)dx =
∫ xi
xi−1
[
f (xi ) + f ′(xi )(x − xi ) + f ′′(xi )(x − xi)
2
2+ · · ·
]
dx
= hf (xi) +h3
24f ′′(xi) +
h5
1920f iv (xi ) + · · · ,
third order accurate!Over the entire domain [a, b],∫ b
a
f (x)dx ≈ h
n∑
i=1
f (xi)+h3
24
n∑
i=1
f ′′(xi ) = h
n∑
i=1
f (xi )+h2
24(b−a)f ′′(ξ),
for ξ ∈ [a, b] (from mean value theorem): second order accurate.
![Page 749: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/749.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 749,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Trapezoidal ruleApproximating function f (x) with a linear interpolation,
∫ xi
xi−1
f (x)dx ≈ h
2[f (xi−1) + f (xi )]
and∫ b
a
f (x)dx ≈ h
[
1
2f (x0) +
n−1∑
i=1
f (xi ) +1
2f (xn)
]
.
![Page 750: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/750.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 750,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Trapezoidal ruleApproximating function f (x) with a linear interpolation,
∫ xi
xi−1
f (x)dx ≈ h
2[f (xi−1) + f (xi )]
and∫ b
a
f (x)dx ≈ h
[
1
2f (x0) +
n−1∑
i=1
f (xi ) +1
2f (xn)
]
.
Taylor series expansions about the mid-point:
f (xi−1) = f (xi )−h
2f ′(xi ) +
h2
8f ′′(xi )−
h3
48f ′′′(xi ) +
h4
384f iv (xi )− · · ·
f (xi ) = f (xi ) +h
2f ′(xi ) +
h2
8f ′′(xi ) +
h3
48f ′′′(xi ) +
h4
384f iv (xi ) + · · ·
⇒ h
2[f (xi−1) + f (xi )] = hf (xi ) +
h3
8f ′′(xi) +
h5
384f iv(xi ) + · · ·
Recall∫ xi
xi−1f (x)dx = hf (xi ) + h3
24 f ′′(xi ) + h5
1920 f iv (xi ) + · · · .
![Page 751: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/751.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 751,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Error estimate of trapezoidal rule
∫ xi
xi−1
f (x)dx =h
2[f (xi−1) + f (xi )]−
h3
12f ′′(xi )−
h5
480f iv (xi) + · · ·
Over an extended domain,
∫ b
a
f (x)dx = h
[
1
2f (x0) + f (xn)+
n−1∑
i=1
f (xi)
]
−h2
12(b−a)f ′′(ξ)+· · · .
The same order of accuracy as the mid-point rule!
![Page 752: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/752.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 752,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Error estimate of trapezoidal rule
∫ xi
xi−1
f (x)dx =h
2[f (xi−1) + f (xi )]−
h3
12f ′′(xi )−
h5
480f iv (xi) + · · ·
Over an extended domain,
∫ b
a
f (x)dx = h
[
1
2f (x0) + f (xn)+
n−1∑
i=1
f (xi)
]
−h2
12(b−a)f ′′(ξ)+· · · .
The same order of accuracy as the mid-point rule!
Different sources of merit
Mid-point rule: Use of mid-point leads to symmetricerror-cancellation.
Trapezoidal rule: Use of end-points allows double utilizationof boundary points in adjacent intervals.
How to use both the merits?
![Page 753: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/753.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 753,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Simpson’s rulesDivide [a, b] into an even number (n = 2m) of intervals.Fit a quadratic polynomial over a panel of two intervals.For this panel of length 2h, two estimates:
M(f ) = 2hf (xi ) and T (f ) = h[f (xi−1) + f (xi+1)]
J = M(f ) +h3
3f ′′(xi ) +
h5
60f iv (xi) + · · ·
J = T (f )− 2h3
3f ′′(xi )−
h5
15f iv (xi ) + · · ·
![Page 754: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/754.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 754,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Simpson’s rulesDivide [a, b] into an even number (n = 2m) of intervals.Fit a quadratic polynomial over a panel of two intervals.For this panel of length 2h, two estimates:
M(f ) = 2hf (xi ) and T (f ) = h[f (xi−1) + f (xi+1)]
J = M(f ) +h3
3f ′′(xi ) +
h5
60f iv (xi) + · · ·
J = T (f )− 2h3
3f ′′(xi )−
h5
15f iv (xi ) + · · ·
Simpson’s one-third rule (with error estimate):∫ xi+1
xi−1
f (x)dx =h
3[f (xi−1) + 4f (xi ) + f (xi+1)]−
h5
90f iv(xi )
Fifth (not fourth) order accurate!
![Page 755: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/755.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 755,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Newton-Cotes Integration Formulae
Simpson’s rulesDivide [a, b] into an even number (n = 2m) of intervals.Fit a quadratic polynomial over a panel of two intervals.For this panel of length 2h, two estimates:
M(f ) = 2hf (xi ) and T (f ) = h[f (xi−1) + f (xi+1)]
J = M(f ) +h3
3f ′′(xi ) +
h5
60f iv (xi) + · · ·
J = T (f )− 2h3
3f ′′(xi )−
h5
15f iv (xi ) + · · ·
Simpson’s one-third rule (with error estimate):∫ xi+1
xi−1
f (x)dx =h
3[f (xi−1) + 4f (xi ) + f (xi+1)]−
h5
90f iv(xi )
Fifth (not fourth) order accurate!
A four-point rule: Simpson’s three-eighth ruleStill higher order rules NOT advisable!
![Page 756: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/756.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 756,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
To determine quantity F using a step size h, estimate F (h) error terms: hp, hq, hr etc (p < q < r) F = limδ→0 F (δ)? plot F (h), F (αh), F (α2h) (with α < 1) and extrapolate?
![Page 757: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/757.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 757,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
To determine quantity F using a step size h, estimate F (h) error terms: hp, hq, hr etc (p < q < r) F = limδ→0 F (δ)? plot F (h), F (αh), F (α2h) (with α < 1) and extrapolate?
F (h) = F + chp +O(hq)
F (αh) = F + c(αh)p +O(hq)
F (α2h) = F + c(α2h)p +O(hq)
![Page 758: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/758.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 758,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
To determine quantity F using a step size h, estimate F (h) error terms: hp, hq, hr etc (p < q < r) F = limδ→0 F (δ)? plot F (h), F (αh), F (α2h) (with α < 1) and extrapolate?
F (h) = F + chp +O(hq)
F (αh) = F + c(αh)p +O(hq)
F (α2h) = F + c(α2h)p +O(hq)
Eliminate c and determine (better estimates of) F :
F1(h) =F (αh)− αpF (h)
1− αp= F + c1h
q +O(hr )
F1(αh) =F (α2h)− αpF (αh)
1− αp= F + c1(αh)q +O(hr )
![Page 759: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/759.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 759,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
To determine quantity F using a step size h, estimate F (h) error terms: hp, hq, hr etc (p < q < r) F = limδ→0 F (δ)? plot F (h), F (αh), F (α2h) (with α < 1) and extrapolate?
1 F (h) = F + chp +O(hq)
2 F (αh) = F + c(αh)p +O(hq)
4 F (α2h) = F + c(α2h)p +O(hq)
Eliminate c and determine (better estimates of) F :
3 F1(h) =F (αh)− αpF (h)
1− αp= F + c1h
q +O(hr )
5 F1(αh) =F (α2h)− αpF (αh)
1− αp= F + c1(αh)q +O(hr )
Still better estimate: 6 F2(h) = F1(αh)−αqF1(h)1−αq = F +O(hr )
![Page 760: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/760.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 760,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
To determine quantity F using a step size h, estimate F (h) error terms: hp, hq, hr etc (p < q < r) F = limδ→0 F (δ)? plot F (h), F (αh), F (α2h) (with α < 1) and extrapolate?
F (h) = F + chp +O(hq)
F (αh) = F + c(αh)p +O(hq)
F (α2h) = F + c(α2h)p +O(hq)
Eliminate c and determine (better estimates of) F :
F1(h) =F (αh)− αpF (h)
1− αp= F + c1h
q +O(hr )
F1(αh) =F (α2h)− αpF (αh)
1− αp= F + c1(αh)q +O(hr )
Still better estimate: F2(h) = F1(αh)−αqF1(h)1−αq = F +O(hr )
Richardson extrapolation
![Page 761: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/761.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 761,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
Trapezoidal rule for J =∫ b
af (x)dx : p = 2, q = 4, r = 6 etc
T (f ) = J + ch2 + dh4 + eh6 + · · ·
With α = 12 , half the sum available for successive levels.
![Page 762: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/762.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 762,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Richardson Extrapolation and Romberg Integration
Trapezoidal rule for J =∫ b
af (x)dx : p = 2, q = 4, r = 6 etc
T (f ) = J + ch2 + dh4 + eh6 + · · ·
With α = 12 , half the sum available for successive levels.
Romberg integration Trapezoidal rule with h = H: find J11. With h = H/2, find J12.
J22 =J12 −
(
12
)2J11
1−(
12
)2 =4J12 − J11
3.
If |J22 − J12| is within tolerance, STOP. Accept J ≈ J22. With h = H/4, find J13.
J23 =4J13 − J12
3and J33 =
J23 −(
12
)4J22
1−(
12
)4=
16J23 − J22
15.
If |J33 − J23| is within tolerance, STOP with J ≈ J33.
![Page 763: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/763.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 763,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Further Issues
Featured functions: adaptive quadrature
With prescribed tolerance ǫ, assign quota ǫi =ǫ(xi−xi−1)
b−aof
error to every interval [xi−1, xi ].
For each interval, find two estimates of the integral andestimate the error.
If error estimate is not within quota, then subdivide.
![Page 764: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/764.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 764,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Further Issues
Featured functions: adaptive quadrature
With prescribed tolerance ǫ, assign quota ǫi =ǫ(xi−xi−1)
b−aof
error to every interval [xi−1, xi ].
For each interval, find two estimates of the integral andestimate the error.
If error estimate is not within quota, then subdivide.
Function as tabulated data
Only trapezoidal rule applicable?
Fit a spline over data points and integrate the segments?
![Page 765: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/765.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 765,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Further Issues
Featured functions: adaptive quadrature
With prescribed tolerance ǫ, assign quota ǫi =ǫ(xi−xi−1)
b−aof
error to every interval [xi−1, xi ].
For each interval, find two estimates of the integral andestimate the error.
If error estimate is not within quota, then subdivide.
Function as tabulated data
Only trapezoidal rule applicable?
Fit a spline over data points and integrate the segments?
Improper integral: Newton-Cotes closed formulae not applicable!
Open Newton-Cotes formulae
Gaussian quadrature
![Page 766: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/766.jpg)
Mathematical Methods in Engineering and Science Basic Methods of Numerical Integration 766,
Newton-Cotes Integration FormulaeRichardson Extrapolation and Romberg IntegrationFurther Issues
Points to note
Definition of an integral and integrability
Closed Newton-Cotes formulae and their error estimates
Richardson extrapolation as a general technique
Romberg integration
Adaptive quadrature
Necessary Exercises: 1,2,3,4
![Page 767: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/767.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 767,
Gaussian QuadratureMultiple IntegralsOutline
Advanced Topics in Numerical Integration*Gaussian QuadratureMultiple Integrals
![Page 768: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/768.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 768,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A typical quadrature formula: a weighted sum∑n
i=0 wi fi fi : function value at i -th sampled point
wi : corresponding weight
Newton-Cotes formulae:
Abscissas (xi ’s) of sampling prescribed
Coefficients or weight values determined to eliminatedominant error terms
![Page 769: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/769.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 769,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A typical quadrature formula: a weighted sum∑n
i=0 wi fi fi : function value at i -th sampled point
wi : corresponding weight
Newton-Cotes formulae:
Abscissas (xi ’s) of sampling prescribed
Coefficients or weight values determined to eliminatedominant error terms
Gaussian quadrature rules:
no prescription of quadrature points
only the ‘number’ of quadrature points prescribed
locations as well as weights contribute to the accuracy criteria
![Page 770: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/770.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 770,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A typical quadrature formula: a weighted sum∑n
i=0 wi fi fi : function value at i -th sampled point
wi : corresponding weight
Newton-Cotes formulae:
Abscissas (xi ’s) of sampling prescribed
Coefficients or weight values determined to eliminatedominant error terms
Gaussian quadrature rules:
no prescription of quadrature points
only the ‘number’ of quadrature points prescribed
locations as well as weights contribute to the accuracy criteria
with n integration points, 2n degrees of freedom
can be made exact for polynomials of degree up to 2n − 1
![Page 771: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/771.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 771,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A typical quadrature formula: a weighted sum∑n
i=0 wi fi fi : function value at i -th sampled point
wi : corresponding weight
Newton-Cotes formulae:
Abscissas (xi ’s) of sampling prescribed
Coefficients or weight values determined to eliminatedominant error terms
Gaussian quadrature rules:
no prescription of quadrature points
only the ‘number’ of quadrature points prescribed
locations as well as weights contribute to the accuracy criteria
with n integration points, 2n degrees of freedom
can be made exact for polynomials of degree up to 2n − 1
best locations: interior points
open quadrature rules: can handle integrable singularities
![Page 772: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/772.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 772,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Gauss-Legendre quadrature
∫ 1
−1f (x)dx = w1f (x1) + w2f (x2)
Four variables: Insist that it is exact for 1, x , x2 and x3.
![Page 773: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/773.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 773,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Gauss-Legendre quadrature
∫ 1
−1f (x)dx = w1f (x1) + w2f (x2)
Four variables: Insist that it is exact for 1, x , x2 and x3.
w1 + w2 =
∫ 1
−1dx = 2,
w1x1 + w2x2 =
∫ 1
−1xdx = 0,
w1x21 + w2x
22 =
∫ 1
−1x2dx =
2
3
and w1x31 + w2x
32 =
∫ 1
−1x3dx = 0.
![Page 774: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/774.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 774,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Gauss-Legendre quadrature
∫ 1
−1f (x)dx = w1f (x1) + w2f (x2)
Four variables: Insist that it is exact for 1, x , x2 and x3.
w1 + w2 =
∫ 1
−1dx = 2,
w1x1 + w2x2 =
∫ 1
−1xdx = 0,
w1x21 + w2x
22 =
∫ 1
−1x2dx =
2
3
and w1x31 + w2x
32 =
∫ 1
−1x3dx = 0.
x1 = −x2, w1 = w2
![Page 775: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/775.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 775,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Gauss-Legendre quadrature
∫ 1
−1f (x)dx = w1f (x1) + w2f (x2)
Four variables: Insist that it is exact for 1, x , x2 and x3.
w1 + w2 =
∫ 1
−1dx = 2,
w1x1 + w2x2 =
∫ 1
−1xdx = 0,
w1x21 + w2x
22 =
∫ 1
−1x2dx =
2
3
and w1x31 + w2x
32 =
∫ 1
−1x3dx = 0.
x1 = −x2, w1 = w2 ⇒ w1 = w2 = 1, x1 = − 1√3, x2 = 1√
3
![Page 776: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/776.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 776,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Two-point Gauss-Legendre quadrature formula∫ 1−1 f (x)dx = f (− 1√
3) + f ( 1√
3)
Exact for any cubic polynomial: parallels Simpson’s rule!
![Page 777: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/777.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 777,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Two-point Gauss-Legendre quadrature formula∫ 1−1 f (x)dx = f (− 1√
3) + f ( 1√
3)
Exact for any cubic polynomial: parallels Simpson’s rule!Three-point quadrature rule along similar lines:
∫ 1
−1f (x)dx =
5
9f
(
−√
3
5
)
+8
9f (0) +
5
9f
(
√
3
5
)
![Page 778: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/778.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 778,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Two-point Gauss-Legendre quadrature formula∫ 1−1 f (x)dx = f (− 1√
3) + f ( 1√
3)
Exact for any cubic polynomial: parallels Simpson’s rule!Three-point quadrature rule along similar lines:
∫ 1
−1f (x)dx =
5
9f
(
−√
3
5
)
+8
9f (0) +
5
9f
(
√
3
5
)
A large number of formulae: Consult mathematical handbooks.
![Page 779: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/779.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 779,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Two-point Gauss-Legendre quadrature formula∫ 1−1 f (x)dx = f (− 1√
3) + f ( 1√
3)
Exact for any cubic polynomial: parallels Simpson’s rule!Three-point quadrature rule along similar lines:
∫ 1
−1f (x)dx =
5
9f
(
−√
3
5
)
+8
9f (0) +
5
9f
(
√
3
5
)
A large number of formulae: Consult mathematical handbooks.For domain of integration [a, b],
x =a + b
2+
b − a
2t and dx =
b − a
2dt
With scaling and relocation,
∫ b
a
f (x)dx =b − a
2
∫ 1
−1f [x(t)]dt
![Page 780: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/780.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 780,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
General Framework for n-point formula
f (x): a polynomial of degree 2n − 1
p(x): Lagrange polynomial through the n quadrature points
f (x)− p(x): a (2n − 1)-degree polynomial having n of its roots atthe quadrature points
![Page 781: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/781.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 781,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
General Framework for n-point formula
f (x): a polynomial of degree 2n − 1
p(x): Lagrange polynomial through the n quadrature points
f (x)− p(x): a (2n − 1)-degree polynomial having n of its roots atthe quadrature points
Then, with φ(x) = (x − x1)(x − x2) · · · (x − xn),
f (x)− p(x) = φ(x)q(x).
Quotient polynomial: q(x) =∑n−1
i=0 αixi
Direct integration:
∫ 1
−1f (x)dx =
∫ 1
−1p(x)dx +
∫ 1
−1
[
φ(x)n−1∑
i=0
αixi
]
dx
![Page 782: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/782.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 782,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
General Framework for n-point formula
f (x): a polynomial of degree 2n − 1
p(x): Lagrange polynomial through the n quadrature points
f (x)− p(x): a (2n − 1)-degree polynomial having n of its roots atthe quadrature points
Then, with φ(x) = (x − x1)(x − x2) · · · (x − xn),
f (x)− p(x) = φ(x)q(x).
Quotient polynomial: q(x) =∑n−1
i=0 αixi
Direct integration:
∫ 1
−1f (x)dx =
∫ 1
−1p(x)dx +
∫ 1
−1
[
φ(x)n−1∑
i=0
αixi
]
dx
How to make the second term vanish?
![Page 783: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/783.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 783,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Choose quadrature points x1, x2, · · · , xn so that φ(x) is orthogonalto all polynomials of degree less than n.
Legendre polynomial
![Page 784: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/784.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 784,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Choose quadrature points x1, x2, · · · , xn so that φ(x) is orthogonalto all polynomials of degree less than n.
Legendre polynomial
Gauss-Legendre quadrature
1. Choose Pn(x), Legendre polynomial of degree n, as φ(x).
2. Take its roots x1, x2, · · · , xn as the quadrature points.
3. Fit Lagrange polynomial of f (x), using these n points.
p(x) = L1(x)f (x1) + L2(x)f (x2) + · · ·+ Ln(x)f (xn)
4.∫ 1
−1f (x)dx =
∫ 1
−1p(x)dx =
n∑
j=1
f (xj )
∫ 1
−1Lj (x)dx
Weight values: wj =∫ 1−1 Lj(x)dx , for j = 1, 2, · · · , n
![Page 785: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/785.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 785,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Weight functions in Gaussian quadrature
What is so great about exact integration of polynomials?
![Page 786: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/786.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 786,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
Weight functions in Gaussian quadrature
What is so great about exact integration of polynomials?
Demand something else: generalization
Exact integration of polynomials times function W (x)
Given weight function W (x) and number (n) of quadrature points,
work out the locations (xj ’s) of the n points and thecorresponding weights (wj ’s), so that integral
∫ b
a
W (x)f (x)dx =
n∑
j=1
wj f (xj)
is exact for an arbitrary polynomial f (x) of degree up to(2n − 1).
![Page 787: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/787.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 787,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A family of orthogonal polynomials with increasing degree:
quadrature points: roots of n-th member of the family.
![Page 788: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/788.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 788,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A family of orthogonal polynomials with increasing degree:
quadrature points: roots of n-th member of the family.
For different kinds of functions and different domains,
Gauss-Chebyshev quadrature
Gauss-Laguerre quadrature
Gauss-Hermite quadrature
· · · · · · · · ·Several singular functions and infinite domains can be handled.
![Page 789: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/789.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 789,
Gaussian QuadratureMultiple IntegralsGaussian Quadrature
A family of orthogonal polynomials with increasing degree:
quadrature points: roots of n-th member of the family.
For different kinds of functions and different domains,
Gauss-Chebyshev quadrature
Gauss-Laguerre quadrature
Gauss-Hermite quadrature
· · · · · · · · ·Several singular functions and infinite domains can be handled.
A very special case:
For W (x) = 1, Gauss-Legendre quadrature!
![Page 790: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/790.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 790,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
S =
∫ b
a
∫ g2(x)
g1(x)f (x , y) dy dx
![Page 791: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/791.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 791,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
S =
∫ b
a
∫ g2(x)
g1(x)f (x , y) dy dx
⇒ F (x) =
∫ g2(x)
g1(x)f (x , y) dy and S =
∫ b
a
F (x)dx
with complete flexibility of individual quadrature methods.
![Page 792: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/792.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 792,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
S =
∫ b
a
∫ g2(x)
g1(x)f (x , y) dy dx
⇒ F (x) =
∫ g2(x)
g1(x)f (x , y) dy and S =
∫ b
a
F (x)dx
with complete flexibility of individual quadrature methods.
Double integral on rectangular domain
Two-dimensional version of Simpson’s one-third rule:
∫ 1
−1
∫ 1
−1f (x , y)dxdy
= w0f (0, 0) + w1[f (−1, 0) + f (1, 0) + f (0,−1) + f (0, 1)]
+ w2[f (−1,−1) + f (−1, 1) + f (1,−1) + f (1, 1)]
![Page 793: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/793.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 793,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
S =
∫ b
a
∫ g2(x)
g1(x)f (x , y) dy dx
⇒ F (x) =
∫ g2(x)
g1(x)f (x , y) dy and S =
∫ b
a
F (x)dx
with complete flexibility of individual quadrature methods.
Double integral on rectangular domain
Two-dimensional version of Simpson’s one-third rule:
∫ 1
−1
∫ 1
−1f (x , y)dxdy
= w0f (0, 0) + w1[f (−1, 0) + f (1, 0) + f (0,−1) + f (0, 1)]
+ w2[f (−1,−1) + f (−1, 1) + f (1,−1) + f (1, 1)]
Exact for bicubic functions: w0 = 16/9, w1 = 4/9 and w2 = 1/9.
![Page 794: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/794.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 794,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
Monte Carlo integration
I =
∫
Ωf (x)dV
![Page 795: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/795.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 795,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
Monte Carlo integration
I =
∫
Ωf (x)dV
Requirements:
a simple volume V enclosing the domain Ω
a point classification scheme
Generating random points in V ,
F (x) =
f (x) if x ∈ Ω,0 otherwise .
![Page 796: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/796.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 796,
Gaussian QuadratureMultiple IntegralsMultiple Integrals
Monte Carlo integration
I =
∫
Ωf (x)dV
Requirements:
a simple volume V enclosing the domain Ω
a point classification scheme
Generating random points in V ,
F (x) =
f (x) if x ∈ Ω,0 otherwise .
I ≈ V
N
N∑
i=1
F (xi )
Estimate of I (usually) improves with increasing N.
![Page 797: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/797.jpg)
Mathematical Methods in Engineering and Science Advanced Topics in Numerical Integration* 797,
Gaussian QuadratureMultiple IntegralsPoints to note
Basic strategy of Gauss-Legendre quadrature
Formulation of a double integral from fundamental principle
Monte Carlo integration
Necessary Exercises: 2,5,6
![Page 798: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/798.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 798,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Outline
Numerical Solution of Ordinary Differential EquationsSingle-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
![Page 799: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/799.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 799,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial value problem (IVP) of a first order ODE:
dy
dx= f (x , y), y(x0) = y0
To determine: y(x) for x ∈ [a, b] with x0 = a.
![Page 800: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/800.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 800,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial value problem (IVP) of a first order ODE:
dy
dx= f (x , y), y(x0) = y0
To determine: y(x) for x ∈ [a, b] with x0 = a.
Numerical solution: Start from the point (x0, y0).
y1 = y(x1) = y(x0 + h) =?
Found (x1, y1).
![Page 801: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/801.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 801,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial value problem (IVP) of a first order ODE:
dy
dx= f (x , y), y(x0) = y0
To determine: y(x) for x ∈ [a, b] with x0 = a.
Numerical solution: Start from the point (x0, y0).
y1 = y(x1) = y(x0 + h) =?
Found (x1, y1). Repeat up to x = b.
![Page 802: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/802.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 802,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial value problem (IVP) of a first order ODE:
dy
dx= f (x , y), y(x0) = y0
To determine: y(x) for x ∈ [a, b] with x0 = a.
Numerical solution: Start from the point (x0, y0).
y1 = y(x1) = y(x0 + h) =?
Found (x1, y1). Repeat up to x = b.
Information at how many points are used at every step?
Single-step method: Only the current value
Multi-step method: History of several recent steps
![Page 803: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/803.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 803,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Euler’s method
At (xn, yn), evaluate slope dydx
= f (xn, yn).
For a small step h,
yn+1 = yn + hf (xn, yn)
Repitition of such steps constructs y(x).
![Page 804: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/804.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 804,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Euler’s method
At (xn, yn), evaluate slope dydx
= f (xn, yn).
For a small step h,
yn+1 = yn + hf (xn, yn)
Repitition of such steps constructs y(x).
First order truncated Taylor’s series:
Expected error: O(h2)
Accumulation over steps
Total error: O(h)
Euler’s method is a first order method.
![Page 805: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/805.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 805,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Euler’s method
At (xn, yn), evaluate slope dydx
= f (xn, yn).
For a small step h,
yn+1 = yn + hf (xn, yn)
Repitition of such steps constructs y(x).
First order truncated Taylor’s series:
Expected error: O(h2)
Accumulation over steps
Total error: O(h)
Euler’s method is a first order method.
Question: Total error = Sum of errors over the steps?Answer: No, in general.
![Page 806: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/806.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 806,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial slope for the entire step: is it a good idea?
C3
x x x
C2
1
2 31
0y
0 x
y
∆ 3y
C
C
y
xO
3
Figure: Euler’s method
![Page 807: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/807.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 807,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial slope for the entire step: is it a good idea?
C3
x x x
C2
1
2 31
0y
0 x
y
∆ 3y
C
C
y
xO
3
Figure: Euler’s method
1C
0y
0 xx1
C
P1
Q
Q
P
Q*
1
Q2
y
xO
Figure: Improved Euler’s method
Improved Euler’s method or Heun’s method
![Page 808: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/808.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 808,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Initial slope for the entire step: is it a good idea?
C3
x x x
C2
1
2 31
0y
0 x
y
∆ 3y
C
C
y
xO
3
Figure: Euler’s method
1C
0y
0 xx1
C
P1
Q
Q
P
Q*
1
Q2
y
xO
Figure: Improved Euler’s method
Improved Euler’s method or Heun’s method
yn+1 = yn + hf (xn, yn)
yn+1 = yn + h2 [f (xn, yn) + f (xn+1, yn+1)]
The order of Heun’s method is two.
![Page 809: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/809.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 809,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Runge-Kutta methodsSecond order method:
k1 = hf (xn, yn), k2 = hf (xn + αh, yn + βk1)k = w1k1 + w2k2,
and xn+1 = xn + h, yn+1 = yn + k
Force agreement up to the second order.
![Page 810: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/810.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 810,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Runge-Kutta methodsSecond order method:
k1 = hf (xn, yn), k2 = hf (xn + αh, yn + βk1)k = w1k1 + w2k2,
and xn+1 = xn + h, yn+1 = yn + k
Force agreement up to the second order.
yn+1
= yn + w1hf (xn, yn) + w2h[f (xn, yn) + αhfx(xn, yn) + βk1fy (xn, yn) + · ·= yn + (w1 + w2)hf (xn, yn) + h2w2[αfx(xn, yn) + βf (xn, yn)fy (xn, yn)] +
From Taylor’s series, using y ′ = f (x , y) and y ′′ = fx + ffy ,
y(xn+1) = yn +hf (xn, yn)+h2
2[fx(xn, yn)+ f (xn, yn)fy (xn, yn)]+ · · ·
![Page 811: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/811.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 811,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Runge-Kutta methodsSecond order method:
k1 = hf (xn, yn), k2 = hf (xn + αh, yn + βk1)k = w1k1 + w2k2,
and xn+1 = xn + h, yn+1 = yn + k
Force agreement up to the second order.
yn+1
= yn + w1hf (xn, yn) + w2h[f (xn, yn) + αhfx(xn, yn) + βk1fy (xn, yn) + · ·= yn + (w1 + w2)hf (xn, yn) + h2w2[αfx(xn, yn) + βf (xn, yn)fy (xn, yn)] +
From Taylor’s series, using y ′ = f (x , y) and y ′′ = fx + ffy ,
y(xn+1) = yn +hf (xn, yn)+h2
2[fx(xn, yn)+ f (xn, yn)fy (xn, yn)]+ · · ·
w1 + w2 = 1, αw2 = βw2 = 12
![Page 812: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/812.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 812,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
Runge-Kutta methodsSecond order method:
k1 = hf (xn, yn), k2 = hf (xn + αh, yn + βk1)k = w1k1 + w2k2,
and xn+1 = xn + h, yn+1 = yn + k
Force agreement up to the second order.
yn+1
= yn + w1hf (xn, yn) + w2h[f (xn, yn) + αhfx(xn, yn) + βk1fy (xn, yn) + · ·= yn + (w1 + w2)hf (xn, yn) + h2w2[αfx(xn, yn) + βf (xn, yn)fy (xn, yn)] +
From Taylor’s series, using y ′ = f (x , y) and y ′′ = fx + ffy ,
y(xn+1) = yn +hf (xn, yn)+h2
2[fx(xn, yn)+ f (xn, yn)fy (xn, yn)]+ · · ·
w1 + w2 = 1, αw2 = βw2 = 12 ⇒ α = β = 1
2w2, w1 = 1− w2
![Page 813: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/813.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 813,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
With continuous choice of w2,
a family of second order Runge Kutta (RK2) formulae
Popular form of RK2: with choice w2 = 1,
k1 = hf (xn, yn), k2 = hf (xn + h2 , yn + k1
2 )xn+1 = xn + h, yn+1 = yn + k2
![Page 814: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/814.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 814,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Single-Step Methods
With continuous choice of w2,
a family of second order Runge Kutta (RK2) formulae
Popular form of RK2: with choice w2 = 1,
k1 = hf (xn, yn), k2 = hf (xn + h2 , yn + k1
2 )xn+1 = xn + h, yn+1 = yn + k2
Fourth order Runge-Kutta method (RK4):
k1 = hf (xn, yn)
k2 = hf (xn + h2 , yn + k1
2 )
k3 = hf (xn + h2 , yn + k2
2 )k4 = hf (xn + h, yn + k3)
k = 16(k1 + 2k2 + 2k3 + k4)
xn+1 = xn + h, yn+1 = yn + k
![Page 815: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/815.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 815,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Question: How to decide whether the error is within tolerance?
![Page 816: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/816.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 816,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Question: How to decide whether the error is within tolerance?Additional estimates:
handle to monitor the error further efficient algorithms
![Page 817: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/817.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 817,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Question: How to decide whether the error is within tolerance?Additional estimates:
handle to monitor the error further efficient algorithms
Runge-Kutta method with adaptive step sizeIn an interval [xn, xn + h],
y(1)n+1 = yn+1 + ch5 + higher order terms
Over two steps of size h2 ,
y(2)n+1 = yn+1 + 2c
(
h
2
)5
+ higher order terms
![Page 818: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/818.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 818,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Question: How to decide whether the error is within tolerance?Additional estimates:
handle to monitor the error further efficient algorithms
Runge-Kutta method with adaptive step sizeIn an interval [xn, xn + h],
y(1)n+1 = yn+1 + ch5 + higher order terms
Over two steps of size h2 ,
y(2)n+1 = yn+1 + 2c
(
h
2
)5
+ higher order terms
Difference of two estimates:
∆ = y(1)n+1 − y
(2)n+1 ≈
15
16ch5
Best available value: y∗n+1 = y
(2)n+1 − ∆
15 =16y
(2)n+1−y
(1)n+1
15
![Page 819: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/819.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 819,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Evaluation of a step:
∆ > ǫ: Step size is too large for accuracy.Subdivide the interval.
∆ << ǫ: Step size is inefficient!
![Page 820: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/820.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 820,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Evaluation of a step:
∆ > ǫ: Step size is too large for accuracy.Subdivide the interval.
∆ << ǫ: Step size is inefficient!
Start with a large step size.Keep subdividing intervals whenever ∆ > ǫ.
Fast marching over smooth segments and small steps inzones featured with rapid changes in y(x).
![Page 821: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/821.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 821,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Practical Implementation of Single-Step Methods
Evaluation of a step:
∆ > ǫ: Step size is too large for accuracy.Subdivide the interval.
∆ << ǫ: Step size is inefficient!
Start with a large step size.Keep subdividing intervals whenever ∆ > ǫ.
Fast marching over smooth segments and small steps inzones featured with rapid changes in y(x).
Runge-Kutta-Fehlberg method
With six function values,
An RK4 formula embedded in an RK5 formula
two independent estimates and an error estimate!
RKF45 in professional implementations
![Page 822: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/822.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 822,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
Methods for a single first order ODE
directly applicable to a first order vector ODE
A typical IVP with an ODE system:
dy
dx= f(x , y), y(x0) = y0
![Page 823: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/823.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 823,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
Methods for a single first order ODE
directly applicable to a first order vector ODE
A typical IVP with an ODE system:
dy
dx= f(x , y), y(x0) = y0
An n-th order ODE: convert into a system of first order ODE’s
Defining state vector z(x) = [y(x) y ′(x) · · · y (n−1)(x)]T ,
work out dzdx
to form the state space equation.
Initial condition: z(x0) = [y(x0) y ′(x0) · · · y (n−1)(x0)]T
![Page 824: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/824.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 824,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
Methods for a single first order ODE
directly applicable to a first order vector ODE
A typical IVP with an ODE system:
dy
dx= f(x , y), y(x0) = y0
An n-th order ODE: convert into a system of first order ODE’s
Defining state vector z(x) = [y(x) y ′(x) · · · y (n−1)(x)]T ,
work out dzdx
to form the state space equation.
Initial condition: z(x0) = [y(x0) y ′(x0) · · · y (n−1)(x0)]T
A system of higher order ODE’s with the highest order derivativesof orders n1, n2, n3, · · · , nk
Cast into the state space form with the state vector ofdimension n = n1 + n2 + n3 + · · ·+ nk
![Page 825: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/825.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 825,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
State space formulation is directly applicable when
the highest order derivatives can be solved explicitly.
The resulting form of the ODE’s: normal system of ODE’s
![Page 826: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/826.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 826,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
State space formulation is directly applicable when
the highest order derivatives can be solved explicitly.
The resulting form of the ODE’s: normal system of ODE’s
Example:
yd2x
dt2− 3
(
dy
dt
)(
dx
dt
)2
+ 2x
(
dx
dt
)
√
d2y
dt2+ 4 = 0
exy d3y
dt3− y
(
d2y
dt2
)3/2
+ 2x + 1 = e−t
![Page 827: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/827.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 827,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Systems of ODE’s
State space formulation is directly applicable when
the highest order derivatives can be solved explicitly.
The resulting form of the ODE’s: normal system of ODE’s
Example:
yd2x
dt2− 3
(
dy
dt
)(
dx
dt
)2
+ 2x
(
dx
dt
)
√
d2y
dt2+ 4 = 0
exy d3y
dt3− y
(
d2y
dt2
)3/2
+ 2x + 1 = e−t
State vector: z(t) =[
x dxdt
y dydt
d2ydt2
]T
With three trivial derivatives z ′1(t) = z2, z ′3(t) = z4 and z ′4(t) = z5
and the other two obtained from the given ODE’s,
we get the state space equations as dzdt
= f(t, z).
![Page 828: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/828.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 828,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Multi-Step Methods*
Single-step methods: every step a brand new IVP!
Why not try to capture the trend?
![Page 829: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/829.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 829,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Multi-Step Methods*
Single-step methods: every step a brand new IVP!
Why not try to capture the trend?
A typical multi-step formula:
yn+1 = yn + h[c0f (xn+1, yn+1) + c1f (xn, yn)
+ c2f (xn−1, yn−1) + c3f (xn−2, yn−2) + · · · ]
Determine coefficients by demanding the exactness for leadingpolynomial terms.
![Page 830: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/830.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 830,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Multi-Step Methods*
Single-step methods: every step a brand new IVP!
Why not try to capture the trend?
A typical multi-step formula:
yn+1 = yn + h[c0f (xn+1, yn+1) + c1f (xn, yn)
+ c2f (xn−1, yn−1) + c3f (xn−2, yn−2) + · · · ]
Determine coefficients by demanding the exactness for leadingpolynomial terms.
Explicit methods: c0 = 0, evaluation easy, but involvesextrapolation.
Implicit methods: c0 6= 0, difficult to evaluate, but better stability.
Predictor-corrector methods
Example: Adams-Bashforth-Moulton method
![Page 831: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/831.jpg)
Mathematical Methods in Engineering and Science Numerical Solution of Ordinary Differential Equations 831,
Single-Step MethodsPractical Implementation of Single-Step MethodsSystems of ODE’sMulti-Step Methods*
Points to note
Euler’s and Runge-Kutta methods
Step size adaptation
State space formulation of dynamic systems
Necessary Exercises: 1,2,5,6
![Page 832: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/832.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 832,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Outline
ODE Solutions: Advanced IssuesStability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
![Page 833: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/833.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 833,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Adaptive RK4 is an extremely successful method.
But, its scope has a limitation.
Focus of explicit methods (such as RK) is accuracy and efficiency.
The issue of stabilty is handled indirectly.
![Page 834: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/834.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 834,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Adaptive RK4 is an extremely successful method.
But, its scope has a limitation.
Focus of explicit methods (such as RK) is accuracy and efficiency.
The issue of stabilty is handled indirectly.
Stabilty of explicit methodsFor the ODE system y′ = f(x , y), Euler’s method gives
yn+1 = yn + f(xn, yn)h +O(h2).
Taylor’s series of the actual solution:
y(xn+1) = y(xn) + f(xn, y(xn))h +O(h2)
![Page 835: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/835.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 835,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Adaptive RK4 is an extremely successful method.
But, its scope has a limitation.
Focus of explicit methods (such as RK) is accuracy and efficiency.
The issue of stabilty is handled indirectly.
Stabilty of explicit methodsFor the ODE system y′ = f(x , y), Euler’s method gives
yn+1 = yn + f(xn, yn)h +O(h2).
Taylor’s series of the actual solution:
y(xn+1) = y(xn) + f(xn, y(xn))h +O(h2)
Discrepancy or error:
∆n+1 = yn+1 − y(xn+1)
= [yn − y(xn)] + [f(xn, yn)− f(xn, y(xn))]h +O(h2)
= ∆n +
[
∂f
∂y(xn, yn)∆n
]
h +O(h2) ≈ (I + hJ)∆n
![Page 836: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/836.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 836,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Euler’s step magnifies the error by a factor (I + hJ).
![Page 837: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/837.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 837,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Euler’s step magnifies the error by a factor (I + hJ).
Using J loosely as the representative Jacobian,
∆n+1 ≈ (I + hJ)n∆1.
For stability, ∆n+1 → 0 as n→∞.
Eigenvalues of (I + hJ) must fall within the unit circle|z | = 1. By shift theorem, eigenvalues of hJ must fallinside the unit circle with the centre at z0 = −1.
|1 + hλ| < 1 ⇒ h <−2Re (λ)
|λ|2
![Page 838: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/838.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 838,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
Euler’s step magnifies the error by a factor (I + hJ).
Using J loosely as the representative Jacobian,
∆n+1 ≈ (I + hJ)n∆1.
For stability, ∆n+1 → 0 as n→∞.
Eigenvalues of (I + hJ) must fall within the unit circle|z | = 1. By shift theorem, eigenvalues of hJ must fallinside the unit circle with the centre at z0 = −1.
|1 + hλ| < 1 ⇒ h <−2Re (λ)
|λ|2
Note: Same result for single ODE w ′ = λw , with complex λ.For second order Runge-Kutta method,
∆n+1 =
[
1 + hλ+h2λ2
2
]
∆n
Region of stability in the plane of z = hλ:∣
∣
∣1 + z + z2
2
∣
∣
∣< 1
![Page 839: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/839.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 839,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
−5 −4 −3 −2 −1 0 1 2 3
−3
−2
−1
0
1
2
3
Re(hλ)
Im(h
λ)
O
RK2
RK4
Euler
UNSTABLE
UNSTABLE
Figure: Stability regions of explicit methods
Question: What do these stability regions mean with reference tothe system eigenvalues?Question: How does the step size adaptation of RK4 operate on asystem with eigenvalues on the left half of complex plane?
![Page 840: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/840.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 840,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stability Analysis
−5 −4 −3 −2 −1 0 1 2 3
−3
−2
−1
0
1
2
3
Re(hλ)
Im(h
λ)
O
RK2
RK4
Euler
UNSTABLE
UNSTABLE
Figure: Stability regions of explicit methods
Question: What do these stability regions mean with reference tothe system eigenvalues?Question: How does the step size adaptation of RK4 operate on asystem with eigenvalues on the left half of complex plane?
Step size adaptation tackles instability by its symptom!
![Page 841: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/841.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 841,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
Backward Euler’s method
yn+1 = yn + f(xn+1, yn+1)h
Solve it?
![Page 842: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/842.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 842,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
Backward Euler’s method
yn+1 = yn + f(xn+1, yn+1)h
Solve it? Is it worth solving?
∆n+1 ≈ yn+1 − y(xn+1)
= [yn − y(xn)] + h[f(xn+1, yn+1)− f(xn+1, y(xn+1))]
= ∆n + hJ(xn+1, yn+1)∆n+1
Notice the flip in the form of this equation.
![Page 843: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/843.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 843,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
Backward Euler’s method
yn+1 = yn + f(xn+1, yn+1)h
Solve it? Is it worth solving?
∆n+1 ≈ yn+1 − y(xn+1)
= [yn − y(xn)] + h[f(xn+1, yn+1)− f(xn+1, y(xn+1))]
= ∆n + hJ(xn+1, yn+1)∆n+1
Notice the flip in the form of this equation.
∆n+1 ≈ (I − hJ)−1∆n
Stability: eigenvalues of (I − hJ) outside the unit circle |z | = 1
|hλ− 1| > 1 ⇒ h >2Re (λ)
|λ|2Absolute stability for a stable ODE, i.e. one with Re (λ) < 0
![Page 844: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/844.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 844,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Re(hλ)
Im(h
λ)
O
STABLE
STABLE
STABLE
UNSTABLE
Figure: Stability region of backward Euler’s method
![Page 845: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/845.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 845,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Re(hλ)
Im(h
λ)
O
STABLE
STABLE
STABLE
UNSTABLE
Figure: Stability region of backward Euler’s method
How to solve g(yn+1) = yn + hf(xn+1, yn+1)− yn+1 = 0 for yn+1?
![Page 846: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/846.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 846,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Implicit Methods
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Re(hλ)
Im(h
λ)
O
STABLE
STABLE
STABLE
UNSTABLE
Figure: Stability region of backward Euler’s method
How to solve g(yn+1) = yn + hf(xn+1, yn+1)− yn+1 = 0 for yn+1?Typical Newton’s iteration:
y(k+1)n+1 = y
(k)n+1 + (I − hJ)−1
[
yn − y(k)n+1 + hf
(
xn+1, y(k)n+1
)]
Semi-implicit Euler’s method for local solution:
yn+1 = yn + h(I − hJ)−1f(xn+1, yn)
![Page 847: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/847.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 847,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stiff Differential Equations
Example: IVP of a mass-spring-damper system:
x + cx + kx = 0, x(0) = 0, x(0) = 1
(a) c = 3, k = 2: x = e−t − e−2t
(b) c = 49, k = 600: x = e−24t − e−25t
0 0.5 1 1.5 2 2.5 3 3.5 4−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
t
x
(a) Case of c = 3, k = 2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
t
x
(b) Case of c = 49, k = 600
Figure: Solutions of a mass-spring-damper system: ordinary situations
![Page 848: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/848.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 848,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stiff Differential Equations
(c) c = 302, k = 600: x = e−2t−e−300t
298
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−4
−3
−2
−1
0
1
2
3
4x 10
−3
t
x
(c) With RK4
Figure: Solutions of a mass-spring-damper system: stiff situation
![Page 849: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/849.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 849,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Stiff Differential Equations
(c) c = 302, k = 600: x = e−2t−e−300t
298
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−4
−3
−2
−1
0
1
2
3
4x 10
−3
t
x
(c) With RK4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−4
−3
−2
−1
0
1
2
3
4x 10
−3
t
x(d) With implicit Euler
Figure: Solutions of a mass-spring-damper system: stiff situation
To solve stiff ODE systems,
use implicit method, preferably with explicit Jacobian.
![Page 850: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/850.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 850,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
A paradigm shift from the initial value problems
A ball is thrown with a particular velocity. What trajectorydoes the ball follow?
How to throw a ball such that it hits a particular window at aneighbouring house after 15 seconds?
![Page 851: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/851.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 851,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
A paradigm shift from the initial value problems
A ball is thrown with a particular velocity. What trajectorydoes the ball follow?
How to throw a ball such that it hits a particular window at aneighbouring house after 15 seconds?
Two-point BVP in ODE’s:
boundary conditions at two values of the independentvariable
![Page 852: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/852.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 852,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
A paradigm shift from the initial value problems
A ball is thrown with a particular velocity. What trajectorydoes the ball follow?
How to throw a ball such that it hits a particular window at aneighbouring house after 15 seconds?
Two-point BVP in ODE’s:
boundary conditions at two values of the independentvariable
Methods of solution
Shooting method
Finite difference (relaxation) method
Finite element method
![Page 853: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/853.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 853,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Shooting method
follows the strategy to adjust trials to hit a target.
Consider the 2-point BVP
y′ = f(x , y), g1(y(a)) = 0, g2(y(b)) = 0,
where g1 ∈ Rn1, g2 ∈ Rn2 and n1 + n2 = n.
Parametrize initial state: y(a) = h(p) with p ∈ Rn2.
Guess n2 values of p to define IVP
y′ = f(x , y), y(a) = h(p).
Solve this IVP for [a, b] and evaluate y(b).
Define error vector E(p) = g2(y(b)).
![Page 854: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/854.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 854,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Objective: To solve E(p) = 0
From current vector p, n2 perturbations as p + eiδ: Jacobian ∂E∂p
Each Newton’s step: solution of n2 + 1 initial valueproblems!
![Page 855: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/855.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 855,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Objective: To solve E(p) = 0
From current vector p, n2 perturbations as p + eiδ: Jacobian ∂E∂p
Each Newton’s step: solution of n2 + 1 initial valueproblems!
Computational cost
Convergence not guaranteed (initial guess important)
![Page 856: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/856.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 856,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Objective: To solve E(p) = 0
From current vector p, n2 perturbations as p + eiδ: Jacobian ∂E∂p
Each Newton’s step: solution of n2 + 1 initial valueproblems!
Computational cost
Convergence not guaranteed (initial guess important)
Merits of shooting method
Very few parameters to start
In many cases, it is found quite efficient.
![Page 857: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/857.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 857,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Finite difference (relaxation) method
adopts a global perspective.
1. Discretize domain [a, b]: grid of pointsa = x0 < x1 < x2 < · · · < xN−1 < xN = b.Function values y(xi ): n(N + 1) unknowns
2. Replace the ODE over intervals by finite difference equations.Considering mid-points, a typical (vector) FDE:
yi−yi−1−hf
(
xi + xi−1
2,yi + yi−1
2
)
= 0, for i = 1, 2, 3, · · · ,N
nN (scalar) equations
3. Assemble additional n equations from boundary conditions.
4. Starting from a guess solution over the grid, solve this system.(Sparse Jacobian is an advantage.)
![Page 858: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/858.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 858,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Boundary Value Problems
Finite difference (relaxation) method
adopts a global perspective.
1. Discretize domain [a, b]: grid of pointsa = x0 < x1 < x2 < · · · < xN−1 < xN = b.Function values y(xi ): n(N + 1) unknowns
2. Replace the ODE over intervals by finite difference equations.Considering mid-points, a typical (vector) FDE:
yi−yi−1−hf
(
xi + xi−1
2,yi + yi−1
2
)
= 0, for i = 1, 2, 3, · · · ,N
nN (scalar) equations
3. Assemble additional n equations from boundary conditions.
4. Starting from a guess solution over the grid, solve this system.(Sparse Jacobian is an advantage.)
Iterative schemes for solution of systems of linear equations.
![Page 859: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/859.jpg)
Mathematical Methods in Engineering and Science ODE Solutions: Advanced Issues 859,
Stability AnalysisImplicit MethodsStiff Differential EquationsBoundary Value Problems
Points to note
Numerical stability of ODE solution methods
Computational cost versus better stability of implicit methods
Multiscale responses leading to stiffness: failure of explicitmethods
Implicit methods for stiff systems
Shooting method for two-point boundary value problems
Relaxation method for boundary value problems
Necessary Exercises: 1,2,3,4,5
![Page 860: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/860.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 860,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Outline
Existence and Uniqueness TheoryWell-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
![Page 861: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/861.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 861,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value ProblemsPierre Simon de Laplace (1749-1827):
”We may regard the present state of the
universe as the effect of its past and the
cause of its future. An intellect which at a
certain moment would know all forces that
set nature in motion, and all positions of all
items of which nature is composed, if this
intellect were also vast enough to submit
these data to analysis, it would embrace in a
single formula the movements of the greatest
bodies of the universe and those of the
tiniest atom; for such an intellect nothing
would be uncertain and the future just like
the past would be present before its eyes.”
![Page 862: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/862.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 862,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Initial value problem
y ′ = f (x , y), y(x0) = y0
From (x , y), the trajectory develops according to y ′ = f (x , y).
The new point: (x + δx , y + f (x , y)δx)The slope now: f (x + δx , y + f (x , y)δx)
Question: Was the old direction of approach valid?
![Page 863: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/863.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 863,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Initial value problem
y ′ = f (x , y), y(x0) = y0
From (x , y), the trajectory develops according to y ′ = f (x , y).
The new point: (x + δx , y + f (x , y)δx)The slope now: f (x + δx , y + f (x , y)δx)
Question: Was the old direction of approach valid?
With δx → 0, directions appropriate, if
limx→x
f (x , y) = f (x , y(x)),
i.e. if f (x , y) is continuous.
![Page 864: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/864.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 864,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Initial value problem
y ′ = f (x , y), y(x0) = y0
From (x , y), the trajectory develops according to y ′ = f (x , y).
The new point: (x + δx , y + f (x , y)δx)The slope now: f (x + δx , y + f (x , y)δx)
Question: Was the old direction of approach valid?
With δx → 0, directions appropriate, if
limx→x
f (x , y) = f (x , y(x)),
i.e. if f (x , y) is continuous.
If f (x , y) =∞, then y ′ =∞ and trajectory is vertical.
For the same value of x, several values of y!
y(x) not a function, unless f (x , y) 6=∞, i.e. f (x , y) is bounded.
![Page 865: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/865.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 865,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value ProblemsPeano’s theorem: If f (x , y) is continuous and bounded in arectangle R = (x , y) : |x − x0| < h, |y − y0| < k, with|f (x , y)| ≤ M <∞, then the IVP y ′ = f (x , y), y(x0) = y0 has asolution y(x) defined in a neighbourhood of x0.
![Page 866: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/866.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 866,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value ProblemsPeano’s theorem: If f (x , y) is continuous and bounded in arectangle R = (x , y) : |x − x0| < h, |y − y0| < k, with|f (x , y)| ≤ M <∞, then the IVP y ′ = f (x , y), y(x0) = y0 has asolution y(x) defined in a neighbourhood of x0.
),y0(x0(x),y0(x0(x
xx0−h x0 x0+h
y0−k
0+k
k
k Mh
Mh
OO
y0−k
y0+k
k
k
Mh
Mh
xx0 xxxxx0x0 −h +h
h h hh
y y
y
(a) Mh <= k (b) Mh >= k
0y
0y
Figure: Regions containing the trajectories
Guaranteed neighbourhood:
[x0 − δ, x0 + δ], where δ = min(h, kM
) > 0
![Page 867: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/867.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 867,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Example:
y ′ =y − 1
x, y(0) = 1
Function f (x , y) = y−1x
undefined at (0, 1).
Premises of existence theorem not satisfied.
![Page 868: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/868.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 868,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Example:
y ′ =y − 1
x, y(0) = 1
Function f (x , y) = y−1x
undefined at (0, 1).
Premises of existence theorem not satisfied.
But, premises here are sufficient, not necessary!
Result inconclusive.
![Page 869: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/869.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 869,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Example:
y ′ =y − 1
x, y(0) = 1
Function f (x , y) = y−1x
undefined at (0, 1).
Premises of existence theorem not satisfied.
But, premises here are sufficient, not necessary!
Result inconclusive.
The IVP has solutions: y(x) = 1 + cx for all values of c .
The solution is not unique.
![Page 870: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/870.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 870,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Example:
y ′ =y − 1
x, y(0) = 1
Function f (x , y) = y−1x
undefined at (0, 1).
Premises of existence theorem not satisfied.
But, premises here are sufficient, not necessary!
Result inconclusive.
The IVP has solutions: y(x) = 1 + cx for all values of c .
The solution is not unique.
Example: y ′2 = |y |, y(0) = 0
Existence theorem guarantees a solution.
But, there are two solutions:
y(x) = 0 and y(x) = sgn(x) x2/4.
![Page 871: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/871.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 871,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Physical system to mathematical model
Mathematical solution Interpretation about the physical system
Meanings of non-uniqueness of a solution
Mathematical model admits of extraneous solution(s)?
Physical system itself can exhibit alternative behaviours?
![Page 872: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/872.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 872,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Physical system to mathematical model
Mathematical solution Interpretation about the physical system
Meanings of non-uniqueness of a solution
Mathematical model admits of extraneous solution(s)?
Physical system itself can exhibit alternative behaviours?
Indeterminacy of the solution
Mathematical model of the system is not complete.
The initial value problem is not well-posed.
![Page 873: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/873.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 873,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Physical system to mathematical model
Mathematical solution Interpretation about the physical system
Meanings of non-uniqueness of a solution
Mathematical model admits of extraneous solution(s)?
Physical system itself can exhibit alternative behaviours?
Indeterminacy of the solution
Mathematical model of the system is not complete.
The initial value problem is not well-posed.
After existence, next important question:
Uniqueness of a solution
![Page 874: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/874.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 874,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Continuous dependence on initial condition
Suppose that for IVP y ′ = f (x , y), y(x0) = y0,
unique solution: y1(x).
Applying a small perturbation to the initial condition, the new IVP:y ′ = f (x , y), y(x0) = y0 + ǫ
unique solution: y2(x)
Question: By how much y2(x) differs from y1(x) for x > x0?
![Page 875: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/875.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 875,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Continuous dependence on initial condition
Suppose that for IVP y ′ = f (x , y), y(x0) = y0,
unique solution: y1(x).
Applying a small perturbation to the initial condition, the new IVP:y ′ = f (x , y), y(x0) = y0 + ǫ
unique solution: y2(x)
Question: By how much y2(x) differs from y1(x) for x > x0?
Large difference: solution sensitive to initial condition
Practically unreliable solution
![Page 876: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/876.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 876,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Well-Posedness of Initial Value Problems
Continuous dependence on initial condition
Suppose that for IVP y ′ = f (x , y), y(x0) = y0,
unique solution: y1(x).
Applying a small perturbation to the initial condition, the new IVP:y ′ = f (x , y), y(x0) = y0 + ǫ
unique solution: y2(x)
Question: By how much y2(x) differs from y1(x) for x > x0?
Large difference: solution sensitive to initial condition
Practically unreliable solution
Well-posed IVP:
An initial value problem is said to be well-posed if thereexists a solution to it, the solution is unique and itdepends continuously on the initial conditions.
![Page 877: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/877.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 877,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
Lipschitz condition:
|f (x , y)− f (x , z)| ≤ L|y − z |L: finite positive constant (Lipschitz constant)
Theorem: If f (x , y) is a continuous function satisfying aLipschitz condition on a stripS = (x , y) : a < x < b,−∞ < y <∞, then for anypoint (x0, y0) ∈ S, the initial value problem ofy ′ = f (x , y), y(x0) = y0 is well-posed.
![Page 878: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/878.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 878,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
Lipschitz condition:
|f (x , y)− f (x , z)| ≤ L|y − z |L: finite positive constant (Lipschitz constant)
Theorem: If f (x , y) is a continuous function satisfying aLipschitz condition on a stripS = (x , y) : a < x < b,−∞ < y <∞, then for anypoint (x0, y0) ∈ S, the initial value problem ofy ′ = f (x , y), y(x0) = y0 is well-posed.
Assume y1(x) and y2(x): solutions of the ODE y ′ = f (x , y) withinitial conditions y(x0) = (y1)0 and y(x0) = (y2)0Consider E (x) = [y1(x)− y2(x)]2.
E ′(x) = 2(y1 − y2)(y′1 − y ′
2) = 2(y1 − y2)[f (x , y1)− f (x , y2)]
![Page 879: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/879.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 879,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
Lipschitz condition:
|f (x , y)− f (x , z)| ≤ L|y − z |L: finite positive constant (Lipschitz constant)
Theorem: If f (x , y) is a continuous function satisfying aLipschitz condition on a stripS = (x , y) : a < x < b,−∞ < y <∞, then for anypoint (x0, y0) ∈ S, the initial value problem ofy ′ = f (x , y), y(x0) = y0 is well-posed.
Assume y1(x) and y2(x): solutions of the ODE y ′ = f (x , y) withinitial conditions y(x0) = (y1)0 and y(x0) = (y2)0Consider E (x) = [y1(x)− y2(x)]2.
E ′(x) = 2(y1 − y2)(y′1 − y ′
2) = 2(y1 − y2)[f (x , y1)− f (x , y2)]
Applying Lipschitz condition,
|E ′(x)| ≤ 2L(y1 − y2)2 = 2LE (x).
Need to consider the case of E ′(x) ≥ 0 only.
![Page 880: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/880.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 880,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
E ′(x)
E (x)≤ 2L ⇒
∫ x
x0
E ′(x)
E (x)dx ≤ 2L(x − x0)
Integrating, E (x) ≤ E (x0)e2L(x−x0).
![Page 881: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/881.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 881,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
E ′(x)
E (x)≤ 2L ⇒
∫ x
x0
E ′(x)
E (x)dx ≤ 2L(x − x0)
Integrating, E (x) ≤ E (x0)e2L(x−x0).
Hence,|y1(x) − y2(x)| ≤ eL(x−x0)|(y1)0 − (y2)0|.
![Page 882: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/882.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 882,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
E ′(x)
E (x)≤ 2L ⇒
∫ x
x0
E ′(x)
E (x)dx ≤ 2L(x − x0)
Integrating, E (x) ≤ E (x0)e2L(x−x0).
Hence,|y1(x) − y2(x)| ≤ eL(x−x0)|(y1)0 − (y2)0|.
Since x ∈ [a, b], eL(x−x0) is finite.
|(y1)0 − (y2)0| = ǫ ⇒ |y1(x)− y2(x)| ≤ eL(x−x0)ǫ
continuous dependence of the solution on initial condition
![Page 883: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/883.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 883,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
E ′(x)
E (x)≤ 2L ⇒
∫ x
x0
E ′(x)
E (x)dx ≤ 2L(x − x0)
Integrating, E (x) ≤ E (x0)e2L(x−x0).
Hence,|y1(x) − y2(x)| ≤ eL(x−x0)|(y1)0 − (y2)0|.
Since x ∈ [a, b], eL(x−x0) is finite.
|(y1)0 − (y2)0| = ǫ ⇒ |y1(x)− y2(x)| ≤ eL(x−x0)ǫ
continuous dependence of the solution on initial condition
In particular, (y1)0 = (y2)0 = y0 ⇒ y1(x) = y2(x) ∀ x ∈ [a, b].
The initial value problem is well-posed.
![Page 884: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/884.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 884,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
A weaker theorem (hypotheses are stronger):
Picard’s theorem: If f (x , y) and ∂f∂y
are continuous andbounded on a rectangleR = (x , y) : a < x < b, c < y < d, then for every(x0, y0) ∈ R, the IVP y ′ = f (x , y), y(x0) = y0 has aunique solution in some neighbourhood |x − x0| ≤ h.
![Page 885: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/885.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 885,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
A weaker theorem (hypotheses are stronger):
Picard’s theorem: If f (x , y) and ∂f∂y
are continuous andbounded on a rectangleR = (x , y) : a < x < b, c < y < d, then for every(x0, y0) ∈ R, the IVP y ′ = f (x , y), y(x0) = y0 has aunique solution in some neighbourhood |x − x0| ≤ h.
From the mean value theorem,
f (x , y1)− f (x , y2) =∂f
∂y(x , ξ)(y1 − y2).
With Lipschitz constant L = sup∣
∣
∣
∂f∂y
∣
∣
∣,
Lipschitz condition is satisfied ‘lavishly’ !
![Page 886: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/886.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 886,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Uniqueness Theorems
A weaker theorem (hypotheses are stronger):
Picard’s theorem: If f (x , y) and ∂f∂y
are continuous andbounded on a rectangleR = (x , y) : a < x < b, c < y < d, then for every(x0, y0) ∈ R, the IVP y ′ = f (x , y), y(x0) = y0 has aunique solution in some neighbourhood |x − x0| ≤ h.
From the mean value theorem,
f (x , y1)− f (x , y2) =∂f
∂y(x , ξ)(y1 − y2).
With Lipschitz constant L = sup∣
∣
∣
∂f∂y
∣
∣
∣,
Lipschitz condition is satisfied ‘lavishly’ !
Note: All these theorems give only sufficient conditions!Hypotheses of Picard’s theorem ⇒ Lipschitz condition ⇒Well-posedness ⇒ Existence and uniqueness
![Page 887: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/887.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 887,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Extension to ODE Systems
For ODE System
dy
dx= f(x , y), y(x0) = y0
Lipschitz condition:
‖f(x , y) − f(x , z)‖ ≤ L‖y − z‖
Scalar function E (x) generalized as
E (x) = ‖y1(x) − y2(x)‖2 = (y1 − y2)T (y1 − y2)
Partial derivative ∂f∂y
replaced by the Jacobian A = ∂f∂y
Boundedness to be inferred from the boundedness of its norm
With these generalizations, the formulations work as usual.
![Page 888: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/888.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 888,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Extension to ODE Systems
IVP of linear first order ODE system
y′ = A(x)y + g(x), y(x0) = y0
Rate function: f(x , y) = A(x)y + g(x)
Continuity and boundedness of the coefficient functionsin A(x) and g(x) are sufficient for well-posedness.
![Page 889: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/889.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 889,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Extension to ODE Systems
IVP of linear first order ODE system
y′ = A(x)y + g(x), y(x0) = y0
Rate function: f(x , y) = A(x)y + g(x)
Continuity and boundedness of the coefficient functionsin A(x) and g(x) are sufficient for well-posedness.
An n-th order linear ordinary differential equation
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = R(x)
State vector: z = [y y ′ y ′′ · · · y (n−1)]T
With z ′1 = z2, z ′2 = z3, · · · , z ′n−1 = zn and z ′n from the ODE,
state space equation in the form z′ = A(x)z + g(x)
Continuity and boundedness of P1(x),P2(x), · · · ,Pn(x)and R(x) guarantees well-posedness.
![Page 890: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/890.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 890,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Closure
A practical by-product of existence and uniqueness results:
important results concerning the solutions
![Page 891: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/891.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 891,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Closure
A practical by-product of existence and uniqueness results:
important results concerning the solutions
A sizeable segment of current research: ill-posed problems
Dynamics of some nonlinear systems Chaos: sensitive dependence on initial conditions
![Page 892: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/892.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 892,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Closure
A practical by-product of existence and uniqueness results:
important results concerning the solutions
A sizeable segment of current research: ill-posed problems
Dynamics of some nonlinear systems Chaos: sensitive dependence on initial conditions
For boundary value problems,
No general criteria for existence and uniqueness
![Page 893: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/893.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 893,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Closure
A practical by-product of existence and uniqueness results:
important results concerning the solutions
A sizeable segment of current research: ill-posed problems
Dynamics of some nonlinear systems Chaos: sensitive dependence on initial conditions
For boundary value problems,
No general criteria for existence and uniqueness
Note: Taking clue from the shooting method, a BVP in ODE’scan be visualized as a complicated root-finding problem!
Multiple solutions or non-existence of solution is no surprise.
![Page 894: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/894.jpg)
Mathematical Methods in Engineering and Science Existence and Uniqueness Theory 894,
Well-Posedness of Initial Value ProblemsUniqueness TheoremsExtension to ODE SystemsClosure
Points to note
For a solution of initial value problems, questions of existence,uniqueness and continuous dependence on initial condition areof crucial importance.
These issues pertain to aspects of practical relevanceregarding a physical system and its dynamic simulation
Lipschitz condition is the tightest (available) criterion fordeciding these questions regarding well-posedness
Necessary Exercises: 1,2
![Page 895: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/895.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 895,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Outline
First Order Ordinary Differential EquationsFormation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the Exact FormFirst Order Linear (Leibnitz) ODE and Associated FormsOrthogonal TrajectoriesModelling and Simulation
![Page 896: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/896.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 896,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Formation of Differential Equations and Their Solutions
A differential equation represents a class of functions.
Example: y(x) = cxk
![Page 897: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/897.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 897,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Formation of Differential Equations and Their Solutions
A differential equation represents a class of functions.
Example: y(x) = cxk
With dydx
= ckxk−1 and d2ydx2 = ck(k − 1)xk−2,
xyd2y
dx2= x
(
dy
dx
)2
− ydy
dx
A compact ‘intrinsic’ description.
![Page 898: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/898.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 898,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Formation of Differential Equations and Their Solutions
A differential equation represents a class of functions.
Example: y(x) = cxk
With dydx
= ckxk−1 and d2ydx2 = ck(k − 1)xk−2,
xyd2y
dx2= x
(
dy
dx
)2
− ydy
dx
A compact ‘intrinsic’ description.
Important terms
Order and degree of differential equations
Homogeneous and non-homogeneous ODE’s
![Page 899: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/899.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 899,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Formation of Differential Equations and Their Solutions
A differential equation represents a class of functions.
Example: y(x) = cxk
With dydx
= ckxk−1 and d2ydx2 = ck(k − 1)xk−2,
xyd2y
dx2= x
(
dy
dx
)2
− ydy
dx
A compact ‘intrinsic’ description.
Important terms
Order and degree of differential equations
Homogeneous and non-homogeneous ODE’s
Solution of a differential equation
general, particular and singular solutions
![Page 900: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/900.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 900,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Separation of Variables
ODE form with separable variables:
y ′ = f (x , y) ⇒ dy
dx=φ(x)
ψ(y)or ψ(y)dy = φ(x)dx
![Page 901: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/901.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 901,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Separation of Variables
ODE form with separable variables:
y ′ = f (x , y) ⇒ dy
dx=φ(x)
ψ(y)or ψ(y)dy = φ(x)dx
Solution as quadrature:
∫
ψ(y)dy =
∫
φ(x)dx + c .
![Page 902: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/902.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 902,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Separation of Variables
ODE form with separable variables:
y ′ = f (x , y) ⇒ dy
dx=φ(x)
ψ(y)or ψ(y)dy = φ(x)dx
Solution as quadrature:
∫
ψ(y)dy =
∫
φ(x)dx + c .
Separation of variables through substitution
Example:y ′ = g(αx + βy + γ)
Substitute v = αx + βy + γ to arrive at
dv
dx= α+ βg(v) ⇒ x =
∫
dv
α+ βg(v)+ c
![Page 903: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/903.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 903,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
ODE’s with Rational Slope Functions
y ′ =f1(x , y)
f2(x , y)
![Page 904: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/904.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 904,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
ODE’s with Rational Slope Functions
y ′ =f1(x , y)
f2(x , y)
If f1 and f2 are homogeneous functions of n-th degree, thensubstitution y = ux separates variables x and u.
dy
dx=φ1(y/x)
φ2(y/x)⇒ u+x
du
dx=φ1(u)
φ2(u)
![Page 905: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/905.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 905,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
ODE’s with Rational Slope Functions
y ′ =f1(x , y)
f2(x , y)
If f1 and f2 are homogeneous functions of n-th degree, thensubstitution y = ux separates variables x and u.
dy
dx=φ1(y/x)
φ2(y/x)⇒ u+x
du
dx=φ1(u)
φ2(u)⇒ dx
x=
φ2(u)
φ1(u)− uφ2(u)du
![Page 906: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/906.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 906,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
ODE’s with Rational Slope Functions
y ′ =f1(x , y)
f2(x , y)
If f1 and f2 are homogeneous functions of n-th degree, thensubstitution y = ux separates variables x and u.
dy
dx=φ1(y/x)
φ2(y/x)⇒ u+x
du
dx=φ1(u)
φ2(u)⇒ dx
x=
φ2(u)
φ1(u)− uφ2(u)du
For y ′ = a1x+b1y+c1a2x+b2y+c2
, coordinate shift
x = X + h, y = Y + k ⇒ y ′ =dy
dx=
dY
dXproduces
dY
dX=
a1X + b1Y + (a1h + b1k + c1)
a2X + b2Y + (a2h + b2k + c2).
Choose h and k such that
a1h + b1k + c1 = 0 = a2h + b2k + c2.
If the system is inconsistent, then substitute u = a2x + b2y .
![Page 907: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/907.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 907,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Clairaut’s equationy = xy ′ + f (y ′)
Substitute p = y ′ and differentiate:
p = p + xdp
dx+ f ′(p)
dp
dx⇒ dp
dx[x + f ′(p)] = 0
![Page 908: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/908.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 908,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Clairaut’s equationy = xy ′ + f (y ′)
Substitute p = y ′ and differentiate:
p = p + xdp
dx+ f ′(p)
dp
dx⇒ dp
dx[x + f ′(p)] = 0
dpdx
= 0 means y ′ = p = m (constant)
family of straight lines y = mx + f (m) as general solution
![Page 909: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/909.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 909,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Clairaut’s equationy = xy ′ + f (y ′)
Substitute p = y ′ and differentiate:
p = p + xdp
dx+ f ′(p)
dp
dx⇒ dp
dx[x + f ′(p)] = 0
dpdx
= 0 means y ′ = p = m (constant)
family of straight lines y = mx + f (m) as general solution
Singular solution:
x = −f ′(p) and y = f (p)− pf ′(p)
Singular solution is the envelope of the family of straightlines that constitute the general solution.
![Page 910: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/910.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 910,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Second order ODE’s with the function not appearingexplicitly
f (x , y ′, y ′′) = 0
Substitute y ′ = p and solve f (x , p, p′) = 0 for p(x).
![Page 911: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/911.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 911,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Second order ODE’s with the function not appearingexplicitly
f (x , y ′, y ′′) = 0
Substitute y ′ = p and solve f (x , p, p′) = 0 for p(x).Second order ODE’s with independent variable not appearingexplicitly
f (y , y ′, y ′′) = 0
Use y ′ = p and
y ′′ =dp
dx=
dp
dy
dy
dx= p
dp
dy⇒ f (y , p, p
dp
dy) = 0.
Solve for p(y).
![Page 912: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/912.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 912,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Some Special ODE’s
Second order ODE’s with the function not appearingexplicitly
f (x , y ′, y ′′) = 0
Substitute y ′ = p and solve f (x , p, p′) = 0 for p(x).Second order ODE’s with independent variable not appearingexplicitly
f (y , y ′, y ′′) = 0
Use y ′ = p and
y ′′ =dp
dx=
dp
dy
dy
dx= p
dp
dy⇒ f (y , p, p
dp
dy) = 0.
Solve for p(y).Resulting equation solved through a quadrature as
dy
dx= p(y) ⇒ x = x0 +
∫
dy
p(y).
![Page 913: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/913.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 913,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Exact Differential Equations and Reduction to the Exact Form
Mdx + Ndy : an exact differential if
M =∂φ
∂xand N =
∂φ
∂y, or,
∂M
∂y=∂N
∂x
![Page 914: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/914.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 914,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Exact Differential Equations and Reduction to the Exact Form
Mdx + Ndy : an exact differential if
M =∂φ
∂xand N =
∂φ
∂y, or,
∂M
∂y=∂N
∂x
M(x , y)dx + N(x , y)dy = 0 is an exact ODE if ∂M∂y
= ∂N∂x
![Page 915: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/915.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 915,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Exact Differential Equations and Reduction to the Exact Form
Mdx + Ndy : an exact differential if
M =∂φ
∂xand N =
∂φ
∂y, or,
∂M
∂y=∂N
∂x
M(x , y)dx + N(x , y)dy = 0 is an exact ODE if ∂M∂y
= ∂N∂x
With M(x , y) = ∂φ∂x
and N(x , y) = ∂φ∂y
,
∂φ
∂xdx +
∂φ
∂ydy = 0 ⇒ dφ = 0.
Solution: φ(x , y) = c
![Page 916: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/916.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 916,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Exact Differential Equations and Reduction to the Exact Form
Mdx + Ndy : an exact differential if
M =∂φ
∂xand N =
∂φ
∂y, or,
∂M
∂y=∂N
∂x
M(x , y)dx + N(x , y)dy = 0 is an exact ODE if ∂M∂y
= ∂N∂x
With M(x , y) = ∂φ∂x
and N(x , y) = ∂φ∂y
,
∂φ
∂xdx +
∂φ
∂ydy = 0 ⇒ dφ = 0.
Solution: φ(x , y) = c
Working rule:
φ1(x , y) =
∫
M(x , y)dx+g1(y) and φ2(x , y) =
∫
N(x , y)dy+g2(x)
Determine g1(y) and g2(x) from φ1(x , y) = φ2(x , y) = φ(x , y).
![Page 917: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/917.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 917,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Exact Differential Equations and Reduction to the Exact Form
Mdx + Ndy : an exact differential if
M =∂φ
∂xand N =
∂φ
∂y, or,
∂M
∂y=∂N
∂x
M(x , y)dx + N(x , y)dy = 0 is an exact ODE if ∂M∂y
= ∂N∂x
With M(x , y) = ∂φ∂x
and N(x , y) = ∂φ∂y
,
∂φ
∂xdx +
∂φ
∂ydy = 0 ⇒ dφ = 0.
Solution: φ(x , y) = c
Working rule:
φ1(x , y) =
∫
M(x , y)dx+g1(y) and φ2(x , y) =
∫
N(x , y)dy+g2(x)
Determine g1(y) and g2(x) from φ1(x , y) = φ2(x , y) = φ(x , y).If ∂M
∂y6= ∂N
∂x, but ∂
∂y(FM) = ∂
∂x(FN)?
F : Integrating factor
![Page 918: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/918.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 918,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
General first order linear ODE:
dy
dx+ P(x)y = Q(x)
Leibnitz equation
![Page 919: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/919.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 919,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
General first order linear ODE:
dy
dx+ P(x)y = Q(x)
Leibnitz equation
For integrating factor F (x),
F (x)dy
dx+ F (x)P(x)y =
d
dx[F (x)y ] ⇒ dF
dx= F (x)P(x).
![Page 920: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/920.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 920,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
General first order linear ODE:
dy
dx+ P(x)y = Q(x)
Leibnitz equation
For integrating factor F (x),
F (x)dy
dx+ F (x)P(x)y =
d
dx[F (x)y ] ⇒ dF
dx= F (x)P(x).
Separating variables,∫
dF
F=
∫
P(x)dx ⇒ lnF =
∫
P(x)dx .
Integrating factor: F (x) = eR
P(x)dx
![Page 921: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/921.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 921,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
General first order linear ODE:
dy
dx+ P(x)y = Q(x)
Leibnitz equation
For integrating factor F (x),
F (x)dy
dx+ F (x)P(x)y =
d
dx[F (x)y ] ⇒ dF
dx= F (x)P(x).
Separating variables,∫
dF
F=
∫
P(x)dx ⇒ lnF =
∫
P(x)dx .
Integrating factor: F (x) = eR
P(x)dx
yeR
P(x)dx =
∫
Q(x)eR
P(x)dxdx + C
![Page 922: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/922.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 922,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
Bernoulli’s equation
dy
dx+ P(x)y = Q(x)yk
![Page 923: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/923.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 923,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
Bernoulli’s equation
dy
dx+ P(x)y = Q(x)yk
Substitution: z = y1−k , dzdx
= (1− k)y−k dydx
gives
dz
dx+ (1− k)P(x)z = (1− k)Q(x),
in the Leibnitz form.
![Page 924: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/924.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 924,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
Bernoulli’s equation
dy
dx+ P(x)y = Q(x)yk
Substitution: z = y1−k , dzdx
= (1− k)y−k dydx
gives
dz
dx+ (1− k)P(x)z = (1− k)Q(x),
in the Leibnitz form.Riccati equation
y ′ = a(x) + b(x)y + c(x)y2
If one solution y1(x) is known, then propose y(x) = y1(x) + z(x).
![Page 925: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/925.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 925,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
First Order Linear (Leibnitz) ODE and Associated Forms
Bernoulli’s equation
dy
dx+ P(x)y = Q(x)yk
Substitution: z = y1−k , dzdx
= (1− k)y−k dydx
gives
dz
dx+ (1− k)P(x)z = (1− k)Q(x),
in the Leibnitz form.Riccati equation
y ′ = a(x) + b(x)y + c(x)y2
If one solution y1(x) is known, then propose y(x) = y1(x) + z(x).
y ′1(x) + z ′(x) = a(x) + b(x)[y1(x) + z(x)] + c(x)[y1(x) + z(x)]2
Since y ′1(x) = a(x) + b(x)y1(x) + c(x)[y1(x)]2,
z ′(x) = [b(x) + 2c(x)y1(x)]z(x) + c(x)[z(x)]2,
in the form of Bernoulli’s equation.
![Page 926: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/926.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 926,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Orthogonal Trajectories
In xy -plane, one-parameter equation φ(x , y , c) = 0:
a family of curves
Differential equation of the family of curves:
dy
dx= f1(x , y)
![Page 927: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/927.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 927,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Orthogonal Trajectories
In xy -plane, one-parameter equation φ(x , y , c) = 0:
a family of curves
Differential equation of the family of curves:
dy
dx= f1(x , y)
Slope of curves orthogonal to φ(x , y , c) = 0:
dy
dx= − 1
f1(x , y)
Solving this ODE, another family of curves ψ(x , y , k) = 0.
Orthogonal trajectories
![Page 928: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/928.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 928,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Orthogonal Trajectories
In xy -plane, one-parameter equation φ(x , y , c) = 0:
a family of curves
Differential equation of the family of curves:
dy
dx= f1(x , y)
Slope of curves orthogonal to φ(x , y , c) = 0:
dy
dx= − 1
f1(x , y)
Solving this ODE, another family of curves ψ(x , y , k) = 0.
Orthogonal trajectories
If φ(x , y , c) = 0 represents the potential lines (contours),then ψ(x , y , k) = 0 will represent the streamlines!
![Page 929: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/929.jpg)
Mathematical Methods in Engineering and Science First Order Ordinary Differential Equations 929,
Formation of Differential Equations and Their SolutionsSeparation of VariablesODE’s with Rational Slope FunctionsSome Special ODE’sExact Differential Equations and Reduction to the ExactFirst Order Linear (Leibnitz) ODE and Associated FoOrthogonal TrajectoriesModelling and Simulation
Points to note
Meaning and solution of ODE’s
Separating variables
Exact ODE’s and integrating factors
Linear (Leibnitz) equations
Orthogonal families of curves
Necessary Exercises: 1,3,5,7
![Page 930: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/930.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 930,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Outline
Second Order Linear Homogeneous ODE’sIntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
![Page 931: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/931.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 931,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Introduction
Second order ODE:f (x , y , y ′, y ′′) = 0
Special case of a linear (non-homogeneous) ODE:
y ′′ + P(x)y ′ + Q(x)y = R(x)
Non-homogeneous linear ODE with constant coefficients:
y ′′ + ay ′ + by = R(x)
![Page 932: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/932.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 932,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Introduction
Second order ODE:f (x , y , y ′, y ′′) = 0
Special case of a linear (non-homogeneous) ODE:
y ′′ + P(x)y ′ + Q(x)y = R(x)
Non-homogeneous linear ODE with constant coefficients:
y ′′ + ay ′ + by = R(x)
For R(x) = 0, linear homogeneous differential equation
y ′′ + P(x)y ′ + Q(x)y = 0
and linear homogeneous ODE with constant coefficients
y ′′ + ay ′ + by = 0
![Page 933: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/933.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 933,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
y ′′ + ay ′ + by = 0
![Page 934: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/934.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 934,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
y ′′ + ay ′ + by = 0
Assumey = eλx ⇒ y ′ = λeλx and y ′′ = λ2eλx .
Substitution: (λ2 + aλ+ b)eλx = 0
Auxiliary equation:λ2 + aλ+ b = 0
Solve for λ1 and λ2:
Solutions: eλ1x and eλ2x
![Page 935: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/935.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 935,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
y ′′ + ay ′ + by = 0
Assumey = eλx ⇒ y ′ = λeλx and y ′′ = λ2eλx .
Substitution: (λ2 + aλ+ b)eλx = 0
Auxiliary equation:λ2 + aλ+ b = 0
Solve for λ1 and λ2:
Solutions: eλ1x and eλ2x
Three cases
Real and distinct (a2 > 4b): λ1 6= λ2
y(x) = c1y1(x) + c2y2(x) = c1eλ1x + c2e
λ2x
![Page 936: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/936.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 936,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
Real and equal (a2 = 4b): λ1 = λ2 = λ = − a2
only solution in hand: y1 = eλx
Method to develop another solution?
![Page 937: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/937.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 937,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
Real and equal (a2 = 4b): λ1 = λ2 = λ = − a2
only solution in hand: y1 = eλx
Method to develop another solution? Verify that y2 = xeλx is another solution.
y(x) = c1y1(x) + c2y2(x) = (c1 + c2x)eλx
![Page 938: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/938.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 938,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
Real and equal (a2 = 4b): λ1 = λ2 = λ = − a2
only solution in hand: y1 = eλx
Method to develop another solution? Verify that y2 = xeλx is another solution.
y(x) = c1y1(x) + c2y2(x) = (c1 + c2x)eλx
Complex conjugate (a2 < 4b): λ1,2 = − a2 ± iω
y(x) = c1e(− a
2+iω)x + c2e
(− a2−iω)x
= e−ax2 [c1(cosωx + i sinωx) + c2(cosωx − i sinωx)]
= e−ax2 [A cosωx + B sinωx ],
with A = c1 + c2, B = i(c1 − c2).
![Page 939: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/939.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 939,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Homogeneous Equations with Constant Coefficients
Real and equal (a2 = 4b): λ1 = λ2 = λ = − a2
only solution in hand: y1 = eλx
Method to develop another solution? Verify that y2 = xeλx is another solution.
y(x) = c1y1(x) + c2y2(x) = (c1 + c2x)eλx
Complex conjugate (a2 < 4b): λ1,2 = − a2 ± iω
y(x) = c1e(− a
2+iω)x + c2e
(− a2−iω)x
= e−ax2 [c1(cosωx + i sinωx) + c2(cosωx − i sinωx)]
= e−ax2 [A cosωx + B sinωx ],
with A = c1 + c2, B = i(c1 − c2). A third form: y(x) = Ce−
ax2 cos(ωx − α)
![Page 940: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/940.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 940,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Euler-Cauchy Equation
x2y ′′ + axy ′ + by = 0
Substituting y = xk , auxiliary (or indicial) equation:
k2 + (a − 1)k + b = 0
![Page 941: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/941.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 941,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Euler-Cauchy Equation
x2y ′′ + axy ′ + by = 0
Substituting y = xk , auxiliary (or indicial) equation:
k2 + (a − 1)k + b = 0
1. Roots real and distinct [(a − 1)2 > 4b]: k1 6= k2.
y(x) = c1xk1 + c2x
k2 .
2. Roots real and equal [(a − 1)2 = 4b]: k1 = k2 = k = − a−12 .
y(x) = (c1 + c2 ln x)xk .
3. Roots complex conjugate [(a − 1)2 < 4b]: k1,2 = − a−12 ± iν.
y(x) = x− a−12 [A cos(ν ln x)+B sin(ν ln x)] = Cx− a−1
2 cos(ν ln x−α).
![Page 942: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/942.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 942,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Euler-Cauchy Equation
x2y ′′ + axy ′ + by = 0
Substituting y = xk , auxiliary (or indicial) equation:
k2 + (a − 1)k + b = 0
1. Roots real and distinct [(a − 1)2 > 4b]: k1 6= k2.
y(x) = c1xk1 + c2x
k2 .
2. Roots real and equal [(a − 1)2 = 4b]: k1 = k2 = k = − a−12 .
y(x) = (c1 + c2 ln x)xk .
3. Roots complex conjugate [(a − 1)2 < 4b]: k1,2 = − a−12 ± iν.
y(x) = x− a−12 [A cos(ν ln x)+B sin(ν ln x)] = Cx− a−1
2 cos(ν ln x−α).
Alternative approach: substitution
x = et ⇒ t = ln x ,dx
dt= et = x and
dt
dx=
1
x, etc.
![Page 943: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/943.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 943,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
y ′′ + P(x)y ′ + Q(x)y = 0
Well-posedness of its IVP:
The initial value problem of the ODE, with arbitraryinitial conditions y(x0) = Y0, y ′(x0) = Y1, has a uniquesolution, as long as P(x) and Q(x) are continuous in theinterval under question.
![Page 944: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/944.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 944,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
y ′′ + P(x)y ′ + Q(x)y = 0
Well-posedness of its IVP:
The initial value problem of the ODE, with arbitraryinitial conditions y(x0) = Y0, y ′(x0) = Y1, has a uniquesolution, as long as P(x) and Q(x) are continuous in theinterval under question.
At least two linearly independent solutions:
y1(x): IVP with initial conditions y(x0) = 1, y ′(x0) = 0
y2(x): IVP with initial conditions y(x0) = 0, y ′(x0) = 1
c1y1(x) + c2y2(x) = 0 ⇒ c1 = c2 = 0
![Page 945: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/945.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 945,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
y ′′ + P(x)y ′ + Q(x)y = 0
Well-posedness of its IVP:
The initial value problem of the ODE, with arbitraryinitial conditions y(x0) = Y0, y ′(x0) = Y1, has a uniquesolution, as long as P(x) and Q(x) are continuous in theinterval under question.
At least two linearly independent solutions:
y1(x): IVP with initial conditions y(x0) = 1, y ′(x0) = 0
y2(x): IVP with initial conditions y(x0) = 0, y ′(x0) = 1
c1y1(x) + c2y2(x) = 0 ⇒ c1 = c2 = 0
At most two linearly independent solutions?
![Page 946: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/946.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 946,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Wronskian of two solutions y1(x) and y2(x):
W (y1, y2) =
∣
∣
∣
∣
y1 y2
y ′1 y ′
2
∣
∣
∣
∣
= y1y′2 − y2y
′1
![Page 947: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/947.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 947,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Wronskian of two solutions y1(x) and y2(x):
W (y1, y2) =
∣
∣
∣
∣
y1 y2
y ′1 y ′
2
∣
∣
∣
∣
= y1y′2 − y2y
′1
Solutions y1 and y2 are linearly dependent, if and only if ∃ x0
such that W [y1(x0), y2(x0)] = 0.
![Page 948: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/948.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 948,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Wronskian of two solutions y1(x) and y2(x):
W (y1, y2) =
∣
∣
∣
∣
y1 y2
y ′1 y ′
2
∣
∣
∣
∣
= y1y′2 − y2y
′1
Solutions y1 and y2 are linearly dependent, if and only if ∃ x0
such that W [y1(x0), y2(x0)] = 0.
W [y1(x0), y2(x0)] = 0 ⇒ W [y1(x), y2(x)] = 0 ∀x .
![Page 949: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/949.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 949,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Wronskian of two solutions y1(x) and y2(x):
W (y1, y2) =
∣
∣
∣
∣
y1 y2
y ′1 y ′
2
∣
∣
∣
∣
= y1y′2 − y2y
′1
Solutions y1 and y2 are linearly dependent, if and only if ∃ x0
such that W [y1(x0), y2(x0)] = 0.
W [y1(x0), y2(x0)] = 0 ⇒ W [y1(x), y2(x)] = 0 ∀x . W [y1(x1), y2(x1)] 6= 0 ⇒ W [y1(x), y2(x)] 6= 0 ∀x , and y1(x)
and y2(x) are linearly independent solutions.
![Page 950: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/950.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 950,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Wronskian of two solutions y1(x) and y2(x):
W (y1, y2) =
∣
∣
∣
∣
y1 y2
y ′1 y ′
2
∣
∣
∣
∣
= y1y′2 − y2y
′1
Solutions y1 and y2 are linearly dependent, if and only if ∃ x0
such that W [y1(x0), y2(x0)] = 0.
W [y1(x0), y2(x0)] = 0 ⇒ W [y1(x), y2(x)] = 0 ∀x . W [y1(x1), y2(x1)] 6= 0 ⇒ W [y1(x), y2(x)] 6= 0 ∀x , and y1(x)
and y2(x) are linearly independent solutions.
Complete solution:
If y1(x) and y2(x) are two linearly independent solutions,then the general solution is
y(x) = c1y1(x) + c2y2(x).
And, the general solution is the complete solution .
No third linearly independent solution. No singular solution.
![Page 951: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/951.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 951,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
If y1(x) and y2(x) are linearly dependent, then y2 = ky1.
W (y1, y2) = y1y′2 − y2y
′1 = y1(ky
′1)− (ky1)y
′1 = 0
In particular, W [y1(x0), y2(x0)] = 0
![Page 952: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/952.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 952,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
If y1(x) and y2(x) are linearly dependent, then y2 = ky1.
W (y1, y2) = y1y′2 − y2y
′1 = y1(ky
′1)− (ky1)y
′1 = 0
In particular, W [y1(x0), y2(x0)] = 0
Conversely, if there is a value x0, where
W [y1(x0), y2(x0)] =
∣
∣
∣
∣
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
∣
∣
∣
∣
= 0,
then for[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
c1
c2
]
= 0,
coefficient matrix is singular.
![Page 953: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/953.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 953,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
If y1(x) and y2(x) are linearly dependent, then y2 = ky1.
W (y1, y2) = y1y′2 − y2y
′1 = y1(ky
′1)− (ky1)y
′1 = 0
In particular, W [y1(x0), y2(x0)] = 0
Conversely, if there is a value x0, where
W [y1(x0), y2(x0)] =
∣
∣
∣
∣
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
∣
∣
∣
∣
= 0,
then for[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
c1
c2
]
= 0,
coefficient matrix is singular.
Choose non-zero
[
c1
c2
]
and frame y(x) = c1y1 + c2y2, satisfying
IVP y ′′ + Py ′ + Qy = 0, y(x0) = 0, y ′(x0) = 0.
![Page 954: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/954.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 954,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
If y1(x) and y2(x) are linearly dependent, then y2 = ky1.
W (y1, y2) = y1y′2 − y2y
′1 = y1(ky
′1)− (ky1)y
′1 = 0
In particular, W [y1(x0), y2(x0)] = 0
Conversely, if there is a value x0, where
W [y1(x0), y2(x0)] =
∣
∣
∣
∣
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
∣
∣
∣
∣
= 0,
then for[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
c1
c2
]
= 0,
coefficient matrix is singular.
Choose non-zero
[
c1
c2
]
and frame y(x) = c1y1 + c2y2, satisfying
IVP y ′′ + Py ′ + Qy = 0, y(x0) = 0, y ′(x0) = 0.
Therefore, y(x) = 0 ⇒ y1 and y2 are linearly dependent.
![Page 955: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/955.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 955,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Pick a candidate solution Y (x), choose a point x0, evaluatefunctions y1, y2, Y and their derivatives at that point, frame
[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
C1
C2
]
=
[
Y (x0)Y ′(x0)
]
and ask for solution
[
C1
C2
]
.
![Page 956: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/956.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 956,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Pick a candidate solution Y (x), choose a point x0, evaluatefunctions y1, y2, Y and their derivatives at that point, frame
[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
C1
C2
]
=
[
Y (x0)Y ′(x0)
]
and ask for solution
[
C1
C2
]
.
Unique solution for C1,C2. Hence, particular solution
y∗(x) = C1y1(x) + C2y2(x)
is the “unique” solution of the IVP
y ′′ + Py ′ + Qy = 0, y(x0) = Y (x0), y ′(x0) = Y ′(x0).
![Page 957: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/957.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 957,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Pick a candidate solution Y (x), choose a point x0, evaluatefunctions y1, y2, Y and their derivatives at that point, frame
[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
C1
C2
]
=
[
Y (x0)Y ′(x0)
]
and ask for solution
[
C1
C2
]
.
Unique solution for C1,C2. Hence, particular solution
y∗(x) = C1y1(x) + C2y2(x)
is the “unique” solution of the IVP
y ′′ + Py ′ + Qy = 0, y(x0) = Y (x0), y ′(x0) = Y ′(x0).
But, that is the candidate function Y (x)!
![Page 958: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/958.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 958,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Theory of the Homogeneous Equations
Pick a candidate solution Y (x), choose a point x0, evaluatefunctions y1, y2, Y and their derivatives at that point, frame
[
y1(x0) y2(x0)y ′1(x0) y ′
2(x0)
] [
C1
C2
]
=
[
Y (x0)Y ′(x0)
]
and ask for solution
[
C1
C2
]
.
Unique solution for C1,C2. Hence, particular solution
y∗(x) = C1y1(x) + C2y2(x)
is the “unique” solution of the IVP
y ′′ + Py ′ + Qy = 0, y(x0) = Y (x0), y ′(x0) = Y ′(x0).
But, that is the candidate function Y (x)! Hence, Y (x) = y∗(x).
![Page 959: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/959.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 959,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
For completely describing the solutions, we need
two linearly independent solutions.
No guaranteed procedure to identify two basis members!
![Page 960: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/960.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 960,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
For completely describing the solutions, we need
two linearly independent solutions.
No guaranteed procedure to identify two basis members!
If one solution y1(x) is available, then to find another?Reduction of order
![Page 961: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/961.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 961,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
For completely describing the solutions, we need
two linearly independent solutions.
No guaranteed procedure to identify two basis members!
If one solution y1(x) is available, then to find another?Reduction of order
Assume the second solution as
y2(x) = u(x)y1(x)
and determine u(x) such that y2(x) satisfies the ODE.
u′′y1 + 2u′y ′1 + uy ′′
1 + P(u′y1 + uy ′1) + Quy1 = 0
![Page 962: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/962.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 962,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
For completely describing the solutions, we need
two linearly independent solutions.
No guaranteed procedure to identify two basis members!
If one solution y1(x) is available, then to find another?Reduction of order
Assume the second solution as
y2(x) = u(x)y1(x)
and determine u(x) such that y2(x) satisfies the ODE.
u′′y1 + 2u′y ′1 + uy ′′
1 + P(u′y1 + uy ′1) + Quy1 = 0
⇒ u′′y1 + 2u′y ′1 + Pu′y1 + u(y ′′
1 + Py ′1 + Qy1) = 0.
Since y ′′1 + Py ′
1 + Qy1 = 0, we have y1u′′ + (2y ′
1 + Py1)u′ = 0
![Page 963: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/963.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 963,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
Denoting u′ = U, U ′ + (2y ′1
y1+ P)U = 0.
Rearrangement and integration of the reduced equation:
dU
U+ 2
dy1
y1+ Pdx = 0 ⇒ Uy2
1 eR
Pdx = C = 1 (choose).
![Page 964: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/964.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 964,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
Denoting u′ = U, U ′ + (2y ′1
y1+ P)U = 0.
Rearrangement and integration of the reduced equation:
dU
U+ 2
dy1
y1+ Pdx = 0 ⇒ Uy2
1 eR
Pdx = C = 1 (choose).
Then,
u′ = U =1
y21
e−R
Pdx ,
Integrating,
u(x) =
∫
1
y21
e−R
Pdxdx ,
and
y2(x) = y1(x)
∫
1
y21
e−R
Pdxdx .
![Page 965: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/965.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 965,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for Solutions
Denoting u′ = U, U ′ + (2y ′1
y1+ P)U = 0.
Rearrangement and integration of the reduced equation:
dU
U+ 2
dy1
y1+ Pdx = 0 ⇒ Uy2
1 eR
Pdx = C = 1 (choose).
Then,
u′ = U =1
y21
e−R
Pdx ,
Integrating,
u(x) =
∫
1
y21
e−R
Pdxdx ,
and
y2(x) = y1(x)
∫
1
y21
e−R
Pdxdx .
Note: The factor u(x) is never constant!
![Page 966: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/966.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 966,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for SolutionsFunction space perspective:
Operator ‘D’ means differentiation, operates on an infinitedimensional function space as a linear transformation.
It maps all constant functions to zero. It has a one-dimensional null space.
![Page 967: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/967.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 967,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for SolutionsFunction space perspective:
Operator ‘D’ means differentiation, operates on an infinitedimensional function space as a linear transformation.
It maps all constant functions to zero. It has a one-dimensional null space.
Second derivative or D2 is an operator that has a two-dimensionalnull space, c1 + c2x , with basis 1, x.
![Page 968: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/968.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 968,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for SolutionsFunction space perspective:
Operator ‘D’ means differentiation, operates on an infinitedimensional function space as a linear transformation.
It maps all constant functions to zero. It has a one-dimensional null space.
Second derivative or D2 is an operator that has a two-dimensionalnull space, c1 + c2x , with basis 1, x.Examples of composite operators
(D + a) has a null space ce−ax .
(xD + a) has a null space cx−a.
![Page 969: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/969.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 969,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Basis for SolutionsFunction space perspective:
Operator ‘D’ means differentiation, operates on an infinitedimensional function space as a linear transformation.
It maps all constant functions to zero. It has a one-dimensional null space.
Second derivative or D2 is an operator that has a two-dimensionalnull space, c1 + c2x , with basis 1, x.Examples of composite operators
(D + a) has a null space ce−ax .
(xD + a) has a null space cx−a.
A second order linear operator D2 + P(x)D + Q(x) possesses atwo-dimensional null space.
Solution of [D2 + P(x)D + Q(x)]y = 0: description of thenull space, or a basis for it..
Analogous to solution of Ax = 0, i.e. development of a basisfor Null(A).
![Page 970: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/970.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Homogeneous ODE’s 970,
IntroductionHomogeneous Equations with Constant CoefficientsEuler-Cauchy EquationTheory of the Homogeneous EquationsBasis for Solutions
Points to note
Second order linear homogeneous ODE’s
Wronskian and related results
Solution basis
Reduction of order
Null space of a differential operator
Necessary Exercises: 1,2,3,7,8
![Page 971: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/971.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 971,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Outline
Second Order Linear Non-Homogeneous ODE’sLinear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
![Page 972: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/972.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 972,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Linear ODE’s and Their SolutionsThe Complete Analogy
Table: Linear systems and mappings: algebraic and differential
In ordinary vector space In infinite-dimensional function space
Ax = b y ′′ + Py ′ + Qy = R
The system is consistent. P(x), Q(x), R(x) are continuous.
A solution x∗ A solution yp(x)
Alternative solution: x Alternative solution: y(x)
x − x∗ satisfies Ax = 0, y(x)− yp(x) satisfies y ′′ + Py ′ + Qy = 0,is in null space of A. is in null space of D2 + P(x)D + Q(x).
Complete solution: Complete solution:x = x∗ +
∑
i ci (x0)i yp(x) +∑
i ciyi(x)
Methodology: Methodology:Find null space of A Find null space of D2 + P(x)D + Q(x)
i.e. basis members (x0)i . i.e. basis members yi(x).Find x∗ and compose. Find yp(x) and compose.
![Page 973: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/973.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 973,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Linear ODE’s and Their SolutionsProcedure to solve y ′′ + P(x)y ′ + Q(x)y = R(x)
1. First, solve the corresponding homogeneous equation, obtain abasis with two solutions and construct
yh(x) = c1y1(x) + c2y2(x).
2. Next, find one particular solution yp(x) of the NHE andcompose the complete solution
y(x) = yh(x) + yp(x) = c1y1(x) + c2y2(x) + yp(x).
3. If some initial or boundary conditions are known, they can beimposed now to determine c1 and c2.
![Page 974: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/974.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 974,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Linear ODE’s and Their SolutionsProcedure to solve y ′′ + P(x)y ′ + Q(x)y = R(x)
1. First, solve the corresponding homogeneous equation, obtain abasis with two solutions and construct
yh(x) = c1y1(x) + c2y2(x).
2. Next, find one particular solution yp(x) of the NHE andcompose the complete solution
y(x) = yh(x) + yp(x) = c1y1(x) + c2y2(x) + yp(x).
3. If some initial or boundary conditions are known, they can beimposed now to determine c1 and c2.
Caution: If y1 and y2 are two solutions of the NHE, thendo not expect c1y1 + c2y2 to satisfy the equation.
![Page 975: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/975.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 975,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Linear ODE’s and Their SolutionsProcedure to solve y ′′ + P(x)y ′ + Q(x)y = R(x)
1. First, solve the corresponding homogeneous equation, obtain abasis with two solutions and construct
yh(x) = c1y1(x) + c2y2(x).
2. Next, find one particular solution yp(x) of the NHE andcompose the complete solution
y(x) = yh(x) + yp(x) = c1y1(x) + c2y2(x) + yp(x).
3. If some initial or boundary conditions are known, they can beimposed now to determine c1 and c2.
Caution: If y1 and y2 are two solutions of the NHE, thendo not expect c1y1 + c2y2 to satisfy the equation.
Implication of linearity or superposition:
With zero initial conditions, if y1 and y2 are responsesdue to inputs R1(x) and R2(x), respectively, then theresponse due to input c1R1 + c2R2 is c1y1 + c2y2.
![Page 976: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/976.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 976,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
y ′′ + ay ′ + by = R(x)
What kind of function to propose as yp(x) if R(x) = xn?
![Page 977: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/977.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 977,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
y ′′ + ay ′ + by = R(x)
What kind of function to propose as yp(x) if R(x) = xn? And what if R(x) = eλx?
![Page 978: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/978.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 978,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
y ′′ + ay ′ + by = R(x)
What kind of function to propose as yp(x) if R(x) = xn? And what if R(x) = eλx? If R(x) = xn + eλx , i.e. in the form k1R1(x) + k2R2(x)?
![Page 979: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/979.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 979,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
y ′′ + ay ′ + by = R(x)
What kind of function to propose as yp(x) if R(x) = xn? And what if R(x) = eλx? If R(x) = xn + eλx , i.e. in the form k1R1(x) + k2R2(x)?
The principle of superposition (linearity)
![Page 980: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/980.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 980,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
y ′′ + ay ′ + by = R(x)
What kind of function to propose as yp(x) if R(x) = xn? And what if R(x) = eλx? If R(x) = xn + eλx , i.e. in the form k1R1(x) + k2R2(x)?
The principle of superposition (linearity)
Table: Candidate solutions for linear non-homogeneous ODE’s
RHS function R(x) Candidate solution yp(x)
pn(x) qn(x)
eλx keλx
cosωx or sinωx k1 cosωx + k2 sinωx
eλx cosωx or eλx sinωx k1eλx cosωx + k2e
λx sinωx
pn(x)eλx qn(x)eλx
pn(x) cos ωx or pn(x) sinωx qn(x) cos ωx + rn(x) sinωx
pn(x)eλx cosωx or pn(x)eλx sinωx qn(x)eλx cosωx + rn(x)eλx sinωx
![Page 981: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/981.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 981,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
![Page 982: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/982.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 982,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
![Page 983: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/983.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 983,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
![Page 984: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/984.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 984,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
(b) y(x) = c1e2x + c2e
3x+
![Page 985: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/985.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 985,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
(b) y(x) = c1e2x + c2e
3x+ xe3x
![Page 986: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/986.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 986,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
(b) y(x) = c1e2x + c2e
3x+ xe3x
(c) y(x) = c1e3x + c2xe
3x+
![Page 987: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/987.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 987,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
(b) y(x) = c1e2x + c2e
3x+ xe3x
(c) y(x) = c1e3x + c2xe
3x+ 12x2e3x
![Page 988: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/988.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 988,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Undetermined Coefficients
Example:(a) y ′′ − 6y ′ + 5y = e3x
(b) y ′′ − 5y ′ + 6y = e3x
(c) y ′′ − 6y ′ + 9y = e3x
In each case, the first official proposal: yp = ke3x
(a) y(x) = c1ex + c2e
5x − e3x/4
(b) y(x) = c1e2x + c2e
3x+ xe3x
(c) y(x) = c1e3x + c2xe
3x+ 12x2e3x
Modification rule
If the candidate function (keλx , k1 cosωx + k2 sinωx ork1e
λx cosωx + k2eλx sinωx) is a solution of the corresponding
HE; with λ, ±iω or λ± iω (respectively) satisfying theauxiliary equation; then modify it by multiplying with x .
In the case of λ being a double root, i.e. both eλx and xeλx
being solutions of the HE, choose yp = kx2eλx .
![Page 989: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/989.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 989,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of Parameters
Solution of the HE:
yh(x) = c1y1(x) + c2y2(x),
in which c1 and c2 are constant ‘parameters’.
![Page 990: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/990.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 990,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of Parameters
Solution of the HE:
yh(x) = c1y1(x) + c2y2(x),
in which c1 and c2 are constant ‘parameters’.
For solution of the NHE,
how about ‘variable parameters’?
Proposeyp(x) = u1(x)y1(x) + u2(x)y2(x)
and force yp(x) to satisfy the ODE.
A single second order ODE in u1(x) and u2(x).We need one more condition to fix them.
![Page 991: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/991.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 991,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of ParametersFrom yp = u1y1 + u2y2,
y ′p = u′
1y1 + u1y′1 + u′
2y2 + u2y′2.
![Page 992: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/992.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 992,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of ParametersFrom yp = u1y1 + u2y2,
y ′p = u′
1y1 + u1y′1 + u′
2y2 + u2y′2.
Condition u′1y1 + u′
2y2 = 0 gives
y ′p = u1y
′1 + u2y
′2.
![Page 993: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/993.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 993,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of ParametersFrom yp = u1y1 + u2y2,
y ′p = u′
1y1 + u1y′1 + u′
2y2 + u2y′2.
Condition u′1y1 + u′
2y2 = 0 gives
y ′p = u1y
′1 + u2y
′2.
Differentiating,
y ′′p = u′
1y′1 + u′
2y′2 + u1y
′′1 + u2y
′′2 .
![Page 994: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/994.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 994,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of ParametersFrom yp = u1y1 + u2y2,
y ′p = u′
1y1 + u1y′1 + u′
2y2 + u2y′2.
Condition u′1y1 + u′
2y2 = 0 gives
y ′p = u1y
′1 + u2y
′2.
Differentiating,
y ′′p = u′
1y′1 + u′
2y′2 + u1y
′′1 + u2y
′′2 .
Substitution into the ODE:
u′1y
′1+u′
2y′2+u1y
′′1 +u2y
′′2 +P(x)(u1y
′1+u2y
′2)+Q(x)(u1y1+u2y2) = R(x)
Rearranging,
u′1y
′1+u′
2y′2+u1(y
′′1 +P(x)y ′
1+Q(x)y1)+u2(y′′2 +P(x)y ′
2+Q(x)y2) = R(x).
As y1 and y2 satisfy the associated HE, u′1y
′1 + u′
2y′2 = R(x)
![Page 995: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/995.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 995,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of Parameters
[
y1 y2
y ′1 y ′
2
] [
u′1
u′2
]
=
[
0R
]
![Page 996: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/996.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 996,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of Parameters
[
y1 y2
y ′1 y ′
2
] [
u′1
u′2
]
=
[
0R
]
Since Wronskian is non-zero, this system has unique solution
u′1 = −y2R
Wand u′
2 =y1R
W.
Direct quadrature:
u1(x) = −∫
y2(x)R(x)
W [y1(x), y2(x)]dx and u2(x) =
∫
y1(x)R(x)
W [y1(x), y2(x)]dx
![Page 997: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/997.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 997,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Method of Variation of Parameters
[
y1 y2
y ′1 y ′
2
] [
u′1
u′2
]
=
[
0R
]
Since Wronskian is non-zero, this system has unique solution
u′1 = −y2R
Wand u′
2 =y1R
W.
Direct quadrature:
u1(x) = −∫
y2(x)R(x)
W [y1(x), y2(x)]dx and u2(x) =
∫
y1(x)R(x)
W [y1(x), y2(x)]dx
In contrast to the method of undetermined multipliers,variation of parameters is general. It is applicable for allcontinuous functions as P(x), Q(x) and R(x).
![Page 998: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/998.jpg)
Mathematical Methods in Engineering and Science Second Order Linear Non-Homogeneous ODE’s 998,
Linear ODE’s and Their SolutionsMethod of Undetermined CoefficientsMethod of Variation of ParametersClosure
Points to note
Function space perspective of linear ODE’s
Method of undetermined coefficients
Method of variation of parameters
Necessary Exercises: 1,3,5,6
![Page 999: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/999.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 999,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Outline
Higher Order Linear ODE’sTheory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
![Page 1000: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1000.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1000,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = R(x)
General solution: y(x) = yh(x) + yp(x), where yp(x): a particular solution yh(x): general solution of corresponding HE
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = 0
![Page 1001: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1001.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1001,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = R(x)
General solution: y(x) = yh(x) + yp(x), where yp(x): a particular solution yh(x): general solution of corresponding HE
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = 0
For the HE, suppose we have n solutions y1(x), y2(x), · · · , yn(x).Assemble the state vectors in matrix
Y(x) =
y1 y2 · · · yn
y ′1 y ′
2 · · · y ′n
y ′′1 y ′′
2 · · · y ′′n
......
. . ....
y(n−1)1 y
(n−1)2 · · · y
(n−1)n
.
![Page 1002: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1002.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1002,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = R(x)
General solution: y(x) = yh(x) + yp(x), where yp(x): a particular solution yh(x): general solution of corresponding HE
y (n)+P1(x)y (n−1)+P2(x)y (n−2)+· · ·+Pn−1(x)y ′+Pn(x)y = 0
For the HE, suppose we have n solutions y1(x), y2(x), · · · , yn(x).Assemble the state vectors in matrix
Y(x) =
y1 y2 · · · yn
y ′1 y ′
2 · · · y ′n
y ′′1 y ′′
2 · · · y ′′n
......
. . ....
y(n−1)1 y
(n−1)2 · · · y
(n−1)n
.
Wronskian:W (y1, y2, · · · , yn) = det[Y(x)]
![Page 1003: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1003.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1003,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
If solutions y1(x), y2(x), · · · , yn(x) of HE are linearlydependent, then for a non-zero k ∈ Rn,n∑
i=1
kiyi (x) = 0 ⇒n∑
i=1
kiy(j)i (x) = 0 for j = 1, 2, 3, · · · , (n − 1)
⇒ [Y(x)]k = 0 ⇒ [Y(x)] is singular
⇒ W [y1(x), y2(x), · · · , yn(x)] = 0.
![Page 1004: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1004.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1004,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
If solutions y1(x), y2(x), · · · , yn(x) of HE are linearlydependent, then for a non-zero k ∈ Rn,n∑
i=1
kiyi (x) = 0 ⇒n∑
i=1
kiy(j)i (x) = 0 for j = 1, 2, 3, · · · , (n − 1)
⇒ [Y(x)]k = 0 ⇒ [Y(x)] is singular
⇒ W [y1(x), y2(x), · · · , yn(x)] = 0.
If Wronskian is zero at x = x0, then Y(x0) is singular and anon-zero k ∈ Null [Y(x0)] gives
∑ni=1 kiyi(x) = 0, implying
y1(x), y2(x), · · · , yn(x) to be linearly dependent. Zero Wronskian at some x = x0 implies zero Wronskian
everywhere. Non-zero Wronskian at some x = x1 ensuresnon-zero Wronskian everywhere and the corrsepondingsolutions as linearly independent.
![Page 1005: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1005.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1005,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Theory of Linear ODE’s
If solutions y1(x), y2(x), · · · , yn(x) of HE are linearlydependent, then for a non-zero k ∈ Rn,n∑
i=1
kiyi (x) = 0 ⇒n∑
i=1
kiy(j)i (x) = 0 for j = 1, 2, 3, · · · , (n − 1)
⇒ [Y(x)]k = 0 ⇒ [Y(x)] is singular
⇒ W [y1(x), y2(x), · · · , yn(x)] = 0.
If Wronskian is zero at x = x0, then Y(x0) is singular and anon-zero k ∈ Null [Y(x0)] gives
∑ni=1 kiyi(x) = 0, implying
y1(x), y2(x), · · · , yn(x) to be linearly dependent. Zero Wronskian at some x = x0 implies zero Wronskian
everywhere. Non-zero Wronskian at some x = x1 ensuresnon-zero Wronskian everywhere and the corrsepondingsolutions as linearly independent.
With n linearly independent solutions y1(x), y2(x), · · · , yn(x)of the HE, we have its general solution yh(x) =
∑ni=1 ciyi (x),
acting as the complementary function for the NHE.
![Page 1006: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1006.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1006,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Homogeneous Equations with Constant Coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = 0
With trial solution y = eλx , the auxiliary equation:
λn + a1λn−1 + a2λ
n−2 + · · · + an−1λ+ an = 0
![Page 1007: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1007.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1007,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Homogeneous Equations with Constant Coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = 0
With trial solution y = eλx , the auxiliary equation:
λn + a1λn−1 + a2λ
n−2 + · · · + an−1λ+ an = 0
Construction of the basis:
1. For every simple real root λ = γ, eγx is a solution.
2. For every simple pair of complex roots λ = µ± iω,eµx cosωx and eµx sinωx are linearly independent solutions.
3. For every real root λ = γ of multiplicity r ; eγx , xeγx , x2eγx ,· · · , x r−1eγx are all linearly independent solutions.
4. For every complex pair of roots λ = µ± iω of multiplicity r ;eµx cosωx , eµx sinωx , xeµx cosωx , xeµx sinωx , · · · ,x r−1eµx cosωx , x r−1eµx sinωx are the required solutions.
![Page 1008: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1008.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1008,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Method of undetermined coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = R(x)
Extension of the second order case
![Page 1009: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1009.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1009,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Method of undetermined coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = R(x)
Extension of the second order caseMethod of variation of parameters
yp(x) =
n∑
i=1
ui(x)yi (x)
![Page 1010: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1010.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1010,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Method of undetermined coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = R(x)
Extension of the second order caseMethod of variation of parameters
yp(x) =
n∑
i=1
ui(x)yi (x)
Imposed condition Derivative∑n
i=1 u′i (x)yi (x) = 0 ⇒ y ′
p(x) =∑n
i=1 ui (x)y ′i (x)
![Page 1011: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1011.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1011,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Method of undetermined coefficients
y (n) + a1y(n−1) + a2y
(n−2) + · · ·+ an−1y′ + any = R(x)
Extension of the second order caseMethod of variation of parameters
yp(x) =
n∑
i=1
ui(x)yi (x)
Imposed condition Derivative∑n
i=1 u′i (x)yi (x) = 0 ⇒ y ′
p(x) =∑n
i=1 ui (x)y ′i (x)
∑ni=1 u′
i (x)y ′i (x) = 0 ⇒ y ′′
p (x) =∑n
i=1 ui (x)y ′′i (x)
· · · · · · · · · ⇒ · · · · · · · · ·∑n
i=1 u′i (x)y
(n−2)i (x) = 0 ⇒ y
(n−1)p (x) =
∑ni=1 ui(x)y
(n−1)i (x)
Finally, y(n)p (x) =
∑ni=1 u′
i (x)y(n−1)i (x) +
∑ni=1 ui (x)y
(n)i (x)
⇒n∑
i=1
u′i (x)y
(n−1)i (x)+
n∑
i=1
ui (x)[
y(n)i + P1y
(n−1)i + · · · + Pnyi
]
= R(x)
![Page 1012: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1012.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1012,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Since each yi (x) is a solution of the HE,
n∑
i=1
u′i(x)y
(n−1)i (x) = R(x).
Assembling all conditions on u′(x) together,
[Y(x)]u′(x) = enR(x).
![Page 1013: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1013.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1013,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Since each yi (x) is a solution of the HE,
n∑
i=1
u′i(x)y
(n−1)i (x) = R(x).
Assembling all conditions on u′(x) together,
[Y(x)]u′(x) = enR(x).
Since Y−1 = adj Ydet(Y) ,
u′(x) =1
det[Y(x)][adj Y(x)]enR(x) =
R(x)
W (x)[last column of adj Y(x)].
Using cofactors of elements from last row only,
u′i (x) =
Wi(x)
W (x)R(x),
with Wi (x) = Wronskian evaluated with en in place of i -th column.
![Page 1014: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1014.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1014,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Non-Homogeneous Equations
Since each yi (x) is a solution of the HE,
n∑
i=1
u′i(x)y
(n−1)i (x) = R(x).
Assembling all conditions on u′(x) together,
[Y(x)]u′(x) = enR(x).
Since Y−1 = adj Ydet(Y) ,
u′(x) =1
det[Y(x)][adj Y(x)]enR(x) =
R(x)
W (x)[last column of adj Y(x)].
Using cofactors of elements from last row only,
u′i (x) =
Wi(x)
W (x)R(x),
with Wi (x) = Wronskian evaluated with en in place of i -th column.
ui(x) =∫ Wi (x)R(x)
W (x) dx
![Page 1015: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1015.jpg)
Mathematical Methods in Engineering and Science Higher Order Linear ODE’s 1015,
Theory of Linear ODE’sHomogeneous Equations with Constant CoefficientsNon-Homogeneous EquationsEuler-Cauchy Equation of Higher Order
Points to note
Wronskian for a higher order ODE
General theory of linear ODE’s Variation for parameters for n-th order ODE
Necessary Exercises: 1,3,4
![Page 1016: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1016.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1016,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Outline
Laplace TransformsIntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
![Page 1017: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1017.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1017,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
Classical perspective
Entire differential equation is known in advance.
Go for a complete solution first.
Afterwards, use the initial (or other) conditions.
![Page 1018: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1018.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1018,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
Classical perspective
Entire differential equation is known in advance.
Go for a complete solution first.
Afterwards, use the initial (or other) conditions.
A practical situation
You have a plant intrinsic dynamic model as well as the starting conditions.
You may drive the plant with different kinds of inputs ondifferent occasions.
![Page 1019: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1019.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1019,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
Classical perspective
Entire differential equation is known in advance.
Go for a complete solution first.
Afterwards, use the initial (or other) conditions.
A practical situation
You have a plant intrinsic dynamic model as well as the starting conditions.
You may drive the plant with different kinds of inputs ondifferent occasions.
Implication
Left-hand side of the ODE and the initial conditions areknown a priori.
Right-hand side, R(x), changes from task to task.
![Page 1020: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1020.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1020,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
IntroductionAnother question: What if R(x) is not continuous?
When power is switched on or off, what happens?
If there is a sudden voltage fluctuation, what happens to theequipment connected to the power line?
Or, does “anything” happen in the immediate future?
![Page 1021: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1021.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1021,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
IntroductionAnother question: What if R(x) is not continuous?
When power is switched on or off, what happens?
If there is a sudden voltage fluctuation, what happens to theequipment connected to the power line?
Or, does “anything” happen in the immediate future?
“Something” certainly happens. The IVP has a solution!
Laplace transforms provide a tool to find the solution, inspite of the discontinuity of R(x).
![Page 1022: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1022.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1022,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
IntroductionAnother question: What if R(x) is not continuous?
When power is switched on or off, what happens?
If there is a sudden voltage fluctuation, what happens to theequipment connected to the power line?
Or, does “anything” happen in the immediate future?
“Something” certainly happens. The IVP has a solution!
Laplace transforms provide a tool to find the solution, inspite of the discontinuity of R(x).
Integral transform:
T [f (t)](s) =
∫ b
a
K (s, t)f (t)dt
s: frequency variable
K (s, t): kernel of the transform
Note: T [f (t)] is a function of s, not t.
![Page 1023: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1023.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1023,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
With kernel function K (s, t) = e−st , and limits a = 0, b =∞,Laplace transform
F (s) = Lf (t) =
∫ ∞
0e−st f (t)dt = lim
b→∞
∫ b
0e−st f (t)dt
When this integral exists, f (t) has its Laplace transform.
![Page 1024: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1024.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1024,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
With kernel function K (s, t) = e−st , and limits a = 0, b =∞,Laplace transform
F (s) = Lf (t) =
∫ ∞
0e−st f (t)dt = lim
b→∞
∫ b
0e−st f (t)dt
When this integral exists, f (t) has its Laplace transform.
Sufficient condition:
f (t) is piecewise continuous, and
it is of exponential order, i.e. |f (t)| < Mect for some (finite)M and c .
![Page 1025: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1025.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1025,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Introduction
With kernel function K (s, t) = e−st , and limits a = 0, b =∞,Laplace transform
F (s) = Lf (t) =
∫ ∞
0e−st f (t)dt = lim
b→∞
∫ b
0e−st f (t)dt
When this integral exists, f (t) has its Laplace transform.
Sufficient condition:
f (t) is piecewise continuous, and
it is of exponential order, i.e. |f (t)| < Mect for some (finite)M and c .
Inverse Laplace transform:
f (t) = L−1F (s)
![Page 1026: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1026.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1026,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Basic Properties and Results
Linearity:
Laf (t) + bg(t) = aLf (t)+ bLg(t)First shifting property or the frequency shifting rule:
Leat f (t) = F (s − a)
Laplace transforms of some elementary functions:
L(1) =
∫ ∞
0e−stdt =
[
e−st
−s
]∞
0
=1
s,
L(t) =
∫ ∞
0e−sttdt =
[
te−st
−s
]∞
0
+1
s
∫ ∞
0e−stdt =
1
s2,
L(tn) =n!
sn+1(for positive integer n),
L(ta) =Γ(a + 1)
sa+1(for a ∈ R+)
and L(eat) =1
s − a.
![Page 1027: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1027.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1027,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Basic Properties and Results
L(cos ωt) =s
s2 + ω2, L(sinωt) =
ω
s2 + ω2;
L(cosh at) =s
s2 − a2, L(sinh at) =
a
s2 − a2;
L(eµt cosωt) =s − µ
(s − µ)2 + ω2, L(eµt sinωt) =
ω
(s − µ)2 + ω2.
![Page 1028: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1028.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1028,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Basic Properties and Results
L(cos ωt) =s
s2 + ω2, L(sinωt) =
ω
s2 + ω2;
L(cosh at) =s
s2 − a2, L(sinh at) =
a
s2 − a2;
L(eµt cosωt) =s − µ
(s − µ)2 + ω2, L(eµt sinωt) =
ω
(s − µ)2 + ω2.
Laplace transform of derivative:
Lf ′(t) =
∫ ∞
0e−st f ′(t)dt
=[
e−st f (t)]∞0
+ s
∫ ∞
0e−st f (t)dt = sLf (t) − f (0)
Using this process recursively,
Lf (n)(t) = snLf (t)−s(n−1)f (0)−s(n−2)f ′(0)−· · ·− f (n−1)(0).
![Page 1029: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1029.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1029,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Basic Properties and Results
L(cos ωt) =s
s2 + ω2, L(sinωt) =
ω
s2 + ω2;
L(cosh at) =s
s2 − a2, L(sinh at) =
a
s2 − a2;
L(eµt cosωt) =s − µ
(s − µ)2 + ω2, L(eµt sinωt) =
ω
(s − µ)2 + ω2.
Laplace transform of derivative:
Lf ′(t) =
∫ ∞
0e−st f ′(t)dt
=[
e−st f (t)]∞0
+ s
∫ ∞
0e−st f (t)dt = sLf (t) − f (0)
Using this process recursively,
Lf (n)(t) = snLf (t)−s(n−1)f (0)−s(n−2)f ′(0)−· · ·− f (n−1)(0).
For integral g(t) =∫ t
0 f (t)dt, g(0) = 0, andLg ′(t) = sLg(t)−g(0) = sLg(t) ⇒ Lg(t) = 1
sLf (t).
![Page 1030: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1030.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1030,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Application to Differential Equations
Example:Initial value problem of a linear constant coefficient ODE
y ′′ + ay ′ + by = r(t), y(0) = K0, y ′(0) = K1
![Page 1031: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1031.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1031,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Application to Differential Equations
Example:Initial value problem of a linear constant coefficient ODE
y ′′ + ay ′ + by = r(t), y(0) = K0, y ′(0) = K1
Laplace transforms of both sides of the ODE:
s2Y (s)− sy(0)− y ′(0) + a[sY (s)− y(0)] + bY (s) = R(s)
⇒ (s2 + as + b)Y (s) = (s + a)K0 + K1 + R(s)
A differential equation in y(t) has been converted to analgebraic equation in Y (s).
![Page 1032: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1032.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1032,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Application to Differential Equations
Example:Initial value problem of a linear constant coefficient ODE
y ′′ + ay ′ + by = r(t), y(0) = K0, y ′(0) = K1
Laplace transforms of both sides of the ODE:
s2Y (s)− sy(0)− y ′(0) + a[sY (s)− y(0)] + bY (s) = R(s)
⇒ (s2 + as + b)Y (s) = (s + a)K0 + K1 + R(s)
A differential equation in y(t) has been converted to analgebraic equation in Y (s).
Transfer function: ratio of Laplace transform of output functiony(t) to that of input function r(t), with zero initial conditions
Q(s) =Y (s)
R(s)=
1
s2 + as + b(in this case)
Y (s) = [(s + a)K0 + K1]Q(s) + Q(s)R(s)
Solution of the given IVP: y(t) = L−1Y (s)
![Page 1033: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1033.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1033,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
Unit step function
u(t − a) =
0 if t < a1 if t > a
Its Laplace transform:
Lu(t−a) =
∫ ∞
0e−stu(t−a)dt =
∫ a
00 ·dt +
∫ ∞
a
e−stdt =e−as
s
![Page 1034: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1034.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1034,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
Unit step function
u(t − a) =
0 if t < a1 if t > a
Its Laplace transform:
Lu(t−a) =
∫ ∞
0e−stu(t−a)dt =
∫ a
00 ·dt +
∫ ∞
a
e−stdt =e−as
s
For input f (t) with a time delay,
f (t − a)u(t − a) =
0 if t < af (t − a) if t > a
has its Laplace transform as
Lf (t − a)u(t − a) =
∫ ∞
a
e−st f (t − a)dt
=
∫ ∞
0e−s(a+τ)f (τ)dτ = e−asLf (t).
Second shifting property or the time shifting rule
![Page 1035: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1035.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1035,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
Define
fk(t − a) =
1/k if a ≤ t ≤ a + k0 otherwise
=1
ku(t − a)− 1
ku(t − a− k)
1k
1k
k1
k1
1k
fk
o a t
1
u(t−a)
fk
o o
1
1
a a+k
u(t−a−k)
u(t−a)
1
a a+k
(t−a)
tt
−−
o a t
1
(t−a)δ
(a) (c) Function (b) Composition (d) Dirac’s − functionδ
Unit step function
Figure: Step and impulse functions
![Page 1036: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1036.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1036,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
Define
fk(t − a) =
1/k if a ≤ t ≤ a + k0 otherwise
=1
ku(t − a)− 1
ku(t − a− k)
1k
1k
k1
k1
1k
fk
o a t
1
u(t−a)
fk
o o
1
1
a a+k
u(t−a−k)
u(t−a)
1
a a+k
(t−a)
tt
−−
o a t
1
(t−a)δ
(a) (c) Function (b) Composition (d) Dirac’s − functionδ
Unit step function
Figure: Step and impulse functions
and note that its integral
Ik =
∫ ∞
0fk(t − a)dt =
∫ a+k
a
1
kdt = 1.
does not depend on k.
![Page 1037: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1037.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1037,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
In the limit,
δ(t − a) = limk→0
fk(t − a)
or, δ(t − a) =
∞ if t = a0 otherwise
and
∫ ∞
0δ(t − a)dt = 1.
Unit impulse function or Dirac’s delta function
![Page 1038: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1038.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1038,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
In the limit,
δ(t − a) = limk→0
fk(t − a)
or, δ(t − a) =
∞ if t = a0 otherwise
and
∫ ∞
0δ(t − a)dt = 1.
Unit impulse function or Dirac’s delta function
Lδ(t − a) = limk→0
1
k[Lu(t − a) − Lu(t − a − k)]
= limk→0
e−as − e−(a+k)s
ks= e−as
![Page 1039: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1039.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1039,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Handling Discontinuities
In the limit,
δ(t − a) = limk→0
fk(t − a)
or, δ(t − a) =
∞ if t = a0 otherwise
and
∫ ∞
0δ(t − a)dt = 1.
Unit impulse function or Dirac’s delta function
Lδ(t − a) = limk→0
1
k[Lu(t − a) − Lu(t − a − k)]
= limk→0
e−as − e−(a+k)s
ks= e−as
Through step and impulse functions, Laplace transformmethod can handle IVP’s with discontinuous inputs.
![Page 1040: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1040.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1040,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
ConvolutionA generalized product of two functions
h(t) = f (t) ∗ g(t) =
∫ t
0f (τ)g(t − τ) dτ
![Page 1041: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1041.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1041,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
ConvolutionA generalized product of two functions
h(t) = f (t) ∗ g(t) =
∫ t
0f (τ)g(t − τ) dτ
Laplace transform of the convolution:
H(s) =
∫ ∞
0e−st
∫ t
0f (τ)g(t−τ)dτ dt
t= τ
τ
ot
=
τ
o t
(a) Original order (b) Changed order
τt
Figure: Region of integration for Lh(t)
![Page 1042: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1042.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1042,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
ConvolutionA generalized product of two functions
h(t) = f (t) ∗ g(t) =
∫ t
0f (τ)g(t − τ) dτ
Laplace transform of the convolution:
H(s) =
∫ ∞
0e−st
∫ t
0f (τ)g(t−τ)dτ dt =
∫ ∞
0f (τ)
∫ ∞
τe−stg(t−τ)dt dτ
t= τ
τ
ot
=
τ
o t
(a) Original order (b) Changed order
τt
Figure: Region of integration for Lh(t)
![Page 1043: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1043.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1043,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Convolution
Through substitution t ′ = t − τ ,
H(s) =
∫ ∞
0f (τ)
∫ ∞
0e−s(t′+τ)g(t ′) dt ′ dτ
=
∫ ∞
0f (τ)e−sτ
[∫ ∞
0e−st′g(t ′) dt ′
]
dτ
![Page 1044: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1044.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1044,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Convolution
Through substitution t ′ = t − τ ,
H(s) =
∫ ∞
0f (τ)
∫ ∞
0e−s(t′+τ)g(t ′) dt ′ dτ
=
∫ ∞
0f (τ)e−sτ
[∫ ∞
0e−st′g(t ′) dt ′
]
dτ
H(s) = F (s)G (s)
Convolution theorem:
Laplace transform of the convolution integral of twofunctions is given by the product of the Laplacetransforms of the two functions.
Utilities:
To invert Q(s)R(s), one can convolute y(t) = q(t) ∗ r(t).
In solving some integral equation.
![Page 1045: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1045.jpg)
Mathematical Methods in Engineering and Science Laplace Transforms 1045,
IntroductionBasic Properties and ResultsApplication to Differential EquationsHandling DiscontinuitiesConvolutionAdvanced Issues
Points to note
A paradigm shift in solution of IVP’s
Handling discontinuous input functions
Extension to ODE systems
The idea of integral transforms
Necessary Exercises: 1,2,4
![Page 1046: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1046.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1046,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Outline
ODE SystemsFundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
![Page 1047: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1047.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1047,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Fundamental Ideas
y′ = f(t, y)
Solution: a vector function y = h(t)
![Page 1048: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1048.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1048,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Fundamental Ideas
y′ = f(t, y)
Solution: a vector function y = h(t)
Autonomous system: y′ = f(y)
Points in y-space where f(y) = 0:
equilibrium points or critical points
![Page 1049: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1049.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1049,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Fundamental Ideas
y′ = f(t, y)
Solution: a vector function y = h(t)
Autonomous system: y′ = f(y)
Points in y-space where f(y) = 0:
equilibrium points or critical points
System of linear ODE’s:
y′ = A(t)y + g(t)
autonomous systems if A and g are constant
homogeneous systems if g(t) = 0
homogeneous constant coefficient systems if A is constantand g(t) = 0
![Page 1050: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1050.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1050,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Fundamental Ideas
For a homogeneous system,
y′ = A(t)y
Wronskian: W (y1, y2, y3, · · · , yn) = |y1 y2 y3 · · · yn|
![Page 1051: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1051.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1051,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Fundamental Ideas
For a homogeneous system,
y′ = A(t)y
Wronskian: W (y1, y2, y3, · · · , yn) = |y1 y2 y3 · · · yn|
If Wronskian is non-zero, then
Fundamental matrix: Y(t) = [y1 y2 y3 · · · yn],
giving a basis.
General solution:
y(t) =
n∑
i=1
ciyi (t) = [Y(t)] c
![Page 1052: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1052.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1052,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
![Page 1053: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1053.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1053,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
![Page 1054: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1054.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1054,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt
![Page 1055: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1055.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1055,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt ⇒ Ax = λx
![Page 1056: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1056.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1056,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt ⇒ Ax = λxIf A is diagonalizable,
n linearly independent solutions yi = xieλi t corresponding to n
eigenpairs
![Page 1057: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1057.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1057,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt ⇒ Ax = λxIf A is diagonalizable,
n linearly independent solutions yi = xieλi t corresponding to n
eigenpairs
If A is not diagonalizable?
![Page 1058: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1058.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1058,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt ⇒ Ax = λxIf A is diagonalizable,
n linearly independent solutions yi = xieλi t corresponding to n
eigenpairs
If A is not diagonalizable?
All xieλi t together will not complete the basis.
Try y = xteµt?
![Page 1059: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1059.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1059,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
y′ = Ay
Non-degenerate case: matrix A non-singular
Origin (y = 0) is the unique equilibrium point.
Attempt y = xeλt ⇒ y′ = λxeλt .
Substitution: Axeλt = λxeλt ⇒ Ax = λxIf A is diagonalizable,
n linearly independent solutions yi = xieλi t corresponding to n
eigenpairs
If A is not diagonalizable?
All xieλi t together will not complete the basis.
Try y = xteµt? Substitution leads to
xeµt + µxteµt = Axteµt ⇒ xeµt = 0 ⇒ x = 0.
Absurd!
![Page 1060: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1060.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1060,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
Try a linearly independent solution in the form
y = xteµt + ueµt .
Linear independence here has two implications: infunction space AND in ordinary vector space!
![Page 1061: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1061.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1061,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
Try a linearly independent solution in the form
y = xteµt + ueµt .
Linear independence here has two implications: infunction space AND in ordinary vector space!
Substitution:
xeµt + µxteµt + µueµt = Axteµt + Aueµt ⇒ (A − µI)u = x
Solve for u, the generalized eigenvector of A.
![Page 1062: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1062.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1062,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Homogeneous Systems with Constant Coefficients
Try a linearly independent solution in the form
y = xteµt + ueµt .
Linear independence here has two implications: infunction space AND in ordinary vector space!
Substitution:
xeµt + µxteµt + µueµt = Axteµt + Aueµt ⇒ (A − µI)u = x
Solve for u, the generalized eigenvector of A.For Jordan blocks of larger sizes,
y1 = xeµt , y2 = xteµt +u1eµt , y3 =
1
2xt2eµt +u1te
µt +u2eµt etc.
Jordan canonical form (JCF) of A provides a set of basisfunctions to describe the complete solution of the ODEsystem.
![Page 1063: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1063.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1063,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
y′ = Ay + g(t)
![Page 1064: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1064.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1064,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
y′ = Ay + g(t)
Complementary function:
yh(t) =
n∑
i=1
ciyi (t) = [Y(t)]c
Complete solution:
y(t) = yh(t) + yp(t)
We need to develop one particular solution yp.
![Page 1065: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1065.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1065,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
y′ = Ay + g(t)
Complementary function:
yh(t) =
n∑
i=1
ciyi (t) = [Y(t)]c
Complete solution:
y(t) = yh(t) + yp(t)
We need to develop one particular solution yp.
Method of undetermined coefficientsBased on g(t), select candidate function Gk(t) and propose
yp =∑
k
ukGk(t),
vector coefficients (uk) to be determined by substitution.
![Page 1066: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1066.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1066,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of diagonalization
If A is a diagonalizable constant matrix, with X−1AX = D,
changing variables to z = X−1y, such that y = Xz,
Xz′ = AXz+g(t) ⇒ z′ = X−1AXz+X−1g(t) = Dz+h(t) (say).
![Page 1067: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1067.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1067,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of diagonalization
If A is a diagonalizable constant matrix, with X−1AX = D,
changing variables to z = X−1y, such that y = Xz,
Xz′ = AXz+g(t) ⇒ z′ = X−1AXz+X−1g(t) = Dz+h(t) (say).
Single decoupled Leibnitz equations
z ′k = dkzk + hk(t), k = 1, 2, 3, · · · , n;
leading to individual solutions
zk(t) = ckedk t + edk t
∫
e−dk thk(t)dt.
![Page 1068: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1068.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1068,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of diagonalization
If A is a diagonalizable constant matrix, with X−1AX = D,
changing variables to z = X−1y, such that y = Xz,
Xz′ = AXz+g(t) ⇒ z′ = X−1AXz+X−1g(t) = Dz+h(t) (say).
Single decoupled Leibnitz equations
z ′k = dkzk + hk(t), k = 1, 2, 3, · · · , n;
leading to individual solutions
zk(t) = ckedk t + edk t
∫
e−dk thk(t)dt.
After assembling z(t), we reconstruct y = Xz.
![Page 1069: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1069.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1069,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of variation of parametersIf we can supply a basis Y(t) of the complementary function yh(t),then we propose
yp(t) = [Y(t)]u(t)
![Page 1070: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1070.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1070,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of variation of parametersIf we can supply a basis Y(t) of the complementary function yh(t),then we propose
yp(t) = [Y(t)]u(t)
Substitution leads to
Y ′u + Yu′ = AYu + g.
Since Y ′ = AY,
Yu′ = g, or, u′ = [Y]−1g.
![Page 1071: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1071.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1071,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Linear Non-Homogeneous Systems
Method of variation of parametersIf we can supply a basis Y(t) of the complementary function yh(t),then we propose
yp(t) = [Y(t)]u(t)
Substitution leads to
Y ′u + Yu′ = AYu + g.
Since Y ′ = AY,
Yu′ = g, or, u′ = [Y]−1g.
Complete solution:
y(t) = yh + yp = [Y]c + [Y]
∫
[Y]−1gdt
This method is completely general.
![Page 1072: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1072.jpg)
Mathematical Methods in Engineering and Science ODE Systems 1072,
Fundamental IdeasLinear Homogeneous Systems with Constant CoefficientsLinear Non-Homogeneous SystemsNonlinear Systems
Points to note
Theory of ODE’s in terms of vector functions
Methods to find complementary functions in the case of constant coefficients particular solutions for all cases
Necessary Exercises: 1
![Page 1073: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1073.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1073,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Outline
Stability of Dynamic SystemsSecond Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
![Page 1074: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1074.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1074,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
A system of two first order linear differential equations:
y ′1 = a11y1 + a12y2
y ′2 = a21y1 + a22y2
or, y′ = Ay
![Page 1075: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1075.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1075,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
A system of two first order linear differential equations:
y ′1 = a11y1 + a12y2
y ′2 = a21y1 + a22y2
or, y′ = Ay
Phase: a pair of values of y1 and y2
Phase plane: plane of y1 and y2
Trajectory: a curve showing the evolution of the system for aparticular initial value problem
Phase portrait: all trajectories together showing the completepicture of the behaviour of the dynamic system
![Page 1076: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1076.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1076,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
A system of two first order linear differential equations:
y ′1 = a11y1 + a12y2
y ′2 = a21y1 + a22y2
or, y′ = Ay
Phase: a pair of values of y1 and y2
Phase plane: plane of y1 and y2
Trajectory: a curve showing the evolution of the system for aparticular initial value problem
Phase portrait: all trajectories together showing the completepicture of the behaviour of the dynamic system
Allowing only isolated equilibrium points, matrix A is non-singular: origin is the only equilibrium point.
Eigenvalues of A:
λ2 − (a11 + a22)λ+ (a11a22 − a12a21) = 0
![Page 1077: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1077.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1077,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
Characteristic equation:
λ2 − pλ+ q = 0,
with p = (a11 + a22) = λ1 + λ2 and q = a11a22 − a12a21 = λ1λ2
![Page 1078: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1078.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1078,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
Characteristic equation:
λ2 − pλ+ q = 0,
with p = (a11 + a22) = λ1 + λ2 and q = a11a22 − a12a21 = λ1λ2
Discriminant D = p2 − 4q and
λ1,2 =p
2±√
(p
2
)2− q =
p
2±√
D
2.
Solution (for diagonalizable A):
y = c1x1eλ1t + c2x2e
λ2t
![Page 1079: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1079.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1079,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
Characteristic equation:
λ2 − pλ+ q = 0,
with p = (a11 + a22) = λ1 + λ2 and q = a11a22 − a12a21 = λ1λ2
Discriminant D = p2 − 4q and
λ1,2 =p
2±√
(p
2
)2− q =
p
2±√
D
2.
Solution (for diagonalizable A):
y = c1x1eλ1t + c2x2e
λ2t
Solution for deficient A:
y = c1x1eλt + c2(tx1 + u)eλt
⇒ y′ = c1λx1eλt + c2(x1 + λu)eλt + λtc2x1e
λt
![Page 1080: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1080.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1080,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear Systems
y1 y1
y1y1
y2 y2
y2
y2
y1
y2
y1
y2
o
oo o
oo
(a) Saddle point (b) Centre (c) Spiral
(d) Improper node (e) Proper node (f) Degenerate node
Figure: Neighbourhood of critical points
![Page 1081: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1081.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1081,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear SystemsTable: Critical points of linear systems
Type Sub-type Eigenvalues Position in p-q chart StabilitySaddle pt real, opposite signs q < 0 unstableCentre pure imaginary q > 0, p = 0 stableSpiral complex, both q > 0, p 6= 0 stable
non-zero components D = p2 − 4q < 0 if p < 0,Node real, same sign q > 0, p 6= 0, D ≥ 0 unstable
improper unequal in magnitude D > 0 if p > 0proper equal, diagonalizable D = 0degenerate equal, deficient D = 0
![Page 1082: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1082.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1082,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Second Order Linear SystemsTable: Critical points of linear systems
Type Sub-type Eigenvalues Position in p-q chart StabilitySaddle pt real, opposite signs q < 0 unstableCentre pure imaginary q > 0, p = 0 stableSpiral complex, both q > 0, p 6= 0 stable
non-zero components D = p2 − 4q < 0 if p < 0,Node real, same sign q > 0, p 6= 0, D ≥ 0 unstable
improper unequal in magnitude D > 0 if p > 0proper equal, diagonalizable D = 0degenerate equal, deficient D = 0
= 0q− 4p2
spiral spiral
o p
q
saddle point
centre
unstablestable
node node
unstable
Figure: Zones of critical points in p-q chart
![Page 1083: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1083.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1083,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Nonlinear Dynamic Systems
Phase plane analysis
Determine all the critical points.
Linearize the ODE system around each of them as
y′ = J(y0)(y − y0).
With z = y − y0, analyze each neighbourhood from z′ = Jz.
Assemble outcomes of local phase plane analyses.
‘Features’ of a dynamic system are typically captured byits critical points and their neighbourhoods.
![Page 1084: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1084.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1084,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Nonlinear Dynamic Systems
Phase plane analysis
Determine all the critical points.
Linearize the ODE system around each of them as
y′ = J(y0)(y − y0).
With z = y − y0, analyze each neighbourhood from z′ = Jz.
Assemble outcomes of local phase plane analyses.
‘Features’ of a dynamic system are typically captured byits critical points and their neighbourhoods.
Limit cycles
isolated closed trajectories (only in nonlinear systems)
![Page 1085: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1085.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1085,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Nonlinear Dynamic Systems
Phase plane analysis
Determine all the critical points.
Linearize the ODE system around each of them as
y′ = J(y0)(y − y0).
With z = y − y0, analyze each neighbourhood from z′ = Jz.
Assemble outcomes of local phase plane analyses.
‘Features’ of a dynamic system are typically captured byits critical points and their neighbourhoods.
Limit cycles
isolated closed trajectories (only in nonlinear systems)
Systems with arbitrary dimension of state space?
![Page 1086: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1086.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1086,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Lyapunov Stability Analysis
Important terms
Stability: If y0 is a critical point of the dynamic systemy′ = f(y) and for every ǫ > 0, ∃ δ > 0 such that
‖y(t0)− y0‖ < δ ⇒ ‖y(t) − y0‖ < ǫ ∀ t > t0,
then y0 is a stable critical point. If, further,y(t)→ y0 as t →∞, then y0 is said to beasymptotically stable.
Positive definite function: A function V (y), with V (0) = 0, iscalled positive definite if
V (y) > 0 ∀y 6= 0.
Lyapunov function: A positive definite function V (y), havingcontinuous ∂V
∂yi, with a negative semi-definite rate of
changeV ′ = [∇V (y)]T f(y).
![Page 1087: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1087.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1087,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Lyapunov Stability Analysis
Lyapunov’s stability criteria:
Theorem: For a system y′ = f(y) with the origin as acritical point, if there exists a Lyapunov function V (y),then the system is stable at the origin, i.e. the origin is astable critical point.Further, if V ′(y) is negative definite, then it isasymptotically stable.
A generalization of the notion of total energy: negativity of its ratecorrespond to trajectories tending to decrease this ‘energy’.
![Page 1088: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1088.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1088,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Lyapunov Stability Analysis
Lyapunov’s stability criteria:
Theorem: For a system y′ = f(y) with the origin as acritical point, if there exists a Lyapunov function V (y),then the system is stable at the origin, i.e. the origin is astable critical point.Further, if V ′(y) is negative definite, then it isasymptotically stable.
A generalization of the notion of total energy: negativity of its ratecorrespond to trajectories tending to decrease this ‘energy’.
Note: Lyapunov’s method becomes particularly important when alinearized model allows no analysis or when its results are suspect.
![Page 1089: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1089.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1089,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Lyapunov Stability Analysis
Lyapunov’s stability criteria:
Theorem: For a system y′ = f(y) with the origin as acritical point, if there exists a Lyapunov function V (y),then the system is stable at the origin, i.e. the origin is astable critical point.Further, if V ′(y) is negative definite, then it isasymptotically stable.
A generalization of the notion of total energy: negativity of its ratecorrespond to trajectories tending to decrease this ‘energy’.
Note: Lyapunov’s method becomes particularly important when alinearized model allows no analysis or when its results are suspect.
Caution: It is a one-way criterion only!
![Page 1090: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1090.jpg)
Mathematical Methods in Engineering and Science Stability of Dynamic Systems 1090,
Second Order Linear SystemsNonlinear Dynamic SystemsLyapunov Stability Analysis
Points to note
Analysis of second order systems
Classification of critical points
Nonlinear systems and local linearization
Phase plane analysis
Examples in physics, engineering, economics,biological and social systems
Lyapunov’s method of stability analysis
Necessary Exercises: 1,2,3,4,5
![Page 1091: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1091.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1091,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Outline
Series Solutions and Special FunctionsPower Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
![Page 1092: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1092.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1092,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodMethods to solve an ODE in terms of elementary functions:
restricted in scope
Theory allows study of the properties of solutions!
![Page 1093: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1093.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1093,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodMethods to solve an ODE in terms of elementary functions:
restricted in scope
Theory allows study of the properties of solutions!
When elementary methods fail, gain knowledge about solutions through properties, and for actual evaluation develop infinite series.
Power series:
y(x) =
∞∑
n=0
anxn = a0 + a1x + a2x
2 + a3x3 + a4x
4 + a5x5 + · · ·
or in powers of (x − x0).
![Page 1094: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1094.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1094,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodMethods to solve an ODE in terms of elementary functions:
restricted in scope
Theory allows study of the properties of solutions!
When elementary methods fail, gain knowledge about solutions through properties, and for actual evaluation develop infinite series.
Power series:
y(x) =
∞∑
n=0
anxn = a0 + a1x + a2x
2 + a3x3 + a4x
4 + a5x5 + · · ·
or in powers of (x − x0).
A simple exercise:
Try developing power series solutions in the above formand study their properties for differential equations
y ′′ + y = 0 and 4x2y ′′ = y .
![Page 1095: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1095.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1095,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series Method
y ′′ + P(x)y ′ + Q(x)y = 0
If P(x) and Q(x) are analytic at a point x = x0,
i.e. if they possess convergent series expansions in powersof (x − x0) with some radius of convergence R,
then the solution is analytic at x0, and a power series solution
y(x) = a0 + a1(x − x0) + a2(x − x0)2 + a3(x − x0)
3 + · · ·is convergent at least for |x − x0| < R .
![Page 1096: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1096.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1096,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series Method
y ′′ + P(x)y ′ + Q(x)y = 0
If P(x) and Q(x) are analytic at a point x = x0,
i.e. if they possess convergent series expansions in powersof (x − x0) with some radius of convergence R,
then the solution is analytic at x0, and a power series solution
y(x) = a0 + a1(x − x0) + a2(x − x0)2 + a3(x − x0)
3 + · · ·is convergent at least for |x − x0| < R .
For x0 = 0 (without loss of generality), suppose
P(x) =
∞∑
n=0
pnxn = p0 + p1x + p2x
2 + p3x3 + · · · ,
Q(x) =
∞∑
n=0
qnxn = q0 + q1x + q2x
2 + q3x3 + · · · ,
and assume y(x) =∑∞
n=0 anxn.
![Page 1097: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1097.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1097,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodDifferentiation of y(x) =
∑∞n=0 anx
n as
y ′(x) =
∞∑
n=0
(n +1)an+1xn and y ′′(x) =
∞∑
n=0
(n +2)(n +1)an+2xn
leads to
P(x)y ′ =
∞∑
n=0
pnxn
[ ∞∑
n=0
(n + 1)an+1xn
]
=
∞∑
n=0
n∑
k=0
pn−k(k + 1)ak+1xn
Q(x)y =∞∑
n=0
qnxn
[ ∞∑
n=0
anxn
]
=∞∑
n=0
n∑
k=0
qn−kakxn
![Page 1098: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1098.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1098,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodDifferentiation of y(x) =
∑∞n=0 anx
n as
y ′(x) =
∞∑
n=0
(n +1)an+1xn and y ′′(x) =
∞∑
n=0
(n +2)(n +1)an+2xn
leads to
P(x)y ′ =
∞∑
n=0
pnxn
[ ∞∑
n=0
(n + 1)an+1xn
]
=
∞∑
n=0
n∑
k=0
pn−k(k + 1)ak+1xn
Q(x)y =∞∑
n=0
qnxn
[ ∞∑
n=0
anxn
]
=∞∑
n=0
n∑
k=0
qn−kakxn
⇒∞∑
n=0
[
(n + 2)(n + 1)an+2 +n∑
k=0
pn−k(k + 1)ak+1 +n∑
k=0
qn−kak
]
xn = 0
![Page 1099: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1099.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1099,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Power Series MethodDifferentiation of y(x) =
∑∞n=0 anx
n as
y ′(x) =
∞∑
n=0
(n +1)an+1xn and y ′′(x) =
∞∑
n=0
(n +2)(n +1)an+2xn
leads to
P(x)y ′ =
∞∑
n=0
pnxn
[ ∞∑
n=0
(n + 1)an+1xn
]
=
∞∑
n=0
n∑
k=0
pn−k(k + 1)ak+1xn
Q(x)y =∞∑
n=0
qnxn
[ ∞∑
n=0
anxn
]
=∞∑
n=0
n∑
k=0
qn−kakxn
⇒∞∑
n=0
[
(n + 2)(n + 1)an+2 +n∑
k=0
pn−k(k + 1)ak+1 +n∑
k=0
qn−kak
]
xn = 0
Recursion formula:
an+2 = − 1
(n + 2)(n + 1)
n∑
k=0
[(k + 1)pn−kak+1 + qn−kak ]
![Page 1100: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1100.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1100,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Frobenius’ Method
For the ODE y ′′ + P(x)y ′ + Q(x)y = 0, a point x = x0 is
ordinary point if P(x) and Q(x) are analytic at x = x0: powerseries solution is analytic
singular point if any of the two is non-analytic (singular) at x = x0
regular singularity: (x − x0)P(x) and(x − x0)
2Q(x) are analytic at the point irregular singularity
![Page 1101: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1101.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1101,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Frobenius’ Method
For the ODE y ′′ + P(x)y ′ + Q(x)y = 0, a point x = x0 is
ordinary point if P(x) and Q(x) are analytic at x = x0: powerseries solution is analytic
singular point if any of the two is non-analytic (singular) at x = x0
regular singularity: (x − x0)P(x) and(x − x0)
2Q(x) are analytic at the point irregular singularity
The case of regular singularity
For x0 = 0, with P(x) = b(x)x
and Q(x) = c(x)x2 ,
x2y ′′ + xb(x)y ′ + c(x)y = 0
in which b(x) and c(x) are analytic at the origin.
![Page 1102: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1102.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1102,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Frobenius’ Method
Working steps:
1. Assume the solution in the form y(x) = x r∑∞
n=0 anxn.
2. Differentiate to get the series expansions for y ′(x) and y ′′(x).
3. Substitute these series for y(x), y ′(x) and y ′′(x) into thegiven ODE and collect coefficients of x r , x r+1, x r+2 etc.
4. Equate the coefficient of x r to zero to obtain an equation inthe index r , called the indicial equation as
r(r − 1) + b0r + c0 = 0;
allowing a0 to become arbitrary.
5. For each solution r , equate other coefficients to obtain a1, a2,a3 etc in terms of a0.
Note: The need is to develop two solutions.
![Page 1103: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1103.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1103,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Defined as Integrals
Gamma function: Γ(n) =∫∞0 e−xxn−1dx , convergent for n > 0.
Recurrence relation Γ(1) = 1, Γ(n + 1) = nΓ(n)allows extension of the definition for the entire realline except for zero and negative integers.Γ(n + 1) = n! for non-negative integers.(A generalization of the factorial function.)
Beta function: B(m, n) =∫ 10 xm−1(1− x)n−1dx =
2∫ π/20 sin2m−1 θ cos2n−1 θ dθ; m, n > 0.
B(m, n) = B(n,m); B(m, n) = Γ(m)Γ(n)Γ(m+n)
Error function: erf (x) = 2√π
∫ x
0 e−t2dt.
(Area under the normal or Gaussian distribution)
Sine integral function: Si (x) =∫ x
0sin tt
dt.
![Page 1104: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1104.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1104,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
In the study of some important problems in physics,
some variable-coefficient ODE’s appear recurrently,
defying analytical solution!
![Page 1105: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1105.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1105,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
In the study of some important problems in physics,
some variable-coefficient ODE’s appear recurrently,
defying analytical solution!
Series solutions ⇒ properties and connections⇒ further problems ⇒ further solutions ⇒ · · ·
![Page 1106: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1106.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1106,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
In the study of some important problems in physics,
some variable-coefficient ODE’s appear recurrently,
defying analytical solution!
Series solutions ⇒ properties and connections⇒ further problems ⇒ further solutions ⇒ · · ·
Table: Special functions of mathematical physics
Name of the ODE Form of the ODE Resulting functions
Legendre’s equation (1 − x2)y′′ − 2xy′ + k(k + 1)y = 0 Legendre functionsLegendre polynomials
Airy’s equation y′′ ± k2xy = 0 Airy functions
Chebyshev’s equation (1 − x2)y′′ − xy′ + k2y = 0 Chebyshev polynomials
Hermite’s equation y′′ − 2xy′ + 2ky = 0 Hermite functionsHermite polynomials
Bessel’s equation x2y′′ + xy′ + (x2− k2)y = 0 Bessel functions
Neumann functionsHankel functions
Gauss’s hypergeometric x(1 − x)y′′ + [c − (a + b + 1)x]y′ − aby = 0 Hypergeometric functionequation
Laguerre’s equation xy′′ + (1 − x)y′ + ky = 0 Laguerre polynomials
![Page 1107: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1107.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1107,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre’s equation
(1− x2)y ′′ − 2xy ′ + k(k + 1)y = 0
P(x) = − 2x1−x2 and Q(x) = k(k+1)
1−x2 are analytic at x = 0 withradius of convergence R = 1.
x = 0 is an ordinary point and a power series solutiony(x) =
∑∞n=0 anx
n is convergent at least for |x | < 1.
![Page 1108: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1108.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1108,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre’s equation
(1− x2)y ′′ − 2xy ′ + k(k + 1)y = 0
P(x) = − 2x1−x2 and Q(x) = k(k+1)
1−x2 are analytic at x = 0 withradius of convergence R = 1.
x = 0 is an ordinary point and a power series solutiony(x) =
∑∞n=0 anx
n is convergent at least for |x | < 1.
Apply power series method:
a2 = −k(k + 1)
2!a0,
a3 = −(k + 2)(k − 1)
3!a1
and an+2 = −(k − n)(k + n + 1)
(n + 2)(n + 1)an for n ≥ 2.
Solution: y(x) = a0y1(x) + a1y2(x)
![Page 1109: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1109.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1109,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre functions
y1(x) = 1− k(k + 1)
2!x2 +
k(k − 2)(k + 1)(k + 3)
4!x4 − · · ·
y2(x) = x − (k − 1)(k + 2)
3!x3 +
(k − 1)(k − 3)(k + 2)(k + 4)
5!x5 − · ·
![Page 1110: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1110.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1110,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre functions
y1(x) = 1− k(k + 1)
2!x2 +
k(k − 2)(k + 1)(k + 3)
4!x4 − · · ·
y2(x) = x − (k − 1)(k + 2)
3!x3 +
(k − 1)(k − 3)(k + 2)(k + 4)
5!x5 − · ·
Special significance: non-negative integral values of k
![Page 1111: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1111.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1111,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre functions
y1(x) = 1− k(k + 1)
2!x2 +
k(k − 2)(k + 1)(k + 3)
4!x4 − · · ·
y2(x) = x − (k − 1)(k + 2)
3!x3 +
(k − 1)(k − 3)(k + 2)(k + 4)
5!x5 − · ·
Special significance: non-negative integral values of k
For each k = 0, 1, 2, 3, · · · ,one of the series terminates at the term containing xk .
Polynomial solution: valid for the entire real line!
![Page 1112: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1112.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1112,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre functions
y1(x) = 1− k(k + 1)
2!x2 +
k(k − 2)(k + 1)(k + 3)
4!x4 − · · ·
y2(x) = x − (k − 1)(k + 2)
3!x3 +
(k − 1)(k − 3)(k + 2)(k + 4)
5!x5 − · ·
Special significance: non-negative integral values of k
For each k = 0, 1, 2, 3, · · · ,one of the series terminates at the term containing xk .
Polynomial solution: valid for the entire real line!
Recurrence relation in reverse:
ak−2 = − k(k − 1)
2(2k − 1)ak
![Page 1113: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1113.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1113,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre polynomialChoosing ak = (2k−1)(2k−3)···3·1
k! ,
Pk(x) =(2k − 1)(2k − 3) · · · 3 · 1
k!
×[
xk − k(k − 1)
2(2k − 1)xk−2 +
k(k − 1)(k − 2)(k − 3)
2 · 4(2k − 1)(2k − 3)xk−4 − · · ·
]
.
This choice of ak ensures Pk(1) = 1 and implies Pk(−1) = (−1)k .
![Page 1114: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1114.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1114,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Legendre polynomialChoosing ak = (2k−1)(2k−3)···3·1
k! ,
Pk(x) =(2k − 1)(2k − 3) · · · 3 · 1
k!
×[
xk − k(k − 1)
2(2k − 1)xk−2 +
k(k − 1)(k − 2)(k − 3)
2 · 4(2k − 1)(2k − 3)xk−4 − · · ·
]
.
This choice of ak ensures Pk(1) = 1 and implies Pk(−1) = (−1)k .Initial Legendre polynomials:
P0(x) = 1,
P1(x) = x ,
P2(x) =1
2(3x2 − 1),
P3(x) =1
2(5x3 − 3x),
P4(x) =1
8(35x4 − 30x2 + 3) etc.
![Page 1115: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1115.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1115,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
P (
x)
P (x)
P (x)
P (x)
P (x)
P (x) P (x)
0
1
n
3
5 4
2
Figure: Legendre polynomials
![Page 1116: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1116.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1116,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
P (
x)
P (x)
P (x)
P (x)
P (x)
P (x) P (x)
0
1
n
3
5 4
2
Figure: Legendre polynomials
All roots of a Legendre polynomial are real and they lie in [−1, 1].
![Page 1117: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1117.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1117,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
P (
x)
P (x)
P (x)
P (x)
P (x)
P (x) P (x)
0
1
n
3
5 4
2
Figure: Legendre polynomials
All roots of a Legendre polynomial are real and they lie in [−1, 1].
Orthogonality?
![Page 1118: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1118.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1118,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Bessel’s equation
x2y ′′ + xy ′ + (x2 − k2)y = 0
x = 0 is a regular singular point.
![Page 1119: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1119.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1119,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Bessel’s equation
x2y ′′ + xy ′ + (x2 − k2)y = 0
x = 0 is a regular singular point.Frobenius’ method: carrying out the early steps,
(r2−k2)a0xr+[(r+1)2−k2]a1x
r+1+
∞∑
n=2
[an−2+r2−k2+n(n+2r)an]xr+n = 0
Indicial equation: r2 − k2 = 0⇒ r = ±k
![Page 1120: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1120.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1120,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Bessel’s equation
x2y ′′ + xy ′ + (x2 − k2)y = 0
x = 0 is a regular singular point.Frobenius’ method: carrying out the early steps,
(r2−k2)a0xr+[(r+1)2−k2]a1x
r+1+
∞∑
n=2
[an−2+r2−k2+n(n+2r)an]xr+n = 0
Indicial equation: r2 − k2 = 0⇒ r = ±kWith r = k, (r + 1)2 − k2 6= 0 ⇒ a1 = 0 and
an = − an−2
n(n + 2r)for n ≥ 2.
Odd coefficients are zero and
a2 = − a0
2(2k + 2), a4 =
a0
2 · 4(2k + 2)(2k + 4), etc.
![Page 1121: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1121.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1121,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Bessel functions:Selecting a0 = 1
2kΓ(k+1)and using n = 2m,
am =(−1)m
2k+2mm!Γ(k + m + 1).
Bessel function of the first kind of order k:
Jk(x) =
∞∑
m=0
(−1)mxk+2m
2k+2mm!Γ(k + m + 1)=
∞∑
m=0
(−1)m(
x2
)k+2m
m!Γ(k + m + 1)
![Page 1122: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1122.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1122,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Special Functions Arising as Solutions of ODE’s
Bessel functions:Selecting a0 = 1
2kΓ(k+1)and using n = 2m,
am =(−1)m
2k+2mm!Γ(k + m + 1).
Bessel function of the first kind of order k:
Jk(x) =
∞∑
m=0
(−1)mxk+2m
2k+2mm!Γ(k + m + 1)=
∞∑
m=0
(−1)m(
x2
)k+2m
m!Γ(k + m + 1)
When k is not an integer, J−k(x) completes the basis.
For integer k, J−k(x) = (−1)kJk(x), linearly dependent!Reduction of order can be used to find another solution.
Bessel function of the second kind or Neumann function
![Page 1123: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1123.jpg)
Mathematical Methods in Engineering and Science Series Solutions and Special Functions 1123,
Power Series MethodFrobenius’ MethodSpecial Functions Defined as IntegralsSpecial Functions Arising as Solutions of ODE’s
Points to note
Solution in power series
Ordinary points and singularities
Definition of special functions
Legendre polynomials
Bessel functions
Necessary Exercises: 2,3,4,5
![Page 1124: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1124.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1124,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Outline
Sturm-Liouville TheoryPreliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
![Page 1125: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1125.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1125,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
A simple boundary value problem:
y ′′ + 2y = 0, y(0) = 0, y(π) = 0
![Page 1126: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1126.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1126,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
A simple boundary value problem:
y ′′ + 2y = 0, y(0) = 0, y(π) = 0
General solution of the ODE:
y(x) = a sin(x√
2) + b cos(x√
2)
Condition y(0) = 0 ⇒ b = 0. Hence, y(x) = a sin(x√
2).
Then, y(π) = 0 ⇒ a = 0. Only solution is y(x) = 0.
![Page 1127: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1127.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1127,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
A simple boundary value problem:
y ′′ + 2y = 0, y(0) = 0, y(π) = 0
General solution of the ODE:
y(x) = a sin(x√
2) + b cos(x√
2)
Condition y(0) = 0 ⇒ b = 0. Hence, y(x) = a sin(x√
2).
Then, y(π) = 0 ⇒ a = 0. Only solution is y(x) = 0.
Now, consider the BVP
y ′′ + 4y = 0, y(0) = 0, y(π) = 0.
The same steps give y(x) = a sin(2x), with arbitrary value of a.
Infinite number of non-trivial solutions!
![Page 1128: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1128.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1128,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Boundary value problems as eigenvalue problemsExplore the possible solutions of the BVP
y ′′ + ky = 0, y(0) = 0, y(π) = 0.
![Page 1129: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1129.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1129,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Boundary value problems as eigenvalue problemsExplore the possible solutions of the BVP
y ′′ + ky = 0, y(0) = 0, y(π) = 0.
With k ≤ 0, no hope for a non-trivial solution. Considerk = ν2 > 0.
Solutions: y = a sin(νx), only for specific values of ν (or k):ν = 0,±1,±2,±3, · · · ; i.e. k = 0, 1, 4, 9, · · · .
![Page 1130: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1130.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1130,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Boundary value problems as eigenvalue problemsExplore the possible solutions of the BVP
y ′′ + ky = 0, y(0) = 0, y(π) = 0.
With k ≤ 0, no hope for a non-trivial solution. Considerk = ν2 > 0.
Solutions: y = a sin(νx), only for specific values of ν (or k):ν = 0,±1,±2,±3, · · · ; i.e. k = 0, 1, 4, 9, · · · .
Question:
For what values of k (eigenvalues), does the given BVPpossess non-trivial solutions, and
what are the corresponding solutions (eigenfunctions), up toarbitrary scalar multiples?
Analogous to the algebraic eigenvalue problem Av = λv!
![Page 1131: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1131.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1131,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Boundary value problems as eigenvalue problemsExplore the possible solutions of the BVP
y ′′ + ky = 0, y(0) = 0, y(π) = 0.
With k ≤ 0, no hope for a non-trivial solution. Considerk = ν2 > 0.
Solutions: y = a sin(νx), only for specific values of ν (or k):ν = 0,±1,±2,±3, · · · ; i.e. k = 0, 1, 4, 9, · · · .
Question:
For what values of k (eigenvalues), does the given BVPpossess non-trivial solutions, and
what are the corresponding solutions (eigenfunctions), up toarbitrary scalar multiples?
Analogous to the algebraic eigenvalue problem Av = λv!Analogy of a Hermitian matrix: self-adjoint differential operator.
![Page 1132: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1132.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1132,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Consider the ODE y ′′ + P(x)y ′ + Q(x)y = 0.Question:
Is it possible to find functions F (x) and G (x) such that
F (x)y ′′ + F (x)P(x)y ′ + F (x)Q(x)y
gets reduced to the derivative of F (x)y ′ + G (x)y?
![Page 1133: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1133.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1133,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Consider the ODE y ′′ + P(x)y ′ + Q(x)y = 0.Question:
Is it possible to find functions F (x) and G (x) such that
F (x)y ′′ + F (x)P(x)y ′ + F (x)Q(x)y
gets reduced to the derivative of F (x)y ′ + G (x)y?
Comparing with
d
dx[F (x)y ′ + G (x)y ] = F (x)y ′′ + [F ′(x) + G (x)]y ′ + G ′(x)y ,
F ′(x) + G (x) = F (x)P(x) and G ′(x) = F (x)Q(x).
![Page 1134: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1134.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1134,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Consider the ODE y ′′ + P(x)y ′ + Q(x)y = 0.Question:
Is it possible to find functions F (x) and G (x) such that
F (x)y ′′ + F (x)P(x)y ′ + F (x)Q(x)y
gets reduced to the derivative of F (x)y ′ + G (x)y?
Comparing with
d
dx[F (x)y ′ + G (x)y ] = F (x)y ′′ + [F ′(x) + G (x)]y ′ + G ′(x)y ,
F ′(x) + G (x) = F (x)P(x) and G ′(x) = F (x)Q(x).
Elimination of G (x):
F ′′(x)− P(x)F ′(x) + [Q(x)− P ′(x)]F (x) = 0
This is the adjoint of the original ODE.
![Page 1135: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1135.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1135,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
The adjoint ODE
The adjoint of the ODE y ′′ + P(x)y ′ + Q(x)y = 0 is
F ′′ + P1F′ + Q1F = 0,
where P1 = −P and Q1 = Q − P ′.
![Page 1136: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1136.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1136,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
The adjoint ODE
The adjoint of the ODE y ′′ + P(x)y ′ + Q(x)y = 0 is
F ′′ + P1F′ + Q1F = 0,
where P1 = −P and Q1 = Q − P ′. Then, the adjoint of F ′′ + P1F
′ + Q1F = 0 is
φ′′ + P2φ′ + Q2φ = 0,
where P2 = −P1 = P andQ2 = Q1 − P ′
1 = Q − P ′ − (−P ′) = Q.
![Page 1137: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1137.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1137,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
The adjoint ODE
The adjoint of the ODE y ′′ + P(x)y ′ + Q(x)y = 0 is
F ′′ + P1F′ + Q1F = 0,
where P1 = −P and Q1 = Q − P ′. Then, the adjoint of F ′′ + P1F
′ + Q1F = 0 is
φ′′ + P2φ′ + Q2φ = 0,
where P2 = −P1 = P andQ2 = Q1 − P ′
1 = Q − P ′ − (−P ′) = Q.
The adjoint of the adjoint of a second order linearhomogeneous equation is the original equation itself.
![Page 1138: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1138.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1138,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
The adjoint ODE
The adjoint of the ODE y ′′ + P(x)y ′ + Q(x)y = 0 is
F ′′ + P1F′ + Q1F = 0,
where P1 = −P and Q1 = Q − P ′. Then, the adjoint of F ′′ + P1F
′ + Q1F = 0 is
φ′′ + P2φ′ + Q2φ = 0,
where P2 = −P1 = P andQ2 = Q1 − P ′
1 = Q − P ′ − (−P ′) = Q.
The adjoint of the adjoint of a second order linearhomogeneous equation is the original equation itself.
When is an ODE its own adjoint? y ′′ + P(x)y ′ + Q(x)y = 0 is self-adjoint only in the trivial case
of P(x) = 0. What about F (x)y ′′ + F (x)P(x)y ′ + F (x)Q(x)y = 0?
![Page 1139: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1139.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1139,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Second order self-adjoint ODE
Question: What is the adjoint of Fy ′′ + FPy ′ + FQy = 0?
![Page 1140: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1140.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1140,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Second order self-adjoint ODE
Question: What is the adjoint of Fy ′′ + FPy ′ + FQy = 0?
Rephrased question: What is the ODE that φ(x) has to satisfy if
φFy ′′ + φFPy ′ + φFQy =d
dx
[
φFy ′ + ξ(x)y]
?
![Page 1141: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1141.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1141,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Second order self-adjoint ODE
Question: What is the adjoint of Fy ′′ + FPy ′ + FQy = 0?
Rephrased question: What is the ODE that φ(x) has to satisfy if
φFy ′′ + φFPy ′ + φFQy =d
dx
[
φFy ′ + ξ(x)y]
?
Comparing terms,
d
dx(φF ) + ξ(x) = φFP and ξ′(x) = φFQ.
Eliminating ξ(x), we have d2
dx2 (φF ) + φFQ = ddx
(φFP).
![Page 1142: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1142.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1142,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Second order self-adjoint ODE
Question: What is the adjoint of Fy ′′ + FPy ′ + FQy = 0?
Rephrased question: What is the ODE that φ(x) has to satisfy if
φFy ′′ + φFPy ′ + φFQy =d
dx
[
φFy ′ + ξ(x)y]
?
Comparing terms,
d
dx(φF ) + ξ(x) = φFP and ξ′(x) = φFQ.
Eliminating ξ(x), we have d2
dx2 (φF ) + φFQ = ddx
(φFP).
Fφ′′ + 2F ′φ′ + F ′′φ+ FQφ = FPφ′ + (FP)′φ
⇒ Fφ′′ + (2F ′ − FP)φ′ +[
F ′′ − (FP)′ + FQ]
φ = 0
This is the same as the original ODE, when F ′(x) = F (x)P(x)
![Page 1143: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1143.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1143,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Casting a given ODE into the self-adjoint form:
Equation y ′′ + P(x)y ′ + Q(x)y = 0 is converted to theself-adjoint form through the multiplication ofF (x) = e
R
P(x)dx .
General form of self-adjoint equations:
d
dx[F (x)y ′] + R(x)y = 0
![Page 1144: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1144.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1144,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Preliminary Ideas
Casting a given ODE into the self-adjoint form:
Equation y ′′ + P(x)y ′ + Q(x)y = 0 is converted to theself-adjoint form through the multiplication ofF (x) = e
R
P(x)dx .
General form of self-adjoint equations:
d
dx[F (x)y ′] + R(x)y = 0
Working rules:
To determine whether a given ODE is in the self-adjoint form,check whether the coefficient of y ′ is the derivative of thecoefficient of y ′′.
To convert an ODE into the self-adjoint form, first obtain theequation in normal form by dividing with the coefficient of y ′′.If the coefficient of y ′ now is P(x), then next multiply theresulting equation with e
R
Pdx .
![Page 1145: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1145.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1145,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville Problems
Sturm-Liouville equation
[r(x)y ′]′ + [q(x) + λp(x)]y = 0,
where p, q, r and r ′ are continuous on [a, b], with p(x) > 0 on[a, b] and r(x) > 0 on (a, b).
![Page 1146: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1146.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1146,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville Problems
Sturm-Liouville equation
[r(x)y ′]′ + [q(x) + λp(x)]y = 0,
where p, q, r and r ′ are continuous on [a, b], with p(x) > 0 on[a, b] and r(x) > 0 on (a, b).
With different boundary conditions,
Regular S-L problem:a1y(a) + a2y
′(a) = 0 and b1y(b) + b2y′(b) = 0,
vectors [a1 a2]T and [b1 b2]
T being non-zero.
Periodic S-L problem: With r(a) = r(b),y(a) = y(b) and y ′(a) = y ′(b).
Singular S-L problem: If r(a) = 0, no boundary condition isneeded at x = a. If r(b) = 0, no boundary conditionis needed at x = b.(We just look for bounded solutions over [a, b].)
![Page 1147: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1147.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1147,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsOrthogonality of eigenfunctions
Theorem: If ym(x) and yn(x) are eigenfunctions(solutions) of a Sturm-Liouville problem corresponding todistinct eigenvalues λm and λn respectively, then
(ym, yn) ≡∫ b
a
p(x)ym(x)yn(x)dx = 0,
i.e. they are orthogonal with respect to the weightfunction p(x).
![Page 1148: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1148.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1148,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsOrthogonality of eigenfunctions
Theorem: If ym(x) and yn(x) are eigenfunctions(solutions) of a Sturm-Liouville problem corresponding todistinct eigenvalues λm and λn respectively, then
(ym, yn) ≡∫ b
a
p(x)ym(x)yn(x)dx = 0,
i.e. they are orthogonal with respect to the weightfunction p(x).
From the hypothesis,
(ry ′m)′ + (q + λmp)ym = 0 ⇒ (q + λmp)ymyn = −(ry ′
m)′yn
(ry ′n)
′ + (q + λnp)yn = 0 ⇒ (q + λnp)ymyn = −(ry ′n)
′ym
![Page 1149: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1149.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1149,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsOrthogonality of eigenfunctions
Theorem: If ym(x) and yn(x) are eigenfunctions(solutions) of a Sturm-Liouville problem corresponding todistinct eigenvalues λm and λn respectively, then
(ym, yn) ≡∫ b
a
p(x)ym(x)yn(x)dx = 0,
i.e. they are orthogonal with respect to the weightfunction p(x).
From the hypothesis,
(ry ′m)′ + (q + λmp)ym = 0 ⇒ (q + λmp)ymyn = −(ry ′
m)′yn
(ry ′n)
′ + (q + λnp)yn = 0 ⇒ (q + λnp)ymyn = −(ry ′n)
′ym
Subtracting,
(λm − λn)pymyn = (ry ′n)
′ym + (ry ′n)y
′m − (ry ′
m)y ′n − (ry ′
m)′yn
=[
r(ymy ′n − yny
′m)]′.
![Page 1150: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1150.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1150,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsIntegrating both sides,
(λm − λn)
∫ b
a
p(x)ym(x)yn(x)dx
= r(b)[ym(b)y ′n(b)− yn(b)y ′
m(b)]− r(a)[ym(a)y ′n(a)− yn(a)y
′m(a)].
![Page 1151: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1151.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1151,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsIntegrating both sides,
(λm − λn)
∫ b
a
p(x)ym(x)yn(x)dx
= r(b)[ym(b)y ′n(b)− yn(b)y ′
m(b)]− r(a)[ym(a)y ′n(a)− yn(a)y
′m(a)].
In a regular S-L problem, from the boundary condition atx = a, the homogeneous system[
ym(a) y ′m(a)
yn(a) y ′n(a)
] [
a1
a2
]
=
[
00
]
has non-trivial solutions.
Therefore, ym(a)y ′n(a)− yn(a)y
′m(a) = 0.
Similarly, ym(b)y ′n(b)− yn(b)y ′
m(b) = 0. In a singular S-L problem, zero value of r(x) at a boundary
makes the corresponding term vanish even without a BC. In a periodic S-L problem, the two terms cancel out together.
![Page 1152: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1152.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1152,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsIntegrating both sides,
(λm − λn)
∫ b
a
p(x)ym(x)yn(x)dx
= r(b)[ym(b)y ′n(b)− yn(b)y ′
m(b)]− r(a)[ym(a)y ′n(a)− yn(a)y
′m(a)].
In a regular S-L problem, from the boundary condition atx = a, the homogeneous system[
ym(a) y ′m(a)
yn(a) y ′n(a)
] [
a1
a2
]
=
[
00
]
has non-trivial solutions.
Therefore, ym(a)y ′n(a)− yn(a)y
′m(a) = 0.
Similarly, ym(b)y ′n(b)− yn(b)y ′
m(b) = 0. In a singular S-L problem, zero value of r(x) at a boundary
makes the corresponding term vanish even without a BC. In a periodic S-L problem, the two terms cancel out together.
Since λm 6= λn, in all cases,∫ b
a
p(x)ym(x)yn(x)dx = 0.
![Page 1153: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1153.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1153,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsExample: Legendre polynomials over [−1, 1]Legendre’s equation
d
dx[(1− x2)y ′] + k(k + 1)y = 0
is self-adjoint and defines a singular Sturm Liouville problem over[−1, 1] with p(x) = 1, q(x) = 0, r(x) = 1− x2 and λ = k(k + 1).
![Page 1154: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1154.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1154,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsExample: Legendre polynomials over [−1, 1]Legendre’s equation
d
dx[(1− x2)y ′] + k(k + 1)y = 0
is self-adjoint and defines a singular Sturm Liouville problem over[−1, 1] with p(x) = 1, q(x) = 0, r(x) = 1− x2 and λ = k(k + 1).
(m−n)(m+n+1)
∫ 1
−1Pm(x)Pn(x)dx = [(1−x2)(PmP ′
n−PnP′m)]1−1 = 0
![Page 1155: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1155.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1155,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsExample: Legendre polynomials over [−1, 1]Legendre’s equation
d
dx[(1− x2)y ′] + k(k + 1)y = 0
is self-adjoint and defines a singular Sturm Liouville problem over[−1, 1] with p(x) = 1, q(x) = 0, r(x) = 1− x2 and λ = k(k + 1).
(m−n)(m+n+1)
∫ 1
−1Pm(x)Pn(x)dx = [(1−x2)(PmP ′
n−PnP′m)]1−1 = 0
From orthogonal decompositions 1 = P0(x), x = P1(x),
x2 =1
3(3x2 − 1) +
1
3=
2
3P2(x) +
1
3P0(x),
x3 =1
5(5x3 − 3x) +
3
5x =
2
5P3(x) +
3
5P1(x),
x4 =8
35P4(x) +
4
7P2(x) +
1
5P0(x) etc;
Pk(x) is orthogonal to all polynomials of degree less than k.
![Page 1156: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1156.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1156,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsReal eigenvalues
Eigenvalues of a Sturm-Liouville problem are real.
![Page 1157: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1157.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1157,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsReal eigenvalues
Eigenvalues of a Sturm-Liouville problem are real.
Let eigenvalue λ = µ+ iν and eigenfunction y(x) = u(x) + iv(x).Substitution leads to
[r(u′ + iv ′)]′ + [q + (µ+ iν)p](u + iv) = 0.
![Page 1158: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1158.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1158,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsReal eigenvalues
Eigenvalues of a Sturm-Liouville problem are real.
Let eigenvalue λ = µ+ iν and eigenfunction y(x) = u(x) + iv(x).Substitution leads to
[r(u′ + iv ′)]′ + [q + (µ+ iν)p](u + iv) = 0.
Separation of real and imaginary parts:
[ru′]′ + (q + µp)u − νpv = 0 ⇒ νpv2 = [ru′]′v + (q + µp)uv
[rv ′]′ + (q + µp)v + νpu = 0 ⇒ νpu2 = −[rv ′]′u − (q + µp)uv
Adding together,
νp(u2 +v2) = [ru′]′v +[ru′]v ′− [rv ′]u′− [rv ′]′u = −[
r(uv ′ − vu′)]′
![Page 1159: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1159.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1159,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Sturm-Liouville ProblemsReal eigenvalues
Eigenvalues of a Sturm-Liouville problem are real.
Let eigenvalue λ = µ+ iν and eigenfunction y(x) = u(x) + iv(x).Substitution leads to
[r(u′ + iv ′)]′ + [q + (µ+ iν)p](u + iv) = 0.
Separation of real and imaginary parts:
[ru′]′ + (q + µp)u − νpv = 0 ⇒ νpv2 = [ru′]′v + (q + µp)uv
[rv ′]′ + (q + µp)v + νpu = 0 ⇒ νpu2 = −[rv ′]′u − (q + µp)uv
Adding together,
νp(u2 +v2) = [ru′]′v +[ru′]v ′− [rv ′]u′− [rv ′]′u = −[
r(uv ′ − vu′)]′
Integration and application of boundary conditions leads to
ν
∫ b
a
p(x)[u2(x) + v2(x)]dx = 0.
ν = 0 and λ = µ
![Page 1160: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1160.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1160,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Eigenfunctions of Sturm-Liouville problems:
convenient and powerful instruments to represent andmanipulate fairly general classes of functions
![Page 1161: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1161.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1161,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Eigenfunctions of Sturm-Liouville problems:
convenient and powerful instruments to represent andmanipulate fairly general classes of functions
y0, y1, y2, y3, · · · : a family of continuous functions over [a, b],mutually orthogonal with respect to p(x).
Representation of a function f (x) on [a, b]:
f (x) =
∞∑
m=0
amym(x) = a0y0(x)+ a1y1(x)+ a2y2(x)+ a3y3(x)+ · · ·
Generalized Fourier seriesAnalogous to the representation of a vector as a linear combinationof a set of mutually orthogonal vectors.
Question: How to determine the coefficients (an)?
![Page 1162: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1162.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1162,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Inner product:
(f , yn) =
∫ b
a
p(x)f (x)yn(x)dx
=
∫ b
a
∞∑
m=0
[amp(x)ym(x)yn(x)]dx =
∞∑
m=0
am(ym, yn) = an‖yn‖2
where
‖yn‖ =√
(yn, yn) =
√
∫ b
a
p(x)y2n (x)dx
Fourier coefficients: an = (f ,yn)‖yn‖2
![Page 1163: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1163.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1163,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Inner product:
(f , yn) =
∫ b
a
p(x)f (x)yn(x)dx
=
∫ b
a
∞∑
m=0
[amp(x)ym(x)yn(x)]dx =
∞∑
m=0
am(ym, yn) = an‖yn‖2
where
‖yn‖ =√
(yn, yn) =
√
∫ b
a
p(x)y2n (x)dx
Fourier coefficients: an = (f ,yn)‖yn‖2
Normalized eigenfunctions:
φm(x) =ym(x)
‖ym(x)‖Generalized Fourier series (in orthonormal basis):
f (x) =
∞∑
m=0
cmφm(x) = c0φ0(x)+c1φ1(x)+c2φ2(x)+c3φ3(x)+ · · ·
![Page 1164: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1164.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1164,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
In terms of a finite number of members of the family φk(x),
ΦN(x) =
N∑
m=0
αmφm(x) = α0φ0(x)+α1φ1(x)+α2φ2(x)+· · ·+αNφN(x).
Error
E = ‖f − ΦN‖2 =
∫ b
a
p(x)
[
f (x)−N∑
m=0
αmφm(x)
]2
dx
![Page 1165: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1165.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1165,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
In terms of a finite number of members of the family φk(x),
ΦN(x) =
N∑
m=0
αmφm(x) = α0φ0(x)+α1φ1(x)+α2φ2(x)+· · ·+αNφN(x).
Error
E = ‖f − ΦN‖2 =
∫ b
a
p(x)
[
f (x)−N∑
m=0
αmφm(x)
]2
dx
Error is minimized when
∂E
∂αn=
∫ b
a
2p(x)
[
f (x)−N∑
m=0
αmφm(x)
]
[−φn(x)]dx = 0
⇒∫ b
a
αnp(x)φ2n(x)dx =
∫ b
a
p(x)f (x)φn(x)dx .
αn = cn
best approximation in the mean or least square approximation
![Page 1166: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1166.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1166,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Using the Fourier coefficients, error
E = (f , f )−2N∑
n=0
cn(f , φn)+N∑
n=0
c2n (φn, φn) = ‖f ‖2−2
N∑
n=0
c2n+
N∑
n=0
c2n
E = ‖f ‖2 −N∑
n=0
c2n ≥ 0.
Bessel’s inequality:
N∑
n=0
c2n ≤ ‖f ‖2 =
∫ b
a
p(x)f 2(x)dx
![Page 1167: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1167.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1167,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Using the Fourier coefficients, error
E = (f , f )−2N∑
n=0
cn(f , φn)+N∑
n=0
c2n (φn, φn) = ‖f ‖2−2
N∑
n=0
c2n+
N∑
n=0
c2n
E = ‖f ‖2 −N∑
n=0
c2n ≥ 0.
Bessel’s inequality:
N∑
n=0
c2n ≤ ‖f ‖2 =
∫ b
a
p(x)f 2(x)dx
Partial sum
sk(x) =k∑
m=0
amφm(x)
Question: Does the sequence of sk converge?Answer: The bound in Bessel’s inequality ensures convergence.
![Page 1168: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1168.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1168,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Question: Does it converge to f ?
limk→∞
∫ b
a
p(x)[sk(x)− f (x)]2dx = 0?
Answer: Depends on the basis used.
![Page 1169: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1169.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1169,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Question: Does it converge to f ?
limk→∞
∫ b
a
p(x)[sk(x)− f (x)]2dx = 0?
Answer: Depends on the basis used.Convergence in the mean or mean-square convergence:
An orthonormal set of functions φk(x) on an intervala ≤ x ≤ b is said to be complete in a class of functions,or to form a basis for it, if the corresponding generalizedFourier series for a function converges in the mean to thefunction, for every function belonging to that class.
Parseval’s identity:∑∞
n=0 c2n = ‖f ‖2
![Page 1170: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1170.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1170,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Eigenfunction Expansions
Question: Does it converge to f ?
limk→∞
∫ b
a
p(x)[sk(x)− f (x)]2dx = 0?
Answer: Depends on the basis used.Convergence in the mean or mean-square convergence:
An orthonormal set of functions φk(x) on an intervala ≤ x ≤ b is said to be complete in a class of functions,or to form a basis for it, if the corresponding generalizedFourier series for a function converges in the mean to thefunction, for every function belonging to that class.
Parseval’s identity:∑∞
n=0 c2n = ‖f ‖2
Eigenfunction expansion: generalized Fourier series in terms ofeigenfunctions of a Sturm-Liouville problem
convergent for continuous functions with piecewise continuousderivatives, i.e. they form a basis for this class.
![Page 1171: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1171.jpg)
Mathematical Methods in Engineering and Science Sturm-Liouville Theory 1171,
Preliminary IdeasSturm-Liouville ProblemsEigenfunction Expansions
Points to note
Eigenvalue problems in ODE’s
Self-adjoint differential operators
Sturm-Liouville problems
Orthogonal eigenfunctions
Eigenfunction expansions
Necessary Exercises: 1,2,4,5
![Page 1172: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1172.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1172,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Outline
Fourier Series and IntegralsBasic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
![Page 1173: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1173.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1173,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
With q(x) = 0 and p(x) = r(x) = 1, periodic S-L problem:
y ′′ + λy = 0, y(−L) = y(L), y ′(−L) = y ′(L)
![Page 1174: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1174.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1174,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
With q(x) = 0 and p(x) = r(x) = 1, periodic S-L problem:
y ′′ + λy = 0, y(−L) = y(L), y ′(−L) = y ′(L)
Eigenfunctions 1, cos πxL, sin πx
L, cos 2πx
L, sin 2πx
L, · · ·
constitute an orthogonal basis for representing functions.
![Page 1175: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1175.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1175,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
With q(x) = 0 and p(x) = r(x) = 1, periodic S-L problem:
y ′′ + λy = 0, y(−L) = y(L), y ′(−L) = y ′(L)
Eigenfunctions 1, cos πxL, sin πx
L, cos 2πx
L, sin 2πx
L, · · ·
constitute an orthogonal basis for representing functions.For a periodic function f (x) of period 2L, we propose
f (x) = a0 +
∞∑
n=1
(
an cosnπx
L+ bn sin
nπx
L
)
and determine the Fourier coefficients from Euler formulae
a0 =1
2L
∫ L
−L
f (x)dx ,
am =1
L
∫ L
−L
f (x) cosmπx
Ldx and bm =
1
L
∫ L
−L
f (x) sinmπx
Ldx .
![Page 1176: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1176.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1176,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
With q(x) = 0 and p(x) = r(x) = 1, periodic S-L problem:
y ′′ + λy = 0, y(−L) = y(L), y ′(−L) = y ′(L)
Eigenfunctions 1, cos πxL, sin πx
L, cos 2πx
L, sin 2πx
L, · · ·
constitute an orthogonal basis for representing functions.For a periodic function f (x) of period 2L, we propose
f (x) = a0 +
∞∑
n=1
(
an cosnπx
L+ bn sin
nπx
L
)
and determine the Fourier coefficients from Euler formulae
a0 =1
2L
∫ L
−L
f (x)dx ,
am =1
L
∫ L
−L
f (x) cosmπx
Ldx and bm =
1
L
∫ L
−L
f (x) sinmπx
Ldx .
Question: Does the series converge?
![Page 1177: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1177.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1177,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Dirichlet’s conditions:
If f (x) and its derivative are piecewise continuous on[−L,L] and are periodic with a period 2L, then the series
converges to the mean f (x+)+f (x−)2 of one-sided limits, at
all points.
Fourier series
![Page 1178: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1178.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1178,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Dirichlet’s conditions:
If f (x) and its derivative are piecewise continuous on[−L,L] and are periodic with a period 2L, then the series
converges to the mean f (x+)+f (x−)2 of one-sided limits, at
all points.
Fourier seriesNote: The interval of integration can be [x0, x0 + 2L] for any x0.
![Page 1179: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1179.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1179,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Dirichlet’s conditions:
If f (x) and its derivative are piecewise continuous on[−L,L] and are periodic with a period 2L, then the series
converges to the mean f (x+)+f (x−)2 of one-sided limits, at
all points.
Fourier seriesNote: The interval of integration can be [x0, x0 + 2L] for any x0.
It is valid to integrate the Fourier series term by term.
The Fourier series uniformly converges to f (x) over aninterval on which f (x) is continuous. At a jump discontinuity,
convergence to f (x+)+f (x−)2 is not uniform. Mismatch peak
shifts with inclusion of more terms (Gibb’s phenomenon).
Term-by-term differentiation of the Fourier series at a pointrequires f (x) to be smooth at that point.
![Page 1180: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1180.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1180,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Multiplying the Fourier series with f (x),
f 2(x) = a0f (x) +
∞∑
n=1
[
anf (x) cosnπx
L+ bnf (x) sin
nπx
L
]
![Page 1181: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1181.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1181,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Multiplying the Fourier series with f (x),
f 2(x) = a0f (x) +
∞∑
n=1
[
anf (x) cosnπx
L+ bnf (x) sin
nπx
L
]
Parseval’s identity:
⇒ a20 +
1
2
∞∑
n=1
(a2n + b2
n) =1
2L
∫ L
−L
f 2(x)dx
The Fourier series representation is complete.
![Page 1182: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1182.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1182,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Multiplying the Fourier series with f (x),
f 2(x) = a0f (x) +
∞∑
n=1
[
anf (x) cosnπx
L+ bnf (x) sin
nπx
L
]
Parseval’s identity:
⇒ a20 +
1
2
∞∑
n=1
(a2n + b2
n) =1
2L
∫ L
−L
f 2(x)dx
The Fourier series representation is complete.
A periodic function f (x) is composed of its mean value andseveral sinusoidal components, or harmonics.
Fourier coefficients are corresponding amplitudes. Parseval’s identity is simply a statement on energy balance!
![Page 1183: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1183.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1183,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Basic Theory of Fourier Series
Multiplying the Fourier series with f (x),
f 2(x) = a0f (x) +
∞∑
n=1
[
anf (x) cosnπx
L+ bnf (x) sin
nπx
L
]
Parseval’s identity:
⇒ a20 +
1
2
∞∑
n=1
(a2n + b2
n) =1
2L
∫ L
−L
f 2(x)dx
The Fourier series representation is complete.
A periodic function f (x) is composed of its mean value andseveral sinusoidal components, or harmonics.
Fourier coefficients are corresponding amplitudes. Parseval’s identity is simply a statement on energy balance!
Bessel’s inequality
a20 +
1
2
N∑
n=1
(a2n + b2
n) ≤1
2L‖f (x)‖2
![Page 1184: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1184.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1184,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Original spirit of Fouries series representation of periodic functions over (−∞,∞).
Question: What about a function f (x) defined only on [−L,L]?
![Page 1185: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1185.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1185,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Original spirit of Fouries series representation of periodic functions over (−∞,∞).
Question: What about a function f (x) defined only on [−L,L]?Answer: Extend the function as
F (x) = f (x) for − L ≤ x ≤ L, and F (x + 2L) = F (x).
Fourier series of F (x) acts as the Fourier series representation off (x) in its own domain.
![Page 1186: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1186.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1186,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Original spirit of Fouries series representation of periodic functions over (−∞,∞).
Question: What about a function f (x) defined only on [−L,L]?Answer: Extend the function as
F (x) = f (x) for − L ≤ x ≤ L, and F (x + 2L) = F (x).
Fourier series of F (x) acts as the Fourier series representation off (x) in its own domain.In Euler formulae, notice that bm = 0 for an even function.
The Fourier series of an even function is a Fouriercosine series
f (x) = a0 +
∞∑
n=1
an cosnπx
L,
where a0 = 1L
∫ L
0 f (x)dx and an = 2L
∫ L
0 f (x) cos nπxL
dx.
![Page 1187: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1187.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1187,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Original spirit of Fouries series representation of periodic functions over (−∞,∞).
Question: What about a function f (x) defined only on [−L,L]?Answer: Extend the function as
F (x) = f (x) for − L ≤ x ≤ L, and F (x + 2L) = F (x).
Fourier series of F (x) acts as the Fourier series representation off (x) in its own domain.In Euler formulae, notice that bm = 0 for an even function.
The Fourier series of an even function is a Fouriercosine series
f (x) = a0 +
∞∑
n=1
an cosnπx
L,
where a0 = 1L
∫ L
0 f (x)dx and an = 2L
∫ L
0 f (x) cos nπxL
dx.
Similarly, for an odd function, Fourier sine series.
![Page 1188: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1188.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1188,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Over [0,L], sometimes we need a series of sine terms only, orcosine terms only!
![Page 1189: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1189.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1189,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Over [0,L], sometimes we need a series of sine terms only, orcosine terms only!
x
s
c
O−L−2L−3L L 2L 3L x
O−3L
L
f(x)
O
x
f (x)
f (x)
3L2L−2L
(b) Even periodic extension
−L L
0,L(a) Function over ( )
(c) Odd periodic extension
Figure: Periodic extensions for cosine and sine series
![Page 1190: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1190.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1190,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Half-range expansions
For Fourier cosine series of a function f (x) over [0,L], evenperiodic extension:
fc(x) =
f (x) for 0 ≤ x ≤ L,f (−x) for −L ≤ x < 0,
and fc(x+2L) = fc(x)
For Fourier sine series of a function f (x) over [0,L], oddperiodic extension:
fs(x) =
f (x) for 0 ≤ x ≤ L,−f (−x) for −L ≤ x < 0,
and fs(x+2L) = fs(x)
![Page 1191: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1191.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1191,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Half-range expansions
For Fourier cosine series of a function f (x) over [0,L], evenperiodic extension:
fc(x) =
f (x) for 0 ≤ x ≤ L,f (−x) for −L ≤ x < 0,
and fc(x+2L) = fc(x)
For Fourier sine series of a function f (x) over [0,L], oddperiodic extension:
fs(x) =
f (x) for 0 ≤ x ≤ L,−f (−x) for −L ≤ x < 0,
and fs(x+2L) = fs(x)
To develop the Fourier series of a function, which is available as aset of tabulated values or a black-box library routine,
integrals in the Euler formulae are evaluated numerically.
![Page 1192: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1192.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1192,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Extensions in Application
Half-range expansions
For Fourier cosine series of a function f (x) over [0,L], evenperiodic extension:
fc(x) =
f (x) for 0 ≤ x ≤ L,f (−x) for −L ≤ x < 0,
and fc(x+2L) = fc(x)
For Fourier sine series of a function f (x) over [0,L], oddperiodic extension:
fs(x) =
f (x) for 0 ≤ x ≤ L,−f (−x) for −L ≤ x < 0,
and fs(x+2L) = fs(x)
To develop the Fourier series of a function, which is available as aset of tabulated values or a black-box library routine,
integrals in the Euler formulae are evaluated numerically.
Important: Fourier series representation is richer and morepowerful compared to interpolatory or least square approximationin many contexts.
![Page 1193: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1193.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1193,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
Question: How to apply the idea of Fourier series to anon-periodic function over an infinite domain?
![Page 1194: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1194.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1194,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
Question: How to apply the idea of Fourier series to anon-periodic function over an infinite domain?Answer: Magnify a single period to an infinite length.
Fourier series of function fL(x) of period 2L:
fL(x) = a0 +
∞∑
n=1
(an cos pnx + bn sinpnx),
where pn = nπL
is the frequency of the n-th harmonic.
![Page 1195: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1195.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1195,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
Question: How to apply the idea of Fourier series to anon-periodic function over an infinite domain?Answer: Magnify a single period to an infinite length.
Fourier series of function fL(x) of period 2L:
fL(x) = a0 +
∞∑
n=1
(an cos pnx + bn sinpnx),
where pn = nπL
is the frequency of the n-th harmonic.
Inserting the expressions for the Fourier coefficients,
fL(x) =1
2L
∫ L
−L
fL(x)dx
+1
π
∞∑
n=1
[
cos pnx
∫ L
−L
fL(v) cos pnv dv + sin pnx
∫ L
−L
fL(v) sin pnv dv
]
∆p,
where ∆p = pn+1 − pn = πL.
![Page 1196: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1196.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1196,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
In the limit (if it exists), as L→∞, ∆p → 0,
f (x) =1
π
∫ ∞
0
[
cos px
∫ ∞
−∞f (v) cos pv dv + sinpx
∫ ∞
−∞f (v) sin pv dv
]
dp
![Page 1197: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1197.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1197,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
In the limit (if it exists), as L→∞, ∆p → 0,
f (x) =1
π
∫ ∞
0
[
cos px
∫ ∞
−∞f (v) cos pv dv + sinpx
∫ ∞
−∞f (v) sin pv dv
]
dp
Fourier integral of f (x):
f (x) =
∫ ∞
0[A(p) cos px + B(p) sin px ]dp,
where amplitude functions
A(p) =1
π
∫ ∞
−∞f (v) cos pv dv and B(p) =
1
π
∫ ∞
−∞f (v) sin pv dv
are defined for a continuous frequency variable p.
![Page 1198: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1198.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1198,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
In the limit (if it exists), as L→∞, ∆p → 0,
f (x) =1
π
∫ ∞
0
[
cos px
∫ ∞
−∞f (v) cos pv dv + sinpx
∫ ∞
−∞f (v) sin pv dv
]
dp
Fourier integral of f (x):
f (x) =
∫ ∞
0[A(p) cos px + B(p) sin px ]dp,
where amplitude functions
A(p) =1
π
∫ ∞
−∞f (v) cos pv dv and B(p) =
1
π
∫ ∞
−∞f (v) sin pv dv
are defined for a continuous frequency variable p.
In phase angle form,
f (x) =1
π
∫ ∞
0
∫ ∞
−∞f (v) cos p(x − v)dv dp.
![Page 1199: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1199.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1199,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
Using cos θ = e iθ+e−iθ
2 in the phase angle form,
f (x) =1
2π
∫ ∞
0
∫ ∞
−∞f (v)[e ip(x−v) + e−ip(x−v)]dv dp.
With substitution p = −q,
∫ ∞
0
∫ ∞
−∞f (v)e−ip(x−v)dv dp =
∫ 0
−∞
∫ ∞
−∞f (v)e iq(x−v)dv dq.
![Page 1200: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1200.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1200,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Fourier Integrals
Using cos θ = e iθ+e−iθ
2 in the phase angle form,
f (x) =1
2π
∫ ∞
0
∫ ∞
−∞f (v)[e ip(x−v) + e−ip(x−v)]dv dp.
With substitution p = −q,
∫ ∞
0
∫ ∞
−∞f (v)e−ip(x−v)dv dp =
∫ 0
−∞
∫ ∞
−∞f (v)e iq(x−v)dv dq.
Complex form of Fourier integral
f (x) =1
2π
∫ ∞
−∞
∫ ∞
−∞f (v)e ip(x−v)dv dp =
∫ ∞
−∞C (p)e ipxdp,
in which the complex Fourier integral coefficient is
C (p) =1
2π
∫ ∞
−∞f (v)e−ipvdv .
![Page 1201: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1201.jpg)
Mathematical Methods in Engineering and Science Fourier Series and Integrals 1201,
Basic Theory of Fourier SeriesExtensions in ApplicationFourier Integrals
Points to note
Fourier series arising out of a Sturm-Liouville problem
A versatile tool for function representation
Fourier integral as the limiting case of Fourier series
Necessary Exercises: 1,3,6,8
![Page 1202: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1202.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1202,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Outline
Fourier TransformsDefinition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
![Page 1203: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1203.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1203,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Complex form of the Fourier integral:
f (t) =1√2π
∫ ∞
−∞
[
1√2π
∫ ∞
−∞f (v)e−iwvdv
]
e iwtdw
Composition of an infinite number of functions in theform e iwt√
2π, over a continuous distribution of frequency w.
![Page 1204: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1204.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1204,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Complex form of the Fourier integral:
f (t) =1√2π
∫ ∞
−∞
[
1√2π
∫ ∞
−∞f (v)e−iwvdv
]
e iwtdw
Composition of an infinite number of functions in theform e iwt√
2π, over a continuous distribution of frequency w.
Fourier transform: Amplitude of a frequency component:
F(f ) ≡ f (w) =1√2π
∫ ∞
−∞f (t)e−iwtdt
Function of the frequency variable.
![Page 1205: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1205.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1205,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Complex form of the Fourier integral:
f (t) =1√2π
∫ ∞
−∞
[
1√2π
∫ ∞
−∞f (v)e−iwvdv
]
e iwtdw
Composition of an infinite number of functions in theform e iwt√
2π, over a continuous distribution of frequency w.
Fourier transform: Amplitude of a frequency component:
F(f ) ≡ f (w) =1√2π
∫ ∞
−∞f (t)e−iwtdt
Function of the frequency variable.
Inverse Fourier transform
F−1(f ) ≡ f (t) =1√2π
∫ ∞
−∞f (w)e iwtdw
recovers the original function.
![Page 1206: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1206.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1206,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Example: Fourier transform of f (t) = 1?
![Page 1207: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1207.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1207,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Example: Fourier transform of f (t) = 1?Let us find out the inverse Fourier transform of f (w) = kδ(w).
f (t) = F−1(f ) =1√2π
∫ ∞
−∞kδ(w)e iwtdw =
k√2π
F(1) =√
2πδ(w)
![Page 1208: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1208.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1208,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Example: Fourier transform of f (t) = 1?Let us find out the inverse Fourier transform of f (w) = kδ(w).
f (t) = F−1(f ) =1√2π
∫ ∞
−∞kδ(w)e iwtdw =
k√2π
F(1) =√
2πδ(w)
Linearity of Fourier transforms:
Fαf1(t) + βf2(t) = αf1(w) + β f2(w)
![Page 1209: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1209.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1209,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Example: Fourier transform of f (t) = 1?Let us find out the inverse Fourier transform of f (w) = kδ(w).
f (t) = F−1(f ) =1√2π
∫ ∞
−∞kδ(w)e iwtdw =
k√2π
F(1) =√
2πδ(w)
Linearity of Fourier transforms:
Fαf1(t) + βf2(t) = αf1(w) + β f2(w)
Scaling:
Ff (at) =1
|a| f(w
a
)
and F−1
f(w
a
)
= |a|f (at)
![Page 1210: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1210.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1210,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Definition and Fundamental Properties
Example: Fourier transform of f (t) = 1?Let us find out the inverse Fourier transform of f (w) = kδ(w).
f (t) = F−1(f ) =1√2π
∫ ∞
−∞kδ(w)e iwtdw =
k√2π
F(1) =√
2πδ(w)
Linearity of Fourier transforms:
Fαf1(t) + βf2(t) = αf1(w) + β f2(w)
Scaling:
Ff (at) =1
|a| f(w
a
)
and F−1
f(w
a
)
= |a|f (at)
Shifting rules:
Ff (t − t0) = e−iwt0Ff (t)F−1f (w − w0) = e iw0tF−1f (w)
![Page 1211: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1211.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1211,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Fourier transform of the derivative of a function:
If f (t) is continuous in every interval and f ′(t) is piecewisecontinuous,
∫∞−∞ |f (t)|dt converges and f (t) approaches zero as
t → ±∞, then
Ff ′(t) =1√2π
∫ ∞
−∞f ′(t)e−iwtdt
=1√2π
[
f (t)e−iwt]∞−∞ −
1√2π
∫ ∞
−∞(−iw)f (t)e−iwtdt
= iw f (w).
![Page 1212: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1212.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1212,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Fourier transform of the derivative of a function:
If f (t) is continuous in every interval and f ′(t) is piecewisecontinuous,
∫∞−∞ |f (t)|dt converges and f (t) approaches zero as
t → ±∞, then
Ff ′(t) =1√2π
∫ ∞
−∞f ′(t)e−iwtdt
=1√2π
[
f (t)e−iwt]∞−∞ −
1√2π
∫ ∞
−∞(−iw)f (t)e−iwtdt
= iw f (w).
Alternatively, differentiating the inverse Fourier transform,
d
dt[f (t)] =
d
dt
[
1√2π
∫ ∞
−∞f (w)e iwtdw
]
=1√2π
∫ ∞
−∞
∂
∂t
[
f (w)e iwt]
dw = F−1iw f (w).
![Page 1213: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1213.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1213,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Under appropriate premises,
Ff ′′(t) = (iw)2 f (w) = −w2f (w).
In general, Ff (n)(t) = (iw)n f (w).
![Page 1214: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1214.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1214,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Under appropriate premises,
Ff ′′(t) = (iw)2 f (w) = −w2f (w).
In general, Ff (n)(t) = (iw)n f (w).
Fourier transform of an integral:
If f (t) is piecewise continuous on every interval,∫∞−∞ |f (t)|dt converges and f (0) = 0, then
F∫ t
−∞f (τ)dτ
=1
iwf (w).
![Page 1215: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1215.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1215,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Under appropriate premises,
Ff ′′(t) = (iw)2 f (w) = −w2f (w).
In general, Ff (n)(t) = (iw)n f (w).
Fourier transform of an integral:
If f (t) is piecewise continuous on every interval,∫∞−∞ |f (t)|dt converges and f (0) = 0, then
F∫ t
−∞f (τ)dτ
=1
iwf (w).
Derivative of a Fourier transform (with respect to the frequencyvariable):
Ftnf (t) = indn
dwnf (w),
if f (t) is piecewise continuous and∫∞−∞ |tnf (t)|dt converges.
![Page 1216: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1216.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1216,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Convolution of two functions:
h(t) = f (t) ∗ g(t) =
∫ ∞
−∞f (τ)g(t − τ)dτ
![Page 1217: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1217.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1217,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Convolution of two functions:
h(t) = f (t) ∗ g(t) =
∫ ∞
−∞f (τ)g(t − τ)dτ
h(w) = Fh(t)
=1√2π
∫ ∞
−∞
∫ ∞
−∞f (τ)g(t − τ)e−iwtdτ dt
=1√2π
∫ ∞
−∞f (τ)e−iwτ
[∫ ∞
−∞g(t − τ)e−iw(t−τ)dt
]
dτ
=
∫ ∞
−∞f (τ)e−iwτ
[
1√2π
∫ ∞
−∞g(t ′)e−iwt′dt ′
]
dτ
![Page 1218: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1218.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1218,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Convolution of two functions:
h(t) = f (t) ∗ g(t) =
∫ ∞
−∞f (τ)g(t − τ)dτ
h(w) = Fh(t)
=1√2π
∫ ∞
−∞
∫ ∞
−∞f (τ)g(t − τ)e−iwtdτ dt
=1√2π
∫ ∞
−∞f (τ)e−iwτ
[∫ ∞
−∞g(t − τ)e−iw(t−τ)dt
]
dτ
=
∫ ∞
−∞f (τ)e−iwτ
[
1√2π
∫ ∞
−∞g(t ′)e−iwt′dt ′
]
dτ
Convolution theorem for Fourier transforms:
h(w) =√
2πf (w)g(w)
![Page 1219: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1219.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1219,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Conjugate of the Fourier transform:
f ∗(w) =1√2π
∫ ∞
−∞f ∗(t)e iwtdt
Inner product of f (w) and g(w):∫ ∞
−∞f ∗(w)g(w)dw =
∫ ∞
−∞
1√2π
∫ ∞
−∞f ∗(t)e iwtdt g(w)dw
=
∫ ∞
−∞f ∗(t)
[
1√2π
∫ ∞
−∞g(w)e iwtdw
]
dt
=
∫ ∞
−∞f ∗(t)g(t)dt.
![Page 1220: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1220.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1220,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Important Results on Fourier Transforms
Conjugate of the Fourier transform:
f ∗(w) =1√2π
∫ ∞
−∞f ∗(t)e iwtdt
Inner product of f (w) and g(w):∫ ∞
−∞f ∗(w)g(w)dw =
∫ ∞
−∞
1√2π
∫ ∞
−∞f ∗(t)e iwtdt g(w)dw
=
∫ ∞
−∞f ∗(t)
[
1√2π
∫ ∞
−∞g(w)e iwtdw
]
dt
=
∫ ∞
−∞f ∗(t)g(t)dt.
Parseval’s identity: For g(t) = f (t) in the above,∫ ∞
−∞‖f (w)‖2dw =
∫ ∞
−∞‖f (t)‖2dt,
equating the total energy content of the frequency spectrum of awave or a signal to the total energy flow over time.
![Page 1221: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1221.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1221,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Consider a signal f (t) from actual measurement or sampling.We want to analyze its amplitude spectrum (versus frequency).
For the FT, how to evaluate the integral over (−∞,∞)?
![Page 1222: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1222.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1222,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Consider a signal f (t) from actual measurement or sampling.We want to analyze its amplitude spectrum (versus frequency).
For the FT, how to evaluate the integral over (−∞,∞)?
Windowing: Sample the signal f (t) over a finite interval.
A window function:
g(t) =
1 for a ≤ t ≤ b0 otherwise
Actual processing takes place on the windowed function f (t)g(t).
![Page 1223: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1223.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1223,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Consider a signal f (t) from actual measurement or sampling.We want to analyze its amplitude spectrum (versus frequency).
For the FT, how to evaluate the integral over (−∞,∞)?
Windowing: Sample the signal f (t) over a finite interval.
A window function:
g(t) =
1 for a ≤ t ≤ b0 otherwise
Actual processing takes place on the windowed function f (t)g(t).
Next question: Do we need to evaluate the amplitude for allw ∈ (−∞,∞)?
![Page 1224: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1224.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1224,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Consider a signal f (t) from actual measurement or sampling.We want to analyze its amplitude spectrum (versus frequency).
For the FT, how to evaluate the integral over (−∞,∞)?
Windowing: Sample the signal f (t) over a finite interval.
A window function:
g(t) =
1 for a ≤ t ≤ b0 otherwise
Actual processing takes place on the windowed function f (t)g(t).
Next question: Do we need to evaluate the amplitude for allw ∈ (−∞,∞)?
Most useful signals are particularly rich only in their owncharacteristic frequency bands.
Decide on an expected frequency band, say [−wc ,wc ].
![Page 1225: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1225.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1225,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Time step for sampling?
![Page 1226: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1226.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1226,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Time step for sampling?
With N sampling over [a, b),
wc∆ ≤ π,
data being collected at t = a, a + ∆, a + 2∆, · · · , a + (N − 1)∆,with N∆ = b − a.
![Page 1227: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1227.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1227,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Time step for sampling?
With N sampling over [a, b),
wc∆ ≤ π,
data being collected at t = a, a + ∆, a + 2∆, · · · , a + (N − 1)∆,with N∆ = b − a.
Nyquist critical frequency
![Page 1228: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1228.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1228,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Time step for sampling?
With N sampling over [a, b),
wc∆ ≤ π,
data being collected at t = a, a + ∆, a + 2∆, · · · , a + (N − 1)∆,with N∆ = b − a.
Nyquist critical frequency
Note the duality.
Decision of sampling rate ∆ determines the band of frequencycontent that can be accommodated.
Decision of the interval [a, b) dictates how finely thefrequency spectrum can be developed.
![Page 1229: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1229.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1229,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
Time step for sampling?
With N sampling over [a, b),
wc∆ ≤ π,
data being collected at t = a, a + ∆, a + 2∆, · · · , a + (N − 1)∆,with N∆ = b − a.
Nyquist critical frequency
Note the duality.
Decision of sampling rate ∆ determines the band of frequencycontent that can be accommodated.
Decision of the interval [a, b) dictates how finely thefrequency spectrum can be developed.
Shannon’s sampling theorem
A band-limited signal can be reconstructed from a finitenumber of samples.
![Page 1230: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1230.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1230,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
![Page 1231: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1231.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1231,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
A similar discrete version of inverse Fourier transform.
![Page 1232: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1232.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1232,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
A similar discrete version of inverse Fourier transform.
Reconstruction: a trigonometric interpolation of sampled data.
Structure of Fourier and inverse Fourier transforms reduces theproblem with a system of linear equations [O(N3) operations]to that of a matrix-vector multiplication [O(N2) operations].
![Page 1233: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1233.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1233,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
A similar discrete version of inverse Fourier transform.
Reconstruction: a trigonometric interpolation of sampled data.
Structure of Fourier and inverse Fourier transforms reduces theproblem with a system of linear equations [O(N3) operations]to that of a matrix-vector multiplication [O(N2) operations].
Structure of matrix[
mkj
]
, with patterns of redundancies,
opens up a trick to reduce it further to O(N log N) operations.
![Page 1234: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1234.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1234,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
A similar discrete version of inverse Fourier transform.
Reconstruction: a trigonometric interpolation of sampled data.
Structure of Fourier and inverse Fourier transforms reduces theproblem with a system of linear equations [O(N3) operations]to that of a matrix-vector multiplication [O(N2) operations].
Structure of matrix[
mkj
]
, with patterns of redundancies,
opens up a trick to reduce it further to O(N log N) operations.
Cooley-Tuckey algorithm:
![Page 1235: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1235.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1235,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier Transform
With discrete data at tk = k∆ for k = 0, 1, 2, 3, · · · ,N − 1,
f(w) =∆√2π
[
mkj
]
f(t),
where mj = e−iwj∆ and[
mkj
]
is an N × N matrix.
A similar discrete version of inverse Fourier transform.
Reconstruction: a trigonometric interpolation of sampled data.
Structure of Fourier and inverse Fourier transforms reduces theproblem with a system of linear equations [O(N3) operations]to that of a matrix-vector multiplication [O(N2) operations].
Structure of matrix[
mkj
]
, with patterns of redundancies,
opens up a trick to reduce it further to O(N log N) operations.
Cooley-Tuckey algorithm:
fast Fourier transform (FFT)
![Page 1236: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1236.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1236,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier TransformDFT representation reliable only if the incoming signal is reallyband-limited in the interval [−wc ,wc ].
Frequencies beyond [−wc ,wc ] distort the spectrum near w = ±wc
by folding back.Aliasing
Detection: a posteriori
![Page 1237: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1237.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1237,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier TransformDFT representation reliable only if the incoming signal is reallyband-limited in the interval [−wc ,wc ].
Frequencies beyond [−wc ,wc ] distort the spectrum near w = ±wc
by folding back.Aliasing
Detection: a posteriori
Bandpass filtering: If we expect a signal having components onlyin certain frequency bands and want to get rid of unwanted noisefrequencies,
for every band [w1,w2] of our interest, we define windowfunction φ(w) with intervals [−w2,−w1] and [w1,w2].
Windowed Fourier transform φ(w)f (w) filters out frequencycomponents outside this band.
![Page 1238: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1238.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1238,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Discrete Fourier TransformDFT representation reliable only if the incoming signal is reallyband-limited in the interval [−wc ,wc ].
Frequencies beyond [−wc ,wc ] distort the spectrum near w = ±wc
by folding back.Aliasing
Detection: a posteriori
Bandpass filtering: If we expect a signal having components onlyin certain frequency bands and want to get rid of unwanted noisefrequencies,
for every band [w1,w2] of our interest, we define windowfunction φ(w) with intervals [−w2,−w1] and [w1,w2].
Windowed Fourier transform φ(w)f (w) filters out frequencycomponents outside this band.
For recovery,
convolve raw signal f (t) with IFT φ(t) of φ(w).
![Page 1239: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1239.jpg)
Mathematical Methods in Engineering and Science Fourier Transforms 1239,
Definition and Fundamental PropertiesImportant Results on Fourier TransformsDiscrete Fourier Transform
Points to note
Fourier transform as amplitude function in Fourier integral
Basic operational tools in Fourier and inverse Fouriertransforms
Conceptual notions of discrete Fourier transform (DFT)
Necessary Exercises: 1,3,6
![Page 1240: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1240.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1240,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationOutline
Minimax Approximation*Approximation with Chebyshev polynomialsMinimax Polynomial Approximation
![Page 1241: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1241.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1241,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Chebyshev polynomials:Polynomial solutions of the singular Sturm-Liouville problem
(1− x2)y ′′ − xy ′ + n2y = 0 or[√
1− x2 y ′]′
+n2
√1− x2
y = 0
over −1 ≤ x ≤ 1, with Tn(1) = 1 for all n.
![Page 1242: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1242.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1242,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Chebyshev polynomials:Polynomial solutions of the singular Sturm-Liouville problem
(1− x2)y ′′ − xy ′ + n2y = 0 or[√
1− x2 y ′]′
+n2
√1− x2
y = 0
over −1 ≤ x ≤ 1, with Tn(1) = 1 for all n.
Closed-form expressions:
Tn(x) = cos(n cos−1 x),
or,
T0(x) = 1, T1(x) = x , T2(x) = 2x2 − 1, T3(x) = 4x3 − 3x , · · · ;
with the three-term recurrence relation
Tk+1(x) = 2xTk(x) − Tk−1(x).
![Page 1243: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1243.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1243,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Immediate observations Coefficients in a Chebyshev polynomial are integers. In
particular, the leading coefficient of Tn(x) is 2n−1. For even n, Tn(x) is an even function, while for odd n it is an
odd function. Tn(1) = 1, Tn(−1) = (−1)n and |Tn(x)| ≤ 1 for −1 ≤ x ≤ 1. Zeros of a Chebyshev polynomial Tn(x) are real and lie inside
the interval [−1, 1] at locations x = cos (2k−1)π2n for
k = 1, 2, 3, · · · , n.These locations are also called Chebyshev accuracy points.Further, zeros of Tn(x) are interlaced by those of Tn+1(x).
Extrema of Tn(x) are of magnitude equal to unity, alternate insign and occur at x = cos kπ
nfor k = 0, 1, 2, 3, · · · , n.
Orthogonality and norms:
∫ 1
−1
Tm(x)Tn(x)√1− x2
dx =
0 if m 6= n,π2 if m = n 6= 0, andπ if m = n = 0.
![Page 1244: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1244.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1244,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
−1.5 −1 −0.5 0 0.5 1 1.5
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
T (
x)
extremazeroes
3
Figure: Extrema and zeros of T3(x)
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
y
P (x)T (x)
8
8
Figure: Contrast: P8(x) and T8(x)
![Page 1245: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1245.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1245,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
−1.5 −1 −0.5 0 0.5 1 1.5
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
T (
x)
extremazeroes
3
Figure: Extrema and zeros of T3(x)
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
y
P (x)T (x)
8
8
Figure: Contrast: P8(x) and T8(x)
Being cosines and polynomials at the same time, Chebyshevpolynomials possess a wide variety of interesting properties!
![Page 1246: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1246.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1246,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
−1.5 −1 −0.5 0 0.5 1 1.5
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
T (
x)
extremazeroes
3
Figure: Extrema and zeros of T3(x)
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x
y
P (x)T (x)
8
8
Figure: Contrast: P8(x) and T8(x)
Being cosines and polynomials at the same time, Chebyshevpolynomials possess a wide variety of interesting properties!
Most striking property:
equal-ripple oscillations, leading to minimax property
![Page 1247: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1247.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1247,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Minimax property
Theorem: Among all polynomials pn(x) of degree n > 0with the leading coefficient equal to unity, 21−nTn(x)deviates least from zero in [−1, 1]. That is,
max−1≤x≤1
|pn(x)| ≥ max−1≤x≤1
|21−nTn(x)| = 21−n.
![Page 1248: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1248.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1248,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Minimax property
Theorem: Among all polynomials pn(x) of degree n > 0with the leading coefficient equal to unity, 21−nTn(x)deviates least from zero in [−1, 1]. That is,
max−1≤x≤1
|pn(x)| ≥ max−1≤x≤1
|21−nTn(x)| = 21−n.
If there exists a monic polynomial pn(x) of degree n such that
max−1≤x≤1
|pn(x)| < 21−n,
then at (n + 1) locations of alternating extrema of 21−nTn(x), thepolynomial
qn(x) = 21−nTn(x)− pn(x)
will have the same sign as 21−nTn(x).
![Page 1249: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1249.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1249,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Minimax property
Theorem: Among all polynomials pn(x) of degree n > 0with the leading coefficient equal to unity, 21−nTn(x)deviates least from zero in [−1, 1]. That is,
max−1≤x≤1
|pn(x)| ≥ max−1≤x≤1
|21−nTn(x)| = 21−n.
If there exists a monic polynomial pn(x) of degree n such that
max−1≤x≤1
|pn(x)| < 21−n,
then at (n + 1) locations of alternating extrema of 21−nTn(x), thepolynomial
qn(x) = 21−nTn(x)− pn(x)
will have the same sign as 21−nTn(x).With alternating signs at (n + 1) locations in sequence, qn(x) willhave n intervening zeros, even though it is a polynomial of degreeat most (n − 1): CONTRADICTION!
![Page 1250: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1250.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1250,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Chebyshev series
f (x) = a0T0(x) + a1T1(x) + a2T2(x) + a3T3(x) + · · ·
with coefficients
a0 =1
π
∫ 1
−1
f (x)T0(x)√1− x2
dx and an =2
π
∫ 1
−1
f (x)Tn(x)√1− x2
dx for n = 1, 2, 3, · ·
![Page 1251: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1251.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1251,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Chebyshev series
f (x) = a0T0(x) + a1T1(x) + a2T2(x) + a3T3(x) + · · ·
with coefficients
a0 =1
π
∫ 1
−1
f (x)T0(x)√1− x2
dx and an =2
π
∫ 1
−1
f (x)Tn(x)√1− x2
dx for n = 1, 2, 3, · ·
A truncated series∑n
k=0 akTk(x):
Chebyshev economization
Leading error term an+1Tn+1(x) deviates least from zero over[−1, 1] and is qualitatively similar to the error function.
![Page 1252: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1252.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1252,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
Chebyshev series
f (x) = a0T0(x) + a1T1(x) + a2T2(x) + a3T3(x) + · · ·
with coefficients
a0 =1
π
∫ 1
−1
f (x)T0(x)√1− x2
dx and an =2
π
∫ 1
−1
f (x)Tn(x)√1− x2
dx for n = 1, 2, 3, · ·
A truncated series∑n
k=0 akTk(x):
Chebyshev economization
Leading error term an+1Tn+1(x) deviates least from zero over[−1, 1] and is qualitatively similar to the error function.
Question: How to develop a Chebyshev series approximation?Find out so many Chebyshev polynomials and evaluate coefficients?
![Page 1253: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1253.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1253,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
For approximating f (t) over [a, b], scale the variable ast = a+b
2 + b−a2 x , with x ∈ [−1, 1].
Remark: The economized series∑n
k=0 akTk(x) gives minimaxdeviation of the leading error term an+1Tn+1(x).
![Page 1254: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1254.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1254,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
For approximating f (t) over [a, b], scale the variable ast = a+b
2 + b−a2 x , with x ∈ [−1, 1].
Remark: The economized series∑n
k=0 akTk(x) gives minimaxdeviation of the leading error term an+1Tn+1(x).
Assuming an+1Tn+1(x) to be the error, at the zeros of Tn+1(x),the error will be ‘officially’ zero, i.e.
n∑
k=0
akTk(xj ) = f (t(xj)),
where x0, x1, x2, · · · , xn are the roots of Tn+1(x).
![Page 1255: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1255.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1255,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationApproximation with Chebyshev polynomials
For approximating f (t) over [a, b], scale the variable ast = a+b
2 + b−a2 x , with x ∈ [−1, 1].
Remark: The economized series∑n
k=0 akTk(x) gives minimaxdeviation of the leading error term an+1Tn+1(x).
Assuming an+1Tn+1(x) to be the error, at the zeros of Tn+1(x),the error will be ‘officially’ zero, i.e.
n∑
k=0
akTk(xj ) = f (t(xj)),
where x0, x1, x2, · · · , xn are the roots of Tn+1(x).
Recall: Values of an n-th degree polynomial at n + 1points uniquely fix the entire polynomial.
Interpolation of these n + 1 values leads to the same polynomial!
Chebyshev-Lagrange approximation
![Page 1256: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1256.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1256,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationMinimax Polynomial Approximation
Situations in which minimax approximation is desirable: Develop the approximation once and keep it for use in future.
Requirement: Uniform quality control over the entire domain
Minimax approximation:
deviation limited by the constant amplitude of ripple
![Page 1257: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1257.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1257,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationMinimax Polynomial Approximation
Situations in which minimax approximation is desirable: Develop the approximation once and keep it for use in future.
Requirement: Uniform quality control over the entire domain
Minimax approximation:
deviation limited by the constant amplitude of ripple
Chebyshev’s minimax theorem
Theorem: Of all polynomials of degree up to n, p(x) isthe minimax polynomial approximation of f (x), i.e. itminimizes
max |f (x)− p(x)|,if and only if there are n + 2 points xi such that
a ≤ x1 < x2 < x3 < · · · < xn+2 ≤ b,
where the difference f (x)− p(x) takes its extreme valuesof the same magnitude and alternating signs.
![Page 1258: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1258.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1258,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationMinimax Polynomial Approximation
Utilize any gap to reduce the deviation at the other extrema withvalues at the bound.
p(x)∆
y
ε
−ε/2
/2
ε
nm wb
x
f(x) − p(x)
laO
d
−d
Figure: Schematic of an approximation that is not minimax
Construction of the minimax polynomial: Remez algorithm
![Page 1259: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1259.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1259,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationMinimax Polynomial Approximation
Utilize any gap to reduce the deviation at the other extrema withvalues at the bound.
p(x)∆
y
ε
−ε/2
/2
ε
nm wb
x
f(x) − p(x)
laO
d
−d
Figure: Schematic of an approximation that is not minimax
Construction of the minimax polynomial: Remez algorithm
Note: In the light of this theorem and algorithm, examine howTn+1(x) is qualitatively similar to the complete error function!
![Page 1260: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1260.jpg)
Mathematical Methods in Engineering and Science Minimax Approximation* 1260,
Approximation with Chebyshev polynomialsMinimax Polynomial ApproximationPoints to note
Unique features of Chebyshev polynomials
The equal-ripple and minimax properties
Chebyshev series and Chebyshev-Lagrange approximation
Fundamental ideas of general minimax approximation
Necessary Exercises: 2,3,4
![Page 1261: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1261.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1261,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Outline
Partial Differential EquationsIntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
![Page 1262: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1262.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1262,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionQuasi-linear second order PDE’s
a∂2u
∂x2+ 2b
∂2u
∂x∂y+ c
∂2u
∂y2= F (x , y , u, ux , uy )
hyperbolic if b2 − ac > 0, modelling phenomena which evolve intime perpetually and do not approach a steady state
parabolic if b2 − ac = 0, modelling phenomena which evolve intime in a transient manner, approaching steady state
elliptic if b2 − ac < 0, modelling steady-state configurations,without evolution in time
![Page 1263: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1263.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1263,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionQuasi-linear second order PDE’s
a∂2u
∂x2+ 2b
∂2u
∂x∂y+ c
∂2u
∂y2= F (x , y , u, ux , uy )
hyperbolic if b2 − ac > 0, modelling phenomena which evolve intime perpetually and do not approach a steady state
parabolic if b2 − ac = 0, modelling phenomena which evolve intime in a transient manner, approaching steady state
elliptic if b2 − ac < 0, modelling steady-state configurations,without evolution in time
If F (x , y , u, ux , uy ) = 0,
second order linear homogeneous differential equation
Principle of superposition: A linear combination of differentsolutions is also a solution.
![Page 1264: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1264.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1264,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionQuasi-linear second order PDE’s
a∂2u
∂x2+ 2b
∂2u
∂x∂y+ c
∂2u
∂y2= F (x , y , u, ux , uy )
hyperbolic if b2 − ac > 0, modelling phenomena which evolve intime perpetually and do not approach a steady state
parabolic if b2 − ac = 0, modelling phenomena which evolve intime in a transient manner, approaching steady state
elliptic if b2 − ac < 0, modelling steady-state configurations,without evolution in time
If F (x , y , u, ux , uy ) = 0,
second order linear homogeneous differential equation
Principle of superposition: A linear combination of differentsolutions is also a solution.Solutions are often in the form of infinite series.
Solution techniques in PDE’s typically attack the boundaryvalue problem directly.
![Page 1265: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1265.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1265,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Introduction
Initial and boundary conditionsTime and space variables are qualitatively different.
![Page 1266: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1266.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1266,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Introduction
Initial and boundary conditionsTime and space variables are qualitatively different.
Conditions in time: typically initial conditions.For second order PDE’s, u and ut over the entire spacedomain: Cauchy conditions
Time is a single variable and is decoupled from the spacevariables.
![Page 1267: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1267.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1267,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Introduction
Initial and boundary conditionsTime and space variables are qualitatively different.
Conditions in time: typically initial conditions.For second order PDE’s, u and ut over the entire spacedomain: Cauchy conditions
Time is a single variable and is decoupled from the spacevariables.
Conditions in space: typically boundary conditions.For u(t, x , y), boundary conditions over the entire curve in thex-y plane that encloses the domain. For second order PDE’s,
Dirichlet condition: value of the function Neumann condition: derivative normal to the boundary Mixed (Robin) condition
![Page 1268: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1268.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1268,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Introduction
Initial and boundary conditionsTime and space variables are qualitatively different.
Conditions in time: typically initial conditions.For second order PDE’s, u and ut over the entire spacedomain: Cauchy conditions
Time is a single variable and is decoupled from the spacevariables.
Conditions in space: typically boundary conditions.For u(t, x , y), boundary conditions over the entire curve in thex-y plane that encloses the domain. For second order PDE’s,
Dirichlet condition: value of the function Neumann condition: derivative normal to the boundary Mixed (Robin) condition
Dirichlet, Neumann and Cauchy problems
![Page 1269: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1269.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1269,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionMethod of separation of variablesFor u(x , y), propose a solution in the form
u(x , y) = X (x)Y (y)
and substitute
ux = X ′Y , uy = XY ′, uxx = X ′′Y , uxy = X ′Y ′, uyy = XY ′′
to cast the equation into the form
φ(x ,X ,X ′,X ′′) = ψ(y ,Y ,Y ′,Y ′′).
![Page 1270: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1270.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1270,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionMethod of separation of variablesFor u(x , y), propose a solution in the form
u(x , y) = X (x)Y (y)
and substitute
ux = X ′Y , uy = XY ′, uxx = X ′′Y , uxy = X ′Y ′, uyy = XY ′′
to cast the equation into the form
φ(x ,X ,X ′,X ′′) = ψ(y ,Y ,Y ′,Y ′′).
If the manoeuvre succeeds then, x and y being independentvariables, it implies
φ(x ,X ,X ′,X ′′) = ψ(y ,Y ,Y ′,Y ′′) = k.
![Page 1271: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1271.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1271,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
IntroductionMethod of separation of variablesFor u(x , y), propose a solution in the form
u(x , y) = X (x)Y (y)
and substitute
ux = X ′Y , uy = XY ′, uxx = X ′′Y , uxy = X ′Y ′, uyy = XY ′′
to cast the equation into the form
φ(x ,X ,X ′,X ′′) = ψ(y ,Y ,Y ′,Y ′′).
If the manoeuvre succeeds then, x and y being independentvariables, it implies
φ(x ,X ,X ′,X ′′) = ψ(y ,Y ,Y ′,Y ′′) = k.
Nature of the separation constant k is decided based on thecontext, resulting ODE’s are solved in consistency with theboundary conditions and assembled to construct u(x , y).
![Page 1272: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1272.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1272,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Transverse vibrations of a string
δx
δxP
Q
θ+δθ
Tθ
T
xLO
u
QP
Figure: Transverse vibration of a stretched string
Small deflection and slope: cos θ ≈ 1, sin θ ≈ θ ≈ tan θ
![Page 1273: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1273.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1273,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Transverse vibrations of a string
δx
δxP
Q
θ+δθ
Tθ
T
xLO
u
QP
Figure: Transverse vibration of a stretched string
Small deflection and slope: cos θ ≈ 1, sin θ ≈ θ ≈ tan θ
Horizontal (longitudinal) forces on PQ balance.From Newton’s second law, vertical (transverse) deflection u(x , t):
T sin(θ + δθ)− T sin θ = ρδx∂2u
∂t2
![Page 1274: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1274.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1274,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Under the assumptions, denoting c2 = Tρ ,
δx∂2u
∂t2= c2
[
∂u
∂x
∣
∣
∣
∣
Q
− ∂u
∂x
∣
∣
∣
∣
P
]
.
In the limit, as δx → 0, PDE of transverse vibration:
∂2u
∂t2= c2∂
2u
∂x2
one-dimensional wave equation
![Page 1275: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1275.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1275,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Under the assumptions, denoting c2 = Tρ ,
δx∂2u
∂t2= c2
[
∂u
∂x
∣
∣
∣
∣
Q
− ∂u
∂x
∣
∣
∣
∣
P
]
.
In the limit, as δx → 0, PDE of transverse vibration:
∂2u
∂t2= c2∂
2u
∂x2
one-dimensional wave equation
Boundary conditions (in this case): u(0, t) = u(L, t) = 0
Initial configuration and initial velocity:
u(x , 0) = f (x) and ut(x , 0) = g(x)
Cauchy problem: Determine u(x , t) for 0 ≤ x ≤ L, t ≥ 0.
![Page 1276: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1276.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1276,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Solution by separation of variables
utt = c2uxx , u(0, t) = u(L, t) = 0, u(x , 0) = f (x), ut(x , 0) = g(x)
Assumingu(x , t) = X (x)T (t),
and substituting utt = XT ′′ and uxx = X ′′T , variables areseparated as
T ′′
c2T=
X ′′
X= −p2.
The PDE splits into two ODE’s
X ′′ + p2X = 0 and T ′′ + c2p2T = 0.
![Page 1277: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1277.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1277,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Solution by separation of variables
utt = c2uxx , u(0, t) = u(L, t) = 0, u(x , 0) = f (x), ut(x , 0) = g(x)
Assumingu(x , t) = X (x)T (t),
and substituting utt = XT ′′ and uxx = X ′′T , variables areseparated as
T ′′
c2T=
X ′′
X= −p2.
The PDE splits into two ODE’s
X ′′ + p2X = 0 and T ′′ + c2p2T = 0.
Eigenvalues of BVP X ′′ + p2X = 0, X (0) = X (L) = 0 are p = nπL
and eigenfunctions
Xn(x) = sin px = sinnπx
Lfor n = 1, 2, 3, · · · .
Second ODE: T ′′ + λ2nT = 0, with λn = cnπ
L
![Page 1278: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1278.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1278,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Corresponding solution:
Tn(t) = An cos λnt + Bn sinλnt
![Page 1279: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1279.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1279,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Corresponding solution:
Tn(t) = An cos λnt + Bn sinλnt
Then, for n = 1, 2, 3, · · · ,
un(x , t) = Xn(x)Tn(t) = (An cos λnt + Bn sinλnt) sinnπx
L
satisfies the PDE and the boundary conditions.
![Page 1280: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1280.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1280,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Corresponding solution:
Tn(t) = An cos λnt + Bn sinλnt
Then, for n = 1, 2, 3, · · · ,
un(x , t) = Xn(x)Tn(t) = (An cos λnt + Bn sinλnt) sinnπx
L
satisfies the PDE and the boundary conditions.
Since the PDE and the BC’s are homogeneous, by superposition,
u(x , t) =
∞∑
n=1
[An cos λnt + Bn sinλnt] sinnπx
L.
Question: How to determine coefficients An and Bn?
![Page 1281: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1281.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1281,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Corresponding solution:
Tn(t) = An cos λnt + Bn sinλnt
Then, for n = 1, 2, 3, · · · ,
un(x , t) = Xn(x)Tn(t) = (An cos λnt + Bn sinλnt) sinnπx
L
satisfies the PDE and the boundary conditions.
Since the PDE and the BC’s are homogeneous, by superposition,
u(x , t) =
∞∑
n=1
[An cos λnt + Bn sinλnt] sinnπx
L.
Question: How to determine coefficients An and Bn?
Answer: By imposing the initial conditions.
![Page 1282: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1282.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1282,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Initial conditions: Fourier sine series of f (x) and g(x)
u(x , 0) = f (x) =
∞∑
n=1
An sinnπx
L
ut(x , 0) = g(x) =
∞∑
n=1
λnBn sinnπx
L
![Page 1283: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1283.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1283,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Initial conditions: Fourier sine series of f (x) and g(x)
u(x , 0) = f (x) =
∞∑
n=1
An sinnπx
L
ut(x , 0) = g(x) =
∞∑
n=1
λnBn sinnπx
L
Hence, coefficients:
An =2
L
∫ L
0f (x) sin
nπx
Ldx and Bn =
2
cnπ
∫ L
0g(x) sin
nπx
Ldx
![Page 1284: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1284.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1284,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Initial conditions: Fourier sine series of f (x) and g(x)
u(x , 0) = f (x) =
∞∑
n=1
An sinnπx
L
ut(x , 0) = g(x) =
∞∑
n=1
λnBn sinnπx
L
Hence, coefficients:
An =2
L
∫ L
0f (x) sin
nπx
Ldx and Bn =
2
cnπ
∫ L
0g(x) sin
nπx
Ldx
Related problems:
Different boundary conditions: other kinds of series Long wire: infinite domain, continuous frequencies and
solution from Fourier integralsAlternative: Reduce the problem using Fourier transforms.
General wave equation in 3-d: utt = c2∇2u Membrane equation: utt = c2(uxx + uyy)
![Page 1285: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1285.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1285,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
D’Alembert’s solution of the wave equation
Method of characteristicsCanonical form
By coordinate transformation from (x , y) to (ξ, η), withU(ξ, η) = u[x(ξ, η), y(ξ, η)],
hyperbolic equation: Uξη = Φ
parabolic equation: Uξξ = Φ
elliptic equation: Uξξ + Uηη = Φ
in which Φ(ξ, η,U,Uξ,Uη) is free from second derivatives.
![Page 1286: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1286.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1286,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
D’Alembert’s solution of the wave equation
Method of characteristicsCanonical form
By coordinate transformation from (x , y) to (ξ, η), withU(ξ, η) = u[x(ξ, η), y(ξ, η)],
hyperbolic equation: Uξη = Φ
parabolic equation: Uξξ = Φ
elliptic equation: Uξξ + Uηη = Φ
in which Φ(ξ, η,U,Uξ,Uη) is free from second derivatives.
For a hyperbolic equation, entire domain becomes a network of ξ-ηcoordinate curves, known as characteristic curves,
along which decoupled solutions can be tracked!
![Page 1287: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1287.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1287,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
For a hyperbolic equation in the form
a∂2u
∂x2+ 2b
∂2u
∂x∂y+ c
∂2u
∂y2= F (x , y , u, ux , uy ),
roots of am2 + 2bm + c are
m1,2 =−b ±
√b2 − ac
a,
real and distinct.Coordinate transformation
ξ = y + m1x , η = y + m2x
leads to Uξη = Φ(ξ, η,U,Uξ ,Uη).
![Page 1288: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1288.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1288,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
For a hyperbolic equation in the form
a∂2u
∂x2+ 2b
∂2u
∂x∂y+ c
∂2u
∂y2= F (x , y , u, ux , uy ),
roots of am2 + 2bm + c are
m1,2 =−b ±
√b2 − ac
a,
real and distinct.Coordinate transformation
ξ = y + m1x , η = y + m2x
leads to Uξη = Φ(ξ, η,U,Uξ ,Uη).For the BVP
utt = c2uxx , u(0, t) = u(L, t) = 0, u(x , 0) = f (x), ut(x , 0) = g(x),
canonical coordinate transformation:
ξ = x − ct, η = x + ct, with x =1
2(ξ + η), t =
1
2c(η − ξ).
![Page 1289: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1289.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1289,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Substitution of derivatives
ux = Uξξx + Uηηx = Uξ + Uη ⇒ uxx = Uξξ + 2Uξη + Uηη
ut = Uξξt + Uηηt = −cUξ + cUη ⇒ utt = c2Uξξ − 2c2Uξη + c2Uηη
into the PDE utt = c2uxx gives
c2(Uξξ − 2Uξη + Uηη) = c2(Uξξ + 2Uξη + Uηη).
Canonical form: Uξη = 0
![Page 1290: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1290.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1290,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Substitution of derivatives
ux = Uξξx + Uηηx = Uξ + Uη ⇒ uxx = Uξξ + 2Uξη + Uηη
ut = Uξξt + Uηηt = −cUξ + cUη ⇒ utt = c2Uξξ − 2c2Uξη + c2Uηη
into the PDE utt = c2uxx gives
c2(Uξξ − 2Uξη + Uηη) = c2(Uξξ + 2Uξη + Uηη).
Canonical form: Uξη = 0
Integration:
Uξ =
∫
Uξηdη + ψ(ξ) = ψ(ξ)
⇒ U(ξ, η) =
∫
ψ(ξ)dξ + f2(η) = f1(ξ) + f2(η)
![Page 1291: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1291.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1291,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Substitution of derivatives
ux = Uξξx + Uηηx = Uξ + Uη ⇒ uxx = Uξξ + 2Uξη + Uηη
ut = Uξξt + Uηηt = −cUξ + cUη ⇒ utt = c2Uξξ − 2c2Uξη + c2Uηη
into the PDE utt = c2uxx gives
c2(Uξξ − 2Uξη + Uηη) = c2(Uξξ + 2Uξη + Uηη).
Canonical form: Uξη = 0
Integration:
Uξ =
∫
Uξηdη + ψ(ξ) = ψ(ξ)
⇒ U(ξ, η) =
∫
ψ(ξ)dξ + f2(η) = f1(ξ) + f2(η)
D’Alembert’s solution: u(x , t) = f1(x − ct) + f2(x + ct)
![Page 1292: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1292.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1292,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Physical insight from D’Alembert’s solution:
f1(x − ct): a progressive wave in forward direction with speed c
Reflection at boundary:
in a manner depending upon the boundary condition
Reflected wave f2(x + ct): another progressive wave, this one inbackward direction with speed c
![Page 1293: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1293.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1293,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Physical insight from D’Alembert’s solution:
f1(x − ct): a progressive wave in forward direction with speed c
Reflection at boundary:
in a manner depending upon the boundary condition
Reflected wave f2(x + ct): another progressive wave, this one inbackward direction with speed c
Superposition of two waves: complete solution (response)
![Page 1294: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1294.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1294,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Hyperbolic Equations
Physical insight from D’Alembert’s solution:
f1(x − ct): a progressive wave in forward direction with speed c
Reflection at boundary:
in a manner depending upon the boundary condition
Reflected wave f2(x + ct): another progressive wave, this one inbackward direction with speed c
Superposition of two waves: complete solution (response)
Note: Components of the earlier solution: with λn = cnπL
,
cosλnt sinnπx
L=
1
2
[
sinnπ
L(x − ct) + sin
nπ
L(x + ct)
]
sinλnt sinnπx
L=
1
2
[
cosnπ
L(x − ct)− cos
nπ
L(x + ct)
]
![Page 1295: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1295.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1295,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Heat conduction equation or diffusion equation:
∂u
∂t= c2∇2u
One-dimensional heat (diffusion) equation:
ut = c2uxx
Heat conduction in a finite bar: For a thin bar of length L withend-points at zero temperature,
ut = c2uxx , u(0, t) = u(L, t) = 0, u(x , 0) = f (x).
![Page 1296: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1296.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1296,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Heat conduction equation or diffusion equation:
∂u
∂t= c2∇2u
One-dimensional heat (diffusion) equation:
ut = c2uxx
Heat conduction in a finite bar: For a thin bar of length L withend-points at zero temperature,
ut = c2uxx , u(0, t) = u(L, t) = 0, u(x , 0) = f (x).
Assumption u(x , t) = X (x)T (t) leads to
XT ′ = c2X ′′T ⇒ T ′
c2T=
X ′′
X= −p2,
giving rise to two ODE’s as
X ′′ + p2X = 0 and T ′ + c2p2T = 0.
![Page 1297: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1297.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1297,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
BVP in the space coordinate X ′′ + p2X = 0, X (0) = X (L) = 0has solutions
Xn(x) = sinnπx
L.
![Page 1298: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1298.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1298,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
BVP in the space coordinate X ′′ + p2X = 0, X (0) = X (L) = 0has solutions
Xn(x) = sinnπx
L.
With λn = cnπL
, the ODE in T (t) has the corresponding solutions
Tn(t) = Ane−λ2
nt .
By superposition,
u(x , t) =
∞∑
n=1
An sinnπx
Le−λ2
nt ,
coefficients being determined from initial condition as
u(x , 0) = f (x) =∞∑
n=1
An sinnπx
L,
a Fourier sine series.As t →∞, u(x , t)→ 0 (steady state)
![Page 1299: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1299.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1299,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Non-homogeneous boundary conditions:
ut = c2uxx , u(0, t) = u1, u(L, t) = u2, u(x , 0) = f (x).
For u1 6= u2, with u(x , t) = X (x)T (t), BC’s do not separate!
![Page 1300: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1300.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1300,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Non-homogeneous boundary conditions:
ut = c2uxx , u(0, t) = u1, u(L, t) = u2, u(x , 0) = f (x).
For u1 6= u2, with u(x , t) = X (x)T (t), BC’s do not separate!Assume
u(x , t) = U(x , t) + uss(x),
where component uss(x), steady-state temperature (distribution),does not enter the differential equation.
u′′ss(x) = 0, uss(0) = u1, uss(L) = u2 ⇒ uss(x) = u1 +
u2 − u1
Lx
![Page 1301: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1301.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1301,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Non-homogeneous boundary conditions:
ut = c2uxx , u(0, t) = u1, u(L, t) = u2, u(x , 0) = f (x).
For u1 6= u2, with u(x , t) = X (x)T (t), BC’s do not separate!Assume
u(x , t) = U(x , t) + uss(x),
where component uss(x), steady-state temperature (distribution),does not enter the differential equation.
u′′ss(x) = 0, uss(0) = u1, uss(L) = u2 ⇒ uss(x) = u1 +
u2 − u1
Lx
Substituting into the BVP,
Ut = c2Uxx , U(0, t) = U(L, t) = 0, U(x , 0) = f (x) − uss(x).
Final solution:
u(x , t) =
∞∑
n=1
Bn sinnπx
Le−λ2
nt + uss(x),
Bn being coefficients of Fourier sine series of f (x)− uss(x).
![Page 1302: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1302.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1302,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Heat conduction in an infinite wire
ut = c2uxx , u(x , 0) = f (x)
![Page 1303: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1303.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1303,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Heat conduction in an infinite wire
ut = c2uxx , u(x , 0) = f (x)
In place of nπL
, now we have continuous frequency p.
Solution as superposition of all frequencies:
u(x , t) =
∫ ∞
0up(x , t)dp =
∫ ∞
0[A(p) cos px+B(p) sin px ]e−c2p2tdp
Initial condition
u(x , 0) = f (x) =
∫ ∞
0[A(p) cos px + B(p) sin px ]dp
gives the Fourier integral of f (x) and amplitude functions
A(p) =1
π
∫ ∞
−∞f (v) cos pv dv and B(p) =
1
π
∫ ∞
−∞f (v) sin pv dv .
![Page 1304: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1304.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1304,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Solution using Fourier transforms
ut = c2uxx , u(x , 0) = f (x)
![Page 1305: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1305.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1305,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Solution using Fourier transforms
ut = c2uxx , u(x , 0) = f (x)
Using derivative formula of Fourier transforms,
F(ut) = c2(iw)2F(u) ⇒ ∂u
∂t= −c2w2u,
since variables x and t are independent.Initial value problem in u(w , t):
∂u
∂t= −c2w2u, u(0) = f (w)
Solution: u(w , t) = f (w)e−c2w2t
![Page 1306: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1306.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1306,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Parabolic Equations
Solution using Fourier transforms
ut = c2uxx , u(x , 0) = f (x)
Using derivative formula of Fourier transforms,
F(ut) = c2(iw)2F(u) ⇒ ∂u
∂t= −c2w2u,
since variables x and t are independent.Initial value problem in u(w , t):
∂u
∂t= −c2w2u, u(0) = f (w)
Solution: u(w , t) = f (w)e−c2w2t
Inverse Fourier transform gives solution of the original problem as
u(x , t) = F−1u(w , t) =1√2π
∫ ∞
−∞f (w)e−c2w2te iwxdw
⇒ u(x , t) =1
π
∫ ∞
−∞f (v)
∫ ∞
0cos(wx − wv)e−c2w2tdw dv .
![Page 1307: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1307.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1307,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Heat flow in a plate: two-dimensional heat equation
∂u
∂t= c2
(
∂2u
∂x2+∂2u
∂y2
)
Steady-state temperature distribution:
∂2u
∂x2+∂2u
∂y2= 0
Laplace’s equation
![Page 1308: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1308.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1308,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Heat flow in a plate: two-dimensional heat equation
∂u
∂t= c2
(
∂2u
∂x2+∂2u
∂y2
)
Steady-state temperature distribution:
∂2u
∂x2+∂2u
∂y2= 0
Laplace’s equationSteady-state heat flow in a rectangular plate:
uxx + uyy = 0, u(0, y) = u(a, y) = u(x , 0) = 0, u(x , b) = f (x);
a Dirichlet problem over the domain 0 ≤ x ≤ a, 0 ≤ y ≤ b.
![Page 1309: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1309.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1309,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Heat flow in a plate: two-dimensional heat equation
∂u
∂t= c2
(
∂2u
∂x2+∂2u
∂y2
)
Steady-state temperature distribution:
∂2u
∂x2+∂2u
∂y2= 0
Laplace’s equationSteady-state heat flow in a rectangular plate:
uxx + uyy = 0, u(0, y) = u(a, y) = u(x , 0) = 0, u(x , b) = f (x);
a Dirichlet problem over the domain 0 ≤ x ≤ a, 0 ≤ y ≤ b.Proposal u(x , y) = X (x)Y (y) leads to
X ′′Y + XY ′′ = 0 ⇒ X ′′
X= −Y ′′
Y= −p2.
![Page 1310: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1310.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1310,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Heat flow in a plate: two-dimensional heat equation
∂u
∂t= c2
(
∂2u
∂x2+∂2u
∂y2
)
Steady-state temperature distribution:
∂2u
∂x2+∂2u
∂y2= 0
Laplace’s equationSteady-state heat flow in a rectangular plate:
uxx + uyy = 0, u(0, y) = u(a, y) = u(x , 0) = 0, u(x , b) = f (x);
a Dirichlet problem over the domain 0 ≤ x ≤ a, 0 ≤ y ≤ b.Proposal u(x , y) = X (x)Y (y) leads to
X ′′Y + XY ′′ = 0 ⇒ X ′′
X= −Y ′′
Y= −p2.
Separated ODE’s:
X ′′ + p2X = 0 and Y ′′ − p2Y = 0
![Page 1311: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1311.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1311,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
From BVP X ′′ + p2X = 0, X (0) = X (a) = 0, Xn(x) = sin nπxa
Corresponding solution of Y ′′ − p2Y = 0:
Yn(y) = An coshnπy
a+ Bn sinh
nπy
a
![Page 1312: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1312.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1312,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
From BVP X ′′ + p2X = 0, X (0) = X (a) = 0, Xn(x) = sin nπxa
Corresponding solution of Y ′′ − p2Y = 0:
Yn(y) = An coshnπy
a+ Bn sinh
nπy
a
Condition Y (0) = 0 ⇒ An = 0, and
un(x , y) = Bn sinnπx
asinh
nπy
a
The complete solution:
u(x , y) =
∞∑
n=1
Bn sinnπx
asinh
nπy
a
![Page 1313: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1313.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1313,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
From BVP X ′′ + p2X = 0, X (0) = X (a) = 0, Xn(x) = sin nπxa
Corresponding solution of Y ′′ − p2Y = 0:
Yn(y) = An coshnπy
a+ Bn sinh
nπy
a
Condition Y (0) = 0 ⇒ An = 0, and
un(x , y) = Bn sinnπx
asinh
nπy
a
The complete solution:
u(x , y) =
∞∑
n=1
Bn sinnπx
asinh
nπy
a
The last boundary condition u(x , b) = f (x) fixes the coefficientsfrom the Fourier sine series of f (x).
![Page 1314: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1314.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1314,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
From BVP X ′′ + p2X = 0, X (0) = X (a) = 0, Xn(x) = sin nπxa
Corresponding solution of Y ′′ − p2Y = 0:
Yn(y) = An coshnπy
a+ Bn sinh
nπy
a
Condition Y (0) = 0 ⇒ An = 0, and
un(x , y) = Bn sinnπx
asinh
nπy
a
The complete solution:
u(x , y) =
∞∑
n=1
Bn sinnπx
asinh
nπy
a
The last boundary condition u(x , b) = f (x) fixes the coefficientsfrom the Fourier sine series of f (x).
Note: In the example, BC’s on three sides were homogeneous.How did it help? What if there are more non-homogeneous BC’s?
![Page 1315: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1315.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1315,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Steady-state heat flow with internal heat generation
∇2u = φ(x , y)
Poisson’s equation
Separation of variables impossible!
![Page 1316: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1316.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1316,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Elliptic Equations
Steady-state heat flow with internal heat generation
∇2u = φ(x , y)
Poisson’s equation
Separation of variables impossible!
Consider function u(x , y) as
u(x , y) = uh(x , y) + up(x , y)
Sequence of steps
one particular solution up(x , y) that may or may not satisfysome or all of the boundary conditions
solution of the corresponding homogeneous equation, namelyuxx + uyy = 0 for uh(x , y)
such that u = uh + up satisfies all the boundary conditions
![Page 1317: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1317.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1317,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Transverse vibration of a rectangular membrane:
∂2u
∂t2= c2
(
∂2u
∂x2+∂2u
∂y2
)
A Cauchy problem of the membrane:
utt = c2(uxx + uyy); u(x , y , 0) = f (x , y), ut(x , y , 0) = g(x , y);u(0, y , t) = u(a, y , t) = u(x , 0, t) = u(x , b, t) = 0.
![Page 1318: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1318.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1318,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Transverse vibration of a rectangular membrane:
∂2u
∂t2= c2
(
∂2u
∂x2+∂2u
∂y2
)
A Cauchy problem of the membrane:
utt = c2(uxx + uyy); u(x , y , 0) = f (x , y), ut(x , y , 0) = g(x , y);u(0, y , t) = u(a, y , t) = u(x , 0, t) = u(x , b, t) = 0.
Separate the time variable from the space variables:
u(x , y , t) = F (x , y)T (t) ⇒ Fxx + Fyy
F=
T ′′
c2T= −λ2
Helmholtz equation:
Fxx + Fyy + λ2F = 0
![Page 1319: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1319.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1319,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Assuming F (x , y) = X (x)Y (y),
X ′′
X= −Y ′′ + λ2Y
Y= −µ2
⇒ X ′′ + µ2X = 0 and Y ′′ + ν2Y = 0,
such that λ =√
µ2 + ν2.
![Page 1320: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1320.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1320,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Assuming F (x , y) = X (x)Y (y),
X ′′
X= −Y ′′ + λ2Y
Y= −µ2
⇒ X ′′ + µ2X = 0 and Y ′′ + ν2Y = 0,
such that λ =√
µ2 + ν2.
With BC’s X (0) = X (a) = 0 and Y (0) = Y (b) = 0,
Xm(x) = sinmπx
aand Yn(y) = sin
nπy
b.
Corresponding values of λ are
λmn =
√
(mπ
a
)2+(nπ
b
)2
with solutions of T ′′ + c2λ2T = 0 as
Tmn(t) = Amn cos cλmnt + Bmn sin cλmnt.
![Page 1321: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1321.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1321,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Composing Xm(x), Yn(y) and Tmn(t) and superposing,
u(x , y , t) =
∞∑
m=1
∞∑
n=1
[Amn cos cλmnt+Bmn sin cλmnt] sinmπx
asin
nπy
b,
coefficients being determined from the double Fourier series
f (x , y) =
∞∑
m=1
∞∑
n=1
Amn sinmπx
asin
nπy
b
and g(x , y) =
∞∑
m=1
∞∑
n=1
cλmnBmn sinmπx
asin
nπy
b.
![Page 1322: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1322.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1322,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Two-Dimensional Wave Equation
Composing Xm(x), Yn(y) and Tmn(t) and superposing,
u(x , y , t) =
∞∑
m=1
∞∑
n=1
[Amn cos cλmnt+Bmn sin cλmnt] sinmπx
asin
nπy
b,
coefficients being determined from the double Fourier series
f (x , y) =
∞∑
m=1
∞∑
n=1
Amn sinmπx
asin
nπy
b
and g(x , y) =
∞∑
m=1
∞∑
n=1
cλmnBmn sinmπx
asin
nπy
b.
BVP’s modelled in polar coordinatesFor domains of circular symmetry, important in many practicalsystems, the BVP is conveniently modelled in polar coordinates,
the separation of variables quite often producing
Bessel’s equation, in cylindrical coordinates, and Legendre’s equation, in spherical coordinates
![Page 1323: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1323.jpg)
Mathematical Methods in Engineering and Science Partial Differential Equations 1323,
IntroductionHyperbolic EquationsParabolic EquationsElliptic EquationsTwo-Dimensional Wave Equation
Points to note
PDE’s in physically relevant contexts
Initial and boundary conditions
Separation of variables
Examples of boundary value problems with hyperbolic,parabolic and elliptic equations
Modelling, solution and interpretation
Cascaded application of separation of variables for problemswith more than two independent variables
Necessary Exercises: 1,2,4,7,9,10
![Page 1324: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1324.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1324,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Outline
Analytic FunctionsAnalyticity of Complex FunctionsConformal MappingPotential Theory
![Page 1325: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1325.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1325,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Function f of a complex variable z
gives a rule to associate a unique complex numberw = u + iv to every z = x + iy in a set.
![Page 1326: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1326.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1326,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Function f of a complex variable z
gives a rule to associate a unique complex numberw = u + iv to every z = x + iy in a set.
Limit: If f (z) is defined in a neighbourhood of z0 (except possiblyat z0 itself) and ∃l ∈ C such that ∀ ǫ > 0, ∃ δ > 0 such that
0 < |z − z0| < δ ⇒ |f (z)− l | < ǫ,
thenl = lim
z→z0
f (z).
![Page 1327: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1327.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1327,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Function f of a complex variable z
gives a rule to associate a unique complex numberw = u + iv to every z = x + iy in a set.
Limit: If f (z) is defined in a neighbourhood of z0 (except possiblyat z0 itself) and ∃l ∈ C such that ∀ ǫ > 0, ∃ δ > 0 such that
0 < |z − z0| < δ ⇒ |f (z)− l | < ǫ,
thenl = lim
z→z0
f (z).
Crucial difference from real functions: z can approach z0 in allpossible manners in the complex plane.
Definition of the limit is more restrictive.
![Page 1328: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1328.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1328,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Function f of a complex variable z
gives a rule to associate a unique complex numberw = u + iv to every z = x + iy in a set.
Limit: If f (z) is defined in a neighbourhood of z0 (except possiblyat z0 itself) and ∃l ∈ C such that ∀ ǫ > 0, ∃ δ > 0 such that
0 < |z − z0| < δ ⇒ |f (z)− l | < ǫ,
thenl = lim
z→z0
f (z).
Crucial difference from real functions: z can approach z0 in allpossible manners in the complex plane.
Definition of the limit is more restrictive.
Continuity: limz→z0 f (z) = f (z0)
Continuity in a domain D: continuity at every point in D
![Page 1329: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1329.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1329,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Derivative of a complex function:
f ′(z0) = limz→z0
f (z)− f (z0)
z − z0= lim
δz→0
f (z0 + δz)− f (z0)
δz
When this limit exists, function f (z) is said to be differentiable.
Extremely restrictive definition!
![Page 1330: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1330.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1330,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Derivative of a complex function:
f ′(z0) = limz→z0
f (z)− f (z0)
z − z0= lim
δz→0
f (z0 + δz)− f (z0)
δz
When this limit exists, function f (z) is said to be differentiable.
Extremely restrictive definition!
Analytic function
A function f (z) is called analytic in a domain D if it isdefined and differentiable at all points in D.
![Page 1331: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1331.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1331,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Derivative of a complex function:
f ′(z0) = limz→z0
f (z)− f (z0)
z − z0= lim
δz→0
f (z0 + δz)− f (z0)
δz
When this limit exists, function f (z) is said to be differentiable.
Extremely restrictive definition!
Analytic function
A function f (z) is called analytic in a domain D if it isdefined and differentiable at all points in D.
Points to be settled later:
Derivative of an analytic function is also analytic.
An analytic function possesses derivatives of all orders.
![Page 1332: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1332.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1332,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Derivative of a complex function:
f ′(z0) = limz→z0
f (z)− f (z0)
z − z0= lim
δz→0
f (z0 + δz)− f (z0)
δz
When this limit exists, function f (z) is said to be differentiable.
Extremely restrictive definition!
Analytic function
A function f (z) is called analytic in a domain D if it isdefined and differentiable at all points in D.
Points to be settled later:
Derivative of an analytic function is also analytic.
An analytic function possesses derivatives of all orders.
A great qualitative difference between functions of a real variableand those of a complex variable!
![Page 1333: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1333.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1333,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Cauchy-Riemann conditionsIf f (z) = u(x , y) + iv(x , y) is analytic then
f ′(z) = limδx ,δy→0
δu + iδv
δx + iδy
along all paths of approach for δz = δx + iδy → 0 or δx , δy → 0.
32
1
5
4
O
y
x
0z
Figure: Paths approaching z0
δz = iδy
δδz = xz0
xO
y
Figure: Paths in C-R equations
![Page 1334: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1334.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1334,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Cauchy-Riemann conditionsIf f (z) = u(x , y) + iv(x , y) is analytic then
f ′(z) = limδx ,δy→0
δu + iδv
δx + iδy
along all paths of approach for δz = δx + iδy → 0 or δx , δy → 0.
32
1
5
4
O
y
x
0z
Figure: Paths approaching z0
δz = iδy
δδz = xz0
xO
y
Figure: Paths in C-R equations
Two expressions for the derivative:
f ′(z) =∂u
∂x+ i
∂v
∂x=∂v
∂y− i
∂u
∂y
![Page 1335: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1335.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1335,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Cauchy-Riemann equations or conditions∂u∂x
= ∂v∂y
and ∂u∂y
= −∂v∂x
are necessary for analyticity.Question: Do the C-R conditions imply analyticity?
![Page 1336: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1336.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1336,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Cauchy-Riemann equations or conditions∂u∂x
= ∂v∂y
and ∂u∂y
= −∂v∂x
are necessary for analyticity.Question: Do the C-R conditions imply analyticity?Consider u(x , y) and v(x , y) having continuous first order partialderivatives that satisfy the Cauchy-Riemann conditions.By mean value theorem,
δu = u(x + δx , y + δy)− u(x , y) = δx∂u
∂x(x1, y1) + δy
∂u
∂y(x1, y1)
with x1 = x + ξδx , y1 = y + ξδy for some ξ ∈ [0, 1]; and
δv = v(x + δx , y + δy)− v(x , y) = δx∂v
∂x(x2, y2) + δy
∂v
∂y(x2, y2)
with x2 = x + ηδx , y2 = y + ηδy for some η ∈ [0, 1].
![Page 1337: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1337.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1337,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Cauchy-Riemann equations or conditions∂u∂x
= ∂v∂y
and ∂u∂y
= −∂v∂x
are necessary for analyticity.Question: Do the C-R conditions imply analyticity?Consider u(x , y) and v(x , y) having continuous first order partialderivatives that satisfy the Cauchy-Riemann conditions.By mean value theorem,
δu = u(x + δx , y + δy)− u(x , y) = δx∂u
∂x(x1, y1) + δy
∂u
∂y(x1, y1)
with x1 = x + ξδx , y1 = y + ξδy for some ξ ∈ [0, 1]; and
δv = v(x + δx , y + δy)− v(x , y) = δx∂v
∂x(x2, y2) + δy
∂v
∂y(x2, y2)
with x2 = x + ηδx , y2 = y + ηδy for some η ∈ [0, 1].Then,
δf =
[
δx∂u
∂x(x1, y1) + iδy
∂v
∂y(x2, y2)
]
+i
[
δx∂v
∂x(x2, y2)− iδy
∂u
∂y(x1, y1)
]
![Page 1338: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1338.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1338,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Using C-R conditions ∂v∂y
= ∂u∂x
and ∂u∂y
= −∂v∂x
,
δf = (δx + iδy)∂u
∂x(x1, y1) + iδy
[
∂u
∂x(x2, y2)−
∂u
∂x(x1, y1)
]
+ i(δx + iδy)∂v
∂x(x1, y1) + iδx
[
∂v
∂x(x2, y2)−
∂v
∂x(x1, y1)
]
⇒ δf
δz=
∂u
∂x(x1, y1) + i
∂v
∂x(x1, y1) +
iδx
δz
[
∂v
∂x(x2, y2)−
∂v
∂x(x1, y1)
]
+ iδy
δz
[
∂u
∂x(x2, y2)−
∂u
∂x(x1, y1)
]
![Page 1339: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1339.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1339,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Using C-R conditions ∂v∂y
= ∂u∂x
and ∂u∂y
= −∂v∂x
,
δf = (δx + iδy)∂u
∂x(x1, y1) + iδy
[
∂u
∂x(x2, y2)−
∂u
∂x(x1, y1)
]
+ i(δx + iδy)∂v
∂x(x1, y1) + iδx
[
∂v
∂x(x2, y2)−
∂v
∂x(x1, y1)
]
⇒ δf
δz=
∂u
∂x(x1, y1) + i
∂v
∂x(x1, y1) +
iδx
δz
[
∂v
∂x(x2, y2)−
∂v
∂x(x1, y1)
]
+ iδy
δz
[
∂u
∂x(x2, y2)−
∂u
∂x(x1, y1)
]
Since∣
∣
δxδz
∣
∣ ,∣
∣
∣
δyδz
∣
∣
∣ ≤ 1, as δz → 0, the limit exists and
f ′(z) =∂u
∂x+ i
∂v
∂x= −i
∂u
∂y+∂v
∂y.
Cauchy-Riemann conditions are necessary and sufficientfor function w = f (z) = u(x , y) + iv(x , y) to be analytic.
![Page 1340: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1340.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1340,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Harmonic functionDifferentiating C-R equations ∂v
∂y= ∂u
∂xand ∂u
∂y= −∂v
∂x,
∂2u
∂x2=
∂2v
∂x∂y,∂2u
∂y2= − ∂2v
∂y∂x,
∂2u
∂y∂x=∂2v
∂y2,
∂2u
∂x∂y= −∂
2v
∂x2
⇒ ∂2u
∂x2+∂2u
∂y2= 0 =
∂2v
∂x2+∂2v
∂y2.
Real and imaginary components of an analytic functionsare harmonic functions.
Conjugate harmonic function of u(x , y): v(x , y)
![Page 1341: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1341.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1341,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Analyticity of Complex Functions
Harmonic functionDifferentiating C-R equations ∂v
∂y= ∂u
∂xand ∂u
∂y= −∂v
∂x,
∂2u
∂x2=
∂2v
∂x∂y,∂2u
∂y2= − ∂2v
∂y∂x,
∂2u
∂y∂x=∂2v
∂y2,
∂2u
∂x∂y= −∂
2v
∂x2
⇒ ∂2u
∂x2+∂2u
∂y2= 0 =
∂2v
∂x2+∂2v
∂y2.
Real and imaginary components of an analytic functionsare harmonic functions.
Conjugate harmonic function of u(x , y): v(x , y)
Families of curves u(x , y) = c and v(x , y) = k are mutuallyorthogonal, except possibly at points where f ′(z) = 0.
Question: If u(x , y) is given, then how to develop the completeanalytic function w = f (z) = u(x , y) + iv(x , y)?
![Page 1342: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1342.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1342,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
Function: mapping of elements in domain to their images in range
![Page 1343: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1343.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1343,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
Function: mapping of elements in domain to their images in rangeDepiction of a complex variable requires a plane with two axes.Mapping of a complex function w = f (z) is shown in two planes.
![Page 1344: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1344.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1344,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
Function: mapping of elements in domain to their images in rangeDepiction of a complex variable requires a plane with two axes.Mapping of a complex function w = f (z) is shown in two planes.Example: mapping of a rectangle under transformation w = ez
−1 −0.5 0 0.5 1 1.5 2−1
−0.5
0
0.5
1
1.5
2
x
y
O A B
C D
(a) The z-plane
−1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5−1
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
u
v
A’ B’ O’
C’
D’
(b) The w -plane
Figure: Mapping corresponding to function w = ez
![Page 1345: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1345.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1345,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
Conformal mapping: a mapping that preserves the angle betweenany two directions in magnitude and sense.Verify: w = ez defines a conformal mapping.
Through relative orientations of curves at the points ofintersection, ‘local’ shape of a figure is preserved.
![Page 1346: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1346.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1346,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
Conformal mapping: a mapping that preserves the angle betweenany two directions in magnitude and sense.Verify: w = ez defines a conformal mapping.
Through relative orientations of curves at the points ofintersection, ‘local’ shape of a figure is preserved.
Take curve z(t), z(0) = z0 and image w(t) = f [z(t)],w0 = f (z0).For analytic f (z), w(0) = f ′(z0)z(0), implying
|w(0)| = |f ′(z0)| |z(0)| and arg w(0) = arg f ′(z0) + arg z(0).
For several curves through z0,
image curves pass through w0 and all of them turn by thesame angle arg f ′(z0).
Cautions f ′(z) varies from point to point. Different scaling and turning
effects take place at different points. ‘Global’ shape changes. For f ′(z) = 0, argument is undefined and conformality is lost.
![Page 1347: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1347.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1347,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
An analytic function defines a conformal mapping exceptat its critical points where its derivative vanishes.
Except at critical points, an analytic function is invertible.
We can establish an inverse of any conformal mapping.
![Page 1348: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1348.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1348,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
An analytic function defines a conformal mapping exceptat its critical points where its derivative vanishes.
Except at critical points, an analytic function is invertible.
We can establish an inverse of any conformal mapping.
Examples Linear function w = az + b (for a 6= 0) Linear fractional transformation
w =az + b
cz + d, ad − bc 6= 0
Other elementary functions like zn, ez etc
![Page 1349: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1349.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1349,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Conformal Mapping
An analytic function defines a conformal mapping exceptat its critical points where its derivative vanishes.
Except at critical points, an analytic function is invertible.
We can establish an inverse of any conformal mapping.
Examples Linear function w = az + b (for a 6= 0) Linear fractional transformation
w =az + b
cz + d, ad − bc 6= 0
Other elementary functions like zn, ez etc
Special significance of conformal mappings:
A harmonic function φ(u, v) in the w-plane is also aharmonic function, in the form φ(x , y) in the z-plane, aslong as the two planes are related through a conformalmapping.
![Page 1350: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1350.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1350,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Potential Theory
Riemann mapping theorem: Let D be a simply connecteddomain in the z-plane bounded by a closed curve C . Then thereexists a conformal mapping that gives a one-to-one correspondencebetween D and the unit disc |w | < 1 as well as between C and theunit circle |w | = 1, bounding the unit disc.
![Page 1351: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1351.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1351,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Potential Theory
Riemann mapping theorem: Let D be a simply connecteddomain in the z-plane bounded by a closed curve C . Then thereexists a conformal mapping that gives a one-to-one correspondencebetween D and the unit disc |w | < 1 as well as between C and theunit circle |w | = 1, bounding the unit disc.
Application to boundary value problems
First, establish a conformal mapping between the givendomain and a domain of simple geometry.
Next, solve the BVP in this simple domain.
Finally, using the inverse of the conformal mapping, constructthe solution for the given domain.
![Page 1352: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1352.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1352,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Potential Theory
Riemann mapping theorem: Let D be a simply connecteddomain in the z-plane bounded by a closed curve C . Then thereexists a conformal mapping that gives a one-to-one correspondencebetween D and the unit disc |w | < 1 as well as between C and theunit circle |w | = 1, bounding the unit disc.
Application to boundary value problems
First, establish a conformal mapping between the givendomain and a domain of simple geometry.
Next, solve the BVP in this simple domain.
Finally, using the inverse of the conformal mapping, constructthe solution for the given domain.
Example: Dirichlet problem with Poisson’s integral formula
f (re iθ) =1
2π
∫ 2π
0
(R2 − r2)f (Re iφ)
R2 − 2Rr cos(θ − φ) + r2dφ
![Page 1353: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1353.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1353,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Potential Theory
Two-dimensional potential flow
Velocity potential φ(x , y) gives velocity components Vx = ∂φ∂x
and Vy = ∂φ∂y
.
A streamline is a curve in the flow field, the tangent to whichat any point is along the local velocity vector.
Stream function ψ(x , y) remains constant along a streamline.
ψ(x , y) is the conjugate harmonic function of φ(x , y).
Complex potential function Φ(z) = φ(x , y) + iψ(x , y) definesthe flow.
![Page 1354: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1354.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1354,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Potential Theory
Two-dimensional potential flow
Velocity potential φ(x , y) gives velocity components Vx = ∂φ∂x
and Vy = ∂φ∂y
.
A streamline is a curve in the flow field, the tangent to whichat any point is along the local velocity vector.
Stream function ψ(x , y) remains constant along a streamline.
ψ(x , y) is the conjugate harmonic function of φ(x , y).
Complex potential function Φ(z) = φ(x , y) + iψ(x , y) definesthe flow.
If a flow field encounters a solid boundary of a complicated shape,
transform the boundary conformally to a simple boundary
to facilitate the study of the flow pattern.
![Page 1355: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1355.jpg)
Mathematical Methods in Engineering and Science Analytic Functions 1355,
Analyticity of Complex FunctionsConformal MappingPotential Theory
Points to note
Analytic functions and Cauchy-Riemann conditions
Conformality of analytic functions
Applications in solving BVP’s and flow description
Necessary Exercises: 1,2,3,4,7,9
![Page 1356: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1356.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1356,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Outline
Integrals in the Complex PlaneLine IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
![Page 1357: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1357.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1357,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Line Integral
For w = f (z) = u(x , y) + iv(x , y), over a smooth curve C ,∫
C
f (z)dz =
∫
C
(u+iv)(dx+idy) =
∫
C
(udx−vdy)+i
∫
C
(vdx+udy).
Extension to piecewise smooth curves is obvious.
![Page 1358: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1358.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1358,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Line Integral
For w = f (z) = u(x , y) + iv(x , y), over a smooth curve C ,∫
C
f (z)dz =
∫
C
(u+iv)(dx+idy) =
∫
C
(udx−vdy)+i
∫
C
(vdx+udy).
Extension to piecewise smooth curves is obvious.
With parametrization, for z = z(t), a ≤ t ≤ b, with z(t) 6= 0,∫
C
f (z)dz =
∫ b
a
f [z(t)]z(t)dt.
![Page 1359: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1359.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1359,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Line Integral
For w = f (z) = u(x , y) + iv(x , y), over a smooth curve C ,∫
C
f (z)dz =
∫
C
(u+iv)(dx+idy) =
∫
C
(udx−vdy)+i
∫
C
(vdx+udy).
Extension to piecewise smooth curves is obvious.
With parametrization, for z = z(t), a ≤ t ≤ b, with z(t) 6= 0,∫
C
f (z)dz =
∫ b
a
f [z(t)]z(t)dt.
Over a simple closed curve, contour integral:∮
Cf (z)dz
![Page 1360: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1360.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1360,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Line Integral
For w = f (z) = u(x , y) + iv(x , y), over a smooth curve C ,∫
C
f (z)dz =
∫
C
(u+iv)(dx+idy) =
∫
C
(udx−vdy)+i
∫
C
(vdx+udy).
Extension to piecewise smooth curves is obvious.
With parametrization, for z = z(t), a ≤ t ≤ b, with z(t) 6= 0,∫
C
f (z)dz =
∫ b
a
f [z(t)]z(t)dt.
Over a simple closed curve, contour integral:∮
Cf (z)dz
Example:∮
Czndz for integer n, around circle z = ρe iθ
∮
C
zndz = iρn+1
∫ 2π
0e i(n+1)θdθ =
0 for n 6= −1,2πi for n = −1.
![Page 1361: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1361.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1361,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Line Integral
For w = f (z) = u(x , y) + iv(x , y), over a smooth curve C ,∫
C
f (z)dz =
∫
C
(u+iv)(dx+idy) =
∫
C
(udx−vdy)+i
∫
C
(vdx+udy).
Extension to piecewise smooth curves is obvious.
With parametrization, for z = z(t), a ≤ t ≤ b, with z(t) 6= 0,∫
C
f (z)dz =
∫ b
a
f [z(t)]z(t)dt.
Over a simple closed curve, contour integral:∮
Cf (z)dz
Example:∮
Czndz for integer n, around circle z = ρe iθ
∮
C
zndz = iρn+1
∫ 2π
0e i(n+1)θdθ =
0 for n 6= −1,2πi for n = −1.
The M-L inequality: If C is a curve of finite length L and|f (z)| < M on C , then
∣
∣
∣
∣
∫
C
f (z)dz
∣
∣
∣
∣
≤∫
C
|f (z)| |dz | < M
∫
C
|dz | = ML.
![Page 1362: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1362.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1362,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
C is a simple closed curve in a simply connected domain D.
Function f (z) = u + iv is analytic in D.
Contour integral∮
Cf (z)dz =?
![Page 1363: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1363.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1363,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
C is a simple closed curve in a simply connected domain D.
Function f (z) = u + iv is analytic in D.
Contour integral∮
Cf (z)dz =?
If f ′(z) is continuous, then by Green’s theorem in the plane,∮
C
f (z)dz =
∫
R
∫ (
−∂v
∂x− ∂u
∂y
)
dxdy+i
∫
R
∫ (
∂u
∂x− ∂v
∂y
)
dxdy ,
where R is the region enclosed by C .
![Page 1364: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1364.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1364,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
C is a simple closed curve in a simply connected domain D.
Function f (z) = u + iv is analytic in D.
Contour integral∮
Cf (z)dz =?
If f ′(z) is continuous, then by Green’s theorem in the plane,∮
C
f (z)dz =
∫
R
∫ (
−∂v
∂x− ∂u
∂y
)
dxdy+i
∫
R
∫ (
∂u
∂x− ∂v
∂y
)
dxdy ,
where R is the region enclosed by C .
From C-R conditions,∮
Cf (z)dz = 0.
Proof by Goursat: without the hypothesis of continuity of f ′(z)
Cauchy-Goursat theorem
If f (z) is analytic in a simply connected domain D, then∮
Cf (z)dz = 0 for every simple closed curve C in D.
Importance of Goursat’s contribution:
continuity of f ′(z) appears as consequence!
![Page 1365: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1365.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1365,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Principle of path independenceTwo points z1 and z2 on the close curve C
two open paths C1 and C2 from z1 to z2
Cauchy’s theorem on C , comprising of C1 in the forward directionand C2 in the reverse direction:∫
C1
f (z)dz−∫
C2
f (z)dz = 0⇒∫ z2
z1
f (z)dz =
∫
C1
f (z)dz =
∫
C2
f (z)dz
![Page 1366: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1366.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1366,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Principle of path independenceTwo points z1 and z2 on the close curve C
two open paths C1 and C2 from z1 to z2
Cauchy’s theorem on C , comprising of C1 in the forward directionand C2 in the reverse direction:∫
C1
f (z)dz−∫
C2
f (z)dz = 0⇒∫ z2
z1
f (z)dz =
∫
C1
f (z)dz =
∫
C2
f (z)dz
For an analytic function f (z) in a simply connecteddomain D,
∫ z2
z1f (z)dz is independent of the path and
depends only on the end-points, as long as the path iscompletely contained in D.
![Page 1367: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1367.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1367,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Principle of path independenceTwo points z1 and z2 on the close curve C
two open paths C1 and C2 from z1 to z2
Cauchy’s theorem on C , comprising of C1 in the forward directionand C2 in the reverse direction:∫
C1
f (z)dz−∫
C2
f (z)dz = 0⇒∫ z2
z1
f (z)dz =
∫
C1
f (z)dz =
∫
C2
f (z)dz
For an analytic function f (z) in a simply connecteddomain D,
∫ z2
z1f (z)dz is independent of the path and
depends only on the end-points, as long as the path iscompletely contained in D.
Consequence: Definition of the function
F (z) =
∫ z
z0
f (ξ)dξ
What does the formulation suggest?
![Page 1368: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1368.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1368,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Indefinite integralQuestion: Is F (z) analytic? Is F ′(z) = f (z)?
![Page 1369: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1369.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1369,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Indefinite integralQuestion: Is F (z) analytic? Is F ′(z) = f (z)?
F (z + δz)− F (z)
δz− f (z) =
1
δz
[∫ z+δz
z0
f (ξ)dξ −∫ z
z0
f (ξ)dξ
]
− f (z)
=1
δz
∫ z+δz
z
[f (ξ)− f (z)]dξ
![Page 1370: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1370.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1370,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Indefinite integralQuestion: Is F (z) analytic? Is F ′(z) = f (z)?
F (z + δz)− F (z)
δz− f (z) =
1
δz
[∫ z+δz
z0
f (ξ)dξ −∫ z
z0
f (ξ)dξ
]
− f (z)
=1
δz
∫ z+δz
z
[f (ξ)− f (z)]dξ
f is continuous ⇒ ∀ǫ,∃δ such that |ξ− z | < δ ⇒ |f (ξ)− f (z)| < ǫ
![Page 1371: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1371.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1371,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Indefinite integralQuestion: Is F (z) analytic? Is F ′(z) = f (z)?
F (z + δz)− F (z)
δz− f (z) =
1
δz
[∫ z+δz
z0
f (ξ)dξ −∫ z
z0
f (ξ)dξ
]
− f (z)
=1
δz
∫ z+δz
z
[f (ξ)− f (z)]dξ
f is continuous ⇒ ∀ǫ,∃δ such that |ξ− z | < δ ⇒ |f (ξ)− f (z)| < ǫChoosing δz < δ,
∣
∣
∣
∣
F (z + δz)− F (z)
δz− f (z)
∣
∣
∣
∣
<ǫ
δz
∫ z+δz
z
dξ = ǫ.
If f (z) is analytic in a simply connected domain D, thenthere exists an analytic function F (z) in D such that
F ′(z) = f (z) and
∫ z2
z1
f (z)dz = F (z2)− F (z1).
![Page 1372: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1372.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1372,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Principle of deformation of paths
f (z) analytic everywhere otherthan isolated points s1, s2, s3
∫
C1
f (z)dz =
∫
C2
f (z)dz =
∫
C3
f (z)dz
Not so for path C ∗.
2z
D
C
C *
sz1
C
C s
s
1
2
33
2
1
Figure: Path deformation
![Page 1373: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1373.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1373,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Principle of deformation of paths
f (z) analytic everywhere otherthan isolated points s1, s2, s3
∫
C1
f (z)dz =
∫
C2
f (z)dz =
∫
C3
f (z)dz
Not so for path C ∗.
2z
D
C
C *
sz1
C
C s
s
1
2
33
2
1
Figure: Path deformation
The line integral remains unaltered through a continuousdeformation of the path of integration with fixedend-points, as long as the sweep of the deformationincludes no point where the integrand is non-analytic.
![Page 1374: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1374.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1374,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Cauchy’s theorem in multiply connected domain
LC
L
L3
C
C2
1
1
2
3
C
Figure: Contour for multiply connected domain∮
C
f (z)dz −∮
C1
f (z)dz −∮
C2
f (z)dz −∮
C3
f (z)dz = 0.
![Page 1375: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1375.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1375,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Theorem
Cauchy’s theorem in multiply connected domain
LC
L
L3
C
C2
1
1
2
3
C
Figure: Contour for multiply connected domain∮
C
f (z)dz −∮
C1
f (z)dz −∮
C2
f (z)dz −∮
C3
f (z)dz = 0.
If f (z) is analytic in a region bounded by the contour Cas the outer boundary and non-overlapping contours C1,C2, C3, · · · , Cn as inner boundaries, then
∮
C
f (z)dz =n∑
i=1
∮
Ci
f (z)dz .
![Page 1376: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1376.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1376,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z): analytic function in a simply connected domain D
For z0 ∈ D and simple closed curve C in D,
∮
C
f (z)
z − z0dz = 2πif (z0).
![Page 1377: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1377.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1377,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z): analytic function in a simply connected domain D
For z0 ∈ D and simple closed curve C in D,
∮
C
f (z)
z − z0dz = 2πif (z0).
Consider C as a circle with centre at z0 and radius ρ,
with no loss of generality (why?).
![Page 1378: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1378.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1378,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z): analytic function in a simply connected domain D
For z0 ∈ D and simple closed curve C in D,
∮
C
f (z)
z − z0dz = 2πif (z0).
Consider C as a circle with centre at z0 and radius ρ,
with no loss of generality (why?).
∮
C
f (z)
z − z0dz = f (z0)
∮
C
dz
z − z0+
∮
C
f (z)− f (z0)
z − z0dz
![Page 1379: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1379.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1379,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z): analytic function in a simply connected domain D
For z0 ∈ D and simple closed curve C in D,
∮
C
f (z)
z − z0dz = 2πif (z0).
Consider C as a circle with centre at z0 and radius ρ,
with no loss of generality (why?).
∮
C
f (z)
z − z0dz = f (z0)
∮
C
dz
z − z0+
∮
C
f (z)− f (z0)
z − z0dz
From continuity of f (z), ∃δ such that for any ǫ,
|z − z0| < δ ⇒ |f (z)− f (z0)| < ǫ and
∣
∣
∣
∣
f (z)− f (z0)
z − z0
∣
∣
∣
∣
<ǫ
ρ,
with ρ < δ.
![Page 1380: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1380.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1380,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z): analytic function in a simply connected domain D
For z0 ∈ D and simple closed curve C in D,
∮
C
f (z)
z − z0dz = 2πif (z0).
Consider C as a circle with centre at z0 and radius ρ,
with no loss of generality (why?).
∮
C
f (z)
z − z0dz = f (z0)
∮
C
dz
z − z0+
∮
C
f (z)− f (z0)
z − z0dz
From continuity of f (z), ∃δ such that for any ǫ,
|z − z0| < δ ⇒ |f (z)− f (z0)| < ǫ and
∣
∣
∣
∣
f (z)− f (z0)
z − z0
∣
∣
∣
∣
<ǫ
ρ,
with ρ < δ. From M-L inequality, the second integral vanishes.
![Page 1381: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1381.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1381,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Direct applications
Evaluation of contour integral: If g(z) is analytic on the contour and in the enclosed region,
the Cauchy’s theorem implies∮
Cg(z)dz = 0.
If the contour encloses a singularity at z0, then Cauchy’sformula supplies a non-zero contribution to the integral, iff (z) = g(z)(z − z0) is analytic.
Evaluation of function at a point: If finding the integral onthe left-hand-side is relatively simple, then we use it toevaluate f (z0).
Significant in the solution of boundary valueproblems!
Example: Poisson’s integral formula
u(r , θ) =1
2π
∫ 2π
0
(R2 − r2)u(R , φ)
R2 − 2Rr cos(θ − φ) + r2dφ
for the Dirichlet problem over a circular disc.
![Page 1382: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1382.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1382,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Poisson’s integral formulaTaking z0 = re iθ and z = Re iφ (with r < R) in Cauchy’s formula,
2πif (re iθ) =
∫ 2π
0
f (Re iφ)
Re iφ − re iθ(iRe iφ)dφ.
How to get rid of imaginary quantities from the expression?
![Page 1383: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1383.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1383,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Poisson’s integral formulaTaking z0 = re iθ and z = Re iφ (with r < R) in Cauchy’s formula,
2πif (re iθ) =
∫ 2π
0
f (Re iφ)
Re iφ − re iθ(iRe iφ)dφ.
How to get rid of imaginary quantities from the expression?Develop a complement. With R2
rin place of r ,
0 =
∫ 2π
0
f (Re iφ)
Re iφ − R2
re iθ
(iRe iφ)dφ =
∫ 2π
0
f (Re iφ)
re−iθ − Re−iφ(ire−iθ)dφ.
Subtracting,
2πif (re iθ) = i
∫ 2π
0f (Re iφ)
[
Re iφ
Re iφ − re iθ+
re−iθ
Re−iφ − re−iθ
]
dφ
= i
∫ 2π
0
(R2 − r2)f (Re iφ)
(Re iφ − re iθ)(Re−iφ − re−iθ)dφ
⇒ f (re iθ) =1
2π
∫ 2π
0
(R2 − r2)f (Re iφ)
R2 − 2Rr cos(θ − φ) + r2dφ.
![Page 1384: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1384.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1384,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Cauchy’s integral formula evaluates contour integral of g(z),
if the contour encloses a point z0 where g(z) isnon-analytic but g(z)(z − z0) is analytic.
If g(z)(z − z0) is also non-analytic, but g(z)(z − z0)2 is analytic?
![Page 1385: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1385.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1385,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Cauchy’s integral formula evaluates contour integral of g(z),
if the contour encloses a point z0 where g(z) isnon-analytic but g(z)(z − z0) is analytic.
If g(z)(z − z0) is also non-analytic, but g(z)(z − z0)2 is analytic?
f (z0) =1
2πi
∮
C
f (z)
z − z0dz ,
f ′(z0) =1
2πi
∮
C
f (z)
(z − z0)2dz ,
f ′′(z0) =2!
2πi
∮
C
f (z)
(z − z0)3dz ,
· · · = · · · · · · · · · ,
f (n)(z0) =n!
2πi
∮
C
f (z)
(z − z0)n+1dz .
![Page 1386: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1386.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1386,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
Cauchy’s integral formula evaluates contour integral of g(z),
if the contour encloses a point z0 where g(z) isnon-analytic but g(z)(z − z0) is analytic.
If g(z)(z − z0) is also non-analytic, but g(z)(z − z0)2 is analytic?
f (z0) =1
2πi
∮
C
f (z)
z − z0dz ,
f ′(z0) =1
2πi
∮
C
f (z)
(z − z0)2dz ,
f ′′(z0) =2!
2πi
∮
C
f (z)
(z − z0)3dz ,
· · · = · · · · · · · · · ,
f (n)(z0) =n!
2πi
∮
C
f (z)
(z − z0)n+1dz .
The formal expressions can be established through differentiationunder the integral sign.
![Page 1387: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1387.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1387,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z0 + δz)− f (z0)
δz=
1
2πiδz
∮
C
f (z)
[
1
z − z0 − δz− 1
z − z0
]
dz
=1
2πi
∮
C
f (z)dz
(z − z0 − δz)(z − z0)
=1
2πi
∮
C
f (z)dz
(z − z0)2+
1
2πi
∮
C
f (z)
[
1
(z − z0 − δz)(z − z0)− 1
(z − z0)2
]
dz
=1
2πi
∮
C
f (z)dz
(z − z0)2+
1
2πiδz
∮
C
f (z)dz
(z − z0 − δz)(z − z0)2
![Page 1388: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1388.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1388,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Cauchy’s Integral Formula
f (z0 + δz)− f (z0)
δz=
1
2πiδz
∮
C
f (z)
[
1
z − z0 − δz− 1
z − z0
]
dz
=1
2πi
∮
C
f (z)dz
(z − z0 − δz)(z − z0)
=1
2πi
∮
C
f (z)dz
(z − z0)2+
1
2πi
∮
C
f (z)
[
1
(z − z0 − δz)(z − z0)− 1
(z − z0)2
]
dz
=1
2πi
∮
C
f (z)dz
(z − z0)2+
1
2πiδz
∮
C
f (z)dz
(z − z0 − δz)(z − z0)2
If |f (z)| < M on C , L is path length and d0 = min |z − z0|,∣
∣
∣
∣
δz
∮
C
f (z)dz
(z − z0 − δz)(z − z0)2
∣
∣
∣
∣
<ML|δz |
d20 (d0 − |δz |)
→ 0 as δz → 0.
An analytic function possesses derivatives of all orders atevery point in its domain.
Analyticity implies much more than mere differentiability!
![Page 1389: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1389.jpg)
Mathematical Methods in Engineering and Science Integrals in the Complex Plane 1389,
Line IntegralCauchy’s Integral TheoremCauchy’s Integral Formula
Points to note
Concept of line integral in complex plane
Cauchy’s integral theorem
Consequences of analyticity
Cauchy’s integral formula
Derivatives of arbitrary order for analytic functions
Necessary Exercises: 1,2,5,7
![Page 1390: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1390.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1390,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Outline
Singularities of Complex FunctionsSeries Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
![Page 1391: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1391.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1391,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Taylor’s series of function f (z), analytic in a neighbourhood of z0:
f (z) =
∞∑
n=0
an(z−z0)n = a0+a1(z−z0)+a2(z−z0)
2+a3(z−z0)3+· · · ,
with coefficients
an =1
n!f (n)(z0) =
1
2πi
∮
C
f (w)dw
(w − z0)n+1,
where C is a circle with centre at z0.Form of the series and coefficients: similar to real functions
![Page 1392: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1392.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1392,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Taylor’s series of function f (z), analytic in a neighbourhood of z0:
f (z) =
∞∑
n=0
an(z−z0)n = a0+a1(z−z0)+a2(z−z0)
2+a3(z−z0)3+· · · ,
with coefficients
an =1
n!f (n)(z0) =
1
2πi
∮
C
f (w)dw
(w − z0)n+1,
where C is a circle with centre at z0.Form of the series and coefficients: similar to real functions
The series representation is convergent within a disc|z − z0| < R, where radius of convergence R is thedistance of the nearest singularity from z0.
![Page 1393: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1393.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1393,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Taylor’s series of function f (z), analytic in a neighbourhood of z0:
f (z) =
∞∑
n=0
an(z−z0)n = a0+a1(z−z0)+a2(z−z0)
2+a3(z−z0)3+· · · ,
with coefficients
an =1
n!f (n)(z0) =
1
2πi
∮
C
f (w)dw
(w − z0)n+1,
where C is a circle with centre at z0.Form of the series and coefficients: similar to real functions
The series representation is convergent within a disc|z − z0| < R, where radius of convergence R is thedistance of the nearest singularity from z0.
Note: No valid power series representation around z0, i.e. inpowers of (z − z0), if f(z) is not analytic at z0
![Page 1394: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1394.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1394,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Taylor’s series of function f (z), analytic in a neighbourhood of z0:
f (z) =
∞∑
n=0
an(z−z0)n = a0+a1(z−z0)+a2(z−z0)
2+a3(z−z0)3+· · · ,
with coefficients
an =1
n!f (n)(z0) =
1
2πi
∮
C
f (w)dw
(w − z0)n+1,
where C is a circle with centre at z0.Form of the series and coefficients: similar to real functions
The series representation is convergent within a disc|z − z0| < R, where radius of convergence R is thedistance of the nearest singularity from z0.
Note: No valid power series representation around z0, i.e. inpowers of (z − z0), if f(z) is not analytic at z0
Question: In that case, what about a series representation thatincludes negative powers of (z − z0) as well?
![Page 1395: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1395.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1395,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Laurent’s series: If f (z) is analytic on circles C1 (outer) and C2
(inner) with centre at z0, and in the annulus in between, then
f (z) =∞∑
n=−∞an(z − z0)
n =∞∑
m=0
bm(z − z0)m +
∞∑
m=1
cm
(z − z0)m;
with coefficients
an =1
2πi
∮
C
f (w)dw
(w − z0)n+1;
or, bm =1
2πi
∮
C
f (w)dw
(w − z0)m+1, cm =
1
2πi
∮
C
f (w)(w − z0)m−1dw ;
the contour C lying in the annulus and enclosing C2.
![Page 1396: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1396.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1396,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Laurent’s series: If f (z) is analytic on circles C1 (outer) and C2
(inner) with centre at z0, and in the annulus in between, then
f (z) =∞∑
n=−∞an(z − z0)
n =∞∑
m=0
bm(z − z0)m +
∞∑
m=1
cm
(z − z0)m;
with coefficients
an =1
2πi
∮
C
f (w)dw
(w − z0)n+1;
or, bm =1
2πi
∮
C
f (w)dw
(w − z0)m+1, cm =
1
2πi
∮
C
f (w)(w − z0)m−1dw ;
the contour C lying in the annulus and enclosing C2.
Validity of this series representation: in annular region obtained bygrowing C1 and shrinking C2 till f (z) ceases to be analytic.
![Page 1397: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1397.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1397,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Laurent’s series: If f (z) is analytic on circles C1 (outer) and C2
(inner) with centre at z0, and in the annulus in between, then
f (z) =∞∑
n=−∞an(z − z0)
n =∞∑
m=0
bm(z − z0)m +
∞∑
m=1
cm
(z − z0)m;
with coefficients
an =1
2πi
∮
C
f (w)dw
(w − z0)n+1;
or, bm =1
2πi
∮
C
f (w)dw
(w − z0)m+1, cm =
1
2πi
∮
C
f (w)(w − z0)m−1dw ;
the contour C lying in the annulus and enclosing C2.
Validity of this series representation: in annular region obtained bygrowing C1 and shrinking C2 till f (z) ceases to be analytic.
Observation: If f (z) is analytic inside C2 as well, then cm = 0 andLaurent’s series reduces to Taylor’s series.
![Page 1398: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1398.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1398,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s seriesCauchy’s integral formula for any point z in the annulus,
f (z) =1
2πi
∮
C1
f (w)dw
w − z− 1
2πi
∮
C2
f (w)dw
w − z.
![Page 1399: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1399.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1399,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s seriesCauchy’s integral formula for any point z in the annulus,
f (z) =1
2πi
∮
C1
f (w)dw
w − z− 1
2πi
∮
C2
f (w)dw
w − z.
Organization of the series:
1
w − z=
1
(w − z0)[1− (z − z0)/(w − z0)]
1
w − z= − 1
(z − z0)[1− (w − z0)/(z − z0)]
C2 1
0
z
z
C
w
Figure: The annulus
![Page 1400: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1400.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1400,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s seriesCauchy’s integral formula for any point z in the annulus,
f (z) =1
2πi
∮
C1
f (w)dw
w − z− 1
2πi
∮
C2
f (w)dw
w − z.
Organization of the series:
1
w − z=
1
(w − z0)[1− (z − z0)/(w − z0)]
1
w − z= − 1
(z − z0)[1− (w − z0)/(z − z0)]
C2 1
0
z
z
C
w
Figure: The annulus
Using the expression for the sum of a geometric series,
1+q+q2+· · ·+qn−1 =1− qn
1− q⇒ 1
1− q= 1+q+q2+· · ·+qn−1+
qn
1− q.
![Page 1401: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1401.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1401,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s seriesCauchy’s integral formula for any point z in the annulus,
f (z) =1
2πi
∮
C1
f (w)dw
w − z− 1
2πi
∮
C2
f (w)dw
w − z.
Organization of the series:
1
w − z=
1
(w − z0)[1− (z − z0)/(w − z0)]
1
w − z= − 1
(z − z0)[1− (w − z0)/(z − z0)]
C2 1
0
z
z
C
w
Figure: The annulus
Using the expression for the sum of a geometric series,
1+q+q2+· · ·+qn−1 =1− qn
1− q⇒ 1
1− q= 1+q+q2+· · ·+qn−1+
qn
1− q.
We use q = z−z0w−z0
for integral over C1 and q = w−z0z−z0
over C2.
![Page 1402: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1402.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1402,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s series (contd)Using q = z−z0
w−z0,
1
w − z=
1
w − z0+
z − z0
(w − z0)2+· · ·+(z − z0)
n−1
(w − z0)n+
(
z − z0
w − z0
)n 1
w − z
⇒ 1
2πi
∮
C1
f (w)dw
w − z= a0 +a1(z− z0)+ · · ·+an−1(z− z0)
n−1 +Tn,
with coefficients as required and
Tn =1
2πi
∮
C1
(
z − z0
w − z0
)n f (w)
w − zdw .
![Page 1403: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1403.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1403,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Proof of Laurent’s series (contd)Using q = z−z0
w−z0,
1
w − z=
1
w − z0+
z − z0
(w − z0)2+· · ·+(z − z0)
n−1
(w − z0)n+
(
z − z0
w − z0
)n 1
w − z
⇒ 1
2πi
∮
C1
f (w)dw
w − z= a0 +a1(z− z0)+ · · ·+an−1(z− z0)
n−1 +Tn,
with coefficients as required and
Tn =1
2πi
∮
C1
(
z − z0
w − z0
)n f (w)
w − zdw .
Similarly, with q = w−z0z−z0
,
− 1
2πi
∮
C2
f (w)dw
w − z= a−1(z − z0)
−1 + · · · + a−n(z − z0)−n + T−n,
with appropriate coefficients and the remainder term
T−n =1
2πi
∮
C2
(
w − z0
z − z0
)n f (w)
z − wdw .
![Page 1404: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1404.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1404,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Convergence of Laurent’s series
f (z) =
n−1∑
k=−n
ak(z − z0)k + Tn + T−n,
where Tn =1
2πi
∮
C1
(
z − z0
w − z0
)n f (w)
w − zdw
and T−n =1
2πi
∮
C2
(
w − z0
z − z0
)n f (w)
z − wdw .
![Page 1405: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1405.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1405,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Convergence of Laurent’s series
f (z) =
n−1∑
k=−n
ak(z − z0)k + Tn + T−n,
where Tn =1
2πi
∮
C1
(
z − z0
w − z0
)n f (w)
w − zdw
and T−n =1
2πi
∮
C2
(
w − z0
z − z0
)n f (w)
z − wdw .
f (w) is bounded
∣
∣
∣
z−z0w−z0
∣
∣
∣< 1 over C1 and
∣
∣
∣
w−z0z−z0
∣
∣
∣< 1 over C2
Use M-L inequality to show that
remainder terms Tn and T−n approach zero as n→∞.
![Page 1406: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1406.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1406,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Series Representations of Complex Functions
Convergence of Laurent’s series
f (z) =
n−1∑
k=−n
ak(z − z0)k + Tn + T−n,
where Tn =1
2πi
∮
C1
(
z − z0
w − z0
)n f (w)
w − zdw
and T−n =1
2πi
∮
C2
(
w − z0
z − z0
)n f (w)
z − wdw .
f (w) is bounded
∣
∣
∣
z−z0w−z0
∣
∣
∣< 1 over C1 and
∣
∣
∣
w−z0z−z0
∣
∣
∣< 1 over C2
Use M-L inequality to show that
remainder terms Tn and T−n approach zero as n→∞.
Remark: For actually developing Taylor’s or Laurent’s series of afunction, algebraic manipulation of known facts are employed quiteoften, rather than evaluating so many contour integrals!
![Page 1407: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1407.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1407,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Zeros and Singularities
Zeros of an analytic function: points where the function vanishes
If, at a point z0,
a function f (z) vanishes along with first m − 1 of itsderivatives, but f (m)(z0) 6= 0;
then z0 is a zero of f (z) of order m, giving the Taylor’s series as
f (z) = (z − z0)mg(z).
![Page 1408: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1408.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1408,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Zeros and Singularities
Zeros of an analytic function: points where the function vanishes
If, at a point z0,
a function f (z) vanishes along with first m − 1 of itsderivatives, but f (m)(z0) 6= 0;
then z0 is a zero of f (z) of order m, giving the Taylor’s series as
f (z) = (z − z0)mg(z).
An isolated zero has a neighbourhood containing no other zero.
For an analytic function, not identically zero, every pointhas a neighbourhood free of zeros of the function, exceptpossibly for that point itself. In particular, zeros of suchan analytic function are always isolated.
Implication: If f (z) has a zero in every neighbourhood aroundz0 then it cannot be analytic at z0, unless it is the zero function[i.e. f (z) = 0 everywhere].
![Page 1409: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1409.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1409,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Zeros and Singularities
Entire function: A function which is analytic everywhereExamples: zn (for positive integer n), ez , sin z etc.
The Taylor’s series of an entire function has an infiniteradius of convergence.
![Page 1410: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1410.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1410,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Zeros and Singularities
Entire function: A function which is analytic everywhereExamples: zn (for positive integer n), ez , sin z etc.
The Taylor’s series of an entire function has an infiniteradius of convergence.
Singularities: points where a function ceases to be analytic
Removable singularity: If f (z) is not defined at z0, but has a limit.Example: f (z) = ez−1
zat z = 0.
Pole: If f (z) has a Laurent’s series around z0, with a finitenumber of terms with negative powers. If an = 0 forn < −m, but a−m 6= 0, then z0 is a pole of order m,limz→z0(z − z0)
mf (z) being a non-zero finite number.A simple pole: a pole of order one.
Essential singularity: A singularity which is neither a removablesingularity nor a pole. If the function has a Laurent’sseries, then it has infinite terms with negativepowers. Example: f (z) = e1/z at z = 0.
![Page 1411: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1411.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1411,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Zeros and Singularities
Zeros and poles: complementary to each other
Poles are necessarily isolated singularities. A zero of f (z) of order m is a pole of 1
f (z) of the same orderand vice versa.
If f (z) has a zero of order m at z0 where g(z) has a pole ofthe same order, then f (z)g(z) is either analytic at z0 or has aremovable singularity there.
Argument theorem:
If f (z) is analytic inside and on a simple closedcurve C except for a finite number of poles insideand f (z) 6= 0 on C, then
1
2πi
∮
C
f ′(z)
f (z)dz = N − P ,
where N and P are total numbers of zeros and polesinside C respectively, counting multiplicities (orders).
![Page 1412: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1412.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1412,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
ResiduesTerm by term integration of Laurent’s series:
∮
Cf (z)dz = 2πia−1
![Page 1413: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1413.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1413,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
ResiduesTerm by term integration of Laurent’s series:
∮
Cf (z)dz = 2πia−1
Residue: Resz0
f (z) = a−1 = 12πi
∮
Cf (z)dz
![Page 1414: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1414.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1414,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
ResiduesTerm by term integration of Laurent’s series:
∮
Cf (z)dz = 2πia−1
Residue: Resz0
f (z) = a−1 = 12πi
∮
Cf (z)dz
If f (z) has a pole (of order m) at z0, then
(z − z0)mf (z) =
∞∑
n=−m
an(z − z0)m+n
is analytic at z0, and
dm−1
dzm−1[(z − z0)
mf (z)] =∞∑
n=−1
(m + n)!
(n + 1)!an(z − z0)
n+1
![Page 1415: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1415.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1415,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
ResiduesTerm by term integration of Laurent’s series:
∮
Cf (z)dz = 2πia−1
Residue: Resz0
f (z) = a−1 = 12πi
∮
Cf (z)dz
If f (z) has a pole (of order m) at z0, then
(z − z0)mf (z) =
∞∑
n=−m
an(z − z0)m+n
is analytic at z0, and
dm−1
dzm−1[(z − z0)
mf (z)] =∞∑
n=−1
(m + n)!
(n + 1)!an(z − z0)
n+1
⇒ Resz0
f (z) = a−1 =1
(m − 1)!lim
z→z0
dm−1
dzm−1[(z − z0)
mf (z)].
![Page 1416: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1416.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1416,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
ResiduesTerm by term integration of Laurent’s series:
∮
Cf (z)dz = 2πia−1
Residue: Resz0
f (z) = a−1 = 12πi
∮
Cf (z)dz
If f (z) has a pole (of order m) at z0, then
(z − z0)mf (z) =
∞∑
n=−m
an(z − z0)m+n
is analytic at z0, and
dm−1
dzm−1[(z − z0)
mf (z)] =∞∑
n=−1
(m + n)!
(n + 1)!an(z − z0)
n+1
⇒ Resz0
f (z) = a−1 =1
(m − 1)!lim
z→z0
dm−1
dzm−1[(z − z0)
mf (z)].
Residue theorem: If f (z) is analytic inside and on simple closedcurve C , with singularities at z1, z2, z3, · · · , zk inside C ; then
∮
C
f (z)dz = 2πi
k∑
i=1
Reszi
f (z).
![Page 1417: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1417.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1417,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
General strategy Identify the required integral as a contour integral of a
complex function, or a part thereof. If the domain of integration is infinite, then extend the
contour infinitely, without enclosing new singularities.
![Page 1418: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1418.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1418,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
General strategy Identify the required integral as a contour integral of a
complex function, or a part thereof. If the domain of integration is infinite, then extend the
contour infinitely, without enclosing new singularities.
Example:
I =
∫ 2π
0φ(cos θ, sin θ)dθ
With z = e iθ and dz = izdθ,
I =
∮
C
φ
[
1
2
(
z +1
z
)
,1
2i
(
z − 1
z
)]
dz
iz=
∮
C
f (z)dz ,
where C is the unit circle centred at the origin.Denoting poles falling inside the unit circle C as pj ,
I = 2πi∑
j
Respj
f (z).
![Page 1419: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1419.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1419,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: For real rational function f (x),
I =
∫ ∞
−∞f (x)dx ,
denominator of f (x) being of degree two higher than numerator.
![Page 1420: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1420.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1420,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: For real rational function f (x),
I =
∫ ∞
−∞f (x)dx ,
denominator of f (x) being of degree two higher than numerator.
Consider contour C enclosing semi-circular region |z | ≤ R , y ≥ 0,large enough to enclose all singularities above the x-axis.
S
iR
−R RO x
p
p
p
R
y
Figure: The contour
![Page 1421: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1421.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1421,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: For real rational function f (x),
I =
∫ ∞
−∞f (x)dx ,
denominator of f (x) being of degree two higher than numerator.
Consider contour C enclosing semi-circular region |z | ≤ R , y ≥ 0,large enough to enclose all singularities above the x-axis.
∮
C
f (z)dz =
∫ R
−R
f (x)dx +
∫
S
f (z)dz
For finite M, |f (z)| < MR2 on C
∣
∣
∣
∣
∫
S
f (z)dz
∣
∣
∣
∣
<M
R2πR =
πM
R.
S
iR
−R RO x
p
p
p
R
y
Figure: The contour
![Page 1422: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1422.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1422,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: For real rational function f (x),
I =
∫ ∞
−∞f (x)dx ,
denominator of f (x) being of degree two higher than numerator.
Consider contour C enclosing semi-circular region |z | ≤ R , y ≥ 0,large enough to enclose all singularities above the x-axis.
∮
C
f (z)dz =
∫ R
−R
f (x)dx +
∫
S
f (z)dz
For finite M, |f (z)| < MR2 on C
∣
∣
∣
∣
∫
S
f (z)dz
∣
∣
∣
∣
<M
R2πR =
πM
R.
I =
∫ ∞
−∞f (x)dx = 2πi
∑
j
Respj
f (z) as R →∞.
S
iR
−R RO x
p
p
p
R
y
Figure: The contour
![Page 1423: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1423.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1423,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: Fourier integral coefficients
A(s) =
∫ ∞
−∞f (x) cos sx dx and B(s) =
∫ ∞
−∞f (x) sin sx dx
![Page 1424: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1424.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1424,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Evaluation of Real Integrals
Example: Fourier integral coefficients
A(s) =
∫ ∞
−∞f (x) cos sx dx and B(s) =
∫ ∞
−∞f (x) sin sx dx
Consider
I = A(s) + iB(s) =
∫ ∞
−∞f (x)e isxdx .
Similar to the previous case,∮
C
f (z)e iszdz =
∫ R
−R
f (x)e isxdx +
∫
S
f (z)e iszdz .
As |e isz | = |e isx | |e−sy | = |e−sy | ≤ 1 for y ≥ 0, we have∣
∣
∣
∣
∫
S
f (z)e iszdz
∣
∣
∣
∣
<M
R2πR =
πM
R,
which yields, as R →∞,
I = 2πi∑
j
Respj
[f (z)e isz ].
![Page 1425: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1425.jpg)
Mathematical Methods in Engineering and Science Singularities of Complex Functions 1425,
Series Representations of Complex FunctionsZeros and SingularitiesResiduesEvaluation of Real Integrals
Points to note
Taylor’s series and Laurent’s series
Zeros and poles of analytic functions
Residue theorem
Evaluation of real integrals through contour integration ofsuitable complex functions
Necessary Exercises: 1,2,3,5,8,9,10
![Page 1426: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1426.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1426,
IntroductionEuler’s EquationDirect Methods
Outline
Variational Calculus*IntroductionEuler’s EquationDirect Methods
![Page 1427: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1427.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1427,
IntroductionEuler’s EquationDirect Methods
Introduction
Consider a particle moving on a smooth surface z = ψ(q1, q2).
With position r = [q1(t) q2(t) ψ(q1(t), q2(t))]T on the surface
and δr = [δq1 δq2 (∇ψ)T δq]T in the tangent plane, length of thepath from qi = q(ti) to qf = q(tf ) is
l =
∫
‖δr‖ =
∫ tf
ti
‖r‖dt =
∫ tf
ti
[
q21 + q2
2 + (∇ψT q)2]1/2
dt.
For shortest path or geodesic, minimize the path length l .
![Page 1428: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1428.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1428,
IntroductionEuler’s EquationDirect Methods
Introduction
Consider a particle moving on a smooth surface z = ψ(q1, q2).
With position r = [q1(t) q2(t) ψ(q1(t), q2(t))]T on the surface
and δr = [δq1 δq2 (∇ψ)T δq]T in the tangent plane, length of thepath from qi = q(ti) to qf = q(tf ) is
l =
∫
‖δr‖ =
∫ tf
ti
‖r‖dt =
∫ tf
ti
[
q21 + q2
2 + (∇ψT q)2]1/2
dt.
For shortest path or geodesic, minimize the path length l .
Question: What are the variables of the problem?
![Page 1429: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1429.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1429,
IntroductionEuler’s EquationDirect Methods
Introduction
Consider a particle moving on a smooth surface z = ψ(q1, q2).
With position r = [q1(t) q2(t) ψ(q1(t), q2(t))]T on the surface
and δr = [δq1 δq2 (∇ψ)T δq]T in the tangent plane, length of thepath from qi = q(ti) to qf = q(tf ) is
l =
∫
‖δr‖ =
∫ tf
ti
‖r‖dt =
∫ tf
ti
[
q21 + q2
2 + (∇ψT q)2]1/2
dt.
For shortest path or geodesic, minimize the path length l .
Question: What are the variables of the problem?
Answer: The entire curve or function q(t).
Variational problem:Optimization of a function of functions, i.e. a functional.
![Page 1430: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1430.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1430,
IntroductionEuler’s EquationDirect Methods
Introduction
Functionals and their extremization
Suppose that a candidate curve is represented as a sequence ofpoints qj = q(tj) at time instants
ti = t0 < t1 < t2 < t3 < · · · < tN−1 < tN = tf .
Geodesic problem: a multivariate optimization problem with the2(N − 1) variables in qj , 1 ≤ j ≤ N − 1.
With N →∞, we obtain the actual function.
![Page 1431: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1431.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1431,
IntroductionEuler’s EquationDirect Methods
Introduction
Functionals and their extremization
Suppose that a candidate curve is represented as a sequence ofpoints qj = q(tj) at time instants
ti = t0 < t1 < t2 < t3 < · · · < tN−1 < tN = tf .
Geodesic problem: a multivariate optimization problem with the2(N − 1) variables in qj , 1 ≤ j ≤ N − 1.
With N →∞, we obtain the actual function.
First order necessary condition: Functional is stationary withrespect to arbitrary small variations in qj.
[Equivalent to vanishing of the gradient]
![Page 1432: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1432.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1432,
IntroductionEuler’s EquationDirect Methods
Introduction
Functionals and their extremization
Suppose that a candidate curve is represented as a sequence ofpoints qj = q(tj) at time instants
ti = t0 < t1 < t2 < t3 < · · · < tN−1 < tN = tf .
Geodesic problem: a multivariate optimization problem with the2(N − 1) variables in qj , 1 ≤ j ≤ N − 1.
With N →∞, we obtain the actual function.
First order necessary condition: Functional is stationary withrespect to arbitrary small variations in qj.
[Equivalent to vanishing of the gradient]
This gives equations for the stationary points.
Here, these equations are differential equations!
![Page 1433: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1433.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1433,
IntroductionEuler’s EquationDirect Methods
IntroductionExamples of variational problems
Geodesic path: Minimize l =∫ b
a‖r′(t)‖dt
Minimal surface of revolution: MinimizeS =
∫
2πyds = 2π∫ b
ay√
1 + y ′2dxThe brachistochrone problem: To find the curve along which the
descent is fastest.
Minimize T =∫
dsv
=∫ b
a
√
1+y ′2
2gy dx
Fermat’s principle: Light takes the fastest path.
Minimize T =∫ u2
u1
√x ′2+y ′2+z ′2
c(x ,y ,z) duIsoperimetric problem: Largest area in the plane enclosed by a
closed curve of given perimeter. By extension,extremize a functional under one or more equalityconstraints.
Hamilton’s principle of least action: Evolution of a dynamicsystem through the minimization of the action
s =
∫ t2
t1
Ldt =
∫ t2
t1
(K − P)dt
![Page 1434: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1434.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1434,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Find out a function y(x), that will make the functional
I [y(x)] =
∫ x2
x1
f [x , y(x), y ′(x)]dx
stationary, with boundary conditions y(x1) = y1 and y(x2) = y2.
![Page 1435: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1435.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1435,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Find out a function y(x), that will make the functional
I [y(x)] =
∫ x2
x1
f [x , y(x), y ′(x)]dx
stationary, with boundary conditions y(x1) = y1 and y(x2) = y2.Consider variation δy(x) with δy(x1) = δy(x2) = 0 and consistentvariation δy ′(x).
δI =
∫ x2
x1
(
∂f
∂yδy +
∂f
∂y ′ δy′)
dx
![Page 1436: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1436.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1436,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Find out a function y(x), that will make the functional
I [y(x)] =
∫ x2
x1
f [x , y(x), y ′(x)]dx
stationary, with boundary conditions y(x1) = y1 and y(x2) = y2.Consider variation δy(x) with δy(x1) = δy(x2) = 0 and consistentvariation δy ′(x).
δI =
∫ x2
x1
(
∂f
∂yδy +
∂f
∂y ′ δy′)
dx
Integration of the second term by parts:∫ x2
x1
∂f
∂y ′ δy′dx =
∫ x2
x1
∂f
∂y ′d
dx(δy)dx =
[
∂f
∂y ′ δy
]x2
x1
−∫ x2
x1
d
dx
∂f
∂y ′ δy dx
With δy(x1) = δy(x2) = 0, the first term vanishes identically, and
δI =
∫ x2
x1
[
∂f
∂y− d
dx
∂f
∂y ′
]
δy dx .
![Page 1437: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1437.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1437,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
For δI to vanish for arbitrary δy(x),ddx
∂f∂y ′ − ∂f
∂y= 0.
![Page 1438: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1438.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1438,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
For δI to vanish for arbitrary δy(x),ddx
∂f∂y ′ − ∂f
∂y= 0.
Functions involving higher order derivatives
I [y(x)] =
∫ x2
x1
f(
x , y , y ′, y ′′, · · · , y (n))
dx
with prescribed boundary values for y , y ′, y ′′, · · · , y (n−1)
![Page 1439: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1439.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1439,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
For δI to vanish for arbitrary δy(x),ddx
∂f∂y ′ − ∂f
∂y= 0.
Functions involving higher order derivatives
I [y(x)] =
∫ x2
x1
f(
x , y , y ′, y ′′, · · · , y (n))
dx
with prescribed boundary values for y , y ′, y ′′, · · · , y (n−1)
δI =
∫ x2
x1
[
∂f
∂yδy +
∂f
∂y ′ δy′ +
∂f
∂y ′′ δy′′ + · · · + ∂f
∂y (n)δy (n)
]
dx
Working rule: Starting from the last term, integrate one term ata time by parts, using consistency of variations and BC’s.
![Page 1440: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1440.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1440,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
For δI to vanish for arbitrary δy(x),ddx
∂f∂y ′ − ∂f
∂y= 0.
Functions involving higher order derivatives
I [y(x)] =
∫ x2
x1
f(
x , y , y ′, y ′′, · · · , y (n))
dx
with prescribed boundary values for y , y ′, y ′′, · · · , y (n−1)
δI =
∫ x2
x1
[
∂f
∂yδy +
∂f
∂y ′ δy′ +
∂f
∂y ′′ δy′′ + · · · + ∂f
∂y (n)δy (n)
]
dx
Working rule: Starting from the last term, integrate one term ata time by parts, using consistency of variations and BC’s.Euler’s equation:
∂f
∂y− d
dx
∂f
∂y ′ +d2
dx2
∂f
∂y ′′ − · · ·+ (−1)ndn
dxn
∂f
∂y (n)= 0,
an ODE of order 2n, in general.
![Page 1441: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1441.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1441,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of a vector function
I [r(t)] =
∫ t2
t1
f (t, r, r)dt
![Page 1442: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1442.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1442,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of a vector function
I [r(t)] =
∫ t2
t1
f (t, r, r)dt
In terms of partial gradients ∂f∂r and ∂f
∂ r ,
δI =
∫ t2
t1
[
(
∂f
∂r
)T
δr +
(
∂f
∂r
)T
δr
]
dt
=
∫ t2
t1
(
∂f
∂r
)T
δrdt +
[
(
∂f
∂r
)T
δr
]t2
t1
−∫ t2
t1
d
dt
(
∂f
∂r
)T
δrdt
=
∫ t2
t1
[
∂f
∂r− d
dt
∂f
∂r
]T
δrdt.
![Page 1443: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1443.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1443,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of a vector function
I [r(t)] =
∫ t2
t1
f (t, r, r)dt
In terms of partial gradients ∂f∂r and ∂f
∂ r ,
δI =
∫ t2
t1
[
(
∂f
∂r
)T
δr +
(
∂f
∂r
)T
δr
]
dt
=
∫ t2
t1
(
∂f
∂r
)T
δrdt +
[
(
∂f
∂r
)T
δr
]t2
t1
−∫ t2
t1
d
dt
(
∂f
∂r
)T
δrdt
=
∫ t2
t1
[
∂f
∂r− d
dt
∂f
∂r
]T
δrdt.
Euler’s equation: a system of second order ODE’s
d
dt
∂f
∂r− ∂f
∂r= 0 or
d
dt
∂f
∂ ri− ∂f
∂ri= 0 for each i .
![Page 1444: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1444.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1444,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
![Page 1445: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1445.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1445,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
Euler’s equation: ∂∂x
∂f∂ux
+ ∂∂y
∂f∂uy− ∂f
∂u= 0
![Page 1446: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1446.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1446,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
Euler’s equation: ∂∂x
∂f∂ux
+ ∂∂y
∂f∂uy− ∂f
∂u= 0
Moving boundariesRevision of the basic case: allowing non-zero δy(x1), δy(x2)
![Page 1447: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1447.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1447,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
Euler’s equation: ∂∂x
∂f∂ux
+ ∂∂y
∂f∂uy− ∂f
∂u= 0
Moving boundariesRevision of the basic case: allowing non-zero δy(x1), δy(x2)
At an end-point, ∂f∂y ′ δy has to vanish for arbitrary δy(x).
∂f∂y ′ vanishes at the boundary.
Euler boundary condition or natural boundary condition
![Page 1448: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1448.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1448,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
Euler’s equation: ∂∂x
∂f∂ux
+ ∂∂y
∂f∂uy− ∂f
∂u= 0
Moving boundariesRevision of the basic case: allowing non-zero δy(x1), δy(x2)
At an end-point, ∂f∂y ′ δy has to vanish for arbitrary δy(x).
∂f∂y ′ vanishes at the boundary.
Euler boundary condition or natural boundary condition
Equality constraints and isoperimetric problems
Minimize I =∫ x2
x1f (x , y , y ′)dx subject to J =
∫ x2
x1g(x , y , y ′)dx = J0.
In another level of generalization, constraint φ(x , y , y ′) = 0.
![Page 1449: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1449.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1449,
IntroductionEuler’s EquationDirect Methods
Euler’s Equation
Functionals of functions of several variables
I [u(x , y)] =
∫
D
∫
f (x , y , u, ux , uy )dx dy
Euler’s equation: ∂∂x
∂f∂ux
+ ∂∂y
∂f∂uy− ∂f
∂u= 0
Moving boundariesRevision of the basic case: allowing non-zero δy(x1), δy(x2)
At an end-point, ∂f∂y ′ δy has to vanish for arbitrary δy(x).
∂f∂y ′ vanishes at the boundary.
Euler boundary condition or natural boundary condition
Equality constraints and isoperimetric problems
Minimize I =∫ x2
x1f (x , y , y ′)dx subject to J =
∫ x2
x1g(x , y , y ′)dx = J0.
In another level of generalization, constraint φ(x , y , y ′) = 0.
Operate with f ∗(x , y , y ′, λ) = f (x , y , y ′) + λ(x)g(x , y , y ′).
![Page 1450: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1450.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1450,
IntroductionEuler’s EquationDirect Methods
Direct MethodsFinite difference methodWith given boundary values y(a) and y(b),
I [y(x)] =
∫ b
a
f [x , y(x), y ′(x)]dx
Represent y(x) by its values over xi = a + ih withi = 0, 1, 2, · · · ,N, where b − a = Nh.
Approximate the functional by
I [y(x)] ≈ φ(y1, y2, y3, · · · , yN−1) =
N∑
i=1
f (xi , yi , y′i )h,
where xi =xi+xi−1
2 , yi =yi+yi−1
2 and y ′i =
yi−yi−1
h.
Minimize φ(y1, y2, y3, · · · , yN−1) with respect to yi ;for example, by solving ∂φ
∂yi= 0 for all i .
![Page 1451: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1451.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1451,
IntroductionEuler’s EquationDirect Methods
Direct MethodsFinite difference methodWith given boundary values y(a) and y(b),
I [y(x)] =
∫ b
a
f [x , y(x), y ′(x)]dx
Represent y(x) by its values over xi = a + ih withi = 0, 1, 2, · · · ,N, where b − a = Nh.
Approximate the functional by
I [y(x)] ≈ φ(y1, y2, y3, · · · , yN−1) =
N∑
i=1
f (xi , yi , y′i )h,
where xi =xi+xi−1
2 , yi =yi+yi−1
2 and y ′i =
yi−yi−1
h.
Minimize φ(y1, y2, y3, · · · , yN−1) with respect to yi ;for example, by solving ∂φ
∂yi= 0 for all i .
Exercise: Show that ∂φ∂yi
= 0 is equivalent to Euler’s equation.
![Page 1452: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1452.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1452,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Rayleigh-Ritz methodIn terms of a set of basis functions, express the solution as
y(x) =
N∑
i=1
αiwi (x).
Represent functional I [y(x)] as a multivariate function φ(α).
Optimize φ(α) to determine αi ’s.
![Page 1453: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1453.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1453,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Rayleigh-Ritz methodIn terms of a set of basis functions, express the solution as
y(x) =
N∑
i=1
αiwi (x).
Represent functional I [y(x)] as a multivariate function φ(α).
Optimize φ(α) to determine αi ’s.
Note: As N →∞, the numerical solution approaches exactitude.For a particular tolerance, one can truncate appropriately.
![Page 1454: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1454.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1454,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Rayleigh-Ritz methodIn terms of a set of basis functions, express the solution as
y(x) =
N∑
i=1
αiwi (x).
Represent functional I [y(x)] as a multivariate function φ(α).
Optimize φ(α) to determine αi ’s.
Note: As N →∞, the numerical solution approaches exactitude.For a particular tolerance, one can truncate appropriately.
Observation: With these direct methods, no need to reduce thevariational (optimization) problem to Euler’s equation!
![Page 1455: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1455.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1455,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Rayleigh-Ritz methodIn terms of a set of basis functions, express the solution as
y(x) =
N∑
i=1
αiwi (x).
Represent functional I [y(x)] as a multivariate function φ(α).
Optimize φ(α) to determine αi ’s.
Note: As N →∞, the numerical solution approaches exactitude.For a particular tolerance, one can truncate appropriately.
Observation: With these direct methods, no need to reduce thevariational (optimization) problem to Euler’s equation!
Question: Is it possible to reformulate a BVP as a variationalproblem and then use a direct method?
![Page 1456: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1456.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1456,
IntroductionEuler’s EquationDirect Methods
Direct MethodsThe inverse problem: From
I [y(x)] ≈ φ(α) =
∫ b
a
f
(
x ,N∑
i=1
αiwi (x),N∑
i=1
αiw′i (x)
)
dx ,
![Page 1457: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1457.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1457,
IntroductionEuler’s EquationDirect Methods
Direct MethodsThe inverse problem: From
I [y(x)] ≈ φ(α) =
∫ b
a
f
(
x ,N∑
i=1
αiwi (x),N∑
i=1
αiw′i (x)
)
dx ,
∂φ
∂αi
=
Z
b
a
2
4
∂f
∂y
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A wi (x) +∂f
∂y′
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A w′
i (x)
3
5 dx.
![Page 1458: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1458.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1458,
IntroductionEuler’s EquationDirect Methods
Direct MethodsThe inverse problem: From
I [y(x)] ≈ φ(α) =
∫ b
a
f
(
x ,N∑
i=1
αiwi (x),N∑
i=1
αiw′i (x)
)
dx ,
∂φ
∂αi
=
Z
b
a
2
4
∂f
∂y
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A wi (x) +∂f
∂y′
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A w′
i (x)
3
5 dx.
Integrating the second term by parts and using wi (a) = wi (b) = 0,
∂φ
∂αi
=
∫ b
a
R[
N∑
i=1
αiwi
]
wi (x)dx ,
where R[y ] ≡ ∂f∂y− d
dx∂f∂y ′ = 0 is the Euler’s equation of the
variational problem.
![Page 1459: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1459.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1459,
IntroductionEuler’s EquationDirect Methods
Direct MethodsThe inverse problem: From
I [y(x)] ≈ φ(α) =
∫ b
a
f
(
x ,N∑
i=1
αiwi (x),N∑
i=1
αiw′i (x)
)
dx ,
∂φ
∂αi
=
Z
b
a
2
4
∂f
∂y
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A wi (x) +∂f
∂y′
0
@x,
NX
i=1
αi wi ,
NX
i=1
αi w′
i
1
A w′
i (x)
3
5 dx.
Integrating the second term by parts and using wi (a) = wi (b) = 0,
∂φ
∂αi
=
∫ b
a
R[
N∑
i=1
αiwi
]
wi (x)dx ,
where R[y ] ≡ ∂f∂y− d
dx∂f∂y ′ = 0 is the Euler’s equation of the
variational problem.Def.: R[z(x)]: residual of the differential equation R[y ] = 0operated over the function z(x)
Residual of the Euler’s equation of a variational problemoperated upon the solution obtained by Rayleigh-Ritzmethod is orthogonal to basis functions wi(x).
![Page 1460: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1460.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1460,
IntroductionEuler’s EquationDirect Methods
Direct MethodsGalerkin methodQuestion: What if we cannot find a ‘corresponding’ variationalproblem for the differential equation?Answer: Work with the residual directly and demand
∫ b
a
R[z(x)]wi (x)dx = 0.
![Page 1461: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1461.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1461,
IntroductionEuler’s EquationDirect Methods
Direct MethodsGalerkin methodQuestion: What if we cannot find a ‘corresponding’ variationalproblem for the differential equation?Answer: Work with the residual directly and demand
∫ b
a
R[z(x)]wi (x)dx = 0.
Freedom to choose two different families of functions as basisfunctions ψj (x) and trial functions wi (x):
∫ b
a
R
∑
j
αjψj(x)
wi(x)dx = 0
![Page 1462: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1462.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1462,
IntroductionEuler’s EquationDirect Methods
Direct MethodsGalerkin methodQuestion: What if we cannot find a ‘corresponding’ variationalproblem for the differential equation?Answer: Work with the residual directly and demand
∫ b
a
R[z(x)]wi (x)dx = 0.
Freedom to choose two different families of functions as basisfunctions ψj (x) and trial functions wi (x):
∫ b
a
R
∑
j
αjψj(x)
wi(x)dx = 0
A singular case of the Galerkin method:
delta functions, at discrete points, as trial functions
![Page 1463: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1463.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1463,
IntroductionEuler’s EquationDirect Methods
Direct MethodsGalerkin methodQuestion: What if we cannot find a ‘corresponding’ variationalproblem for the differential equation?Answer: Work with the residual directly and demand
∫ b
a
R[z(x)]wi (x)dx = 0.
Freedom to choose two different families of functions as basisfunctions ψj (x) and trial functions wi (x):
∫ b
a
R
∑
j
αjψj(x)
wi(x)dx = 0
A singular case of the Galerkin method:
delta functions, at discrete points, as trial functions
Satisfaction of the differential equation exactly at the chosenpoints, known as collocation points:
Collocation method
![Page 1464: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1464.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1464,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Finite element methods
discretization of the domain into elements of simple geometry
basis functions of low order polynomials with local scope
design of basis functions so as to achieve enough order ofcontinuity or smoothness across element boundaries
piecewise continuous/smooth basis functions for entiredomain, with a built-in sparse structure
some weighted residual method to frame the algebraicequations
solution gives coefficients which are actually the nodal values
![Page 1465: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1465.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1465,
IntroductionEuler’s EquationDirect Methods
Direct Methods
Finite element methods
discretization of the domain into elements of simple geometry
basis functions of low order polynomials with local scope
design of basis functions so as to achieve enough order ofcontinuity or smoothness across element boundaries
piecewise continuous/smooth basis functions for entiredomain, with a built-in sparse structure
some weighted residual method to frame the algebraicequations
solution gives coefficients which are actually the nodal values
Suitability of finite element analysis in software environments
effectiveness and efficiency
neatness and modularity
![Page 1466: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1466.jpg)
Mathematical Methods in Engineering and Science Variational Calculus* 1466,
IntroductionEuler’s EquationDirect Methods
Points to note
Optimization with respect to a function
Concept of a functional
Euler’s equation
Rayleigh-Ritz and Galerkin methods
Optimization and equation-solving in the infinite-dimensionalfunction space: practical methods and connections
Necessary Exercises: 1,2,4,5
![Page 1467: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1467.jpg)
Mathematical Methods in Engineering and Science Epilogue 1467,
Outline
Epilogue
![Page 1468: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1468.jpg)
Mathematical Methods in Engineering and Science Epilogue 1468,
Epilogue
Source for further information:
http://home.iitk.ac.in/˜ dasgupta/MathBook
Destination for feedback:
![Page 1469: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1469.jpg)
Mathematical Methods in Engineering and Science Epilogue 1469,
Epilogue
Source for further information:
http://home.iitk.ac.in/˜ dasgupta/MathBook
Destination for feedback:
Some general courses in immediate continuation
Advanced Mathematical Methods
Scientific Computing
Advanced Numerical Analysis
Optimization
Advanced Differential Equations
Partial Differential Equations
Finite Element Methods
![Page 1470: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1470.jpg)
Mathematical Methods in Engineering and Science Epilogue 1470,
Epilogue
Some specialized courses in immediate continuation
Linear Algebra and Matrix Theory
Approximation Theory
Variational Calculus and Optimal Control
Advanced Mathematical Physics
Geometric Modelling
Computational Geometry
Computer Graphics
Signal Processing
Image Processing
![Page 1471: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1471.jpg)
Mathematical Methods in Engineering and Science Selected References 1471,
Outline
Selected References
![Page 1472: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1472.jpg)
Mathematical Methods in Engineering and Science Selected References 1472,
Selected References I
F. S. Acton.Numerical Methods that usually Work.The Mathematical Association of America (1990).
C. M. Bender and S. A. Orszag.Advanced Mathematical Methods for Scientists and Engineers.
Springer-Verlag (1999).
G. Birkhoff and G.-C. Rota.Ordinary Differential Equations.John Wiley and Sons (1989).
G. H. Golub and C. F. Van Loan.Matrix Computations.The John Hopkins University Press (1983).
![Page 1473: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1473.jpg)
Mathematical Methods in Engineering and Science Selected References 1473,
Selected References II
M. T. Heath.Scientific Computing .Tata McGraw-Hill Co. Ltd (2000).
E. Kreyszig.Advanced Engineering Mathematics.John Wiley and Sons (2002).
E. V. Krishnamurthy and S. K. Sen.Numerical Algorithms.Affiliated East-West Press Pvt Ltd (1986).
D. G. Luenberger.Linear and Nonlinear Programming .Addison-Wesley (1984).
P. V. O’Neil.Advanced Engineering Mathematics.Thomson Books (2004).
![Page 1474: Mathematical Methods in Engineering and Science](https://reader031.vdocument.in/reader031/viewer/2022012320/5540b2634a7959e10e8b4ad1/html5/thumbnails/1474.jpg)
Mathematical Methods in Engineering and Science Selected References 1474,
Selected References III
W. H. Press, S. A. Teukolsky, W. T. Vellerling and B. P.Flannery.Numerical Recipes.Cambridge University Press (1998).
G. F. Simmons.Differential Equations with Applications and Historical Notes.Tata McGraw-Hill Co. Ltd (1991).
J. Stoer and R. Bulirsch.Introduction to Numerical Analysis.Springer-Verlag (1993).
C. R. Wylie and L. C. Barrett.Advanced Engineering Mathematics.Tata McGraw-Hill Co. Ltd (2003).