Properties of Matrix Operations

 

Properties of Addition

The basic properties of addition for real numbers also hold true for matrices. 

Let A, B and C be m x n matrices

1. A + B  =  B + A    commutative

2. A + (B + C)  =  (A + B) + C    associative3. There is a unique m x n matrix O with

        A + O  =  A        additive identity4. For any  m x n matrix A there is an m x n matrix B (called -A) with

       A + B  =  O        additive inverse

 

The proofs are all similar.  We will prove the first property.

 

Proof of Property 1

We have

        (A + B)ij  =  Aij + Bij     definition of addition of matrices

        =  Bij + Aij         commutative property of addition for real numbers

       =  (B + A)ij       definition of addition of matrices

Notice that the zero matrix is different for different m and n.  For example

       

Properties of Matrix Multiplication

Unlike matrix addition, the properties of multiplication of real numbers do not all generalize to matrices.  Matrices rarely commute even if AB and BA are both defined.  There often is no multiplicative inverse of a matrix, even if the matrix is a square matrix.  There are a few properties of multiplication of real numbers that generalize to matrices.  We state them now.

Let A, B and C be matrices of dimensions such that the following are defined.  Then

1. A(BC)  =  (AB)C                 associative

2. A(B + C)  =  AB + AC        distributive3. (A + B)C  =  AC + BC        distributive4. There are unique matrices Im and In with

        Im A  =  A In  =  A        multiplicative identity

We will often omit the subscript and write I for the identity matrix.  The identity matrix is a square scalar matrix with 1's along the diagonal.  For example

       

We will prove the second property and leave the rest for you.

 

Proof of Property 2

Again we show that the general element of the left hand side is the same as the right hand side.  We have

        (A(B + C))ij  = (Aik(B + C)kj)        definition of matrix multiplication

        =  (Aik(Bkj + Ckj))        definition of matrix addition

        =  (AikBkj + AikCkj)       distributive property of the real numbers

        =  AikBkj + AikCkj     commutative property of the real numbers

        =  (AB)ij + (AC)ij        definition of matrix multiplication

where the sum is taken from 1 to k.

Example

We will demonstrate property 1 with

       

We have

       

so that

       

We have

       

so that

       

Properties of Scalar Multiplication

Since we can multiply a matrix by a scalar, we can investigate the properties that this multiplication has.  All of the properties of multiplication of real numbers generalize.  In particular, we have

Let r and s be real numbers and A and B be matrices.  Then

1. r(sA)  =  (rs)A

   2. (r + s)A  =  rA + sA3. r(A + B)  =  rA + rB4. A(rB)  =  r(AB)  =  (rA)B

 

We will prove property 3 and leave the rest for you.  We have

        (r(A + B))ij  =  (r)(A + B)ij          definition of scalar multiplication

        =  (r)(Aij + Bij)        definition of addition of matrices

        =  rAij + rBij        distributive property of the real numbers

        =  (rA)ij + (rB)ij        definition of scalar multiplication

        =  (rA + rB)ij        definition of addition of matrices

Properties of the Transpose of a Matrix

Recall that the transpose of a matrix is the operation of switching rows and columns.  We state the following properties.  We proved the first property in the last section.

Let r be a real number and A and B be matrices.  Then

1. (AT)T  =  A

2. (A + B)T  =  AT + BT

3. (AB)T  =  BTAT

4. (rA)T  =  rAT

Transpose

The transpose of a matrix is a new matrix whose rows are the columns of the original (which makes its columns the rows of the original). Here is a matrix and its transpose:

The superscript "T" means "transpose". Another way to look at the transpose is that the element at row r column c in the original is placed at row c column r of the transpose. The element arc of the original matrix becomes element acr in the transposed matrix.

Usually we will work with square matrices, and it is usually square matrices that will be transposed. However, non-square matrices can be transposed, as well:

Transpose of a Matrix

A matrix which is formed by turning all the rows of a given matrix into columns and vice-versa. The transpose of matrix A is written AT.

 

Inverse of a MatrixMatrix Inverse Multiplicative Inverse of a Matrix

For a square matrix A, the inverse is written A-1. When A is multiplied by A-1 the result is the identity matrix I. Non-square matrices do not have inverses.

Note: Not all square matrices have inverses. A square matrix which has an inverse is called invertible or nonsingular, and a square matrix without an inverse is called noninvertible or singular.

 

AA-1 = A-1A = I

Example: For matrix , its inverse is since

  AA-1 =

and A-1A = .

 

Here are three ways to find the inverse of a matrix:

1. Shortcut for 2x2 matrices

For , the inverse can be found using this formula:

Example:

2. Augmented matrix method

Use Gauss-Jordan elimination to transform [ A | I ] into [ I | A-1 ].

Example: The following steps result in .

so we see that .

3. Adjoint method

A-1 = (adjoint of A)   or   A-1 = (cofactor matrix of A)T

Example: The following steps result in A-1 for .

The cofactor matrix for A is , so the adjoint is

. Since det A = 22, we get

.

Matrix Inverse

The inverse of a square matrix , sometimes called a reciprocal matrix, is a matrix such that

(1)

where is the identity matrix. Courant and Hilbert (1989, p. 10) use the notation to denote the inverse matrix.

A square matrix has an inverse iff the determinant (Lipschutz 1991, p. 45). A matrix possessing an inverse is called nonsingular, or invertible.

The matrix inverse of a square matrix may be taken in Mathematica using the function Inverse[m].

For a matrix

(2)

the matrix inverse is

(3) (4)

For a matrix

(5)

the matrix inverse is

(6)

A general matrix can be inverted using methods such as the Gauss-Jordan elimination, Gaussian elimination, or LU decomposition.

The inverse of a product of matrices and can be expressed in terms of and . Let

(7)

Then

(8)

and

(9)

Therefore,

(10)

so

(11)

where is the identity matrix, and

(12)

SEE ALSO: Gauss-Jordan Elimination, Gaussian Elimination, LU Decomposition, Matrix, Matrix 1-Inverse, Matrix Addition, Matrix Multiplication, Moore-Penrose Matrix Inverse, Nonsingular Matrix, Pseudoinverse, Singular Matrix, Strassen Formulas REFERENCES:

Ayres, F. Jr. Schaum's Outline of Theory and Problems of Matrices. New York: Schaum, p. 11, 1962.

Ben-Israel, A. and Greville, T. N. E. Generalized Inverses: Theory and Applications. New York: Wiley, 1977.

Courant, R. and Hilbert, D. Methods of Mathematical Physics, Vol.   1. New York: Wiley, 1989.

Jodár, L.; Law, A. G.; Rezazadeh, A.; Watson, J. H.; and Wu, G. "Computations for the Moore-Penrose and Other Generalized Inverses." Congress. Numer. 80, 57-64, 1991.

Lipschutz, S. "Invertible Matrices." Schaum's Outline of Theory and Problems of Linear Algebra, 2nd ed. New York: McGraw-Hill, pp. 44-45, 1991.

Nash, J. C. Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation, 2nd ed. Bristol, England: Adam Hilger, pp. 24-26, 1990.

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. "Is Matrix Inversion an Process?" §2.11 in Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 95-98, 1992.

Rosser, J. B. "A Method of Computing Exact Inverses of Matrices with Integer Coefficients." J. Res. Nat. Bur. Standards Sect. B. 49, 349-358, 1952.

Referenced on Wolfram|Alpha: Matrix Inverse

TransposeThe object obtained by replacing all elements with . For a second-tensor rank tensor , the tensor transpose is simply . The matrix transpose, most commonly written , is the matrix obtained by exchanging 's rows and columns, and satisfies the identity

(1)

Unfortunately, several other notations are commonly used, as summarized in the following table. The notation is used in this work.

notation referencesThis work; Golub and Van Loan (1996), Strang (1988)Arfken (1985, p. 201), Griffiths (1987, p. 223)Ayres (1962, p. 11), Courant and Hilbert (1989, p. 9)

The transpose of a matrix or tensor is implemented in Mathematica as Transpose[A].

The product of two transposes satisfies

(2) (3) (4) (5) (6

)

where Einstein summation has been used to implicitly sum over repeated indices. Therefore,

(7)

SEE ALSO: Antisymmetric Matrix, Congruent Matrices, Conjugate Matrix, Conjugate Transpose, Symmetric Matrix REFERENCES:

Arfken, G. Mathematical Methods for Physicists, 3rd ed. Orlando, FL: Academic Press, p. 201, 1985.

Ayres, F. Jr. Schaum's Outline of Theory and Problems of Matrices. New York: Schaum, pp. 11-12, 1962.

Boothroyd, J. "Algorithm 302: Transpose Vector Stored Array." Comm. ACM 10, 292-293, May 1967.

Brenner, N. "Algorithm 467: Matrix Transposition N Place [F1]." Comm. ACM 16, 692-694, Nov. 1973.

Cate, E. G. and Twigg, D. W. "Algorithm 513: Analysis of In-Situ Transposition." ACM Trans. Math. Software 3, 104-110, March 1977.

Courant, R. and Hilbert, D. Methods of Mathematical Physics, Vol.   1. New York: Wiley, 1989.

Golub, G. H. and Van Loan, C. F. Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins, 1989.

Griffiths, D. J. Introduction to Elementary Particles. New York: Wiley, p. 220, 1987.

Knuth, D. E. "Transposing a Rectangular Matrix." Ch. 1.3.3 Ex. 12. The Art of Computer Programming, Vol.   1: Fundamental Algorithms, 3rd ed. Reading, MA: Addison-Wesley, pp. 182 and 523, 1997.

Laflin, S. and Brebner, M. A. "Algorithm 380: In-Situ Transposition of a Rectangular Matrix. [F1]." Comm. ACM 13, 324-326, May 1970.

Strang, G. Introduction to Linear Algebra. Wellesley, MA: Wellesley-Cambridge Press, 1993.

Strang, G. Linear Algebra and its Applications, 3rd ed. Philadelphia, PA: Saunders, 1988.

Windley, P. F. "Transposing Matrices in a Digital Computer." Computer J. 2, 47-48, Apr. 1959.

Referenced on Wolfram|Alpha: Transpose

Matrix

A matrix is a concise and useful way of uniquely representing and working with linear transformations. In particular, every linear transformation can be represented by a matrix, and every matrix corresponds to a unique linear transformation. The matrix, and its close relative the determinant, are extremely important concepts in linear algebra, and were first formulated by Sylvester (1851) and Cayley.

In his 1851 paper, Sylvester wrote, "For this purpose we must commence, not with a square, but with an oblong arrangement of terms consisting, suppose, of lines and columns. This will not in itself represent a determinant, but is, as it were, a Matrix out of which we may form various systems of determinants by fixing upon a number , and selecting at will lines and columns, the squares corresponding of th order." Because Sylvester was interested in the determinant formed from the rectangular array of number and not the array itself (Kline 1990, p. 804), Sylvester used the term "matrix" in its conventional usage to mean "the place from which something else originates" (Katz 1993). Sylvester (1851) subsequently used the term matrix informally, stating "Form the rectangular matrix consisting of rows and columns.... Then all the determinants that can be formed by rejecting any one column at pleasure out of this matrix are identically zero." However, it remained up to Sylvester's collaborator Cayley to use the terminology in its modern form in papers of 1855 and 1858 (Katz 1993).

In his 1867 treatise on determinants, C. L. Dodgson (Lewis Carroll) objected to the use of the term "matrix," stating, "I am aware that the word 'Matrix' is already in use to express the very meaning for which I use the word 'Block'; but surely the former word means rather the mould, or form, into which algebraical quantities may be introduced, than an actual assemblage of such quantities...." However, Dodgson's objections have passed unheeded and the term "matrix" has stuck.

The transformation given by the system of equations

(1) (2) (3) (4)

is represented as a matrix equation by

(5)

where the are called matrix elements.

An matrix consists of rows and columns, and the set of matrices with real coefficients is sometimes denoted . To remember which index refers to which direction, identify the indices of the last (i.e., lower right) term, so the indices of the last element in the above matrix identifies it as an matrix.

A matrix is said to be square if , and rectangular if . An matrix is called a column vector, and a matrix is called a row vector. Special types of square matrices include the identity matrix , with (where is the Kronecker delta) and the diagonal matrix (where are a set of constants).

In this work, matrices are represented using square brackets as delimiters, but in the general literature, they are more commonly delimited using parentheses. This latter convention introduces the unfortunate notational ambiguity between matrices of the

form and the binomial coefficient

(6)

When referenced symbolically in this work, matrices are denoted in a sans serif font, e.g, , , etc. In this concise notation, the transformation given in equation (5) can be written

(7)

where and are vectors and is a matrix. A number of other notational conventions also exist, with some authors preferring an italic typeface.

It is sometimes convenient to represent an entire matrix in terms of its matrix elements. Therefore, the th element of the matrix could be written , and the

matrix composed of entries could be written as , or simply for short.

Two matrices may be added (matrix addition) or multiplied (matrix multiplication) together to yield a new matrix. Other common operations on a single matrix are matrix diagonalization, matrix inversion, and transposition.

The determinant or of a matrix is a very important quantity which appears in many diverse applications. The sum of the diagonal elements of a square matrix is known as the matrix trace and is also an important quantity in many sorts of computations.

SEE ALSO: Adjacency Matrix, Adjoint, Alternating Sign Matrix, Antisymmetric Matrix, Block Matrix, Bohr Matrix, Bourque-Ligh Conjecture, Cartan Matrix, Circulant Matrix, Condition Number, Cramer's Rule, Determinant, Diagonal Matrix, Dirac Matrices, Eigen Decomposition Theorem, Eigenvector, Elementary Matrix, Elementary Row and Column Operations, Equivalent Matrix, Fourier Matrix, Gram Matrix, Hilbert Matrix, Hypermatrix, Identity Matrix, Ill-Conditioned Matrix, Incidence Matrix, Irreducible Matrix, Kac Matrix, Least Common Multiple Matrix, LU Decomposition, Matrix Addition, Matrix Inverse, Matrix Multiplication, Matrix Trace, McCoy's Theorem, Minimal Matrix, Normal Matrix, Pauli Matrices, Permutation Matrix, Positive Definite Matrix, Random Matrix, Rational Canonical Form, Reducible Matrix, Roth's Removal Rule, Shear Matrix, Singular Matrix, Smith Normal Form, Sparse Matrix, Special Matrix, Square Matrix, Stochastic Matrix, Submatrix, Symmetric Matrix, Tournament Matrix REFERENCES:

Arfken, G. "Matrices." §4.2 in Mathematical Methods for Physicists, 3rd ed. Orlando, FL: Academic Press, pp. 176-191, 1985.

Bapat, R. B. Linear Algebra and Linear Models, 2nd ed. New York: Springer-Verlag, 2000.

Dodgson, C. L. An Elementary Treatise on Determinants, with Their Application to Simultaneous Linear Equations and Algebraical Geometry. London: Macmillan, 1867.

Frazer, R. A.; Duncan, W. J.; and Collar, A. R. Elementary Matrices and Some Applications to Dynamics and Differential Equations. Cambridge, England: Cambridge University Press, 1955.

Katz, V. J. A History of Mathematics. An Introduction. New York: HarperCollins, 1993.

Kline, M. Mathematical Thought from Ancient to Modern Times. Oxford, England: Oxford University Press, 1990.

Lütkepohl, H. Handbook of Matrices. New York: Wiley, 1996.

Meyer, C. D. Matrix Analysis and Applied Linear Algebra. Philadelphia, PA: SIAM, 2000.

Sylvester, J. J. "Additions to the Articles 'On a New Class of Theorems' and 'On Pascal's Theorem.' " Philos. Mag., 363-370, 1850. Reprinted in J. J. Sylvester's Mathematical Papers, Vol. 1. Cambridge, England: At the University Press, pp. 145-151, 1904.

Sylvester, J. J. An Essay on Canonical Forms, Supplement to a Sketch of a Memoir on Elimination, Transformation and Canonical Forms. London, 1851. Reprinted in J. J. Sylvester's Collected Mathematical Papers, Vol. 1. Cambridge, England: At the University Press, p. 209, 1904.

Wolfram, S. A New Kind of Science. Champaign, IL: Wolfram Media, p. 1168, 2002.

Zhang, F. Matrix Theory: Basic Results and Techniques. New York: Springer-Verlag, 1999.

Referenced on Wolfram|Alpha: Matrix