Report

Matrix Algebra Matrix algebra is a means of expressing large numbers of calculations made upon ordered sets of numbers. Often referred to as Linear Algebra Many equations would be completely intractable if scalar mathematics had to be used. It is also important to note that the scalar algebra is under there somewhere. Definitions - scalar scalar - a number denoted with regular type as is scalar algebra [1] or [a] Definitions - vector vector - a single row or column of numbers denoted with bold small letters row vector a = 1 2 3 4 5 column vector x = x1 x 2 x3 x4 x5 Definitions - Matrix A matrix is a set of rows and columns of numbers 1 2 3 4 5 6 Denoted with a bold Capital letter All matrices (and vectors) have an order - that is the number of rows x the number of columns. Thus A = 1 2 3 4 5 6 2 x 3 Matrix Equality Two matrices are equal iff (if and only if) all of their elements are identical Note: your data set is a matrix. Matrix Operations Addition and Subtraction Multiplication Transposition Inversion Addition and Subtraction Two matrices may be added iff they are the same order. Simply add the corresponding elements a11 a12 b11 a b a 22 21 21 a31 a32 b31 b12 c11 c12 b22 c 21 c 22 b32 c31 c32 Addition and Subtraction (cont.) Where a11 b11 c11 a12 b12 c12 a21 b21 c 21 a22 b22 c 22 a31 b31 c31 a32 b32 c32 Hence 1 2 4 6 5 8 3 4 4 6 7 10 5 6 4 6 9 12 Scalar Multiplication To multiply a scalar times a matrix, simply multiply each element of the matrix by the scalar quantity a11 a12 2a11 2 a21 a22 2a21 2a12 2a22 Matrix Multiplication (cont.) To multiply a matrix times a matrix, we write A times B as AB This is pre-multiplying multiplying A by B. B by A, or post- Matrix Multiplication (cont.) In order to multiply matrices, they must be conformable (the number of columns in A must equal the number of rows in B.) an (mxn) x (nxp) = (mxp) an (mxn) x (pxn) = cannot be done a (1xn) x (nx1) = a scalar (1x1) Matrix Multiplication (cont.) The general rule for matrix multiplication is: N cij aik bkj wherei 1,2,...,M , and j 1,2,...,P k 1 Matrix multiplication is not Commutative AB does not necessarily equal BA (BA may even be an impossible operation) Yet matrix multiplication is Associative A(BC) = (AB)C Special matrices There are a number of special matrices Square Diagonal Symmetric Null Identity Square matrix A square matrix is just what it sounds like, an nxn matrix a11 a 21 a31 a41 a12 a13 a22 a23 a32 a33 a42 a43 a14 a24 a34 a44 Square matrices are quite useful for describing the properties or interrelationships among a set of things – like the correlation matric for your dataset. Diagonal Matrices A diagonal matrix is a square matrix that has values on the diagonal with all offdiagonal entities being zero. a11 0 0 a 22 0 0 0 0 0 0 a33 0 0 0 0 a44 Symmetric Matrix All of the elements in the upper right portion of the matrix are identical to those in the lower left. For example, the correlation matrix Identity Matrix The identity matrix I is a diagonal matrix where the diagonal elements all equal one. It is used in a fashion analogous to multiplying through by "1" in scalar math. 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 Null Matrix A square matrix where all elements equal zero. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Not usually ‘used’ so much as sometimes the result of a calculation. Analogous to “a+b=0” The Transpose of a Matrix A' Taking the transpose is an operation that creates a new matrix based on an existing one. The rows of A = the columns of A' Hold upper left and lower right corners and rotate 180 degrees. Example of a transpose 1 4 1 2 3 A 2 5, A' 4 5 6 3 6 The Transpose of a Matrix A' If A = A', then A is symmetric (i.e. correlation matrix) If AA’ = A then A' is idempotent The transpose of a sum = sum of transposes (and A' = A) Idempotent means applying the same operation gives invariant results – e.g. 1 x 1 = 1 (see further…) (A + B + C)’ = A’ + B’ + C’ The transpose of a product = the product of the transposes in reverse order (ABC)’ = C’B’A’ An example: Suppose that you wish to obtain the sum of squared errors from the vector e. Simply pre-multiply e by its transpose e'. which, in matrices looks like e' e e e2 ..en 2 1 2 2 An example - cont Since the matrix product is a scalar found by summing the elements of the vector squared. The Determinant of a Matrix The determinant of a matrix A is denoted by |A|. Determinants exist only for square matrices. They are a matrix characteristic, and they are also difficult to compute The Determinant for a 2x2 matrix If A = Then a11 a 21 a12 a22 A a11a22 a12a21 That one is easy The Determinant for a 3x3 matrix If A = a11 a 21 a31 a12 a22 a32 a13 a23 a33 Then A = a11a22 a33 - a11a23a32 + a12 a23a31 - a12 a21a33 + a13a21a32 - a13a22 a31 Determinants For 4 x 4 and up don't try. For those interested, expansion by minors and cofactors is the preferred method. (However the spaghetti method works well! Simply duplicate all but the last column of the matrix next to the original and sum the products of the diagonals along the following pattern.) Spaghetti Method of |A| A = a11a22 a33 - a11a23a32 + a12 a23a31 - a12 a21a33 + a13a21a32 - a13a22 a31 a11 a12 a a 21 22 a31 a32 a13 a11 a12 a23 a21 a22 a33 a31 a32 Spaghetti Method of |A| A = a11a22 a33 - a11a23a32 + a12 a23a31 - a12 a21a33 + a13a21a32 - a13a22 a31 a11 a12 a a 21 22 a31 a32 a13 a11 a12 a23 a21 a22 a33 a31 a32 Properties of Determinates Determinants have several mathematical properties which are useful in matrix manipulations. 1 |A|=|A'|. 2. If a row of A = 0, then |A|= 0. 3. If every value in a row is multiplied by k, then |A| = k|A|. 4. If two rows (or columns) are interchanged the sign, but not value, of |A| changes. 5. If two rows are identical, |A| = 0. Properties of Determinates 6. |A| remains unchanged if each element of a row or each element multiplied by a constant, is added to any other row. 7. Det of product = product of Det's |AB| = |A| |B| 8. Det of a diagonal matrix = product of the diagonal elements The Inverse of a Matrix (A-1) For an nxn matrix A, there may be a B such that AB = I = BA. (The inverse is analogous to a reciprocal) A matrix which has an inverse is nonsingular. A matrix which does not have an inverse is singular. An inverse exists only if A 0 Inverse by Row or column operations Set up a tableau matrix A tableau for inversions consists of the matrix to be inverted post multiplied by a conformable identity matrix. Matrix Inversion by Tableau Method Rules: You may interchange rows. You may multiply a row by a scalar. You may replace a row with the sum of that row and another row multiplied by a scalar. Every operation performed on A must be performed on I When you are done; A = I & I = A-1 The Tableau Method of Matrix Inversion: An Example Step 1: Set up Tableau 3 1 0 0 1 4 2 5 0 1 0 4 1 3 2 0 0 1 Step 2: Add –2(Row 1) to Row 2 3 1 0 0 1 4 0 3 2 2 1 0 1 3 2 0 0 1 Matrix Inversion – cont. 3 1 0 0 1 4 0 3 2 2 1 0 1 3 2 0 0 1 Step 3: Add –1(Row 1) to Row 3 3 1 0 0 1 4 0 3 2 2 1 0 0 7 5 1 0 1 Step 4: Multiply Row 2 by –1/3 3 1 0 0 1 4 0 1 2 / 3 2 / 3 1 / 3 0 0 7 5 1 0 1 Matrix Inversion – cont. 3 1 0 0 1 4 0 1 2 / 3 2 / 3 1 / 3 0 0 7 5 1 0 1 Step 5: Add –4 (Row 2) 1 0 to Row 1 1 / 3 5 / 3 4 / 3 0 1 2 / 3 2 / 3 1 / 3 0 0 7 5 1 0 1 Step 6: Add 7(Row 2) to Row 3 0 1 0 1 / 3 5 / 3 4 / 3 0 0 1 2 / 3 2 / 3 1 / 3 0 0 0 1 / 3 11/ 3 7 / 3 1 Matrix Inversion – cont. 1 0 1 / 3 5 / 3 4 / 3 0 0 1 2 / 3 2 / 3 1 / 3 0 0 0 1 / 3 11/ 3 7 / 3 1 0 2 1 1 2 / 3 2 / 3 1 / 3 0 0 0 1 / 3 11/ 3 7 / 3 1 Step 7: Add Row 3 1 0 0 1 to Row 1 0 2 1 1 1 0 8 Step 9: Add 2(Row 0 1 0 5 2 3) to Row 2 0 0 1 / 3 11/ 3 7 / 3 1 Matrix Inversion – cont. Step 9: Multiply Row 3 by -3 The original matrix is now an identity, and the original identity has been transformed to the Inverse 0 2 1 1 1 0 0 1 8 0 5 2 0 0 1 / 3 11/ 3 7 / 3 1 1 1 1 0 0 2 0 1 0 8 5 2 0 0 1 11 7 3 Checking the calculation Remember AA-1=I 3 2 1 1 1 0 0 1 4 2 5 8 5 2 0 1 0 4 1 3 2 11 7 3 0 0 1 Thus 1 * 2 4 * 8 3 * 11 1 1 * 1 4 * 5 3 * 7 0 etc Using Stata to do matrix Algebra Try the following command set matrix matrix matrix matrix matrix matrix matrix matrix matrix input A= (1, 2 \ 3, 4) list A input B= (5, 6 \ 7, 8) list B define C = A + B list C X = (1, 4, 3 \ 2, 5, 4 \ 1, -3, -2) Xinv = inv(X) list Xinv The Matrix Model The multiple regression model may be easily represented in matrix terms. Y XB e Where the Y, X, B and e are all matrices of data, coefficients, or residuals The Matrix Model (cont.) The matrices in represented by Y1 Y 2 Y Yn X 11 X 12 X X 21 22 X ... ... X n1 X n 2 Y XB e ... X ik ... X 2 k ... ... ... X nk B1 B 2 B Bk are e1 e 2 e en Note that we postmultiply X by B since this order makes them conformable. The Assumptions of the Model Scalar Version 1. The ei's are normally distributed. 2. E(ei) = 0 3. E(ei2) = 2 4. E(eiej) = 0 (ij) 5. X's are nonstochastic with values fixed in repeated samples and (Xik-Xbark)2/n is a finite nonzero number. 6. The number of observations is greater than the number of coefficients estimated. 7. No exact linear relationship exists between any of the explanatory variables. The Assumptions of the Model: The Matrix Version These same assumptions expressed in matrix format are: 1. e N(0,) 2. = 2I 3. The elements of X are fixed in repeated samples and (1/ n)X'X is nonsingular and its elements are finite Derivation of B's in matrix notation Given the matrix algebra model we can replicate the least squares normal equations in matrix format. We need to minimize ee’ which is the sum of squared errors Setting the derivative equal to 0 we ultimately get B (X' X)1 X' Y Note that X’X is called the sums-of-squares and cross-products matrix.