Report

Chapter 2 Multivariate Distributions Math 6203 Fall 2009 Instructor: Ayona Chatterjee Random Vector • Given a random experiment with a sample space C. Consider two random variables X1 and X2 which assign to each element c of C one and only one ordered pair of numbers X1(c)=x1 and X2(c)=x2. Then we say that (X1, X2) is a random vector. • The space of (X1, X2) is the set of ordered pairs D={(x1, x2) : X1(c)=x1 and X2(c)=x2 } Cumulative Distribution Function • The joint cumulative distribution function of (X1, X2) is denoted by FX1,X2 (x1, x2) and is given as FX1,X2 (x1, x2) =P[X1≤x1, X2 ≤x2)]. • A random vector (X1, X2 )is a discrete random variable is its space D is finite or countable. • A random vector (X1, X2 ) with space D is continuous if its cdf FX1,X2 (x1, x2) is continuous. Probability Mass Function • For discrete random variables X1 and X2, the joint pmf is defined as p X 1 X 2 ( x1 , x 2 ) P [ X 1 x1 , X 2 x 2 ] Note * 0 p X 1 X 2 ( x1 , x 2 ) 1 * D p X 1 X 2 ( x1 , x 2 ) 1 Probability Density Function • For a continuous random vector x1 x 2 F X 1 X 2 ( x1 , x 2 ) f X 1 X 2 ( w1 , w 2 ) dw 1 dw 2 F X 1 X 2 ( x1 , x 2 ) 2 x1 x 2 f X 1 X 2 ( x1 , x 2 ) Note * f X 1 X 2 ( x1 , x 2 ) 0 * f X 1 X 2 ( x1 , x 2 ) dx 1 dx 2 1 D Marginals • The marginal distributions can be obtained from the joint probability density function. • For a discrete and continuous random vector the marginals can be obtained as below: p X 1 ( x1 ) x2 p X 1 X 2 ( x1 , x 2 ) f X 1 ( x1 ) f X 1 X 2 ( x1 , x 2 ) dx 2 Expectation • Suppose (X1, X2) is of the continuous type. Then E(Y) exists if g ( x1 , x 2 ) f X 1 X 2 ( x1 , x 2 ) dx 1 dx 2 then E (Y ) g(x , x 1 2 ) f X 1 X 2 ( x1 , x 2 ) dx 1 dx 2 Theorem • Let (X1, X2) be a random vector. Let Y1 = g1(X1, X2) and Y2 = g2 (X1, X2) be a random variable whose expectations exits. Then for any real numbers k1 and k2. E(k1 Y1 + k2 Y2 )= k1E(Y1 ) + k2 E(Y2 ) Note E ( g ( X 2 )) g(x 2 ) f X 1 X 2 ( x1 , x 2 ) dx 1 dx 2 g(x 2 ) f X 2 ( x 2 ) dx 2 Moment Generating Function • Let X = (X1. X2 )’ be a random vector. If E(et1x1+t2x2 ) exists for |t1 |<h1 and |t2 |<h2 where h1 and h2 are positive, the mgf is given as M X1X 2 (t ) E [e ' t X ] where t ( t1 , t 2 ) M X1X 2 ' ( t1 , 0 ) is the mgf of X 1 and M X1X 2 ( 0 , t 2 ) is the mgf of X 2 . 2.3 CONDITIONAL DISTRIBUTIONS AND EXPECTATIONS • So far we know – How to find marginals given the joint distribution. • Now – Look at conditional distribution, distribution of one of the random variable when the other has a specific value. Conditional pmf • We define p X 2 | X 1 ( x 2 | x1 ) P ( X 1 x1 , X 2 x 2 ) P ( X 1 x1 ) p X 1 , X 2 ( x1 , x 2 ) p X 1 ( x1 ) x2 S X 2 • SX2 is the support of X2. • Here we assume pX1 (x1) > 0. • Thus conditional probability is the joint divvied by the marginal. Conditional pdf • Let fX1x2 (x1, x2 ) be the joint pdf and fx1 (x1) and fx2 (x2) be the marginals for X1 and X2 respectively then the conditional pdf of X2, given X1 is f X 2 | X 1 ( x 2 | x1 ) f X 1 ( x1 ) 0 f X 1 , X 2 ( x1 , x 2 ) f X 1 ( x1 ) f 2 |1 ( x 2 | x1 ) Note * f 2 |1 ( x 2 | x1 ) 0 * f 2 |1 ( x 2 | x1 ) dx 2 1 Conditional Expectation and Variance If u(X 2 ) is a function of X 2 , the conditiona X 1 x1if it exists, is given by u(X 2 ), given that E [ u ( X 2 ) | x1 ] l expectatio n of u(x 2 ) f 2 |1 ( x 2 | x1 ) dx 2 var( X 2 | x1 ) E ( X 2 2 | x1 ) [ E ( X 2 | x1 )] 2 Theorem • Let (X1, X2) be a random vector such that the variance of X2 is finite. Then – E[E(X2 |X1)]=E(X2) – Var[E(X2 |X1 )]≤ var(X2 ) 2.4 The Correlation Coefficient E [( X 1 )( Y 2 )] E ( XY ) E ( X ) E (Y ) Cov ( X , Y ) E [( X 1 )( Y 2 )] Cov ( X , Y ) 1 2 Here ρ is called the correlation coefficient of X and Y. Cov(X,Y) is the covariance between X and Y. The Correlation Coefficient • Note that -1 ≤ ρ≤ 1. • For the bivariate case – If ρ = 1, the graph of the line y = a + bx (b > 0) contains all the probability of the distribution of X and Y. – For ρ = -1, the above is true for the line y = a + bx with b < 0. – For the non-extreme case, ρ can be looked as a measure of the intensity of the concentration of the probability of X and Y about a line y = a + bx. Theorem • Suppose (X,Y) have a joint distribution with the variance of X and Y finite and positive. Denote the means and variances of X and Y by µ1 , µ2 and σ12 , σ22 respectively, and let ρ be the correlation coefficient between X and Y. If E(Y|X) is linear in X then E (Y | X ) 2 2 1 ( X 1 ) E (var( Y | X )) 2 (1 ) 2 2 2.5 Independent random Variables • If the conditional pdf f2|1 (x2|x1) does not depend upon x1 then the marginal pdf of X2 equals the conditional pdf f2|1 (x2|x1) . • Let the random variables X and Y have joint pdf f(x,y) and the marginals fx (x) and fy (y) respectively. The random variables X and Y are said to be independent if and only if – f(x,y)= fx (x) fy (y) – Similar defintion can be wriiten for discrete random variables. – Random variables that are not independent are said to be dependent. Theorem • Let the random variables X and Y have support S1 and S2, respectively and have the joint pdf f(x,y). Then X and Y are independent if and only if f(x,y) can be written as a product of a nonnegative function of x and a nonnegative function of y. That is f(x,y)=g(x)h(y) where g(x)>0 and h(y)>0. Note • In general X and Y must be dependent of the space of positive probability density of X and Y is bounded by a curve that is neither a horizontal or vertical line. • Example; f(x,y)=8xy, 0< x< y < 1 – S={(x,y): 0< x< y < 1} This is not a product space. Theorems • Let (X, Y) have the joint cfd F(x,y) and let A and Y have the marginal cdfs Fx (x) and Fy (y) respectively. Then X and Y are independent if and only if – F(x,y)= Fx (x)Fy (y) • The random variable X and Y are independent if and only if the following condition holds. – P(a < X≤ b, c < Y ≤ d)= P(a < X≤ b)P( c < Y ≤ d) – For ever a < b, c < d and a,b,c and are constants. Theorems • Suppose X and Y are independent and that E(u(X)) and E(v(Y)) exist, then – E[u(x), v(Y)]=E[u(X)]E[v(Y)] • Suppose the joint mgf M(t1,t2) exists for the random variables X and Y. Then X and Y are independent if and only if – M(t1,t2) = M(t1,0)M(0,t2) • That is the joint mfg if the product of the marginal mgfs. Note • If X and Y are independent then the correlation coefficient is zero. • However a zero correlation coefficient does not imply independence.