Category: Dependence, Correlation, and Conditional Expectation

In Section 7.9 we introduced stochastic processes as sequences of random variables; assuming a discrete-time stochastic process, we have a sequence of the form Xt, t = 1,2,3,…. We have also pointed out that we cannot characterize a stochastic process in terms of the marginal distributions of each variable Xt. In principle, we should assign the joint distribution of all the…

We have introduced the exponential distribution in Section 7.6.3, where we also pointed out its link with the Poisson distribution and the Poisson process. The standard use of exponential variables to model random time between events relies on its memoryless property, which we are now able to appreciate. Consider an exponential random variable X with parameter λ, and say that X models…

In this section we take advantage of a fundamental theorem concerning iterated expectation. Before formalizing the idea, let us illustrate it by a simple example. Example 8.7 You are lost in an underground mine and stand in front of two tunnels. One of the two tunnels will lead you to the surface after a 5-hour walk; the…

We are already familiar with the concept of conditional probability when events are involved. When dealing with random variables X and Y, we might wonder whether knowing something about Y, possibly even its realized value, can help us in predicting the value of X. To introduce the concepts in the simplest way, it is a good idea to work with…

A detailed coverage of multivariate distributions is beyond the scope but we should at least consider a generalization of normal distribution. A univariate normal distribution is characterized by its expected value μ and by its variance σ2. In the multivariate case, we have a vector of expected values μ and a covariance matrix Σ. We consider a random vector taking values in :…

The covariance is a generalization of variance. Hence, it is not surprising that it shares a relevant shortcoming: Its value depends on the unit of measurement of the underlying quantities. We recall that it is impossible to say whether a variance of 10,000 is large or small; a similar consideration applies to standard deviation, which…

In Section 7.7 we dealt with sums of random variables, under the restrictive assumption of independence. Finally, armed with covariance, we may tackle the general case. THEOREM 8.4 (Variance of the sum/difference of two random variables) Given two random variables X and Y, the variance of their sum and difference is respectively. This theorem is somewhat reassuring; if…

If two random variables are not independent, it is natural to investigate their degree of dependence, which means finding a way to measure it and to take advantage of it. The second task leads to statistical modeling, which we will investigate later in the simplest case of linear regression. The first task is not as easy…

In the previous section we formally introduced the concept of the joint cumulative distribution function (CDF). In the case of two random variables, X and Y, this is a function FX,Y(x, y) of two arguments, giving the probability of the joint event {X ≤ x, Y ≤ y}: The joint CDF tells the whole story about how the two random variables are linked. Then,…

In order to fully appreciate the issues involved in characterizing the dependence of random variables, as well as to appreciate the role of independence, we should have some understanding of how to characterize the joint distribution of random variables.1 For the sake of simplicity, we will deal only with the case of two random variables with…