Estimating covariance and related issues

Just as we have defined sample variance, we may define sample covariance S_XY between random variables X and Y:

where n is the size of the sample, i.e., the number of observed pairs (X_i, Y_i). Sample covariance can also be rewritten as follows:

To see this, we note the following:

This rewriting mirrors the relationship σ_XY = E[XY] − μ_X μ_Y from probability theory. It is important to realize that our sample must consist of joint realizations of variables X and Y. If we want to investigate the impact of temperature on ice cream demand, we must have pairs of observations taken in the same place at the same time; clearly, mixing observations is no use.

This definition is consistent with sample variance, since . Yet, one thing that may not look that obvious is why we should divide by n − 1. When dealing with variance, a common justification is that we lose one degree of freedom because we estimate one parameter, the unknown expected value. This simple rule does not sound convincing here, as we use two sample means  and  as estimators of the respective expected values. Indeed, the best idea is to really prove that the estimator above is unbiased.

THEOREM 9.6 The sample covariance (9.24) is an unbiased estimator of covariance, i.e., E[S_XY] = σ_SY.

PROOF Using (9.25) and the fact that the pairs (X_i, Y_i) are identically distributed, we obtain

Since the pairs are also independent, for i ≠ j we have

There are n² − n = n(n − 1) such terms in the last double sum in Eq. (9.26), hence

So we see that

which proves the result.

Now let us consider a quite practical question. If we have a large set of jointly distributed random variables, what is the required effort if we want to estimate their covariance structure? Equivalently, how many correlations we need? The covariance matrix is symmetric; hence, if we have n random variables, we have to estimate n variances and n(n − l)/2 covariances. This amounts to

entries. Hence, if n = 1,000, we should estimate 500,500 entries in the covariance matrix. A daunting task, indeed! If you think that such a case will never occur in practice, please consider Example 8.5, on portfolio management. You might well consider 1,000 assets for inclusion in the portfolio. In such a case, can we estimate the covariance matrix? What you know about statistical inference tells that you might need a lot of data to come up with a reliable estimate of a parameter. If you have to estimate a huge number of parameters, you need a huge collection of historical data. Unfortunately, many of them would actually tell us nothing useful: Would you use data from the 1940s to characterize the distribution of returns for IBM stock shares now? We need a completely different approach to reduce our estimation requirements.

Example 9.21 (Single-factor models for portfolio management) The returns of a stock share are influenced by many factors. Some are general economic factors, such as inflation and economic growth. Others are peculiar factors of a single firm, depending on its management strategy, product port-folio, etc. In between, we may have some factors that are quite relevant for a group of firms within a specific industrial sectors, much less for others; think of the impact of oil prices on energy or telecommunications.

Rather than modeling uncertain returns individually, we might try to take advantage of this structure of general and specific factors. Let us take the idea to an extreme and build a simple model whereby there is one factor common to all of stock shares, and a specific factor for each single firm. Formally, we represent the random return for the stock share of firm i as

where

• α_i and β_i are parameters to be estimated.
• R_m is a random variable representing the common risk factor; the subscript m stands for market; indeed, financial theory suggests that a suitable common factor could be the return of a market portfolio consisting of all stock shares, with a proportion depending on their relative capitalization with respect to the whole market.
•  is a random variable representing individual risk, which in financial parlance is referred to as idiosyncratic risk; a natural assumption about these variables is that  (otherwise, we would include the expected value into α_i). Another requirement is that the common factor really captures whatever the stock returns have in common, and that the specific factors are independent. Typically, we do not require independence, but only lack of correlation. These requirements can be formalized as: Where we deal with linear regression, we will see that condition (9.27) is actually ensured by model estimation procedures based on least squares. On the contrary, condition (9.28) is just an assumption, resulting in a so-called diagonal model, since the covariance matrix of specific factors is diagonal.

The model above is called single-factor model for obvious reasons, but what are its advantages from a statistical perspective? Let us check how many unknown parameters we should estimate in order to evaluate expected return and variance of return for an arbitrary portfolio. To begin with, observe that for a portfolio with weights w_i, we have

Then,

where μ_m is expected return of the market portfolio (more generally, the expected value of whatever common risk factor we choose). From this, we see that we need to estimate:

• n parameters α_i, i = 1,…, n
• n parameters β_i, i = 1,…, n
• The expected value μ_m

These add up to 2n + 1 parameters. Variance is a bit trickier, but we may use the diagonality condition (9.28) to eliminate covariances and obtain

where  is the variance of the common risk factor and  is the variance of each idiosyncratic risk factor, i = 1,…, n. They amount to n + 1 additional parameters that we should estimate, bringing the total to 3n + 2 parameters. In the case of n = 1,000 assets, the we have a grand total of 3,002 parameters; this is a large number, anyway, but pales when compared with the 500,500 entries of the full covariance matrix.

This example is quite instructive. Please keep in mind that we are not estimating parameters for fun. This is a book about management, and estimates are supposed to be used as an input to decision-making procedures; as you may expect, wrong estimates may lead to poor decisions. As it often turns out in practice, the more sophisticated the decision process, the larger the impact of wrong estimates.

The single-factor model we just outlined has a pervasive impact in both portfolio management and corporate finance. In fact, it is the starting point of a well-known equilibrium model, the capital asset pricing model (CAPM). To put it simply, CAPM is based on rewriting the single-factor model in terms of excess returns with respect to the risk-free return r_f:

By taking expected values, we obtain

It is useful to interpret the expected excess return μ_i − r_f as a risk premium, since it is the return above the risk-free rate that an investor expects if she holds the risky asset i; by the same token, μ_m − r_f is the risk premium from holding the market portfolio, i.e., a broadly diversified portfolio reflecting the relative market capitalization of firms. Then, according to CAPM, the following conditions hold at equilibrium in Eq. (9.29):

These conditions state that there is no specific risk premium for asset i; the risk premium is only due to the correlation of its return with the general market return. This is a controversial result, relying on many debatable assumptions, and we cannot really discuss it in any detail. However, there are two possibilities about CAPM: Either it is valid model, or it is not.

1. If it is a valid model, the practical implication is that you should not pay any financial analyst, since the best that you can do is to invest in a market portfolio, surrogated by a wide market index. There is no hidden alpha to take advantage of by active stock picking, and only efficiency in passive portfolio management matters. In fact, this is why exchange traded funds (ETFs) are so popular; they are passive funds tracking broad market indexes at low cost. Of course, if one believes CAPM, statistics should be used to support the thesis empirically.
2. If it is not a valid model, then you should try to manage a portfolio actively. This means that you should try to use statistics to estimate parameters in order to gain differential knowledge that can be used to make money. Furthermore, from an organizational perspective this task may be decomposed in two parts, one pertaining to general market conditions (involving macroeconomic factors), and one pertaining to specific information about a firm.

This brief discussion should convince you about the fundamental role of quantitative methods in practice, and their impact on organization of a portfolio management firm.

More generally, this motivates the use of statistical models, which we begin discussing simple linear regression. As you may imagine, it is hard to believe that one single factor can really capture everything stock shares may have in common. As a result, we do not have a diagonal model, since there is some commonality left unexplained by a single factor. This leads us to consider multiple factor models