Author: haroonkhan
-
Using regression for forecasting and explanation purposes
We have seen that omitting variables may result in biased estimates, or even in debatable models where significant coefficients are associated with regressor variables that may even have no real impact on the response variable. However, a rather cynical point of view could be that, as long as the model does a good job at…
-
Testing a multiple regression model
To investigate the statistical validity of a multiple regression model, the first step is to check the variance of the estimators. In this case, we have multiple estimators, so we should check their covariance matrix: Using Eq. (16.4), we see that The familiar assumptions about errors, in the multivariate case, can be expressed as i.e., the…
-
Selecting explanatory variables: collinearity
When selecting variables, there are a few issues and tradeoffs involved. Apparently, we should aim at finding the model with the largest R2 coefficient, and, arguably, the more variables we include, the better model we obtain. However, the following examples show that subtle difficulties may be encountered. Example 16.1 (Omitted variables and bias) Let us consider the sample…
-
BUILDING, TESTING, AND USING MULTIPLE LINEAR REGRESSION MODELS
The least-squares approach to estimating parameters of a multiple regression model is a fairly straightforward extension of simple linear regression. What is not so easy is the extension of the statistical testing procedures, which present more variants when multiple variables are involved. Nevertheless, the necessary intuition for understanding what commercially available statistical software tools offer…
-
MULTIPLE LINEAR REGRESSION BY LEAST SQUARES
Running a linear regression with multiple explanatory variables is a rather straightforward extension of what especially if we assume fixed, deterministic values of the regressors. The underlying statistical model is We avoid using α to denote a constant term, so that we may group parameters into a vector . The model is estimated on the basis…
-
Introduction
We extend the simple linear regression concepts that were introduced. The first quite natural idea is building a linear regression model involving more than one regressor. Finding the parameters by ordinary least squares (OLS) is a rather straightforward exercise, as we see in Section 16.1. What is much less straightforward is the statistical side of the…
-
Covariance matrices
Given a random vector with expected value μ, the covariance matrix can be expressed as Note that inside the expectation we are multiplying a column vector p × 1 and a row vector 1 × p, which does result in a square matrix p × p. It may also be worth noting that there is a slight inconsistency of notation, since we denote…
-
MATRIX ALGEBRA AND MULTIVARIATE ANALYSIS
In this section we discuss a few more concepts that are useful in multivariate analysis. Unfortunately, when moving to multivariate statistics, we run out of notation. As usual, capital letters will refer to random quantities, with boldface reserved for random vectors such as X and Z; elements of these vectors will be denoted by Xi and Zi, and scalar random variables will…
-
Correspondence analysis
Correspondence analysis is a graphical technique for representing the information included in a two-way contingency table containing frequency counts. For example, Table 15.2 lists the number of times an attribute (crispy, sugar-free, good with coffee, etc.) is used by consumers to describe a snack (cookies, candies, muffins, etc.).5 The method deals with two categorical or discrete quantitative variables…
-
Multidimensional scaling
Multidimensional scaling is a family of procedures that aim at producing a low-dimensional representation of object similarity/dissimilarity. Consider n brands and a similarity matrix, whose entry dij measures the distance between brands i and j, as perceived by consumers. This matrix is a direct input of multidimensional scaling, whereas other methods aim at computing distances. Then, we want to find a representation…