Category: Simple Linear Regression

The aim of this section is to broaden our view about linear regression by analyzing it in the light of some concepts from linear algebra. In fact, linear regression can be regarded as a sort of orthogonal projection within suitably chosen vector spaces. To see this, let us group observations and residuals into vectors as…

In this section we outline what happens when we relax a bit our assumptions about the underlying statistical model: The first thing to note is that now the explanatory variable is random. This is certainly going to make things a tad more complicated, but we do not want to change our assumptions substantially, which is…

Regression models can be used in a variety of ways, but the essential possibilities are In the first case, we are actually concerned with the estimate of slope; the idea is that understanding the phenomenon can lead to knowledge and to better policies. Apparently, there is little difference from the second case since, after all,…

All of the theory we have built so far relies on specific assumptions about the errors , which we recall here once again for convenience: Since errors are not observable directly, we must settle for a check based on residuals. The check can exploit sound statistical procedures, which are beyond the scope for our purposes, it…

Testing the significance of a slope coefficient is a first diagnostic check of the suitability of a regression model. However, per se, this test does not tell us much about the predictive power of the model as a whole. The reader may better appreciate the point by thinking about a multiple regression model involving several…

Now we are armed with the necessary knowledge to draw statistical inferences about the regression parameters. Mirroring what we did with the estimation of expected value, we should The technicalities involved here are essentially the same as those involved in dealing with estimation of the expected value, and we avoid repeating the reasoning. In the…

Equations (10.16) and (10.18) help us in assessing the uncertainty about the estimate of unknown regression parameters. A missing piece in this puzzle, however, is the standard deviation  of the random errors, which are not directly observable. The only viable approach we have is to rely on the residuals  as a proxy for the errors . The intuition is…

In this section we want to tackle a few statistical issues concerning the estimation of the unknown parameters of the data-generating model, featuring a nonstochastic regressor and homoskedastic errors: As we said, the values xi are numbers and the errors  are independent and identically distributed random variables satisfying the following assumptions: Our task mirrors what we did when…

So far, we have regarded linear regression as a numerical problem, which is fairly easy to solve by least squares. However, this is a rather limited view. To begin with, it would be useful to gain a deeper insight into the meaning of the regression coefficients, in particular the slope b. The careful reader might have…

If we label a model like y = a + bx as linear, no eyebrow should be raised. Now, consider a regression model involving a squared explanatory variable: Is this linear? Actually it is, in terms of the factor that matters most: fitting model coefficients. True, the model is nonlinear in terms of the explanatory variable x, but the actual unknowns when…