Lesson 6.2: Simple Linear Regression

Readings 📖 and Instructional Video 🎦

Least Squares Regression

Read Section 8.2: Least Squares Regression in OpenIntro Statistics (Diez et al., 2019) CC BY-SA 3.0. In Lesson 6.1 we considered the linear association between two numerical variables in terms of a scatterplot and correlation. This lesson builds on that by developing the least squares regression model, also known as simple linear regression. Here, one of the numerical variables, the predictor or explanatory variable (x), is used to predict the other numerical variable, the response variable (y). The least squares regression model can also handle a categorical predictor variable with two categories if we re-code it as an indicator variable (a variable that is 0 for one category and 1 for the other category). As you read, look up new terminology in the Glossary and self-assess your understanding by attempting the guided practice exercises.

Watch the following video, Fitting a Line with Least Squares Regression (Barr et al., 2014-a), on how to fit a least squares regression line to a predictor variable (x) and a response variable (y) (duration 00:06:48).

You can also watch the 44-minute video, OS4 Section 8.2 Least Squares Regression—Overview by an Author (OpenIntroOrg, 2021), for a walk-through of the content in Section 8.2. This video provides a less-formal discussion of the textbook material.

Influential Points

Read Section 8.3: Types of Outliers in Linear Regression in OpenIntro Statistics (Diez et al., 2019) CC BY-SA 3.0. When fitting a least squares regression model, we should always graph the data in a scatterplot with the least squares regression line. This allows us to see whether there are any points that stand-out from the overall pattern in the point cloud. Such points, called outliers, may be influential in the sense of having an outsized effect on the fit of the model. Again, as you read, look up new terminology in the Glossary and self-assess your understanding by attempting the guided practice exercises.

Watch the following video, Types of Outliers in Linear Regression (Barr et al., 2014-b), on the different types of outliers in linear regression (duration 00:02:52).

Inference for Linear Regression

Inference for linear regression lies beyond the scope of this course, so don’t worry about reading Section 8.4 in OpenIntro Statistics. For the purposes of this course, all we need to know about inference for regression is contained in Example 8.11 in Section 8.24: “The second column is a standard error for this point estimate [of β₁]: SE_b1 = 0.0108. The third column is a t-test statistic for the null hypothesis that β₁ = 0: T = −3.98. The last column is the p-value for the t-test statistic for the null hypothesis β₁ = 0 and a two-sided alternative hypothesis: 0.0002″ (Diez et al., 2019) CC BY-SA 3.0. Roughly speaking, if this p-value is small (say smaller than 0.05), then there is a statistically significant linear association between y and x.

Least Squares Regression and Influential Points

Read Supplementary Notes 6.2, which gives a whole lot more detail and examples of least squares regression and influential points.

Virtual Statistical Software Lab 💻

Work through the virtual statistical software lab: Software Lab 6.2. In this lab you’ll analyze some simple linear regression models fit to “personal freedom” data for different countries around the world. As you work through the lab, answer the exercises in the shaded boxes. These exercises are not graded but the solutions are available: Software Lab 6.2 Solutions. The lab should take you no more than 45 minutes to complete.