Class 03: The Lady Tasting Tea
Today we are going to fill in some of the gaps left from the last class about building exploratory graphics and linear models. The title for this class comes from an experiment described in the 1935 text The Design of Experiments by Sir Ronald Fisher, one of the founding fathers of modern statistics.
We are going to look at a different dataset of teas. Specifically, tea reviews from the Adagio Tea website. I collected this dataset about 12 months ago, so it should be similar but not exactly the same as what is one the site today. Let’s read the data into R from my website as we have done with the mammals and abalone datasets:
Opening the data in the data viewer, we see the required first two columns and the response of interest: the average score assigned to the tea by customers.
Variables available to predict the output are the type of tea, the number of reviews received the price of the tea. The latter is given in estimated cents per cup as reported on the site. We also have the full name of the tea.
Before doing anything else, let’s try to understand the distribution of each of the variables.
The score values are generally very high, with most of them above 88. All of the scores are whole integers, and the most common values are between 92 and 95.
The price variable is heavily skewed, with a few very expensive teas. Most are well under a quarter per cup.
The number of reviews also have a bit of skew, but not nearly as strongly as the price variable.
There are twelve types of tea, with some having only a few samples and others having over thirty.
Now, we can proceed to bivariate plots showing the relationship between each variable and the response.
There seems to be a slight positive relationship between the number of reviews and the score.
Given the skew of the plot, it is hard to figure out the exact relationship between price and score.
Last time I used the
geom_smooth function to put what I
called a best-fit line onto the plot. We then used the
function to estimate the slope and intercept of this line
numerically. Let’s try to formalize this concept now.
The classical linear regression model assumes that the average value, or mean, of the response Y is a linear function of X. Symbolically, with the index i representing the i’th sample, this gives:
Similarly, we can write that Y is equal to a fixed linear effect dependent on X plus a random variable epsilon with zero mean.
The estimation task here is to find reasonable estimates for the alpha and beta components given pairs of observations (X, Y). There are many ways of doing this but by far the most common is to use what is known as Ordinary Least Squares or OLS. This selects the alpha and beta that minimize the squared errors implied by the linear model. As before, let’s write this down symbolically:
A natural question is to ask why we are interested in the squared errors. We could just as likely ask for the minimizer of the absolute value of the errors or the minimizer of the maximum errors.
I can offer two justifications of the squared error, one numerical and one statistical. Using the squared error, unlike the absolute errors or maximum errors, produces a smooth function. That is, a function that has an infinite number of derivatives at every point (although, all derivatives after 2 are equal to zero). This makes it easier to find a solution to the linear equation. Secondly, if the errors are distributed as a normal random variable the OLS estimator is equivalent to the maximum likelihood estimator (MLE).
Linear Models - Visually
In order to better understand linear models, it helps to see a picture. Below I have drawn a line through our dataset and indicated the errors (also known as residuals) that the ordinary least squares is concerned with minimizing. Note: don’t worry much about the code producing this graphic, concentrate just on the output for now.
Notice that this line under-guesses most of the score of teas, particularly if the number of reviews is low.
How can we figure out what the best alpha and beta are for this model? The most straightforward way would be to just try a large number of values and see which one minimizes the response. This is largely impractical but useful to see in this small example. It will also let you see some more involved R code.
To start, we construct variables to hold the best sum of squares, alpha, and beta values. We then create a double loop cycling over all combinations of alpha and beta and testing whether the given configuration gives a better sum of squares compared to our current best value.
Now, let’s compare the brute force algorithm with the lm function:
Our brute-force solution is actually quote close to this! Of course, it depended on me knowing approximately what the right slope and intercept should be. In this case that was easy graphically, but in more complex example it is a much more challenging task.
Let’s use the predict function again to fit this model to our data:
And now plot the fitted values. Does it visually correspond to where you would expect the best fit line to run?
How should be evaluate the predictive ability of a model? The method of ordinary least squares suggests that we should consider the sum of squared errors. A challenge with this is that the value of a sum will grow in proportion to how many observations there are. This makes it hard to compare results across data sources and subsets. A simple solution is to use the average value of the squared errors, known as the mean squared error or MSE:
This works as a general measurement of error, but the units are a bit strange as they are given in squared scores. A simple solution to this exists by taking the square root of the MSE, resulting in the root mean squared error (RMSE):
You’ll find references to the RMSE through the literature on machine learning as it is by far the most common measurement for predictiveness of continuous responses.
How good of a result is our RMSE of 1.99? It’s hard to say for sure, but one easy thing to do is to compare it to the simplest possible model: the one that predicts every score will be equal to the average score. The RMSE of this estimator is given by:
So, we have improved on the baseline model, but not by a large amount. Note: if you are familiar with the standard error, the RMSE of the mean is almost equal to the standard error of the response.
Multivariate linear regression
The linear regression model I just introduced is known as simple linear regression because there is only one explanatory variable. We can easy consider multivariate models; for instance, we can be write a two variable model mathematically as follows:
The geometric interpretation of this is that we have plane in place of the line in the simple linear regression model.
When I teach an applied statistics course, we spend a lot of time working up to multivariate regression models. The interpretation of multivariate models can quickly become quite complex. However, using multivariate models in statistical learning is much easier to understand: each slope coefficient (beta and gamma here) corresponds to a weight placed on how much the response changes with each predictor variable.
Fitting multivariate models is also quite easy with the
lm function. Simply add the variables together that
you would like to use for prediction. Here we use both
the number of reviews and the price of the tea:
Using the color aesthetics we learned last class, we can visualize the predicted values of this model. Don’t get hung up on the code below; concentrate on the output.
Also, predictiveness of the model has been greatly improved over the simple linear regression from before:
Notes on Labs
- you should see your completion marker increased from
- you should also see that your first two labs now have scores
in the file
- you may have entries in
results.md, but still missing the completion scores
- if there are any issues:
- make sure that the files have correct extensions (
csv), including capitalization
- if files are misnamed at the moment, please move them to the correct ones
- I will re-update the completions indicator this evening; please try to have labs 1 and 2 fixed by then
- make sure that the files have correct extensions (
- due to naming issues, I’ve decided it will be easier to post lab assignments on the website rather than pushing the templates to GitHub
- some version of R and RStudio produce
.htmlfiles rather than
nb.htmlfiles; this is fine - just put the html files online instead. The grading script has been modified to catch this.
- if you had error messages about “curl” when downloading files,
you can run
install.packages("devtools")once to get the packages it is complaining about