The Hitchhiker’s Guide to Linear Models is now complete
My new book.
The book can be downloaded for free but you will need a Leanpub account, same if you buy it.
The Hitchhiker’s Guide to Linear Models is finally complete. It took me a while to finish it but I’m happy with the result. I hope you enjoy it as much as I did writing it.
The Github repository contains the code for the book so that the readers can avoid copy and paste from the PDF.
Table of contents:
Contents |
|
|
|
Preface | i |
|
|
1 R Setup | 1 |
1.1 R and Rstudio | 1 |
1.2 Installing R | 1 |
1.2.1 Windows and Mac | 1 |
1.2.2 Linux | 1 |
1.3 Installing RStudio | 2 |
1.3.1 Windows and Mac | 2 |
1.3.2 Linux | 2 |
1.4 Installing R Packages. | 2 |
1.5 Changing RStudio colors and font | 4 |
1.6 Installing Quarto | 4 |
1.6.1 Windows and Mac | 4 |
1.6.2 Linux1 | 4 |
|
|
2 Linear algebra review | 5 |
2.1 Using R as a calculator | 5 |
2.2 System of linear equations | 5 |
2.3 Matrix | 5 |
2.4 Transpose matrix | 6 |
2.5 Matrix multiplication | 6 |
2.6 Matrix representation of a system of linear equations | 6 |
2.7 Identity matrix | 7 |
2.8 Inverse matrix | 7 |
2.9 Solving systems of linear equations | 7 |
|
|
3 Statistics review |
|
3.1 Using R as a calculator | 11 |
3.2 Data and dataset | 11 |
3.3 Summation | 11 |
3.4 Probability | 11 |
3.5 Descriptive statistics | 13 |
3.5.1 Mean | 13 |
3.5.2 Variance | 13 |
3.5.3 Standard deviation | 14 |
3.5.4 Covariance | 15 |
3.5.5 Correlation | 16 |
3.6 Distributions | 20 |
3.6.1 Normal distribution | 20 |
3.6.2 Poisson distribution | 22 |
3.6.3 Student’s t-distribution | 23 |
3.6.4 Computing probabilities with the normal distribution | 24 |
3.6.5 Computing probabilities with the Poisson distribution | 27 |
3.6.6 Computing probabilities with the t-distribution | 28 |
3.7 Sample size | 29 |
|
|
4 Recommended workflow | 30 |
4.1 Creating projects | 30 |
4.2 Creating scripts | 30 |
4.3 Creating notebooks | 32 |
4.4 Organizing code sections | 33 |
4.5 Customizing notebooks’ output | 34 |
|
|
5 Read, Manipulate, and Plot Data | 35 |
5.1 The datasauRus dataset in R format. | 35 |
5.2 The Quality of Government dataset in CSV format. | 40 |
5.3 The Quality of Government dataset in SAV (SPSS) format | 44 |
5.4 The Quality of Government dataset in DTA (Stata) format | 48 |
5.5 The Freedom House dataset in XLSX (Excel) format | 50 |
|
|
6 Linear Model with One Explanatory Variable | 60 |
6.1 Model specification | 60 |
6.2 The Galton dataset | 64 |
6.3 A word of caution about Galton’s work | 64 |
6.4 Loading the Galton dataset | 65 |
6.5 Estimating linear models’ coefficients | 66 |
6.5.1 Linear model as correlation | 66 |
6.5.2 Linear model as matrix multiplication | 67 |
6.5.3 Relation between correlation and matrix multiplication | 71 |
6.5.4 Computational note | 75 |
6.6 Logarithmic transformations | 75 |
6.7 Plotting model results | 76 |
6.8 Linear model does not equal straight line | 81 |
6.9 Transforming variables | 85 |
6.10 Regression with weights | 89 |
|
|
7 Linear Model with Multiple Explanatory Variables | 91 |
7.1 Model specification | 91 |
7.2 Life expectancy, GDP and well-being in the Quality of Government dataset | 94 |
7.3 Estimating linear models’ coefficients | 96 |
7.4 Model accuracy | 103 |
7.4.1 Root Mean Squared Error and Mean Absolute Error | 103 |
7.4.2 RMSE and MAE interpretation | 104 |
7.5 Model summary | 107 |
7.5.1 Coefficient’s standard error | 107 |
7.5.2 Coefficient’s t-statistic | 108 |
7.5.3 Coefficient’s p-value | 108 |
7.5.4 Residual standard error | 109 |
7.5.5 Model’s multiple R-squared (or unadjusted R-squared) | 109 |
7.5.6 Model’s adjusted R-squared | 110 |
7.5.7 Model’s F-statistic | 111 |
7.6 Error’s assumptions | 111 |
7.6.1 Error’s normality | 112 |
7.6.2 Error’s homoscedasticity (homogeneous variance) | 113 |
|
|
8 Linear Model with Binary and Categorical Explanatory Variables | 114 |
8.1 Model specification with binary variables | 114 |
8.1.1 ANOVA is a particular case of a linear model with binary variables | 114 |
8.1.2 Corruption and popular vote in the Quality of Government dataset | 114 |
8.1.3 Estimating a linear model and ANOVA with one predictor and two categories | 116 |
8.1.4 Corruption and regime type in the Quality of Government dataset | 118 |
8.1.5 Estimating a linear model and ANOVA with one predictor and multiple categories | 120 |
8.1.6 Estimating a linear model with continuous and categorical predictors | 126 |
8.2 Model specification with binary interactions | 128 |
8.2.1 Corruption and interaction variables in the Quality of Government dataset | 128 |
8.2.2 Estimating a linear model with binary interactions | 131 |
8.2.3 Confidence intervals with binary interactions | 133 |
8.3 Model specification with categorical interactions | 136 |
8.3.1 Estimating a linear model with categorical interactions | 136 |
8.3.2 Confidence intervals with categorical interactions | 137 |
|
|
9 Linear Model with Fixed Effects | 140 |
9.1 Year fixed effects | 140 |
9.1.1 Model specification | 140 |
9.1.2 Corruption and popular vote in the Quality of Government dataset | 140 |
9.1.3 Estimating year fixed effects’ coefficients | 142 |
9.2 Country fixed effects | 145 |
9.2.1 Model specification | 145 |
9.2.2 Corruption and popular vote in the Quality of Government dataset | 145 |
9.2.3 Estimating country-time fixed effects’ coefficients | 145 |
9.3 Country-year fixed effects | 148 |
9.3.1 Model specification | 148 |
9.3.2 Corruption and popular vote in the Quality of Government dataset | 149 |
9.3.3 Estimating country-time fixed effects’ coefficients | 149 |
|
|
10 Generalized Linear Model with One Explanatory Variable | 152 |
10.1 Model specification | 152 |
10.2 Model families. | 152 |
10.2.1 Gaussian model | 153 |
10.2.2 Poisson model | 153 |
10.2.3 Quasi-Poisson model | 154 |
10.2.4 Binomial model (or logit model) | 157 |
|
|
11 Generalized Linear Model with Multiple Explanatory Variables | 165 |
11.1 Obtaining the original codes and data | 165 |
11.2 Loading the original data | 165 |
11.3 Ordinary Least Squares | 166 |
11.4 Poisson Pseudo Maximum Likelihood | 167 |
11.5 Tobit | 169 |
11.6 Reporting multiple models | 170 |
|
|
References | 172 |
Don’t panic!