D01-05AM: Multiple Regression Analysis: Estimation, Diagnostics and Modeling

The course starts with a discussion of the logic of the multiple regression model and the central assumptions underlying the ordinary least squares approach. This part includes an extensive discussion of the derivation of the ordinary least squares estimator, its standard error and summary statistics. Then it proceeds with testing for the adequacy of the assumptions and suitable corrections and extensions to the estimation techniques in the context of cross-sectional data. Particular emphasis will be laid on influential cases, multicollinearity, heteroskedasticity, and autocorrelation. Finally, categorical predictors, interactions, and nonlinearities will be considered.

Multiple regression analysis has become the workhorse of quantitative analysis in many social sciences. In the process of learning statistics, a thorough discussion of the multiple regression model constitutes the bridge between basic statistics and advanced statistical and econometric techniques, which are all in one way or another extensions or alternatives to the linear ordinary least squares approach of multiple regression. Hence mastering multiple regression opens the avenue to a wide variety of more sophisticated techniques for quantitative social science data analysis.

The course starts by discussing the derivation of the ordinary least squares principle in a bivariate model and then extends the idea to the matrix representation of the multivariate model. A brief introduction to the matrix algebra of regression analysis is included. Then, the standard errors are derived and fundamental criteria for model adequacy (t-test, F-test, coefficient of determination) are discussed. It then proceeds to diagnosing model robustness and fit in more detail, covering multicollinearity, including tests and remedial actions, as well as measures for leverage and influence of observations. Furthermore, residual structures will be discussed. The assumption that standard errors are identically distributed across observations (homoscedasticity) and the assumption that residuals are uncorrelated (no autocorrelation) will be covered along with test procedures and remedial treatments. Finally, some deviations from the linearity assumption will be considered. We will first discuss situations with categorical independent, then explore situations in which the effect of one variable depends on the level of another variables, and end by studying nonlinear associations between the independent variables and the dependent variable.

In the lab exercises, several data sets, in particular individual-level data from the European Social Survey will be used. This data is freely available at www.europeansocialsurvey.org. Topics will typically be introduced with a dataset containing aggregate political economy data. All lab sessions will use the software package R (www.r-project.org). The course is set up as follows: First a topic is presented in the lecture, then students are asked to work on a set of tasks in which they stepwise develop and test a regression model, using the ideas and formulae presented in the lecture. Participants have to bring a laptop computer with a recent version of R installed.

The course assumes proficiency with descriptive and inferential statistics at the level of test theory.

Course Reading

Field, A., Miles, J. and Field, Z. (2012): Discovering Statistics Using R. Sage.

This description is subject to change at the discretion of the Instructor.

2 Credits

For completion of all work before and during the course, as outlined by the Instructor, and 90% participation and attendance of the course.

2 Additional Credits

Course specific extra assignments. These can include submitting assignments before the course, daily assignments, and/or a final assignment to be completed after the course as decided by the Instructor.  This description is subject to change at the discretion of the Instructor.