D02-01AM: Advanced Regression

The course builds on a basic understanding of regression analysis and linear models and provides a more advanced understanding of how these models can be assessed and improved. We will look in detail at the model assumptions, how violations can be detected, and discuss how such violations can be fixed. Heterogeneity of effects will also be discussed in some detail, in particular interaction terms and how they should be used in practice. In the second part of the course, we will then cover generalized linear models and situation with limited dependent variables. After discussing maximum likelihood estimation, we will introduce the logit and the probit model and provide a detailed understanding of how to interpret these models. Then more complex limited dependent variables (count variables, categorical variables) are discussed, and the corresponding models are introduced.

The course is designed to provide participants with a more advanced understanding of regression models. At the end of the course, the students will be able to understand their linear models in detail and assess model violations and learn how to deal with them. Models that do not assume a linear relationship will also be covered in this course, and students will be able to use and build models for limited dependent variables using generalized linear models. We will discuss model fit and model selection, and cover interaction terms in some detail. Each day has a theoretical and a practical component (1.5 hours each), students should bring their own laptop to class with R installed. A basic level of R and a basic understanding of linear regression are prerequisites for this course. Advanced mathematical knowledge or detailed knowledge of estimation theory is not required.

On day one, we will start with a brief review of the linear regression, the model assumptions and the properties of the OLS estimator (unbiasedness, efficiency). In this session we will also discuss model fit and model (mis)specification, potential sources of bias, regression outliers, leverage, and influential data. We will also have a detailed look at R’s linear model object, and how the information stored in that object can be used for model assessment, but also for effective model presentation.

In the second class, the aim is to assess three out of the four model assumptions (linearity, normality of the errors, homoscedasticity). For each of these assumptions, we will discuss how to detect potential violations, and how to correct them. We will discuss both visual and statistical tools to detect these violations. In addition, we will cover distributions and transformations to help improve the models.

On day three we discuss the independence assumption and cover different ways to deal with situations when this assumption is violated (clustered standard errors, fixed effects, random effects), and their advantages and disadvantages. Then we will discuss in some detail situations when the effect of a variable depends on another variable (interaction effect), how such interaction effects can be meaningfully interpreted and presented, and some recent developments in the literature on that issue (what to do if the interaction effect is non-linear, binning and kernel estimator, generalized additive models).

The fourth class will start discussing non-linear models. First maximum likelihood estimation will be introduced and, based on that, generalized linear models and link functions. We will then discuss situations with dichotomous dependent variables (that is variables that can only take on two values) and cover the logit and the probit model. We will learn how to make sense of these models, and how to effectively present them using predicted probabilities.

On the final day we will start by discussing interaction terms in generalized linear models, and how to make sense of them (particularly for the logit model). Finally, we will look at different situations of limited dependent variables, such as count response variables, or categorical response variables, the appropriate link functions in such situations, and the corresponding models.

This description is subject to change at the discretion of the Instructor

2 Credits

For completion of all work before and during the course, as outlined by the Instructor, and 90% participation and attendance of the course.

2 Additional Credits

Course specific extra assignments. These can include submitting assignments before the course, daily assignments, and/or a final assignment to be completed after the course as decided by the Instructor.