Today’s post will share some pictures from Berlin, but also pick up where an earlier post on covariates in diff-in-diff left off. This time, I just wanted to blend formalism with some thoughts I have. I will show you the derivation for when covariate imbalance will break parallel trends and when it won’t. And then I’ll just share my opinions on critiquing the weakness of a diff in diff. So if you are interested in either check it out.
But I also flipped a coin 3x (or had python do it anyway), and it came up two heads out of three which means this’ll be paywalled. Thank you again for your support!
Leaving Berlin
Guten tag! After four days and three nights in beautiful Berlin. I am heading to the airport where I’ll hop on a plane to Pisa — back to Tuscany! I just left there! — and then book it over to Lucca. I spent two weeks in Lucca, Italy last year and had such a nice time. I’ll be presenting my paper on discretion and the behavior of AI agents. Ask IMT about it and see if they’ll let you peek your head in!
But I had an absolutely fantastic time at the Berlin School of Economics. I presented at DIW Berlin, and met new people. It was really amazing. We are talking about doing it again next year even.
But for now, I move on. I have two more things before I’m done and head to San Sebastián in the Basque Country, known for its fabulous concha bay, amazing food and synthetic control. This weeks it’s IMT in Lucca, and then starting this weekend, Belgium, to KU Leuven.
But let me pause there, because what I want to do now is pick up on a substack from last week and show you formally the situations where it is mandatory to include coverages in a difference-in-differences.
The core reason is that you need the covariates to satisfy conditional parallel trends, which we discuss in section 4 of our JEL. There’s three separate bias terms though. And they are:
The covariates you need to satisfy conditional parallel trends are imbalanced between treatment and control.
Heterogenous treatment effects across the dimensions of those covariates.
Conditional parallel trends is based on time INVARIANT covariates.
Selection on levels versus selection on trends
It’s quite common to hear someone say that they don’t need to control for covariates because everything that determines the outcome that doesn’t change over time gets deleted with the first difference. Or, put another way, it’s getting absorbed in the unit fixed effect.
But this is not accurate. Diff-in-diff is robust to variables that determine Y(0), the level of the outcome, but recall the bias of diff-in-diff: it is not based on level differences. It is based on trend differences in Y(0).
Thus if you need them to satisfy conditional parallel trends, then of course deleting them won’t help. You’ll need to figure out a way to keep them in.
Imbalance in covariates with changing returns on Y(0) over time
Well related to that time invariant case is the more general problem: imbalance. And this time I want to show you formally what happens, as last time it was only a simple numerical example. But I enjoy working from the regression equation to the formal depiction of the problem and I bet others do as well.
Let’s start out with a two people model. Since we are including covariates, we cannot simply write it down as four averages and three subtractions, so let’s instead write it down as a two way fixed effects regression with additive covariates to illustrate the problem with “changing returns to the covariate over time”. Here is that regression in the first period, period 1.
And here it is in period 2.
So notice that I put subscripts 1 and 2 on \beta. What does that mean? That means that the effect in period 1, \beta_1, may or may not be what it became later in period 2, \beta_2.
