Looks Can Be Deceiving: Why Visual Pre-trend Checks Aren’t Enough in Difference-in-Differences

Difference-in-differences (diff-in-diff) is a method whose intuition and simplicity can sometimes mislead the researcher. One of the things that economists frequently do to help pin down the credibility of their diff-in-diff design, though, is rely on visual inspection of event-study plots. The notion is that if the two groups were trending together on average before they were treated, then they’d continue to afterwards.

But in this substack I’ll present a simulation that shows that by relying exclusively on visual checks, one can major mistake. That’s because the pre-trends can conceal significant methodological pitfalls. This substack is just me trying to again emphasize the importance of understanding the role that covariates play in satisfying conditional parallel trends, but it’s also a substack where I emphasize something that gets less attention and that’s the role of “covariate-specific trends in the missing potential outcome”. This post in other words will be about how parallel pre-trends can mask problems related to:

1. Conditional parallel trends

2. Covariate-specific trends

But before we get into it, we have to flip a coin! Best two out of three. Heads I paywall, tails I don’t. Drum roll…

Heads it is! So this is paywalled. But if ever there was a substack post that would entice you to become a paying subscriber, this is it. As I’ve said before, I think differential timing has kind of taken all the oxygen out of the room. There’s some very basic issues that I think are getting neglected, and that is with the notion of our heuristics like event studies for even the simplest of all designs — controlling for stuff in the 2x2. So thank you if you’re a paying subscriber! And others should consider becoming one too!