Deciphering the ATE in LaLonde's Non-Experimental NSW Sample using a Simple Decomposition Method
A Causal Inference Puzzle
When I was a little kid, I had a class where we were given logic puzzles to teach us critical thinking skills. I loved doing them. The ones I loved were relationship puzzles and an example might be this.
Alex is Barbara’s brother.
Dan is Carolyn’s father.
Barbara is Dan’s daughter.
Eric is Alex’s nephew.
Carolyn is not Barbara’s sister.
And my job was to determine how Eric was related to Dan.
I loved those puzzles. We’d get them in this one class in the fourth grade, and it was my favorite of all the assignments in that class. I was routinely the best at it, and as I was not the best at many things, I was amazed there even was something they prioritized teaching us that I was so good at it, even if I didn’t understand what this was generally was. The missing information was always present if you knew how to reason. I just remember the feeling more than anything — a feeling of euphoria and relief at the same time. I didn’t like puzzles as a kid or now, but I loved these code breaking games.
Today’s substack is like me doing those old puzzles; consider it a causal inference cypher. I will use the two datasets that Bob LaLonde put together — one experimental, one non-experimental — and with the simple difference in means decomposition, I’ll plug in missing information contained in each to deduce a plausible estimate of the average treatment effect (ATE) in the non-experimental group and I’ll suggest that makes interpreting LaLonde’s results a little different than maybe you’ve ordinarily thought.
I’ve never seen anyone do this before, which I think has around four explanations. It could mean
it’s not interesting, and no has ever done it as a result
someone has done it before but I just have never seen it,
it is interesting, but I’m about to do it incorrectly,
it’s kind of interesting and maybe I basically get it right.
Regardless of which it is, I’m going to use all the information I can find from two datasets, fill out as much as I can with certainty, make a plausible conjecture on one missing thing, and then assert that the ATE in the non-experimental dataset is a negative $4,892. In other words, the ATE in the non-experimental dataset was always negative, not positive, and definitely not the ATE found in the RCT, +$886. How? Let me explain my reasoning.