Deciphering the ATE in LaLonde's Non-Experimental NSW Sample using a Simple Decomposition Method

A Causal Inference Puzzle

When I was a little kid, I had a class where we were given logic puzzles to teach us critical thinking skills. I loved doing them. The ones I loved were relationship puzzles and an example might be this.

• Alex is Barbara’s brother.

• Dan is Carolyn’s father.

• Barbara is Dan’s daughter.

• Eric is Alex’s nephew.

• Carolyn is not Barbara’s sister.

And my job was to determine how Eric was related to Dan.

I loved those puzzles. We’d get them in this one class in the fourth grade, and it was my favorite of all the assignments in that class. I was routinely the best at it, and as I was not the best at many things, I was amazed there even was something they prioritized teaching us that I was so good at it, even if I didn’t understand what this was generally was. The missing information was always present if you knew how to reason. I just remember the feeling more than anything — a feeling of euphoria and relief at the same time. I didn’t like puzzles as a kid or now, but I loved these code breaking games.

Today’s substack is like me doing those old puzzles; consider it a causal inference cypher. I will use the two datasets that Bob LaLonde put together — one experimental, one non-experimental — and with the simple difference in means decomposition, I’ll plug in missing information contained in each to deduce a plausible estimate of the average treatment effect (ATE) in the non-experimental group and I’ll suggest that makes interpreting LaLonde’s results a little different than maybe you’ve ordinarily thought.

I’ve never seen anyone do this before, which I think has around four explanations. It could mean

1. it’s not interesting, and no has ever done it as a result

2. someone has done it before but I just have never seen it,

3. it is interesting, but I’m about to do it incorrectly,

4. it’s kind of interesting and maybe I basically get it right.

Regardless of which it is, I’m going to use all the information I can find from two datasets, fill out as much as I can with certainty, make a plausible conjecture on one missing thing, and then assert that the ATE in the non-experimental dataset is a negative $4,892. In other words, the ATE in the non-experimental dataset was always negative, not positive, and definitely not the ATE found in the RCT, +$886. How? Let me explain my reasoning.