Mixtape Mailbag #3: How Might DAGs Win Hearts and Minds in Program Evaluation?
Skepticism and Conjectures about DAGs and Design
The Mixtape Mailbag comes out every Monday morning. It’s a place where readers write me their questions and I try to tackle them. Sometimes I bring a guest with me, but this week I come alone. If you would like me to tackle a question you have, just shoot me an email at scunning@gmail.com. I’ll add it to the queue and work my way to it soon.
In this week’s mailbag, a reader named ST asked about me directed acyclic graphs, or DAGs. DAGs, for those who don’t know, are a graphical approach to causal inference in which a theoretical description of the causal links between the variables is established ex ante, and then knowledge about those links are used to identify various causal effects with do calculus. It was developed by many people, notably Sewell Wright and later the computer scientist, Judea Pearl and his students and collaborators. It represents one of the most important contributions to causal inference to date, next to the experimental design itself. Listen to ST’s letter. They ask me more or less what would it take for DAGs to become more influential. I try to tackle the question and ended up writing, as before, a long answer. Here’s ST’s letter, followed by my response. Most of the response is behind the paywall, but this time I left you with a little teaser. Cheers!
Dear Scott,
I hope this is the right way to contact you regarding the mixtape mailbag.
My question is this:
There seems to be a lot of skepticism about graphical models for causal inference in economics, probably most famously discussed in Imben's 2020 paper.
Based on your knowledge of the history of economic thought, what would it take for graphical inference to "break through" in econometrics? Or do you think this will never happen?
I look forward to your thoughts!
Best regards
ST
Dear ST,
It’s 5am right now and I have three kitties asleep nearby. Ronnie and Betty, both littermates and sisters, are asleep on the bed, and Clara, the new rescue kitten from outside, is on the corner. They still haven’t quite accepted her. Simba, the orange rescue kitty, is asleep somewhere around here. This was his second night in a row to spend the night. I’m not sure I’m going to let him back out this time.
I love your question. Thinking about it took me down a rabbit trail about the necessary conditions of scientific revolutions in general, and within econometrics specifically. The scientific revolution of causal inference within economics is a subject I think about often, but I hadn’t thought of the thought experiment involving its replacement before, and in trying to answer your question, I was forced to remember what little I could remember of Thomas Kuhn. While this answer may not be fully satisfying, I hope it is at least entertaining and thought provoking to read.
DAGs Would Replace Design One Funeral At A Time
Why do scientific revolutions happen and when will one happen within econometrics again? To frame the question in this way is to suggest that scientific progress is a kind of creative destruction where antiquated methods are replaced by presumably better ones. One way we will know that DAGs have replaced the dominance of the experimental design is when we see the dead bodies of proponents of design because ultimately scientific revolutions happen one funeral at a time. While grisly to talk this way, it is probably worthwhile to start my response on the more realistic footing from a historical perspective.
Keep reading with a 7-day free trial
Subscribe to Scott's Substack to keep reading this post and get 7 days of free access to the full post archives.