At the Northwestern University annual conference on causal inference’s main event last week, we were fortunate enough to have the newly tenured associate professor at Brandeis University, Tymon Slozynski.
Scott when I think about Angrist’s JMP, if the LATE is so interesting because the draft itself is interesting, then why do we even need the IV framework? Why not just focus on the reduced form? Shouldn’t then we interpret the first stage as less of an instrument for service and more of a rescaling of how many people were actually affected by the draft?
I think that’s a good question. The RF would give us the effect of the draft but not the effect of military service so it wouldn’t technically be the LATE. So the question then is do you want to know the effect of military service or the draft. But this is where the exclusion restriction comes in
Let me try to ask this in a different way that might be useful. What is the counterfactual policy that targets the same population from the LATE that isn't a draft (the instrument being studied)?
I mostly just think this is actually quite a bad example of the point you're trying to make. The Angrist paper is the right LATE precisely because there's really only one policy (or class of policy) of interest where the answer to his question is interesting *and* the LATE is the right one, which is inducing involuntary service. (I think. Please you or someone else should feel free to push back on this!) But then why do we care about the effect of military service relative to just the effect of the draft itself? My instinct is that we don't.
This strikes me as a general issue with serious IV designs (as well as RD designs). You have to argue that the compliers are not only compliers for your IV of choice, but also for a counterfactual policy, and usually one that is quite different from the policy being studied, especially if you are trying to estimate general parameters like elasticities of demand or supply. I'm curious if you have a better example in mind where this assumption seems more clear.
To be clear, I am totally with Imbens (and you) that LATEs are still very informative, and I have written both RD and IV papers of my own, that make general claims to external validity! But I do wonder if this issue is why (in my mind) DD designs have overtaken the other two in recent years.
Well part of the reason DD probably took over isn’t because of a concern in the ATT over the LATE imo. The discussion of the ATT by econometricians according to Pinto actually doesn’t get focused attention until the mid “ought as” (eg 2005+). But DD is much older than that. The more likely explanation is: 1) hard to defend exclusion, 2) hard to find great instruments, 3) increased availability of rich panel data, 4) perceived credibility due to event study pre trends, 5) may be the only realistic way to study large government programs, 6) widespread belief TWFE was sufficient for identification. But if you read closely early or even honestly pre-2018 did papers, you will rarely see nuanced statements like ATT. Much more likely someone says “the” causal effect (ie constant treatment effects). And if you do believe in constant treatment effects, then the LATE is the ATE and it solved the problem of selection. Let me read your other part now.
Scott when I think about Angrist’s JMP, if the LATE is so interesting because the draft itself is interesting, then why do we even need the IV framework? Why not just focus on the reduced form? Shouldn’t then we interpret the first stage as less of an instrument for service and more of a rescaling of how many people were actually affected by the draft?
I think that’s a good question. The RF would give us the effect of the draft but not the effect of military service so it wouldn’t technically be the LATE. So the question then is do you want to know the effect of military service or the draft. But this is where the exclusion restriction comes in
Let me try to ask this in a different way that might be useful. What is the counterfactual policy that targets the same population from the LATE that isn't a draft (the instrument being studied)?
I mostly just think this is actually quite a bad example of the point you're trying to make. The Angrist paper is the right LATE precisely because there's really only one policy (or class of policy) of interest where the answer to his question is interesting *and* the LATE is the right one, which is inducing involuntary service. (I think. Please you or someone else should feel free to push back on this!) But then why do we care about the effect of military service relative to just the effect of the draft itself? My instinct is that we don't.
This strikes me as a general issue with serious IV designs (as well as RD designs). You have to argue that the compliers are not only compliers for your IV of choice, but also for a counterfactual policy, and usually one that is quite different from the policy being studied, especially if you are trying to estimate general parameters like elasticities of demand or supply. I'm curious if you have a better example in mind where this assumption seems more clear.
To be clear, I am totally with Imbens (and you) that LATEs are still very informative, and I have written both RD and IV papers of my own, that make general claims to external validity! But I do wonder if this issue is why (in my mind) DD designs have overtaken the other two in recent years.
Well part of the reason DD probably took over isn’t because of a concern in the ATT over the LATE imo. The discussion of the ATT by econometricians according to Pinto actually doesn’t get focused attention until the mid “ought as” (eg 2005+). But DD is much older than that. The more likely explanation is: 1) hard to defend exclusion, 2) hard to find great instruments, 3) increased availability of rich panel data, 4) perceived credibility due to event study pre trends, 5) may be the only realistic way to study large government programs, 6) widespread belief TWFE was sufficient for identification. But if you read closely early or even honestly pre-2018 did papers, you will rarely see nuanced statements like ATT. Much more likely someone says “the” causal effect (ie constant treatment effects). And if you do believe in constant treatment effects, then the LATE is the ATE and it solved the problem of selection. Let me read your other part now.
Thanks Scott! I appreciate your insights here.