Mixtape Mailbag: Suggestions for Synthetic Control, as well as Callaway and Sant'Anna, on a Reader's Project with a Few Treated Units

Discussing some of the ways to present CS, what it is and is not doing, whether it can incorporate state and year fixed effects, and how I might go about it differently.

It’s been so long since I did a Mixtape Mailbag. And for those who have subscribed lately, that’s when someone writes in and write a response. I’ve been letting the letters pile up and am trying to work through them. Today is an email I got from a postdoc researcher who attended an old workshop on synthetic control I did earlier this year. They have a question about evaluating something called “Universal Licensing Recognition Laws”. They sent me a deck of slides and asked for my feedback and advice. So today I’m going to look over their slides and answer their question. Let’s see if I can get all this done before my conference. So let’s listen to “Synth in Seattle”.

Dear Scott

I want to first thank you for your amazing workshop, this is probably the best workshop I have ever attended after grad school, I really like it and I learned a lot!

I am thinking of a specific legislation that has yet to be adopted in Illinois, but currently there are other 21 states that have already adopted a similar policy since 2013. More details are in the attached slides for a conference (I actually used what I learned in your workshop to produce the event study graphs!).

So basically, I want to focus on Illinois, and compare it to other states that have already adopted this policy. In my current analysis, I use neighboring states of MO and IA as the comparison group, which I am now thinking might not be that good of a choice, since MO and IA are less developed than IL which has Chicago, and they are also less pro-labor relative to the pro-labor movements in IL. I plan to switch to CO and PA which are more comparable to IL when using csdid.

But I am also thinking that whether I can use a synthetic control method, to treat IL as a treatment group with no treatment, and all of the other 21 states as the control group with treatment and apply different weights to those states. Based on what we talked about today, an ideal output graph would be the spaghetti graph you have demonstrated in Stata. But in today’s example, we see that Texas clearly has a treatment effect after treatment, while other control states do not have that clear of an effect. Thinking of using synthetic control reversely, the graph would be that there is no clear effect for IL, but clear effects for other states that have adopted this policy. But since in my case the adoption time for the policy is different for different states, I am not sure this method would work (although my understanding is that we will talk about different variations of synthetic control tomorrow).

Sorry for sending you so many details about this question, and thanks again for this amazing workshop. I look forward to your opinion and feedback on this issue.

Signed,

Synth in Seattle

Dear Synth in Seattle,

It’s been several months since you wrote me this letter, and I apologize for being so delinquent in my response. I’m hoping that in that time, though, you have made progress on this project and that this letter will not be too disruptive. Take everything I say with a grain of salt. As I am not an expert in your area, I wouldn’t take it too seriously if I end up making suggestions that are on a different road than what you have undertaken since then.