Mixtape Mailbag: When Firms Choose Their Own Treatments, Will That Violate Parallel Trends? Sometimes.

Here is a letter for today’s Mixtape Mailbag. I’m excited to dig into this question and think a little bit more about the issues the reader raises about selection. I could’ve put this just as easily in the “Pedro’s Checklist” subsection, but it technically came to me in a mailbag, so I got to put it here. This is for the paying subscribers only, but I do think it’s worth getting the 7-day trial, at least, as it’s very relevant to a lot of situations I see all the time which is the situation when some units endogenously select their own treatment in a panel. When can I and when can I not trust that parallel trends will knowing nothing more than the “treatment assignment mechanism” to use Imbens and Rubin nomenclature, or “selection” using economists’ more generally language? I think this is a general interest post, so consider it.

Dear Scott,
 
Thanks for your great Substack! I’ve been reading it with great gusto for a little while now.
 
I have a question regarding firm financing and counterfactual methods, which I have been working on for a while. In Finland, we have nice registry-based micro datas at our disposal and, thus, we are able to conduct pretty nice (I think) TWFE, staggered TWFE, matching and so on type of analysis with it.
 
To concretize my puzzle, I’d like to draw your attention to this recent study by Bertoni et al (2023) that was published in the Journal of Corporate Finance: The long-term effects of loan guarantees on SME performance - ScienceDirect. The main analysis is a mix of CEM, PSM and did to analyze the changes in sales growth, employment growth and assets growth. The treatment variable is whether or not a firm received a bank loan guarantee. A bank loan guarantee basically means that firm gets a finance, that is (partly) guaranteed by the European Investment Fund (EIF) in this case. As an alternative analysis, the authors also do a TWFE, staggered TWFE (Sun and Abraham) and IV. The results in almost all specifications are positive, that is, in general the effect of a bank loan guarantee produced positive effects on firm outcomes.
 
The thing that puzzles me a little bit is the identification of TWFE model in the presence of selection. It’s obvious, I think, that the firms that receive finance are quite different from other firms. For example, they are probably more growth oriented than other firms. Furthermore, it’s kind tautological that if a firm received finance for, say, investing in new machinery, the turnover and sales will increase as the production potential of the firm grows. Also, investments are not made randomly, but the firm owners probably already feel or at least anticipate a growing demand for their products. In other words, I think that the selection bias can be potentially strong.
 
My core question is, assuming there is no good natural experiment at hand, how should one approach the estimation of causal effect when there clearly is selection? Given your knowledge on the matter, do you think there are methods that are “exactly” suited for this type of problem?
 
I apologize if my description of the problem is a bit disorderly, but I’m happy to clarify and elaborate if needed. 😊
 
Best,

Analytical Financier

Saying Back the Question

This is an interesting question. I didn’t have access to the full paper at the moment that Analytical Financier is referencing unfortunately, but I got the gist of it. And unfortunately, I don’t have a finance background, so this is my best effort to understand the institutional context of the original paper (which is linked to above in the quote).

The data is from France, but the intervention is a bit complex and I think matters for the reader’s question. Briefly, the European Investment Fund (EIF) has chosen to cover higher risk loans, which here is called a guaranteed loan. At first I thought it was France, bc the data is from France, but it's actually the European Investment Fund is doing this but the authors are focused on France, presumably because that's the context where they've collected data.

The nature of the loan guarantee is that the EIF is going cover anywhere from 50-75% of the potential losses on the principal of these loans, which therefore means -- I think -- that the banks are taking on higher risk loans. These are clients with higher risk on default, in other words, but they are lower risk to the bank given the guarantee as a large share of the losses is borne by a third party -- i.e., the EIF. Ordinarily, a lender bears all the risk, and thus these loans have become ironically safer which I'm assuming should lead to the bank holding more of them in their portfolio.

So that’s sort of the blanketing program covering all of France, and since it covers all of France, how then will the authors of the original study use difference-in-differences? And that’s where I think the authors’ concern comes in. The treated banks will be those banks who endogenously sorted into offering these loans. If you, like me, need an analogy for this, here is one. Let’s say that you are estimating the returns to college major, and you have data on students earnings in a panel, both before they majored and then after. You want to know the effect of majoring in mathematics. Can you just use difference-in-differences? Can you use difference-in-differences, even though they are technically not being told to major in mathematics, but presumably choosing the major based on advice from their guidance counselor or their own introspection and if so when can you and if not when can you not and why?

This is a question about selection and parallel trends. Seems more and more like this is coming up, and maybe it is coming up because I keep writing about this paper by Ghanem, Sant’Anna and Wüthrich, or maybe it’s just in the water. I’ll call it GSW just to keep it separate from the corporate finance paper. But the point is, endogenous selection of a treatment can pose problems for difference-in-differences as it may violate parallel trends, but it will ultimately depend on why some banks chose to take advantage of this program but not others.

The Selection Setup