Returning home update
Greetings folks. I am back from Europe. I left on May 9th (not the 11th as I previously said) and got back on the 4th of July to Boston. That’s 56 days in Europe across I think I counted half a dozen countries and approximately 10 cities. Quite a romp. I made new friends, saw new places, visited old ones, including Amelia in San Sebastián, my favorite restaurant in the world in my favorite city in the world, and gave a lot of talks about diff-in-diff, as well as my research on the behavior of AI agents, as well as just explaining AI agents for research more generally. I got to teach single day workshop on diff-in-diff, 2-day workshops on diff-in-diff, 4-day workshops on diff-in-diff, 5-day workshops on diff-in-diff, and 75 minute talks on the behavior of AI agents multiple times. It was quite a lot of teaching compressed into a short period of time.
Here was my new jacket I got in Italy that I wore to Amelia. Can you believe I actually picked this entire thing out when I was in Arono? I had to lug that jacket around everywhere, but when I saw it in the window at this little place, I went inside and immediately the guy there started helping me, and it was like he just somehow knew what I liked and didn’t like even better than I knew, which is admittedly a low bar, but alas, he did and I am grateful. I love the fit. I see now how guys wear this kind of thing every day. I’m going to have to figure out a way to justify doing it. He tailored it for me and it just feels like I am wearing the jacket and pants I was born in.
But now I’m back, and I knew when I returned to Boston that something was different because of two things. First, the moment I got off the plane, the experiences of the last two months started to fade experientially and epistemologically. By which I mean the feelings of the travel started to vanish. Alas, the consequences of being 50 years old, having fairly severe myopia from ADHD maybe, and who knows what else. But the second thing I noticed is the experience of Boston-as-home. That too felt different. I left feeling like Boston was home, and I came back feeling like Boston would require getting to know here again, and I knew that because I couldn’t quite remember which key opened which door in my apartment building, and I couldn’t quite remember the name of all the stops on the green and red line.
But it’ll come back to me, and most likely already has. Last night I went to Banner’s bar downtown by the Garden and watched the Mexico-England game on the big screen. Man was that ever an amazing game! The entire place was full of Mexico fans. Check out dude behind me wearing a Mexican wrestler mask.
I was pulling for Mexico because 1) they are our neighbors, 2) I have become more and more interested in reggaeton and while I’m not positive that Bad Bunny is popular in Mexico, I basically now draw a fairly large circle and have a newfound appreciation for all of Latin America including Puerto Rico, Mexico, and all of South America that seems entirely driven by listening constantly to reggaeton, Bad Bunny in particular, and going to see Buena Vista Social Club on Broadway earlier this semester. And 3) we celebrated independence from England on the 4th, so I kind of felt like that should dominate my love of England football that mainly comes from my deep, deep, abiding joy and love for Ted Lasso season 1. Still, that was an amazing game. Give Mexico another 5 minutes in the first half and another 5 minutes in the second half, and I bet that goes a different way. They aggressive, and that header that almost went in were it not for that unbelievable block by England’s goalie was just incredible. But man — Bellingham. What a holy terror he is.
But now I pull for the US, and one they get beat, Spain, and if they’re beat, Argentina. Or maybe in a different order than that, I haven’t decided yet.
Claude Code
But let’s get into Claude Code stuff. I had something I wanted to share. I think I’ve been waiting to share it mainly because I wasn’t sure when or how to share it, because this is sort of connected to the struggles I’ve been facing in projects over the last 9 months as I worked more intensively. It’s more or less about this idea I have been talking about for several months in my talks: the cleavage between “production” of cognitive output and “verification” of cognitive output as a consequence of AI agents like Claude Code.
Recall that all Claude Code posts here will remain free to everyone until after a few days at which point it goes beyond a paywall. This is the 56th essay I’ve written on Claude Code since mid November 2025, and won’t be the last. But if you find this kind of writing valuable, please consider becoming a paying subscriber.
So in my talks, what I often start with are the big conceptual frameworks for thinking about AI agents for empirical research. And those are this:
I show an isoquant for “producing a single unit of cognitive output” which can be anything from a table of regression coefficients, to a poem, to an entirely worked out manuscript.
This isoquant is concave and assumed to come as a level surface from a quasi-concave production function in which a single piece of cognitive output can be produced with different ratios of machine time (M) to human time (H), but which I suggest historically, before AI agents, was extremely “human time intensive”. The machine time was typically just the time it took for Stata to invert a matrix (as an example).
Then I draw a straight line for simplification and say post-AI (agents specifically), the production functions had straightened out and for simplification purposes the level surfaces were now linear.
The relevance for empirical research is that production functions with linear isoquants are called perfect substitutes, and the long run equilibrium in kind of cost-minimizing situation is to choose the mix of M/H that is the lowest cost way of producing that single unit of cognitive output which is to probably go as close to M* (the vertical intercept) as humanly possible.
But see this is entirely about production. The production function is not the verification function. What do I mean? I mean that we can now produce cognitive output very cheaply, but that’s not the only thing that has been done. What else has been done is that AI agents, by taking over large portions of the production of cognitive output, has created an even stronger wedge between the production side of our cognitive output and our ability to evaluate whether that production is accurate, or frankly, even what we intended to make in the first place. The latter is particularly gnarly to be honest, and a bit disorienting. Why does this happen? Let me pivot into a metaphor before pivoting back to this.
Amelia and the production function of culinary experiences
At Amelia restaurant in San Sebastián, there’s several people who are head and executive chefs. The two I have learned most are the executive chef Richard Zamora, a young man from the south of Spain. The other is Marina Serina-Zapper, who is head chef. She’s from, I think, Germany. Mark Alexander Klaar is also head chef, and he’s from Sweden. But then there is also Mariana Tapia, who is the sommelier and manager. Here’s a picture of Richard, but I didn’t get a picture of Marina, Mariana or Mark apparently.
But to my point, they are technically making the food. And yet, if you look closely at their aprons look what it says: Amelia, Paulo Airaudo. Who is Paulo Airaundo? He is the founder of Amelia, sort of your Thomas Keller type from French Laundry in California. Paulo has had an illustrious career in food, one of which was the Michelin star restaurant Arzak, also in San Sebastián.
I actually don’t go to Michelin restaurants for whatever it’s worth as I’m not really a foodie. I ate a bag of Doritos last night. My favorite meal appears to be a smash burger by revealed preference. But I have gone to Amelia every year for three years, and it’s at least on my bucket list to go nine more times through my 50s until 59.
But back to Paolo. Paolo has started several restaurants, including Amelia which is now Michelin 2.
And so, while he did not technically produce the food I ate last Wednesday, it is still the case that without him there would not have been a single meal made. He is the CEO of the kitchen, even if he is now no longer the executive chef or the head chef there. You can see him here, though, standing off to the right.
So, there’s an org chart in the restaurant from your Paolo Airudo, to your Richards and Marinas and Marianas, to everyone else back there cooking and creating. They are all part of the production function of the meal I had that night if “meal” is even the right word for it. Maybe “experience” is the better word, but the point is, they are all part of it. But there is a clear separation, nonetheless, between the various factors of production.
Production functions are not verification functions
Well, consider this possibility. In tech, there are often thought to be two types of economists, and I’m going to extend that idea here. There are the managers and there are the individual contributors. What’s the difference exactly? Well, the managers are sort of like the people who manage research assistants in academia, or perhaps run labs. Think of Raj Chetty and Opportunity Insights. Is Dr. Chetty himself writing the actual code anymore? I don’t know, but if I had to guess, I’d say probably not. My hunch is that his lab has dozens of RAs. And part of what Dr. Chetty’s human capital has been focused on in the scientific production function is in the management side of producing research. This is obviously a historic part of the production function of science too. Let me explain this part by referencing Amelia restaurant.
Well, that’s what I mean here. Raj Chetty is the Paolo Airudo, undoubtedly, in the production technology that Opportunity Insights produces. Opportunity Insights is arguably the Amelia, a Michelin 2 or Michelin 3 in Dr. Chetty’s case probably, of a production function that produces cognitive output.
But what’s different I think is that AI has not, and probably will not any time soon, infiltrated restaurants like Amelia. I mean, I take that back. I bet you good money some kid out there is the next Paolo Airudo, and they are absolutely using AI agents to develop innovative recipes and culinary experiences never before seen, and probably as a result of that, they may even get to “skip the line” altogether and never step foot in a place like Arzak, French Laundry, or Amelia. They just have a $200 a month subscription to Claude Max, and use it intensively to learn everything they can to innovate that historically otherwise required touring a global circuit of Michelin restaurants throughout their formative years, moving up from dishwasher to sous chef, to they hope, maybe one day head chef. Their “long hours” look very different, I suspect, than their counterparts even one generation earlier.
And yet what I mean is that still, at this exact moments, our robot technology is pretty primitive. Even if robots probably can reliably crack an egg and mix and cook and all that, I don’t think we are anywhere close to that taking over the kitchen, except for maybe your Taco Bells or something. I wouldn’t be surprised if that actually was on the immediate horizon, but for the top — no, I’m skeptical I’ll see that any time soon.
I digress. My point is that Raj Chetty is probably a manager type. He’s the founder and CEO of Opportunity Insights. He is not making the rolls — not directly. Now he had made the rolls — he had his own global circuit of working his way up, but eventually he worked his way up, not to being the one responsible for physically making the rolls, but rather being the one who conceived of the roles and hiring the talent, and designing the entire experience, and then probably most of all — managing its execution in various subtle and explicit ways. But he is not himself directly coding. He is not in that sense producing it. He is not himself the intermedia level of production.
And that is because cognitive output is like many modern production functions something which passes through hands at different stages of production. There is the flour, there is the water, there is bowl, there is the flat, clean surface, there is the oven, there is the kneading, there is the electricity powering the oven, there is the human being expertly mixing precise ingredients, there is the time spent waiting as energy forces the conversion of all those mixed ingredients into the rolls.
Food is interesting. The production side and the verification side of those meals happens on two levels.
Customers as the verification function. Customers do not produce the food. They verify the food by consuming it. They announce whether it is good or bad, either by writing reviews as critics, or simply by enjoying or hating the meal. This is akin to the role of peer review, editors and referees. Customers and referees are the demand side of science and play important roles in the overall delivery of meals to the community.
Chefs as the verification function. But even before the food ever goes to the customer, verification is happening still on the supply side itself. Chefs are also playing a role, internally, in the verification process. It’s just not the same kind of verification. They psychologically know whether the food is adequate or not simply by being the ones who made it. Why? Well, for one thing, they use their eyes to see if the hue around the rolls are beautiful. They know what the rolls should look like because they have made them and they possess so much human capital around roll production that it is borderline impossible to explain it to anyone which I think is probably why the apprenticing of chefs is such an explicit part of the acquisition of human capital in culinary production. You do not merely go to the Culinary Institute of America to become a chef. You work in kitchens, under other chefs, who teach you through experience and learning by doing in such an intensive environment that by end the human capital that cannot easily be conveyed through cookbooks and classes is simply learned. The Polyani Paradox, in other words, according to David Autor.
AI Agents have ripped apart production and verification within the supply side
So, here is my overarching point. Historically, the individual contributors knew their work because they made it. And over time, they learned more and more skills about verification because they made it. The “psychology of feeling-knowing” is such a crucial part of verification and you gained that kind of deep feeling-knowing by doing.
And historically it was quite difficult to do anything because the marginal cost of producing cognitive output was pretty high. I made this graphic (with Claude Code) to illustrate my point.
When the marginal cost of producing cognitive output was MC_0, then we produce Q_c* cognitive output, measured as papers, patent applications, inventions and grant applications. They are not yet verified on the actual demand-side necessarily, which is not a stage I’m focused on, but they are produced, they are costly to do, and they use production technologies that require attention and human time, both of which create skills, human capital, and ultimately the verification of accuracy and fidelity.
But now, the individual contributor — put aside your Raj Chetty’s for a moment — does not do. Rather they coauthor and direct Claude Code to do. And they can produce virtually anything in their imagination. They are the Green Lantern of science possessing that space ring. People often think Green Lantern’s power comes from the ring, but it both does and doesn’t. The ring itself is inert, powerless, dumb even. What the ring does is that it activates the wielder’s imagination. Hal Jordan was thought to be a powerful Green Lantern in the Green Lantern Corp not because his ring was more powerful, but because his imagination was, his courage was, his resilience was. His calm under pressure was.
Well, AI Agents have made all of us Green Lanterns. They are our ring, and we can do anything — I think literally anything. Which in my metaphor means that the marginal cost of producing cognitive output has plummeted to probably epsilon for all practical purposes. The private costs I mean; the marginal social costs, well that’s a different matter entirely. For all we know the private marginal costs and social marginal costs are the same, or they are extremely different, or will be priced in, or won’t. Even if OpenAI and Anthropic ultimately price private use at marginal cost, that likely won’t internalize future costs like existential risk as the property rights over that are basically undefined entirely to nonexistent. We are at the beginning of figuring that out, and theorists and activists and legislators are fast at work. But I’m not talking about the social marginal costs. I am talking about the private marginal costs and those have fallen by a lot.
So, what has that done? I had Claude make this picture for us. This is basic intermediate microeconomics. When marginal cost fell to epsilon, we gained two types of output. First, we gained the dark blue colored rectangle that I am for now calling “Old work, now nearly free”. This is the stuff we were already doing, we just do it using far less human time. These are our regression coefficients in a table that would’ve taken us a long time to make, but we were going to make them. We just used almost no time.
But then there is the green triangle which I’ve just called “New work that would have never been done at all”. And that’s the production of new work. That’s a reduced form oriented PhD student deciding to tackle theorems for a game theoretic model. That’s the structural microeconomist deciding to write a paper using regression adjustment to estimate the ATT for a job training program. That’s some young woman from Siberia running regressions for the very first time. This is new work entirely.
Okay, so her is the issue. The dark blue and the green are all “new output”. But see, all of the output has been produced with very little to almost no human time, which means all of the “verification” that comes from simply doing — the kind of Polyani-style knowledge you simply know because you made it — is not there. But I’m not exactly sure how best to convey this idea, so for now I have this graphic. I reserve the right to tinker with it, but I’m trying to just say that it’s entirely possible that you can evaluate the old stuff done with less time fine. And maybe you can verify some of that new stuff. But you really aren’t tasting anything when the individual contributor is producing with the AI agent as their own personal magic wand. And so there is going to be things both that they cannot verify because they did not directly make it, which will actually include even the old light blue stuff. And probably some of that new output they theoretically could verify if they had the time to do it, or used the time to do it. After all, regression adjustment isn’t rocket science.
But there’s also going to be new stuff they cannot verify at all, which is the hashed stuff below.
Aggregate Errors
So here is my final point and I’m done. Errors. Total production of errors. If the marginal cost of production has fallen by a lot, but the demand curve if steep, then the elasticity of supply is actually small and the amount of additional cognitive output we is small enough that we can figure out probably the verification part rather easily. That’s something like this.
I mean, all of it has to be verified keep in mind, and so it’s not actually straightforward how to verify even the old Q_c* cognitive output if you were not the ones who actually made it, but that’s fine. Let’s just assume that part away for a second. Let’s ignore depreciation of human capital and skill that can easily creep in if a person literally is not doing the actual production themselves, but relies excessively on the machine time for the production of cognitive output. Let’s ignore the important human capital acquisition that comes from mixing and kneading oneself.
We just have to verify the old Q_c* stuff plus some small additional Q*_ai - Q*c stuff. I mean all of it has to be verified, but that’s the bump we get from agents. And I think some people think that bump is just that — a bump. And maybe in the short-run it will be. But in the long-run, if all we get is that bump, then it would be the first time ever that it was like that. Nowhere else is it the case that when the factors of production are no longer fixed, that we continue to exist on the same point of an isoquant as when they were fixed. I personally am not going to adopt such a naive view of the world anyway. I think this is probably more likely an accurate description of the near future if not the current present.
I think we are going to see a lot of “new work”. And so we have to do two things. We have to verify all the old work, Q*_c, which we did not personally hand make and thus the latent knowledge that you get from hand making is not there to help us. But now we have to verify new work that we never would’ve made, which already implies either that the time constraints were excessive but now have been relaxed (for sure) thus enabling us to tackle new stuff, but also things we are reaching for that would’ve taken “infinite man hours” without AI which is to say it was impossible for all practical purposes.
That is the reality in my opinion. That is where we are, and that is a nontrivial problem because even if the error rate falls from AI agents, the fact that verification is disconnected from first hand production will mean there is still some error rate and it’ll be non-zero, and therefore depending on these implied supply elasticities themselves, the amount of additional errors as a function of that Q_ai - Q_c can mean more aggregate errors in the scientific record.
To me, this is not a fatal problem. Not at all. I mean it is and it isn’t. If we think our work matters, then it is a fatal problem, but we have had fatal problems before. The invention of the automobile led to the invention of the seat belt. It led to the invention of better braking technologies. But, you know what else it led to? Moral hazard. I remember one economist (was it Arrow?) who said if you really wanted to get automobile accidents to fall, you didn’t want seat belts — you wanted a giant dagger protruding from the center of the steering wheel aimed directly at the driver’s heart. That’ll make them slow down, and that’ll make them safer — for themselves and others.
But you know, daggers and seat belts are not the only solutions. Round-abouts were a solution for increased safety too. Stop signs were a solution. Fines for drinking and driving were a solution. And yet, automobile fatalities remain an extremely large problem in the United States. Why? Because the elasticity of demand for driving is so massive that when the cars get safer, we drive more, further distances, and more recklessly.
I think this is a solvable problem, but to pretend it is not a problem in the first place I think is a mistake. So somehow we have to figure out — those of us who are individual contributors that is who have never had resources for RAs (me) and have had to do it themselves (me) — how to reintroduce the psychology of knowing that assists us in verification when we are not driving the car on producing research. And I think there are solutions, but we are still working on them. But they are crucial because I do not think at the moment whatsoever that AI agents are perfect substitutes for verification even if they are near perfect substitutes for production.
I believe it was Peltzman not Arrow who recommended the spike