Scott's Mixtape Substack

Scott's Mixtape Substack

Claude Code

Claude Code 15: The Results Are In: Can LLMs Replicate a PNAS Paper? (Part 2)

69% agreement, $11, and 2.6 hours later

scott cunningham's avatar
scott cunningham
Feb 06, 2026
∙ Paid

TL;DR

I used gpt-4o-mini to replicate the text classification from Card et al.’s PNAS paper on 140 years of immigration rhetoric. Here’s what happened:

  • Total Cost of this Exercise: $10.99

  • Total Time: 1 hour set up (yesterday’s video), 2.6 hours (we expected 24), 1 hour of analysis today = 4.6 hours.

  • Agreement with original classifier: 69%

  • Key finding: The po…

User's avatar

Continue reading this post for free, courtesy of scott cunningham.

Or purchase a paid subscription.
© 2026 scott cunningham · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture