6 Comments
User's avatar
Tomer Stern's avatar

I think it'd be fun to replicate this paper with an ELO approach, like what Joe Weisenthal is doing with measuring the orality of text.

scott cunningham's avatar

Not a bad idea. I can totally do that.

Dr Sam Illingworth's avatar

This is genius, Scott. I've been trialling something myself recently with some qualitative research. What I found really helpful is to set up a Claude agent to effectively learn the methodology from a keystone paper or systematic review paper, and then save that and implement it across the project. Still tinkering, but it seems to be highly effective.

Scott Hancock's avatar

You've mentioned before that so much of the materials on using tools like Claude Code is oriented towards software developers, and that's most of what I've read, and why I've subscribed to your Substack.

At around the 46-minute mark, you ask Referee 2 to review the work, but you don't create a fresh context window. I was surprised to see that, but perhaps you haven't found it to be a practical issue? That's something the software developers are always hyper-vigilant about, so I've just followed that advice when I have been able to play around with CC. But maybe I should just go yolo.

You mentioned powerpoint; I wonder if you can combine your "Rhetoric of Decks" skill with Anthropic's 'pptx' skill. https://github.com/anthropics/skills/tree/main/skills/pptx

scott cunningham's avatar

Oh did I not do that? Good catch. I should've done that. If I didn't do that, then it speaks to a workflow problem. I may not be realizing I'm not in a different window. I have to think about why I missed that.

Doing (Undergraduate) Econ's avatar

“To me, the real mystery of Claude Code is why does the copy-paste method of coding seem to actually make me less attentive, but Claude Code for some reason keeps me more engaged, more attentive?”

I suspect it’s the same psychological force as requiring fresh eggs be added to cake mix. Per AI:

“In the early 1950s, Betty Crocker’s "just add water" cake mix was a market failure because housewives felt it was too easy, resembling a deceptive shortcut rather than "real" baking. Sales soared only after marketers removed the powdered egg, forcing users to add fresh eggs, which provided a sense of pride, ownership, and effort, overcoming the guilt of convenience.”

See also the “IKEA effect.”