The model I recommended was not the model I used

What?

Midway through my doctorate, in August 2023, I benchmarked Claude against GPT-3 and GPT-4 on 370,000 words of interview transcripts from my IPA study of 40 participants. I found Claude clearly better for nuanced qualitative analysis, convinced enough that I emailed Anthropic’s press office about it. But the analysis in my thesis, as executed, ran on GPT-3/4: the work was already under way, and I have always stated that plainly.

So what?

The temptation in research is to let the tidier story win, to imply you used the best tool because you knew which was best. The discipline I am most proud of is the duller one: describing what actually ran, not what I would have chosen with hindsight. In qualitative work the instrument is part of the finding, so being honest about the instrument is not a footnote, it is the integrity of the result.

Now what?

I now write the tooling into the method up front: which model, which version, what it touched and what it did not, so the provenance is legible to a reader and to a machine. And when I advocate for a tool, as I did for Claude, I keep the two claims separate: this is what I recommend, and this, separately, is what this piece of work used.