01
What can be checked publicly
The public archive contains response files for a shared 44-probe battery across Haiku 4.5, Sonnet 4.6, and Opus 4.7. For this check, I selected the latest committed run for each tier with all 44 responses and no recorded errors. The receipt names the exact files and source commit.
One earlier Opus file contains 44 errors and no successful responses. It is excluded. I am comparing saved responses, not rerunning the models or making a claim about their current behavior.
02
What “drift” means here
The comparator flags a changed refusal label, a response-length change of at least 50%, a change in the first 100 characters for persona/identity prompts, or low word-set overlap. The aggregate is the mean of rounded per-category flag rates.
That is a screening heuristic. Two answers can mean the same thing while using different words, and a change in length can trigger a flag before any other comparison is considered. The length calculation also depends on which response is treated as the baseline, so pair order matters.
The repository’s severity labels are thresholds on that heuristic. They are not measurements of safety risk.
03
What the archived comparison returned
In the selected pair order, Haiku–Sonnet scores 0.878, Haiku–Opus 0.792, and Sonnet–Opus 0.798. These are category-average flag rates, not the fraction of answers judged substantively wrong or different.
All identity, refusal-category, and safety-adjacent responses are flagged in these comparisons. Crucially, none of the selected comparisons records a refusal-label flip. “100% drift on refusal probes” therefore does not mean “100% changed refusal decisions.”
The seven capability prompts have zero or one flag per pair. That is too small and too narrowly scored a sample to establish a shared capability backbone. It tells me where to look at the responses more carefully.
04
What I changed after checking
The earlier version quoted expanded-battery scores of 0.91–0.93, a 0.17 baseline, and a strong repeatability claim. The public validation scripts reference later snapshots, including the expanded battery, that are not included in the checked commit. I cannot reconstruct those claims from this public release, so I have removed them from the result presented here.
The earlier interpretation also reached too far into the system. API responses do not isolate weights from provider policy or other runtime behavior. I can describe the observed text differences without claiming to have located their internal cause.
05
The question that remains
I still want to know which differences matter when someone switches models or a product routes requests between them. This battery gives an initial set of responses to inspect. A stronger study would need a justified semantic scoring method, more representative tasks, repeated runs, and a clearer account of the surrounding system.
For now, the useful result is smaller: preserve the prompts, outputs, scoring rule, and file selection, so someone else can check what the number actually says.