01
Start with a comparison
A refusal is an observation. To learn from it, I need a comparison: the same prompt at another time, another system under the same setup, or a controlled change to the context. None gives me direct access to the provider’s intentions.
I would track refusal rates, framing, and changes over time separately. A single label such as “suppression” can conceal several explanations, including a capability limit, a policy, or a change in the surrounding product.
02
What the record can support
The probe battery, system versions, sampling settings, and scoring procedure need to travel with the result. A repeated difference can support a claim about behavior under those conditions. It does not, by itself, tell us which internal layer caused it.
The tier comparison is a related example. Its central limitation matters here too: observing an API response does not cleanly separate training from runtime policy.
03
The accountability question
I think behavioral measurement can complement access to documents and explanations. It cannot replace them. Providers can restrict access, and a missing baseline may make a question impossible to answer from outputs alone.
What I would like to see is a public claim about a system paired with a test someone else can inspect and challenge. The open question is which comparisons are informative enough to support that challenge.