Plenty of people modify open models to remove their refusal behaviour. Almost nobody reports what it cost, and the ones who do usually report differences their measurement cannot actually resolve. This site is the other thing: the method, the tooling, and the numbers with their error bars attached.
That last figure is the point. Forty-nine of fifty-one measurements land inside the band where the benchmark cannot tell two runs apart — so the honest summary is removing refusals costs no capability this instrument can detect, which is a different claim from “it costs nothing”, and a much more defensible one.
Every model on all three axes — maths (GSM8K), instruction-following (IFEval) and code (HumanEval) — each drawn against its own noise floor, with footprints and context lengths.
Open the catalogue →The full write-up: what refusal removal costs across three axes, why KL divergence does not predict damage, and the measurement bugs that produced nearly every dramatic result along the way.
Read the findings →Six modules, each from something that cost hours: models that cannot be modified at all, ones that still refuse afterwards, and exports that die after the run has already succeeded.
Work through the modules →What each model costs to actually run: how much conversation 16 simultaneous users each get, and how fast. Memory per token varies 18× across the fleet, and it does not track model size.
See the numbers →Decide whether a model can be modified before spending GPU hours finding out. 23 checkpoints in 48 seconds, no weights loaded.
Source comingA 200-question benchmark carries roughly ±8 questions of run-to-run noise. That is large enough to swallow most of the effects people report, and small enough that a three-question gap looks meaningful when it isn’t.
The tell is simple: when a modified model beats its original — and here it regularly does — you are reading noise. No amount of refusal removal teaches a model arithmetic.
So every number here is a count out of its total, with the band drawn. 156 and 157 out of 200 are obviously the same result. 78.0% and 78.5% are not obviously anything.