Why your modified model still refuses

The run completed, the metrics looked fine, and it still will not answer. Almost always the same cause.

You measured the reasoning, not the answer

Some models think out loud before answering. They write a long stretch of working-out first, then give the actual answer at the end. If you stop reading before they reach the answer, everything you measured was the working-out — and the tool then spends its whole run improving the wrong thing, perfectly diligently.

The specific trap: some chat templates open a reasoning block in the prompt. The generation prompt ends mid-thought, so the model continues deliberating from the first token, and nothing it produces during scoring is an answer.

What this cost us

400 trials on one model, optimising against chain-of-thought. The run looked healthy throughout. It reached 110 refusals out of 200 and would not improve. After the fix, 50 trials reached zero.

Look at the tail of your prompt

One line tells you:

from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained(path)
print(repr(tok.apply_chat_template(
    [{"role": "user", "content": "hi"}],
    tokenize=False, add_generation_prompt=True)[-40:]))

If it ends with an unclosed reasoning marker, you have the problem:

'<|im_start|>assistant\n<think>\n'          ← open. broken.
'<|im_start|>assistant\n<think>\n\n</think>\n\n'  ← already closed. fine.
'<|im_start|>assistant\n'                    ← no reasoning block. fine.

This differs inside one model family

Do not copy a sibling’s configuration. In the Qwen3.5 family the 4B, 9B and 27B leave the block open while the 2B and 0.8B emit a closed one. Applying the larger models’ fix to the 2B injects a second, unmatched closing tag into every prompt. Check per checkpoint.

Derive the fix, do not guess it

The instinct is to append the closing tag. That is wrong for most models. The correct prefix is whatever the no-thinking render adds after the opener, which you can compute:

msg = [{"role": "user", "content": "hi"}]
think   = tok.apply_chat_template(msg, tokenize=False, add_generation_prompt=True)
nothink = tok.apply_chat_template(msg, tokenize=False, add_generation_prompt=True,
                                  enable_thinking=False)

i = think.rfind("<think>") + len("<think>")
j = nothink.rfind("<think>") + len("<think>")
if nothink[j:].startswith(think[i:]):
    print(repr(nothink[j:][len(think[i:]):]))   # the prefix you need

For the Qwen family this yields '\n</think>\n\n', not the bare tag. Anchor the comparison at the opener rather than the start of the string — at least one model also alters its system prompt when thinking is enabled, so a whole-string diff finds nothing.

Two more traps in the same family

enable_thinking=False is widely accepted and widely ignored. Assert that it actually changes the render. And if a reasoning model scores near zero on anything, suspect the block before suspecting the model — check whether the answer field is empty while the reasoning field is full. The findings page has all three costumes this bug wore.