hbin0701/sft-model-1008

Rewind run4 SFT model ([rewound] marker, single-token <rewind>)

Qwen3-1.7B fine-tuned to write a <rewind> action while thinking. When the tag is written, an inference controller deletes the last 4 reasoning sections, appends a [rewound] line, and generation continues from there. This is the SFT stage of run4 (before any RL), trained with LOSS=v2 of the code below.

What changed vs. Sangsang/rewind-run1-sft-model

	run1 SFT	this model
<rewind>	4 ordinary text tokens	one added token, id 151669 (a spare embedding row, initialised from <)
after a rewind	nothing marks it	[rewound] is appended where the erased text was
SFT loss	likelihood	likelihood + "don't fire here": -log(1 - p(<rewind>)) at paragraph starts of non-trigger rows (weight 1)
epochs	2	1

Training data: the same 584 rows as Sangsang/rewind-run1-sft-data (146 trigger, 146 post-rewind, 146 negative, 146 clean); post-rewind prompts get [rewound] appended.

System prompt (used in training; use it at inference)

You may write while reasoning. It erases the last part of your reasoning (about the last four steps) so you can continue from the earlier point where things were still on track. Use it only when you realize your reasoning has gone wrong and continuing will not lead to a correct answer; do not use it for routine double-checking. You can use it at most once per problem, and never after . After a rewind, [rewound] marks where the erased part was.

Thinking mode on: the assistant turn starts with an open <think>\n.

Running it

A plain generate() does not perform the rewind. A controller must:

1. stop generation on token id 151669 (<rewind>);
2. delete the last 4 sections of the reasoning (paragraph runs about one topic; see section_starts in the code) and append [rewound]\n\n;
3. resume from that text; after one rewind, ban token 151669 (e.g. logit_bias={151669: -100}).

The 16,384-token budget counts the tokens on the page, so the erased tokens are given back. rewind/eval_rewind.py in the code is a complete controller (vLLM).

Known issue

This checkpoint fires far too often. Measured on 6 correct solutions, P(<rewind>) is 0.15-0.49 at almost every paragraph break after the first, so in RL nearly every solution rewound ~200 tokens in. Cause: the "don't fire here" term is averaged over each row's paragraph starts, which balances at about 146 / (146 + 438) = 0.25 per break. Summing it instead is the planned fix. End-of-SFT log: P(tag | trigger point) ~0.19, highest P(tag) at a "keep going" paragraph start ~0.47.

Training

setting	value
base	Qwen/Qwen3-1.7B, full fine-tuning, fp32 master weights, bf16 compute
epochs / steps	1 / 73 (global batch 8, 584 examples)
lr	2e-6, 8 warm-up steps then cosine, AdamW (wd 0.01), grad clip 1.0
loss	completion-only, mean per example; trigger target = the single <rewind> token
hardware	2x RTX PRO 6000 Blackwell, 19 min

training_log.jsonl has the per-step loss, P(tag | trigger) and the highest P(tag) at non-trigger paragraph starts.

Code

github.com/hbin0701/sclm, branch rl-resolving, commit c427255: LOSS=v2 TAG_TOKEN=1 SFT_EPOCHS=1 bash run.sh reproduces this stage. The private repo needs an account with access.