Topic detectors on SAE features, dynamic per-token selection

📄 Paper · 📝 Blog post · 💻 Code · 🤗 Collection

Small causal detectors that label the topic of every token Gemma 3 reads or writes, from the model's own activations. Each token gets one of seven business topics: Enterprise documents, General news & content, Customer service, Legal, Financial, HR & people operations, and Healthcare. A detector uses only the current and earlier tokens, so it can run while the model generates and its label can switch partway through a response.

This repository holds the best detector for each backbone with this input, chosen by test macro-F1 from a GRU, a causal Transformer and a causal ConvNeXt trained on the same features.

Input

For every token and layer, that token's own 150 strongest Gemma Scope 2 SAE features. Each kept value is written into a column that always belongs to that feature. The columns are the features picked by training tokens, capped at the 6,000 most frequent per layer, about 54,000 columns in total.

All detectors read the last ten layers of the backbone, which stays frozen. Only the detector is trained.

Detectors

Folder Backbone Layers Detector Input width Parameters Learning rate Batch Best epoch Test macro-F1 Test overlap (IoU)
gemma-3-1b/ google/gemma-3-1b-it 16–25 ConvNeXt 54,036 17.0M 0.001 8 15 97.3 94.8
gemma-3-4b/ google/gemma-3-4b-it 24–33 ConvNeXt 54,100 17.0M 0.001 8 15 97.1 94.5

Scores are percentages on the 1,500-record test split of Mercity/topic-shift-10k, pooled over every labelled prompt and response token. Overlap is the per-topic intersection-over-union of predicted and true tokens, averaged over topics. Each prediction is the argmax label, with no smoothing.

Training

Detectors were trained on the 7,000-record train split with prompt and response topic tokens supervised, using AdamW for 15 epochs with weight decay 0.01, dropout 0.1 and gradient clipping at 1.0. Learning rate and batch size are in the table above. The checkpoint with the best validation macro-F1 was kept.

Files

Each backbone folder contains:

  • checkpoint_best.pt: a dict with state_dict, architecture, model (the constructor arguments) and classes (the topic order).
  • results.json: the validation and test metrics recorded at training time.
  • selected_features.npz: for each layer (layer_<n>), the SAE feature ids that own a fixed input column, in column order. A token's top-150 features per layer are written into these columns; features outside the list are dropped.
  • meta.json: the feature cache's settings, including K (features kept per token per layer) and the total input width.

Loading

The architectures live in the project code.

import torch
from huggingface_hub import hf_hub_download
from models import build_model  # run from the root of the SAE-Guardrails repo

path = hf_hub_download("Mercity/online-guardrails-sae-dynamic", "gemma-3-1b/checkpoint_best.pt")
checkpoint = torch.load(path, map_location="cpu", weights_only=False)
detector = build_model(checkpoint["architecture"], checkpoint["model"])
detector.load_state_dict(checkpoint["state_dict"])
detector.eval()  # input (batch, tokens, features) -> logits (batch, tokens, 7)

Part of the collection Online Guardrails using Sparse Autoencoders.

Citation

@misc{mallagundla2026topicscope,
  title         = {Train on prefill, monitor at decode: Using sparse autoencoders for topic monitoring},
  author        = {Mallagundla, Rishikesh and Patel, Pranav},
  year          = {2026},
  eprint        = {2611.12935},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2611.12935}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Mercity/online-guardrails-sae-dynamic

Finetuned
(585)
this model

Dataset used to train Mercity/online-guardrails-sae-dynamic

Collection including Mercity/online-guardrails-sae-dynamic

Paper for Mercity/online-guardrails-sae-dynamic