Oxidane/tmr-ai-text-detector

TMR: Target Mining RoBERTa - AI Text Detector

A robust AI-generated text detector based on RoBERTa-base, trained with Focal Loss and Self-Hard-Negative iterative mining on the RAID dataset.

Model Description

TMR (Target Mining RoBERTa) is designed to detect AI-generated text with high accuracy while maintaining low false positive rates. The model uses:

• Architecture: RoBERTa-base (125M parameters)
• Loss Function: Focal Loss (gamma=2.0, alpha=[0.85, 0.15]) to focus on hard examples
• Training Strategy: Self-Hard-Negative (Self-HN) iterative mining
• Training Data: 50,000 stratified samples from RAID (45% human, 55% AI)

Performance

RAID Leaderboard (Official Results)

Metric	All Settings	No Adversarial
AUROC	99.28%	99.85%
TPR @ 5% FPR	95.79%	99.65%
TPR @ 1% FPR	90.17%	98.56%

Results from RAID Benchmark evaluation on 672,000 test samples (including adversarial attacks).

Held-out Evaluation (100k samples)

Metric	Score
AUROC	99.69%
Accuracy	97.42%
FPR	2.61%
FNR	2.58%

Held-out evaluation on RAID train split (seed=999, excluded training/validation samples).

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_path = "Oxidane/tmr-ai-text-detector"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)

# Predict
text = "Your text here..."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512, padding=True)

with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    probs = torch.softmax(logits, dim=-1)

# Probability that text is AI-generated
ai_probability = probs[0][1].item()
print(f"AI probability: {ai_probability:.4f}")

# Binary classification (threshold=0.5)
is_ai = ai_probability > 0.5
print(f"Prediction: {'AI-generated' if is_ai else 'Human-written'}")

Training

Trained on the RAID dataset (ACL 2024) with Self-Hard-Negative mining: iteratively identifying human samples misclassified as AI, then retraining with these hard examples.

Limitations

• Language: Primarily trained on English text
• Domain: Best performance on text similar to RAID training domains (news, books, abstracts, reviews, recipes, Wikipedia, poetry, Reddit)
• Threshold: Optimized for threshold=0.5
• Out-of-distribution: May have higher false positive rates on casual conversation, short text, or domains not seen during training

License

MIT License

Citation

If you use this model, please cite:

@misc{tmr-ai-text-detector,
  title={TMR: Target Mining RoBERTa for AI Text Detection},
  author={Oxidane},
  year={2025},
  url={https://huggingface.co/Oxidane/tmr-ai-text-detector}
}

Contact

For questions, contact me@oxidane.net