espnet/bagpiper-tts-sft

Bagpiper-TTS SFT

Bagpiper-TTS is an 8B natural-language-guided universal speech-synthesis model. It interprets a free-form text request, produces an intermediate plan and rich caption, and then generates audio. It supports classical TTS, multi-talker dialogue, intent-to-speech, role-play, singing, and general speech scenes.

Bagpiper-TTS accepts text instructions only. It does not accept reference audio and is not a voice-cloning system.

Files and format

• model.pt: native ESPnet weight-only checkpoint, stored as {"module": state_dict}.
• train_bagpiper_tts.yaml: sanitized configuration saved with the selected checkpoint, named consistently with the pre-trained release layout.
• inference.yaml: paper-aligned text-then-audio decoding configuration.
• MANIFEST.json and SHA256SUMS: provenance and integrity records.

model.pt contains 1,382 tensors: the 1,381 BF16 tensors from training plus the deterministically reconstructed FP32 vocab_weight buffer. It contains no optimizer or exact-resume state. It is not a Transformers from_pretrained directory and no vLLM compatibility is claimed.

Download

hf download espnet/bagpiper-tts-sft --local-dir bagpiper-tts-sft
sha256sum -c bagpiper-tts-sft/SHA256SUMS

ESPnet runtime

The release was strict-loaded with:

• ESPnet PR #6503, commit 1992821bfb6d68c6636d1b75340b8b2b591bc015;
• Python 3.11.15, PyTorch 2.12.1+cu129, Transformers 5.5.4;
• FlashAttention-3 3.0.0 at source commit 77aacb68d194ba9af1010eda5eac3e7c0df8e6f6; and
• one NVIDIA H100 80GB HBM3.

git clone --branch titan-pt-weightonly-init https://github.com/jctian98/espnet.git
cd espnet
git checkout 1992821bfb6d68c6636d1b75340b8b2b591bc015
python -m pip install -e '.[speechlm]'
python -m pip install --no-build-isolation \
  'flash-attn-3 @ git+https://github.com/Dao-AILab/flash-attention.git@77aacb68d194ba9af1010eda5eac3e7c0df8e6f6#subdirectory=hopper'

Use espnet2/speechlm/bin/inference.py with train_bagpiper_tts.yaml, inference.yaml, and model.pt. Training backend configuration belongs to the ESPnet recipe and is not duplicated in this model repository.

Paper-reported results

These values are from the paper's checkpoint and protocols; they were not rerun as part of the Hub upload.

Evaluation	Result
Seed-TTS-Eval English WER	1.7%
Mean advanced-task fulfillment, Gemini-3-Flash	4.09 / 5
Mean advanced-task human score	3.69 / 5

The general-purpose tier was evaluated qualitatively. Judge-based and small-sample subjective results do not guarantee performance on arbitrary prompts.

Provenance and validation

• Selected checkpoint: step_276698/global_step276686.
• Raw source: JinchuanTian/bagpipier_tts_ckpt at revision d70ecb45d579242d67da948d98a31ced3f854702.
• Raw SHA-256: d8a53f857760aab476896e486733f34c3ac9945dbbeedeea702c66eb75d457eb.
• model.pt SHA-256: 168fe0aca32f4b1636e4ee2e63ef3880427be352bd6369a1f5ba3b49750f91da.
• All 1,381 source tensors matched the converted file exactly with torch.equal; the only added key was vocab_weight.
• The 1,382-entry state dict passed the native ESPnet strict inference loader.

Limitations and responsible use

The model can hallucinate attributes, mispronounce or alter text, confuse speakers, fail on timing/pitch, and produce artifacts or unsafe content. The headline benchmark is English. Natural-language controls are less deterministic than fixed metadata. Descriptive prompts can still request recognizable personas or styles even without reference audio. Do not use outputs for impersonation, deceptive media, harassment, surveillance, authentication, or to imply that a real person said something they did not say. Disclose synthetic audio where it could be mistaken for a real recording.

Training-data availability and terms are documented at Bagpiper-TTS SFT Data. Third-party component terms are listed in THIRD_PARTY_NOTICES.md.

Citation

@unpublished{tian2026bagpipertts,
  title={Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis},
  author={Tian, Jinchuan and Wang, Haoran and Arora, Siddhant and others},
  year={2026}
}

@inproceedings{anonymous2026bagpiper,
  title={Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions},
  author={Jinchuan Tian and Haoran Wang and Bo-Hao Su and Chien-yu Huang and
          Qingzheng Wang and Jiatong Shi and William Chen and Xun Gong and
          Siddhant Arora and Chin-Jou Li and Masao Someki and Takashi Maekaku and
          Keita Goto and Yusuke Shinohara and Jin Sakuma and
          Chao-Han Huck Yang and Shinji Watanabe},
  booktitle={Third Conference on Language Modeling},
  year={2026},
  url={https://openreview.net/forum?id=FuHs64E3X6}
}