zortzi-tts

Basque (eu) fine-tune of Audio8-TTS-Preview-0.6b — a 0.6B DualAR text-to-speech model. This checkpoint speaks only Basque with two voices: Maider (female) and Antton (male), anchored to the HiTZ-Aholab speakers.

🏗️ Source code & training recipe: github.com/itzune/zortzi-tts

---

Model details

Architecture	DualAR TTS (Qwen2-style decoder + codec)
Parameters	601M (AR) + codec
Dtype	BF16
Vocab size	155,776 (BPE, Qwen2-style)
Codebooks	10 × 4096 entries
Audio	44.1 kHz, ~21.5 frames/s
Max sequence length	2048 (~95s hard ceiling; keep clips ≤20s for stable training)
Base model	Audio8/Audio8-TTS-Preview-0.6b (Apache-2.0)
License	Apache-2.0 (inherited from base)

Voices

Only two voices are available. They are carried by reference audio clips (their codec codes are the primary voice carrier; the speaker token is always <|speaker:0|>).

Voice	Gender	Reference clip	Reference text
Maider	Female	voices/maider.wav	Aurrelaria prest dago jokatzeko.
Antton	Male	voices/antton.wav	Inguru hura gerrillarien esku izan da denbora luzean.

These reference clips are included in this repository under voices/. They were selected with find_neutral_ref.py (pitch-range/median ratio < 0.5) so that the reference does not bleed expressive style into the output.

---

Quick start (GPU — PyTorch)

This is the highest-fidelity path: full BF16, no quantization, KV-cache on device.

Install

# Clone the Audio8_TTS runtime (provides audio8_tts_infer.py + custom modeling code)
git clone https://github.com/Audio8-AI/Audio8_TTS.git
cd Audio8_TTS

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python --torch-backend=cu124 \
    transformers soundfile torch

Synthesize

.venv/bin/python audio8_tts_infer.py \
    --model itzune/zortzi-tts \
    --text "Kaixo mundua! Nire izena Maider da." \
    --reference-audio voices/maider.wav \
    --reference-text "Aurrelaria prest dago jokatzeko." \
    --output out.wav \
    --temperature 0.8 --top-p 0.95

⚠️ Sampling is required. Do not pass --greedy. Greedy decoding causes repetition loops that never reach EOS. Always use --temperature 0.8 --top-p 0.95.

Switch voices

Change the --reference-audio and --reference-text to the Antton clip:

.venv/bin/python audio8_tts_infer.py \
    --model itzune/zortzi-tts \
    --text "Kaixo mundua! Nire izena Antton da." \
    --reference-audio voices/antton.wav \
    --reference-text "Inguru hura gerrillarien esku izan da denbora luzean." \
    --output out.wav \
    --temperature 0.8 --top-p 0.95

Programmatic usage

import torch
from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained(
    "itzune/zortzi-tts",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).cuda()
processor = AutoProcessor.from_pretrained("itzune/zortzi-tts", trust_remote_code=True)

See the upstream audio8_tts_infer.py for the full generation loop.

---

ONNX deployment (CPU)

For CPU deployment without PyTorch/GPU, see the companion model card: itzune/zortzi-tts-onnx — INT4-quantized ONNX models (575 MB total) that run on the Audio8 ONNX runtime.

---

Training summary

A two-phase curriculum with full fine-tuning (not LoRA) on an NVIDIA L40 (48 GB). Full details in the source repo.

Phase 1 — phonotactics base (Common Voice 26.0)

Learned the text→semantic (phonotactics) mapping for Basque across many speakers, with no reference audio.

• Data: Mozilla Common Voice 26.0 Basque, 134,531 validated clips (CC0) (v26.0 was downloaded directly from the Common Voice website; the closest HuggingFace mirror is common_voice_17_0)
• Config: LR 1e-5 cosine, batch 4×4, max_steps 6000
• Runtime: ~64 min
• Key insight: stopped at step 6000 (the "Goldilocks" zone) to avoid prosodic hysteresis — CV is flat read-speech, and training too long entrenches a neutral prosody prior that later phases cannot fully reverse

Phase 2 — voice anchor + prosody polish (HiTZ, merged)

Anchored Maider/Antton voice identity and restored expressive prosody (especially wh-question intonation) in a single pass.

• Data: HiTZ-Aholab Basque TTS (Maider + Antton, 27,000 clips, CC BY 4.0), upsampled to 41,300 clips: declarative ×1, interrogative ×3, exclamative ×3 (functionally equivalent to sequence-level loss weighting)
• References: cross-clip self-conditioning (ref = a different clip from the same speaker — teaches the zero-shot voice-cloning path)
• Config: LR 5e-6 linear decay, batch 4×4, 3 epochs, warmup 0.03
• Started from: Phase 1 step-6000 checkpoint
• Final: 7,746 steps, train_loss=29.96, slow_accuracy=30.9%

Key decisions

Decision	Rationale
Full fine-tune over LoRA	New-language phonotactics require full weight updates
Two-phase curriculum	Phase 1 (CV) learns phonotactics; Phase 2 (HiTZ) anchors voice + prosody
Stop Phase 1 at step 6000	Avoid prosodic hysteresis from flat CV read-speech
Upsample interrogatives/exclamatives 3×	Sequence-level loss weighting for prosody
Cross-clip self-conditioning	Teaches zero-shot voice-cloning path
Sampling (temp 0.8, top-p 0.95)	Greedy causes infinite repetition loops

---

Limitations & known issues

• Numerals are not pronounced in Basque. Even with text normalization (digits expanded to Basque words), the model pronounces numbers in a mix of languages rather than Basque. Spell out numbers manually if correct pronunciation is required.
• No built-in text normalization at inference. Audio8_TTS performs no language-specific G2P, number-to-words, or abbreviation expansion. Text is BPE-tokenized verbatim. Acronyms like "TTS" should be expanded to "te te ese" upstream of the model.
• Two voices only. This checkpoint speaks only Basque with Maider and Antton. No other voices are registered.
• Sampling required. Greedy decoding produces repetition loops.

---

Datasets, licenses & acknowledgements

This model is a fine-tuned derivative of the Audio8 TTS preview model. The fine-tuned weights are released under Apache-2.0, inherited from the upstream model. The training data carries its own obligations:

Dataset	Role	License	Clips
Audio8-TTS-Preview-0.6b	Base model	Apache-2.0	—
Mozilla Common Voice 26.0 — Basque ¹	Phase 1: phonotactics	CC0 1.0	134,531
HiTZ-Aholab Basque TTS (Maider + Antton)	Phase 2: voice + prosody	CC BY 4.0	27,000

¹ Common Voice 26.0 was obtained directly from the Common Voice website. The closest HuggingFace dataset mirror is mozilla-foundation/common_voice_17_0.

Redistribution note. Because the CC BY 4.0 HiTZ-Aholab data conditions the voice of this model (Maider, Antton), any redistribution of the fine-tuned weights or generated audio must carry the HiTZ attribution and citation below.

Funding acknowledgements

The HiTZ-Aholab speech synthesis dataset was developed with funding from:

The Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia — Funded by EU — NextGenerationEU within the framework of the project ILENIA (ref. 2022/TL22/00215335), and by a grant from the Department of Culture and Language Policy of the Basque Government (IKER-GAITU project).

The zortzi-tts fine-tuning work itself is independent and not directly funded by the above grants; they are acknowledged here solely because the voice-anchor dataset carries the obligation.

Citation

If you use this model or build on its training recipe, please cite the base model, the datasets, and this work:

@dataset{itzune_zortzi_tts,
    author    = {{itzune}},
    title     = {{zortzi-tts: Basque fine-tune of Audio8-TTS-Preview-0.6b}},
    year      = 2025,
    publisher = {Hugging Face},
    url       = {https://huggingface.co/itzune/zortzi-tts}
}

@inproceedings{ardila-etal-2020-common,
    title     = "Common Voice: A Massively-Multilingual Speech Corpus",
    author    = "Ardila, Rosana and Branson, Megan and Davis, Kelly and
                 Kohler, Michael and Meyer, Josh and Henretty, Michael and
                 Morais, Reuben and Saunders, Lindsay and Tyers, Francis and
                 Weber, Gregor",
    booktitle = "Proceedings of the Twelfth Language Resources and Evaluation
                 Conference (LREC)",
    month     = may, year = "2020",
    address   = "Marseille, France",
    publisher = "European Language Resources Association",
    pages     = "4218--4222",
    url       = "https://aclanthology.org/2020.lrec-1.520/"
}

@dataset{navas_hernaez_2025_17952596,
    author    = {Navas, Eva and Hernaez Rioja, Inmaculada and
                 Saratxaga, Ibon and Sanchez, Jon and
                 García Romillo, Víctor and Flores Ríos, Mariana and
                 Bellanco, Aitor},
    title     = {{HiTZ-Aholab speech synthesis dataset in Basque}},
    month     = dec, year      = 2025,
    publisher = {Zenodo},
    version   = {1.0},
    doi       = {10.5281/zenodo.17952596},
    url       = {https://doi.org/10.5281/zenodo.17952596}
}