LFM2.5-Audio-1.5B-ONNX-Vulkan (deck profile, Tier-1)

The audio role of the dual-role LFM2.5 Steam-Deck bundle, prepared for the onnx-vulkan-rs Vulkan runtime. Derived from the pinned official release LiquidAI/LFM2.5-Audio-1.5B-ONNX @ 62318d95ddf42a65e742cdd6fd33df91874a801d (every stock file below is byte-identical to that revision) with two Tier-1 memory reductions (2026-08-23):

1. decoder_q4: lm_head f32 -> symmetric q4 MatMulNBits. The stock release quantized its 92 layer GEMMs but left the weight-tied lm_head as an f32 table (536,870,912 B, on every generation step). It is now one 3-input symmetric MatMulNBits node (com.microsoft domain, same per-32-block convention as the release's own 92 MMBs: scale = max/7, q = clip(round(w/s)+8, 0..15), two int4 per byte low-nibble-first, dequant (q-8)*s): quant uint8 [65536,64,16] (67,108,864 B) + scales f32 [65536,64] (16,777,216 B). Data file: 1,217,650,688 -> 764,665,856 B.
2. host embedding tables: f32 -> native bfloat16. embed_tokens.bin (536,870,912 -> 268,435,456 B) and audio_embedding.bin (134,283,264 -> 67,141,632 B) now store the original bf16 checkpoint bytes. The f32 bins were the exact bf16->f32 bit upcast, so this is bit-exact (all three verified bit-identical before conversion); consumers upcast at lookup. dtype in the adjacent .json records the form.

Total model data: 3.55 GiB -> ~2.5 GiB for the dual bundle (the sister instruct repo carries the matching instruct role).

Files

file	bytes	sha256
decoder_q4.onnx	169,217	6d27b280e1d621ba466d499f786efe4f3573a018f167eed36e570ef08086c9bf
decoder_q4.onnx_data	764,665,856	dbc832557de2e0bf848f837d158cb0ed5f93b93a6beb621556aca431965ef740
audio_encoder_q4.onnx	477,419	2f3080ea1a316d19951a73a161d505b9f7c5cd43996fafc77515d846e68fb500
audio_encoder_q4.onnx_data	139,110,400	edb1205829a3c43ff2695c2d711ff78dcb5ee2e1e3af1fa2bc9ec15322f2e2ac
audio_detokenizer_q4.onnx	74,691	347b6c9f0eab569352b629af5fb52bade0e0ce73d71ade768efbbbccc623bb00
audio_detokenizer_q4.onnx_data	56,481,416	36c42737ed1113d0f9c375313b3dbfb0c312a82e2ce7bc14728d1856115e6aec
vocoder_depthformer_q4.onnx	54,511	00d4c69b64eb335c97552c679930f93e537ca4ba6437fb9afc76ded522e28f40
vocoder_depthformer_q4.onnx_data	187,166,720	d67cbae1a4b1fb2b47bf7f89490384ca8ece5bfbee7a269fe55ace1d0355d2ed
audio_embedding_q4.onnx	358	d111221f32cc0ffd9fba1e88af1436bfa202d4a21013c8c5d27850a54bd55f4b
audio_embedding_q4.onnx_data	134,283,264	3f03eb90911edd35413e9ee415a4999dd3e8320886ae56911520706612eed3cc
embed_tokens.bin (bf16)	268,435,456	50e8189fe981903582a2703ebd96907988080d7160fadce7dded598e0f99cd45
embed_tokens.json	544	0340643b52fe7554d97443977e2f1c00e4f8a119af36a7e2c82f2878101187c3
audio_embedding.bin (bf16)	67,141,632	d4c1a541246b1f28e5b17595e642488061b3af4e83749f21b0a2306c719cd67a
audio_embedding.json	600	ee3a3d0b6f7fa5d4d49442db395d7098131506e4e405753adf5fb35434668f15
mel_config.json	246	4ffa79932b1d744deb7339ee6b07febd1bef57f1bb342f3cc58a841af03eaba8
tokenizer.json	4,732,404	fbb4071ea653afe958c24562139c2c09618f90bb6bf50d7669dfe10a0fa02e68
config.json	2,640	a057946a5f4db083b5ad5add201c9193ade532aa1faefca91c7168fe3d7caadc
LICENSE	(LFM Open License v1.0)	61d7e939a05911c765b7e98ffaa1ab5ca6c0174a65350766c25cb10197d19fc8

Records: dual_role_manifest.json (deck manifest v6), audio_pins.json (stock pin table + the decoder_lmhead_q4 derivation block), audio_stack_provenance.json, lmhead_q4_provenance.json, tables16_provenance.json.

Load contract

• 24 graph inputs, inputs_embeds (f32) first - the host feeds embeddings from the bf16 tables (exact upcast at lookup); KV/conv caches are zeroed per turn and threaded past_* <- present.*.
• 93 MatMulNBits nodes (92 stock + the new lm_head), all 3-input symmetric, bits=4, block_size=32, com.microsoft domain (this ORT build resolves that form; a plain ai.onnx MatMulNBits node fails to load in ORT 1.26).
• One pre-existing custom op: SimplifiedLayerNormalization (com.microsoft) - the generic onnx.checker flags it; the Vulkan runtime implements it natively (the stock graph fails the checker identically).
• The runtime is the onnx-vulkan-rs fork (matbeedotcom/onnx-vulkan-rs-edge, main at 2d660f2) - the deck also needs its role-aware decoder multiplexer; the graphs themselves only use release-native ops.

Validation

• Convention check: dequantizing a stock release MMB (layer-9 up_proj) with this recipe against the bf16 checkpoint lands within half a quantization step (max_err 0.0230713 <= half-step 0.0306396).
• CPU ORT greedy parity vs the stock f32-lm_head graph: 4/4 prompts bit-identical (text-first prompts; the TTS prompt stops at AUDIO_START as expected). The q4 lm_head changes no greedy token on the covered prompts.
• Instruct role (sister repo): 21/21 bit-identical vs bf16 PyTorch reference, re-confirmed after the Tier-1 table conversion.
• GPU showcase (2026-08-23, Vulkan, full dual bundle): 14/14 ALL-PASS on this exact decoder. The five TTS turns (the q4-lm_head quality gate) all pass with decode_rtf 0.72-0.95 and are at or below the stock-lm_head run (t3 12.5 s -> 10.9 s).

Reproduction

audio_stack_export.py (pinned stock) -> audio_lmhead_q4_export.py --install (lm_head q4) -> f16_tables_export.py --install (bf16 tables); all in the source repo's dual_role/, hash-pinned end to end.