LFM2.5-350M-Instruct-ONNX-Vulkan

A 4-bit quantized ONNX export of the LiquidAI LFM2.5-350M instruct decoder, built for the LFM2.5 dual-role deck stack (remotemedia). It is a template-replication of LiquidAI's 1.2B instruct q4 graph contract: identical input/output names, KV/conv-state threading, opset, and op layout (com.microsoft GroupQueryAttention with in-op RoPE, asymmetric MatMulNBits block-32, SimplifiedLayerNormalization), scaled to hidden 1024.

Files

file	bytes	notes
decoder_q4.onnx	141,181	graph proto (external-data refs)
decoder_q4.onnx_data	260,075,520	weights: 93 asymmetric-q4 MMBs (incl. weight-tied lm_head) + f32 norms/conv/RoPE caches
embed_tokens.bin	134,217,728	exact bf16 bytes of model.embed_tokens.weight (host-fed inputs_embeds)
embed_tokens.json	640	loader meta: vocab 65536, hidden 1024, head_dim 64, bf16, sha256
instruct350_provenance.json	19,546	full export audit: pins, contract checks, dequant audit (93/93 within one step)
onnx_parity_report.json	6,900	CPU ORT vs PyTorch bf16 greedy parity (6-prompt battery)
onnx_grammar_battery.json	4,193	constrained-greedy grammar + grounding-gate battery, 11/11

Total: 395 MB (vs the 1.2B instruct dir's ~1.14 GB).

Model contract (decoder)

• 24 inputs: inputs_embeds [B,S,1024] f32 (host embedding table lookup, not an in-graph gather), attention_mask [B,total] i64, 10x past_conv.N [B,1024,3] (conv layers only), 6x past_key_values.N.key/value [B,8,P,64] (attention layers 2/5/8/10/12/14)
• 23 outputs: logits [B,S,65536] f32 (head width 65536, vocab 64402) + present states in per-layer order
• 14 hybrid layers (10 short-conv + 6 attention), SwiGLU FFN, GQA 16q/8kv x 64
• lm_head weight-tied to the embedding table and quantized in-graph (asymmetric MatMulNBits) - no separate head weights
• RoPE: theta 1e6, head_dim 64, max_pos 128000 (caches byte-copied from the 1.2B instruct release; identical parameters)
• Quantization: per (row, 32-block) asymmetric - s=(max-min)/15, zp=round(-min/s), packed int4 (low nibble first), zp one byte per two blocks; all 93 weights audited within one full step of the bf16 source

Provenance

• Source checkpoint: LiquidAI/LFM2.5-350M @ 9e6c6ccf47cd318696e137d381a7ded8fe4df09f (model.safetensors sha256 1c9c77a4471a7f59..., 708,984,464 B, 148 tensors)
• Contract source: LiquidAI/LFM2.5-1.2B-Instruct-ONNX @ 10f72e70abf67ac0fd7ebf15bc5854726891d864 (graph templates; the 1.2B release's own s/zp recipe reproduces 100% here)
• Reproducible export: remotemedia/lfm25-audio-onnx repo, dual_role/instruct350_export.py (run with --install to regenerate the bundle; pinned snapshot, no network at build time beyond the pinned cache)

Validation

• CPU ORT vs PyTorch bf16 greedy (same 6-prompt battery, canonical deck framing): B4 tool call 13/13 bit-identical, T8c round-trip 17/17 bit-identical; structured prompts otherwise agree except near-boundary tail tokens (onnx_parity_report.json)
• Grammar + grounding-gate battery (token-level legal sets on this exact graph): 11/11 - B3 JSON exact-forced, B4 protocol forced to the measured golden, T7a fabrication persists in free mode, T7b gated refusal, T8c round, empty-lexicon control (onnx_grammar_battery.json)
• GPU showcase (dual-role deck, Vulkan): 14/14 ALL-PASS with this variant as the instruct role - forced get_weather(city="Paris") tool call, grounded-refusal discipline, and the full agentic voice loop

Usage in the deck

Point the remotemedia lfm25-audio-onnx plugin's instruct_model_dir at this directory (the audio role keeps the stock 1.5B bundle). The loader requires the onnx-vulkan-rs fork (matbeedotcom/onnx-vulkan-rs-edge, main @ 74ca614 or later) for the 4-input asymmetric MatMulNBits kernels. The host embeds come from embed_tokens.bin (exact bf16, upcast at lookup); the deck's shared tokenizer (64,402 vocab) applies unchanged.

License

Inherits the LFM Open License v1.0 of the source checkpoint (LiquidAI/LFM2.5-350M); see that repo's LICENSE for the full terms. Derived model (quantized ONNX export); no additional terms.