altaidevorg/turkish-bge-m3-model2vec-turboquant-4bit

🇹🇷 Turkish BGE-M3 Model2Vec (TurboQuant 4-bit)

Ultra-compact 4-bit INT4 quantized Turkish static sentence embedding model using Google TurboQuant (ICLR 2026) vector quantization.

• Model Size: 4.92 MB (447x smaller than 2.2 GB BGE-M3)
• Embedding Dimension: 256
• Inference Speed: 24,248 sentences/sec on CPU
• Accuracy (STSb-TR gorkem): 91.79% (Zero quality degradation vs FP16)
• Deployment: Ideal for mobile applications (iOS / Android) and local desktop apps.

---

📊 Benchmark Comparison

Model	Quantization	Size	STSb-TR (gorkemergune/stsb-tr)	Speed
BAAI/bge-m3 (Teacher)	FP32	~2,200 MB	96.35%	79 sent/s
turkish-bge-m3-model2vec (Base)	FP16	19.36 MB	91.36%	63,012 sent/s
🥈 turkish-bge-m3-model2vec-turboquant-4bit	INT4 (4-bit)	4.92 MB	91.79%	24,248 sent/s

---

🚀 Usage

# Clone repo & install library from https://github.com/altaidevorg/model2vec_experiments
from src.turboquant import TurboQuantStaticModel

# Load 4-bit model directly
model = TurboQuantStaticModel.from_pretrained("altaidevorg/turkish-bge-m3-model2vec-turboquant-4bit")

sentences = ["Hafif vektör modelleri mobil cihazlarda mükemmel çalışır."]
embeddings = model.encode(sentences)
print(f"Shape: {embeddings.shape}")  # (1, 256)

---

📄 License

Apache-2.0