eulogik/Bharat-Tiny-LLM-v2-MLX

🇮🇳 Bharat-Tiny-LLM v2 — MLX

1.5B Hindi/Hinglish LLM with Brahmi token injection. 880 MB, fully offline on Apple Silicon.

---

✨ What is Brahmi injection?

300 high-frequency Devanagari subword tokens injected into the Qwen2.5 tokenizer (+0.2% vocab), embeddings initialized as the mean of constituent byte tokens, recovered with embedding warmup + LoRA.

Measured benchmarks (held-out data; methodology)

Metric	Base Qwen2.5-1.5B	v2	Δ
Tokens / 1000 Devanagari chars	1041.9	661.6	−36.5%
Tokens / 1000 chars (mixed Hinglish+Hindi)	593.3	486.6	−18.0%
Bits / char (held-out Hindi, fair protocol)	1.6531	1.4878	−10.0%
Warmup val loss (injected rows)	2.776	1.320	−52.5%

End-to-end savings are lower than script-only because Latin/Hinglish spans gain nothing from Devanagari tokens.

🚀 Quick Start

The easy way — bharat CLI

pip install "bharat-tiny-llm[mlx]"
bharat chat        # interactive Hindi/Hinglish REPL, works offline
bharat ask "नमस्ते!"

Python

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load(
    "eulogik/Bharat-Tiny-LLM-v2-MLX",
    adapter_path="eulogik/Bharat-Tiny-LLM-v2-MLX",  # see note below
)

The LoRA adapter ships in the lora_adapter/ folder of this repo. mlx_lm.load needs a local directory for adapters — easiest is:

from huggingface_hub import snapshot_download
adir = snapshot_download("eulogik/Bharat-Tiny-LLM-v2-MLX",
                         allow_patterns=["lora_adapter/*"]) + "/lora_adapter"
model, tokenizer = load("eulogik/Bharat-Tiny-LLM-v2-MLX", adapter_path=adir)

sampler = make_sampler(temp=0.3)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Chai peete hain?"}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128, sampler=sampler))

⚠️ Important usage notes

1. No system prompts — v2 was trained without system turns; adding one degrades output.
2. Use temp ≤ 0.3 with top-p 0.85 and repetition penalty ≥ 1.2. Higher temperatures produce garbled out-of-script tokens.
3. Adapter required — the base expanded model without the LoRA adapter generates poorly.
4. Adapter fixed Aug 24, 2026: if you downloaded earlier, re-pull lora_adapter/.

🎯 Training

Phase	Params	Hardware	Result
Embedding warmup (300 rows)	614K (0.04%)	Colab T4, 3K steps	val 2.776 → 1.320
LoRA (rank 8, scale 20, 16 layers)	~5M	Mac Mini M4, 500 steps ≈ 35 min	val 1.837 @ step 400

📜 License & attribution

Apache 2.0. Base: Qwen2.5-1.5B © Alibaba Cloud, Apache 2.0. Built by eulogik · GitHub · PyPI

Citation

@techreport{kishore2026brahmilite,
    title={Brahmi-Lite: Minimal-Budget Devanagari Token Injection for Edge LLMs},
    author={Gautam Kishore},
    year={2026},
    institution={eulogik},
    url={https://github.com/eulogik/Bharat-Tiny-LLM}
}