Merge multimodal TranslateGemma artifact docs

CLAUDE.md

@@ -6,6 +6,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Python toolkit that converts Google's TranslateGemma 4B IT model (HuggingFace) into on-device inference bundles for Android. Google's official TFLite files only support WebGPU — this project produces CPU/XNNPACK-compatible `.litertlm` (LiteRT-LM) and `.task` (MediaPipe) files with proper KV-cache prefill/decode signatures.
 
+The primary Hugging Face app artifact is the multimodal image+text LiteRT-LM bundle:
+`artifacts/int4-multimodal/translategemma-4b-it-int4-multimodal.litertlm` (~2.76 GB). It supports structured text translation and image translation through LiteRT-LM vision, and is intended for PrivateAITranslate. If this file is absent from the Hugging Face repo, the app download URL for the multimodal model will 404.
+
 ## Common Commands
 
 ### Single quantization conversion (produces `.task`)
@@ -109,6 +112,7 @@ In Google AI Edge Gallery **Prompt Lab** mode, paste this as the System Prompt w
 - `conv-venv/` — virtualenv with all deps (`litert-torch==0.8.0`, `mediapipe`, `ai_edge_torch`, `transformers`)
 - `/tmp/litert-lm-pkg/` — manually assembled package from cloned LiteRT-LM repo with compiled FlatBuffer (`flatc -p --gen-onefile`) and proto (`protoc`) Python bindings; required by `bundle_litertlm.py` at runtime; NOT persistent across reboots
 - `/tmp/litert-lm/` — cloned `google-ai-edge/LiteRT-LM` repo (schema source for rebuilding the package)
-- Conversion requires ~128 GB RAM; 4B model loads ~46 GB
+- Multimodal conversion observed peak RSS around 73.9 GiB; require 96 GB minimum RAM, with 128 GB preferred. Ideal builder is a Vast.ai PyTorch image or Ubuntu-based NVIDIA/PyTorch image rather than a bare CUDA runtime image.
+- CPU real-device validation has passed on Pixel 10 and Galaxy S22 / S22 Ultra-class targets. GPU initialization currently fails during validation and should be considered experimental and not validated.
 - `translategemma-4b-it/tokenizer.model` is the SentencePiece binary used by both `.task` and `.litertlm` bundlers; `ensure_tokenizer_model()` auto-converts from `tokenizer.json` if missing
 - HuggingFace repo: `barakplasma/translategemma-4b-it-android-task-quantized`; upload token in `HF_TOKEN` env var

README.md

@@ -19,7 +19,7 @@ On-device translation model for Android using [Google AI Edge](https://ai.google
 Converts [google/translategemma-4b-it](https://huggingface.co/google/translategemma-4b-it) (55 languages, 4B params)
 into formats that run locally on Android without internet or cloud APIs.
 
-Google only publishes WebGPU-only TFLite files. This repo bridges that gap with CPU/XNNPACK-compatible `.litertlm` bundles (LiteRT-LM format) with embedded chat template.
+Google only publishes WebGPU-only TFLite files. This repo bridges that gap with CPU/XNNPACK-compatible `.litertlm` bundles (LiteRT-LM format) with embedded chat template, including a multimodal image+text Android bundle for PrivateAITranslate.
 
 ---
 
@@ -29,8 +29,11 @@ Google only publishes WebGPU-only TFLite files. This repo bridges that gap with
 |------|------|-------|
 | `artifacts/int4-generic/translategemma-4b-it-int4-generic.litertlm` | ~2 GB | INT4 blockwise quant — faster, lower RAM |
 | `artifacts/dynamic_int8-generic/translategemma-4b-it-dynamic_int8-generic.litertlm` | ~4 GB | Dynamic INT8 — better quality |
+| `artifacts/int4-multimodal/translategemma-4b-it-int4-multimodal.litertlm` | ~2.76 GB | INT4 multimodal image+text bundle for structured text translation and LiteRT-LM vision image translation |
 
-**Start with INT4** if you're unsure — it loads faster and uses less RAM. Use dynamic_int8 for better translation quality.
+**Start with INT4** if you're unsure — it loads faster and uses less RAM. Use dynamic_int8 for better text translation quality.
+
+The multimodal artifact is the bundle expected by PrivateAITranslate for image translation. If `artifacts/int4-multimodal/translategemma-4b-it-int4-multimodal.litertlm` is not present in this Hugging Face repo, the app download URL for that model will return 404.
 
 ---
 
@@ -68,6 +71,10 @@ Use standard ISO 639-1 language codes: `en`, `he`, `fr`, `es`, `de`, `ar`, `zh`,
 
 Plain text (no tags) is also accepted — the model will attempt translation based on context.
 
+### Image translation
+
+The multimodal `.litertlm` bundle uses LiteRT-LM vision support for image+text translation flows in PrivateAITranslate. It is intended for structured text translation and image translation, not general image captioning.
+
 ---
 
 ## Device Requirements
@@ -79,6 +86,8 @@ Plain text (no tags) is also accepted — the model will attempt translation bas
 | OS | Android 10+ |
 | Runtime | Google AI Edge Gallery or LiteRT-LM SDK |
 
+CPU real-device validation passed on Pixel 10 and Galaxy S22 / S22 Ultra-class targets. GPU execution currently fails initialization in validation and should be treated as experimental and not validated.
+
 ---
 
 ## What's Different From Google's Official Files
@@ -106,7 +115,17 @@ The `scripts/` folder contains the full conversion pipeline:
 
 ### Reproduce a build
 
-Requirements: ~128 GB RAM, Python 3.12, `litert-torch==0.8.0`
+Requirements: 96 GB minimum system RAM, 128 GB preferred, Python 3.12, `litert-torch==0.8.0`.
+
+Observed multimodal export peak RSS was about 73.9 GiB, so 96 GB is the practical floor once Python, model cache, filesystem cache, and conversion overhead are included. Use 128 GB when running multiple quantization attempts or keeping extra build artifacts.
+
+Ideal Vast.ai image:
+
+- Vast.ai PyTorch image or an Ubuntu-based NVIDIA/PyTorch CUDA image, not a bare CUDA runtime image.
+- Python 3.12 with `uv`/pip, Git, Git LFS, Hugging Face CLI, and build tools available.
+- CUDA/PyTorch wheel support matching the rented GPU architecture; use CUDA 12.8+ PyTorch wheels on Blackwell GPUs.
+- 96+ GB RAM and at least 80 GB free disk; 150+ GB disk is safer for source checkpoint, caches, exported TFLite files, and `.litertlm` bundles.
+- Persistent `/workspace` volume if the instance may be stopped/recycled before upload.
 
 ```bash
 # Clone LiteRT-LM builder (needed by bundle_litertlm.py)
@@ -117,7 +136,7 @@ pip install litert-torch==0.8.0 mediapipe transformers huggingface-hub
 # Download model
 huggingface-cli download google/translategemma-4b-it --local-dir ./translategemma-4b-it
 
-# Convert to TFLite with KV cache (~30-60 min, needs ~128 GB RAM)
+# Convert to TFLite with KV cache (~30-60 min, needs 96 GB minimum RAM)
 python scripts/convert_translategemma_android.py \
   --model-dir ./translategemma-4b-it \
   --tflite-dir ./tflite_output/dynamic_int8 \

artifacts/int4-multimodal/translategemma-4b-it-int4-multimodal.litertlm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bc8e828fbe56dfb503ac033ac84a9eb941d85a9c196803167fa39b35f7fb033
+size 2757894128

scripts/multi_quant_build_upload.py

@@ -196,7 +196,10 @@ def plan_quants(requested_quants, native_ok: bool):
 
 def main():
     ap = argparse.ArgumentParser(
-        description="Run multi-quant conversion+bundle and upload successful artifacts to HF."
+        description=(
+            "Run multi-quant conversion+bundle and upload successful artifacts to HF. "
+            "Requires 96 GB minimum system RAM; 128 GB is preferred for headroom."
+        )
     )
     ap.add_argument("--converter-script", default="/home/ubuntu/convert_translategemma_android.py")
     ap.add_argument("--model-id", default="google/translategemma-4b-it")
@@ -477,4 +480,4 @@ def main():
 
 
 if __name__ == "__main__":
-    main()
+    main()