Pi0.5 AXIS V6 — benchmark reference checkpoint
这是 AXIS × OpenRoboto 本地评测中，在首轮 30 个任务上测得 63/600（10.5%） 成功率的 Pi0.5 V6 权重。
This repository preserves the exact V6 checkpoint evaluated in the local AXIS benchmark. The finetuning data covers tasks 501 and 502, while the baseline evaluation covers all 30 tasks. This is not the later all-30-task finetuning attempt.
Measured results
| Evaluation group | Successes / trials | Success rate |
|---|---|---|
| Finetuned tasks 501 and 502 | 40 / 40 | 100.0% |
| Remaining 28 baseline tasks | 23 / 560 | 4.1% |
| Complete 30-task baseline | 63 / 600 | 10.5% |
| Five added evaluation tasks | 0 / 100 | 0.0% |
| Expanded 35-task set | 63 / 700 | 9.0% |
The same model was used before and after expansion. On the RTX 4090 run, the original 30-task outcomes reproduced exactly. Task 501's success predicate entails task 505's predicate, so these are not independent capabilities. The baseline uses fixed scenes and initial states; these results do not measure broad randomization robustness or long-term ranking validity.
Training and format
- Backbone: Pi0.5; OpenPI configuration:
pi05_axis_joint. - Local finetuning tasks: 501 and 502; 77 episodes, 1,270 examples, 3,000 updates (
global_step=2999). - Batch size: 8; training seed: 42; discrete state input enabled.
- Format: JAX/Orbax OCDBT parameters plus normalization assets and original AXIS metadata.
- Inference-only checkpoint; optimizer/training-resume state is not included.
- OpenPI source revision recorded by the checkpoint:
15a9616a00943ada6c20a0f158e3adb39df2ccac.
The original eligible_for_scoring=false and benchmark-development provenance are preserved in the metadata. This is a local evaluation reference, not an already-approved subnet leaderboard submission.
Files and integrity
The original checkpoint is under checkpoint/. Pass that directory to the validator, rather than the repository root.
checkpoint/params/: Orbax weights.checkpoint/assets/: normalization statistics.checkpoint/axis_vla_metadata.json: original training and input metadata.checkpoint_inventory.json: byte sizes and SHA-256 hashes for all 32 checkpoint files.evaluation/: task scores, training scope, benchmark manifest and evaluation receipt.
Checkpoint size: 12,440,763,327 bytes. Canonical original file-inventory digest:
d4b2c28abfabe43f5567781b1fc3e0c5034cf2e9575d8ca9732d28887b7e29a1
Evaluation in the validator workspace
Use the AXIS/OpenPI runtime and pi05_axis_joint adapter from the validator project. This is not a Transformers from_pretrained checkpoint. After downloading, submit the checkpoint/ directory:
uv run tools/axis_demo_submission.py submit \
--checkpoint /path/to/pi05-axis-v6-benchmark/checkpoint \
--output eval_runs_dev/hf-v6-submission.json
OPENPI_DIR=/path/to/openpi uv run tools/axis_demo_submission.py evaluate \
--submission eval_runs_dev/hf-v6-submission.json \
--manifest configs/benchmarks/axis-v13.0.json \
--output eval_runs_dev/hf-v6-baseline \
--trials 20 --seed 20260907 --gpus 0 --record-trials 2 \
--replan-steps 10 --gripper-mode continuous --policy-samples 1
Recorded protocol: MuJoCo 3.11.0, OSMesa, 256×256 RGB, 5 Hz, up to 120 control steps per trial, action replanning every 10 steps, continuous gripper decoding, one policy sample. Each baseline task has 20 trials.