Microduck Lab: run the real shipped RL policies from a gr.Workflow

README.md

@@ -1,13 +1,102 @@
 ---
-title: Gr Workflow Microduck Lab
-emoji: 🦀
-colorFrom: indigo
+title: Microduck Lab · gr.Workflow
+emoji: 🦆
+colorFrom: yellow
 colorTo: blue
 sdk: gradio
-sdk_version: 6.26.0
-python_version: '3.13'
+sdk_version: 6.22.0
 app_file: app.py
 pinned: false
+hf_oauth: true
+hf_oauth_scopes:
+  - inference-api
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# 🦆 Microduck Lab
+
+Describe a routine in plain English. A language model compiles it into a move
+plan, and the **real reinforcement-learning policies that ship with
+[Microduck](https://pollen-robotics.com/microduck/)** execute it against
+MuJoCo physics — then you get the video, the telemetry and a report.
+
+```
+[routine text] ─▶ (fn) choreograph ─┬─▶ 📝 Move plan
+                    LLM             │
+                                    ▼
+[seed] ────────────────▶ (fn) perform ─┬─▶ 🎬 Routine video
+              MuJoCo + 6 ONNX policies ├─▶ 📊 Telemetry
+                                       └─▶ 📝 Report
+```
+
+## This is not a re-implementation
+
+Nothing about the duck is faked or hand-animated. At runtime the app downloads,
+from the public Space [`multimodalart/microduck-ar`](https://huggingface.co/spaces/multimodalart/microduck-ar):
+
+- `robot_allcollisions.xml` — the MJCF for the real robot, plus its 38 meshes
+- six trained ONNX policies — `BEST_alpha_walking`, `BEST_alpha_stand`,
+  `ball_kick_left`, `ball_kick_right`, `roulade`, `alpha_ground_pick`
+
+and steps them exactly the way the robot's runtime does:
+
+| | |
+|---|---|
+| physics | MuJoCo, timestep 0.005 s, decimation 4 → **50 Hz** control |
+| observation | 61-D: `ang_vel(3) + projected_gravity(3) + joint_pos(14) + joint_vel(14) + last_action(14) + command(13)` |
+| action | `ctrl[j] = DEFAULT_POSE[j] + action[j]` |
+| commands | forward 0.25 m/s, backward −0.2 m/s, yaw ±1.0 rad/s |
+
+The observation layout, the action scaling, the one-shot state machines
+(kick = 25 steps, ground-pick = a `[cos, sin]` phase clock over a 4 s period
+exiting at 0.7) and the fall-recovery machine are ported from that Space's
+`src/sim.js`, which is the reference implementation.
+
+Falls are real. Ask for a barrel roll and the duck tumbles, the fall detector
+fires, and the `stand` policy picks it back up — you can watch that happen in
+the uprightness trace.
+
+## Moves
+
+Timed (take `seconds`): `forward`, `backward`, `strafe_left`, `strafe_right`,
+`turn_left`, `turn_right`, `stand`.
+
+One-shot (run their trained cycle): `kick_left`, `kick_right` (a ball is
+spawned in front), `roll`, `pick`.
+
+## Running it
+
+```bash
+pip install -r apps/09_microduck_lab/requirements.txt
+python apps/09_microduck_lab/app.py
+```
+
+First launch downloads ~10 MB of meshes and policies into
+`.microduck_cache/` (override with `MICRODUCK_CACHE`).
+
+The choreographer uses `Qwen/Qwen3-4B-Instruct-2507` through HF Inference
+Providers when a token is available — sign in on the Space, or set `HF_TOKEN`
+locally. **Without a token it still works**: it falls back to a keyword
+planner, so nothing is behind a sign-in wall.
+
+## Files
+
+| file | what it is |
+|---|---|
+| `duck.py` | asset fetch, MJCF assembly, the 50 Hz policy/physics loop, rendering |
+| `routine.py` | the move vocabulary, plan parsing, rollout + telemetry |
+| `nodes.py` | the two bound fn nodes: `choreograph`, `perform` |
+| `build_workflow.py` | generates `workflow.json` — edit this, not the JSON |
+| `test_pipelines.py` | runs every subject through the real `WorkflowExecutor` |
+
+## Notes for anyone extending it
+
+- Both operators are `fn` nodes on purpose. The canvas rewrites `model` node
+  ports to the endpoint schema, so anything wanting a richer control surface
+  has to be an `fn`.
+- Structured data travels as `text`, never `json` or `dataframe` — the canvas
+  serializes those with `String(obj)` and the receiver gets `"[object Object]"`.
+- The video and chart outputs are returned as `{"path", "url"}` dicts. The REST
+  API reads `path`; the canvas reads `url` and needs a `data:` URI.
+- `workflow.json` is **autosaved by the canvas** whenever a dev server is
+  running. Kill every server before regenerating it, or your graph gets
+  written back over.

app.py

@@ -0,0 +1,27 @@
+import os
+import sys
+
+import gradio as gr
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+
+from nodes import choreograph, perform  # noqa: E402
+
+# Pull the MJCF, meshes and policies during boot (~10 MB, ~45 s cold) so the
+# first visitor's run doesn't pay for it. Non-fatal: the sim re-fetches lazily.
+try:
+    import duck
+    duck.ensure_assets()
+    print("microduck assets ready")
+except Exception as e:
+    print("microduck asset prefetch failed (" + type(e).__name__ + ": "
+          + str(e) + "); will retry on first run")
+
+demo = gr.Workflow(
+    os.path.join(HERE, "workflow.json"),
+    bind={"choreograph": choreograph, "perform": perform},
+)
+
+if __name__ == "__main__":
+    demo.launch()

build_workflow.py

@@ -0,0 +1,153 @@
+"""Generate workflow.json. Run this, don't hand-edit the JSON.
+
+    python build_workflow.py
+
+Note the canvas AUTOSAVES workflow.json while a dev server is running, so kill
+every stray server before regenerating, or the old graph will be written back.
+"""
+
+import json
+import os
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+DEFAULT_ROUTINE = (
+    "waddle forward, turn left, then kick the ball with your right foot "
+    "and finish with a barrel roll"
+)
+
+
+def port(pid, label, ptype, required=False, output_index=None):
+    p = {"id": pid, "label": label, "type": ptype}
+    if required:
+        p["required"] = True
+    if output_index is not None:
+        p["output_index"] = output_index
+    return p
+
+
+def edge(eid, src, src_port, tgt, tgt_port, etype):
+    return {"id": eid, "from_node_id": src, "from_port_id": src_port,
+            "to_node_id": tgt, "to_port_id": tgt_port, "type": etype}
+
+
+GRAPH = {
+    "schema_version": "2",
+    "name": "Microduck Lab",
+    "references": [
+        {
+            "id": "ref_routine", "role": "reference", "label": "Routine",
+            "asset_type": "text",
+            "inputs": [port("in", "Routine", "text")],
+            "outputs": [port("out", "Routine", "text")],
+            "data": {"out": DEFAULT_ROUTINE},
+            "x": 40, "y": 60, "width": 250, "height": 150,
+        },
+        {
+            "id": "ref_seed", "role": "reference", "label": "Seed",
+            "asset_type": "number",
+            "inputs": [port("in", "Seed", "number")],
+            "outputs": [port("out", "Seed", "number")],
+            "data": {"out": 0},
+            "x": 40, "y": 300, "width": 250, "height": 110,
+        },
+    ],
+    "operators": [
+        {
+            "id": "op_plan", "role": "operator", "kind": "fn", "fn": "choreograph",
+            "label": "choreograph (LLM)",
+            "inputs": [port("in_routine", "routine", "text", required=True)],
+            "outputs": [port("out_0", "plan", "text", output_index=0)],
+            "data": {}, "x": 350, "y": 80, "width": 250, "height": 130,
+        },
+        {
+            "id": "op_run", "role": "operator", "kind": "fn", "fn": "perform",
+            "label": "perform (MuJoCo + ONNX)",
+            "inputs": [port("in_plan", "plan", "text", required=True),
+                       port("in_seed", "seed", "number")],
+            "outputs": [port("out_0", "video", "video", output_index=0),
+                        port("out_1", "chart", "image", output_index=1),
+                        port("out_2", "report", "text", output_index=2)],
+            "data": {}, "x": 660, "y": 150, "width": 260, "height": 180,
+        },
+    ],
+    "subjects": [
+        {
+            "id": "sub_video", "role": "subject", "label": "Routine video",
+            "asset_type": "video",
+            "inputs": [port("in", "Video", "video")],
+            "outputs": [], "data": {},
+            "x": 990, "y": 40, "width": 280, "height": 230,
+        },
+        {
+            "id": "sub_chart", "role": "subject", "label": "Telemetry",
+            "asset_type": "image",
+            "inputs": [port("in", "Telemetry", "image")],
+            "outputs": [], "data": {},
+            "x": 990, "y": 300, "width": 280, "height": 210,
+        },
+        {
+            "id": "sub_report", "role": "subject", "label": "Report",
+            "asset_type": "text",
+            "inputs": [port("in", "Report", "text")],
+            "outputs": [], "data": {},
+            "x": 990, "y": 540, "width": 280, "height": 180,
+        },
+        {
+            "id": "sub_plan", "role": "subject", "label": "Move plan",
+            "asset_type": "text",
+            "inputs": [port("in", "Plan", "text")],
+            "outputs": [], "data": {},
+            "x": 350, "y": 300, "width": 250, "height": 200,
+        },
+    ],
+    "edges": [
+        edge("e1", "ref_routine", "out", "op_plan", "in_routine", "text"),
+        edge("e2", "op_plan", "out_0", "op_run", "in_plan", "text"),
+        edge("e3", "op_plan", "out_0", "sub_plan", "in", "text"),
+        edge("e4", "ref_seed", "out", "op_run", "in_seed", "number"),
+        edge("e5", "op_run", "out_0", "sub_video", "in", "video"),
+        edge("e6", "op_run", "out_1", "sub_chart", "in", "image"),
+        edge("e7", "op_run", "out_2", "sub_report", "in", "text"),
+    ],
+}
+
+
+def verify(graph):
+    """Cheap structural checks: no dangling edges, no canvas-hostile port types."""
+    nodes = {n["id"]: n for n in
+             graph["references"] + graph["operators"] + graph["subjects"]}
+    problems = []
+    for e in graph["edges"]:
+        src, tgt = nodes.get(e["from_node_id"]), nodes.get(e["to_node_id"])
+        if src is None or tgt is None:
+            problems.append("edge " + e["id"] + " points at a missing node")
+            continue
+        if e["from_port_id"] not in {p["id"] for p in src["outputs"]}:
+            problems.append("edge " + e["id"] + ": no output port "
+                            + e["from_port_id"] + " on " + e["from_node_id"])
+        if e["to_port_id"] not in {p["id"] for p in tgt["inputs"]}:
+            problems.append("edge " + e["id"] + ": no input port "
+                            + e["to_port_id"] + " on " + e["to_node_id"])
+    for n in nodes.values():
+        for p in n["inputs"] + n["outputs"]:
+            # The canvas serializes these with String(obj) -> "[object Object]".
+            if p["type"] in ("json", "dataframe"):
+                problems.append(n["id"] + "." + p["id"] + " uses canvas-broken type "
+                                + p["type"])
+    return problems
+
+
+if __name__ == "__main__":
+    issues = verify(GRAPH)
+    if issues:
+        raise SystemExit("refusing to write workflow.json:\n  "
+                         + "\n  ".join(issues))
+    out = os.path.join(HERE, "workflow.json")
+    with open(out, "w", encoding="utf-8") as f:
+        json.dump(GRAPH, f, indent=2)
+        f.write("\n")
+    print("wrote " + out)
+    print("  %d references, %d operators, %d subjects, %d edges" % (
+        len(GRAPH["references"]), len(GRAPH["operators"]),
+        len(GRAPH["subjects"]), len(GRAPH["edges"])))

duck.py

@@ -0,0 +1,344 @@
+"""Microduck simulation core.
+
+Runs the REAL policies Hugging Face / Pollen Robotics ship with Microduck: the
+MJCF and the ONNX checkpoints are fetched at runtime from the public Space
+`multimodalart/microduck-ar`, so nothing is vendored here.
+
+Physics: MuJoCo, timestep 0.005 s, decimation 4 -> 50 Hz control.
+The observation layout (61D), the action scaling and the one-shot state
+machines are ported from that Space's `src/sim.js`, the reference
+implementation:
+
+    obs = [base_ang_vel(3), projected_gravity(3), joint_pos(14),
+           joint_vel(14), last_action(14), command(13)]
+    ctrl[j] = DEFAULT_POSE[j] + action[j] * ACTION_SCALE
+"""
+
+import os
+import sys
+import xml.etree.ElementTree as ET
+
+import numpy as np
+
+# A CPU Space has no GPU and no display, so MuJoCo's default EGL/GLFW backends
+# cannot make a context. osmesa is the software rasteriser (libosmesa6 in
+# packages.txt). Must be set before mujoco is imported.
+if sys.platform.startswith("linux") and not os.environ.get("MUJOCO_GL"):
+    os.environ["MUJOCO_GL"] = "osmesa"
+    os.environ.setdefault("PYOPENGL_PLATFORM", "osmesa")
+
+# -- Constants (from src/sim.js) -----------------------------------------
+JOINT_NAMES = [
+    "left_hip_yaw", "left_hip_roll", "left_hip_pitch", "left_knee", "left_ankle",
+    "neck_pitch", "head_pitch", "head_yaw", "head_roll",
+    "right_hip_yaw", "right_hip_roll", "right_hip_pitch", "right_knee", "right_ankle",
+]
+DEFAULT_POSE = np.array([
+    0, -0.08726646259971647, -0.457924, -0.004940, 0.452984,
+    0.3490658503988659, 0.3490658503988659, 0, 0,
+    0, 0.08726646259971647, 0.457924, 0.004940, -0.452984,
+], dtype=np.float32)
+
+NUM_JOINTS = 14
+OBS_SIZE = 61
+CMD_SIZE = 13
+ACTION_SCALE = 1.0
+TIMESTEP = 0.005
+DECIMATION = 4
+CTRL_DT = TIMESTEP * DECIMATION          # 50 Hz
+
+VEL_FWD, VEL_BACK, VEL_ANG = 0.25, -0.2, 1.0
+BALL_RADIUS = 0.05
+BALL_PARK = "50 0 0.05"
+FENCE_HALF, WALL_T, WALL_H = 1.0, 0.025, 0.125
+
+KICK_STEPS = 25
+POST_KICK_LOCK_STEPS = 20
+GROUND_PICK_PERIOD_S = 4.0
+GROUND_PICK_END_PHASE = 0.7
+FALL_DEBOUNCE_STEPS = 10
+FALL_SETTLE_STEPS = 15
+RECOVER_UPRIGHT_STEPS = 50
+RECOVER_GIVEUP_STEPS = 300
+
+POLICY_FILES = {
+    "walk": "BEST_alpha_walking.onnx",
+    "stand": "BEST_alpha_stand.onnx",
+    "kickL": "ball_kick_left.onnx",
+    "kickR": "ball_kick_right.onnx",
+    "roll": "roulade.onnx",
+    "groundpick": "alpha_ground_pick.onnx",
+}
+
+SPACE = "https://huggingface.co/spaces/multimodalart/microduck-ar/resolve/main"
+CACHE = os.environ.get("MICRODUCK_CACHE") or os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), ".microduck_cache")
+
+
+# -- Asset fetch ---------------------------------------------------------
+def _get(rel, dest):
+    """Download `rel` from the microduck-ar Space into `dest` once."""
+    if os.path.exists(dest) and os.path.getsize(dest) > 0:
+        return dest
+    import urllib.request
+    os.makedirs(os.path.dirname(dest), exist_ok=True)
+    tmp = dest + ".part"
+    with urllib.request.urlopen(SPACE + "/" + rel, timeout=120) as r:
+        blob = r.read()
+    with open(tmp, "wb") as f:
+        f.write(blob)
+    os.replace(tmp, dest)
+    return dest
+
+
+def ensure_assets():
+    """Fetch MJCF + meshes + policies. Returns (mjcf_path, mesh_dir, policy_dir)."""
+    mjcf = _get("public/robot/mjlab/robot_allcollisions.xml",
+                os.path.join(CACHE, "robot_allcollisions.xml"))
+    mesh_dir = os.path.join(CACHE, "assets")
+    root = ET.parse(mjcf).getroot()
+    for m in root.find("asset").iter("mesh"):
+        _get("public/robot/mjlab/meshes/" + m.get("file"),
+             os.path.join(mesh_dir, m.get("file")))
+    policy_dir = os.path.join(CACHE, "policies")
+    for fn in POLICY_FILES.values():
+        _get("public/policies/" + fn, os.path.join(policy_dir, fn))
+    return mjcf, mesh_dir, policy_dir
+
+
+# -- Model assembly ------------------------------------------------------
+def build_xml(mjcf_path, mesh_dir):
+    """Robot MJCF + floor, fence, ball, lights, chase camera and a STAND key.
+
+    Mirrors buildPhysicsXml() in sim.js, except the `visual` geoms are KEPT:
+    they are contype=0/conaffinity=0 so they cost nothing physically, and they
+    are what makes the render look like an actual duck.
+    """
+    root = ET.parse(mjcf_path).getroot()
+    root.find("compiler").set("meshdir", mesh_dir)
+    ET.SubElement(root, "option", {"timestep": str(TIMESTEP)})
+
+    asset = root.find("asset")
+    ET.SubElement(asset, "texture", {
+        "name": "sky", "type": "skybox", "builtin": "gradient",
+        "rgb1": "0.30 0.50 0.82", "rgb2": "0.88 0.94 1.0",
+        "width": "256", "height": "256"})
+    ET.SubElement(asset, "texture", {
+        "name": "grid", "type": "2d", "builtin": "checker",
+        "width": "512", "height": "512",
+        "rgb1": "0.93 0.93 0.96", "rgb2": "0.78 0.81 0.88"})
+    ET.SubElement(asset, "material", {
+        "name": "gridmat", "texture": "grid", "texrepeat": "14 14",
+        "reflectance": "0.08"})
+
+    world = root.find("worldbody")
+    ET.SubElement(world, "geom", {
+        "name": "floor", "type": "plane", "size": "0 0 0.05",
+        "pos": "0 0 0", "material": "gridmat"})
+    ET.SubElement(world, "light", {
+        "pos": "0.6 -0.6 1.8", "dir": "-0.3 0.3 -1",
+        "directional": "true", "diffuse": "0.75 0.75 0.75"})
+    ET.SubElement(world, "light", {
+        "pos": "-1 1 1.2", "dir": "0.5 -0.5 -1",
+        "directional": "true", "diffuse": "0.25 0.25 0.3"})
+
+    off, span = FENCE_HALF + WALL_T, FENCE_HALF + 0.05
+    for name, pos, size in [
+        ("wall_px", str(off) + " 0 " + str(WALL_H), str(WALL_T) + " " + str(span) + " " + str(WALL_H)),
+        ("wall_nx", str(-off) + " 0 " + str(WALL_H), str(WALL_T) + " " + str(span) + " " + str(WALL_H)),
+        ("wall_py", "0 " + str(off) + " " + str(WALL_H), str(span) + " " + str(WALL_T) + " " + str(WALL_H)),
+        ("wall_ny", "0 " + str(-off) + " " + str(WALL_H), str(span) + " " + str(WALL_T) + " " + str(WALL_H)),
+    ]:
+        # Invisible fence keeps the duck and ball in frame. group 3 = not drawn.
+        ET.SubElement(world, "geom", {
+            "name": name, "type": "box", "pos": pos, "size": size, "group": "3"})
+
+    ball = ET.SubElement(world, "body", {"name": "ball", "pos": BALL_PARK})
+    ET.SubElement(ball, "freejoint", {"name": "ball_freejoint"})
+    ET.SubElement(ball, "geom", {
+        "name": "ball_geom", "type": "sphere", "size": str(BALL_RADIUS),
+        "mass": "0.03", "friction": "0.4 0.01 0.003", "solref": "0.03 0.4",
+        "condim": "6", "rgba": "0.95 0.35 0.15 1"})
+
+    # STAND keyframe: freejoint(7) + every hinge in document order + ball(7).
+    pose = dict(zip(JOINT_NAMES, DEFAULT_POSE))
+    order = [j.get("name") for body in root.iter("body")
+             for j in body if j.tag == "joint"]
+    qj = " ".join(str(float(pose.get(n, 0.0))) for n in order)
+    kf = ET.SubElement(root, "keyframe")
+    ET.SubElement(kf, "key", {
+        "name": "STAND",
+        "qpos": "0 0 0.12 1 0 0 0 " + qj + " " + BALL_PARK + " 1 0 0 0",
+        "ctrl": " ".join(str(float(x)) for x in DEFAULT_POSE)})
+    return ET.tostring(root, encoding="unicode")
+
+
+# -- The sim -------------------------------------------------------------
+class Microduck:
+    def __init__(self, width=640, height=360, render=True):
+        import mujoco
+        import onnxruntime as ort
+        self.mj = mujoco
+        mjcf, mesh_dir, policy_dir = ensure_assets()
+        self.model = mujoco.MjModel.from_xml_string(build_xml(mjcf, mesh_dir))
+        self.data = mujoco.MjData(self.model)
+
+        opts = ort.SessionOptions()
+        opts.intra_op_num_threads = 1
+        self.sessions = {
+            k: ort.InferenceSession(
+                os.path.join(policy_dir, f), sess_options=opts,
+                providers=["CPUExecutionProvider"])
+            for k, f in POLICY_FILES.items()}
+
+        self.qadr = [self.model.jnt(n).qposadr[0] for n in JOINT_NAMES]
+        self.dadr = [self.model.jnt(n).dofadr[0] for n in JOINT_NAMES]
+        self.gyro = self.model.sensor("imu_ang_vel").adr[0]
+        self.trunk = mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_BODY, "trunk_base")
+        self.stand_key = mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_KEY, "STAND")
+        self.ball_q = self.model.jnt("ball_freejoint").qposadr[0]
+        self.ball_d = self.model.jnt("ball_freejoint").dofadr[0]
+
+        # Shadow mapping costs ~90 ms/frame here and only buys a blocky
+        # contact shadow, so it is off; multisampling is cheap and kept.
+        self.model.vis.quality.shadowsize = 0
+        self.model.vis.quality.offsamples = 4
+
+        self.renderer = mujoco.Renderer(self.model, height, width) if render else None
+        self.opt = mujoco.MjvOption()
+        for g in range(len(self.opt.geomgroup)):
+            self.opt.geomgroup[g] = 1 if g <= 2 else 0   # hide collision + fence
+        self.cam = mujoco.MjvCamera()
+        self.cam.type = mujoco.mjtCamera.mjCAMERA_TRACKING
+        self.cam.trackbodyid = self.trunk
+        self.cam.distance = 0.62
+        self.cam.elevation = -10.0
+        self.cam.lookat[:] = [0.0, 0.0, 0.05]
+        self.cam_offset = 125.0     # degrees behind-left of the duck's heading
+        self._az = None
+        self.reset()
+
+    # -- state ------------------------------------------------------------
+    def reset(self):
+        self.mj.mj_resetDataKeyframe(self.model, self.data, self.stand_key)
+        self.mj.mj_forward(self.model, self.data)
+        self.last_action = np.zeros(NUM_JOINTS, dtype=np.float32)
+        self.recovery = None
+        self.fall_debounce = 0
+        self.post_kick_lock = 0
+        self.ball_active = False
+
+    def proj_gravity(self):
+        w, x, y, z = self.data.body(self.trunk).xquat
+        R = np.array([
+            [1 - 2 * (y * y + z * z), 2 * (x * y - w * z), 2 * (x * z + w * y)],
+            [2 * (x * y + w * z), 1 - 2 * (x * x + z * z), 2 * (y * z - w * x)],
+            [2 * (x * z - w * y), 2 * (y * z + w * x), 1 - 2 * (x * x + y * y)]])
+        return R.T @ np.array([0.0, 0.0, -1.0])
+
+    def yaw(self):
+        q = self.data.qpos
+        return float(np.arctan2(2 * (q[3] * q[6] + q[4] * q[5]),
+                                1 - 2 * (q[5] ** 2 + q[6] ** 2)))
+
+    def spawn_ball(self, rng):
+        q, v = self.data.qpos, self.data.qvel
+        heading = self.yaw() + (rng.random() - 0.5) * 0.5
+        dist = 0.32 + (rng.random() - 0.5) * 0.08
+        margin = BALL_RADIUS + 0.05
+        lo, hi = -FENCE_HALF + margin, FENCE_HALF - margin
+        q[self.ball_q] = float(np.clip(q[0] + np.cos(heading) * dist, lo, hi))
+        q[self.ball_q + 1] = float(np.clip(q[1] + np.sin(heading) * dist, lo, hi))
+        q[self.ball_q + 2] = BALL_RADIUS + 0.02
+        q[self.ball_q + 3:self.ball_q + 7] = [1, 0, 0, 0]
+        v[self.ball_d:self.ball_d + 6] = 0
+        self.mj.mj_forward(self.model, self.data)
+        self.ball_active = True
+
+    def park_ball(self):
+        q, v = self.data.qpos, self.data.qvel
+        q[self.ball_q:self.ball_q + 3] = [50, 0, BALL_RADIUS]
+        q[self.ball_q + 3:self.ball_q + 7] = [1, 0, 0, 0]
+        v[self.ball_d:self.ball_d + 6] = 0
+        self.mj.mj_forward(self.model, self.data)
+        self.ball_active = False
+
+    # -- one control step (50 Hz) -----------------------------------------
+    def control_step(self, mode, cmd3, pick_phase=None):
+        if self.recovery is not None:
+            policy = "stand" if self.recovery["state"] == "recovering" else None
+        else:
+            policy = mode
+
+        if policy is not None:
+            obs = np.zeros(OBS_SIZE, dtype=np.float32)
+            i = 0
+            obs[i:i + 3] = self.data.sensordata[self.gyro:self.gyro + 3]
+            i += 3
+            obs[i:i + 3] = self.proj_gravity()
+            i += 3
+            obs[i:i + NUM_JOINTS] = self.data.qpos[self.qadr] - DEFAULT_POSE
+            i += NUM_JOINTS
+            obs[i:i + NUM_JOINTS] = self.data.qvel[self.dadr]
+            i += NUM_JOINTS
+            obs[i:i + NUM_JOINTS] = self.last_action
+            i += NUM_JOINTS
+            cmd = np.zeros(CMD_SIZE, dtype=np.float32)
+            if mode == "groundpick" and pick_phase is not None:
+                # Phase clock encoded as [cos, sin] in the velocity slots.
+                a = 2 * np.pi * pick_phase
+                cmd[0], cmd[1] = np.cos(a), np.sin(a)
+            elif self.recovery is None and self.post_kick_lock == 0:
+                cmd[:3] = cmd3
+            obs[i:i + CMD_SIZE] = cmd
+
+            act = self.sessions[policy].run(None, {"obs": obs.reshape(1, -1)})[0][0]
+            self.last_action = act.astype(np.float32)
+            self.data.ctrl[:NUM_JOINTS] = DEFAULT_POSE + act * ACTION_SCALE
+
+        for _ in range(DECIMATION):
+            self.mj.mj_step(self.model, self.data)
+
+        if self.post_kick_lock > 0:
+            self.post_kick_lock -= 1
+        self._update_recovery(mode)
+
+    def _update_recovery(self, mode):
+        z, gz = float(self.data.qpos[2]), float(self.proj_gravity()[2])
+        if not np.isfinite(z) or not np.isfinite(gz):
+            self.reset()
+            return
+        fallen = gz > -0.5 or z < 0.02
+        if self.recovery is not None:
+            self.recovery["steps"] += 1
+            if self.recovery["state"] == "fallen":
+                if self.recovery["steps"] >= FALL_SETTLE_STEPS:
+                    self.recovery = {"state": "recovering", "steps": 0, "upright": 0}
+                    self.last_action[:] = 0
+            else:
+                self.recovery["upright"] = (
+                    self.recovery["upright"] + 1 if gz < -0.85 else 0)
+                if self.recovery["upright"] >= RECOVER_UPRIGHT_STEPS:
+                    self.recovery = None
+                    self.last_action[:] = 0
+                elif self.recovery["steps"] >= RECOVER_GIVEUP_STEPS:
+                    self.reset()
+        elif fallen and mode in ("walk", "stand") and self.post_kick_lock == 0:
+            self.fall_debounce += 1
+            if self.fall_debounce >= FALL_DEBOUNCE_STEPS:
+                self.fall_debounce = 0
+                self.recovery = {"state": "fallen", "steps": 0}
+        else:
+            self.fall_debounce = 0
+
+    def frame(self):
+        """Render one frame from a tracking camera that eases in behind the duck."""
+        target = np.degrees(self.yaw()) + self.cam_offset
+        if self._az is None:
+            self._az = target
+        else:
+            # Shortest-arc smoothing, so a turn pans instead of snapping.
+            self._az += 0.05 * (((target - self._az + 180.0) % 360.0) - 180.0)
+        self.cam.azimuth = self._az
+        self.renderer.update_scene(self.data, camera=self.cam, scene_option=self.opt)
+        return self.renderer.render()

nodes.py

@@ -0,0 +1,236 @@
+"""The bound fn nodes for the Microduck Lab workflow.
+
+Two operators:
+  choreograph(routine)        -> plan JSON as TEXT
+  perform(plan, seed)         -> (video, chart image, report text)
+
+Everything structured travels as `text`, and both media outputs are returned
+as FileData-shaped dicts, because the workflow canvas string-coerces `json`
+and `dataframe` fn outputs and needs `{"path", "url"}` for media.
+"""
+
+import base64
+import json
+import os
+import sys
+import tempfile
+from typing import Optional
+
+import numpy as np
+from gradio import OAuthToken
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+import routine as R  # noqa: E402
+
+LLM = "Qwen/Qwen3-4B-Instruct-2507"
+
+SYSTEM = """You choreograph routines for Microduck, a 25 cm bipedal robot.
+
+Reply with ONLY a JSON array of steps. Each step is {"move": <name>} plus
+"seconds" for the timed moves. Allowed moves:
+
+  timed (need "seconds", 0.2-8.0):
+    forward, backward, strafe_left, strafe_right,
+    turn_left, turn_right, stand
+  one-shot (no "seconds", they run their trained cycle):
+    kick_left, kick_right   - kicks a ball, one is spawned in front
+    roll                    - a roulade / forward tumble
+    pick                    - bends down and picks an object off the floor
+
+Rules:
+- Include EVERY action the user asks for, in the order they ask for it.
+  Never drop one, never reorder them.
+- Only add a move the user did not ask for if the routine needs it to make
+  sense (e.g. walking up to a ball before kicking it).
+- Keep the total under 20 seconds, and end with a short "stand" so the
+  routine settles.
+
+Output the JSON array and nothing else."""
+
+# One worked example. Without it both Qwen3-4B and Gemma-3-27B silently drop
+# the first move of a multi-part request; with it, both follow the order.
+FEWSHOT = [
+    {"role": "user",
+     "content": "waddle forward a bit, then turn right and kick the ball"},
+    {"role": "assistant",
+     "content": '[{"move":"forward","seconds":2.5},'
+                '{"move":"turn_right","seconds":1.5},'
+                '{"move":"kick_right"},'
+                '{"move":"stand","seconds":1.0}]'},
+]
+
+# Words -> moves, for the no-token fallback choreographer.
+_KEYWORDS = [
+    (("kick left", "left foot", "kick with its left"), {"move": "kick_left"}),
+    (("kick", "ball", "shoot", "football", "soccer", "penalty"), {"move": "kick_right"}),
+    (("roll", "roulade", "somersault", "tumble", "flip", "barrel"), {"move": "roll"}),
+    (("pick", "grab", "fetch", "sock", "pick up", "bend"), {"move": "pick"}),
+    (("turn left", "left turn", "veer left"), {"move": "turn_left", "seconds": 1.5}),
+    (("turn right", "right turn", "veer right"), {"move": "turn_right", "seconds": 1.5}),
+    (("spin", "pirouette", "circle", "twirl"), {"move": "turn_left", "seconds": 3.0}),
+    (("back", "reverse", "retreat"), {"move": "backward", "seconds": 2.0}),
+    (("wait", "pause", "stand", "still", "idle"), {"move": "stand", "seconds": 1.5}),
+    (("walk", "waddle", "forward", "march", "stroll", "ahead"),
+     {"move": "forward", "seconds": 2.5}),
+]
+
+
+def _fallback_plan(routine_text):
+    """Keyword choreographer, used when no HF token is available.
+
+    Keeps the demo usable signed-out; the LLM path is the upgrade.
+    """
+    text = (routine_text or "").lower()
+    steps, seen = [], set()
+    # Preserve the order the user mentioned things in.
+    hits = []
+    for words, step in _KEYWORDS:
+        pos = min((text.find(w) for w in words if w in text), default=-1)
+        if pos >= 0 and step["move"] not in seen:
+            seen.add(step["move"])
+            hits.append((pos, step))
+    for _, step in sorted(hits, key=lambda h: h[0]):
+        steps.append(dict(step))
+    if not steps:
+        steps = [{"move": "forward", "seconds": 2.5}]
+    if steps[0]["move"] in R.ONE_SHOT:
+        steps.insert(0, {"move": "forward", "seconds": 1.5})
+    steps.append({"move": "stand", "seconds": 1.0})
+    return steps
+
+
+def choreograph(routine_text: str, oauth_token: Optional[OAuthToken] = None) -> str:
+    """Turn a plain-English routine into a Microduck move plan (JSON text)."""
+    routine_text = (routine_text or "").strip() or "waddle forward and look around"
+    token = getattr(oauth_token, "token", None) or os.environ.get("HF_TOKEN")
+    if token is None:
+        try:
+            from huggingface_hub import get_token
+            token = get_token()
+        except Exception:
+            token = None
+
+    steps = None
+    if token:
+        try:
+            from huggingface_hub import InferenceClient
+            reply = InferenceClient(token=token).chat_completion(
+                model=LLM,
+                messages=([{"role": "system", "content": SYSTEM}] + FEWSHOT
+                          + [{"role": "user", "content": routine_text}]),
+                max_tokens=400,
+                temperature=0.2,
+            ).choices[0].message.content
+            parsed, _ = R.parse_plan(reply)
+            if parsed:
+                steps = parsed
+        except Exception:
+            steps = None          # any provider hiccup -> keyword fallback
+
+    if steps is None:
+        steps = _fallback_plan(routine_text)
+        parsed, _ = R.parse_plan(json.dumps(steps))
+        steps = parsed or [{"move": "forward", "seconds": 2.5}]
+    return json.dumps(steps, indent=2)
+
+
+def _filedata(path, mime):
+    """The one output shape both the canvas and the REST API accept."""
+    with open(path, "rb") as f:
+        b64 = base64.b64encode(f.read()).decode()
+    return {"path": path, "url": "data:" + mime + ";base64," + b64}
+
+
+def _chart(tel):
+    """Ground track + uprightness, drawn as an image (dataframe/json ports are
+    string-coerced by the canvas, image ports are not)."""
+    import matplotlib
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+
+    track = np.array(tel["track"]) if tel["track"] else np.zeros((1, 2))
+    upright = np.array(tel["upright"]) if tel["upright"] else np.zeros(1)
+    t = np.arange(len(upright)) / max(tel["fps"], 1)
+
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 4.0), dpi=110)
+    fig.patch.set_facecolor("white")
+
+    rec = np.array([m == "recover" for m in tel["modes"]])
+    ax1.plot(track[:, 0], track[:, 1], lw=2.0, color="#f0a500", zorder=2)
+    if rec.any():
+        ax1.scatter(track[rec, 0], track[rec, 1], s=14, color="#d94a2b",
+                    zorder=3, label="recovering")
+        ax1.legend(loc="best", fontsize=8, frameon=False)
+    ax1.scatter([track[0, 0]], [track[0, 1]], s=70, marker="o",
+                color="#2e7d32", zorder=4)
+    ax1.scatter([track[-1, 0]], [track[-1, 1]], s=90, marker="*",
+                color="#1565c0", zorder=4)
+    ax1.set_title("ground track (start ● / end ★)", fontsize=10)
+    ax1.set_xlabel("x (m)"); ax1.set_ylabel("y (m)")
+    ax1.set_aspect("equal", adjustable="datalim")
+    ax1.grid(alpha=0.25)
+
+    ax2.plot(t, upright, lw=1.6, color="#37474f")
+    ax2.axhline(-1.0, ls=":", lw=1, color="#2e7d32")
+    ax2.axhline(-0.5, ls="--", lw=1, color="#d94a2b")
+    ax2.fill_between(t, -0.5, upright, where=(upright > -0.5),
+                     color="#d94a2b", alpha=0.18)
+    ax2.set_ylim(1.05, -1.05)
+    ax2.set_title("uprightness  (-1 = standing, > -0.5 = down)", fontsize=10)
+    ax2.set_xlabel("t (s)"); ax2.set_ylabel("projected gravity z")
+    ax2.grid(alpha=0.25)
+
+    fig.tight_layout()
+    png = tempfile.NamedTemporaryFile(suffix=".png", delete=False).name
+    fig.savefig(png, facecolor="white")
+    plt.close(fig)
+    return png
+
+
+def _report(plan_steps, tel):
+    lines = [
+        "Microduck routine - " + str(tel["duration_s"]) + " s of simulated time",
+        "",
+        "policies: the six ONNX checkpoints shipped with Microduck, stepped at",
+        "50 Hz against MuJoCo physics (timestep 0.005 s, decimation 4).",
+        "",
+        "step                t (s)          moved     ",
+        "-------------------------------------------- ",
+    ]
+    for entry in tel["log"]:
+        span = str(entry["t_start"]) + "-" + str(entry["t_end"])
+        lines.append("%-18s %-14s %6.3f m" % (entry["move"], span, entry["travelled_m"]))
+    lines += [
+        "",
+        "net displacement : " + str(tel["distance_m"]) + " m",
+        "falls + recoveries: " + str(tel["falls"]),
+        "ended upright    : " + ("yes" if tel["final_upright"] < -0.85 else "no")
+        + "  (projected gravity z = " + str(tel["final_upright"]) + ")",
+    ]
+    return "\n".join(lines)
+
+
+def perform(plan: str, seed: float = 0, fps: float = 25):
+    """Run a plan through the real policies. Returns (video, chart, report)."""
+    steps, notes = R.parse_plan(plan)
+    if not steps:
+        raise ValueError("No runnable moves in the plan. " + "; ".join(notes))
+
+    frames, tel = R.run(steps, seed=int(seed or 0), width=640, height=360,
+                        fps=int(fps or 25))
+
+    import imageio
+    mp4 = tempfile.NamedTemporaryFile(suffix=".mp4", delete=False).name
+    imageio.mimsave(mp4, frames, fps=tel["fps"], codec="libx264",
+                    quality=7, macro_block_size=1)
+
+    report = _report(steps, tel)
+    if notes:
+        report += "\n\nnotes: " + "; ".join(notes)
+    return (_filedata(mp4, "video/mp4"),
+            _filedata(_chart(tel), "image/png"),
+            report)
+
+
+BIND = {"choreograph": choreograph, "perform": perform}

packages.txt

@@ -0,0 +1,3 @@
+libosmesa6
+libgl1
+libglx-mesa0

requirements.txt

@@ -0,0 +1,7 @@
+mujoco>=3.2
+onnxruntime
+numpy
+matplotlib
+imageio
+imageio-ffmpeg
+PyOpenGL

routine.py

@@ -0,0 +1,205 @@
+"""Turn a routine (a list of moves) into a rollout of the real policies.
+
+A plan is a JSON list of steps, e.g.
+
+    [{"move": "forward", "seconds": 2.5},
+     {"move": "turn_left", "seconds": 1.2},
+     {"move": "kick_right"},
+     {"move": "roll"}]
+
+Duration-based moves (`forward`..`stand`) take `seconds`; the one-shots
+(`kick_*`, `roll`, `pick`) run for exactly as long as their trained cycle
+lasts, so `seconds` is ignored for them.
+"""
+
+import json
+
+import numpy as np
+
+import duck as D
+
+# move -> (policy mode, command [vx, vy, wz])
+MOVES = {
+    "forward":    ("walk", (D.VEL_FWD, 0.0, 0.0)),
+    "backward":   ("walk", (D.VEL_BACK, 0.0, 0.0)),
+    "strafe_left":  ("walk", (0.0, 0.15, 0.0)),
+    "strafe_right": ("walk", (0.0, -0.15, 0.0)),
+    "turn_left":  ("walk", (0.0, 0.0, D.VEL_ANG)),
+    "turn_right": ("walk", (0.0, 0.0, -D.VEL_ANG)),
+    "stand":      ("walk", (0.0, 0.0, 0.0)),
+    "kick_left":  ("kickL", (0.0, 0.0, 0.0)),
+    "kick_right": ("kickR", (0.0, 0.0, 0.0)),
+    "roll":       ("roll", (0.0, 0.0, 0.0)),
+    "pick":       ("groundpick", (0.0, 0.0, 0.0)),
+}
+ONE_SHOT = {"kick_left", "kick_right", "roll", "pick"}
+MAX_SECONDS = 30.0
+
+# Loose spellings the LLM (or a human) may produce.
+ALIASES = {
+    "walk": "forward", "walk_forward": "forward", "forwards": "forward",
+    "back": "backward", "backwards": "backward", "reverse": "backward",
+    "left": "turn_left", "right": "turn_right",
+    "rotate_left": "turn_left", "rotate_right": "turn_right",
+    "spin": "turn_left", "spin_left": "turn_left", "spin_right": "turn_right",
+    "sidestep_left": "strafe_left", "sidestep_right": "strafe_right",
+    "wait": "stand", "idle": "stand", "pause": "stand", "look": "stand",
+    "kick": "kick_right", "shoot": "kick_right",
+    "roulade": "roll", "somersault": "roll", "flip": "roll", "tumble": "roll",
+    "pickup": "pick", "pick_up": "pick", "grab": "pick", "fetch": "pick",
+}
+
+
+def parse_plan(text):
+    """Parse a plan from JSON text. Returns (steps, notes).
+
+    Tolerant on purpose: the plan usually comes from an LLM. Unknown moves are
+    dropped and reported in `notes` rather than raising.
+    """
+    notes = []
+    raw = (text or "").strip()
+    if not raw:
+        return [], ["empty plan"]
+    # Tolerate ```json fences and any prose around the array.
+    if "```" in raw:
+        raw = raw.split("```")[1]
+        if raw.lstrip().lower().startswith("json"):
+            raw = raw.lstrip()[4:]
+    start, end = raw.find("["), raw.rfind("]")
+    if start != -1 and end > start:
+        raw = raw[start:end + 1]
+    try:
+        parsed = json.loads(raw)
+    except json.JSONDecodeError as e:
+        return [], ["could not parse plan as JSON: " + str(e)]
+    if isinstance(parsed, dict):
+        parsed = parsed.get("plan") or parsed.get("steps") or [parsed]
+    if not isinstance(parsed, list):
+        return [], ["plan is not a list"]
+
+    steps, total = [], 0.0
+    for item in parsed:
+        if isinstance(item, str):
+            item = {"move": item}
+        if not isinstance(item, dict):
+            continue
+        name = str(item.get("move") or item.get("action") or "").strip().lower()
+        name = name.replace("-", "_").replace(" ", "_")
+        name = ALIASES.get(name, name)
+        if name not in MOVES:
+            notes.append("skipped unknown move " + repr(name))
+            continue
+        if name in ONE_SHOT:
+            secs = 0.0
+        else:
+            try:
+                secs = float(item.get("seconds", item.get("duration", 1.5)))
+            except (TypeError, ValueError):
+                secs = 1.5
+            secs = float(np.clip(secs, 0.2, 8.0))
+        if total + max(secs, 3.0) > MAX_SECONDS:
+            notes.append("plan truncated at " + str(MAX_SECONDS) + "s")
+            break
+        total += secs if secs else 3.0
+        steps.append({"move": name, "seconds": secs})
+    if not steps:
+        notes.append("no runnable moves found")
+    return steps, notes
+
+
+def run(plan_steps, seed=0, width=640, height=360, fps=25, on_frame=None):
+    """Roll the plan out through the real policies.
+
+    Returns (frames, telemetry). `telemetry` carries the ground track, the
+    upright measure and a per-step log.
+    """
+    sim = D.Microduck(width=width, height=height, render=on_frame is None)
+    rng = np.random.default_rng(seed)
+    every = max(1, int(round((1.0 / D.CTRL_DT) / fps)))   # 50 Hz -> every 2nd
+
+    frames, track, upright, modes = [], [], [], []
+    log = []
+    tick = 0
+
+    def settle(min_steps=25, max_steps=400):
+        """Hand back to the walk policy, and if the move ended on the floor let
+        the stand policy finish standing up before the next move starts."""
+        for n in range(max_steps):
+            do_step("walk", (0.0, 0.0, 0.0))
+            if n >= min_steps and sim.recovery is None:
+                return
+
+    def do_step(mode, cmd, phase=None):
+        nonlocal tick
+        sim.control_step(mode, cmd, phase)
+        if tick % every == 0:
+            img = sim.frame() if on_frame is None else on_frame(sim)
+            if img is not None:
+                frames.append(img)
+            track.append((float(sim.data.qpos[0]), float(sim.data.qpos[1])))
+            upright.append(float(sim.proj_gravity()[2]))
+            modes.append("recover" if sim.recovery else mode)
+        tick += 1
+
+    for step in plan_steps:
+        name = step["move"]
+        mode, cmd = MOVES[name]
+        t0 = tick * D.CTRL_DT
+        x0, y0 = float(sim.data.qpos[0]), float(sim.data.qpos[1])
+
+        if name in ("kick_left", "kick_right"):
+            if not sim.ball_active:
+                sim.spawn_ball(rng)
+            for _ in range(D.KICK_STEPS):
+                do_step(mode, cmd)
+            sim.post_kick_lock = D.POST_KICK_LOCK_STEPS
+            for _ in range(D.POST_KICK_LOCK_STEPS):
+                do_step("walk", (0.0, 0.0, 0.0))
+
+        elif name == "roll":
+            tipped, n = False, 0
+            while n < 150:
+                do_step(mode, cmd)
+                n += 1
+                gz = float(sim.proj_gravity()[2])
+                if gz > -0.3:
+                    tipped = True
+                if tipped and gz < -0.85 and n >= 40:
+                    break
+            sim.last_action[:] = 0
+            settle()
+
+        elif name == "pick":
+            phase, n = 0.0, 0
+            while phase < D.GROUND_PICK_END_PHASE and n < 300:
+                do_step(mode, cmd, phase)
+                phase += D.CTRL_DT / D.GROUND_PICK_PERIOD_S
+                n += 1
+            settle()
+
+        else:
+            for _ in range(int(round(step["seconds"] / D.CTRL_DT))):
+                do_step(mode, cmd)
+
+        log.append({
+            "move": name,
+            "t_start": round(t0, 2),
+            "t_end": round(tick * D.CTRL_DT, 2),
+            "travelled_m": round(float(np.hypot(sim.data.qpos[0] - x0,
+                                                sim.data.qpos[1] - y0)), 3),
+            "recovered": bool(sim.recovery),
+        })
+
+    telemetry = {
+        "track": track,
+        "upright": upright,
+        "modes": modes,
+        "log": log,
+        "duration_s": round(tick * D.CTRL_DT, 2),
+        "fps": fps,
+        "falls": sum(1 for a, b in zip(["walk"] + modes, modes)
+                     if b == "recover" and a != "recover"),
+        "distance_m": round(float(np.hypot(sim.data.qpos[0], sim.data.qpos[1])), 3),
+        "final_upright": round(float(sim.proj_gravity()[2]), 3),
+    }
+    return frames, telemetry

test_pipelines.py

@@ -0,0 +1,141 @@
+"""End-to-end test: every subject in workflow.json through the real
+`WorkflowExecutor` — the same code path the canvas and the REST API use.
+
+    python apps/09_microduck_lab/test_pipelines.py
+
+Outputs land in ./_test_output for eyeballing. The choreographer hits an HF
+Inference Provider when a token is present and otherwise falls back to the
+keyword planner, so this runs offline too (after the first asset download).
+"""
+
+import json
+import logging
+import os
+import shutil
+import sys
+import time
+import types
+import warnings
+
+warnings.filterwarnings("ignore")
+logging.disable(logging.CRITICAL)
+
+for _s in (sys.stdout, sys.stderr):
+    try:
+        _s.reconfigure(encoding="utf-8", errors="replace")
+    except (AttributeError, ValueError):
+        pass
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+
+import gradio.workflow as W  # noqa: E402
+from gradio.helpers import special_args  # noqa: E402
+from gradio.workflow_api import (  # noqa: E402
+    WorkflowExecutor,
+    WorkflowGraph,
+    group_free_inputs,
+)
+
+import nodes as N  # noqa: E402
+
+try:
+    from huggingface_hub import get_token
+    _tok = get_token()
+except Exception:
+    _tok = None
+TOKEN = types.SimpleNamespace(token=_tok or os.environ.get("HF_TOKEN"))
+
+OUTDIR = os.path.join(HERE, "_test_output")
+os.makedirs(OUTDIR, exist_ok=True)
+
+
+def call_fn(data, request=None, token=None):
+    """Mirror of gradio's bound-function server fn (workflow.py `call_fn`)."""
+    name = data[0] if data else ""
+    fn = N.BIND.get(name)
+    if fn is None:
+        return json.dumps({"error": "No function '" + str(name) + "' bound"})
+    try:
+        args = json.loads(data[1] if len(data) > 1 else "[]")
+        if not isinstance(args, list):
+            args = [args]
+        args, *_ = special_args(fn, args, request, None, token=token)
+        result = fn(*args)
+        return json.dumps(list(result) if isinstance(result, (list, tuple)) else [result])
+    except Exception as e:
+        return json.dumps({"error": type(e).__name__ + ": " + str(e)})
+
+
+CALLERS = {"fn": call_fn, "model": W.call_model,
+           "space": W.call_space, "dataset": W.fetch_dataset}
+
+with open(os.path.join(HERE, "workflow.json"), encoding="utf-8") as f:
+    GRAPH = WorkflowGraph.from_json(f.read())
+
+
+def seed_for(subject_id):
+    node = GRAPH.node_by_id[subject_id]
+    inputs = {}
+    for free in group_free_inputs(GRAPH, [node]):
+        ref = free["node"]
+        inputs[ref["id"]] = (ref.get("data") or {}).get("out")
+    return inputs
+
+
+def save(subject_id, value):
+    """Persist an output so it can be looked at; return a one-line summary."""
+    if isinstance(value, dict) and value.get("path"):
+        src = value["path"]
+        ext = os.path.splitext(src)[1] or ".bin"
+        dst = os.path.join(OUTDIR, subject_id + ext)
+        shutil.copyfile(src, dst)
+        kb = os.path.getsize(dst) // 1024
+        url = str(value.get("url") or "")
+        shape = "path+url" if url.startswith("data:") else "path only (CANVAS WILL BREAK)"
+        return ext[1:].upper() + ", " + str(kb) + " KB, " + shape + " -> " + os.path.basename(dst)
+
+    if isinstance(value, str) and os.path.isfile(value):
+        dst = os.path.join(OUTDIR, subject_id + os.path.splitext(value)[1])
+        shutil.copyfile(value, dst)
+        return "file -> " + os.path.basename(dst)
+
+    text = str(value)
+    with open(os.path.join(OUTDIR, subject_id + ".txt"), "w", encoding="utf-8") as f:
+        f.write(text)
+    first = text.strip().splitlines()[0][:70] if text.strip() else "(empty)"
+    return "text (" + str(len(text)) + " chars): " + first
+
+
+def main():
+    executor = WorkflowExecutor(GRAPH, CALLERS)
+    targets = [s["id"] for s in GRAPH.subjects]
+    if len(sys.argv) > 1:
+        f = [a.lower() for a in sys.argv[1:]]
+        targets = [t for t in targets if any(a in t.lower() for a in f)]
+
+    print("\nRunning " + str(len(targets)) + " output(s) through WorkflowExecutor")
+    print("outputs -> " + os.path.relpath(OUTDIR, os.getcwd()) + "\n")
+
+    passed, failed = [], []
+    for sid in targets:
+        label = GRAPH.node_by_id[sid]["label"]
+        t0 = time.time()
+        try:
+            value = executor.run(sid, seed_for(sid), request=None, token=TOKEN)
+            if value is None or value == "":
+                raise AssertionError("output was empty")
+            print("  PASS  %-14s %-16s %6.1fs  %s"
+                  % (sid, label, time.time() - t0, save(sid, value)))
+            passed.append(sid)
+        except Exception as e:
+            print("  FAIL  %-14s %-16s %6.1fs  %s: %s"
+                  % (sid, label, time.time() - t0, type(e).__name__, e))
+            failed.append(sid)
+
+    print("\n" + str(len(passed)) + " passed, " + str(len(failed)) + " failed")
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

workflow.json

@@ -0,0 +1,271 @@
+{
+  "schema_version": "2",
+  "name": "Microduck Lab",
+  "references": [
+    {
+      "id": "ref_routine",
+      "role": "reference",
+      "label": "Routine",
+      "asset_type": "text",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Routine",
+          "type": "text"
+        }
+      ],
+      "outputs": [
+        {
+          "id": "out",
+          "label": "Routine",
+          "type": "text"
+        }
+      ],
+      "data": {
+        "out": "waddle forward, turn left, then kick the ball with your right foot and finish with a barrel roll"
+      },
+      "x": 40,
+      "y": 60,
+      "width": 250,
+      "height": 150
+    },
+    {
+      "id": "ref_seed",
+      "role": "reference",
+      "label": "Seed",
+      "asset_type": "number",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Seed",
+          "type": "number"
+        }
+      ],
+      "outputs": [
+        {
+          "id": "out",
+          "label": "Seed",
+          "type": "number"
+        }
+      ],
+      "data": {
+        "out": 0
+      },
+      "x": 40,
+      "y": 300,
+      "width": 250,
+      "height": 110
+    }
+  ],
+  "operators": [
+    {
+      "id": "op_plan",
+      "role": "operator",
+      "kind": "fn",
+      "fn": "choreograph",
+      "label": "choreograph (LLM)",
+      "inputs": [
+        {
+          "id": "in_routine",
+          "label": "routine",
+          "type": "text",
+          "required": true
+        }
+      ],
+      "outputs": [
+        {
+          "id": "out_0",
+          "label": "plan",
+          "type": "text",
+          "output_index": 0
+        }
+      ],
+      "data": {},
+      "x": 350,
+      "y": 80,
+      "width": 250,
+      "height": 130
+    },
+    {
+      "id": "op_run",
+      "role": "operator",
+      "kind": "fn",
+      "fn": "perform",
+      "label": "perform (MuJoCo + ONNX)",
+      "inputs": [
+        {
+          "id": "in_plan",
+          "label": "plan",
+          "type": "text",
+          "required": true
+        },
+        {
+          "id": "in_seed",
+          "label": "seed",
+          "type": "number"
+        }
+      ],
+      "outputs": [
+        {
+          "id": "out_0",
+          "label": "video",
+          "type": "video",
+          "output_index": 0
+        },
+        {
+          "id": "out_1",
+          "label": "chart",
+          "type": "image",
+          "output_index": 1
+        },
+        {
+          "id": "out_2",
+          "label": "report",
+          "type": "text",
+          "output_index": 2
+        }
+      ],
+      "data": {},
+      "x": 660,
+      "y": 150,
+      "width": 260,
+      "height": 180
+    }
+  ],
+  "subjects": [
+    {
+      "id": "sub_video",
+      "role": "subject",
+      "label": "Routine video",
+      "asset_type": "video",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Video",
+          "type": "video"
+        }
+      ],
+      "outputs": [],
+      "data": {},
+      "x": 990,
+      "y": 40,
+      "width": 280,
+      "height": 230
+    },
+    {
+      "id": "sub_chart",
+      "role": "subject",
+      "label": "Telemetry",
+      "asset_type": "image",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Telemetry",
+          "type": "image"
+        }
+      ],
+      "outputs": [],
+      "data": {},
+      "x": 990,
+      "y": 300,
+      "width": 280,
+      "height": 210
+    },
+    {
+      "id": "sub_report",
+      "role": "subject",
+      "label": "Report",
+      "asset_type": "text",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Report",
+          "type": "text"
+        }
+      ],
+      "outputs": [],
+      "data": {},
+      "x": 990,
+      "y": 540,
+      "width": 280,
+      "height": 180
+    },
+    {
+      "id": "sub_plan",
+      "role": "subject",
+      "label": "Move plan",
+      "asset_type": "text",
+      "inputs": [
+        {
+          "id": "in",
+          "label": "Plan",
+          "type": "text"
+        }
+      ],
+      "outputs": [],
+      "data": {},
+      "x": 350,
+      "y": 300,
+      "width": 250,
+      "height": 200
+    }
+  ],
+  "edges": [
+    {
+      "id": "e1",
+      "from_node_id": "ref_routine",
+      "from_port_id": "out",
+      "to_node_id": "op_plan",
+      "to_port_id": "in_routine",
+      "type": "text"
+    },
+    {
+      "id": "e2",
+      "from_node_id": "op_plan",
+      "from_port_id": "out_0",
+      "to_node_id": "op_run",
+      "to_port_id": "in_plan",
+      "type": "text"
+    },
+    {
+      "id": "e3",
+      "from_node_id": "op_plan",
+      "from_port_id": "out_0",
+      "to_node_id": "sub_plan",
+      "to_port_id": "in",
+      "type": "text"
+    },
+    {
+      "id": "e4",
+      "from_node_id": "ref_seed",
+      "from_port_id": "out",
+      "to_node_id": "op_run",
+      "to_port_id": "in_seed",
+      "type": "number"
+    },
+    {
+      "id": "e5",
+      "from_node_id": "op_run",
+      "from_port_id": "out_0",
+      "to_node_id": "sub_video",
+      "to_port_id": "in",
+      "type": "video"
+    },
+    {
+      "id": "e6",
+      "from_node_id": "op_run",
+      "from_port_id": "out_1",
+      "to_node_id": "sub_chart",
+      "to_port_id": "in",
+      "type": "image"
+    },
+    {
+      "id": "e7",
+      "from_node_id": "op_run",
+      "from_port_id": "out_2",
+      "to_node_id": "sub_report",
+      "to_port_id": "in",
+      "type": "text"
+    }
+  ]
+}