Sync heatmap-generation from metro-analytics-catalog

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+expected_output_dlstreamer.gif filter=lfs diff=lfs merge=lfs -text
+expected_output_openvino.gif filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Intel Corporation.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE

README.md

@@ -0,0 +1,407 @@
+---
+license: mit
+license_link: LICENSE
+library_name: openvino
+pipeline_tag: object-detection
+tags:
+  - openvino
+  - intel
+  - yolo
+  - yolo26
+  - heatmap
+  - speed
+  - traffic
+  - tracking
+  - edge-ai
+  - metro
+  - dlstreamer
+language:
+  - en
+---
+
+# Heatmap Generation
+
+| Property | Value |
+|---|---|
+| **Category** | Object Detection + Speed Heatmap Aggregation |
+| **Base Model** | [YOLO26](https://docs.ultralytics.com/models/yolo26/) (Ultralytics) |
+| **Source Framework** | PyTorch (Ultralytics) |
+| **Supported Precisions** | FP32, FP16, INT8 (mixed-precision) |
+| **Inference Engine** | OpenVINO |
+| **Hardware** | CPU, GPU, NPU |
+| **Detected Class(es)** | All 80 COCO classes (heatmap colored by object speed) |
+
+---
+
+## Overview
+
+Heatmap Generation is a Metro Analytics use case that detects objects across
+video frames and colors each region of the scene by how fast traffic moves
+through it. It is built on [YOLO26](https://docs.ultralytics.com/models/yolo26/), a
+state-of-the-art real-time object detector, quantized to INT8 for efficient
+inference on Intel hardware.
+
+Each detection's per-frame displacement is used as a speed estimate, deposited
+over the object's footprint and averaged per location with Gaussian smoothing
+into a color-coded overlay.
+
+The overlay uses the following color scheme:
+
+- **Red** -- fast-moving traffic.
+- **Yellow / green** -- medium speed.
+- **Blue** -- slow-moving or stationary traffic.
+
+Typical Metro deployments include:
+
+- **Traffic Speed Mapping** -- highlight fast corridors and slow/congested lanes.
+- **Congestion Detection** -- surface persistently slow (blue) areas for safety planning.
+- **Pedestrian Flow Analysis** -- compare fast throughways against lingering areas.
+- **Incident Spotting** -- flag unusually fast or stalled movement.
+
+Available variants: `yolo26n`, `yolo26s`, `yolo26m`, `yolo26l`, `yolo26x`.
+Smaller variants (`yolo26n`, `yolo26s`) are recommended for high-FPS edge
+deployment; larger variants improve recall for small or distant objects.
+
+---
+
+## Prerequisites
+
+- Python 3.11+
+- [Install OpenVINO](https://docs.openvino.ai/2026/get-started/install-openvino.html) (latest version)
+- [Install Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/get_started/install/install_guide_ubuntu.html) (latest version)
+
+Create and activate a Python virtual environment before running the scripts:
+
+```bash
+python3 -m venv .venv --system-site-packages
+source .venv/bin/activate
+```
+
+> **Note:** The `--system-site-packages` flag is required so the virtual
+> environment can access the system-installed OpenVINO and DLStreamer Python
+> packages.
+
+---
+
+## Getting Started
+
+### Download and Quantize Model
+
+Run the provided script to download, export to OpenVINO IR, and optionally quantize:
+
+```bash
+chmod +x export_and_quantize.sh
+./export_and_quantize.sh
+```
+
+This exports the default **yolo26n** model in **FP16** precision.
+
+#### Optional: Select a Different Variant or Precision
+
+```bash
+./export_and_quantize.sh yolo26n FP32   # full-precision
+./export_and_quantize.sh yolo26n INT8   # quantized
+./export_and_quantize.sh yolo26s        # larger variant, default FP16
+```
+
+The script performs the following steps:
+
+1. Installs dependencies (`openvino`, `ultralytics`; adds `nncf` for INT8).
+2. Downloads a sample test image (`test.jpg`) and a sample test video (`test_video.mp4`).
+3. Downloads the PyTorch weights and exports to OpenVINO IR.
+4. *(INT8 only)* Quantizes the model using NNCF post-training quantization.
+
+Output files:
+
+- `yolo26n_openvino_model/` -- FP32 or FP16 OpenVINO IR model directory.
+- `yolo26n_heatmap_int8.xml` / `.bin` -- INT8 quantized model *(only when `INT8` is selected)*.
+
+#### Precision / Device Compatibility
+
+| Precision | CPU | GPU | NPU |
+|---|---|---|---|
+| FP32 | Yes | Yes | No |
+| FP16 | Yes | Yes | Yes |
+| INT8 | Yes | Yes | Yes |
+
+### OpenVINO Sample
+
+The sample below runs YOLO26 inference on a video, estimates each object's
+speed from its per-frame displacement, and writes a speed-colored heatmap
+overlay (red = fast, blue = slow) to `output_openvino.mp4`.
+It also saves the final speed heatmap as `heatmap.jpg`.
+Change the `device` string to run on CPU, GPU, or NPU.
+
+```python
+import cv2
+import numpy as np
+import openvino as ov
+
+CONF_THRESHOLD = 0.4
+INPUT_SIZE = 640
+HEATMAP_ALPHA = 0.55
+MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
+MAX_SPEED = 20.0    # px/frame that maps to full red
+
+
+def render_speed_heatmap(frame, speed_sum, count, alpha):
+    """Color traffic by average speed: blue = slow, yellow = medium,
+    red = fast. Only regions where vehicles were seen are tinted, so
+    empty background keeps its original color."""
+    avg = np.zeros_like(speed_sum)
+    seen = count > 0
+    avg[seen] = speed_sum[seen] / count[seen]
+    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
+    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
+    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
+    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
+    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
+    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
+    return overlay.astype(np.uint8), color
+
+
+core = ov.Core()
+model = core.read_model("yolo26n_openvino_model/yolo26n.xml")
+
+# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
+compiled = core.compile_model(model, "CPU")
+
+cap = cv2.VideoCapture("test_video.mp4")
+fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
+width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+writer = cv2.VideoWriter(
+    "output_openvino.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (width, height))
+
+speed_sum = np.zeros((height, width), dtype=np.float32)
+count = np.zeros((height, width), dtype=np.float32)
+prev_centroids = []
+heatmap_color = None
+frame_idx = 0
+total_dets = 0
+
+while True:
+    ok, frame = cap.read()
+    if not ok:
+        break
+    frame_idx += 1
+    h0, w0 = frame.shape[:2]
+    sx, sy = w0 / INPUT_SIZE, h0 / INPUT_SIZE
+
+    blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
+    blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
+    blob = blob.transpose(2, 0, 1)[np.newaxis, ...]
+
+    output = compiled([blob])[compiled.output(0)][0]
+    dets = output[output[:, 4] >= CONF_THRESHOLD]
+    total_dets += len(dets)
+
+    cur_centroids = []
+    for det in dets:
+        x1, y1 = int(det[0] * sx), int(det[1] * sy)
+        x2, y2 = int(det[2] * sx), int(det[3] * sy)
+        x1, x2 = max(0, x1), min(width, x2)
+        y1, y2 = max(0, y1), min(height, y2)
+        cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
+        cur_centroids.append((cx, cy))
+
+        # Speed = displacement from the nearest detection in the previous frame.
+        speed = 0.0
+        if prev_centroids:
+            d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
+            if d <= MATCH_DIST:
+                speed = d
+        speed_sum[y1:y2, x1:x2] += speed
+        count[y1:y2, x1:x2] += 1.0
+    prev_centroids = cur_centroids
+
+    overlay, heatmap_color = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)
+    cv2.putText(overlay, f"Detections: {len(dets)}", (10, 30),
+                cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 255), 2)
+    writer.write(overlay)
+
+cap.release()
+writer.release()
+
+if heatmap_color is not None:
+    cv2.imwrite("heatmap.jpg", heatmap_color)
+    print("Saved: heatmap.jpg")
+
+print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)
+```
+
+**Device targets:**
+
+- `"CPU"` -- default, works on all Intel platforms.
+- `"GPU"` -- Intel integrated or discrete GPU.
+- `"NPU"` -- Intel NPU (validate with `benchmark_app -d NPU`).
+
+#### Expected Output
+
+![OpenVINO expected output](expected_output_openvino.gif)
+
+### DLStreamer Sample
+
+The pipeline below runs the FP16 YOLO26 detector via `gvadetect`.
+A buffer probe estimates each object's speed from its per-frame displacement
+and overlays a speed-colored heatmap (red = fast, blue = slow) on each frame
+before encoding to `output_dlstreamer.mp4`.
+
+> **Notes on running this sample:**
+>
+> - Use the FP16 IR (`yolo26n_openvino_model/yolo26n.xml`). Class names are
+>   read automatically from the model's embedded `metadata.yaml` by
+>   DLStreamer 2026.0+ -- no external `labels-file` is required.
+> - Export `PYTHONPATH` so the DLStreamer Python module is importable:
+>
+>   ```bash
+>   source /opt/intel/openvino_2026/setupvars.sh
+>   source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
+>   export PYTHONPATH=/opt/intel/dlstreamer/python:\
+>   /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
+>   ```
+
+```python
+import gi
+
+gi.require_version("Gst", "1.0")
+gi.require_version("GstAnalytics", "1.0")
+from gi.repository import Gst, GLib, GstAnalytics
+
+import numpy as np
+
+Gst.init([])
+
+# Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
+import cv2
+
+INPUT_VIDEO = "test_video.mp4"
+HEATMAP_ALPHA = 0.55
+MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
+MAX_SPEED = 20.0    # px/frame that maps to full red
+
+
+def render_speed_heatmap(frame, speed_sum, count, alpha):
+    """Color traffic by average speed: blue = slow, yellow = medium,
+    red = fast. Only regions where vehicles were seen are tinted, so
+    empty background keeps its original color."""
+    avg = np.zeros_like(speed_sum)
+    seen = count > 0
+    avg[seen] = speed_sum[seen] / count[seen]
+    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
+    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
+    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
+    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
+    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
+    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
+    return overlay.astype(np.uint8)
+
+
+# For CPU: change device=GPU to device=CPU.
+# For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
+pipeline_str = (
+    f"filesrc location={INPUT_VIDEO} ! decodebin3 ! "
+    "videoconvert ! video/x-raw,format=BGR ! "
+    "gvadetect model=yolo26n_openvino_model/yolo26n.xml "
+    "device=GPU "
+    "threshold=0.4 ! queue ! "
+    "appsink name=sink emit-signals=false sync=false"
+)
+pipeline = Gst.parse_launch(pipeline_str)
+sink = pipeline.get_by_name("sink")
+pipeline.set_state(Gst.State.PLAYING)
+
+speed_sum = None
+count = None
+prev_centroids = []
+writer = None
+frame_idx = 0
+total_dets = 0
+
+while True:
+    sample = sink.emit("pull-sample")
+    if sample is None:
+        break
+    buf = sample.get_buffer()
+    caps = sample.get_caps().get_structure(0)
+    width = caps.get_value("width")
+    height = caps.get_value("height")
+
+    if speed_sum is None:
+        speed_sum = np.zeros((height, width), dtype=np.float32)
+        count = np.zeros((height, width), dtype=np.float32)
+
+    ok, mapinfo = buf.map(Gst.MapFlags.READ)
+    if not ok:
+        continue
+    frame = np.ndarray((height, width, 3), dtype=np.uint8,
+                       buffer=mapinfo.data).copy()
+    buf.unmap(mapinfo)
+    frame_idx += 1
+
+    rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
+    cur_centroids = []
+    det_count = 0
+    if rmeta is not None:
+        idx = 1
+        while True:
+            ok_od, od = rmeta.get_od_mtd(idx)
+            if not ok_od:
+                break
+            _, x, y, w, h, _ = od.get_location()
+            x1, y1 = max(0, int(x)), max(0, int(y))
+            x2, y2 = min(width, int(x + w)), min(height, int(y + h))
+            cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
+            cur_centroids.append((cx, cy))
+
+            # Speed = displacement from the nearest detection last frame.
+            speed = 0.0
+            if prev_centroids:
+                d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
+                if d <= MATCH_DIST:
+                    speed = d
+            speed_sum[y1:y2, x1:x2] += speed
+            count[y1:y2, x1:x2] += 1.0
+            det_count += 1
+            idx += 1
+    prev_centroids = cur_centroids
+    total_dets += det_count
+
+    overlay = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)
+
+    if writer is None:
+        writer = cv2.VideoWriter(
+            "output_dlstreamer.mp4", cv2.VideoWriter_fourcc(*"mp4v"),
+            30.0, (width, height))
+    writer.write(overlay)
+    print(f"Frame {frame_idx}: detections={det_count}", flush=True)
+
+pipeline.set_state(Gst.State.NULL)
+if writer:
+    writer.release()
+print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)
+```
+
+**Device targets:**
+
+- `device=GPU` -- default in the sample code.
+- `device=CPU` -- change `device=GPU` to `device=CPU`.
+- `device=NPU` -- change `device=GPU` to `device=NPU`; use `batch-size=1` and `nireq=4` for best NPU utilization.
+
+#### Expected Output
+
+![DLStreamer expected output](expected_output_dlstreamer.gif)
+
+---
+
+## License
+
+Licensed under the MIT License. See [LICENSE](LICENSE) for details.
+
+## References
+
+- [YOLO26 Documentation](https://docs.ultralytics.com/models/yolo26/)
+- [Ultralytics Heatmap Guide](https://docs.ultralytics.com/guides/heatmaps/)
+- [OpenCV Color Maps](https://docs.opencv.org/4.x/d3/d50/group__imgproc__colormap.html)
+- [OpenVINO Documentation](https://docs.openvino.ai/)
+- [Intel DLStreamer](https://docs.openedgeplatform.intel.com/2026.0/edge-ai-libraries/dlstreamer/index.html)

export_and_quantize.sh

@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: MIT
+# Copyright (C) Intel Corporation
+#
+# Export a YOLO26 detector to OpenVINO IR for the heatmap-generation use case.
+# Usage: ./export_and_quantize.sh [MODEL_VARIANT] [PRECISION]
+# Example: ./export_and_quantize.sh yolo26n FP16
+
+set -euo pipefail
+
+MODEL_NAME="${1:-yolo26n}"
+PRECISION="${2:-FP16}"
+PRECISION="$(echo "${PRECISION}" | tr '[:lower:]' '[:upper:]')"
+
+if [[ "${PRECISION}" != "FP32" && "${PRECISION}" != "FP16" && "${PRECISION}" != "INT8" ]]; then
+    echo "ERROR: unsupported precision '${PRECISION}'. Choose FP32, FP16, or INT8." >&2
+    exit 1
+fi
+
+echo "--- Installing dependencies ---"
+if [[ "${PRECISION}" == "INT8" ]]; then
+    pip install -qU openvino nncf ultralytics
+else
+    pip install -qU openvino ultralytics
+fi
+
+# Ask for approval before downloading models and sample files
+echo ""
+echo "This script will download:"
+echo "  - Model weights and/or sample files"
+echo ""
+read -p "Continue with downloads? (yes/no): " APPROVAL
+if [[ "${APPROVAL}" != "yes" ]]; then
+    echo "Download cancelled by user."
+    exit 0
+fi
+echo ""
+echo "--- Downloading sample test image ---"
+if [[ ! -f test.jpg ]]; then
+    wget -q -O test.jpg https://ultralytics.com/images/bus.jpg
+    echo "Downloaded: test.jpg"
+else
+    echo "Already present: test.jpg"
+fi
+echo ""
+echo "--- Downloading sample test video ---"
+if [[ ! -f test_video.mp4 ]]; then
+    wget -q -O test_video.mp4 \
+        "https://www.pexels.com/download/video/12399570/"
+    echo "Downloaded: test_video.mp4"
+else
+    echo "Already present: test_video.mp4"
+fi
+
+if [[ "${PRECISION}" == "FP32" ]]; then
+    HALF_FLAG="False"
+    EXPORT_LABEL="FP32"
+else
+    HALF_FLAG="True"
+    EXPORT_LABEL="FP16"
+fi
+
+echo "--- Exporting ${MODEL_NAME} to OpenVINO IR (${EXPORT_LABEL}) ---"
+python3 -c "
+from ultralytics import YOLO
+
+model = YOLO('${MODEL_NAME}.pt')
+model.export(format='openvino', half=${HALF_FLAG}, dynamic=False, imgsz=640)
+print('Export complete: ${MODEL_NAME}_openvino_model/')
+"
+
+if [[ "${PRECISION}" == "INT8" ]]; then
+    echo "--- Quantizing to INT8 with NNCF ---"
+    python3 -c "
+import nncf
+import openvino as ov
+import numpy as np
+import cv2
+
+core = ov.Core()
+model = core.read_model('${MODEL_NAME}_openvino_model/${MODEL_NAME}.xml')
+
+img = cv2.imread('test.jpg')
+img = cv2.resize(img, (640, 640))
+img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
+img = img.transpose(2, 0, 1)[np.newaxis, ...]
+
+def transform_fn(data_item):
+    return img
+
+calibration_dataset = nncf.Dataset(list(range(300)), transform_fn)
+
+quantized = nncf.quantize(
+    model,
+    calibration_dataset,
+    preset=nncf.QuantizationPreset.MIXED,
+    subset_size=300,
+)
+
+ov.save_model(quantized, '${MODEL_NAME}_heatmap_int8.xml')
+print('Quantization complete: ${MODEL_NAME}_heatmap_int8.xml')
+"
+fi
+echo "--- Done ---"