Hugging Face's logo

ussoewwin
/
SeedVR2-VAE-TenorRT-Engine-for-Blackwell

Video-to-Video
SeedVR
TensorRT
seedvr2
vae
rtxplan
blackwell
rtx-5090
rtx-5080
rtx-5070
rtx-5060
rtx-5050
video-upscaling
image-to-image
comfyui
Model card Files
xet
 Community

SeedVR2 VAE TensorRT Engines for NVIDIA Blackwell

High-performance, pre-compiled NVIDIA TensorRT engine plans (.rtxplan) built for the official SeedVR2 VAE (seedvr2_ema_vae_fp16), specifically compiled and optimized for the NVIDIA Blackwell GPU architecture (RTX 5090, RTX 5080, RTX 5070, RTX 5060, RTX 5050, and Blackwell workstation / data center GPUs, Compute Capability sm_100 / sm_120).

🌟 Overview

SeedVR2 is a state-of-the-art diffusion-based video restoration and super-resolution model developed by ByteDance. In video upscaling workflows, spatial-temporal VAE encoding (RGB frames to latents) and decoding (latents back to RGB frames) represent primary computational and memory bottlenecks.

This repository provides pre-compiled TensorRT execution engine plans (.rtxplan) covering both VAE Decoding and VAE Encoding, delivering full end-to-end acceleration on NVIDIA Blackwell hardware:

  • Target Architecture: NVIDIA Blackwell (sm_100 / sm_120, RTX 50-Series: RTX 5090, RTX 5080, RTX 5070, RTX 5060, RTX 5050).
  • Dual-Phase VAE Coverage: Dedicated engines for VAE Encoding (Phase 1) and VAE Decoding (Phase 3).
  • Dual Spatial Tile Tiers:
    • Tile 1024 (1024×1024): High-throughput single-image (1f) spatial processing; enables ultra-high resolution processing in a single pass without tiling seams.
    • Tile 512 (512×512): Standard spatial fidelity for temporal sequences (21f, 45f); delivers maximum throughput and seamless boundary blending on Blackwell architecture.
  • Core Temporal Sequences ($4n+1$): High-speed single-image (1f), standard batch (21f), and extended sequence (45f).
  • Base Checkpoint: Comfy-Org/SeedVR2 (seedvr2_ema_vae_fp16) / ByteDance SeedVR2.
  • ComfyUI Custom Node: ComfyUI-SeedVR2-VideoUpscaler-with-TensorRT

📦 Available TensorRT Engine Plans (6 Engines)

📊 Engine Matrix Overview

Module Spatial Tile Temporal Frames ($4n+1$) Count Engine Size Naming Format Primary Use Case
VAE Decoder 1024×1024 1f 1 ~309.8 MB vae_decoder_tile_1024_1f.rtxplan Single-frame / image super-resolution, seam-free 1024px decode
VAE Decoder 512×512 21f, 45f 2 ~308.3 MB each vae_decoder_tile_512_21f.rtxplan
vae_decoder_tile_512_45f.rtxplan		 Standard (21f) & extended (45f) high-throughput video decoding
VAE Encoder 1024×1024 1f 1 ~213.6 MB vae_encoder_1f_tile1024.rtxplan Single-frame / image super-resolution, seam-free 1024px encode
VAE Encoder 512×512 21f, 45f 2 ~218.7 MB / ~231.3 MB vae_encoder_21f_tile512.rtxplan
vae_encoder_45f_tile512.rtxplan		 Standard (21f) & extended (45f) high-throughput video encoding
🔍 Click to expand complete list of all 6 individual engine files

1. VAE Decoder Engines 

vae_decoder_tile_1024_1f.rtxplan
vae_decoder_tile_512_21f.rtxplan
vae_decoder_tile_512_45f.rtxplan

2. VAE Encoder Engines 

vae_encoder_1f_tile1024.rtxplan
vae_encoder_21f_tile512.rtxplan
vae_encoder_45f_tile512.rtxplan

Note on 4n+1 Temporal Structure: SeedVR2 utilizes a 3D causal temporal downsampling VAE architecture. The temporal latent dimension corresponds to $(T - 1) / 4 + 1$. Consequently, video frame sequences are compiled and processed in $4n + 1$ frame blocks. Single image inference uses the dedicated 1f engine.

🛠️ Architectural & Performance Advantages

  1. Native Blackwell Kernel Tuning: Built and tuned with the TensorRT Blackwell compilation pipeline, fully utilizing Blackwell 5th-Generation Tensor Cores, enhanced memory bandwidth, and fused convolution-activation operators across the entire RTX 50-Series lineup (RTX 5090, RTX 5080, RTX 5070, RTX 5060, RTX 5050).
  2. Native Tile 1024 Support for Single-Frame Acceleration: Single-image upscaling operations (1f) execute within a 1024×1024 spatial window in a single TensorRT pass, completely avoiding spatial tiling boundaries and seam artifacts while maximizing memory bandwidth utilization.
  3. Optimized 21f & 45f Video Engine Suite: Provides the standard 21f batch size for high-speed, memory-efficient video generation alongside the 45f extended batch size for maximal temporal coherence across continuous scenes.
  4. Decoupled Dual-Phase VAE Configuration: Enables independent selection of engine frame sizes and tile configurations for encoding and decoding.
  5. Dynamic 1-Shot Pad & Crop Execution: Short or non-matching batch lengths are dynamically padded to the nearest valid $4n+1$ sequence, executed in a single high-throughput TensorRT pass, and cropped back with pixel precision without falling back to slow FP16 CPU/PyTorch loops.
  6. Studio-Grade Memory & Execution Safety:
    • ExecutionContext Lock & Stream Synchronization: Dedicated mutex locks and explicit stream synchronization eliminate race conditions and memory corruption (black tiles) during concurrent or chained executions.
    • Deterministic Tri-Partite Memory Cleanup: Immediate release of intermediate tensors (del, gc.collect(), torch.cuda.empty_cache(), comfy.model_management.soft_empty_cache) prevents progressive VRAM fragmentation across multi-batch sequences.

🚀 Usage in ComfyUI

These TensorRT engine plans (.rtxplan) are directly loaded and executed via the dedicated custom node:

1. Installation

Clone the custom node repository into your ComfyUI custom_nodes/ directory:

cd ComfyUI/custom_nodes
git clone https://github.com/ussoewwin/ComfyUI-SeedVR2-VideoUpscaler-with-TensorRT.git

2. Engine Placement

Place the downloaded .rtxplan engine files into either of the following supported directories:

ComfyUI/custom_nodes/ComfyUI-SeedVR2-VideoUpscaler-with-TensorRT/tensorrt_backend/artifacts/

or

ComfyUI/models/tensorrt/seedvr2/

Upon launching ComfyUI, all detected .rtxplan files will automatically populate the engine_frames and engine_tile dropdown lists of the respective loader nodes:

  • vae_decoder_tile_*_*f.rtxplan populates SeedVR2 Load TensorRT VAE Decoder
  • vae_encoder_*f_tile*.rtxplan populates SeedVR2 Load TensorRT VAE Encoder

3. Node Integration

Connect both loader nodes directly to the SeedVR2 Video Upscaler node:

  • Connect the SEEDVR2_VAE output of SeedVR2 Load TensorRT VAE Encoder to the vae_encode input.
  • Connect the SEEDVR2_VAE output of SeedVR2 Load TensorRT VAE Decoder to the vae_decode input.
+------------------------------------+
| SeedVR2 Load TensorRT VAE Encoder  |
| [engine_frames: 21 / 45 / 1]       |--(SEEDVR2_VAE)---+
| [engine_tile: 512 / 1024]          |                  |
+------------------------------------+                  |
                                                        v [vae_encode]
                                         +----------------------------+
                                         |   SeedVR2 Video Upscaler   |
                                         +----------------------------+
                                                        ^ [vae_decode]
+------------------------------------+                  |
| SeedVR2 Load TensorRT VAE Decoder  |                  |
| [engine_frames: 21 / 45 / 1]       |--(SEEDVR2_VAE)---+
| [engine_tile: 512 / 1024]          |
+------------------------------------+

4. Workflow Examples

Complete Video Upscaler Workflow (TensorRT VAE & Quantized Models)

Workflow JSON: example_workflows/SeedVR2_tensorrt_decode.json

Usage Example - Full Workflow

TensorRT VAE Decoder & Encoder Node Integration

Connect the SEEDVR2_VAE outputs from SeedVR2 Load TensorRT VAE Decoder and SeedVR2 Load TensorRT VAE Encoder directly into the SeedVR2 Video Upscaler node:

Usage Example - TensorRT VAE Nodes

🔧 Building Custom Frame Engines on Demand

The custom node repository also includes a dedicated SeedVR2 Build TensorRT VAE Engines node to compile custom frame sizes directly inside ComfyUI:

  • Supported Frames: Any custom frame count following the $4n+1$ format (e.g. 5f, 21f, 45f, 65f, 81f).
  • Supported Tile Sizes: 256, 512 (recommended for standard sequences), and 1024 (optimal for single-frame inference).
  • Supported Targets: decoder, encoder, or both.

Built engines are saved directly into tensorrt_backend/artifacts/ and become immediately selectable upon restarting ComfyUI.

📜 Credits & References

Downloads last month
-
Inference Providers NEW
Video-to-Video
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support