[Feature Request] Support Tencent HunYuan world-model workloads (HunyuanVideo-1.5/WorldPlay, HY-World-2.0, HunyuanWorld-Mirror)

[xiaofei-zheng]

[Feature Request] Support Tencent HunYuan world-model workloads

1. Background

We are profiling and optimizing Tencent's HunYuan world-model family on AMD ROCm GPUs and integrating TraceLens into an automated optimization pipeline (roofline analysis, kernel-agent prioritization, trace comparison). The current TraceLens pipeline works well for LLM inference (vLLM/SGLang), but HunYuan workloads expose multiple gaps that prevent end-to-end analysis or produce degraded/misleading results.

This issue requests TraceLens support for three related model families:

Model	Repo	Workload Type	Key Characteristics
HunyuanVideo-1.5 (base)	Tencent-Hunyuan/HunyuanVideo-1.5	Video generation (T2V/I2V)	8.3B DiT, denoising-step loop, 3D conv + temporal attention + VAE
HY-WorldPlay (WorldPlay)	Tencent-Hunyuan/HY-WorldPlay	Real-time interactive world modeling	Streaming autoregressive video diffusion at 24 FPS, built on HunyuanVideo-1.5 backbone
HY-World-2.0	Tencent-Hunyuan/HY-World-2.0	3D world asset generation	Produces editable meshes / 3DGS via WorldMirror 2.0 feed-forward model
HunyuanWorld-Mirror	Tencent-Hunyuan/HunyuanWorld-Mirror	Multi-view 3D reconstruction	~1.3B feed-forward model, single forward pass per scene (NOT an LLM)

These workloads share a common theme: they are not autoregressive LLM decode workloads, and their trace structure differs fundamentally from the prefill/decode steady-state that TraceLens currently optimizes for.

2. Symptoms Observed

2.1 Steady-state splitting failures (video/diffusion workloads)

When running trace_analyze on HunyuanVideo-1.5 / WorldPlay traces, the steady-state splitter fails because it expects LLM autoregressive decode structure:

• steady_state_chunk_empty / steady_state_chunk_missing
• Final failure: trace_split_no_steady_state

This is closely related to #784 (xDiT/Diffusion workloads) but extends to video diffusion (3D conv, temporal attention, VAE decode) and streaming frame generation (WorldPlay).

2.2 HIP/CUDA Graph attribution degradation

When profiling under torch.compile(mode='reduce-overhead') or CUDA/HIP graph capture (common in served WorldPlay and WorldMirror configs), kernels collapse into hipGraphLaunch / graph replay events. TraceLens then reports:

• 0 identifiable hot kernels in the session-level roofline composite
• "No roofline snapshot" in final reports
• Eager-profile roofline renders correctly, but served (compiled) configs — which represent production performance — cannot be analyzed

Related: #591 (CUDA Graph benchmark), #608 (framework-agnostic graph capture correlation).

2.3 Vendor library kernels fall into other or are skipped

Hot kernels in Hunyuan workloads are predominantly vendor/library binaries with no resolvable source:

• 3D convolutions via MIOpen/CK (igemm_fwd_*, grouped-conv) — categorized as aten::convolution backed by PyTorch Native, skipped or low-efficiency attribution
• GEMMs via hipBLASLt/Tensile — partial roofline but often in 30–60% efficiency band without actionable bound type
• SDPA/flash attention (aten::_scaled_dot_product_flash_attention, attn_fwd) — source not found, skipped by kernel-agent downstream tools

2.4 Wrong workload metadata in output

Executive summaries default to token-oriented metrics (tok/s, TPOT). Hunyuan workloads need:

Workload	Throughput Unit	Steady-State Unit
HunyuanVideo-1.5 (T2V/I2V)	images/sec or frames/sec	Denoising step period
WorldPlay (streaming)	fps	Per-frame generation period
HunyuanWorld-Mirror	recon/sec (1/mean_latency)	Single forward pass
HY-World-2.0	assets/sec	Single forward pass

3. Requirements

Requirement 1 (primary): Multi-modal steady-state splitting

Extend steady-state detection beyond LLM prefill/decode. Support at least:

• Option A (recommended): Generic "split by externally-provided annotation markers" mode. Caller emits record_function ranges (e.g., denoise_step, frame_generate, forward_pass) and TraceLens splits on those markers.
• Option B: Periodicity detection over kernel signatures for repeated denoising-step / per-frame patterns.

Each HunYuan model family has a distinct periodic structure:

• HunyuanVideo-1.5: N denoising steps per video (highly repetitive full DiT forward passes)
• WorldPlay: Per-frame streaming generation with context memory rebuild
• HunyuanWorld-Mirror / HY-World-2.0: Single forward pass (no periodic loop — treat entire steady window as one unit)

Requirement 2: PerfModel coverage for video/3D operators

Extend perf models beyond transformer-decoder + MoE. Priority operators:

• 3D convolutions (VAE encoder/decoder, DPT heads) — Conv3d, ConvTranspose3d, Im2Col/Im3d2Col paths
• Temporal/spatial attention — non-causal full-sequence self-attention over latent tokens (N = H×W/patch², no KV growth)
• Cross-attention to text/image embeddings
• adaLN / modulation operators in DiT blocks
• Feed-forward 3D reconstruction heads — DPT-style dense prediction, Gaussian splatting render paths

Requirement 3: HIP/CUDA Graph kernel attribution

When traces contain graph capture/replay events, TraceLens should:

• Correlate hipGraphLaunch / cudaGraphLaunch back to constituent kernels (or provide a degraded-but-useful attribution mode)
• Support framework-agnostic graph classification (not only vLLM/SGLang patterns — see Refactor graph capture-replay correlation to be framework-agnostic #608)
• Emit a clear cuda_graph_attribution_degraded flag when full attribution is impossible, with guidance to profile eager for roofline snapshot

Requirement 4: Workload-type metadata in output

Explicitly emit in analysis.md / JSON output:

• workload_type (e.g., video_diffusion, streaming_world_model, 3d_reconstruction)
• throughput_unit (e.g., fps, images/sec, recon/sec)
• Avoid forcing downstream parsers to infer workload type from operator mix

4. Acceptance Criteria

1. HunyuanVideo-1.5 T2V trace: trace_analyze completes without trace_split_no_steady_state; steady-state chunks align with denoising-step periods.
2. WorldPlay streaming trace: Per-frame steady-state extraction works (or annotation-based splitting succeeds).
3. HunyuanWorld-Mirror trace: Single-forward-pass workload analyzed end-to-end with workload_type=3d_reconstruction, throughput_unit=recon/sec.
4. HIP Graph traces: Either full kernel attribution under graph capture, or explicit degraded mode with actionable guidance (not silent 0-kernel roofline).
5. PerfModel: 3D conv, temporal attention, and cross-attention operators receive bound type and efficiency metrics (not other/skipped).
6. Output metadata: workload_type and throughput_unit present in report output for all three model families.

5. Related Issues

• [Feature Request] Support xDiT / Diffusion workloads in trace analysis & roofline #784 — xDiT/Diffusion steady-state splitting (overlapping denoising-step requirement)
• Add benchmark for CUDA Graph / HIP Graph trace processing performance #591 — CUDA/HIP Graph trace processing benchmark
• Refactor graph capture-replay correlation to be framework-agnostic #608 — Framework-agnostic graph capture-replay correlation
• Add Triton/torch.compile perf models (parity with ATen perf models) #619 — Triton/torch.compile perf models

6. Reproduction Materials (available on request)

• PyTorch profiler traces (*.pt.trace.json.gz) for HunyuanVideo-1.5 (WorldPlay bench), HunyuanWorld-Mirror (30-scene suite), and HY-World-2.0
• Corresponding trace_analyze failure logs and roofline composite degradation evidence
• Model architecture metadata (model_arch.json with layer mix, precision, operator shapes)
• Annotation-instrumented trace samples (if Option A is chosen)

[kyle-hoffmeyer]

Hi @xiaofei-zheng, thank you for opening this issue. What docker image(s) are you using for these models?

[tsrikris]

Hi @xiaofei-zheng, @lishuoshuo-amd; @Ahmedhasssan-aig and @kyle-hoffmeyer are going to be working on this feature.

The tasks for bringing up a workload was previously shared by @devalshahamd

1. Investigate profiling
2. Trace Splitting
3. Update perf models
4. Sanitize TraceLens Agent
5. Integrate all steps into Hyperloom, clear out existing pathway

Timeline will be shared once the scope of the effort becomes clear

[xiaofei-zheng]

Thanks @tsrikris for picking this up, and welcome @Ahmedhasssan-aig / @kyle-hoffmeyer.

Docker image

@kyle-hoffmeyer — we did not build a dedicated image for these workloads. We run
them directly in the SGLang image:

primussafe/sglang:v0.5.12-rocm720-mi35x-profilerfix

(ROCm 7.2.0, MI35x, with the profiler fix.) A few Python packages need to be
downgraded for the HunYuan repos to run; the exact versions are reported by the
runtime itself when you launch the models, so it is straightforward to follow the
prompts. We can share our resolved package set if you prefer a pinned starting point.

Two requirements not covered by the 5-step plan

The 5-step plan maps cleanly onto Requirement 1 (trace splitting) and Requirement 2
(perf models). Two other items from the issue are not explicitly represented, and
both matter for our end-to-end pipeline:

Requirement 3 — HIP/CUDA Graph kernel attribution. Our served configurations use
torch.compile reduce-overhead / HIP graph capture, and those are the configurations
that represent production performance. Under graph capture, kernels collapse into
hipGraphLaunch events, and the session-level roofline composite ends up with 0
identifiable hot kernels and reports "No roofline snapshot" without any error — so a
failed attribution is indistinguishable from a genuinely empty result. Eager profiles
render correctly, but eager is not the configuration we need to optimize. Full
attribution would be ideal; a documented degraded mode with an explicit
cuda_graph_attribution_degraded flag would already be enough for our downstream
tooling to handle the case correctly. This overlaps with #608 (framework-agnostic
capture-replay correlation).

Requirement 4 — workload_type / throughput_unit in output. Small addition, but
it lets our automation stop inferring workload type from the operator mix. Today the
executive summary defaults to tok/s and TPOT, which are not meaningful for video
diffusion or 3D reconstruction.

Repro materials

Available whenever you want them:

• PyTorch profiler traces (*.pt.trace.json.gz) for HunyuanVideo-1.5 (WorldPlay
  bench), HunyuanWorld-Mirror (30-scene suite), and HY-World-2.0
• Matching trace_analyze failure logs (trace_split_no_steady_state) and the
  roofline degradation evidence from the graph-captured runs
• model_arch.json per model

We can also produce annotation-instrumented traces with record_function ranges
(denoise_step / frame_generate / forward_pass) if you go with Option A for
steady-state splitting. That is the cheapest path for us to unblock on our side, so
an early signal on Option A vs Option B would help us prepare the right traces.