Test CI with comment-only change#1
Draft
Phil-amd wants to merge 44 commits into
Draft
Conversation
Add a detect-changes job (dorny/paths-filter) to the Fly DSL test workflow. Docs-only / non-code changes skip the test and multi-gpu jobs at the job level, so no GPU runner is used. workflow_dispatch forces a full run, and a detection failure falls back to running the tests.
…#803) `_collect_dependency_sources` used a `visited` set to guard against cycles, but the JitFunction branch went through `val._ensure_cache_manager()` -> `_jit_function_cache_key` -> `_collect_dependency_sources(<fresh visited>)`, bypassing the caller's `visited`. During that computation `manager_key` is still `None`, so there was no reentrancy guard: any jit self-reference, jit<->jit mutual reference, or a `.launch` attribute name colliding with a same-named top-level `@flyc.jit` would recurse until `RecursionError`. Guard reentrancy with a thread-local set of the `id(func)` whose keys are currently being computed on this thread: `_jit_function_cache_key` adds its func on entry and discards it in `finally`, and the dependency traversal, when it meets a JitFunction already being keyed (a cycle), emits a stable `jit-recursive:<name>:<label>` placeholder instead of reentering. Each cycle member's own source still enters the key via its own `_get_func_source`, so the placeholder only breaks the cycle. The stack is thread-local (not a per-instance flag) so that a JitFunction being keyed on one thread is not mistaken for a cycle by a concurrent compilation on another thread — that would otherwise substitute the placeholder for a real dependency and make keys timing-dependent. Acyclic code never hits the placeholder branch, so existing cache keys (and on-disk cache) are unchanged. Add regression tests for top-level `launch` attribute collision, jit self-reference, jit<->jit mutual reference, thread-local isolation of the guard, and no-leak cleanup of the in-progress stack. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ROCm#780) * [Kernel] Add compile_mxfp6_gemm: MXFP6×MXFP4 preshuffle GEMM (gfx950) compile_mxfp4_gemm and compile_mxfp6_gemm share a single private implementation _compile_mxfp_blockscale_gemm(a_dtype='fp4'|'fp6'). use_async_copy (default True) selects buffer_load_lds vs coop gmem->VGPR->LDS. A-operand (fp6): per_1x32_f6_quant / pack_fp6_e2m3 produce tight-packed fp6 (24 B per K=32 chunk) + 8 B zero-pad. read_a takes the first 6 of 8 DWORDs from LDS → i32[6] for MFMA_Scale(Float6E2M3FN, Float4E2M1FN). Tile selection: MXFP6_TUNED_CONFIGS + _pick_mxfp6_tiles heuristic. compile_mxfp6_gemm(M_hint, N, K) auto-selects tiles when not provided. Performance on gfx950, (M,N,K)=(x,8192,8192), TFLOPS: fp4-sync fp4-dma fp6-sync fp6-dma M=32: 185 220 90 204 M=64: 352 394 181 367 M=128: 632 699 335 659 M=256: 521 1333 671 856 M=512: 1459 1658 948 1267 M=1024: 2916 2737 1708 1643 M=2048: 3612 3591 2337 2661 Tests: test_mfma_a6w4_preshuffle (10 cases, gfx950 only). Signed-off-by: Shreyas Atre <satre@amd.com> Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * [Kernel] Simplify mxfp GEMM: fold wrappers, tighten fp6 read_a Collapse _compile_mxfp_blockscale_gemm + the fp4/fp6 pass-through wrappers into a single public compile_mxfp4_gemm(a_dtype='fp4'|'fp6'); compile_mxfp6_gemm is now a thin **kw delegator. Drop redundant _raw() calls in the fp6 read_a repack (Vec.from_elements already coerces elements) and use a comprehension. Net -79 LOC. Verified no codegen change: final gfx950 ISA and LLVM IR are byte-for-byte identical for both fp4 and fp6 (tile 64x128x128, M/N/K=64/8192/8192). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Shreyas Atre <satre@amd.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Felix Li <felix.li@amd.com>
Replace direct @flyc.jit __call__ dispatch at PA decode launch call sites with _run_compiled(), which caches a CompiledFunction on first call and skips the per-call sig.bind/cache-key overhead on repeat decode steps. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Add the gfx1250 MX-scaled WMMA MMA atom (E8M0 block scale, block-16/32, 16x16x128 + 32x16x128 fp4) and the N-D TDM (Tensor Data Mover) whole-tile Global<->LDS copy atom (1-5D, descriptor as atom state), plus the getCopyRank whole-tile lowering path. Also adds the int4 (iu4) plain WMMA config. FileCheck + l0 coverage; device tests validated on gfx1250.
…monolith module (ROCm#814) * [kernels]: unify arch helper naming and dedup arithmetic helpers Structure-only cleanup; generated code unchanged: - unify get_hip_arch alias to get_rocm_arch across 18 modules - fp8_gemm_utils: ceildiv now delegates to common.utils.cdiv; drop the builtin-shadowing divmod (callers use the builtin) - rename the preshuffle-specific crd2idx in mma to preshuffle_crd2idx to disambiguate it from common.layout_utils.crd2idx * [kernels]: split moe_gemm_2stage monolith into a package Structure-only split; generated code unchanged. The 3586-line moe_gemm_2stage.py becomes a package with the three compile functions in separate modules; public API is re-exported from __init__ so all importers keep working: moe_gemm_2stage/{_gemm1,_gemm2,_reduction}.py Function bodies are byte-identical to the pre-split module (verified), so the trace-based DSL emits identical IR. * [kernels]: split mixed_moe and moe_blockscale monoliths into packages Structure-only split; generated code unchanged, same pattern as the moe_gemm_2stage split: mixed_moe_gemm_2stage/{_common,_gemm1,_gemm2}.py moe_blockscale_2stage/{_gemm1,_gemm2,_reduction}.py Public API re-exported from each package __init__; all function bodies are byte-identical to the pre-split modules (verified). * [kernels]: extract shared gfx950 dualwave_swp helpers into attention Move the DUALWAVE_SWP flash-attention primitives (ds_read transpose, s_waitcnt encoding, exec-mask read, LDS alias scopes, split-K workspace sizing) out of flash_attn_gfx950 into kernels/attention/dualwave_swp_common.py so the bf16/fp16 and fp8 gfx950 kernels share one copy. flash_attn_gfx950 stays a single flat module and re-exports dualwave_splitk_workspace_elems for its public API. * [kernels]: collapse flash_attn_fp8_gfx950 to a flat module on shared attention Add the fp8-only ds_read_b64_tr_b8 primitive to kernels/attention/dualwave_swp_common.py and point flash_attn_fp8_gfx950 at the shared helpers. The kernel stays a single flat module (no per-kernel package) and re-exports dualwave_splitk_workspace_elems for its public API. * [kernels]: restore divmod helper for DSL Int32 operands The earlier cleanup dropped fp8_gemm_utils.divmod assuming callers could use the builtin, but that is wrong for DSL scalar types: kernels call divmod() on Int32 values and the builtin rejects them, so restore it as (a // b, a % b) using the overloaded DSL operators and re-add it to the fp8/fp4 4wave/8wave imports. * [kernels]: share MoE vector bitcast helpers and drop submodule underscores Hoist the pure vector bitcast repack helpers into kernels/moe/moe_common.py so the three 2-stage MoE packages share one copy, and rename their private submodules (_gemm1/_gemm2/_reduction/_common) to drop the underscore. Public __init__ facades unchanged; generated IR is byte-identical.
…Cm#840) importorskip only covers a missing aiter module; the top-level symbol import still hard-fails when the attention API drifts (e.g. hk_mla_decode_fwd renamed), raising ImportError during collection and taking down the entire test session (exit 2). Wrap it in try/except and skip the module instead so the mismatch degrades to a clean per-module skip.
…OCm#842) run_benchmark.sh invokes each kernel test as a plain script (python3 tests/kernels/test_*.py), not through pytest. When such a file reaches a pytest.skip()/pytest.importorskip() at runtime -- e.g. a single-GPU runner hitting the layernorm multi-GPU test ("needs >=2 GPUs"), an arch guard, or the mla_decode test hitting an aiter API drift (hk_mla_decode_fwd removed) -- the skip raises an uncaught Skipped exception, the script exits non-zero, and the op was counted as "failed", turning the whole benchmark job (and thus CI) red. The pytest suite already tolerates these (they show up as skips), so only the benchmark step was going red; this made main and several unrelated PRs fail for purely environmental reasons. Add a _fail_or_skip helper that downgrades a non-zero benchmark exit to "skipped" when its log contains a "Skipped:" marker (mirroring the existing grep -q "Skipped:" handling already used for hgemm/fp8), and route the plain failure branches (softmax, layernorm, rmsnorm, flash_attn, mla, gemm, gemm_async, hgemm, moe, moe_w4a16, RDNA WMMA) through it. Genuine failures still count and dump their log tail. Report a Skipped count in the summary; the job still fails iff FAIL_COUNT > 0. The arch-specific blocks (fp8_8wave_rowscale, gemm_fp4, moe_fp4, moe_a8w4) keep their existing bespoke skip detection.
) Move get_llvm_ptr / atomic_add / pointer / global-load helpers into a single kernels/common/mem_ops.py; kernels_common and utils re-export them, so existing imports are unchanged. Add global_store and buffer_atomic_add as single entry points, an __all__ documenting the preferred vs legacy names, and an ir_value unwrap in global_store to match atomic_add. Route the hgemm_splitk semaphore through atomic_add, and every raw_ptr_buffer_atomic_fadd site (MoE gemm/blockscale/mixed, conv3d and conv3d_fp8, gfx1250 common) through buffer_atomic_add. Codegen-neutral: final ISA is byte-identical before/after for every touched kernel. gfx942 tests pass (hgemm_splitk, rmsnorm, moe gemm/ blockscale); gfx950 tests pass (moe gemm suite, conv3d, conv3d_fp8, split-k flash). Co-authored-by: Felix Li <felix.li@amd.com>
Keep a single docs index and merge the Kernel Authoring description. Signed-off-by: Fujun Han <fujun.han@iluvatar.com>
* feat: add type info to dsl_loc_tracing Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> * fix: propagate types in dsl_wrap_result and coerce_int_tuple_args Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> --------- Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> Co-authored-by: Felix Li <felix.li@amd.com>
- detect-changes: use dorny/paths-filter predicate-quantifier 'every' so a file counts as code only when it is not a docs/markdown/image path. The default 'some' quantifier matched every file via '**', so the skip never triggered. - test-skip: shim job that reuses the 'test' matrix name to report the required per-runner checks as passed on ubuntu when the GPU tests are skipped, so branch protection isn't left waiting on missing statuses.
Move mma/ to kernels/common/mma/ and rename pipeline_utils.py to pipeline_utils_gfx1250.py; update all importers. No functional change.
* [CI] Make benchmark baselines fork-safe and bounded Fetch the exact pull request base commit from the base repository instead of using the head fork's origin/main. Run that baseline once, fetch tags from the canonical repository, and bound the baseline step to 45 minutes. Apply the same source selection to the multi-GPU comparison. Allow concurrency cancellation to stop the real GPU jobs while retaining always-run cleanup steps. Emit annotations when optional comparisons cannot run. Refs ROCm#856. Signed-off-by: Jin Pan <47354855+jhinpan@users.noreply.github.com> * [CI] Quarantine the unhealthy Navi runner GPU Recent linux-flydsl-navi-2 jobs hang whenever free-VRAM selection chooses GPU 1, while GPU 2 and GPU 3 complete the same tests in about one minute. Keep this runner on proven healthy GPU 3 until the runner is repaired. Also cap the single-GPU test step at 60 minutes so a bad device cannot occupy a runner until GitHub's default six-hour job limit. Refs ROCm#858. Signed-off-by: Jin Pan <47354855+jhinpan@users.noreply.github.com> --------- Signed-off-by: Jin Pan <47354855+jhinpan@users.noreply.github.com>
Co-authored-by: solin <bingzhou@amd.com>
Added project status and compatibility notice to CONTRIBUTING.md. Signed-off-by: James Tang <james.tang@amd.com>
…OCm#855) * [Perf] Add staged dweight reduction for RMSNorm backward * [Perf] Vectorize persistent RMSNorm backward * [Perf] Reduce RMSNorm backward launch overhead * [Refactor] Share RMSNorm backward vector/scalar flow * [Refactor] Reuse RMSNorm backward compiled launcher cache
* [Feat] Export shuffle xor/up/down/idx ops * update
* Fix Vector list()/iteration hang (ROCm#793) Vector defined __getitem__ but neither __iter__ nor __len__, so list(Vec(buffer_load(vec_width>1))) fell back to CPython's legacy sequence protocol (v[0], v[1], ...). That protocol only stops on IndexError, and the integer branch of __getitem__ never raised one, so tracing spun forever emitting vector.extract ops and never finalized any IR. - Add __len__ (returns numel) and __iter__ (yields the numel lanes). - Bounds-check the integer branch of __getitem__: normalize negative indices and raise IndexError when out of range. - Add tests/unit/test_vector_iteration.py regression coverage. * Apply black formatting to test_vector_iteration.py Add blank line after the docstring in test_negative_index to satisfy the repo's black style check (pre-checks CI). * Address review: real tuple-unpacking test + original index in error - Split the iteration test: keep list-comprehension coverage as test_vector_iteration and add test_vector_unpacking exercising the tuple-unpacking path (a, b, c = v). - Report the caller-supplied index in the out-of-range IndexError instead of the negative-normalized value (e.g. v[-5] now says -5, not -1). --------- Co-authored-by: adityas-amd <adityas-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
Made with Cursor