MoRI v1.2.2

MoRI v1.2.2

Feature-heavy release since v1.2.1: the new CCO (Collective Communication Object) module lands, EPv2 dispatch/combine gains a FlyDSL intranode path and ships as an installable package, a new fabric (UALink scale-up) IO backend is added, plus broad EP/IO/UMBP performance work, RDMA correctness fixes, and CI hardening.

Highlights

• CCO (Collective Communication Object) module — mori's self-contained collective/communication layer (host + device API, LSA intra-node, GDA scale-out) (#449).
• EPv2 dispatch/combine: FlyDSL intranode MoE dispatch/combine (preview) (#448), now distributed as an installable package (#482), with combine gather inner-unroll + tuned geometry by default and gfx1250 EP4 retune (#484), and gfx1250 + cross-node fabric support (#466).
• IO: fabric (UALink scale-up) transfer backend (#467).
• UMBP: cache remote fetches into local DRAM (dual-scheme) (#442), lower master RPC latency under load + kv-event bench (#440), and a distributed-mode flush API with hot-path optimizations (#419).

Features

• feat(EPv2): combine gather inner-unroll + tuned geometry by default + gfx1250 EP4 retune (#484)
• feat(ep): distribute dispatch_combine_v2 as an installable package (#482)
• feat(io): add fabric (UALink scale-up) transfer backend (#467)
• feat(EPv2): gfx1250 + cross-node fabric support (#466)
• feat(ep): add blockwise-FP8 combine to the AsyncLL (low-latency) path (#465)
• feat(ep): add blockwise FP4 (E2M1) combine transport (#461)
• feat(EPv2): [preview] FlyDSL intranode MoE dispatch/combine (#448)
• feat(cco): add Collective Communication Object (CCO) module (#449)
• feat(umbp): cache remote fetches into local DRAM (dual-scheme) (#442)
• feat(rdma): add dmabuf-first MR registration with MORI_DISABLE_DMABUF_REG (#355)
• feat(ccl): add param-contiguous SDMA allgather
• feat(shmem): bind ranks to GPU-local CPUs by default (#401)
• feat(sdma): use KFD recommended engine mask for SDMA engine selection (#429)
• feat(io): enable worker chunking for GPU single-NIC sessions (#436)
• feat(umbp): benchmark / distributed-mode flush API / hot-path optimization (#419)

Performance

• perf(rdma): default to ibv_reg_mr, opt into dmabuf via MORI_ENABLE_DMABUF_REG (#453)
• perf(io): eliminate SQ full spin with futex admission (#439)
• perf(umbp): reduce master RPC latency under load, add kv-event bench (#440)
• perf(ep): improve intranode dispatch kernel load balancing and eliminate repeated loads (#412)
• tune(cco): re-tune MI355X (gfx950) EP8 schedule for vec4 combine (#472)

Bug Fixes

• fix(cco): guard hipMemAllocationTypeUncached for older ROCm (#480)
• fix(io/xgmi): honor sub-region base offset in IPC remapping (#416, fixes #415)
• fix(jit): invalidate kernel cache on #included source changes (#469)
• fix(cco): serialize handle import under the VMM process lock (#476)
• fix(cco): deterministic release of per-op symmetric memory (#474)
• fix(cco): fix HIP VMM memory leak and crash (#455)
• fix(umbp): fix umbp tests error (#470)
• fix(bnxt): only define true/false for C in bnxt_re_hsi.h (#457)
• fix(ionic): cap max_send_wr to 32767 (#451)
• fix(mlx5): fix collapsed-CQ bandwidth spikes (#435)
• fix(rdma): harden RDMA control-plane resilience (#432)
• fix(io): set QP max_inline_data for notification SENDs (#424)

CI / Build / Tools

• ci(nightly): install wheels from artifact, stop bloating gh-pages with wheels (#483)
• tools: add NIC firmware version checks + mori check CI step (#471)
• ci(cco): move runner to MI355X-AINIC (#477)
• ci: prevent GPU VRAM leaks via tini reaping + graceful container teardown (#473)
• ci: recover podman state before internode build on MI355X_AINIC (#462)
• ci(cco): enable cco python test (#456)
• feat(cco): update mori-cco dockerfile (#458)
• ci: move ci runner 325 -> 108 (#447)
• ci: migrate bnxt 300x CI to 325 platform (#438)

Full changelog: v1.2.1...v1.2.2

MoRI v1.2.1

MoRI v1.2.1

Maintenance and feature release on top of v1.2.0, with cached routing replay for training, EP/UMBP/IO performance work, broader hardware/topology robustness, rocm7.14 compatibility support and a number of RDMA correctness fixes.

Features

• feat(ep): add AccumNum=9 vec8 intranode combine for shared-expert fusion (#396)
• feat(umbp): pluggable master eviction strategy (#399)
• feat(ep): optimize warp id allocation under dispatch LL kernel (#408)
• feat(umbp): improve BatchRoutePut placement with select-algo and node-affinity knobs (#382)
• feat(io): add rail affinity for RDMA endpoint connection (#370)
• feat(rdma): load libibverbs at runtime via dlopen instead of linking (#373)
• feat(ep): add cached routing replay for training dispatch/combine (#335)

Performance

• perf(umbp): optimize PoolClient BatchPut/BatchGet (#403)
• perf(umbp): DRAM parallel copy (#392)
• feat(io/rdma): reuse thread-local scratch buffers on batch hot path (#389)

Bug Fixes

• fix(io): RDMA QP load imbalance (#420)
• fix(io): RDMA chunking for oversized multi-SGE WRs (#409)
• fix(shmem): public RDMA quiet returned before the caller's WQEs completed (#404)
• fix(jit): reuse the already-loaded libamdhip64 to avoid HIP 709 on framework streams (#402)
• fix: detect same physical node by MORI_NODE_ID/boot_id, with hostname fallback (#387)
• fix(topo): recognize VGA (0x0300) and 3D (0x0302) PCI classes as GPU (#381)
• fix(topology): avoid infinite loop on PCI bridge with subordinate bus 0xff (#378)
• fix(umbp): qp-per-transfer default value & add logs (#384)
• fix: mori bug on ROCm 7.1.4 (#414)
• fix: tool bug when tested in 7.1.4 docker (#418)

Refactor

• refactor(core,application): include hygiene + host/device separation (#405)
• refactor(bnxt): collapsed-CQ drain, drop outstandingWqe, fix per-lane PSN (#395)

Tests

• test(umbp): use kernel-assigned ports in distributed tests to fix shared-CI flakiness (#417)
• test(umbp): align distributed tests with current API and regroup CMake (#380)
• test(io): add --target-dev-offset to MORI-IO benchmark (#379)

CI

• ci: add CodeQL SAST workflow and fix all high-severity findings (#385)

Tools

• tools: add Broadcom (bnxt) support to env_check.sh and env_setup.sh (#386)

Docs

• docs: add mori sglang news (#423)

Full changelog: v1.2.0...v1.2.1

v1.2.0

Highlights

• MORI-UMBP (new): Unified Memory/Bandwidth Pool — tiered memory/SSD storage with distributed key-value access for scalable memory management; IUMBPClient interface for a dual-scheme architecture (#257). Now built by default (BUILD_UMBP=ON); the SPDK/SSD tier is opt-in via BUILD_UMBP_SPDK=ON.
• MORI-CCL (new): SDMA-based collective communication framework — AllGather, All2All, AllReduce with sync/async paths, pybind + Python wrappers, examples & benchmarks (#235); plus C++ AllGatherIntoTensor over SDMA (#303).
• MORI-IO: single-transfer chunking with adaptive multi-NIC striping (#351); new sync transfer wait API (#341).
• MORI-EP: FP8 blockwise quantization for IntraNode combine (#311, #315); small-token dispatch perf optimization (#333).
• MORI-SHMEM: single-process multi-thread (SPMT) support via MORI_MULTITHREAD_SUPPORT, currently for JAX/XLA (#308).
• Ionic/CCQE: runtime CCQE capability detection — one binary supports both CCQE and non-CCQE deployments (#307).
• Packaging: pre-built wheels on PyPI (amd-mori) and nightly builds (amd-mori-nightly); dynamic versioning via setuptools_scm; mori CLI for env helpers (#289, #297).

Features

• feat(umbp): enable UMBP by default; add BUILD_UMBP_SPDK opt-in for the SPDK tier (#372)
• feat(ccl): SDMA collective framework + AllGather/All2All/AllReduce (#235)
• feat(ccl): C++ AllGatherIntoTensor over SDMA (#303)
• feat(io): single-transfer chunking + adaptive multi-NIC striping (#351)
• feat(io): add sync transfer wait API (#341)
• feat(EP): FP8 blockwise quantization for IntraNode combine (#311)
• feat(EP): decouple fp8_blockwise combine scale_dim from user config (#315)
• feat(shmem): MORI_MULTITHREAD_SUPPORT for SPMT (JAX/XLA) (#308)
• feat(cli): mori console entry point for env helper scripts (#297)
• feat(jit): enable hip kernel debug info (#318)

Performance

• perf(ep): optimize dispatch intranode kernel for small-token cases (#333)

Bug Fixes

• fix(ep): guard dispatch kernels against out-of-range expert id — fixes HSA page fault on wide-ep=16 (#363)
• fix(ep): mlx5 collapsed CQ + dedicated dispatch send buffer — fixes internode-v1 silent corruption (#366)
• fix(ep): safe D2H read in pybind getters & tune test block_num (#314)
• fix(io): make io worker cpu pinning opt-in via MORI_CORE_OFFSET (#364)
• fix(io): bind worker threads within allowed cpuset (#358)
• fix(io): XGMI fallback (#316)
• fix(mlx5): program RoCE QoS (MORI_RDMA_TC/SL) + selectable UDP sport on DEVX QPs (#362)
• fix(rdma): bnxt PollCq/QP bugs + ibverbs MORI_IB_PATH_MTU override (#298)
• fix(memory): skip RDMA MR registration when no RDMA peers exist (#344)
• fix(shmem): missing include in anvil.cpp + SIOF singleton fix (#339)
• fix(ccl): fail fast when MORI_ENABLE_SDMA is not enabled (#302)
• fix: compiled spdlog with hidden visibility to prevent symbol collision (#332)
• fix: XLA build, shmem_api includes separation (#295)
• fix(env): reorder DSCP mapping before scheduling in env_setup.sh (#320)
• fix(env): cache MORI_ENABLE_SDMA / MORI_DISABLE_P2P at Context construction (#317)
• fix(env): env_check.sh/env_setup.sh errors in docker env (#301)
• fix(ep): warn when AsyncLL runs without SDMA

Refactor

• refactor(jit): detect CCQE support at runtime instead of compile time (#307)
• refactor(jit): silence Ionic CCQE probe logs and clean up caching (#347)
• refactor(ccl): migrate collective kernels from AOT to JIT compilation (#319)
• refactor: decouple device compilation and public headers from host-only deps (#296)
• refactor: update FlyDSL extern integration (#299)

CI / Build

• ci: add nightly wheel build workflow + PyPI publish (#289, #327)
• ci: preserve nightly/ dir when deploying docs to gh-pages (#324)
• ci: add UMBP python tests + intranode/internode coverage to nightly
• chore(ci): use next patch version for nightly dev builds (#334)
• ci: bump ROCm base images from 7.2.1 to 7.2.4 (#361)
• ci: stabilize umbp_client_metrics test (port collision) (#368)

Docs

• docs: PyPI / nightly install instructions + conflict warning (#325, #329)
• docs: list libibverbs packages in README prerequisites (#349)
• docs: add recent mori news — RTP-LLM / DeepSpeed / OpenUCL / Triton-dist (#331)

Install

pip install amd-mori==1.2.0        # stable
pip install --pre amd-mori-nightly # nightly

Full changelog: v1.1.1...v1.2.0

v1.1.1

Highlights

• Profiler Support — New profiler integration for EP intranode dispatch/combine with JIT kernel support and proper packaging
• EP Auto-Tuning — New tuning config system that stores benchmark-derived optimal launch parameters (block_num, warp_per_block, etc.) in JSON files, auto-loaded at runtime based on GPU arch, EP size, token count, and hidden dim. Enable with MORI_EP_LAUNCH_CONFIG_MODE=AUTO, with fallback to hardcoded defaults
• AINIC Environment Auto-Detection Tool — Two new scripts for AINIC (ionic) NIC setup and validation:
  • env_check.sh — 6-step validation (firmware, QoS/DSCP/PFC, DCQCN, intra/inter-node BW & latency). Usage: ./tools/env_check.sh [peer_ip] (omit peer_ip for intra-node checks only)
  • env_setup.sh — configures DCQCN and exports MORI_RDMA_SL/MORI_RDMA_TC. Usage: source tools/env_setup.sh [mori|dcqcn|all]
• SGLang EPLB Support — New call_local_expert_count API for SGLang Expert-Parallel Load Balancing, exposing per-rank local expert counts via op.local_expert_count

---

What's Changed

EP (Expert Parallelism)

• Add profiler support to intranode dispatch/combine (#285)
• Add tuning config system for dispatch/combine (#242)
• Add sgl EPLB support (#254)
• Fix EP index overflow (#260)

IO / RDMA

• Improve RDMA flush diagnostics and log hygiene (#277)
• Disable auto XGMI backend for cross-process IPC transfers (#267)
• Avoid QPN collisions in multi-NIC setups (#261)

Shmem

• Add FlyDSL integration for mori shmem device API (#280)

Profiler

• Fix profile mode detection and doc corrections (#281)
• Fix profile mode for JIT kernel (#265)
• Enable JAX gfx950 CI tests and wrap profiler macros (#282)
• Include tools/profiler in JIT sources (#270)

UMBP

• Release GIL for all UMBPClient methods to fix distributed-mode scheduler starvation (#262)

Packaging & Build

• Only pull SPDK submodule when BUILD_UMBP=ON (#283)

Tools

• Auto detect ionic env problem for AINIC platforms (#278)

CI & Benchmark

• Add MI355X-AINIC platform with podman, fix build for non-mlx5 NICs (#263)
• Add MI300X_BNXT platform with Thor2 NIC support (#291)
• Switch intranode tests to MI355X-AINIC-TW (#288)
• Add shmem benchmark tests (#272, #290)

Doc

• Update arch image & add UMBP description (#287)

v1.1.0

Highlights

• Universal Wheel Packaging — Torch-free pybind with framework-agnostic GPU tensor interop, host/device separation (zero hipcc at install), and JIT hardware auto-detection for GPU arch and NIC type
• EP Enhancements — C++ launch API, async kernel optimization, and memory footprint reduction
• JAX Support — JAX integration via XLA FFI custom calls

EP (Expert Parallelism)

• C++ launch API with AOT + JIT cache support (#195)
• Async kernel optimization (#185) and memory footprint reduction (#245)
• Runtime hidden_dim (#189), runtime dtype (#221), independent combine dtype (#239)
• float8_e8m0fnu support (#215)
• Fix intranode overflow when token >= 65536 (#252), async kernel recv launch bug (#256)

IO / RDMA / XGMI

• SQ-depth admission control with robust batch posting/rollback (#188)
• Optimize XGMI batch_write/batch_read for non-contiguous buffer transfers (#205)
• JIT-compile XGMI scatter/gather kernel (#232)
• Harden RDMA CQE handling (#241)
• Fix XGMI cross-process GPU routing and polling hangs (#247)
• Hidden-device XGMI IPC support (#258)
• Runtime dlopen for vendor dv libraries, removing compile-time linking (#237)

Shmem

• GET (remote read) device API with blocking and non-blocking modes (#193)
• Increase default static heap to 4GB and VMM heap to 16GB (#204)
• Expose mori_shmem_free_tensor for triton-dist low-latency kernel (#224)

Packaging & Build

• Universal wheel with host/device separation and JIT hardware auto-detection (#182)
• CMake C++ integration, lazy imports, optional MPI dependency (#203)
• Build mori as a pip package (#236)
• Unified NIC detection logic (#186, #212)

JAX

• XLA FFI custom calls integration (#226)
• JAX intranode test job (#243)

UMBP (WIP)

• Local client with DRAM/SSD tiered storage (#191)
• Segmented SSD log with pluggable IO backend (#206)
• Control plane implementation (#180) and distributed integration (#209)
• SPDK storage backend (#213) and multitenant SPDK proxy (#223)

CI & Tooling

• Overhaul CI workflow with hybrid container and multi-runner support (#217)
• Pre-commit CI with auto-fix bot (#219, #230)
• Sphinx documentation website (#181)

release v0.1.0

release v0.1.0