-
Notifications
You must be signed in to change notification settings - Fork 101
[ROCm][MoRI] WRITE mode support (layerwise xfer) #157
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
BugenZhao
merged 32 commits into
vllm-project:main
from
simondanielsson:feature/mori-write-mode
May 6, 2026
+815
−304
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
7eac3b7
Initial commit
simondanielsson a6374a4
feat: first impl of async (WRITE) mode and raise if mori used without…
simondanielsson 1584560
fix: add do_remote_* to WRITE path creation of kv transfer params
simondanielsson 19a72b4
refactor: use tokio::join! rather than spawn to avoid race condition
simondanielsson 462f6b1
Merge remote-tracking branch 'upstream/main' into feature/mori-write-…
simondanielsson da39023
chore: move docstring to generate_transfer_id
simondanielsson 7d2f7e2
chroe: rename build_decode_kv_params
simondanielsson 0846eed
docs: mention mori in module docstring in vllm_service_discovery
simondanielsson 79d7efd
chore: cargo fmt
simondanielsson 12f7fba
chore: shorten docstring for MoriIOServiceRegistratoin
simondanielsson 6ddf9af
chore: add trackking issue in comment for why remote_tp_size is 1
simondanielsson d605ce2
feat: error if dispatching before registration
simondanielsson 4528365
refactor: make more DRY with extract_base_http_and_dp_ran
simondanielsson 0f1d3fd
Move prefill stop profiling into big else block
simondanielsson 988d995
chore: cargo fmt
simondanielsson ec42d8a
feat: support prefill profiling in WRITE mode
simondanielsson 7a42f50
refactor: make more DRY by adding handle_decode_response
simondanielsson e39f17a
fix: revert all changes in process_vllm_two_stage_request as mori doe…
simondanielsson 2b497bd
refactor: make more DRY!
simondanielsson d3d6233
chore: fix clippy
simondanielsson 898650b
fix: propagate prefill errors
simondanielsson 1a820dd
fix: support logsprobs from prefill also in WRITE mode
simondanielsson b04a049
fix: enforce mori runs with zmq svc discovery
simondanielsson 0d08cda
fix: commit xfer mode only after registration validation succeeds
simondanielsson 6657952
fix: use send_client_request for prefill request in WRITE mode
simondanielsson 3dbd46e
fix: record failed prefills in WRITE mode with record_pd_prefill_error
simondanielsson 45ee5b4
fix: stop profile decode instance upon prefill error in WRITE mode
simondanielsson 11dba53
chore: update comments and rename variables to concurrent dispatch is…
simondanielsson 14d2d2f
Merge remote-tracking branch 'upstream/main' into feature/mori-write-…
simondanielsson 8a5a448
Merge remote-tracking branch 'upstream/main' into feature/mori-write-…
simondanielsson a800f2d
fix: stop profiling decode upon decode error
simondanielsson ffbc9ca
fix: pass in stored transfer mode to parse_registration and improve t…
simondanielsson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.