Skip to content

refactor(sglang): split remote backend from sidecar launcher#11634

Closed
connorcarpenter15 wants to merge 1 commit into
ai-dynamo:mainfrom
connorcarpenter15:refactor/sglang-remote-backend
Closed

refactor(sglang): split remote backend from sidecar launcher#11634
connorcarpenter15 wants to merge 1 commit into
ai-dynamo:mainfrom
connorcarpenter15:refactor/sglang-remote-backend

Conversation

@connorcarpenter15

Copy link
Copy Markdown
Member

Summary

  • rename the Rust adapter, crate, library, executable, and engine around the remote-backend abstraction:
    • dynamo-sglang-remote
    • dynamo_sglang_remote
    • SglangRemoteEngine
  • compile the remote engine into the Python runtime and expose dynamo._core.backend._run_sglang_remote(argv)
  • add dynamo.sglang.sidecar.main(argv) as the actual SGLang-managed sidecar entrypoint
  • rename the aggregated and disaggregated launch scripts and update CODEOWNERS

Why

Follow-up to #11408.

The native gRPC adapter is useful as a standalone remote backend and only becomes a sidecar when SGLang manages it together with Dynamo's unified worker lifecycle. Calling the adapter itself a sidecar blurred those two responsibilities, especially for SGLang's --sidecar module contract.

This change keeps transport, discovery, request conversion, health gating, cancellation, and disaggregation in the remote-backend crate. The small Python module owns the SGLang sidecar composition.

Interfaces

Standalone remote backend:

dynamo-sglang-remote --sglang-endpoint http://127.0.0.1:30001

SGLang-managed sidecar:

python3 -m sglang.launch_server \
    <args> \
    --grpc-port 30001 \
    --sidecar dynamo.sglang.sidecar

The native launcher prepends a stable argv[0], releases the GIL, and runs the shared dynamo_backend_common::run worker lifecycle.

Validation

  • cargo test -p dynamo-sglang-remote — 21 passed
  • cargo test --manifest-path lib/bindings/python/Cargo.toml sglang_remote_argv_prepends_program_name --lib
  • cargo fmt --all -- --check
  • cargo fmt --manifest-path lib/bindings/python/Cargo.toml -- --check
  • Ruff lint and format checks for the changed Python files
  • Python bytecode compilation
  • bash -n for both renamed launch scripts
  • stale-name scan for the removed crate, executable, type, paths, and script names

GPU end-to-end serving and released-wheel installation are left to CI/integration environments.

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor documentation Improvements or additions to documentation backend::sglang Relates to the sglang backend labels Jul 14, 2026
@datadog-official

This comment has been minimized.

Copy link
Copy Markdown
Member Author

Superseded by #11636, which uses the internal ai-dynamo/refactor/sglang-remote-backend branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::sglang Relates to the sglang backend documentation Improvements or additions to documentation external-contribution Pull request is from an external contributor refactor size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant