Add blog post: Inside the vLLM-Omni Architecture: Serving Qwen3-Omni#236
Add blog post: Inside the vLLM-Omni Architecture: Serving Qwen3-Omni#236itigges22 wants to merge 2 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 744eaee922
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| Some resources are linked below if you want to try it out yourself. | ||
|
|
||
| **Docker Hub**: [https://hub.docker.com/repository/docker/jitigges/vllm-omni-demo/general](https://hub.docker.com/repository/docker/jitigges/vllm-omni-demo/general) |
There was a problem hiding this comment.
For readers who are not signed in as the repository owner, this /repository/docker/.../general URL is the Docker Hub management/dashboard route rather than the public repository page, so the “try it out” link can take them to a login or inaccessible page. Use the public /r/<namespace>/<repo> page or include the docker pull command so the demo container is reachable from the published blog post.
Useful? React with 👍 / 👎.
Signed-off-by: Johnathon Isaac Tigges <207649357+itigges22@users.noreply.github.com>
itigges22
force-pushed
the
add-vllm-omni-architecture-post
branch
from
744eaee to
bb1ee2e
Compare
Signed-off-by: Johnathon Isaac Tigges <207649357+itigges22@users.noreply.github.com>
|
we have more to update and will open a new PR recently about serving qwen3-omni |
|
@hsliuustc0106 Looking forward to it. Would you want to coordinate so this does not overlap with your PR? It could potentially sit alongside an official serving guide as a use-case attached rather than duplicate it? But I am happy to update the demo to track the latest vLLM-Omni work once your PR is up. Just let me know! |
we are working on the async scheduling for multi-stage pipeline, please check our recent PRs and issues |
2 participants
Summary
Adds a new blog post: Inside the vLLM-Omni Architecture: Serving Qwen3-Omni, by Isaac Tigges.
The post walks through an insurance-claim triage demo built on a single NVIDIA B200, separates what the Qwen3-Omni model does from what the engine does, and digs into how vLLM-Omni serves the heterogeneous pipeline (Thinker → Talker → Code2Wav) — covering the stage graph, OmniConnector shared-memory transport, async chunking, prefix caching, and benchmarks vs HuggingFace Transformers.
Changes
_posts/2026-06-08-inside-vllm-omni-architecture.mdassets/figures/inside-vllm-omni-architecture/(hero, stage-graph, async-chunking, qwen3-tts-benchmarks, broad-model-support)Test plan
bundle installsucceedsbundle exec jekyll servebuilds the site without errors/2026/06/08/inside-vllm-omni-architecture.html(HTTP 200)