Fix missing Ascend RT launch hooks for vLLM native kernels

[ltaodream]

Motivation

hami-vnpu-core limits Ascend compute usage by intercepting selected Ascend RT launch APIs and calling the existing token limiter before forwarding the launch to the real runtime API.

The current hook set already covers:

• rtAicpuKernelLaunchExWithArgs
• rtAicpuKernelLaunchWithFlag
• rtKernelLaunchWithFlagV2
• rtKernelLaunchWithHandleV2
• rtModelExecute

However, vLLM-Ascend native kernels can also launch through RT symbols that are not currently intercepted by hami-vnpu-core.

In the local vLLM-Ascend environment, libvllm_ascend_kernels.so imports the following RT launch symbols:

rtVectorCoreKernelLaunchWithHandle
rtKernelLaunchWithFlagV2
rtKernelLaunchWithHandle
rtKernelLaunchWithHandleV2

Before this PR, rtKernelLaunchWithHandle and rtVectorCoreKernelLaunchWithHandle were missing from the hook layer. Kernels launched through these symbols could bypass npu_limiter().wait_for_token(stm), so compute throttling would not apply to those paths.

This matters for serving stacks such as vLLM-Ascend and SGLang-on-Ascend, where native custom kernels may be used by inference workloads. In that case, memory quota enforcement can still work, but compute isolation may be ineffective if the hot launch path is not intercepted.

This PR adds the two missing RT launch hooks and keeps the implementation aligned with the existing RT-layer interception design.

Modifications

This PR adds hooks for:

• rtKernelLaunchWithHandle
• rtVectorCoreKernelLaunchWithHandle

Both hooks follow the existing pattern used by other RT launch hooks:

npu_limiter().wait_for_token(stm);
return passthrough!(...);

No new scheduling policy is introduced. The change only extends coverage of the existing compute limiter to additional RT launch APIs.

Main changes:

• Add rtKernelLaunchWithHandle hook.
• Add rtVectorCoreKernelLaunchWithHandle hook.
• Keep return type and argument style consistent with the existing RT hooks in hook.rs.
• Do not add ACL-layer hooks.
• Do not add RTS-layer hooks.
• Do not change memory quota logic.
• Do not change graph/model execution behavior.

Validation

Static Symbol Check

Checked that vLLM-Ascend imports the missing RT launch symbols:

readelf -Ws /vllm-workspace/vllm-ascend/vllm_ascend/libvllm_ascend_kernels.so \
  | grep -E 'UND.*(rtKernelLaunchWithHandle|rtVectorCoreKernelLaunchWithHandle|rtKernelLaunchWithHandleV2|rtKernelLaunchWithFlagV2)'

Result:

UND rtVectorCoreKernelLaunchWithHandle
UND rtKernelLaunchWithFlagV2
UND rtKernelLaunchWithHandle
UND rtKernelLaunchWithHandleV2

This confirms that vLLM-Ascend native kernels can use the two symbols added by this PR.

Build Check

Built hami-vnpu-core locally:

cargo build --release

Result: passed.

Confirmed the new symbols are exported by libvnpu.so:

nm -D target/release/libvnpu.so \
  | grep -E ' rtKernelLaunchWithHandle$| rtVectorCoreKernelLaunchWithHandle$| rtKernelLaunchWithHandleV2$| rtKernelLaunchWithFlagV2$'

Result:

T rtKernelLaunchWithFlagV2
T rtKernelLaunchWithHandle
T rtKernelLaunchWithHandleV2
T rtVectorCoreKernelLaunchWithHandle

Diff Check

Checked the patch for whitespace issues:

git diff --check

Result: passed.

Runtime Test Environment

Runtime validation was performed on a real Ascend NPU environment.

Environment details:

• NPU: Ascend 910B2C
• CANN used for runtime test: cann-8.5.1
• Test workload: vLLM-Ascend native op
• Hook library: target/release/libvnpu.so
• Compute limiter mode: two concurrent processes sharing the same NPU_GLOBAL_SHM_PATH
• Priorities tested: NPU_PRIORITY=20 and NPU_PRIORITY=80

NPU status after testing:

npu-smi info

Result:

Health: OK
No running processes found

Runtime Workload

The runtime test used a vLLM-Ascend native custom op:

torch.ops._C_ascend.get_masked_input_and_mask(...)

The op is provided by:

import vllm_ascend.vllm_ascend_C

This path uses the vLLM-Ascend native kernel library that imports the missing RT launch symbols.

The test repeatedly launched the same native op from two concurrent processes.

Each process used:

LD_PRELOAD=target/release/libvnpu.so
NPU_GLOBAL_SHM_PATH=<same global path>
NPU_LOCAL_SHM_PATH=<different local path>
NPU_MEM_QUOTA=65536

The only intended difference between the two processes was:

NPU_PRIORITY=20
NPU_PRIORITY=80

Baseline Without LD_PRELOAD

First, the same two-process workload was run without LD_PRELOAD.

Result:

process	priority	throughput
low	unset	33018 ops/s
high	unset	32967 ops/s

The two processes had effectively equal throughput without the limiter.

Original hami-vnpu-core Behavior

Then the same workload was run with the original hami-vnpu-core hook library before this PR.

Result:

process	priority	throughput
low	20	25984 ops/s
high	80	28584 ops/s

The throughput ratio was only about 1.10x, so the workload was not meaningfully throttled according to the 20/80 priority split.

This shows that the original hook set did not intercept the relevant vLLM-Ascend native launch path.

Behavior With This PR

Finally, the same workload was run with this PR applied.

Result:

process	priority	throughput
low	20	5250 ops/s
high	80	21665 ops/s

The throughput ratio was about 4.13x, matching the expected 20/80 priority ratio.

This confirms that adding rtKernelLaunchWithHandle and rtVectorCoreKernelLaunchWithHandle makes the existing compute limiter effective for this vLLM-Ascend native kernel path.

Result Summary

Observed throughput:

test case	low process	high process	effect
no LD_PRELOAD	33018 ops/s	32967 ops/s	no throttling
original hook set	25984 ops/s	28584 ops/s	no meaningful priority split
this PR	5250 ops/s	21665 ops/s	priority split effective

Conclusion:

This PR fixes a real RT-layer interception gap for vLLM-Ascend native kernels. The original hook set did not meaningfully limit this workload, while this PR makes the existing priority-based compute limiter effective.

Summary by CodeRabbit

• New Features
  • Added support for two additional kernel launch entry points, expanding compatibility with more runtime calls.
  • These launches now respect the same scheduling limits and token-waiting behavior as existing kernel execution paths.

[hami-robot]

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.

📝 Please follow instructions in the contributing guide to update your commits with the DCO

Full details of the Developer Certificate of Origin can be found at developercertificate.org.

The list of commits missing DCO signoff:

• 2f686e6 Add missing Ascend RT launch hooks

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[hami-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ltaodream

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hami-robot]

Welcome @ltaodream! It looks like this is your first PR to Project-HAMi/hami-vnpu-core 🎉

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5229f761-e8ca-439c-af7a-e9b301f45c13

📥 Commits

Reviewing files that changed from the base of the PR and between 962cb48 and 2f686e6.

📒 Files selected for processing (1)

• crates/hook/src/hook.rs

---

📝 Walkthrough

Walkthrough

Two new exported FFI wrapper functions, rtKernelLaunchWithHandle and rtVectorCoreKernelLaunchWithHandle, are added to crates/hook/src/hook.rs. Each waits on npu_limiter().wait_for_token(stm) before forwarding arguments to passthrough! with a corresponding ABI signature.

Changes

Kernel launch hooks

Layer / File(s)	Summary
New kernel-launch wrapper functions crates/hook/src/hook.rs	Adds rtKernelLaunchWithHandle and rtVectorCoreKernelLaunchWithHandle extern "C" functions that wait for a token via npu_limiter().wait_for_token(stm) before invoking passthrough! with their respective argument signatures; existing rtKernelLaunchWithHandleV2 remains unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant HookFFI
  participant NpuLimiter
  participant Passthrough

  Caller->>HookFFI: rtKernelLaunchWithHandle(handle, tilingKey, blockDim, argsInfo, smDesc, stm, kernelInfo)
  HookFFI->>NpuLimiter: wait_for_token(stm)
  NpuLimiter-->>HookFFI: token acquired
  HookFFI->>Passthrough: forward args
  Passthrough-->>Caller: return u64

Loading

Poem

A rabbit hops through kernel gates,
Two new handles, token it awaits,
Passthrough whispers, "go ahead!"
Stream by stream, the tasks are fed,
Hop, hop, hooray, the code is spread! 🐇⚡

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding missing Ascend RT launch hooks for vLLM native kernels.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}