[TENT] Bind transport policies to intent type#2839
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces intent-based policy binding to the transport selector, allowing requests to match policies based on business intent (such as foreground get or background prefetch). It updates the selection context, policy definitions, and configuration parsing logic, and adds comprehensive unit tests. Feedback highlights a critical issue where intent_type is ignored during request merging, which could cause requests with different intents to be incorrectly merged and executed under the wrong policy. Additionally, a simplification is suggested for the integer parsing logic in parseIntentType to improve readability.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| ctx.priority_level = | ||
| request.priority; // Use request priority for selection | ||
| ctx.policy_name = request.policy_name; // Optional: bind to specific policy | ||
| ctx.intent_type = request.intent_type; // Business intent policy filter |
There was a problem hiding this comment.
While ctx.intent_type is correctly populated here, there is a critical gap in mergeRequests (and its sorting comparator) where intent_type is completely ignored.
If two requests with different intent_type values are submitted in the same batch, they can be merged into a single request. When merged, the second request's intent_type is lost, and its transfer will be executed under the first request's intent_type policy (including its QP pool, service level, traffic class, etc.).
To fix this, intent_type (and ideally policy_name as well) must be added to the sorting comparator and the can_merge lambda in mergeRequests to prevent merging requests with different intents.
| if (value.is_number_unsigned()) { | ||
| const auto raw = value.get<uint64_t>(); | ||
| if (raw <= static_cast<uint64_t>(IntentType::STAGING_INTERNAL)) { | ||
| return static_cast<IntentType>(raw); | ||
| } | ||
| return std::nullopt; | ||
| } | ||
|
|
||
| if (value.is_number_integer()) { | ||
| const auto raw = value.get<int64_t>(); | ||
| if (raw >= static_cast<int64_t>(IntentType::INTENT_UNSPEC) && | ||
| raw <= static_cast<int64_t>(IntentType::STAGING_INTERNAL)) { | ||
| return static_cast<IntentType>(raw); | ||
| } | ||
| } |
There was a problem hiding this comment.
The parsing logic for unsigned and signed integers can be simplified and made more consistent by using an else if structure and removing the redundant return std::nullopt; inside the first block.
if (value.is_number_unsigned()) {
const auto raw = value.get<uint64_t>();
if (raw <= static_cast<uint64_t>(IntentType::STAGING_INTERNAL)) {
return static_cast<IntentType>(raw);
}
} else if (value.is_number_integer()) {
const auto raw = value.get<int64_t>();
if (raw >= static_cast<int64_t>(IntentType::INTENT_UNSPEC) &&
raw <= static_cast<int64_t>(IntentType::STAGING_INTERNAL)) {
return static_cast<IntentType>(raw);
}
}471ddd3 to
964a590
Compare
|
CI note: the first This is unrelated to the intent-policy path:
I will rerun the failed job once the current workflow attempt finishes. No |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Cluster validation on two H20 nodes for commit Environment:
Passed:
Notes / not counted as pass criteria for this PR:
Current GitHub CI status at the time of this comment: no failures observed; format/docs/tent-ci/build-flags/wheels are passing, several build/test-wheel jobs are still pending. |
|
This PR was replaced by #2847 because the head branch was renamed from |
Summary
Request::intent_typeinto transport policy selectionintent_typefilter in configured TENT policiespolicy_nameoverridesCompatibility
Policies without
intent_typematch every intent exactly as before. Requestswith
INTENT_UNSPECtherefore retain the existing selection behavior unless anoperator explicitly configures an
intent_unspecrule.Test plan
tent_transport_selector_testandtent_intent_type_testServer validation used commit
964a590d, fetched directly from GitHub. Theupstream GitHub Actions matrix is still running; this PR remains Draft until
the TENT CUDA-on/off jobs complete.