Skip to content

[TENT] Bind transport policies to intent type#2848

Closed
catyans wants to merge 3 commits into
kvcache-ai:mainfrom
catyans:yanshu/intent-policy-binding
Closed

[TENT] Bind transport policies to intent type#2848
catyans wants to merge 3 commits into
kvcache-ai:mainfrom
catyans:yanshu/intent-policy-binding

Conversation

@catyans

@catyans catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • carry Request::intent_type into transport policy selection
  • allow an optional intent_type filter in configured TENT policies
  • preserve catch-all behavior for existing policies and explicit policy_name overrides
  • reject invalid intent filters fail-closed by skipping the malformed policy
  • document intent-to-transport/QP-pool/SL/TC binding semantics

Compatibility

Policies without intent_type match every intent exactly as before. Requests
with INTENT_UNSPEC therefore retain the existing selection behavior unless an
operator explicitly configures an intent_unspec rule.

Test plan

  • clang-format 20
  • pre-commit static checks and codespell
  • clean TENT CUDA-off build on Ubuntu 24.04 / GCC 13
  • six new intent-policy cases repeated 100 times
  • tent_transport_selector_test and tent_intent_type_test
  • full TENT regression suite: 22/22 passed
  • ASan + UBSan build/tests: 2/2 passed, leak detection enabled

Server validation used commit 964a590d, fetched directly from GitHub. The
upstream GitHub Actions matrix is still running; this PR remains Draft until
the TENT CUDA-on/off jobs complete.

@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

This replaces #2839 after migrating the source branch to yanshu/intent-policy-binding. The code commit is unchanged (194ea22); prior discussion and review remain available on #2839.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces business-intent-based policy binding (intent_type) to the transport selector, allowing requests to match specific QoS and transport policies (such as foreground_get or checkpoint). It also adds command-line benchmark support and comprehensive unit tests. The review feedback highlights a critical issue where request merging does not group by intent_type, potentially causing QoS leakage. Additionally, the reviewer suggests making command-line intent parsing case-insensitive to prevent crashes and explicitly handling null JSON values for intent_type to avoid unexpectedly skipping catch-all policies.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

ctx.priority_level =
request.priority; // Use request priority for selection
ctx.policy_name = request.policy_name; // Optional: bind to specific policy
ctx.intent_type = request.intent_type; // Business intent policy filter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

While intent_type is successfully propagated to the selection context here, the request merging logic in mergeRequests (and its helper can_merge) in this file does not check or sort by intent_type. Consequently, requests with different business intents (e.g., foreground_get and background_prefetch) can be merged into a single transfer task, causing one of the intents to be discarded and routed under the other's policy. To prevent this QoS/intent leakage, mergeRequests should be updated to group and merge requests by intent_type as well.

Comment on lines +106 to +109
auto it = kIntentTypes.find(intent_type);
LOG_ASSERT(it != kIntentTypes.end())
<< "Invalid --tent_intent_type=" << intent_type;
return it->second;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The lookup in kIntentTypes is case-sensitive. If a user passes --tent_intent_type with mixed or uppercase characters (e.g., Foreground_Get), the lookup will fail and trigger a crash via LOG_ASSERT. Converting the input string to lowercase before performing the lookup makes the command-line interface more robust and user-friendly.

Suggested change
auto it = kIntentTypes.find(intent_type);
LOG_ASSERT(it != kIntentTypes.end())
<< "Invalid --tent_intent_type=" << intent_type;
return it->second;
std::string lower_intent = intent_type;
std::transform(lower_intent.begin(), lower_intent.end(), lower_intent.begin(),
[](unsigned char c) { return std::tolower(c); });
auto it = kIntentTypes.find(lower_intent);
LOG_ASSERT(it != kIntentTypes.end())
<< "Invalid --tent_intent_type=" << intent_type;
return it->second;

Comment on lines +249 to +257
if (policy_json.contains("intent_type")) {
auto intent = parseIntentType(policy_json["intent_type"]);
if (!intent.has_value()) {
LOG(WARNING)
<< "Skip policy " << policy.name << ": invalid intent_type";
continue;
}
policy.intent_type = *intent;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If a user explicitly configures "intent_type": null in the JSON policy, parseIntentType will return std::nullopt and cause the entire policy to be skipped with a warning. Explicitly handling null values as std::nullopt (which represents a catch-all/no-filter intent) is more robust and prevents unexpected policy skipping.

        if (policy_json.contains("intent_type")) {
            const auto& val = policy_json["intent_type"];
            if (val.is_null()) {
                policy.intent_type = std::nullopt;
            } else {
                auto intent = parseIntentType(val);
                if (!intent.has_value()) {
                    LOG(WARNING)
                        << "Skip policy " << policy.name << ": invalid intent_type";
                    continue;
                }
                policy.intent_type = *intent;
            }
        }

@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by ; the final source branch is intent-policy-binding, with the same commit 194ea22.

@catyans catyans closed this Jul 10, 2026
@catyans
catyans deleted the yanshu/intent-policy-binding branch July 10, 2026 15:20
@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci Transfer Engine labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci Transfer Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant