Skip to content

[TENT] Bind transport policies to intent type#2839

Closed
catyans wants to merge 3 commits into
kvcache-ai:mainfrom
catyans:codex/intent-policy-binding
Closed

[TENT] Bind transport policies to intent type#2839
catyans wants to merge 3 commits into
kvcache-ai:mainfrom
catyans:codex/intent-policy-binding

Conversation

@catyans

@catyans catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • carry Request::intent_type into transport policy selection
  • allow an optional intent_type filter in configured TENT policies
  • preserve catch-all behavior for existing policies and explicit policy_name overrides
  • reject invalid intent filters fail-closed by skipping the malformed policy
  • document intent-to-transport/QP-pool/SL/TC binding semantics

Compatibility

Policies without intent_type match every intent exactly as before. Requests
with INTENT_UNSPEC therefore retain the existing selection behavior unless an
operator explicitly configures an intent_unspec rule.

Test plan

  • clang-format 20
  • pre-commit static checks and codespell
  • clean TENT CUDA-off build on Ubuntu 24.04 / GCC 13
  • six new intent-policy cases repeated 100 times
  • tent_transport_selector_test and tent_intent_type_test
  • full TENT regression suite: 22/22 passed
  • ASan + UBSan build/tests: 2/2 passed, leak detection enabled

Server validation used commit 964a590d, fetched directly from GitHub. The
upstream GitHub Actions matrix is still running; this PR remains Draft until
the TENT CUDA-on/off jobs complete.

@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci Transfer Engine labels Jul 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces intent-based policy binding to the transport selector, allowing requests to match policies based on business intent (such as foreground get or background prefetch). It updates the selection context, policy definitions, and configuration parsing logic, and adds comprehensive unit tests. Feedback highlights a critical issue where intent_type is ignored during request merging, which could cause requests with different intents to be incorrectly merged and executed under the wrong policy. Additionally, a simplification is suggested for the integer parsing logic in parseIntentType to improve readability.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

ctx.priority_level =
request.priority; // Use request priority for selection
ctx.policy_name = request.policy_name; // Optional: bind to specific policy
ctx.intent_type = request.intent_type; // Business intent policy filter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

While ctx.intent_type is correctly populated here, there is a critical gap in mergeRequests (and its sorting comparator) where intent_type is completely ignored.

If two requests with different intent_type values are submitted in the same batch, they can be merged into a single request. When merged, the second request's intent_type is lost, and its transfer will be executed under the first request's intent_type policy (including its QP pool, service level, traffic class, etc.).

To fix this, intent_type (and ideally policy_name as well) must be added to the sorting comparator and the can_merge lambda in mergeRequests to prevent merging requests with different intents.

Comment on lines +86 to +100
if (value.is_number_unsigned()) {
const auto raw = value.get<uint64_t>();
if (raw <= static_cast<uint64_t>(IntentType::STAGING_INTERNAL)) {
return static_cast<IntentType>(raw);
}
return std::nullopt;
}

if (value.is_number_integer()) {
const auto raw = value.get<int64_t>();
if (raw >= static_cast<int64_t>(IntentType::INTENT_UNSPEC) &&
raw <= static_cast<int64_t>(IntentType::STAGING_INTERNAL)) {
return static_cast<IntentType>(raw);
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The parsing logic for unsigned and signed integers can be simplified and made more consistent by using an else if structure and removing the redundant return std::nullopt; inside the first block.

    if (value.is_number_unsigned()) {
        const auto raw = value.get<uint64_t>();
        if (raw <= static_cast<uint64_t>(IntentType::STAGING_INTERNAL)) {
            return static_cast<IntentType>(raw);
        }
    } else if (value.is_number_integer()) {
        const auto raw = value.get<int64_t>();
        if (raw >= static_cast<int64_t>(IntentType::INTENT_UNSPEC) &&
            raw <= static_cast<int64_t>(IntentType::STAGING_INTERNAL)) {
            return static_cast<IntentType>(raw);
        }
    }

@catyans catyans force-pushed the codex/intent-policy-binding branch from 471ddd3 to 964a590 Compare July 10, 2026 13:11
@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

CI note: the first tent-ci (cuda-off) attempt failed only in the existing
RuntimeQueueDispatch.DispatchesOnlyOneWindowOnSubmit test. The background
progress worker submitted the second window before the test's immediate
submit_calls == 1 assertion, producing 2.

This is unrelated to the intent-policy path:

  • all new intent-policy tests passed in that CI job;
  • tent-ci (cuda-on) passed;
  • the PR commit reproduced the existing race at repeat 992;
  • an unmodified clean checkout of upstream/main@4188e3ae reproduced the same
    assertion at repeat 1473.

I will rerun the failed job once the current workflow attempt finishes. No
production-code workaround is being added for this pre-existing test race.

@codecov-commenter

codecov-commenter commented Jul 10, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
mooncake-transfer-engine/benchmark/utils.cpp 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Cluster validation on two H20 nodes for commit 194ea22:

Environment:

  • Target/initiator: 33.255.68.172 and 33.255.68.173, both 8x NVIDIA H20
  • TENT backend, P2P metadata, RDMA only (tcp/shm/nvlink/mnnvl/gds/io_uring disabled)
  • RDMA bind addresses: 26.248.13.19 <-> 26.248.13.15
  • NIC whitelist: mlx5_bond_0 (bond1, GID index 3)
  • tebench built with USE_TENT=ON, USE_CUDA=ON, and TENT RDMA enabled

Passed:

  • DRAM -> DRAM foreground_get, 172 -> 173, --check_consistency, 64 KiB mixed read/write: rc=0, ~3.01 GB/s, P99 tx ~19 us
  • DRAM -> DRAM foreground_get, 173 -> 172, --check_consistency, 64 KiB mixed read/write: rc=0, ~3.07 GB/s, P99 tx ~18 us
  • DRAM -> DRAM checkpoint intent with policy intent_type=checkpoint, transports=[rdma], --check_consistency, 64 KiB mixed read/write: rc=0, ~2.24 GB/s
  • Negative intent-routing check: policy intent_type=checkpoint, transports=[tcp] while TCP is disabled and buffers are RDMA-registered only. --tent_intent_type=checkpoint fails as expected (rc=1, no registered buffer for selected transport), showing checkpoint did not fall through to the generic memory/RDMA policy.
  • VRAM -> VRAM foreground_get over RDMA/GDR, mixed read/write without consistency check, 64 KiB..1 MiB: rc=0, up to ~12.76 GB/s.

Notes / not counted as pass criteria for this PR:

  • A policy using qp_pool=background for checkpoint matched the checkpoint intent but timed out on this cluster (transfer timeout (software)). A checkpoint policy without qp_pool succeeds. This looks like an adjacent RDMA QoS/QP-pool issue rather than the intent-to-transport binding added here.
  • VRAM --check_consistency failed with an inconsistent-data check in tebench. VRAM non-consistency RDMA/GDR transfers complete successfully; DRAM consistency was used for correctness validation.

Current GitHub CI status at the time of this comment: no failures observed; format/docs/tent-ci/build-flags/wheels are passing, several build/test-wheel jobs are still pending.

@catyans catyans closed this Jul 10, 2026
@catyans catyans deleted the codex/intent-policy-binding branch July 10, 2026 15:09
@catyans catyans restored the codex/intent-policy-binding branch July 10, 2026 15:10
@catyans catyans reopened this Jul 10, 2026
@catyans catyans closed this Jul 10, 2026
@catyans catyans deleted the codex/intent-policy-binding branch July 10, 2026 15:10
@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2848. The source branch was migrated without code changes because branch names must not use the codex prefix; both PRs point to commit 194ea22.

@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

This PR was replaced by #2847 because the head branch was renamed from codex/intent-policy-binding to intent-policy-binding to avoid the codex/ prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci Transfer Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants