Skip to content

[TENT] Bind transport policies to intent type#2847

Open
catyans wants to merge 4 commits into
kvcache-ai:mainfrom
catyans:intent-policy-binding
Open

[TENT] Bind transport policies to intent type#2847
catyans wants to merge 4 commits into
kvcache-ai:mainfrom
catyans:intent-policy-binding

Conversation

@catyans

@catyans catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • carry Request::intent_type into transport policy selection
  • allow an optional intent_type filter in configured TENT policies
  • preserve catch-all behavior for existing policies and explicit policy_name overrides
  • reject invalid intent filters fail-closed by skipping the malformed policy
  • document intent-to-transport/QP-pool/SL/TC binding semantics

Compatibility

Policies without intent_type match every intent exactly as before. Requests
with INTENT_UNSPEC therefore retain the existing selection behavior unless an
operator explicitly configures an intent_unspec rule.

Test plan

  • clang-format 20
  • pre-commit static checks and codespell
  • clean TENT CUDA-off build on Ubuntu 24.04 / GCC 13
  • six new intent-policy cases repeated 100 times
  • tent_transport_selector_test and tent_intent_type_test
  • full TENT regression suite: 22/22 passed
  • ASan + UBSan build/tests: 2/2 passed, leak detection enabled

Server validation used commit 964a590d, fetched directly from GitHub. The
upstream GitHub Actions matrix is still running; this PR remains Draft until
the TENT CUDA-on/off jobs complete.

@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Replacement for #2839 using branch intent-policy-binding (renamed to avoid the codex/ prefix).

Cluster validation on two H20 nodes for commit 194ea22:

Environment:

  • Target/initiator: 33.255.68.172 and 33.255.68.173, both 8x NVIDIA H20
  • TENT backend, P2P metadata, RDMA only (tcp/shm/nvlink/mnnvl/gds/io_uring disabled)
  • RDMA bind addresses: 26.248.13.19 <-> 26.248.13.15
  • NIC whitelist: mlx5_bond_0 (bond1, GID index 3)
  • tebench built with USE_TENT=ON, USE_CUDA=ON, and TENT RDMA enabled

Passed:

  • DRAM -> DRAM foreground_get, 172 -> 173, --check_consistency, 64 KiB mixed read/write: rc=0, ~3.01 GB/s, P99 tx ~19 us
  • DRAM -> DRAM foreground_get, 173 -> 172, --check_consistency, 64 KiB mixed read/write: rc=0, ~3.07 GB/s, P99 tx ~18 us
  • DRAM -> DRAM checkpoint intent with policy intent_type=checkpoint, transports=[rdma], --check_consistency, 64 KiB mixed read/write: rc=0, ~2.24 GB/s
  • Negative intent-routing check: policy intent_type=checkpoint, transports=[tcp] while TCP is disabled and buffers are RDMA-registered only. --tent_intent_type=checkpoint fails as expected (rc=1, no registered buffer for selected transport), showing checkpoint did not fall through to the generic memory/RDMA policy.
  • VRAM -> VRAM foreground_get over RDMA/GDR, mixed read/write without consistency check, 64 KiB..1 MiB: rc=0, up to ~12.76 GB/s.

Notes / not counted as pass criteria for this PR:

  • A policy using qp_pool=background for checkpoint matched the checkpoint intent but timed out on this cluster (transfer timeout (software)). A checkpoint policy without qp_pool succeeds. This looks like an adjacent RDMA QoS/QP-pool issue rather than the intent-to-transport binding added here.
  • VRAM --check_consistency failed with an inconsistent-data check in tebench. VRAM non-consistency RDMA/GDR transfers complete successfully; DRAM consistency was used for correctness validation.

@catyans catyans marked this pull request as draft July 10, 2026 15:11

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for business intent-based policy binding (intent_type) in the transport selector, allowing requests to match specific transport policies based on their intent (e.g., foreground_get, background_prefetch, checkpoint). The changes span the policy configuration parser, selection matching logic, benchmark tool, documentation, and unit tests. Feedback suggests converting the command-line argument --tent_intent_type to lowercase in the benchmark backend to ensure case-insensitive parsing consistency with the policy configuration parser.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread mooncake-transfer-engine/benchmark/tent_backend.cpp
@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci Transfer Engine labels Jul 10, 2026
@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up validation for the mixed-case intent fix at 65ee26f:

  • built tebench and tent_intent_type_test with TENT enabled
  • ctest -R tent_intent_type_test --output-on-failure: 1/1 passed
  • clang-format 20 check passed for the changed file

All review threads are resolved. Full GitHub CI is now running/queued.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
mooncake-transfer-engine/benchmark/utils.cpp 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci Transfer Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants