Skip to content

[TENT] Add QoS metrics baseline to tebench#2841

Closed
catyans wants to merge 10 commits into
kvcache-ai:mainfrom
catyans:codex/qos-benchmark-baseline
Closed

[TENT] Add QoS metrics baseline to tebench#2841
catyans wants to merge 10 commits into
kvcache-ai:mainfrom
catyans:codex/qos-benchmark-baseline

Conversation

@catyans

@catyans catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a generic --qos_classes=name:threads:slo_us:weight[:isolated_gbps],... workload contract to tebench
  • report per-class SLO attainment, P99 latency, goodput, and isolation leakage
  • report aggregate weighted goodput, weighted Jain fairness, and total link utilization
  • optionally append schema-versioned JSONL records; unavailable baseline-dependent metrics are emitted as null/N/A
  • retain link capacity and isolated baselines in JSONL so all derived metrics are reproducible
  • document metric definitions and matching isolated/mixed-run requirements

Metric semantics

  • SLO attainment is the fraction of completed batches with end-to-end transfer latency at or below the configured class SLO.
  • Jain fairness is computed over per-class throughput / weight.
  • Isolation leakage is max(0, 1 - mixed_throughput / isolated_throughput) and is only emitted when the caller supplies a matching isolated baseline.
  • Total utilization requires an explicit usable link capacity.

Validation

  • classic and USE_TENT=ON tebench targets build successfully with GCC 13.3 in a clean Ubuntu 24.04 container.
  • tebench_qos_metrics_test passes in both builds, covering contract parsing, validation, metric math, JSON validity, and null semantics.
  • TENT SHM target/initiator smoke completed about 1 million mixed batches; isolated-to-mixed recomputation matched all six derived metrics exactly.
  • classic TCP target/initiator smoke passed and produced valid two-class JSONL.
  • cross-host TENT/TCP on sinian-public-1 -> sinian-platform passed. With 64 KiB x batch 4 and four workers, the strict 1 ms foreground SLO attained 17.65%; Jain fairness was 0.8338, maximum isolation leakage 41.21%, and utilization 68.45%. All values were independently recomputed from the JSONL inputs.
  • RDMA-enabled build and metric tests pass; both hosts discover five HCAs. Cross-host RDMA transfer is not claimed as passed because the available backup pair reports peer GID/bootstrap mismatch and WQE retry timeouts.
  • format, codespell, Sphinx, and path checks pass.

Dependency

This draft is stacked on #2794 because that PR introduces the reusable tebench deadline workload and per-request deadline plumbing. Once #2794 merges, this branch will be rebased onto main before review.

彦纾 and others added 8 commits July 8, 2026 15:17
…i#2792)

Within a priority tier, when several flows contend for one NIC the bandwidth
is split blindly/equally today, with no way to give a flow about to miss its
deadline a larger share. This adds an opt-in, deadline-aware arbitration.

The ordering acts at the per-NIC-path slice-post point: after slices are
grouped by (local NIC -> remote NIC), they are sorted most-urgent-first by
predicted MLU (predicted transfer time / remaining deadline window, reusing
the MLU notion from kvcache-ai#2618) before submitSlices() hands them to the QP budget.
That is the point where same-tier flows actually contend for the shared NIC;
an earlier prototype that reordered at the priority-queue pop had no effect
because the QP budget, not the tier queue, is the contention point.

  * bw_arbitration.h: pure OrderByUrgency() policy (unit-testable, no RDMA
    deps) + PredictedMlu(). bw<=0 or no-deadline degrade to original order.
  * workers.cpp: opt-in wiring via transports/rdma/deadline_bw_arbitration
    (default false = byte-identical FIFO / equal split).
  * transfer_engine_bench: --deadline_us / --deadline_tight_threads to tag N
    threads as tight-deadline flows + per-flow tight/loose throughput report.
  * bw_arbitration_test: 7 cases (tighter-first, no-deadline-last, past-due,
    FIFO ties, zero-bandwidth no-op, size-weighting).

Measured (H20/ConnectX 200G RoCE, 2 nodes, 16 threads = 4 tight + 12 loose,
1 MB): tight flows 12.1 -> 44.6 GB/s (+269%), loose yield, total throughput
unchanged (48.4 GB/s). At light load it is a no-op, as intended.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
#	mooncake-transfer-engine/tent/tests/CMakeLists.txt
@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci Transfer Engine labels Jul 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces QoS metrics baseline support and deadline-aware NIC bandwidth arbitration within a priority tier for the transfer engine benchmark tool (tebench). It adds command-line options to configure QoS classes, calculate metrics (such as SLO attainment, Jain's fairness, and isolation leakage), and output versioned JSONL records. Additionally, it implements a deadline-aware scheduling policy for RDMA transport workers to prioritize urgent flows. The review feedback highlights a potential undefined behavior bug when parsing empty SLO fields and suggests avoiding hardcoded /tmp paths in unit tests to prevent permission conflicts in shared environments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread mooncake-transfer-engine/benchmark/qos_metrics.cpp Outdated
Comment thread mooncake-transfer-engine/benchmark/tests/qos_metrics_test.cpp Outdated
@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Cluster validation update:

  • Full classic and TENT-on builds: PASS
  • QoS metric unit/JSON tests: PASS in both builds
  • TENT SHM isolated -> mixed with independent formula recomputation: PASS
  • Classic TCP two-class smoke: PASS
  • Cross-host TENT/TCP isolated -> mixed: PASS (64 KiB x batch 4, 4 workers, 1 ms foreground SLO attainment 17.65%, Jain 0.8338, max leakage 41.21%, utilization 68.45%)
  • RDMA-enabled build and HCA discovery: PASS (5 devices per host)
  • Cross-host RDMA data path: ENVIRONMENT BLOCKED; the available backup pair has peer GID/bootstrap mismatch followed by WQE retry timeout, so no RDMA performance claim is made.

The cluster run also exposed and fixed two reporting issues: JSONL now retains isolated_throughput_gbps and link_capacity_gbps for exact recomputation, and best-effort text leakage keeps six-digit precision.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 64.42049% with 132 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
mooncake-transfer-engine/benchmark/main.cpp 0.00% 76 Missing ⚠️
mooncake-transfer-engine/benchmark/qos_metrics.cpp 79.47% 39 Missing ⚠️
mooncake-transfer-engine/benchmark/utils.cpp 0.00% 17 Missing ⚠️

📢 Thoughts on this report? Let us know!

@catyans catyans closed this Jul 10, 2026
@catyans
catyans deleted the codex/qos-benchmark-baseline branch July 10, 2026 15:09
@catyans

catyans commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2845. The fork branch was renamed to remove the disallowed codex prefix; GitHub automatically closed this PR because its original head ref no longer exists. Commit history is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation run-ci Transfer Engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants