feat(simq): add search statistics#2418
Conversation
|
/label S-ready-to-merge |
Merge Protections🟢 All 2 merge protections satisfied — ready to merge. Show 2 satisfied protections🟢 Require kind label
🟢 Require version label
|
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive search statistics tracking for the SIMQ algorithm, including coarse distance comparisons, probe counts, candidate counts, and filtered candidates, which are serialized to JSON and attached to the result datasets. Corresponding unit tests have been added to validate these statistics. Feedback is provided regarding a potential null pointer dereference in the newly added read_dist_cmp helper function if the input dataset is null.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
There was a problem hiding this comment.
Pull request overview
Adds SIMQ-specific search statistics to result datasets (KNN + range) so SIMQ behavior can be observed and compared consistently with other indexes that already populate Dataset::Statistics.
Changes:
- Extend SIMQ coarse search to accumulate coarse-stage probe and distance-comparison counters.
- Populate SIMQ-specific statistic fields (e.g., coarse/rerank/filtered/result counts and limited-size truncation flag) into the returned dataset statistics JSON.
- Add regression assertions in SIMQ tests to require statistics presence and basic invariants.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
tests/test_simq.cpp |
Adds helpers to parse/require SIMQ statistics and asserts stats are present in KNN/range tests. |
src/algorithm/simq/simq.h |
Extends coarse_search signature to optionally return coarse-stage counters. |
src/algorithm/simq/simq.cpp |
Implements SIMQ statistics dumping and wires coarse/rerank/filter/result counters into KNN + range search outputs (including empty-index results). |
d5cb70a to
ac37cfe
Compare
Populate Dataset statistics for SIMQ KNN and range search results, including coarse-search and rerank counters for observability. Fixes antgroup#2417 Signed-off-by: LHT129 <tianlan.lht@antgroup.com> Assisted-by: OpenCode:codex-1/gpt-5.5
ac37cfe to
1f8f826
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
tests/test_simq.cpp:316
- The new statistics assertions don’t cover the empty-index path. Since this PR explicitly changes SIMQ to return non-empty statistics even when
total_count_ == 0/rep_hgraph_ == nullptr, a regression test should verify thatKnnSearchon an unbuilt/empty SIMQ index still returns a dataset whose statistics JSON contains the expected SIMQ keys with zero counts (instead of "{}").
// ─────────────────────────────────────────────────────────────────────────────
// Test cases
// ─────────────────────────────────────────────────────────────────────────────
Summary
Fixes #2417
Test Plan