feat: unify memory statistics API signatures

[LHT129]

Background

The Index public API provides 4 memory statistics interfaces with inconsistent return types, parameter types, and naming conventions.

Problem

1. Inconsistent return types: GetMemoryUsage returns int64_t, GetMemoryUsageDetail returns std::string (JSON), EstimateMemory returns uint64_t, GetEstimateBuildMemory returns int64_t. Memory size is inherently non-negative.
2. Inconsistent parameter types: EstimateMemory(uint64_t) vs GetEstimateBuildMemory(int64_t).
3. Inconsistent naming: GetMemoryUsage (Get prefix), EstimateMemory (no prefix), GetEstimateBuildMemory (mixed).
4. GetMemoryUsageDetail implementation inconsistency: HGraph used CalcSerializeSize() for detail but GetMemoryUsage() for total.
5. GetMemoryUsageDetail incomplete coverage: Only HGraph had a real implementation.
6. JsonType not exposed: Public API returned std::string JSON but JsonType was never exported.

Proposed Changes

• GetMemoryUsage: int64_t -> uint64_t
• GetMemoryUsageDetail: std::string (JSON) -> std::unordered_map<std::string, uint64_t>
• GetEstimateBuildMemory -> EstimateBuildMemory, int64_t -> uint64_t
• current_memory_usage_ atomic: int64_t -> uint64_t
• HGraph GetMemoryUsageDetail now uses GetMemoryUsage() per component for runtime consistency
• Updated all index implementations, datacell layer, tests, and eval tools

Fixes #2387

[mergify]

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Require kind label

• label~=^kind/

🟢 Require version label

• label~=^version/

[gemini-code-assist]

Code Review

This pull request refactors memory usage tracking across the codebase by changing the return types and parameters of memory-related functions (such as GetMemoryUsage and EstimateBuildMemory) from int64_t to uint64_t. Additionally, GetMemoryUsageDetail has been updated to return a std::map<std::string, uint64_t> instead of a JSON string. Feedback on these changes highlights a failing assertion in tests/test_hgraph.cpp due to the renaming of the "route_graph" key to indexed keys, and a type mismatch in src/impl/reverse_edge.cpp where a local variable remains int64_t despite the function's return type being updated to uint64_t.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR standardizes the memory statistics APIs across the public Index interface and internal implementations by making memory sizes consistently unsigned (uint64_t), renaming the build-memory estimator, and replacing JSON-string “detail” output with a structured map.

Changes:

• Updated memory-related return/param types to uint64_t and renamed GetEstimateBuildMemory → EstimateBuildMemory.
• Changed GetMemoryUsageDetail from JSON std::string to std::map<std::string, uint64_t> and updated callers/tests.
• Implemented/adjusted component-level memory detail reporting (notably in HGraph) for runtime consistency.

Reviewed changes

Copilot reviewed 68 out of 68 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/eval/case/search_eval_case.cpp	Adapts eval output to new GetMemoryUsageDetail() map shape
tools/eval/case/build_eval_case.cpp	Adapts eval output to new GetMemoryUsageDetail() map shape
tests/test_simple_index.cpp	Updates test index API overrides and build-memory estimate test
tests/test_random_index.cpp	Updates build-memory estimator name in assertion
tests/test_hgraph.cpp	Updates HGraph detail assertions for map-based detail
src/vsag_ext.cpp	Updates IndexHandler::GetMemoryUsage() to uint64_t
src/utils/visited_list.h	Updates resource memory usage return type to uint64_t
src/utils/resource_object.h	Updates ResourceObject::GetMemoryUsage() and docs to uint64_t
src/utils/lock_strategy.h	Updates mutex memory usage signatures to uint64_t
src/utils/lock_strategy.cpp	Updates PointsMutex::GetMemoryUsage() to uint64_t
src/io/io_context.h	Updates IOContext::GetMemoryUsage() to uint64_t
src/index/index_impl.h	Forwards renamed/ret-typed memory APIs to inner index
src/index/hnsw.h	Updates HNSW::GetMemoryUsage() return type
src/index/diskann.h	Updates DiskANN memory APIs and renames estimator
src/index/diskann.cpp	Updates DiskANN build-memory estimator to uint64_t
src/impl/reverse_edge_test.cpp	Updates test variable type for memory usage
src/impl/reverse_edge.h	Updates ReverseEdge::GetMemoryUsage() return type
src/impl/reverse_edge.cpp	Updates ReverseEdge::GetMemoryUsage() return type/casts
src/impl/label_table/label_table.h	Updates label table memory usage return type
src/impl/label_table/label_remap.h	Updates label remap memory usage return type
src/impl/bitset/sparse_bitset.h	Updates sparse bitset memory usage return type
src/impl/bitset/sparse_bitset.cpp	Updates sparse bitset memory usage return type/cast
src/impl/bitset/fast_bitset.h	Updates fast bitset memory usage return type
src/impl/bitset/fast_bitset.cpp	Updates fast bitset memory usage return type/cast
src/impl/bitset/computable_bitset.h	Updates abstract bitset memory usage signature to uint64_t
src/datacell/sparse_vector_datacell.inl	Updates memory usage accumulator/return type to uint64_t
src/datacell/sparse_vector_datacell.h	Updates memory usage return type to uint64_t
src/datacell/sparse_term_datacell.h	Updates memory usage return type to uint64_t
src/datacell/sparse_term_datacell.cpp	Updates memory usage return type/cast to uint64_t
src/datacell/sparse_graph_datacell.h	Updates memory usage return type to uint64_t
src/datacell/sparse_graph_datacell.cpp	Updates memory usage return type/cast to uint64_t
src/datacell/rabitq_split_datacell.h	Updates memory usage return types/accumulators to uint64_t
src/datacell/multi_vector_datacell.inl	Updates memory usage accumulator/return type to uint64_t
src/datacell/multi_vector_datacell.h	Updates memory usage return type to uint64_t
src/datacell/graph_interface.h	Updates virtual GetMemoryUsage() return type
src/datacell/graph_datacell.h	Updates memory usage accumulator/return type to uint64_t
src/datacell/flatten_interface.h	Updates virtual GetMemoryUsage() return type
src/datacell/flatten_datacell.h	Updates flatten datacell memory usage types
src/datacell/extra_info_interface.h	Updates virtual GetMemoryUsage() return type
src/datacell/extra_info_datacell.h	Updates extra info datacell memory usage types
src/datacell/compressed_graph_datacell.h	Updates compressed graph datacell memory usage signature
src/datacell/compressed_graph_datacell.cpp	Updates compressed graph datacell memory usage return type/cast
src/datacell/bucket_interface.h	Updates bucket interface memory usage signature
src/datacell/bucket_datacell.h	Updates bucket datacell accumulator/return type
src/datacell/attribute_inverted_interface.h	Updates attribute inverted interface memory usage signature
src/datacell/attribute_bucket_inverted_datacell.h	Updates attribute inverted datacell memory usage signature
src/datacell/attribute_bucket_inverted_datacell.cpp	Updates attribute inverted datacell memory usage return type/cast
src/attr/multi_bitset_manager.h	Updates multibitset manager memory usage signature
src/attr/multi_bitset_manager.cpp	Updates multibitset manager memory usage return type/cast
src/attr/attr_value_map.h	Updates attr value map memory usage signature
src/attr/attr_value_map.cpp	Updates attr value map memory usage accumulator/return type
src/algorithm/sindi/sindi.h	Updates GetMemoryUsageDetail() stub to return an empty map
src/algorithm/lazy_hgraph/lazy_hgraph.h	Updates lazy hgraph memory usage signature
src/algorithm/lazy_hgraph/lazy_hgraph.cpp	Updates lazy hgraph memory usage return type
src/algorithm/ivf/ivf_partition_strategy.h	Updates IVF partition memory usage signature
src/algorithm/ivf/ivf_nearest_partition.h	Updates IVF nearest partition memory usage signature
src/algorithm/ivf/ivf_nearest_partition.cpp	Updates IVF nearest partition memory usage return type/cast
src/algorithm/ivf/ivf.h	Updates IVF memory usage signature
src/algorithm/ivf/ivf.cpp	Updates IVF GetMemoryUsage() return type
src/algorithm/inner_index_interface_test.cpp	Updates estimator name in interface tests
src/algorithm/inner_index_interface.h	Renames estimator, updates memory APIs, changes atomic type to uint64_t
src/algorithm/hgraph/hgraph_serialize.cpp	Replaces JSON detail with map-based per-component memory usage
src/algorithm/hgraph/hgraph.h	Updates HGraph detail signature to map
src/algorithm/bruteforce/bruteforce.h	Updates brute force memory usage signature
src/algorithm/bruteforce/bruteforce.cpp	Updates brute force GetMemoryUsage() return type
mockimpl/vsag/simpleflat.h	Updates mock index memory usage signature
include/vsag/vsag_ext.h	Updates public extension handler memory usage signature
include/vsag/index.h	Updates public API signatures/docs for memory stats APIs

Comments suppressed due to low confidence (5)

tools/eval/case/search_eval_case.cpp:1

• Index::GetMemoryUsageDetail() (public API) still throws std::runtime_error by default (see include/vsag/index.h), but this eval tool no longer catches it, so it'll terminate when detail isn't supported. Also, casting uint64_t size to int64_t can overflow (becoming negative) for large values. Fix by catching std::exception (or std::runtime_error) here and by writing an unsigned value into the JSON result (or rendering as string) instead of static_cast<int64_t>(size).
  tools/eval/case/build_eval_case.cpp:1
• Same issue as search_eval_case.cpp: the public default implementation throws std::runtime_error, but only vsag::VsagException is caught now, so unsupported detail can crash eval runs. Also uint64_t → int64_t may overflow when the memory value exceeds INT64_MAX. Catch std::exception (or include std::runtime_error) and avoid narrowing to signed.
  src/impl/reverse_edge.cpp:1
• ReverseEdge::GetMemoryUsage() now returns uint64_t, but the accumulator is still int64_t. The += mixes signed/unsigned, which can overflow or produce incorrect results, and then return usage; converts a potentially-negative int64_t to uint64_t. Change usage to uint64_t (and remove signed casts) so all arithmetic stays unsigned.
  src/impl/reverse_edge.cpp:1
• ReverseEdge::GetMemoryUsage() now returns uint64_t, but the accumulator is still int64_t. The += mixes signed/unsigned, which can overflow or produce incorrect results, and then return usage; converts a potentially-negative int64_t to uint64_t. Change usage to uint64_t (and remove signed casts) so all arithmetic stays unsigned.
  tests/test_random_index.cpp:1
• This comparison now mixes uint64_t with a floating-point multiplier (* 1.2), which forces conversion to floating point and can lose precision for large memory values. Prefer an integer-safe comparison (e.g., compare against EstimateBuildMemory(max_elements) * 6 / 5) or cast to a wider floating type explicitly if a ratio-based tolerance is required.

[LHT129]

All review comments addressed and resolved in the latest push:

1. route_graph key (gemini): Route graph memory is now aggregated under a single "route_graph" key instead of per-index keys like "route_graph_0". This matches existing test expectations.
2. reverse_edge.cpp int64_t (gemini): Changed usage to uint64_t and removed unnecessary static_cast.
3. cal_memory_usage() signed cast (copilot): Removed static_cast<int64_t> in cal_memory_usage() across IVF, HGraph, SINDI, and Pyramid. Also fixed the same pattern found in pyramid.cpp.
4. attr_value_map helper (copilot): Changed get_memory_usage() helper return type and accumulator from int64_t to uint64_t.

[Copilot]

Pull request overview

Copilot reviewed 71 out of 71 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 71 out of 71 changed files in this pull request and generated 3 comments.

[LHT129]

All review feedback addressed in the latest push:

1. eval case int64_t overflow (search_eval_case.cpp / build_eval_case.cpp): Removed static_cast<int64_t> — now stores uint64_t directly in JSON. Also changed catch(VsagException) to catch(const std::exception&) to handle std::runtime_error from unsupported implementations.
2. PR description alignment (index.h): Updated PR body to say std::unordered_map instead of std::map. The API intentionally uses std::unordered_map for O(1) lookups. Most index types do not need to override GetMemoryUsageDetail — the default throw behavior is intentional for indexes without component-level breakdown.

Rebased onto latest origin/main (includes #1984).

[Copilot]

Pull request overview

Copilot reviewed 73 out of 73 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

tools/eval/case/search_eval_case.cpp:1

• Catching only std::exception may stop handling vsag::VsagException if it does not derive from std::exception (the previous code explicitly caught vsag::VsagException). To preserve behavior (and avoid unexpected termination), add an explicit catch (const vsag::VsagException& e) before the std::exception catch (and apply the same change in tools/eval/case/build_eval_case.cpp).
  tools/eval/case/search_eval_case.cpp:1
• Iterating a std::unordered_map yields non-deterministic key order, which can make the emitted JSON unstable across runs (problematic for report diffs / golden outputs). Consider sorting keys before writing to result["memory_detail(B)"] (e.g., collect keys, sort, then emit) to make eval outputs reproducible.

[Copilot]

Pull request overview

Copilot reviewed 73 out of 73 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (4)

tools/eval/case/search_eval_case.cpp:1

• This changes the exception handling behavior vs. the previous catch (vsag::VsagException&) branch. If vsag::VsagException does not derive from std::exception, it will no longer be caught here (potentially terminating the tool). Consider either adding back a catch (const vsag::VsagException& e) branch, or ensuring VsagException inherits from std::exception so that catch (const std::exception&) is correct.
  tools/eval/case/search_eval_case.cpp:1
• Iterating an std::unordered_map produces non-deterministic key order, which can make eval output unstable across runs (noisy diffs, hard-to-compare baselines). Consider emitting keys in a stable order (e.g., copy to a vector and sort by name, or return/convert to std::map at the boundary where output is generated).
  tools/eval/case/build_eval_case.cpp:1
• Same as search_eval_case.cpp: switching to only catch (const std::exception&) may stop catching vsag::VsagException if it is not derived from std::exception. Consider restoring an explicit catch (const vsag::VsagException&) or aligning the exception hierarchy so this handler remains effective.
  tools/eval/case/build_eval_case.cpp:1
• Eval output may become non-deterministic due to unordered_map iteration order. For reproducible evaluation reports, consider sorting component names before writing them into result (or using an ordered map for output).

[LHT129]

Addressed Review Comments (round 3)

All 5 unresolved review threads have been addressed:

1. route_graph key stability (threads 11, 13): Fixed — removed the conditional. is now always included in the map for a stable key schema.
2. Exception message (thread 12): Fixed — changed to .
3. Breaking API/ABI (thread 14): Intentional — this is a pre-1.0 library and the task explicitly requires these signature changes.
4. Unit tests (thread 15): Existing tests already cover the scenarios. With the fix, the key set is now stable.

Also added documentation for and in both English and Chinese docs.

Rebased onto latest (), single commit .

[vsag-bot]

/label S-ready-to-merge
/waiting-on maintainer
/request-review @jiaweizone
/request-review @inabao

[Copilot]

Pull request overview

Copilot reviewed 73 out of 73 changed files in this pull request and generated 5 comments.

[LHT129]

Round 5 review fixes

• Docs (en+zh): Changed "throw " to "throw an exception" for both and , since factory-created indexes actually throw via .
• Test: Changed to in to avoid implicit float promotion of .

[wxyucs]

lgtm

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request