Skip to content

refactor(param): introduce declarative schema and migrate BruteForceParameter#2376

Open
LHT129 wants to merge 1 commit into
antgroup:mainfrom
LHT129:opencode/pr1-param-schema-bruteforce
Open

refactor(param): introduce declarative schema and migrate BruteForceParameter#2376
LHT129 wants to merge 1 commit into
antgroup:mainfrom
LHT129:opencode/pr1-param-schema-bruteforce

Conversation

@LHT129

@LHT129 LHT129 commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

What

Add a small declarative schema framework under src/param/ and use it for BruteForceParameter as the first migration. The framework lets a parameter class describe its own fields once and have FromJson, ToJson, CheckCompatibility (and unknown-key detection) derived from that single source of truth.

Why

BruteForceParameter (and every other *_parameter.{h,cpp} that will follow) hand-writes three near-identical blocks: a FromJson with CHECK_ARGUMENT(json.Contains(...)), a ToJson with the matching json[KEY].SetJson(...), and a CheckCompatibility that chains CHECK_FIELD_EQ / CHECK_SUB_PARAM. The boilerplate grows linearly with the field count and is silent on unknown keys, so typos like {"pq_dim": 8} on a BruteForce index parse cleanly today.

A schema-driven approach lets each class declare the fields once and produce the three behaviours from that single declaration. Range<T> covers numeric validation, and a ConstantStringField handles the common "type": "brute_force" tag without giving it special-case treatment.

Schema framework (src/param/)

  • validators.h provides a header-only Range<T> validator that callers can attach to a scalar field. It throws VsagException with ErrorType::INVALID_ARGUMENT if the value is outside the closed [min, max] range, mirroring the existing CHECK_ARGUMENT style.
  • schema.h exposes Schema, SchemaBuilder<Owner>, and a small set of VSAG_PARAM* macros. Two field kinds are supported in this PR:
    • ScalarField<T> for uint64_t / int64_t / std::string / bool / float (the types BruteForce-class parameters use today).
    • SubParamField<Sub> for Parameter-derived sub-objects that are constructed via a factory (e.g. FlattenInterfaceParameter for BruteForceParameter::base_codes_param).
    • A ConstantStringField covers the "type": "brute_force" tag pattern that several index parameters share.
    • Schema::Parse calls each field's parser, fills the C++ member, runs the per-field validator, and refuses unknown JSON keys via CheckUnknownKeys. Schema::Serialize writes the fields back in the order they were declared so the JSON layout stays stable. Schema::Equal walks the fields once and stops at the first mismatch, replacing manual CHECK_FIELD_EQ / CHECK_SUB_PARAM ladders.
  • schema.cpp implements CheckUnknownKeys using JsonWrapper::GetInnerJson()->items() so callers can register the set of keys the base class already consumes (extra_known_keys) without forcing every parameter to inherit a common JSON pre-filter.

The framework is intentionally narrow: it does not touch any InnerIndexParameter field, does not add a Validate hook (left for PR #2), and does not migrate any of the other parameter classes. That restraint is what keeps the BruteForce-side diff small.

BruteForce migration

BruteForceParameter now declares its schema in-class via VSAG_PARAM_SCHEMA_DECL() and defines it out-of-line in the .cpp:

VSAG_PARAM_SCHEMA(BruteForceParameter)
    VSAG_PARAM_TYPE_TAG(TYPE_KEY, INDEX_TYPE_BRUTE_FORCE)
    VSAG_PARAM_SUBPARAM_REQUIRED(FlattenInterfaceParameter, base_codes_param, BASE_CODES_KEY, CreateFlattenParam)
VSAG_PARAM_SCHEMA_END()

Three overrides (FromJson / ToJson / CheckCompatibility) delegate to schema().Parse / schema().Serialize / schema().Equal after forwarding to the InnerIndexParameter base, so inherited fields keep their existing behaviour. The hand-written CHECK_ARGUMENT(json.Contains(BASE_CODES_KEY)), CHECK_SUB_PARAM(*this, *p, base_codes_param), and json[BASE_CODES_KEY].SetJson(base_codes_param->ToJson()) blocks are gone.

JSON compatibility

The schema keeps the field order stable and uses the same keys / types as the hand-written version, so BruteForceParameter::FromJson(user_json) followed by ToJson() produces byte-identical output before and after this PR. Unknown keys ({"pq_dim": 8} etc.) now throw INVALID_ARGUMENT instead of being silently accepted.

Tests

  • src/param/schema_test.cpp exercises the framework end-to-end with a minimal in-process TestParam (one scalar with a Range, one string, one sub-parameter) and covers four behaviours: scalar + sub-parameter parse, JSON round-trip stability, Range rejection, and unknown-key rejection. Both rejection paths assert VsagException.
  • The existing BruteForce Parameters CheckCompatibility test in src/algorithm/bruteforce/bruteforce_parameter_test.cpp is unchanged and continues to exercise the public surface of BruteForceParameter (now backed by the schema).

Build wiring

  • src/param/CMakeLists.txt follows the project convention: an OBJECT library param for production code and a STATIC library param_test for the unit tests, gated on ENABLE_TESTS.
  • src/CMakeLists.txt registers param as a subdirectory and adds it to VSAG_DEP_LIBS so vsag / vsag_static pick it up.
  • tests/CMakeLists.txt links param_test into unittests with the same --whole-archive (Linux) / -Wl,-force_load (macOS) treatment used for the other *_test static libraries.

Verification

  • cmake -B build -DCMAKE_BUILD_TYPE=Debug -DENABLE_TESTS=ON -G 'Unix Makefiles' -S. succeeded.
  • cmake --build build --target unittests built successfully (including param, param_test, bruteforce, algorithm_test).
  • Ran the targeted filter [ut][param][schema],[ut][BruteForceParameter],[brute_force_threshold]:
    All tests passed (17 assertions in 6 test cases)
    
  • make fmt (clang-format v15) clean.

Signed-off-by: claude <claude@anthropic.com>
Signed-off-by: LHT129 <tianlan.lht@antgroup.com>

@LHT129 LHT129 added kind/improvement Optimizations, UX polish, or minor improvements 性能优化、体验打磨或细节改良 version/1.0 labels Jun 30, 2026
@mergify

mergify Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 All 2 merge protections satisfied — ready to merge.

Show 2 satisfied protections

🟢 Require kind label

  • label~=^kind/

🟢 Require version label

  • label~=^version/

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a declarative, macro-driven parameter schema framework under src/param to automate JSON parsing, serialization, and compatibility checks, migrating BruteForceParameter as the first user. While the design is a great improvement, there are two key issues: first, BruteForceParameter::FromJson fails to call schema().CheckUnknownKeys(json), meaning unknown keys are still silently accepted; second, the Range validator's use of static_cast<T> can lead to silent overflow, underflow, or truncation bugs when comparing different numeric types.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/algorithm/bruteforce/bruteforce_parameter.cpp
Comment thread src/param/validators.h Outdated
@LHT129 LHT129 self-assigned this Jun 30, 2026
@LHT129 LHT129 force-pushed the opencode/pr1-param-schema-bruteforce branch 2 times, most recently from c632815 to e85f1d2 Compare July 1, 2026 02:52
Copilot AI review requested due to automatic review settings July 1, 2026 02:52
@LHT129 LHT129 force-pushed the opencode/pr1-param-schema-bruteforce branch from e85f1d2 to b9f2105 Compare July 1, 2026 02:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@LHT129 LHT129 force-pushed the opencode/pr1-param-schema-bruteforce branch from b9f2105 to 357d56a Compare July 1, 2026 03:53
Copilot AI review requested due to automatic review settings July 1, 2026 04:28
@LHT129 LHT129 force-pushed the opencode/pr1-param-schema-bruteforce branch from 357d56a to 4b4b164 Compare July 1, 2026 04:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…ruteForceParameter

Add a declarative schema framework under src/param/ to automate JSON
parsing, serialization, and compatibility checks for parameter classes.
Migrate BruteForceParameter as the first user.

New files:
- src/param/validators.h: Range<T> validator with type-safe comparisons
- src/param/schema.h: Schema and VSAG_PARAM* macro definitions
- src/param/schema.cpp: Schema implementation
- src/param/schema_test.cpp: Schema framework unit tests
- src/param/CMakeLists.txt: Build configuration

Modified files:
- src/algorithm/bruteforce/bruteforce_parameter.{h,cpp}: Use schema
- src/inner_string_params.h: Add INDEX_BRUTE_FORCE to DEFAULT_MAP
- src/CMakeLists.txt: Add param subdirectory
- tests/CMakeLists.txt: Add param_test to unittests

Fixes CI failure: LazyHGraph test was failing because DEFAULT_MAP
was missing INDEX_BRUTE_FORCE entry, causing template expansion to
produce literal string "{INDEX_BRUTE_FORCE}" instead of "brute_force".

Signed-off-by: LHT129 <tianlan.lht@antgroup.com>
@LHT129 LHT129 force-pushed the opencode/pr1-param-schema-bruteforce branch from 4b4b164 to 6f25428 Compare July 2, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/improvement Optimizations, UX polish, or minor improvements 性能优化、体验打磨或细节改良 module/testing size/XL version/1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants