chore(xtest): Shared Scenario/Instance Pydantic schema in otdf-sdk-mgr#450
chore(xtest): Shared Scenario/Instance Pydantic schema in otdf-sdk-mgr#450dmihalcik-virtru wants to merge 5 commits into
Conversation
…X-3302)
Introduces otdf_sdk_mgr.schema as the canonical Pydantic v2 model layer
for the multi-instance test harness. Both otdf-sdk-mgr and otdf-local
will read scenarios.yaml / instance.yaml through these models so the
on-disk YAML format has exactly one definition.
Models:
- SourceRef, PlatformPin, KasPin, SdkPin (with mutually-exclusive
dist|source|image validation on the platform/KAS pins)
- PortsConfig, Metadata, Fixtures
- Instance (apiVersion/kind/metadata/platform/ports/kas/...)
- ScenarioSdks (encrypt + decrypt maps mirroring xtest's
--sdks-encrypt / --sdks-decrypt convention)
- Suite (pytest select + flags)
- Scenario (composes Instance + ScenarioSdks + Suite)
Includes load_scenario / load_instance / dump_instance helpers and a
`python -m otdf_sdk_mgr.schema validate <path>` CLI entry that dispatches
on `kind:` so the same command validates both Scenario and Instance YAML.
Adds pydantic + ruamel.yaml to otdf-sdk-mgr's deps and a 6-test smoke
suite covering round-trips, pin validation, encrypt/decrypt union dedup,
and unknown-field rejection.
Refs: https://virtru.atlassian.net/browse/DSPX-3302
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new schema module for OpenTDF scenarios and instances, utilizing Pydantic models to ensure a canonical YAML definition across different tools. The changes include adding pydantic and ruamel.yaml as dependencies, implementing load/dump logic, and providing comprehensive smoke tests. The review feedback suggests several improvements to the implementation, specifically regarding the enforcement of UTF-8 encoding for file operations, the removal of redundant YAML configuration settings, and the use of more specific exception handling for validation and parsing errors.
There was a problem hiding this comment.
Pull request overview
First PR of a five-part stack introducing a multi-instance test harness for OpenTDF. This change adds only the shared Pydantic v2 schema in otdf-sdk-mgr (no consumers yet), plus a small CLI validator and unit tests. It establishes the on-disk shape for scenarios.yaml / instance.yaml so downstream PRs in the stack (otdf-local, xtest/conftest.py, Claude plugin) can import a single canonical definition.
Changes:
- Adds
otdf_sdk_mgr.schemawith strict (extra="forbid") v2 models:Scenario,Instance,PlatformPin,KasPin,SdkPin,ScenarioSdks,Suite, plus helpersload_scenario/load_instance/dump_instanceand apython -m otdf_sdk_mgr.schema validate <path>entrypoint. - Adds
pydantic>=2.6.0andruamel.yaml>=0.18.0to project dependencies (with correspondinguv.lockentries). - Adds 6 schema unit tests covering scenario round-trip, pin "exactly one source" invariant, KasPin features pass-through, SDK union, instance dump/load, and unknown-field rejection.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| otdf-sdk-mgr/pyproject.toml | Adds pydantic and ruamel.yaml runtime dependencies. |
| otdf-sdk-mgr/uv.lock | Auto-generated lockfile updates for the new dependencies (pydantic, pydantic-core, annotated-types, typing-inspection, ruamel-yaml). |
| otdf-sdk-mgr/src/otdf_sdk_mgr/schema.py | New module with Pydantic models, YAML load/dump helpers, and a validate CLI entry point. |
| otdf-sdk-mgr/tests/test_schema.py | Smoke tests for the new schema (round-trip, pin invariants, union, extra-forbid rejection). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… format xtest's --sdks / --sdks-encrypt / --sdks-decrypt accept whitespace- separated `sdk@version` tokens after #446 (e.g. `go@v0.24.0`, `go@main`, `go@*`). The version segment must match an actual directory under `xtest/sdk/<lang>/dist/`. Scenario version fields can be aliases (`lts`, `tip`) that only resolve to a concrete dist name once `otdf-sdk-mgr install scenario` runs, so we can't translate scenarios → pytest args from the scenario YAML alone. Adds two helpers so the scenario→pytest bridge has one canonical implementation: installed_json_for(scenario_path): The conventional sibling file `otdf-sdk-mgr install scenario` writes. `xtest/scenarios/x.yaml` → `xtest/scenarios/x.installed.json`. scenario_to_pytest_sdks(scenario, installed_json_path) -> dict: Returns `{"encrypt": ["go@v0.24.0", ...], "decrypt": [...]}`, reading the dist directory names recorded in installed.json. Raises FileNotFoundError with a `run install scenario first` hint when the record is missing (aliases can't be passed verbatim to xtest, so a clean error beats a confusing pytest failure). Raises ValueError when the scenario references an SDK the install record doesn't cover. Both `otdf-local scenario run` and `xtest/conftest.py`'s `--scenario`-default path will switch to this helper in the following PRs so they no longer drop the version when forwarding to pytest. Refs: https://virtru.atlassian.net/browse/DSPX-3302 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
dmihalcik-virtru
changed the title
Summary
First PR in a five-part stack that introduces a multi-instance test harness and a Claude plugin for OpenTDF bug reproduction. This PR adds only the shared Pydantic schema in
otdf-sdk-mgr— no consumers yet.otdf_sdk_mgr.schemawith v2 models:Scenario,Instance,PlatformPin,KasPin,SdkPin,ScenarioSdks,Suite, etc.ScenarioSdks.encrypt/.decryptmirror xtest's existing--sdks-encrypt/--sdks-decryptconvention so a→b-only scenarios are first-class.python -m otdf_sdk_mgr.schema validate <path>validates either a Scenario or an Instance file based on itskind:.pydantic+ruamel.yamltootdf-sdk-mgr/pyproject.toml.Stack
install scenarioinotdf-sdk-mgr(builds on this)otdf-localmulti-instance refactor + new CLI subcommandsxtest/conftest.pyintegration (--scenario,--instance).claude/skills/, settings, plugin manifest)Test plan
cd otdf-sdk-mgr && uv run pytest tests/test_schema.py— all 6 passuv run python -m otdf_sdk_mgr.schema validate <path>accepts a valid scenarios.yaml and rejects unknown fieldsJira: https://virtru.atlassian.net/browse/DSPX-3302
🤖 Generated with Claude Code