Skip to content

fix(nvca): preserve MiniService spec when saving workload config - #626

Open
sbaum1994 wants to merge 2 commits into
mainfrom
sbaum/fix/miniservice-workload-config-patch
Open

fix(nvca): preserve MiniService spec when saving workload config#626
sbaum1994 wants to merge 2 commits into
mainfrom
sbaum/fix/miniservice-workload-config-patch

Conversation

@sbaum1994

@sbaum1994 sbaum1994 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Customer Summary

NVCA now reads and saves per-workload configuration without removing required MiniService fields, allowing affected Helm workloads to continue installation.

TL;DR

Persist spec.workloadConfig through the MiniService compatibility serializer and replace the partial server-side apply with a merge patch based on the complete existing object.

Additional Details

MiniServiceSpec has custom JSON compatibility logic for canonical and legacy ICMS request names. The newer WorkloadConfig field was missing from that wire representation, so NVCA could neither serialize nor deserialize it.

saveWorkloadConfig also constructed a partial MiniService and force-applied it with the controller's existing field manager. Its serialized zero values replaced required fields such as spec.namespace and spec.icmsRequestName, leaving reconciliation terminally stuck.

This change:

  • Adds WorkloadConfig to the compatibility JSON representation in both directions.
  • Computes a merge patch between immutable copies of the complete MiniService, so only spec.workloadConfig changes.
  • Adds regression coverage for JSON round trips, merge-patch contents, and preservation of namespace, request name, and Helm configuration.

No dependencies or generated API types changed.

Observed test sequence

The AWS BYOC test exercised this sequence:

  1. Create a Helm function whose deployment configuration sets nvcfWorkloadConfig.featureFlags.StatusByWorkerReadiness=true.
  2. Render the chart successfully, including the nvcf-workload-config control ConfigMap plus the workload objects.
  3. Decode and remove that control ConfigMap from the objects that will be applied to the workload namespace.
  4. Persist the decoded flag into MiniService.spec.workloadConfig before applying the remaining workload objects.
  5. Wait for the function to become ACTIVE, then establish a successful inference baseline.
  6. Cordon and drain one decode-worker node and verify serving and health behavior throughout the degraded window.

The observed failure occurred at step 4, before workload application, baseline inference, or cordon/drain. The workload-config patch emptied spec.namespace and spec.icmsRequestName while failing to persist spec.workloadConfig. The next reconcile returned miniservice <name> has no namespace, the MiniService remained Installing, and the function remained DEPLOYING until the test timeout.

For the Reviewer

Please focus on:

  • pkg/apis/nvca/v1alpha1/miniservice_json.go for compatibility serialization completeness.
  • internal/miniservice/reconcile.go for the field-ownership change from server-side apply to merge patch.

For QA

Local verification completed:

go test ./internal/miniservice ./pkg/apis/nvca/v1alpha1 -run '^(TestSaveWorkloadConfigPreservesExistingSpec|TestMiniServiceSpecRoundTripsWorkloadConfig)$' -count=1 -ldflags '...version=v25.8.0'
go test ./internal/miniservice/... ./pkg/apis/nvca/v1alpha1/... -count=1 -ldflags '...version=v25.8.0'
golangci-lint run -c .golangci.yml --new-from-rev=origin/main

The full affected package run used Kubernetes 1.34.1 envtest assets and passed. Changed-code lint reported 0 issues. The unfiltered full-tree lint currently reports 36 pre-existing findings on origin/main; none are on changed lines.

Runtime QA is still needed before this draft is ready to merge. Build and deploy the patched NVCA, install a Helm workload that renders nvcf-workload-config with StatusByWorkerReadiness: true, and verify:

  • spec.workloadConfig contains the flag.
  • Existing MiniService spec fields remain unchanged.
  • Reconciliation proceeds past installation and applies the workload objects.
  • Function cleanup removes its namespace.

Issues

Fixes #625

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Serialize workload config through the MiniService compatibility wire type and persist it with a merge patch so unrelated spec fields retain their values and field ownership.

Closes #625

Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds WorkloadConfig to MiniService compatibility JSON handling and updates saveWorkloadConfig to use a merge patch from the existing MiniService. Tests cover JSON round trips and preservation of existing specification fields.

Changes

Workload configuration persistence

Layer / File(s) Summary
WorkloadConfig compatibility serialization
src/compute-plane-services/nvca/pkg/apis/nvca/v1alpha1/miniservice_json.go, src/compute-plane-services/nvca/pkg/apis/nvca/v1alpha1/miniservice_json_test.go
The compatibility JSON payload now includes WorkloadConfig. Marshal and unmarshal operations preserve workload feature flags.
Merge-patch workload configuration persistence
src/compute-plane-services/nvca/internal/miniservice/reconcile.go, src/compute-plane-services/nvca/internal/miniservice/reconcile_test.go
saveWorkloadConfig now uses a merge patch based on the existing MiniService, deep-copies the workload configuration, and updates the caller after success. Tests verify that existing specification fields remain unchanged.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: kristinapathak

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes address [#625] by preserving existing MiniService fields, serializing WorkloadConfig, and adding regression coverage.
Out of Scope Changes check ✅ Passed All changes directly support workload configuration persistence and regression coverage described in [#625].
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes the MiniService spec preservation fix.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sbaum/fix/miniservice-workload-config-patch

Comment @coderabbitai help to get the list of available commands.

@sbaum1994
sbaum1994 marked this pull request as ready for review August 3, 2026 22:37
@sbaum1994
sbaum1994 requested a review from a team as a code owner August 3, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVCA workload config persistence prunes required MiniService spec fields

2 participants