fix(nvca): preserve MiniService spec when saving workload config - #626
Open
sbaum1994 wants to merge 2 commits into
Open
fix(nvca): preserve MiniService spec when saving workload config#626sbaum1994 wants to merge 2 commits into
sbaum1994 wants to merge 2 commits into
Conversation
Serialize workload config through the MiniService compatibility wire type and persist it with a merge patch so unrelated spec fields retain their values and field ownership. Closes #625 Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
📝 WalkthroughWalkthroughThe change adds ChangesWorkload configuration persistence
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
sbaum1994
marked this pull request as ready for review
August 3, 2026 22:37
kristinapathak
approved these changes
Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Customer Summary
NVCA now reads and saves per-workload configuration without removing required MiniService fields, allowing affected Helm workloads to continue installation.
TL;DR
Persist
spec.workloadConfigthrough the MiniService compatibility serializer and replace the partial server-side apply with a merge patch based on the complete existing object.Additional Details
MiniServiceSpechas custom JSON compatibility logic for canonical and legacy ICMS request names. The newerWorkloadConfigfield was missing from that wire representation, so NVCA could neither serialize nor deserialize it.saveWorkloadConfigalso constructed a partial MiniService and force-applied it with the controller's existing field manager. Its serialized zero values replaced required fields such asspec.namespaceandspec.icmsRequestName, leaving reconciliation terminally stuck.This change:
WorkloadConfigto the compatibility JSON representation in both directions.spec.workloadConfigchanges.No dependencies or generated API types changed.
Observed test sequence
The AWS BYOC test exercised this sequence:
nvcfWorkloadConfig.featureFlags.StatusByWorkerReadiness=true.nvcf-workload-configcontrol ConfigMap plus the workload objects.MiniService.spec.workloadConfigbefore applying the remaining workload objects.ACTIVE, then establish a successful inference baseline.The observed failure occurred at step 4, before workload application, baseline inference, or cordon/drain. The workload-config patch emptied
spec.namespaceandspec.icmsRequestNamewhile failing to persistspec.workloadConfig. The next reconcile returnedminiservice <name> has no namespace, the MiniService remainedInstalling, and the function remainedDEPLOYINGuntil the test timeout.For the Reviewer
Please focus on:
pkg/apis/nvca/v1alpha1/miniservice_json.gofor compatibility serialization completeness.internal/miniservice/reconcile.gofor the field-ownership change from server-side apply to merge patch.For QA
Local verification completed:
The full affected package run used Kubernetes 1.34.1 envtest assets and passed. Changed-code lint reported
0 issues. The unfiltered full-tree lint currently reports 36 pre-existing findings onorigin/main; none are on changed lines.Runtime QA is still needed before this draft is ready to merge. Build and deploy the patched NVCA, install a Helm workload that renders
nvcf-workload-configwithStatusByWorkerReadiness: true, and verify:spec.workloadConfigcontains the flag.Issues
Fixes #625
Checklist