fix: validate unified agent definitions in doctor#9254
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
📋 Prioritization NoteThanks for the contribution! The linked issue isn't in the current milestone yet. |
There was a problem hiding this comment.
Pull request overview
Updates doctor validation to use the same unified agent-definition resolver as runtime and deployment.
Changes:
- Supports inline, referenced, and legacy definitions.
- Uses source-neutral diagnostic wording.
- Expands tests for resolution precedence and multi-service failures.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
checks_project.go |
Resolves and validates unified definitions. |
checks_project_test.go |
Adds definition-resolution test coverage. |
checks_manual_env.go |
Updates skip wording. |
checks_manual_env_test.go |
Updates skip assertion. |
checks_local.go |
Registers the renamed check. |
checks_local_test.go |
Updates check metadata assertion. |
doctor_format_test.go |
Updates capitalization coverage. |
jongio
left a comment
There was a problem hiding this comment.
The switch to project.LoadAgentDefinition here only inspects the returned error, so an inline definition whose kind is anything other than hosted passes this check without being validated. In agentDefinitionFromStruct (internal/project/agent_definition.go:596-598) a non-hosted kind returns (zero, false, nil) before the validation block runs. That covers both workflow (a valid kind, whose name check is then skipped) and unknown values like nonsense.
The retained disk fallback still validates strictly through ValidateAgentDefinition, which rejects unknown kinds, so an invalid kind now fails when it lives in a legacy agent.yaml on disk but passes when it is inline in azure.yaml. Since doctor is a validation command, worth deciding whether it should reject a resolved-but-unvalidated non-hosted inline definition, or whether validating unknown/workflow kinds belongs in the shared resolver (which would also cover run and deploy). Not blocking, and the fuller fix is arguably out of scope for this alignment change.
The problem
Modern agent projects keep their agent definition inline in
azure.yaml. Deploy and run already understand this, butazd ai agent doctordidn't — it still went looking for a standaloneagent.yamlfile on disk and failed the check when it couldn't find one.The result: a perfectly valid project that deploys successfully would still get a red FAIL from doctor. That's misleading and makes people distrust the command (see #9223).
The fix
Point doctor's definition check at the same resolver that run and deploy use —
project.LoadAgentDefinition, added in #9149 — instead of readingagent.yamloff disk itself.The check now passes when a definition is found in any legitimate place:
azure.yamlazure.yamlagent.yaml/agent.ymlon diskIf a service has no definition anywhere, doctor still fails — but the message and suggestion no longer assume the old
agent.yamlfile.Everything else about the check is unchanged: same check ID (
local.agent-yaml-valid), same--local-onlybehavior, and it still aggregates failures across services in a stable, name-sorted order.Why this approach
Reusing the runtime resolver keeps doctor honest: "doctor passes" now means "deploy will resolve the same definition," so the two can't quietly drift apart. It also avoids maintaining a second, doctor-only YAML parser that we'd have to keep in sync by hand.
One side effect worth calling out (manual-env check)
Making the definition check pass for inline-only projects has a knock-on effect on the next check,
local.manual-env-vars:<service>/agent.yamlfrom disk. For inline projects it finds nothing and reports "no manual env vars are missing."This is expected, not a regression introduced here. Teaching manual-env to read the resolved definition is separate work tracked in #8710. The existing skip-cascade guard is kept, so when the definition check itself fails or skips, manual-env still skips as before (now with matching wording).
Testing
Unit tests:
go test ./internal/cmd/doctor ./internal/cmdpass, with new coverage for inline definitions, inline-over-stale-file precedence, and the service-name sort helper.I also built the extension and ran
azd ai agent doctor --local-onlyend-to-end against sample projects covering each definition source, to confirm azd really surfaces the inline definition to the check (not just hand-built fixtures):agent definition validresultazure.yaml, inline definition, noagent.yamlon disk (the #9223 case)agent.yamlon diskagent.yaml-only service (no inline definition)On the running extension I also confirmed the check ID and
--local-onlybehavior are unchanged, the failure suggestion no longer tells modern projects to create anagent.yaml, and the manual-env behavior change above is observable.Closes #9223.
Part of #8710.