[Eval framework] Document current eval coverage and gaps

[helen229]

Parent workstream: #16345
Parent epic: #16344

Goal

Publish and maintain a stakeholder-readable inventory of Azure SDK Agent eval coverage so teams can see what is tested today, identify gaps, and prioritize new scenarios.

Scope

• Inventory current eval surfaces: MCP tool-selection, workflow scenarios, skill trigger/routing, skill capability, mock, and live tiers.
• Map covered skills, public MCP tools, workflow scenarios, repositories, and test tiers.
• Distinguish deterministic coverage checks from model-graded behavioral coverage.
• Document known gaps, exclusions, owners, and the issue tracking each gap.
• Include completed versus pending framework capabilities and links to representative pipeline results.
• Define how the inventory stays current as skills/tools/scenarios are added.
• Share the document with the Shanga team, Laurent, Justin, and other eval stakeholders for review.

Deliverable

A checked-in document in azure-sdk-tools linked from the eval README and epic #16344, with tables for:

1. Eval type and purpose.
2. Skills and scenarios currently covered.
3. MCP tool and mock-handler coverage.
4. Cross-repository coverage.
5. Known gaps and follow-up issues.

Acceptance criteria

• Current eval types and tiers are documented.
• Covered skills, tools, and workflow scenarios are listed.
• Missing/partial coverage is explicit and linked to issues.
• Specs and language-repo rollout status is visible.
• Stakeholder review feedback is incorporated.
• A maintenance owner/process is documented.

Related

• [Eval framework] Enforce skill, MCP tool, and mock-handler eval coverage #16345 — automated coverage enforcement
• Skill eval suite: Fix and raise the bar  #15871 — per-skill eval quality
• [Eval framework] Complete cross-repo support and pilot in azure-rest-api-specs #16347 — specs-repo pilot
• [Eval framework] Roll out the shared eval pipeline to Azure SDK language repositories #16348 — language-repo rollout