Skip to content

Replace feature: field in QA YAML with first-class labels #604

@couimet

Description

@couimet

Why this came up

Discovered while working on issue #249 (PR-1 adopting the Unreleased pattern). The QA YAML schema has both feature: (a single string per TC) and labels: (a list per TC). With labels now driving real behavior in resolve-qa-labels.js (filtering via --label, --exclude-label, --assisted, --exclude-assisted) and feeding into generate-qa-issue.sh for grouped GitHub issue creation, the feature: field has become quasi-redundant. Each TC effectively gets categorized twice: once by feature: string, once by labels: list, and the two never cross-reference each other.

Current state of feature:

  • Every TC in qa-test-cases-unreleased.yaml has a feature: field. Examples: 'R-M Status Bar Menu', 'Bind to Destination', 'Bookmarks'.
  • Consumers of feature::
    • scripts/generate-qa-issue.sh (around line 113-120): groups TCs by ID prefix (e.g., bind-to-destination-001 → group key bind-to-destination) and picks the most common feature: value in each group to use as the group's display name. So feature: becomes a human label for ID-prefix groups.
    • scripts/resolve-qa-labels.js: does not appear to read feature: at all (it parses TC IDs, labels, automated status, non_automatable_reason). Worth confirming as part of scoping.
    • The yaml header schema doc enumerates feature: but does not say how it is used.
  • TC ID prefix (e.g., bind-to-destination) already encodes the same grouping as feature:. The redundancy is between the ID prefix and the feature: field, not between feature: and labels: exactly.

What labels: already gives us

  • Multi-dimensional categorization (a TC can carry cursor, ubuntu, requires-extensions, etc.).
  • Filter primitives: --label X, --exclude-label X, --assisted, --exclude-assisted.
  • Cross-cutting concerns (platform, IDE, prerequisites) that do not fit the single-axis feature: field.

Where the redundancy actually is

labels: is not a drop-in replacement for feature:. They serve different purposes:

  • feature: = single-value subsystem grouping ("which area of the product does this exercise")
  • labels: = multi-value cross-cutting tags ("what platform/setup/assistance dimension")

The real redundancy is feature: vs <feature-slug>-NNN TC ID. The ID already encodes the subsystem. feature: is a human-readable expansion of the slug. Removing feature: and deriving the human display name from the slug (e.g., bind-to-destinationBind to Destination via title-case + dehyphenation) eliminates the duplication. Labels stay as-is for cross-cutting tags.

Sketch of the change

Three potential shapes, ordered from most to least conservative:

  1. Derive display name from TC ID slug, keep labels: unchanged: drop feature:; update generate-qa-issue.sh to title-case the ID prefix for group display (bind-to-destinationBind to Destination). Labels untouched. Minimal scope. Risk: some current feature: strings have non-trivial words (R-M Status Bar Menu) that title-casing the slug would not reproduce; the script would need either a mapping table or accept slightly different display names.

  2. Convention: namespaced label for feature: keep one structured label per TC like feature:bind-to-destination alongside other tags. Resolver code stays roughly the same; display logic strips the feature: prefix. Allows multiple feature labels per TC (e.g., a TC that exercises both bind and bookmarks) which feature: cannot. Risk: another concept to remember.

  3. Drop feature: entirely without replacement: rely on the slug, no display niceness. Smallest schema. Risk: GitHub issues lose readable section headers.

Option 1 with a small static mapping table for the exceptions probably hits the best ratio of cleanup to risk, but the implementation should explore all three.

Migration considerations

  • The YAML file has 70+ TCs with feature: set. Whichever path is picked, the migration is mechanical and probably scriptable.
  • CLAUDE.md <qa-yaml> schema doc references feature: indirectly through QA001/QA002/etc. No rule appears to depend on feature: value semantics. Worth verifying.
  • BATS fixtures in tests/shell/ set feature: on test YAMLs. Those would need to stay valid against the new schema or get updated.
  • Downstream consumers: validate-qa-coverage.sh reads automated markers and TC IDs only — likely a no-op there. The release-testing-instructions script does not read feature: at all.

Open questions

  • Does resolve-qa-labels.js actually touch feature: anywhere? Verify before scoping.
  • Are there any external tools (Linear, project boards, release notes generation) that depend on the feature: string?
  • If we go with Option 1, do we accept that the GitHub issue section headers might read slightly differently from today, or do we maintain a mapping table to preserve current wording?
  • Does the deferred-version (Unreleased) pattern from Prepare for extension's 2.0.0 release #249 interact with this? Not believed to — labels and the placeholder are independent concerns.

Not in scope

  • Anything that requires shipping before 2.0.0. This is a quality-of-life cleanup, not a release blocker.
  • Changes to labels: semantics or the filter API. Those work fine as-is.
  • Renaming TC IDs (QA001 forbids that anyway).

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions