Suppress flaky tests errors in AzDO UI

[paulmedynski]

Description

This PR adds distinct flaky test targets to build.proj that run the flaky tests and ignore all errors entirely. This suppresses spurious errors/warnings from the flaky tests in the Azure DevOps UI for pipeline runs, and avoids the orange (!) and yellow status throughout the various pipelines. We know the flaky tests will fail - we don't need to see it on every pipeline run. Seeing all of their errors risks hiding/masking other legitimate problems that should be noticed.

Testing

Normal PR/CI runs will confirm.

[Copilot]

Pull request overview

This PR modifies the Azure DevOps pipeline configuration to suppress spurious errors and warnings from flaky tests in the pipeline UI. The changes implement two distinct strategies depending on the pipeline context:

Changes:

• Created new dedicated MSBuild targets (RunFlakyUnitTests, RunFlakyFunctionalTests, RunFlakyManualTests) in build.proj that use IgnoreExitCode and IgnoreStandardErrorWarningFormat to suppress error output
• Updated the PR/CI pipeline template (run-all-tests-step.yml) to use the new flaky test targets and removed explicit filter parameters
• Removed flaky test execution entirely from the MDS Official pipeline templates (netfx and netcore), and updated regular tests to exclude flaky tests
• Added whitespace cleanup across multiple pipeline files

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
eng/pipelines/common/templates/steps/run-all-tests-step.yml	Updated flaky unit test tasks to use new RunFlakyUnitTests target; removed explicit filter parameters; cleaned up trailing whitespace
eng/pipelines/common/templates/steps/build-and-run-tests-netfx-step.yml	Removed separate flaky test tasks; added filters to regular tests to exclude flaky tests; added pipeline identifier comment
eng/pipelines/common/templates/steps/build-and-run-tests-netcore-step.yml	Removed separate flaky test tasks; added filters to regular tests to exclude flaky tests; added pipeline identifier comment
eng/pipelines/common/templates/jobs/run-tests-package-reference-job.yml	Whitespace cleanup and added pipeline identifier comment
build.proj	Added new MSBuild targets for running flaky tests with error suppression; updated comments to clarify non-flaky test targets

Comments suppressed due to low confidence (1)

eng/pipelines/common/templates/steps/run-all-tests-step.yml:145

• The flaky functional tests task is still using the old target name -t:RunFunctionalTests with a filter instead of the new dedicated target -t:RunFlakyFunctionalTests. This is inconsistent with the changes in the Linux/macOS section (line 242) and the build.proj file, where new dedicated targets for flaky tests were created. The target should be changed to -t:RunFlakyFunctionalTests and the -p:Filter="category=flaky" line should be removed to match the pattern used for unit tests in this same file.

          -t:RunFunctionalTests
          -p:TF=${{ parameters.targetFramework }}
          -p:TestSet=${{ parameters.testSet }}
          -p:ReferenceType=${{ parameters.referenceType }}
          -p:MdsPackageVersion=${{ parameters.mdsPackageVersion }}
          -p:Filter="category=flaky"

[paulmedynski]

Official builds don't need to be running flaky tests. We currently don't run any tests in the official MDS builds anyway (an issue we should probably fix).

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[paulmedynski]

Abandoning this effort. I think we want to take a different approach to flaky tests.