Flash backend, PyOCD and MPBuild integration

[Josverl]

This PR introduces

• a new flash backend architecture to make it simpler to add additional flash methods without needing spagetti code.
• Add a pyOCD backend with dynamic target detection
• Addresses known dependency issues with libusb on Windows for Python 3.14.
• Integrates mpbuild.

This is inspired by, and an alternative to #36 and #37

[codecov]

Codecov Report

❌ Patch coverage is 73.02885% with 561 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.63%. Comparing base (98c1e0b) to head (25c2e41).

Files with missing lines	Patch %	Lines
mpflash/flash/builtins/pyocd/flash.py	40.12%	174 Missing and 14 partials ⚠️
mpflash/flash/builtins/pyocd/core.py	66.40%	93 Missing and 35 partials ⚠️
mpflash/build.py	78.76%	31 Missing and 24 partials ⚠️
mpflash/flash/__init__.py	78.90%	18 Missing and 9 partials ⚠️
mpflash/cli_pyocd.py	82.19%	14 Missing and 12 partials ⚠️
mpflash/cli_flash.py	77.63%	10 Missing and 7 partials ⚠️
mpflash/flash/builtins/uf2_backend.py	61.53%	12 Missing and 3 partials ⚠️
mpflash/bootloader/registry.py	77.77%	8 Missing and 4 partials ⚠️
mpflash/bootloader/activate.py	56.52%	5 Missing and 5 partials ⚠️
mpflash/flash/builtins/dfu_backend.py	85.50%	5 Missing and 5 partials ⚠️
... and 14 more

Additional details and impacted files

@@             Coverage Diff             @@
##           preview      #96      +/-   ##
===========================================
+ Coverage    75.60%   75.63%   +0.03%     
===========================================
  Files           54       74      +20     
  Lines         3033     4978    +1945     
  Branches       489      809     +320     
===========================================
+ Hits          2293     3765    +1472     
- Misses         621      964     +343     
- Partials       119      249     +130     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR refactors MPFlash’s flashing workflow into a pluggable backend architecture, adds an opt-in pyOCD backend (with probe/target handling), and integrates mpbuild-based local firmware building—alongside new unit/integration and optional hardware-in-the-loop tests.

Changes:

• Introduces a backend registry (FlashBackend, FlashContext, select_backend) and routes flashing through it.
• Adds a pyOCD backend + CLI commands and options (--method pyocd, --probe-id, --target, CMSIS pack install).
• Integrates mpbuild into the flash CLI flow and adds HIL test scaffolding + udev rules/docs for Linux USB permissions.

Reviewed changes

Copilot reviewed 82 out of 88 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tests/test_build.py	Tests mpbuild integration helpers
tests/mpremoteboard/test_mprb.py	Adds mpremote reconnect/parse tests
tests/integration/test_cli_integration.py	CLI integration tests for pyOCD flags
tests/hw/test_hw_backends.py	Adds hardware smoke tests per backend
tests/hw/conftest.py	HIL fixtures gated by env vars
tests/flash/test_worklist_refactored.py	Tests new firmware ranking behavior
tests/flash/test_uf2_windows.py	Updates UF2 import paths
tests/flash/test_uf2_mac.py	Updates UF2 import paths
tests/flash/test_uf2_linux.py	Updates UF2 import paths (commented)
tests/flash/test_uf2_boardid.py	Updates UF2 import paths
tests/flash/test_stm32_dfu.py	Updates DFU import paths
tests/flash/test_registry.py	Tests backend registry + selection rules
tests/flash/test_flash_uf2_A.py	Updates UF2 mocking paths
tests/flash/test_flash_tasks.py	Adds path normalization + backend compatibility tests
tests/flash/test_flash_esp.py	Updates esptool import paths
tests/flash/test_flash_1.py	Updates bootloader/flash patch targets
tests/flash/test_boot_touch1200.py	Updates touch1200 import paths
tests/fixtures/mock_pyocd_data.py	Adds pyOCD mock datasets/MCUs
tests/db/test_loader.py	Replaces mock with stdlib mock
tests/db/test_core.py	Replaces mock with stdlib mock
tests/conftest.py	Loads .env, adds pyOCD fixtures, resets probe registry
tests/cli/test_cli_flash.py	Adds CLI behavior coverage for build/serial filtering
tests/cli/test_cli_download.py	Replaces mock with stdlib mock
tests/bootloader/test_bootloader_registry.py	Tests bootloader activator registry
tests/basicgit_test.py	Replaces mock with stdlib mock
tests.md	Documents HIL markers/env vars and usage
pyproject.toml	Adds python-dotenv, defines pyocd/build extras, adds markers
mpflash/udev_rules/README.md	Adds udev rules documentation
mpflash/udev_rules/65-mpflash-stm32-dfu.rules	Adds STM32 DFU udev rule
mpflash/udev_rules/50-picoprobe.rules	Adds Picoprobe udev rules
mpflash/udev_rules/50-cmsis-dap.rules	Adds CMSIS-DAP udev rules
mpflash/udev_rules/49-wch-link.rules	Adds WCH-Link udev rule
mpflash/udev_rules/49-vtlinkii.rules	Adds ESLinkII udev rule
mpflash/udev_rules/49-stlinkv3.rules	Adds STLinkV3 udev rules
mpflash/udev_rules/49-stlinkv2.rules	Adds STLinkV2 udev rule
mpflash/udev_rules/49-stlinkv2-1.rules	Adds STLinkV2-1 udev rules
mpflash/mpremoteboard/init.py	Improves parsing/logging + restart probing behavior
mpflash/flash/worklist.py	Adds firmware ranking + path normalization
mpflash/flash/services.py	Adds backend services bundle (bootloader, platform, reenum)
mpflash/flash/registry.py	Adds backend registry + entry-point discovery
mpflash/flash/context.py	Adds FlashContext / FlashResult / Reason
mpflash/flash/builtins/uf2/wsl2.py	Adds UF2 support for WSL2 mounts
mpflash/flash/builtins/uf2/windows.py	Adds UF2 Windows helper module
mpflash/flash/builtins/uf2/volume.py	Adds platform-aware UF2 volume handling
mpflash/flash/builtins/uf2/uf2disk.py	Adds UF2 disk helper class
mpflash/flash/builtins/uf2/macos.py	Adds UF2 macOS helper module
mpflash/flash/builtins/uf2/linux.py	Adds UF2 Linux helper module
mpflash/flash/builtins/uf2/boardid.py	Adds UF2 board-id parser
mpflash/flash/builtins/uf2/init.py	Documents UF2 module role + retains UF2 logic
mpflash/flash/builtins/uf2_backend.py	Adds UF2 backend wrapper + registration
mpflash/flash/builtins/pyocd/probes.py	Adds pyOCD probe abstraction/registry
mpflash/flash/builtins/pyocd/init.py	Adds pyOCD package marker
mpflash/flash/builtins/pyocd_backend.py	Adds pyOCD backend wrapper + selection rules
mpflash/flash/builtins/esptool_backend.py	Adds esptool backend wrapper + registration
mpflash/flash/builtins/esp/init.py	Documents esp backend as internal
mpflash/flash/builtins/dfu/stm32_dfu.py	Adds DFU polling + udev guidance integration
mpflash/flash/builtins/dfu/init.py	Documents DFU module role
mpflash/flash/builtins/dfu_backend.py	Adds DFU backend wrapper + readiness check
mpflash/flash/builtins/init.py	Registers built-in backends on import
mpflash/flash/base.py	Adds FlashBackend ABC contract
mpflash/flash/init.py	Replaces port-ladder flashing with backend dispatcher
mpflash/custom/naming.py	Minor log formatting cleanup
mpflash/common.py	Adds FlashMethod enum + udev help builder
mpflash/cli_pyocd.py	Adds pyOCD CLI commands
mpflash/cli_plugins.py	Adds mpflash plugins backend listing
mpflash/cli_main.py	Loads .env early + registers new CLI modules
mpflash/cli_flash.py	Adds --method pyocd, probe/target flags, --build integration
mpflash/bootloader/registry.py	Adds bootloader activator registry
mpflash/bootloader/detect.py	Routes bootloader readiness via backend readiness probes
mpflash/bootloader/builtins/touch1200.py	Converts touch1200 into activator + registers it
mpflash/bootloader/builtins/mpy.py	Adds MicroPython activator implementation
mpflash/bootloader/builtins/manual.py	Converts manual activator + registers it
mpflash/bootloader/builtins/init.py	Imports built-in activators
mpflash/bootloader/base.py	Adds BootloaderActivator ABC
mpflash/bootloader/activate.py	Orchestrates bootloader entry via activator registry
justfile	Adds pyocd extra + HIL recipes
docs/stm32_udev_rules.md	Updates DFU udev rules documentation
.vscode/settings.json	Updates VS Code settings + MCP sampling config
.vscode/mcp.json	Adds local debug MCP server config
.github/workflows/pytest_mpflash.yml	Installs all extras in CI
.env	Adds HIL env var examples

[Josverl]

@pi-anl,

I think you mentioned

I'm not sure what your appetite is for large changes

I'm afraid its gotten bigger as I had progressed with restructuring the database before I got around to this.
I want to expand to support more MCU families , and found that it got rather messy in all the decision trees.
So I tried to refactor into a (supposedly) pluggable flash-backend architecture ,
PyOCD is now refactored into one of the built-in plugins

$ mpflash plugins
                                      Flash backends (wsl2)                                      
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Name    ┃ Ports          ┃ Formats          ┃ Platforms       ┃ Prio ┃ Bootloader ┃ Available ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ uf2     │ nrf, rp2, samd │ .uf2             │ linux, macos,   │   10 │    yes     │    yes    │
│         │                │                  │ windows, wsl2   │      │            │           │
│ dfu     │ stm32          │ .dfu, .bin       │ linux, macos,   │   10 │    yes     │    yes    │
│         │                │                  │ windows, wsl2   │      │            │           │
│ esptool │ esp32, esp8266 │ .bin             │ linux, macos,   │   10 │     no     │    yes    │
│         │                │                  │ windows, wsl2   │      │            │           │
│ pyocd   │ *              │ .bin, .hex,      │ linux, macos,   │  -10 │     no     │    yes    │
│         │                │ .elf, .axf       │ windows, wsl2   │      │            │           │
└─────────┴────────────────┴──────────────────┴─────────────────┴──────┴────────────┴───────────┘

I tested PyOCD on a limited number of probes and it works on my machine , but I would appreciate if you can check if it still works for your initial intent.

I also added the mpbuild integration, and added a --clean option and fixed several minor gotchas wrt to variants and versions , but that seems to work quite well.

I`m still cleaning things up , and will need to clean up the commits and write some documentation on the plugin stuff to make that useful.