Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
deee9a6
Fix KiCad SWIG plugin: cross-platform launch, IPC auto-discovery, Qt …
Apr 1, 2026
6202372
Fix track/via extraction and board viewer rendering
Apr 1, 2026
8f88912
Add copper zone extraction and rendering
Apr 1, 2026
17dba92
feat: extract keepout rule areas from KiCad and block routing through…
Apr 1, 2026
283d52f
fix: draw keepout rule areas when show_keepouts checkbox is enabled
Apr 1, 2026
b232d83
feat: right-click context menu shows keepout constraints on PCB viewer
Apr 1, 2026
a7a7bb3
fix: downgrade per-paint-event INFO logs to DEBUG in draw methods
Apr 1, 2026
15889c4
fix: include orthoroute.json in build package and add to validation
Apr 1, 2026
70868a9
docs: update copilot-instructions with current codebase state
Apr 1, 2026
d8fd59b
perf: live-run GPU profiling — MULTI-LAUNCH overhead is real bottleneck
Apr 3, 2026
947797a
perf: reclassify cuda_dijkstra.py logging -- 210 INFO->DEBUG, documen…
Apr 3, 2026
98f1fa4
perf: enable persistent CUDA kernel -- 2x speedup (48min -> 25min)
Apr 3, 2026
31addf0
docs: update perf docs -- persistent kernel results + next bottleneck
Apr 3, 2026
dbaeac9
perf: bitmap GPU-resident + vectorized owner bitmap build (no net spe…
Apr 3, 2026
24b0b27
perf: vectorize _path_to_edges and commit_path (180x and 9x speedup)
Apr 3, 2026
4bdae97
logging: ORTHO_DEBUG controls log level, screenshots, and profile_time
Apr 3, 2026
5fd7f75
docs: add TODO for automated regression test on TestBackplane
Apr 3, 2026
1255f67
fix: 3 KiCad IPC bugs - retry on connection refused, get_open_documen…
Apr 5, 2026
30e637f
fix: pipeline.py imports, rrg.py attribute names, logging to ERROR-on…
Apr 5, 2026
b156dc4
feat: unified_pathfinder returns iteration_metrics for regression tes…
Apr 5, 2026
ce7bd51
docs: rewrite copilot-instructions.md; update contributing.md to refe…
Apr 5, 2026
288bee6
chore: add agent customizations, instructions, prompts, and skills fo…
Apr 5, 2026
ab42031
chore: add launch_kicad_debug.ps1 debug launch script and pytest.ini
Apr 5, 2026
76d194d
test: update TestBackplane with results from successful GPU routing r…
Apr 5, 2026
7b35fc3
test: add full test suite (was gitignored) + fix regression conftest …
Apr 5, 2026
7a3646e
test: un-ignore test_*.py inside tests/ folder; add all test files to…
Apr 5, 2026
cd801d2
test: regression - board load tests from .kicad_pcb, log_routing_resu…
Apr 5, 2026
15e1930
test: add TestLatticeSize group + log_lattice fixture; add Apr-5 perf…
Apr 5, 2026
c7caa92
test: parse end-of-run banners from log; 25 skipped -> 0 skipped
Apr 5, 2026
3d68559
docs: add full convergence trend table to Apr-5 baseline
Apr 5, 2026
8b38742
Add unit tests and update optimization baseline (April 8, 2026)
Apr 10, 2026
3054ac1
feat: Add comprehensive test suite with golden result baseline
Apr 10, 2026
591ac17
Fix KiCad pad coordinate transform and add headless regression runner
Apr 10, 2026
e836eb1
Add automated optimization workflow with smoke test validation
Apr 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/agents/routing-debugger.agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
description: "Use when: diagnosing routing failures, analyzing congestion, interpreting log files, reading PROFILE timings, explaining why nets failed to route, reviewing debug screenshots. Read-only analysis of logs, debug_output, and routing algorithm code."
name: Routing Debugger
tools: [read, search]
user-invocable: true
---

You are a read-only routing failure analyst for OrthoRoute. Your job is to diagnose why nets failed to route, identify congestion hotspots, and explain performance regressions — without modifying any source files.

## Scope

You may read from:
- `logs/` — `latest.log`, `run_<timestamp>.log`
- `debug_output/` — iteration screenshots, any JSON/CSV dumps
- `orthoroute/algorithms/manhattan/` — routing algorithm source for cross-referencing

You must NOT edit files, run the router, or suggest code changes. Report findings only.

## Workflow

When given a routing failure or performance question:

1. **Locate the relevant log** — prefer `logs/latest.log`; fall back to most recent timestamped log.

2. **Find the failure signature** — search for:
- `FAILED` or `UNROUTED` lines (unrouted nets at end of run)
- `[ITER N]` lines to establish per-iteration timing
- `[PROFILE]` lines for hot functions (`[PROFILE] func: Xms`)
- `WARNING` or `ERROR` lines for unexpected conditions
- `congestion_ratio` values across iterations (rising = converging, plateau = stuck)

3. **Identify root cause category:**

| Symptom | Likely cause |
|---------|-------------|
| Same nets fail every iteration | Structural congestion — insufficient routing channels |
| Net count drops then plateaus | Ripup/reroute loop stuck in local minimum |
| `[PROFILE] _build_owner_bitmap_for_fullgraph` high | Per-net bitmap rebuild (known issue — ~0.9ms × net-count) |
| Iter time spikes after N iterations | GPU memory pressure or fallback to CPU |
| `keepout` in failed net path | Net endpoint inside or adjacent to keepout area |
| `via` conflicts in log | Blind/buried via layer assignment conflicts |

4. **Report findings** as a structured summary:
- **Run summary**: total nets, routed %, iterations completed, total time
- **Failed nets**: list with net name and failure reason if discernible
- **Performance hotspots**: top `[PROFILE]` entries by time
- **Congestion trend**: `congestion_ratio` first → last, direction
- **Recommendations**: parameter changes to try (reference [docs/tuning_guide.md](../../docs/tuning_guide.md))

## Log Format Reference

```
[ROUTING START] nets=512 layers=32 grid=0.05mm
[ITER 1] routed=389/512 (76.0%) congestion_ratio=1.42 iter=11.2s total=11.2s
[PROFILE] _path_to_edges: 42ms
[PROFILE] commit_path: 18ms
[PROFILE] _build_owner_bitmap_for_fullgraph: 461ms ← watch this one
[ROUTING DONE] routed=501/512 (97.8%) total=143s
UNROUTED: GND_17, VCC_3, NET_204 ...
```

## Constraints

- Do **not** suggest changes to `unified_pathfinder.py` without the user explicitly asking.
- Do **not** run any commands. Read only.
- If a log is missing or `ORTHO_DEBUG` was not set, explain what to enable and stop: `$env:ORTHO_DEBUG = '1'`, then re-run.
- If asked to fix a bug, respond: "I'm a read-only analyst. Use the default agent to apply changes."
129 changes: 129 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# OrthoRoute — Agent Workspace Instructions

GPU-accelerated PCB autorouter for KiCad (PathFinder / Manhattan lattice, up to 32 layers, 3,200 pads). Research code transitioning to production. See [README](../README.md) and the [build log](https://bbenchoff.github.io/pages/OrthoRoute.html) for background.

---

## Agent Workflow — REQUIRED

**For every code or config change:**

1. **Edit** source files in the repo
2. **Sync** — run `.\copy_to_kicad.ps1` immediately after every change
3. **Launch KiCad** — run `.\launch_kicad_debug.ps1` (debug mode) or ask user to start KiCad
4. **Wait** — do nothing until the user reports back from KiCad
5. **Commit** only after explicit approval ("commit", "looks good", etc.)
6. **Never push** to remote without explicit instruction

```powershell
.\copy_to_kicad.ps1 # sync after every change
.\launch_kicad_debug.ps1 # launch KiCad with ORTHO_DEBUG=1 for testing
```

---

## Build & Run

```powershell
pip install -r requirements.txt # Install deps
python build.py # Build: build/OrthoRoute-1.0.0.zip
python build.py --deploy # Build + install to KiCad plugin folder
.\copy_to_kicad.ps1 # Fast dev sync (Python + config only)
python main.py plugin # Run with GUI (KiCad must be open)
python main.py --test-manhattan # Built-in GUI acceptance test
pytest tests/ # Run test suite (unit + regression)
```

**Debug mode:** `.\launch_kicad_debug.ps1` or `$env:ORTHO_DEBUG = '1'` → full DEBUG log + screenshots in `debug_output/`.
Tear down: `Remove-Item Env:ORTHO_DEBUG, Env:ORTHO_SCREENSHOT_FREQ, Env:ORTHO_SCREENSHOT_SCALE -ErrorAction SilentlyContinue`

**Testing:** 167 unit tests + 63 regression tests. See [tests/README.md](../tests/README.md) and [tests/run_golden_regression.md](../tests/run_golden_regression.md).

---

## Architecture

Four-layer clean architecture — **strict dependency rule: flow inward only**:

```
presentation/ → application/ → domain/ → infrastructure/
```

`domain/` has **zero** infrastructure imports — enforced by `.github/instructions/domain-layer.instructions.md`. Use DI via `application/interfaces/`.

Key patterns: Repository, Strategy (`RoutingEngine`), CQRS, `EventBus`.

### Naming Conventions

- **Interfaces**: ABCs in `application/interfaces/` (e.g., `BoardRepository`, `EventPublisher`)
- **Commands**: `<Action><Entity>Command` (e.g., `RouteNetCommand`)
- **Events**: `<Entity><Action>` (e.g., `NetRouted`, `RoutingStarted`)
- **Repositories**: `Memory<Entity>Repository` for in-memory implementations
- **Models**: One class per file in `domain/models/`; `@dataclass(frozen=True)` for value objects

---

## Key Files

| File | Purpose |
|------|---------|
| [main.py](../main.py) | Entry point (CLI, plugin, tests) |
| [orthoroute.json](../orthoroute.json) | Default config (included in built package via `build.py`) |
| [orthoroute/presentation/pipeline.py](../orthoroute/presentation/pipeline.py) | Shared execution pipeline (CLI + GUI) |
| [orthoroute/presentation/plugin/kicad_plugin.py](../orthoroute/presentation/plugin/kicad_plugin.py) | KiCad plugin entry point |
| [orthoroute/algorithms/manhattan/unified_pathfinder.py](../orthoroute/algorithms/manhattan/unified_pathfinder.py) | Main routing engine (~5,967 lines — do NOT refactor without tests) |
| [orthoroute/infrastructure/kicad/rich_kicad_interface.py](../orthoroute/infrastructure/kicad/rich_kicad_interface.py) | IPC board data extraction (pads, tracks, vias, zones, keepouts) |
| [orthoroute/presentation/gui/main_window.py](../orthoroute/presentation/gui/main_window.py) | PCB viewer (rendering + display controls) |
| [orthoroute/shared/utils/performance_utils.py](../orthoroute/shared/utils/performance_utils.py) | `@profile_time` → logs `[PROFILE] func: Xms` at WARNING — **only when `ORTHO_DEBUG=1`** |
| [orthoroute/shared/utils/logging_utils.py](../orthoroute/shared/utils/logging_utils.py) | `init_logging()` — active entry point; console ERROR+, file ERROR (normal) or DEBUG (`ORTHO_DEBUG=1`) |

**Test board:** `TestBoards/TestBackplane.kicad_pcb` — 18-layer backplane (73.1×97.3mm), 1,604 pads, 512 nets. **Golden result:** 18.4 min, 512/512 nets routed, zero overuse (Apr 10, 2026).

---

## KiCad Integration

**Plugin Installation:**
- Plugin folder: `Documents/KiCad/9.0/3rdparty/plugins/com_github_bbenchoff_orthoroute`
- Name must use underscores (not dots) for Python import compatibility
- Manual install required due to [KiCad bug #19465](https://gitlab.com/kicad/code/kicad/-/issues/19465)
- Icons (`icon-24.png`, `icon-64.png`) + metadata (`plugin.json`, `metadata.json`) required for toolbar button
- See [docs/plugin_manager_integration.md](../docs/plugin_manager_integration.md)

**Board Data Adapters (tried in order):**
1. **IPC API** (KiCad 9.0+, preferred) — via `rich_kicad_interface.py`
2. **SWIG** (pcbnew module) — legacy fallback
3. **File Parser** — ⚠️ currently broken, returns 0 pads/nets

**Logs:**
- Plugin mode: `<plugin_dir>/logs/latest.log`, `<plugin_dir>/logs/run_<timestamp>.log`
- Repo mode: `<repo>/logs/latest.log`

**GPU:** 127× speedup requires NVIDIA + CuPy. Fallback: `--cpu-only`. See [docs/cloud_gpu_setup.md](../docs/cloud_gpu_setup.md).

---

## Critical Pitfalls

1. **Logging regressions** — Console must show ERROR+ only. Per-paint-event calls in `_draw_tracks`, `_draw_vias`, `_draw_zones` must stay at DEBUG. If WARNING/INFO appears on the console after a change, revert the logger level.

2. **`UnifiedPathFinder` is a 5,967-line monolith** — Do NOT refactor without tests. Follow `.github/instructions/refactor-pathfinder.instructions.md`: one extraction at a time, test before and after.

3. **Dependency violations** — `domain/` importing from `infrastructure/` is always a bug. See `.github/instructions/domain-layer.instructions.md`.

4. **Plugin not appearing** — Check `Preferences → Plugins → Enable Python API` and restart KiCad. Check `<plugin_dir>/logs/latest.log`.

5. **`_build_owner_bitmap_for_fullgraph`** called per-net (~0.9ms × 512 = ~460ms/iter) — known optimization candidate; do not add more per-net calls in this pattern.

---

## Docs Reference

- [docs/contributing.md](../docs/contributing.md) — project status, test gaps, contribution priorities
- [docs/tuning_guide.md](../docs/tuning_guide.md) — PathFinder parameter tuning
- [docs/congestion_ratio.md](../docs/congestion_ratio.md) — convergence metrics
- [docs/barrel_conflicts_explained.md](../docs/barrel_conflicts_explained.md) — via conflict resolution
- [docs/ORP_ORS_file_formats.md](../docs/ORP_ORS_file_formats.md) — headless cloud routing formats
- [docs/layer_compaction.md](../docs/layer_compaction.md) — layer reduction strategies
- [docs/optimization/](../docs/optimization/) — profiling baselines and logging review
- [docs/optimization/golden_result_2026-04-10.md](../docs/optimization/golden_result_2026-04-10.md) — **GOLDEN STANDARD**: regression baseline, 512/512 nets, zero overuse, GPU benchmarks
36 changes: 36 additions & 0 deletions .github/instructions/domain-layer.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
applyTo: "orthoroute/domain/**"
---

# Domain Layer Rules

The `domain/` layer is the **innermost layer** — it must have zero knowledge of infrastructure, frameworks, or KiCad.

## Hard Rules

- **No infrastructure imports.** Never import from `orthoroute.infrastructure.*` inside `domain/`.
- **No application imports.** `domain/` cannot import from `orthoroute.application.*`.
- **No KiCad bindings.** No `pcbnew`, `kiutils`, or any KiCad-specific type anywhere in this layer.
- **No I/O.** No file reads, network calls, or subprocess usage.

## Allowed Dependencies

```python
# ✅ Standard library + pure third-party only
import dataclasses, typing, enum, math
import numpy as np # pure computation only

# ✅ Within domain
from orthoroute.domain.models.board import Board
from orthoroute.domain.services.routing_engine import RoutingEngine # abstract base
```

## Patterns

- Models: `@dataclass(frozen=True)` for value objects; one class per file.
- Services: Abstract base classes (ABCs) only — concrete implementations live in `infrastructure/`.
- Events: Plain dataclasses, no event bus wiring (bus lives in `application/`).

## When you need external data

Define an interface in `application/interfaces/` and inject it via the constructor. Never reach out directly.
94 changes: 94 additions & 0 deletions .github/instructions/kicad-ipc.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
description: "Use when reading or writing KiCad IPC adapter code: pads, tracks, vias, zones, keepouts, layer names, coordinate units, protobuf fields. Covers rich_kicad_interface.py, ipc_adapter.py, and anything under infrastructure/kicad/."
applyTo: "orthoroute/infrastructure/kicad/**"
---

# KiCad IPC Adapter Conventions

These rules apply to all code under `orthoroute/infrastructure/kicad/`.

## Adapter Priority

Three adapters are tried in order — never skip the fallback chain:

1. **IPC API** (`ipc_adapter.py`) — KiCad 9.0+ socket/HTTP API. Preferred.
2. **SWIG** (`swig_adapter.py`) — Python bindings. Legacy fallback.
3. **File Parser** (`file_parser.py`) — Direct `.kicad_pcb` parsing. Always available.

Changes to one adapter must not break the others. Each implements the same interface from `application/interfaces/board_repository.py`.

## Coordinate Units

- KiCad IPC returns coordinates in **nanometres (nm)** as integers.
- All domain models use **millimetres (mm)** as floats.
- Convert at the adapter boundary — **never pass raw nm into domain objects**:

```python
# ✅ Correct
pad_x_mm = pad.position.x / 1_000_000 # nm → mm

# ❌ Wrong — nm leaks into domain
pad.position.x # raw nm value
```

## Layer Name Normalisation

IPC API returns layer names with a bus-prefix artifact. Strip it before storing:

```python
# IPC returns: "BL_F_Cu", "BL_In1_Cu", "BL_B_Cu"
# Domain needs: "F.Cu", "In1.Cu", "B.Cu"

layer_name = raw_name.removeprefix("BL_").replace("_", ".", 1)
```

Always normalize before passing layer names to domain models or the router.

## Protobuf Field Names

Key fields used in `rich_kicad_interface.py` — use these exact names, do not guess:

| Object | Proto field | Domain meaning |
|--------|-------------|----------------|
| Zone | `ZT_RULE_AREA` | Keepout / rule area (not copper fill) |
| Zone | `rule_area_settings` | Contains all five keepout constraint flags |
| Keepout flags | `keepout_tracks` | Blocks routing tracks |
| Keepout flags | `keepout_vias` | Blocks vias |
| Keepout flags | `keepout_copper` | Blocks copper fill (KiCad DRC only, router ignores) |
| Keepout flags | `keepout_pads` | Blocks pads |
| Keepout flags | `keepout_footprints` | Blocks footprints |
| Via | `drill_diameter` | Via drill size (nm) |
| Pad | `orientation` | Pad rotation in degrees |

## Keepout Extraction Pattern

Keepouts are extracted in `_extract_zones()` and returned as dicts — preserve this schema exactly:

```python
{
'name': str, # Zone name (may be empty string)
'layers': List[str], # Normalised layer names, e.g. ['In1.Cu', 'In2.Cu']
'outline': [[x, y], ...], # Polygon vertices in mm (already converted)
'keepout_tracks': bool,
'keepout_vias': bool,
'keepout_copper': bool, # Router does NOT enforce this flag
'keepout_pads': bool,
'keepout_footprints': bool,
}
```

## Router Enforcement Boundary

The router (`UnifiedPathFinder.initialize_graph`) enforces only `keepout_tracks` and `keepout_vias`.
`keepout_copper` is for KiCad DRC zone fills — do not add router enforcement for it.

## Error Handling

- IPC connection errors must be caught and trigger fallback to SWIG, not crash.
- Log failures at WARNING with the adapter name: `logger.warning("IPC adapter failed: %s", e)`.
- Never swallow errors silently in the fallback chain — always log before falling through.

## Tests

Mock the IPC socket at the adapter boundary; do not launch KiCad in unit tests.
Use `tests/infrastructure/kicad/` for adapter tests. See [docs/contributing.md](../../../docs/contributing.md).
41 changes: 41 additions & 0 deletions .github/instructions/refactor-pathfinder.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
applyTo: "**/unified_pathfinder.py"
---

# Refactoring Rules for unified_pathfinder.py

This file is a **3,900+ line monolith**. It works. Treat it with care.

## Prime Directive

> **Never restructure the whole file in one pass.** One extraction at a time, one test before each extraction.

## Safe Workflow

1. **Identify one cohesive chunk** (≤200 lines, single responsibility).
2. **Write a test** that exercises the current behavior of that chunk.
3. **Run the test** — confirm it passes against the original code.
4. **Extract** the chunk into a new class/module under `orthoroute/algorithms/manhattan/pathfinder/`.
5. **Replace** the original with a call to the new class.
6. **Run the test again** — must still pass.
7. Stop. Do not continue to the next chunk in the same session.

## What NOT to Do

- Do not rename methods across the whole file to fix style.
- Do not add type annotations file-wide.
- Do not reorder methods for readability.
- Do not change any algorithm logic while refactoring — structure only.

## Good First Extractions (pre-identified)

| Chunk | Approximate lines | Suggested class name |
|-------|-------------------|----------------------|
| Cost matrix initialization | ~150 | `CostMatrixBuilder` |
| Keepout obstacle marking | ~120 | `KeepoutObstacleApplier` |
| Pad-to-lattice mapping | ~180 | `PadLatticeMapper` |
| Batch net scheduling | ~200 | `NetBatchScheduler` |

## After Extraction

Place new files in `orthoroute/algorithms/manhattan/pathfinder/` (directory already exists). Update `__init__.py` exports. Do not change the public API of `UnifiedPathFinder` itself.
Loading