gf180mcu DRC performance regression

[mole99]

Hello!

I stumbled across what seems to be a performance regression when I tried to update the wafer.space project template to the latest LibreLane version.

Previously, KLayout DRC took around 1 - 1.5 hours on my machine. After updating KLayout to 0.30.6, the DRC check has been running for 10 hours and still hasn't completed.

I've tested the following KLayout versions:

• 0.30.4 - good
• 0.30.5 - good
• 0.30.6 - bad
• 0.30.7 - bad

For example, the following rules now take a considerable amount of time: DV.3 (more than 2 hours), ESD.8 (1h+)

You can find a standalone reproducible here: https://drive.proton.me/urls/M2D0EQTZVW#19J3oG6G5cuY

[klayoutmatthias]

I'll take a look. Whatever went wrong here ... but even 1 to 1.5hours is too much for a single run.

[klayoutmatthias]

The patch should bring performance back to 0.30.5 level. Still runtimes are too long to my taste. The new version and 0.30.5 take about 4.5 hours for a full run on the test case with 4 threads.

[mole99]

Thank you so much, Matthias!

I can confirm that the patch fixes the performance regression. The runtime is now 01:05:48 on 24 threads.

---

Yes, there may be ways to improve the performance further, both by improving KLayout and by refining the DRC run sets.

However, we're trying a third approach that is also compatible with the first two: Clyde has been looking into explicitly parallelizing the DRC run. The general idea is that gf180mcu.drc first performs netlist extraction and general derivations. Then we fork the Ruby process for each DRC rule deck. The rule decks run independently of each other, each on a single core, but they share read-only memory in RAM due to Copy-on-Write (CoW) of the OS. After a runset finishes, the DRC result is captured and merged using your new ReportDatabase#merge function :)

This way, we should be able to utilize most of the available cores. Let's see how it goes!

[klayoutmatthias]

Very good :)

I think improvement is possible both in the deck (for example in MSLOT), but also on my side. And of course parallelization helps. I noticed that some of the operations degrade into almost flat operations, which is not.optimal.for.deep.mode and creates a large memory footprint. Maybe it is possible to optimize this case.

Matthias