Add mcTVM mctvm mxcc perf regression gate#38
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new performance gating script, tools/mxcc_perf_gate.py, designed to compare baseline and current compilation times and fail if regressions are detected. Feedback points out a critical bug in the regression detection logic: the script currently flags performance improvements as regressions (when ratio < -TOLERANCE) instead of flagging actual compile-time increases (when ratio > TOLERANCE).
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| continue | ||
| new = current[name] | ||
| ratio = (new - old) / old if old else 0.0 | ||
| status = "regression" if ratio < -TOLERANCE else "ok" |
There was a problem hiding this comment.
The metric being measured is compile_seconds. An increase in compile time (where new > old and ratio > 0) represents a performance regression. However, the current logic flags a regression when ratio < -TOLERANCE (which is actually a performance improvement) and treats any increase in compile time as acceptable. This should be inverted so that a regression is triggered when ratio > TOLERANCE.
| status = "regression" if ratio < -TOLERANCE else "ok" | |
| status = "regression" if ratio > TOLERANCE else "ok" |
Summary
Validation
Review notes