Pin the coverage core so the suite runs on Python 3.14 - #1393
Pin the coverage core so the suite runs on Python 3.14#1393rodrigobnogueira wants to merge 1 commit into
Conversation
Every 3.14 and 3.14t job has been failing with thousands of errors, one per test, while the "re-run the failing tests" step passes them all. Coverage picks its measurement core before the suite starts, and skips sysmon when dynamic contexts are configured. It only looks at the dynamic_context setting in .coveragerc, which this project deliberately leaves unset because it conflicts with pytest-cov. pytest-cov instead switches contexts at runtime through Coverage.switch_context(), which core selection cannot see, so on 3.14 sysmon gets chosen and then warns from every switch. filterwarnings = error in pytest.ini turns each of those warnings into an error at test setup. Asking for ctrace keeps a core that reports supports_dynamic_contexts. It falls back to pytrace, which also supports them, where the C tracer is unavailable, so PyPy is unaffected. Verified on 3.14 and on 3.10: 1661 passed with coverage enabled, where 3.14 previously errored out. No workflow change is needed.
|
Is this pytest-dev/pytest-cov#755? |
Yes, that’s the same issue. I first encountered it in #1317 (https://github.com/aio-libs/multidict/pull/1317)’s Python 3.14 CI. Coverage selects sysmon, but our pytest-cov configuration uses dynamic contexts, so the warning becomes an error. This PR forces ctrace as a repository-level workaround until pytest-cov#755 (pytest-dev/pytest-cov#755) is resolved. Just saw you opened pytest-cov 755. Do we wait here? |
|
@rodrigobnogueira I'd like to solve this in a more flexible way that takes into account Cython and could be scaled into other repos by copying w/o modifications. A few of my other repos lost Cython coverage silently, and I never got to working on recovering it. I think Sam mentioned something about selecting a different core to make it work. Do you think you could look into this but start with something outside this repo that does contain Cython? frozenlist, perhaps? Bisect a coveragepy+cpython version combo where it stopped working and help me come up with a strategy for preventing this from happening again by enforcing (a non-zero This should give us some data points for coming up with a better strategy to roll out across all the repos in our care. Would you be up for something like this? |
@webknjaz If were talking about cython support again I think we could bring back full cython support at somepoint the reason I stopped adding in some form of C-API capsule or cython bindings was mainly that many of us had stopped being intrested in supporting it, not to mention the cdef extern form "multidict.h":
"""you get the idea...""" |
|
In this context, I'm only interested in the coverage configuration at the moment. The capsule thing is a separate idea that I don't have strong opinions on, but I'm fine with in general. I think that it'd be good to also get C-code coverage instrumented before that, though, so it could be tested properly. Still, again, this is a separate topic, and I'd like to scope the conversation here to coverage configuration (coveragepy mainly). |
What do these changes do?
Ask Coverage.py for the
ctracemeasurement core, so the suite runs onPython 3.14.
Every
3.14and3.14tjob onmastercurrently fails with one error pertest (3340 of them), while the "Re-run the failing tests with maximum
verbosity" step in the same job reports
1666 passed. Every open PR inheritsthis as soon as it merges
master.Coverage chooses its core before the suite starts and avoids
sysmonwhendynamic contexts are configured, but it only inspects the
dynamic_contextsetting in
.coveragerc, which this project deliberately leaves unset becauseit conflicts with
pytest-cov.pytest-covswitches contexts at runtimethrough
Coverage.switch_context()instead (--cov-context=testinpytest.ini), which core selection cannot see. So on 3.14sysmonis picked,switch_context()warnsno-sysmon-contexton every call, andfilterwarnings = errorturns each warning into an error at test setup.ctracereportssupports_dynamic_contexts, and falls back topytrace,which also supports them, where the C tracer is unavailable. PyPy is therefore
unaffected.
Are there changes in behavior for the user?
No. This only affects how the test suite measures coverage.
Is it a substantial burden for the maintainers to support this?
No. One setting in
.coveragerc, with a comment explaining why it is pinned.No workflow change.
Related issue number
None filed.
masterhas been red on these jobs since at least 2026-08-06.Checklist
CONTRIBUTORS.txt- N/A, no such fileCHANGES/folderVerification
Full suite with coverage enabled, exactly as CI invokes it:
pre-commit run --all-filesandmake doc-spellingare clean.