Skip to content

Pin the coverage core so the suite runs on Python 3.14 - #1393

Open
rodrigobnogueira wants to merge 1 commit into
aio-libs:masterfrom
rodrigobnogueira:fix-coverage-core-314
Open

Pin the coverage core so the suite runs on Python 3.14#1393
rodrigobnogueira wants to merge 1 commit into
aio-libs:masterfrom
rodrigobnogueira:fix-coverage-core-314

Conversation

@rodrigobnogueira

Copy link
Copy Markdown
Member

What do these changes do?

Ask Coverage.py for the ctrace measurement core, so the suite runs on
Python 3.14.

Every 3.14 and 3.14t job on master currently fails with one error per
test (3340 of them), while the "Re-run the failing tests with maximum
verbosity" step in the same job reports 1666 passed. Every open PR inherits
this as soon as it merges master.

Coverage chooses its core before the suite starts and avoids sysmon when
dynamic contexts are configured, but it only inspects the dynamic_context
setting in .coveragerc, which this project deliberately leaves unset because
it conflicts with pytest-cov. pytest-cov switches contexts at runtime
through Coverage.switch_context() instead (--cov-context=test in
pytest.ini), which core selection cannot see. So on 3.14 sysmon is picked,
switch_context() warns no-sysmon-context on every call, and
filterwarnings = error turns each warning into an error at test setup.

ctrace reports supports_dynamic_contexts, and falls back to pytrace,
which also supports them, where the C tracer is unavailable. PyPy is therefore
unaffected.

Are there changes in behavior for the user?

No. This only affects how the test suite measures coverage.

Is it a substantial burden for the maintainers to support this?

No. One setting in .coveragerc, with a comment explaining why it is pinned.
No workflow change.

Related issue number

None filed. master has been red on these jobs since at least 2026-08-06.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist - N/A, this is a test-infrastructure fix
  • Documentation reflects the changes - N/A
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt - N/A, no such file
  • Add a new news fragment into the CHANGES/ folder

Verification

Full suite with coverage enabled, exactly as CI invokes it:

Python before after
3.14 errors, one per test 1661 passed
3.10 1661 passed 1661 passed

pre-commit run --all-files and make doc-spelling are clean.

Every 3.14 and 3.14t job has been failing with thousands of errors, one
per test, while the "re-run the failing tests" step passes them all.

Coverage picks its measurement core before the suite starts, and skips
sysmon when dynamic contexts are configured. It only looks at the
dynamic_context setting in .coveragerc, which this project deliberately
leaves unset because it conflicts with pytest-cov. pytest-cov instead
switches contexts at runtime through Coverage.switch_context(), which
core selection cannot see, so on 3.14 sysmon gets chosen and then warns
from every switch. filterwarnings = error in pytest.ini turns each of
those warnings into an error at test setup.

Asking for ctrace keeps a core that reports supports_dynamic_contexts.
It falls back to pytrace, which also supports them, where the C tracer
is unavailable, so PyPy is unaffected.

Verified on 3.14 and on 3.10: 1661 passed with coverage enabled, where
3.14 previously errored out. No workflow change is needed.
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Aug 9, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 242 untouched benchmarks


Comparing rodrigobnogueira:fix-coverage-core-314 (898fbc8) with master (41c1b91)

Open in CodSpeed

@webknjaz

Copy link
Copy Markdown
Member

Is this pytest-dev/pytest-cov#755?

@rodrigobnogueira

rodrigobnogueira commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

Is this pytest-dev/pytest-cov#755?

Yes, that’s the same issue. I first encountered it in #1317 (https://github.com/aio-libs/multidict/pull/1317)’s Python 3.14 CI. Coverage selects sysmon, but our pytest-cov configuration uses dynamic contexts, so the warning becomes an error. This PR forces ctrace as a repository-level workaround until pytest-cov#755 (pytest-dev/pytest-cov#755) is resolved.

Just saw you opened pytest-cov 755. Do we wait here?

@rodrigobnogueira
rodrigobnogueira marked this pull request as ready for review August 11, 2026 02:03
@webknjaz

Copy link
Copy Markdown
Member

@rodrigobnogueira I'd like to solve this in a more flexible way that takes into account Cython and could be scaled into other repos by copying w/o modifications. A few of my other repos lost Cython coverage silently, and I never got to working on recovering it. I think Sam mentioned something about selecting a different core to make it work. Do you think you could look into this but start with something outside this repo that does contain Cython? frozenlist, perhaps? Bisect a coveragepy+cpython version combo where it stopped working and help me come up with a strategy for preventing this from happening again by enforcing (a non-zero *.cyx coverage in Codecov reports, for example).

This should give us some data points for coming up with a better strategy to roll out across all the repos in our care. Would you be up for something like this?

@Vizonex

Vizonex commented Aug 12, 2026

Copy link
Copy Markdown
Member

@rodrigobnogueira I'd like to solve this in a more flexible way that takes into account Cython and could be scaled into other repos by copying w/o modifications. A few of my other repos lost Cython coverage silently, and I never got to working on recovering it. I think Sam mentioned something about selecting a different core to make it work. Do you think you could look into this but start with something outside this repo that does contain Cython? frozenlist, perhaps? Bisect a coveragepy+cpython version combo where it stopped working and help me come up with a strategy for preventing this from happening again by enforcing (a non-zero *.cyx coverage in Codecov reports, for example).

This should give us some data points for coming up with a better strategy to roll out across all the repos in our care. Would you be up for something like this?

@webknjaz If were talking about cython support again I think we could bring back full cython support at somepoint the reason I stopped adding in some form of C-API capsule or cython bindings was mainly that many of us had stopped being intrested in supporting it, not to mention the _testcapi module was never going to work and that it needed to be in the same directory as the multidict source files. There might be an easier way of adding multidict in that doesn't involve a C-API capsule known as extern where we just need the headerfile of multidict to get it to work properly. Know however that it will not support the Pythonized version of multidict and will only work with the C extension of Multidict. However on the bright side this speeds up parsing and writing http requests by around 30% inside of aiohttp and a workaround for the non-CAPI wouldn't be hard to come up with.

cdef extern form "multidict.h":
     """you get the idea..."""

@webknjaz

Copy link
Copy Markdown
Member

In this context, I'm only interested in the coverage configuration at the moment. The capsule thing is a separate idea that I don't have strong opinions on, but I'm fine with in general. I think that it'd be good to also get C-code coverage instrumented before that, though, so it could be tested properly. Still, again, this is a separate topic, and I'd like to scope the conversation here to coverage configuration (coveragepy mainly).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants