Skip to content

Drop brittle == 7 in Point Mutation test for Julia 1.11+ compat#135

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:smc/fix-point-mutation-test
May 26, 2026
Merged

Drop brittle == 7 in Point Mutation test for Julia 1.11+ compat#135
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:smc/fix-point-mutation-test

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Summary

Please ignore this PR until reviewed by @ChrisRackauckas.

Fixes the 10 pre-existing Point Mutation test failures on Julia 1.11+ (visible in any recent CI run on master where the matrix includes Julia 1.x — and called out in #133).

Root cause

The test in test/mutations.jl asserts:

@test length(ex) == length(mex) == 7

The == 7 was the tree length produced by rand(rng, tr, 2) on Julia 1.10. Julia 1.11 reimplemented rand(::AbstractRNG, ::Dict/Set/KeySet) with a more efficient algorithm that consumes the RNG stream differently. Evolutionary.rand(rng, ::TreeGP, H) in src/gp.jl calls rand(rng, keys(t.terminals)) / rand(rng, keys(t.functions)) — i.e. rand over a KeySet. On Julia 1.11+ that produces a different (length-5) tree even with StableRNG(42), because StableRNGs stabilizes the raw RNG number stream, not how Base's collection sampling consumes it.

The behavioral invariant being tested is that point mutation preserves the node count of the input expression. length(ex) == length(mex) already captures that; the == 7 was hard-coding an implementation artifact specific to 1.10's Dict-sampling RNG sequence.

Change

One-line test change plus a 4-line comment explaining the WHY (so a future reader doesn't re-add the constant):

+        # The invariant is length(ex) == length(mex); the original `== 7` was the
+        # length on Julia 1.10 only. Julia 1.11 changed how `rand` samples from
+        # `Dict`/`KeySet`, so `rand(rng, tr, H)` yields a different tree on 1.11+
+        # even under StableRNG.
-            @test length(ex) == length(mex) == 7
+            @test length(ex) == length(mex)

No source code changes — this is a brittle test, not a bug.

Test plan

  • Local Pkg.test() on Julia 1.10.11 (was already passing) — still passes.
  • Local Pkg.test() on Julia 1.12.6 (was failing with 10 Point Mutation failures) — now passes.
  • CI on this PR.

Context

Discovered while modernizing CI infrastructure in #134 (which expanded the test matrix to include pre/1/lts on multiple OSes and surfaced these failures consistently). This fix is split out as its own PR because it's a test correctness change, separate from CI-infra work.

🤖 Generated with Claude Code

The test in test/mutations.jl asserts `length(ex) == length(mex) == 7`,
but the `== 7` was the tree length on Julia 1.10 only. Julia 1.11
reimplemented `rand(::AbstractRNG, ::Dict/Set/KeySet)` with a more
efficient algorithm that consumes the RNG stream differently, so
`rand(rng, tr, H)` in src/gp.jl produces a different-length tree on
1.11+ even with StableRNG (StableRNGs stabilizes the raw RNG number
stream, not how Base's collection sampling consumes it).

The behavioral invariant being tested is that point mutation preserves
the node count of the input expression, which `length(ex) == length(mex)`
already captures. Drop the hard-coded `== 7`.

Verified locally: tests pass on both Julia 1.10.11 and 1.12.6.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review May 26, 2026 20:56
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Heads-up on the CI status: this PR's branch is based on master, so it runs the legacy .github/workflows/CI.yml. GitHub Actions has now auto-deprecated actions/cache@v1 (used by that workflow), so the Julia matrix jobs auto-fail before any test code executes — see the "This request has been automatically failed because it uses a deprecated version of actions/cache: v1" annotation. This is the same infra failure that motivated #134, not a problem with the fix itself.

The fix has been verified locally on:

  • Julia 1.10.11 — passes (was already passing)
  • Julia 1.12.6 — passes (was failing with 10 Point Mutation errors)

Once #134 lands, I'll rebase this onto the new master and the modern Tests workflow will pick up the fix; alternatively, this can be merged on local-verification trust given the trivial nature of the change.

@ChrisRackauckas ChrisRackauckas merged commit 44f7e8e into SciML:master May 26, 2026
2 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants