When an averaged indexed sum has a coefficient that depends on the summation index, the scope is dropped and the index dangles free. The representation (a SumIndices/SumNonEqual metadata tag on an average leaf) cannot survive SymbolicUtils Add/Mul canonicalization for the general case. The fix is a dedicated moment-layer sum node (one atomic Term, mirroring the existing average node), which can carry an arbitrary summed body robustly.
This surfaced via QuantumCumulants issue #198 (time-dependent indexed coupling), but the root cause is in SQA's average/sum representation.
Minimal reproduction (SQA only)
using SecondQuantizedAlgebra
using Symbolics: @variables
@variables L::Real
ha = NLevelSpace(:atoms, 2); hb = FockSpace(:bus); h = hb ⊗ ha
k = Index(h, :k, L, ha); kk = Index(h, :kk, L, ha)
u(x, y) = DoubleIndexedVariable(:u, x, y)
σ11(z) = IndexedOperator(Transition(h, :σ, 1, 1), z)
σ22(z) = IndexedOperator(Transition(h, :σ, 2, 2), z)
average(Σ(u(kk, k), k)) # want Σ(k) u(kk,k); on some builds returns bare u(kk,k)
average(Σ(u(kk, k) * σ11(kk), k)) # coefficient u(kk,k) hoisted OUTSIDE the Σ
Two observed symptoms: a pure-c-number sum collapses to a single representative term (u(kk,k) with no Σ), and an index-dependent coefficient is pulled outside the sum so k dangles. get_indices does not see the dangling k (it lives in a c-number coefficient), so the malformed expression looks well-formed and only fails later at ODEProblem. The result is also order/version fragile: the same call can land the coefficient inside or outside depending on SymbolicUtils argument ordering.
Root cause
average(::QAdd) (src/average.jl) stamps SumIndices/SumNonEqual metadata on the average leaf and emits coeff * leaf. Whether the index-dependent coeff ends up inside the sum is then left to commutative-* term construction. More fundamentally:
- SymbolicUtils
Add stores a monomial as base => numericCoeff, rebuilding base and discarding metadata on any composite (*/+) node and on any node multiplied by a numeric coefficient. Metadata only survives on an atomic dict-key Sym.
- Therefore the metadata representation cannot express
Σ_k (numeric · f(k) · g(k)) when two factors share the sum index k (e.g. a time-dependent coupling sin(δ(kk,k)t) · u(kk,k)): the carrier would have to be the product, but only atomic factors survive, and stamping one factor leaves the other dangling while stamping all yields a wrong decoupled Σ_k f · Σ_k g.
There is also a known correctness trap already documented downstream: a bare setmetadata leaf is isequal to the un-summed leaf (metadata is ignored by isequal/hash), so the two never cancel in a later subtraction, which can over-count.
Proposed design: a moment-layer sum node
Keep the operator layer unchanged (QAdd.indices is a plain struct field and is robust). Replace the leaf-metadata smear at the moment layer with one atomic node, modeled exactly on the existing AvgFunc average node:
struct SumFunc end; const sym_sum = SumFunc() # operation singleton (cf. AvgFunc)
# body is ANY averaged expression; indices/ne ride as metadata on the node
_indexed_sum(body, indices, ne) =
Term{SymReal}(sym_sum, [body]; type = Number) |>
node -> setmetadata(setmetadata(node, SumIndices, indices), SumNonEqual, ne)
SymbolicUtils.show_call(io, ::SumFunc, x; kw...) = ... # renders Σ(k=1:L)(k≠…) body
Because the node's operation is sym_sum (not +/*), Add/Mul treat it as an opaque base, so its metadata and arbitrary multi-factor body survive canonicalization. The node is not isequal to its body (different operation), which fixes the cancellation trap.
The SumIndices/SumNonEqual metadata keys are reused (now carried on the node), so consumers that read them need a locator change, not a semantics change.
Consumer changes (each: recognize one new node head)
average(::QAdd): per scoped term, emit _indexed_sum(average(scoped_body), indices, ne) instead of stamping a leaf. Multi-factor coefficients and numeric prefactors live inside body.
undo_average: sym_sum case rebuilds Σ(undo_average(body), indices...) (operator QAdd).
- printing:
show_call(::SumFunc, …).
get_order / get_indices / leaf detection: recurse into body.
make_time_dependent: already generic (descends args), recurses into body.
Downstream (QuantumCumulants, separate PR)
evaluate (_materialise_scoped, _propagate_sum_scope), cumulant_expansion/_stamp_sum_to_first_leaves (wrap the factorized product in the node rather than re-stamping leaves), _reduce_ground_in_drift, and scaling operate on body / match the node head. These are tracked in the QC issue.
Migration
- Introduce the node,
average/undo_average/printing/order/indices, plus tests (single- and multi-factor, numeric prefactors, c-number sums, the cancellation/isequal case).
- Switch internal consumers to the node.
- Remove the leaf-metadata emission path.
Out of scope / non-goals
Operator-layer sums (QAdd.indices) are unchanged. This does not reintroduce the v0.5 "type zoo": it is a single uniform node, not per-shape types.
When an averaged indexed sum has a coefficient that depends on the summation index, the scope is dropped and the index dangles free. The representation (a
SumIndices/SumNonEqualmetadata tag on an average leaf) cannot survive SymbolicUtilsAdd/Mulcanonicalization for the general case. The fix is a dedicated moment-layer sum node (one atomicTerm, mirroring the existing average node), which can carry an arbitrary summed body robustly.This surfaced via QuantumCumulants issue #198 (time-dependent indexed coupling), but the root cause is in SQA's
average/sum representation.Minimal reproduction (SQA only)
Two observed symptoms: a pure-c-number sum collapses to a single representative term (
u(kk,k)with noΣ), and an index-dependent coefficient is pulled outside the sum sokdangles.get_indicesdoes not see the danglingk(it lives in a c-number coefficient), so the malformed expression looks well-formed and only fails later atODEProblem. The result is also order/version fragile: the same call can land the coefficient inside or outside depending on SymbolicUtils argument ordering.Root cause
average(::QAdd)(src/average.jl) stampsSumIndices/SumNonEqualmetadata on the average leaf and emitscoeff * leaf. Whether the index-dependentcoeffends up inside the sum is then left to commutative-*term construction. More fundamentally:Addstores a monomial asbase => numericCoeff, rebuildingbaseand discarding metadata on any composite (*/+) node and on any node multiplied by a numeric coefficient. Metadata only survives on an atomic dict-key Sym.Σ_k (numeric · f(k) · g(k))when two factors share the sum indexk(e.g. a time-dependent couplingsin(δ(kk,k)t) · u(kk,k)): the carrier would have to be the product, but only atomic factors survive, and stamping one factor leaves the other dangling while stamping all yields a wrong decoupledΣ_k f · Σ_k g.There is also a known correctness trap already documented downstream: a bare
setmetadataleaf isisequalto the un-summed leaf (metadata is ignored byisequal/hash), so the two never cancel in a later subtraction, which can over-count.Proposed design: a moment-layer sum node
Keep the operator layer unchanged (
QAdd.indicesis a plain struct field and is robust). Replace the leaf-metadata smear at the moment layer with one atomic node, modeled exactly on the existingAvgFuncaverage node:Because the node's
operationissym_sum(not+/*),Add/Multreat it as an opaque base, so its metadata and arbitrary multi-factorbodysurvive canonicalization. The node is notisequalto its body (different operation), which fixes the cancellation trap.The
SumIndices/SumNonEqualmetadata keys are reused (now carried on the node), so consumers that read them need a locator change, not a semantics change.Consumer changes (each: recognize one new node head)
average(::QAdd): per scoped term, emit_indexed_sum(average(scoped_body), indices, ne)instead of stamping a leaf. Multi-factor coefficients and numeric prefactors live insidebody.undo_average:sym_sumcase rebuildsΣ(undo_average(body), indices...)(operatorQAdd).show_call(::SumFunc, …).get_order/get_indices/ leaf detection: recurse intobody.make_time_dependent: already generic (descends args), recurses intobody.Downstream (QuantumCumulants, separate PR)
evaluate(_materialise_scoped,_propagate_sum_scope),cumulant_expansion/_stamp_sum_to_first_leaves(wrap the factorized product in the node rather than re-stamping leaves),_reduce_ground_in_drift, andscalingoperate onbody/ match the node head. These are tracked in the QC issue.Migration
average/undo_average/printing/order/indices, plus tests (single- and multi-factor, numeric prefactors, c-number sums, the cancellation/isequalcase).Out of scope / non-goals
Operator-layer sums (
QAdd.indices) are unchanged. This does not reintroduce the v0.5 "type zoo": it is a single uniform node, not per-shape types.