test(edr_provider): broaden solx provider-path stack-trace coverage - #1552
test(edr_provider): broaden solx provider-path stack-trace coverage#1552nebasuke wants to merge 4 commits into
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1552 +/- ##
==========================================
+ Coverage 79.77% 79.89% +0.11%
==========================================
Files 453 452 -1
Lines 79038 78956 -82
Branches 79038 78956 -82
==========================================
+ Hits 63050 63079 +29
+ Misses 13810 13694 -116
- Partials 2178 2183 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
e7c64db to
dc3c75e
Compare
e09dae7 to
ab5a7d7
Compare
ab5a7d7 to
90deafb
Compare
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Improves solx stack-trace attribution/parity (especially around compiler-generated “declaration-attributed” locations) and adds tooling/fixtures to regenerate and test these behaviors against updated solx versions.
Changes:
- Extend solx trace strategy + DWARF decoding to better handle declaration-padding / unmapped locations and line-0 call sites.
- Add a CLI command to regenerate solx compiler-output fixtures, plus a new StackTraceScenarios fixture corpus.
- Expand provider integration tests and update sweep/docs to reflect improved parity and solx 0.1.6 behavior.
Reviewed changes
Copilot reviewed 15 out of 17 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| js/integration-tests/solx-parity-sweep/test/sweep.ts | Removes scenarios that no longer diverge from solc, tightening parity checks. |
| js/integration-tests/solx-parity-sweep/README.md | Updates pinned-divergence documentation to reflect solx 0.1.6 parity changes. |
| crates/tool/cli/src/update.rs | Fixes repo-root discovery by adjusting ancestor depth. |
| crates/tool/cli/src/solx_fixtures.rs | Adds fixture-regeneration helper that runs solx --standard-json and rewrites outputs. |
| crates/tool/cli/src/main.rs | Exposes fixture regeneration as a CLI subcommand. |
| crates/edr_solidity/src/trace_strategy.rs | Adds strategy hooks for location equivalence and declaration-attributed inference; improves revert attribution. |
| crates/edr_solidity/src/error_inferrer.rs | Uses new strategy hooks to keep stack traces stable and avoid declaration-location regressions. |
| crates/edr_solidity/src/debug_info/dwarf.rs | Handles solx 0.1.6 call_line=0 by falling back to decl sites; adds targeted tests. |
| crates/edr_solidity/fixtures/sources/StackTraceScenarios.sol | Adds new Solidity scenario corpus targeting provider-path stack-trace variants. |
| crates/edr_solidity/fixtures/solx_compiler_output_stack_trace_scenarios.json | Adds compiled solx output fixture for StackTraceScenarios. |
| crates/edr_solidity/fixtures/solx_compiler_output.json | Updates Counter fixture output to new solx/toolchain settings. |
| crates/edr_solidity/fixtures/solx_compiler_input_stack_trace_scenarios.json | Adds committed standard-json input for StackTraceScenarios (with contents scrubbed). |
| crates/edr_solidity/fixtures/solx_compiler_input.json | Pins optimizer mode in Counter standard-json input for fixture stability. |
| crates/edr_solidity/fixtures/README.md | Documents new gen-solx-fixtures regeneration path and the new fixture pair. |
| crates/edr_provider/tests/integration/solx_stack_trace.rs | Broadens provider-level integration tests for more stack-trace variants and new fixture. |
| .changeset/solx-declaration-attributed-inference.md | Adds changelog entry for the solx stack-trace inference fixes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7dde91b to
39b9358
Compare
a4d67d2 to
b1de264
Compare
39b9358 to
33c8322
Compare
b1de264 to
a0d6af5
Compare
33c8322 to
0d2f30a
Compare
a0d6af5 to
fb576c7
Compare
0d2f30a to
052a2b6
Compare
052a2b6 to
b8faf71
Compare
Groundwork for the coverage in this PR, with no change in what the existing tests assert: - The three fixture loaders differed only in paths and source key; they now delegate to `assemble_build_info`. The `include_str!` literals stay at the call sites — the macro needs a literal path. - Drop the local `deploy` in favour of `test_utils::deploy_contract`, which does the same send-and-read-receipt dance. - Drop the local `source_reference_of` match in favour of the existing `StackTraceEntry::source_reference` method. - Extract `stack_trace_from_failure` from `expect_failed_call_stack_trace` and route the latter through a value-taking variant, so the deployment-failure and payability helpers added later can reuse both. - Calldata construction moves into `call` / `encode_call_u256` / `encode_call_address` instead of being spelled out per test.
One test per DWARF-decoding axis over the existing scenarios fixture (no fixture changes): panic sub-codes (0x01/0x12/0x21/0x31/0x32), CREATE-path reverts (plain and internal-helper constructors — creation-bytecode `debugInfo` was previously untested), modifier bodies, cross-contract and mutually recursive CALL chains, external-recursion frame counts, internal helpers/recursion, internal libraries, fallback/receive bodies, and per-statement line discrimination (two `require`s in one function; the trace must point at the second). The provider path is what Hardhat 3's JS-test flow (`hardhat test` + ethers/viem) hits. #1425 covers the solidity-test-runner path via the JS parity sweep, but the sweep is CI-skipped until `@nomicfoundation/hardhat-solx` is published — so these are the trace assertions that actually run in CI today. Assertions stay at entry-variant + source line, except for the two call shapes where frame gains/losses are the point (cross-contract call, external recursion): those pin the full rendered shape via `assert_trace_shape`. Exhaustive frame parity with solc remains the sweep's job.
… artifacts The StackTraceScenarios fixture's dispatch-level contracts, so far unused: the entry variants the solc corpus exercises but no solx test reached. FunctionNotPayableError, UnrecognizedFunctionWithoutFallbackError, MissingFallbackOrReceiveError, both FallbackNotPayable variants, InvalidParamsError (truncated calldata), plus an externally linked library exercising `linkReferences` placeholder substitution and DELEGATECALL decoding into the library's own `debugInfo`, and DirectLibraryCallError for a direct call to a deployed library. `link_library` bails on bytecode referencing more than one distinct library instead of silently mis-linking — it takes a single address.
…erence pipeline Organize the file so coverage is auditable against edr_solidity's inference stages: all helpers in one block up front, then one test section per stage, with a file-level coverage map that also states what is deliberately left to the solc corpus or follow-ups. Fold the two variant-only variance-axis tests into their stronger families (overflow becomes panic code 0x11; the custom-error test now pins the decoded message and statement line). Strengthen the panic family and dispatch guards to pin source anchors, not just entry variants. Drop the unused bail_on_call_failure from the test provider config — the eth_call channel is follow-up material.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (1)
crates/edr_provider/tests/integration/solx_stack_trace.rs:74
assemble_build_infousesunwrap()when replacing the source content. If the fixture format changes or thesource_keyis wrong, this will panic without a helpful error message, making failures harder to diagnose in CI. Prefer returning a contextualanyhowerror instead.
input.sources.get_mut(source_key).unwrap().content = source_content.to_owned();
Adding wide test coverage for stack trace tests, along with some test refactoring for more re-use. This is a follow-up of #1572. I've tried to put tests into blocks to make it a bit easier to follow what's currently covered. I could split into files instead if that is easier.
Note that I've removed two tests as they were already covered by a more generic test, see also the test folding Claude section below.
This is the first PR of a planned few. I'm trying to do a structured expansion of stack-trace coverage. The idea here is that this functions both as increased coverage for solx/solc, and some of the pre-work for being able to migrate away from hardhat-tests.
Planned next, roughly in order:
0x51(calling a function variable that was never assigned). Both are reported through fallback code paths no test reaches today.eth_call: the tests so far only cover failing transactions; failing read-only calls take a separate path that we should either cover or explicitly leave out of scope.Claude summary
Rebased onto
mainnow that #1577 (the DWARF attribution fixes + fixture infrastructure) is merged. This PR is tests only: it broadens the provider-path (JSON-RPC) stack-trace coverage for solx artifacts from 11 tests to 36, all against fixtures already onmain— and organizes the file so the coverage is auditable against the inference pipeline instead of reading as a blob.The provider path is what Hardhat 3's JS-test flow (
hardhat test+ ethers/viem) hits. #1425 covers the solidity-test-runner path via the JS parity sweep, but the sweep is CI-skipped until@nomicfoundation/hardhat-solxis published — so these are the trace assertions that actually run in CI today.File layout mirrors the inference pipeline
solx_stack_trace.rsputs all helpers in one block up front, then one test section peredr_solidityinference stage, so reviewing the file is walking the pipeline:infer_before_tracing_call_message: payability, missing function/fallback/receive, direct library callscheck_last_instruction:InvalidParamsErrorfor truncated calldatacheck_revert_or_invalid_opcode: six panic codes with statement anchors, custom-error argument decoding, revert-line discrimination between requiresfilter_redundant_frames: cross-contract frames, external/internal/mutual recursion, internal helpers and libraries, linked external library (linkReferences+ DELEGATECALL decode), fallback/receive bodiesfix_initial_modifier+SolxTraceStrategyattribution: plain, nested, cross-contract, bare-revertdebugInfo: reverting constructors, plain and via internal helpercheck_last_submessage: returndata-size errors (real callee and EOA target)-O3twins (themode3_*tests)-O3; hardhat-solx defaults to-O1). Since solx 0.1.6 the-O1DWARF maps compiler helpers to real statement lines, so only these more aggressively optimized artifacts still reach the fallback inference paths: walking back to the last statement when a revert is attributed to a declaration line, and recovering the failing function for unmapped bare revertsEvery test now asserts a source anchor (function/contract/statement line), not just the entry variant — e.g. each panic code pins the statement it's attributed to, and the dispatch guards pin the declaration they anchor at. The two call shapes where frame gains/losses are the point (cross-contract call, external recursion) pin the full rendered trace via
assert_trace_shape. Line pins are goldens: a shifted anchor after a solx upgrade is expected drift (update the pin), a lost frame is a regression.Deliberately not covered — location-free classification paths already pinned by the solc corpus, plus follow-up material: message-kind dispatch (precompiles, unrecognized contracts, contract-too-large), create-side guards, the solc-opcode-pattern rewrites in
mapped_inline_internal_functions_heuristics, out-of-gas rewrites, proxy propagation, and theeth_callchannel.bail_on_call_failurewas dropped from the test provider config since nothing exercised it.The external-recursion test already caught one real regression: a late refactor in #1425 dropped the per-strategy recursion carve-out in
filter_redundant_frames, collapsing any external-recursion depth to a single frame. That fix was upstreamed separately as #1569.Two tests folded, one renamed (no coverage removed)
The diff deletes two tests by name; both were variant-only assertions strictly subsumed by a stronger replacement:
panic_error_variant_surfaces_for_overflow_scenarioasserted only that somePanicErrorentry appears forOverflowTest— a weaker duplicate of the panic-code family right next to it. It's nowpanic_code_surfaces_for_arithmetic_overflowin that family, additionally pinning the code (0x11) and the statement the panic is attributed to (x = x + 1, line 26).custom_error_variant_surfaces_for_custom_error_scenarioasserted only that someCustomErrorentry appears. It's nowcustom_error_decodes_name_and_args, additionally pinning the decoded message (reverted with custom error 'MyError(42, "custom error")') and therevertstatement line — the known-selector argument-decoding path had no coverage before.revert_error_variant_surfaces_for_counter, is kept asrevert_error_surfaces_end_to_end_for_counter: it's the only test on the minimal Counter fixture (a second, independent artifact assembly), so it stays as the plumbing smoke test.Accepted gaps
Two
SolxTraceStrategyfallbacks remain untested:unresolved_callstack_entryand the panic-helper selector fallback. Both are last resorts for instructions whose DWARF location doesn't resolve to a known function, and no fixture produces that — not even at-O3(MutualRecursionTest, the closest candidate, reached parity without them). Auditing that solx covers every reachable PC with aDW_TAG_subprogrambelongs in solx's own test suite, not EDR scenarios.Verification
solx_stack_trace.rs: 36/36 against the 0.1.6 fixtures (including the-O3twins from fix(edr_solidity): solx stack-trace attribution for declaration-level and line-0 DWARF #1577).edr_providerintegration harness: 216/216 (fork tests requireALCHEMY_URL).cargo +nightly fmt --checkandcargo clippy -p edr_provider --tests --features test-utilsclean.