Conversation
…ange Follows the CryptoProvider pattern established in zaino: zingolabs/zaino@a62dcd3 rustls 0.23 auto-selects a provider only when exactly one of its ring / aws_lc_rs features is enabled. The workspace pinned rustls with the ring feature while rustls's default features already enable aws-lc-rs, so feature unification put both providers in the graph and nothing installed a process-level default - four zingo-netutils tests panicked with "Could not automatically determine the process-level CryptoProvider". Changes: - zingo-netutils gains crypto::ensure_default_crypto_provider(): a guarded, first-install-wins install of aws-lc-rs, called where the crate builds rustls configs. An embedder (e.g. zingo-mobile) that installs its own provider first keeps it. Re-exported through zingolib. - The seven scattered ad-hoc ring provider installs (LightClient constructors, sync_example_wallet, zingo-cli main, libtonode tests) are replaced with that one helper. - Workspace features: rustls drops ring and gains prefer-post-quantum (the X25519MLKEM768 hybrid group leads outbound handshakes; ML-KEM is only available on aws-lc-rs); hyper-rustls and tokio-rustls switch ring -> aws-lc-rs; tonic gains tls-aws-lc so the provider is asserted in the feature graph rather than inherited. - zingo-price: reqwest switches to rustls-tls-no-provider (never auto-selects) and installs the provider before fetching; it sits below zingo-netutils in the graph, so it carries a mirrored helper. - Real crypto use excised: the legacy BIP32 HMAC-SHA512 derivation in extended_transparent.rs moves from ring::hmac to the RustCrypto hmac + sha2 crates. The stored-wallet regression tests (verify_example_wallet_*) confirm derivation is unchanged. - The ring and rustls dependencies are dropped from zingolib, zingo-cli, and libtonode-tests; ring leaves the workspace dependency table. - zcash_proofs's download-params feature (whose minreq -> rustls 0.21 -> ring subtree is the sole ring residue) is now enabled only for the build script that fetches sapling params; the runtime dependency graph is 100% aws-lc-rs. Verified: the four previously failing zingo-netutils tests pass (including the live mainnet-indexer connect); 95/95 unit tests across zingo-netutils, zingo-price, and zingolib; cargo clippy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit de11e16)
Flaky tests should surface as failures instead of being masked by automatic retries. - ZINGOLIB_NEXTEST_RETRIES defaults to "0" and is now a conditional [env] entry (env_not_set), matching ZINGOLIB_NEXTEST_FILTER: cargo-make otherwise clobbers inherited environment variables, so this is what makes a per-run `ZINGOLIB_NEXTEST_RETRIES=N makers test ...` override actually work when investigating a flake. - The two hardcoded `--retries 2` nextest invocations in .github/workflows/test.yaml (which bypass makers) become `--retries 0`. - `makers help` example lines updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ersions - zcash_local_net + zingo_test_vectors: git-pinned to the infrastructure repo's bump_to_NU6.3 branch (0dc4a51), replacing the stale rev pin. - ZAINO_IMAGE_TAG 0.3.1 -> 0.6.0-rc.1-no-tls (same digest published under zingodevops/zaino and zingodevops/zainod, so the docker-ci FROM line still resolves). - ZEBRA_VERSION=6.0.0-rc.0 added; not yet consumed, reserved for the zebrad provisioning wiring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit 0983747)
- libtonode-tests no longer pins test_lwd_zcashd; it runs the no-feature default network combo, the Core stack (zainod + zebrad). - zingolib_testutils: delete the test_zainod_zcashd and test_lwd_zcashd features; test_lwd_zebrad survives (behind the infrastructure repo's legacy-stack feature) until the Legacy stack is removed as a unit. - scenarios.rs: four network_combo cfg blocks reduced to two; stale zcashd doc comments rewritten in Validator/Indexer terms. - CONTEXT.md: seed the glossary (Network combo, Faucet; Core/Legacy stack deferred to the infrastructure repo's glossary). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Darkside is lightwalletd's darksidewalletd mode, and the bump_to_NU6.3 infrastructure gates the lightwalletd indexer behind the opt-in legacy-stack feature (scheduled for removal there). Without it the darkside suite no longer compiles; this completes the zebrad port for that suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The zainod+zebrad default network combo needs a zebrad binary, but the
CI image never contained one, so every test that launches a local net
died with the infrastructure's "Test executable must be set in
TEST_BINARIES_DIR" spawn panic.
Wire ZEBRA_VERSION from .env.testing-artifacts through the whole chain,
following zaino's pattern:
- docker-ci/Dockerfile: zebrad-downloader stage from
docker.io/zfnd/zebra:${ZEBRA_VERSION}; the binary is copied from
/usr/local/bin/zebrad into /usr/bin/, with a provenance LABEL.
- Makefile.toml: ZEBRA_VERSION joins the build-image --build-args and
echo output, the container_image_tag inputs (so a zebra bump rebuilds
the image), and /usr/bin/zebrad joins the container symlink list that
populates TEST_BINARIES_DIR.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`add_subtree_roots` fails only inside the CI container, with the wallet
consistently 32 sapling shards short of the server's count (1096 vs
1128); the same commit passes on the host. pepper-sync's subtree-root
fetch accepts a clean-but-premature stream end as complete, so the open
question is what ends the stream early in the container.
Add an #[ignore]d diagnostic, run manually in the environment under
investigation:
ZINGOLIB_NEXTEST_FILTER='' makers container-test -p libtonode-tests \
-E 'test(diagnose_subtree_root_stream)' --run-ignored all --no-capture
For three fresh connections it prints the sapling subtree-root count,
the chain height where the stream ended, and how it ended — then the
decisive experiment: resuming from start_index = count on the same
connection. Zero additional roots means that backend genuinely has only
that many (a lagging backend behind the zec.rocks load balancer);
roughly 32 more means the stream is being cut mid-flight in the
container's network path. Finally it reports where pepper-sync's own
fetch path lands the wallet.
On the host all connections currently report 1128 roots, clean end,
resume 0, wallet 1128.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing it get_subtree_roots accepted a clean-but-premature stream end as complete: its retry logic only fires on DeadlineExceeded / EOF-decode errors, so a stream cut mid-flight (proxy, flow control) silently truncated the wallet's shard tree instead of surfacing. For unbounded requests (max_entries == 0, the only in-tree callers), a clean end is now trusted only after a resume pass from the current index comes back empty; a cut stream resumes where it stopped. Costs one empty round-trip on the happy path. Bounded requests keep single-pass semantics — resuming would fetch past the caller's cap. Defense-in-depth arising from the add_subtree_roots container failures (wallet 1096 vs server 1128 sapling shards); the evidence for that incident points to a stuck backend behind the zec.rocks load balancer, which no client-side change can paper over — see issue #2440 and the diagnose_subtree_root_stream diagnostic added in the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…drop obsolete shield step Two fixes discovered by the first zebrad-backed scenario runs: - Scenario defaults passed the all-at-1 ActivationHeights::default(), but zebrad rejects every NU6.x activation block whose lockbox/funding-stream config doesn't pair with the heights, stalling the chain at genesis. default_test_activation_heights() now returns the harness's regtest fixture shape (all=1, nu5=2, nu6=2, nu6_1/2=5) — the only shape its subsidy fixtures support — with nu6_3 unactivated: this branch's wallet builds against released librustzcash (NU6.2-max branch ids), so an NU6.3-active chain rejects every wallet transaction. Drop the set_nu6_3(None) override when the ironwood wallet branch lands. - zebrad_shielded_funds no longer shields before offloading: zebrad mines to Orchard natively (the "does not support shielded mining" comment was stale), so coinbase already lands in the pool and the shield found no transparent funds (InsufficientFunds(0)). fast::basic_scenario passes end-to-end on zainod + zebrad with this (launch, orchard mining, 100-block maturity, sync, sends). Exact-balance assertions calibrated for the streamless zcashd stack (e.g. 1_875_000_000 in mine_to_transparent) still need recalibration for the deferred funding stream (1/100 of subsidy from height 2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The deferred (lockbox) funding stream in the harness fixture skims 1/100 of the block subsidy from height 2 onward, and orchard coinbase accrues from the block after NU5 activation (height 3) — both confirmed to the zatoshi by server-run failures. The zcashd-era expectations assumed full subsidy per block from height 1. - zingolib_testutils::scenarios gains DEFERRED_STREAM_SKIM, POST_STREAM_BLOCK_REWARD, FUND_OFFLOAD_AMOUNT, mined_block_rewards_total, FUNDED_FAUCET_SETUP_HEIGHT and funded_faucet_orchard_balance so expectations are derived, not magic. - Recalibrated: mine_to_transparent, mine_to_transparent_coinbase_maturity, mine_to_transparent_and_shield, mine_to_transparent_and_propose_shielding, value_transfers, filter_empty_messages, mine_to_orchard (un-ignored: its "zebrad cannot mine shielded" reason is disproven), send_orchard_back_and_forth (faucet-as-miner collects coinbase + fees). - Sapling mining is the one true zebrad gap: send_mined_sapling_to_orchard becomes send_mined_orchard_to_orchard (the spend-mined-funds state transition it exercises is pool-agnostic); unfunded_mobileclient mines transparent (no client holds the mining key); mine_to_sapling (pre-existing ignore, reason corrected) and send_heartwood_sapling_funds (also needs Canopy above 1, which the config writer refuses) stay ignored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tions The fast-set run showed two failure mechanisms beyond the funding-stream arithmetic (which mine_to_transparent_and_shield/propose confirmed): 1. Sync race: bare sync_and_await reaches whatever the Indexer has ingested, which lags the Validator right after generate_blocks — mine_to_transparent observed one block's coinbase at a height-3 tip. New scenarios::sync_client_to_validator_tip loops sync until the client fully scans the Validator's tip; every scenario-setup sync and the failing tests' post-generate syncs now use it. 2. Receiver selection: with an Orchard miner, block 1 (pre-NU5) pays the miner's SAPLING receiver the full pre-stream subsidy (observed s_balance: 625000000). Orchard-scenario expectations now include BLOCK_ONE_SAPLING_COINBASE. Model constants are hypothesis-encoded so the next run adjudicates each with a distinct delta: ORCHARD_COINBASE_START_HEIGHT=2 (off-by-one POST_STREAM_BLOCK_REWARD means flip to 3), fee-returns-to-miner (10_000 deltas), and a new fast::orchard_miner_coinbase_distribution probe asserts the full o/s/t distribution at a known tip. Also: sync_all_epochs_from_heartwood -> sync_all_expressible_epochs (canopy=3 and staggered NU heights are inexpressible: the config writer requires Canopy at 1 and the subsidy fixtures pair only with the fixture shape); unified_address_discovery syncs to tip before asserting discovery (same race). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shielded coinbase has no 100-block maturity rule (transparent-only); the server run proved it: value_transfers spends orchard coinbase notes 1-3 blocks old. The 100-block generation in zebrad_shielded_funds was a relic of the old shield-transparent-coinbase flow, costing every funded scenario minutes of mining plus a 100-block wallet sync. Funded setup now ends at height 4 (launch + 2 setup + offload confirm). Every derived expectation recomputes through FUNDED_FAUCET_SETUP_HEIGHT. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All ten funded scenarios failed at the offload with zebra's "rejected from the mempool until the next chain tip block ... could not validate orchard proof" once the maturity dance was removed: at a height-3 tip the offload had to spend the tip block's own note. Ordinary sends of blocks-old notes pass (value_transfers), so the rule being tripped is specifically about the tip block. zebrad_shielded_funds now syncs first and mines one block before the spend, so the wallet's selectable notes and anchor sit at least one block behind zebra's tip. Funded setup ends at height 5; expectations recompute through FUNDED_FAUCET_SETUP_HEIGHT. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adder The tip-separation attempt refuted itself informatively: holding the wallet one block behind made zebra reject the offload with "incorrect consensus branch id" — the wallet signs for its-tip+1, and the fixture ladder activates NU6.1/NU6.2 at height 5, exactly where the streamlined setup operates. Tests that sync to the real tip before sending (value_transfers) never hit this. zebrad_shielded_funds now mines two extra blocks first (tip 5, clearing the ladder so tip+1 and the inclusion block share the NU6.2 branch id), syncs to the real tip, then offloads — with four pre-tip notes so oldest-first selection never touches the tip block's note (the original "could not validate orchard proof" rejection). Funded setup ends at height 6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…IGHT The height-6 funded setup shifted every absolute block height in tests built on faucet_funded_recipient_default by +3 (funding tx confirms at setup+2, observed as "Confirmed at 8"). Rewrite the literals in received_tx_status_pending_to_confirmed_with_mempool_monitor and send_to_transparent_and_sapling_maintain_balance as offsets from scenarios::FUNDED_FAUCET_SETUP_HEIGHT so future setup-height changes recompute instead of breaking. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sends in one transaction, then mines one block at a time — waiting for
the sender's wallet to reach each new height — until the transaction is
confirmed. Owns both send hazards discovered on the zainod+zebrad
stack, so callers need no choreography:
- indexer tip-lag (generate_blocks returns on the validator's height;
the indexer's poll-based ingestion lags 100-500ms; a sync begun in
that window silently misses the newest block);
- zebra's tip-block spend rejection ("rejected from the mempool until
the next chain tip block"): on exactly that error, one separation
block is mined and the send retried once, mirroring the
zebrad_shielded_funds fix.
Confirmation is asserted (up to five blocks of tolerance), so a
transaction that misses its block for any new reason fails at the send
site with its txids, not downstream at an unrelated balance assert.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the historical unified_address_discovery failure, established by direct measurement: after generate_blocks returns, the validator has the new block but the indexer's poll-based ingestion reports the old tip for 100-500ms (10/10 rounds, max ~490ms, zainod 0.4.3 and 0.6.0 alike). A wallet sync begun inside that window completes against the stale tip and silently misses the newest block. Exonerated with evidence along the way: discovery bookkeeping (works when the block is visible), zebra's block templates (the send IS in the next block), zainod 0.6.0 specifically, and generate_blocks height accounting. Deliberately RED until the harness (or its callers) wait for indexer convergence after block generation; the assertion message carries the mechanism. Until then, tests must use the wallet-height-aware helpers (increase_height_and_wait_for_client, sync_client_to_validator_tip, send_and_bump). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ten sites used bare generate_blocks + sync_and_await (or quick_send + generate_blocks(1)), each a latent indexer tip-lag or tip-block-spend flake with retries at 0: - unified_address_discovery: setup mining via increase_height_and_wait_for_client; the four-output send via send_and_bump (confirmation now asserted at the send site). - Three LibtonodeEnvironment sites (messages_*/value_transfers): bare recipient sync after increase_chain_height (which waits only on the validator) becomes sync_client_to_validator_tip. - by_address_finsight: three quick_send + mine + sync triplets become send_and_bump. - send_and_sync_with_multiple_notes_no_panic: three generate_n_blocks + dual-sync groups become increase_height_and_wait_for_client for the first client and sync_client_to_validator_tip for the second. Left alone, deliberately: second-client syncs that follow a wait-for-client on another client (ordering makes them safe — the indexer demonstrably has the block once any wallet synced to it), pure mempool-pickup syncs after unmined sends, and propose-time generate-without-sync sites whose semantics depend on the wallet NOT seeing the new blocks. Verified: unified_address_discovery green through send_and_bump on the host stack that previously failed deterministically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gfr8) Clears the runtime copy flagged by the high-severity Dependabot alert (panic on malformed CRL BIT STRING). Defense-in-depth only: no zls code configures CRLs or revocation checking, so the vulnerable parse is unreachable — there is no input channel that delivers a CRL to webpki. The second locked copy (0.101.7) cannot be bumped: it belongs to the build-time-only zcash_proofs -> minreq -> rustls 0.21 subtree (pinned to webpki ^0.101, no patched release in that line) and goes away only when upstream drops minreq or we drop download-params. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ensure_taddrs_from_old_seeds_work and address_generation_deterministic_and_coherent assert key derivations against fixed vectors — nothing they check touches a chain, yet each spent ~12s launching zebrad+zainod for scaffolding it never used. They are now zingolib unit tests against LightWallet directly (21ms total). The one behavioral subtlety is made explicit: the original "next address index is 2" came from the scenario ClientBuilder generating an extra sapling-only address after construction, not from LightClient itself — the unit test reproduces that step with a comment so the vector-pinned indices and their diversified derivations are unchanged. Two fewer LocalNet launches per fast-suite run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
15 chain_generics tests failed at fund_client_orchard's bare sync_and_await: the Indexer lags the Validator after generate_blocks (the new diagnostic measures up to ~515ms), so the faucet signs from a stale tip and zebra rejects the send near the NU6.1/NU6.2 activation boundary. ConductChain gains sync_client_to_tip with a bare-sync default (darkside unchanged); the generic funding flow calls it, and LibtonodeEnvironment overrides it with scenarios::sync_client_to_validator_tip — the one shared deterministic-sync implementation. Also: verify_old_wallet_uses_server_height_in_send's stale height literal (8, pre-dating the funded-setup change) is now derived from FUNDED_FAUCET_SETUP_HEIGHT. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
filter_empty_messages and message_thread assert message semantics — empty-memo filtering, address filtering, and message/value-transfer ordering — which are pure summary derivation over wallet transaction records. As integration tests they cost 49s and 132s of LocalNet mining and syncing (message_thread alone set the fast suite's wall-clock); as zingolib unit tests they run in 54ms combined. pepper-sync gains three test-features constructors (WalletNote, OutgoingNote, and WalletTransaction::new_for_test_with_orchard_notes) so tests can fabricate received/sent transaction records — including the recipient_full_unified_address that sent-side filtering matches against — without a chain. The unit tests reproduce the integration tests' exact shapes: two empty memos then Hello+empty for filtering; the five-transaction Alice/Bob/Charlie thread with "Reply to" memos for counting (bob 3, charlie 2) and bidirectional height ordering. Two fewer LocalNet launches and ~180s less node time per fast-suite run; the fast set drops from 18 to 16 tests with coverage relocated, not removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test asserts the absence of a lag that is known to exist (measured 10/10 rounds on zainod 0.4.3 and 0.6.0 alike), so in a normal run it only restates a documented harness limitation. Keep it as an on-demand probe (cargo nextest run --run-ignored) to re-measure after a zcash_local_net indexer-convergence barrier lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test was marked "temporary for infrastructure integration": it ran faucet_funded_recipient_default and asserted nothing. During the zebrad port it earned its keep as the canonical probe — it was the first test to surface the crypto-provider panic, the heights/subsidy mismatch, and the funding-flow rework. All of that is now permanently covered: every funded test exercises the same setup path and then asserts on it, and orchard_miner_coinbase_distribution asserts the coinbase model explicitly. What remained was ~70s of assertion-free LocalNet runtime per run and one extra exposure to the launch port flake. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…et suite create_send_to_self_with_zfz_active asserts value-transfer KIND classification: a self-send yields SendToSelf(Basic), and the Zennies-for-Zingo output yields Sent(Send) addressed to the ZFZ address. Both are pure summary derivation over a transaction record — one fabricated SendToSelf transaction whose outgoing orchard notes are (a) a note to the wallet's own orchard receiver and (b) a note carrying the decoded ZFZ regtest unified address. 72s of LocalNet becomes 14ms. Deliberately NOT claimed by this port: the propose_send_all -> send_stored_proposal execution the integration test drove incidentally; that pipeline remains covered by the chain-bound send tests (send_to_tex and the funded-send family). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simpool_insufficient_1_sapling_to_orchard flaked with "status regression to Transmitted" in the with_assertions confirmation poll. It cannot be a regression: pepper-sync's update_status is monotonic. An observed Transmitted means the wallet has not yet seen the tx anywhere — under instant regtest mining a tx is often mined before the mempool monitor observes it (status path Transmitted -> Confirmed, skipping Mempool), and the check can land in the measured ~500ms Indexer-lag window before the confirming block is scanned. Treat Transmitted like Mempool: keep polling. The existing patience bound converts a persistent Transmitted (a genuine broadcast failure) into a test failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…Net suite spendable_balance_includes_notes_in_incomplete_shards asserts that a confirmed, positioned note whose block has no completed orchard shard still counts as spendable — spendable-balance composition over wallet state, no chain required. 70s of LocalNet becomes 17ms, and the port is stricter than the original: the integration test only obtained an incomplete shard incidentally from regtest's tiny tree, while the unit test constructs the condition explicitly (empty orchard shard ranges, fully-scanned birthday..tip, sapling tip checkpoint for the anchor). Rig additions: pepper-sync's WalletNote::new_for_test gains a position parameter, and SyncState::new_for_test (test-features) builds sync state from explicit scan ranges — unlocking future spendability and witness-gating tests without a chain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
zancas
changed the title
Member
Author
|
230 / 247 pass at this point. Rather than debugging these in place, I am going to invest in extending the mock framework. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds atop, and requires #2437 merge that first.
Fixes: #2439
230 / 247 ran on dev when I started