Skip to content

Add Block Number Cache to Staged Stream Sync#4942

Merged
GheisMohammadi merged 17 commits intodevfrom
feature/sss_block_cache
Feb 20, 2026
Merged

Add Block Number Cache to Staged Stream Sync#4942
GheisMohammadi merged 17 commits intodevfrom
feature/sss_block_cache

Conversation

@GheisMohammadi
Copy link
Copy Markdown
Collaborator

@GheisMohammadi GheisMohammadi commented Sep 2, 2025

During staged stream sync, the node repeatedly queries every connected peer for their current block height — both in identifySyncedStreams (to filter peers that are synced past the target) and in estimateCurrentNumber (to determine the sync target).

For long-range syncs, these queries are almost entirely redundant. If a peer reports block 1,200,000 and our current target is 500,000, there is no reason to ask that peer again until we actually catch up to 1,200,000.

This PR introduces a per-stream block number cache that eliminates those repeated round-trips, significantly reducing network load on both the syncing node and its peers.

The BlockNumberCache stores each peer's last-reported block height keyed by stream ID.

  • On a cache hit — when the cached value already meets or exceeds the current target — the cached value is returned immediately with no network call.
  • On a cache miss, the cache queries that specific peer using a whitelist option and stores the result.
  • The cache includes configurable max size with LRU eviction, TTL-based expiry with background cleanup, and hit/miss stats for observability.

To support querying a specific peer on cache miss, the p2p request manager's pickAvailableStream was updated to properly respect whitelists:

  • When a request carries a whitelist, only those streams are considered as candidates.
  • The whitelist and blacklist data structures were also changed from plain maps to SafeMap for thread safety.

The integration touches three files in the staged sync pipeline:

  • In stage_bodies.go, identifySyncedStreams now calls bnCache.GetBlockNumber() per stream instead of making a direct protocol request.
  • In syncing.go, estimateCurrentNumber routes through bnCache.doGetCurrentNumberRequest() which queries any available stream with high priority and caches the responding stream's result.
  • The cache itself is initialized with default configuration in staged_stream_sync.go and lives on the StagedStreamSync struct.

Comprehensive unit tests cover cache hits, misses, threshold logic, eviction, expiry, invalidation, and reset.

@GheisMohammadi GheisMohammadi self-assigned this Sep 2, 2025
@GheisMohammadi GheisMohammadi marked this pull request as draft September 2, 2025 23:15
@GheisMohammadi GheisMohammadi force-pushed the feature/sss_block_cache branch 2 times, most recently from f90ed09 to 6207229 Compare September 4, 2025 07:37
@GheisMohammadi GheisMohammadi force-pushed the feature/sss_block_cache branch from 6207229 to 96ad6c7 Compare December 29, 2025 11:18
@GheisMohammadi GheisMohammadi force-pushed the feature/sss_block_cache branch from 96ad6c7 to c27e6d2 Compare February 16, 2026 09:02
@GheisMohammadi GheisMohammadi marked this pull request as ready for review February 16, 2026 09:10
@mur-me
Copy link
Copy Markdown
Collaborator

mur-me commented Feb 20, 2026

Devnet is looking good, let's put it on the testnet

@GheisMohammadi GheisMohammadi merged commit 8c3ad2f into dev Feb 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants